alexharv074.github.io

My blog

View on GitHub
18 August 2018

Creating a ZIP file for an AWS Lambda Python function

by Alex Harvey

I found the documentation a bit confusing for creating an AWS Lambda ZIP file for a Python function plus dependencies, so this post will document the procedure.

Note about virtualenv and large zip files

I found this post at codeburst.io by Alexandra Johnson quite useful, although in the end it created a zip file that was very large due to inclusion of the whole virtualenv, and actually involved more steps than required. That is no doubt because the Amazon documentation also talks about zipping up a virtualenv.

Meanwhile, the AWS Lambda limits are documented here. On the allowed size of the zip file:

If the size of your Lambda function’s zipped deployment packages exceeds 3MB, you will not be able to use the inline code editing feature in the Lambda console. You can still use the console to invoke your Lambda function.

Including the whole virtualenv can easily lead to the 3MB size being exceeded, so this procedure avoids using the virtualenv altogether.

Create a zip file

About this example

In this example I am creating a simple zip file with a single Python pip package, xlrd.

Create requirements.txt

So, I created a simple requirements.txt at the root of my project:

xlrd

Note about ZIP file folder structure

The final folder structure is expected to look like this:

$ tree -L 1 .
.
├── lambda_function.py
├── xlrd
└── xlrd-1.1.0.dist-info

Workaround for Homebrew’s Python on Mac OS X

At the time of writing, Homebrew’s Python on Mac OS X is broken. Based on this Stack Overflow post, I had to create a file setup.cfg at the root of my project:

[install]
prefix=

Without this, a pip error message is seen:

must supply either home or prefix/exec-prefix — not both

Install dependencies

To install:

$ pip install -r requirements.txt -t .

Now my project root looks like this:

$ tree -L 1 .
.
├── bin
├── requirements.txt
├── setup.cfg
├── xlrd
└── xlrd-1.1.0.dist-info

Create the zip file

To create the zip file:

$ zip -r9 ../lambda.zip * -x "bin/*" requirements.txt setup.cfg

Note that it is not obvious how to exclude directories from a zip file. See here for more on this.

Update a zip file

One more useful thing to know is how to update the lambda function itself in the zip file without having to pip install and update everything else. The command for that is:

$ zip -g ../lambda.zip lambda_function.py
tags: lambda