Lambda Layers: How to Create Them? — Python Version

[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function':
No module named 'SmbClient'

Well, The above error what made me write about Lambda Layers, what are they? and why we need to use them ? and how to create them?

Hopefully by the end of this article, you will get all the answers for the above questions and you will be ready to use layers in your Lambda functions.

What is Lambda Layers?

Based on this,

A Lambda layer is an archive containing additional code, such as libraries, dependencies, or even custom runtimes. When you include a layer in a function, the contents are extracted to the /opt directory in the execution environment. You can include up to five layers per function.

In other words, it is the place where we can add any code dependencies the Lambda function needs.

The following diagram illustrates the high-level architectural differences between two functions that share dependencies. One uses Lambda layers, and the other does not.

image source

So how to create these layers ?

One thing to keep in mind when creating these layers is that Lambda extracts the layer contents into the /opt directory in your function’s execution environment.

1. Let's create the virtual environment and activate it

python3 -m venv venv
source venv/bin/activate

2. Create a directory with the name “Python” and change to it

mkdir python
cd python

Its important you create a directory named python. This is mandatory when working with lambda layers.

pip3 install pip_module_name -t .
pip3 install smbclient -t .

When installing the packages, make sure to install it in the current directory

Wait till the installation is done , and to see the packages, run the command ls

$ ls
_cffi_backend.cpython-310-x86_64-linux-gnu.so cffi.libs
pycparser-2.21.dist-info   smbprotocol-1.10.1.dist-info
bin      cryptography     pyspnego-0.9.1.dist-info   spnego
cffi      cryptography-37.0.4.dist-info   smbclient
cffi-1.15.0.dist-info    pycparser     smbprotocol

Now go above level, out of the Python directory

cd ..

Then zip the Python folder

zip -r name_of_deployment_package.zip python

Now that zip file is the file you will need to upload to the AWS Lambda layers.

One small note to add here , is that because AWS Lambda are running on Linux and sometimes — well in my case I was installing the packages on Mac OS — and sometimes because the binaries included in the deployment package were built for a platform other than Linux, you might be getting the below error

Invalid ELF header

How to fix it ?

Well, it is easy, using Docker.

1. Let's run a docker container using using ubuntu image

docker run -v lambda_code:/python -it --rm ubuntu

2. Get the container up-to-date and install python3, pip, and zip.

apt-get update && apt-get install -y -qq python3-pip git \
&& cd /usr/local/bin && ln -s /usr/bin/python3 python \
&& python3 -m pip install --upgrade pip \
&& python3 -m pip install ipython \
&& rm -rf /var/lib/apt/lists/*
apt-get update && apt-get install zip

3. Change into the directory and install the needed packages

cd /python
pip install -t . pip_module_name

4. Move up a level out of the Python directory and zip the packages

cd ..
zip -r9 name_of_deployment_package.zip python

Now you have the .zip file to upload to your AWS Lambda Layer.

Happy Coding!