The idea behind AWS Lambda is great — no servers or software to manage sounds very attractive to me as a developer. But it turned out that managing Lambda functions is not trivial as well. Apart from many new concepts and tools provided by AWS, there are several 3rd-party tools built to help you deal with AWS Lambda. Those tools require you to learn many new concepts and interfaces which may be tough for a beginner. Therefore, I created a starter kit project which will help you to start using NodeJS Lambda functions and learning the AWS Lambda stack.

The starter kit uses the official AWS CLI and CloudFormation to manage Lambda resources. No other dependencies are required. The starter kit glues various AWS CLI commands via several very basic shell scripts so it is easy to learn and change. Additionally, the starter kit unlike many other projects out there is using YAML for writing CloudFormation templates so that templates become much easier to read.

TL;DR Source Code

Conventions

The starter kit is based on several conventions:

1. The kit is a project consisting of several Lambda functions.
2. All Lambda functions are created/updated/deleted using CloudFormation as the same time.
3. The source code of Lambda functions is stored in AWS S3.
4. Lambda functions reside in individual folders; each one of them is a separate NodeJS project.

Prerequisites

To start using the starter kit, you need to have an AWS account and have the AWS CLI installed and configured. The following guides will be helpful: Installing the AWS Command Line Interface, Configuring the AWS Command Line Interface.

Once the AWS CLI is installed you can clone the starter kit:

git clone git@github.com:OrKoN/aws-lambda-nodejs-cloudformation-starter-kit.git lambda-starter
cd lambda-starter

By default, the kit contains one Lambda function called FooFunction. The function outputs foo when invoked.

The source code of Lambda functions is stored in S3 so you need to create an S3 bucket first. The bucket name (S3_BUCKET_NAME) is configured in config.env. Feel free to change it as well as the STACK_NAME.

Deploying the project

To deploy the project functions, use the following command:

npm run create-stack

This command creates zip archives with your functions and uploads them to S3. The command generates two files for every functions FunctionName-latest.zip and FunctionName-Version.zip, where Version is the version number of the function. It can be an md5 hash of the zip file or anything else. There is another convention: the version number for a function is provided by a build.sh that is function-specific and that is supposed to output only the version number.

After zip files are uploaded to S3, the command creates a new CloudFormation stack which, in turn, creates the actual Lambda resources (or other resources) on AWS. Here is how the stack appears in the CloudFormation console:

Updating the project

Once you modified the source code of your project and want to deploy the new version:

npm run update-stack

This will upload new versions of the functions to S3 and update the same CloudFormation stack so that changes are actually deployed.

Deleting the project

You can always delete all resources belonging to the project by running

npm run delete-stack

Internals

Every function is supposed to have a build.sh file which builds zip files and uploads them to S3 and outputs the version of the function. Here is how it may look like:

#!/bin/bash

# creating a zip file and suppressing the output
{
  rm FooFunction.zip
  cd FooFunction; zip -r ../FooFunction.zip *; cd ..;
} &> /dev/null


# version is just an md5 hash
Version=$(md5sum -b FooFunction.zip | awk '{print $1}')

# uploading zip files to S3
{
  aws s3 cp FooFunction.zip s3://$S3_BUCKET_NAME/FooFunction-$Version.zip
  aws s3 cp FooFunction.zip s3://$S3_BUCKET_NAME/FooFunction-latest.zip
  rm FooFunction.zip
} &> /dev/null

# build.sh should output the version only
echo $Version

The source of create-stack, update-stack and delete-stack is located in the bin folder. create.sh is used to create the stack for the first time:

#!/bin/bash

./FooFunction/build.sh # build functions and upload to S3
# TODO: add other functions here

# aws cli create-stack is invoked
aws cloudformation create-stack --stack-name=$STACK_NAME \
  --template-body=file://cloudformation.yaml \
  --capabilities CAPABILITY_IAM \ # to create roles for Lambda functions
  --parameters \
    ParameterKey=S3BucketName,ParameterValue=$S3_BUCKET_NAME

echo "Creating..."

# wait till the creation is finished
aws cloudformation wait stack-create-complete --stack-name $STACK_NAME

update.sh is similar but it makes use of the version parameter to deploy the right version:

#!/bin/bash

FOO_FUNCTION_VERSION=$(./FooFunction/build.sh)
# TODO: add other functions here

aws cloudformation update-stack \
  --stack-name=$STACK_NAME \
  --template-body=file://cloudformation.yaml \
  --capabilities CAPABILITY_IAM \
  --parameters \
  ParameterKey=S3BucketName,ParameterValue=$S3_BUCKET_NAME \
  ParameterKey=FooFunctionVersion,ParameterValue=$FOO_FUNCTION_VERSION
  # TODO: add version parameters of other functions here

echo "Updating..."

aws cloudformation wait stack-update-complete --stack-name $STACK_NAME

delete.sh is simple:

#!/bin/bash

aws cloudformation delete-stack \
  --stack-name=$STACK_NAME

echo "Deleting..."

aws cloudformation wait stack-delete-complete --stack-name $STACK_NAME

Cloudformation

AWS resources are defined in cloudformation.yaml. This file describes all functions of the project and this implies that all functions and all resources are updated at the same time. Here is how the default cloudformation.yaml looks like:

Description: >
  AWS Lambda Nodejs Starter project.
  To add a new function define the function role and the function itself following
  the template of FooFunction.

Parameters:

  FooFunctionVersion:
    Description: "Version of the lambda function required to update existing stack"
    Type: String
    Default: "latest"

  S3BucketName:
    Description: "S3BucketName"
    Type: "String"

Resources:
  FooFunctionRole:
    Type: "AWS::IAM::Role"
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          -
            Effect: "Allow"
            Principal:
              Service:
                - "lambda.amazonaws.com"
            Action:
              - "sts:AssumeRole"
      Policies:
        -
          PolicyName: "FooFunctionPolicy"
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
            - Effect: Allow
              Action:
              - logs:CreateLogGroup
              - logs:CreateLogStream
              - logs:PutLogEvents
              Resource: "*"

  FooFunction:
    Type: "AWS::Lambda::Function"
    Properties:
      Code:
        S3Bucket: !Ref S3BucketName
        S3Key: !Sub FooFunction-${FooFunctionVersion}.zip
      Description: Lambda function that schedules updates
      Environment:
        Variables:
          NODE_ENV: production
      FunctionName: FooFunction
      Handler: "index.handler"
      MemorySize: 128
      Role: !GetAtt FooFunctionRole.Arn
      Runtime: "nodejs4.3"
      Timeout: 100

Here, you can tweak FooFunction to change the parameters of the Lambda function and change FooFunctionRole to grant additional permissions (such as DynamoDB or SNS access) to your function.

If you don’t want to create a new role, you can re-use existing one and delete FooFunctionRole. In this case, you don’t need to specify the --capabilities CAPABILITY_IAM parameter in the create and update operations.

Adding New Functions

To add a new function you need to create a new folder where the function will be located and add references to the new function wherever you find a TODO comment in the shell scripts. Additionally, you need to create a build.sh which will build your function and output its version. Also the function resource and its role need to be added to the cloudformation.yaml.

The source code can be found on Github: https://github.com/OrKoN/aws-lambda-nodejs-cloudformation-starter-kit

I hope this starter kit will be helpful to anyone who starts working with AWS Lambda and that it is a good introduction to more complicated technologies like terraform, apex or node-lambda. Thanks for reading!