React to AWS CodeCommit events with AWS Lambda

AWS CodeCommit Overview

AWS CodeCommit lets you store and version your source code. You can also configure it to generate triggers in response to specific events, independent of your Git flow.

Some of the event types are:

  • Branch/Tag creation
  • Branch/Tag deletion
  • Push operations

You can define up to 10 different triggers for each CodeCommit repository. Also, you can optionally configure a trigger to react only to some tags or branches.

At the moment, you can implement a trigger with two AWS services:

  • Amazon SNS - Send predefined notifications, for example via email
  • AWS Lambda - Implement any custom logic, such as other AWS services or 3rd-party integrations

Here we will create a new CodeCommit repository and configure a custom AWS Lambda trigger to send enhanced email notifications. By default, SNS will include in each email only a link to the updated repository and the IAM user who performed the operation. With our custom notification, we will also include a list of commits with the corresponding author and message.

In more advanced scenarios, you may want to use Lambda to send Slack messages, automatically close tickets or any other integration that makes a developer’s life easier.

Creating a new CodeCommit repository

First of all, we have to create a new CodeCommit repository.

After clicking Create repository, a Connect to your repository pop-up window is opened. It is OK to click Close and ignore the pop-up.

Once you’ve created the repository, you can clone it and start pushing new commits. Normally, you would accomplish this on your local machine. However, for simplicity sake, Cloud Academy has launched a pre-configured Linux instance that has the AWS Command Line Interface (CLI) and Git already installed for you.

Navigate to Services > Compute > EC2, then click Instances. You should see one Linux instance running. You will need to connect to the instance with an SSH client and the proper SSH key. How to do this differs depending if your local host is Linux or Windows based. Please follow the instructions below depending on your operating system.

Login to the EC2 instance and then run aws configure to configure your account details. Then run the following to setup your git cli.

[ec2-user@ip-10-10-10-220 ~]$ git config --global credential.helper '!aws codecommit credential-helper $@'
[ec2-user@ip-10-10-10-220 ~]$ git config --global credential.UseHttpPath true
[ec2-user@ip-10-10-10-220 ~]$ git config --global user.email "shravan@gmail.com"
[ec2-user@ip-10-10-10-220 ~]$ git config --global user.name "shravan k"

[ec2-user@ip-10-10-10-220 ~]$ git clone https://git-codecommit.us-west-2.amazonaws.com/v1/repos/CloudAcademyLab
Cloning into 'CloudAcademyLab'...
warning: You appear to have cloned an empty repository.
[ec2-user@ip-10-10-10-220 ~]$

Create a new SNS topic and subscription

As a second step, we have to create a new SNS topic to publish notifications about our CodeCommit repository.

We can now create a new subscription to this topic and connect it to your personal email address so that you will receive new updates about the CodeCommit repository while you complete this lab.

Click Topics in the left navigation pane. From the SNS Topics listing page, select the new CodeCommits Topic and click Create subscription.

The Topic ARN field will already be filled for you. Select Email for the notification protocol and then enter your personal email as the Endpoint.

Note that you will need to confirm the subscription by clicking on the confirmation link that you’ll find in your inbox. Confirmation emails from Amazon SNS typically arrive within a minute or two, but may take up to five minutes. If you don’t see it, be sure to check your Spam folder.

The body of the email will looks similar to the following:

After clicking on confirmation, you will see something like this:

Once CodeCommit and SNS are properly configured, we can proceed with the creation of new triggers.

Implementing a Lambda Function to process CodeCommit events

We are ready to create the processing logic. Create a new Lambda function shown below

The processing logic will need to fetch the commit data for each record given by CodeCommit. Note that each record might contain multiple commits since developers often push multiple commits at once. For each commit, we will retrieve the related data (author and message), and then publish a new enhanced SNS notification. The Lambda implementation shown below is repository agnostic, meaning that it can be attached to any CodeCommit repository and handle events from multiple repositories.

import boto3

codecommit = boto3.client('codecommit')
sns = boto3.resource('sns')
topic = sns.Topic('arn:aws:sns:us-west-2:753761729251:CodeCommits')

REPO_LINK = "https://console.aws.amazon.com/codecommit/home?region=%s#/repository/%s"
BRANCH_LINK = REPO_LINK + "/browse/%s/--/"

def lambda_handler(event, context):
    print("Processing %s record(s)" % len(event['Records']))
    messages, regions = process_records(event['Records'])
    print("Sending %s notification(s)" % len(messages))
    send_notifications(messages, regions)
    return True

def process_records(records):
    messages = {}
    regions = {}

    for record in records:
        # extract record info
        repo_name = record['eventSourceARN'].split(':')[-1]

        regions[repo_name] = record['awsRegion']
        if repo_name not in messages:
            messages[repo_name] = {}

        for ref in record['codecommit']['references']:
            # extract commit info
            commit_id = ref['commit']
            branch = ref['ref'].split('/')[-1]

            # fetch commit data
            data = codecommit.get_commit(
                repositoryName=repo_name,
                commitId=commit_id,
            )

            author = data['commit']['author']['name']
            message = data['commit']['message']

            if branch not in messages[repo_name]:
                messages[repo_name][branch] = []

            # store (author, message) tuple
            messages[repo_name][branch].append((author, message.strip()))

    return messages, regions

def send_notifications(messages, regions):

    for repo_name in messages:
        repo_url = REPO_LINK % (regions[repo_name], repo_name)
        email_text = "New commits on %s.\n\n" % repo_name
        email_text += "Repository URL: %s\n\n" % repo_url
        for branch in messages[repo_name]:
            branch_url = BRANCH_LINK % (regions[repo_name], repo_name, branch)
            email_text += "----------------------------------------\n"
            email_text += "Branch %s\n\n" % branch
            for author, msg in messages[repo_name][branch]:
                email_text += ' - %s: "%s"\n' % (author, msg)
            email_text += "\n%s\n\n" % branch_url

        topic.publish(
            Message=email_text,
            Subject='AWS CodeCommit - %s' % repo_name,
        )

The code is organized into three Python functions:

  • lambda_handler: The Main Lambda logic that will take care of logging and invoking the other functions
  • process_records: Extracts the author and messages data for each commit using the CodeCommit API
  • send_notifications: Publishes a new SNS notification for each repository, building a plain-text body

Go under Designer section. Select CodeCommit from the triggers dropdown.

We want to be notified only for the Push to existing branch event, on All branches. Optionally, we could configure additional custom data, but in our case we don’t need it.

The Configure triggers step of the Create function wizard should look similar to:

Unfortunately, the function is hard to test with fake data, as it needs to fetch real commits from CodeCommit. We will need to push at least one commit to our new repository to properly test the Lambda Function. Here is an example of the event structure it will receive:

{
    "Records": [
        {
            "eventId": "5a824061-17ca-46a9-bbf9-114edeadbeef",
            "eventVersion": "1.0",
            "eventTime": "2016-01-01T23:59:59.000+0000",
            "eventTriggerName": "my-trigger",
            "eventPartNumber": 1,
            "codecommit": {
                "references": [
                    {
                        "commit": "de884dc40ad5ce0a999bb35787f2eea72d5db091",
                        "ref": "refs/heads/master"
                    },
                    {
                        "commit": "5e6c7124d95a62625693314c3b5bca4355f5984a",
                        "ref": "refs/heads/master"
                    }
                ]
            },
            "eventName": "TriggerEventTest",
            "eventTriggerConfigId": "5a824061-17ca-46a9-bbf9-114edeadbeef",
            "eventSourceARN": "arn:aws:codecommit:us-east-1:123456789012:CloudAcademyLab",
            "userIdentityARN": "arn:aws:iam::123456789012:root",
            "eventSource": "aws:codecommit",
            "awsRegion": "us-west-2",
            "eventTotalParts": 1
        }
    ]
}

AWS recommends you test the trigger first. Although there is a test button right from the confirmation screen, and additional testing capabilities under Actions > Configure test event, it does take additional configuration.

In the next step, we will test the trigger by pushing new commits to our new repository and check the enhanced SNS notifications.

Test the CodeCommit trigger

Let’s recap what we have created so far:

  • A CodeCommit repository to host our source code
  • An SNS Topic to publish enhanced notifications
  • An SNS Subscription bound to our personal email
  • A Lambda Function to process CodeCommit events and publish enhanced SNS notifications

We are now ready to test the system and push new commits to CodeCommit.

[ec2-user@ip-10-10-10-220 CloudAcademyLab]$ pwd
/home/ec2-user/CloudAcademyLab
[ec2-user@ip-10-10-10-220 CloudAcademyLab]$ git status
On branch master

No commits yet

nothing to commit (create/copy files and use "git add" to track)
[ec2-user@ip-10-10-10-220 CloudAcademyLab]$ echo "Hello Shravan" > README.md
[ec2-user@ip-10-10-10-220 CloudAcademyLab]$ git add .
[ec2-user@ip-10-10-10-220 CloudAcademyLab]$ git commit -m "Initial commit"
[master (root-commit) bf70c86] Initial commit
 1 file changed, 1 insertion(+)
 create mode 100644 README.md
[ec2-user@ip-10-10-10-220 CloudAcademyLab]$ git push origin master
Counting objects: 3, done.
Writing objects: 100% (3/3), 225 bytes | 225.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To https://git-codecommit.us-west-2.amazonaws.com/v1/repos/CloudAcademyLab
 * [new branch]      master -> master
[ec2-user@ip-10-10-10-220 CloudAcademyLab]$

When performing these commands in succession, you will create and push a new README file. Therefore, CodeCommit will invoke our Lambda Function and pass only one commit ID. The corresponding email will contain only one commit.

As a second test, perform the following in sequence:

  • Edit your README file and save the changes
  • Commit your edits without pushing (use a descriptive message)
  • edit your README file again
  • Commit your edits
  • Run git push

The sequence above illustrates that the trigger is performed once per push, not once per commit.

If everything is correctly configured, you should receive two emails similar to the following:

Alternatively, you may want to improve the system and use Amazon Simple Email Service (SES) instead of SNS to send HTML-based emails instead of plain-text notifications.

In the event that you don’t receive an email, you can troubleshoot the problem by inspecting Lambda’s logs.