Automated backup job using AWS Lambda

Aim: To automate AWS instance backup job with some retention period.

How to Automate AMI Backups & Cleanups, using AWS Lambda (Serverless)? Is it possible to automate AWS AMI creation and deletion using Lambda funtion?

Yes, we can completely automate backup stuff in any AWS account using Lambda function. This is completely an automated thing and we can avoid human errors at any level.

How it works? And what are the prerequisites?

We have Python codes to create and remove AMI and snapshots, which execute using Lambda function with the help of CloudWatch event. I got this code from Git hub community and the original scrip creates AMIs of instances with specific TAG name.

These are modified scripts which create AMIs of all EC2 instances from any region.

Prerequisites

  • IAM user.
  • Lambda functions.
  • CloudWatch Events.

What is AWS Lambda function?

AWS Lambda is an event-driven, serverless computing platform provided by Amazon Web Services. Introduced in 2014 by AWS, Lambda simplifies the process of building smaller, on-demand applications that are responsive to events and new information.

It runs code in response to events and automatically manages compute resources required by the code. You can start a Lambda instance within milliseconds! To top it all, it supports Node.js, Python and Java, as of 2016.

Please do follow the steps pasted below for setting up the functions correctly:

I. Setup IAM user role and attach a policy to this role.

Why need of IAM role?

IAM user role is required to give proper permissions for our AWS Lambda functions for creating and removing instance backups.

You need to create a policy and attach that policy to IAM role. Please see the following steps:

Step 1: Log into AWS console.

Step 2: Click on roles, create role.

Step 3: Select AWS Lambda as the Role Type and then proceed to create a role.

Step 4: Click on create policy.

Step 5: Go to Json editor format and paste the following rule:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:*"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        },
        {
            "Effect": "Allow",
            "Action": "ec2:*",
            "Resource": "*"
        }
    ]
}

We have just created a role which gives permissions to EC2 instances and view logs in Cloudwatch.

Attach the above policy to the role we created earlier.

II. Lambda function to create backup.

We can create the Lambda function in Python. This is a modified code. The original one, which I got from Git creates backups of instances with a particular TAG value. In this code, it creates AMIs of all instances from all region.

You can check the original code from this link.

To create all instances backup, you can use the following code:

AWS AMI creation Python code

How it works?

The Python script searches instances from all region and as soon as it has the instances list, it loops through each instance and then creates an AMI of it. After creating the backup, it creates a TAG for those AMIs. This TAG is based on the retention value we have give in the code.

It creates a TAG “DeleteOn” with value based on that retention days we have given.

Create Lambda function.

Login to your AWS Management console, Go to Services, and click on Lambda under Compute.

> Click on Functions Menu on the left, and click on Create a Lambda Function
> Select Blank Function and proceed with lambda
> Give a name for it – AMIBackups
> Select Python 2.7 as a Runtime option * You’ll have to write a code next. You can use the above code.
> Select the previously created IAM role
> Click Next and Create Function

III. Lambda function to delete backup.

You can follow the same procedure to create this lambda function. Here I am sharing the modified code. This code filter today’s date and list out all AMIs for deletion.

The original code (from here) does not have this option and that removes all AMIs with Deleteon TAG is equal to today’s date and all previous AMIs with DeleteOn TAG.

You can use the following code:

AWS AMI deletion Python code

> This script filter all the AMIs with DeleteOn TAG is equal to today’s date.
> It checks that the latest daily backup succeeded then it stores every image that’s reached its DeleteOn tag’s date for deletion.
> It then loops through the AMIs, de-registers them and removes all the snapshots associated with that AMI.

That’s it!

Add CloudWatch events and schedule above jobs accordingly.

Arunlal Ashok

Linux Cloud Infrastructure / DevOps Engineer. I'm dealing Linux servers since 2012. I started this blog to share and discuss my ideas.

Always happy for an open discussion! Write to arun (@) crybit (dot) com. Check about me for more details. About this blog and our strong members, check The team CryBit.com

2 Responses

  1. Bestcustomessay says:

    Thank you for this article.

  2. Bestcustomessay says:

    Arunlal Ashok, thanks for the article post.Really thank you! Great.

Leave a Reply

Your email address will not be published. Required fields are marked *