Automated backup job using AWS Lambda

Aim: To automate AWS instance backup job with some retention period.

Automated backup job using AWS Lambda. How to Automate AMI Backups & Cleanups, using AWS Lambda (Serverless)? Is it possible to automate AWS AMI creation and deletion using Lambda funtion?

Yes, we can completely automate backup stuff in any AWS account using Lambda function. This is completely an automated thing and we can avoid human errors at any level.

Automated backup job using AWS Lambda.

How it works? And what are the prerequisites?

We have Python codes to create and remove AMI and snapshots, which execute using Lambda function with the help of CloudWatch event. I got this code from Git hub community and the original scrip creates AMIs of instances with specific TAG name.

These are modified scripts which create AMIs of all EC2 instances from any region.

Prerequisites

• IAM user.
• Lambda functions.
• CloudWatch Events.

What is AWS Lambda function?

AWS Lambda is an event-driven, serverless computing platform provided by Amazon Web Services. Introduced in 2014 by AWS, Lambda simplifies the process of building smaller, on-demand applications that are responsive to events and new information.

It runs code in response to events and automatically manages compute resources required by the code. You can start a Lambda instance within milliseconds! To top it all, it supports Node.js, Python and Java, as of 2016.

Please do follow the steps pasted below for setting up the functions correctly: Okay let’s start Automated backup job using AWS Lambda.

I. Setup IAM user role and attach a policy to this role.

Why need of IAM role?

IAM user role is required to give proper permissions for our AWS Lambda functions for creating and removing instance backups.

You need to create a policy and attach that policy to IAM role. Please see the following steps:

• Log into AWS console.
• Click on roles, create role.
• Select AWS Lambda as the Role Type and then proceed to create a role.
• Click on create policy.
• Go to Json editor format and paste the following rule:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:*"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        },
        {
            "Effect": "Allow",
            "Action": "ec2:*",
            "Resource": "*"
        }
    ]
}

We have just created a role which gives permissions to EC2 instances and view logs in Cloudwatch.

Attach the above policy to the role we created earlier.

II. Lambda function to create backup.

We can create the Lambda function in Python. This is a modified code. The original one, which I got from Git creates backups of instances with a particular TAG value. In this code, it creates AMIs of all instances from all region.

You can check the original code from this link.

To create all instances backup, you can use the following code:

AWS AMI creation Python code

How it works?

The Python script searches instances from all region and as soon as it has the instances list, it loops through each instance and then creates an AMI of it. After creating the backup, it creates a TAG for those AMIs. This TAG is based on the retention value we have give in the code.

It creates a TAG “DeleteOn” with value based on that retention days we have given.

Create Lambda function.

Login to your AWS Management console, Go to Services, and click on Lambda under Compute.

Click on Functions Menu on the left, and click on Create a Lambda Function
> Select Blank Function and proceed with lambda
> Give a name for it – AMIBackups
> Select Python 2.7 as a Runtime option * You’ll have to write a code next. You can use the above code.
> Select the previously created IAM role
> Click Next and Create Function

How to redirect non www request to www domain – AWS route 53

This is a simple workaround to redirect all non www request to www domain, from AWS Route 53. In this case your, I am considering that, your APP/Site is directly connecting from some ELB to direct App/Web port. No web-servers running on server to do the redirection.

III. Lambda function to delete backup.

You can follow the same procedure to create this lambda function. Here I am sharing the modified code. This code filter today’s date and list out all AMIs for deletion.

The original code (from here) does not have this option and that removes all AMIs with Deleteon TAG is equal to today’s date and all previous AMIs with DeleteOn TAG.

You can use the following code:

AWS AMI deletion Python code

> This script filter all the AMIs with DeleteOn TAG is equal to today’s date.
> It checks that the latest daily backup succeeded then it stores every image that’s reached its DeleteOn tag’s date for deletion.
> It then loops through the AMIs, de-registers them and removes all the snapshots associated with that AMI.

That’s it! Alright we completed the Automated backup job using AWS Lambda.

Add CloudWatch events and schedule above jobs accordingly.