Lab Infrastructure Automation using AWS CloudFormation

In this post I would be talking about using AWS CloudFormation for the automation of various AWS resources. This is a project in which we provide users with hands on practice for different AWS services by facilitating them with step by step instructions to perform various labs. When the user starts the lab in the backend all the AWS resources needed for that particular lab are allocated using the AWS CloudFormation.

What is AWS CloudFormation?

AWS CloudFormation is an AWS service which allows you to create a collection of related AWS resources and provision and manage them in an orderly and predictable manner. AWS CloudFormation allows you to deploy and update various resources like compute, database and many other, by creating a template which would describe all the resources you want and CloudFormation will take care of provisioning and configuring them for you. It eliminates the need of creating and configuring resources individually as well as you don’t have to worry about what resources are dependent on what other resources.

AWS CloudFormation Templates

In the previous section we talked about deploying resources using a template. Here, is a brief explanation about the templates, CloudFormation templates are JSON or YAML-formatted text files which comprises of five types of elements:

An optional list of template parameters: Template Parameters are used for getting user input at the time of stack creation. The parameters contain a list of attributes which have its values. You can also add specific constraints against the input values if you wish to.

An optional list of output values: With the Outputs you can conveniently present the stack’s key resources such as the Public IPv4 address of your EC2 Instance to user via AWS Management console or command line tool.

An optional list of Mappings used to look up static configuration values: With the Mappings section you can match a key with its corresponding set of named values. For instance, if you want Mappings for a particular region you can specify region name as key and values which would be specific for each region.

The list of AWS resources and their configuration values: In the Resources section comes all your AWS resources that you want to include in your stack and there specific configurations if there are any.

A template file format version number: The AWSTemplateFormatVersion section which is an optional section is used to identify the capabilities of the template.

AWS CloudFormation uses these templates as blueprints for building your AWS resources. We can create stack from AWS management console as well as using AWS Command Line Interface. A stack is a collection of AWS resources that you can manage as a single unit. Here in the next section I would be showing how one can configure AWS CLI and use it to create stack using AWS CLI commands.

Sample Template:

Template1.yaml

AWSTemplateFormatVersion: 2010-09-09
Parameters:
 KeyName:
   Description: Name of existing EC2 KeyPair to enable SSH access   into the Server
   Type: AWS::EC2::KeyPair::KeyName
Mappings:
 RegionMap:
  us-east-1:
   AMI: ami-1853ac65
  us-west-1:
   AMI: ami-bf5548df
  eu-west-1:
   AMI: ami-bf5548df
Resources:
 VPC:
  Type: AWS::EC2::VPC
  Properties:
   CidrBlock: 10.0.0.0/16
   EnableDnsHostnames: true
   Tags:
   - Key: Name
   Value: Lab VPC
 
 WebServerSG:
  Type: 'AWS::EC2::SecurityGroup'
  Properties:
   GroupDescription: Enable SSH access via port 22 and 80
   VpcId: !Ref VPC
   SecurityGroupIngress:
    - IpProtocol: tcp
      FromPort: '22'
      ToPort: '22'
      CidrIp: 0.0.0.0/0
    - IpProtocol: tcp
      FromPort: '80'
      ToPort: '80'
      CidrIp: 0.0.0.0/0
 
 EC2Instance:
  Type: 'AWS::EC2::Instance'
  Properties:
   InstanceType: t2.micro
   ImageId:
    Fn::FindInMap:
     - RegionMap
     - !Ref AWS::Region
     - AMI
   SecurityGroupIds:
    - !Ref WebServerSG
   Tags:
    -Key:"Name"
     Value: Lab EC2
   KeyName: !Ref KeyName
Outputs:
 Website:
  Description: Public DNS for EC2 EC2Instance
  Value: !Sub 'http://${EC2Instance.PublicDnsName}'

Code above is small template whose explanation is follows:

Parameters: EC2 Key Pair is taken as user input when stack is being created.

Mappings: The Mappings section maps AMI IDs of EC2 instance for the particular region, for example if region is us-east-1, AMI with id ami-1853ac65 will be launched.

Resources: In the resources section, VPC with CIDR block 10.0.0.0/16 is created and its name is Lab VPC. A Security Group with name WebServerSG which allows access via port 22 and 80 is created in Lab VPC. Next resource in resources section is EC2 Instance. An EC2 Instance of type t2.micro with security group WebServerSG and name Lab EC2 is launched.

Outputs: The Outputs section, gives public URL of the EC2 instance as output.

Install and Configure AWS Command Line Interface

To create stack using AWS CLI on Windows, first you need to install AWS CLI version 2 and configure it.

Step 1:

For Windows(64-bit) download the AWS CLI MSI installer from the link below:

https://awscli.amazonaws.com/AWSCLIV2.msi

Run the downloaded MSI installer and follow on-screen instructions.

Step 2:

To configure the AWS CLI you need Access Key ID and Secret Access Key. To generate Access Key ID and Secret Access Key login to your AWS Console. In the navigation pane choose Users. Click on Security Credentials tab and in Access Keys section click on Create New Access Key and download the CSV file.

Step 3:

Next step, in the command prompt write command aws configure and press enter which is shown below.

AWS Access Key: AKIAIOSFODNN7EXAMPLE 
AWS Secret Access Key ID: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY 
Default region name: us-east-1 
Default output format: json 

In the above code enter the details of Access Key, Secret Access Key ID from the csv file you downloaded, default region name will be the according to the region which you want to be selected by default and Output format will be json or yaml format in which you want the output to be formatted in.

Create Stack using AWS CLI

In order to create a stack which has a Parameter of entering key pair, the command is as follows:

aws cloudformation create-stack --stack-name RDS --template-body file://C:\ AWS\Template1.yaml –parameters ParameterKey=KeyName,ParameterValue=demo

The above command creates a stack with name YourStackName and C:\AWS\Template1.yaml is the path to your file. On the successful creation of the stack you will get Stack Arn in the output.

Delete Stack using AWS CLI

To delete the stack that you created the CLI command is:

aws cloudformation delete-stack --stack-name YourStackName

Final Outcome of the Project

In this part I would be showing you the final result of the project and how users can use it to have hands-on practice for the various AWS resources.

Home Page

Once the user is logged-in, the homepage provides them with various labs that are available. The users can choose which lab they want to perform with help of the list of labs which has a small description of which AWS service is used in that particular lab.

Homepage of the AWSLAB

Lab Page

Once the users choose which lab they want to perform, by clicking on View Lab the instructions page for that particular lab will be displayed. The instructions page provides users with the each and every minute step they have to follow in order to complete the from clicking on the Start Lab button to start the lab to clicking on the End Lab button to complete the lab. In this page the users have to carefully follow every step stated in order to complete the lab. Before starting the lab, the users can have a quick glance on what task they have to perform from the task menu given on the right side of the page.

Instructions page for lab of Hosting WordPress

What happens in the backend?

Now once the user clicks on the Start lab button, what exactly happens in the backend is a CloudFormation template for that particular lab will be launched, which will have all the necessary resources created for the execution of that particular lab. For instance, talking about the lab of Hosting WordPress on EC2 in the backend necessary resources like security group, EC2 instance will be automatically created by the template and the users don’t have to care about all these and just have to follow the instructions to setup WordPress.

Snapshot of the resources been created in the backend

Once the users click on the End Lab button, at that time the entire resources created in the backend will be deleted.

The created resources been deleted

Summary

In this post I have talked about AWS CloudFormation service and how we can use the service to automate deployment of various resources and use of AWS CLI to create, delete stacks using CLI commands and how the we have implemented the project.