Automating AWS infrastructure using SaltStack

The infrastructure management and provisioning is moving to the next big thing: Infrastructure As A Code. This post is an example of how infrastructure as a code can work.I am using SaltStack and Amazon Web Services.

This tutorial is taken from SaltStack For DevOps book, a fruit of long hours of work and self-learning. You can visit the website for more information.

Using Salt Cloud With Amazon Web Services (AWS)

Actually, working with ec2 virtual machines is almost similar to working with Linode VMs, I am telling this because my first experience of provisioning infrastructure using code and configurations was with Linode virtual machines.

AWS Provider Configuration

First thing we are going to do here is to check our key pairs.

If you have multiple key pairs, you are going to use one of them in the Salt Cloud Provider configuration, that’s why you should search for your key name.

You can use this link to list all of your key pairs:

If you are using a different region, change the eu-west-1 by the region you are considering. In the next example we are using us-east-1 as the region:

To check which key name corresponds to the used key pair, use this command (after installing aws kit):

This will help you match your key pair to the fingerprint and allows you to check the name of your key.

In this example our key is called kp and it is located under:

Now go and get your security credentials for accessing your ec2 instances, you can find the id here:

You can not get your key from there, but normally you would have kept this in a secret place, if you lost this, you should generate two other key pairs.

You should also know other things like the name of the security group you want to use and your ssh_username:

  • Amazon Linux > ec2-user
  • RHEL > ec2-user
  • CentOS > ec2-user
  • Ubuntu > ubuntu
  • etc ..

Another thing to set up is the ssh_interface that could have two different values:

  • private_ips > The salt-cloud command is run inside the EC2
  • public_ips > The salt-cloud command is run outside of EC2

This is a generic example of a provider configuration using private_ips:

With a public_ips, we will have something similar to this configuration:

Please note two things:

  • Previously, the suggested provider for AWS EC2 was the aws provider. This has been deprecated in favor of the ec2 provider.
  • The provider parameter in cloud provider definitions was renamed to driver (since the version 2015.8.0).

AWS Profile Configuration

Let’s set up the profile to provide more “ec2-specific” configuration options.

In the profile configuration we should provide the provider, the image id, the size of the instance and the ssh_username which is Ubuntu since out image is also based on Ubuntu.

If we want to add a volume (10Gb), we can do it like this:

Suppose we want to add two other volumes while choosing the iops (Input/Output per second), we could add a similar configuration to the next one:

Note that to use an EBS optimised ec2 instance we may use:

We can also add tags to our new instance and it will be applied to all ec2 instances created using this profile:

We have the possibility to force grains synchronization by adding:

One thing that I automate is setting my own configurations, like the .vimrc file, you can automate things like this by adding a script that will be executed:

Network configuration is also accessible using Salt Cloud, here is an example where the primary IP address is the private address and the ec2 instance will have a public IP (not an elastic IP) with subnet id and a security group id:

If you prefer the EIP (Elastic IP):

We want to delete root volume when we destroy our ec2 instance:

When a machine is terminated, we want to delete all not-root EBS volumes for an instance:

Now we have a functional ec2 profile:

When we want to create a similar profile to the last one but we would like to change one or two options, we can use extends like in the following example:

> The last example was just provided to help you understand the use of extends, it was not tested.

Using Salt Cloud To Automate AWS EC2 Creation

Starting a private ec2 instance can be done like this:

and launching a public one can be done using this command:

Like we have done when we saw how to use Salt Maps with Linode, nothing is different from using it with aws:

and then we can start our ec2 instances using:

Salt Cloud allows getting, setting and deleting tags after launching the ec2 instance using the instance name (or the instance id):

It allows also to rename the machine:

To enable termination protection, Salt Cloud can be used like in the following command:

Other options are available:

using Salt Cloud from command line allows adding volumes and specific configurations like choosing a snapshot to create a volume:

Creating a simple volume in a specific zone:

Adding size:

Choosing a snapshot:

Selecting the type (standard, gp2, io1 ..etc):

Detaching a volume then deleting it:

This tutorial is taken from SaltStack For DevOps book, a fruit of long hours of work and self-learning. You can visit the website for more information.

You may also like...