Elastic Compute Cloud
EC2 is one of the most common compute services used within AWS. EC2 is arguably the first compute service that you will encounter when working with AWS. It allows you to deploy virtual servers within your AWS environment and most people will require an EC2 instance within their environment as a part of at least one of their solutions. There are a number of elements in creating your EC2 instance, which I want to break down and explain. The EC2 service can be broken down into the following components:
- Amazon machine images, AMIs,
- instant types,
- instance purchasing options,
- user data,
- storage options and
Let’s look at each of these individually.
These are essentially templates of pre-configured EC2 instances which allow you to quickly launch a new EC2 instance based on the configuration within the AMI. This prevents you from having to install an operating system or any other common applications that you might need to install on a number of other EC2 instances. From a high level perspective an AMI is an image baseline that will include an operating system and applications along with any custom configuration.
AWS provides a large number of AMIs covering different operating systems from Linux to Red Hat to Microsoft Windows among others. When configuring your EC2 instance, selecting your AMI is the first configuration choice you’ll need to make. You can also create your own AMI images to help you speed up your own deployment.
For example you would start with selecting an AWS AMI, let’s say a Linux server. And then once it is up and running you may then need to install a number of your own custom applications and make specific configuration changes. Now if you needed another server to perform the same functionality, you could go through the same process of selecting a Linux AWS AMI, and again manually installing the applications and making your configurations. Or once you have made those changes on the first instance you can then simply create a brand new AMI or template of that instance with all the applications installed and configurations already made. Then if you need another instance of the same configuration all you would need to do is to select your custom AMI as the base image for your instance and it will launch with Linux server, your custom applications already installed and any configurations already made. As you can see this has many benefits and certainly comes in useful when implementing auto scaling.
In addition to both AWS managed and your custom managed AMIs, you could also select an AMI from the AWS marketplace. The AWS marketplace is essentially an online store that allows you to purchase AMIs from trusted vendors like Cisco, Citrix, Alert Logic et cetera. These vendor AMIs may have specific applications and configurations already made, such as instances that are optimized with built-in security and monitoring tools or contained database migration systems. Lastly community AMIs also exists which are a repository of AMIs that have been created and shared by other AWS members.
Let’s now take a look at instance types, once you have selected your AMI from any of the different sources discussed above, you must then select an instance type. An instance type simply defines the size of the instance based on a number of different parameters, these being
- ECUs: This defines a number of EC2 compute units for instance
- vCPUs: This is the number of virtual CPUs on the instance.
- Physical Processor: This is the process speed used on the instance.
- Clock Speed: This is the clock speed in gigahertz
- Memory: The amount of memory associated.
- Instance Storage: This is the capacity of the local instance store volumes available.
- EBS optimized available: This defines if the instance supports EBS optimized storage or not.
- Network Performance: This shows the performance level of rate of data transfer.
- IPV6 support: This simply indicates if the instance type supports IPV6.
- Processor Architecture: This shows the architected type of the processor.
- AES-NI: This stands for advanced encryption standard new instructions and it shows if the instance supports it for enhanced data protection.
- AVX: This indicates if the instance supports AVX which is advanced vector extensions, which are primary used for applications focused on audio and video, scientific calculations and 3D modeling analysis.
- Turbo: which shows if the instance supports intel turbo boost and AMD turbo core technologies.
The key parameters here to primarily be aware of for general usage of an EC2 instance, could be summarized as vCPUs and memory, instant storage and network performance. But obviously this really depends on your actual usage and application. Having this flexibility of variant instances allows you to select the most appropriate size or power of an instance that you need for optimal performance within your applications.
These different instance types are categorized into different family types that offer distinct performance benefits, which again helps you to select the most appropriate instance for your needs. Within each of these instance families you will have a range of instant types with varied CPU, memory, storage and network performance et cetera.
These instance families can be summarized as follows.
Instance purchasing options
On-demand instances: These are EC2 instances that you can launch at any time and have it provisioned and available to you within minutes. You can use this instance for a shorter time or for as long as you need before terminating the instance. These instances have a flat rate and is determined on the instance type selected and is paid by the second. On-demand instances are typically used for short term uses where workloads can be irregular and where workload can be interrupted. Many users of AWS use on-demand instances within their testing and development environments. And when you stop or terminate your on-demand instance you’ll stop paying for the compute resource.
Reserved instances: Reserved instances allow you to purchase a discount for an incidents type with set criteria for a set period of time in return for a reduced cost compared to on-demand instances. This reduction can be as much as 75%. These reservations against instances must be purchased in either one or three year time frames. Further reductions can be achieved with reserved instances depending on which payment methods you select. There are three options available to you, firstly all upfront. The complete payment for the one or three year reservation is paid. And this offers the largest discount and no further payment is required regardless of the number of hours the instance is used. Partial upfront, here a smaller upfront payment is made and then a discount is applied to any hours used by the instance during the term. And finally no upfront, no upfront or partial payments are made and the smallest discount of the three models is applied to any hours used by the instance. Reserved instances are used for long-term predictable workloads allowing you to make full use of the cost savings to be had when using compute resources offered by EC2.
Scheduled instances: Scheduled instances, these are similar to reserved instances and the fact that you pay for the reservations of an instance on a recurring schedule, either daily, weekly or monthly. For example you might have a weekly task that is scheduled that performs some kind of bulk processing for a number of hours at the same time every week. With scheduled instances you could set up a scheduled instance to run during that set timeframe once a week. And this prevents you for having to use the on-demand instances which would incur a higher price. You should note that when using scheduled instances but even if you didn’t use the instance you would still be charged. This allows you to provision instances for scheduled workloads that are not continuously running. Which is where a reserved instance would be the preferred choice.
Spot instances: Spot instances allows you to bid for unused EC2 compute resources, however your resource is not guaranteed for a fixed period of time. To you to spot instance you must bid higher than the current spot price which is set by AWS. And this spot price fluctuates depending on supply and demand of the unused resource. If your bid price for an instance type is higher than the spot price, then you’ll purchase that instance. But as soon as your bid price becomes lower than the fluctuating spot price, you will be issued a two-minute warning before the instance automatically terminates and is removed from your AWS environment. The bonus for spot instances is that you can bid for large EC2 instances at a very low cost point saving a huge amount on cost. Due to the nature of how the instances can be suddenly removed from your environment, spot instances are only useful for processing data and applications that can be suddenly interrupted. Such as batch jobs and background processing of data.
On-demand capacity reservations: Capacity reservations allows you to reserve capacity for your EC2 instances based on different attributes. Such as instance type, platform and tenancy et cetera. Within a particular availability zone for any period of time. This ensures that you always have the available number of instances you require within a specific availability zone immediately. This capacity reservation could also be used in conjunction with your reserved instances discount providing you additional savings.
EC2 tenancy and this relates to what underlying host your EC2 instance will reside on. So essentially the physical server within an AWS data center.
Shared Tenancy: Shared tenancy, this option will launch your EC2 instance on any available host with the specified resources required for your selected instance type. Regardless of which other customers and users also have EC2 instances running on the same host, hence the share tenancy name. AWS implement advanced security mechanisms to prevent one EC2 instance from accessing another on the same host. How the security is applied and operated is out of scope of this post and it is maintained by AWS themselves.
Dedicated Tenancy: Dedicated tenancy, this includes both dedicated instances and dedicated hosts. Dedicated instances are hosted on hardware that no other customer can access. It can only be accessed by your own AWS account. You may be required to launch your instances as a dedicated instance due to internal security policies or external compliance controls. Dedicated instances do incur additional charges due to the fact you are preventing other customers from running EC2 instances on the same hardware and so there will likely be unused capacity remaining. However the hardware might be shared by other resources you have running in your own account.
Dedicated host: A dedicated host is effectively the same as dedicated instances. However they offer additional visibility and control, how you can place your instances on the physical host. They also allow you to use your existing licenses, such as PA-VM license or Windows Server licenses et cetera. Using dedicated hosts give you the ability to use the same host for a number of instances that you want to launch and align with any compliance and regulatory requirements.
If you don’t need to address any compliance or security issues that require dedicated tenancy, then I recommend using shared tenancy to reduce your overall costs.
User data, during the configuration of your EC2 instance there is a section called user data. Which allows you to enter commands that will run during the first boot cycle of the instance. This is a great way to automatically perform functions upon boot, such as to pull down any additional software you want installing from any software repositories you may have. You could also download and get the latest OS updates during boot. For example you could enter
yum update -y, for a Linux instance which will then update its own software automatically at the time of boot.
Storage options, as a part the configuration when setting up an EC2 instance, you are asked to select and configure your storage requirements.
Selecting storage for your EC2 instance will depend on the instance selected, what you intend to use the instance for and how critical the data is. Storage for EC2 can be classified between two distinct categories, persistent storage and ephemeral storage. Persistent storage is available by attaching elastic block storage EBS volumes. And a ephemeral storage is created by some EC2 instances themselves using a local storage on the underlying host known as instance back storage. Let’s look at each of these storage options in greater depth.
EBS volumes are separate devices from the EC2 instance itself. And so it’s not physically attached like ephemeral storage is. EBS volumes are considered network attached storage devices which are then logically attached to the EC2 instance via the AWS network. This principle is not dissimilar to attaching an external hard disk to your home laptop or PC. Where the external hard disk represent your EBS volume and your PC represents your EC2 instance. The data on EBS volumes are automatically replicated to other EBS volumes within the same availability zone for resiliency which is managed by AWS.
You can disconnect an EBS volume from your EC2 instance and the data will remain intact. Allowing you to reattach it to another EC2 instance if required. You can also implement encryption on these volumes if needed and take backup snapshots of all the data on the volume to S3. EBS volumes can be created in different sizes again with different performance capabilities depending on your requirements.
Ephemeral storage or instance backed storage is the storage that is physically attached to the underlying host on which the EC2 instance resides on. Looking back at our previous example, this would be similar to your own laptop or PC’s hard disk. There is a difference here though, with AWS EC2 instances as soon as the instance is stopped or terminated all saved data on a ephemeral storage is lost.
If you reboot your instance then the data will remain but not if you stop it. Therefore if you have data that you need to retain it is not recommended that you use instance backed storage for this data. Instead use EBS volumes for persistent data storage. Unlike EBS volumes you are unable to detach ephemeral instance store volumes from the instance.
Security, security is fundamental with any AWS deployment. As so I just want to highlight a couple of points relating specifically to EC2 security. Firstly during creation of your EC2 instance you will be asked to select a security group for your instance.
A security group is essentially an instance level firewall allowing you to restrict both ingress and egress traffic by specifying what traffic allowed to communicate with it.
You can restrict this communication by source ports and protocols for both inbound and outbound communication. Your instances are then associated with this security group.
At the very end of your EC2 instance creation, you will need to select an existing key pair or create and download a new one.
But what is a key pair and what is it used for?
A key pair, as the name implies, is made up of two components, a public key and a private key. The function of key pairs is to encrypt the login information for Linux and Windows EC2 instances. And then decrypt the same information allowing you to authenticate onto the instance. The public key encrypts data such as the username and password. For Windows instances, the private key is used to decrypt this data allowing you to gain access to the login credentials including the password. For Linux instances the private key is used to remotely connect onto the instance via SSH. The public key is held and kept by AWS, and the private key is your responsibility to keep and ensure that it is not lost or compromised.
So going back to when you create your EC2 instance and a new key pair. You’re given the opportunity to download the key pair, once you have done this you must keep that file safe until you’re ready to log on to the associated EC2 instance. It’s worth noting that you can use the same key pair on multiple instances to save you managing multiple private keys. Do bear in mind however should the private key become compromised access could be gained to all the instances where that key pair was used.
Once you have authenticated to the EC2 instance the first time, you can set up additional less privileged access controls such as local windows accounts allowing other users to connect and authenticate to or even utilize Microsoft Active Directory. One final point regarding security on your EC2 instance it is your responsibility to maintain and install the latest OS and security patches released by the OS vendor as dictated within the AWS shared responsibility model.