 |
From Learn Amazon Web Services in a Month of Lunches by David Clinton
Have you been wondering what EC2 is, what it does, and what you can do with it? This article lays it all out for you! |
Save 37% on Learn Amazon Web Services in a Month of Lunches. Just enter code fccclinton into the discount code box at checkout at manning.com.
EC2 (short for “Elastic Compute Cloud”) is a hugely successful service at the very core of the AWS platform that allows you to effectively rent units of compute power, storage space, and network connectivity running within AWS’s vast infrastructure. If you need to provide a web-based service to your customers, users, or readers – anything from a community wiki to a fully integrated, multi-level commercial mobile application – you can use those resources to create a virtual machine (called an “instance” in AWS circles) that, to the outside world, will act and feel just like the real thing. There are even different instance types that you can explore while you discover more about the AWS cloud. You can read this guide to changing EC2 instance types to learn more. For us, as we take our first steps in the AWS cloud, what’s really nice is that the servers require so little time and effort to launch, so if we make a mistake or somehow mess up some important configuration files, we can easily pull the plug on an instance and have a replacement up and running – usually in less than a minute.
If you don’t already have an AWS account, head over to aws.amazon.com right now and sign up. While you will have to give them your credit card information, you won’t be charged anything unless you actively launch resources (by default, nothing is running on a new account – you are only responsible for the services you activate). Not only that, all the resources we’re going to launch in this article will fall into Amazon’s generous ‘free’ tier that’s available to all accounts in their first year. So don’t worry about charges.
Security
NOTE: since this AWS account will be associated with your credit card, you should exercise normal precautions, including using a good password made up of both upper and lowercase characters, at least one number, and one non-alphanumeric character
Launching an AWS Instance
We’re going to begin the process of starting up a new virtual Linux server (don’t be concerned if you haven’t got any previous Linux experience; it’ll be a pretty shallow learning curve). Those interested in a Linux VPS can get help from IO Zoom with regards to hosting one – their plans suit all budgets and come with a range of features that will enhance your experience with a virtual private server including protection through attacks, free backups, SSD technology, and more. Why Linux? Because it can be freely adapted to fill just about any role, and, in most cases, the operating system itself is available for free. And besides, the last time I checked with The Cloud Market (thecloudmarket.com/stats#/totals), more than 92% of all EC2 instances launched are running one flavor or another or Linux. You wouldn’t want to be antisocial, would you?
Although we’ll be focusing on accessing and managing AWS-based Linux instances (along with all the Linux-specific skills you’ll need to keep up), there’s nothing to prevent you running Windows in your EC2 instance – even in the Free Tier.
Once your account has been activated, point your web browser to the AWS console (http://console.aws.amazon.com) and click on the EC2 icon (see figure 1).
Figure 1 Part of the AWS console displaying key services grouped by theme…with compute and storage services highlighted.
There’ll be a big blue button marked *Launch Instance* on the EC2 Dashboard. Click it. Scroll down just a bit until you see an Ubuntu Server 16.04 LTS volume type and click on the blue “Select” button (visible in figure 2) to the right. That 16.04 version will probably be replaced by Ubuntu Server 18.04 sometime in late 2018. I use Ubuntu, but, with a few minor variations, everything we’ll do here will work just as well using any flavor of Linux.
Figure 2 Select an Ubuntu Amazon Machine Image (AMI) – note also some of the alternative operating systems that are available as AMIs.
On the Choose an Instance Type page that will follow, the General Purpose t2.micro “Free tier eligible” type will probably be pre-selected. Because our server is going to be exposed to only very light use, that will give us plenty of power and cost us the least, so make sure it’s selected. For an instance profile called “micro,” the t2 packs quite a wallop; you might not want to use a computer built with such specs for your high-end gaming, but I was pleasantly surprised just how well this little fellow can handle the demands of a relatively light website workload.
Now let’s click on the “Next: Configure Instance Details” button at the bottom-right. The Configure Instance Details page lets us customize the network neighborhood within which our instance will live. Edits are necessary for some for the more advanced scenarios, but for now, the preset values for settings like VPC, subnet, and shutdown will work well for us.
We’ll click on “Next: Add Storage” to move on to Add Storage. The default Storage value we’re offered – an eight gigabyte SSD (solid state drive) to host our base operating system – is also fine. In fact, all the software that will make up our Linux server will probably use less than half of the space we’ve given it. And, unless we make some really bad choices later about retaining old log data or where to store large application and database files, eight GB should remain more than enough into the foreseeable future.
The Tag Instance page, which comes up next, gives us the opportunity to assign an identifying name to our instance. As we’ll learn later in the book, tags can be really useful for quickly finding specific objects from among long lists of resources. For now, though, feel free to type any useful identifying name (like, say, “MyInstance”) – or nothing at all – in this field.
Next we have the “Configure Security Group” page whose settings should protect us from attack or, in other words, define exactly who gets access to each of our project’s resources. AWS Security Groups work much the same way as the various kinds of network firewalls which you might already be familiar with, but you can also think of them as filters that only allow objects of just the right size and weight through to the other side.
I’ll illustrate how this works with a simple – and important – example. The way things are set by default, anyone with a copy of your private authentication key (which we’ll discuss soon) will be able to get into your server from anywhere in the world. Assuming you’re careful not to carelessly expose your private key, that’s not really quite as scary as it sounds. But restricting SSH access to only those requests coming from behind the unique public IP address used by your own desktop PC is a good idea and something AWS strongly recommends.
Go ahead and click on the drop down menu under the “Source” column marked “Anywhere” and select “My IP” instead. AWS will automatically populate the field with the public IP address given to you by your Internet provider. A custom IP address could also be specified if, say, you wanted to give access to members of your team toiling away in a remote office.
Got multiple IP addresses?
TIP: If you need access from machines using more than one IP address, you can create multiple security group rules, one for each address. Also, if the IP address you use is dynamic (meaning, it changes from time to time – usually whenever your modem/router is rebooted), then you’ll have to update the value in your Security Group before you’ll be able to access the instance.
Let’s make one more edit to the Security Group. Since we’re building a web server – whose purpose is, after all, to serve pages using the HTTP protocol – we’ll need to create a new rule to open up the default HTTP port (80 – the port used by most unencrypted browser-based web traffic). Click once on “Add Rule” and then on the drop down marked “Custom TCP Rule”. Select HTTP from the list that will appear. Notice in figure 3 how the value of “Port Range” for this rule is automatically set to 80 and “Source” is Anywhere (i.e., 0.0.0.0/0 – meaning any users anywhere on the Internet will have access to our content). Besides HTTP, you can choose from a long list of common protocols, including various email and database standards.
Figure 3 AWS Security Group configuration page
Let’s click “Review and Launch” and then “Launch” – although, technically, nothing will actually happen until after one final step: choosing a key pair. If you can contain your curiosity for just a paragraph or two, I’ll explain what that’s all about. In the meantime, let’s select “Create a new key pair” and give it a name – perhaps something that reflects the project like: book-sales-site-key. I’ll just use “keyname”. Click “Download Key Pair” to download the key to your local computer and a dialog will pop up asking you where you want to save your new file. I would suggest that you save it to your user’s home directory, so that you’ll have easy access to it right away whenever you open your terminal. Note that the new key has a .pem extension.
When you’re done with that, click “Launch Instance” and you’re done. Your first EC2 instance! It will take a minute or two before the instance will be ready for action, but in the meantime, the quickest way to get to the EC2 instances dashboard – with all of its important login information – is by clicking on the “View Instances” button that will appear. Once you’re on the Instances dashboard (see figure 4), you will know the instance is ready when you see its public IP address appear in the Instance Description tab of the dashboard.
Figure 4 EC2 Instance details page which displays important information – including the instance’s public IP address – and links to instance-related resources.
Since this website won’t have the usual human readable URL (like google.com), the IP (107.22.95.59, in this case) is the address you’ll use to log in to the instance itself, and to load the web page we will soon create.
Accessing your AWS instance
While your instance is now active and running, it’s not doing anything on behalf of humankind. If you want to get it working for you, you’ll have to actually log in and configure some software. The way you gain access to a remote server like those on AWS will depend on two main factors: what operating system is running on the server; and what operating system is running on your local desktop machine.
Since our EC2 server is uses Linux, it’s expecting our visit by way of the secure remote management tool – SSH.
Using SSH (the “Secure SHell” protocol)
You may be surprised to learn that unencrypted data moving between computers is easily visible to just about anyone with physical access to any point along the network. To ensure that the private data or passwords that you exchange remain safe, you absolutely must employ some kind of encryption tool – like key pairs – to effectively scramble your data while it’s in transit and then reliably decode it once it reaches the other end. The OpenSSH program is a tool for just that purpose, which has proven so popular that even Microsoft has begun supporting SSH natively within Windows.
A key pair is a pair of text files containing long strings of plain-text characters. Here’s an (inactive) example of the public half of a pair:
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKu2XRjOEtABbkatlP S7/3CkqSo5tdhuMOKVgJGRpgIBe/riiby6PCTc53maT28hD5AfM1yV1lba1iUEZyh3yvaoCuC3jZ3+kiTyTYm/dpZ5eInMMAAj5DFhev4sTFfzWClo3BuvOS2U9/5fRPTz8YPYuuYnTr+84Kta28Yd6/IqDb7hz6UaMNv+Z36MhG0NOXJhRBBYiyPYQFd9nxE+cyMZ+FmGjMz6y8Q8dS4v5f6cnz8V3mbSFdZyn3OSMD0DxDxEu47GRNUYw+i9x7DkAKc/DKNN94RPD3bQm1W9koNL0MQNHgp4LJfvMKbo1852FNBbd1OIPTVBvlLjH39wbF root@container
The strings represent digital keys whose contents are, for all intents and purposes, impossible to guess. The file designated as the public part of the pair can be copied to the remote computer you’d like to later log in to. The existence of both halves of the pair on the two computers assures OpenSSH (and its users) that it’s safe to allow the session to begin.
For Amazon instances, AWS will embed the public half of a pair on your instance as it starts up. You will only need to safely download and store the private key on any computer from which you’d like to connect. Assuming you don’t already have one, AWS will also help you create a new key pair.
Accessing from a Linux or OS X machine
Now it’s time to open a terminal window so we can connect to our new AWS instance. Depending on the Linux distribution you’ve got running on your machine, this might work through the CTRL+ALT-t keyboard combination or through a desktop menu. Type “ls” to list the contents of your current directory, confirming that your key pair .pem file is there.
ls
If the key is not there, you’ll have to locate it. On a Linux machine, predictably enough, you can do that using the *locate* tool.
sudo updatedb locate keyname.pem
What is this “sudo” thing-y?
For many practical reasons, using an operating system account that enjoys full administration powers is unnecessarily risky, as anyone who find himself logged in (including work colleagues wandering by a momentarily unattended computer) will have automatic complete control. On the other hand, restricting yourself to a non-administration account will make it pretty much impossible to get anything done.
Many flavors of Linux solve this problem by providing selected accounts with admin authority, which, under most circumstances is purely theoretical, but can be invoked when necessary by prefacing a command with the word “sudo” (like the example just above). Once you confirm your identity by providing your password, your command will be treated as though it was issued by the root user.
Want to see how it works? Try running the above command without sudo.
Once you find your key, you’ll need to tighten up the file’s permissions (meaning, set it so that only its owner – you – has access of any kind). Without this, your AWS Ubuntu server simply won’t let you log in. On either Linux or OS X, you can change the file permissions using a basic Unix command line program – called Change Mode – that’s part of the basic Bash environment on all Unix-based systems. Substitute the actual name you gave your key for “keyname”:
chmod 400 keyname.pem
Make a note of the path to the file (probably something like /home/yourname/Downloads), as you’ll need it later.
We can now log in to our instance using our key pair (using the full path you found through locate, if it’s not in the current directory), the username “ubuntu”, and the IP address you just saw on the EC2 dashboard.
ssh -i keyname.pem ubuntu@54.152.9.184
Here (figure 5) You will be asked to authenticate the host by typing the word “yes” – all three letters of the word “yes”. I remember a very smart DevOps professional turning to me for help after being unable to log in to a remote server after ten or fifteen frustrating minutes because he kept responding with just a “y”.
Figure 5 SSH login screen to an AWS EC2 instance
If you received an error telling you something like “openssh not installed” then, obviously, you’ll have to install it. On an Ubuntu, Mint, or Debian system, do that with:
sudo apt install openssh-server sudo systemctl start ssh
On some older Ubuntu systems, that last command will be “sudo service ssh start”.
On CentOS Linux, use Yum instead:
sudo yum -y install openssh-server openssh-clients chkconfig sshd on service sshd start
Accessing from a Windows machine
While Windows 10 now allows you to install a Linux BASH interface, through which you can use SSH as we saw above, you might prefer to work with the Windows SSH client, PuTTY. You’ll first need to download the free PuTTY remote access program from its web site:
http://www.chiark.greenend.org.uk/~sgtatham/putty/
Next, you will convert the .pem key that you generated in AWS, to PuTTY’s .ppk format. From the Start menu, click on All Programs > PuTTY > PuTTYgen.
For “Type of key to generate”, I would choose the most commonly used encryption algorithm, SSH-2 RSA. Most of the others are just there for backwards compatibility. Then click on the “Load” button, make sure that “All Files (*.*)” will be displayed, and select your .pem file, then click “Save private key” to start the conversion (make sure you use the same name for your key that you used for the .pem version). You don’t need to add a passphrase if you don’t want to, but getting your local machine to prompt you for a passphrase each time to initiate a remote SSH login does add an extra layer of security.
You can now open the PuTTY program itself through All Programs > PuTTY > PuTTY.
Since the instance we’re running on AWS is an Ubuntu server, the Host Name will be ubuntu. Choose SSH for Connection Type and make sure that your port is set to 22 (which is the default port used by SSH connections). You will select your .ppk file through the Category pane on the left. Click to expand Connection, and then SSH, then click on Auth. From there, browse to the folder where you saved your .ppk file and select it. Finally, click Yes and your session should open.
Building an Ubuntu Linux web server
Installing the software
If everything went as planned, your terminal should now be connected to your Amazon Ubuntu server instance and you’ll be all set to begin getting things ready to host your website. That means it’s time to start installing the software we’ll be using. First, we’ll update the software repository. Type “sudo apt update”; here is a sample of the output you’ll get:
sudo apt update Ign http://us-east-1.ec2.archive.ubuntu.com trusty InRelease Get:1 http://us-east-1.ec2.archive.ubuntu.com trusty-updates InRelease [65.9 kB] Get:2 http://us-east-1.ec2.archive.ubuntu.com trusty-backports InRelease [65.9 kB] Hit http://us-east-1.ec2.archive.ubuntu.com trusty Release.gpg Hit http://us-east-1.ec2.archive.ubuntu.com trusty Release Get:3 http://security.ubuntu.com trusty-security InRelease [65.9 kB] Get:4 http://us-east-1.ec2.archive.ubuntu.com trusty-updates/main Sources [383 kB] Get:5 http://us-east-1.ec2.archive.ubuntu.com trusty-updates/restricted Sources [5,360 B] [...] Get:36 http://us-east-1.ec2.archive.ubuntu.com trusty/universe Sources [6,399 kB] Get:37 http://us-east-1.ec2.archive.ubuntu.com trusty/multiverse Sources [174 kB] Hit http://us-east-1.ec2.archive.ubuntu.com trusty/main amd64 Packages Hit http://us-east-1.ec2.archive.ubuntu.com trusty/restricted amd64 Packages [...] Hit http://us-east-1.ec2.archive.ubuntu.com trusty/universe Translation-en Ign http://us-east-1.ec2.archive.ubuntu.com trusty/main Translation-en_US [...] Ign http://us-east-1.ec2.archive.ubuntu.com trusty/universe Translation-en_US Fetched 11.7 MB in 4s (2,720 kB/s) Reading package lists... Done
What is a software repository and why does it need updating?
Most of the tens of thousands of free programs that help make Linux as secure, efficient, and useful as it is, are curated and managed by one of two groups of trusted administrators. The online software archives run by these administrators are known as repositories. So that your copy of Linux knows what software is available and how it can be found, from time to time (ideally, before each software installation) you need to update the local version of the repository information that’s kept in an index on your computer. Ubuntu, as part of the Debian family of distributions, uses the *apt* command line interface to manage packages, while yum handles the job for the Red Hat/CentOS repositories.
That one hardly hurt at all. Now we’re going to install the three software packages that will power our web site. These three packages in particular are used together with Linux so often that the combination has even been given its own acronym: LAMP (Linux, Apache, MySQL, and PHP).
- The Apache web server management tool (to point inbound Internet visitors to your website resources).
- The MySQL database (so any software we’ll use later – including WordPress – will have a platform on which to build the databases they’ll need). If you are unfamiliar with WordPress, click here.
- The PHP scripting language (also necessary for many applications, including WordPress).
Don’t leave out the caret (^) character at the end of your command. Here’s how the command (and a very small selection of the output) will look. Note how you are prompted to confirm the installation by typing y (for “yes”):
sudo apt install lamp-server^ Reading package lists... Done Building dependency tree Reading state information... Done Note, selecting 'mysql-server-core-5.5' for task 'lamp-server' Note, selecting 'mysql-server-5.5' for task 'lamp-server' [...] 0 upgraded, 27 newly installed, 0 to remove and 49 not upgraded. Need to get 15.2 MB of archives. After this operation, 123 MB of additional disk space will be used. *Do you want to continue? [Y/n]* [...] apache2_switch_mpm Switch to prefork * Restarting web server apache2 [ OK ] apache2_invoke: Enable module php5 * Restarting web server apache2 [ OK ] Setting up php5-json (1.3.2-2build1) ... php5_invoke: Enable module json for cli SAPI php5_invoke: Enable module json for apache2 SAPI Processing triggers for libc-bin (2.19-0ubuntu6.9) ...
What? No password?
Didn’t I just say that invoking sudo would always require I enter my password? Another question: just what is our password on this AWS instance? The short answer: for security reasons, you don’t have a password. The (slightly) longer answer: because you managed to log in using your unique and highly secure keypair, AWS assumes that you are who you claim to be.
During the installation process, you’ll be asked to create a password for your MySQL database. Make sure you don’t lose it, as you’ll need it later on.
So we can begin setting up our new web site, we’ll visit our new web root directory which, by default, is where all the files accessed by visitors to our site will be kept. By “web root directory” I simply mean the directory within our Linux filesystem where Apache will look to find the files that will make up our website. On an Ubuntu machine, this will normally be /var/www/html/, but you could configure Apache to expect it just about anywhere you like.
cd /var/www/html
Type “ls” to list the current directory contents.
ls
As you can see in figure 6, the only file there right now is called index.html – web browsers will generally load this file first whenever visiting a site.
Figure 6 Contents of the /var/www/html directory. – note the “ls” command displaying the existing index.html file.
Creating the website
To see what the index.html file produces, type the IP address AWS assigned your server – it’s the same four-part numeric address you used to open your SSH shell session from your terminal – into the browser URL field to visit your site. As you can see from figure 7, the current index.html will display the Apache-Ubuntu welcome and orientation page.
Figure 7 The default Apache welcome page that includes important information on how Apache is run from an Ubuntu server.
Let’s go back to our terminal’s command line to do some customization. Use the mv (move) command to rename the default index.html file to index.html.backup. This way, we’ll be free to create a new index.html file while preserving the old one in its original state for future reference. When working with configuration files on servers, it’s always a good idea to make backup copies of older versions rather than simply overwriting them.
sudo mv index.html index.html.backup
We can now to write a simple html page of our own using your favorite text editor – mine is Nano.
sudo nano index.html
Type whatever you want into the editor. Just so you can see how it works, try adding a little HTML formatting, perhaps something like this:
Welcome!
We hope you really enjoy your stay here.
When you’re done, hit CTRL+x and then “y” to save the file and close Nano. To view your new page, hit the refresh button on the browser tab that, until now, had displayed your site’s Apache page.
That’s it – your very own web server!
Since it’s never a good idea to leave unused AWS instances running (they well eventually start to cost you a lot of money and can be a security risk), once you’ve finished admiring it from all angles, you should shut it down. Go back to the EC2 console (console.aws.amazon.com/ec2), make sure that your instance is highlighted, click on the “Actions” button near the top of the page, hover your mouse over the Instance State menu item, and click on “Terminate.” You will be warned that your EBS and all its contents will be lost for good, and your SSH session, if it’s still active, will now end.
9:58…9:59…10:00 minutes!
So that’s that. While we haven’t produced anything all that useful yet, I would imagine that you can already see some of the potential of AWS, and more importantly – how you can use AWS to get your stuff done faster and better than ever before.
If you want to know more, check out the book on liveBook here and see this slide deck.