Creating an Amazon EC2 Instance
Login into our AWS account.
· Go to EC2, Virtual Servers in the Cloud
· Once you are at EC2 Dashboard, you can go to Instances and select Launch Instance.
· Choose Amazon Linux AMI (first one)
(NOTE: All instructions will assume a specific baseline image/operating system for consistency. You are welcome to create another image, but please watch for high-end instance costs and for instances left running idle after you are done)
· Choose “General purpose” instance category and select instance size t2.small should be sufficient. The smaller instances may have insufficient resources (RAM, in particular) to run Hadoop jobs.
· Click “Next: Configure Instance Details”
· Make sure that “Auto-assign Public IP” option is enabled (the default value should be fine, but if you see “Disable” there, you should change it)
· Click “Next: Add Storage”. Change this storage to at least 12GB (or 16GB). This cannot be changed later without creating a brand new instance.
· Click “Next: Tag instance”
· Add your name or some other easily identifiable marker as the Value corresponding to the Name key.
· Click on “Review and Launch” (we are not going to worry about setting up security groups until we get to multi-node clusters, so you can ignore the “Next: Configure Security Group” for the time being)
· (Choose “Review and Launch”)
· Click on “Launch”
At this point you will need to generate a key pair if you do not yet have one (you will not be able to use key pairs for which you do not have the original .pem file). We will talk more about the relevant cryptography concepts later in class – but for now, you just need to generate a pair of values, where the “private key” file will serve as your password, allowing you to login into your instance. I will review the process in class.
· Choose “Create a new key pair” option and give this key pair a name (use your user name so that you are able to identify that key pair later)
· Click on “Download Key Pair”. Please be sure to remember where you saved the .pem file that you got (if you lose the file, the associated instance will not be accessible ever again)
· Once you have generated the key, you can click on “Launch Instance” and next (while the instance is initiating) you will need to export the private key for PuTTy:
· Assuming you use PuTTy terminal (let me know if you need help with another environment), you will need to download putty.exe and puttygen.exe from http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
Remember that your user name for Amazon instance is ec2-user (not root or your CampusConnect or anything else). Do not replace “user” in ec2-user with anything else.
If you are using Linux/Mac, you do not need to use PuTTy software and go through the instructions that follow. Instead, you can simply use the following command:
ssh -i private_key_file.pem ec2-user@your_instance_ip_address
in the terminal, which will connect you to the instance. If you get a message about permission errors and file being “too open”, that can be resolved with changing permissions to be more restrictive: chmod 600 private_key_file.pem
You can get more details by clicking Connect button in AWS:
· Assuming that you are on a Windows computer, run puttygen.exe (in order to export the private key into putty-specific format)
· Click “Load” to open the .pem file you received (be sure to choose AllFiles filter if you cannot see the amazon AWS key file in the dialog box)
· Click “Save private key” (for simplicity, you can ignore the warning about using passphrase to protect the private key).
· That saved file has a .ppk extension, putty private key, which we will use as a password (you do not need to export the private key again)
· Now, you can run putty.exe to setup a new connection
· Name your connection session as you wish (“Saved Sessions”)
· Next go to “Data” (under “Connection” menu on the left) and put in ec2-user for the “Auto-login username” field
· Open the “SSH” drop down menu (also under “Connection”) and click on “Auth” sub-menu. Here, you will specify the name of the private key file we have just exported. Click on “Browse” and choose the key file (.ppk file)
· Go back to the “Session” menu (top menu on the left) and fill in the “Host name or IP address” field
· For that, you need to go back to EC2 AWS console, find the running instance you just started and copy the contents of the “Public IP” entry
· Now you should save your session (you can save session at any time, but that’s the final result). Once you have saved the session, you can use it to quickly login to your instance.
· Finally, assuming that the Amazon instance has fully initiated, you can double-click on your newly saved session to automatically connect to your instance! You can click away the one-time “trust the host?” warning dialog.