How to migrate website to Amazon EC2 and RWS

Recommended Ubuntu book

We have just migrated a site from Windows Server to Amazon RDS and EC2. Basically, we needed to move to the cloud separating database from the web server. RDS now runs MySQL 5.7 and EC2 runs on Amazon Linux with Apache. Before the start, on our Windows server, we installed putty to be able to connect to the web server and WinSCP to transfer all the files.

Initial desicion was to keep MySQL, but when choosing between Linux distributions, Amazon Linux seemed like a good option. It is based on CentOS, however, it has been heavily modified to suit AWS. After reading quite a bit about it and looking at different benchmark case studies, it looked like a good choice. It is fast and responsive. It is built and maintained by Amazon for better integration with management tools pre-installed. It is getting updated on a 6-month release cycle and upgrading usually goes without any problems. However, this may be not the best option if you need special packages and community support. It took us some time to decide between AL and Ubuntu.

Step 1, Set Up and Adding Security Groups.

Reference http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_SettingUp.html if needed to sign up, create user and login. Once logged in, select the region where our servers will reside in the top right corner, in our case, Oregon.  We now need to create two security groups, one used for RDS database instance and one for EC2 web instance. Click home button and navigate to “VPC” under “Networking”.

Go to “Security Groups” -> “Create Security Group” to create new security group for our RDS instance. Give it a name and description, such as site-name-rds. Select VPC and click create.

Select created group and edit inbound rules. We need to add ip address from which we will be connecting, which is the server we will be transferring database from.

Your ip will be masked as XXX.XXX.XXX.XXX, replace as needed.  Select MySQL/Aurora (3306), add XXX.XXX.XXX.XXX/32 ip address. It is followed by /32 for exact match ip. Use XXX.XXX.XXX.0/24 for a range of all ip addresses in the subnet.

We will later add one more ip of EC2 web instance to this group.

Once again, go to “Security Groups” -> “Create Security Group” to create new security group for our EC2 instance. Give it a name and description, such as site-name-web. Select VPC and click create.

To allow all incoming traffic over HTTP and HTTPS lets add:

HTTP(80) 0.0.0.0/0 and HTTPS(443) 0.0.0.0/0.

Next, we need to be able to SSH, so lets add:

SSH(22) XXX.XXX.XXX.XXX/32

Save.

Step 2, Set Up RDS Instance.

Reference http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_GettingStarted.CreatingConnecting.MySQL.html if needed.

Make sure that the right region where our servers will reside is selected in the top right corner, in our case, Oregon. Click home button and navigate to “RDS” under “Database”.

Click Launch DB Instance. Select MySQL and then MySQL version. We will use 5.7.11, db.t2.small that will give us 1 vCPU and 2Gib RAM. Since this is a production instance, we will use Multi-AZ Deployment for maximum availability. We will also use 12GB on General Purpose SSD for our databases.

Next, we name our insance site-name-cloud-db, chose db username and password and proceed to “Next Step”. Here we pick the right VPC, Subnet Group, leave it publicly accessible and pick our site-name-rds security group. We leave everything else as is, changing only backup window to 14 days.

Launch DB Instance. It is now being created. You can click “View Your DB Instances” and watch the status. Once changed to “available”, we can start using it.

Step 3, Set Up EC2 Instance.

Reference http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Tutorials.WebServerDB.CreateWebServer.html if needed.

Make sure that the right region where our servers will reside is selected in the top right corner, in our case, Oregon. Click home button and navigate to “EC2” under “Compute”.

Before we begin, lets get a key pair that we will need to connect to the server once it is set up. For reference, use http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/get-set-up-for-amazon-ec2.html#create-a-key-pair . On the left side menu, under “network & security” click “Key Pairs”. Click create key pair, give it a name, such as my-site-key-pair, click create and it will be automatically downloaded. Save it in the safe location on your machine.

Next, click on “Instances” on the left menu and click “Launch Instance”.

Select Amazon Linux AMI. For our simple server we pick t2.micro with 1 vCPU and 1GiB of Memory.

Click “Next” to configure instance details, where we left everything default. On the next screen we changed the storage to suit our needs, which is 100GiB general purpose SSD. On the next screen we create a tag of site-name-web and move forward. Next screen gives us an option to pick existing security group site-name-web create earlier.

Click Launch, verify that information is correct, pick key pair created earlier and click “Launch Instances”. It is now being created. Click “View Instances” and watch the status. Once changed to “running”, we can start using it.

Step 4, Add ip of new EC2 Instance to RDS Security Group.

Run this command on EC2 server to get the ip address and get ip in inet addr under eth0

ifconfig

Edit security group for your site-name-rds and add ip or subnet as:

MySQL/Aurora (3306) 172.XX.X.0/24

Save.

Step 5, Set Up putty and Connect.

Launch PuTTYgen. Make sure SSH-2-RSA option is checked. Click “Load”, in the drop down option select “All Files”, locate the .pem file downloaded earlier after creating Key Pairs, “save private key” with the same name and .ppk extension.

Now you need to get your public dns name. Once EC2 instance is loaded, click on it and in the “Description” below copy Public DNS.

Start putty. In the Host Name field enter user@public-dns, such as ec2-user@ec2-XX-XX-XXX-XX.us-west-2.compute.amazonaws.com. Amazon Linux instances get created with ec2-user by default. Confirm that selected port is 22 and connection type is SSH.

Enter the name in “saved sessions”, such as site-name-aws and click save.

On the left hand side menu in putty navigate to Connection – SSH – Auth. Click Browse and locate .ppk key created using PuTTYgen earlier. Go back to “Session” menu, click save and then open. Click “Yes” if it displays a message and you should be on the server and ready to configure.

Step 6, Set Up Web Server.

First, lets update the packages

sudo yum update -y

Next, install apache, php 5.6 and mysql driver

sudo yum install -y httpd24 php56 php56-mysqlnd

Next, additional packages, if needed. For instance, we use gd image library and need to add it

sudo yum install php56-gd

Soap package:

sudo yum install php56-soap

Imagick, which requires php devel, php pear and a few others

sudo yum install kernel-devel gcc gcc-c++

sudo yum install php56-devel

sudo yum install php-pear # This line installs pecl as well as pear

sudo yum install ImageMagick-devel

sudo pecl install imagick

which requires us to open php.ini and append a line at the end of the file to load the library

sudo vi /etc/php.ini

press “i” to edit the file, add this line at the end

extension=imagick.so

click “esc” and “zz” while holding “shift” to save.

Lets start apache

sudo service httpd start

Change config to autostart on boot

sudo chkconfig httpd on

Add www group and current user to that group along with apache group

sudo groupadd www

sudo usermod -a -G www ec2-user

Lets change permissions on www folder, exit the server and re-login

sudo chown -R root:www /var/www

exit

This is it for the web server, however, we host a couple websites on it. We will create virtual hosts next

cd /var/www/

sudo mkdir vhosts

cd vhosts

sudo mkdir site-1(actual name of the site)

sudo mkdir site-2(actual name of the site)

sudo chown -R root:www /var/www

sudo chmod 2775 /var/www

find /var/www -type d -exec sudo chmod 2775 {} \;

find /var/www -type f -exec sudo chmod 0664 {} \;

cd /etc/httpd/conf.d

sudo touch vhost.conf

sudo vi /etc/httpd/conf.d/vhost.conf

paste and replace site-1.com and site-2.com with actual sites:


# Leave this alone. This setting tells Apache that

# this vhost should be used as the default if nothing

# more appropriate is available.

ServerName default:80

# REQUIRED. Set this to the directory you want to use for

# your “default” site files.

DocumentRoot /var/www/html

# Optional. Uncomment this and set it to your admin email

# address, if you have one. If there is a server error,

# this is the address that Apache will show to users.

#ServerAdmin you@example.com

# Optional. Uncomment this if you want to specify

# a different error log file than the default. You will

# need to create the error file first.

#ErrorLog /var/www/vhosts/logs/error_log

 

# REQUIRED. Set this to the host/domain/subdomain that

# you want this VirtualHost record to handle.

ServerName site-1.com

# Optional. You can specify additional host names that

# serve up the same site. This can be top-level, domains,

# sub-domains, and can even use wildcard subdomains such

# as *.yourdomain.com – just separate each host name

# with a single space.

ServerAlias www.site-1.com

# REQUIRED. Set this to the directory you want to use for

# this vhost site’s files.

DocumentRoot /var/www/vhosts/site-1

# Optional. Uncomment this and set it to your admin email

# address, if you have one. If there is a server error,

# this is the address that Apache will show to users.

#ServerAdmin you@example.com

# Optional. Uncomment this if you want to specify

# a different error log file than the default. You will

# need to create the error file first.

ErrorLog /var/www/vhosts/logs/site-1_error_log

# REQUIRED. Let’s make sure that .htaccess files work on

# this site. Don’t forget to change the file path to

# match your DocumentRoot setting above.

AllowOverride All

 

# REQUIRED. Set this to the host/domain/subdomain that

# you want this VirtualHost record to handle.

ServerName site-2.com

# Optional. You can specify additional host names that

# serve up the same site. This can be top-level, domains,

# sub-domains, and can even use wildcard subdomains such

# as *.yourdomain.com – just separate each host name

# with a single space.

ServerAlias www.site-2.com

# REQUIRED. Set this to the directory you want to use for

# this vhost site’s files.

DocumentRoot /var/www/vhosts/site-2

# Optional. Uncomment this and set it to your admin email

# address, if you have one. If there is a server error,

# this is the address that Apache will show to users.

#ServerAdmin you@example.com

# Optional. Uncomment this if you want to specify

# a different error log file than the default. You will

# need to create the error file first.

ErrorLog /var/www/vhosts/logs/site-2_error_log

# REQUIRED. Let’s make sure that .htaccess files work on

# this site. Don’t forget to change the file path to

# match your DocumentRoot setting above.

AllowOverride All

Next, lets create log directory

cd /var/www/vhosts/

sudo mkdir logs

Restart apache

sudo service httpd restart

Step 7, Set Up SSL Keys and SSL Virtual Directories.

Download and install WinSCP and import connection configuration from Putty.

Export current certificates from existing site. In our case it was windows server and we used this resource for reference – https://www.sslshopper.com/move-or-copy-an-ssl-certificate-from-a-windows-server-to-an-apache-server.html that gave us site.pfx file. Upload the file to /home/ec2-user via WinSCP.

Now, in our ssh session (putty) we need to convert and store the keys:

cd /home/ec2-user/

openssl pkcs12 -in estoresbyzome.pfx -out estoresbyzome.txt -nodes

copy each key and certificate from and including opening —- , to and including closing of each section while pasting it into site.key, site.crt, and if there is more than 2 – intermidiateCA.crt. Then move them to private folder and change permissions.

sudo cp site.key /etc/pki/tls/private/site.key

sudo cp site.crt /etc/pki/tls/private/site.crt

sudo cp intermediateCA.crt /etc/pki/tls/private/intermediateCA.crt

cd /etc/pki/tls/private/

sudo chmod 600 site.key

sudo chmod 600 site.crt

sudo chmod 644 intermediateCA.crt

Install mod ssl for apache and restart

sudo yum install -y mod24_ssl

sudo service httpd restart

Next, we need to change ssl certificate and key in ssl.conf

sudo vi /etc/httpd/conf.d/ssl.conf

to


SSLCertificateFile /etc/pki/tls/private/site.crt

SSLCertificateKeyFile /etc/pki/tls/private/site.key

sudo service httpd restart

Add virtual host entries:

cd /etc/httpd/conf.d/

sudo vi vhost.conf

At the end of the file add:


ServerName site-1.com

ServerAlias www.site-1.com

DocumentRoot /var/www/vhosts/site-1

SSLEngine on

SSLCertificateFile /etc/pki/tls/private/site.crt

SSLCertificateKeyFile /etc/pki/tls/private/site.key

ErrorLog /var/www/vhosts/logs/site-1_error_log

 

ServerName site-2.com

ServerAlias www.site-2.com

DocumentRoot /var/www/vhosts/site-2

SSLEngine on

SSLCertificateFile /etc/pki/tls/private/site.crt

SSLCertificateKeyFile /etc/pki/tls/private/site.key

ErrorLog /var/www/vhosts/logs/site-2_error_log

Save and restart apache, this should be it

sudo service httpd restart

Step 8, Display Maintenance Message on Live Site.

Since our site uses the same header, displaying maintenance message is as easy as pasting html and php exit at the top of it:


<!doctype html>
<title>Site Maintenance</title>
<style>
body { text-align: center; padding: 150px; }
h1 { font-size: 50px; }
body { font: 20px Helvetica, sans-serif; color: #333; }
article { display: block; text-align: left; width: 650px; margin: 0 auto; }
a { color: #dc8100; text-decoration: none; }
a:hover { color: #333; text-decoration: none; }
</style>

<p><img src="img/logo.gif" alt="Site" title="Site" width="150" height="100" align="left"></p>
<article>
<h1>We&rsquo;ll be back soon!</h1>
<div>
<p>Sorry for the inconvenience but we&rsquo;re performing some maintenance at the moment. We&rsquo;ll be back online shortly!</p>
<p>&mdash; Site-1 Team</p>
</div>
</article>
<?php
exit;
?>

Step 9, Backup Database.

Stop the production db and start with locked tables if exit in the header doesn’t stop complete access before the backup.

mysqldump -u root -p -h 127.0.0.1 site-1-db > c:\Backups\site-1-db(build_date).sql

mysqldump -u root -p -h 127.0.0.1 site-2-db > c:\Backups\site-2-db(build_date).sql

Step 10, Stop Production Database.

Since we are running a Windows server, run this command in command line

net stop mysql

Step 11, Create, Restore the Database and Update Schema.

mysql -u zome -p -h site-cloud-db.your-database-endpoint.us-west-2.rds.amazonaws.com

create database site-2;

exit;

mysql -u zome -p -h site-cloud-db.your-database-endpoint.us-west-2.rds.amazonaws.com site-1-db < c:\Backups\right_backup.sql

mysql -u zome -p -h site-cloud-db.your-database-endpoint.us-west-2.rds.amazonaws.com site-2-db < c:\Backups\right_backup_2.sql

We also had a couple of MyISAM tables and converted them to InnoDB, connect to instance

mysql -u zome -p -h site-cloud-db.your-database-endpoint.us-west-2.rds.amazonaws.com

login and check if there are any MyISAM

use INFORMATION_SCHEMA;

SELECT * FROM `TABLES` where engine = 'MyISAM' and (TABLE_SCHEMA = 'site-1-db' OR TABLE_SCHEMA = 'site-2-db’ ) ORDER BY `ENGINE` DESC

use site-1-db;

ALTER TABLE `myisam-table1` ENGINE=INNODB;

ALTER TABLE `myisam-table2` ENGINE=INNODB;

etc.

Step 12, Transfer Files and change db connections.

Login via WinSCP and transfer all the files to the right vhost directories. Change site configurations to point to the new database.

Step 13, change permissions to write directories for apache.

To make directories writable to apache execute the following command on the directories that need write access:
sudo chown -R root:apache directory-name

Step 14, Test.

To test before going live edit the c:\windows\system32\drivers\etc\hosts file on local machine to point to the right server by specifying ip address:
XXX.XXX.XXX.XXX site-1.com www.site-1.com
XXX.XXX.XXX.XXX site-2.com www.site-2.com

Step 15, Change DNS to Point to the New Site.

If you are using AWS for DNS, navigate to Route 53, if not, point your domain A record to the new ip address. Perform the last set of testing and the site in now on AWS.

This is it


One Response to “How to migrate website to Amazon EC2 and RWS