JZ dot C | Script to Backup Files Mysql to S3 and to Restore Them

Here is my script that will backup files & MySQL db’s to Amazon S3, and restore them.

Thanks to the team over at Chatterfly, check them out for sponsoring this scripts development.

The use case I wrote this for was having an auto-scaling group that included the latest database. For infrequently changing LAMP sites like my Wordpress blog, which include a MySQL database, you run the script to update S3 after making changes. Then when traffic gets heavier, your auto-scaling group launches more instances, they boot up and download the latest files and db, and then join the ELB pool.

It also works well for backing up a non-EC2 Linux server to S3 for Disaster Recovery (DR).

Steps to setup backups to Amazon S3 using this script:

Download script from github, here https://github.com/jonzobrist/Bash-Admin-Scripts/tree/master/bluesun-setup

Copy scripts somewhere on your server, I use /etc/init.d. Mark script bluesun-setup.sh executable.

Create directory /etc/bluesun-setup and put server.conf in it.

Edit /etc/bluesun-setup/server.conf. Read the comments, it is heavily commented and includes use cases.

Download and setup Tim Kay’s excellent Amazon AWS tools. Follow Tim’s instructions for setting up your AWS credentials file.

Setup a mysql user, and put the credentials in root’s my.cnf.

Create an Amazon S3 bucket to store your files.

Push your files to Amazon S3 manually.

Setup a cron job to push them however more frequently you like.

Steps to restore from Amazon S3 using this script.

Follow steps 1-6 from above.

Copy the /etc/bluesun-setup/server.conf from the original server to the restore server.

Restore from S3.

Detailed version of backup to Amazon S3 using this script

1. Download script from github, here https://github.com/jonzobrist/Bash-Admin-Scripts/tree/master/bluesun-setup
2. Copy scripts somewhere on your server, I use /etc/init.d. Mark script bluesun-setup.sh executable.
3. Create directory /etc/bluesun-setup and put server.conf in it.
4. Edit /etc/bluesun-setup/server.conf. Read the comments, it is heavily commented and includes use cases.
5. Download and setup Tim Kay’s excellent Amazon AWS tools. Follow Tim’s instructions for setting up your AWS credentials file.
6. Setup a mysql user, and put the credentials in root’s my.cnf.
7. Create an Amazon S3 bucket to store your files.
8. Push your files to Amazon S3 manually.
9. Setup a cron job to push them however more frequently you like.

cd /etc/init.d
wget https://github.com/jonzobrist/Bash-Admin-Scripts/raw/master/bluesun-setup/bluesun-setup.sh
chmod uog+x bluesun-setup.sh
mkdir -p /etc/bluesun-setup
cd /etc/bluesun-setup
wget https://github.com/jonzobrist/Bash-Admin-Scripts/raw/master/bluesun-setup/server.conf
#Edit server.conf. At a minimum set S3_BUCKET, DIRS and one of MYSQL_FILENAME or MYSQL_DATABASES.
mkdir -p /root/bin
cd /root/bin
curl https://raw.github.com/timkay/aws/master/aws -o aws
chmod uog+x aws
perl aws --install
touch /root/.awssecret
chmod og-rwx /root/.awssecret
#Edit /root/.awssecret. Add your AWS credentials, put the ACCESS KEY ID on the first line, and the SECRET KEY on the second line
s3mkdir backups.example.com
touch /root/.my.cnf
chmod og-rwx /root/.my.cnf
#Create your mysql user, if not using root. A typical grant for a read only backup user could be
# mysql> GRANT SELECT, LOCK TABLES ON *.* TO user@'localhost' IDENTIFIED BY 'password';
#Edit /root/.my.cnf so it has a mysql [client] section that includes your username and password, so it looks like this:
[client]
user="user"
password="password"
#Now backup to your new S3 bucket with
/etc/init.d/bluesun-setup.sh updateS3
#Check that your files are there
s3ls backups.example.com
#Setup a cron job for more backups
crontab -e
#Add this line & save
0 * * * * /etc/init.d/bluesun-setup.sh updateS3
To restore follow all the same steps except manually run

/etc/init.d/bluesun-setup.sh start

Also, don’t setup that in a cron job 😉