This post is part of a series on automatically backing up your website:
- Part 1: Getting Started with Amazon S3 and S3Fox
- Part 2: Using Amazon S3 from the Command Line with s3sync
- Part 3: Backing Up Your Data with Astrails-Safe
- Part 4: Automating Your Backup with Cron
What Is Amazon S3?
Amazon S3 is a web service that lets you store data in the cloud for very cheap (my October bill was $0.05). This makes it a great place to backup files.
Signing Up for Amazon S3
The first step is to get an Amazon S3 account if you don’t already have one. After you sign up, Amazon will give you an access key ID and a secret access key. You can think of these as a login and password, respectively. The access key ID tells Amazon who you are, and the secret access key authenticates you. Guard your secret access key as you would a password. Anyone who has your secret access key can access your S3 data.
S3Fox is not necessary for backing up your data on S3, but I found it very useful. That’s because Amazon S3 is just a web service; it has no GUI. S3Fox is a Firefox add-on that gives you a nice GUI for interacting with your S3 data. This is great if you’re a visual thinker like me. Check out the video preview of S3Fox before you install it.
Update: Amazon now provides a GUI for S3 at console.aws.amazon.com/s3.
Setting Up Your Buckets
All S3 data is stored in buckets. A bucket is simply a collection of data objects which have unique names. For example, a bucket may contain an image named
foo.gif and a text file named
bar.txt. Buckets have a flat structure; it’s like having one directory full of many files. However, we can fake a hierarchical directory structure if we give the data objects clever names like
workfiles/bar.txt. S3Fox does this for us automatically.
Buckets have some constraints:
- Your account is limited to 100 buckets (but each bucket can hold an unlimited number of objects).
- An object in a bucket can’t be more than 5 gigabytes terabytes.
- Everyone in the whole wide world shares a global bucket namespace.
Because of this last constraint, it is recommended that you name your buckets after domain names that you own. That way we avoid namespace collisions. For example, I created a bucket named
backup.techiferous.com for this blog.
Go ahead and use S3Fox to create a bucket and experiment with adding and deleting files and directories from it.
Next Step: Your Web Server
Now that you have an S3 account and have tested it out, let’s learn how to interact with S3 from the command line of your web server.