The Computer Oracle

Can I use rsync for backing up at Amazon S3

--------------------------------------------------
Rise to the top 3% as a developer or hire one of them at Toptal: https://topt.al/25cXVn
--------------------------------------------------

Music by Eric Matyas
https://www.soundimage.org
Track title: Puzzle Island

--

Chapters
00:00 Can I Use Rsync For Backing Up At Amazon S3
00:32 Accepted Answer Score 28
01:32 Answer 2 Score 1
01:50 Answer 3 Score 6
02:02 Answer 4 Score 8
02:32 Thank you

--

Full question
https://superuser.com/questions/288270/c...

--

Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...

--

Tags
#backup #ftp #amazons3

#avk47



ACCEPTED ANSWER

Score 28


No, you cannot use rsync to transfer files to Amazon. It uses its own protocol for the service. But you can access their storage via third-party services, like www.s3rsync.com, then you'll be able to use rsync, and your data will be finally transfered to S3 storage.

Or you can use special utilities designed for S3 storage. There are: s3sync, s3command, s3cp, tarsnap (tarsnap is a third party service, like s3rsync).

Another rsync-like tool, duplicity, supports S3 storage as a backend as well as many other backup backends, including RackSpace Cloud Files (another cloud storage service, priced similarly to Amazon S3).

Backup to S3:

duplicity /home/me s3+http://bucketname/prefix

or to Rackspace's Cloud Files:

duplicity /home/me cf+http://container_name



ANSWER 2

Score 8


Amazon's official command line interface worked well for me. It can sync from local to an S3 bucket, and also sync from an S3 bucket back to local files.

1) install Amazon's command line interface, from https://aws.amazon.com/cli/

2) from a shell,

export AWS_ACCESS_KEY_ID=(access key here)
export AWS_SECRET_ACCESS_KEY=(secret key here)
export AWS_DEFAULT_REGION=(region here)
cd dirtoupload
aws s3 sync . s3://nameOfBucket

More useful options like --dryrun and --delete (to mirror) listed at http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html.




ANSWER 3

Score 6


You can simply mount your bucket locally using s3fs and then use rsync to copy to that mount point.




ANSWER 4

Score 1


Jungle Disk is an easy way to back up files on Amazon S3 (subscription including S3 account, but you can use any other AWS account). You may also be might want to have a look at Super Flexible (software only).