This special remote type stores file contents in a bucket in Amazon S3 or a similar service.
See using Amazon S3 and Internet Archive via S3 for usage examples.
configuration
The standard environment variables AWS_ACCESS_KEY_ID
and
AWS_SECRET_ACCESS_KEY
are used to supply login credentials
for Amazon. When encryption is enabled, they are stored in encrypted form
by git annex initremote
. Without encryption, they are stored in a
file only you can read inside the local git repository. So you do not
need to keep the environment variables set after the initial
initalization of the remote.
A number of parameters can be passed to git annex initremote
to configure
the S3 remote.
encryption
- Required. Either "none" to disable encryption (not recommended), or a value that can be looked up (using gpg -k) to find a gpg encryption key that will be given access to the remote. Note that additional gpg keys can be given access to a remote by rerunning initremote with the new key id. See encryption.datacenter
- Defaults to "US". Other values include "EU", "us-west-1", and "ap-southeast-1".storageclass
- Default is "STANDARD". If you have configured git-annex to preserve multiple copies, consider setting this to "REDUCED_REDUNDANCY" to save money.host
andport
- Specify in order to use a different, S3 compatable service.bucket
- S3 requires that buckets have a globally unique name, so by default, a bucket name is chosen based on the remote name and UUID. This can be specified to pick a bucket name.fileprefix
- By default, git-annex places files in a tree rooted at the top of the S3 bucket. When this is set, it's prefixed to the filenames used. For example, you could set it to "foo/" in one special remote, and to "bar/" in another special remote, and both special remotes could then use the same bucket.x-amz-*
are passed through as http headers when storing keys in S3.
ANNEX_S3_ACCESS_KEY_ID
andANNEX_S3_SECRET_ACCESS_KEY
seem to have been changed toAWS_ACCESS_KEY_ID
andAWS_SECRET_ACCESS_KEY
it'd be really nice being able to configure a S3 remote of the form
<bucket>/<folder>
(not really a folder, of course, just the usual prefix trick used to simulate folders at S3). The remote = bucket architecture is not scalable at all, in terms of number of repositories.how hard would it be to support this?
thanks, this is the only thing that's holding us back from using git-annex, nice tool!
fileprefix
setting. Note that I have not tested it, beyond checking it builds, since I let my S3 account expire. Your testing would be appreciated.Any chance I could bribe you to setup Rackspace Cloud Files support? We are using them and would hate to have a S3 bucket only for this.
https://github.com/rackspace/python-cloudfiles