This special remote type stores file contents in directory.
One use case for this would be if you have a removable drive that you want to use it to sneakernet files between systems (possibly with encrypted contents). Just set up both systems to use the drive's mountpoint as a directory remote.
If you just want two copies of your repository with the files "visible"
in the tree in both, the directory special remote is not what you want.
Instead, you should use a regular git clone
of your git-annex repository.
configuration
These parameters can be passed to git annex initremote
to configure the
remote:
encryption
- Required. Either "none" to disable encryption, or a value that can be looked up (using gpg -k) to find a gpg encryption key that will be given access to the remote, or "shared" which allows every clone of the repository to decrypt the encrypted data.Note that additional gpg keys can be given access to a remote by running enableremote with the new key id. See encryption.
chunksize
- Avoid storing files larger than the specified size in the directory. For use on directories on mount points that have file size limitations. The default is to never chunk files.
The value can use specified using any commonly used units. Example:chunksize=100 megabytes
Note that enabling chunking on an existing remote with non-chunked files is not recommended.
Setup example:
# git annex initremote usbdrive type=directory directory=/media/usbdrive/ encryption=none
# git annex describe usbdrive "usb drive on /media/usbdrive/"
Thanks for this great tool! I was wondering what the differences are between using
type=directory
,type=rsync
, or a bare git repo for directories?I guess I can't use just a regular repo because my USB drive is formatted as
vfat
-- which threw me for a loop the first time I heard aboutgit-annex
about a year ago, because I followed the walkthrough, and it didn't work as expected and gave up (now I know it was just a case of PEBKAC). It might be worth adding a note about vfat to the "Adding a remote" section of the walkthrough, since the unstated assumption there is that the USB drive is formatted as a filesystem that supports symlinks.Thanks again, my scientific data management just got a lot more sane!
The directory and rsync special remotes intentionally use the same layout. So the same directory could be set up as both types of special remotes.
The main reason to use this rather than a bare git repo is that it supports encryption.
git annex move --from $remote
to get all the content out of it, thengit annex dead $remote
and finally you cangit remote rm $remote
I tried the suggestion on comment 4, but when I add again a remote with the same path, it gets the same repository identifier and is considered dead. Is that expected?
My use case: I use a usb drive to transfer some large files from one git annex to another, then I use the usb drive for something else and the special remote is removed. Later I want to use the same usb drive again, but when I create the repository, it starts in the dead state.
@nicolas, I suspect you are using
git annex initremote
with the same name that you used for the now dead-and-buried remote. That causes it to be reanimated, which is not what you want.Since git-annex version 4.20130501,
git annex initremote
is reserved for creating new remotes, not enabling old ones, so it will refuse to do this. That's to avoid exactly this confusion.Using
git annex initremote
with a different remote name, and the same directory should work just fine.This is great work. I've developed a serious annex-addiction and now I want to use it everywhere! In particular I was hoping to apply it to this use case:
I have large files/directories (approx 5 TB) on an nfs mount to which is a) write-protected (think "read-only medium") and b) used by non-git users. Both reasons prevent me from setting up a git-annex repos there. However, I would like to use git-annex to keep track of the paths and get/drop files from my different computers.
On one of my servers, I set up a git annex repos, hoping to only manage the structure, the locations, and the number of copies. I don't want to have copies of the 5TB files in that repository, as disk space is not unlimited (just for the sake of making them available to my laptop).
I as banking on using a special remote (either directory or rsync) to tell the git-annex repos where the actual data is.
I am not concerned with data loss, as it is backed up in regular time intervals by our sysadmin.
I tried both directory remote and rsync remote but there seems to be a missing piece (I suppose its add). Any ideas?
This is what I did:
I added the directory remote and an rsync remote
the copy command fails without complaints $ git annex copy --from collections
I tried adding virtual files to git annex
but still any kind of get/copy command does not get any new files
It would be awesome if I could use git-annex for this, to keep track of my copies and copies of copies. And then I could also keep track of data on my write-protected DVDs.
Is there any chance?
Thanks a lot!
-- Laura
@Laura the directory special remote requires files to be in a particular directory structure with special names git-annex comes up with. So you can't use it on an existing tree of files like that.
What you can do is use the web special remote, with a
file://
url to point to the files wherever they are stored. So for example,git annex addurl file:///media/dvd/file
Using the web remote is a pretty nice trick!
Thanks, Joey - I would not have guessed that.
-- Laura