1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
|
A walkthrough of the basic features of git-annex.
[[!toc]]
## creating a repository
This is very straightforward. Just tell it a description of the repository.
# mkdir ~/annex
# cd ~/annex
# git init
# git annex init "my laptop"
## adding a remote
Like any other git repository, git-annex repositories have remotes.
Let's start by adding a USB drive as a remote.
# sudo mount /media/usb
# cd /media/usb
# git clone ~/annex
# cd annex
# git annex init "portable USB drive"
# git remote add home ~/annex
# cd ~/annex
# git remote add usbdrive /media/usb
This is all standard ad-hoc distributed git repository setup.
The only git-annex specific part is telling it the name
of the new repository created on the USB drive.
Notice that both repos are set up as remotes of the other one. This lets
either get annexed files from the other. You'll want to do that even
if you are using a centralized bare repository.
## adding files
# cd ~/annex
# cp /tmp/big_file .
# cp /tmp/debian.iso .
# git annex add .
add big_file ok
add debian.iso ok
# git commit -a -m added
Notice you commit at the end, this checks in git-annex's record of the
files but not their actual, large, content.
## renaming files
# cd ~/annex
# git mv big_file my_cool_big_file
# mkdir iso
# git mv debian.iso iso
# git annex fix .
fix iso/debian.iso ok
# git commit -m moved
You can use any normal git operations to move files around, or even
make copies or delete them. `git-annex fix` needs to be run if a file
is moved into a different directory, in order to fix up the symlink
pointing to the file's content.
## getting file content
A repository does not always have all annexed file contents available.
When you need the content of a file, you can use "git annex get" to
make it available.
We can use this to copy everything in the laptop's home annex to the
USB drive.
# cd /media/usb/annex
# git pull home master
# git annex get .
get my_cool_big_file (copying from home...) ok
get iso/debian.iso (copying from home...) ok
Notice that you had to git pull from home first, this lets git-annex know
what has changed in home, and so it knows about the files present there and
can get them. See below for an easier way.
## transferring files: When things go wrong
After a while, you'll have serveral annexes, with different file contents.
You don't have to try to keep all that straight; git-annex does
[[location_tracking]] for you. If you ask it to get a file and the drive
or file server is not accessible, it will let you know what it needs to get
it:
# git annex get video/hackity_hack_and_kaxxt.mov
get video/_why_hackity_hack_and_kaxxt.mov (not available)
I was unable to access these remotes: server
Try making some of these repositories available:
5863d8c0-d9a9-11df-adb2-af51e6559a49 -- my home file server
58d84e8a-d9ae-11df-a1aa-ab9aa8c00826 -- portable USB drive
ca20064c-dbb5-11df-b2fe-002170d25c55 -- backup SATA drive
failed
# sudo mount /media/usb
# git annex get video/hackity_hack_and_kaxxt.mov
get video/hackity_hack_and_kaxxt.mov (copying from usbdrive...) ok
# git commit -a -m "got a video I want to rewatch on the plane"
## removing files
You can always drop files safely. Git-annex checks that some other annex
has the file before removing it.
# git annex drop iso/debian.iso
drop iso/Debian_5.0.iso ok
# git commit -a -m "freed up space"
## removing files: When things go wrong
Before dropping a file, git-annex wants to be able to look at other
remotes, and verify that they still have a file. After all, it could
have been dropped from them too. If the remotes are not mounted/available,
you'll see something like this.
# git annex drop important_file other.iso
drop important_file (unsafe)
Could only verify the existence of 0 out of 1 necessary copies
I was unable to access these remotes: usbdrive
Try making some of these repositories available:
58d84e8a-d9ae-11df-a1aa-ab9aa8c00826 -- portable USB drive
ca20064c-dbb5-11df-b2fe-002170d25c55 -- backup SATA drive
(Use --force to override this check, or adjust annex.numcopies.)
failed
drop other.iso (unsafe)
Could only verify the existence of 0 out of 1 necessary copies
No other repository is known to contain the file.
(Use --force to override this check, or adjust annex.numcopies.)
failed
Here you might --force it to drop `important_file` if you trust your backup.
But `other.iso` looks to have never been copied to anywhere else, so if
it's something you want to hold onto, you'd need to transfer it to
some other repository before dropping it.
## moving file content to another repository
Often you will want to transfer some file contents from a repository to
some other one, and then drop it from the first repository. For example,
your laptop's disk is getting full; time to move some files to an external
disk. Doing that by hand is possible, but a bit of a pain. `git annex move`
makes it very easy.
# git annex move my_cool_big_file --to usbdrive
move my_cool_big_file (to usbdrive...) ok
## using ssh remotes
So far git-annex has been used with a remote repository on a USB drive.
But it can also be used with a remote that is truely remote, a host
accessed by ssh.
Say you have a desktop on the same network as your laptop and want
to clone the laptop's annex to it:
# git clone ssh://mylaptop/home/me/annex ~/annex
# cd ~/annex
# git annex init "my desktop"
Now you can get files and they will be transferred by `scp`:
# git annex get my_cool_big_file
get my_cool_big_file (getting UUIDs for origin...) (copying from origin...)
WORM:1285650548:2159:my_cool_big_file 100% 2159 2.1KB/s 00:00
ok
When you drop files, git-annex will ssh over to the remote and make
sure the file's content is still there before removing it locally:
# git annex drop my_cool_big_file
drop my_cool_big_file (checking origin..) ok
Note that normally git-annex prefers to use non-ssh remotes, like
a USB drive, before ssh remotes. They are assumed to be faster/cheaper to
access, if available. There is a annex-cost setting you can configure in
`.git/config` to adjust which repositories it prefers. See
[[the_man_page|git-annex]] for details.
Also, note that you need full shell access for this to work --
git-annex needs to be able to ssh in and run commands.
## using the URL backend
git-annex has multiple key-value [[backends]]. So far this walkthrough has
demonstrated the default, WORM (Write Once, Read Many) backend.
Another handy backend is the URL backend, which can fetch file's content
from remote URLs. Here's how to set up some files in your repository
that use this backend:
# git annex fromkey --backend=URL --key=http://www.archive.org/somefile somefile
fromkey somefile ok
# git commit -m "added a file from the Internet Archive"
Now you if you ask git-annex to get that file, it will download it,
and cache it locally.
# git annex get somefile
get somefile (downloading)
#########################################################################100.0%
ok
You can always drop files downloaded by the URL backend. It is assumed
that the URL is stable; no local backup is kept.
# git annex drop somefile
drop somefile (ok)
|
