| Commit message (Collapse) | Author | Age |
|---|
| ... | |
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Make sanity checker run git annex unused daily, and queue up transfers
of unused files to any remotes that will have them. The transfer retrying
code works for us here, so eg when a backup disk remote is plugged in,
any transfers to it are done. Once the unused files reach a remote,
they'll be removed locally as unwanted.
If the setup does not cause unused files to go to a remote, they'll pile
up, and the sanity checker detects this using some heuristics that are
pretty good -- 1000 unused files, or 10% of disk used by unused files,
or more disk wasted by unused files than is left free. Once it detects
this, it pops up an alert in the webapp, with a button to take action.
TODO: Webapp UI to configure this, and also the ability to launch an
immediate cleanup of all unused files.
This commit was sponsored by Simon Michael.
|
| |
|
|
|
|
|
| |
With a really nice optimisation that keeps it from having any overhead
in normal operation!
This commit was sponsored by Ulises Vitulli.
|
| |
|
|
|
|
|
|
|
|
| |
This will be used in expiring old unused objects. The timestamp is when it
was first noticed it was unused.
Backwards compatability: It supports reading old format unused log files.
The old version of git-annex will ignore lines in log files written by the
new version, so the worst interop problem would be git annex dropunused not
knowing some numbers that git-annex unused reported.
|
| | |
|
| |
|
|
| |
This includes several bug fixes.
|
| |
|
|
|
|
|
| |
Move stuff into Logs.NumCopies. Add a NumCopies newtype.
Better names for various serialization classes that are specific to one
thing or another.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* numcopies: New command, sets global numcopies value that is seen by all
clones of a repository.
* The annex.numcopies git config setting is deprecated. Once the numcopies
command is used to set the global number of copies, any annex.numcopies
git configs will be ignored.
* assistant: Make the prefs page set the global numcopies.
This global numcopies setting is needed to let preferred content
expressions operate on numcopies.
It's also convenient, because typically if you want git-annex to preserve N
copies of files in a repo, you want it to do that no matter which repo it's
running in. Making it global avoids needing to warn the user about gotchas
involving inconsistent annex.numcopies settings.
(See changes to doc/numcopies.mdwn.)
Added a new variety of git-annex branch log file, that holds only 1 value.
Will probably be useful for other stuff later.
This commit was sponsored by Nicolas Pouillard.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This allows a remote to store a piece of arbitrary state associated with a
key. This is needed to support Tahoe, where the file-cap is calculated from
the data stored in it, and used to retrieve a key later. Glacier also would
be much improved by using this.
GETSTATE and SETSTATE are added to the external special remote protocol.
Note that the state is left as-is even when a key is removed from a remote.
It's up to the remote to decide when it wants to clear the state.
The remote state log, $KEY.log.rmt, is a UUID-based log. However,
rather than using the old UUID-based log format, I created a new variant
of that format. The new varient is more space efficient (since it lacks the
"timestamp=" hack, and easier to parse (and the parser doesn't mess with
whitespace in the value), and avoids compatability cruft in the old one.
This seemed worth cleaning up for these new files, since there could be a
lot of them, while before UUID-based logs were only used for a few log
files at the top of the git-annex branch. The transition code has also
been updated to handle these new UUID-based logs.
This commit was sponsored by Daniel Hofer.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This was unexpectedly difficult because of a depdenency cycle. To parse a
preferred content expression involves several things that need to operate
on the list of remotes. Which needs Remote.External. The only way to avoid
this cycle (I tried breaking it at several points) was to skip parsing the
expression in SETWANTED.
That's sorta ok, because git-annex already has to deal with unparsable
preferred content expressions being stored, in order to handle eg,
upgrades. But I'm still not very happy that I cannot check it.
I feel this is a strong indication that I need to beware of further
bloating the special remote protocol interface.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
Fixed up a number of things that had worked around there not being a way to
get that.
Most notably, transfer info files on windows now include the process id,
since no locking is currently done. This means the file format varies
between windows and unix.
|
| |
|
|
|
|
|
|
| |
Windows has a larger (unsigned) PID space, so cannot use the unix CInt
there.
Note that TransferInfo does not yet ever get the TransferPid populated,
as there is missing locking.
|
| | |
|
| | |
|
| |
|
|
| |
Caused by bug fixed in 4807612214df574e1dd11dff68c7c55dbff73108
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
Mostly because it gives me an excuse and a hook to document the schedule
expression format.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Once I built the basic widget, it turned out to be rather easy to replicate
it once per scheduled activity and wire it all up to a fully working UI.
This does abuse yesod's form handling a bit, but I think it's ok.
And it would be nice to have it all ajax-y, so that saving one modified
form won't lose any modifications to other forms. But for now, a nice
simple 115 line of code implementation is a win.
This late night hack session commit was sponsored by Andrea Rota.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
| |
I forgot I had <$$> hidden away in Utility.Applicative.
It allows doing the same kind of currying as does >=*>
and I found using it made the code more readable for me.
(*>=> was not used)
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
scan. This prevents repeated retries to download files that are not available, or are not referenced by the current git tree.
This is motivated by a user report that the assistant was repeatedly
retrying transfers of files that had been deleted (in direct mode, so
removing the only copy).
Note that the glacier code retries failed transfers after a while to retry
downloads that have aged long enough to be available. This is ok; if we're
doing a full transfer scan we'll retry on every file that is still in the
git tree.
Also note that this makes the assistant less likely to get every file
referenced by old revs of the git tree. Not something the assistant tries
to ensure anyway, so I feel this is acceptable.
|
| |
|
|
| |
from git, which can be so large it runs out of memory.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
been marked as dead from the git-annex branch.
Wrote nice pure transition calculator, and ugly code to stage its results
into the git-annex branch. Also had to split up several Log modules
that Annex.Branch needed to use, but that themselves used Annex.Branch.
The transition calculator is limited to looking at and changing one file at
a time. While this made the implementation relatively easy, it precludes
transitions that do stuff like deleting old url log files for keys that are
being removed because they are no longer present anywhere.
|
| |
|
|
|
|
|
| |
Having one module that knows about all the filenames used on the branch
allows working back from an arbitrary filename to enough information about
it to implement dropping dead remotes and doing other log file compacting
as part of a forget transition.
|
| |
|
|
|
|
|
|
|
| |
Works, more or less. --dead is not implemented, and so far a new branch
is made, but keys no longer present anywhere are not scrubbed.
git annex sync fails to push the synced/git-annex branch after a forget,
because it's not a fast-forward of the existing synced branch. Could be
fixed by making git-annex sync use assistant-style sync branches.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
When quvi is installed, git-annex addurl automatically uses it to detect
when an page is a video, and downloads the video file.
web special remote: Also support using quvi, for getting files,
or checking if files exist in the web.
This commit was sponsored by Mark Hepburn. Thanks!
|
| | |
|
| |
|
|
|
| |
The latter is harder for me to remember, but avoids build failures in code
used by the configure program.
|
| | |
|
| | |
|
| |
|
|
| |
next file, rather than dying.
|
| | |
|
| |
|
|
| |
last run of git annex unused. Supported by fsck, get, move, copy.
|
| |
|
|
|
|
|
|
|
| |
Made fromDirect check that a file in the tree has good content (and is not
a broken symlink either) before copying it to another file that has the
same key.
Made replaceFile clean up the temp file if the action that creates it, or
the file replacement action fails.
|
| | |
|
| | |
|
| | |
|
| | |
|
Joey Hess
Richard Hartmann
Joey Hess