summaryrefslogtreecommitdiff
path: root/Logs
Commit message (Collapse)AuthorAge
* Keys marked as dead are now skipped by --all.Gravatar Joey Hess2017-05-09
| | | | | | | | | | | | | | | | | | | | | | | | | fsck already special-cased dead keys to make --all not report errors with them, and it makes sense to also expand that to whereis. I think it makes sense for dead keys to be skipped by all uses of --all, so mistakes can be completely forgotten about and not come back to haunt us. The speed impact of testing if the key is dead is negligible for fsck and whereis, since they use the location log anyway and it gets cached. This does slow down a few commands that support --all, in particular metadata --all runs around 2x as slow. I don't think metadata --all is often used though. It might slow down copy/move/mirror --all and get --all. log --all is not affected (does not use the normal --all machinery). Dead keys will still be processed by --incomplete, --branch, --failed, and --key. Although it would be unlikely for a dead key to ave in incomplete or failed transfer. It seems to make perfect sense for --branch to process keys on the branch, even if dead. (fsck's special-casing of dead keys was left in, so if one of these options causes a dead key to be fscked, there will be a nice message.) This commit was supported by the NSF-funded DataLad project.
* multicast: New command, uses uftp to multicast annexed files, for eg a ↵Gravatar Joey Hess2017-03-30
| | | | | | classroom setting. This commit was supported by the NSF-funded DataLad project.
* Drop support for building with old versions of directory, feed, and http-types.Gravatar Joey Hess2017-03-10
|
* AssociatedFile newtypeGravatar Joey Hess2017-03-10
| | | | | | To prevent any further mistakes like 1a497cefb47557f0b4788c606f9071be422b2511 This commit was sponsored by Francois Marier on Patreon.
* fix up Read instance incompatability caused by recent commitGravatar Joey Hess2017-02-24
| | | | | | | | | | | | | | | | | | | 2f868db90c7ba16eee901b9b1472b1e1a889dd93 changed the Read instance for Key. I've checked all uses of that instance (by removing it and seeing what breaks), and they're all limited to the webapp, except one. That is GitAnnexDistribution's Read instance. So, 2f868db90c7ba16eee901b9b1472b1e1a889dd93 would have broken upgrades of git-annex from downloads.kitenet.net. Once the .info files there got updated for a new release, old releases would have failed to parse them and never upgraded. To fix this, I found a way to make the .info files that contain GitAnnexDistribution values be readable by the old version of git-annex. This commit was sponsored by Ewen McNeill.
* annex.autocommit can be configured via git-annex configGravatar Joey Hess2017-02-03
| | | | | | | | | | | | | | | | | ... to control the default behavior in all clones of a repository. This includes a new Configurable data type, so the GitConfig type indicates which values can be configured this way. The implementation should be quite efficient; the config log is only read once, and only when a Configurable value has not already been set by git-config. Indeed, it would be nice in the future to extend this, so that git-config is itself only read on demand. Some commands may not need to look at the git configuration at all. This commit was sponsored by Trenton Cronholm on Patreon.
* Some optimisations to string splitting code.Gravatar Joey Hess2017-01-31
| | | | | | | | | | | | | | | | | | | Turns out that Data.List.Utils.split is slow and makes a lot of allocations. Here's a much simpler single character splitter that behaves the same (even in wacky corner cases) while running in half the time and 75% the allocations. As well as being an optimisation, this helps move toward eliminating use of missingh. (Data.List.Split.splitOn is nearly as slow as Data.List.Utils.split and allocates even more.) I have not benchmarked the effect on git-annex, but would not be surprised to see some parsing of eg, large streams from git commands run twice as fast, and possibly in less memory. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
* forgot to add this new source fileGravatar Joey Hess2017-01-30
|
* vicfg: Include the numcopies configuation.Gravatar Joey Hess2017-01-30
| | | | | | | | | Docs say vicfg can configure everything from git-annex branch, so it ought to configure numcopies. Note that commenting out existing numcopies does not unset it. This commit was sponsored by Thom May on Patreon.
* Always use filesystem encoding for all file and handle reads and writes.Gravatar Joey Hess2016-12-24
| | | | | This is a big scary change. I have convinced myself it should be safe. I hope!
* Avoid backtraces on expected failures when built with ghc 8; only use ↵Gravatar Joey Hess2016-11-15
| | | | | | | | | | | | | backtraces for unexpected errors. ghc 8 added backtraces on uncaught errors. This is great, but git-annex was using error in many places for a error message targeted at the user, in some known problem case. A backtrace only confuses such a message, so omit it. Notably, commands like git annex drop that failed due to eg, numcopies, used to use error, so had a backtrace. This commit was sponsored by Ethan Aubin.
* Optimised git-annex branch log file timestamp parsing. 10% speedupGravatar Joey Hess2016-09-29
| | | | | | This sped up git annex find --not --in web from 6.64s to 5.69s. The optimised parser is probably more like 50% faster than the general one it replaced.
* move Arbitrary instance to new Types.Transfer moduleGravatar Joey Hess2016-09-05
| | | | Avoid orphan instance warning
* avoid warning about orphan Arbirary instanceGravatar Joey Hess2016-09-05
|
* get, move, copy, mirror: Added --failed switch which retries failed copies/movesGravatar Joey Hess2016-08-03
| | | | | | | | | Note that get --from foo --failed will get things that a previous get --from bar tried and failed to get, etc. I considered making --failed only retry transfers from the same remote, but it was easier, and seems more useful, to not have the same remote requirement. Noisy due to some refactoring into Types/
* log: Added --all option.Gravatar Joey Hess2016-07-17
|
* remove \r from Arbitrary for log testsGravatar Joey Hess2016-05-27
|
* Split lines in the git-annex branch on \r as well as \n, to deal with \r\n ↵Gravatar Joey Hess2016-05-27
| | | | | | | | | | | | | | | terminated lines written by some versions of git-annex on Windows. This fixes strange displays in some cases, including whereis showing many duplicate locations, and showing more total copies than actually exist. It's unknown if that lead to data loss when eg, dropping. At the moment, it seems unlikely it could, since the UUID with \r's appended is not the same as a UUID without, and so no remote matches it. It's also unknown if \r's can leak in on windows, perhaps when merging the git-annex branch.
* nub transitionList to avoid ugly message after repeated transitions, and ↵Gravatar Joey Hess2016-05-18
| | | | avoid redundant work for repeated ForgetDeadRemotes transitions
* reinject: Added new mode which can reinject known files into the annex.Gravatar Joey Hess2016-04-22
| | | | For example: git-annex reinject --known /mnt/backup/*
* Limit annex.largefiles parsing to the subset of preferred content ↵Gravatar Joey Hess2016-02-03
| | | | | | expressions that make sense in its context. So, not "standard" or "lackingcopies", etc.
* simplifyGravatar Joey Hess2016-02-03
|
* remove 163 lines of code without changing anything except importsGravatar Joey Hess2016-01-20
|
* refactorGravatar Joey Hess2016-01-19
|
* avoid unnecessary reading of git-annex branch data when matching on ↵Gravatar Joey Hess2015-12-04
| | | | | | | annex.largefiles This makes git annex clean not look at the git-annex branch at all, and so speeds it up by 50% or more.
* fix use of hifalutin terminologyGravatar Joey Hess2015-11-16
|
* convert from Utility.LockPool to Annex.LockPool everywhereGravatar Joey Hess2015-11-12
|
* Avoid unncessary write to the location log when a file is unlocked and then ↵Gravatar Joey Hess2015-10-12
| | | | | | | | | | | | | | | added back with unchanged content. Implemented with no additional overhead of compares etc. This is safe to do for presence logs because of their locality of change; a given repo's presence logs are only ever changed in that repo, or in a repo that has just been actively changing the content of that repo. So, we don't need to worry about a split-brain situation where there'd be disagreement about the location of a key in a repo. And so, it's ok to not update the timestamp when that's the only change that would be made due to logging presence info.
* oops, didn't mean to commit this debugGravatar Joey Hess2015-10-06
|
* analysisGravatar Joey Hess2015-10-06
|
* Fix bug in combination of preferred and required content settings. When one ↵Gravatar Joey Hess2015-09-15
| | | | | | | was set to the empty string and the other set to some expression, this bug caused all files to be wanted, instead of only files matching the expression. Avoid: MAny `MOr` otherexpression Which matches anything.
* add test case that all standard group preferred content expressions parseGravatar Joey Hess2015-06-17
|
* 4 ns optimisation of repeated calls to hasDifference on the same DifferencesGravatar Joey Hess2015-06-11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | I want this as fast as possible, so it can be added to code paths without slowing them down. Avoid the set lookup, and rely on laziness, drops runtime from 14.37 ns to 11.03 ns according to this criterion benchmark: import Criterion.Main import qualified Types.Difference as New import qualified Types.DifferenceOld as Old main :: IO () main = defaultMain [ bgroup "hasDifference" [ bench "new" $ whnf (New.hasDifference New.OneLevelObjectHash) new , bench "old" $ whnf (Old.hasDifference Old.OneLevelObjectHash) old ] ] where s = "fromList [ObjectHashLower, OneLevelObjectHash, OneLevelBranchHash]" new = New.readDifferences s old = Old.readDifferences s A little bit of added boilerplate, but I suppose it's worth it to not need to worry about set lookup overhead. Note that adding more differences would slow down the old implementation; the new implementation will run the same speed.
* dead --key: Can be used to mark a key as dead.Gravatar Joey Hess2015-06-09
|
* fsck: Ignore keys that are known to be dead when running in --all mode or a ↵Gravatar Joey Hess2015-06-09
| | | | in a bare repo. Otherwise, still reports files with lost contents, even if the content is dead.
* parse X in location log file as indicating a dead keyGravatar Joey Hess2015-06-09
| | | | | | | | | | | | | | A dead key is both not present at the location that thinks it has a copy, and also is assumed to probably not be present anywhere else. Although there may be lurking disconnected repos that somehow still have a copy. Suprisingly few changes needed for this! This is because the presence log code only really concerns itself with keys that are present, and dead keys are not present. Note that both the location and web log can be parsed as having a dead key. I don't see any value to having keys listed as dead in the web log, but since it doesn't change any behavior, there was no point in not parsing it.
* remove back-compat code for old version of containersGravatar Joey Hess2015-06-06
| | | | Already b-d on a newer version.
* ignore failure to clean up stale transfer lock fileGravatar Joey Hess2015-05-19
| | | | | Perhaps due to permissions problem, or perhaps a race with another process also cleaning up.
* fix crash in stale transfer lockfile cleanup codeGravatar Joey Hess2015-05-19
| | | | | Need to differentiate between the lockfile not being locked, and it not existing.
* use lock pools throughout git-annexGravatar Joey Hess2015-05-19
| | | | | | | | | | | | | The one exception is in Utility.Daemon. As long as a process only daemonizes once, which seems reasonable, and as long as it avoids calling checkDaemon once it's already running as a daemon, the fcntl locking gotchas won't be a problem there. Annex.LockFile has it's own separate lock pool layer, which has been renamed to LockCache. This is a persistent cache of locks that persist until closed. This is not quite done; lockContent stil needs to be converted.
* lock pools to work around non-concurrency/composition safety of POSIX fcntlGravatar Joey Hess2015-05-18
|
* Stale transfer lock and info files will be cleaned up automatically when ↵Gravatar Joey Hess2015-05-12
| | | | | | get/unused/info commands are run. Deleting lock files is tricky, tricky stuff. I think I got it right!
* an optimization that also fixes a reversionGravatar Joey Hess2015-05-12
| | | | | | | | | | | | | | | | | | | | This is a little optimisation; avoid loading the info file for the download of the current key when checking for other downloads. The reversion it fixes is sorta strange. b94eafec8c4a7868da753f9b22ca823552e9764c broke checking for transfers that were already in progress. Indeed, the transfer lock was not held after getTransfers was called. Why? I think it's magic in ghc's handling of getLock and setLock, although it's hard to tell since those functions are almost entirely undocumented as to their semantics. Something, either the RTS (or maybe it's linux?) notices that the same process has taken a lock and is now calling getLock on a FD attached to the same file. So, it drops the lock. So, this optimisation avoids that problematic behavior.
* Take space that will be used by running downloads into account when checking ↵Gravatar Joey Hess2015-05-12
| | | | annex.diskreserve.
* couple of AMP warnings I missed beforeGravatar Joey Hess2015-05-10
|
* rejigger imports for clean build with ghc 7.10's AMP changesGravatar Joey Hess2015-05-10
| | | | | The explict import Prelude after import Control.Applicative is a trick to avoid a warning.
* more time-1.5 fixesGravatar Joey Hess2015-05-10
|
* fix build warning with time 1.5Gravatar Joey Hess2015-05-10
|
* forgot to add new moduleGravatar Joey Hess2015-05-10
|
* support time-1.5.0Gravatar Joey Hess2015-05-10
| | | | | | | This no longer uses old-locale's defaultTimeLocale, but provides one of its own. Factored out a Logs.TimeStamp.