summaryrefslogtreecommitdiff
path: root/Utility
Commit message (Collapse)AuthorAge
* relay external special remote stderr through progress suppression machinery ↵Gravatar Joey Hess2015-04-04
| | | | | | | | | | (eep!) It sounds worse than it is. ;) Some external special remotes may run commands that display progress on stderr. If git-annex is run with --quiet, this should filter out such displays while letting the errors through.
* well along the way to fully quiet --quietGravatar Joey Hess2015-04-04
| | | | | | | Came up with a generic way to filter out progress messages while keeping errors, for commands that use stderr for both. --json mode will disable command outputs too.
* make downloadQuiet quiet againGravatar Joey Hess2015-04-03
| | | | This was broken in commit 95418cc430284b65af13105f7c63da08908dd826
* fix stderr handlingGravatar Joey Hess2015-04-03
|
* WIP on making --quiet silence progress, and infra for concurrent progress barsGravatar Joey Hess2015-04-03
|
* rename bothHandles -> ioHandlesGravatar Joey Hess2015-04-03
|
* Significantly sped up processing of large numbers of directories passed to a ↵Gravatar Joey Hess2015-04-02
| | | | | | | | single git-annex command. (try 2) New approach is to do it the expensive way for the first 100 paths on the command line, but then assume the user doesn't care about order too much and fall back to the cheap way that does not preserve order.
* Revert "Significantly sped up processing of large numbers of directories ↵Gravatar Joey Hess2015-04-02
| | | | | | | | | passed to a single git-annex command." This reverts commit 5492d8f4ddbd398e0188da9daed840908d1198c0. Whoops, git ls-files does not always output in the input ordering. That's why all this work is needed. Urk.
* commentGravatar Joey Hess2015-04-02
|
* minor optGravatar Joey Hess2015-04-02
|
* make segmentXargs preserve orderGravatar Joey Hess2015-04-02
|
* Significantly sped up processing of large numbers of directories passed to a ↵Gravatar Joey Hess2015-04-02
| | | | single git-annex command.
* micro-optGravatar Joey Hess2015-04-01
|
* Work around curl bug when asked to download an empty url to a file.Gravatar Joey Hess2015-03-27
| | | | | | | | | | | | | | | | | | | | | In this situation, curl -o exits successfully without creating the output file. There was already a workaround for curl file:/// but I did not realize this also affected regular url downloads. To fix it, pre-create the destination file before starting curl. Since we cannot always know the size of an url before trying to download it, let's always do this. Note that since curl is told -C -, we have to consider if this makes curl try to do a ranged download, which might fail on some servers where a regular download would have succeeded. My testing indicates this isn't a problem; since the file is empty, curl seems to not try to do a ranged download. Original report: https://github.com/datalad/datalad/issues/79 Curl bug report: https://github.com/bagder/curl/issues/183
* update comment for segmentXargsGravatar Joey Hess2015-03-26
|
* metadata: Fix encoding problem that led to mojibake when storing metadata ↵Gravatar Joey Hess2015-03-04
| | | | | | | | | | | | | | | | | | | | | | | | | strings that contained both unicode characters and a space (or '!') character. The fix is to stop using w82s, which does not properly reconstitute unicode strings. Instrad, use utf8 bytestring to get the [Word8] to base64. This passes unicode through perfectly, including any invalid filesystem encoded characters. Note that toB64 / fromB64 are also used for creds and cipher embedding. It would be unfortunate if this change broke those uses. For cipher embedding, note that ciphers can contain arbitrary bytes (should really be using ByteString.Char8 there). Testing indicated it's not safe to use the new fromB64 there; I think that characters were incorrectly combined. For credpair embedding, the username or password could contain unicode. Before, that unicode would fail to round-trip through the b64. So, I guess this is not going to break any embedded creds that worked before. This bug may have affected some creds before, and if so, this change will not fix old ones, but should fix new ones at least.
* fix intentional typo left in after debugging session, broke quvi support ↵Gravatar Joey Hess2015-02-19
| | | | (last release is ok)
* bittorrent: Fix mojibake introduced in parsing arai2c progress output.Gravatar Joey Hess2015-02-10
| | | | | | | | | | | | hGetSomeString reads one byte at a time, so unicode bytes are not composed. The problem comes when outputting that to the console with hPut; that tried to apply the handle's encoding, and so we get mojibake. Instead, use ByteStrings, and only convert it to a string for parsing, not for display. Note that there are a couple of other things that use hGetSomeString, which I've left as-is for now.
* addurl: Avoid crash if quvi is not installed, when git-annex was built with ↵Gravatar Joey Hess2015-02-10
| | | | | | | process-1.2 createProcess has been changed to throw an exception if the program is not in path.
* Windows: Fix bug in dropping an annexed file, which caused a symlink to be ↵Gravatar Joey Hess2015-02-09
| | | | staged that contained backslashes.
* Added MD5 and MD5E backends.Gravatar Joey Hess2015-02-04
|
* Remove support for building without cryptohash.Gravatar Joey Hess2015-02-04
| | | | | This will prevent backporting to wheezy, but it's time to simplify the code.
* import Data.Default in CommonGravatar Joey Hess2015-01-28
|
* addurl: When a Content-Disposition header suggests a filename to use, addurl ↵Gravatar Joey Hess2015-01-22
| | | | will consider using it, if it's reasonable and doesn't conflict with an existing file. (--file overrides this)
* excess indentGravatar Joey Hess2015-01-22
|
* merge change from propellorGravatar Joey Hess2015-01-21
| | | | exporting Unit allows custom data units
* Revert "remove absNormPathUnix, using my absPathFrom replacement"Gravatar Joey Hess2015-01-21
| | | | | | | | | This reverts commit b6b368ed036f2e34ee4b7d39e5b41b1ba2d0a76c. Consider: relPathDirToFile (absPathFrom "/tmp/repo/xxx" "y/bar") "/tmp/repo/.git/annex/objects/xxx" This needs to always yield "../../../.git/annex/objects/xxx" but on Windows, it is "..\\..\\/tmp/repo/.git/annex/objects/xxx"
* remove absNormPathUnix, using my absPathFrom replacementGravatar Joey Hess2015-01-21
|
* update my email address and homepage urlGravatar Joey Hess2015-01-21
|
* on second thought, InodeCache should use getFileSizeGravatar Joey Hess2015-01-20
| | | | | | | | This is necessary for interop between inode caches created on unix and windows. Which is more important than supporting inodecaches for large keys with the wrong size, which are broken anyway. There should be no slowdown from this change, except on Windows.
* build fixGravatar Joey Hess2015-01-20
|
* fix upFrom of "." (and upFrom of "/" on Windows)Gravatar Joey Hess2015-01-20
|
* add getFileSize, which can get the real size of a large file on WindowsGravatar Joey Hess2015-01-20
| | | | | | | | | | | | | | Avoid using fileSize which maxes out at just 2 gb on Windows. Instead, use hFileSize, which doesn't have a bounded size. Fixes support for files > 2 gb on Windows. Note that the InodeCache code only needs to compare a file size, so it doesn't matter it the file size wraps. So it has been left as-is. This was necessary both to avoid invalidating existing inode caches, and because the code passed FileStatus around and would have become more expensive if it called getFileSize. This commit was sponsored by Christian Dietrich.
* convert parentDir to be based on takeDirectory, but fixed for trailing /Gravatar Joey Hess2015-01-09
|
* revert parentDir changeGravatar Joey Hess2015-01-09
| | | | | | | | Reverts 2bba5bc22d049272d3328bfa6c452d3e2e50e86c Unfortunately, this caused breakage on Windows, and possibly elsewhere, because parentDir and takeDirectory do not behave the same when there is a trailing directory separator.
* fix build failureGravatar Joey Hess2015-01-09
|
* Merge branch 'master' into relativepathsGravatar Joey Hess2015-01-06
|\ | | | | | | | | | | Conflicts: Locations.hs debian/changelog
| * build fixGravatar Joey Hess2015-01-06
| |
| * made parentDir return a Maybe FilePath; removed most uses of itGravatar Joey Hess2015-01-06
| | | | | | | | | | | | | | | | parentDir is less safe than takeDirectory, especially when working with relative FilePaths. It's really only useful in loops that want to terminate at / This commit was sponsored by Audric SCHILTKNECHT.
* | Switch to using relative paths to the git repository.Gravatar Joey Hess2015-01-06
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | This allows the git repository to be moved while git-annex is running in it, with fewer problems. On Windows, this avoids some of the problems with the absurdly small MAX_PATH of 260 bytes. In particular, git-annex repositories should work in deeper/longer directory structures than before. See http://git-annex.branchable.com/bugs/__34__git-annex:_direct:_1_failed__34___on_Windows/ There are several possible ways this change could break git-annex: 1. If it changes its working directory while it's running, that would be Bad News. Good news everyone! git-annex never does so. It would also break thread safety, so all such things were stomped out long ago. 2. parentDir "." -> "" which is not a valid path. I had to fix one instace of this, and I should probably wipe all calls to parentDir out of the git-annex code base; it was never a good idea. 3. Things like relPathDirToFile require absolute input paths, and code assumes that the git repo path is absolute and passes it to it as-is. In the case of relPathDirToFile, I converted it to not make this assumption. Currently, the test suite has 16 failures.
* Fix build with process 1.2.1.0.Gravatar Joey Hess2015-01-04
|
* OSX: Switched away from deprecated statfs64 interface.Gravatar Joey Hess2014-12-31
| | | | | | | | | | Getting rid of build warning warning: 'statfs64' is deprecated: first deprecated in OS X 10.6 [-Wdeprecated-declarations] 10.6 is much older than the oldest git-annex OSX port, so won't break anything.
* Windows: Got the rsync special remote working.Gravatar Joey Hess2014-12-30
| | | | | | | | | More aggressive rsync params fixup for windows. Param may contain a url, or a file path, so check if it looks like a local file path and if so, fix it up. On windows only, rsyncUrlIsPath will treat c:foo as a path, rather than as a rsyncurl starting with a host "c".
* Windows: Fix local rsync filepath munging (fixes 26 test suite failures).Gravatar Joey Hess2014-12-30
|
* final warning fixGravatar Joey Hess2014-12-29
|
* fixGravatar Joey Hess2014-12-29
|
* fix more warnings on BSD/OSXGravatar Joey Hess2014-12-29
|
* one more warningGravatar Joey Hess2014-12-29
|
* fixed all remaining build warnings on WindowsGravatar Joey Hess2014-12-29
|
* fix a OSX C compiler warning about null pointerGravatar Joey Hess2014-12-29
|