summaryrefslogtreecommitdiff
path: root/Command
Commit message (Collapse)AuthorAge
* addurl: Register transfer so the webapp can see it.Gravatar Joey Hess2013-04-11
| | | | | | * addurl: Register transfer so the webapp can see it. * addurl: Automatically retry downloads that fail, as long as some additional content was downloaded.
* addurl: Bugfix: Did not properly add file in direct mode.Gravatar Joey Hess2013-04-11
|
* webapp: New --listen= option allows running the webapp on one computer and ↵Gravatar Joey Hess2013-04-08
| | | | | | | connecting to it from another. Does not yet use HTTPS. I'd need to generate a certificate, and I'm not sure what's the best way to do that.
* Bugfix: Direct mode no longer repeatedly checksums duplicated files.Gravatar Joey Hess2013-04-06
| | | | | | | | | | | | | | | | | Fixed by storing a list of cached inodes for a key, instead of just one. Backwards compatability note: An old git-annex version will fail to parse an inode cache file that has been written by a new version, and has multiple items. It will succees if just one. So old git-annexes will have even worse behavior when there are duplicated files, if that is possible. I don't think it will be a problem. (Famous last words.) Also, note that it doesn't expire old and unused inode caches for a key. It would be possible to add this if needed; just look through the associated files for a key and if there are more cached inodes, throw out any not corresponding to associated files. Unless a file is being copied repeatedly and the old copy deleted, this lack of expiry should not be a problem.
* Use lower case hash directories for storing files on crippled filesystems, ↵Gravatar Joey Hess2013-04-04
| | | | | | | | | | | | | | | same as is already done for bare repositories. * since this is a crippled filesystem anyway, git-annex doesn't use symlinks on it * so there's no reason to use the mixed case hash directories that we're stuck using to avoid breaking everyone's symlinks to the content * so we can do what is already done for all bare repos, and make non-bare repos on crippled filesystems use the all-lower case hash directories * which are, happily, all 3 letters long, so they cannot conflict with mixed case hash directories * so I was able to 100% fix this and even resuming `git annex add` in the test case will recover and it will all just work.
* clean up urlrenderer handling when the webapp is not builtGravatar Joey Hess2013-04-03
|
* squelch protocol error on shutdownGravatar Joey Hess2013-04-02
|
* Update working tree files fully atomicallyGravatar Joey Hess2013-04-02
| | | | | | | | | | | This avoids commit churn by the assistant when eg, replacing a file with a symlink. But, just as importantly, it prevents the working tree being left with a deleted file if git-annex, or perhaps the whole system, crashes at the wrong time. (It also probably avoids confusing displays in file managers.)
* drop --auto: Fix bug that prevented dropping files from untrusted repositories.Gravatar Joey Hess2013-04-01
| | | | This is a corresponding bug to the one I fixed yesterday in the assistant.
* webapp: Added UI to delete repositories. Closes: #689847Gravatar Joey Hess2013-03-31
|
* minor refactoringGravatar Joey Hess2013-03-30
|
* let's use words and unwords rather than split and intercalate " "Gravatar Joey Hess2013-03-30
|
* Make git-annex-shell call the command with its (safe) options.Gravatar guilhem2013-03-30
|
* add --force overrides annex.largefilesGravatar Joey Hess2013-03-29
|
* New annex.largefiles setting, which configures which files `git annex add` ↵Gravatar Joey Hess2013-03-29
| | | | | | | | | | | | | and the assistant add to the annex. I would have sort of liked to put this in .gitattributes, but it seems it does not support multi-word attribute values. Also, making this a single config setting makes it easy to only parse the expression once. A natural next step would be to make the assistant `git add` files that are not annex.largefiles. OTOH, I don't think `git annex add` should `git add` such files, because git-annex command line tools are not in the business of wrapping git command line tools.
* webapp: Progess bar fixes for many types of special remotes.Gravatar Joey Hess2013-03-28
| | | | | | | | | | | | | There was confusion in different parts of the progress bar code about whether an update contained the total number of bytes transferred, or the number of bytes transferred since the last update. One way this bug showed up was progress bars that seemed to stick at zero for a long time. In order to fix it comprehensively, I add a new BytesProcessed data type, that is explicitly a total quantity of bytes, not a delta. Note that this doesn't necessarily fix every problem with progress bars. Particularly, buffering can now cause progress bars to seem to run ahead of transfers, reaching 100% when data is still being uploaded.
* Per-command usage messages.Gravatar Joey Hess2013-03-27
|
* add section metadata to all commandsGravatar Joey Hess2013-03-24
| | | | Not yet used .. mindless train work.
* 2 minor fixes to transferkeys, otherwise it was perfect 1st time!Gravatar Joey Hess2013-03-20
| | | | | | | | Needed to send a trailing NUL to end a request, and set the read handle non-blocking. Also, set fileSystemEncoding on all handles, since there's a filename in there.
* maintain pools of running transferkeys processes (untested)Gravatar Joey Hess2013-03-19
|
* implement transferkeys plumbing commandGravatar Joey Hess2013-03-19
|
* map: Combine duplicate repositories, for a nicer looking map.Gravatar Joey Hess2013-03-16
|
* fix relaxed with existing fileGravatar Joey Hess2013-03-12
|
* addurl: Add --relaxed option.Gravatar Joey Hess2013-03-11
|
* finish fast direct mode rename handling. wow, it's fastGravatar Joey Hess2013-03-11
|
* remove Eq instance for InodeCacheGravatar Joey Hess2013-03-11
| | | | | | | | | There are two types of equality here, and which one is right varies, so this forces me to consider and choose between them. Based on this, I learned that the commit in git anex sync was always doing a strong comparison, even when in a repository where the inodes had changed. Fixed that.
* skip non-directories in statusGravatar Joey Hess2013-03-11
|
* status: Can now be run with a directory path to show only the status of that ↵Gravatar Joey Hess2013-03-11
| | | | directory, rather than the whole annex.
* bugfix: drop --from an unavailable remote no longer updates the location ↵Gravatar Joey Hess2013-03-10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | log, incorrectly, to say the remote does not have the key. The comments correctly noted that the remote could drop the key and yet False be returned due to some problem that occurred afterwards. For example, if it's a network remote, it could drop the key just as the network goes down, and so things timeout and a nonzero exit from ssh is propigated through and False returned. However... Most of the time, this scenario will not have happened. False will mean the remote was not available or could not drop the key at all. So, instead of assuming the worst, just trust the status we have. If we get it wrong, and the scenario above happened, our location log will think the remote has the key. But the remote's location log (assuming it has one) will know it dropped it, and the next sync will regain consistency. For a special remote, with no location log, our location log will be wrong, but this is no different than the situation where someone else dropped the key from the remote and we've not synced with them. The standard paranoia about not trusting the location log to be the last word about whether a remote has a key will save us from these situations. Ie, if we try to drop the file, we'll actively check the remote, and determine the inconsistency then.
* add build flags to version outputGravatar Joey Hess2013-03-09
|
* two types of byNameGravatar Joey Hess2013-03-05
| | | | | | | | Clean up from 5123a1a83aa3b954fe67629508bab5ccea0e4148. In some cases, looking up a remote by name even though it has no UUID is desirable. This includes git annex sync, which can operate on remotes without an annex, and XMPP pairing, which runs addRemote (with calls byName) before the UUID of the XMPP remote has been configured in git.
* webapp: New preferences page allows enabling/disabling debug logging at ↵Gravatar Joey Hess2013-03-03
| | | | runtime, as well as configuring numcopies and diskreserve.
* git subcommand cleanupGravatar Joey Hess2013-03-03
| | | | | | Pass subcommand as a regular param, which allows passing git parameters like -c before it. This was already done in the pipeing set of functions, but not the command running set.
* break dependency cycle by special casing running of test commandGravatar Joey Hess2013-02-27
|
* embed test suite into git annex; available by running: git annex testGravatar Joey Hess2013-02-27
| | | | | | | | | | | | | | I have seen some other programs do this, and think it's pretty cool. Means you can test wherever it's deployed, as well as at build time. My other reason for doing it is less happy. Cabal's handling of test suites sucks, requiring duplicated info, and even when that's done, it fails to preprocess hsc files here. Building it in avoids that and avoids having to explicitly tell cabal to enable test suites, which would then make it link the test executable every time, which is unnecessarily slow. This also has the benefit that now "make fast test" does a max speed build and tests it.
* Makefile now builds using cabal, taking advantage of cabal's automatic ↵Gravatar Joey Hess2013-02-27
| | | | | | | | | detection of appropriate build flags. The only thing lost is ./ghci Speed: make fast used to take 20 seconds here, when rebuilding from touching Command/Unused.hs. With cabal, it's 29 seconds.
* annex.version is now set to 4 for direct mode repositoriesGravatar Joey Hess2013-02-26
| | | | | | | To avoid old versions of git-annex getting confused. There is no upgrade required though. We switch back to 3 when going from direct to indirect.
* copy: Update location log when no copy was performed, if the location log ↵Gravatar Joey Hess2013-02-26
| | | | was out of date.
* make adding modified files work on crippled filesystemsGravatar Joey Hess2013-02-20
|
* Direct mode: Fix support for adding a modified file.Gravatar Joey Hess2013-02-20
| | | | | | | | | | | Adding a file that is already annexed, but has been modified, was broken in direct mode. This fix makes the new content be added. It does have the problem that re-running `git annex add` will checksum and re-add the content repeatedly, until it's committed. This happens because the key associated with the file does not change until the new one gets committed, so it keeps thinking the file has changed.
* fully support core.symlinks=false in all relevant symlink handling codeGravatar Joey Hess2013-02-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refactored annex link code into nice clean new library. Audited and dealt with calls to createSymbolicLink. Remaining calls are all safe, because: Annex/Link.hs: ( liftIO $ createSymbolicLink linktarget file only when core.symlinks=true Assistant/WebApp/Configurators/Local.hs: createSymbolicLink link link test if symlinks can be made Command/Fix.hs: liftIO $ createSymbolicLink link file command only works in indirect mode Command/FromKey.hs: liftIO $ createSymbolicLink link file command only works in indirect mode Command/Indirect.hs: liftIO $ createSymbolicLink l f refuses to run if core.symlinks=false Init.hs: createSymbolicLink f f2 test if symlinks can be made Remote/Directory.hs: go [file] = catchBoolIO $ createSymbolicLink file f >> return True fast key linking; catches failure to make symlink and falls back to copy Remote/Git.hs: liftIO $ catchBoolIO $ createSymbolicLink loc file >> return True ditto Upgrade/V1.hs: liftIO $ createSymbolicLink link f v1 repos could not be on a filesystem w/o symlinks Audited and dealt with calls to readSymbolicLink. Remaining calls are all safe, because: Annex/Link.hs: ( liftIO $ catchMaybeIO $ readSymbolicLink file only when core.symlinks=true Assistant/Threads/Watcher.hs: ifM ((==) (Just link) <$> liftIO (catchMaybeIO $ readSymbolicLink file)) code that fixes real symlinks when inotify sees them It's ok to not fix psdueo-symlinks. Assistant/Threads/Watcher.hs: mlink <- liftIO (catchMaybeIO $ readSymbolicLink file) ditto Command/Fix.hs: stopUnless ((/=) (Just link) <$> liftIO (catchMaybeIO $ readSymbolicLink file)) $ do command only works in indirect mode Upgrade/V1.hs: getsymlink = takeFileName <$> readSymbolicLink file v1 repos could not be on a filesystem w/o symlinks Audited and dealt with calls to isSymbolicLink. (Typically used with getSymbolicLinkStatus, but that is just used because getFileStatus is not as robust; it also works on pseudolinks.) Remaining calls are all safe, because: Assistant/Threads/SanityChecker.hs: | isSymbolicLink s -> addsymlink file ms only handles staging of symlinks that were somehow not staged (might need to be updated to support pseudolinks, but this is only a belt-and-suspenders check anyway, and I've never seen the code run) Command/Add.hs: if isSymbolicLink s || not (isRegularFile s) avoids adding symlinks to the annex, so not relevant Command/Indirect.hs: | isSymbolicLink s -> void $ flip whenAnnexed f $ only allowed on systems that support symlinks Command/Indirect.hs: whenM (liftIO $ not . isSymbolicLink <$> getSymbolicLinkStatus f) $ do ditto Seek.hs:notSymlink f = liftIO $ not . isSymbolicLink <$> getSymbolicLinkStatus f used to find unlocked files, only relevant in indirect mode Utility/FSEvents.hs: | Files.isSymbolicLink s = runhook addSymlinkHook $ Just s Utility/FSEvents.hs: | Files.isSymbolicLink s -> Utility/INotify.hs: | Files.isSymbolicLink s -> Utility/INotify.hs: checkfiletype Files.isSymbolicLink addSymlinkHook f Utility/Kqueue.hs: | Files.isSymbolicLink s = callhook addSymlinkHook (Just s) change all above are lower-level, not relevant Audited and dealt with calls to isSymLink. Remaining calls are all safe, because: Annex/Direct.hs: | isSymLink (getmode item) = This is looking at git diff-tree objects, not files on disk Command/Unused.hs: | isSymLink (LsTree.mode l) = do This is looking at git ls-tree, not file on disk Utility/FileMode.hs:isSymLink :: FileMode -> Bool Utility/FileMode.hs:isSymLink = checkMode symbolicLinkMode low-level Done!!
* proper fix for dropunusedGravatar Joey Hess2013-02-15
| | | | | | | | | | Now getKeysPresent checks that the key's content, not only its directory, exists. In direct mode, the inode cache file is used as a standin for the content. removeAnnex always removes the inode cache file, and drop and move --from always call removeAnnex, even if the object does not seem to be inAnnex, to ensure it's always deleted.
* Revert "Clean up direct mode cache and mapping info when dropping keys."Gravatar Joey Hess2013-02-15
| | | | | | | | | This reverts commit f797939d15a2b414e62b28ccb0bd9e5b77978d76. This was buggy, it caused the direct mode cache to be lost when dropping keys, so when the file is gotten back, it's stored in indirect mode. Note to self: Do not attempt bug fixes at 6 am!
* don't allow setting indirect mode on a crippled filesystemGravatar Joey Hess2013-02-15
|
* migrate, rekey: copy rather than hard linking in crippled filesystem modeGravatar Joey Hess2013-02-15
|
* add: Improved detection of files that are modified while being added.Gravatar Joey Hess2013-02-14
| | | | | | | | | | | | | In indirect mode, now checks the inode cache to detect changes to a file. Note that a file can still be changed if a process has it open for write, after landing in the annex. In direct mode, some checking of the inode cache was done before, but from a much later point, so fewer modifications could be detected. Now it's as good as indirect mode. On crippled filesystems, no lock down is done before starting to add a file, so checking the inode cache is the only protection we have.
* split out Utility.InodeCacheGravatar Joey Hess2013-02-14
|
* crippled filesystem support, probing and initial supportGravatar Joey Hess2013-02-14
| | | | | | | | | | | | git annex init probes for crippled filesystems, and sets direct mode, as well as `annex.crippledfilesystem`. Avoid manipulating permissions of files on crippled filesystems. That would likely cause an exception to be thrown. Very basic support in Command.Add for cripped filesystems; avoids the lock down entirely since doing it needs both permissions and hard links. Will make this better soon.
* can now build Android targeted binaryGravatar Joey Hess2013-02-10
| | | | | | | | | | | | | | | | Various things that don't work on Android are just ifdefed out. * the webapp (needs template haskell for arm) * --include and --exclude globbing (needs libpcre, which is not ported; probably I'll make it use the pure haskell glob library instead) * annex.diskreserve checking (missing sys/statvfs.h) * timestamp preservation support (yawn) * S3 * WebDAV * XMPP The resulting 17mb binary has been tested on Android, and it is able to, at least, print its usage message.
* Clean up direct mode cache and mapping info when dropping keys.Gravatar Joey Hess2013-02-07
| | | | | | | | | | | | | | These files were left behind, and made getKeysPresent find keys that were not present. It would be expensive to make getKeysPresent check that the actual key files are present (it just lists the directories). But that's not needed if we just clean up the stale cache and mapping files. To handle systems that were in direct mode and got switched back with stale direct mode files, made cleanObjectLoc remove all files in the key's directory. git annex unused will still list keys that are gone but for which the stale direct mode files exists. To deal with that, made dropunused remove the key's directory even if the key does not seem to be present.