summaryrefslogtreecommitdiff
path: root/Annex/Direct.hs
Commit message (Collapse)AuthorAge
* finish fixing direct mode merge bug involving unstaged local filesGravatar Joey Hess2014-03-04
| | | | | | | | | | Added test cases for both ways this can happen, with a conflict involving a file, or a directory. Cleaned up resolveMerge to not touch the work tree in direct mode, which turned out to be the only way to handle things.. And makes it much nicer. Still need to run test suite on windows.
* fix a9067868a8594577ead2ecbe55f9563bef12f26d to handle annexed files in ↵Gravatar Joey Hess2014-03-03
| | | | | | | | | | | | | | | conflicted merge In the case of a conflicted merge where the remote adds a directory, and we have a file (which is checked in), resolveMerge' will create the link, and so the fix for a9067868a8594577ead2ecbe55f9563bef12f26d looked at that, thought it was an unannexed file (it's not in the oldref), and preserved it. This is a hacky fix. It would be better for resolveMerge' to not update the work tree, at least in direct mode, and only stage the changes, which mergeDirectCleanUp could then move into tree. I want to make that change, but this is not the time to do it.
* sync: Fix bug in direct mode that caused a file not checked into git to be ↵Gravatar Joey Hess2014-03-03
| | | | deleted when merging with a remote that added a file by the same name. (Thanks, jkt)
* remove Read instance for RefGravatar Joey Hess2014-02-19
| | | | | | | | Removed instance, got it all to build using fromRef. (With a few things that really need to show something using a ref for debugging stubbed out.) Then added back Read instance, and made Logs.View use it for serialization. This changes the view log format.
* random hlint (to give the autobuilder something new to build)Gravatar Joey Hess2014-02-11
|
* Fix direct mode's handling when modifications to non-annexed files are ↵Gravatar Joey Hess2013-12-12
| | | | pulled from a remote. A bug prevented the files from being updated in the work tree, and this caused the modification to be reverted.
* format commentGravatar Joey Hess2013-12-12
|
* Avoid using git commit in direct mode, since in some situations it will read ↵Gravatar Joey Hess2013-12-01
| | | | | | | | | | | | the full contents of files in the tree. The assistant's commit code also always avoids git commit, for simplicity. Indirect mode sync still does a git commit -a to catch unstaged changes. Note that this means that direct mode sync no longer runs the pre-commit hook or any other hooks git commit might call. The git annex pre-commit hook action for direct mode is however explicitly run. (The assistant already ran git commit with hooks disabled, so no change there.)
* Direct mode .git/annex/objects directories are no longer left writableGravatar Joey Hess2013-11-15
| | | | | | | | | Because that allowed writing to symlinks of files that are not present, which followed the link and put bad content in an object location. fsck: Fix up .git/annex/object directory permissions. This commit was sponsored by an anonymous bitcoin donor.
* Fix direct mode merge bug when a direct mode file was deleted and replaced ↵Gravatar Joey Hess2013-11-15
| | | | with a directory. An ordering problem caused the directory to not get created in this case. Thanks to Tim for the test cases.
* work around lack of receive.denyCurrentBranch in direct modeGravatar Joey Hess2013-11-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | Now that direct mode sets core.bare=true, git's normal prohibition about pushing into the currently checked out branch doesn't work. A simple fix for this would be an update hook which blocks the pushes.. but git hooks must be executable, and git-annex needs to be usable on eg, FAT, which lacks x bits. Instead, enabling direct mode switches the branch (eg master) to a special purpose branch (eg annex/direct/master). This branch is not pushed when syncing; instead any changes that git annex sync commits get written to master, and it's pushed (along with synced/master) to the remote. Note that initialization has been changed to always call setDirect, even if it's just setDirect False for indirect mode. This is needed because if the user has just cloned a direct mode repo, that nothing has synced with before, it may have no master branch, and only a annex/direct/master. Resulting in that branch being checked out locally too. Calling setDirect False for indirect mode moves back out of this branch, to a new master branch, and ensures that a manual "git push" doesn't push changes directly to the annex/direct/master of the remote. (It's possible that the user makes a commit w/o using git-annex and pushes it, but nothing I can do about that really.) This commit was sponsored by Jonathan Harrington.
* v5 for direct mode, with automatic upgradeGravatar Joey Hess2013-11-05
| | | | | This includes storing the current state of the HEAD ref, which git annex sync is going to need, but does not make sync use it.
* update for DiffTree type change (which fixes assistant in subdir confusion bug)Gravatar Joey Hess2013-10-17
|
* ensure merge directory is empty before starting mergeGravatar Joey Hess2013-10-16
| | | | Don't want some past failed merge to lead to bad results, potentially.
* more completely solve catKey memory leakGravatar Joey Hess2013-09-19
| | | | | | | | | | | | | | | | | | | Done using a mode witness, which ensures it's fixed everywhere. Fixing catFileKey was a bear, because git cat-file does not provide a nice way to query for the mode of a file and there is no other efficient way to do it. Oh, for libgit2.. Note that I am looking at tree objects from HEAD, rather than the index. Because I cat-file cannot show a tree object for the index. So this fix is technically incomplete. The only cases where it matters are: 1. A new large file has been directly staged in git, but not committed. 2. A file that was committed to HEAD as a symlink has been staged directly in the index. This could be fixed a lot better using libgit2.
* sync, pre-commit, indirect: Avoid unnecessarily catting non-symlink files ↵Gravatar Joey Hess2013-09-19
| | | | from git, which can be so large it runs out of memory.
* sync: Fix bug that caused direct mode mappings to not be updated when ↵Gravatar Joey Hess2013-09-13
| | | | merging files into the tree on Windows.
* importfeed: git-annex becomes a podcatcher in 150 LOCGravatar Joey Hess2013-07-28
|
* fsck: Don't claim to fix direct mode when run on a symlink whose content is ↵Gravatar Joey Hess2013-07-08
| | | | not present.
* make direct mode merge cleanup more robustGravatar Joey Hess2013-05-25
| | | | | | | | If the cleanup of a single file fails for some reason, continue to clean up other files. This could happen because of a race. The merge pulls in a change to a file, which gets changed locally at the same time.
* improve robustness of fromDirect and replaceFileGravatar Joey Hess2013-05-25
| | | | | | | | | Made fromDirect check that a file in the tree has good content (and is not a broken symlink either) before copying it to another file that has the same key. Made replaceFile clean up the temp file if the action that creates it, or the file replacement action fails.
* update inode cache after copying contentGravatar Joey Hess2013-05-20
| | | | | | | This was also tripped by the test suite's automatic conflict resolution test. Which also shows BTW that an unnecessary copy of content is done sometimes when merging in direct mode. Not going to try to speed that up now.
* didn't quite get removeDirect right before, this passes test suiteGravatar Joey Hess2013-05-20
|
* minor optimisation and warning fixGravatar Joey Hess2013-05-20
|
* direct mode bug fix: After a conflicted merge was automatically resolved, ↵Gravatar Joey Hess2013-05-20
| | | | | | | | | | | | | | | | | the content of a file that was already present could incorrectly be replaced with a symlink. The bug was in movein, which just replaceFile'd the file with a symlink, even if it already had the desired content, before trying to pull the content out of the annex and replace the symlink with it. That was ok-ish for non conflicted merges, where if the file existed it would be an old version of the content. But for conflicted merges, the automatic merge resolver has already run, and will have already put the desired content into the file for the local variant. Also, made removeDirect not trust that the associated files map is correct. Only if it can verify that another file has the content will it not move it into .git/annex/objects.
* test suite passes in direct modeGravatar Joey Hess2013-05-17
| | | | | | | | | | | This fixes a bug with git annex add in direct mode. If some files already existed in the tree pointing at the same key as a file that was just added, and their content was not present, add neglected to copy the content to those files. I also changed the behavior of moveAnnex slightly: When content is moved into the annex in direct mode, it does not overwrite any content already present in direct mode files. That content may be modified after all.
* fix toDirectGen bug introduced in 7d345fcb307cfecb5fa834a0b17be01e4e6f5df8Gravatar Joey Hess2013-05-15
|
* direct: Fix a bug that could cause some files to be left in indirect mode.Gravatar Joey Hess2013-05-06
| | | | | | | | | It's possible for files in indirect mode to have a direct mode mapping file. Probably from when they were in direct mode. In this case, toDirectGen tried to copy the content from the direct mode file that the mapping said had it. But, being in indirect mode, it didn't really have the content. So it did nothing. This fix makes it always move the content from .git/annex/objects/ when it's there.
* thaw content directory in direct mode tooGravatar Joey Hess2013-04-30
| | | | | | | | A content directory can be frozen in direct mode. One way this can happen is if the content is transferred before direct mode has a mapping for it, so it's stored in the content directory. So, we need to thaw the content directory before doing things with it.
* sync: Bug fix, avoid adding to the annex the dummy symlinks used on crippled ↵Gravatar Joey Hess2013-04-23
| | | | | | | | | | | | | | | | | | | filesystems. The root of the problem is that toInodeCache sees a non-symlink, and so goes on and generates a new inode cache for the dummy symlink. Any place that toInodeCache, or sameFileStatus, or genInodeCache are called may need to deal with this case. Although many of them are ok. For example, prepSendAnnex calls sameInodeCache, which calls genInodeCache.. but if the file content is not present, the InodeCache generated for its standin file is appropriately not the same, and so it returns Nothing. I've audited some, but have to say I'm not happy with this; it should be handled at the type level somehow, or a toInodeCache wrapper be used that is aware of dummy symlinks. (The Watcher already dealt with it, via the guardSymlinkStandin function.)
* Bugfix: Direct mode no longer repeatedly checksums duplicated files.Gravatar Joey Hess2013-04-06
| | | | | | | | | | | | | | | | | Fixed by storing a list of cached inodes for a key, instead of just one. Backwards compatability note: An old git-annex version will fail to parse an inode cache file that has been written by a new version, and has multiple items. It will succees if just one. So old git-annexes will have even worse behavior when there are duplicated files, if that is possible. I don't think it will be a problem. (Famous last words.) Also, note that it doesn't expire old and unused inode caches for a key. It would be possible to add this if needed; just look through the associated files for a key and if there are more cached inodes, throw out any not corresponding to associated files. Unless a file is being copied repeatedly and the old copy deleted, this lack of expiry should not be a problem.
* Use lower case hash directories for storing files on crippled filesystems, ↵Gravatar Joey Hess2013-04-04
| | | | | | | | | | | | | | | same as is already done for bare repositories. * since this is a crippled filesystem anyway, git-annex doesn't use symlinks on it * so there's no reason to use the mixed case hash directories that we're stuck using to avoid breaking everyone's symlinks to the content * so we can do what is already done for all bare repos, and make non-bare repos on crippled filesystems use the all-lower case hash directories * which are, happily, all 3 letters long, so they cannot conflict with mixed case hash directories * so I was able to 100% fix this and even resuming `git annex add` in the test case will recover and it will all just work.
* hlintGravatar Joey Hess2013-04-03
|
* Update working tree files fully atomicallyGravatar Joey Hess2013-04-02
| | | | | | | | | | | This avoids commit churn by the assistant when eg, replacing a file with a symlink. But, just as importantly, it prevents the working tree being left with a deleted file if git-annex, or perhaps the whole system, crashes at the wrong time. (It also probably avoids confusing displays in file managers.)
* remove Eq instance for InodeCacheGravatar Joey Hess2013-03-11
| | | | | | | | | There are two types of equality here, and which one is right varies, so this forces me to consider and choose between them. Based on this, I learned that the commit in git anex sync was always doing a strong comparison, even when in a repository where the inodes had changed. Fixed that.
* Direct mode: Support filesystems like FAT which can change their inodes each ↵Gravatar Joey Hess2013-02-19
| | | | time they are mounted.
* hlintGravatar Joey Hess2013-02-18
|
* fully support core.symlinks=false in all relevant symlink handling codeGravatar Joey Hess2013-02-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refactored annex link code into nice clean new library. Audited and dealt with calls to createSymbolicLink. Remaining calls are all safe, because: Annex/Link.hs: ( liftIO $ createSymbolicLink linktarget file only when core.symlinks=true Assistant/WebApp/Configurators/Local.hs: createSymbolicLink link link test if symlinks can be made Command/Fix.hs: liftIO $ createSymbolicLink link file command only works in indirect mode Command/FromKey.hs: liftIO $ createSymbolicLink link file command only works in indirect mode Command/Indirect.hs: liftIO $ createSymbolicLink l f refuses to run if core.symlinks=false Init.hs: createSymbolicLink f f2 test if symlinks can be made Remote/Directory.hs: go [file] = catchBoolIO $ createSymbolicLink file f >> return True fast key linking; catches failure to make symlink and falls back to copy Remote/Git.hs: liftIO $ catchBoolIO $ createSymbolicLink loc file >> return True ditto Upgrade/V1.hs: liftIO $ createSymbolicLink link f v1 repos could not be on a filesystem w/o symlinks Audited and dealt with calls to readSymbolicLink. Remaining calls are all safe, because: Annex/Link.hs: ( liftIO $ catchMaybeIO $ readSymbolicLink file only when core.symlinks=true Assistant/Threads/Watcher.hs: ifM ((==) (Just link) <$> liftIO (catchMaybeIO $ readSymbolicLink file)) code that fixes real symlinks when inotify sees them It's ok to not fix psdueo-symlinks. Assistant/Threads/Watcher.hs: mlink <- liftIO (catchMaybeIO $ readSymbolicLink file) ditto Command/Fix.hs: stopUnless ((/=) (Just link) <$> liftIO (catchMaybeIO $ readSymbolicLink file)) $ do command only works in indirect mode Upgrade/V1.hs: getsymlink = takeFileName <$> readSymbolicLink file v1 repos could not be on a filesystem w/o symlinks Audited and dealt with calls to isSymbolicLink. (Typically used with getSymbolicLinkStatus, but that is just used because getFileStatus is not as robust; it also works on pseudolinks.) Remaining calls are all safe, because: Assistant/Threads/SanityChecker.hs: | isSymbolicLink s -> addsymlink file ms only handles staging of symlinks that were somehow not staged (might need to be updated to support pseudolinks, but this is only a belt-and-suspenders check anyway, and I've never seen the code run) Command/Add.hs: if isSymbolicLink s || not (isRegularFile s) avoids adding symlinks to the annex, so not relevant Command/Indirect.hs: | isSymbolicLink s -> void $ flip whenAnnexed f $ only allowed on systems that support symlinks Command/Indirect.hs: whenM (liftIO $ not . isSymbolicLink <$> getSymbolicLinkStatus f) $ do ditto Seek.hs:notSymlink f = liftIO $ not . isSymbolicLink <$> getSymbolicLinkStatus f used to find unlocked files, only relevant in indirect mode Utility/FSEvents.hs: | Files.isSymbolicLink s = runhook addSymlinkHook $ Just s Utility/FSEvents.hs: | Files.isSymbolicLink s -> Utility/INotify.hs: | Files.isSymbolicLink s -> Utility/INotify.hs: checkfiletype Files.isSymbolicLink addSymlinkHook f Utility/Kqueue.hs: | Files.isSymbolicLink s = callhook addSymlinkHook (Just s) change all above are lower-level, not relevant Audited and dealt with calls to isSymLink. Remaining calls are all safe, because: Annex/Direct.hs: | isSymLink (getmode item) = This is looking at git diff-tree objects, not files on disk Command/Unused.hs: | isSymLink (LsTree.mode l) = do This is looking at git ls-tree, not file on disk Utility/FileMode.hs:isSymLink :: FileMode -> Bool Utility/FileMode.hs:isSymLink = checkMode symbolicLinkMode low-level Done!!
* start to support core.symlinks=falseGravatar Joey Hess2013-02-15
| | | | | Utility functions to handle no symlink mode, and converted Annex.Content to use them; still many other places to convert.
* add: Improved detection of files that are modified while being added.Gravatar Joey Hess2013-02-14
| | | | | | | | | | | | | In indirect mode, now checks the inode cache to detect changes to a file. Note that a file can still be changed if a process has it open for write, after landing in the annex. In direct mode, some checking of the inode cache was done before, but from a much later point, so fewer modifications could be detected. Now it's as good as indirect mode. On crippled filesystems, no lock down is done before starting to add a file, so checking the inode cache is the only protection we have.
* split out Utility.InodeCacheGravatar Joey Hess2013-02-14
|
* crippled filesystem support, probing and initial supportGravatar Joey Hess2013-02-14
| | | | | | | | | | | | git annex init probes for crippled filesystems, and sets direct mode, as well as `annex.crippledfilesystem`. Avoid manipulating permissions of files on crippled filesystems. That would likely cause an exception to be thrown. Very basic support in Command.Add for cripped filesystems; avoids the lock down entirely since doing it needs both permissions and hard links. Will make this better soon.
* ensure that content directory is thawed when writing direct mode mapping and ↵Gravatar Joey Hess2013-01-26
| | | | cache files
* some more direct mode fixesGravatar Joey Hess2013-01-18
| | | | | | | | Avoid a crash if a mapping contains files that no longer exist. This could happen because eg, one was deleted and a commit has not yet been done to update the mapping. Fix path calculation.
* Fix direct mode mapping code to always store direct mode filenames relative ↵Gravatar Joey Hess2013-01-18
| | | | to the top of the repository, even when operating inside a subdirectory.
* In direct mode, files with the same key are no longer hardlinked, as that ↵Gravatar Joey Hess2013-01-14
| | | | would cause a surprising behavior if modifying one, where the other would also change.
* add works in direct modeGravatar Joey Hess2013-01-06
| | | | | Also, changed sync to no longer automatically add files in direct mode. That was only necessary before because add didn't work.
* direct: Avoid hardlinking symlinks that point to the same content when the ↵Gravatar Joey Hess2013-01-06
| | | | content is not present.
* assistant direct mode file add/change bookkeepingGravatar Joey Hess2012-12-25
| | | | | | | | | | | | | | When a file is changed in direct mode, the old content is probably lost (at least from the local repo), and bookeeping needs to be updated to reflect this. Also, synthetic add events are generated at assistant startup, so make it detect when the file has not really changed, and avoid re-adding it. This does add the overhead of querying the runing git cat-file for the key that's recorded in git for the file, each time a file is added or modified in direct mode.
* commentGravatar Joey Hess2012-12-19
|