summaryrefslogtreecommitdiff
path: root/Annex
Commit message (Collapse)AuthorAge
* Use lower case hash directories for storing files on crippled filesystems, ↵Gravatar Joey Hess2013-04-04
| | | | | | | | | | | | | | | same as is already done for bare repositories. * since this is a crippled filesystem anyway, git-annex doesn't use symlinks on it * so there's no reason to use the mixed case hash directories that we're stuck using to avoid breaking everyone's symlinks to the content * so we can do what is already done for all bare repos, and make non-bare repos on crippled filesystems use the all-lower case hash directories * which are, happily, all 3 letters long, so they cannot conflict with mixed case hash directories * so I was able to 100% fix this and even resuming `git annex add` in the test case will recover and it will all just work.
* hlintGravatar Joey Hess2013-04-03
|
* cleanupGravatar Joey Hess2013-04-02
|
* Update working tree files fully atomicallyGravatar Joey Hess2013-04-02
| | | | | | | | | | | This avoids commit churn by the assistant when eg, replacing a file with a symlink. But, just as importantly, it prevents the working tree being left with a deleted file if git-annex, or perhaps the whole system, crashes at the wrong time. (It also probably avoids confusing displays in file managers.)
* New annex.largefiles setting, which configures which files `git annex add` ↵Gravatar Joey Hess2013-03-29
| | | | | | | | | | | | | and the assistant add to the annex. I would have sort of liked to put this in .gitattributes, but it seems it does not support multi-word attribute values. Also, making this a single config setting makes it easy to only parse the expression once. A natural next step would be to make the assistant `git add` files that are not annex.largefiles. OTOH, I don't think `git annex add` should `git add` such files, because git-annex command line tools are not in the business of wrapping git command line tools.
* cleanup debug printGravatar Joey Hess2013-03-28
|
* comment typoGravatar Joey Hess2013-03-18
|
* move comment to right placeGravatar Joey Hess2013-03-18
|
* remove old debug printGravatar Joey Hess2013-03-16
|
* Bugfix: Fix bug in inode cache sentinal check, which broke copying to local ↵Gravatar Joey Hess2013-03-12
| | | | repos if the repo being copied from had moved to a different filesystem or otherwise changed all its inodes'
* detect renames during commit, and .. um, do nothing special because it's ↵Gravatar Joey Hess2013-03-11
| | | | | | lunch time But I'm well set up to fast-track direct mode adds for renames now.
* remove Eq instance for InodeCacheGravatar Joey Hess2013-03-11
| | | | | | | | | There are two types of equality here, and which one is right varies, so this forces me to consider and choose between them. Based on this, I learned that the commit in git anex sync was always doing a strong comparison, even when in a repository where the inodes had changed. Fixed that.
* tag xmpp pushes with jidGravatar Joey Hess2013-03-06
| | | | | | | | | | This fixes the issue mentioned in the last commit. Turns out just collecting UUID of clients behind a XMPP remote is insufficient (although I should probably still do it for other reasons), because a single remote repo might be connected via both XMPP and local pairing. So a way is needed to know when a push was received from any client using a given XMPP remote over XMPP, as opposed to via ssh.
* assistant: Get back in sync with XMPP remotes after network reconnection, ↵Gravatar Joey Hess2013-03-06
| | | | | | | | | | | | | | | | | | | | | and on startup. Make manualPull send push requests over XMPP. When reconnecting with remotes, those that are XMPP remotes cannot immediately be pulled from and scanned, so instead maintain a set of (probably) desynced remotes, and put XMPP remotes on it. (This set could be used in other ways later, if we can detect we're out of sync with other types of remotes.) The merger handles detecting when a XMPP push is received from a desynced remote, and triggers a scan then, if they have in fact diverged. This has one known bug: A single XMPP remote can have multiple clients behind it. When this happens, only the UUID of one client is recorded as the UUID of the XMPP remote. Pushes from the other XMPP clients will not trigger a scan. If the client whose UUID is expected responds to the push request, it'll work, but when that client is offline, we're SOL.
* Run ssh with -T to avoid tty allocation and any login scripts that may do ↵Gravatar Joey Hess2013-03-04
| | | | undesired things with it.
* git subcommand cleanupGravatar Joey Hess2013-03-03
| | | | | | Pass subcommand as a regular param, which allows passing git parameters like -c before it. This was already done in the pipeing set of functions, but not the command running set.
* Makefile now builds using cabal, taking advantage of cabal's automatic ↵Gravatar Joey Hess2013-02-27
| | | | | | | | | detection of appropriate build flags. The only thing lost is ./ghci Speed: make fast used to take 20 seconds here, when rebuilding from touching Command/Unused.hs. With cabal, it's 29 seconds.
* annex.version is now set to 4 for direct mode repositoriesGravatar Joey Hess2013-02-26
| | | | | | | To avoid old versions of git-annex getting confused. There is no upgrade required though. We switch back to 3 when going from direct to indirect.
* fixGravatar Joey Hess2013-02-24
|
* hopefully fix a bugGravatar Joey Hess2013-02-24
|
* remove debug printGravatar Joey Hess2013-02-23
|
* squelch warningGravatar Joey Hess2013-02-22
|
* fix inverted logicGravatar Joey Hess2013-02-22
|
* fix sameInodeCache to check the inode change sentinalGravatar Joey Hess2013-02-22
| | | | | This should fix the problem where the assistant, on Android, re-adds every file on startup.
* work around broken getEnvironment on Android in the most important place: ↵Gravatar Joey Hess2013-02-22
| | | | | | | | git annex init This resulted in a lot of user complains that git annex init had git telling them they needed to run git config --global user.email .. which didn't work because even HOME was not passed into git.
* only create inode sentinal file when initializing a new repoGravatar Joey Hess2013-02-20
|
* shorter directory to external ssh socketGravatar Joey Hess2013-02-19
| | | | Before it was too long to be used.
* Direct mode: Support filesystems like FAT which can change their inodes each ↵Gravatar Joey Hess2013-02-19
| | | | time they are mounted.
* Android: Support ssh connection caching.Gravatar Joey Hess2013-02-19
|
* set fileSystemEncoding when reading files that might be binaryGravatar Joey Hess2013-02-18
|
* hlintGravatar Joey Hess2013-02-18
|
* typesGravatar Joey Hess2013-02-18
|
* fully support core.symlinks=false in all relevant symlink handling codeGravatar Joey Hess2013-02-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refactored annex link code into nice clean new library. Audited and dealt with calls to createSymbolicLink. Remaining calls are all safe, because: Annex/Link.hs: ( liftIO $ createSymbolicLink linktarget file only when core.symlinks=true Assistant/WebApp/Configurators/Local.hs: createSymbolicLink link link test if symlinks can be made Command/Fix.hs: liftIO $ createSymbolicLink link file command only works in indirect mode Command/FromKey.hs: liftIO $ createSymbolicLink link file command only works in indirect mode Command/Indirect.hs: liftIO $ createSymbolicLink l f refuses to run if core.symlinks=false Init.hs: createSymbolicLink f f2 test if symlinks can be made Remote/Directory.hs: go [file] = catchBoolIO $ createSymbolicLink file f >> return True fast key linking; catches failure to make symlink and falls back to copy Remote/Git.hs: liftIO $ catchBoolIO $ createSymbolicLink loc file >> return True ditto Upgrade/V1.hs: liftIO $ createSymbolicLink link f v1 repos could not be on a filesystem w/o symlinks Audited and dealt with calls to readSymbolicLink. Remaining calls are all safe, because: Annex/Link.hs: ( liftIO $ catchMaybeIO $ readSymbolicLink file only when core.symlinks=true Assistant/Threads/Watcher.hs: ifM ((==) (Just link) <$> liftIO (catchMaybeIO $ readSymbolicLink file)) code that fixes real symlinks when inotify sees them It's ok to not fix psdueo-symlinks. Assistant/Threads/Watcher.hs: mlink <- liftIO (catchMaybeIO $ readSymbolicLink file) ditto Command/Fix.hs: stopUnless ((/=) (Just link) <$> liftIO (catchMaybeIO $ readSymbolicLink file)) $ do command only works in indirect mode Upgrade/V1.hs: getsymlink = takeFileName <$> readSymbolicLink file v1 repos could not be on a filesystem w/o symlinks Audited and dealt with calls to isSymbolicLink. (Typically used with getSymbolicLinkStatus, but that is just used because getFileStatus is not as robust; it also works on pseudolinks.) Remaining calls are all safe, because: Assistant/Threads/SanityChecker.hs: | isSymbolicLink s -> addsymlink file ms only handles staging of symlinks that were somehow not staged (might need to be updated to support pseudolinks, but this is only a belt-and-suspenders check anyway, and I've never seen the code run) Command/Add.hs: if isSymbolicLink s || not (isRegularFile s) avoids adding symlinks to the annex, so not relevant Command/Indirect.hs: | isSymbolicLink s -> void $ flip whenAnnexed f $ only allowed on systems that support symlinks Command/Indirect.hs: whenM (liftIO $ not . isSymbolicLink <$> getSymbolicLinkStatus f) $ do ditto Seek.hs:notSymlink f = liftIO $ not . isSymbolicLink <$> getSymbolicLinkStatus f used to find unlocked files, only relevant in indirect mode Utility/FSEvents.hs: | Files.isSymbolicLink s = runhook addSymlinkHook $ Just s Utility/FSEvents.hs: | Files.isSymbolicLink s -> Utility/INotify.hs: | Files.isSymbolicLink s -> Utility/INotify.hs: checkfiletype Files.isSymbolicLink addSymlinkHook f Utility/Kqueue.hs: | Files.isSymbolicLink s = callhook addSymlinkHook (Just s) change all above are lower-level, not relevant Audited and dealt with calls to isSymLink. Remaining calls are all safe, because: Annex/Direct.hs: | isSymLink (getmode item) = This is looking at git diff-tree objects, not files on disk Command/Unused.hs: | isSymLink (LsTree.mode l) = do This is looking at git ls-tree, not file on disk Utility/FileMode.hs:isSymLink :: FileMode -> Bool Utility/FileMode.hs:isSymLink = checkMode symbolicLinkMode low-level Done!!
* proper fix for dropunusedGravatar Joey Hess2013-02-15
| | | | | | | | | | Now getKeysPresent checks that the key's content, not only its directory, exists. In direct mode, the inode cache file is used as a standin for the content. removeAnnex always removes the inode cache file, and drop and move --from always call removeAnnex, even if the object does not seem to be inAnnex, to ensure it's always deleted.
* Revert "Clean up direct mode cache and mapping info when dropping keys."Gravatar Joey Hess2013-02-15
| | | | | | | | | This reverts commit f797939d15a2b414e62b28ccb0bd9e5b77978d76. This was buggy, it caused the direct mode cache to be lost when dropping keys, so when the file is gotten back, it's stored in indirect mode. Note to self: Do not attempt bug fixes at 6 am!
* start to support core.symlinks=falseGravatar Joey Hess2013-02-15
| | | | | Utility functions to handle no symlink mode, and converted Annex.Content to use them; still many other places to convert.
* add: Improved detection of files that are modified while being added.Gravatar Joey Hess2013-02-14
| | | | | | | | | | | | | In indirect mode, now checks the inode cache to detect changes to a file. Note that a file can still be changed if a process has it open for write, after landing in the annex. In direct mode, some checking of the inode cache was done before, but from a much later point, so fewer modifications could be detected. Now it's as good as indirect mode. On crippled filesystems, no lock down is done before starting to add a file, so checking the inode cache is the only protection we have.
* split out Utility.InodeCacheGravatar Joey Hess2013-02-14
|
* crippled filesystem support, probing and initial supportGravatar Joey Hess2013-02-14
| | | | | | | | | | | | git annex init probes for crippled filesystems, and sets direct mode, as well as `annex.crippledfilesystem`. Avoid manipulating permissions of files on crippled filesystems. That would likely cause an exception to be thrown. Very basic support in Command.Add for cripped filesystems; avoids the lock down entirely since doing it needs both permissions and hard links. Will make this better soon.
* Now uses the Haskell uuid library, rather than needing a uuid program.Gravatar Joey Hess2013-02-10
| | | | | | | | Been meaning to do this for some time; Android port was last straw. Note that newer versions of the uuid library have a Data.UUID.V4 that generates random UUIDs slightly more cleanly, but Debian has an old version of the library, so I do it slightly round-about.
* Clean up direct mode cache and mapping info when dropping keys.Gravatar Joey Hess2013-02-07
| | | | | | | | | | | | | | These files were left behind, and made getKeysPresent find keys that were not present. It would be expensive to make getKeysPresent check that the actual key files are present (it just lists the directories). But that's not needed if we just clean up the stale cache and mapping files. To handle systems that were in direct mode and got switched back with stale direct mode files, made cleanObjectLoc remove all files in the key's directory. git annex unused will still list keys that are gone but for which the stale direct mode files exists. To deal with that, made dropunused remove the key's directory even if the key does not seem to be present.
* Deal with stale mappings for deleted file in direct mode.Gravatar Joey Hess2013-02-05
| | | | | | | | | | | | The most common way for a mapping to be stale is when a file was deleted, or renamed. Nothing updates the mappings for deletions yet. But they can also become stale in other ways. For example a file can be modified. So, the mapping is not trusted to be consistent. When we get a key, only replace symlinks that still point to that key with its content. When we drop a key, only put back symlinks for files that still have the direct mode content.
* add another setting to GitConfigGravatar Joey Hess2013-01-28
|
* ensure that content directory is thawed when writing direct mode mapping and ↵Gravatar Joey Hess2013-01-26
| | | | cache files
* allow lazy reading of map contentsGravatar Joey Hess2013-01-18
| | | | Don't explicitly close; hGetContents will close when read is done.
* some more direct mode fixesGravatar Joey Hess2013-01-18
| | | | | | | | Avoid a crash if a mapping contains files that no longer exist. This could happen because eg, one was deleted and a commit has not yet been done to update the mapping. Fix path calculation.
* Avoid filename encoding errors when writing direct mode mappings.Gravatar Joey Hess2013-01-18
|
* Fix direct mode mapping code to always store direct mode filenames relative ↵Gravatar Joey Hess2013-01-18
| | | | to the top of the repository, even when operating inside a subdirectory.
* In direct mode, files with the same key are no longer hardlinked, as that ↵Gravatar Joey Hess2013-01-14
| | | | would cause a surprising behavior if modifying one, where the other would also change.
* check for direct mode file change when copying to a local git remoteGravatar Joey Hess2013-01-10
|