summaryrefslogtreecommitdiff
path: root/Backend
Commit message (Collapse)AuthorAge
* SHA*E backends: Exclude non-alphanumeric characters from extensions.Gravatar Joey Hess2012-12-20
| | | | | | * SHA*E backends: Exclude non-alphanumeric characters from extensions. * migrate: Remove leading \ in SHA* checksums, and non-alphanumerics from extensions of SHA*E keys.
* handle sha*sum's leading \ in checksum with certian unsual filenamesGravatar Joey Hess2012-12-20
| | | | | | | | * Bugfix: Remove leading \ from checksums output by sha*sum commands, when the filename contains \ or a newline. Closes: #696384 * fsck: Still accept checksums with a leading \ as valid, now that above bug is fixed. * migrate: Remove leading \ in checksums
* where indentingGravatar Joey Hess2012-11-11
|
* Avoid crashing on encoding errors in filenames when writing transfer info ↵Gravatar Joey Hess2012-09-16
| | | | files and reading from checksum commands.
* SHA256E is new default backendGravatar Joey Hess2012-09-12
| | | | | | | | The default backend used when adding files to the annex is changed from SHA256 to SHA256E, to simplify interoperability with OSX, media players, and various programs that needlessly look at symlink targets. To get old behavior, add a .gitattributes containing: * annex.backend=SHA256
* Bugfix: Fix fsck in SHA*E backends, when the key contains composite ↵Gravatar Joey Hess2012-08-24
| | | | extensions, as added in 3.20120721.
* better readProcessGravatar Joey Hess2012-07-19
|
* add back debug loggingGravatar Joey Hess2012-07-19
| | | | | | | | | | | | | Make Utility.Process wrap the parts of System.Process that I use, and add debug logging to them. Also wrote some higher-level code that allows running an action with handles to a processes stdin or stdout (or both), and checking its exit status, all in a single function call. As a bonus, the debug logging now indicates whether the process is being run to read from it, feed it data, chat with it (writing and reading), or just call it for its side effect.
* switch from System.Cmd.Utils to System.ProcessGravatar Joey Hess2012-07-18
| | | | | | | | | | | | | | | | | | Test suite now passes with -threaded! I traced back all the hangs with -threaded to System.Cmd.Utils. It seems it's just crappy/unsafe/outdated, and should not be used. System.Process seems to be the cool new thing, so converted all the code to use it instead. In the process, --debug stopped printing commands it runs. I may try to bring that back later. Note that even SafeSystem was switched to use System.Process. Since that was a modified version of code from System.Cmd.Utils, it needed to be converted too. I also got rid of nearly all calls to forkProcess, and all calls to executeFile, which I'm also doubtful about working well with -threaded.
* fix leading period before two-element extensionsGravatar Joey Hess2012-07-06
|
* SHAnE backends are now smarter about composite extensions, such as .tar.gz ↵Gravatar Joey Hess2012-07-05
| | | | Closes: #680450
* Use SHA library for files less than 50 kb in size, at which point it's ↵Gravatar Joey Hess2012-07-04
| | | | faster than forking the more optimised external program.
* When shaNsum commands cannot be found, use the Haskell SHA library (already ↵Gravatar Joey Hess2012-07-04
| | | | | | | | a dependency) to do the checksumming. This may be slower, but avoids portability problems. Using Crypto's version of the hashes would be another option. I need to benchmark it. The SHA2 library (which provides SHA1 also, confusing name) may be the fastest option, but is not currently in Debian.
* maintain set of files pendingAddGravatar Joey Hess2012-06-20
| | | | | | | | Kqueue needs to remember which files failed to be added due to being open, and retry them. This commit gets the data in place for such a retry thread. Broke KeySource out into its own file, and added Eq and Ord instances so it can be stored in a Set.
* separate source of content from the filename associated with the key when ↵Gravatar Joey Hess2012-06-05
| | | | | | generating a key This already made migrate's code a lot simpler.
* Require that the SHA256 backend can be used when building, since it's the ↵Gravatar Joey Hess2012-05-31
| | | | default.
* handle really long urlsGravatar Joey Hess2012-02-16
| | | | | Using the whole url as a key can make the filename too long. Truncate and use a md5sum for uniqueness if necessary.
* addurl --fast: Verifies that the url can be downloaded (only getting its ↵Gravatar Joey Hess2012-02-10
| | | | head), and records the size in the key.
* fsck --fromGravatar Joey Hess2012-01-19
| | | | | | | | | | | | | | | | Fscking a remote is now supported. It's done by retrieving the contents of the specified files from the remote, and checking them, so can be an expensive operation. (Several optimisations are possible, to speed it up, of course.. This is the slow and stupid remote fsck to start with.) Still, if the remote is a special remote, or a git repository that you cannot run fsck in locally, it's nice to have the ability to fsck it. If you have any directory special remotes, now would be a good time to fsck them, in case you were hit by the data loss bug fixed in the previous release!
* convert fsckKey to a MaybeGravatar Joey Hess2012-01-19
| | | | This way it's clear when a backend does not implement its own fsck checks.
* type alias cleanupGravatar Joey Hess2011-12-31
|
* more partial function removalGravatar Joey Hess2011-12-15
| | | | | Left a few Prelude.head's in where it was checked not null and too hard to remove, etc.
* Prevent key names from containing newlines.Gravatar Joey Hess2011-12-06
| | | | | | | | | There are several places where it's assumed a key can be written on one line. One is in the format of the .git/annex/unused files. The difficult one is that filenames derived from keys are fed into git cat-file --batch, which has a line based input. (And no -z option.) So, for now it's best to block such keys being created.
* add support for using hashDirLower in addition to hashDirMixedGravatar Joey Hess2011-11-28
| | | | | | | | | | Supporting multiple directory hash types will allow converting to a different one, without a flag day. gitAnnexLocation now checks which of the possible locations have a file. This means more statting of files. Several places currently use gitAnnexLocation and immediately check if the returned file exists; those need to be optimised.
* reorder repo parameters lastGravatar Joey Hess2011-11-08
| | | | | | | | | | | | | Many functions took the repo as their first parameter. Changing it consistently to be the last parameter allows doing some useful things with currying, that reduce boilerplate. In particular, g <- gitRepo is almost never needed now, instead use inRepo to run an IO action in the repo, and fromRepo to get a value from the repo. This also provides more opportunities to use monadic and applicative combinators.
* use SHA256 by defaultGravatar Joey Hess2011-11-04
| | | | | | | | | | | | | | | | | | | | | | | | To get old behavior, add a .gitattributes containing: * annex.backend=WORM I feel that SHA256 is a better default for most people, as long as their systems are fast enough that checksumming their files isn't a problem. git-annex should default to preserving the integrity of data as well as git does. Checksum backends also work better with editing files via unlock/lock. I considered just using SHA1, but since that hash is believed to be somewhat near to being broken, and git-annex deals with large files which would be a perfect exploit medium, I decided to go to a SHA-2 hash. SHA512 is annoyingly long when displayed, and git-annex displays it in a few places (and notably it is shown in ls -l), so I picked the shorter hash. Considered SHA224 as it's even shorter, but feel it's a bit weird. I expect git-annex will use SHA-3 at some point in the future, but probably not soon! Note that systems without a sha256sum (or sha256) program will fall back to defaulting to SHA1.
* Record uuid when auto-initializing a remote so it shows in status.Gravatar Joey Hess2011-11-02
|
* playing with >=>Gravatar Joey Hess2011-10-31
| | | | | Apparently in haskell if you teach a man to fish, he'll write more pointfree code.
* minor syntax changesGravatar Joey Hess2011-10-11
|
* renameGravatar Joey Hess2011-10-05
|
* renameGravatar Joey Hess2011-10-04
|
* factor out common importsGravatar Joey Hess2011-10-03
| | | | no code changes
* go go gadget hlintGravatar Joey Hess2011-09-20
|
* split groups of related functions out of UtilityGravatar Joey Hess2011-08-22
|
* moved files aroundGravatar Joey Hess2011-08-20
|
* addurl: --fast can be used to avoid immediately downloading the url.Gravatar Joey Hess2011-08-06
| | | | | | The tricky part about this is that to generate a key, the file must be present already. Worked around by adding (back) an URL key type, which is used for addurl --fast.
* fooGravatar Joey Hess2011-08-06
|
* unify elipsis handlingGravatar Joey Hess2011-07-19
| | | | | And add a simple dots-based progress display, currently only used in v2 upgrade.
* hlint tweaksGravatar Joey Hess2011-07-15
| | | | Did all sources except Remotes/* and Command/*
* remove unused backend machineryGravatar Joey Hess2011-07-05
| | | | | | | | | | | | | The only remaining vestiage of backends is different types of keys. These are still called "backends", mostly to avoid needing to change user interface and configuration. But everything to do with storing keys in different backends was gone; instead different types of remotes are used. In the refactoring, lots of code was moved out of odd corners like Backend.File, to closer to where it's used, like Command.Drop and Command.Fsck. Quite a lot of dead code was removed. Several data structures became simpler, which may result in better runtime efficiency. There should be no user-visible changes.
* wordingGravatar Joey Hess2011-07-01
|
* remove URL backendGravatar Joey Hess2011-07-01
|
* add web special remoteGravatar Joey Hess2011-07-01
| | | | | Generalized LocationLog to PresenceLog, and use a presence log to record urls for the web special remote.
* renamed GitRepo to GitGravatar Joey Hess2011-06-30
| | | | It was always imported qualified as Git anyway
* tweak fsck wording so file is at the end of the lineGravatar Joey Hess2011-06-23
|
* avoid unnecessary read of trust.logGravatar Joey Hess2011-06-23
|
* fixed logFileGravatar Joey Hess2011-06-22
|
* move LocationLog into Annex monad from IOGravatar Joey Hess2011-06-22
| | | | It will need to run in Annex so it can use Branch
* Bugfix: Fix fsck to not think all SHAnE keys are bad.Gravatar Joey Hess2011-06-10
|
* get --from is the same as copy --fromGravatar Joey Hess2011-06-09
| | | | | | | | get not honoring --from has surprised me a few times, so least surprise suggests it should just behave like copy --from. This leaves the difference between get and copy being that copy always requires the remote to copy from, while get will decide whether to get a file from a key/value store or a remote.