summaryrefslogtreecommitdiff
path: root/Backend/WORM.hs
Commit message (Collapse)AuthorAge
* rename fsckKey to verifyKeyContentGravatar Joey Hess2015-10-01
| | | | No behavior changes.
* update my email address and homepage urlGravatar Joey Hess2015-01-21
|
* add getFileSize, which can get the real size of a large file on WindowsGravatar Joey Hess2015-01-20
| | | | | | | | | | | | | | Avoid using fileSize which maxes out at just 2 gb on Windows. Instead, use hFileSize, which doesn't have a bounded size. Fixes support for files > 2 gb on Windows. Note that the InodeCache code only needs to compare a file size, so it doesn't matter it the file size wraps. So it has been left as-is. This was necessary both to avoid invalidating existing inode caches, and because the code passed FileStatus around and would have become more expensive if it called getFileSize. This commit was sponsored by Christian Dietrich.
* Generate shorter keys for WORM and URL, avoiding keys that are longer than ↵Gravatar Joey Hess2015-01-06
| | | | used for SHA256, so as to not break on systems like Windows that have very small maximum path length limits.
* WORM backend: Switched to include the relative path to the file inside the ↵Gravatar Joey Hess2014-09-11
| | | | repository, rather than just the file's base name. Note that if you're relying on such things to keep files separate with WORM, you should really be using a better backend.
* WORM backend: When adding a file in a subdirectory, avoid including the ↵Gravatar Joey Hess2014-08-12
| | | | subdirectory in the key name.
* add key stability checking interfaceGravatar Joey Hess2014-07-27
| | | | | | | Needed for resuming from chunks. Url keys are considered not stable. I considered treating url keys with a known size as stable, but just don't feel that is enough information.
* add chunk metadata to KeyGravatar Joey Hess2014-07-24
| | | | | | | | Added new fields for chunk number, and chunk size. These will not appear in normal keys ever, but will be used for chunked data stored on special remotes. This commit was sponsored by Jouni K Seppanen.
* migrate: Avoid re-checksumming when migrating from hashE to hash backend.Gravatar Joey Hess2014-07-10
|
* Better sanitization of problem characters when generating URL and WORM keys.Gravatar Joey Hess2013-10-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | FAT has a lot of characters it does not allow in filenames, like ? and * It's probably the worst offender, but other filesystems also have limitiations. In 2011, I made keyFile escape : to handle FAT, but missed the other characters. It also turns out that when I did that, I was also living dangerously; any existing keys that contained a : had their object location change. Oops. So, adding new characters to escape to keyFile is out. Well, it would be possible to make keyFile behave differently on a per-filesystem basis, but this would be a real nightmare to get right. Consider that a rsync special remote uses keyFile to determine the filenames to use, and we don't know the underlying filesystem on the rsync server.. Instead, I have gone for a solution that is backwards compatable and simple. Its only downside is that already generated URL and WORM keys might not be able to be stored on FAT or some other filesystem that dislikes a character used in the key. (In this case, the user can just migrate the problem keys to a checksumming backend. If this became a big problem, fsck could be made to detect these and suggest a migration.) Going forward, new keys that are created will escape all characters that are likely to cause problems. And if some filesystem comes along that's even worse than FAT (seems unlikely, but here it is 2013, and people are still using FAT!), additional characters can be added to the set that are escaped without difficulty. (Also, made WORM limit the part of the filename that is embedded in the key, to deal with filesystem filename length limits. This could have already been a problem, but is more likely now, since the escaping of the filename can make it longer.) This commit was sponsored by Ian Downes
* fix permission damage (thanks, Windows)Gravatar Joey Hess2013-05-11
|
* clean up from windows portingGravatar Joey Hess2013-05-11
|
* git-annex now builds on Windows (doesn't work)Gravatar Joey Hess2013-05-11
|
* handle sha*sum's leading \ in checksum with certian unsual filenamesGravatar Joey Hess2012-12-20
| | | | | | | | * Bugfix: Remove leading \ from checksums output by sha*sum commands, when the filename contains \ or a newline. Closes: #696384 * fsck: Still accept checksums with a leading \ as valid, now that above bug is fixed. * migrate: Remove leading \ in checksums
* maintain set of files pendingAddGravatar Joey Hess2012-06-20
| | | | | | | | Kqueue needs to remember which files failed to be added due to being open, and retry them. This commit gets the data in place for such a retry thread. Broke KeySource out into its own file, and added Eq and Ord instances so it can be stored in a Set.
* separate source of content from the filename associated with the key when ↵Gravatar Joey Hess2012-06-05
| | | | | | generating a key This already made migrate's code a lot simpler.
* convert fsckKey to a MaybeGravatar Joey Hess2012-01-19
| | | | This way it's clear when a backend does not implement its own fsck checks.
* type alias cleanupGravatar Joey Hess2011-12-31
|
* renameGravatar Joey Hess2011-10-05
|
* renameGravatar Joey Hess2011-10-04
|
* factor out common importsGravatar Joey Hess2011-10-03
| | | | no code changes
* hlint tweaksGravatar Joey Hess2011-07-15
| | | | Did all sources except Remotes/* and Command/*
* remove unused backend machineryGravatar Joey Hess2011-07-05
| | | | | | | | | | | | | The only remaining vestiage of backends is different types of keys. These are still called "backends", mostly to avoid needing to change user interface and configuration. But everything to do with storing keys in different backends was gone; instead different types of remotes are used. In the refactoring, lots of code was moved out of odd corners like Backend.File, to closer to where it's used, like Command.Drop and Command.Fsck. Quite a lot of dead code was removed. Several data structures became simpler, which may result in better runtime efficiency. There should be no user-visible changes.
* rename modules for data types into Types/ directoryGravatar Joey Hess2011-06-01
|
* check key size when available, no matter the backendGravatar Joey Hess2011-03-23
| | | | | Now that SHA and other backends can have size info, fsck should check it whenever available.
* rename fileGravatar Joey Hess2011-03-15
|
* first pass at using new keysGravatar Joey Hess2011-03-15
| | | | | | | It compiles. It sorta works. Several subcommands are FIXME marked and broken, because things that used to accept separate --backend and --key params need to be changed to accept just a --key that encodes all the key info, now that there is metadata in keys.
* Rethink filename encoding handling for display. Since filename encoding may ↵0.23Gravatar Joey Hess2011-03-12
| | | | or may not match locale settings, any attempt to decode filenames will fail for some files. So instead, do all output in binary mode.
* only enable SHA backends that configure finds support forGravatar Joey Hess2011-03-02
|
* update unicode FilePath handlingGravatar Joey Hess2011-02-11
| | | | | | | | Based on http://hackage.haskell.org/trac/ghc/ticket/3307 , whether FilePath contains decoded unicode varies by OS. So, add a configure check for it. Also, renamed showFile to filePathToString
* Fix display of unicode filenames.Gravatar Joey Hess2011-02-10
| | | | | | | | | Internally, the filenames are stored as un-decoded unicode. I tried decoding them, but then haskell tries to access the wrong files. Hmm. So, I've unhappily chosen option "B", which is to decode filenames before they are displayed.
* better directory handlingGravatar Joey Hess2011-01-27
| | | | | | | | | | Rename Locations functions for better consitency, and make their values more consistent too. Used </> rather than manually building paths. There are still more places that manually do so, but are tricky, due to the behavior of </> when the second FilePath is absolute. So I only changed places where it obviously was relative.
* rename TypeInternals to BackendTypesGravatar Joey Hess2011-01-26
| | | | Now that it only contains types used by the backends
* successfully split Annex and AnnexState out of TypeInternalsGravatar Joey Hess2011-01-25
|
* parameterize Backend typeGravatar Joey Hess2011-01-25
| | | | | This allows the Backend type to not depend on the Annex type, and so the Annex type can later be moved out of TypeInternals.
* got rid of Core moduleGravatar Joey Hess2011-01-16
| | | | Most of it was to do with managing annexed Content, so put there
* finished hlintingGravatar Joey Hess2010-11-22
|
* fsck: Print warnings to stderr; --quiet can now be used to only see problems.Gravatar Joey Hess2010-11-15
|
* fsck bugfixesGravatar Joey Hess2010-11-13
|
* fsck improvementsGravatar Joey Hess2010-11-13
|
* fsck improvementsGravatar Joey Hess2010-11-13
| | | | | | | | | * fsck: Check if annex.numcopies is satisfied. * fsck: Verify the sha1 of files when the SHA1 backend is used. * fsck: Verify the size of files when the WORM backend is used. * fsck: Allow specifying individual files to fsk if fscking everything is not desired. * fsck: Fix bug, introduced in 0.04, in detection of unused data.
* more Wall cleaningGravatar Joey Hess2010-10-31
|
* copyright statements0.02Gravatar Joey Hess2010-10-27
|
* gratuitous renameGravatar Joey Hess2010-10-18
|
* remove checksum from WORMGravatar Joey Hess2010-10-17
| | | | with size and mtime in the key, it's redundant
* add visible size and time to worm keysGravatar Joey Hess2010-10-17
|
* tweaksGravatar Joey Hess2010-10-16
|
* rename backends moreGravatar Joey Hess2010-10-15