| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
| |
New table needed to look up what filenames are used in the currently
exported tree, for reasons explained in export.mdwn.
Also, added smart constructors for ExportLocation and ExportDirectory to
make sure they contain filepaths with the right direction slashes.
And some code refactoring.
This commit was sponsored by Francois Marier on Patreon.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
backtraces for unexpected errors.
ghc 8 added backtraces on uncaught errors. This is great, but git-annex was
using error in many places for a error message targeted at the user, in
some known problem case. A backtrace only confuses such a message, so omit it.
Notably, commands like git annex drop that failed due to eg, numcopies,
used to use error, so had a backtrace.
This commit was sponsored by Ethan Aubin.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
representable in the current locale.
This is a mostly backwards compatable change. I broke backwards
compatability in the case where a filename starts with double-quote.
That seems likely to be very rare, and v6 unlocked files are a new feature
anyway, and fsck needs to fix missing associated file mappings anyway. So,
I decided that is good enough.
The encoding used is to just show the String when it contains a problem
character. While that adds some overhead to addAssociatedFile and
removeAssociatedFile, those are not called very often. This approach has
minimal decode overhead, because most filenames won't be encoded that way,
and it only has to look for the leading double-quote to skip the expensive
read. So, getAssociatedFiles remains fast.
I did consider using ByteString instead, but getting a FilePath converted
with all chars intact, even surrigates, is difficult, and it looks like
instance PersistField ByteString uses Text, which I don't trust for problem
encoded data. It would probably be slower too, and it would make the
database less easy to inspect manually.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This breaks any existing keys database!
IKey serializes more efficiently than SKey, although this limits the
use of its Read/Show instances.
This makes the keys database use less disk space, and so should be a win.
Updated benchmark:
benchmarking keys database/getAssociatedFiles from 1000 (hit)
time 64.04 μs (63.95 μs .. 64.13 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 64.02 μs (63.96 μs .. 64.08 μs)
std dev 218.2 ns (172.5 ns .. 299.3 ns)
benchmarking keys database/getAssociatedFiles from 1000 (miss)
time 52.53 μs (52.18 μs .. 53.21 μs)
0.999 R² (0.998 R² .. 1.000 R²)
mean 52.31 μs (52.18 μs .. 52.91 μs)
std dev 734.6 ns (206.2 ns .. 1.623 μs)
benchmarking keys database/getAssociatedKey from 1000 (hit)
time 64.60 μs (64.46 μs .. 64.77 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 64.74 μs (64.57 μs .. 65.20 μs)
std dev 900.2 ns (389.7 ns .. 1.733 μs)
benchmarking keys database/getAssociatedKey from 1000 (miss)
time 52.46 μs (52.29 μs .. 52.68 μs)
1.000 R² (0.999 R² .. 1.000 R²)
mean 52.63 μs (52.35 μs .. 53.37 μs)
std dev 1.362 μs (562.7 ns .. 2.608 μs)
variance introduced by outliers: 24% (moderately inflated)
benchmarking keys database/addAssociatedFile to 1000 (old)
time 487.3 μs (484.7 μs .. 490.1 μs)
1.000 R² (0.999 R² .. 1.000 R²)
mean 490.9 μs (487.8 μs .. 496.5 μs)
std dev 13.95 μs (6.841 μs .. 22.03 μs)
variance introduced by outliers: 20% (moderately inflated)
benchmarking keys database/addAssociatedFile to 1000 (new)
time 6.633 ms (5.741 ms .. 7.751 ms)
0.905 R² (0.850 R² .. 0.965 R²)
mean 8.252 ms (7.803 ms .. 8.602 ms)
std dev 1.126 ms (900.3 μs .. 1.430 ms)
variance introduced by outliers: 72% (severely inflated)
benchmarking keys database/getAssociatedFiles from 10000 (hit)
time 65.36 μs (64.71 μs .. 66.37 μs)
0.998 R² (0.995 R² .. 1.000 R²)
mean 65.28 μs (64.72 μs .. 66.45 μs)
std dev 2.576 μs (920.8 ns .. 4.122 μs)
variance introduced by outliers: 42% (moderately inflated)
benchmarking keys database/getAssociatedFiles from 10000 (miss)
time 52.34 μs (52.25 μs .. 52.45 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 52.49 μs (52.42 μs .. 52.59 μs)
std dev 255.4 ns (205.8 ns .. 312.9 ns)
benchmarking keys database/getAssociatedKey from 10000 (hit)
time 64.76 μs (64.67 μs .. 64.84 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 64.67 μs (64.62 μs .. 64.72 μs)
std dev 177.3 ns (148.1 ns .. 217.1 ns)
benchmarking keys database/getAssociatedKey from 10000 (miss)
time 52.75 μs (52.66 μs .. 52.82 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 52.69 μs (52.63 μs .. 52.75 μs)
std dev 210.6 ns (173.7 ns .. 265.9 ns)
benchmarking keys database/addAssociatedFile to 10000 (old)
time 489.7 μs (488.7 μs .. 490.7 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 490.4 μs (489.6 μs .. 492.2 μs)
std dev 3.990 μs (2.435 μs .. 7.604 μs)
benchmarking keys database/addAssociatedFile to 10000 (new)
time 9.994 ms (9.186 ms .. 10.74 ms)
0.959 R² (0.928 R² .. 0.979 R²)
mean 9.906 ms (9.343 ms .. 10.40 ms)
std dev 1.384 ms (1.051 ms .. 2.100 ms)
variance introduced by outliers: 69% (severely inflated)
|
| |
|
|
|
|
|
|
|
| |
Renamed the db to keys, since it is various info about a Keys.
Dropping a key will update its pointer files, as long as their content can
be verified to be unmodified. This falls back to checksum verification, but
I want it to use an InodeCache of the key, for speed. But, I have not made
anything populate that cache yet.
|
|
|
Did not keep backwards compat for sticky bit records. An incremental fsck
that is already in progress will start over on upgrade to this version.
This is not yet ready for merging. The autobuilders need to have sqlite
installed.
Also, interrupting a fsck --incremental does not commit the database.
So, resuming with fsck --more restarts from beginning.
Memory: Constant during a fsck of tens of thousands of files.
(But, it does seem to buffer whole transation in memory, so
may really scale with number of files.)
CPU: ?
|
Joey Hess