| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This breaks any existing keys database!
IKey serializes more efficiently than SKey, although this limits the
use of its Read/Show instances.
This makes the keys database use less disk space, and so should be a win.
Updated benchmark:
benchmarking keys database/getAssociatedFiles from 1000 (hit)
time 64.04 μs (63.95 μs .. 64.13 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 64.02 μs (63.96 μs .. 64.08 μs)
std dev 218.2 ns (172.5 ns .. 299.3 ns)
benchmarking keys database/getAssociatedFiles from 1000 (miss)
time 52.53 μs (52.18 μs .. 53.21 μs)
0.999 R² (0.998 R² .. 1.000 R²)
mean 52.31 μs (52.18 μs .. 52.91 μs)
std dev 734.6 ns (206.2 ns .. 1.623 μs)
benchmarking keys database/getAssociatedKey from 1000 (hit)
time 64.60 μs (64.46 μs .. 64.77 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 64.74 μs (64.57 μs .. 65.20 μs)
std dev 900.2 ns (389.7 ns .. 1.733 μs)
benchmarking keys database/getAssociatedKey from 1000 (miss)
time 52.46 μs (52.29 μs .. 52.68 μs)
1.000 R² (0.999 R² .. 1.000 R²)
mean 52.63 μs (52.35 μs .. 53.37 μs)
std dev 1.362 μs (562.7 ns .. 2.608 μs)
variance introduced by outliers: 24% (moderately inflated)
benchmarking keys database/addAssociatedFile to 1000 (old)
time 487.3 μs (484.7 μs .. 490.1 μs)
1.000 R² (0.999 R² .. 1.000 R²)
mean 490.9 μs (487.8 μs .. 496.5 μs)
std dev 13.95 μs (6.841 μs .. 22.03 μs)
variance introduced by outliers: 20% (moderately inflated)
benchmarking keys database/addAssociatedFile to 1000 (new)
time 6.633 ms (5.741 ms .. 7.751 ms)
0.905 R² (0.850 R² .. 0.965 R²)
mean 8.252 ms (7.803 ms .. 8.602 ms)
std dev 1.126 ms (900.3 μs .. 1.430 ms)
variance introduced by outliers: 72% (severely inflated)
benchmarking keys database/getAssociatedFiles from 10000 (hit)
time 65.36 μs (64.71 μs .. 66.37 μs)
0.998 R² (0.995 R² .. 1.000 R²)
mean 65.28 μs (64.72 μs .. 66.45 μs)
std dev 2.576 μs (920.8 ns .. 4.122 μs)
variance introduced by outliers: 42% (moderately inflated)
benchmarking keys database/getAssociatedFiles from 10000 (miss)
time 52.34 μs (52.25 μs .. 52.45 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 52.49 μs (52.42 μs .. 52.59 μs)
std dev 255.4 ns (205.8 ns .. 312.9 ns)
benchmarking keys database/getAssociatedKey from 10000 (hit)
time 64.76 μs (64.67 μs .. 64.84 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 64.67 μs (64.62 μs .. 64.72 μs)
std dev 177.3 ns (148.1 ns .. 217.1 ns)
benchmarking keys database/getAssociatedKey from 10000 (miss)
time 52.75 μs (52.66 μs .. 52.82 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 52.69 μs (52.63 μs .. 52.75 μs)
std dev 210.6 ns (173.7 ns .. 265.9 ns)
benchmarking keys database/addAssociatedFile to 10000 (old)
time 489.7 μs (488.7 μs .. 490.7 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 490.4 μs (489.6 μs .. 492.2 μs)
std dev 3.990 μs (2.435 μs .. 7.604 μs)
benchmarking keys database/addAssociatedFile to 10000 (new)
time 9.994 ms (9.186 ms .. 10.74 ms)
0.959 R² (0.928 R² .. 0.979 R²)
mean 9.906 ms (9.343 ms .. 10.40 ms)
std dev 1.384 ms (1.051 ms .. 2.100 ms)
variance introduced by outliers: 69% (severely inflated)
|
| | |
|
| |\ |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
benchmarking keys database/addAssociatedFile to 1000 (old)
time 516.1 μs (514.7 μs .. 517.4 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 514.0 μs (512.1 μs .. 515.2 μs)
std dev 4.740 μs (2.972 μs .. 7.068 μs)
benchmarking keys database/addAssociatedFile to 1000 (new)
time 5.750 ms (4.857 ms .. 6.885 ms)
0.815 R² (0.698 R² .. 0.904 R²)
mean 7.858 ms (7.311 ms .. 8.421 ms)
std dev 1.684 ms (1.383 ms .. 2.027 ms)
variance introduced by outliers: 88% (severely inflated)
benchmarking keys database/addAssociatedFile to 10000 (old)
time 515.7 μs (514.8 μs .. 516.5 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 515.4 μs (513.7 μs .. 516.6 μs)
std dev 4.824 μs (2.957 μs .. 7.167 μs)
benchmarking keys database/addAssociatedFile to 10000 (new)
time 8.934 ms (7.779 ms .. 10.05 ms)
0.868 R² (0.751 R² .. 0.934 R²)
mean 11.51 ms (10.66 ms .. 12.26 ms)
std dev 2.174 ms (1.816 ms .. 2.747 ms)
variance introduced by outliers: 82% (severely inflated)
|
| | |
| |
| |
| | |
my usecase
|
| |/ |
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This is a schema change so will break any existing keys databases. But,
it's not been released yet, so I'm still able to make such changes.
This speeds up the benchmark quite nicely:
benchmarking keys database/getAssociatedKey from 1000 (hit)
time 91.65 μs (91.48 μs .. 91.81 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 91.78 μs (91.66 μs .. 91.94 μs)
std dev 468.3 ns (353.1 ns .. 624.3 ns)
benchmarking keys database/getAssociatedKey from 1000 (miss)
time 53.33 μs (53.23 μs .. 53.40 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 53.43 μs (53.36 μs .. 53.53 μs)
std dev 274.2 ns (211.7 ns .. 361.5 ns)
benchmarking keys database/getAssociatedKey from 10000 (hit)
time 92.99 μs (92.74 μs .. 93.27 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 92.90 μs (92.76 μs .. 93.16 μs)
std dev 608.7 ns (404.1 ns .. 963.5 ns)
benchmarking keys database/getAssociatedKey from 10000 (miss)
time 53.12 μs (52.91 μs .. 53.39 μs)
1.000 R² (0.999 R² .. 1.000 R²)
mean 52.84 μs (52.68 μs .. 53.16 μs)
std dev 715.4 ns (400.4 ns .. 1.370 μs)
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The benchmark shows that the database access is quite fast indeed!
And, it scales linearly to the number of keys, with one exception,
getAssociatedKey.
Based on this benchmark, I don't think I need worry about optimising
for cases where all files are locked and the database is mostly empty.
In those cases, database access will be misses, and according to this
benchmark, should add only 50 milliseconds to runtime.
(NB: There may be some overhead to getting the database opened and locking
the handle that this benchmark doesn't see.)
joey@darkstar:~/src/git-annex>./git-annex benchmark
setting up database with 1000
setting up database with 10000
benchmarking keys database/getAssociatedFiles from 1000 (hit)
time 62.77 μs (62.70 μs .. 62.85 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 62.81 μs (62.76 μs .. 62.88 μs)
std dev 201.6 ns (157.5 ns .. 259.5 ns)
benchmarking keys database/getAssociatedFiles from 1000 (miss)
time 50.02 μs (49.97 μs .. 50.07 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 50.09 μs (50.04 μs .. 50.17 μs)
std dev 206.7 ns (133.8 ns .. 295.3 ns)
benchmarking keys database/getAssociatedKey from 1000 (hit)
time 211.2 μs (210.5 μs .. 212.3 μs)
1.000 R² (0.999 R² .. 1.000 R²)
mean 211.0 μs (210.7 μs .. 212.0 μs)
std dev 1.685 μs (334.4 ns .. 3.517 μs)
benchmarking keys database/getAssociatedKey from 1000 (miss)
time 173.5 μs (172.7 μs .. 174.2 μs)
1.000 R² (0.999 R² .. 1.000 R²)
mean 173.7 μs (173.0 μs .. 175.5 μs)
std dev 3.833 μs (1.858 μs .. 6.617 μs)
variance introduced by outliers: 16% (moderately inflated)
benchmarking keys database/getAssociatedFiles from 10000 (hit)
time 64.01 μs (63.84 μs .. 64.18 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 64.85 μs (64.34 μs .. 66.02 μs)
std dev 2.433 μs (547.6 ns .. 4.652 μs)
variance introduced by outliers: 40% (moderately inflated)
benchmarking keys database/getAssociatedFiles from 10000 (miss)
time 50.33 μs (50.28 μs .. 50.39 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 50.32 μs (50.26 μs .. 50.38 μs)
std dev 202.7 ns (167.6 ns .. 252.0 ns)
benchmarking keys database/getAssociatedKey from 10000 (hit)
time 1.142 ms (1.139 ms .. 1.146 ms)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.142 ms (1.140 ms .. 1.144 ms)
std dev 7.142 μs (4.994 μs .. 10.98 μs)
benchmarking keys database/getAssociatedKey from 10000 (miss)
time 1.094 ms (1.092 ms .. 1.096 ms)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.095 ms (1.095 ms .. 1.097 ms)
std dev 4.277 μs (2.591 μs .. 7.228 μs)
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| |/ |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
User-Agent: git-annex
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |\ |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
The crash turned out to be caused by the sqlite database being deleted out
from under sqlite before it was done with it. Since multiple git_annex
calls are done in the same process while running the test suite, the
DbHandle could linger until GCed, and the test repo, and thus sqlite
database be deleted before the workerThread was done.
|
| | | |
|
| |/ |
|
| |
|
|
|
|
|
|
|
| |
The homomorphs are back, just encoded such that it doesn't crash in LANG=C
However, I noticed a bug in the old escaping; [pseudoSlash] was escaped the
same as ['/','/']. Fixed by using '%' to escape pseudoSlash. Which requires
doubling '%' to escape it, but that's already done in the escaping of
worktree filenames in a view, so is probably ok.
|
| |
|
|
| |
ugly escaping, as the unicode method doesn't work on non-unicode supporting systems.
|
| | |
|
| | |
|
Joey Hess
https://me.yahoo.com/a/EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s-#7c0f4
jhannwong@c9c7a67b5632a4bbc0c959cfeb3d340e02f28565
sameerds
emanuele.ruffaldi@56b9c9a5c1f873994b869248e9b53aa20f437d20
pkitslaar
git-annex.branchable.com@69b3f2da2837a3d9de8eab0b93771008294b7d69
wsha.code+ga@b38779424f41c5701bbe5937340be43ff1474b2d
G.nius.ck@d885edcfde63422ee84e5ee501b7aa545e91298d
https://openid.stackexchange.com/user/e65e6d0e-58ba-41de-84cc-1f2ba54cf574
mark@6b90344cdab3158eacb94a3944460d138afc9bef
oliverconzen@8cff8c5ab3868de177f748566f90c79720f9cf4b
Alan