summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorGravatar webanck <webanck@web>2017-11-07 16:03:17 +0000
committerGravatar admin <admin@branchable.com>2017-11-07 16:03:17 +0000
commit2d86f05a7a6aea745453997444d5bfbf26c165c3 (patch)
tree73f6d86db3837f255e05469144390e86f8925ce4
parent8d9f826eac3029874b584c7d73523371b1e5136f (diff)
-rw-r--r--doc/bugs/Massive_git_add_produces_sqlite_crashes.mdwn57
1 files changed, 57 insertions, 0 deletions
diff --git a/doc/bugs/Massive_git_add_produces_sqlite_crashes.mdwn b/doc/bugs/Massive_git_add_produces_sqlite_crashes.mdwn
new file mode 100644
index 000000000..dbf46a03b
--- /dev/null
+++ b/doc/bugs/Massive_git_add_produces_sqlite_crashes.mdwn
@@ -0,0 +1,57 @@
+### Please describe the problem.
+
+When adding plenty of files to my git annex repository, I encounter recurring sqlite errors.
+
+### What steps will reproduce the problem?
+Create a git annex repo, add thousands of annexed binary files, and add thousands of small files tracked only with git.
+
+
+### What version of git-annex are you using? On what operating system?
+
+ >git annex version
+ git-annex version: 6.20171003
+ build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Quvi
+ dependency versions: aws-0.13.0 bloomfilter-2.0.1.0 cryptonite-0.10 DAV-1.2 feed-0.3.10.4 ghc-7.10.3 http-client-0.4.26.2 persistent-sqlite-2.2 torrent-10000.0.0 uuid-1.3.11 yesod-1.4.2
+ key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+ remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+ local repository version: 6
+ supported repository versions: 3 5 6
+ upgrade supported from repository versions: 0 1 2 3 4 5
+ operating system: linux x86_64
+
+ >lsb_release -d
+ Description: Ubuntu 16.04.3 LTS
+
+ >uname -r
+ 4.4.0-43-Microsoft
+ #(I am using Bash on Windows.)
+
+### Please provide any additional information below.
+
+
+ # If you can, paste a complete transcript of the problem occurring here.
+ >git ls-files --others | grep txt | wc -l
+ 1953
+ >git add $(git ls-files --others | grep txt)
+ sqlite worker thread crashed: SQLite3 returned ErrorIO while attempting to perform prepare "SELECT null from content limit 1": disk I/O error
+ git-annex: sqlite query crashed
+ error: external filter git-annex smudge --clean %f failed 1
+ error: external filter git-annex smudge --clean %f failed
+ # [...] plenty of errors follow
+ # End of transcript or log.
+
+
+Triying to solve this problem, I found a part of answer in the form of a similar problem encounter here : <https://git-annex.branchable.com/forum/Problem_with_corrupt_SQLite_DB/>
+
+Deleting git annex databases and running git annex fsck didnt do the trick:
+
+ rm -rf .git/annex/keys/db .git/annex/keys/db-wal
+ git annex fsck --incremental -J4
+ git add $(git ls-files --others | grep txt)
+ # Again, plenty of sqlite errors :()
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+Oh yeah, I am still discovering this powerfull git annex tool.
+
+In fact, collegues and I are forming a group during the process to exchange about different use cases, encountered problems and help each other.
+