aboutsummaryrefslogtreecommitdiff
path: root/doc/bugs/enormous_fsck_output_OOM
diff options
context:
space:
mode:
authorGravatar Joey Hess <joeyh@joeyh.name>2016-04-19 13:46:11 -0400
committerGravatar Joey Hess <joeyh@joeyh.name>2016-04-19 13:55:29 -0400
commitdbfc00e2a6ad99200da35f75f889174cd7bfd195 (patch)
tree9d8a9d1353ab10376183c1bda881fface04b6fcb /doc/bugs/enormous_fsck_output_OOM
parent41b7950285ef1e91b80c441c2be68a1ef4d0d27c (diff)
remove old closed bugs and todo items to speed up wiki updates and reduce size
Remove closed bugs and todos that were last edited or commented before Q3 2015. Command line used: for f in $(grep -l '\[\[done\]\]' -- *.mdwn); do d="$(echo "$f" | sed 's/.mdwn$//')"; if [ -z "$(git log --since=09-09-2015 --pretty=oneline -- "$f")" -a -z "$(git log --since=09-09-2015 --pretty=oneline -- "$d")" ]; then git rm -- "$f"; git rm -rf "$d"; fi; done for f in $(grep -l '|done\]\]' -- *.mdwn); do d="$(echo "$f" | sed 's/.mdwn$//')"; if [ -z "$(git log --since=09-09-2015 --pretty=oneline -- "$f")" -a -z "$(git log --since=09-09-2015 --pretty=oneline -- "$d")" ]; then git rm -- "$f"; git rm -rf "$d"; fi; done
Diffstat (limited to 'doc/bugs/enormous_fsck_output_OOM')
-rw-r--r--doc/bugs/enormous_fsck_output_OOM/comment_1_490b8bfe95b01a23408ecb5d63dcd40b._comment10
-rw-r--r--doc/bugs/enormous_fsck_output_OOM/comment_2_2666c135dd3378cf6301aa4957049fbd._comment10
-rw-r--r--doc/bugs/enormous_fsck_output_OOM/comment_3_dfb169c441215b671f8c971184de3e16._comment10
-rw-r--r--doc/bugs/enormous_fsck_output_OOM/comment_4_19ef90803aa7ce158bce02378e18ea0f._comment30
-rw-r--r--doc/bugs/enormous_fsck_output_OOM/comment_5_2b5406768fff2834f7aefa76ef949de2._comment12
-rw-r--r--doc/bugs/enormous_fsck_output_OOM/comment_6_0997f1a94c2fda9fe69824e074011518._comment18
-rw-r--r--doc/bugs/enormous_fsck_output_OOM/comment_7_2cdc79f1e0f72693814e91dc88a758e1._comment10
-rw-r--r--doc/bugs/enormous_fsck_output_OOM/comment_8_b9aab0aba4dab30260371b4762e0e51d._comment8
-rw-r--r--doc/bugs/enormous_fsck_output_OOM/comment_9_8de694dff75e27856c8282d1f2d120b6._comment14
9 files changed, 0 insertions, 122 deletions
diff --git a/doc/bugs/enormous_fsck_output_OOM/comment_1_490b8bfe95b01a23408ecb5d63dcd40b._comment b/doc/bugs/enormous_fsck_output_OOM/comment_1_490b8bfe95b01a23408ecb5d63dcd40b._comment
deleted file mode 100644
index 73d218bac..000000000
--- a/doc/bugs/enormous_fsck_output_OOM/comment_1_490b8bfe95b01a23408ecb5d63dcd40b._comment
+++ /dev/null
@@ -1,10 +0,0 @@
-[[!comment format=mdwn
- username="http://joeyh.name/"
- ip="209.250.56.172"
- subject="comment 1"
- date="2014-02-23T18:51:45Z"
- content="""
-I suspect that git fsck is outputting so many lines about problems that it's taking more memory than it's limited to using to hold them all.
-
-Can you paste the output of: git fsck --no-dangling --no-reflogs
-"""]]
diff --git a/doc/bugs/enormous_fsck_output_OOM/comment_2_2666c135dd3378cf6301aa4957049fbd._comment b/doc/bugs/enormous_fsck_output_OOM/comment_2_2666c135dd3378cf6301aa4957049fbd._comment
deleted file mode 100644
index f2028c91f..000000000
--- a/doc/bugs/enormous_fsck_output_OOM/comment_2_2666c135dd3378cf6301aa4957049fbd._comment
+++ /dev/null
@@ -1,10 +0,0 @@
-[[!comment format=mdwn
- username="http://joeyh.name/"
- ip="209.250.56.172"
- subject="comment 2"
- date="2014-02-23T19:09:30Z"
- content="""
-Erm, that output is liable to be big, I only care how many lines and characters of output there are!
-
- git fsck --no-dangling --no-reflogs |wc
-"""]]
diff --git a/doc/bugs/enormous_fsck_output_OOM/comment_3_dfb169c441215b671f8c971184de3e16._comment b/doc/bugs/enormous_fsck_output_OOM/comment_3_dfb169c441215b671f8c971184de3e16._comment
deleted file mode 100644
index 96b1ac9cd..000000000
--- a/doc/bugs/enormous_fsck_output_OOM/comment_3_dfb169c441215b671f8c971184de3e16._comment
+++ /dev/null
@@ -1,10 +0,0 @@
-[[!comment format=mdwn
- username="http://joeyh.name/"
- ip="209.250.56.172"
- subject="comment 3"
- date="2014-02-23T19:12:10Z"
- content="""
-Also, you can build git-annex from source with the RTS options enabled by running `cabal install git-annex --ghc-options=-rtsopts`
-
-(or just build git-repair which has the repository repair parts of git-annex)
-"""]]
diff --git a/doc/bugs/enormous_fsck_output_OOM/comment_4_19ef90803aa7ce158bce02378e18ea0f._comment b/doc/bugs/enormous_fsck_output_OOM/comment_4_19ef90803aa7ce158bce02378e18ea0f._comment
deleted file mode 100644
index 69774073d..000000000
--- a/doc/bugs/enormous_fsck_output_OOM/comment_4_19ef90803aa7ce158bce02378e18ea0f._comment
+++ /dev/null
@@ -1,30 +0,0 @@
-[[!comment format=mdwn
- username="https://www.google.com/accounts/o8/id?id=AItOawnFjuvfPpi1kf6l54bxfFUm0Aw_Gf_IO0o"
- nickname="Aaron"
- subject="Didn't work"
- date="2014-03-01T22:40:36Z"
- content="""
-Many thanks for your input, Joey.
-
-That didn't seem to work.
-
-$ git fsck --no-dangling --no-reflogs |wc
-error: unknown option `no-dangling'
-usage: git fsck [options] [<object>...]
-
- -v, --verbose be verbose
- --unreachable show unreachable objects
- --tags report tags
- --root report root nodes
- --cache make index objects head nodes
- --reflogs make reflogs head nodes (default)
- --full also consider packs and alternate objects
- --strict enable more strict checking
- --lost-found write dangling objects in .git/lost-found
- --progress show progress
-
- 0 0 0
-
-$ git --version
-git version 1.7.9.5
-"""]]
diff --git a/doc/bugs/enormous_fsck_output_OOM/comment_5_2b5406768fff2834f7aefa76ef949de2._comment b/doc/bugs/enormous_fsck_output_OOM/comment_5_2b5406768fff2834f7aefa76ef949de2._comment
deleted file mode 100644
index 923e1ccba..000000000
--- a/doc/bugs/enormous_fsck_output_OOM/comment_5_2b5406768fff2834f7aefa76ef949de2._comment
+++ /dev/null
@@ -1,12 +0,0 @@
-[[!comment format=mdwn
- username="http://joeyh.name/"
- ip="209.250.56.146"
- subject="comment 5"
- date="2014-03-05T16:53:33Z"
- content="""
-You have an older version of git, which does not support --no-dangling.
-
-git-annex will detect that, and omit the option. This does make it more likely that git fsck is outputing a ton of information, so I continue to think that's the most likely cause of the memory use.
-
-Please verify with: git fsck --no-reflogs |wc
-"""]]
diff --git a/doc/bugs/enormous_fsck_output_OOM/comment_6_0997f1a94c2fda9fe69824e074011518._comment b/doc/bugs/enormous_fsck_output_OOM/comment_6_0997f1a94c2fda9fe69824e074011518._comment
deleted file mode 100644
index ddf69f94b..000000000
--- a/doc/bugs/enormous_fsck_output_OOM/comment_6_0997f1a94c2fda9fe69824e074011518._comment
+++ /dev/null
@@ -1,18 +0,0 @@
-[[!comment format=mdwn
- username="https://www.google.com/accounts/o8/id?id=AItOawnFjuvfPpi1kf6l54bxfFUm0Aw_Gf_IO0o"
- nickname="Aaron"
- subject="comment 6"
- date="2014-03-08T00:08:22Z"
- content="""
-That seemed to work, thanks:
-
- $ git fsck --no-reflogs |wc
- Checking object directories: 100% (256/256), done.
- error: refs/gcrypt/gitception+ does not point to a valid object!
- error: refs/remotes/Beta/git-annex does not point to a valid object!
- error: refs/remotes/Beta/master does not point to a valid object!
- Checking connectivity: 128728, done.
- 369082 1165340 20898546
-
-Thanks for your help!
-"""]]
diff --git a/doc/bugs/enormous_fsck_output_OOM/comment_7_2cdc79f1e0f72693814e91dc88a758e1._comment b/doc/bugs/enormous_fsck_output_OOM/comment_7_2cdc79f1e0f72693814e91dc88a758e1._comment
deleted file mode 100644
index 14bdc17b8..000000000
--- a/doc/bugs/enormous_fsck_output_OOM/comment_7_2cdc79f1e0f72693814e91dc88a758e1._comment
+++ /dev/null
@@ -1,10 +0,0 @@
-[[!comment format=mdwn
- username="http://joeyh.name/"
- ip="108.236.230.124"
- subject="comment 7"
- date="2014-03-10T18:02:46Z"
- content="""
-Ok, that verifies my hypothesis that fsck is outputting a lot of lines. 369082 lines to be precise, comprising 20 mb of data in all. So it's not too surprising this ends up blowing up into a bad amount of memory use.
-
-I'm going to move this from a forum post over to a bug: <http://git-annex.branchable.com/bugs/enormous_fsck_output_OOM>
-"""]]
diff --git a/doc/bugs/enormous_fsck_output_OOM/comment_8_b9aab0aba4dab30260371b4762e0e51d._comment b/doc/bugs/enormous_fsck_output_OOM/comment_8_b9aab0aba4dab30260371b4762e0e51d._comment
deleted file mode 100644
index fa437311f..000000000
--- a/doc/bugs/enormous_fsck_output_OOM/comment_8_b9aab0aba4dab30260371b4762e0e51d._comment
+++ /dev/null
@@ -1,8 +0,0 @@
-[[!comment format=mdwn
- username="http://joeyh.name/"
- ip="108.236.230.124"
- subject="comment 8"
- date="2014-03-10T18:12:29Z"
- content="""
-In a quick test with a 32 mb dummy fsck output, git-repair ballooned up to 1.7 gb. Clearly something not happy there, although in my case this did not cause it to crash.
-"""]]
diff --git a/doc/bugs/enormous_fsck_output_OOM/comment_9_8de694dff75e27856c8282d1f2d120b6._comment b/doc/bugs/enormous_fsck_output_OOM/comment_9_8de694dff75e27856c8282d1f2d120b6._comment
deleted file mode 100644
index a26e58c58..000000000
--- a/doc/bugs/enormous_fsck_output_OOM/comment_9_8de694dff75e27856c8282d1f2d120b6._comment
+++ /dev/null
@@ -1,14 +0,0 @@
-[[!comment format=mdwn
- username="http://joeyh.name/"
- ip="108.236.230.124"
- subject="comment 9"
- date="2014-03-10T19:17:34Z"
- content="""
-I've gotten this down to 900 mb used, in the case where every single line lists a different sha. Possibly more important, if lines repeat shas, or are extraneous, memory usage will be significantly lower. This might be enough to get it working in Aaron's repository, especially if the bulk of the git fsck output was about dangling objects, which are now ignored without buffering them all in memory.
-
-The memory usage is just about as low as is possible; it takes a fair amount of memory just to hold 300 thousand shas in memory. And the git repair process needs to keep track of every broken sha. (Maybe there's a way to stream them, but I don't immediately see one.)
-
-I hesitate to say this means the problem is truly fixed. I have some much larger repositories with eg, `git count-objects -v` reporting 2 million objects. If they all went corrupt, it would still use too much memory.
-
-One improvement would be to store Shas in packed memory, rather than as strings like they are now. That would probably half the memory used. It still does not seem like a full solution.
-"""]]