Merge branch 'master' into tasty-tests

Conflicts:
	Test.hs

6 files changed, 124 insertions, 0 deletions

diff --git a/doc/bugs/Resource_exhausted/comment_10_bccf9528ffe963154c92ce49762e7ea6._comment b/doc/bugs/Resource_exhausted/comment_10_bccf9528ffe963154c92ce49762e7ea6._comment
new file mode 100644
index 000000000..da2a15ba2
--- /dev/null
+++ b/doc/bugs/Resource_exhausted/comment_10_bccf9528ffe963154c92ce49762e7ea6._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawln3ckqKx0x_xDZMYwa9Q1bn4I06oWjkog"
+ nickname="Michael"
+ subject="comment 10"
+ date="2013-08-26T18:33:40Z"
+ content="""
+@Joey: it was a \"pretty large\" transfer, several hundred gigabytes in perhaps ~100000 files.  The copying was going to a GPG-encrypted directory remote.
+The error only happened once or twice so far.  Point taken about find in /proc; I'll do that if it happens next time.
+
+"""]]
diff --git a/doc/bugs/Resource_exhausted/comment_5_f55d933bce77fd2185ebd0cc46fe57ec._comment b/doc/bugs/Resource_exhausted/comment_5_f55d933bce77fd2185ebd0cc46fe57ec._comment
new file mode 100644
index 000000000..19bcdb376
--- /dev/null
+++ b/doc/bugs/Resource_exhausted/comment_5_f55d933bce77fd2185ebd0cc46fe57ec._comment
@@ -0,0 +1,64 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawln3ckqKx0x_xDZMYwa9Q1bn4I06oWjkog"
+ nickname="Michael"
+ subject="I just reproduced this with the latest version"
+ date="2013-08-11T03:17:11Z"
+ content="""
+At some point during a large copy, there's an ever increasing number of pipes in /proc/git-annex-pid/fd
+As soon as it hits the limit (1023 in my case), copies start failing
+
+    etc
+    r-x------ 1 michael michael 64 Aug 10 20:14 851 -> pipe:[2250609]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 852 -> pipe:[2251549]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 853 -> pipe:[2251550]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 854 -> pipe:[2250612]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 855 -> pipe:[2250613]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 856 -> pipe:[2246639]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 857 -> pipe:[2246640]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 858 -> pipe:[2246642]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 859 -> pipe:[2246643]
+    l-wx------ 1 michael michael 64 Aug 10 20:13 86 -> pipe:[2241378]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 860 -> pipe:[2246645]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 861 -> pipe:[2246646]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 862 -> pipe:[2246648]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 863 -> pipe:[2246649]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 864 -> pipe:[2246653]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 865 -> pipe:[2246654]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 866 -> pipe:[2249407]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 867 -> pipe:[2251789]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 868 -> pipe:[2250627]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 869 -> pipe:[2250628]
+    lr-x------ 1 michael michael 64 Aug 10 20:13 87 -> pipe:[2241379]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 870 -> pipe:[2251778]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 871 -> pipe:[2251779]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 872 -> pipe:[2251781]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 873 -> pipe:[2251782]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 874 -> pipe:[2250635]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 875 -> pipe:[2250636]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 876 -> pipe:[2251575]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 877 -> pipe:[2251576]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 878 -> pipe:[2251785]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 879 -> pipe:[2251786]
+    l-wx------ 1 michael michael 64 Aug 10 20:13 88 -> pipe:[2247853]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 880 -> pipe:[2249430]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 881 -> pipe:[2249431]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 882 -> pipe:[2251581]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 883 -> pipe:[2251582]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 884 -> pipe:[2250653]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 885 -> pipe:[2250654]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 886 -> pipe:[2251790]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 887 -> pipe:[2250670]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 888 -> pipe:[2250663]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 889 -> pipe:[2250664]
+    lr-x------ 1 michael michael 64 Aug 10 20:13 89 -> pipe:[2247854]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 890 -> pipe:[2250668]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 891 -> pipe:[2250669]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 892 -> pipe:[2250671]
+    l-wx------ 1 michael michael 64 Aug 10 20:14 894 -> pipe:[2251601]
+    lr-x------ 1 michael michael 64 Aug 10 20:14 895 -> pipe:[2251602]
+    lr-x------ 1 michael michael 64 Aug 10 20:13 9 -> pipe:[2241268]
+    lr-x------ 1 michael michael 64 Aug 10 20:13 90 -> pipe:[2242273]
+    l-wx------ 1 michael michael 64 Aug 10 20:13 91 -> pipe:[2245776]
+    etc
+
+"""]]
diff --git a/doc/bugs/Resource_exhausted/comment_6_26c98fca45b029a527f9684873db4be5._comment b/doc/bugs/Resource_exhausted/comment_6_26c98fca45b029a527f9684873db4be5._comment
new file mode 100644
index 000000000..b95b6cfa2
--- /dev/null
+++ b/doc/bugs/Resource_exhausted/comment_6_26c98fca45b029a527f9684873db4be5._comment
@@ -0,0 +1,18 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawln3ckqKx0x_xDZMYwa9Q1bn4I06oWjkog"
+ nickname="Michael"
+ subject="comment 6"
+ date="2013-08-11T03:20:37Z"
+ content="""
+with every
+
+    git annex
+    copy blah/blah/blah (to testremote...) 
+    git-annex: runInteractiveProcess: pipe: Too many open files
+    failed
+
+The number of open fd's by git-annex increases by 1.
+
+4.20130802 built with cabal on Ubuntu 13.04
+
+"""]]
diff --git a/doc/bugs/Resource_exhausted/comment_7_8bab413b472f900e04977db2bc3951b6._comment b/doc/bugs/Resource_exhausted/comment_7_8bab413b472f900e04977db2bc3951b6._comment
new file mode 100644
index 000000000..1b050aa33
--- /dev/null
+++ b/doc/bugs/Resource_exhausted/comment_7_8bab413b472f900e04977db2bc3951b6._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawln3ckqKx0x_xDZMYwa9Q1bn4I06oWjkog"
+ nickname="Michael"
+ subject="comment 7"
+ date="2013-08-11T03:21:45Z"
+ content="""
+...increases in the failing case, that is. This doesn't happen all the time.
+"""]]
diff --git a/doc/bugs/Resource_exhausted/comment_8_e9bec0b80179b1229b6af0979a21c727._comment b/doc/bugs/Resource_exhausted/comment_8_e9bec0b80179b1229b6af0979a21c727._comment
new file mode 100644
index 000000000..2135211cd
--- /dev/null
+++ b/doc/bugs/Resource_exhausted/comment_8_e9bec0b80179b1229b6af0979a21c727._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawln3ckqKx0x_xDZMYwa9Q1bn4I06oWjkog"
+ nickname="Michael"
+ subject="comment 8"
+ date="2013-08-11T20:44:19Z"
+ content="""
+It also looks like the location log has got corrupted (files are actually present, but not recorded in the location log) somewhere along the lines as I was trying to get/drop to figure out what's going on.
+Explicitly dropping files then getting files fixes the location log issue.
+"""]]
diff --git a/doc/bugs/Resource_exhausted/comment_9_419e24e0b91f569294ece28c42daa246._comment b/doc/bugs/Resource_exhausted/comment_9_419e24e0b91f569294ece28c42daa246._comment
new file mode 100644
index 000000000..0e06f5af9
--- /dev/null
+++ b/doc/bugs/Resource_exhausted/comment_9_419e24e0b91f569294ece28c42daa246._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.154.0.63"
+ subject="comment 9"
+ date="2013-08-24T19:10:21Z"
+ content="""
+@Michael how large a copy are you doing? And what kind of remote are you copying the files to?
+It would be helpful if you could be more specific about something I could do to reproduce the problem. Without a test case, I am unlikely to fix the bug. With a test case, I'd be surprised if it took long to fix it.
+
+If you have a process running that is experiencing the problem, you can also narrow it down a *lot* by looking at what these leaking pipe file descriptors are pipes to.  For example, if you have:
+
+lr-x------ 1 michael michael 64 Aug 10 20:14 895 -> pipe:[2251602]
+
+You can run `find /proc/ -ls 2251602` and find the process at other end of the pipe, and look its pid up in ps to see what command it is.
+"""]]