diff options
Diffstat (limited to 'doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock')
12 files changed, 147 insertions, 0 deletions
diff --git a/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_10_d44de6a250694b25ce9c3169d62db8d1._comment b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_10_d44de6a250694b25ce9c3169d62db8d1._comment new file mode 100644 index 000000000..f2ba4335c --- /dev/null +++ b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_10_d44de6a250694b25ce9c3169d62db8d1._comment @@ -0,0 +1,20 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 10""" + date="2016-03-01T20:52:38Z" + content=""" + 2456732 openat(AT_FDCWD, ".git/annex/ssh/", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = -1 ENOENT (No such file or directory) + 2456732 mkdir(".git/annex/ssh", 0777) = 0 + 2456732 open(".git/annex/ssh/smaug.lock", O_RDONLY|O_CREAT, 0666) = 11 + 2456732 fcntl(11, F_GETFD) = 0 + 2456732 fcntl(11, F_SETFD, FD_CLOEXEC) = 0 + 2456732 close(11) = 0 + +Backs up what I thought git-annex should be doing; it's not fcntl locking that file. + +Ah, I'll bet it's not git-annex at all this time. +It runs ssh with -S .git/annex/ssh/smaug, and ssh probably +does its own locking around setting up that control socket. + +If so, disabling annex.sshcaching will avoid the problem. +"""]] diff --git a/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_11_56ae0f15bbdea2331df3b261b74d0b0b._comment b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_11_56ae0f15bbdea2331df3b261b74d0b0b._comment new file mode 100644 index 000000000..2ee2e6aa8 --- /dev/null +++ b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_11_56ae0f15bbdea2331df3b261b74d0b0b._comment @@ -0,0 +1,7 @@ +[[!comment format=mdwn + username="https://me.yahoo.com/a/EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s-#7c0f4" + subject="comment 11" + date="2016-03-01T22:40:44Z" + content=""" +would then may be annex not to use sshcaching if operating under pidlock, unless some nfs specific flag is used to tease it apart +"""]] diff --git a/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_12_d499a2f44e8ee2f39ee959f4d6373570._comment b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_12_d499a2f44e8ee2f39ee959f4d6373570._comment new file mode 100644 index 000000000..5ab7c9e11 --- /dev/null +++ b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_12_d499a2f44e8ee2f39ee959f4d6373570._comment @@ -0,0 +1,13 @@ +[[!comment format=mdwn + username="https://me.yahoo.com/a/EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s-#7c0f4" + subject="comment 12" + date="2016-03-10T04:36:12Z" + content=""" +so -- shouldn't annex at least upon init sense if repo is under nfs? if to be done platform independent way then it could do smth like + +[[!format sh \"\"\" +python -c 'from glob import glob; import os; f=open(\".tmp-test\", \"w\"); os.unlink(\".tmp-test\"); assert(not glob(\".nfs*\"))' +\"\"\"]] + +somewhere under .git/annex/tmp .. so if .nfs* file gets generated -- under nfs. Seems to work for me in limited set of tests -- assertion fails all the time under NFS ;) +"""]] diff --git a/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_1_a98a54c04fa4e81f35fe958e746d61cb._comment b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_1_a98a54c04fa4e81f35fe958e746d61cb._comment new file mode 100644 index 000000000..21304e2e8 --- /dev/null +++ b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_1_a98a54c04fa4e81f35fe958e746d61cb._comment @@ -0,0 +1,7 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2016-03-01T14:36:25Z" + content=""" +FYI, I think you could remove the .nfs files to free up space. +"""]] diff --git a/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_2_18169e7bbd2caba5ee4bb0286961ac95._comment b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_2_18169e7bbd2caba5ee4bb0286961ac95._comment new file mode 100644 index 000000000..42b3265bd --- /dev/null +++ b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_2_18169e7bbd2caba5ee4bb0286961ac95._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 2""" + date="2016-03-01T15:35:48Z" + content=""" +Oddly, I cannot reproduce this, although I can reproduce the behavior in +<http://git-annex.branchable.com/bugs/thread_blocked_indefinitely_in_an_STM_transaction__while_moving_within__a_local_clone/>a + +(smaug:/tmp/123 has permissions that do not let me access it.) +"""]] diff --git a/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_3_e3b623ff6714a9fe5fa0d332c72fe32f._comment b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_3_e3b623ff6714a9fe5fa0d332c72fe32f._comment new file mode 100644 index 000000000..51f71c7ac --- /dev/null +++ b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_3_e3b623ff6714a9fe5fa0d332c72fe32f._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 3""" + date="2016-03-01T16:52:16Z" + content=""" +I've fixed the STM transaction bug. Need either more info to reproduce this +bug, or you could test and see if it still occurs when git-annex is +upgraded to ad888a6b760e8f9d31f8d99c51912bcdaa7fb0c1 +"""]] diff --git a/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_4_58eebd8cfd664b32ef6fd0ddc34c5e86._comment b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_4_58eebd8cfd664b32ef6fd0ddc34c5e86._comment new file mode 100644 index 000000000..7a562f7ef --- /dev/null +++ b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_4_58eebd8cfd664b32ef6fd0ddc34c5e86._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="https://me.yahoo.com/a/EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s-#7c0f4" + subject="more info" + date="2016-03-01T17:31:29Z" + content=""" +If we could remove those .nfs* files, it would indeed be not that bad but we can't + +smaug:/tmp/123 -- sorry about permissions but it is a regular annex nothing special, so the bug should show itself with other repos as well I think. I gave you access to it now and also there is /tmp/123.tar.gz archive of it just in case. +"""]] diff --git a/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_5_e5e24428ac02b78d38cd4f197ae3807b._comment b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_5_e5e24428ac02b78d38cd4f197ae3807b._comment new file mode 100644 index 000000000..75a45bdad --- /dev/null +++ b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_5_e5e24428ac02b78d38cd4f197ae3807b._comment @@ -0,0 +1,31 @@ +[[!comment format=mdwn + username="https://me.yahoo.com/a/EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s-#7c0f4" + subject="recent snapshot seems has fixed it" + date="2016-03-01T18:52:27Z" + content=""" +[[!format sh \"\"\" +$> git clone smaug:/tmp/123 123-clone && cd 123-clone && git config annex.pidlock true && echo 124 > 124.dat && git annex add 124.dat && git commit -m 'added 124' && git annex move --to=origin 124.dat +Cloning into '123-clone'... +remote: Counting objects: 22, done. +remote: Compressing objects: 100% (16/16), done. +remote: Total 22 (delta 3), reused 0 (delta 0) +Receiving objects: 100% (22/22), done. +Resolving deltas: 100% (3/3), done. +Checking connectivity... done. +total 1 +1 123.dat@ 1 README.txt +(merging origin/git-annex into git-annex...) +(recording state in git...) +add 124.dat ok +(recording state in git...) +[master 6eca577] added 124 + 1 file changed, 1 insertion(+) + create mode 120000 124.dat +move 124.dat (checking origin...) ok +(recording state in git...) + +$> git annex version +git-annex version: 6.20160301+gitg647fffd-1~ndall+1 + +\"\"\"]] +"""]] diff --git a/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_6_01dc7a1ff67783ce672d72cefe7b4bb5._comment b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_6_01dc7a1ff67783ce672d72cefe7b4bb5._comment new file mode 100644 index 000000000..20eca2f62 --- /dev/null +++ b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_6_01dc7a1ff67783ce672d72cefe7b4bb5._comment @@ -0,0 +1,7 @@ +[[!comment format=mdwn + username="https://me.yahoo.com/a/EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s-#7c0f4" + subject="comment 6" + date="2016-03-01T18:54:15Z" + content=""" +but then I found ./.git/annex/ssh/.nfs000000000000f41600003608.lock left behind (removable, luckily to me ;) ) +"""]] diff --git a/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_7_458518805b8d6613930b38b9ccc3c1bc._comment b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_7_458518805b8d6613930b38b9ccc3c1bc._comment new file mode 100644 index 000000000..756b64250 --- /dev/null +++ b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_7_458518805b8d6613930b38b9ccc3c1bc._comment @@ -0,0 +1,7 @@ +[[!comment format=mdwn + username="https://me.yahoo.com/a/EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s-#7c0f4" + subject="comment 7" + date="2016-03-01T18:58:20Z" + content=""" +and those are breeding with next subsequent --move +"""]] diff --git a/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_8_853bc273b19bd6d84ca8f5da6c3dfb56._comment b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_8_853bc273b19bd6d84ca8f5da6c3dfb56._comment new file mode 100644 index 000000000..d24d78d51 --- /dev/null +++ b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_8_853bc273b19bd6d84ca8f5da6c3dfb56._comment @@ -0,0 +1,20 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 8""" + date="2016-03-01T20:17:35Z" + content=""" +That ssh lock file is created by this code: + + -- The posix lock file is created even when using pid locks, in order to + -- avoid complicating any code that might expect to be able to see that + -- lock file. But, it's not locked. + dummyPosixLock :: Maybe FileMode -> LockFile -> IO () + dummyPosixLock m f = closeFd =<< openLockFile ReadLock m f + +But, that does not ever actually take a lock on the file, so +NFS should not make its .nfs thing in this case. Unless NFS does it when a +FD is simply opened with close-on-exec set. + +Can you get a strace of the creation of files under .git/annex/ssh/ +that result in these .nfs things? +"""]] diff --git a/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_9_86656a409ab25c7fa24de8ac3e68b254._comment b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_9_86656a409ab25c7fa24de8ac3e68b254._comment new file mode 100644 index 000000000..b862174b1 --- /dev/null +++ b/doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock/comment_9_86656a409ab25c7fa24de8ac3e68b254._comment @@ -0,0 +1,7 @@ +[[!comment format=mdwn + username="https://me.yahoo.com/a/EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s-#7c0f4" + subject="comment 9" + date="2016-03-01T20:43:17Z" + content=""" +ok -- see on smaug /mnt/nfs/scrap/datalad/test_nfs/123-clone-move.strace . Now you can experiment there as well -- the entire /mnt/btrfs/scrap is mounted also via nfs (under /mnt/nfs/scrap) +"""]] |