diff options
Diffstat (limited to 'doc/bugs')
74 files changed, 1739 insertions, 0 deletions
diff --git a/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library.mdwn b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library.mdwn new file mode 100644 index 000000000..5f7e490ca --- /dev/null +++ b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library.mdwn @@ -0,0 +1,62 @@ +### Please describe the problem. +I have lots of content stored in Amazon S3. Using git-annex from before commit 911ba8d972e4e7b151385d30c198598e1a0dfaca, I am able to ``git annex get`` from S3 and files are downloaded. +Using a more recent version (eg that commit, or the current master, or release 20150409), I am unable to download the content. + +I'm not sure if my repo or remote is somehow misconfigured, or if there's something else going on here. + +--Walter + +### What steps will reproduce the problem? +Use a version of git-annex with s3-aws + +### What version of git-annex are you using? On what operating system? +Debian, versions as above + +### Please provide any additional information below. + +[[!format sh """ +# If you can, paste a complete transcript of the problem occurring here. +# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log + +git annex get . --from cloud --debug +[2015-04-19 22:23:37 BST] read: git ["--git-dir=../../../.git","--work-tree=../../..","--literal-pathspecs","show-ref","git-annex"] +[2015-04-19 22:23:37 BST] read: git ["--git-dir=../../../.git","--work-tree=../../..","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"] +[2015-04-19 22:23:37 BST] read: git ["--git-dir=../../../.git","--work-tree=../../..","--literal-pathspecs","log","refs/heads/git-annex..a51a912223d3d86f19762e387e3eae23c3024d2c","-n1","--pretty=%H"] +[2015-04-19 22:23:37 BST] chat: git ["--git-dir=../../../.git","--work-tree=../../..","--literal-pathspecs","cat-file","--batch"] +[2015-04-19 22:23:37 BST] read: git ["--git-dir=../../../.git","--work-tree=../../..","--literal-pathspecs","ls-files","--cached","-z","--","."] +get IMG_7079.JPG (from cloud...) [2015-04-19 22:23:37 BST] chat: gpg ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] + +failed +get IMG_7080.JPG (from cloud...) +failed + + + +git annex info cloud +remote: cloud +description: [cloud] +uuid: be992080-b1db-11e1-8f79-1b10bb4092ef +trust: semitrusted +cost: 250.0 +type: S3 +creds: embedded in git repository (gpg encrypted) +bucket: ffffffffffffffffffffffffff +partsize: unlimited +encryption: encrypted (to gpg keys: FFFFFFFFFFFF) (hybrid mode) +chunking: none +remote annex keys: 0 +remote annex size: 0 bytes + + +git annex fsck -f cloud +fsck IMG_6876.JPG (checking cloud...) (StatusCodeException (Status {statusCode = 301, statusMessage = "Moved Permanently"}) [("x-amz-request-id","275ADF5B1B77D514"),("x-amz-id-2","flWGBHOZYEZAohygAzBIZAYd7nBGkm3HpSMfJuhgRp3txXx20yJz7S4yRlNLwCs1cHUMyWc9JbA="),("Content-Type","application/xml"),("Transfer-Encoding","chunked"),("Date","Sun, 19 Apr 2015 22:23:15 GMT"),("Server","AmazonS3")] (CJ {expose = []})) failed + + + +# End of transcript or log. +"""]] + +> I think I've made all the git-annex improvements that are going to come +> from this bug report. There's still the open bug on the aws library to better +> follow these redirects. Anyway, I think it makes sense to call this +> bug [[done]]. --[[Joey]] diff --git a/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_11_f30f86482d48ff8d658a4ee74d4c8586._comment b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_11_f30f86482d48ff8d658a4ee74d4c8586._comment new file mode 100644 index 000000000..2f2a3f0c6 --- /dev/null +++ b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_11_f30f86482d48ff8d658a4ee74d4c8586._comment @@ -0,0 +1,21 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawnSenxKyE_2Z6Wb-EBMO8FciyRywjx1ZiQ" + nickname="Walter" + subject="comment 11" + date="2015-04-23T20:29:17Z" + content=""" +I think I may have not been entirely clear previously; the file \"GPGHMACSHA1--417830f4c50a2887674917abd2c18c522853255a\" was not in the bucket, but git annex said it was. That is, the upload failed, but git annex thought it succeeded. + +Similarly, all the files I recently added we not actually uploaded, but git annex thought they were. I ``git annex fsck``ed them, which was fast because it failed to download any of them. fscking other files is slow, as it has to download them of course. + +Maybe to reproduce this, you could try: + +[[!format sh \"\"\" +git annex initremote cloud datacenter=ap-southeast-1 +git annex add file +git annex copy --to cloud file +git annex drop file +git annex enableremote could dataceter=ap-southeast-2 +git annex get file +\"\"\"]] +"""]] diff --git a/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_12_1ba3271a62b8ce8088c67e4741ecf385._comment b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_12_1ba3271a62b8ce8088c67e4741ecf385._comment new file mode 100644 index 000000000..3b416f09b --- /dev/null +++ b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_12_1ba3271a62b8ce8088c67e4741ecf385._comment @@ -0,0 +1,32 @@ +[[!comment format=mdwn + username="joey" + subject="""fully reproduced""" + date="2015-04-24T16:33:02Z" + content=""" +I was able to fully reproduce this bug! I installed the old version of +git-annex that used the S3 library, and made a remote: + + joey@darkstar:~/tmp/rrold>git annex initremote S3 type=S3 encryption=none datacenter=ap-southeast-1 + initremote S3 (checking bucket...) (creating bucket in ap-southeast-1...) ok + joey@darkstar:~/tmp/rrold>git annex move me --to S3 + move me (checking S3...) (to S3...) + ok + +Retrieval then failed using current git-annex. + +Also, a remote made with the old git-annex with datacenter=ap-southeast-2 +fails with the new git-annex. + +Hypothesis: Either the new or the old S3 library must be confusing between +ap-southeast-1/2. My guess is the old library was just creating and using +buckets in the wrong place, at least when told to use ap-southeast-*. + +--- + +I cannot reproduce anything about "the upload failed, but git annex thought it succeeded", +nor do I see any indications in comments 11 or 12 that git-annex's location +log is failing in any way. The sequence of commands in comment 11 ends with the +get failing, as it should, since the remote has been switched to a different +datacenter. I don't understand what you're seeing in comment #12 at all; +it seems to just show it getting a file successfully. +"""]] diff --git a/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_12_a208344e552bfe6b8d9f409560c9a515._comment b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_12_a208344e552bfe6b8d9f409560c9a515._comment new file mode 100644 index 000000000..030e82d9d --- /dev/null +++ b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_12_a208344e552bfe6b8d9f409560c9a515._comment @@ -0,0 +1,61 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawnSenxKyE_2Z6Wb-EBMO8FciyRywjx1ZiQ" + nickname="Walter" + subject="comment 12" + date="2015-04-23T21:03:36Z" + content=""" +For completeness, here is the output when I get a file that *is* properly in the bucket (and you could use for any further testing you need to do). + +While this may have been caused by some misconfiguration on my part (though I'm not entirely sure how that could happen, strangely it would be easier to muck up now enableremote doesn't create a new bucket), I feel the potential harm here (the location information being wrong) is quite serious. (I'm sure this point does not escape you). + +[[!format sh \"\"\" +>git annex get --force --debug file.jpg --from cloud +[2015-04-23 21:52:41 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"git-annex\"] +[2015-04-23 21:52:41 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"] +[2015-04-23 21:52:41 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..cb0f954d09e3ea28171434e0e7499c84d1722fce\",\"-n1\",\"--pretty=%H\"] +[2015-04-23 21:52:41 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..573f75e01681e9bf2b513bc85e18fc250298a4d3\",\"-n1\",\"--pretty=%H\"] +[2015-04-23 21:52:41 BST] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"cat-file\",\"--batch\"] +[2015-04-23 21:52:41 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"ls-files\",\"--cached\",\"-z\",\"--\",\"file.jpg\"] +[2015-04-23 21:52:41 BST] chat: gpg [\"--batch\",\"--no-tty\",\"--use-agent\",\"--quiet\",\"--trust-model\",\"always\",\"--decrypt\"] +(checking cloud...) [2015-04-23 21:52:42 BST] String to sign: \"HEAD\n\n\nThu, 23 Apr 2015 20:52:42 GMT\n/BUCKET/GPGHMACSHA1--08b3dee71059819e3558ac9ef8b82ad87e2d8951\" +[2015-04-23 21:52:42 BST] Host: \"BUCKET.s3-ap-southeast-2.amazonaws.com\" +[2015-04-23 21:52:42 BST] Path: \"/GPGHMACSHA1--08b3dee71059819e3558ac9ef8b82ad87e2d8951\" +[2015-04-23 21:52:42 BST] Query string: \"\" +[2015-04-23 21:52:42 BST] Response status: Status {statusCode = 200, statusMessage = \"OK\"} +[2015-04-23 21:52:42 BST] Response header 'x-amz-id-2': 'f8bEclNud1KNHevvGPVHutG3V0TH/ixnMSuu3NBhEKRrWaUYtENbKyA5PyxCdSrz0REgq/Bgu1w=' +[2015-04-23 21:52:42 BST] Response header 'x-amz-request-id': '7A344C3C3A27308E' +[2015-04-23 21:52:42 BST] Response header 'Date': 'Thu, 23 Apr 2015 20:52:43 GMT' +[2015-04-23 21:52:42 BST] Response header 'Last-Modified': 'Fri, 31 Oct 2014 07:03:03 GMT' +[2015-04-23 21:52:42 BST] Response header 'ETag': '\"66a85b0007a52d82e5bd29192ebdb510\"' +[2015-04-23 21:52:42 BST] Response header 'Accept-Ranges': 'bytes' +[2015-04-23 21:52:42 BST] Response header 'Content-Type': '' +[2015-04-23 21:52:42 BST] Response header 'Content-Length': '46058' +[2015-04-23 21:52:42 BST] Response header 'Server': 'AmazonS3' +[2015-04-23 21:52:42 BST] Response metadata: S3: request ID=<none>, x-amz-id-2=<none> +get file.jpg (from cloud...) +[2015-04-23 21:52:42 BST] String to sign: \"GET\n\n\nThu, 23 Apr 2015 20:52:42 GMT\n/BUCKET/GPGHMACSHA1--08b3dee71059819e3558ac9ef8b82ad87e2d8951\" +[2015-04-23 21:52:42 BST] Host: \"BUCKET.s3-ap-southeast-2.amazonaws.com\" +[2015-04-23 21:52:42 BST] Path: \"/GPGHMACSHA1--08b3dee71059819e3558ac9ef8b82ad87e2d8951\" +[2015-04-23 21:52:42 BST] Query string: \"\" +[2015-04-23 21:52:43 BST] Response status: Status {statusCode = 200, statusMessage = \"OK\"} +[2015-04-23 21:52:43 BST] Response header 'x-amz-id-2': 'LRDMgQAj+F81m3UqDebJ5CoZdyM/c2tMaFUvhjn8kjqq3x2Evy7O+wgLUiwE7lqascd0yrHR+xA=' +[2015-04-23 21:52:43 BST] Response header 'x-amz-request-id': '068D946E995E7473' +[2015-04-23 21:52:43 BST] Response header 'Date': 'Thu, 23 Apr 2015 20:52:44 GMT' +[2015-04-23 21:52:43 BST] Response header 'Last-Modified': 'Fri, 31 Oct 2014 07:03:03 GMT' +[2015-04-23 21:52:43 BST] Response header 'ETag': '\"66a85b0007a52d82e5bd29192ebdb510\"' +[2015-04-23 21:52:43 BST] Response header 'Accept-Ranges': 'bytes' +[2015-04-23 21:52:43 BST] Response header 'Content-Type': '' +[2015-04-23 21:52:43 BST] Response header 'Content-Length': '46058' +[2015-04-23 21:52:43 BST] Response header 'Server': 'AmazonS3' +[2015-04-23 21:52:43 BST] Response metadata: S3: request ID=068D946E995E7473, x-amz-id-2=LRDMgQAj+F81m3UqDebJ5CoZdyM/c2tMaFUvhjn8kjqq3x2Evy7O+wgLUiwE7lqascd0yrHR+xA= +99% 22.5KB/s 0s[2015-04-23 21:52:44 BST] chat: gpg [\"--batch\",\"--no-tty\",\"--use-agent\",\"--quiet\",\"--trust-model\",\"always\",\"--batch\",\"--passphrase-fd\",\"14\",\"--decrypt\"] +ok +[2015-04-23 21:52:44 BST] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"hash-object\",\"-w\",\"--stdin-paths\",\"--no-filters\"] +[2015-04-23 21:52:44 BST] feed: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"update-index\",\"-z\",\"--index-info\"] +[2015-04-23 21:52:44 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"] +(recording state in git...) +[2015-04-23 21:52:44 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"write-tree\"] +[2015-04-23 21:52:44 BST] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"commit-tree\",\"444e504a0ab73d01df08ef731e691205cfd485f5\",\"--no-gpg-sign\",\"-p\",\"refs/heads/git-annex\"] +[2015-04-23 21:52:44 BST] call: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"update-ref\",\"refs/heads/git-annex\",\"6e57ed008525cd58641c54a5ac6f07a960a7dc5c\"] +\"\"\"]] +"""]] diff --git a/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_14_035a933289f95c365ce2c851f6946181._comment b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_14_035a933289f95c365ce2c851f6946181._comment new file mode 100644 index 000000000..96d212a39 --- /dev/null +++ b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_14_035a933289f95c365ce2c851f6946181._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawnSenxKyE_2Z6Wb-EBMO8FciyRywjx1ZiQ" + nickname="Walter" + subject="comment 14" + date="2015-04-24T20:13:31Z" + content=""" +Playing around with it, I also can't reproduce it (using new or old versions of git-annex; it may be, as you allude, a problem in an old version of the s3 library). + +Anyway, I'm happy that it's working now. +"""]] diff --git a/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_16_2f16c63e539e51d9d1c0f85605e6d1e8._comment b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_16_2f16c63e539e51d9d1c0f85605e6d1e8._comment new file mode 100644 index 000000000..e5141a30b --- /dev/null +++ b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_16_2f16c63e539e51d9d1c0f85605e6d1e8._comment @@ -0,0 +1,22 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 16""" + date="2015-04-25T01:18:51Z" + content=""" +Investigating further, when I create a bucket with the AWS library +in ap-southeast-2, `s3cmd info` shows it is located there. + +When I create a bucket with hS3 in ap-southeast-2, I get this +interesting output: + + joey@darkstar:~>s3cmd info s3://s3-43302240-076c-4420-8099-f2ef0b517e5f + s3://s3-43302240-076c-4420-8099-f2ef0b517e5f/ (bucket): + Location: ap-southeast-2 + WARNING: Redirected to: s3-43302240-076c-4420-8099-f2ef0b517e5f.s3-ap-southeast-2.amazonaws.com + Expiration Rule: none + policy: none + ACL: joeyhess: FULL_CONTROL + +So, it's apparently in the datacenter I asked for when making it, +but here's a redirect again. +"""]] diff --git a/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_1_533c4a26200501486a9ec103e1301391._comment b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_1_533c4a26200501486a9ec103e1301391._comment new file mode 100644 index 000000000..8921214a9 --- /dev/null +++ b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_1_533c4a26200501486a9ec103e1301391._comment @@ -0,0 +1,18 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2015-04-21T19:23:18Z" + content=""" +So the http 301 redirect seems to be the relevant clue. + +Perhaps the old S3 library followed such redirects and the new aws library +does not. Although AFAICS, http-client should follow redirects by default. + +Are you able to store new files in this remote and get them back out? + +Which Amazon datacenter is the bucket at? + +I've added some more debugging to the S3 remote; --debug will now +have it log the http requests and responses. Perhaps this will give us +more clues to the problem. +"""]] diff --git a/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_2_24224679a1516e2852b43624c787f639._comment b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_2_24224679a1516e2852b43624c787f639._comment new file mode 100644 index 000000000..2bafdf7ea --- /dev/null +++ b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_2_24224679a1516e2852b43624c787f639._comment @@ -0,0 +1,63 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawnSenxKyE_2Z6Wb-EBMO8FciyRywjx1ZiQ" + nickname="Walter" + subject="comment 2" + date="2015-04-21T22:53:21Z" + content=""" +It's in bucket ap-southeast-2, which conflicts with the info in the log below. I'm not sure why they disagree. + +This was a new file, which I also \"uploaded\" with a ``git annex copy --to cloud test_file``, and ``git annex whereis`` says is in cloud. However, using ``s3cmd`` to retrieve it (using the info from the log below), it claims it doesn't exist (404). So, I don't think it got uploaded correctly. I don't seem to get any useful logs when forcing an upload with ``--debug`` (as in no S3-related logs, but I've included that at the very bottom). + +[[!format sh \"\"\" +> git annex get test_file +get test_file (from cloud...) + + Unable to access these remotes: cloud + + Try making some of these repositories available: + be992080-b1db-11e1-8f79-1b10bb4092ef -- [cloud] + + (Note that these git remotes have annex-ignore set: origin) +failed +git-annex: get: 1 failed +walter@kronos:~/NewPics$ git annex get test_file --debug +[2015-04-21 23:39:57 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"ls-files\",\"--cached\",\"-z\",\"--\",\"test_file\"] +get test_file [2015-04-21 23:39:57 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"git-annex\"] +[2015-04-21 23:39:57 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"] +[2015-04-21 23:39:57 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..347f154f5dfa9e41dc459eda328421741e1e90a6\",\"-n1\",\"--pretty=%H\"] +[2015-04-21 23:39:57 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..83793925a571a3228cc64e204598f8c54203b1f7\",\"-n1\",\"--pretty=%H\"] +[2015-04-21 23:39:57 BST] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"cat-file\",\"--batch\"] +(from cloud...) [2015-04-21 23:39:57 BST] chat: gpg [\"--batch\",\"--no-tty\",\"--use-agent\",\"--quiet\",\"--trust-model\",\"always\",\"--decrypt\"] + +[2015-04-21 23:39:57 BST] String to sign: \"GET\n\n\nTue, 21 Apr 2015 22:39:57 GMT\n/BUCKETID/GPGHMACSHA1--417830f4c50a2887674917abd2c18c522853255a\" +[2015-04-21 23:39:57 BST] Host: \"BUCKETID.s3-ap-southeast-1.amazonaws.com\" +[2015-04-21 23:39:57 BST] Path: \"/GPGHMACSHA1--417830f4c50a2887674917abd2c18c522853255a\" +[2015-04-21 23:39:57 BST] Query string: \"\" +[2015-04-21 23:39:57 BST] Response status: Status {statusCode = 301, statusMessage = \"Moved Permanently\"} +[2015-04-21 23:39:57 BST] Response header 'x-amz-request-id': 'C2825FBB20ED22B4' +[2015-04-21 23:39:57 BST] Response header 'x-amz-id-2': 'I93feDTHOrPR+bwVqoMBuEEwYQAN7ZfjOq0jdIJ6ywzOPYYxTfqZg9OR+M0L+MFdilHKRJ+CEv8=' +[2015-04-21 23:39:57 BST] Response header 'Content-Type': 'application/xml' +[2015-04-21 23:39:57 BST] Response header 'Transfer-Encoding': 'chunked' +[2015-04-21 23:39:57 BST] Response header 'Date': 'Tue, 21 Apr 2015 22:39:56 GMT' +[2015-04-21 23:39:57 BST] Response header 'Server': 'AmazonS3' +[2015-04-21 23:39:57 BST] Response metadata: S3: request ID=C2825FBB20ED22B4, x-amz-id-2=I93feDTHOrPR+bwVqoMBuEEwYQAN7ZfjOq0jdIJ6ywzOPYYxTfqZg9OR+M0L+MFdilHKRJ+CEv8= + + Unable to access these remotes: cloud + + Try making some of these repositories available: + be992080-b1db-11e1-8f79-1b10bb4092ef -- [cloud] + + (Note that these git remotes have annex-ignore set: origin) +failed +git-annex: get: 1 failed + +> git annex copy --to cloud --force --debug test_file +[2015-04-21 23:47:24 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"git-annex\"] +[2015-04-21 23:47:24 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"] +[2015-04-21 23:47:24 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..347f154f5dfa9e41dc459eda328421741e1e90a6\",\"-n1\",\"--pretty=%H\"] +[2015-04-21 23:47:24 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..83793925a571a3228cc64e204598f8c54203b1f7\",\"-n1\",\"--pretty=%H\"] +[2015-04-21 23:47:24 BST] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"cat-file\",\"--batch\"] +[2015-04-21 23:47:24 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"ls-files\",\"--cached\",\"-z\",\"--\",\"test_file\"] + +\"\"\"]] +"""]] diff --git a/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_3_9665a868f786afa445f5e3b0fbd20011._comment b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_3_9665a868f786afa445f5e3b0fbd20011._comment new file mode 100644 index 000000000..8b92ca305 --- /dev/null +++ b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_3_9665a868f786afa445f5e3b0fbd20011._comment @@ -0,0 +1,29 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 3""" + date="2015-04-23T17:00:20Z" + content=""" +ap-southeast-2 vs ap-southeast-1 must have something to do with this +problem. + +I tried making a S3 remote with datacenter=ap-southeast-2 and it didn't +have this problem, and used s3-ap-southeast-2.amazonaws.com, so doesn't +seem that git-annex is getting the mapping to endpoints wrong. + +I also tried forcing git-annex to use ap-southeast-1 instead of the right +datacenter, and I don't get a redirect, getting a file just fails in that +misconfiguration. + +So, it seems there is something special about your bucket; git-annex thinks +it's in ap-southeast-1, S3 apparently redirects to somewhere else. Sounds +like the AWS interface shows you the bucket is located in ap-southeast-2? + +This is looking more like a bug with the haskell-AWS library. If AWS can +send redirects here, it ought to follow them. It would be really helpful if +I had a way to reproduce the problem. Since your remote is encrypted +anyway, is there any chance you could generate AWS creds that I could use +to access that bucket to try to get files from it? + +(The `copy --to cloud` didn't do anything because git-annex already thinks +the file is there (--force has no effect on copy --to).) +"""]] diff --git a/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_4_b70a0075e61565e4501f0b5b143b222d._comment b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_4_b70a0075e61565e4501f0b5b143b222d._comment new file mode 100644 index 000000000..2f862c560 --- /dev/null +++ b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_4_b70a0075e61565e4501f0b5b143b222d._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 4""" + date="2015-04-23T18:17:23Z" + content=""" +I agree, enableremote should not create the bucket. Made that change. Also, +`git annex info $s3remote` will show some more info about it, including its +endpoint. +"""]] diff --git a/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_4_d37d9b008e2e8f570af86e620ad6064f._comment b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_4_d37d9b008e2e8f570af86e620ad6064f._comment new file mode 100644 index 000000000..b79f5812d --- /dev/null +++ b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_4_d37d9b008e2e8f570af86e620ad6064f._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawnSenxKyE_2Z6Wb-EBMO8FciyRywjx1ZiQ" + nickname="Walter" + subject="comment 4" + date="2015-04-23T17:58:14Z" + content=""" +I've sent an email regarding this. + +One comment, it seems that it's not possible to change the configuration of an S3 remote, as ``enableremote`` causes git-annex to try to create a new bucket (which fails, as the name is already used). Otherwise, I could try to tell git-annex it has the region wrong. Perhaps an option ``use-existing-bucket`` or something would work? +"""]] diff --git a/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_6_7caef30897eabf04faab12f4b4a16916._comment b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_6_7caef30897eabf04faab12f4b4a16916._comment new file mode 100644 index 000000000..fd55b5134 --- /dev/null +++ b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_6_7caef30897eabf04faab12f4b4a16916._comment @@ -0,0 +1,29 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 6""" + date="2015-04-23T18:44:02Z" + content=""" +I reproduced the problem using the creds, and a modified version of +GetObject.hs from haskell-aws. + +I had to upgrade from aws-0.9.2 to 0.11.4 to see the 301 redirect in the +debug mesages, but both versions seemed to fail the same way, it's just the +newer version added more debugging output. + +There's an exception, which git-annex obscured: + + Response metadata: S3: request ID=5B5B9AE39E0C9729, x-amz-id-2=t6xToNbirPnwgEhTdbFr+Ncq5cGr3fMCRNq5WlFLjEk3ZJtan5aCotcsCS3GMTMgjsP/MNcOWUw= + GetObject: HeaderException {headerErrorMessage = "ETag missing"} + +I've seen this "ETag missing" before when an object was genuinely missing, but +I'm not sure what it indicates in this case. + +This was using s3-ap-southeast-1.amazonaws.com. If I use southeast-2, it +just fails with "NoSuchKey". So I think that the 1-vs-2 was a red herring; +git-annex is using the right endpoint. + +I have forwarded this bug report to: <https://github.com/aristidb/aws/issues/160> + +It might be good to get in touch with the haskell-aws maintainer and provide the +creds so they can reproduce it too. +"""]] diff --git a/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_7_cd943597e319c94e91b17f24346be456._comment b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_7_cd943597e319c94e91b17f24346be456._comment new file mode 100644 index 000000000..171b2b371 --- /dev/null +++ b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_7_cd943597e319c94e91b17f24346be456._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 7""" + date="2015-04-23T18:59:10Z" + content=""" +Hrm, I get an identical error message if I try to request a file that +is certianly not there! + +So maybe the 301 is a red herring? +"""]] diff --git a/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_8_13f862524d4aa503fc998ede41617942._comment b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_8_13f862524d4aa503fc998ede41617942._comment new file mode 100644 index 000000000..531e8c79d --- /dev/null +++ b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_8_13f862524d4aa503fc998ede41617942._comment @@ -0,0 +1,12 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawnSenxKyE_2Z6Wb-EBMO8FciyRywjx1ZiQ" + nickname="Walter" + subject="comment 8" + date="2015-04-23T19:02:18Z" + content=""" +Ok, so I did ``git annex enableremote cloud datacenter=ap-southeast-2``, and can now get files properly. So from that point of view it now works. And I guess that provides an easy way to reproduce (just set a working S3 remote to the wrong datacenter). I'm prepared to accept that this was something I did somehow (at some point I manually moved files from one S3 bucket (actually account) to another, but it seems that git-annex would have created the bucket, so I'm not sure how the datacenter could be wrong.) + +In any case, now I'm not sure exactly which files did get uploaded properly, so will run a ``fsck``. I guess it would be good to either return an error when this happens, or follow the redirect. + +Also, I really appreciate the quick response you have to bugs! +"""]] diff --git a/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_8_44915e2b9e871c16861223e1e2a0827b._comment b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_8_44915e2b9e871c16861223e1e2a0827b._comment new file mode 100644 index 000000000..dd6f2f7a6 --- /dev/null +++ b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_8_44915e2b9e871c16861223e1e2a0827b._comment @@ -0,0 +1,13 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 8""" + date="2015-04-23T19:06:02Z" + content=""" +Glad that resolved it, but I'm still confused why I can't seem to get the +test file when I have my test program use ap-southeast-2. (But maybe those +creds you gave me just don't let me or something..) + +If you're up for some sleuthing, you could check out the git-annex branch +and look through `git blame remotes.log`. And changes that git-annex ever +made to the config of the remote will be in the history of that file. +"""]] diff --git a/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_9_769de1e47221dfb6c810665e3704bbb2._comment b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_9_769de1e47221dfb6c810665e3704bbb2._comment new file mode 100644 index 000000000..788ccf7ed --- /dev/null +++ b/doc/bugs/Can__39__t_get_content_from_S3_with_s3-aws_library/comment_9_769de1e47221dfb6c810665e3704bbb2._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawnSenxKyE_2Z6Wb-EBMO8FciyRywjx1ZiQ" + nickname="Walter" + subject="comment 9" + date="2015-04-23T19:07:56Z" + content=""" +Is it possible to do a fast ``fsck`` on an S3 remote? Because I don't want to download all the files again, it would be nice to just have the option to check it it exists. + +I get a ``failed to download file from remote`` error when I try it. +"""]] diff --git a/doc/bugs/Data_loss_when_doing___96__git_annex_import_--force__96__.mdwn b/doc/bugs/Data_loss_when_doing___96__git_annex_import_--force__96__.mdwn new file mode 100644 index 000000000..60e04fc14 --- /dev/null +++ b/doc/bugs/Data_loss_when_doing___96__git_annex_import_--force__96__.mdwn @@ -0,0 +1,62 @@ +### Please describe the problem. + +Calling `git annex import --force file-in-working-copy` removes `file-in-working-copy` from disk. + +My workflow is: + +1) copy the CR2 files from a card to the desired directory structure using a tool of my choice, +2) import the created directory layout to git-annex + +### What version of git-annex are you using? On what operating system? + +[[!format sh """ +$ git-annex version +git-annex version: 5.20150327 +build flags: Pairing Testsuite S3 DBus DNS Feeds Quvi TDFA +key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E MD5E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 MD5 WORM URL +remote types: git gcrypt S3 bup directory rsync web bittorrent tahoe glacier ddar hook external +local repository version: 5 +supported repository version: 5 +upgrade supported from repository versions: 0 1 2 4 +"""]] + +### How to reproduce: + +[[!format sh """ +jkt@svist ~/temp $ mkdir annex-add-force-data-loss +jkt@svist ~/temp $ cd annex-add-force-data-loss/ +jkt@svist ~/temp/annex-add-force-data-loss $ git init +Initialized empty Git repository in /home/jkt/temp/annex-add-force-data-loss/.git/ +jkt@svist ~/temp/annex-add-force-data-loss $ echo 1234 > foo +jkt@svist ~/temp/annex-add-force-data-loss $ ls -al +total 16K +drwxr-xr-x 3 jkt jkt 27 May 8 13:54 . +drwx------ 55 jkt jkt 8.0K May 8 13:54 .. +drwxr-xr-x 6 jkt jkt 96 May 8 13:54 .git +-rw-r--r-- 1 jkt jkt 5 May 8 13:54 foo +jkt@svist ~/temp/annex-add-force-data-loss $ git annex import --force foo +git-annex: First run: git-annex init +jkt@svist ~/temp/annex-add-force-data-loss $ git annex init +init ok +(recording state in git...) +jkt@svist ~/temp/annex-add-force-data-loss $ ls -al +total 16K +drwxr-xr-x 3 jkt jkt 27 May 8 13:54 . +drwx------ 55 jkt jkt 8.0K May 8 13:54 .. +drwxr-xr-x 8 jkt jkt 119 May 8 13:54 .git +-rw-r--r-- 1 jkt jkt 5 May 8 13:54 foo +jkt@svist ~/temp/annex-add-force-data-loss $ git annex import --force foo +import foo +git-annex: foo: rename: does not exist (No such file or directory) +failed +git-annex: import: 1 failed +jkt@svist ~/temp/annex-add-force-data-loss $ ls -al +total 12K +drwxr-xr-x 3 jkt jkt 17 May 8 13:55 . +drwx------ 55 jkt jkt 8.0K May 8 13:54 .. +drwxr-xr-x 8 jkt jkt 119 May 8 13:55 .git +"""]] +...and the file is gone :(. + +> You should use `git annex add` in this case, not import. +> I've made import refuse to run in this case. [[done]] --[[Joey]] diff --git a/doc/bugs/Git-Annex_requires_all_repositories_to_repair.mdwn b/doc/bugs/Git-Annex_requires_all_repositories_to_repair.mdwn index 7087247f5..eef705e69 100644 --- a/doc/bugs/Git-Annex_requires_all_repositories_to_repair.mdwn +++ b/doc/bugs/Git-Annex_requires_all_repositories_to_repair.mdwn @@ -1,3 +1,6 @@ I recently had my git-annex repository die and it needed to be repaired. Two of my repositories are external hard drives. When I tried to use git-annex repair, it would churn for some hours, then error because the external hard drives were not plugged in. When I brought the two hard drives home from the various places that they are (safely) stored, it all worked fine, but it would have been great if git-annex repair could somehow do what it could with what was connected and do the rest as and when the other drives are plugged in. This must only become more of a problem as git-annex is used for longer, as one may have a handful of USB keys storing a little on each. [[!taglink moreinfo]] + +> With the lack of an error message or any followup, it's hard to take +> this bug seriously, so [[done]] --[[Joey]] diff --git a/doc/bugs/Proxy_support.mdwn b/doc/bugs/Proxy_support.mdwn index ae6eaf689..2af63d706 100644 --- a/doc/bugs/Proxy_support.mdwn +++ b/doc/bugs/Proxy_support.mdwn @@ -17,3 +17,26 @@ Please provide any additional information below. I don't use networkmanager if proxy information is obtained from it. There should be a fallback to environment variables. [[!tag confirmed]] + +> Here's a patch that shows how to enable proxy support for the +> parts of git-annex that use http-client and http-conduit: +> <https://github.com/kiwiroy/git-annex/commit/18a3ceda1beb7c356541270f37ae4c0d56ada726> +> +> Other parts of git-annex use wget/curl and should already support +> the environment variables. + +>> I had a closer look at this, and it turns out that http-client +>> now enables http proxy support by default, when the environment variable +>> is set. Since version 0.4.7. See `defaultProxy`. +>> +>> git-annex uses http-client for all its http access (either directly +>> or through layers like http-conduit and DAV). +>> +>> So I don't need to change +>> git-annex at all; it just needs to be built with a new enough version +>> of http-client. This will happen for the linux autobuilders once the new +>> version reaches Debian. I've updated the OSX autobuilder's http-client +>> so it will already support proxying and anyone can built git-annex +>> using the new http-client and get proxy support. +>> +>> Calling this [[done]] as nothing remains to do in git-annex. --[[Joey]] diff --git a/doc/bugs/Repository_Information_Is_Lost.mdwn b/doc/bugs/Repository_Information_Is_Lost.mdwn index 7b11ac4cf..cbaf866c8 100644 --- a/doc/bugs/Repository_Information_Is_Lost.mdwn +++ b/doc/bugs/Repository_Information_Is_Lost.mdwn @@ -31,3 +31,9 @@ Clones of my repositories lost all track of other repositories they only seem to """]] [[!tag moreinfo]] + +> No followup for over a year, and not enough information in the intial +> report to know if there's a bug at all. It occurs to me that the user +> might have just forgotten to push the git-annex branch to wherever they +> later cloned the repo from. `git annex sync` would fix that mistake up. +> Anyway, [[done]]. --[[Joey]] diff --git a/doc/bugs/Small_archive_behaving_like_archive.mdwn b/doc/bugs/Small_archive_behaving_like_archive.mdwn index 384a75baf..947777279 100644 --- a/doc/bugs/Small_archive_behaving_like_archive.mdwn +++ b/doc/bugs/Small_archive_behaving_like_archive.mdwn @@ -31,3 +31,6 @@ Mac OSX 10.8.3 (Build 12D78) """]] [[!tag moreinfo]] + +> Since there's a plausible explanation in my comment and no followup, +> [[done]] --[[Joey]] diff --git a/doc/bugs/__34__transfers_in_progress__34___gets_the_direction_wrong.mdwn b/doc/bugs/__34__transfers_in_progress__34___gets_the_direction_wrong.mdwn new file mode 100644 index 000000000..e0b02a704 --- /dev/null +++ b/doc/bugs/__34__transfers_in_progress__34___gets_the_direction_wrong.mdwn @@ -0,0 +1,55 @@ +### Please describe the problem. + + 07:32 warp@18-volt:/mnt/zooyork/annex/somerepo (master)$ git annex info --fast + repository mode: indirect + trusted repositories: 0 + semitrusted repositories: 4 + 00000000-0000-0000-0000-000000000001 -- web + 3dca408f-ccfd-48df-a8a0-b7d517a482ef -- zooyork [here] + 9fbb4e2c-4ed9-47b4-877c-74d2eeea6296 -- [kirby] + d1b57403-2bc2-41d8-9a8e-eec0d3f31e67 -- [pucca] + untrusted repositories: 0 + transfers in progress: + downloading folder/some-large-file.mp4 from pucca + available local disk space: 232.2 gigabytes (+1 megabyte reserved) + +The "transfers in progress" section of the above "git annex info" output describes a file being downloaded from pucca, presumably to [here], which is zooyork. This does not describe the transfer correctly, when I ran the "git annex info --fast" command another git annex process was downloading folder/some-large-file.mp4 from zooyork to pucca (so, in the other direction than git annex info described). + +### What steps will reproduce the problem? + +1. Make two regular git annex repositories, add some big files so a transfer takes long enough that you can run some git annex commands while the transfer is ongoing +2. cd /repo/a ; git annex get a-big-file.mp4 --from=b +3. (in a different window), cd /repo/b ; git annex info --fast + +### What version of git-annex are you using? On what operating system? + + 07:37 warp@18-volt:/mnt/zooyork/annex$ git annex version + git-annex version: 5.20141105-g8b19598 + build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3 WebDAV Inotify DBus DesktopNotify XMPP DNS Feeds Quvi TDFA CryptoHash + key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 WORM URL + remote types: git gcrypt S3 bup directory rsync web webdav tahoe glacier ddar hook external + + 07:37 warp@18-volt:/mnt/zooyork/annex$ lsb_release -a + No LSB modules are available. + Distributor ID: Ubuntu + Description: Ubuntu 14.04 LTS + Release: 14.04 + Codename: trusty + + +### Please provide any additional information below. + +If you need more info, I'm "kuno" in the #git-annex IRC channel. + +[[!format sh """ +# If you can, paste a complete transcript of the problem occurring here. +# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log + + +# End of transcript or log. +"""]] + +> When I follow those steps, I see "uploading bigfile" in repo B, +> which is indeed the case since repo A is downloading that file from B. +> +> I suspect you're the one who got confused. [[done]] --[[Joey]] diff --git a/doc/bugs/_git-annex:_bad_url_ssh:__47____47__git__64__gitlab.com:__126____47__gitlabname__47__reponame.git.mdwn b/doc/bugs/_git-annex:_bad_url_ssh:__47____47__git__64__gitlab.com:__126____47__gitlabname__47__reponame.git.mdwn new file mode 100644 index 000000000..74e9235dd --- /dev/null +++ b/doc/bugs/_git-annex:_bad_url_ssh:__47____47__git__64__gitlab.com:__126____47__gitlabname__47__reponame.git.mdwn @@ -0,0 +1,33 @@ +### Please describe the problem. +An encrypted remote is added to a working git annex repository (mind ":~/" in the remote add command). However, after that any git annex command fails. + +### What steps will reproduce the problem? + > git remote add encrypted gcrypt::ssh://git@gitlab.com:~/gitlabname/reponame.git + > git push encrypted master + gcrypt: Repository not found: ssh://git@gitlab.com:~/gitlabname/reponame.git + gcrypt: Setting up new repository + gcrypt: Remote ID is :id:abcdefghijklmnopqrst + Counting objects: 53, done. + Compressing objects: 100% (52/52), done. + Total 53 (delta 12), reused 0 (delta 0) + gcrypt: Encrypting to: --throw-keyids --default-recipient-self + gcrypt: Requesting manifest signature + ... + To gcrypt::ssh://git@gitlab.com:~/gitlabname/reponame.git + * [new branch] master -> master + > + > git annex sync + git-annex: bad url ssh://git@gitlab.com:~/gitlabname/reponame.git + +### What version of git-annex are you using? On what operating system? +5.20150419-g900e1b6 on Mac OS X + +### Please provide any additional information below. + +[[!format sh """ +# If you can, paste a complete transcript of the problem occurring here. +# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log +- + +# End of transcript or log. +"""]] diff --git a/doc/bugs/_git-annex:_bad_url_ssh:__47____47__git__64__gitlab.com:__126____47__gitlabname__47__reponame.git/comment_1_60db0d6bfc71a62b3c1021527a8d2d60._comment b/doc/bugs/_git-annex:_bad_url_ssh:__47____47__git__64__gitlab.com:__126____47__gitlabname__47__reponame.git/comment_1_60db0d6bfc71a62b3c1021527a8d2d60._comment new file mode 100644 index 000000000..4b7b0e90a --- /dev/null +++ b/doc/bugs/_git-annex:_bad_url_ssh:__47____47__git__64__gitlab.com:__126____47__gitlabname__47__reponame.git/comment_1_60db0d6bfc71a62b3c1021527a8d2d60._comment @@ -0,0 +1,23 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2015-04-29T15:16:31Z" + content=""" +According to the git-fetch man page, the syntax to use +for this kind of url is: + + ssh://[user@]host.xz[:port]/~[user]/path/to/repo.git/ + +Your url is missing the leading slash before the `~`, and has +a : with no port specified. + + ssh://git@gitlab.com:~/gitlabname/reponame.git + +It is in fact, not a legal url. + +Now, git might accept it despite not documenting it as an accepted form, +but why wander into undefined territory when there are legal ways to write +this url that work fine? + +Does GitLab promote using these malformed urls? +"""]] diff --git a/doc/bugs/_git-annex:_bad_url_ssh:__47____47__git__64__gitlab.com:__126____47__gitlabname__47__reponame.git/comment_2_49682a91147990a578929678880bd226._comment b/doc/bugs/_git-annex:_bad_url_ssh:__47____47__git__64__gitlab.com:__126____47__gitlabname__47__reponame.git/comment_2_49682a91147990a578929678880bd226._comment new file mode 100644 index 000000000..e93708644 --- /dev/null +++ b/doc/bugs/_git-annex:_bad_url_ssh:__47____47__git__64__gitlab.com:__126____47__gitlabname__47__reponame.git/comment_2_49682a91147990a578929678880bd226._comment @@ -0,0 +1,26 @@ +[[!comment format=mdwn + username="http://rfhbuk.pip.verisignlabs.com.pip.verisignlabs.com/" + nickname="rfhb" + subject="comment 2" + date="2015-04-29T15:45:34Z" + content=""" +Thank you. I was wondering about the URL as flagged above. When I specify the URL as I did, the git commands work but when I change it to ssh://git@gitlab.com:/~/gitlabname/reponame.git, git command fails with + + > git push encrypted master + gcrypt: Development version -- Repository format MAY CHANGE + gcrypt: Repository not found: ssh://git@gitlab.com:/~/gitlabname/reponame.git + gcrypt: Setting up new repository + gcrypt: Remote ID is :id:xxxxxxxxxxxxxxxxx + Counting objects: 3, done. + Total 3 (delta 0), reused 0 (delta 0) + gcrypt: Encrypting to: --throw-keyids --default-recipient-self + gcrypt: Requesting manifest signature + .... passphase entered ... + GitLab: No such project + fatal: Could not read from remote repository. + Please make sure you have the correct access rights + and the repository exists. + error: failed to push some refs to 'gcrypt::ssh://git@gitlab.com:/~/gitlabname/reponame.git' + +So perhaps I need to investigate how to get gcrypt to work with remote git(lab) repositories. Thanks. +"""]] diff --git a/doc/bugs/_git-annex:_bad_url_ssh:__47____47__git__64__gitlab.com:__126____47__gitlabname__47__reponame.git/comment_3_bf0fa06596338444a3772d1865b17e26._comment b/doc/bugs/_git-annex:_bad_url_ssh:__47____47__git__64__gitlab.com:__126____47__gitlabname__47__reponame.git/comment_3_bf0fa06596338444a3772d1865b17e26._comment new file mode 100644 index 000000000..0d7fb9c2d --- /dev/null +++ b/doc/bugs/_git-annex:_bad_url_ssh:__47____47__git__64__gitlab.com:__126____47__gitlabname__47__reponame.git/comment_3_bf0fa06596338444a3772d1865b17e26._comment @@ -0,0 +1,7 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 3""" + date="2015-04-29T17:09:07Z" + content=""" +This works: ssh://git@gitlab.com/username/reponame.git +"""]] diff --git a/doc/bugs/addurl_+_sync_vs_addurl_+_commit.mdwn b/doc/bugs/addurl_+_sync_vs_addurl_+_commit.mdwn new file mode 100644 index 000000000..21aa96952 --- /dev/null +++ b/doc/bugs/addurl_+_sync_vs_addurl_+_commit.mdwn @@ -0,0 +1,27 @@ +### Please describe the problem. + +I think this is what happened; I need to go back and check this again (maybe I was just misreading something) but I want to get it written down first. + +I've got a git repository, and I just ran git annex init. Then I ran git annex addurl a bunch of times, followed by git annex sync. The result was apparently a repository where the files downloaded by addurl were added using the SHA256 backend rather than the URL backend. I deleted the branches and tried again, but this time after calling git annex addurl a bunch of times I did a normal git commit. This time everything looked fine; the files were all listed in as present in the web remote. + +### What steps will reproduce the problem? + +[[!format sh """ +git annex init +git annex addurl "https://archive.org/download/emularity_engine_jsmess/messnapple2e.js.gz" --file "messnapple2e.js.gz" +git annex sync +"""]] + +### What version of git-annex are you using? On what operating system? + + git-annex version: 5.20150412-g2be4834 + build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3 WebDAV Inotify DBus DesktopNotify XMPP DNS Feeds Quvi TDFA TorrentParser + key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E MD5E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 MD5 WORM URL + remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external + local repository version: 5 + supported repository version: 5 + upgrade supported from repository versions: 0 1 2 4 +This is on Linux. + +> In the lack of any followups, I'm confident this was a case of user +> error, so closing. [[done]] --[[Joey]] diff --git a/doc/bugs/addurl_+_sync_vs_addurl_+_commit/comment_1_513d89ddd728dc9c6cc9fbf8af18afb0._comment b/doc/bugs/addurl_+_sync_vs_addurl_+_commit/comment_1_513d89ddd728dc9c6cc9fbf8af18afb0._comment new file mode 100644 index 000000000..48ce2354d --- /dev/null +++ b/doc/bugs/addurl_+_sync_vs_addurl_+_commit/comment_1_513d89ddd728dc9c6cc9fbf8af18afb0._comment @@ -0,0 +1,17 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2015-04-18T18:44:16Z" + content=""" +If you use `git annex addurl`, the file will use the SHA256 backend. + +If you use `git annex addurl --fast`, it will use the URL backend. + +**Either way**, git-annex will know that you added the file from the web, +and `git-annex whereis` will tell you the file is in the web remote, +and tell you its url. + +I don't think there's a bug here. I think you got confused by +seeing URL some of the time and SHA256 some of the time. +If you disagree, please post a transcript demonstrating your problem. +"""]] diff --git a/doc/bugs/clean-duplicates_causes_data_loss.mdwn b/doc/bugs/clean-duplicates_causes_data_loss.mdwn new file mode 100644 index 000000000..c5d545420 --- /dev/null +++ b/doc/bugs/clean-duplicates_causes_data_loss.mdwn @@ -0,0 +1,30 @@ +### Please describe the problem. + +Use of git-annex import --clean-duplicates can cause data loss, where git-annex deletes content that it doesn't actually have a copy of (i.e. there is no duplicate). + +### What steps will reproduce the problem? + +I've written a quick'n'dirty test script which goes through a bunch of combinations and tests --clean-duplicates. Given an 'origin' repo containing 'a' and 'b' content and a clone of it ('import') which doesn't contain 'a' and 'b' content. + +g-a import --clean-duplicates ~/tmp/importme (containing a, b and c) into 'import' after: + + Origin is set to trusted in import, b is dropped from within origin: + b is deleted from importme even though no annexes have copies (reasonable, as origin is set to trusted and import thinks it has the content). + + Origin is set to semitrusted in import, b is dropped within origin: + b is deleted from importme even though no annexes have copies (this is most likely one to bite people). + + Origin is set to untrusted in import, b is dropped within origin: + b is deleted from importme even though no annexes have copies and git-annex has been explicitly told to not trust information about origin :( This is really surprising behaviour! + +### What version of git-annex are you using? On what operating system? + +* 5.20150409 +* Arch Linux (git-annex-bin) + +### Please provide any additional information below. + +I can provide the script if it is wanted (coded in Perl, couple of non-core dependencies). + +> Decided to go ahead and make it check remotes like drop does, so [[done]] +> --[[Joey]] diff --git a/doc/bugs/clean-duplicates_causes_data_loss/comment_1_0c5da8b1285d16967a0423a0e259e06b._comment b/doc/bugs/clean-duplicates_causes_data_loss/comment_1_0c5da8b1285d16967a0423a0e259e06b._comment new file mode 100644 index 000000000..689cef97a --- /dev/null +++ b/doc/bugs/clean-duplicates_causes_data_loss/comment_1_0c5da8b1285d16967a0423a0e259e06b._comment @@ -0,0 +1,36 @@ +[[!comment format=mdwn + username="CandyAngel" + subject="comment 1" + date="2015-04-24T11:57:35Z" + content=""" +Command to exemplify the \"worst case\" (untrusted causing deletion): + + mkdir /tmp/ga-icd + cd /tmp/ga-icd + + git init origin + cd origin + git commit -m create + git annex init origin + echo a > a + echo b > b + git annex add . + git commit -m files + + mkdir /tmp/ga-icd/importme + echo a > a + echo b > b + echo c > c + + cd /tmp/ga-icd + git clone origin import + git annex init import + +So we now have origin (with content for 2 files), import which knows origin has content for both files and directory we want to clean up. The following causes 'b' to be lost (hope you have backups!). + + cd /tmp/ga-icd/origin + git annex drop b --force + cd /tmp/ga-icd/import + git annex untrust origin + git annex import --clean-duplicates /tmp/ga-icd/importme +"""]] diff --git a/doc/bugs/clean-duplicates_causes_data_loss/comment_2_14c5c3417f107a3619a4402f75bc0002._comment b/doc/bugs/clean-duplicates_causes_data_loss/comment_2_14c5c3417f107a3619a4402f75bc0002._comment new file mode 100644 index 000000000..c99c7699f --- /dev/null +++ b/doc/bugs/clean-duplicates_causes_data_loss/comment_2_14c5c3417f107a3619a4402f75bc0002._comment @@ -0,0 +1,17 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 2""" + date="2015-04-29T17:18:17Z" + content=""" +--clean-duplicates is not really intended to go do a full verification +that the annexed content is present. I can see how this is surprising; +it would probably be better if it were named "--clean-seen-content" +or something like that. + +But, --deduplicate has the same behavior for such files, and I can't see a +way to rename that more effectively. + +For now I have improved the git-annex-import man page to note that +annexed content that's been removed would make a file be considered a +duplicate. +"""]] diff --git a/doc/bugs/clean-duplicates_causes_data_loss/comment_3_ed3f40b5798d9e770657e9c6b8e28039._comment b/doc/bugs/clean-duplicates_causes_data_loss/comment_3_ed3f40b5798d9e770657e9c6b8e28039._comment new file mode 100644 index 000000000..f4c367d9c --- /dev/null +++ b/doc/bugs/clean-duplicates_causes_data_loss/comment_3_ed3f40b5798d9e770657e9c6b8e28039._comment @@ -0,0 +1,62 @@ +[[!comment format=mdwn + username="CandyAngel" + subject="comment 3" + date="2015-04-29T18:42:38Z" + content=""" +Well, yeah, it is quite surprising as you don't like git-annex messing about with files outside the annex, but one of the commands that does so will cause wanton destruction with barely any checks against data loss. + +Oddly, if the origin is marked as dead instead of untrusted, --clean-duplicates doesn't remove anything from /tmp/ga-icd/importme, even though 'import' still knows about the files and has enough information to delete them. Weird. + + + mkdir /tmp/ga-icd + + cd /tmp/ga-icd + + git init origin + Initialized empty Git repository in /tmp/ga-icd/origin/.git/ + + cd origin + + git commit -m create --allow-empty + [master (root-commit) 6cfdbc1] create + + git annex init origin + init origin ok + (recording state in git...) + + echo a + + echo b + + git annex add . + add a ok + add b ok + (recording state in git...) + + git commit -m files + [master 2c2ed64] files + 2 files changed, 2 insertions(+) + create mode 120000 a + create mode 120000 b + + mkdir /tmp/ga-icd/importme + + cd /tmp/ga-icd/importme + + echo a + + echo b + + echo c + + cd /tmp/ga-icd + + git clone origin import + Cloning into 'import'... + done. + + cd import + + git annex init import + init import (merging origin/git-annex into git-annex...) + ok + (recording state in git...) + + cd /tmp/ga-icd/origin + + git annex drop b --force + drop b ok + (recording state in git...) + + cd /tmp/ga-icd/import + + git annex dead origin + dead origin ok + (recording state in git...) + + git annex import --clean-duplicates /tmp/ga-icd/importme + + ls /tmp/ga-icd/import + a b + + ls /tmp/ga-icd/importme/ + a b c + +I'm think I'm just going to steer clear of it completely (and roll my own) until it is as fussy about preserving data as the rest of git-annex. + +(Also, apologies for the original name, I didn't realise it would cause any problems.) +"""]] diff --git a/doc/bugs/commitBuffer:_invalid_argument___40__invalid_character__41__2.mdwn b/doc/bugs/commitBuffer:_invalid_argument___40__invalid_character__41__2.mdwn index 7a67a7509..1be88f4a1 100644 --- a/doc/bugs/commitBuffer:_invalid_argument___40__invalid_character__41__2.mdwn +++ b/doc/bugs/commitBuffer:_invalid_argument___40__invalid_character__41__2.mdwn @@ -20,3 +20,5 @@ It will fail at entry "DR14: Verschwörungstheorien" ### What version of git-annex are you using? On what operating system? 5.20150327-g19a1a35 (standalone build) on Fedora 21 + +[[!tag moreinfo]] diff --git a/doc/bugs/corrupt_backend_upon_sync__63__.mdwn b/doc/bugs/corrupt_backend_upon_sync__63__.mdwn index b669c22ec..e99dfcc88 100644 --- a/doc/bugs/corrupt_backend_upon_sync__63__.mdwn +++ b/doc/bugs/corrupt_backend_upon_sync__63__.mdwn @@ -74,3 +74,5 @@ $ git annex get --from=laptop # End of transcript or log. """]] + +[[!tag moreinfo]] diff --git a/doc/bugs/dolphin_integration_file_is_broken.mdwn b/doc/bugs/dolphin_integration_file_is_broken.mdwn new file mode 100644 index 000000000..2f2fc1c4e --- /dev/null +++ b/doc/bugs/dolphin_integration_file_is_broken.mdwn @@ -0,0 +1,37 @@ +### Please describe the problem. + +git annex will automatically create the file + +.kde/share/kde4/services/ServiceMenus/git-annex.desktop + +However the actions created do not work because the variable used is %U (file:/// style URL) which git annex does not understand. + +According to http://standards.freedesktop.org/desktop-entry-spec/latest/ar01s06.html + +Also the escaping seems broken. The following line is one that works for me. + + Exec=sh -c 'cd "$(dirname -- "$1")" && git-annex get --notify-start --notify-finish -- "$1"' command_string_is_ignored %f + +or simply + + Exec=git-annex get --notify-start --notify-finish -- %F + +### What steps will reproduce the problem? + + +### What version of git-annex are you using? On what operating system? + +5.20141125 + +### Please provide any additional information below. + +[[!format sh """ +# If you can, paste a complete transcript of the problem occurring here. +# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log + + +# End of transcript or log. +"""]] + +> [[fixed|done]]; confirmed the new version still works on filenames with +> spaces in them. --[[Joey]] diff --git a/doc/bugs/dolphin_integration_file_is_broken/comment_1_65c4fd7547be9c16abadaf5fc9d0b793._comment b/doc/bugs/dolphin_integration_file_is_broken/comment_1_65c4fd7547be9c16abadaf5fc9d0b793._comment new file mode 100644 index 000000000..15077277e --- /dev/null +++ b/doc/bugs/dolphin_integration_file_is_broken/comment_1_65c4fd7547be9c16abadaf5fc9d0b793._comment @@ -0,0 +1,11 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2015-04-29T16:49:22Z" + content=""" +You forgot to say what version of dolphin you are having trouble with. +I've tested it working with version 4.14.2. + +I don't understand what the "command_string_is_ignored" in your example +is supposed to do, or why you'd not want to quote the filename. +"""]] diff --git a/doc/bugs/dolphin_integration_file_is_broken/comment_2_237fdaddaa80c1e4d262d724d9aed82b._comment b/doc/bugs/dolphin_integration_file_is_broken/comment_2_237fdaddaa80c1e4d262d724d9aed82b._comment new file mode 100644 index 000000000..3dcaa7c21 --- /dev/null +++ b/doc/bugs/dolphin_integration_file_is_broken/comment_2_237fdaddaa80c1e4d262d724d9aed82b._comment @@ -0,0 +1,21 @@ +[[!comment format=mdwn + username="silvio" + subject="comment 2" + date="2015-04-29T19:24:55Z" + content=""" +I currently use dolphin Version 14.12.3 but it's been broken for me forever. I always had my own fix but now it keeps getting overwritten by git-annex. + +the current line in my git-annex version is: + +sh -c 'cd \"$(dirname '%U')\" && git-annex get --notify-start --notify-finish -- %U' + +The problem is that %U gives an url style file (file:///some/path) which git-annex doesn't understand. + +Further more the quoting is broken because and i quote: + +\"Field codes (%F,%U) must not be used inside a quoted argument, the result of field code expansion inside a quoted argument is undefined.\" + +Additionally I don't think it's possible to prevent an arbitrary command injection unless you have the Field code as a single argument. + +The command_string_is_ignored is just the string that will be assigned to $0 which is not used. You could use it instead of $1 but didn't like using the command string as an argument. +"""]] diff --git a/doc/bugs/dolphin_integration_file_is_broken/comment_3_89c166c4e44380b8f354f2c6b5e42fad._comment b/doc/bugs/dolphin_integration_file_is_broken/comment_3_89c166c4e44380b8f354f2c6b5e42fad._comment new file mode 100644 index 000000000..fdbc3beb1 --- /dev/null +++ b/doc/bugs/dolphin_integration_file_is_broken/comment_3_89c166c4e44380b8f354f2c6b5e42fad._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 3""" + date="2015-04-29T19:37:42Z" + content=""" +So a much newer version than the one in Debian yet.. + +I tried your command line with the old version in Debian, and it seems to +work ok, so thanks for the improvement! +"""]] diff --git a/doc/bugs/dolphin_integration_file_is_broken/comment_3_f7624258e8c2269f3ccdab683df8bb3d._comment b/doc/bugs/dolphin_integration_file_is_broken/comment_3_f7624258e8c2269f3ccdab683df8bb3d._comment new file mode 100644 index 000000000..a6437e470 --- /dev/null +++ b/doc/bugs/dolphin_integration_file_is_broken/comment_3_f7624258e8c2269f3ccdab683df8bb3d._comment @@ -0,0 +1,7 @@ +[[!comment format=mdwn + username="silvio" + subject="comment 3" + date="2015-04-29T19:39:33Z" + content=""" +Also %U gives a list of urls and basename of a list doesn't give back anything useful. I don't see how this could possibly work for you. +"""]] diff --git a/doc/bugs/dolphin_integration_file_is_broken/comment_4_3e12372915c4c72b05db2a1c28a9daf7._comment b/doc/bugs/dolphin_integration_file_is_broken/comment_4_3e12372915c4c72b05db2a1c28a9daf7._comment new file mode 100644 index 000000000..783fd6cb1 --- /dev/null +++ b/doc/bugs/dolphin_integration_file_is_broken/comment_4_3e12372915c4c72b05db2a1c28a9daf7._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="silvio" + subject="comment 4" + date="2015-04-29T19:47:21Z" + content=""" +sorry for so many posts :) + +As for not quoting the file name I think it doesn't need to be. It will still be just one argument. Exec doesn't work like sh. The standard doesn't mention anything about single quote quoting either but it seems to work and there is no good alternative. +"""]] diff --git a/doc/bugs/failure_to_find_file_that___34__should__34___exist_in_remote_is_silent.mdwn b/doc/bugs/failure_to_find_file_that___34__should__34___exist_in_remote_is_silent.mdwn index 5718e5011..6197861e1 100644 --- a/doc/bugs/failure_to_find_file_that___34__should__34___exist_in_remote_is_silent.mdwn +++ b/doc/bugs/failure_to_find_file_that___34__should__34___exist_in_remote_is_silent.mdwn @@ -35,3 +35,9 @@ My particular issue has probably existed through a few version upgrades though. # End of transcript or log. """]] + +> Since neither of us can think of a better behavior for `git annex +> get/copy > --from remote` in this case, I've been reduced to documenting +> it better. The docs now mention that --from will cause it to silently +> skip files that are not present in the specified remote. So, [[done]] +> --[[Joey]] diff --git a/doc/bugs/failure_to_return_to_indirect_mode_on_usb.mdwn b/doc/bugs/failure_to_return_to_indirect_mode_on_usb.mdwn index f2a11eea8..a61c67254 100644 --- a/doc/bugs/failure_to_return_to_indirect_mode_on_usb.mdwn +++ b/doc/bugs/failure_to_return_to_indirect_mode_on_usb.mdwn @@ -17,3 +17,7 @@ git-annex: indirect: 1 failed """]] [[!tag moreinfo]] + +> I don't like closing bug reports, but after over a year with no followup, +> and with the original bug report lacking even an error message, closing +> it seems like the best course of action. [[done]] --[[Joey]] diff --git a/doc/bugs/fsck_--from___36__remote__44___copies_symlinks_instead_of_content.mdwn b/doc/bugs/fsck_--from___36__remote__44___copies_symlinks_instead_of_content.mdwn new file mode 100644 index 000000000..732af2990 --- /dev/null +++ b/doc/bugs/fsck_--from___36__remote__44___copies_symlinks_instead_of_content.mdwn @@ -0,0 +1,80 @@ +### Please describe the problem. + + git annex fsck --from $remote + +copies the symlinks from the remote into *.git/annex/tmp*, which then fail to fsck as they don't point to content. + +In the transcript below, the 'a' file should fail fsck, but the rest pass. + +### What steps will reproduce the problem? + +See transcript + +### What version of git-annex are you using? On what operating system? + +* git-annex version: 5.20150409-g3575ee5 +* Arch Linux (git-annex-bin) + +### Please provide any additional information below. + + ## git init origin + ## cd corrupt/ + ## git annex init corrupt + ## echo a > a.txt + ## echo b > b.txt + ## echo c > c.txt + + ## git annex add . + add a.txt ok + add b.txt ok + add c.txt ok + (recording state in git...) + + ## git commit -m "add files" + [master (root-commit) 1d670a5] add files + 3 files changed, 3 insertions(+) + create mode 120000 a.txt + create mode 120000 b.txt + create mode 120000 c.txt + +("corrupting" a file not needed but here for completeness) + + ## chmod +w .git/annex/objects/41/pJ/SHA256E-s2--87428fc522803d31065e7bce3cf03fe475096631e5e07bbd7a0fde60c4cf25c7.txt/ + ## echo 'd' > .git/annex/objects/41/pJ/SHA256E-s2--87428fc522803d31065e7bce3cf03fe475096631e5e07bbd7a0fde60c4cf25c7.txt/SHA256E-s2--87428fc522803d31065e7bce3cf03fe475096631e5e07bbd7a0fde60c4cf25c7.txt + + ## cd .. + + ## git clone corrupt/ recovery/ + Cloning into 'recovery'... + done. + + ## cd recovery/ + + ## git annex init recovery + init recovery (merging origin/git-annex into git-annex...) + (recording state in git...) + ok + (recording state in git...) + + ## git annex fsck --from origin + fsck a.txt + git-annex: .git/annex/tmp/fsck24477.SHA256E-s2--87428fc522803d31065e7bce3cf03fe475096631e5e07bbd7a0fde60c4cf25c7.txt: getFileStatus: does not exist (No such file or directory) + failed + fsck b.txt + git-annex: .git/annex/tmp/fsck24477.SHA256E-s2--0263829989b6fd954f72baaf2fc64bc2e2f01d692d4de72986ea808f6e99813f.txt: getFileStatus: does not exist (No such file or directory) + failed + fsck c.txt + git-annex: .git/annex/tmp/fsck24477.SHA256E-s2--a3a5e715f0cc574a73c3f9bebb6bc24f32ffd5b67b387244c2c909da779a1478.txt: getFileStatus: does not exist (No such file or directory) + failed + (recording state in git...) + git-annex: fsck: 3 failed + + ## ls -lah .git/annex/tmp/ + total 20K + drwxr-xr-x 2 gemma users 4.0K Apr 18 17:27 .. + drwxr-xr-x 5 gemma users 4.0K Apr 18 17:27 .. + lrwxrwxrwx 1 gemma users 197 Apr 18 17:27 fsck24477.SHA256E-s2--0263829989b6fd954f72baaf2fc64bc2e2f01d692d4de72986ea808f6e99813f.txt -> ../corrupt/.git/annex/objects/x7/01/SHA256E-s2--0263829989b6fd954f72baaf2fc64bc2e2f01d692d4de72986ea808f6e99813f.txt/SHA256E-s2--0263829989b6fd954f72baaf2fc64bc2e2f01d692d4de72986ea808f6e99813f.txt + lrwxrwxrwx 1 gemma users 197 Apr 18 17:27 fsck24477.SHA256E-s2--87428fc522803d31065e7bce3cf03fe475096631e5e07bbd7a0fde60c4cf25c7.txt -> ../corrupt/.git/annex/objects/41/pJ/SHA256E-s2--87428fc522803d31065e7bce3cf03fe475096631e5e07bbd7a0fde60c4cf25c7.txt/SHA256E-s2--87428fc522803d31065e7bce3cf03fe475096631e5e07bbd7a0fde60c4cf25c7.txt + lrwxrwxrwx 1 gemma users 197 Apr 18 17:27 fsck24477.SHA256E-s2--a3a5e715f0cc574a73c3f9bebb6bc24f32ffd5b67b387244c2c909da779a1478.txt -> ../corrupt/.git/annex/objects/Vw/zz/SHA256E-s2--a3a5e715f0cc574a73c3f9bebb6bc24f32ffd5b67b387244c2c909da779a1478.txt/SHA256E-s2--a3a5e715f0cc574a73c3f9bebb6bc24f32ffd5b67b387244c2c909da779a1478.txt + +> reproduced, test cased, fixed, [[done]] --[[Joey]] diff --git a/doc/bugs/git-annex_deletes_file_when_using___34__git_annex_get__34___after___34__git_annex_addurl_--file__34__.mdwn b/doc/bugs/git-annex_deletes_file_when_using___34__git_annex_get__34___after___34__git_annex_addurl_--file__34__.mdwn new file mode 100644 index 000000000..1081b7acf --- /dev/null +++ b/doc/bugs/git-annex_deletes_file_when_using___34__git_annex_get__34___after___34__git_annex_addurl_--file__34__.mdwn @@ -0,0 +1,112 @@ +### Please describe the problem. + +When using "`git annex addurl --file`" with an ftp url, the committed +file is deleted after dropping the contents with --force (because +git-annex can't determine if the ftp server contains a valid copy) and +executing "`git annex get`". It's the "`git annex get`" command that +deletes the file. + +This does not happen when using an http url. + +### What steps will reproduce the problem? + +`git clone https://gist.github.com/sunny256/24f6c29645efd0aab4d9` + +and execute the bash script `runme`. There's more info in a long comment +there, plus various flags you can enable/disable to test under different +conditions. + +### What version of git-annex are you using? On what operating system? + +Using the newest git-annex from <https://downloads.kitenet.net/.git/> in +directory git-annex/linux/current/, 5.20150420-gb0ebb23. + +Have tested with versions way back to v5.20131221, they all behave the +same. + +Using Debian GNU/Linux 7.8 (wheezy) on x86_64 with brand new git 2.4.0. + +### Please provide any additional information below. + +[[!format sh """ +# If you can, paste a complete transcript of the problem occurring here. +# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log + +$ ./runme + +================== git init ================== +Initialized empty Git repository in /home/sunny/src/git/ga-bug/tmpdirawedsfkn/.git/ + +================== git annex init ================== +init ok +(recording state in git...) + +================== git commit, empty start commit ================== +[master (root-commit) 6d5d623] Empty startcommit + +================== git annex addurl ================== +addurl README (downloading ftp://ftp.funet.fi/pub/Linux/mirrors/debian/README ...) +--2015-05-02 03:28:59-- ftp://ftp.funet.fi/pub/Linux/mirrors/debian/README + => '.git/annex/tmp/URL--ftp&c%%ftp.funet.fi%pub%Linux%mirrors%debian%README' +Resolving ftp.funet.fi (ftp.funet.fi)... 193.166.3.2, 2001:708:10:9::20:2 +Connecting to ftp.funet.fi (ftp.funet.fi)|193.166.3.2|:21... connected. +Logging in as anonymous ... Logged in! +==> SYST ... done. ==> PWD ... done. +==> TYPE I ... done. ==> CWD (1) /pub/Linux/mirrors/debian ... done. +==> SIZE README ... 1495 +==> PASV ... done. ==> RETR README ... done. +Length: 1495 (1.5K) (unauthoritative) + +100%[================================================>] 1,495 --.-K/s in 0.01s + +2015-05-02 03:29:00 (125 KB/s) - '.git/annex/tmp/URL--ftp&c%%ftp.funet.fi%pub%Linux%mirrors%debian%README' saved [1495] + +ok +(recording state in git...) + +================== git commit, add README ================== +[master 264d597] Add README + 1 file changed, 1 insertion(+) + create mode 120000 README + +================== git annex drop --force ================== +drop README ok +(recording state in git...) + +================== git annex get ================== +get README (from web...) +--2015-05-02 03:29:00-- ftp://ftp.funet.fi/pub/Linux/mirrors/debian/README + => '.git/annex/tmp/SHA256-s1495--8822780b87a880ca9956ac108812557044618859cecb07df488df57e8134e34f' +Resolving ftp.funet.fi (ftp.funet.fi)... 193.166.3.2, 2001:708:10:9::20:2 +Connecting to ftp.funet.fi (ftp.funet.fi)|193.166.3.2|:21... connected. +Logging in as anonymous ... Logged in! +==> SYST ... done. ==> PWD ... done. +==> TYPE I ... done. ==> CWD (1) /pub/Linux/mirrors/debian ... done. +==> SIZE README ... 1495 +==> PASV ... done. ==> RETR README ... done. +Length: 1495 (1.5K) (unauthoritative) + +100%[================================================>] 1,495 --.-K/s in 0s + +2015-05-02 03:29:02 (73.1 MB/s) - '.git/annex/tmp/SHA256-s1495--8822780b87a880ca9956ac108812557044618859cecb07df488df57e8134e34f' saved [1495] + +ok +(recording state in git...) + +================== ls -l ================== +total 0 + +README is gone, should not happen + +Reached the end +$ + +# End of transcript or log. +"""]] + +> workaround in place; [[done]] --[[Joey]] + +> Also, fixed it to allow dropping the file if the ftp server seems +> to reply with a successful result (it's replying with 350, which is not +> unambiguously good, but since curl is able to get the right file length, +> the file is presumably still on the ftp server. --[[Joey]] diff --git a/doc/bugs/git-annex_deletes_file_when_using___34__git_annex_get__34___after___34__git_annex_addurl_--file__34__/comment_1_dca81d5db9a966fc992ed28bb3c2a242._comment b/doc/bugs/git-annex_deletes_file_when_using___34__git_annex_get__34___after___34__git_annex_addurl_--file__34__/comment_1_dca81d5db9a966fc992ed28bb3c2a242._comment new file mode 100644 index 000000000..65ebab1f2 --- /dev/null +++ b/doc/bugs/git-annex_deletes_file_when_using___34__git_annex_get__34___after___34__git_annex_addurl_--file__34__/comment_1_dca81d5db9a966fc992ed28bb3c2a242._comment @@ -0,0 +1,19 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2015-05-05T17:11:38Z" + content=""" +Thanks for a great bug report! + +Unfortunately, this turns out to be a bug in wget, as shown by this transcript: + + joey@darkstar:~/tmp/y>ls + README@ + joey@darkstar:~/tmp/y>wget -q --show-progress --clobber -c -O .git/annex/tmp/SHA256E-s1495--8822780b87a880ca9956ac108812557044618859cecb07df488df57e8134e34f ftp://ftp.funet.fi/pub/Linux/mirrors/debian/README --user-agent git-annex/5.20150505-gcdb212f + joey@darkstar:~/tmp/y>ls + joey@darkstar:~/tmp/y> + +I have filed a bug report on wget <http://bugs.debian.org/784348>, +and I guess I'll try to work around it in git-annex by running wget +inside an empty temp directory. +"""]] diff --git a/doc/bugs/git-annex_deletes_file_when_using___34__git_annex_get__34___after___34__git_annex_addurl_--file__34__/comment_2_c683d729fb0b3285ba9946539fe50d0d._comment b/doc/bugs/git-annex_deletes_file_when_using___34__git_annex_get__34___after___34__git_annex_addurl_--file__34__/comment_2_c683d729fb0b3285ba9946539fe50d0d._comment new file mode 100644 index 000000000..8c757b8a3 --- /dev/null +++ b/doc/bugs/git-annex_deletes_file_when_using___34__git_annex_get__34___after___34__git_annex_addurl_--file__34__/comment_2_c683d729fb0b3285ba9946539fe50d0d._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://sunny256.wordpress.com/" + nickname="sunny256" + subject="Thanks" + date="2015-05-07T14:20:49Z" + content=""" +Thank you for looking into this and coming up with a fix. Awesome. :) +"""]] diff --git a/doc/bugs/git-annex_fails_to_parse_external_remotes__39_____34__CHECKPRESENT-SUCCESS__34___response.mdwn b/doc/bugs/git-annex_fails_to_parse_external_remotes__39_____34__CHECKPRESENT-SUCCESS__34___response.mdwn new file mode 100644 index 000000000..f18d7a522 --- /dev/null +++ b/doc/bugs/git-annex_fails_to_parse_external_remotes__39_____34__CHECKPRESENT-SUCCESS__34___response.mdwn @@ -0,0 +1,56 @@ +### Please describe the problem. + +During a git annex fsck --fast --from <someexternalremote>, any CHECKPRESENT-SUCCESS responses are considered failures: + + fsck file + failed to download file from remote + failed + (recording state in git...) + git-annex: fsck: 1 failed + +It doesn't appear that the external protocol is being violated in any way; it occurs both with my special external (https://git.encryptio.com/slime/blob/refs/heads/master:/misc/git-annex-remote-slime/main.go) and with the example external remote. Making these dump their IO to stderr shows they're behaving correctly, as far as I can tell. + +`git annex testremote` passes on these remotes too, so it's not catching the issue (though it probably should.) + +These implementations used to work fine (~6mo ago? not sure); I haven't been able to get git-annex to build (cabal woes), so I can't bisect it myself. + +### What steps will reproduce the problem? + +With https://git-annex.branchable.com/special_remotes/external/example.sh/ installed as "git-annex-remote-externaltest", this script shows the issue: + +[[!format sh """ +#!/bin/sh +set -e + +[ -e "annex-test-dirs/repo/.git/annex/objects" ] && ( + find "annex-test-dirs/repo/.git/annex/objects" -type f -exec chmod 0644 {} \; + find "annex-test-dirs/repo/.git/annex/objects" -type d -exec chmod 0755 {} \; +) + +rm -rf annex-test-dirs +mkdir -p annex-test-dirs/{repo,data} + +cd annex-test-dirs/repo +git init +git annex init +MYPASSWORD=a MYLOGIN=b git annex initremote ext type=external encryption=none externaltype=externaltest directory="$(pwd)/../data" + +echo "data" > file +git annex add file +git commit -m message + +git annex copy --to ext + +# this works: +git annex fsck --from ext + +# but this incorrectly fails and marks the file "not present": +git annex fsck --fast --from ext +"""]] + +### What version of git-annex are you using? On what operating system? + +git-annex 5.20150420-gb0ebb23, from the linux standalone tarball, on linux. + +> Upgrade, this was fixed earlier this week, in [[!commit +> cfbeb1e7b74aa9759bd62b342e80869de582f10d]]; [[done]] afaik --[[Joey]] diff --git a/doc/bugs/git-annex_happily_deleted_most_of_my_files___36____35____38____33__/comment_5_fe2ba9550fc9dc2dc877965b436fe823._comment b/doc/bugs/git-annex_happily_deleted_most_of_my_files___36____35____38____33__/comment_5_fe2ba9550fc9dc2dc877965b436fe823._comment new file mode 100644 index 000000000..d5d2c35d2 --- /dev/null +++ b/doc/bugs/git-annex_happily_deleted_most_of_my_files___36____35____38____33__/comment_5_fe2ba9550fc9dc2dc877965b436fe823._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawkW9u-8uqR62QBZjeTNCXsL7Ds55dAMGbA" + nickname="Horea" + subject="comment 5" + date="2015-04-17T22:15:07Z" + content=""" +I am experiencing the same thing (and have experienced it 1.5 years ago as well, right before deciding to never again use git-annex). Enough info or not is it simply unacceptable for your software to do this and you should look into it as best you can. Try to repeat the steps which he described. I for one, am getting much the same results. +"""]] diff --git a/doc/bugs/git-annex_happily_deleted_most_of_my_files___36____35____38____33__/comment_6_f36c0ab40bac68445fa0f205661c9f51._comment b/doc/bugs/git-annex_happily_deleted_most_of_my_files___36____35____38____33__/comment_6_f36c0ab40bac68445fa0f205661c9f51._comment new file mode 100644 index 000000000..d794c23f9 --- /dev/null +++ b/doc/bugs/git-annex_happily_deleted_most_of_my_files___36____35____38____33__/comment_6_f36c0ab40bac68445fa0f205661c9f51._comment @@ -0,0 +1,16 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 6""" + date="2015-04-18T18:52:20Z" + content=""" +It's approximately 1000 times easier for someone who is experiencing a +problem to paste a tracript into this bug report than it is for me to guess +what someone might be doing and reproduce their bug (or problem, or mistake +or oops, or whatever) from basically no information. + +This is why I ask for bugs to be filed with either transcripts, or the +steps that can be used to reproduce them. + +I think you'll find that if you post enough information for a bug to be +reproduced, it'll get fixed quite fast. +"""]] diff --git a/doc/bugs/git_annex_assistant_--stop_fails_if_not_run_from_inside_a_git_annex_repository.mdwn b/doc/bugs/git_annex_assistant_--stop_fails_if_not_run_from_inside_a_git_annex_repository.mdwn new file mode 100644 index 000000000..61a01d73f --- /dev/null +++ b/doc/bugs/git_annex_assistant_--stop_fails_if_not_run_from_inside_a_git_annex_repository.mdwn @@ -0,0 +1,21 @@ +I set up a systemd unit to start the assistant, which works fine (it uses --autostart). When it comes time to stop the unit, however, it calls git annex assistant --stop, which fails because the current directory doesn't have a git annex repository in it. + + git-annex version: 5.20140717 + build flags: Assistant Inotify DBus Quvi TDFA + key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SHA256 SHA1 SHA512 SHA224 SHA384 WORM URL + remote types: git gcrypt bup directory rsync web tahoe glacier ddar hook external + +Here's my unit file: +[[!format ini """ +[Service] +ExecStart=/usr/bin/git-annex assistant --autostart +ExecStop=/usr/bin/git-annex assistant --stop + +[Install] +WantedBy=default.target +"""]] + +> This seems to overlap with [[todo/server-level_daemon__63__/]] in some way... --[[anarcat]] + +> Added --autostop and improved the docuemntation for --stop. [[done]] +> --[[Joey]] diff --git a/doc/bugs/git_annex_assistant_--stop_fails_if_not_run_from_inside_a_git_annex_repository/comment_1_a4cb3f759a1b076851c92fa8f9cca1be._comment b/doc/bugs/git_annex_assistant_--stop_fails_if_not_run_from_inside_a_git_annex_repository/comment_1_a4cb3f759a1b076851c92fa8f9cca1be._comment new file mode 100644 index 000000000..24f24455f --- /dev/null +++ b/doc/bugs/git_annex_assistant_--stop_fails_if_not_run_from_inside_a_git_annex_repository/comment_1_a4cb3f759a1b076851c92fa8f9cca1be._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2015-04-29T15:22:57Z" + content=""" +Well, --stop is intended to stop the daemon in the current git repository, +not all of them. That would be --autostop or something not implemented. + +I don't see how this is a bug. Feature request at best. +"""]] diff --git a/doc/bugs/git_annex_assistant_--stop_fails_if_not_run_from_inside_a_git_annex_repository/comment_2_953e6d1262728da6d30f4d3f1dd8fc29._comment b/doc/bugs/git_annex_assistant_--stop_fails_if_not_run_from_inside_a_git_annex_repository/comment_2_953e6d1262728da6d30f4d3f1dd8fc29._comment new file mode 100644 index 000000000..a58c1094b --- /dev/null +++ b/doc/bugs/git_annex_assistant_--stop_fails_if_not_run_from_inside_a_git_annex_repository/comment_2_953e6d1262728da6d30f4d3f1dd8fc29._comment @@ -0,0 +1,11 @@ +[[!comment format=mdwn + username="db48x" + subject="comment 2" + date="2015-04-29T22:56:47Z" + content=""" +Well, the usage just says + + --stop stop daemon + +so naturally I assumed that this is how you stop the daemon. It's cool though; systemd can just send it a SIGKILL or whatever. +"""]] diff --git a/doc/bugs/git_annex_import:_ignored_names_fatal.mdwn b/doc/bugs/git_annex_import:_ignored_names_fatal.mdwn index e96ca2acb..8981cadd8 100644 --- a/doc/bugs/git_annex_import:_ignored_names_fatal.mdwn +++ b/doc/bugs/git_annex_import:_ignored_names_fatal.mdwn @@ -36,3 +36,6 @@ Can't the ignored files just be ignored? # End of transcript or log. """]] + +> Made git-annex import check for gitignored files before +> moving them into the work tree. [[done]] --[[Joey]] diff --git a/doc/bugs/git_annex_import:_ignored_names_fatal/comment_2_99f575f424ec2f0d739fa7296f0d0086._comment b/doc/bugs/git_annex_import:_ignored_names_fatal/comment_2_99f575f424ec2f0d739fa7296f0d0086._comment new file mode 100644 index 000000000..0806cdef0 --- /dev/null +++ b/doc/bugs/git_annex_import:_ignored_names_fatal/comment_2_99f575f424ec2f0d739fa7296f0d0086._comment @@ -0,0 +1,11 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 2""" + date="2015-04-29T17:35:22Z" + content=""" +I cannot reproduce it repeating; the file is actually moved out of the +import location, so importing again won't do anything with it. You can +`git annex add --force` the file to bypass the .gitignore. + +`git annex import --force` also bypasses the problem entirely. +"""]] diff --git a/doc/bugs/git_annex_importfeed_not_working_on_youtube_playlists.mdwn b/doc/bugs/git_annex_importfeed_not_working_on_youtube_playlists.mdwn new file mode 100644 index 000000000..2306642fc --- /dev/null +++ b/doc/bugs/git_annex_importfeed_not_working_on_youtube_playlists.mdwn @@ -0,0 +1,39 @@ +### Please describe the problem. +The [podcasts page](https://git-annex.branchable.com/tips/downloading_podcasts/) mentions that you should be able to use `git annex importfeed` on youtube playlists, however, that doesn't seem to work (or I am using it incorrectly). + +For example: + + » git annex importfeed --fast "https://www.youtube.com/playlist?list=PLz8YL4HVC87X3tYVYOjOLzSasPwgUsMPT" + (checking known urls...) + importfeed https://www.youtube.com/playlist?list=PLz8YL4HVC87X3tYVYOjOLzSasPwgUsMPT + /var/folders/kb/fplmcbhs4z52p4x59cqgm4kw0000gn/T/f [ <=> ] 460.08K 1.87MB/s in 0.2s + + warning: bad feed content + ok + +### What steps will reproduce the problem? + +1. Enter a git annex repository +2. Attempt to import a youtube playlist feed (see command above) with `--fast` + +### What version of git-annex are you using? On what operating system? + + » git annex version + git-annex version: 5.20150205 + build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3 WebDAV FsEvents XMPP DNS Feeds Quvi TDFA TorrentParser + key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E MD5E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 MD5 WORM URL + remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external + local repository version: 5 + supported repository version: 5 + upgrade supported from repository versions: 0 1 2 4 + +This is on Mac OSX 10.10.3, with git-annex installed via homebrew + +> You have to give it an url to a RSS feed containing the playlist. +> +> For example, `git annex importfeed +> 'http://gdata.youtube.com/feeds/api/playlists/PLz8ZG1e9MPlzefklz1Gv79icjywTXycR-'` +> still works despite there having been various news stories about youtube +> removing RSS functionality. +> +> I've added this example to the documentation. [[done]] --[[Joey]] diff --git a/doc/bugs/git_annex_list__47__whereis_and_uncommited_local_changes.mdwn b/doc/bugs/git_annex_list__47__whereis_and_uncommited_local_changes.mdwn index 691591519..cf7ce7e5e 100644 --- a/doc/bugs/git_annex_list__47__whereis_and_uncommited_local_changes.mdwn +++ b/doc/bugs/git_annex_list__47__whereis_and_uncommited_local_changes.mdwn @@ -28,3 +28,6 @@ git-annex 5.20140421 Linux 3.14.3 [[!tag confirmed]] + +> I think it's reasonable for whereis to show location tracking information +> which may be out of date for many reasons, and so, [[done]] --[[Joey]] diff --git a/doc/bugs/local_pair_fails_if_non-ascii_characters_present_on_annex_path/comment_2_89f25f787558c77201fa6226cc7af5f5._comment b/doc/bugs/local_pair_fails_if_non-ascii_characters_present_on_annex_path/comment_2_89f25f787558c77201fa6226cc7af5f5._comment new file mode 100644 index 000000000..d922d1722 --- /dev/null +++ b/doc/bugs/local_pair_fails_if_non-ascii_characters_present_on_annex_path/comment_2_89f25f787558c77201fa6226cc7af5f5._comment @@ -0,0 +1,14 @@ +[[!comment format=mdwn + username="http://xgm.de/oid/" + nickname="Horus" + subject="comment 2" + date="2015-05-05T17:35:37Z" + content=""" +I have the same problem without any special characters in my path (besides ~ of $HOME) +Debug Log gives: + + illegal control characters in pairing message; ignoring + [2015-05-05 19:11:40 CEST] PairListener: received \"PairMsg (Verifiable {verifiableVal = (PairReq,PairData {remoteHostName = Just \\"asaru\\", remoteUserName = \\"florian\\", remoteDirectory = \\"~/Desktop/annex\\", remoteSshPubKey = \\"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDE8cd+cThzgRD+9RuiFhbL6UbPP+gvyNcUdrwVZoqfn2AE0niOe6XwsvqNrL4BZE50ySIo71XHyyAtRPiW3h0R8NjJo8+VFha2KL9vCXySNjq0Ib6HinfCDNUp5hI35F+LnUtUAVkhhhVqfJj4C6K3JTjXQ9J/hgiYRpNCY+2hV0+sF/e643SsyNlkUhiNxfCd4LQ5bedX6FeSCYBwteVgtZQzyByeawFpj1uajqBbDgDBLmclXDNrb4DwqavLRj+L+XxPtNqSKXSp8Q2/oypr/GQeTjmHEb8K/7qSjNHcBDAHH9fUI5lhDDhyxc4lMfap0lseSWtlwldhKjGqPnB9 florian@asaru\\n\\", pairUUID = UUID \\"0b1e8007-4a8d-4cc4-9ca1-320f4f700081\\"},IPv4Addr 347252928), verifiableDigest = \\"dff64a76c0333223cc3909f13bbdb3e1a70ddaa4\\"})\" + +I'll be very happy to provide any other help... +"""]] diff --git a/doc/bugs/man_page_has_example_lines_joined_into_a_single_line.mdwn b/doc/bugs/man_page_has_example_lines_joined_into_a_single_line.mdwn new file mode 100644 index 000000000..397666a2c --- /dev/null +++ b/doc/bugs/man_page_has_example_lines_joined_into_a_single_line.mdwn @@ -0,0 +1,18 @@ + +[[!format sh """ +# If you can, paste a complete transcript of the problem occurring here. +# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log +$> git annex help schedule | grep -A3 examples +man: can't set the locale; make sure $LC_* and $LANG are correct + Some examples: + + fsck self 30m every day at any time fsck self 1h every month at 3 AM fsck self 1h on day 1 of every month at any time fsck self 1h every week divisible by 2 at any time + +$> git annex version +git-annex version: 5.20150409+git126-ga29f683-1~ndall+1 +# about to build a fresh standlone + +# End of transcript or log. +"""]] + +> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/regression:_behavior_when_files_to_add_do_not_exist.mdwn b/doc/bugs/regression:_behavior_when_files_to_add_do_not_exist.mdwn new file mode 100644 index 000000000..9421dc66d --- /dev/null +++ b/doc/bugs/regression:_behavior_when_files_to_add_do_not_exist.mdwn @@ -0,0 +1,103 @@ +### Please describe the problem. + +When adding a list of files, where some exist and some don't, annex claims to add some of the files until it encounters the first missing file. Then it bails out, leaving files hashed but not added. + +### What steps will reproduce the problem? + +Create a file, ask annex to add the file and a non-existant file + +Expected and historic behavior: annex adds the file + +Actual behavior: annex hashes but does not add the file + +### What version of git-annex are you using? On what operating system? + +git-annex version: 5.20150420-gb0ebb23 +standalone linux amd64 + +### Please provide any additional information below. + +[[!format sh """ +# If you can, paste a complete transcript of the problem occurring here. +# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log + +$ git annex version +git-annex version: 5.20150420-gb0ebb23 +[ . . . ] + +$ git init asdf +Initialized empty Git repository in /tmp/asdf/.git/ + +$ cd asdf + +$ git annex init +init ok +(recording state in git...) + +$ touch asdf + +$ git add asdf qwer +fatal: pathspec 'qwer' did not match any files + +$ git annex add asdf qwer +add asdf ok +git-annex: qwer not found + +$ file * | sed -e 's/`.*//' +asdf: symbolic link to + +$ git status +On branch master + +Initial commit + +Untracked files: + (use "git add <file>..." to include in what will be committed) + + asdf + +nothing added to commit but untracked files present (use "git add" to track) + + +# End of transcript or log. +"""]] + +Older version of git-annex: + +[[!format sh """ + +$ git annex version +git-annex version: 5.20140412ubuntu1 +[ . . . ] + +$ git init asdf +Initialized empty Git repository in /tmp/asdf/.git/ + +$ cd asdf + +$ git annex init asdf +init asdf ok +(Recording state in git...) + +$ touch asdf + +$ git annex add asdf qwer +add asdf ok +git-annex: qwer not found +(Recording state in git...) + +$ file * | sed -e 's/`.*//' +asdf: symbolic link to + +$ git status +On branch master + +Initial commit + +Changes to be committed: + (use "git rm --cached <file>..." to unstage) + + new file: asdf +"""]] + +> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/regression:_behavior_when_files_to_add_do_not_exist/comment_1_3e444d500071779bcbfbc781b4756daf._comment b/doc/bugs/regression:_behavior_when_files_to_add_do_not_exist/comment_1_3e444d500071779bcbfbc781b4756daf._comment new file mode 100644 index 000000000..6fc7f84db --- /dev/null +++ b/doc/bugs/regression:_behavior_when_files_to_add_do_not_exist/comment_1_3e444d500071779bcbfbc781b4756daf._comment @@ -0,0 +1,30 @@ +[[!comment format=mdwn + username="clacke" + subject="Inconsistent between batches of files" + date="2015-04-22T12:56:42Z" + content=""" +If you add enough files, annex gets past the first `(Recording state in git...)` and then breaks on only the last portion, so some files are added and some are only hashed: + +[[!format sh \"\"\" +$ touch {10000..20240} 20242 + +$ git annex add {10000..20242} +[ . . . ] +add 20240 ok +(recording state in git...) +add 20242 ok +git-annex: 20241 not found + +$ file 20240 20242 | sed -e 's/`.*//' +20240: symbolic link to +20242: symbolic link to + +$ git status | tail -n 7 + new file: 20240 + +Untracked files: + (use \"git add <file>...\" to include in what will be committed) + + 20242 +\"\"\"]] +"""]] diff --git a/doc/bugs/regression:_behavior_when_files_to_add_do_not_exist/comment_2_4e2bd0704377c9bdc9fdb06a078fbc62._comment b/doc/bugs/regression:_behavior_when_files_to_add_do_not_exist/comment_2_4e2bd0704377c9bdc9fdb06a078fbc62._comment new file mode 100644 index 000000000..3e77d3a39 --- /dev/null +++ b/doc/bugs/regression:_behavior_when_files_to_add_do_not_exist/comment_2_4e2bd0704377c9bdc9fdb06a078fbc62._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 2""" + date="2015-04-22T20:27:29Z" + content=""" +This behavior could stand to be improved, but it's easy to deal with: Just +git annex add the files again, listing only the ones that actually exist, +and it will proceed where it was interrupted by the error. +"""]] diff --git a/doc/bugs/regression:_behavior_when_files_to_add_do_not_exist/comment_3_9ea62129cc568b07994b7e3d517a100c._comment b/doc/bugs/regression:_behavior_when_files_to_add_do_not_exist/comment_3_9ea62129cc568b07994b7e3d517a100c._comment new file mode 100644 index 000000000..20de644e9 --- /dev/null +++ b/doc/bugs/regression:_behavior_when_files_to_add_do_not_exist/comment_3_9ea62129cc568b07994b7e3d517a100c._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 3""" + date="2015-04-29T18:03:17Z" + content=""" +Also, I'm not so sure it's a regression, at least back at version +5.20141125 it behaved the same way. +"""]] diff --git a/doc/bugs/regression:_behavior_when_files_to_add_do_not_exist/comment_4_42481bb2f6f625a9891e59ec97574164._comment b/doc/bugs/regression:_behavior_when_files_to_add_do_not_exist/comment_4_42481bb2f6f625a9891e59ec97574164._comment new file mode 100644 index 000000000..4755498c1 --- /dev/null +++ b/doc/bugs/regression:_behavior_when_files_to_add_do_not_exist/comment_4_42481bb2f6f625a9891e59ec97574164._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 4""" + date="2015-04-30T18:56:31Z" + content=""" +Regression or not, it would be useful if git-annex continued past such +not-existing files to process the rest of the specified files, and only +set the error flag. +"""]] diff --git a/doc/bugs/rsync_on_windows_broken_by_upgrade.mdwn b/doc/bugs/rsync_on_windows_broken_by_upgrade.mdwn new file mode 100644 index 000000000..929770ccc --- /dev/null +++ b/doc/bugs/rsync_on_windows_broken_by_upgrade.mdwn @@ -0,0 +1,69 @@ +rsync on windows has been broken by the upgrade to a newer version of +cygwin. `rsync user@host:file file` opens the ssh connection, but hangs +up with a protocol error. Apparently it doesn't get even the protocol +version message from the server. + +Problem doesn't seem to affect the bundled ssh, just rsync. --[[Joey]] + +> Update: Apparently there are two ssh's! msysgit bundles one (did it used +> to in PATH?) and git-annex bundles one from cygwin. msysgit's ends up +> in Git/bin and git-annex's in Git/cmd. +> +> Seems that cygwin's rsync cannot use git's ssh for whatever reason. +> +> So the workaround is to +> delete Git/bin/ssh.exe and leave Git/cmd/ssh.exe. Then rsync works. +> However, this may screw up git's use of ssh or other stuff. +> +> Particularly, cygwin's ssh doesn't honor HOME anymore, instead using +> the getpwent home, which doesn't exist. +> +> Also, see +> [[webapp_fails_to_connect_to_ssh_repository___40__windows__41__]] +> which is the inverse of this bug perhaps, or at least seems related. +> +> Using 2 ssh's that try to use config from different places seems like +> a losing propisition. Need to find an rsync that works with git's ssh. +> --[[Joey]] +> +> > Update: The git bin/ directory is only in PATH when inside "git bash". +> > This bug only seems to affect using git-annex that way. The git bash +> > PATH has `bin` before `cmd`. +> > +> > Also, git seems to work ok using the newer ssh from cygwin. +> > However, that ssh tries to write to a .ssh/known_hosts +> > in a cygwin location and so doesn't remember hosts. +> > +> > What a mess. You can install any version of Linux and get rsync, ssh, +> > git that all integrate and work together. Or you can use Windows and +> > enjoy the pain(TM) --[[Joey]] + +>>> Possible fixes: +>>> +>>> * Roll the bundled ssh and rsync back to the older versions. +>>> +>>> **This works**. And, seems that the older version of ssh from cygwin +>>> looks at HOME, rather than getpwent home which the newer +>>> cygwin ssh does. +>>> +>>> * Roll the bundled rsync back, drop ssh. Rely on msysgit's bundled ssh, +>>> copying it into cmd so it's in PATH. Check: Does this combo work? +>>> +>>> **This works**! rsync 3.0.9 works ok with msysgit's bundled ssh. +>>> rsync 3.1.1 is the one that needs a newer ssh. **[[done]]** +>>> +>>> Note that this means we're using an old version of rsync +>>> from cygwin with libraries from a newer cygwin. That might prove +>>> fragile as cygwin is upgraded. +>>> +>>> * Hope that msysgit gets updated to include a newer version of ssh +>>> which works with the new rsync. +>>> +>>> (Seems reasonable as a long-term plan, assuming that the +>>> new rsync's problem with ssh is that it needs a new one, and not some +>>> special cygwin thing.) +>>> +>>> * Get rsync from somewhere else, perhaps msysgit. (Maybe also get ssh +>>> from msysgit?) +>>> * Keep the new rsync from cygwin, and build ssh from source, +>>> patching it to use HOME in preference to getpwent home. diff --git a/doc/bugs/rsync_on_windows_broken_by_upgrade/comment_1_de67cd10dd73fdc4d43eeb7ffbe67568._comment b/doc/bugs/rsync_on_windows_broken_by_upgrade/comment_1_de67cd10dd73fdc4d43eeb7ffbe67568._comment new file mode 100644 index 000000000..85c5e740f --- /dev/null +++ b/doc/bugs/rsync_on_windows_broken_by_upgrade/comment_1_de67cd10dd73fdc4d43eeb7ffbe67568._comment @@ -0,0 +1,7 @@ +[[!comment format=mdwn + username="anton" + subject="Thanks!" + date="2015-05-08T07:11:34Z" + content=""" +Thank you Joey for the great support and all your work on git-annex in general! +"""]] diff --git a/doc/bugs/ssh_fails_in_windows_nightly_build.mdwn b/doc/bugs/ssh_fails_in_windows_nightly_build.mdwn new file mode 100644 index 000000000..a40e3ea71 --- /dev/null +++ b/doc/bugs/ssh_fails_in_windows_nightly_build.mdwn @@ -0,0 +1,10 @@ +### Please describe the problem. + +After installing a nightly build from https://qa.nest-initiative.org/view/msysGit/job/msysgit-git-annex-assistant-test/, ssh fails to run when launched by the webapp. It shows a system error dialog with the message "The program can't start because msys-crypto-1.0.0.dll is missing...". The dll is present in $PROGRAMFILES/Git/bin, and ssh works when run from the command line. + +[[!format sh """ +Version: 5.20150508-g8e96a31 +Build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3 WebDAV DNS Feeds Quvi TDFA TorrentParser +"""]] + +> sigh.. [[fixed|done]] now.. --[[Joey]] diff --git a/doc/bugs/ssh_portnum_bugs.mdwn b/doc/bugs/ssh_portnum_bugs.mdwn index 4f24a9945..0d7199d8e 100644 --- a/doc/bugs/ssh_portnum_bugs.mdwn +++ b/doc/bugs/ssh_portnum_bugs.mdwn @@ -13,3 +13,6 @@ Change Port 22 in /etc/ssh/sshd_config to Port 9999, restart ssh on both compute When I was experiencing this issue, I was running the default on Jessie/Wheezy. Now I'm running the latest (via auto-update and distributed binary) but don't know if this is still an issue with latest versions (I switched to 22 as a workaround). [[!tag moreinfo]] + +> Closing as it seems likely this is an old bug fixed after wheezy. +> [[done]] --[[Joey]] diff --git a/doc/bugs/ssh_portnum_bugs/comment_6_5360e3c8db03402d2b7d81c0bbe8b35e._comment b/doc/bugs/ssh_portnum_bugs/comment_6_5360e3c8db03402d2b7d81c0bbe8b35e._comment new file mode 100644 index 000000000..cfe43137c --- /dev/null +++ b/doc/bugs/ssh_portnum_bugs/comment_6_5360e3c8db03402d2b7d81c0bbe8b35e._comment @@ -0,0 +1,7 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 6""" + date="2015-05-01T18:21:09Z" + content=""" +Any chance of a followup? +"""]] diff --git a/doc/bugs/ssh_portnum_bugs/comment_7_4157337c26a4267323da78dc7609a834._comment b/doc/bugs/ssh_portnum_bugs/comment_7_4157337c26a4267323da78dc7609a834._comment new file mode 100644 index 000000000..2657443a4 --- /dev/null +++ b/doc/bugs/ssh_portnum_bugs/comment_7_4157337c26a4267323da78dc7609a834._comment @@ -0,0 +1,7 @@ +[[!comment format=mdwn + username="jamieson" + subject="Follow up" + date="2015-05-01T20:40:49Z" + content=""" +Sorry about that! please close and I'll follow up if I have any more issues. Thanks!! +"""]] diff --git a/doc/bugs/webapp_fails_to_connect_to_ssh_repository___40__windows__41__.mdwn b/doc/bugs/webapp_fails_to_connect_to_ssh_repository___40__windows__41__.mdwn new file mode 100644 index 000000000..83fec53ab --- /dev/null +++ b/doc/bugs/webapp_fails_to_connect_to_ssh_repository___40__windows__41__.mdwn @@ -0,0 +1,40 @@ +### Please describe the problem. + +I added a remote repository, and it successfully tested the ssh connection to the server. Nothing happens, however, once it comes to actually creating the remote repository (or merging with an existing one). It'll just sit there forever, never actually connecting to the server. + +A quick look in process explorer shows something of what's going on: git-annex has launched ssh, and ssh is spamming subprocesses. It's launching ssh.exe which is launching git-annex.exe (yes, on the local machine.) It exits right away, but the command line is "C:\Program Files (x86)\Git\cmd\git-annex.exe" "Please type 'yes' or 'no': ". I've no idea why it's doing that though. + +If I kill that parent ssh process, I get this error message in the transcript: + + Could not create directory '/home/db48x/.ssh'. + +This seems a bit dubious; both my local computer and the remote computer have a ~/.ssh directory. + +In order to rule out a problem with my local computer (an ancient install of Git or cygwin/msys or something, we've tried this on fresh computers which have never had git installed; we get exactly the same problem there. + +### What steps will reproduce the problem? + +Create or connect to a repository via SSH. + +### What version of git-annex are you using? On what operating system? + +Windows 7 + + Version: 5.20150420-gb0ebb23 + Build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3 WebDAV DNS Feeds Quvi TDFA TorrentParser + +### Please provide any additional information below. + +While this is going on, the log has is showing that it's executing the following command: + +[[!format sh """ +[2015-04-28 22:34:16 Pacific Daylight Time] chat: ssh ["-oNumberOfPasswordPrompts=1","-p","22","db48x@eambar.db48x.net","sh -c 'mkdir -p '\"'\"'annex'\"'\"'&&cd '\"'\"'annex'\"'\"'&&if [ ! -d .git ]; then if [ -x ~/.ssh/git-annex-wrapper ]; then ~/.ssh/git-annex-wrapper git init --bare --shared; else git init --bare --shared; fi && if [ -x ~/.ssh/git-annex-wrapper ]; then ~/.ssh/git-annex-wrapper git config receive.denyNonFastforwards; else git config receive.denyNonFastforwards; fi ;fi&&mkdir -p ~/.ssh&&if [ ! -e ~/.ssh/git-annex-shell ]; then (echo '\"'\"'#!/bin/sh'\"'\"';echo '\"'\"'set -e'\"'\"';echo '\"'\"'if [ \"x$SSH_ORIGINAL_COMMAND\" != \"x\" ]; then'\"'\"';echo '\"'\"'exec git-annex-shell -c \"$SSH_ORIGINAL_COMMAND\"'\"'\"';echo '\"'\"'else'\"'\"';echo '\"'\"'exec git-annex-shell -c \"$@\"'\"'\"';echo '\"'\"'fi'\"'\"') > ~/.ssh/git-annex-shell; fi&&chmod 700 ~/.ssh/git-annex-shell&&touch ~/.ssh/authorized_keys&&chmod 600 ~/.ssh/authorized_keys&&echo '\"'\"'command=\"env GIT_ANNEX_SHELL_DIRECTORY='\"'\"'\"'\"'\"'\"'\"'\"'annex'\"'\"'\"'\"'\"'\"'\"'\"' ~/.ssh/git-annex-shell\",no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-pty ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDiPFdIMyYCBmc14f9cUZaG36Zw+NziqX9Z+xGl5GAYq16nORxVc+70Bj+A9cHoHLzTMBJnw7G/f7xJNGbKNgKUPKZohT8AQfg8lnyK8qpyzI2dJB3R0vPBMPxZCBm4IOpdB6ad3B6dUiyNtyMn1hza7GVhIFOsHfGG+K3PGtFgwOz/Zj+2zmcZIL/BHObFsba/yhQxbsjLYPI7mmNV9CLb1+XcR0z2okWvxu28lOrcIXDAdEhp1cjjjpBhwTH1F8/gJcJ6ENBa4JiGt/oEKb1b/pXLaMX6dRjc/gYoy7z0Hw7RD73hH+UtPj5TAeKwoNdaVXdqSsVI+3ql+O5PFTxt db48x@caradhras\n'\"'\"' >>~/.ssh/authorized_keys'"] +"""]] + +> This sounds like it's all down to the newer ssh from cygwin ignoring HOME +> and trying to use /home/user which doesn't work very well outside cygwin. +> +> Since git-annex has switched to not include that ssh any longer, and +> instead use the ssh that's bundled with msysgit, I think this bug is +> closed. [[done]] Upgrading to the latest daily build should fix your +> system's ssh. Please followup if I'm wrong. --[[Joey]] diff --git a/doc/bugs/webapp_fails_to_connect_to_ssh_repository___40__windows__41__/comment_1_c89c2fcd8d93bc64b80749b4207a3ebd._comment b/doc/bugs/webapp_fails_to_connect_to_ssh_repository___40__windows__41__/comment_1_c89c2fcd8d93bc64b80749b4207a3ebd._comment new file mode 100644 index 000000000..6db718bf2 --- /dev/null +++ b/doc/bugs/webapp_fails_to_connect_to_ssh_repository___40__windows__41__/comment_1_c89c2fcd8d93bc64b80749b4207a3ebd._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="anton" + subject="me too" + date="2015-05-07T09:50:09Z" + content=""" +I get similar results on Windows, but I only use the command line. For some reason git-annex ignores the ssh-agent settings (SSH_AUTH_SOCK=...) and uses the wrong path for the ssh config dir -- /home/username/.ssh (that probably doesn't exist) -- instead of c:/Users/username/.ssh (or whatever it really is). Your issue is probably that ssh wrongly looks for known_hosts in /home/username/.ssh and asks whether you wan't to accept the unknown host key. + +SSH works when ran by git itself (like git clone/push/fetch), also for rsync, but seemingly not git-annex. +"""]] diff --git a/doc/bugs/webapp_fails_to_connect_to_ssh_repository___40__windows__41__/comment_2_0562eb168f7a35262a24346030c4fa9f._comment b/doc/bugs/webapp_fails_to_connect_to_ssh_repository___40__windows__41__/comment_2_0562eb168f7a35262a24346030c4fa9f._comment new file mode 100644 index 000000000..9089921ec --- /dev/null +++ b/doc/bugs/webapp_fails_to_connect_to_ssh_repository___40__windows__41__/comment_2_0562eb168f7a35262a24346030c4fa9f._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 2""" + date="2015-05-07T18:26:25Z" + content=""" +See [[rsync_on_windows_broken_by_upgrade]] which is the inverse of this +bug. +"""]] diff --git a/doc/bugs/windows_ssh_webapp_password_entry_broken.mdwn b/doc/bugs/windows_ssh_webapp_password_entry_broken.mdwn new file mode 100644 index 000000000..e734e007b --- /dev/null +++ b/doc/bugs/windows_ssh_webapp_password_entry_broken.mdwn @@ -0,0 +1,5 @@ +Recent changes to ssh on Windows have broken the webapps's support for +entering a password when adding a ssh remote. + +Using ssh on windows with an existing remote does work. So as a workaround, +set up a passwordless ssh key that can log into the ssh server. --[[Joey]] diff --git a/doc/bugs/wrong_synopsis_in_manpage_of_git_annex_pre-commit.mdwn b/doc/bugs/wrong_synopsis_in_manpage_of_git_annex_pre-commit.mdwn new file mode 100644 index 000000000..f94786b03 --- /dev/null +++ b/doc/bugs/wrong_synopsis_in_manpage_of_git_annex_pre-commit.mdwn @@ -0,0 +1,36 @@ +### Please describe the problem. + +The synopsis in the manpage of git annex pre-commit mentions +git annex instead of git annex pre-commit. + +I'll attach a patch to fix the manpage, below: + +[[!format diff """ +From f5fed6ccdcef3bcb8be07691265842d437037dec Mon Sep 17 00:00:00 2001 +From: Felix Gruber <felgru@gmx.de> +Date: Fri, 1 May 2015 02:05:32 +0200 +Subject: [PATCH] fix synopsis in manpage of git annex pre-commit + +--- + doc/git-annex-pre-commit.mdwn | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/doc/git-annex-pre-commit.mdwn b/doc/git-annex-pre-commit.mdwn +index e0f6fdb..bc1e86e 100644 +--- a/doc/git-annex-pre-commit.mdwn ++++ b/doc/git-annex-pre-commit.mdwn +@@ -4,7 +4,7 @@ git-annex pre-commit - run by git pre-commit hook + + # SYNOPSIS + +-git annex `[path ...]` ++git annex pre-commit `[path ...]` + + # DESCRIPTION + +-- +2.1.4 +"""]] + +> You know, this is a wiki, you could fix it yourself. With git push even. +> In any case [[done]] now. --[[Joey]] |