diff options
author | Joey Hess <joey@kitenet.net> | 2013-12-15 13:37:12 -0400 |
---|---|---|
committer | Joey Hess <joey@kitenet.net> | 2013-12-15 13:37:12 -0400 |
commit | ad6ec99c4ce229e6f553566d20fbfd80f22b431f (patch) | |
tree | 81b68067128f556dc5f529a314ae579f24a23725 | |
parent | b3bf180933609c259fff2b3d3969284cbbc0050d (diff) | |
parent | a58a3707325e65de460bb384b431ba4b96692f14 (diff) |
Merge branch 'master' of ssh://git-annex.branchable.com
11 files changed, 277 insertions, 1 deletions
diff --git a/doc/bugs/Endless_SSH_password_prompts/comment_5_7490ca530d4e7d49eaa264eb5880dd17._comment b/doc/bugs/Endless_SSH_password_prompts/comment_5_7490ca530d4e7d49eaa264eb5880dd17._comment new file mode 100644 index 000000000..7990d0ff2 --- /dev/null +++ b/doc/bugs/Endless_SSH_password_prompts/comment_5_7490ca530d4e7d49eaa264eb5880dd17._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawkdiXn5s1wX2Zncy1S7_h-yPTv2gt0bmmY" + nickname="Jacob" + subject="Same problem on ubuntu" + date="2013-12-15T14:22:55Z" + content=""" +Same problem, I even manually setup ssh keys between the two servers but it just loops and keeps asking for a password every sync. +"""]] diff --git a/doc/bugs/OSX_build_broken.mdwn b/doc/bugs/OSX_build_broken.mdwn new file mode 100644 index 000000000..ac729b429 --- /dev/null +++ b/doc/bugs/OSX_build_broken.mdwn @@ -0,0 +1,11 @@ +### Please describe the problem. + +The OSX webapp doesn't start on the current OSX 10.7.5 builds. + +### What steps will reproduce the problem? + +Download git-annex from http://downloads.kitenet.net/git-annex/OSX/current/10.7.5_Lion/git-annex.dmg and install it by copying it to Application. Try to launch it then. The result is a message in the system log saying "git-annex unknown command webapp" and then the usage information. + +### What version of git-annex are you using? On what operating system? + +The current version as downloaded from http://downloads.kitenet.net/git-annex/OSX/current/10.7.5_Lion/git-annex.dmg on OSX 10.7.5 diff --git a/doc/bugs/Syncing_of_file_contents_seems_to_be_broken_in_recent_versions_of_the_assistant.mdwn b/doc/bugs/Syncing_of_file_contents_seems_to_be_broken_in_recent_versions_of_the_assistant.mdwn new file mode 100644 index 000000000..c14ad54f8 --- /dev/null +++ b/doc/bugs/Syncing_of_file_contents_seems_to_be_broken_in_recent_versions_of_the_assistant.mdwn @@ -0,0 +1,132 @@ +### Please describe the problem. + +Since I last tested git-annex (version 4.20130516.1) my setup seems to no longer work in current versions (5.20131130) + +### What steps will reproduce the problem? + +[[!format sh """ +$ mkdir 1 2 +$ cd 1; git init; git-annex init; git annex direct; echo "file added to 1" > file_from_1; cd .. +Initialized empty Git repository in /home/pedrocr/testannexbug/1/.git/ +init ok +(Recording state in git...) +commit +# On branch master +# +# Initial commit +# +nothing to commit (create/copy files and use "git add" to track) +ok +direct ok +$ cd 2; git init; git-annex init; git annex direct; echo "file added to 2" > file_from_2; cd .. +Initialized empty Git repository in /home/pedrocr/testannexbug/2/.git/ +init ok +(Recording state in git...) +commit +# On branch master +# +# Initial commit +# +nothing to commit (create/copy files and use "git add" to track) +ok +direct ok +$ cd 1; git remote add 2 ssh://localhost/~pedrocr/Hacks/test-git-annex/2; git annex assistant; cd .. +$ cd 2; git remote add 1 ssh://localhost/~pedrocr/Hacks/test-git-annex/1; git annex assistant; cd .. +$ ls -la 1 +total 20 +drwxrwxr-x 3 pedrocr pedrocr 4096 Dec 15 11:33 . +drwxrwxr-x 4 pedrocr pedrocr 4096 Dec 15 11:33 .. +-rw-rw-r-- 1 pedrocr pedrocr 16 Dec 15 11:33 file_from_1 +-rw-r--r-- 1 pedrocr pedrocr 16 Dec 15 11:33 file_from_2 +drwxrwxr-x 9 pedrocr pedrocr 4096 Dec 15 11:33 .git +$ ls -la 2 +total 20 +drwxrwxr-x 3 pedrocr pedrocr 4096 Dec 15 11:33 . +drwxrwxr-x 4 pedrocr pedrocr 4096 Dec 15 11:33 .. +-rw-r--r-- 1 pedrocr pedrocr 16 Dec 15 11:33 file_from_1 +-rw-rw-r-- 1 pedrocr pedrocr 16 Dec 15 11:33 file_from_2 +drwxrwxr-x 9 pedrocr pedrocr 4096 Dec 15 11:33 .git +"""]] + +so far so good + +[[!format sh """ +$ date | tee 1/file_from_1 +Sun Dec 15 11:53:18 WET 2013 +$ date | tee 2/file_from_2 +Sun Dec 15 11:53:22 WET 2013 +$ cat 1/* +Sun Dec 15 11:53:18 WET 2013 +file added to 2 +$ cat 2/* +file added to 1 +Sun Dec 15 11:53:22 WET 2013 +$ +"""]] + +But then syncing doesn't work. There doesn't seem to be anything special in the logs + +[[!format sh """ +$ tail -n 20 1/.git/annex/daemon.log +(Recording state in git...) +add file_from_1 (checksum...) [2013-12-15 11:53:18 WET] Committer: Committing changes to git +[2013-12-15 11:53:18 WET] Pusher: Syncing with 2 + +file_from_1 + 29 100% 0.00kB/s 0:00:00 (xfer#1, to-check=0/1) + +sent 104 bytes received 31 bytes 270.00 bytes/sec +total size is 29 speedup is 0.21 +[2013-12-15 11:53:18 WET] Transferrer: Uploaded file_from_1 +Already up-to-date. +To ssh://localhost/~pedrocr/Hacks/test-git-annex/2 + f025062..431ac37 git-annex -> synced/git-annex + d393bb3..fef60ca annex/direct/master -> synced/master +Already up-to-date. +[2013-12-15 11:53:21 WET] Pusher: Syncing with 2 +To ssh://localhost/~pedrocr/Hacks/test-git-annex/2 + 431ac37..3cc7045 git-annex -> synced/git-annex + + +$ tail -n 20 2/.git/annex/daemon.log +(Recording state in git...) +add file_from_2 (checksum...) [2013-12-15 11:53:22 WET] Committer: Committing changes to git +[2013-12-15 11:53:22 WET] Pusher: Syncing with 1 + +file_from_2 + 29 100% 0.00kB/s 0:00:00 (xfer#1, to-check=0/1) +Already up-to-date. + +sent 104 bytes received 31 bytes 270.00 bytes/sec +total size is 29 speedup is 0.21 +[2013-12-15 11:53:22 WET] Transferrer: Uploaded file_from_2 +To ssh://localhost/~pedrocr/Hacks/test-git-annex/1 + ebbcd4c..dae7a77 git-annex -> synced/git-annex + ffd5a90..67ad7c1 annex/direct/master -> synced/master +Already up-to-date. +[2013-12-15 11:53:25 WET] Pusher: Syncing with 1 +To ssh://localhost/~pedrocr/Hacks/test-git-annex/1 + dae7a77..976c899 git-annex -> synced/git-annex +"""]] + + + +### What version of git-annex are you using? On what operating system? + +I'm running git-annex from the Ubuntu PPA on Ubuntu 12.04 LTS. + +[[!format sh """ +$ lsb_release -v +No LSB modules are available. +$ lsb_release -a +No LSB modules are available. +Distributor ID: Ubuntu +Description: Ubuntu 12.04.3 LTS +Release: 12.04 +Codename: precise +$ git annex version +git-annex version: 5.20131130 +build flags: Assistant Webapp Pairing S3 WebDAV Inotify DBus XMPP Feeds Quvi TDFA +key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SHA256 SHA1 SHA512 SHA224 SHA384 WORM URL +remote types: git gcrypt S3 bup directory rsync web webdav glacier hook +"""]] diff --git a/doc/bugs/direct_mode_sync_should_avoid_git_commit/comment_1_d286800081f019651cd40355b7d5518a._comment b/doc/bugs/direct_mode_sync_should_avoid_git_commit/comment_1_d286800081f019651cd40355b7d5518a._comment new file mode 100644 index 000000000..98897593c --- /dev/null +++ b/doc/bugs/direct_mode_sync_should_avoid_git_commit/comment_1_d286800081f019651cd40355b7d5518a._comment @@ -0,0 +1,27 @@ +[[!comment format=mdwn + username="TroisSinges" + ip="82.227.207.5" + subject="Almost done" + date="2013-12-15T10:13:50Z" + content=""" +Hi! + +Thank you for the fix, it's much better now. However, in some case, it doesn't work. For example, when I try to add inject a file in the direct repository (rm symlinkedFile ; mv /tmp/myFile symlinkedFile ; git annex add symlinkedFile), it tries to map each file in memory (which takes a very long time through the network). + + # git annex add bigFile --debug + [2013-12-15 11:00:00 CET] read: git [\"--git-dir=/Volumes/Video/Annex/.git\",\"--work-tree=/Volumes/Video/Annex\",\"-c\",\"core.bare=false\",\"ls-files\",\"--others\",\"--exclude-standard\",\"-z\",\"--\",\"bigFile\"] + [2013-12-15 11:00:00 CET] read: git [\"--git-dir=/Volumes/Video/Annex/.git\",\"--work-tree=/Volumes/Video/Annex\",\"-c\",\"core.bare=false\",\"ls-files\",\"--modified\",\"-z\",\"--\",\"bigFile\"] + +If I use lsof, each another file in repository is mapped in memory: + + # lsof -p 30130 + git 30130 troissinges 3r REG 46,11 4519526640 3093837953332665982 /Volumes/Video/Annex/AnotherBigFile + +My version of software: + + git annex version + git-annex version: 5.20131213 + build flags: Assistant Webapp Pairing Testsuite S3 WebDAV FsEvents XMPP DNS Feeds Quvi TDFA CryptoHash + key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 WORM URL + remote types: git gcrypt S3 bup directory rsync web webdav glacier hook +"""]] diff --git a/doc/bugs/feature_request:_addhash.mdwn b/doc/bugs/feature_request:_addhash.mdwn new file mode 100644 index 000000000..fca74200e --- /dev/null +++ b/doc/bugs/feature_request:_addhash.mdwn @@ -0,0 +1,27 @@ +### Use case 1 + +I have a big repo using a SHA256E back-end. Along comes a new shiny SKEIN512E back-end and I would like to transition to using that, because it's faster and moves from ridiculously to ludicrously low risk of collisions. + +I can set `.gitattributes` to use the new back-end for any new files added, but then I when I import some arbitrary mix of existing and new files to the repo it will not deduplicate any more, it will add all the files under the new hash scheme. + +### Use case 2 + +I have a big repo of files I have added using `addurl --fast`. I download the files, and they are in the repo. + + - I cannot verify later that none of them have been damaged. + - If I come across an offline collection of some of the files, I cannot easily get them into the annex by a simple import. + +### Workaround + +In both these cases, what I can do is unlock (maybe?) or unannex (definitely) all of the files, and then re-add them using the new hash. In both cases this means I now risk having duplicates in various clones of the repo, and would have to clean them up with `drop-unused` -- after first having re-copied them from a repo that has them under the new hash. I also lose the continuity of the history of that object. + +In use case 2 I also lose the URLs of the files and would have to re-add them using `addurl`. + +... which brings me to the proposed feature. + +### addhash + +Symmetrical to `addurl`, which in one form can take an existing hashed (or URL-sourced) file and add an URL source to it, `addhash` can take an existing URL-sourced (or hashed) file and add a hash reference to it (given that the file is in the annex so that the hash may be calculated) -- an alias under which it may also be identified, in addition to the existing URL or hash. + + - Any file added to the annex after `addhash` will use the symlink name of the original hash if their hash matches the `addhash`ed one. + - An `fsck` run will use one of the available hashes to verify the integrity of the file, maybe according to some internal order of preference, or possibly a configurable one. diff --git a/doc/bugs/git_annex_get___60__file__62___should_verify_file_hash.mdwn b/doc/bugs/git_annex_get___60__file__62___should_verify_file_hash.mdwn new file mode 100644 index 000000000..f680729ac --- /dev/null +++ b/doc/bugs/git_annex_get___60__file__62___should_verify_file_hash.mdwn @@ -0,0 +1,32 @@ +### Please describe the problem. +git annex get fileName- should perform a hash check on the file content before adding to the local repository + + +### What steps will reproduce the problem? +Two scenarios: +1) Malicious user and owner of repository being pulled from can edit his/her local .git/annex/objects directory +to alter the file content. For src code, this could be to insert a bug, insert a backdoor, or for example +to replace an image file artifact for a website, with a pornographic image. +The user pulling the file content with "git annex get fileName" might not be aware of the file contents +until they actually examine the file or perform an fsck or commit locally. +In the meantime a kiddy porn image could be sitting in their repository or a src code backdoor can get incorporated and deployed etc. +2) a file could also simply get corrupted during download. An inherent hash check during the 'annex get' would +point out the problem immediately. +To reproduce: create repoNum1, and clone it to create repoNum2. manual edit/replace content in repoNum1/.git/annex/objects/... +then perform a 'git annex get <fileName>' from repoNum2 on the file that has been manipulated + + +### What version of git-annex are you using? On what operating system? +3.2012112ubuntu2 on running linux mint + + +### Please provide any additional information below. + +Aside: Thanks Joey - this is fantastic work you are doing. You have really improved git. The ability to checkout +an entire tree - but selectively get only the content actually needed is a real killer feature. +Kudos and again many many thanks +M. + + +# End of transcript or log. +"""]] diff --git a/doc/forum/Can_Not_Sync_to_Git_Repo/comment_8_36abd829ea71a44c7cded1123a7c913d._comment b/doc/forum/Can_Not_Sync_to_Git_Repo/comment_8_36abd829ea71a44c7cded1123a7c913d._comment new file mode 100644 index 000000000..66f75327e --- /dev/null +++ b/doc/forum/Can_Not_Sync_to_Git_Repo/comment_8_36abd829ea71a44c7cded1123a7c913d._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawniayrgSdVLUc3c6bf93VbO-_HT4hzxmyo" + nickname="Tobias" + subject="comment 8" + date="2013-12-14T13:15:13Z" + content=""" +The master branch was checkout out (I don't know why). Checking out the mentioned branch fixed the syncing issue: `git checkout annex/direct/master` +"""]] diff --git a/doc/forum/S3_Host_Question/comment_3_797edf3ad41561ab8960f3b28d20611e._comment b/doc/forum/S3_Host_Question/comment_3_797edf3ad41561ab8960f3b28d20611e._comment new file mode 100644 index 000000000..a0dbae5bd --- /dev/null +++ b/doc/forum/S3_Host_Question/comment_3_797edf3ad41561ab8960f3b28d20611e._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://me.yahoo.com/a/FHnTlSBo1eCGJRwueeKeB6.RCaPbGMPr5jxx8A--#ce0d8" + nickname="Hamza" + subject="comment 3" + date="2013-12-14T09:05:26Z" + content=""" +remote info is recorded in git once you clone the git repo it knows about machine a and s3 repo. +"""]] diff --git a/doc/forum/local_pairing_with_2_mac/comment_10_508585e72c81d197a9a1e193c25a702a._comment b/doc/forum/local_pairing_with_2_mac/comment_10_508585e72c81d197a9a1e193c25a702a._comment new file mode 100644 index 000000000..1d69f41fd --- /dev/null +++ b/doc/forum/local_pairing_with_2_mac/comment_10_508585e72c81d197a9a1e193c25a702a._comment @@ -0,0 +1,15 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawnfC02DTOZygFDDvvg0maRciYYMhEIFgkw" + nickname="Utku" + subject="comment 10" + date="2013-12-15T09:51:28Z" + content=""" +unfortunately, it is not working on the computer where ssh was not working +I click an got annex and nothing seems to open +I try to go on the system logs but I see nothing really obvious. The only log that may be related would be : +15/12/13 10:46:23,230 Dock[166]: find_shared_window: WID 21425 +15/12/13 10:46:23,230 Dock[166]: CGSGetWindowTags: Invalid window 0x53b1 + +if you tell me where to look at I can give you something more useful. + +"""]] diff --git a/doc/forum/local_pairing_with_2_mac/comment_9_f7a1ce9627ebfc854dfde2f6c924db80._comment b/doc/forum/local_pairing_with_2_mac/comment_9_f7a1ce9627ebfc854dfde2f6c924db80._comment new file mode 100644 index 000000000..deaf9b68c --- /dev/null +++ b/doc/forum/local_pairing_with_2_mac/comment_9_f7a1ce9627ebfc854dfde2f6c924db80._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.87" + subject="comment 9" + date="2013-12-13T18:52:40Z" + content=""" +As a workaround, I have removed ssh-keygen and ssh from the OSX DMG. AIUI, OSX ships ssh by default anyway. +"""]] diff --git a/doc/install/OSX.mdwn b/doc/install/OSX.mdwn index 1671476ae..784e42c83 100644 --- a/doc/install/OSX.mdwn +++ b/doc/install/OSX.mdwn @@ -34,7 +34,7 @@ the app for OSX Lion. <pre> brew update -brew install haskell-platform git ossp-uuid md5sha1sum coreutils libgsasl gnutls libidn libgsasl pkg-config libxml2 +brew install haskell-platform git ossp-uuid md5sha1sum coreutils gnutls libidn gsasl pkg-config libxml2 brew link libxml2 --force cabal update mkdir $HOME/bin |