### Please describe the problem. "git annex --pathdepth=3 addurl URL" displays this error message and reports "failed", even though the operation succeeds when using --fast: git-annex: /media/usb32g-1/annex/musikk/.git/annex/tmp/2325: rename: does not exist (No such file or directory) ### What steps will reproduce the problem? I have lots of downloaded files I'd like to add an URL to, so I'm doing this: $ ga addurl --fast --pathdepth=3 $(for f in *.mp3; do echo http://traffic.libsyn.com/geologicpodcast/$f; done) addurl git-annex: /media/usb32g-1/annex/musikk/.git/annex/tmp/2325: rename: does not exist (No such file or directory) failed addurl git-annex: /media/usb32g-1/annex/musikk/.git/annex/tmp/2325: rename: does not exist (No such file or directory) failed addurl git-annex: /media/usb32g-1/annex/musikk/.git/annex/tmp/2325: rename: does not exist (No such file or directory) [... Delete 166 lines ...] addurl git-annex: /media/usb32g-1/annex/musikk/.git/annex/tmp/2325: rename: does not exist (No such file or directory) failed addurl git-annex: /media/usb32g-1/annex/musikk/.git/annex/tmp/2325: rename: does not exist (No such file or directory) failed (Recording state in git...) git-annex: addurl: 60 failed $ This "ga" command is a wrapper around git-annex to make it work everywhere with the .tar.gz version: #!/bin/bash #======================================================================= # ga # File ID: e89047ce-29d1-11e2-bb6f-00c0a8deee11 #======================================================================= if test "$1" = "sync"; then stat_output="$(git status --porcelain | grep -v '^??')" test -n "$stat_output" && { echo ga: Repo has modifications, will not git-annex sync >&2; exit 1; } fi /opt/git-annex/runshell /opt/git-annex/git-annex "$@" $PATH is also set up to include /opt/git-annex/bin/ , and all other operations work nicely. In my ~/bin/ directory I've also symlinked this script to "git-annex". When dropping "--fast", this happens: $ ga addurl --pathdepth=3 $(for f in *.mp3; do echo http://traffic.libsyn.com/geologicpodcast/$f; done | tail -20 | head -1) addurl (downloading http://traffic.libsyn.com/geologicpodcast/GeologicPodcast302-Feb28-13.mp3 ...) --2013-07-04 14:08:01-- http://traffic.libsyn.com/geologicpodcast/GeologicPodcast302-Feb28-13.mp3 Resolving traffic.libsyn.com (traffic.libsyn.com)... 204.16.245.39 Connecting to traffic.libsyn.com (traffic.libsyn.com)|204.16.245.39|:80... connected. HTTP request sent, awaiting response... 302 Found Location: http://ec.libsyn.com/p/6/9/b/69b9837556ed5238/GeologicPodcast302-Feb28-13.mp3?d13a76d516d9dec20c3d276ce028ed5089ab1ce3dae902ea1d01cf853ed5cd5c0f85&c_id=5440885 [following] --2013-07-04 14:08:02-- http://ec.libsyn.com/p/6/9/b/69b9837556ed5238/GeologicPodcast302-Feb28-13.mp3?d13a76d516d9dec20c3d276ce028ed5089ab1ce3dae902ea1d01cf853ed5cd5c0f85&c_id=5440885 Resolving ec.libsyn.com (ec.libsyn.com)... 68.232.34.133 Connecting to ec.libsyn.com (ec.libsyn.com)|68.232.34.133|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 53173384 (51M) [audio/mpeg] Saving to: `/media/usb32g-1/annex/musikk/.git/annex/tmp/URL--http&c%%traffic.libsyn.com%geologicpodcast%GeologicPodcast302-Feb28-13.mp3' 100%[==================================================================================================================================================================>] 53,173,384 3.50M/s in 45s 2013-07-04 14:08:47 (1.13 MB/s) - `/media/usb32g-1/annex/musikk/.git/annex/tmp/URL--http&c%%traffic.libsyn.com%geologicpodcast%GeologicPodcast302-Feb28-13.mp3' saved [53173384/53173384] (checksum...) git-annex: /media/usb32g-1/annex/musikk/.git/annex/tmp/3970: rename: does not exist (No such file or directory) failed (Recording state in git...) git-annex: addurl: 1 failed $ git-annex now believes that the file is stored locally, but an fsck shows how things really are: $ ga fsck fsck GeologicPodcast02-Feb21-07.mp3 ok fsck GeologicPodcast03-Feb28-07.mp3 ok [... Delete 35 lines ...] fsck GeologicPodcast301-Feb22-13.mp3 ok fsck GeologicPodcast302-Feb28-13.mp3 (fixing location log) ** Based on the location log, GeologicPodcast302-Feb28-13.mp3 ** was expected to be present, but its content is missing. failed fsck GeologicPodcast303-Mar07-13.mp3 ok fsck GeologicPodcast303.1_Fancast6-Mar09-13.mp3 ok [... Delete 16 lines ...] fsck GeologicPodcast72-Jul03-08.mp3 ok (Recording state in git...) git-annex: fsck: 3 failed $ After this fsck, everything is OK again; the location log is correct and "get" from this URL works. The file data didn't exist locally (it's stored on another disk) when doing this, but I also experimented what would happen if the file data did exist: $ rm GeologicPodcast297-Jan24-13.mp3 $ wget http://traffic.libsyn.com/geologicpodcast/GeologicPodcast297-Jan24-13.mp3 --2013-07-04 14:35:15-- http://traffic.libsyn.com/geologicpodcast/GeologicPodcast297-Jan24-13.mp3 Resolving traffic.libsyn.com (traffic.libsyn.com)... 204.16.245.39 Connecting to traffic.libsyn.com (traffic.libsyn.com)|204.16.245.39|:80... connected. HTTP request sent, awaiting response... 302 Found Location: http://hwcdn.libsyn.com/p/9/d/f/9df13b1a6d622c46/GeologicPodcast297-Jan24-13.mp3?c_id=5341445&expiration=1372945406&hwt=476d03abba40d71119011bf7cb51f68a [following] --2013-07-04 14:35:16-- http://hwcdn.libsyn.com/p/9/d/f/9df13b1a6d622c46/GeologicPodcast297-Jan24-13.mp3?c_id=5341445&expiration=1372945406&hwt=476d03abba40d71119011bf7cb51f68a Resolving hwcdn.libsyn.com (hwcdn.libsyn.com)... 69.16.175.42, 69.16.175.10 Connecting to hwcdn.libsyn.com (hwcdn.libsyn.com)|69.16.175.42|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 49299974 (47M) [audio/mpeg] Saving to: ‘GeologicPodcast297-Jan24-13.mp3’ 100%[===========================================================================================================================================>] 49,299,974 5.88MB/s in 30s $ ga add add GeologicPodcast297-Jan24-13.mp3 (checksum...) ok (Recording state in git...) $ ga addurl --pathdepth=3 http://traffic.libsyn.com/geologicpodcast/GeologicPodcast297-Jan24-13.mp3 At this stage, the data for this .mp3 file exist locally, but git-annex is either unaware of it, or it insists on verifying the data. Anyhow, it downloads the file: $ ga addurl --pathdepth=3 http://traffic.libsyn.com/geologicpodcast/GeologicPodcast297-Jan24-13.mp3 addurl (downloading http://traffic.libsyn.com/geologicpodcast/GeologicPodcast297-Jan24-13.mp3 ...) --2013-07-04 14:42:40-- http://traffic.libsyn.com/geologicpodcast/GeologicPodcast297-Jan24-13.mp3 Resolving traffic.libsyn.com (traffic.libsyn.com)... 204.16.245.39 Connecting to traffic.libsyn.com (traffic.libsyn.com)|204.16.245.39|:80... connected. HTTP request sent, awaiting response... 302 Found Location: http://ec.libsyn.com/p/9/d/f/9df13b1a6d622c46/GeologicPodcast297-Jan24-13.mp3?d13a76d516d9dec20c3d276ce028ed5089ab1ce3dae902ea1d01cf853ed5cc54b707&c_id=5341445 [following] --2013-07-04 14:42:41-- http://ec.libsyn.com/p/9/d/f/9df13b1a6d622c46/GeologicPodcast297-Jan24-13.mp3?d13a76d516d9dec20c3d276ce028ed5089ab1ce3dae902ea1d01cf853ed5cc54b707&c_id=5341445 Resolving ec.libsyn.com (ec.libsyn.com)... 68.232.34.133 Connecting to ec.libsyn.com (ec.libsyn.com)|68.232.34.133|:80... connected. HTTP request sent, awaiting response... 206 Partial Content Length: 49299974 (47M), 48727654 (46M) remaining [audio/mpeg] Saving to: `/media/usb32g-1/annex/musikk/.git/annex/tmp/URL--http&c%%traffic.libsyn.com%geologicpodcast%GeologicPodcast297-Jan24-13.mp3' 100%[====================================================================================================================================================================>] 49,299,974 2.07M/s in 88s 2013-07-04 14:44:10 (538 KB/s) - `/media/usb32g-1/annex/musikk/.git/annex/tmp/URL--http&c%%traffic.libsyn.com%geologicpodcast%GeologicPodcast297-Jan24-13.mp3' saved [49299974/49299974] (checksum...) git-annex: /media/usb32g-1/annex/musikk/.git/annex/objects/gp/7Z/SHA256-s49299974--ff6758202d7168a0548b004d0c6e79a095b7738f442aa14d2603665914bee7e0/SHA256-s49299974--ff6758202d7168a0548b004d0c6e79a095b7738f442aa14d2603665914bee7e0: rename: does not exist (No such file or directory) failed (Recording state in git...) git-annex: addurl: 1 failed 2013-07-04 14:44:11 sunny@dg-vbox:/media/usb32g-1/annex/musikk/podcast/Geologic Podcast (master) $ The error message appears again, but because the data did exist from before, git-annex fsck has nothing to complain about: $ ga fsck [...] fsck GeologicPodcast297-Jan24-13.mp3 (checksum...) ok [...] 2013-07-04 14:44:33 sunny@dg-vbox:/media/usb32g-1/annex/musikk/podcast/Geologic Podcast (master) $ ### What version of git-annex are you using? On what operating system? v4.20130627 Linux i386 and AMD64. Both versions behave the same way. Tested on two computers: Computer #1 (AMD64 version): $ uname -a Linux dg-vbox 3.8.0-19-generic #30-Ubuntu SMP Wed May 1 16:35:23 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux $ lsb_release -a No LSB modules are available. Distributor ID: LinuxMint Description: Linux Mint 15 Olivia Release: 15 Codename: olivia Computer #2 (i386 version): $ uname -a Linux linode 2.6.39.1-linode34 #1 SMP Tue Jun 21 10:29:24 EDT 2011 i686 GNU/Linux $ lsb_release -a Distributor ID: Ubuntu Description: Ubuntu 10.04.4 LTS Release: 10.04 Codename: lucid No LSB modules are available. Also tested with v4.20130601, same thing happens. ### Please provide any additional information below. [[!format sh """ # If you can, paste a complete transcript of the problem occurring here. # If the problem is with the git-annex assistant, paste in .git/annex/daemon.log All interesting transcripts are posted above with cruft removed. # End of transcript or log. """]] > The cause of this bug is using --pathdepth=3 when the url > only has two components in its path. So, kinda GIGO, but > let's fix this anyway. ;) > > [[fixed|done]] --[[Joey]]