Add bug report about "git annex addurl"
This commit is contained in:
http://sunny256.sunbase.org/
admin
parent
d21aab2d3b
commit
3bdf00b0c0
1 changed files with 191 additions and 0 deletions
|
|
@ -0,0 +1,191 @@
|
|||
### Please describe the problem.
|
||||
|
||||
"git annex --pathdepth=3 addurl URL" displays this error message and reports "failed", even though the operation succeeds when using --fast:
|
||||
|
||||
git-annex: /media/usb32g-1/annex/musikk/.git/annex/tmp/2325: rename: does not exist (No such file or directory)
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
I have lots of downloaded files I'd like to add an URL to, so I'm doing this:
|
||||
|
||||
$ ga addurl --fast --pathdepth=3 $(for f in *.mp3; do echo http://traffic.libsyn.com/geologicpodcast/$f; done)
|
||||
addurl
|
||||
git-annex: /media/usb32g-1/annex/musikk/.git/annex/tmp/2325: rename: does not exist (No such file or directory)
|
||||
failed
|
||||
addurl
|
||||
git-annex: /media/usb32g-1/annex/musikk/.git/annex/tmp/2325: rename: does not exist (No such file or directory)
|
||||
failed
|
||||
addurl
|
||||
git-annex: /media/usb32g-1/annex/musikk/.git/annex/tmp/2325: rename: does not exist (No such file or directory)
|
||||
[... Delete 166 lines ...]
|
||||
addurl
|
||||
git-annex: /media/usb32g-1/annex/musikk/.git/annex/tmp/2325: rename: does not exist (No such file or directory)
|
||||
failed
|
||||
addurl
|
||||
git-annex: /media/usb32g-1/annex/musikk/.git/annex/tmp/2325: rename: does not exist (No such file or directory)
|
||||
failed
|
||||
(Recording state in git...)
|
||||
git-annex: addurl: 60 failed
|
||||
$
|
||||
|
||||
This "ga" command is a wrapper around git-annex to make it work everywhere with the .tar.gz version:
|
||||
|
||||
#!/bin/bash
|
||||
|
||||
#=======================================================================
|
||||
# ga
|
||||
# File ID: e89047ce-29d1-11e2-bb6f-00c0a8deee11
|
||||
#=======================================================================
|
||||
|
||||
if test "$1" = "sync"; then
|
||||
stat_output="$(git status --porcelain | grep -v '^??')"
|
||||
test -n "$stat_output" && { echo ga: Repo has modifications, will not git-annex sync >&2; exit 1; }
|
||||
fi
|
||||
/opt/git-annex/runshell /opt/git-annex/git-annex "$@"
|
||||
|
||||
$PATH is also set up to include /opt/git-annex/bin/ , and all other operations work nicely. In my ~/bin/ directory I've also symlinked this script to "git-annex".
|
||||
|
||||
When dropping "--fast", this happens:
|
||||
|
||||
$ ga addurl --pathdepth=3 $(for f in *.mp3; do echo http://traffic.libsyn.com/geologicpodcast/$f; done | tail -20 | head -1)
|
||||
addurl (downloading http://traffic.libsyn.com/geologicpodcast/GeologicPodcast302-Feb28-13.mp3 ...) --2013-07-04 14:08:01-- http://traffic.libsyn.com/geologicpodcast/GeologicPodcast302-Feb28-13.mp3
|
||||
Resolving traffic.libsyn.com (traffic.libsyn.com)... 204.16.245.39
|
||||
Connecting to traffic.libsyn.com (traffic.libsyn.com)|204.16.245.39|:80... connected.
|
||||
HTTP request sent, awaiting response... 302 Found
|
||||
Location: http://ec.libsyn.com/p/6/9/b/69b9837556ed5238/GeologicPodcast302-Feb28-13.mp3?d13a76d516d9dec20c3d276ce028ed5089ab1ce3dae902ea1d01cf853ed5cd5c0f85&c_id=5440885 [following]
|
||||
--2013-07-04 14:08:02-- http://ec.libsyn.com/p/6/9/b/69b9837556ed5238/GeologicPodcast302-Feb28-13.mp3?d13a76d516d9dec20c3d276ce028ed5089ab1ce3dae902ea1d01cf853ed5cd5c0f85&c_id=5440885
|
||||
Resolving ec.libsyn.com (ec.libsyn.com)... 68.232.34.133
|
||||
Connecting to ec.libsyn.com (ec.libsyn.com)|68.232.34.133|:80... connected.
|
||||
HTTP request sent, awaiting response... 200 OK
|
||||
Length: 53173384 (51M) [audio/mpeg]
|
||||
Saving to: `/media/usb32g-1/annex/musikk/.git/annex/tmp/URL--http&c%%traffic.libsyn.com%geologicpodcast%GeologicPodcast302-Feb28-13.mp3'
|
||||
|
||||
100%[==================================================================================================================================================================>] 53,173,384 3.50M/s in 45s
|
||||
|
||||
2013-07-04 14:08:47 (1.13 MB/s) - `/media/usb32g-1/annex/musikk/.git/annex/tmp/URL--http&c%%traffic.libsyn.com%geologicpodcast%GeologicPodcast302-Feb28-13.mp3' saved [53173384/53173384]
|
||||
|
||||
(checksum...)
|
||||
git-annex: /media/usb32g-1/annex/musikk/.git/annex/tmp/3970: rename: does not exist (No such file or directory)
|
||||
failed
|
||||
(Recording state in git...)
|
||||
git-annex: addurl: 1 failed
|
||||
$
|
||||
|
||||
git-annex now believes that the file is stored locally, but an fsck shows how things really are:
|
||||
|
||||
$ ga fsck
|
||||
fsck GeologicPodcast02-Feb21-07.mp3 ok
|
||||
fsck GeologicPodcast03-Feb28-07.mp3 ok
|
||||
[... Delete 35 lines ...]
|
||||
fsck GeologicPodcast301-Feb22-13.mp3 ok
|
||||
fsck GeologicPodcast302-Feb28-13.mp3 (fixing location log)
|
||||
** Based on the location log, GeologicPodcast302-Feb28-13.mp3
|
||||
** was expected to be present, but its content is missing.
|
||||
failed
|
||||
fsck GeologicPodcast303-Mar07-13.mp3 ok
|
||||
fsck GeologicPodcast303.1_Fancast6-Mar09-13.mp3 ok
|
||||
[... Delete 16 lines ...]
|
||||
fsck GeologicPodcast72-Jul03-08.mp3 ok
|
||||
(Recording state in git...)
|
||||
git-annex: fsck: 3 failed
|
||||
$
|
||||
|
||||
After this fsck, everything is OK again; the location log is correct and "get" from this URL works.
|
||||
|
||||
The file data didn't exist locally (it's stored on another disk) when doing this, but I also experimented what would happen if the file data did exist:
|
||||
|
||||
$ rm GeologicPodcast297-Jan24-13.mp3
|
||||
$ wget http://traffic.libsyn.com/geologicpodcast/GeologicPodcast297-Jan24-13.mp3
|
||||
--2013-07-04 14:35:15-- http://traffic.libsyn.com/geologicpodcast/GeologicPodcast297-Jan24-13.mp3
|
||||
Resolving traffic.libsyn.com (traffic.libsyn.com)... 204.16.245.39
|
||||
Connecting to traffic.libsyn.com (traffic.libsyn.com)|204.16.245.39|:80... connected.
|
||||
HTTP request sent, awaiting response... 302 Found
|
||||
Location: http://hwcdn.libsyn.com/p/9/d/f/9df13b1a6d622c46/GeologicPodcast297-Jan24-13.mp3?c_id=5341445&expiration=1372945406&hwt=476d03abba40d71119011bf7cb51f68a [following]
|
||||
--2013-07-04 14:35:16-- http://hwcdn.libsyn.com/p/9/d/f/9df13b1a6d622c46/GeologicPodcast297-Jan24-13.mp3?c_id=5341445&expiration=1372945406&hwt=476d03abba40d71119011bf7cb51f68a
|
||||
Resolving hwcdn.libsyn.com (hwcdn.libsyn.com)... 69.16.175.42, 69.16.175.10
|
||||
Connecting to hwcdn.libsyn.com (hwcdn.libsyn.com)|69.16.175.42|:80... connected.
|
||||
HTTP request sent, awaiting response... 200 OK
|
||||
Length: 49299974 (47M) [audio/mpeg]
|
||||
Saving to: ‘GeologicPodcast297-Jan24-13.mp3’
|
||||
|
||||
100%[===========================================================================================================================================>] 49,299,974 5.88MB/s in 30s
|
||||
|
||||
$ ga add
|
||||
add GeologicPodcast297-Jan24-13.mp3 (checksum...) ok
|
||||
(Recording state in git...)
|
||||
$ ga addurl --pathdepth=3 http://traffic.libsyn.com/geologicpodcast/GeologicPodcast297-Jan24-13.mp3
|
||||
|
||||
At this stage, the data for this .mp3 file exist locally, but git-annex is either unaware of it, or it insists on verifying the data. Anyhow, it downloads the file:
|
||||
|
||||
$ ga addurl --pathdepth=3 http://traffic.libsyn.com/geologicpodcast/GeologicPodcast297-Jan24-13.mp3
|
||||
addurl (downloading http://traffic.libsyn.com/geologicpodcast/GeologicPodcast297-Jan24-13.mp3 ...) --2013-07-04 14:42:40-- http://traffic.libsyn.com/geologicpodcast/GeologicPodcast297-Jan24-13.mp3
|
||||
Resolving traffic.libsyn.com (traffic.libsyn.com)... 204.16.245.39
|
||||
Connecting to traffic.libsyn.com (traffic.libsyn.com)|204.16.245.39|:80... connected.
|
||||
HTTP request sent, awaiting response... 302 Found
|
||||
Location: http://ec.libsyn.com/p/9/d/f/9df13b1a6d622c46/GeologicPodcast297-Jan24-13.mp3?d13a76d516d9dec20c3d276ce028ed5089ab1ce3dae902ea1d01cf853ed5cc54b707&c_id=5341445 [following]
|
||||
--2013-07-04 14:42:41-- http://ec.libsyn.com/p/9/d/f/9df13b1a6d622c46/GeologicPodcast297-Jan24-13.mp3?d13a76d516d9dec20c3d276ce028ed5089ab1ce3dae902ea1d01cf853ed5cc54b707&c_id=5341445
|
||||
Resolving ec.libsyn.com (ec.libsyn.com)... 68.232.34.133
|
||||
Connecting to ec.libsyn.com (ec.libsyn.com)|68.232.34.133|:80... connected.
|
||||
HTTP request sent, awaiting response... 206 Partial Content
|
||||
Length: 49299974 (47M), 48727654 (46M) remaining [audio/mpeg]
|
||||
Saving to: `/media/usb32g-1/annex/musikk/.git/annex/tmp/URL--http&c%%traffic.libsyn.com%geologicpodcast%GeologicPodcast297-Jan24-13.mp3'
|
||||
|
||||
100%[====================================================================================================================================================================>] 49,299,974 2.07M/s in 88s
|
||||
|
||||
2013-07-04 14:44:10 (538 KB/s) - `/media/usb32g-1/annex/musikk/.git/annex/tmp/URL--http&c%%traffic.libsyn.com%geologicpodcast%GeologicPodcast297-Jan24-13.mp3' saved [49299974/49299974]
|
||||
|
||||
(checksum...)
|
||||
git-annex: /media/usb32g-1/annex/musikk/.git/annex/objects/gp/7Z/SHA256-s49299974--ff6758202d7168a0548b004d0c6e79a095b7738f442aa14d2603665914bee7e0/SHA256-s49299974--ff6758202d7168a0548b004d0c6e79a095b7738f442aa14d2603665914bee7e0: rename: does not exist (No such file or directory)
|
||||
failed
|
||||
(Recording state in git...)
|
||||
git-annex: addurl: 1 failed
|
||||
2013-07-04 14:44:11 sunny@dg-vbox:/media/usb32g-1/annex/musikk/podcast/Geologic Podcast (master)
|
||||
$
|
||||
|
||||
The error message appears again, but because the data did exist from before, git-annex fsck has nothing to complain about:
|
||||
|
||||
$ ga fsck
|
||||
[...]
|
||||
fsck GeologicPodcast297-Jan24-13.mp3 (checksum...) ok
|
||||
[...]
|
||||
2013-07-04 14:44:33 sunny@dg-vbox:/media/usb32g-1/annex/musikk/podcast/Geologic Podcast (master)
|
||||
$
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
v4.20130627 Linux i386 and AMD64. Both versions behave the same way. Tested on two computers:
|
||||
|
||||
Computer #1 (AMD64 version):
|
||||
|
||||
$ uname -a
|
||||
Linux dg-vbox 3.8.0-19-generic #30-Ubuntu SMP Wed May 1 16:35:23 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
|
||||
$ lsb_release -a
|
||||
No LSB modules are available.
|
||||
Distributor ID: LinuxMint
|
||||
Description: Linux Mint 15 Olivia
|
||||
Release: 15
|
||||
Codename: olivia
|
||||
|
||||
Computer #2 (i386 version):
|
||||
|
||||
$ uname -a
|
||||
Linux linode 2.6.39.1-linode34 #1 SMP Tue Jun 21 10:29:24 EDT 2011 i686 GNU/Linux
|
||||
$ lsb_release -a
|
||||
Distributor ID: Ubuntu
|
||||
Description: Ubuntu 10.04.4 LTS
|
||||
Release: 10.04
|
||||
Codename: lucid
|
||||
No LSB modules are available.
|
||||
|
||||
Also tested with v4.20130601, same thing happens.
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
|
||||
All interesting transcripts are posted above with cruft removed.
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
Loading…
Reference in a new issue