-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (GNU/Linux)
iQIVAwUAUh0JyckQ2SIlEuPHAQihEQ/+Ix0nlizcigYfHqmqea8DQTE4mlIlGEng
IkUX5e1dir1JqqHk+4jjSdrjuopAnZ5wsf+bWnMEC7yOYftq/59OGowDIy9jWqqf
oN0BQDNIwgR8+5WETewM9ZWc+I5yCcj6sEyf0/BrJJY5QUMENDsRKLF79ADPHWuU
5EGezOHTD0m+KBqSw+s+Cj+XUmZHJgPq2j6kFqlrFMc7qv7rLnj3bI3Mc7aHfWua
g0oHMyXA2jMiP2UwjMU/hAPq6dqt0fjeJ7vyYyIsETQ1jZsdZQtRjaSSxNAEBEJ1
eGD0zYJ6GdusD0IpI+UVp6Wkw3y0kKA0WsYXtu3vVqyEjO7uHBGcWPePiGISd7nd
GPW7KBftKFBa7DQe56d/Ztk05Aze0eeek50DGMovzKfOj/FysNn3ystUy43RBSF8
40kCb8S/Rs2VPVOWNrcccYo2Hx5SFaYG+YBHJTxYNdwsx4DyMB1iGowe3fASJgnF
VMhqkKWn55i/CdJG2jlWr4wSiW44iY3b/na/f3Ktb1FamveYCxJIH+trFBAv3qn3
EWh0wuFeg8VF/WDK/IRcuH12jup784JhIEjTeTvURd4/TJXkNtxO1xHV7BG1VuhR
rSr9AUNfdiPwRZViK4jQUd6x9OF+Qil8pSe3XbvIsJLfpWkiK80kcbFCFmXaJIrQ
5cspJ23ALTI=
=gpx9
-----END PGP SIGNATURE-----
Merge tag 'debian/4.20130827' into debian-wheezy-backport
Release 4.20130827 for sid [dgit]
# gpg: Signature made Tue Aug 27 20:19:21 2013 UTC using RSA key ID 2512E3C7
# gpg: Good signature from "Joey Hess <joeyh@debian.org>"
# gpg: aka "Joey Hess <joey@kitenet.net>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: E85A 5F63 B31D 24C1 EBF0 D81C C910 D922 2512 E3C7
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
iQIVAwUAUgyVackQ2SIlEuPHAQi3ORAAsNQE4iju50yznUQcCRHNG4HXNuZoqj+T
kMyIDQ0bGuI1nlt9Ams9jZM3y7HoUfjiOOQgeoiLQyGHm6jXreiT6LCZIfAcpE7w
2OY9nqmUns6C7JRDkXG+1ONcawJFqKJVtCeuAruFbLf5CLKErOYJnJNY6zRj7zvB
H4hEoGsW/z1CDBWNd5Zh6KGWshJMkCanpNAUBu8IiBc65wdFBkg5xj0MvnmYwZHC
peKPvap6C2SXL1noTLC2o5UVCboZKrOcFp3g6Xy0zNaHHYPMFvktzZ16gD/VlHJa
i5YStKg4VkVl8R5sPGOxqkNZIDnh4+qKSIivYWUBZJPTUMMfDbAQC+FzOdryCHj8
uZJNY/MGGrqg8jKWNwubZY6klK1Ou0HtpFNy5ANudU0EgE0JBgg95LfbnbgWIRrf
FcrJg3VM1zkkHEi3iZmSlZuOST1nO5c+Y71XH0PT6mntrNh65fn/d5a+DcYFx22L
rv3asKVZkAI45BQ/yMDKMLiZufjPVduEXK54hq05QFgSfIDtxzv+BdlTZF/PwMlf
3VuC9ish71+CJVhO7e1qk29ubJtKXfDExcZDhbs0k6IS1jtUNnfgHDnNhrckjgnf
2Vwh4zHDOwWJ2le9Bj6NqjHjsckgCUewwCN+P/7bI+X4YX9xnHEid58YG1S/gxFi
QzCRcLXZr2E=
=SNiD
-----END PGP SIGNATURE-----
Merge tag '4.20130815' into debian-wheezy-backport
tagging version 4.20130815
# gpg: Signature made Thu Aug 15 08:46:33 2013 UTC using RSA key ID 2512E3C7
# gpg: Good signature from "Joey Hess <joeyh@debian.org>"
# gpg: aka "Joey Hess <joey@kitenet.net>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: E85A 5F63 B31D 24C1 EBF0 D81C C910 D922 2512 E3C7
When quvi is installed, git-annex addurl automatically uses it to detect
when an page is a video, and downloads the video file.
web special remote: Also support using quvi, for getting files,
or checking if files exist in the web.
This commit was sponsored by Mark Hepburn. Thanks!
<RichiH> i richih@eudyptes (git)-[master] ~git/debconf-share/debconf13/photos/chrysn % rm /home/richih/work/git/debconf-share/.git/annex/tmp/SHA256E-s3044235--693b74fcb12db06b5e79a8b99d03e2418923866506ee62d24a4e9ae8c5236758.JPG
<RichiH> richih@eudyptes (git)-[master] ~git/debconf-share/debconf13/photos/chrysn % git annex get P8060008.JPG
<RichiH> get P8060008.JPG (from website...) --2013-08-21 21:42:45-- http://annex.debconf.org/debconf-share/.git//annex/objects/1a4/67d/SHA256E-s3044235--693b74fcb12db06b5e79a8b99d03e2418923866506ee62d24a4e9ae8c5236758.JPG/SHA256E-s3044235--693b74fcb12db06b5e79a8b99d03e2418923866506ee62d24a4e9ae8c5236758.JPG
<RichiH> Resolving annex.debconf.org (annex.debconf.org)... 5.153.231.227, 2001:41c8:1000:19::227:2
<RichiH> Connecting to annex.debconf.org (annex.debconf.org)|5.153.231.227|:80... connected.
<RichiH> HTTP request sent, awaiting response... 404 Not Found
<RichiH> 2013-08-21 21:42:45 ERROR 404: Not Found.
<RichiH> File `/home/richih/work/git/debconf-share/.git/annex/tmp/SHA256E-s3044235--693b74fcb12db06b5e79a8b99d03e2418923866506ee62d24a4e9ae8c5236758.JPG' already there; not retrieving.
<RichiH> Unable to access these remotes: website
<RichiH> Try making some of these repositories available:
<RichiH> 3e0356ac-0743-11e3-83a5-1be63124a102 -- website (annex.debconf.org)
<RichiH> a7495021-9f2d-474e-80c7-34d29d09fec6 -- chrysn@hephaistos:~/data/projects/debconf13/debconf-share
<RichiH> eb8990f7-84cd-4e6b-b486-a5e71efbd073 -- joeyh passport usb drive
<RichiH> f415f118-f428-4c68-be66-c91501da3a93 -- joeyh laptop
<RichiH> failed
<RichiH> git-annex: get: 1 failed
<RichiH> richih@eudyptes (git)-[master] ~git/debconf-share/debconf13/photos/chrysn %
I was not able to reproduce the failure, but I did reproduce that
wget -O http://404/ results in an empty file being written.
This is a simple approach for setting up a mirroring repository.
It will work with any type of remotes.
Mirror --from is more expensive than mirror --to in general.
OTOH, mirror --from will get the file from any remote that has it, not only
the named mirror remote. And if the named mirror remote is not the fastest
available remote with a file, that can speed things up.
It would be possible to make the assistant or watch command do a more
dynamic mirroring, that didn't need to scan every time.
Requires git 1.8.4 or newer. When it's installed, a background
git check-ignore process is run, and used to efficiently check ignores
whenever a new file is added.
Thanks to Adam Spiers, for getting the necessary support into git for this.
A complication is what to do about files that are gitignored but have
been checked into git anyway. git commands assume the ignore has been
overridden in this case, and not need any more overriding to commit a
changed version.
However, for the assistant to do the same, it would have to run git ls-files
to check if the ignored file is in git. This is somewhat expensive. Or it
could use the running git-cat-file process to query the file that way,
but that requires transferring the whole file content over a pipe, so it
can be quite expensive too, for files that are not git-annex
symlinks.
Now imagine if the user knows that a file or directory tree will be getting
frequent changes, and doesn't want the assistant to sync it, so gitignores
it. The assistant could overload the system with repeated ls-files checks!
So, I've decided that the assistant will not automatically commit changes
to files that are gitignored. This is a tradeoff. Hopefully it won't be a
problem to adjust .gitignore settings to not ignore files you want the
assistant to autocommit, or to manually git annex add files that are listed
in .gitignore.
(This could be revisited if git-annex gets access to an interface to check
the content of the index w/o forking a git command. This could be libgit2,
or perhaps a separate git cat-file --batch-check process, so it wouldn't
need to ship over the whole file content.)
This commit was sponsored by Francois Marier. Thanks!
This runs git-cat-file in non-batch mode for all files with spaces.
If a directory tree has a lot of them, and is in direct mode, even "git
annex add" when there are few new files will need a *lot* of forks!
The only reason buffering the whole file content to get the sha is not a
memory leak is that git-annex only ever uses this on symlinks.
This needs to be reverted as soon as a fix is available in git!
A git pathspec is a filename, except when it starts with ':', it's taken
to refer to a branch, etc. Rather than special case ':', any filename
starting with anything unusual is prefixed with "./"
This could have been a real mess to deal with, but luckily SafeCommand
is already extensively used and so we know at the type level the difference
between parameters that are files, and parameters that are command options.
Testing did show that Git.Queue was not using SafeCommand on
filenames fed to xargs. (Filenames starting with '-' worked before only
because -- was used to separate filenames from options when calling eg git
add.)
The test suite now passes with filenames starting with ':'. However, I did
not keep that change to it, because such filenames are probably not legal
on windows, and I have enough ugly windows ifdefs in there as it is.
This commit was sponsored by Otavio Salvador. Thanks!
This includes recovery from the ssh-agent problem that led to many reporting
http://git-annex.branchable.com/bugs/Internal_Server_Error:_Unknown_UUID/
(Including fixing up .ssh/config to set IdentitiesOnly.)
Remotes that have no known uuid are now displayed in the webapp as
"unfinished". There's a link to check their status, and if the remote
has been set annex-ignore, a retry button can be used to unset that and
try again to set up the remote.
As this bug has shown, the process of adding a ssh remote has some failure
modes that are not really ideal. It would certianly be better if, when
setting up a ssh remote it would detect if it's failed to get the UUID,
and handle that in the remote setup process, rather than waiting until
later and handling it this way.
However, that's hard to do, particularly for local pairing, since the
PairListener runs as a background thread. The best it could do is pop up an
alert if there's a problem. This solution is not much different.
Also, this solution handles cases where the user has gotten their repo into
a mess manually and let's the assistant help with cleaning it up.
This commit was sponsored by Chia Shee Liang. Thanks!
When setting up a dedicated ssh key to access the annex on a host,
set IdentitiesOnly to prevent the ssh-agent from forcing use of a different
ssh key.
That behavior could result in unncessary password prompts. I remember
getting a message or two from people who got deluged with password
prompts and I couldn't at the time see why.
Also, it would prevent git-annex-shell from being run on the remote host,
when git-annex was installed there by unpacking the standalone tarball,
since the authorized_keys line for the dedicated ssh key, which sets
up calling git-annex-shell when it's not in path, wouldn't be used.
This fixes
http://git-annex.branchable.com/bugs/Internal_Server_Error:_Unknown_UUID
but I've not closed that bug yet since I should still:
1. Investigate why the ssh remote got set up despite being so broken.
2. Make the webapp not handle the NoUUID state in such an ugly way.
3. Possibly add code to fix up systems that encountered the problem.
Although since it requires changes to .ssh/config this may be one for
the release notes.
Thanks to TJ for pointing me in the right direction to understand what
was happening here.
Started with a problem when running addurl on a really long url,
because the whole url is munged into the filename. Ended up doing
a fairly extensive review for places where filenames could get too large,
although it's hard to say I'm not missed any..
Backend.Url had a 128 character limit, which is fine when the limit is 255,
but not if it's a lot shorter on some systems. So check the pathconf()
limit. Note that this could result in fromUrl creating different keys
for the same url, if run on systems with different limits. I don't see
this is likely to cause any problems. That can already happen when using
addurl --fast, or if the content of an url changes.
Both Command.AddUrl and Backend.Url assumed that urls don't contain a
lot of multi-byte unicode, and would fail to truncate an url that did
properly.
A few places use a filename as the template to make a temp file.
While that's nice in that the temp file name can be easily related back to
the original filename, it could lead to `git annex add` failing to add a
filename that was at or close to the maximum length.
Note that in Command.Add.lockdown, the template is still derived from the
filename, just with enough space left to turn it into a temp file.
This is an important optimisation, because the assistant may lock down
a bunch of files all at once, and using the same template for all of them
would cause openTempFile to iterate through the same set of names,
looking for an unused temp file. I'm not very happy with the relatedTemplate
hack, but it avoids that slowdown.
Backend.WORM does not limit the filename stored in the key.
I have not tried to change that; so git annex add will fail on really long
filenames when using the WORM backend. It seems better to preserve the
invariant that a WORM key always contains the complete filename, since
the filename is the only unique material in the key, other than mtime and
size. Since nobody has complained about add failing (I think I saw it
once?) on WORM, probably it's ok, or nobody but me uses it.
There may be compatability problems if using git annex addurl --fast
or the WORM backend on a system with the 255 limit and then trying to use
that repo in a system with a smaller limit. I have not tried to deal with
those.
This commit was sponsored by Alexander Brem. Thanks!
This bug was introduced in 82a6db8fe8,
which improved handling of adding very large numbers of files by ensuring
that a minimum number of max size commits (5000 files each) were done.
I accidentially made it wait for another change to appear after such a max
size commit, even if a lot of queued changes were already accumulated.
That resulted in a stall when it got to the end. Now fixed to not wait
any longer than necessary to ensure the watcher has had time to wake back
up after the max size commit.
This commit was sponsored by Michael Linksvayer. Thanks!
This is a laziness problem. Despite the bang pattern on newfiles, the list
was not being fully evaluated before cleanup was called. Moving cleanup out
to after the list is actually used fixes this.
More evidence that I should be using ResourceT or pipes, if any was needed.
This affected both the hourly NetWatcherFallback thread and the syncing
when network connection is detected.
It was a reversion of sorts, introduced in
8861e270be, when annex-ignore was changed to
not control git syncing. I forgot to make it check annex-sync at that
point.
This was the last place in git-annex that could remove data referred to by
the git history, without being forced.
Like drop, dropunused checks remotes, and honors the global annex.numcopies
setting. (However, .gitattributes settings cannot apply to unused files.)
