Commit graph

39484 commits

Author SHA1 Message Date
Joey Hess
31eb5fddf3
borg: Fix a bug that prevented importing keys of type URL and WORM
Keys stored on the filesystem are mangled by keyFile to avoid problem
chars. So, that mangling has to be reversed when parsing files from a
borg backup back to a key.

The directory special remote also so mangles them. Some other special
remotes do not; eg S3 just serializes the key -- but S3 object names are
not limited to filesystem valid filenames anyway, so a S3 server must
not map them directly to files in any case. It seems unlikely that a
borg backup of some such special remote will get broken by this change.

This commit was sponsored by Graham Spencer on Patreon.
2021-03-26 12:07:00 -04:00
parhuzamos
72f5088d34 2021-03-26 12:34:01 +00:00
parhuzamos
2187892a81 2021-03-26 12:31:47 +00:00
Lukey
79adbbadce Added a comment 2021-03-25 18:26:32 +00:00
branchable@652bb419541f29c1d6b1fc16bd84c3867fbda245
252551e670 2021-03-25 17:25:29 +00:00
Joey Hess
537f9d9a11
Improved display of errors when accessing a git http remote fails.
New error message:

  Remote foo not usable by git-annex; setting annex-ignore

  http://localhost/foo/config download failed: Configuration of annex.security.allowed-ip-addresses does not allow accessing address ::1

If git config parse fails, or the git config file is not available at the url,
a better error message for that is also shown.

This commit was sponsored by Mark Reidenbach on Patreon.
2021-03-24 14:19:32 -04:00
Joey Hess
fdf1ccbe3f
move comment 2021-03-24 13:57:00 -04:00
Joey Hess
97021435de
Merge branch 'master' of ssh://git-annex.branchable.com 2021-03-24 12:12:08 -04:00
Joey Hess
4631d1ab56
Fix build with attoparsec-0.14
It changed parseOnly in the ByteString.Lazy module to take a lazy, not
strict ByteString. In all these cases though, we actually had a strict
ByteString, so the most efficient fix, which also happens to avoid needing
ifdefs, is to use the non-lazy module instead.

This commit was sponsored by Denis Dzyubenko on Patreon.
2021-03-24 12:11:50 -04:00
Joey Hess
41d9148c72
fix attoparsec lower bound
needed for parseOnly
2021-03-24 11:54:16 -04:00
git-annex@6f13b739194f758abc0b86556b7ce966c1bf3c00
14b846e9fa Added a comment: borg hardlinks 2021-03-24 10:29:16 +00:00
git-annex@6f13b739194f758abc0b86556b7ce966c1bf3c00
d4a3465f8b Added a comment: / vs. % in key 2021-03-24 10:07:30 +00:00
git-annex@6f13b739194f758abc0b86556b7ce966c1bf3c00
cc9d6889e2 2021-03-24 09:44:49 +00:00
tomdhunt
ffccfff618 Added a comment 2021-03-23 22:59:07 +00:00
Ilya_Shlyakhter
fc598d56e9 posted forum question re: git-annex-sync and git-annex-config 2021-03-23 20:13:18 +00:00
Ilya_Shlyakhter
a4cc0c95b4 added suggestion for additional git-annex-config settings 2021-03-23 20:11:39 +00:00
Ilya_Shlyakhter
547a5a8ca8 Added a comment: annex.supportunlocked=false 2021-03-23 20:02:19 +00:00
Joey Hess
f19271c5d9
comment 2021-03-23 15:51:21 -04:00
Joey Hess
806b6f77b9
Merge branch 'master' of ssh://git-annex.branchable.com 2021-03-23 15:47:21 -04:00
Joey Hess
1e3b228154
speed up init
This was making it git checkout master when that branch was already
checked out, for apparently no good reason at all. In a 100,000 file
repo, that takes about 1 second.

Note, I'm not sure why it checks out the branch in the Nothing case, so
I left that alone.
2021-03-23 15:43:42 -04:00
Ilya_Shlyakhter
3925235805 Added a comment: annex.supportunlocked 2021-03-23 19:30:44 +00:00
Joey Hess
5d78cd9d08
Sped up git-annex init in a clone of an existing repository
Seems that hasOrigin was never finding origin's git-annex branch, so a new
one got created each time. And so then it later needed to merge the two
branches, which is expensive.

Added --no-track to git branch to avoid it displaying a message about
setting up tracking branches. Of course there's no reason to make the
git-annex branch a tracking branch since git-annex auto-merges it.
2021-03-23 15:23:13 -04:00
yarikoptic
ed5fd5b896 Added a comment 2021-03-23 18:43:32 +00:00
Joey Hess
798f685077
New annex.supportunlocked config
Can beet to false to avoid some expensive things needed to support unlocked
files.

See my comment for why this only controls what init sets up, and not other
behavior.

I didn't bother with making the v5 upgrade code path look at this, though
it easily could, because the docs say to run git-annex init after setting
it to make it take effect.
2021-03-23 14:04:34 -04:00
Joey Hess
af96b49145
comment 2021-03-23 13:53:32 -04:00
Joey Hess
a8b837aaef
add git ls-tree --long parser
Not yet used, but allows getting the size of items in the tree fairly
cheaply.

I noticed that CmdLine.Seek uses ls-tree and the feeds the files into
another long-running process to check their size. That would be an
example of a place that might be sped up by using this. Although in that
particular case, it only needs to know the size of unlocked files, not
locked. And since enabling --long probably doubles the ls-tree runtime
or more, the overhead of using it there may outwweigh the benefit.
2021-03-23 12:47:00 -04:00
Joey Hess
d89a9b0f78
comments 2021-03-23 12:05:05 -04:00
Joey Hess
e09560eea2
open a bug based on comment thread 2021-03-23 11:50:07 -04:00
Joey Hess
0af8df6714
Merge remote-tracking branch 'origin/master' 2021-03-23 11:49:43 -04:00
Lukey
2ac2cc79c2 Added a comment 2021-03-23 07:19:40 +00:00
Joey Hess
c68ba7d893
whereis: Don't include yt: prefix when showing url to content retrieved with youtube-dl
I don't think this was really intentional behavior. It may be that it was
useful to include it so it could be passed to rmurl, since without it rmurl
would not actually remove the url. Since that was changed earlier today,
now seems like a good time to clean up the display of these urls.

This commit was sponsored by Jochen Bartl on Patreon.
2021-03-22 19:56:24 -04:00
tomdhunt
a50071b60f Added a comment 2021-03-22 23:35:32 +00:00
Ilya_Shlyakhter
3b748d522b clarified git-annex-unlock docs re: git add 2021-03-22 19:58:43 +00:00
Ilya_Shlyakhter
420e402779 Added a comment: replacing a locked file with new content 2021-03-22 19:50:27 +00:00
Joey Hess
29f9cf188d
comment 2021-03-22 15:20:28 -04:00
Joey Hess
3a125a8a9a
Merge branch 'master' of ssh://git-annex.branchable.com 2021-03-22 15:09:22 -04:00
Joey Hess
637229c593
fix fsck --from --all to not fall over trying to check required content
fsck: When --from is used in combination with --all or similar options, do
not verify required content, which can't be checked properly when operating
on keys.

This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2021-03-22 15:08:07 -04:00
kyle
e15a185107 Added a comment: Re: push doc-only changes 2021-03-22 18:58:36 +00:00
Joey Hess
b8df74c016
comment 2021-03-22 14:44:24 -04:00
Joey Hess
da989596cd
git-annex is in openbsd ports 2021-03-22 14:37:06 -04:00
Joey Hess
e8ee1a4506
reorder
it seems to make sense to have linux and osx first since a lot of users
use those, and windows at the end because well, it's windows. but it
seemed odd to start with osx, so moved linux up.

alphabetical would maybe be better, but then android comes first, which
just feels weird since it's such a niche port. also the linux distro
sublist has a good reason not to be alphabetical, eg docker and conda
are on there as kind of last resorts
2021-03-22 14:30:23 -04:00
Joey Hess
9b8d6a86e8
freebsd is not a linux distribution 2021-03-22 14:28:03 -04:00
Joey Hess
5545e78a1e
Make --debug also enable debugging in child git-annex processes
Especially necessary with stalldetection using child processes for
transfers.

This commit was sponsored by Jack Hill on Patreon.
2021-03-22 14:25:28 -04:00
Joey Hess
1ddaa6b541
comment 2021-03-22 13:52:11 -04:00
Joey Hess
2af5bc8fa2
Merge branch 'master' of ssh://git-annex.branchable.com 2021-03-22 13:50:58 -04:00
Joey Hess
3a3832fcb4
remove link to old windows 7 build
that autobuilder is no longer available
2021-03-22 13:48:46 -04:00
Joey Hess
79b3e0dca9
comment 2021-03-22 13:43:58 -04:00
Lukey
25cf5e39df Added a comment 2021-03-22 17:19:37 +00:00
Joey Hess
5d75cbcdcf
webdav: deal with buggy webdav servers in renameExport
box.com already had a special case, since its renaming was known buggy.
In its case, renaming to the temp file succeeds, but then renaming the temp
file to final destination fails.

Then this 4shared server has buggy handling of renames across directories.
While already worked around with for the temp files when storing exports
now being in the same directory as the final filename, that also affected
renameExport when the file moves between directories.

I'm not entirely clear what happens on the 4shared server when it fails
this way. It kind of looks like it may rename the file to destination and
then still fail.

To handle both, when rename fails, delete both the source and the
destination, and fall back to uploading the content again. In the box.com
case, the temp file is the source, and deleting it makes sure the temp file
gets cleaned up. In the 4shared case, the file may have been renamed to the
destination and so cleaning that up avoids any interference with the
re-upload to the destination.
2021-03-22 13:08:18 -04:00
Joey Hess
0af9d1dcb6
unregisterurl: remove all forms of an url, no matter what the downloader is set to
unregisterurl: Fix a bug that caused an url to not be unregistered when it
is claimed by a special remote other than the web.

See commit f175d4cc90 for rationalle.
2021-03-22 12:17:17 -04:00