Commit graph

34273 commits

Author SHA1 Message Date
Joey Hess
e1ab9a6d41
Merge branch 'master' of ssh://git-annex.branchable.com 2019-03-11 14:30:50 -04:00
Joey Hess
91f0f76252
need to write a tip
deferred because a tip that is limited to the directory special remote
is not very interesting to most
2019-03-11 14:30:05 -04:00
Joey Hess
5dd21116b7
update for deprecation of --tracking option 2019-03-11 14:27:01 -04:00
Joey Hess
4d610ee98a
Merge branch 'importtree' 2019-03-11 14:17:59 -04:00
Joey Hess
353e4f6d24
update changelog 2019-03-11 14:17:49 -04:00
Joey Hess
ffec06aaec
update 2019-03-11 14:13:58 -04:00
Joey Hess
28e46d947a
avoid sync --content trying to sendKey to exporttree remotes 2019-03-11 14:09:46 -04:00
Joey Hess
057999f0fc
fix sync --content with remote.name.annex-tracking-branch=master:subdir
It was exporting the whole tree not just the subdir. Now tested fully
working in both directions.
2019-03-11 14:07:52 -04:00
Joey Hess
8ae0db925b
fix name of annex-tracking-branch config 2019-03-11 13:56:59 -04:00
Joey Hess
e1fdc8b374
record only subtree in export db and log after importing a subtree 2019-03-11 13:45:01 -04:00
Joey Hess
e46e40bf05
fix update of export tracking branch when exporting branch:subdir 2019-03-11 13:44:12 -04:00
Joey Hess
2912429640
better indicate when special remotes do not support renameExport
Avoid a warning message when renameExport is not supported, and just
fallback to deleting with a subsequent re-upload. Especially needed for
importtree remotes, where renameExport needs to be disabled.

This changes the external special remote protocol, but in a
backwards-compatible way. A reply of UNSUPPORTED-REQUEST to an older
version of git-annex will cause it to make renameExport return False.
2019-03-11 12:53:24 -04:00
Joey Hess
c755788256
sync: import when annex-tracking-branch is configured
This works, and tested syncing both gets changes from a special remote
and sends changes to it, keeping it fully in sync nicely!

But have not tried it with a subdir configured.
2019-03-09 13:57:49 -04:00
Joey Hess
ca1a3caaa8
refactor 2019-03-09 13:34:57 -04:00
Joey Hess
633021e135
--no-push and remote.name.annex-push prevent exporting trees to special remotes
Users may want sync to only export, or only import and this is broadly
analagous to push and pull, so it makes sense to use the same
configuration for it.
2019-03-09 13:21:49 -04:00
Joey Hess
d9ee048d85
doc updates for import 2019-03-09 13:10:30 -04:00
publiwebmaxter@739143dcf7422b8de1502bf2925640bed6ef7014
0941ff5190 Added a comment 2019-03-09 08:33:49 +00:00
Joey Hess
6149a3b9bd
added progress bars
clean up old todo item I checked earlier, see commit
dec30d2b14
2019-03-08 12:43:03 -04:00
Joey Hess
e412129523
concurrency and status messages when downloading from import 2019-03-08 12:33:44 -04:00
Joey Hess
033e47b7d7
update 2019-03-07 19:44:00 -04:00
Joey Hess
33d79d84ff
Merge branch 'master' of ssh://git-annex.branchable.com 2019-03-07 16:15:52 -04:00
Joey Hess
a66dd19e15
devblog 2019-03-07 16:15:28 -04:00
anarcat
dba0e89b71 Added a comment 2019-03-07 20:15:10 +00:00
Joey Hess
ee5f1422df
remove debug print 2019-03-07 16:08:58 -04:00
Joey Hess
e3a704224f
fix export db locking deadlock 2019-03-07 16:06:02 -04:00
Joey Hess
7e35c81ada
locking problem 2019-03-07 15:22:23 -04:00
Joey Hess
4efd431136
remove obsolete TODO
updateExportDb runs addExportedLocation
2019-03-07 15:11:24 -04:00
anarcat
b43d382841 Added a comment: or -c annex.verify=false 2019-03-07 18:23:02 +00:00
anarcat
afeb00b19a Added a comment: parallelizing checksum and get 2019-03-07 18:21:22 +00:00
Joey Hess
9a72785307
fixes to export db lookup when accessing importtree=yes
Now in a fresh clone with a importtree=yes remote enabled,
git annex fsck --from the remote works.
2019-03-07 14:10:56 -04:00
Joey Hess
93025dd59f
add missing locking of ContentIdentifier database when writing
This is not super efficient; it would be better to lock the database
once and build up a queue of changes and flush once.

But, storeExportWithContentIdentifier is likely going to be the really
expensive part, so let's do the simple thing and only optimise later if
needed.
2019-03-07 13:32:33 -04:00
Joey Hess
3f449f845e
update 2019-03-07 13:28:18 -04:00
Joey Hess
50797ee2c5
remove obsolete comment 2019-03-07 13:02:46 -04:00
Joey Hess
71fec9060c
move 2019-03-07 12:56:40 -04:00
Joey Hess
68d1661251
cross-repo import now working correctly 2019-03-07 12:31:35 -04:00
crest
2cf3d68fe0 2019-03-07 10:48:01 +00:00
501st_alpha1
5843c0f905 Added a comment: Figured out how to sync with Keybase 2019-03-07 06:52:25 +00:00
Joey Hess
ee251b2e2e
implement updating the ContentIdentifier db with info from the git-annex branch
untested

This won't be super slow, but it does need to diff two likely large
trees, and since the git-annex branch rarely sits still, it will most
likely be run at the beginning of every import.

A possible speed improvement would be to only run this when the database
did not contain a ContentIdentifier. But that would only speed up
imports when there is no new version of a file on the special remote,
at most renames of existing files being imported.

A better speed improvement would be to record something in the git-annex
branch that indicates when an import has been run, and only do the diff
if the git-annex branch has record of a newer import than we've seen
before. Then, it would only run when there is in fact new
ContentIdentifier information available from a remote. Certianly doable,
but didn't want to complicate things yet.
2019-03-06 18:04:30 -04:00
Joey Hess
12e4906657
refactor
locationLogFileKey had an out of date list of toplevel log files to skip
over, and was only not broken because the other toplevel log files don't
look like keys. Fixed that too.
2019-03-06 17:54:29 -04:00
Joey Hess
dec30d2b14
updates
Note that I tried an evil remote that lists ImportLocations with
../../../ in them and indeed this resulted in git blowing up and the
import failing, and not writing outside the repo.
2019-03-06 17:07:36 -04:00
Joey Hess
8e9713b769
add export+import test case 2019-03-06 16:49:33 -04:00
Joey Hess
3b412aaae0
simplify Applicative instance 2019-03-06 16:44:17 -04:00
Joey Hess
1b3d04979e
speed up slow quickcheck test
Only test parsing of ContentIdentifier lists, not the whole log.
2019-03-06 16:43:41 -04:00
Joey Hess
c6c5f6336b
avoid whitespace in Arbitrary UUID and empty UUID 2019-03-06 15:44:27 -04:00
Joey Hess
6ef38df881
fix another parser bug 2019-03-06 14:51:31 -04:00
Joey Hess
b3d30e7d70
remove unncessary locking of ContentIdentifier db
Remote.Helper.ExportImport only reads from it, and locking is only
needed when writing.
2019-03-06 14:36:57 -04:00
Joey Hess
c0bd202147
fix failing test case
An empty list of [ContentIdenfier] serialized to the same thing
as a single ContentIdentifier "". Avoid this ambiguity by requiring the
list be non-empty.
2019-03-06 14:27:15 -04:00
Joey Hess
be6085cfe5
fix option parser
Alternative doesn't combine the subparsers the way I wanted.
Unfortunately this new parser has suboptimal usage because everything is
all jumbled together.
2019-03-06 13:10:29 -04:00
yarikoptic
9f908c8856 Added a comment: more details on coreutils cp treatment of reflink 2019-03-06 16:00:35 +00:00
Joey Hess
b0fe4916b7
forgot to change list delimiter in parser 2019-03-06 11:59:42 -04:00