Commit graph

25464 commits

Author SHA1 Message Date
Joey Hess
aaacf431d8
handle importtree=yes config
For now, it's only allowed when exporttree=yes is also set.
That simplified the implementation, but could later be changed if
there's a remote that makes sense to be an import but not an export.
However, it may work just as well to make a remote be readonly to
prevent export to it while still allowing import.
2019-03-04 16:07:35 -04:00
Joey Hess
5f17a9cc50
docs for importtree config 2019-03-04 15:39:19 -04:00
Joey Hess
3cd19fb4d0
use InodeCache to avoid races in import from directory special remote
This does not avoid all possible races, but it does avoid all likely
ones, and is demonstratably better than git's own handling of races
where files get modified at the same time as it's updating the working
tree.

The main thing this won't detect are not unlikely races where part
of a file gets changed while it's being copied and then the file is
restored to its original condition before the modification check.
No, it's more likely that the limitations of checking inode, size,
and mtime won't detect certian modifications, involving eg mmapped
files.
2019-03-04 13:57:23 -04:00
Joey Hess
51fc969b66
notes 2019-03-01 16:44:34 -04:00
Joey Hess
18d7a1dbbb
make export and sync update special remote tracking branch
The branch is only updated once the export is 100% complete. This way,
if an export is started but interrupted and so the remote does not yet
contain some of the files, an import will make a commit on the old
branch, and so won't delete the missing files.
2019-03-01 16:35:48 -04:00
Joey Hess
d28b0a8bd0
use disconnected history for import tracking branch
This avoids the first merge from it deleting all files in the current
branch, which was very surpring and unwanted behavior.
2019-03-01 14:33:29 -04:00
Joey Hess
740f957cef
new problem 2019-03-01 13:49:26 -04:00
Joey Hess
1c8793691a
import: update location log for removed files 2019-03-01 13:26:59 -04:00
Joey Hess
5402c39882
plan for item 2019-03-01 12:56:20 -04:00
Joey Hess
a3f6e07fec
todo 2019-02-27 15:56:32 -04:00
Joey Hess
b1f10fbb4d
update location log during import 2019-02-27 13:58:03 -04:00
Joey Hess
e2e57f8556
initial export support for directory special remote
This does not guard against race condition yet, it's only for testing
purposes.
2019-02-27 13:42:34 -04:00
Joey Hess
45aacd888b
import downloader complete (untested)
Made some api changes.

listImportableContents needs to provide the size
of the data, so the downloader can check disk free space.

retrieveExportWithContentIdentifier is passed the filepath to write to

Use temporary "CID" key during download of a ContentIdentifier from a
remote, so withTmp can be used and then move the content to the real key
once it's known.
2019-02-27 13:15:02 -04:00
Joey Hess
f4b773e9a1
incomplete action to download files from import 2019-02-26 15:25:28 -04:00
Joey Hess
e4e464da65
import command is updating tracking branch 2019-02-26 13:15:48 -04:00
Joey Hess
760f26ebc6
Merge branch 'master' into importtree 2019-02-26 11:36:36 -04:00
sameerds
22e9a49bd9 Added a comment: +1 2019-02-24 17:06:30 +00:00
david.j.buckmaster@984ff2704feacab53415ac5647206517d18f88f8
ea3e6322e7 Added a comment 2019-02-24 07:01:35 +00:00
david.j.buckmaster@984ff2704feacab53415ac5647206517d18f88f8
1202a2a48c removed 2019-02-24 06:59:21 +00:00
david.j.buckmaster@984ff2704feacab53415ac5647206517d18f88f8
24ebc3d12e Added a comment 2019-02-24 06:56:43 +00:00
david.j.buckmaster@984ff2704feacab53415ac5647206517d18f88f8
8c777cb36d removed 2019-02-24 06:43:34 +00:00
david.j.buckmaster@984ff2704feacab53415ac5647206517d18f88f8
eb6c0608e6 Added a comment 2019-02-24 06:41:01 +00:00
david.j.buckmaster@984ff2704feacab53415ac5647206517d18f88f8
3f450e5271 Added a comment 2019-02-24 01:20:15 +00:00
Joey Hess
89ee156059
Merge branch 'master' of ssh://git-annex.branchable.com 2019-02-23 16:04:15 -04:00
Joey Hess
15a4f91d07
devblog 2019-02-23 16:03:29 -04:00
Joey Hess
7072f7a071
note 2019-02-23 15:57:18 -04:00
Joey Hess
5bac8babdb
doc updated for import tree
Deprecated git annex export --tracking because it makes sense to have a
single configuration of tracking for both imports and exports.
2019-02-23 15:46:03 -04:00
david.j.buckmaster@984ff2704feacab53415ac5647206517d18f88f8
663c62d7ce 2019-02-23 18:38:42 +00:00
Joey Hess
464485bffe
Merge branch 'master' into importtree 2019-02-23 13:58:22 -04:00
Joey Hess
d685b119df
more design 2019-02-23 13:55:26 -04:00
Joey Hess
c5ceefdca6
Merge branch 'master' into importtree 2019-02-22 22:03:00 -04:00
Joey Hess
3c405838f8
more design 2019-02-22 22:02:50 -04:00
Joey Hess
4e0d08b66b
Merge branch 'master' into importtree 2019-02-22 21:18:13 -04:00
Joey Hess
200dc632f5
more design 2019-02-22 21:18:01 -04:00
Joey Hess
8c836623b7
design work 2019-02-22 16:18:09 -04:00
Joey Hess
d7e5a884f7
response 2019-02-22 16:03:19 -04:00
gan
5b7daecbd3 removed 2019-02-22 18:03:32 +00:00
gan
776916d5fe Added a comment: Clarification 2019-02-22 18:03:16 +00:00
gan
9e38dfd700 Added a comment: Clarification 2019-02-22 18:03:03 +00:00
gan
baa1699570 Added a comment: Provide flags to youtube-dl? 2019-02-22 18:01:26 +00:00
Joey Hess
d67494716c
Merge branch 'master' of ssh://git-annex.branchable.com 2019-02-21 17:46:09 -04:00
Joey Hess
433fef865f
devblog 2019-02-21 17:45:47 -04:00
Joey Hess
8fdea8f444
WIP
Added graftTree but it's buggy.

Should use graftTree in Annex.Branch.graftTreeish; it will be faster
than the current implementation there.

Started Annex.Import, but untested and it doesn't yet handle tree
grafting.
2019-02-21 17:32:59 -04:00
62.226.58.176
8428a74a0e poll vote (My phone (or MP3 player)) 2019-02-21 18:36:04 +00:00
Joey Hess
56137ce0d2
use colon not space to delimit content identifier list
InodeCache serializes to a value with spaces, and seems likely other
things will too, and want to avoid unncessary base64 of content
identifiers when possible.
2019-02-21 13:45:16 -04:00
Joey Hess
a66ab6a9eb
Merge branch 'master' of ssh://git-annex.branchable.com 2019-02-20 17:36:14 -04:00
Joey Hess
c605478e41
devblog 2019-02-20 17:34:45 -04:00
Joey Hess
7392304f1c
notes 2019-02-20 17:06:36 -04:00
Joey Hess
e8bfc3640b
storing ContentIdentifier in the git-annex branch 2019-02-20 15:40:07 -04:00
Joey Hess
0442842622
add import tree interface to Remote 2019-02-20 15:35:22 -04:00