Commit graph

2468 commits

Author SHA1 Message Date
Joey Hess
8c54604e67
import+export from directory special remote fully working
Had to add two more API calls to override export APIs that are not safe
for use in combination with import.

It's unfortunate that removeExportDirectory is documented to be allowed
to remove non-empty directories. I'm not entirely sure why it's that
way, my best guess is it was intended to make it easy to implement with
just rm -rf.
2019-03-05 14:20:14 -04:00
Joey Hess
554b7b7f3e
fix todo 2019-03-04 18:20:12 -04:00
Joey Hess
cd3a2b023a
initial try at using storeExportWithContentIdentifier
Untested, and I'm not sure about the locking of the ContentIdentifier db.
2019-03-04 17:50:41 -04:00
Joey Hess
aaacf431d8
handle importtree=yes config
For now, it's only allowed when exporttree=yes is also set.
That simplified the implementation, but could later be changed if
there's a remote that makes sense to be an import but not an export.
However, it may work just as well to make a remote be readonly to
prevent export to it while still allowing import.
2019-03-04 16:07:35 -04:00
Joey Hess
3cd19fb4d0
use InodeCache to avoid races in import from directory special remote
This does not avoid all possible races, but it does avoid all likely
ones, and is demonstratably better than git's own handling of races
where files get modified at the same time as it's updating the working
tree.

The main thing this won't detect are not unlikely races where part
of a file gets changed while it's being copied and then the file is
restored to its original condition before the modification check.
No, it's more likely that the limitations of checking inode, size,
and mtime won't detect certian modifications, involving eg mmapped
files.
2019-03-04 13:57:23 -04:00
Joey Hess
51fc969b66
notes 2019-03-01 16:44:34 -04:00
Joey Hess
18d7a1dbbb
make export and sync update special remote tracking branch
The branch is only updated once the export is 100% complete. This way,
if an export is started but interrupted and so the remote does not yet
contain some of the files, an import will make a commit on the old
branch, and so won't delete the missing files.
2019-03-01 16:35:48 -04:00
Joey Hess
d28b0a8bd0
use disconnected history for import tracking branch
This avoids the first merge from it deleting all files in the current
branch, which was very surpring and unwanted behavior.
2019-03-01 14:33:29 -04:00
Joey Hess
740f957cef
new problem 2019-03-01 13:49:26 -04:00
Joey Hess
1c8793691a
import: update location log for removed files 2019-03-01 13:26:59 -04:00
Joey Hess
5402c39882
plan for item 2019-03-01 12:56:20 -04:00
Joey Hess
a3f6e07fec
todo 2019-02-27 15:56:32 -04:00
Joey Hess
b1f10fbb4d
update location log during import 2019-02-27 13:58:03 -04:00
Joey Hess
e2e57f8556
initial export support for directory special remote
This does not guard against race condition yet, it's only for testing
purposes.
2019-02-27 13:42:34 -04:00
Joey Hess
45aacd888b
import downloader complete (untested)
Made some api changes.

listImportableContents needs to provide the size
of the data, so the downloader can check disk free space.

retrieveExportWithContentIdentifier is passed the filepath to write to

Use temporary "CID" key during download of a ContentIdentifier from a
remote, so withTmp can be used and then move the content to the real key
once it's known.
2019-02-27 13:15:02 -04:00
Joey Hess
f4b773e9a1
incomplete action to download files from import 2019-02-26 15:25:28 -04:00
Joey Hess
e4e464da65
import command is updating tracking branch 2019-02-26 13:15:48 -04:00
Joey Hess
7072f7a071
note 2019-02-23 15:57:18 -04:00
Joey Hess
4e0d08b66b
Merge branch 'master' into importtree 2019-02-22 21:18:13 -04:00
Joey Hess
8c836623b7
design work 2019-02-22 16:18:09 -04:00
Joey Hess
8fdea8f444
WIP
Added graftTree but it's buggy.

Should use graftTree in Annex.Branch.graftTreeish; it will be faster
than the current implementation there.

Started Annex.Import, but untested and it doesn't yet handle tree
grafting.
2019-02-21 17:32:59 -04:00
Joey Hess
7392304f1c
notes 2019-02-20 17:06:36 -04:00
Joey Hess
d128c8c3ec
add design document for import tree 2019-02-20 12:12:32 -04:00
Joey Hess
2f67c4ac87
minor tweaks to transition plan 2019-02-20 11:28:00 -04:00
Joey Hess
02c1542886
Merge branch 'master' of ssh://git-annex.branchable.com 2019-02-13 16:29:28 -04:00
Joey Hess
94d8bfb158
finally an API happy with 2019-02-13 16:28:02 -04:00
Ilya_Shlyakhter
3caae7ea81 alternate way to support batch operations by remotes 2019-02-13 16:55:34 +00:00
Ilya_Shlyakhter
1b279ec738 added suggestion for batch-mode operations for remotes 2019-02-13 16:52:09 +00:00
Ilya_Shlyakhter
23ef159352 Added a comment 2019-02-13 16:48:08 +00:00
Ilya_Shlyakhter
9d192e1cbb Added a comment 2019-02-13 16:46:26 +00:00
Grothausmann.Roman@343b033fef650d362015d5593ec935121340e082
a716b2cff9 Added a comment: Up-vote for sftp special remote 2019-02-13 12:11:05 +00:00
Grothausmann.Roman@343b033fef650d362015d5593ec935121340e082
bd06bcaebe 2019-02-13 11:21:10 +00:00
Joey Hess
2514c26ed8
update 2019-02-12 22:02:02 -04:00
Ilya_Shlyakhter
7e8016f104 added suggestion for creating simpler-to-use trusted export remotes 2019-02-12 21:09:22 +00:00
Ilya_Shlyakhter
024120065e universal batch mode -- clarified difference from current batch modes 2019-02-12 00:38:14 +00:00
Ilya_Shlyakhter
ea00337468 universal batch mode 2019-02-12 00:35:16 +00:00
Joey Hess
87987c78cf
starting api design 2019-02-11 15:47:18 -04:00
Joey Hess
b7991248db
simplify 2019-02-11 15:16:35 -04:00
Joey Hess
5a36f85c16
thoughts 2019-02-11 14:14:44 -04:00
Joey Hess
2f117ec7b7
thought 2019-02-09 14:03:59 -04:00
Joey Hess
b6df092f56
close 2019-02-07 16:12:07 -04:00
Joey Hess
d5c435d3dc
add 2019-02-07 12:16:23 -04:00
Ilya_Shlyakhter
c33793f983 Added a comment 2019-02-05 20:55:59 +00:00
Joey Hess
c3f47ba389
make .noannex file prevent repo fixups
Avoid performing repository fixups for submodules and git-worktrees
when there's a .noannex file that will prevent git-annex from being
used in the repository.

This change is ok as long as the .noannex file is really going to prevent
git-annex from being used. But, init --force could override the file.
Which would result in the repo being initialized without the fixups
having run.

To avoid that situation decided to change init, to not let --force be used
to override a .noannex file. Instead the user can just delete the file.
2019-02-05 14:43:23 -04:00
Joey Hess
b080699a95
fromkey --json
* fromkey: Added --json.
* fromkey --batch output changed to support using it with --json.
  The old output was not parseable for any useful information, so
  this is not expected to break anything.
2019-02-05 14:03:29 -04:00
Joey Hess
7b46b43c48
fromkey: Made idempotent
If the worktree file already exists, and is annexed and uses the same
key, avoid failing, nothing needs to be done.

Had to add lookupFileNotHidden to handle the case where an adjust --hide-missing
is in use, and the worktree file was hidden due to the object content
being missing. lookupFile would return the key of the hidden file,
but it makes sense that after fromkey succeeds, the worktree must
contain the file it was supposed to set up.
2019-02-05 13:13:13 -04:00
Ilya_Shlyakhter
1b3c4d9a56 git-annex-fromkey behavior when target file exists 2019-02-02 23:13:55 +00:00
Ilya_Shlyakhter
f0dea0ba06 reporting the status of each operation in a batch; completing doable ops even if others fail. 2019-01-30 22:20:10 +00:00
kyle
64ae689f21 2019-01-29 16:26:06 +00:00
yarikoptic
76652f0907 Added a comment 2019-01-26 00:46:04 +00:00