Commit graph

2519 commits

Author SHA1 Message Date
Joey Hess
0a14dfd383
comment 2019-04-09 11:08:18 -04:00
Joey Hess
4af55c1f30
Merge branch 'master' of ssh://git-annex.branchable.com 2019-04-05 11:41:46 -04:00
yarikoptic
7db8eaf512 initial question about possible "globus" special remote 2019-04-05 02:25:12 +00:00
Joey Hess
1f3245ddf5
close as basis of this is wrong 2019-04-04 12:50:55 -04:00
Joey Hess
727ac0451a
comment 2019-04-03 13:14:54 -04:00
Joey Hess
bc302b56ae
test patch 2019-03-28 16:16:28 -04:00
Joey Hess
c68ae14268
further thought 2019-03-28 15:46:14 -04:00
Joey Hess
b09c6e3016
todo item based on behavior yoh showed me 2019-03-28 14:04:20 -04:00
Joey Hess
e035bc5324
minor typos 2019-03-27 11:15:20 -04:00
Ilya_Shlyakhter
1c334f74d6 Added a comment 2019-03-26 18:27:04 +00:00
Ilya_Shlyakhter
9b4e06d8c2 fixed a typo 2019-03-26 17:51:19 +00:00
Ilya_Shlyakhter
38669f0817 Added a comment: simplifying the interface 2019-03-26 17:40:33 +00:00
Ilya_Shlyakhter
72b788dfaf re: documenting git-annex dependencies 2019-03-24 18:47:28 +00:00
Joey Hess
9ada4b38c1
comment 2019-03-22 10:30:22 -04:00
Joey Hess
59c8119b2a
comment 2019-03-22 10:18:07 -04:00
Joey Hess
5fea7efee7
comment and toddo 2019-03-22 09:23:31 -04:00
Ilya_Shlyakhter
438ff50013 Added a comment 2019-03-19 20:40:50 +00:00
Joey Hess
caa5cca47f
Merge branch 'master' of ssh://git-annex.branchable.com 2019-03-19 15:57:19 -04:00
Ilya_Shlyakhter
10291efa55 added suggestion for git-annex-test to do simple sanity checks on external utils before running tests that depend on them 2019-03-19 18:55:59 +00:00
Joey Hess
2118006711
comment 2019-03-19 13:53:32 -04:00
Joey Hess
1c2f6aee11
comment 2019-03-19 13:45:28 -04:00
Joey Hess
738325ffb7
comment 2019-03-19 13:40:46 -04:00
Joey Hess
cce81f9b0c
comment 2019-03-19 13:34:45 -04:00
Joey Hess
059ff44e8b
comment 2019-03-19 13:28:08 -04:00
Ilya_Shlyakhter
56fd24de0d added suggestion for importing files from special remotes without downloading them 2019-03-12 17:18:28 +00:00
Joey Hess
91f0f76252
need to write a tip
deferred because a tip that is limited to the directory special remote
is not very interesting to most
2019-03-11 14:30:05 -04:00
Joey Hess
ffec06aaec
update 2019-03-11 14:13:58 -04:00
Joey Hess
057999f0fc
fix sync --content with remote.name.annex-tracking-branch=master:subdir
It was exporting the whole tree not just the subdir. Now tested fully
working in both directions.
2019-03-11 14:07:52 -04:00
Joey Hess
e1fdc8b374
record only subtree in export db and log after importing a subtree 2019-03-11 13:45:01 -04:00
Joey Hess
e46e40bf05
fix update of export tracking branch when exporting branch:subdir 2019-03-11 13:44:12 -04:00
Joey Hess
2912429640
better indicate when special remotes do not support renameExport
Avoid a warning message when renameExport is not supported, and just
fallback to deleting with a subsequent re-upload. Especially needed for
importtree remotes, where renameExport needs to be disabled.

This changes the external special remote protocol, but in a
backwards-compatible way. A reply of UNSUPPORTED-REQUEST to an older
version of git-annex will cause it to make renameExport return False.
2019-03-11 12:53:24 -04:00
Joey Hess
c755788256
sync: import when annex-tracking-branch is configured
This works, and tested syncing both gets changes from a special remote
and sends changes to it, keeping it fully in sync nicely!

But have not tried it with a subdir configured.
2019-03-09 13:57:49 -04:00
Joey Hess
6149a3b9bd
added progress bars
clean up old todo item I checked earlier, see commit
dec30d2b14
2019-03-08 12:43:03 -04:00
Joey Hess
e412129523
concurrency and status messages when downloading from import 2019-03-08 12:33:44 -04:00
Joey Hess
ee5f1422df
remove debug print 2019-03-07 16:08:58 -04:00
Joey Hess
7e35c81ada
locking problem 2019-03-07 15:22:23 -04:00
Joey Hess
9a72785307
fixes to export db lookup when accessing importtree=yes
Now in a fresh clone with a importtree=yes remote enabled,
git annex fsck --from the remote works.
2019-03-07 14:10:56 -04:00
Joey Hess
93025dd59f
add missing locking of ContentIdentifier database when writing
This is not super efficient; it would be better to lock the database
once and build up a queue of changes and flush once.

But, storeExportWithContentIdentifier is likely going to be the really
expensive part, so let's do the simple thing and only optimise later if
needed.
2019-03-07 13:32:33 -04:00
Joey Hess
3f449f845e
update 2019-03-07 13:28:18 -04:00
Joey Hess
68d1661251
cross-repo import now working correctly 2019-03-07 12:31:35 -04:00
Joey Hess
ee251b2e2e
implement updating the ContentIdentifier db with info from the git-annex branch
untested

This won't be super slow, but it does need to diff two likely large
trees, and since the git-annex branch rarely sits still, it will most
likely be run at the beginning of every import.

A possible speed improvement would be to only run this when the database
did not contain a ContentIdentifier. But that would only speed up
imports when there is no new version of a file on the special remote,
at most renames of existing files being imported.

A better speed improvement would be to record something in the git-annex
branch that indicates when an import has been run, and only do the diff
if the git-annex branch has record of a newer import than we've seen
before. Then, it would only run when there is in fact new
ContentIdentifier information available from a remote. Certianly doable,
but didn't want to complicate things yet.
2019-03-06 18:04:30 -04:00
Joey Hess
dec30d2b14
updates
Note that I tried an evil remote that lists ImportLocations with
../../../ in them and indeed this resulted in git blowing up and the
import failing, and not writing outside the repo.
2019-03-06 17:07:36 -04:00
Joey Hess
8e9713b769
add export+import test case 2019-03-06 16:49:33 -04:00
Joey Hess
b3d30e7d70
remove unncessary locking of ContentIdentifier db
Remote.Helper.ExportImport only reads from it, and locking is only
needed when writing.
2019-03-06 14:36:57 -04:00
Joey Hess
f957f64278
add todo 2019-03-06 11:24:06 -04:00
Joey Hess
0db393d82f
add bug 2019-03-05 17:19:26 -04:00
Joey Hess
5767b1b00d
avoid updating tracking branch when transfer to export throws exception 2019-03-05 16:51:13 -04:00
Joey Hess
dc278c059c
fix STM crash
git-annex: thread blocked indefinitely in an STM transaction
failed

git-annex: sqlite query crashed
CallStack (from HasCallStack):
  error, called at ./Database/Handle.hs:98:42 in main:Database.Handle
failed

This needs further investigation.
2019-03-05 16:37:40 -04:00
Joey Hess
46d33e804a
added checkPresentExportWithContentIdentifier
Ugh, don't like needing to add this, but I can't see a way around it.
2019-03-05 16:03:03 -04:00
Joey Hess
fd2a1aaa17
avoid using renameExport on import remotes 2019-03-05 14:57:48 -04:00