Commit graph

29098 commits

Author SHA1 Message Date
Joey Hess
6280af2901
generate more compact git-annex branch for imports
Especially from borg, where the content identifier logs
all end up being the same identical file!

But also, for other imports, the location tracking logs can,
in some cases, be identical files.

Bonus optimisation: Avoid looking up (and parsing when set)
GIT_ANNEX_VECTOR_CLOCK env var every time a log is written to.
Although the lookup does happen at startup even when no
log will be written now.
2020-12-23 15:25:16 -04:00
Joey Hess
7916fc98a3
graft in imported tree to avoid gc
Fix a bug that could prevent getting files from an importtree=yes remote,
because the imported tree was allowed to be garbage collected.
2020-12-23 14:27:38 -04:00
Joey Hess
c6e693b25d
remove ContentIndentifiersCidRemoteIndex uniqueness constraint
For reasons explained in the bug report.

Implemented using a persistent migration, which works fine. It may add a
little startup overhead when a remote is enabled that uses this, but
probably un-noticable.

On the next major version, it would be fine to delete this database,
and regenerate it from the git-annex branch information. Then this
change could be reverted.

Did nothing about adding back the data that got dropped from the db
due to the bug. Only the borg special remote was probably affected,
and it's not been released yet. rm -rf .git/annex/cidsdb does work.
2020-12-23 14:03:33 -04:00
Joey Hess
b370e6b0ad
bug 2020-12-23 13:41:02 -04:00
Joey Hess
e3d356fe84
borg: add subdir= config
Note that, after changing it with enableremote, syncing won't rescan
known archives in the borg repo using the changed config. Probably not a
problem?

Also used File in some places where filenames that could theoretically
start with - are passed to borg, to avoid it confusing them with
options.
2020-12-23 13:12:11 -04:00
Joey Hess
1574972ba9
make sync --content get from third-party populated remotes like borg 2020-12-23 12:10:39 -04:00
Joey Hess
d239a55bd1
Merge branch 'master' of ssh://git-annex.branchable.com 2020-12-22 16:55:15 -04:00
Joey Hess
91624f918d
devblog 2020-12-22 16:53:40 -04:00
Joey Hess
79729bcd76
todo 2020-12-22 16:37:19 -04:00
Joey Hess
a2fe994ebb
move unimplemented option to todo 2020-12-22 16:28:13 -04:00
Joey Hess
cd4c68924b
merged borg
Still a couple related todos, but it's basically usable now.
2020-12-22 16:22:44 -04:00
Joey Hess
310d3c3823
Merge branch 'borg' 2020-12-22 16:19:32 -04:00
Joey Hess
2335476e1e
todo 2020-12-22 16:19:02 -04:00
Joey Hess
4254e2297d
implement retrieveExportWithContentIdentifier
Moved out an XXX to a todo

This seems about ready to merge..
2020-12-22 16:16:48 -04:00
Joey Hess
f31bdd0b19
todo 2020-12-22 15:01:07 -04:00
Joey Hess
82e43da936
todo 2020-12-22 15:00:11 -04:00
Joey Hess
4f9969d0a1
optimisation for borg
Skip needing to list importable contents when unchanged since last time.
2020-12-22 15:00:05 -04:00
Joey Hess
c2d6f335a6
notes on ImportableContents history not being used for retrieval 2020-12-22 11:24:11 -04:00
eric.w@eee65cd362d995ced72640c7cfae388ae93a4234
ce740fb22e 2020-12-21 20:59:21 +00:00
jkniiv
21302c00ef Added a comment: well done! 2020-12-21 18:14:26 +00:00
amerlyq+annex@12121d70a602f094228689a0a24d348d478a8af1
825ff92a40 Added a comment: "mirror" workflow: usage of "wanted" expression is inconsistent with regular annex remote 2020-12-21 07:34:28 +00:00
amerlyq+annex@12121d70a602f094228689a0a24d348d478a8af1
f0490ebad1 Added a comment: adb special remote 2020-12-21 06:36:33 +00:00
Joey Hess
bd7b85371f
bug report 2020-12-20 16:27:32 -04:00
dzhu
9fffe46b0d 2020-12-19 23:05:32 +00:00
Joey Hess
3207e8293b
start borg special remote
Compiles, but unusable so far.
2020-12-18 16:03:51 -04:00
Joey Hess
909318dcee
Merge branch 'master' into borg 2020-12-18 15:27:24 -04:00
Joey Hess
e998320318
Merge branch 'master' of ssh://git-annex.branchable.com 2020-12-18 15:14:10 -04:00
Joey Hess
f62aee0525
fix handling of importtree-only remotes
Don't want to try to use these remotes as key/value remotes, which will
surely fail. It only recently became possible for importtree to be set
w/o exporttree, so before this code was ok.

(cherry picked from commit 97599cb0f7f4115aa5a3e81a91ee3d1d6c52dc84)
2020-12-18 15:13:30 -04:00
Joey Hess
037f8b6863
update 2020-12-18 11:06:23 -04:00
Ilya_Shlyakhter
738d919df3 Added a comment: encryption=onlycreds 2020-12-17 21:01:33 +00:00
Joey Hess
f0a495fa05
Merge branch 'master' into borg 2020-12-17 16:36:15 -04:00
Joey Hess
e5ef8aea9a
Merge branch 'master' of ssh://git-annex.branchable.com 2020-12-17 16:35:10 -04:00
Joey Hess
4c63cab467
todo 2020-12-17 16:30:51 -04:00
kyle
48da7e002d Added a comment 2020-12-17 19:05:13 +00:00
dscheffy@c203b7661ec8c1ebd53e52627c84536c5f0c9026
c53057caa3 Added a comment 2020-12-17 18:14:16 +00:00
kyle
49c4d471ff Added a comment 2020-12-17 17:41:36 +00:00
Joey Hess
ef8c36254a
docs for borg special remote
(which DNE yet)
2020-12-17 13:12:35 -04:00
Joey Hess
e9af56fef1
typo 2020-12-17 12:53:47 -04:00
Joey Hess
53fd1564b1
improve synopsis 2020-12-17 12:51:49 -04:00
Joey Hess
7c7486a45f
response 2020-12-17 12:47:07 -04:00
Joey Hess
c52550a6a8
Merge branch 'master' of ssh://git-annex.branchable.com 2020-12-17 12:45:07 -04:00
Joey Hess
00352ebe37
man page improvement 2020-12-17 12:17:58 -04:00
kyle
54dc20fb74 Added a comment 2020-12-17 16:06:44 +00:00
dscheffy@c203b7661ec8c1ebd53e52627c84536c5f0c9026
3f95fa2a3a Added a comment: How do I man annex config options? 2020-12-17 10:39:40 +00:00
dscheffy@c203b7661ec8c1ebd53e52627c84536c5f0c9026
a2d5fd0aa6 2020-12-16 18:53:21 +00:00
dscheffy@c203b7661ec8c1ebd53e52627c84536c5f0c9026
05ee618d11 Added a comment: Duplicate content creates frustrating cycles 2020-12-16 17:10:52 +00:00
jwodder
cf43362cae Add post metadata 2020-12-16 16:03:55 +00:00
jwodder
ad180cedc7 2020-12-16 16:03:04 +00:00
Joey Hess
2abda21123
update 2020-12-15 16:35:06 -04:00
Joey Hess
117d270bb4
comment 2020-12-15 16:34:16 -04:00