Added a comment: progress?
This commit is contained in:
parent
bb9580ba66
commit
2c1a0af295
1 changed files with 65 additions and 0 deletions
|
@ -0,0 +1,65 @@
|
|||
[[!comment format=mdwn
|
||||
username="anarcat"
|
||||
avatar="http://cdn.libravatar.org/avatar/4ad594c1e13211c1ad9edb81ce5110b7"
|
||||
subject="progress?"
|
||||
date="2018-11-27T06:47:26Z"
|
||||
content="""
|
||||
How's that remote going, RonnyPfannschmidt? :) I can't tell from the [homepage](https://github.com/RonnyPfannschmidt/git-annex-borg/) but from the source code, it looks like initremote is supported so far, but not much else...
|
||||
|
||||
From what I remember, borg supports storing arbitrary blobs with the `borg debug-put-obj` function, and retrieve one with `borg debug-get-obj`. Here's an example of how this could work:
|
||||
|
||||
[1145]anarcat@angela:test$ sha256sum /etc/motd
|
||||
a378977155fb42bb006496321cbe31f74cbda803c3f6ca590f30e76d1afad921 /etc/motd
|
||||
[1146]anarcat@angela:test$ borg init -e none repo
|
||||
[1147]anarcat@angela:test$ borg debug-put-obj repo /etc/motd
|
||||
object a378977155fb42bb006496321cbe31f74cbda803c3f6ca590f30e76d1afad921 put.
|
||||
[1148]anarcat@angela:test$ borg debug-get-obj repo a378977155fb42bb006496321cbe31f74cbda803c3f6ca590f30e76d1afad921 tmp
|
||||
object a378977155fb42bb006496321cbe31f74cbda803c3f6ca590f30e76d1afad921 fetched.
|
||||
[1149]anarcat@angela:test$ sha256sum tmp
|
||||
a378977155fb42bb006496321cbe31f74cbda803c3f6ca590f30e76d1afad921 tmp
|
||||
|
||||
This assumes the underlying blob ID in borg is a SHA256 hash, but that
|
||||
seems like a fair assumption to make. Naturally, this could cause
|
||||
problems with git-annex, which supports multiple hashing algorithms
|
||||
thanks to the multiple [[backends]] support. But maybe this can just
|
||||
work this out by refusing to store non-matchin backends.
|
||||
|
||||
That is, if borg actually worked that way. Unfortunately, while the
|
||||
above actually works, the resulting repository is not quite right:
|
||||
|
||||
$ borg debug dump-repo-objs .
|
||||
Dumping 000000_0000000000000000000000000000000000000000000000000000000000000000.obj
|
||||
Data integrity error: Chunk a378977155fb42bb006496321cbe31f74cbda803c3f6ca590f30e76d1afad921: Invalid encryption envelope
|
||||
|
||||
So borg does not like the repository at all... I'm not sure why, but
|
||||
it sure looks like borg \"objects\" are not as transparent as I
|
||||
hoped and that this low-level interface will not be suitable for
|
||||
git-annex.
|
||||
|
||||
The higher level interface is \"archives\", which have (more or less) a
|
||||
CRUD interface (without the U, really) through the
|
||||
\"create/list/extract/prune\" interface. It's far from what we need:
|
||||
items are deplicated across archives so it means it is impossible to
|
||||
reliably delete a key unless we walk (and modify!) the entire archive list, which is
|
||||
slow and impractical. But it *could* definitely be used to add keys to
|
||||
a repository, using:
|
||||
|
||||
$ time borg create --stdin-name SHA256-a378977155fb42bb006496321cbe31f74cbda803c3f6ca590f30e76d1afad921 .::'{utcnow}' - < /etc/motd
|
||||
1.30user 0.10system 0:01.62elapsed 86%CPU (0avgtext+0avgdata 81464maxresident)k
|
||||
72inputs+1496outputs (0major+31135minor)pagefaults 0swaps
|
||||
|
||||
As you can see, however, that is *slow* (although arguably not slower
|
||||
than `debug-put-obj` which is surprising).
|
||||
|
||||
But even worse, that blob is now hidden behind that archive - you'd
|
||||
need to list all archives (which is also expensive) to find it.
|
||||
|
||||
So I hit a dead end so I'm curious to hear how you were planning to
|
||||
implement this, Ronny. :) Presumably there should be a way to generate
|
||||
an object compatible with `debug-put-obj`, but that interface seems
|
||||
very brittle and has all sorts of warnings all around it... And on the
|
||||
other hand, the archive interface is clunky and slow... I wish there
|
||||
was a better way, and suspect it might be worth talking with upstream
|
||||
(which I'm not anymore) to see if there's a better way to work this
|
||||
problem. -- [[anarcat]]
|
||||
"""]]
|
Loading…
Reference in a new issue