Added a comment: progress?
This commit is contained in:
parent
bb9580ba66
commit
2c1a0af295
1 changed files with 65 additions and 0 deletions
|
@ -0,0 +1,65 @@
|
||||||
|
[[!comment format=mdwn
|
||||||
|
username="anarcat"
|
||||||
|
avatar="http://cdn.libravatar.org/avatar/4ad594c1e13211c1ad9edb81ce5110b7"
|
||||||
|
subject="progress?"
|
||||||
|
date="2018-11-27T06:47:26Z"
|
||||||
|
content="""
|
||||||
|
How's that remote going, RonnyPfannschmidt? :) I can't tell from the [homepage](https://github.com/RonnyPfannschmidt/git-annex-borg/) but from the source code, it looks like initremote is supported so far, but not much else...
|
||||||
|
|
||||||
|
From what I remember, borg supports storing arbitrary blobs with the `borg debug-put-obj` function, and retrieve one with `borg debug-get-obj`. Here's an example of how this could work:
|
||||||
|
|
||||||
|
[1145]anarcat@angela:test$ sha256sum /etc/motd
|
||||||
|
a378977155fb42bb006496321cbe31f74cbda803c3f6ca590f30e76d1afad921 /etc/motd
|
||||||
|
[1146]anarcat@angela:test$ borg init -e none repo
|
||||||
|
[1147]anarcat@angela:test$ borg debug-put-obj repo /etc/motd
|
||||||
|
object a378977155fb42bb006496321cbe31f74cbda803c3f6ca590f30e76d1afad921 put.
|
||||||
|
[1148]anarcat@angela:test$ borg debug-get-obj repo a378977155fb42bb006496321cbe31f74cbda803c3f6ca590f30e76d1afad921 tmp
|
||||||
|
object a378977155fb42bb006496321cbe31f74cbda803c3f6ca590f30e76d1afad921 fetched.
|
||||||
|
[1149]anarcat@angela:test$ sha256sum tmp
|
||||||
|
a378977155fb42bb006496321cbe31f74cbda803c3f6ca590f30e76d1afad921 tmp
|
||||||
|
|
||||||
|
This assumes the underlying blob ID in borg is a SHA256 hash, but that
|
||||||
|
seems like a fair assumption to make. Naturally, this could cause
|
||||||
|
problems with git-annex, which supports multiple hashing algorithms
|
||||||
|
thanks to the multiple [[backends]] support. But maybe this can just
|
||||||
|
work this out by refusing to store non-matchin backends.
|
||||||
|
|
||||||
|
That is, if borg actually worked that way. Unfortunately, while the
|
||||||
|
above actually works, the resulting repository is not quite right:
|
||||||
|
|
||||||
|
$ borg debug dump-repo-objs .
|
||||||
|
Dumping 000000_0000000000000000000000000000000000000000000000000000000000000000.obj
|
||||||
|
Data integrity error: Chunk a378977155fb42bb006496321cbe31f74cbda803c3f6ca590f30e76d1afad921: Invalid encryption envelope
|
||||||
|
|
||||||
|
So borg does not like the repository at all... I'm not sure why, but
|
||||||
|
it sure looks like borg \"objects\" are not as transparent as I
|
||||||
|
hoped and that this low-level interface will not be suitable for
|
||||||
|
git-annex.
|
||||||
|
|
||||||
|
The higher level interface is \"archives\", which have (more or less) a
|
||||||
|
CRUD interface (without the U, really) through the
|
||||||
|
\"create/list/extract/prune\" interface. It's far from what we need:
|
||||||
|
items are deplicated across archives so it means it is impossible to
|
||||||
|
reliably delete a key unless we walk (and modify!) the entire archive list, which is
|
||||||
|
slow and impractical. But it *could* definitely be used to add keys to
|
||||||
|
a repository, using:
|
||||||
|
|
||||||
|
$ time borg create --stdin-name SHA256-a378977155fb42bb006496321cbe31f74cbda803c3f6ca590f30e76d1afad921 .::'{utcnow}' - < /etc/motd
|
||||||
|
1.30user 0.10system 0:01.62elapsed 86%CPU (0avgtext+0avgdata 81464maxresident)k
|
||||||
|
72inputs+1496outputs (0major+31135minor)pagefaults 0swaps
|
||||||
|
|
||||||
|
As you can see, however, that is *slow* (although arguably not slower
|
||||||
|
than `debug-put-obj` which is surprising).
|
||||||
|
|
||||||
|
But even worse, that blob is now hidden behind that archive - you'd
|
||||||
|
need to list all archives (which is also expensive) to find it.
|
||||||
|
|
||||||
|
So I hit a dead end so I'm curious to hear how you were planning to
|
||||||
|
implement this, Ronny. :) Presumably there should be a way to generate
|
||||||
|
an object compatible with `debug-put-obj`, but that interface seems
|
||||||
|
very brittle and has all sorts of warnings all around it... And on the
|
||||||
|
other hand, the archive interface is clunky and slow... I wish there
|
||||||
|
was a better way, and suspect it might be worth talking with upstream
|
||||||
|
(which I'm not anymore) to see if there's a better way to work this
|
||||||
|
problem. -- [[anarcat]]
|
||||||
|
"""]]
|
Loading…
Reference in a new issue