106 lines
5 KiB
Markdown
106 lines
5 KiB
Markdown
git-remote-annex will be a program that allows push/pull/clone of a git
|
|
repository to many types of git-annex special remote.
|
|
|
|
This is a redesign and reimplementation of git-remote-datalad-annex.
|
|
It will be a safer implementation, will support incremental pushes, and
|
|
will be available to users who don't use datalad.
|
|
--[[Joey]]
|
|
|
|
---
|
|
|
|
This is implememented and working. Remaining todo list for it:
|
|
|
|
* Bug: Problem with forced push:
|
|
|
|
joey@darkstar:~/tmp/bench5/a#ook>git push d ook --force
|
|
fatal: bad revision 'refs/namespaces/git-remote-annex/d5a263c6-1c28-432a-a161-914476ae5390/refs/heads/git-annex'
|
|
Push failed (user error (git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","bundle","create","--quiet","/home/joey/tmp/GITBUNDLE2690124-1","--stdin"] exited 128))
|
|
warning: helper reported unexpected status of push
|
|
Everything up-to-date
|
|
|
|
This was preceeded by pushing the git-annex branch and master,
|
|
then making 3 commits and pushing each of them in turn.
|
|
Then reset back one commit, try to push (which fails as
|
|
non-fast-forward), and force push as shown then fails.
|
|
|
|
So the problem is not the forced push itself, which works
|
|
if a non-forced push is not tried before it, but something
|
|
with that specific situation.
|
|
|
|
* Cloning from an annex:: url with importtree=yes doesn't work
|
|
(with or without exporttree=yes). This is because the ContentIdentifier
|
|
db is not populated.
|
|
|
|
* It would be nice if git-annex could generate an annex:: url
|
|
for a special remote and show it to the user, eg when
|
|
they have set the shorthand "annex::" url, so they know the full url.
|
|
`git-annex info $remote` could also display it.
|
|
Currently, the user has to remember how the special remote was
|
|
configured and replicate it all in the url.
|
|
|
|
There are some difficulties to doing this, including that
|
|
RemoteConfig can have hidden fields that should be omitted.
|
|
|
|
* initremote/enableremote could have an option that configures the url to a
|
|
special remote to a annex:: url. This would make it easier to use
|
|
git-remote-annex, since the user would not need to set up the url
|
|
themselves. (Also it would then avoid setting `skipFetchAll = true`)
|
|
|
|
* Improve recovery from interrupted push by using outManifest to clean up
|
|
after it. (Requires populating outManifest.)
|
|
|
|
* See XXX in uploadManifest about recovering from a situation
|
|
where the remote is left with a deleted manifest when a push
|
|
is interrupted part way through. This should be recoverable
|
|
by caching the manifest locally and re-uploading it when
|
|
the remote has no manifest or prompting the user to merge and re-push.
|
|
|
|
* datalad-annex supports cloning from the web special remote,
|
|
using an url that contains the result of pushing to eg, a directory
|
|
special remote.
|
|
`datalad-annex::https://example.com?type=web&url={noquery}`
|
|
Supporting something like this would be good.
|
|
|
|
* Improve behavior in push races. A race can overwrite a change
|
|
to the MANIFEST and lose work that was pushed from the other repo.
|
|
From the user's perspective, that situation is the same as if one repo
|
|
pushed new work, then the other repo did a git push --force, overwriting
|
|
the first repo's push. In the first repo, another push will then fail as
|
|
a non fast-forward, and the user can recover as usual. This is probably
|
|
okish.
|
|
|
|
But.. a MANIFEST overwrite will leave bundle files in the remote that
|
|
are not listed in the MANIFEST. It seems likely that git-annex could
|
|
detect that after the fact and clean it up. Eg, if it caches
|
|
the last MANIFEST it uploaded, next time it downloads the MANIFEST
|
|
it can check if there are bundle files in the old one that are not
|
|
in the new one. If so, it can drop those bundle files from the remote.
|
|
|
|
* A push race can also appear to the user as if they pushed a ref, but then
|
|
it got deleted from the remote. This happens when two pushes are
|
|
pushing different ref names. This might be harder for the user to
|
|
notice; git fetch does not indicate that a remote ref got deleted.
|
|
They would have to use git fetch --prune to notice the deletion.
|
|
Once the user does notice, they can re-push their ref to recover.
|
|
Can this be improved?
|
|
|
|
* The race condition described in
|
|
[[!commit 797f27ab0517e0021363791ff269300f2ba095a5]]
|
|
where before git-annex init is run in a repo,
|
|
using git-remote-annex and at the same time git-annex init can lose
|
|
changes that the latter command (and ones after it) write to the
|
|
git-annex branch.
|
|
|
|
This should be fixable by making git-remote-annex not write to the
|
|
git-annex branch, but to eg, a temporary journal directory.
|
|
|
|
Also, cloning currently writes the special remote config into remote.log,
|
|
which might be slightly different in some way than the config in
|
|
remote.log for the same remote. cloning should not change the stored
|
|
config of a remote, but that branch write was necessary. So throwing
|
|
away the branch write is also good for this case.
|
|
|
|
Also, when the remote uses importtree=yes, pushing to it updates
|
|
content identifiers, which currently get recorded in the git-annex
|
|
branch. It would be good to avoid that being written as well.
|
|
|