Merge branch 'master' of ssh://git-annex.branchable.com
This commit is contained in:
commit
e444fb5d84
6 changed files with 158 additions and 0 deletions
|
@ -0,0 +1,56 @@
|
|||
### Please describe the problem.
|
||||
|
||||
git-annex failed to update the local adjusted branch after `git annex drop` stopped abruptly. In turn, `git annex drop` stopped abruptly because a `git annex copy --from X --to Y` had to be stopped (CTRL-C) because of lack of space.
|
||||
|
||||
(Perhaps there are two issues here: `drop` being unable to cope with the unexpected situation, and the adjustment code being unable to cope with the weird stat left by `drop`'s abrupt interruption.)
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
```
|
||||
$ git copy -J4 --from X --to Y
|
||||
[notice that git-annex starts saying "not enough free space, need 27.87 MB more"]
|
||||
[kill process with CTRL-C]
|
||||
$ git annex drop dirF/
|
||||
drop dirF/a/a1.txt ok
|
||||
drop dirF/a/a2.txt ok
|
||||
[...]
|
||||
error: Your local changes to the following files would be overwritten by checkout:
|
||||
dirF/a/a1.txt
|
||||
dirF/a/a2.txt
|
||||
[...]
|
||||
|
||||
Aborting
|
||||
|
||||
Updating adjusted branch failed.
|
||||
(recording state in git...)
|
||||
$ git status
|
||||
HEAD detached at 9d92415fb
|
||||
nothing to commit, working tree clean
|
||||
$ git annex status
|
||||
$ git branch
|
||||
* (HEAD detached at 9d92415fb)
|
||||
adjusted/master(unlocked)
|
||||
adjusted/master(unlockpresent)
|
||||
git-annex
|
||||
master
|
||||
synced/master
|
||||
```
|
||||
|
||||
`dirF/` is the directory whose files were being copied when the process has been stopped with `CTRL-C`.
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
```
|
||||
git-annex version: 10.20230215-gd24914f2a
|
||||
[...]
|
||||
operating system: linux x86_64
|
||||
supported repository versions: 8 9 10
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10
|
||||
local repository version: 8
|
||||
```
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
git-annex is too good. It so rarely causes problems that one does not develop the "git-annex troubleshooting muscle". :)
|
||||
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="hurlebouc"
|
||||
avatar="http://cdn.libravatar.org/avatar/bda734a6d937c1fe0c9778a6eaefffbc"
|
||||
subject="comment 8"
|
||||
date="2023-03-22T05:47:16Z"
|
||||
content="""
|
||||
Thank you very much for the analysis! This reinforces my conviction that git-annex is a very high quality project.
|
||||
"""]]
|
38
doc/bugs/gcrypt_remotes_using_relative_paths.mdwn
Normal file
38
doc/bugs/gcrypt_remotes_using_relative_paths.mdwn
Normal file
|
@ -0,0 +1,38 @@
|
|||
### Please describe the problem.
|
||||
`git annex sync` is not automatically run for gcrypt remotes using rsync with a relative path
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
Flow 1 (relative path, broken)
|
||||
|
||||
* `git remote add test gcrypt::rsync://user@user.rsync.net:relative/path/to/repo`
|
||||
* `git annex sync` -> DOES NOT SYNC to test remote
|
||||
* Nothing has been synced so I CANNOT successfully clone from the test remote with `git clone gcrypt::rsync://user@user.rsync.net:relative/path/to/repo`
|
||||
* `git push test git-annex master`
|
||||
* I can successfully clone from the test remote with `git clone gcrypt::rsync://user@user.rsync.net:relative/path/to/repo`
|
||||
|
||||
Flow 2 (absolute path, working)
|
||||
|
||||
* `git remote add test gcrypt::rsync://user@user.rsync.net/full/path/to/repo`
|
||||
* `git annex sync` -> DOES SYNC to test remote
|
||||
* I can successfully clone from the test remote with `git clone gcrypt::rsync://user@user.rsync.net:relative/path/to/repo`
|
||||
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
* Debian 11
|
||||
* git-annex version: 10.20230227
|
||||
* git-remote-gcrypt version 1.5
|
||||
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
I am VERY happy with git annex and am using it successfully with a gcrypt remote using an absolute path :)
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
Hello. I am a kinda-new git-annex user. I think git-annex is very difficult to understand. Thus, in this topic, I would like to write out my explanation of how git-annex works and how I can work with it, so that you can tell me if I am misunderstanding something.
|
||||
|
||||
I am a machine learning engineer. I use git-annex to store code together data (large images) and artefacts (large images and model weights). Basically, I am trying to use git-annex as normal git, but with the ability to store large binary files. There is a central bare repository and a bunch of non-bare repositories cloned from it on different computers. Usually, the non-bare repositories connect only to the central bare repository, but sometimes I push/fetch between non-bare repositories directly.
|
||||
|
||||
I can fetch/push/merge/pull the normal branches I work in just as I do with normal git (without annex). However, the git-annexed files aren't fetched/pushed this way because the branch actually contains only symlinks, not the files themselves. I have disabled all the automatic mergin, pulling, etc. functionality of git because I like to have total control. So, I merge or rebase everything manually when there is a need for it. And I never use pull.
|
||||
|
||||
In each repository, there is a branch called git-annex. It contains some metadata that git-annex uses. While in a repository repo1, I can do `git annex sync --only-annex --no-content --no-commit --no-pull --no-push --no-resolvemerge repo2` and git-annex will use some magic (consisting of pushing, pulling, and some kind of automatic merging) to sync the git-annex branch of repo1 and repo2, i.e. it will make them contain the same metadata. The options `--no-commit --no-pull --no-push --no-resolvemerge` are needed to disable the dark magic that is useful for casual users but not for software developers who use git-annex as a git addon. The option `--only-annex` prevents git-annex from creating "synced" branches which are, as far as I understand, another piece of dark magic useful for casual users but not for software developers. If I want, I can remove the `--no-content` flag and git-annex will also download and/or upload the annexed data (does it affect only the data available in the current branch? or is it all data? i'm not sure). This is the only command I need to know to sync the git-annex branch. Supposedly, it's possible to do via normal git fetches, pushes, merges, and maybe pulls, but I don't know how to do that.
|
||||
|
||||
The actual annexed data is stored somewhere in the .git directory. I don't need to worry where. What I need to know is that I can use `git annex copy`, `git annex get`, and `git annex sync --only-annex --no-commit --no-pull --no-push --no-resolvemerge otherrepo` with appropriate paths to copy the annexed data between repositories.
|
||||
|
||||
Ok, so I can use `git annex sync` with a bunch of flags to sync the git-annex branch, I can use `git annex sync`, `git annex get`, `git annex copy` to copy the data around, and I don't need the synced branches. Is my understanding correct?
|
|
@ -0,0 +1,28 @@
|
|||
Due to the issue reported in [[bugs/Failed_adjusted_branch_update_after_error_in_drop]], my git-annex repo is now is a limbo state:
|
||||
|
||||
|
||||
```
|
||||
$ git branch --show-current
|
||||
adjusted/master(unlockpresent)
|
||||
|
||||
$ git annex drop dirF/
|
||||
drop dirF/a/a1.txt ok
|
||||
drop dirF/a/a2.txt ok
|
||||
[...]
|
||||
error: Your local changes to the following files would be overwritten by checkout:
|
||||
dirF/a/a1.txt
|
||||
dirF/a/a2.txt
|
||||
[...]
|
||||
|
||||
Aborting
|
||||
|
||||
Updating adjusted branch failed.
|
||||
(recording state in git...)
|
||||
$ git status
|
||||
HEAD detached at 9d92415fb
|
||||
nothing to commit, working tree clean
|
||||
```
|
||||
|
||||
What is the recommended course of action to (safely) return to the `adjusted/master(unlockpresent)` branch?
|
||||
|
||||
Is it safe to just use `git annex adjust --unlock-present`?
|
|
@ -0,0 +1,17 @@
|
|||
[[!comment format=mdwn
|
||||
username="carlos@4c213b52601d57b650b22d9a246c59aea2c8f859"
|
||||
nickname="carlos"
|
||||
avatar="http://cdn.libravatar.org/avatar/ef0152bec3818b24d9318f6e7013e104"
|
||||
subject="comment 2"
|
||||
date="2023-03-21T21:34:01Z"
|
||||
content="""
|
||||
Hi Joey,
|
||||
|
||||
This is my current effort to copy ~6.7 GB using Datalad (git annex with rclone+Gdrive in the background)
|
||||
|
||||
```
|
||||
Total: 53%|█████████████████████████████████████████████████████████████████▏ | 3.49G/6.53G [8:25:38<7:21:03, 115k Bytes/s]
|
||||
```
|
||||
|
||||
There are many small files in this repository for sure, but I haven't been able to get specific advice as to whether these very small speeds (even copying to an external drive took many hours) are expected given the bottlenecks you mention. Any feedback is welcome.
|
||||
"""]]
|
Loading…
Add table
Add a link
Reference in a new issue