Merge branch 'master' of ssh://git-annex.branchable.com
This commit is contained in:
Joey Hess
commit
08eb87fc47
8 changed files with 820 additions and 1 deletions
doc
bugs
SMB__58___git_annex_clone_works__44___get_fails_on_transfer_lock
add_fails_with_v6_repo_when_four_levels_deep
git-annex_in_nixpkgs_fails_with_git-2.13.0.mdwnglacier.mdwnsync_claims_data_loss_but_seems_to_just_lose_tracking.mdwnsync_claims_data_loss_but_seems_to_just_lose_tracking
forum
git-annex-adjust.mdwn|
|
@ -0,0 +1,86 @@
|
|||
[[!comment format=mdwn
|
||||
username="ewen"
|
||||
avatar="http://cdn.libravatar.org/avatar/605b2981cb52b4af268455dee7a4f64e"
|
||||
subject="dtruss OS X 10.11 git annex on SMB mount"
|
||||
date="2017-05-14T05:04:29Z"
|
||||
content="""
|
||||
A tar file with:
|
||||
|
||||
ewen@ashram:~/Desktop$ tar -tzf dtruss-git-annex-osx-smb-2017-05-14.tar.gz
|
||||
dtruss-git-annex-osx-smb-2017-05-14/
|
||||
dtruss-git-annex-osx-smb-2017-05-14/00-ga-test-smb-osx-symlinks-2017-05-14.script
|
||||
dtruss-git-annex-osx-smb-2017-05-14/01-ga-test-smb-osx-symlinks-dtruss-git-annex-init-2017-05-14.txt
|
||||
dtruss-git-annex-osx-smb-2017-05-14/02-ga-test-smb-osx-symlinks-dtruss-git-annex-sync-2017-05-14.txt
|
||||
dtruss-git-annex-osx-smb-2017-05-14/03-ga-test-smb-osx-symlinks-dtruss-git-annex-get-all-2017-05-14.txt
|
||||
dtruss-git-annex-osx-smb-2017-05-14/04-ga-test-smb-osx-symlinks-dtruss-git-annex-sync-other-way-2017-05-14.txt
|
||||
dtruss-git-annex-osx-smb-2017-05-14/05-ga-test-smb-osx-symlinks-dtruss-git-annex-copy-to-smb-2017-05-14.txt
|
||||
dtruss-git-annex-osx-smb-2017-05-14/dtruss-ga
|
||||
ewen@ashram:~/Desktop$
|
||||
|
||||
is available at [dtruss-git-annex-osx-smb-2017-05-14.tar.gz](http://ewen.mcneill.gen.nz/temp/dtruss-git-annex-osx-smb-2017-05-14.tar.gz) (at least until I next tidy up the `/temp/` directory :-) ).
|
||||
|
||||
The script file is the output of \"`script ....`\" showing all the commands (including typos :-) ) run, except the `dtruss` ones themselves. The various `0n-ga-tst-smb-osx-symlinks-dtruss...` files contain the output of \"`sudo dtruss -af PID`\" where the `PID` was printed out by the script `dtruss-ga` that is included in the archive. `dtruss-ga` was used to enable running git-annex as non-root (to ensure that the SMB access wasn't altered by running as a different user), since `dtruss` itself can only run as root. The script is just:
|
||||
|
||||
ewen@ashram:~/Desktop$ cat dtruss-git-annex-osx-smb-2017-05-14/dtruss-ga
|
||||
#! /bin/sh
|
||||
# Allow dtruss of non-root git annex
|
||||
#
|
||||
# Based on hints at:
|
||||
# http://sec.cs.ucl.ac.uk/users/smurdoch/blog/2015/08/fixing-slow-start-mc-on-osx.html
|
||||
#---------------------------------------------------------------------------
|
||||
|
||||
echo \"Run: sudo dtruss -af -p $$\"
|
||||
echo \"Then press enter\"
|
||||
read
|
||||
exec git annex \"$@\"
|
||||
ewen@ashram:~/Desktop$
|
||||
|
||||
so basically \"print PID, wait for go ahead, run git annex command\".
|
||||
|
||||
From a quick glance through, it looks like:
|
||||
|
||||
7003/0x16c3126: 121576 14 11 write(0x12, \"PidLock {lockingPid = 7003, lockingHost = \\"ashram\\"}\0\", 0x33) = 51 0
|
||||
7003/0x16c3126: 121775 3350 191 close(0x12) = 0 0
|
||||
7003/0x16c3126: 121792 10 6 stat64(\"/Volumes/music/ga-test/.git/annex/locktmp16807282475249\0\", 0x200004940, 0x33) = 0 0
|
||||
7003/0x16c3126: 122041 2341 235 link(\"/Volumes/music/ga-test/.git/annex/locktmp16807282475249\0\", \"/Volumes/music/ga-test/.git/annex/locktmp16807282475249.lnk\0\") = -1 Err#45
|
||||
7003/0x16c3126: 122115 885 49 unlink(\"/Volumes/music/ga-test/.git/annex/locktmp16807282475249.lnk\0\", 0x200004D80, 0x33) = -1 Err#2
|
||||
|
||||
(in `03-...`, the first `get`) probably isn't helping -- that's trying to do a *hardlink* on a SMB file system, and appearing to give up when it fails... (`/Volumes/music` is the SMB mount I was using for testing).
|
||||
|
||||
Also since I didn't realise until after the `script` file/`tar` file were made, there *is* an `annex/pidlock` file:
|
||||
|
||||
ewen@ashram:~$ ls -l /Volumes/music/ga-test/.git/annex/pidlock
|
||||
-rwx------ 1 ewen staff 51 14 May 16:25 /Volumes/music/ga-test/.git/annex/pidlock
|
||||
ewen@ashram:~$ cat /Volumes/music/ga-test/.git/annex/pidlock
|
||||
PidLock {lockingPid = 7143, lockingHost = \"ashram\"}ewen@ashram:~$
|
||||
ewen@ashram:~$
|
||||
|
||||
created within the SMB `git-annex`, which looks vaguely sensible. And FTR, there is *not* one in the HFS (local OS X) `git-annex`:
|
||||
|
||||
ewen@ashram:~$ ls -l /var/tmp/ga-test/.git/annex/pidlock
|
||||
ls: /var/tmp/ga-test/.git/annex/pidlock: No such file or directory
|
||||
ewen@ashram:~$
|
||||
|
||||
so it definitely seems to *try* to do pidlocking on the SMB mount on transfers, but it looks like maybe it's also trying to do something *not* supported.
|
||||
|
||||
Looking through the OS X `open(2)` manpage, it appears that:
|
||||
|
||||
O_EXCL error if O_CREAT and the file exists
|
||||
[...]
|
||||
O_EXLOCK atomically obtain an exclusive lock
|
||||
|
||||
might be plausible non-hardlink locking primitives that'd work better than nothing. The manpage also says:
|
||||
|
||||
When opening a file, a lock with flock(2) semantics can be obtained by
|
||||
setting O_SHLOCK for a shared lock, or O_EXLOCK for an exclusive lock.
|
||||
If creating a file with O_CREAT, the request for the lock will never fail
|
||||
(provided that the underlying filesystem supports locking).
|
||||
|
||||
so maybe that's something to work with? (SMB as a protocol supports All Kinds Of Locking (tm), and hopefully the OS X SMB file system driver would also translate O_EXLOCK, etc, through to those...)
|
||||
|
||||
Hope that's of some help.
|
||||
|
||||
Ewen
|
||||
|
||||
PS: Yes, I do plan to try to install the stand alone x86-64 git-annex on my Synology when I get a spare few hours to sort out all the details. It seems like it would be the most robust solution for my specific problem. But generalised SMB support could also be useful, even if it was \"user needs to be careful about running commands in parallel as there are races\".
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
[[!comment format=mdwn
|
||||
username="t.z.mates"
|
||||
avatar="http://cdn.libravatar.org/avatar/90f15fad216078fd08d62cc676487925"
|
||||
subject="comment 9"
|
||||
date="2017-05-13T20:59:34Z"
|
||||
content="""
|
||||
Thanks for the response! I've [linked](https://github.com/tzmates/GitAnnexStrace) to a repo where I posted the strace results (only for the offending line, otherwise there were a lot of files) as well as the code that generated it. I'm not very familiar with strace results, so let me know if I should supply anything else.
|
||||
|
||||
It certainly could be a BTRFS bug, though it doesn't seem to be tied to a specific kernel version; I've been continually updating kernels for the last 2 years now.
|
||||
|
||||
If it's any help, I've never had this sort of problem using `cp --reflink` day-to-day. I have no experience with `mmap` though, so that's a distinct possibility.
|
||||
|
||||
Thanks for all your input on this so far!
|
||||
"""]]
|
||||
357
doc/bugs/git-annex_in_nixpkgs_fails_with_git-2.13.0.mdwn
Normal file
357
doc/bugs/git-annex_in_nixpkgs_fails_with_git-2.13.0.mdwn
Normal file
|
|
@ -0,0 +1,357 @@
|
|||
### Please describe the problem.
|
||||
|
||||
Tests fail on Nix when git is upgraded to 2.13.
|
||||
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
On NixOS or a system with Nix installed, check out nixpkgs ce8662e693b5756e8457d94bb1765d853310afdb, try to build git-annex (`nix-build -I . -A pkgs.gitAndTools.gitAnnex`).
|
||||
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
git-annex 6.20170321 on Nix ce8662e6.
|
||||
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
I ran a git bisect that concluded:
|
||||
|
||||
[[!format sh """
|
||||
8 out of 285 tests failed (281.73s)
|
||||
(This could be due to a bug in git-annex, or an incompatibility
|
||||
with utilities, such as git, installed on this system.)
|
||||
builder for ‘/nix/store/908y9923fnjmi87apji6q14smgc2rf3d-git-annex-6.20170321.drv’ failed with exit code 1
|
||||
error: build of ‘/nix/store/908y9923fnjmi87apji6q14smgc2rf3d-git-annex-6.20170321.drv’ failed
|
||||
ce8662e693b5756e8457d94bb1765d853310afdb is the first bad commit
|
||||
commit ce8662e693b5756e8457d94bb1765d853310afdb
|
||||
Author: Tim Steinbach <tim@nequissimus.com>
|
||||
Date: Tue May 9 21:57:24 2017 -0400
|
||||
|
||||
git: 2.12.2 -> 2.13.0
|
||||
|
||||
:040000 040000 4155c091e7156a4e15cf073640e0be3c76b1b7f3 48ca917e5578b405cf8517e1f113433b2d076e15 M pkgs
|
||||
bisect run success
|
||||
"""]]
|
||||
|
||||
|
||||
[[!format sh """
|
||||
adjusted branch merge regression: Switched to branch 'adjusted/master(unlocked)'
|
||||
[adjusted/master(unlocked) ca9719e] git-annex in .t/tmprepo39
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 100644 conflictor
|
||||
From ../../.t/tmprepo40
|
||||
* [new branch] git-annex -> r2/git-annex
|
||||
* [new branch] master -> r2/master
|
||||
To ../../.t/tmprepo40
|
||||
* [new branch] git-annex -> synced/git-annex
|
||||
* [new branch] master -> synced/master
|
||||
Switched to branch 'adjusted/master(unlocked)'
|
||||
[adjusted/master(unlocked) b29e8de] git-annex in .t/tmprepo40
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 100644 conflictor
|
||||
merge: refs/heads/synced/master - not something we can merge
|
||||
From ../../.t/tmprepo39
|
||||
* [new branch] adjusted/master(unlocked) -> r1/adjusted/master(unlocked)
|
||||
* [new branch] git-annex -> r1/git-annex
|
||||
* [new branch] master -> r1/master
|
||||
* [new branch] synced/master -> r1/synced/master
|
||||
merge: refs/remotes/r1/master - not something we can merge
|
||||
merge: refs/remotes/r1/synced/master - not something we can merge
|
||||
To ../../.t/tmprepo39
|
||||
* [new branch] git-annex -> synced/git-annex
|
||||
! [rejected] master -> synced/master (non-fast-forward)
|
||||
error: failed to push some refs to '../../.t/tmprepo39'
|
||||
hint: Updates were rejected because a pushed branch tip is behind its remote
|
||||
hint: counterpart. Check out this branch and integrate the remote changes
|
||||
hint: (e.g. 'git pull ...') before pushing again.
|
||||
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
|
||||
To ../../.t/tmprepo39
|
||||
! [rejected] master -> master (non-fast-forward)
|
||||
error: failed to push some refs to '../../.t/tmprepo39'
|
||||
hint: Updates were rejected because a pushed branch tip is behind its remote
|
||||
hint: counterpart. Check out this branch and integrate the remote changes
|
||||
hint: (e.g. 'git pull ...') before pushing again.
|
||||
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
|
||||
Pushing to r1 failed.
|
||||
FAIL (2.32s)
|
||||
"""]]
|
||||
|
||||
|
||||
[[!format sh """
|
||||
conflict resolution (adjusted branch): [master b0dd758] git-annex in .t/tmprepo44
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 100644 conflictor
|
||||
[master 8295bae] git-annex in .t/tmprepo45
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 100644 conflictor
|
||||
Switched to branch 'adjusted/master(unlocked)'
|
||||
On branch master
|
||||
nothing to commit, working tree clean
|
||||
From ../../.t/tmprepo45
|
||||
* [new branch] adjusted/master(unlocked) -> r2/adjusted/master(unlocked)
|
||||
* [new branch] git-annex -> r2/git-annex
|
||||
* [new branch] master -> r2/master
|
||||
* [new branch] synced/master -> r2/synced/master
|
||||
Auto-merging conflictor
|
||||
CONFLICT (add/add): Merge conflict in conflictor
|
||||
Automatic merge failed; fix conflicts and then commit the result.
|
||||
conflictor: needs merge
|
||||
[master 14ffa24] git-annex automatic merge conflict fix
|
||||
Already up-to-date.
|
||||
To ../../.t/tmprepo45
|
||||
8295bae..14ffa24 master -> synced/master
|
||||
* [new branch] git-annex -> synced/git-annex
|
||||
On branch adjusted/master(unlocked)
|
||||
nothing to commit, working tree clean
|
||||
merge: refs/heads/synced/master - not something we can merge
|
||||
From ../../.t/tmprepo44
|
||||
* [new branch] git-annex -> r1/git-annex
|
||||
* [new branch] master -> r1/master
|
||||
* [new branch] synced/master -> r1/synced/master
|
||||
merge: refs/remotes/r1/master - not something we can merge
|
||||
merge: refs/remotes/r1/synced/master - not something we can merge
|
||||
FAIL (2.34s)
|
||||
"""]]
|
||||
|
||||
[[!format sh """
|
||||
adjusted branch merge regression: Switched to branch 'adjusted/master(unlocked)'
|
||||
[adjusted/master(unlocked) 7ccad36] git-annex in .t/tmprepo39
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 100644 conflictor
|
||||
From ../../.t/tmprepo40
|
||||
* [new branch] git-annex -> r2/git-annex
|
||||
* [new branch] master -> r2/master
|
||||
To ../../.t/tmprepo40
|
||||
* [new branch] git-annex -> synced/git-annex
|
||||
* [new branch] master -> synced/master
|
||||
Switched to branch 'adjusted/master(unlocked)'
|
||||
[adjusted/master(unlocked) cfb04fd] git-annex in .t/tmprepo40
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 100644 conflictor
|
||||
merge: refs/heads/synced/master - not something we can merge
|
||||
From ../../.t/tmprepo39
|
||||
* [new branch] adjusted/master(unlocked) -> r1/adjusted/master(unlocked)
|
||||
* [new branch] git-annex -> r1/git-annex
|
||||
* [new branch] master -> r1/master
|
||||
* [new branch] synced/master -> r1/synced/master
|
||||
merge: refs/remotes/r1/master - not something we can merge
|
||||
merge: refs/remotes/r1/synced/master - not something we can merge
|
||||
To ../../.t/tmprepo39
|
||||
* [new branch] git-annex -> synced/git-annex
|
||||
! [rejected] master -> synced/master (non-fast-forward)
|
||||
error: failed to push some refs to '../../.t/tmprepo39'
|
||||
hint: Updates were rejected because a pushed branch tip is behind its remote
|
||||
hint: counterpart. Check out this branch and integrate the remote changes
|
||||
hint: (e.g. 'git pull ...') before pushing again.
|
||||
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
|
||||
To ../../.t/tmprepo39
|
||||
! [rejected] master -> master (non-fast-forward)
|
||||
error: failed to push some refs to '../../.t/tmprepo39'
|
||||
hint: Updates were rejected because a pushed branch tip is behind its remote
|
||||
hint: counterpart. Check out this branch and integrate the remote changes
|
||||
hint: (e.g. 'git pull ...') before pushing again.
|
||||
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
|
||||
Pushing to r1 failed.
|
||||
FAIL (1.79s)
|
||||
"""]]
|
||||
|
||||
[[!format sh """
|
||||
conflict resolution (adjusted branch): [master ea63ee5] git-annex in .t/tmprepo44
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 120000 conflictor
|
||||
[master 07d129c] git-annex in .t/tmprepo45
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 120000 conflictor
|
||||
Switched to branch 'adjusted/master(unlocked)'
|
||||
On branch master
|
||||
nothing to commit, working tree clean
|
||||
From ../../.t/tmprepo45
|
||||
* [new branch] adjusted/master(unlocked) -> r2/adjusted/master(unlocked)
|
||||
* [new branch] git-annex -> r2/git-annex
|
||||
* [new branch] master -> r2/master
|
||||
* [new branch] synced/master -> r2/synced/master
|
||||
Auto-merging conflictor
|
||||
CONFLICT (add/add): Merge conflict in conflictor
|
||||
Automatic merge failed; fix conflicts and then commit the result.
|
||||
conflictor: needs merge
|
||||
[master 5d5a674] git-annex automatic merge conflict fix
|
||||
Already up-to-date.
|
||||
To ../../.t/tmprepo45
|
||||
07d129c..5d5a674 master -> synced/master
|
||||
* [new branch] git-annex -> synced/git-annex
|
||||
On branch adjusted/master(unlocked)
|
||||
nothing to commit, working tree clean
|
||||
merge: refs/heads/synced/master - not something we can merge
|
||||
From ../../.t/tmprepo44
|
||||
* [new branch] git-annex -> r1/git-annex
|
||||
* [new branch] master -> r1/master
|
||||
* [new branch] synced/master -> r1/synced/master
|
||||
merge: refs/remotes/r1/master - not something we can merge
|
||||
merge: refs/remotes/r1/synced/master - not something we can merge
|
||||
FAIL (1.35s)
|
||||
"""]]
|
||||
|
||||
[[!format sh """
|
||||
adjusted branch merge regression: Switched to branch 'adjusted/master(unlocked)'
|
||||
[adjusted/master(unlocked) 6601699] git-annex in .t/tmprepo39
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 100644 conflictor
|
||||
From ../../.t/tmprepo40
|
||||
* [new branch] git-annex -> r2/git-annex
|
||||
* [new branch] master -> r2/master
|
||||
To ../../.t/tmprepo40
|
||||
* [new branch] git-annex -> synced/git-annex
|
||||
* [new branch] master -> synced/master
|
||||
Switched to branch 'adjusted/master(unlocked)'
|
||||
[adjusted/master(unlocked) b37ecc1] git-annex in .t/tmprepo40
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 100644 conflictor
|
||||
merge: refs/heads/synced/master - not something we can merge
|
||||
From ../../.t/tmprepo39
|
||||
* [new branch] adjusted/master(unlocked) -> r1/adjusted/master(unlocked)
|
||||
* [new branch] git-annex -> r1/git-annex
|
||||
* [new branch] master -> r1/master
|
||||
* [new branch] synced/master -> r1/synced/master
|
||||
merge: refs/remotes/r1/master - not something we can merge
|
||||
merge: refs/remotes/r1/synced/master - not something we can merge
|
||||
To ../../.t/tmprepo39
|
||||
* [new branch] git-annex -> synced/git-annex
|
||||
! [rejected] master -> synced/master (non-fast-forward)
|
||||
error: failed to push some refs to '../../.t/tmprepo39'
|
||||
hint: Updates were rejected because a pushed branch tip is behind its remote
|
||||
hint: counterpart. Check out this branch and integrate the remote changes
|
||||
hint: (e.g. 'git pull ...') before pushing again.
|
||||
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
|
||||
To ../../.t/tmprepo39
|
||||
! [rejected] master -> master (non-fast-forward)
|
||||
error: failed to push some refs to '../../.t/tmprepo39'
|
||||
hint: Updates were rejected because a pushed branch tip is behind its remote
|
||||
hint: counterpart. Check out this branch and integrate the remote changes
|
||||
hint: (e.g. 'git pull ...') before pushing again.
|
||||
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
|
||||
Pushing to r1 failed.
|
||||
FAIL (1.54s)
|
||||
"""]]
|
||||
|
||||
[[!format sh """
|
||||
conflict resolution (adjusted branch): [master dcd2368] git-annex in .t/tmprepo44
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 120000 conflictor
|
||||
[master c933362] git-annex in .t/tmprepo45
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 120000 conflictor
|
||||
Switched to branch 'adjusted/master(unlocked)'
|
||||
On branch master
|
||||
nothing to commit, working tree clean
|
||||
From ../../.t/tmprepo45
|
||||
* [new branch] adjusted/master(unlocked) -> r2/adjusted/master(unlocked)
|
||||
* [new branch] git-annex -> r2/git-annex
|
||||
* [new branch] master -> r2/master
|
||||
* [new branch] synced/master -> r2/synced/master
|
||||
Auto-merging conflictor
|
||||
CONFLICT (add/add): Merge conflict in conflictor
|
||||
Automatic merge failed; fix conflicts and then commit the result.
|
||||
conflictor: needs merge
|
||||
[master 9f19aa7] git-annex automatic merge conflict fix
|
||||
Already up-to-date.
|
||||
To ../../.t/tmprepo45
|
||||
c933362..9f19aa7 master -> synced/master
|
||||
* [new branch] git-annex -> synced/git-annex
|
||||
On branch adjusted/master(unlocked)
|
||||
nothing to commit, working tree clean
|
||||
merge: refs/heads/synced/master - not something we can merge
|
||||
From ../../.t/tmprepo44
|
||||
* [new branch] git-annex -> r1/git-annex
|
||||
* [new branch] master -> r1/master
|
||||
* [new branch] synced/master -> r1/synced/master
|
||||
merge: refs/remotes/r1/master - not something we can merge
|
||||
merge: refs/remotes/r1/synced/master - not something we can merge
|
||||
FAIL (1.44s)
|
||||
"""]]
|
||||
|
||||
[[!format sh """
|
||||
adjusted branch merge regression: On branch master
|
||||
nothing to commit, working tree clean
|
||||
On branch master
|
||||
nothing to commit, working tree clean
|
||||
Already on 'adjusted/master(unlocked)'
|
||||
[adjusted/master(unlocked) 70b8469] git-annex in .t/tmprepo39
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 100644 conflictor
|
||||
From ../../.t/tmprepo40
|
||||
* [new branch] annex/direct/master -> r2/annex/direct/master
|
||||
* [new branch] git-annex -> r2/git-annex
|
||||
* [new branch] master -> r2/master
|
||||
To ../../.t/tmprepo40
|
||||
* [new branch] git-annex -> synced/git-annex
|
||||
* [new branch] master -> synced/master
|
||||
Already on 'adjusted/master(unlocked)'
|
||||
[adjusted/master(unlocked) 353957d] git-annex in .t/tmprepo40
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 100644 conflictor
|
||||
merge: refs/heads/synced/master - not something we can merge
|
||||
From ../../.t/tmprepo39
|
||||
* [new branch] adjusted/master(unlocked) -> r1/adjusted/master(unlocked)
|
||||
* [new branch] git-annex -> r1/git-annex
|
||||
* [new branch] master -> r1/master
|
||||
* [new branch] synced/master -> r1/synced/master
|
||||
merge: refs/remotes/r1/master - not something we can merge
|
||||
merge: refs/remotes/r1/synced/master - not something we can merge
|
||||
To ../../.t/tmprepo39
|
||||
* [new branch] git-annex -> synced/git-annex
|
||||
! [rejected] master -> synced/master (non-fast-forward)
|
||||
error: failed to push some refs to '../../.t/tmprepo39'
|
||||
hint: Updates were rejected because a pushed branch tip is behind its remote
|
||||
hint: counterpart. Check out this branch and integrate the remote changes
|
||||
hint: (e.g. 'git pull ...') before pushing again.
|
||||
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
|
||||
To ../../.t/tmprepo39
|
||||
! [rejected] master -> master (non-fast-forward)
|
||||
error: failed to push some refs to '../../.t/tmprepo39'
|
||||
hint: Updates were rejected because a pushed branch tip is behind its remote
|
||||
hint: counterpart. Check out this branch and integrate the remote changes
|
||||
hint: (e.g. 'git pull ...') before pushing again.
|
||||
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
|
||||
Pushing to r1 failed.
|
||||
FAIL (1.87s)
|
||||
"""]]
|
||||
|
||||
[[!format sh """
|
||||
conflict resolution (adjusted branch): On branch master
|
||||
nothing to commit, working tree clean
|
||||
On branch master
|
||||
nothing to commit, working tree clean
|
||||
Already on 'adjusted/master(unlocked)'
|
||||
From ../../.t/tmprepo45
|
||||
* [new branch] adjusted/master(unlocked) -> r2/adjusted/master(unlocked)
|
||||
* [new branch] git-annex -> r2/git-annex
|
||||
* [new branch] master -> r2/master
|
||||
* [new branch] synced/master -> r2/synced/master
|
||||
Auto-merging conflictor
|
||||
CONFLICT (add/add): Merge conflict in conflictor
|
||||
Automatic merge failed; fix conflicts and then commit the result.
|
||||
conflictor: needs merge
|
||||
To ../../.t/tmprepo45
|
||||
372f97c..cebbf06 annex/direct/master -> synced/master
|
||||
* [new branch] git-annex -> synced/git-annex
|
||||
On branch adjusted/master(unlocked)
|
||||
nothing to commit, working tree clean
|
||||
merge: refs/heads/synced/master - not something we can merge
|
||||
From ../../.t/tmprepo44
|
||||
* [new branch] annex/direct/master -> r1/annex/direct/master
|
||||
* [new branch] git-annex -> r1/git-annex
|
||||
* [new branch] master -> r1/master
|
||||
* [new branch] synced/master -> r1/synced/master
|
||||
merge: refs/remotes/r1/master - not something we can merge
|
||||
merge: refs/remotes/r1/synced/master - not something we can merge
|
||||
FAIL (1.50s)
|
||||
"""]]
|
||||
|
||||
[[!format sh """
|
||||
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
git-annex is an essential building block in my digital life style! It keeps backups of all my precious family photos. I'm a big git-annex shill when I get the chance, especially to nix, guix and decentralized hacker types.
|
||||
99
doc/bugs/glacier.mdwn
Normal file
99
doc/bugs/glacier.mdwn
Normal file
|
|
@ -0,0 +1,99 @@
|
|||
### Please describe the problem.
|
||||
|
||||
Amazon Glacier remote doesn't work as expected. It seems like glacier-cli no longer works with git-annex.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
1. use git-annex from archlinux repo
|
||||
2. download and install this with pip: https://github.com/basak/glacier-cli
|
||||
3. follow these instructions https://git-annex.branchable.com/tips/using_Amazon_Glacier/
|
||||
4. the problems occurs when trying to move or copy content to glacier
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
version: 6.20160418-geff8673
|
||||
on: Linux 4.10.13-1-ARCH #1 SMP PREEMPT Thu Apr 27 12:15:09 CEST 2017 x86_64 GNU/Linux
|
||||
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
|
||||
# with python 3.6
|
||||
% git annex copy file.mp3 --to glacier
|
||||
copy file.mp3 (checking glacier...) (to glacier...)
|
||||
98% 0.0 B/s 0sTraceback (most recent call last):
|
||||
File "/home/me/.local/bin/glacier", line 736, in <module>
|
||||
main()
|
||||
File "/home/me/.local/bin/glacier", line 732, in main
|
||||
App().main()
|
||||
File "/home/me/.local/bin/glacier", line 718, in main
|
||||
self.args.func()
|
||||
File "/home/me/.local/bin/glacier", line 500, in archive_upload
|
||||
file_obj=self.args.file, description=name)
|
||||
File "/usr/lib/python3.6/site-packages/boto/glacier/vault.py", line 178, in create_archive_from_file
|
||||
writer.close()
|
||||
File "/usr/lib/python3.6/site-packages/boto/glacier/writer.py", line 231, in close
|
||||
self.partitioner.flush()
|
||||
File "/usr/lib/python3.6/site-packages/boto/glacier/writer.py", line 79, in flush
|
||||
self._send_part()
|
||||
File "/usr/lib/python3.6/site-packages/boto/glacier/writer.py", line 64, in _send_part
|
||||
data = b''.join(self._buffer)
|
||||
TypeError: sequence item 0: expected a bytes-like object, str found
|
||||
failed
|
||||
git-annex: copy: 1 failed
|
||||
|
||||
# --------------------------
|
||||
# with python2.7
|
||||
% git annex copy file.m3 --to glacier
|
||||
copy file.mp3 (checking glacier...) (to glacier...)
|
||||
98% 0.0 B/s 0sTraceback (most recent call last):
|
||||
File "/home/me/.local/bin/glacier", line 736, in <module>
|
||||
main()
|
||||
File "/home/me/.local/bin/glacier", line 732, in main
|
||||
App().main()
|
||||
File "/home/me/.local/bin/glacier", line 718, in main
|
||||
self.args.func()
|
||||
File "/home/me/.local/bin/glacier", line 500, in archive_upload
|
||||
file_obj=self.args.file, description=name)
|
||||
File "/usr/lib/python2.7/site-packages/boto/glacier/vault.py", line 178, in create_archive_from_file
|
||||
writer.close()
|
||||
File "/usr/lib/python2.7/site-packages/boto/glacier/writer.py", line 228, in close
|
||||
self.partitioner.flush()
|
||||
File "/usr/lib/python2.7/site-packages/boto/glacier/writer.py", line 79, in flush
|
||||
self._send_part()
|
||||
File "/usr/lib/python2.7/site-packages/boto/glacier/writer.py", line 75, in _send_part
|
||||
self.send_fn(part)
|
||||
File "/usr/lib/python2.7/site-packages/boto/glacier/writer.py", line 222, in _upload_part
|
||||
self.uploader.upload_part(self.next_part_index, part_data)
|
||||
File "/usr/lib/python2.7/site-packages/boto/glacier/writer.py", line 129, in upload_part
|
||||
content_range, part_data)
|
||||
File "/usr/lib/python2.7/site-packages/boto/glacier/layer1.py", line 1279, in upload_part
|
||||
response_headers=response_headers)
|
||||
File "/usr/lib/python2.7/site-packages/boto/glacier/layer1.py", line 114, in make_request
|
||||
data=data)
|
||||
File "/usr/lib/python2.7/site-packages/boto/connection.py", line 1071, in make_request
|
||||
retry_handler=retry_handler)
|
||||
File "/usr/lib/python2.7/site-packages/boto/connection.py", line 943, in _mexe
|
||||
request.body, request.headers)
|
||||
File "/usr/lib/python2.7/httplib.py", line 1042, in request
|
||||
self._send_request(method, url, body, headers)
|
||||
File "/usr/lib/python2.7/httplib.py", line 1082, in _send_request
|
||||
self.endheaders(body)
|
||||
File "/usr/lib/python2.7/httplib.py", line 1038, in endheaders
|
||||
self._send_output(message_body)
|
||||
File "/usr/lib/python2.7/httplib.py", line 880, in _send_output
|
||||
msg += message_body
|
||||
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8c in position 0: ordinal not in range(128)
|
||||
failed
|
||||
git-annex: copy: 1 failed
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
Yes it's a great tool :). Thanks!
|
||||
AWS Glacier integration would be a perfect addition and I could employ git-annex at my company.
|
||||
|
||||
|
|
@ -0,0 +1,215 @@
|
|||
### Please describe the problem.
|
||||
Sync / Assistant between a `source` and `transfer` repo results in zero copies for whereis, but data is available.
|
||||
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
See a script below...
|
||||
|
||||
``` sh
|
||||
#!/bin/sh
|
||||
|
||||
TMP=$(mktemp -d)
|
||||
|
||||
|
||||
##########################################
|
||||
# Set up the repos
|
||||
##########################################
|
||||
|
||||
|
||||
###
|
||||
# Create working directories
|
||||
cd ${TMP}
|
||||
mkdir source transfer
|
||||
|
||||
|
||||
###
|
||||
# Set up the source repo
|
||||
cd ${TMP}/source
|
||||
git init
|
||||
git annex init source
|
||||
git-annex group here source
|
||||
git annex wanted here standard
|
||||
|
||||
|
||||
###
|
||||
# Set up the transfer repo
|
||||
# N.B. It can see the source repo
|
||||
cd ${TMP}/transfer
|
||||
git init
|
||||
git remote add source ${TMP}/source
|
||||
git annex init transfer
|
||||
git-annex group here transfer
|
||||
git annex wanted here \
|
||||
'not (inallgroup=client and copies=client:1) and ((include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1)'
|
||||
|
||||
|
||||
##########################################
|
||||
# Add some files and run the assistant
|
||||
##########################################
|
||||
|
||||
###
|
||||
# Make sure it works by adding some files to the source...
|
||||
cd ${TMP}/source
|
||||
date > file.1
|
||||
date > file.2
|
||||
date > file.3
|
||||
|
||||
# Run the assistant in the source...
|
||||
(cd ${TMP}/source && \
|
||||
git annex assistant && \
|
||||
sleep 30 && \
|
||||
git annex assistant --stop && \
|
||||
git annex whereis file.*)
|
||||
# PASS: All files are in source.
|
||||
|
||||
# Now run it in the transfer group
|
||||
(cd ${TMP}/transfer && \
|
||||
git annex assistant && \
|
||||
sleep 30 && \
|
||||
git annex assistant --stop && \
|
||||
git annex whereis file.*)
|
||||
# PASS: All files are in transfer
|
||||
|
||||
# Check the source
|
||||
(cd ${TMP}/source && \
|
||||
git annex whereis file.*)
|
||||
# FAIL: Only file.1 was updated to be in source.
|
||||
|
||||
# whereis file.1 (1 copy)
|
||||
# 64b193e0-86f0-4349-b073-70af919ce628 -- transfer
|
||||
# ok
|
||||
# whereis file.2 (0 copies) failed
|
||||
# whereis file.3 (0 copies) failed
|
||||
# git-annex: whereis: 2 failed
|
||||
|
||||
|
||||
##########################################
|
||||
# Start to scratch head and ask 'Why?'
|
||||
##########################################
|
||||
|
||||
##########
|
||||
# What's happening???
|
||||
|
||||
cd ${TMP}/source && git annex sync
|
||||
# Source thinks it is updated - working tree clean
|
||||
|
||||
cd ${TMP}/transfer && git annex sync
|
||||
# Transfer pulls from source, so run again
|
||||
git annex sync
|
||||
# OK... working tree clean
|
||||
|
||||
|
||||
# Check again
|
||||
cd ${TMP}/source && git annex whereis file.*
|
||||
# whereis file.1 (1 copy)
|
||||
# 64b193e0-86f0-4349-b073-70af919ce628 -- transfer
|
||||
# ok
|
||||
# whereis file.2 (0 copies) failed
|
||||
# whereis file.3 (0 copies) failed
|
||||
# git-annex: whereis: 2 failed
|
||||
|
||||
cd ${TMP}/transfer && git annex whereis file.*
|
||||
# whereis file.1 (1 copy)
|
||||
# 64b193e0-86f0-4349-b073-70af919ce628 -- transfer [here]
|
||||
# ok
|
||||
# whereis file.2 (0 copies) failed
|
||||
# whereis file.3 (0 copies) failed
|
||||
# git-annex: whereis: 2 failed
|
||||
|
||||
|
||||
# HOWEVER... Files are definitely in the transfer repo
|
||||
|
||||
cd ${TMP}/source && cat file.*
|
||||
# cat: file.1: No such file or directory
|
||||
# cat: file.2: No such file or directory
|
||||
# cat: file.3: No such file or directory
|
||||
|
||||
|
||||
cd ${TMP}/transfer && cat file.*
|
||||
# Sun May 14 21:22:02 PDT 2017
|
||||
# Sun May 14 21:22:03 PDT 2017
|
||||
# Sun May 14 21:22:06 PDT 2017
|
||||
```
|
||||
|
||||
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
#### Version:
|
||||
```
|
||||
git-annex version: 6.20170510
|
||||
build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV FsEvents ConcurrentOutput TorrentParser MagicMime Feeds Quvi
|
||||
dependency versions: aws-0.16 bloomfilter-2.0.1.0 cryptonite-0.23 DAV-1.3.1 feed-0.3.12.0 ghc-8.0.2 http-client-0.5.6.1 persistent-sqlite-2.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.4.5
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
|
||||
local repository version: 5
|
||||
supported repository versions: 3 5 6
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5
|
||||
operating system: darwin x86_64
|
||||
```
|
||||
|
||||
#### OS:
|
||||
MacOS 10.12.4, but seems to apply to Debian Jessie too.
|
||||
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
Annex assistant daemon.log for `source` repo:
|
||||
|
||||
``` sh
|
||||
[2017-05-15 14:11:24.150202] main: starting assistant version 6.20170510
|
||||
[2017-05-15 14:11:24.196597] Cronner: You should enable consistency checking to protect your data.
|
||||
(scanning...) [2017-05-15 14:11:24.257123] Watcher: Performing startup scan
|
||||
(started...)
|
||||
[2017-05-15 14:11:45.505855] Committer: Adding file.1 file.2 file.3
|
||||
add file.1 ok
|
||||
add file.2 ok
|
||||
add file.3 ok
|
||||
[2017-05-15 14:11:45.544277] Committer: Committing changes to git
|
||||
(recording state in git...)
|
||||
(recording state in git...)
|
||||
```
|
||||
|
||||
Annex assistant daemon.log for `transfer` repo:
|
||||
|
||||
``` sh
|
||||
[2017-05-15 14:12:09.127406] main: starting assistant version 6.20170510
|
||||
[2017-05-15 14:12:09.157611] Cronner: You should enable consistency checking to protect your data.
|
||||
[2017-05-15 14:12:09.177452] TransferScanner: Syncing with source
|
||||
(scanning...) [2017-05-15 14:12:09.210605] Watcher: Performing startup scan
|
||||
(started...)
|
||||
warning: no common commits
|
||||
From /Users/olaf/tmp/git-annex/play/bug-play/source
|
||||
* [new branch] git-annex -> source/git-annex
|
||||
* [new branch] master -> source/master
|
||||
* [new branch] synced/master -> source/synced/master
|
||||
(merging source/git-annex into git-annex...)
|
||||
(recording state in git...)
|
||||
|
||||
|
||||
Already up-to-date.
|
||||
[2017-05-15 14:12:10.256195] Committer: Committing changes to git
|
||||
(recording state in git...)
|
||||
(checksum...) [2017-05-15 14:12:10.425494] Transferrer: Downloaded file.1
|
||||
[2017-05-15 14:12:11.17664] Pusher: Syncing with source
|
||||
(recording state in git...)
|
||||
[2017-05-15 14:12:11.356447] Committer: Committing changes to git
|
||||
(recording state in git...)
|
||||
To /Users/olaf/tmp/git-annex/play/bug-play/source
|
||||
* [new branch] git-annex -> synced/git-annex
|
||||
(checksum...) [2017-05-15 14:12:11.428676] Transferrer: Downloaded file.2
|
||||
(checksum...) [2017-05-15 14:12:11.451457] Transferrer: Downloaded file.3
|
||||
drop source file.1 ok
|
||||
drop source file.2 ok
|
||||
drop source file.3 ok
|
||||
[2017-05-15 14:12:13.477636] Pusher: Syncing with source
|
||||
(recording state in git...)
|
||||
To /Users/olaf/tmp/git-annex/play/bug-play/source
|
||||
825d3dd..60c96cb git-annex -> synced/git-annex
|
||||
```
|
||||
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
Yes. Love it. Donated. Have been using it for years. Recommend it and get(/force) my collaborators to use it. ;-)
|
||||
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
[[!comment format=mdwn
|
||||
username="leavingchicago@c04c893e78d1c4c76cb3e32b5c227cf42bbf7682"
|
||||
nickname="leavingchicago"
|
||||
avatar="http://cdn.libravatar.org/avatar/4ae498d3d6ee558d6b65caa658f72572"
|
||||
subject="Related..."
|
||||
date="2017-05-15T04:43:44Z"
|
||||
content="""
|
||||
I think this bug is related to the problems that I'm having with [/forum/Get source group to automatically drop with assistant](https://git-annex.branchable.com/forum/Get___39__source__39___group_to_automatically_drop_with_assistant/)
|
||||
|
||||
"""]]
|
||||
38
doc/forum/Fix_wrong_rsync_with_git_annex_reinit__63__.mdwn
Normal file
38
doc/forum/Fix_wrong_rsync_with_git_annex_reinit__63__.mdwn
Normal file
|
|
@ -0,0 +1,38 @@
|
|||
# Context
|
||||
|
||||
* Had two repo A and B.
|
||||
* A had a lot of content not yet tracked and thus not available on B.
|
||||
* Filesystem problem on A: (relatively small) part of big repo content is lost.
|
||||
* In emergency, used rsync to propagate yet untracked content from failing A to sane B.
|
||||
* But forgot `ignore .git`, so rsync also partly rsync'ed `.git/annex` .
|
||||
* Now sane repo B believes it is the source one (same UUID A) as shown by `git annex info --fast`.
|
||||
|
||||
Both A and B show:
|
||||
|
||||
UUID-of-A -- A [here]
|
||||
UUID-of-B -- B
|
||||
|
||||
# Actions done
|
||||
|
||||
Search on Google, see [[https://git-annex.branchable.com/git-annex-reinit/]] .
|
||||
|
||||
On B:
|
||||
|
||||
git annex reinit UUID-of-B
|
||||
git annex fsck
|
||||
|
||||
Now `git annex info -fast` on B shows as expected:
|
||||
|
||||
UUID-of-A -- A
|
||||
UUID-of-B -- B [here]
|
||||
|
||||
|
||||
I'm currently doing `git annex copy --from B` to repropagate lost parts of A.
|
||||
|
||||
# Question
|
||||
|
||||
* Were actions done best thing to do?
|
||||
* Is there something more to do?
|
||||
* Something to check?
|
||||
|
||||
Thanks.
|
||||
|
|
@ -4,7 +4,7 @@ git-annex adjust - enter an adjusted branch
|
|||
|
||||
# SYNOPSIS
|
||||
|
||||
git annex adjust --unlock|--fix`
|
||||
`git annex adjust --unlock|--fix`
|
||||
|
||||
# DESCRIPTION
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue