Merge branch 'master' of ssh://git-annex.branchable.com

This commit is contained in:
Joey Hess 2016-05-17 13:49:16 -04:00
commit 9492c8202b
Failed to extract signature
4 changed files with 168 additions and 0 deletions

View file

@ -0,0 +1,94 @@
### Please describe the problem.
`git annex whereis` reports the same UUID multiple times, for example for one file:
```
whereis file1 (20 copies)
0866153a-19e5-4382-aeb6-30e8210706cc
0866153a-19e5-4382-aeb6-30e8210706cc
0866153a-19e5-4382-aeb6-30e8210706cc
0866153a-19e5-4382-aeb6-30e8210706cc
0dc538bf-5015-474a-965b-f59a678c2606
0dc538bf-5015-474a-965b-f59a678c2606
0dc538bf-5015-474a-965b-f59a678c2606
0dc538bf-5015-474a-965b-f59a678c2606
42c9b8ae-7fe5-452c-8965-146077b567fc
42c9b8ae-7fe5-452c-8965-146077b567fc
42c9b8ae-7fe5-452c-8965-146077b567fc
42c9b8ae-7fe5-452c-8965-146077b567fc
99f82d90-85a1-498e-a149-b5d21a0c4540
99f82d90-85a1-498e-a149-b5d21a0c4540
99f82d90-85a1-498e-a149-b5d21a0c4540
99f82d90-85a1-498e-a149-b5d21a0c4540
9b4679b3-95dd-4e20-8349-7a9ed4d9ff6a
9b4679b3-95dd-4e20-8349-7a9ed4d9ff6a
9b4679b3-95dd-4e20-8349-7a9ed4d9ff6a
9b4679b3-95dd-4e20-8349-7a9ed4d9ff6a
ok
```
For another file:
```
whereis file2 (44 copies)
0866153a-19e5-4382-aeb6-30e8210706cc
0866153a-19e5-4382-aeb6-30e8210706cc
0866153a-19e5-4382-aeb6-30e8210706cc
0866153a-19e5-4382-aeb6-30e8210706cc
0dc538bf-5015-474a-965b-f59a678c2606
0dc538bf-5015-474a-965b-f59a678c2606
42c9b8ae-7fe5-452c-8965-146077b567fc
42c9b8ae-7fe5-452c-8965-146077b567fc
42c9b8ae-7fe5-452c-8965-146077b567fc
42c9b8ae-7fe5-452c-8965-146077b567fc
57c272a3-4146-4796-8c4d-349725e11153
57c272a3-4146-4796-8c4d-349725e11153
57c272a3-4146-4796-8c4d-349725e11153
57c272a3-4146-4796-8c4d-349725e11153
6edd8523-6321-4d60-ada1-364489093075
6edd8523-6321-4d60-ada1-364489093075
6edd8523-6321-4d60-ada1-364489093075
6edd8523-6321-4d60-ada1-364489093075
8ca47082-542f-4935-bd4d-71b3d8071f97
8ca47082-542f-4935-bd4d-71b3d8071f97
8ca47082-542f-4935-bd4d-71b3d8071f97
8ca47082-542f-4935-bd4d-71b3d8071f97
99c9cb4b-3d7d-406b-b68d-c30b62072d25
99c9cb4b-3d7d-406b-b68d-c30b62072d25
99c9cb4b-3d7d-406b-b68d-c30b62072d25
99c9cb4b-3d7d-406b-b68d-c30b62072d25
99f82d90-85a1-498e-a149-b5d21a0c4540
99f82d90-85a1-498e-a149-b5d21a0c4540
99f82d90-85a1-498e-a149-b5d21a0c4540
99f82d90-85a1-498e-a149-b5d21a0c4540
9b4679b3-95dd-4e20-8349-7a9ed4d9ff6a -- repo1
9b4679b3-95dd-4e20-8349-7a9ed4d9ff6a
9b4679b3-95dd-4e20-8349-7a9ed4d9ff6a
9b4679b3-95dd-4e20-8349-7a9ed4d9ff6a
9b4679b3-95dd-4e20-8349-7a9ed4d9ff6a
9e5cb42d-7a5a-4a99-b176-f0980d9b7265
9e5cb42d-7a5a-4a99-b176-f0980d9b7265
9e5cb42d-7a5a-4a99-b176-f0980d9b7265
9e5cb42d-7a5a-4a99-b176-f0980d9b7265
b1e180f9-3a47-4178-a360-043e731a4f5b -- local_repo [here]
b5472406-01a1-4cd8-a5b4-bb3d914264d2
b5472406-01a1-4cd8-a5b4-bb3d914264d2
b5472406-01a1-4cd8-a5b4-bb3d914264d2
b5472406-01a1-4cd8-a5b4-bb3d914264d2
ok
```
Furthermore as can be seen it doesn't even recognise that I have added repo1 as a remote for file1, actually several other of the repositories are available as remotes but git annex doesn't recognise them.
I have tried deleting the repository and cloning it again on windows, but that just seemed to create even more UUID repeats. So right now I am a bit vary of trying anything out on Windows on the repository.
I have tried running `git annex repair` on Linux and it reported no problems, even though it also shows repeated UUIDs if I run `git annex whereis`.
### What version of git-annex are you using? On what operating system?
I am running "git-annex version: 6.20160511-g4633f0b" on Windows, but I have been having troubles with this for some time, though only on Windows.
### Please provide any additional information below.
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
Yes it is working very nicely on all my linux computers, and right now I am mostly concerned that I might have messed up the repository by trying out Windows :-(.

View file

@ -0,0 +1,15 @@
[[!comment format=mdwn
username="Gus"
subject="Restarting"
date="2016-05-17T10:36:55Z"
content="""
My attempt to redo the podcast repository adding the `--fast` flag seems to be successful. Thank you.
However, I would still like to ask you one thing about the delete operations.
When would you use `git rm` instead of `git-annex drop` (in a repository that uses only git-annex to store the files)?
Oh, and I'll also sneak in a small question that is unrelated to the above, since I don't think it deserves its own topic:
Is git-annex safe to operate in parallel tasks? Can I be adding file A and move file B and rename file C from the same repository at the same time, without the integrity of the repository being compromised?
"""]]

View file

@ -0,0 +1,48 @@
I still have little experience with git-annex, so I may be missing something fairly obvious.
I have been moving files between two repositories. Things *seemed* to be going well, but today I noticed that I was missing some content that I have just moved. I would like some help to figure out where I went wrong to avoid doing worse mistakes in the future.
What I did was to run the command `git-annex move --from=toshiba` to move a bunch of files from my USB unit to my current repository and then I ran `git-annex sync` on both ends. Afterwards I noticed that the content of the files was not available, so I tried to track one of them.
What I could see in my local indirect repository was a broken symbolic link pointing to `../.git/annex/objects/ZQ/WF/SHA256E-s241--f3d7e5d1f788235b8eec0af58cc0c526b112b9e834a47ba7a475876c49dce343.jpg/SHA256E-s241--f3d7e5d1f788235b8eec0af58cc0c526b112b9e834a47ba7a475876c49dce343.jpg`.
`git-annex info` gave me similar information:
file: Heavy Metal 1981.jpg
size: 241 bytes
key: SHA256E-s241--f3d7e5d1f788235b8eec0af58cc0c526b112b9e834a47ba7a475876c49dce343.jpg
`git-annex whereis` locates the file still on its origin:
whereis Heavy Metal 1981.jpg (1 copy)
49b5b3a4-56ac-4cf2-aed9-1c23d3181c97 -- Toshiba USB HDD [toshiba]
ok
On my external HDD drive, where I have an indirect mode repository, the file has already been replaced by a reference (241 bytes).
$ cat "Heavy Metal 1981.jpg"
../../../../../../../../../media/TOSHIBA EXT/annex/.git/annex/objects/4z/gf/SHA256E-s245--ae647e7ad31089255413a9290ca9344542f3cd15ecef66884613bf776387633d.jpg/SHA256E-s245--ae647e7ad31089255413a9290ca9344542f3cd15ecef66884613bf776387633d.jpg
`git-annex log` remembers something about the operation:
+ Tue, 17 May 2016 12:22:43 WEST Heavy Metal 1981.jpg | 49b5b3a4-56ac-4cf2-aed9-1c23d3181c97 -- Toshiba USB HDD [toshiba]
I tried to `git-annex get "Heavy Metal 1981.jpg"`, and now I have a working symlink on my PC. However it does not point to the image, but to the same 241 bytes reference file that I have on my external HDD.
`git-annex whereis` now mentions 2 locations to my file, but none of the working dirs holds its contents. So, where are the contents of the file? Lost somewhere? It appears that git-annex took the indirect mode reference file and took it for the real file contents -- and that is not good.
I looked at the output of `git-annex unused`, cross-referenced with `git log --stat -S` and I managed to find it somewhere in the list of unused files in my PC's repository, but not on the external drive.
Still, at this point I'm a little worried. I would like to understand what I could have done to cause this mess. Also, how I can clean it up (the rest of the files remain as broken links at the moment).
I have a couple more drives I wanted to add to this setup, but you can understand that I hesitate a little bit at the moment. Maybe I have "lost" more data than I realize.
The version details:
git-annex version: 6.20160511
build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify XMPP ConcurrentOutput TorrentParser MagicMime Feeds Quvi
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
local repository version: 5
supported repository versions: 5 6
upgrade supported from repository versions: 0 1 2 4 5
operating system: linux x86_64

View file

@ -0,0 +1,11 @@
[[!comment format=mdwn
username="openmedi"
subject="comment 6"
date="2016-05-16T23:11:34Z"
content="""
I did assume so, because b2s process seems to be running when the assistant fails to shut down. But a new, maybe related problem might have come to light: I have/had a gitlab remote, that grew so big (30 gig) that they shut it down, in the sense that I can't access it anymore (even reading from it seems to have stoped working). I believe that this gitlab remote has some objects that aren't available anymore and git annex seems to try to repair stuff that is not repairable. As always: This is all said with a big \"maybe\", since I wouldn't know how to test my repo against that hypothesis. I have now done `git annex untrust gitlab` and will try if that helps. All this seem very mysterious to me, since I can't believe that I failed remote should prevent the repo to find an acceptable state.
- git annex fsck states that there are 60 failed files (\"Only 1 of 2 trustworthy copies exist of…\" for all of them as far as I can see in the log)
- git annex version 6.20160418 installed through homebrew
- git annex repo version is 6
"""]]