Merge branch 'master' of ssh://git-annex.branchable.com

This commit is contained in:
Joey Hess 2019-08-13 12:35:33 -04:00
commit 8e92eaf0ab
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
5 changed files with 171 additions and 0 deletions

View file

@ -0,0 +1,31 @@
might be a flaky test/operation, happened while building in i386 buster environment:
[[!format sh """
neurodebian@smaug ..7.20190730+git103-g070fbd693-1~ndall+1 % grep -B20 -A1 FAIL git-annex_7.20190730+git103-g070fbd693-1~ndall+1_i386.build
20 100% 0.00kB/s 0:00:00 (xfr#1, to-chk=0/1)
(checksum...) (recording state in git...)
ok
(recording state in git...)
OK (1.00s)
move (ssh remote): move foo (from origin...)
(checksum...) ok
(recording state in git...)
move foo (to origin...)
ok
(recording state in git...)
OK (0.99s)
copy: copy foo (from origin...)
SHA256E-s20--e394a389d787383843decc5d3d99b6d184ffa5fddeec23b911f9ee7fc8b9ea77
20 100% 0.00kB/s 0:00:00 (xfr#1, to-chk=0/1)
SHA256E-s20--e394a389d787383843decc5d3d99b6d184ffa5fddeec23b911f9ee7fc8b9ea77
20 100% 0.00kB/s 0:00:00 (xfr#1, to-chk=0/1)
failed
copy: 1 failed
FAIL (0.34s)
Test.hs:530:
"""]]
I have tried to rebuild (`debian/rules binary`) second time while in the same environment -- didn't succeed... but then I proceeded to build amd64 fine, and redid i386 build just fine
[[!meta author=yoh]]

View file

@ -0,0 +1,90 @@
### Please describe the problem.
I had thought that, after running git-annex-sync in a repo, running it again should always do nothing (if no remote has changed).
But I'm seeing an example where I have to run git-annex-sync twice for subsequent runs to do nothing.
### What steps will reproduce the problem?
See below. The repo has one submodule, in the directory viral-ngs/ .
### What version of git-annex are you using? On what operating system?
[[!format sh """
(master_env_v150_py36) 16:44 [viral-ngs-benchmarks] $ uname -a
Linux ip-172-31-80-189.ec2.internal 4.14.133-113.112.amzn2.x86_64 #1 SMP Tue Jul 30 18:29:50 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
(master_env_v150_py36) 16:46 [viral-ngs-benchmarks] $ git annex version
git-annex version: 7.20190730-g1030771
build flags: Assistant Webapp Pairing S3 WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite
dependency versions: aws-0.21.1 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.3 feed-1.0.1.0 ghc-8.6.5 http-client-0.5.14 persistent-sql\
ite-2.9.3 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_2\
24 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE\
2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224\
BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external
operating system: linux x86_64
supported repository versions: 5 7
upgrade supported from repository versions: 0 1 2 3 4 5 6
local repository version: 5
"""]]
### Please provide any additional information below.
[[!format sh """
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
(master_env_v150_py36) 16:44 [viral-ngs-benchmarks] $ git annex sync
On branch is-import-rabies200
Your branch is up to date with 'origin/is-import-rabies200'.
It took 5.25 seconds to enumerate untracked files. 'status -uno'
may speed it up, but you have to be careful not to forget to add
new files yourself (see 'git help status').
nothing to commit, working tree clean
commit ok
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 1 (delta 0), reused 1 (delta 0), pack-reused 0
Unpacking objects: 100% (1/1), done.
From github.com:broadinstitute/viral-ngs-benchmarks
1d808d3a08c..de7758ca784 is-import-rabies200 -> origin/is-import-rabies200
1d808d3a08c..de7758ca784 synced/is-import-rabies200 -> origin/synced/is-import-rabies200
Updating 1d808d3a08c..de7758ca784
Fast-forward
viral-ngs | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
Already up to date.
pull origin ok
(master_env_v150_py36) 16:44 [viral-ngs-benchmarks] $ git annex sync
[is-import-rabies200 f3e0898cd33] git-annex in viral-ngs-benchmarks copy on /ilya
1 file changed, 1 insertion(+), 1 deletion(-)
commit ok
pull origin ok
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Delta compression using up to 8 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (2/2), 946 bytes | 946.00 KiB/s, done.
Total 2 (delta 1), reused 0 (delta 0)
remote: Resolving deltas: 100% (1/1), completed with 1 local object.
To github.com:broadinstitute/viral-ngs-benchmarks.git
63fe6905f53..1d808d3a08c is-import-rabies200 -> synced/is-import-rabies200
push origin ok
(master_env_v150_py36) 16:42 [viral-ngs-benchmarks] $ git annex sync
On branch is-import-rabies200
Your branch is up to date with 'origin/is-import-rabies200'.
It took 5.36 seconds to enumerate untracked files. 'status -uno'
may speed it up, but you have to be careful not to forget to add
new files yourself (see 'git help status').
nothing to commit, working tree clean
commit ok
pull origin ok
# End of transcript or log.
"""]]
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)

View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="yarikoptic"
avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
subject="comment 3"
date="2019-08-12T20:15:01Z"
content="""
Could there be some point release to facilitate Windows getting a version with not broken dependency? We are still experiencing tests failures while installing/using [https://downloads.kitenet.net/git-annex/windows/current/git-annex-installer.exe](https://downloads.kitenet.net/git-annex/windows/current/git-annex-installer.exe) which reports git annex
7.20190709-g15a972dd3. Ref: https://ci.appveyor.com/project/mih/datalad/builds/26646252/job/yvrb8hxqjo4ly6v1
"""]]

View file

@ -0,0 +1,24 @@
[[!comment format=mdwn
username="Horus"
avatar="http://cdn.libravatar.org/avatar/8f0ee08b98ea5bba76c3fe112c08851c"
subject="comment 3"
date="2019-08-12T07:40:57Z"
content="""
It gives me:
```
% git annex info StorageBox
uuid: 00638e8a-ce29-4c41-b0df-e38bbbea303c
description: [StorageBox]
remote: StorageBox
trust: semitrusted
cost: 250.0
type: webdav
creds: embedded in git repository (gpg encrypted)
url: https://xyz.your-storagebox.de/annex/Documents
encryption: hybrid (to gpg keys: 4F4FB5E450C86B8F)
chunking: 10 megabyteschunks
remote annex keys: 9629
remote annex size: 44.27 gigabytes
```
"""]]

View file

@ -0,0 +1,16 @@
I think git-annex is an excellent project and could solve all my backup problems if I can just figure out how to use it properly!
My primary usage is as a backup tool: I'm a commercial artist and animator so I have a large archive of projects and assets to manage. About two months ago I found out about git-annex and could see how it could help me create a distributed backup system and easily locate which external HDD the data is stored on.
The tools I use don't play nicely with symlinks, so my local repository is upgraded to v7 and set to annex.thin.
I had some issues with speed so I followed some steps [here](https://git-annex.branchable.com/tips/Repositories_with_large_number_of_files/) to get backups to complete in less than 30mins.
Now a few months later, I've completed some projects and want to free up the space on my local HDD and just keep the backups - still being able to pull them down when I need to. I moved the files I want to drop into a directory '_archive' inside the repository. I ran 'git annex sync --content --to backupHDD01' at this point
I tried running 'git annex drop _archive' but I get no output and the size of the directory stays the same.
Then I tried locking every file in the directory and dropping it but again, no change.
Looking through the directory, some of the files are broken symlinks (this is what I wanted) but most are still just hardlinks. Furthermore,'git annex whereis file.ext' only returns output on some of the files. 'git status' reports a clean working tree.
Have I misinterpreted this whole annex.thin thing? Is there a better way to do what I want to do?
Thanks! J.