mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

Merge branch 'master' of ssh://git-annex.branchable.com

Browse source
This commit is contained in:
Joey Hess 2018-11-19 12:40:12 -04:00
commit 0bdf01079a
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
12 changed files with 201 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

5
doc/bugs/Please_include_version_of_used_to_build_ghc_within___34__git_annex_version__34__.mdwn Normal file
View file

@ -0,0 +1,5 @@
Since git-annex version is not always sufficient to pin point the source of regression (e.g. see [the get -J issue](http://git-annex.branchable.com/bugs/multiple_ssh_prompts__44___and_thread_blocked_indefinitely_in_an___63____63____63___transaction/)), it would be really nice if the version of ghc (and thus all its libraries if I get it right) was included in the output of `git annex version`
[[!meta author=yoh]]
> Closing as I was blind -- there is `dependency versions` reported [[done]] --[[yarikoptic]]

15
doc/bugs/Too_difficult_if_not_impossible_to_explicitly_add__47__keep_file_under_git___40__not_annex__41___in_v6_without_employing_.gitattributes/comment_14_cfab68df294195498fe71daa376e9f68._comment Normal file
View file

@ -0,0 +1,15 @@
[[!comment format=mdwn
username="michael.hanke@c60e12358aa3fc6060531bdead1f530ac4d582ec"
nickname="michael.hanke"
avatar="http://cdn.libravatar.org/avatar/f881df265a423e4f24eff27c623148fd"
subject="git annex init --version=6 leaves repo dirty"
date="2018-11-18T17:00:21Z"
content="""
With this command in an empty directory, I can more or less reliably cause the issue to happen (again)
`sudo rm -rf .git .datalad .gitattributes * ; git clone https://github.com/datalad/testrepo--minimalds.git . && git annex init --version=6; git status`
using 6.20181011+git124-g94aa0e2f6-1~ndall+1 or 6.20180913-1
Full story is here: [#datalad/2998](https://github.com/datalad/datalad/issues/2998)
"""]]

8
doc/bugs/impossible_to_login_to_the_website_at_times/comment_4_dc940fed174b32dfdb4b4ff8c10529f0._comment Normal file
View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="andrew"
avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
subject="comment 4"
date="2018-11-17T19:07:07Z"
content="""
and again. 2018-11-17 2:05pm EST, if the time helps for spelunking the logs.
"""]]

58
doc/bugs/multiple_ssh_prompts__44___and_thread_blocked_indefinitely_in_an___63____63____63___transaction/comment_17_0e4efb1686564f4d8e834b1e0843d397._comment Normal file
View file

@ -0,0 +1,58 @@
[[!comment format=mdwn
username="yarikoptic"
avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
subject="comment 17"
date="2018-11-16T22:12:58Z"
content="""
ok, not sure yet if it is git-annex change or ghc change or both here is the point where things changed radically:
going from 6.20180519+gitgf6f199be3 build which used ghc 8.0.1-17+b1 and where we had a nice single ssh password prompt:
[[!format sh \"\"\"
[yhalchen@discovery7 QA]$ PATH=$HOME/6.20180519+gitgf6f199be3-1~ndall+1/usr/lib/git-annex.linux:$PATH git annex get -J2 sub-*
get sub-amit/ses-20180508/anat/sub-amit_ses-20180508_acq-MPRAGE_T1w.nii.gz get sub-amit/ses-20180508/dwi/sub-amit_ses-20180508_acq-DTIX30Xp2Xs4_dwi.nii.gz (from origin...)
(from origin...)
yoh@falkor.datalad.org's password:
\"\"\"]]
to the next build I have 6.20180521+gitg0e3bebd71 which used ghc 8.0.2-11 and with which locking madness starts to kick in:
[[!format sh \"\"\"
[yhalchen@discovery7 QA]$ PATH=$HOME/6.20180521+gitg0e3bebd71-1~ndall+1/usr/lib/git-annex.linux:$PATH git annex get -J2 sub-*
get sub-amit/ses-20180508/anat/sub-amit_ses-20180508_acq-MPRAGE_T1w.nii.gz get sub-amit/ses-20180508/dwi/sub-amit_ses-20180508_acq-DTIX30Xp2Xs4_dwi.nii.gz (transfer already in progress, or unable to take transfer lock)
Unable to access these remotes: origin
Try making some of these repositories available:
6384a551-a41d-4290-b186-9258befede97 -- bids@rolando:/inbox/BIDS/dbic/QA
7d9ed214-3e5f-4cc8-ac88-f397145b2d4c -- yoh@falkor:/srv/datasets.datalad.org/www/dbic/QA [origin]
ba8f2cea-f229-422c-82be-6580e5e07ed5 -- yoh@smaug:/mnt/datasets/datalad/crawl/dbic/QA
failed
get sub-amit/ses-20180508/func/sub-amit_ses-20180508_task-rest_acq-p2_bold.nii.gz (from origin...)
(from origin...)
thread blocked indefinitely in an STM transaction
thread blocked indefinitely in an MVar operation
Unable to access these remotes: origin
Unable to access these remotes: origin
Try making some of these repositories available:
6384a551-a41d-4290-b186-9258befede97 -- bids@rolando:/inbox/BIDS/dbic/QA
7d9ed214-3e5f-4cc8-ac88-f397145b2d4c -- yoh@falkor:/srv/datasets.datalad.org/www/dbic/QA [origin]
ba8f2cea-f229-422c-82be-6580e5e07ed5 -- yoh@smaug:/mnt/datasets/datalad/crawl/dbic/QA
failed
Try making some of these repositories available:
6384a551-a41d-4290-b186-9258befede97 -- bids@rolando:/inbox/BIDS/dbic/QA
7d9ed214-3e5f-4cc8-ac88-f397145b2d4c -- yoh@falkor:/srv/datasets.datalad.org/www/dbic/QA [origin]
ba8f2cea-f229-422c-82be-6580e5e07ed5 -- yoh@smaug:/mnt/datasets/datalad/crawl/dbic/QA
failed
git-annex: thread blocked indefinitely in an STM transaction
\"\"\"]]
I hope that would give enough clues, Joey?
"""]]

10
doc/bugs/multiple_ssh_prompts__44___and_thread_blocked_indefinitely_in_an___63____63____63___transaction/comment_18_01371e8b9d56d4470af5277456763d12._comment Normal file
View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="yarikoptic"
avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
subject="comment 18"
date="2018-11-19T04:08:37Z"
content="""
FWIW 1, it was a switch from ghc 8.0.1-17+b1 because it was when (I guess) I switched from Debian stable (stretch) to testing (buster) build environment since build-dependencies were no longer satisfiable!
FWIW 2, I have now tested the build in debian unstable with ghc 8.4.3+dfsg1-4 -- the same issue with locks
"""]]

36
doc/bugs/multiple_ssh_prompts__44___and_thread_blocked_indefinitely_in_an___63____63____63___transaction/comment_19_a171911a9bb7a278e673e4b6905c971d._comment Normal file
View file

@ -0,0 +1,36 @@
[[!comment format=mdwn
username="yarikoptic"
avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
subject="dependencies information"
date="2018-11-19T15:54:14Z"
content="""
not sure how I was blind that `git annex version` outputs dependencies information, so here is too nearby versions where
this one is good (no lock issues, but double password prompt):
[[!format sh \"\"\"
*[yhalchen@discovery7 QA]$ git annex version
git-annex version: 6.20180926-gc906aaf
build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify ConcurrentOutput TorrentParser MagicMime Feeds Testsuite
dependency versions: aws-0.17.1 bloomfilter-2.0.1.0 cryptonite-0.23 DAV-1.3.1 feed-0.3.12.0 ghc-8.0.2 http-client-0.5.7.0 persistent-sqlite-2.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.4.5
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external
operating system: linux x86_64
supported repository versions: 3 5 6
upgrade supported from repository versions: 0 1 2 3 4 5
local repository version: 5
\"\"\"]]
and here is a bad one (locks issue)
[[!format sh \"\"\"
*[yhalchen@discovery7 QA]$ PATH=$HOME/6.20180926+git93-gdc47dfe/usr/lib/git-annex.linux:$PATH git annex version
git-annex version: 6.20180926+git93-gdc47dfe90-1~ndall+1
build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Testsuite
dependency versions: aws-0.19 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.2 feed-1.0.0.0 ghc-8.2.2 http-client-0.5.12 persistent-sqlite-2.8.1.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external
operating system: linux x86_64
supported repository versions: 3 5 6
upgrade supported from repository versions: 0 1 2 3 4 5
local repository version: 5
\"\"\"]]
"""]]

33
doc/bugs/standalone_debian_pkg_build_fails_in_sid.mdwn Normal file
View file

@ -0,0 +1,33 @@
### Please describe the problem.
Trying to build debian pkg from current master in sid which has newer ghc (8.4.3+dfsg1-4) to see if it would somehow affect [multiple ssh prompt/locking](http://git-annex.branchable.com/bugs/multiple_ssh_prompts__44___and_thread_blocked_indefinitely_in_an___63____63____63___transaction/) issue. But build fails with unclear to me
[[!format sh """
OK (0.68s)
addurl: On branch master
nothing to commit, working tree clean
Configuration does not allow accessing file:///build/git-annex-7.20181105+git134-gf39db41d2/.t/tmprepo388/myurl
download failed: Configuration does not allow accessing file:///build/git-annex-7.20181105+git134-gf39db41d2/.t/tmprepo388/myurl
OK (0.33s)
All 293 tests passed (264.37s)
git-annex: ExitSuccess
failed
git-annex: test: 1 failed
git-annex: ExitFailure 1
failed
git-annex: test: 1 failed
make[1]: *** [Makefile:78: test] Error 1
make[1]: Leaving directory '/build/git-annex-7.20181105+git134-gf39db41d2'
dh_auto_test: make V=1 -j1 test returned exit code 2
make: *** [debian/rules:15: build-arch] Error 2
"""]]
So it is `All 293 tests passed (264.37s)` but then some additional test fails
Note that was 7.20181105+git134-gf39db41d2 and i386

8
doc/bugs/standalone_debian_pkg_build_fails_in_sid/comment_1_a159b01bd28fde7544343ef2fcfbea56._comment Normal file
View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="yarikoptic"
avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
subject="comment 1"
date="2018-11-19T03:18:23Z"
content="""
apparently doesn't build in buster either -- so it is the problem of the 7.20181105+git134-gf39db41d2 not the environment
"""]]

3
doc/forum/export_single_file.mdwn Normal file
View file

@ -0,0 +1,3 @@
I am following the instructions at [tips/publishing_your_files_to_the_public](https://git-annex.branchable.com/tips/publishing_your_files_to_the_public/) to export trees to a public remote. Often I want to just share a single file publicly with someone, not an entire folder (which is what export supports).
For example I am using [git-annex-remote-googledrive](https://github.com/Lykos153/git-annex-remote-googledrive) and I have created a new folder with public read access from drive.google.com. If I set this folder up as an exporttree remote I could then export a local annexed sub-folder to it. What I really want though is to pick a single file to share publicly with someone and send that to my public folder. Does anyone have thoughts on an existing workflow that might achieve this?

8
doc/forum/export_single_file/comment_1_36d97e3c3f07411f3d367da546759ea5._comment Normal file
View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="andrew"
avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
subject="comment 1"
date="2018-11-17T18:17:27Z"
content="""
And some more detail… I am implementing a feature in [git-annex-turtle](https://github.com/andrewringler/git-annex-turtle) where you can right-click on a file, select Share and have that file uploaded to some public place, then i'll let the user know the public URL so they can share it (perhaps by placing it in their copy-paste buffer). I thought at first I could use [git-annex export](http://git-annex.branchable.com/git-annex-export/) but I don't think it is really designed for this use case. I think export is meant to export a tree to a non-git-ish place and then merge changes into that remote tree as you make changes locally.
"""]]

10
doc/forum/export_single_file/comment_2_ec3d3fb7beb3312571ffa471c39aaf95._comment Normal file
View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="andrew"
avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
subject="comment 2"
date="2018-11-17T19:13:24Z"
content="""
OK. It looks like one way I could achieve this is by following these instructions [old method](https://git-annex.branchable.com/tips/publishing_your_files_to_the_public/old_method/). Basically just `git annex copy --to` the file to a public s3 repo without encryption and without chunking and with `public=yes` then grab the URL.
It seems this should also work for any remote that supports `WHEREIS Key` for getting a URL from any key.
"""]]

7
doc/todo/add_prefix_option_to_export.mdwn Normal file
View file

@ -0,0 +1,7 @@
Could we add a prefix option to [git-annex-export](https://git-annex.branchable.com/git-annex-export/)?
Something like `git annex export master:some-videos --to myexport --prefix share-with-john` would create a new subdirectory called `share-with-john` on the `myexport` exporttree remote and copy all files from the local `some-videos` directory into the new `share-with-john` directory.
I could then do another export using the same remote like `git annex export master:some-other-videos --to myexport --prefix share-with-bill` which wouldn't touch any of the videos I previously shared with john but would create a new export into a new `share-with-bill` directory.
My goal with the prefix option is to setup an exporttree remote one time, but then be able to re-use this same remote multiple times to create independent publicly shared folders.