mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

Merge branch 'master' of ssh://git-annex.branchable.com

Browse source
This commit is contained in:
Joey Hess 2018-11-20 20:23:09 -04:00
commit 16873782e5
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
7 changed files with 207 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

21
doc/bugs/assistant_cannot_sync_files_to_empty_indirect_repo/comment_1_af9b44674bafb63de05b888b90878325._comment Normal file
View file

@ -0,0 +1,21 @@
[[!comment format=mdwn
username="andrew"
avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
subject="comment 1"
date="2018-11-20T21:42:11Z"
content="""
Also note from the log above that `whereis` on repo1 reports that repo2 has a copy of the file even though git failed to merge into repo1 and even though repo1 does not actually have the file.
andrew@bumblebee /Users/Shared/repo1$ git-annex whereis a.txt
whereis a.txt (2 copies)
746bb051-e2c5-463b-8d4c-7d4eee6de855 -- andrew@bumblebee.local:/Users/Shared/repo1 [here]
87af3666-1f63-4ad1-8a1e-a1d260c90751 -- andrew@bumblebee.local:/Users/Shared/repo2 [repo2]
ok
andrew@bumblebee /Users/Shared/repo1$ cd ../repo2
andrew@bumblebee /Users/Shared/repo2$ date
Mon Nov 12 11:29:27 EST 2018
andrew@bumblebee /Users/Shared/repo2$ ls
andrew@bumblebee /Users/Shared/repo2$ git-annex whereis a.txt
git-annex: a.txt not found
git-annex: whereis: 1 failed
"""]]

8
doc/bugs/assistant_cannot_sync_files_to_empty_indirect_repo/comment_2_789149eb53162a11cbafde399215e40e._comment Normal file
View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="andrew"
avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
subject="comment 2"
date="2018-11-20T21:56:03Z"
content="""
I have the same issue with two version 7 repos.
"""]]

130
doc/bugs/dropunused_does_nothing.mdwn Normal file
View file

@ -0,0 +1,130 @@
### Please describe the problem.
there are files in my repo that i cant get rid of.
$ git annex unused
unused . (checking for unused data...) (checking master...)
Some annexed data is no longer used by any files:
NUMBER KEY
1 SHA256E-s5480575--396855045e70ff04dd233f9fcf4e0b42cd25c73a0b0144de53c44bb3ed67972f.done.tif
2 SHA256E-s6133555--b394dd9889feded875396fc32ce98ff9b206bc203ad826b316f0da98969ba5f9.done.tif
3 SHA256E-s5409067--6db75a42d002f6ab52ff5dc57ad9c65bcb34fa6371f22134532546b269cd47f6.done.tif
4 SHA256E-s6431447--76582d83e519d5c1286d073f330b10213ae05c17a4baf5434fe786fce1ac7c23.done.tif
5 SHA256E-s6234195--019cb086986c258e746dbff9f188d58f0f561824ca2d2a888fafb12e7ac1c6ce.done.tif
6 SHA256E-s6993729--79c73a0402181b0c557249243fa5f593b6284027d5fb7f74fb9ac6deb8800235.done.tif
7 SHA256E-s6046931--68522cefd3b009367c1efcc3ae002dce9d405fd3ae27b52ae7c9076349d2c58b.done.tif
8 SHA256E-s5909711--8541a1d4aa4ef1d6a0b6de9f13648f8c97aedd0d371f26ddf1c4a52fb05063b6.done.tif
9 SHA256E-s5872619--5e2ba7852213574878a72e5a1867c6c0154ed487fa633def60e5398bd44ca68c.done.tif
10 SHA256E-s6473539--2353ac9d339afe417e9069e7a7d69b8a8cf1370fef489e27f0e19cee7a3e86bd.done.tif
11 SHA256E-s6218095--0f32ecb974fa9ed1c7736f728385cd1ee3b89e4d6b41f2c5e4c00497e40932cd.done.tif
12 SHA256E-s6573461--202751374b7f1d5872c0e6afc251467036dae782eb2b301a1b79a8927abe0180.done.tif
13 SHA256E-s6441483--bbda783c7f27e90180de717b09c412a3b3500cdc7cdcf84dddb6f222232c9156.done.tif
14 SHA256E-s7265337--abfe156e61f18b9cb93ecf64ffe77e206782554dbebe05b9118057f9cb4a97a1.done.tif
15 SHA256E-s6779531--a6f9912f86d6f734b031e0deb76d7a4015c2884e0e21b22464656056bd993357.done.tif
16 SHA256E-s5216075--8c5a57b75eb8687643f139e7ae82742253e6dfc11b6b76e8053f806257d18999.done.tif
17 SHA256E-s5559823--2a7276cae88871489c8ae20c2164f4f4fa66ad06b79db63daa74de35ed433d55.done.tif
18 SHA256E-s6055855--f9cd8d75dd568abf3980f0c0374d95b66f7f196639142efb0fa37b2917650ec0.done.tif
19 SHA256E-s6506577--a9b79e417801ac365f6ab509ff4cadf713e74fe83bdae601fe5c35b3114705a1.done.tif
(To see where data was previously used, try: git log --stat -S'KEY')
To remove unwanted data: git-annex dropunused NUMBER
and i do a git annex dropunused --force all i get this
dropunused 1 ok
dropunused 2 ok
dropunused 3 ok
dropunused 4 ok
dropunused 5 ok
dropunused 6 ok
dropunused 7 ok
dropunused 8 ok
dropunused 9 ok
dropunused 10 ok
dropunused 11 ok
dropunused 12 ok
dropunused 13 ok
dropunused 14 ok
dropunused 15 ok
dropunused 16 ok
dropunused 17 ok
dropunused 18 ok
dropunused 19 ok
and i have this again:
$ git annex unused
unused . (checking for unused data...) (checking master...)
Some annexed data is no longer used by any files:
NUMBER KEY
1 SHA256E-s5480575--396855045e70ff04dd233f9fcf4e0b42cd25c73a0b0144de53c44bb3ed67972f.done.tif
2 SHA256E-s6133555--b394dd9889feded875396fc32ce98ff9b206bc203ad826b316f0da98969ba5f9.done.tif
3 SHA256E-s5409067--6db75a42d002f6ab52ff5dc57ad9c65bcb34fa6371f22134532546b269cd47f6.done.tif
4 SHA256E-s6431447--76582d83e519d5c1286d073f330b10213ae05c17a4baf5434fe786fce1ac7c23.done.tif
5 SHA256E-s6234195--019cb086986c258e746dbff9f188d58f0f561824ca2d2a888fafb12e7ac1c6ce.done.tif
6 SHA256E-s6993729--79c73a0402181b0c557249243fa5f593b6284027d5fb7f74fb9ac6deb8800235.done.tif
7 SHA256E-s6046931--68522cefd3b009367c1efcc3ae002dce9d405fd3ae27b52ae7c9076349d2c58b.done.tif
8 SHA256E-s5909711--8541a1d4aa4ef1d6a0b6de9f13648f8c97aedd0d371f26ddf1c4a52fb05063b6.done.tif
9 SHA256E-s5872619--5e2ba7852213574878a72e5a1867c6c0154ed487fa633def60e5398bd44ca68c.done.tif
10 SHA256E-s6473539--2353ac9d339afe417e9069e7a7d69b8a8cf1370fef489e27f0e19cee7a3e86bd.done.tif
11 SHA256E-s6218095--0f32ecb974fa9ed1c7736f728385cd1ee3b89e4d6b41f2c5e4c00497e40932cd.done.tif
12 SHA256E-s6573461--202751374b7f1d5872c0e6afc251467036dae782eb2b301a1b79a8927abe0180.done.tif
13 SHA256E-s6441483--bbda783c7f27e90180de717b09c412a3b3500cdc7cdcf84dddb6f222232c9156.done.tif
14 SHA256E-s7265337--abfe156e61f18b9cb93ecf64ffe77e206782554dbebe05b9118057f9cb4a97a1.done.tif
15 SHA256E-s6779531--a6f9912f86d6f734b031e0deb76d7a4015c2884e0e21b22464656056bd993357.done.tif
16 SHA256E-s5216075--8c5a57b75eb8687643f139e7ae82742253e6dfc11b6b76e8053f806257d18999.done.tif
17 SHA256E-s5559823--2a7276cae88871489c8ae20c2164f4f4fa66ad06b79db63daa74de35ed433d55.done.tif
18 SHA256E-s6055855--f9cd8d75dd568abf3980f0c0374d95b66f7f196639142efb0fa37b2917650ec0.done.tif
19 SHA256E-s6506577--a9b79e417801ac365f6ab509ff4cadf713e74fe83bdae601fe5c35b3114705a1.done.tif
(To see where data was previously used, try: git log --stat -S'KEY')
To remove unwanted data: git-annex dropunused NUMBER
### What version of git-annex are you using? On what operating system?
im using a ubuntu bionic with the latest standalone version:
$ git annex version
git-annex version: 7.20181106-g352f88226
build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite
dependency versions: aws-0.19 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.2 feed-1.0.0.0 ghc-8.2.2 http-client-0.5.13 persistent-sqlite-2.8.1.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B25»
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external
operating system: linux x86_64
supported repository versions: 5 7
upgrade supported from repository versions: 0 1 2 3 4 5 6
local repository version: 7
$ git annex info
repository mode: indirect
trusted repositories: 0
semitrusted repositories: 5
00000000-0000-0000-0000-000000000001 -- web
00000000-0000-0000-0000-000000000002 -- bittorrent
007097b0-39f5-42c8-b85f-76f128cdaffa -- [jay]
72033f8d-940b-4999-9a18-d1897c217308 -- preuss@CKC-BS-N0288:~/Bilder [here]
d9effa4c-961c-4521-a2c7-38f4ebbc455e -- marv@rorschach:~/Bilder [rorschach]
untrusted repositories: 0
transfers in progress: none
available local disk space: 155.98 gigabytes (+1 megabyte reserved)
local annex keys: 18096
local annex size: 86.67 gigabytes
annexed files in working tree: 18531
size of annexed files in working tree: 86.84 gigabytes
bloom filter size: 32 mebibytes (3.6% full)
backend usage:
SHA256E: 18531
### Please provide any additional information below.
i have recorded a asciinema: https://asciinema.org/a/O8ZjZp2TVO3mnuB4ZtmJxgceC
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
yeah. i try to use git-annex for just everything. lately im building a little script of using git-annex to save content from my blog. i love it alot :)

12
doc/devblog/day_552_523__v7_release_prep/comment_1_31c6692172dbc1217d0a7ab90c074823._comment Normal file
View file

@ -0,0 +1,12 @@
[[!comment format=mdwn
username="anarcat"
avatar="http://cdn.libravatar.org/avatar/4ad594c1e13211c1ad9edb81ce5110b7"
subject="apenwarr on mtimes"
date="2018-11-20T19:16:59Z"
content="""
this is pretty much the most thorough overview of mtime and related issues I've ever read:
https://apenwarr.ca/log/20181113
warmly recommend it. he basically gave up on mtime: they're tracked and important, but not in isolation. he also tracks file size, inode numbers, etc.
"""]]

8
doc/forum/Assistant_in_windows_only_watches_one_repo_on_startup/comment_1_48b43fe2ec186cea5930bf641522081d._comment Normal file
View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="andrew"
avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
subject="comment 1"
date="2018-11-20T20:43:50Z"
content="""
What is the output of `git annex assistant --autostart --debug` and `git annex version`?
"""]]

23
doc/todo/add_prefix_option_to_export/comment_3_c76aa611a2b389b9d7a0512a733c0d03._comment Normal file
View file

@ -0,0 +1,23 @@
[[!comment format=mdwn
username="andrew"
avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
subject="comment 3"
date="2018-11-20T17:45:34Z"
content="""
OK. Excellent, yes `git mktree` sounds promising, although I might still need an additional option to `export` to achieve my goals (maybe `--append-only` see last paragraph). Here is more detail on my use case:
I am looking to add a `Share` feature to [git-annex-turtle](https://github.com/andrewringler/git-annex-turtle). Often, I have one file, or a few files that I wish to share with people. Typically I will upload these to google drive and then create a public share link from the entire folder they are in, then share that with people over email. With `git-annex-turtle` I would like a user to be able to right-click on a file, a multiple-file selection or a folder, choose `Share` and then have these files shared somewhere publicly, then report back to the user the public URL of this share (via their clipboard or a dialog).
I don't know what remote types would be desirable for `git-annex-turtle` users, but I was hoping that I could find a solution that could work for any remote type that already supports public file downloads and the `exporttree` option. Probably rsync, s3 and google drive would be a good first start. s3 and google drive already support public downloads and a savy user could certainly make their rsync remote support public downloads as well.
The issue with *“initializing one remote per export directory”* is that I would like to minimize effort for my `git-annex-turtle` users. I think it is reasonable to ask them to setup one public remote one time, then they can re-use that remote anytime they want to share something new. But I wouldn't want to have to ask them to create a new remote everytime they click the `Share` button. It is certainly plausible that I could automate this process, but then I would need to investigate storing security credentials and the various authentication mechanism for various supported remotes and write code to auto-generate new remotes of various types on the fly.
Using `git mktree` seems very promising since I could pass it an arbitrary set of 1 or more files or folders that are already present in `git`, create a new tree then share that tree using the export command. One problem with this is that it becomes tricky when I want to share another tree, without deleting previously exported trees (right?). I think, if I could keep track of all trees previously shared I could create a new tree containing all of the old trees and the new tree. But, I don't want to do this, because `git-annex-turtle` is designed to store no critical file or content information that can't be automatically recreated from the git repositories themselves.
I am wondering if adding an option like `--append-only` to the `export` command would resolve this issue? This option would disable the entire merge process, never deleting content from a remote, only ever adding. I could then create new trees using `git mktree` anytime I want to do a new share and just do the export of that new tree with the `--append-only` option and not have to worry about `git annex` trying to merge changes and delete previously exported trees? Or perhaps this isn't any easier than a `--prefix` option since the `export` command needs to locally keep track of what it exported? Perhaps there could be a new command instead of `export`? Some kind of command that supports any remote that already supports the `exporttree` option? Perhaps something like `git annex copy-tree treeish --to the-public-remote` would copy a tree to a remote using something similar to the `export` mechanism but would never attempt to do any merging and would never keep track of what was uploaded. Or perhaps the `--append-only` option to `export` could behave similarly, never keeping track locally of what was uploaded.
"""]]

5
doc/todo/assistant_should_detect_added_remotes.mdwn Normal file
View file

@ -0,0 +1,5 @@
I think it would be useful if the assistant (when monitoring a repo) could automatically detect when the user adds a new remote to that repo. For example, right now, if I have two local repos (`repo1` and `repo2`) neither with any remotes and each is listed in `~/.config/git-annex/autostart` and I run `git annex assistant --autostart` then the assistant will detect files added to each remote but won't sync between them (since they have no remotes) per design.
If I then add each repo as a remote of the other (from the command-line), assistant will still not sync files between the repos until I stop all the assistants running and then restart them. Presumably only on launch does the assistant check the list of remotes?
I think this is perhaps causing issues for users not just on the command-line but also for users who create multiple local remotes from the webapp and then combine them, since the webapp is perhaps not restarting the assistant daemons after the combine operation? I'm not sure about this…