Merge branch 'master' of ssh://git-annex.branchable.com

This commit is contained in:
Joey Hess 2020-12-22 16:55:15 -04:00
commit d239a55bd1
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
5 changed files with 213 additions and 0 deletions

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="jkniiv"
avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d"
subject="well done!"
date="2020-12-21T18:14:26Z"
content="""
Version 8.20201128-g6b1357482 seems to work exactly as it's written on the tin. Thanks, Joey!
"""]]

View file

@ -0,0 +1,84 @@
### Please describe the problem.
Matching queries containing `-(`/`-)` without an explicit `--and`/`--or` next to them are interpreted strangely.
(I'm not sure of the full space of queries that have an issue, but that describes what I've seen so far.)
### What steps will reproduce the problem?
[[!format sh """
cd $(mktemp -d)
git init
git annex init
touch a.1 a.2 b.1 b.2
git annex add
PS4='================ '
set -x
git annex find --include=a.* --include=*.1 # 'a.1' -- good (implicit --and)
git annex find --include=a.* --and '-(' --include=*.1 --or --include=*.2 '-)' # 'a.1 a.2' -- good
git annex find --include=a.* --or '-(' --include=*.1 --or --include=*.2 '-)' # 'a.1 a.2 b.1 b.2' -- good
git annex find --include=a.* '-(' --include=*.1 --or --include=*.2 '-)' # 'a.1 a.2 b.2' -- ???
"""]]
Leaving out the `--and`/`--or` yields a result that is not the same as providing either one explicitly. The output is the same as `--include=a.* --or --include=*.2`, or perhaps `-( --include=a.* --include=*.1 -) --or --include=*.2`.
`git annex find --want-get` with the equivalent preferred content expressions gives the same results.
[[!format sh """
git annex wanted . 'include=a.* include=*.1'; git annex find --want-get
git annex wanted . 'include=a.* and (include=*.1 or include=*.2)'; git annex find --want-get
git annex wanted . 'include=a.* or (include=*.1 or include=*.2)'; git annex find --want-get
git annex wanted . 'include=a.* (include=*.1 or include=*.2)'; git annex find --want-get
"""]]
### What version of git-annex are you using? On what operating system?
Freshly built from source on Arch Linux:
[[!format text """
git-annex version: 8.20201128-ge0062c4f9
build flags: Assistant Webapp Pairing Inotify TorrentParser Feeds Testsuite S3 WebDAV
dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso hook external
operating system: linux x86_64
supported repository versions: 8
upgrade supported from repository versions: 0 1 2 3 4 5 6 7
local repository version: 8
"""]]
### Please provide any additional information below.
Output of the first script above:
[[!format text """
Initialized empty Git repository in /tmp/tmp.ZdtfLaDkA8/.git/
init (scanning for unlocked files...)
ok
(recording state in git...)
add a.1
ok
add a.2
ok
add b.1
ok
add b.2
ok
(recording state in git...)
================ git annex find '--include=a.*' '--include=*.1'
a.1
================ git annex find '--include=a.*' --and '-(' '--include=*.1' --or '--include=*.2' '-)'
a.1
a.2
================ git annex find '--include=a.*' --or '-(' '--include=*.1' --or '--include=*.2' '-)'
a.1
a.2
b.1
b.2
================ git annex find '--include=a.*' '-(' '--include=*.1' --or '--include=*.2' '-)'
a.1
a.2
b.2
"""]]
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
Yes! [My description from before](https://git-annex.branchable.com/bugs/async_external_special_remote__39__s_stdin_not_closed/) still applies. And I've continued growing my annex and expanded onto a new drive since then.

View file

@ -0,0 +1,84 @@
[[!comment format=mdwn
username="amerlyq+annex@12121d70a602f094228689a0a24d348d478a8af1"
nickname="amerlyq+annex"
avatar="http://cdn.libravatar.org/avatar/3d63c9f436b45570d45bd003e468cbd3"
subject="adb special remote"
date="2020-12-21T06:36:32Z"
content="""
Does fixed \"largefiles\" currently works only for \"directory\" remote?
What about \"adb\"? Or the next problem is the general one?
# [How-To Reproduce]: git-annex version: 8.20201127-gdad8442572
cd /tmp
mkdir books
cd books
git init .
git config --local user.name annex
git config --local user.email test
git annex init test
git annex config --set annex.largefiles 'not mimetype=text/*'
echo hello > README.rst
git add README.rst
git commit -m init
mkdir new old
for f in old/1 old/2 new/3 new/4; do (head -c100 /dev/zero;echo $f) > $f; done
git annex add
git commit -m annex
git-annex initremote android type=adb androiddirectory=/storage/0000-0000/books encryption=none exporttree=yes importtree=yes
git config remote.android.annex-tracking-branch master
Syncing twice will result in replacing \"git\" file by \"annexed\" one.
Moreover, it will be using SHA1 instead of default SHA256E (and totally ignore annex.backend option), which is confusing.
$ git annex sync --content
commit
On branch master
nothing to commit, working tree clean
ok
list android ok
update refs/remotes/android/master ok
Already up to date!
Merge made by the 'recursive' strategy.
export android README.rst
/tmp/amer/export140847-1: 1 file pushed, 0 skipped. 0.1 MB/s (6 bytes in 0.000s)
ok
export android new/3
.git/annex/objects/kJ/Q4/SHA256E-s106--29...0 skipped. 1.3 MB/s (106 bytes in 0.000s)
ok
export android new/4
.git/annex/objects/5X/pv/SHA256E-s106--be...0 skipped. 1.9 MB/s (106 bytes in 0.000s)
ok
export android old/1
.git/annex/objects/1k/J6/SHA256E-s106--d5...0 skipped. 2.3 MB/s (106 bytes in 0.000s)
ok
export android old/2
.git/annex/objects/Gw/31/SHA256E-s106--52...0 skipped. 1.7 MB/s (106 bytes in 0.000s)
ok
(recording state in git...)
$ git annex sync --content
commit
On branch master
nothing to commit, working tree clean
ok
list android ok
update refs/remotes/android/master ok
Updating 803974d..f340cca
Fast-forward
README.rst | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
mode change 100644 => 120000 README.rst
(recording state in git...)
$ readlink README.rst
.git/annex/objects/gK/P3/SHA1--ce013625030ba8dba906f756967f9e9ca394464a/SHA1--ce013625030ba8dba906f756967f9e9ca394464a
I seem not able to find related bug/todo/forum and can't wrap around my head what's going on here, assuming it was fixed.
"""]]

View file

@ -0,0 +1,9 @@
What is the best way to balance files among your remotes?
use case:
I have git annex remotes on N external drives, I would like to git annex move --auto --from=$most_used_repo --to= $least_used_repo
expected result:
the free space on all my remotes should be more or less equal.
is there a way to do this currently?

View file

@ -0,0 +1,28 @@
[[!comment format=mdwn
username="amerlyq+annex@12121d70a602f094228689a0a24d348d478a8af1"
nickname="amerlyq+annex"
avatar="http://cdn.libravatar.org/avatar/3d63c9f436b45570d45bd003e468cbd3"
subject=""mirror" workflow: usage of "wanted" expression is inconsistent with regular annex remote"
date="2020-12-21T07:34:27Z"
content="""
Desired workflow with \"adb\":
* my repo of books is large :: dld/ new/ old/ read/ ... (so can't sync whole repo 1:1 to phone and *must exclude old/ directory*)
* usually I download books on phone, rarely on PC, and place them into dld/, then promising ones are moved into new/ and then read/ and each month lukewarm remainings in dld/ are moved into old/$date
* so files are never edited but moved extensively between folders -- and books added from both phone and PC are expected to have the same backend (currently PC uses SHA256 but why phone is unconditionally prefers SHA1 ???)
* this repo also contains textual journal with recorded progress (so I set largefiles to *not mimefile=text/* and failed [Mixed_content_repos_with_import_and_export](https://git-annex.branchable.com/forum/Mixed_content_repos_with_import_and_export/#comment-764ac971faf756140055333649ffb94c))
* expecting to fully mirror _hierarchy_ of files and its changes on both phone and PC (therefore I *use \"master\"* instead of \"master:android\" to prevent manually syncing duplicate hierarchy from subdir)
* expecting to NOT copy unwanted old/ files from PC to phone when exporting (works OK)
* expecting to auto-move all old/ files from phone to PC when importing (like usual \"wanted\" expression for \"group=source\" -- but they are ignored instead)
* expecting \"delete\" on phone will delete file only in \"android/master\" BUT not in \"master\" -- e.g. add symlinks back in merge commit when doing \"git annex sync --content --not-only-annex\" like described in \"import after limited export\"
Concerns:
* FAIL:(special-remote): using \"git annex wanted android 'exclude=old/\*'\" **ignores** present files in old/ directory on phone and **deletes** all old/ symlinks on PC after sync.
* BUT:(git-annex): proper remote with same \"wanted\" expression behaves differently -- it tranfers old/ files content to other remotes and then drops them in current one (but keeps symlink untouched, not deleted like above)
* Why \"import after limited export\" is marked \"done\" if unwanted files are deleted? Or is it my misconception? <https://git-annex.branchable.com/todo/export_preferred_content/>
So, maybe expressions for \"wanted content on remote\" and \"completely ignored in ALL remotes\" must be two different options?
However the current behavior of this \"ignored\" thing is too much counter-intuitive, making its existence questionable.
Otherwise how can I accomplish abovementioned workflow i.e. make \"adb special remote\" to visibly behave during sync as regular annex repo w/o listed quirks?
"""]]