Merge branch 'master' of ssh://git-annex.branchable.com
This commit is contained in:
Joey Hess
commit
9e8f884739
9 changed files with 239 additions and 0 deletions
152
doc/bugs/annex-rsync-upload-options_are_ignored.mdwn
Normal file
152
doc/bugs/annex-rsync-upload-options_are_ignored.mdwn
Normal file
|
|
@ -0,0 +1,152 @@
|
|||
### Please describe the problem.
|
||||
git-annex doesn't seem to honor the remote.\<name\>.annex-rsync-upload-options or annex.rsync-upload-options anymore. The problem seems to be on the server side. I noticed it when upgrading the ancient git-annex on my server from the version in stretch (6.20170101, --bwlimit worked fine with that) to the version in stretch-backports (7.20190129, --bwlimit broken). On the client I'm running bullseye and git-annex version 8.20210223. I then upgraded git-annex on the server to 8.20210903, the rsync-options don't work with that either.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
#### Client:
|
||||
|
||||
root@dition:/mnt/sneakerdisk9# mkdir testannex
|
||||
root@dition:/mnt/sneakerdisk9# cd testannex
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git init
|
||||
hint: Using 'master' as the name for the initial branch. This default branch name
|
||||
hint: is subject to change. To configure the initial branch name to use in all
|
||||
hint: of your new repositories, which will suppress this warning, call:
|
||||
hint:
|
||||
hint: git config --global init.defaultBranch <name>
|
||||
hint:
|
||||
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
|
||||
hint: 'development'. The just-created branch can be renamed via this command:
|
||||
hint:
|
||||
hint: git branch -m <name>
|
||||
Initialized empty Git repository in /mnt/sneakerdisk9/testannex/.git/
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git annex init dition
|
||||
init dition (scanning for unlocked files...)
|
||||
ok
|
||||
(recording state in git...)
|
||||
root@dition:/mnt/sneakerdisk9/testannex# dd if=/dev/random of=bigtestfile bs=1MiB count=500
|
||||
500+0 records in
|
||||
500+0 records out
|
||||
524288000 bytes (524 MB, 500 MiB) copied, 6.00526 s, 87.3 MB/s
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git annex add bigtestfile
|
||||
add bigtestfile
|
||||
ok
|
||||
(recording state in git...)
|
||||
|
||||
#### Server:
|
||||
|
||||
dition@holm:/mnt/btrfs/testannex.git$ git init --bare
|
||||
Initialized empty Git repository in /mnt/btrfs/testannex.git/
|
||||
|
||||
#### Client:
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git remote add holm ssh://dition@holm:1322/mnt/btrfs/testannex.git
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git push --all holm
|
||||
Enumerating objects: 10, done.
|
||||
Counting objects: 100% (10/10), done.
|
||||
Delta compression using up to 4 threads
|
||||
Compressing objects: 100% (7/7), done.
|
||||
Writing objects: 100% (10/10), 871 bytes | 217.00 KiB/s, done.
|
||||
Total 10 (delta 0), reused 0 (delta 0), pack-reused 0
|
||||
To ssh://holm:1322/mnt/btrfs/testannex.git
|
||||
* [new branch] git-annex -> git-annex
|
||||
|
||||
#### Server:
|
||||
|
||||
dition@holm:/mnt/btrfs/testannex.git$ git annex init holm
|
||||
init holm ok
|
||||
(recording state in git...)
|
||||
|
||||
#### Client:
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git config --add annex.rsync-upload-options "--bwlimit 1024"
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git annex sync
|
||||
commit
|
||||
[master (root-commit) d0f28fe] git-annex in dition
|
||||
Committer: root <root@dition.tulpe>
|
||||
Your name and email address were configured automatically based
|
||||
on your username and hostname. Please check that they are accurate.
|
||||
You can suppress this message by setting them explicitly. Run the
|
||||
following command and follow the instructions in your editor to edit
|
||||
your configuration file:
|
||||
|
||||
git config --global --edit
|
||||
|
||||
After doing this, you may fix the identity used for this commit with:
|
||||
|
||||
git commit --amend --reset-author
|
||||
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 120000 bigtestfile
|
||||
ok
|
||||
pull holm
|
||||
remote: Counting objects: 3, done.
|
||||
remote: Compressing objects: 100% (3/3), done.
|
||||
remote: Total 3 (delta 0), reused 0 (delta 0)
|
||||
Unpacking objects: 100% (3/3), 363 bytes | 121.00 KiB/s, done.
|
||||
From ssh://holm:1322/mnt/btrfs/testannex
|
||||
5ace97f..70df03f git-annex -> holm/git-annex
|
||||
ok
|
||||
(merging holm/git-annex into git-annex...)
|
||||
push holm
|
||||
Enumerating objects: 3, done.
|
||||
Counting objects: 100% (3/3), done.
|
||||
Delta compression using up to 4 threads
|
||||
Compressing objects: 100% (2/2), done.
|
||||
Writing objects: 100% (3/3), 314 bytes | 314.00 KiB/s, done.
|
||||
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
|
||||
To ssh://holm:1322/mnt/btrfs/testannex.git
|
||||
* [new branch] git-annex -> synced/git-annex
|
||||
* [new branch] master -> synced/master
|
||||
ok
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git annex copy --to holm bigtestfile
|
||||
copy bigtestfile (to holm...)
|
||||
4% 19.27 MiB 3 MiB/s 2m31s
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
#### Server:
|
||||
Debian stretch amd64 with git-annex from the neurodebian repo
|
||||
|
||||
dition@holm:/mnt/btrfs/testannex.git$ git annex version
|
||||
git-annex version: 8.20210903-1~ndall+1
|
||||
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV
|
||||
dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1
|
||||
persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 S
|
||||
HA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B51
|
||||
2 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256
|
||||
BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WOR
|
||||
M URL X*
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg
|
||||
hook external
|
||||
operating system: linux x86_64
|
||||
supported repository versions: 8
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5 6 7
|
||||
|
||||
#### Client:
|
||||
|
||||
Debian bullseye arm64
|
||||
|
||||
root@dition:/mnt/sneakerdisk9# git annex version
|
||||
git-annex version: 8.20210223
|
||||
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV
|
||||
dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
|
||||
operating system: linux aarch64
|
||||
supported repository versions: 8
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5 6 7
|
||||
|
||||
I can also reproduce this with a debian buster client on amd64 with git-annex 8.20200330
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
As mentioned, the problem was introduced somewhere between 6.20170101 and 7.20190129.
|
||||
|
||||
annex.rsync-download-options "--bwlimit 1024" is ignored as well:
|
||||
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git config --add annex.rsync-download-options "--bwlimit 1024"
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git annex get --from holm bigtestfile
|
||||
get bigtestfile (from holm...)
|
||||
10% 51.82 MiB 19 MiB/s 23s
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
I've been using git-annex for many years with great success. It's a pleasure to use, I don't know how I ever managed to get by without it!
|
||||
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
Hello, thanks for all the work in Git-annex! We've been using since 2018 for Integrated Circuits (ASICs) design with a lot of success.
|
||||
|
||||
One feature that would help us would be to be able to Drop Unused, but not all Unused.
|
||||
In our design flow, every 2 or 3 weeks, we update a collection of Objects, replacing, "overwriting" the objects with newer version.
|
||||
So the older becomes Unused.
|
||||
|
||||
But we want to keep a certain history depth of Unused. And this cannot be based on dates.
|
||||
Because when we stop working on a certain block/module, the Unused Objects can become quite old (like 6 months), but we still want to keep like for example 3x Unused Objects.
|
||||
|
||||
Example: we have overwritten 12x times the same Object "module_a.jpg". So we have 11x Unused versions of "module_a.jpg".
|
||||
We want to keep the latest and +3x latest Unused. So on total only 8x Unused copies would get Dropped.
|
||||
|
||||
Is there a way to do that with Git-annex?
|
||||
Or we would need to create a custom script to parse the git-repository, list all the 12x keys of "module_a.jpg", and pick the 8x for Dropping?
|
||||
|
||||
Thank you.
|
||||
Best regards.
|
||||
Davi Castro.
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="Lukey"
|
||||
avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
|
||||
subject="comment 1"
|
||||
date="2021-09-16T15:00:49Z"
|
||||
content="""
|
||||
Yes, by setting the `annex.addunlocked` config option to `false`
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
[[!comment format=mdwn
|
||||
username="weinzwang"
|
||||
avatar="http://cdn.libravatar.org/avatar/e73d7d9e358f3b974d283fb0834cc5d9"
|
||||
subject="comment 2"
|
||||
date="2021-09-16T20:56:29Z"
|
||||
content="""
|
||||
Thanks for the hint - this doesn't seem to make any difference though. If I unterstand the manpage correctly, adding files in locked form is the default anyway. I tried it to be sure:
|
||||
|
||||
# git config --get annex.addunlocked
|
||||
false
|
||||
|
||||
and restarted the assistant. The behaviour is the same as before, files are added unlocked.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="Lukey"
|
||||
avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
|
||||
subject="comment 3"
|
||||
date="2021-09-17T15:24:47Z"
|
||||
content="""
|
||||
It could also be due to being on a adjusted unlocked branch. Try `git checkout master`
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
[[!comment format=mdwn
|
||||
username="weinzwang"
|
||||
avatar="http://cdn.libravatar.org/avatar/e73d7d9e358f3b974d283fb0834cc5d9"
|
||||
subject="comment 4"
|
||||
date="2021-09-17T18:57:41Z"
|
||||
content="""
|
||||
That's also not it:
|
||||
|
||||
### git branch
|
||||
git-annex
|
||||
* master
|
||||
synced/master
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="joshmen"
|
||||
avatar="http://cdn.libravatar.org/avatar/b182b7f2c9a8b168dd912bfe55bf2c2c"
|
||||
subject="Same here"
|
||||
date="2021-09-19T17:03:26Z"
|
||||
content="""
|
||||
Confirming that assistant also adds files unlocked for me and I haven't found a way to change this (this is one of the main reasons I've moved away from using assistant to manual sync). If there is a way to change this behavior, I'd be glad to know about it!
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
The [[walkthrough|https://git-annex.branchable.com/walkthrough/#index3h2]] says:
|
||||
|
||||
> Notice that both repos are set up as remotes of one another. This lets either get annexed files from the other. You'll want to do that even if you are using git in a more centralized fashion.
|
||||
|
||||
This is not always practical. I use a centralized setup where all of my devices sync (manually) with a bare repo on a server at home over the internet. Clients can access the server but the server cannot access the clients.
|
||||
|
||||
Is it safe to use Git Annex in this way? Are there any pitfalls or gotchas I should be aware of? Is this setup any more (or less) risky when using a non-bare central repo?
|
||||
|
||||
Thanks for this great piece of software!
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
[[!comment format=mdwn
|
||||
username="gabrielhidasy@c3d26e2c0b3e669d012f06736616088b42ad0dbe"
|
||||
nickname="gabrielhidasy"
|
||||
avatar="http://cdn.libravatar.org/avatar/8ae80704e96034c418e126326da2e7c8"
|
||||
subject="Closing the loop"
|
||||
date="2021-09-17T23:56:46Z"
|
||||
content="""
|
||||
The metadata solution worked well, except that my cloud remote is unable to run the assistant to pull files from the desktop, and the assistant in the desktop was not pushing.
|
||||
That is fixed by running ```git annex sync --content cloud``` every few minutes in the desktop.
|
||||
"""]]
|
||||
Loading…
Reference in a new issue