diff --git a/doc/bugs/10.20220525_cannot_probe_for_untrusted_remote_key.mdwn b/doc/bugs/10.20220525_cannot_probe_for_untrusted_remote_key.mdwn deleted file mode 100644 index 499a732e69..0000000000 --- a/doc/bugs/10.20220525_cannot_probe_for_untrusted_remote_key.mdwn +++ /dev/null @@ -1,26 +0,0 @@ -### Please describe the problem. -10.20220525 forcefully untrusts `exporttree=yes`, causing `fsck` to unconditionally fail when probing for a key that only exists on such a remote. - -The full story is at https://github.com/datalad/datalad-next/issues/72 - -The summary is that an application needs to know whether a particular key is on such a remote, and it can only be on that remote (there is no other). Because the availability info from `whereis` cannot be trusted (same rational as the one causing the change in git-annex implementation) it uses `annex fsck --fast --key -f ...` which now fails. - -Parsing the JSON output of the failing `fsck` call seems to be the only way left to accomplish this goal. - -``` -% git -C myclone/.git/dl-repoannex/origin/repoannex -c diff.ignoreSubmodules=none annex fsck -f origin --fast --key XDLRA--refs -c annex.dotfiles=true --json - Only these untrusted locations may have copies of XDLRA--refs - d8735795-663e-4702-8f7e-8c684264a9df -- [origin] - Back it up to trusted locations with git-annex copy. -{"dead":[],"command":"fsck","note":"(Avoid this check by running: git annex dead --key )","success":false,"input":[],"untrusted":[{"here":false,"uuid":"d8735795-663e-4702-8f7e-8c684264a9df","description":"[origin]"}],"key":"XDLRA--refs","error-messages":[],"file":null} -fsck: 1 failed -``` - -That however, is substantially more complex than looking at the exit code -- given that the entire call is about a single key. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -99.9% are just splendid. Wondering here whether this particular consequence of the change was intentional and avoidable? - - -> [[notabug|done]] --[[Joey]] diff --git a/doc/bugs/10.20220525_cannot_probe_for_untrusted_remote_key/comment_1_e35bef8fb0b9a40938022c43c87df597._comment b/doc/bugs/10.20220525_cannot_probe_for_untrusted_remote_key/comment_1_e35bef8fb0b9a40938022c43c87df597._comment deleted file mode 100644 index 5df59eb22b..0000000000 --- a/doc/bugs/10.20220525_cannot_probe_for_untrusted_remote_key/comment_1_e35bef8fb0b9a40938022c43c87df597._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-06-09T17:19:07Z" - content=""" -That was a fix for an old reversion, so trying to use fsck for that would -have also failed before 8.20201129. - -Fsck is not the right tool for the job. Fscking can fail for any of a -variety of reasons, including probably reasons that have not been thought -of or implemented yet, so taking failure to mean only one possible thing is -doomed. - -A better tool exists: `git-annex checkpresentkey` -"""]] diff --git a/doc/bugs/Adjust_--unlock_not_using_--reflink__63__.mdwn b/doc/bugs/Adjust_--unlock_not_using_--reflink__63__.mdwn deleted file mode 100644 index ad2187ae8b..0000000000 --- a/doc/bugs/Adjust_--unlock_not_using_--reflink__63__.mdwn +++ /dev/null @@ -1,20 +0,0 @@ -### Please describe the problem. - -Running adjust --unlock is unexpectedly slow and seems to use a lot of space, even on BTRFS, suggesting it probably does not use --reflink=auto like most other commands. - -### What steps will reproduce the problem? - -Run adjust --unlock with very large files. - -### What version of git-annex are you using? On what operating system? - -6.20170101-1+deb9u1 on Debian Stretch - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Yes I have! I've used it manage lots of video editing disks before, and am now migrating several slightly different copies of 15TB sized documentary footage from random USB3 disks and LTO tapes to a RAID server with BTRFS. - -> Yay, this got fixed some time ago, by making the smudge filter not output -> the file content at all, but instead the link, and having a git hook -> later run git-annex to populate the files, which it does use reflink for -> when possible. [[done]] --[[Joey]] diff --git a/doc/bugs/Adjust_--unlock_not_using_--reflink__63__/comment_1_c42743828ddc62f7df7f612c713ee309._comment b/doc/bugs/Adjust_--unlock_not_using_--reflink__63__/comment_1_c42743828ddc62f7df7f612c713ee309._comment deleted file mode 100644 index e30471b84c..0000000000 --- a/doc/bugs/Adjust_--unlock_not_using_--reflink__63__/comment_1_c42743828ddc62f7df7f612c713ee309._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2017-12-05T17:24:24Z" - content=""" -What happens is `git annex adjust` creates a branch and runs `git checkout` -to check it out. Then git calls `git-annex smudge` on files, and -unfortunately the git smudge interface requires that git-annex output the -whole content of the file, to stdout, which git then writes to disk. - -So yes, this doesn't use reflinks, but it's worse than that, a better -interface would let git-annex simply move or hard-link the file into place, -even on filesystems not supporting reflinks. - -This is discussed in detail in [[todo/smudge]]. Unfortunately it will need -a better interface in git to be addressed, and that's why v6 is still an -experimental feature. -"""]] diff --git a/doc/bugs/Android_termux_install_fails_with_Bad_system_call.mdwn b/doc/bugs/Android_termux_install_fails_with_Bad_system_call.mdwn deleted file mode 100644 index 819bab56f0..0000000000 --- a/doc/bugs/Android_termux_install_fails_with_Bad_system_call.mdwn +++ /dev/null @@ -1,16 +0,0 @@ -Following the installation instructions for termux fails on multiple -phones (including my own) with "Bad system call" when it comes to running -git-annex. - -My guess as to what has happened is that the arm64 build has been updated -to use a newer version of glibc etc, and that is trying to use a syscall -that the Android kernel is too old to support. (Kernel 4.19 in my phone's -case.) - -Unsure when it broke, but the arm64 build is built on Debian testing -for whatever reason, so it's will need a reasonably recent kernel. Perhaps -there should be an equivilant of the i386-ancient build for older arm64 -systems? arm64 has by now been around as long as the Debian jessie used for -i386-ancient. --[[Joey]] - -> [[done]] --[[Joey]] diff --git a/doc/bugs/Broken_symlinks_in_directory_remote_causes_crash.mdwn b/doc/bugs/Broken_symlinks_in_directory_remote_causes_crash.mdwn deleted file mode 100644 index db5ebbddb8..0000000000 --- a/doc/bugs/Broken_symlinks_in_directory_remote_causes_crash.mdwn +++ /dev/null @@ -1,37 +0,0 @@ -### Please describe the problem. - -I have a directory remote with importtree=yes. In that remote, I have some symlinks that are broken. (Long story; this is a file server and they work on the system that has mounted them, but are broken here.) - -### What steps will reproduce the problem? - -I've added it with `git config remote.source.annex-tracking-branch main:$REPO`. When I run `git annex sync`, I get: - -``` -commit -On branch adjusted/main(unlocked) -nothing to commit, working tree clean -ok -list source -git-annex: Unable to list contents of source: [redacted]: getFileStatus: does not exist (No such file or directory) -failed -git-annex: sync: 1 failed -``` - -### What version of git-annex are you using? On what operating system? - -8.20210223-2 on Debian - -### Please provide any additional information below. - -I would like git-annex to either: - -1. Store the symlink as a symlink, or -2. Ignore bad symlinks - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Loading in other parts of my photo collection as we speak! - -> [[fixed|done]], by ignoring the broken symlink. (There is a todo -> about importing symlinks, ->[[todo/import_symlinks_when_importing_from_directory]])--[[Joey]] diff --git a/doc/bugs/CRLF_breaks_interoperability_Win-Ux__58__..post-receive.mdwn b/doc/bugs/CRLF_breaks_interoperability_Win-Ux__58__..post-receive.mdwn deleted file mode 100644 index 06d01733c7..0000000000 --- a/doc/bugs/CRLF_breaks_interoperability_Win-Ux__58__..post-receive.mdwn +++ /dev/null @@ -1,64 +0,0 @@ -### Please describe the problem. - -I have created a git-annex via Windows. If I use sshfs to mount this -onto a Linux server, and then setup a new remote, I encounter the -error: - - fatal: cannot run hooks/post-receive: No such file or directory - -I've already identified that, due to the Windows created repository, -the script uses crlf line-endings. And of course, the Linux kernel -interprets the carriage-return as a trailing component of the shebang - - #!/bin/sh^M - -and obviously can't find the executable. - -I've already worked around this by using a binfmt_misc approach -workaround, registering the shebang with ^M to trigger a wrapper -script. But I'm seeking the permanent fix of git-annex creating these -hooks with unix line-endings only. I think it's fair to say, there's -currently no approach that allows hooks to be in any other form than -unix scripts, and I'm happy to stand corrected, that it makes little -sense to have any other line-endings. - -### What steps will reproduce the problem? - -Run git annex sync against a Windows created remote git-annex, however -that is mounted, on a Linux/unix host. - -### What version of git-annex are you using? On what operating system? - - $ git annex version - git-annex version: 8.20210223 - -on a Debian bullseye operating system. - -### Please provide any additional information below. - -[[!format sh """ -$ git annex sync -On branch master -Your branch is ahead of 'origin/master' by 1 commit. - (use "git push" to publish your local commits) - - -It took 3.28 seconds to enumerate untracked files. 'status -uno' -may speed it up, but you have to be careful not to forget to add -new files yourself (see 'git help status'). -nothing to commit, working tree clean -ok -pull origin -ok -push origin -fatal: cannot run hooks/post-receive: No such file or directory -To /var/tmp/mnt/winhost-w10-5920/cygdrive/e/my.gitannex/ - a0ebd1826d..15cfd91da4 git-annex -> synced/git-annex -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Yes. I think git-annex is a hidden gem of the open source -community. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/CRLF_breaks_interoperability_Win-Ux__58__..post-receive/comment_1_a9501e2f347b957e7dc53a06fe59cff4._comment b/doc/bugs/CRLF_breaks_interoperability_Win-Ux__58__..post-receive/comment_1_a9501e2f347b957e7dc53a06fe59cff4._comment deleted file mode 100644 index dadc7b49e4..0000000000 --- a/doc/bugs/CRLF_breaks_interoperability_Win-Ux__58__..post-receive/comment_1_a9501e2f347b957e7dc53a06fe59cff4._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-11-28T17:57:19Z" - content=""" -You can edit the hooks that git-annex installs however you need to. It will -not overwrite modified hook files. If you edit this hook to have unix line -endings, does the hook still work when using this repository on windows? - -Actually, I'm having difficulty seeing how the CR could have gotten into -that hook. `mkHookScript` uses `unlines` which behaves the same on Windows -as on Linux, so in either case lines should be separated only by `\n`. - -This makes me wonder if something else modified the hook after git-annex -wrote it. If so, `git-annex init` should complain that the hook is -modified. -"""]] diff --git a/doc/bugs/CRLF_breaks_interoperability_Win-Ux__58__..post-receive/comment_2_e4cac0d18fd6cde1084e93d0fe997f40._comment b/doc/bugs/CRLF_breaks_interoperability_Win-Ux__58__..post-receive/comment_2_e4cac0d18fd6cde1084e93d0fe997f40._comment deleted file mode 100644 index a9deb1126d..0000000000 --- a/doc/bugs/CRLF_breaks_interoperability_Win-Ux__58__..post-receive/comment_2_e4cac0d18fd6cde1084e93d0fe997f40._comment +++ /dev/null @@ -1,25 +0,0 @@ -[[!comment format=mdwn - username="beryllium@5bc3c32eb8156390f96e363e4ba38976567425ec" - avatar="http://cdn.libravatar.org/avatar/62b67d68e918b381e7e9dd6a96c16137" - subject="comment 2" - date="2023-01-05T06:48:58Z" - content=""" -Apologies for the delay in response. - -> You can edit the hooks that git-annex installs however you need to. It will not overwrite modified hook files. If you edit this hook to have unix line endings, does the hook still work when using this repository on windows? - -Thanks, that aligns with what I surmised. I didn't apply that type of workaround for the sake of the report being as linear as possible (no alterations outside of running the git-annex commands). - -Yes, of course, if I run dos2unix on the hook file, the repository works for both Linux and Windows. The latter, I understand because it is using msys to run the script, which is flexible with both forms of line-endings. - -> Actually, I'm having difficulty seeing how the CR could have gotten into that hook. mkHookScript uses unlines which behaves the same on Windows as on Linux, so in either case lines should be separated only by \n. - -My understanding has always been... native implementations on Windows (ie. no posix emulation linkage), if you don't call open() for \"binary\", your line-feeds indicate line termination only, and then the native line ending is written to the file, ie. Windows text files. So Windows native Python, Java, etc implementations, behave this way. - -If I understand correctly, git-annex is relying on a native build of GHC? And it's library behaves that way too? Is that right? - -> This makes me wonder if something else modified the hook after git-annex wrote it. If so, git-annex init should complain that the hook is modified. - -As per above, I doubt that there was any alterations here. I tried again with a clean scenario, and indeed, the hook files were written as described. - -"""]] diff --git a/doc/bugs/CRLF_breaks_interoperability_Win-Ux__58__..post-receive/comment_3_a3a702176a316d6b599eb2a149a293ad._comment b/doc/bugs/CRLF_breaks_interoperability_Win-Ux__58__..post-receive/comment_3_a3a702176a316d6b599eb2a149a293ad._comment deleted file mode 100644 index 48585a67b2..0000000000 --- a/doc/bugs/CRLF_breaks_interoperability_Win-Ux__58__..post-receive/comment_3_a3a702176a316d6b599eb2a149a293ad._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="beryllium@5bc3c32eb8156390f96e363e4ba38976567425ec" - avatar="http://cdn.libravatar.org/avatar/62b67d68e918b381e7e9dd6a96c16137" - subject="comment 3" - date="2023-01-05T07:16:30Z" - content=""" -I should add, and this is not a commentary on the implementation of git-annex, etc... but having limited experience with BSD, and understanding that it uses different line endings for text files (???)... and other considerations... the git hooks mechanism is powerful, but seemingly presents challenges for interoperability. - -Maybe the right approach is actually... as I did... let the end-user find their own workaround. It seems it's not really a git-annex fault after all. - -I initially thought that git-annex should write the file out correctly... but it kind of is. It's writing it out for the local sensibilities. - -In this particular case, it's my challenge to solve, not the tools... if you agree, I'd be content to have this closed with no action. - -But I leave it to your better judgement. - -"""]] diff --git a/doc/bugs/CRLF_breaks_interoperability_Win-Ux__58__..post-receive/comment_4_82103077cde68c3b6a70dcf70c4ed837._comment b/doc/bugs/CRLF_breaks_interoperability_Win-Ux__58__..post-receive/comment_4_82103077cde68c3b6a70dcf70c4ed837._comment deleted file mode 100644 index a54f932f79..0000000000 --- a/doc/bugs/CRLF_breaks_interoperability_Win-Ux__58__..post-receive/comment_4_82103077cde68c3b6a70dcf70c4ed837._comment +++ /dev/null @@ -1,22 +0,0 @@ -[[!comment format=mdwn - username="sb-beryllium@6e2c477eac63b823bd315ef8aaf5f93173c1f15b" - nickname="sb-beryllium" - avatar="http://cdn.libravatar.org/avatar/ef62105380b73ef91d760ec327c14e22" - subject="comment 4" - date="2023-01-07T13:05:23Z" - content=""" -Apologies, this is beryllium (by alias)... due to unforeseen circumstances, I have had to register with a different email address. I am hoping this is a temporary situation. - -Further apologies... I acknowledge that I am vacillating somewhat, but I've reverted to thinking that the hooks should be generated with unix line-endings (LF). - -The reason I say this is, it does appear that the pseudo-standard for files under the .git directory is that they use unix line-endings only (perhaps an actual, documented standard... I'm not sure where to look/ask to confirm it). - -This is the case for files such as .git/config, and .git/refs/** etc... under Git for Windows, and even JGit running with Windows native Java. - -So to me.. it seems to make sense that hook files should have unix line-endings exclusively. - -There are possible other mitigations... but that's where I stand with that finally. I'm still not fussed if no actions is determined to be the best course of action. - -Thanks regardless. - -"""]] diff --git a/doc/bugs/CRLF_breaks_interoperability_Win-Ux__58__..post-receive/comment_5_c22f9e3c5e438b2cc05008bc3f91b7fc._comment b/doc/bugs/CRLF_breaks_interoperability_Win-Ux__58__..post-receive/comment_5_c22f9e3c5e438b2cc05008bc3f91b7fc._comment deleted file mode 100644 index de5861f990..0000000000 --- a/doc/bugs/CRLF_breaks_interoperability_Win-Ux__58__..post-receive/comment_5_c22f9e3c5e438b2cc05008bc3f91b7fc._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2023-11-29T17:29:08Z" - content=""" -[[!commit 4e35067325022e4d8790d45557156fd1166484df]] fixed this. -Previously, git-annex did write hook scripts with CRLF line endings. - -Existing hook scripts won't be changed by git-annex, but if you delete -them, and re-run `git-annex init` it will reinstall it without the CRLF. -"""]] diff --git a/doc/bugs/Can__39__t_add_a_git_repo_to_git_annex__58_____34__Invalid_path_repo__47__.git__47__X__34___for_many_X.mdwn b/doc/bugs/Can__39__t_add_a_git_repo_to_git_annex__58_____34__Invalid_path_repo__47__.git__47__X__34___for_many_X.mdwn deleted file mode 100644 index 95a04740d7..0000000000 --- a/doc/bugs/Can__39__t_add_a_git_repo_to_git_annex__58_____34__Invalid_path_repo__47__.git__47__X__34___for_many_X.mdwn +++ /dev/null @@ -1,39 +0,0 @@ -I have some git repositories I don't edit often that I'd like to back up. I'd like to add these to my git annex, so I don't have to resort to a time-consuming hack (such as setting up a proper submodule, or bundling the repositories). - -But when I try to add a .git directory to git annex, I get a bunch of errors of the form - - git-annex: user error (xargs ["-0","git","--git-dir=/tmp/tmp.LhGN3nT9uM/annex/.git","--work-tree=/tmp/tmp.LhGN3nT9uM/annex","add","--"] exited 123) - failed - git-annex: add: 1 failed - add repo/.git/hooks/pre-push.sample ok - (Recording state in git...) - error: Invalid path 'repo/.git/hooks/pre-push.sample' - error: unable to add repo/.git/hooks/pre-push.sample to index - fatal: adding files failed - -STR: - - $ mkdir annex - $ cd annex - annex$ git init - annex$ git annex init - annex$ cd .. - - $ mkdir repo - $ cd repo - repo$ git init - repo$ cd .. - - $ mv repo annex - $ cd annex - annex$ find repo | xargs -n1 git annex add - # Lots of errors of the form above. - -You can't simply do `git annex add repo` because that will ignore the .git directory. Similarly,` git annex add .git` (which I'd think really should try to add the contents of the .git directory) ignores everything. - -I don't know what this error means. Is there a right way to work around this? - -> [[!meta title="cannot add .git/ to a git repository. even when using git-annex."]] -> [[!taglink confirmed]] (but may be out of scope for git-annex) --[[Joey]] -> > -> > [[closing|done]] as out of scope diff --git a/doc/bugs/Can__39__t_add_a_git_repo_to_git_annex__58_____34__Invalid_path_repo__47__.git__47__X__34___for_many_X/comment_1_5d23b1f02c6a0db0b43939210c427994._comment b/doc/bugs/Can__39__t_add_a_git_repo_to_git_annex__58_____34__Invalid_path_repo__47__.git__47__X__34___for_many_X/comment_1_5d23b1f02c6a0db0b43939210c427994._comment deleted file mode 100644 index ff4b30122f..0000000000 --- a/doc/bugs/Can__39__t_add_a_git_repo_to_git_annex__58_____34__Invalid_path_repo__47__.git__47__X__34___for_many_X/comment_1_5d23b1f02c6a0db0b43939210c427994._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="andy" - avatar="http://cdn.libravatar.org/avatar/1f1c630290d8b02d7faddc9fa4482c8d" - subject="comment 1" - date="2017-02-16T06:34:50Z" - content=""" -I've had the same problem; I solve it by packing the repositories into a compressed file of some sort (currently a 7z archive) and committing that to Git Annex. I have a script that makes that a one-line job. If I need to make changes to the repo, I can unpack the archive and work on the repo as usual. -"""]] diff --git a/doc/bugs/Can__39__t_add_a_git_repo_to_git_annex__58_____34__Invalid_path_repo__47__.git__47__X__34___for_many_X/comment_2_6ede677f8113fcd6215ecd68de237f06._comment b/doc/bugs/Can__39__t_add_a_git_repo_to_git_annex__58_____34__Invalid_path_repo__47__.git__47__X__34___for_many_X/comment_2_6ede677f8113fcd6215ecd68de237f06._comment deleted file mode 100644 index 260ea540e1..0000000000 --- a/doc/bugs/Can__39__t_add_a_git_repo_to_git_annex__58_____34__Invalid_path_repo__47__.git__47__X__34___for_many_X/comment_2_6ede677f8113fcd6215ecd68de237f06._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Lukey" - avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b" - subject="comment 2" - date="2022-01-13T18:10:22Z" - content=""" -I found an even better workaround, see [[tips/using_nested_git_repositories/]] -"""]] diff --git a/doc/bugs/Can__39__t_add_a_git_repo_to_git_annex__58_____34__Invalid_path_repo__47__.git__47__X__34___for_many_X/comment_3_3377120e8e246f5ec74d372015590a6c._comment b/doc/bugs/Can__39__t_add_a_git_repo_to_git_annex__58_____34__Invalid_path_repo__47__.git__47__X__34___for_many_X/comment_3_3377120e8e246f5ec74d372015590a6c._comment deleted file mode 100644 index 9ba55f760a..0000000000 --- a/doc/bugs/Can__39__t_add_a_git_repo_to_git_annex__58_____34__Invalid_path_repo__47__.git__47__X__34___for_many_X/comment_3_3377120e8e246f5ec74d372015590a6c._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2022-01-25T16:39:05Z" - content=""" -I think this is out of scope for git-annex. It's not its place to change -what files git allows to be version controlled. -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX.mdwn b/doc/bugs/Changing_sync_to_include_content_breaks_UX.mdwn deleted file mode 100644 index 14380e6964..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX.mdwn +++ /dev/null @@ -1,68 +0,0 @@ -### Please describe the problem. - -After upgrading to git-annex 10.20230626, running `git annex sync` reports: - - git-annex sync will change default behavior to operate on --content in a future version of git-annex. Recommend you explicitly use --no-content (or -g) to prepare for that change. (Or you can configure annex.synccontent) - -which appears to be a barely documented change plan (at least I cannot find it in [the git-annex dev blog](https://git-annex.branchable.com/devblog/), only in the [latest change log]((https://hackage.haskell.org/package/git-annex-10.20230626/changelog)). - -From the little that is said in [the 10.20230626 changelog](https://hackage.haskell.org/package/git-annex-10.20230626/changelog), it appears the intention is to -- **after 10 years** -- fairly quickly switch from `git annex sync` just syncing metadata (allowing git annexes to easily hold partial subsets of content), to doing a full content sync bidirectionally (apparently not allowing git annexes to hold partial subsets of content without explicit countermeasures for this behaviour breaking change). - -I can understand why users might want a `git annex sync` that syncs content. And even maybe why it might want to be the default for those users who expect, eg, "Dropbox like behaviour". - -But **changing the git annex sync default after 10 years** seems extremely user hostile. - -Especially so when changing it from something that does not copy much data (default `git annex sync --no-content`) to something which (a) potentially copies a lot of data (over what might be a slow/expensive link), and (b) will potentially fill up drives due to repopulating entire large annexes which have historically relied on having only a subset of the content available locally, if the change in behaviour (after 10 years) is not caught in time. - -The idea that users should go around every single git annex (I have dozens, with copies of those on dozens of machines and a bunch of offline drives), and make sure each one has `annex.synccontent` set, or every script that runs `git annex sync` has `git annex sync --no-content` on it, just to *restore the default of 10 years* is a pretty rough transition, and not a great user experience. - -I would really strongly suggest that you do not change the behaviour of `sync` in this way *after 10 years*. And if you want a full sync option for user friendliness, then create, eg, `git annex fullsync` which is an alias for `git annex sync --content`. - -If you no longer want to support the user model of having "incomplete annexes" (ie, all copies of a git annex must contain local copies of all data except changes made since the last sync), then the deprecation should be explicitly documented with advanced warning. - -At minimum this signficant behaviour breaking change needs to be communicated *way* better than a random change log entry, and suddenly appearing in `git annex` output as a warning the world is going to break. And it shouldn't be necessary to, eg, trawl through the git source history to try to find any context for a major planned change. - -Some slight saving graces: - -* Fortunately `git annex sync --no-content` seems to be accepted at least back to git-annex 6.x, so at least it can be added into scripts without having to also check which git annex version is running; it's just a "no op" option in everything prior to git-annex 10.20030626. - -* It looks like [`git annex config --set annex.synccontent false`](https://git-annex.branchable.com/git-annex-config/) might be carried with the repository (across syncs), reducing the number of places the new "override, back to the old default behaviour" setting has to be set (but it has to be set on every existing and new git annex, just to restore the 10 year historical behaviour) - -* The [git annex source commit](https://git.kitenet.net/index.cgi/git-annex.git/commit/?id=f93a7fce1d5272c3282ce234053d26b10dd44198) has a tiny bit more context about there needing to be a "Debian Stable release" before the default changes, which doesn't seem to be documented anywhere else; if true, then since [Debian Bookworm just released, with git-annex 10.20230126](https://packages.debian.org/bookworm/git-annex) then the change in behaviour is at least 2-3 years away, at Debian's normal stable release schedule. But this doesn't seem to be documented anywhere else. - -If the plan is that this change in default behaviour will be, eg, in 2H 2025, then I'd suggest (a) putting that planned date in the warning being issued on every run, and (b) putting that date in the documentation for [git annex sync](https://git-annex.branchable.com/git-annex-sync/) which currently just says "will change in a future version of git-annex" (which is very vague, and could be next month or a decade away). However as stated above, I'd really really strongly suggest just creating a new command, like `fullsync` for the new default behaviour, and *not* breaking backwards compatibility in behaviour. - - -### What steps will reproduce the problem? - -Upgrade to git-annex 10.20230626, run `git annex sync` in a git annex repository, without having set `annex.synccontent` (i the git-annex config, or the git config). - -Try to find documentation on this pending change; find nothing other than the changelog note foreshadowing a major change in behaviour after 10 years, and some comments in the source code. - - -### What version of git-annex are you using? On what operating system? - -git-annex 10.20230626, on macOS, installed from Home Brew. - -``` -ewen@basadi:~$ git annex version -git-annex version: 10.20230626 -build flags: Assistant Webapp Pairing FsEvents TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV -dependency versions: aws-0.24.1 bloomfilter-2.0.1.0 cryptonite-0.30 DAV-1.3.4 feed-1.3.2.1 ghc-9.4.4 http-client-0.7.13.1 persistent-sqlite-2.13.1.1 torrent-10000.1.3 uuid-1.3.15 yesod-1.6.2.1 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: darwin x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -ewen@basadi:~$ -``` - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Absolutely. I've been using git-annex since nearly the beginning; and use it extensively to maintain partial copies of large annexes on laptops/desktops (and other space constrained systems). Hence this foreshadowed sudden change in behaviour being extremely surprising, and somewhat alarming. - -Thanks for writing git-annex. It's the reason I've been sponsoring you on Patreon for years. - -Please don't break backwards compatibility. Even in the user experience :-) - -> [[closing|done]] per new plan --[[Joey]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_10_ec80bdff87c7b70e4235cd5b2ba6ed05._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_10_ec80bdff87c7b70e4235cd5b2ba6ed05._comment deleted file mode 100644 index 65d31dd535..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_10_ec80bdff87c7b70e4235cd5b2ba6ed05._comment +++ /dev/null @@ -1,51 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 10" - date="2023-07-12T16:25:48Z" - content=""" -@nobodyinperson, @ewen: Thank you for a balanced discussion! I'm not as elegant with words -but here is my take on your ideas. - -@nobodyinperson: - -> How about: - -> - `git annex assist` (exists, works): - -Yes, I understand the need for `assist` -- not in any way opposed to it. However, I think both -`assist` and `sync` ought learn a new option called `--edit` (or, `--edit-message` but that's wordy) -to be used in a similar fashion to `git commit -m msg --edit`, ie. it would open the user's preferred editor -to edit the commit message. - -> - `git annex metasync` (new proposal): - -You mention that this command would be \"Option-wise as 'dumb' as git annex assist\" but I disagree: as a true -replacement for `sync` it definitely should have at least the `--no-commit` flag as an additional option so that those -used to a `git annex add; git commit -m foo [--edit]; git annex sync --no-content --no-commit` workflow could -migrate to it. Also, no auto-adding (please!) if this is supposed be a replacement for `sync --no-content`. - -> - `git annex sync` (exists, has many options and configs): - -You mention that this command ought to be \"deprecated eventually (in many years), with a big warning upon execution now already\". -Yes, I agree but only if we get a `metasync` with a `--no-commit` flag for reasons above. - ---- - -@ewen: - -> Firstly, if `git annex assist` has been added to create the magical \"Dropbox like experience\", I don't see the need for changing the default behaviour of `git annex sync` at all. (For the record I'm glad `git annex assist` exists; I don't need it, or want it, but it clearly solves a bunch of UI problems for some users, and that's great. This is entirely about backwards incompatible UX changes in existing commands.) - -I think you have a point there that the situation pre-10.20230626 would be preferable. -But I can live with Yann's (@nobodyinperson) suggestions if there is some amount of compromising. - -> Secondly, your (@nobodyinperson) suggestion for \"improving\" the current situation seems to be: - -> 1. Entirely remove `git annex sync` (entirely breaking backwards compatiblity) and/or have it permanently display a warning that cannot be removed (annoying, incompatible with anything looking at output); and - -> 2. Add a new (ie, not in any older version) `git annex metasync` which has some of the existing functionality, and some random other (unwanted by me at least) additional functionality (auto-adding any files in a directory where git annex is run, which I definitely do not want) - -Yes, a replacement for `sync` should not in any case auto-add files. We want more control, not less. - -(continued...) -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_11_82e32aca2c23a1e27553bcd9708870b2._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_11_82e32aca2c23a1e27553bcd9708870b2._comment deleted file mode 100644 index d236976601..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_11_82e32aca2c23a1e27553bcd9708870b2._comment +++ /dev/null @@ -1,25 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 11""" - date="2023-07-12T17:03:27Z" - content=""" -Please don't engage in unwarranted speculation about strange desires of mine -to eliminate features that are crucial to git-annex. In general, avoid -inflammatory language on this website. Thank you. - -It's important that git-annex be able to make changes that make it better. -Backwards compatability is not everything. Improvements can make git-annex -more cohesive and easier to understand for new users. -This kind of pre-warned transition is commonly used by git when making -similarly impactful changes. - -annex.synccontent can be set in your global git config. It can also be set -with `git-annex config` which will affect all clones of that repo. This -will be a minor amount of work for most users to deal with. - -BTW, this transition plan actually started 6 years ago when I added -annex.syncontent configuration and --no-content. -[[!commit b77903af48e650dbb777f29e98d0c7b388353ebd]] -As you note, it's important that's already available in all commonly used -old versions of git-annex. -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_12_4e0746d00d8a596d878c1fe12ec68849._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_12_4e0746d00d8a596d878c1fe12ec68849._comment deleted file mode 100644 index 43a368a7f1..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_12_4e0746d00d8a596d878c1fe12ec68849._comment +++ /dev/null @@ -1,48 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 12" - date="2023-07-12T17:30:52Z" - content=""" -(...continued) - -@ewen: - -> My suggestion for preserving backwards compatibilty would be: - -> 1. Move the behaviour change out of `git annex sync` and into `git annex init` (rationale: it's run once per annex/machine pair, not many times per day/week, *and* it makes the change \"new annexes only\" which is much closer to \"opt in\") - -> 2. From git annex version N + 1, have git annex automatically set `annex.synccontent=true` at `git annex init` of a *new* repo (ie, not just naming the local copy of the annex), and for version N+1 to N+3 (or higher) have `git annex init` issue a warning it has defaulted sync content on now (unlike earlier versions), and describe how to find out how to turn it off (unless a global git setting is making `annex.synccontent=true` the user preferred default anyway, in which case the user doesn't need a warning of a setting matching their preferred option). - -> 3. For any annex where `annex.synccontent` is *not* set, assume it's an older annex, and use the backwards compatible, historical, default (false) *without* issuing any warnings about \"this is going to change\" (and never change that behaviour for historical annexes) - -> 4. Retain `git annex sync` forever, with it obeying `annex.synccontent` (default: false, but set to true in newly created annexes by default) - -> 5. Remove the newly added warning in `git annex sync` entirely, and just keep the functionality of the version before 10.20230623 - -Sounds good. I like the concept of an opt-in the way you laid it out. It's well thought out. - -> This would: - -> 1. Allow changing the `git annex sync` default from `metadata only` to `full content` in a \"new usage, opt in only\" way - -> 2. Provide a clear path to opt out, and a clear path to opt in existing repos to match - -> 3. Avoid needing to issue warnings to users on every run of a very core command until they set config - -> 4. Avoid breaking existing usage, and use cases - -(Yes, that warning is annoying to say the least.) - -I'm willing to accept some breakage to my workflows but I'd rather still see a separate -`git annex add`, `git commit -m ...` and \"sync metadata with one command\" workflow being supported. -The name of that sync command is not that important for me but I understand why it is for you. - -> The git annex \"meta data sync\" dance (which effectively allows `git push` into a repo that has a working directory attached, something that is normally otherwise difficult) was great, and the thing which made git annex attractive to me over other options. It'd be a shame to render that great functionality unreachable without unwanted \"some users wanted this, so all users must have this\" functionality. - -I agree to the strongest degree. I'm not a very good git user, but the core parts of git-annex -like `add` and `sync` have made me feel comfortable to a greater extent than the porcelain of core git. -I'm sure I can adapt to newfangled ways of doing my routine things with git-annex but as on \"old dog\" I'd -rather not adopt those new ways if I can avoid it. - -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_13_80ba2e4241372ff3ce1505900e17c22f._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_13_80ba2e4241372ff3ce1505900e17c22f._comment deleted file mode 100644 index fb4dc59c76..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_13_80ba2e4241372ff3ce1505900e17c22f._comment +++ /dev/null @@ -1,32 +0,0 @@ -[[!comment format=mdwn - username="ewen" - avatar="http://cdn.libravatar.org/avatar/605b2981cb52b4af268455dee7a4f64e" - subject="Plumbing commands vs Porcelain commands" - date="2023-07-13T00:26:20Z" - content=""" -@joey (re [comment 11](http://git-annex.branchable.com/bugs/Changing_sync_to_include_content_breaks_UX/#comment-e4557f089ec22304ff506f09f791bb81)): thank you for replying. I apologise if my tone sounded accusatory; in hindsight writing late at night after a stressful year probably wasn't the best idea. - -I don't know what your intentions are here, which is part of the problem. From the outside, it feels like the git-annex assistant functionality is now the dominant use case which is guiding the development changes. (And that's understandable if it's most/all of your funding to work on the project.) That seems to be leading towards the \"plumbing\" low level commands being transitioned into higher level (\"porcelain\" in git terminology) commands, which do multiple things; and access to the low level functionality being decreased or removed. - -Yes, git has done a few \"UX breaking\" transitions in the past. But in all the cases I can think of the change in functionality was accompanied with obvious documentation on why the change was being made, and how to adapt. For instance the [git safe directory behaviour change (CVE-2022-24765)](https://github.blog/2022-04-12-git-security-vulnerability-announced/) prompted a lot of documentation about why it was changed (eg, [git config manpage](https://git-scm.com/docs/git-config#Documentation/git-config.txt-safedirectory)), and specifically how users could restore the existing functionality where that was appropriate (and, eg, it gave an exact command to run in the warning output). I've been running into that problem repeatedly for a year (including supporting other users), but at least knowing why, and that the recommended \"fix command\" is the ideal solution, makes it quick to explain/fix/carry on. - -In the case of this git-annex change to `sync` (from a \"just plumbing\" meta data sync command, to a \"porcelain\" git annex assistant like command) there doesn't seem to be any clear documentation on why it is changing, or the recommended process to preserve (or recreate) the existing default functionality (meta data sync only), particularly in a backwards compatible way. - -The reference to the [six year old commit](http://source.git-annex.branchable.com/?p=source.git;a=commitdiff;h=b77903af48e650dbb777f29e98d0c7b388353ebd) which you see as \"starting this change\" adds some more context, but almost all of the \"documentation\" about this change appears to be scattered between some source code commits, some short changelog entries, and a bug that started being about something else. As someone who has followed your personal blog and the git-annex dev blog for years, the change of \"sync\" from a low level plumbing command (with optional \"please do more\") to a high level porcelain command (with optional \"please skip most of the things you want to do) still seemed to appear very suddenly, without any foreshadowing it was going to change. And I remain unclear on the intended timeline for the (fairly fundamental, IMHO) change in default behaviour. - -For my purposes I think from here on I'll be doing *all* of the following: - -1. On any system I install git-annex, running `git config --global annex.synccontent false` to set my per-user, per-machine state to equivalent to now - -2. On any git-annex wrapper scripts I have that run `sync`, explicitly calling `git annex sync --no-content` instead of a bare `git-annex sync` as before - -3. On any new git-annex repo that I set up, running `git annex config --set annex.synccontent false` - -4. On the existing git-annex repos, as I interact with them again and remember, run `git annex config --set annex.synccontent false` to maintain the existing default - -5. On new/existing git-annex repos, consider also running `git annex wanted . present` as well, since that seems to effectively match my current policy (ie, what is there is what is supposed to be there, because I put it there). - -(I appreciate that mostly any one of those \"should be sufficient\". But with a lot of existing git annex repos, spread across lots of machines/drives, some of which are offline, the risks of overlooking one or more of them later on is non-trivial. So belt and braces here.) - -Ewen -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_14_08f3c3d26d13fd44ae9c26791b90a8fc._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_14_08f3c3d26d13fd44ae9c26791b90a8fc._comment deleted file mode 100644 index f40ea41eec..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_14_08f3c3d26d13fd44ae9c26791b90a8fc._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 14""" - date="2023-07-14T00:01:27Z" - content=""" -> From the outside, it feels like the git-annex assistant functionality is -> now the dominant use case which is guiding the development changes. - -Based on what, the changes in the last release? - -You're seeing a pattern where none exists. - -> (And that's understandable if it's most/all of your funding to work on the project.) - -No such funding, not that I allow funding to determine overall direction of -this project. -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_15_38a0eafe91e05a7784dd5e305f38a8f3._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_15_38a0eafe91e05a7784dd5e305f38a8f3._comment deleted file mode 100644 index a2e6d36f5c..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_15_38a0eafe91e05a7784dd5e305f38a8f3._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 15""" - date="2023-07-15T16:09:46Z" - content=""" -My current thinking is that when a repository does not have a preferred -content expression, a content sync could avoid adding content to it. - -This would avoid surprising behavior after the transition, except for -repositories that did at some time have a preferred content expression set, -but one that no longer reflects the use of the repository. - -However, current users of `sync --content` (or of annex.synccontent) do -expect it to fill their repositories that don't have preferred content -configured. So would a sync that defaults to content syncing then need to -behave differently than an explicit `sync --content`? Or should changing -behavior in that case also be part of the transition? -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_16_ee529ac7b3d27f1cdc7edf5154c0da8f._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_16_ee529ac7b3d27f1cdc7edf5154c0da8f._comment deleted file mode 100644 index 8eeeeb694b..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_16_ee529ac7b3d27f1cdc7edf5154c0da8f._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 16" - date="2023-07-15T17:04:38Z" - content=""" -> My current thinking is that when a repository does not have a preferred content expression, a content sync could avoid adding content to it. - -Technically, that would make the default preferred content expression `present` instead of `include=*`, right? I'd like that. Less unwanted content after the initial clone when running `git annex assist` or `git annex sync --content`. Goes into the direction of [specifying the default preferred content expression](https://git-annex.branchable.com/todo/Setting_default_preferred_content_expressions/). Users having freshly cloned a repo can then explicitly `get` files or set up their preferred content expression. - -Many annexed repos will be huge in size and rarely one will want *everything* from it initially. Also great for DataLad datasets. -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_17_d60adea3f0c0030a403c33c57f27abda._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_17_d60adea3f0c0030a403c33c57f27abda._comment deleted file mode 100644 index 61bbf12ed9..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_17_d60adea3f0c0030a403c33c57f27abda._comment +++ /dev/null @@ -1,24 +0,0 @@ -[[!comment format=mdwn - username="ewen" - avatar="http://cdn.libravatar.org/avatar/605b2981cb52b4af268455dee7a4f64e" - subject="unspecified default preferred content = present" - date="2023-07-17T00:50:09Z" - content=""" -I would also be in favour of changing the default preferred content expression to be \"present\" at the same time as `sync` was changed to automatically sync content as well as metadata. The effective user experience of a user with the existing default out of box configuration would be basically equivalent (now: `sync` does not sync content by default; then: `sync` will sync content if a policy to indicate desired content is set, but the default policy is \"whatever is here is correct\"). - -If that change were made (default preferred content expression is \"present\") then: - -* I think the current warning on `sync` without `--no-content` could be removed (as the effective default change from the historical defaults is small) - -* There should probably be a warning on `git annex sync` *with* a preferred content policy explicitly set (*other than* `present`, the new default), that the `sync` without `--no-content` will \"soon\" automatically include that preferred content - -* There should probably be a warning on `git annex sync --content` where there is no preferred content policy explicitly set, that the default preferred content policy is changing, and the user should set an explicit one - -That should result in \"I do everything manually\" users (who haven't set a preferred content policy, or set it to `present`) having basically \"no effective change\". And the special cases of users (relying on the default `sync` not including their preferred content policy; relying on the existing default preferred content policy to work with `git annex sync --content`) get a reminder they'll need a smaller tweak to their config. - -I'd also be fine if *new* git annexes (ie `git annex init` after new release) were to set an explicit preferred content policy to the current default (just leaving \"no content policy set\" as the implied `present` suggested above). So new annexes would still get the \"gets everything by default\" behaviour that it seems some users want. (I do *not* want everything by default as almost always I use annexes for content that is too large to keep on my laptop/desktop.) - -Thanks for considering reducing the size of the UX change here. - -Ewen -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_18_594ae4ce426c7fd9963a22def58904d8._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_18_594ae4ce426c7fd9963a22def58904d8._comment deleted file mode 100644 index ff734a4ec7..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_18_594ae4ce426c7fd9963a22def58904d8._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 18" - date="2023-07-17T06:12:26Z" - content=""" -> I think the current warning on sync without --no-content could be removed (as the effective default change from the historical defaults is small) - -The effect in term of file location will be the same after a `git annex symc` (with `present` as new defsult - or let's call it fallback - preferred content expression) and a `git annex sync --content`. **But**: There's one thing that `--content` also changes in terms of UX: speed/runtime. In my large annex repos with many files (~5 figures), a `git annex sync --content` takes *much* longer, even if no content is eventually synced. Git Annex apparently needs to go through all files whether they want to be copied somewhere. **Maybe a short-circuit path could be introduced here when all preferred content expressions effectively turn out to be `present`**. Users might wonder why `git annex sync` suddenly takes so long now. - -In general, **a progress bar or if that's impossible another indication of what's going on between metadata syncing and content syncing** would be helpful. With this, I could agree that the warning can go away for a simple `git annex sync` with no preferred content expressions set. -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_19_bdc211b026d3b05de127f6ed5a3d19b0._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_19_bdc211b026d3b05de127f6ed5a3d19b0._comment deleted file mode 100644 index 611d045637..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_19_bdc211b026d3b05de127f6ed5a3d19b0._comment +++ /dev/null @@ -1,21 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 19""" - date="2023-08-14T16:56:48Z" - content=""" -Changing the default preferred content expression to `present` rather than -`anything` would affect other things like `get --auto` and the assistant. I -don't think that is a good idea. - -My idea in comment #15 is a special case for sync and not a wider change. - -It will have a special case either way. The current special case means -that users who have set preferred content need to remember to use `sync ---content`. The new special case will only manifest as a difference in -behavior between `sync` and `sync --content` (and `get --auto` etc) -for users with no preferred content configured. - -I do think that's a valuable narrowing of the special case. - -Gone ahead and changed the warning. -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_1_1541a9323009b20f527501ebedc9d395._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_1_1541a9323009b20f527501ebedc9d395._comment deleted file mode 100644 index a57481d6fb..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_1_1541a9323009b20f527501ebedc9d395._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 1" - date="2023-07-12T06:49:46Z" - content=""" -Simultaneously, a new command `git annex assist` has been added that does basically a single step of the assistant, so roughly `git annex add;git add -A;git annex sync --content`. That goes into the direction of your `fullsync` suggestion. - -You can read more about the reasoning for `git annex sync` syncing content by default [here](https://git-annex.branchable.com/todo/Having___39__git_annex_sync__39___optionally_add/#comment-fc813a22c713490156234567ed211277), where I suggested to add a config to `git annex sync` that would optionally add new files as well. That turned out to launch an avalanche of new commands and changes (git annex pull, git annex push, git annex assist, sync syncing conent, etc.). 😅 - -IIUC, Joeys reasoning is that `git annex sync` was incomplete/inconsistent from the start and did too many configurable things. `git annex assist` syncs the entire repo state - as the assistant. Preferred content expressions can be used to specify what files a repo wants. If you set that to `git annex wanted REPO present`, a content sync won't affect it anyway. I still think that a configurable default preferred content expression for new repos is very important, now that syncinc content by default might get the default: https://git-annex.branchable.com/todo/Setting_default_preferred_content_expressions/ -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_2_c06017f123f1ee34c7310f923086d09a._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_2_c06017f123f1ee34c7310f923086d09a._comment deleted file mode 100644 index 10b153608b..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_2_c06017f123f1ee34c7310f923086d09a._comment +++ /dev/null @@ -1,21 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="I have to agree that this change to `sync` is annoying" - date="2023-07-12T07:18:54Z" - content=""" -Frankly, as a fellow user of \"incomplete annexes\" I also find it rather jarring that having gotten used to git committing my own changes -on the command line and not relying on mere `sync` to do that (or the assistant -- I haven't found a use -for the assistant yet) but then also preferring to use invocations of `git annex sync --no-commit` (which I've abbreviated to -`git annex-sync` by way of a git alias) rather freely to record my git-level changes across remotes, -I would from here on out be forced to add the `-g` flag to all my invocations just to make sure that I don't get -an annoying warning or my then current repo doesn't cause possibly huge content to flow across the remotes because I haven't -remembered to set `annex.synccontent` appropriately. I know the ideal is to have preferred content settings -for all annexes but not every git-annex user have an innate sense of all the intricacies involved in doing that -and for them to be forced to do so is a bit too much, in my humble opinion. - -@Joey, I have a suggestion: instead of breaking the UX of `sync`, why not do that on a lesser used command `mirror` and change it -to sync git-level data in addition to files (ie. I suggest that `mirror` could be a better/shorter name for OP's `fullsync`). -Then add a flag called `--only-content` to `mirror` to restore its old behavior. I bet that would have a smaller impact on git-annex's -users than making us count our blessings with `-g`s and all. -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_3_c03b45321728155eae9470cd8bb2cc93._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_3_c03b45321728155eae9470cd8bb2cc93._comment deleted file mode 100644 index 11fcf6f9cb..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_3_c03b45321728155eae9470cd8bb2cc93._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 3" - date="2023-07-12T07:33:16Z" - content=""" -> IIUC, Joeys reasoning is that git annex sync was incomplete/inconsistent from the start and did too many configurable things. - -That's not a reason to break backwards compatibility for such a prominent command as `sync`. Just no. - -> git annex assist syncs the entire repo state - as the assistant. - -I don't want to use the assistant -- or it's CLI alter ego assist -- I want my own manicured commit messages, puh-lease! The assistant -is a whole another use case, just not mine at the moment. - -Just give us our old `sync` back. Rename it to `synk` (or, `sc`/`sk` -- as in \"sync for [k]urmudgeons\") if you want but let us have it. :) -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_4_bb6972e9870a68a98d90c6847228ab31._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_4_bb6972e9870a68a98d90c6847228ab31._comment deleted file mode 100644 index f121140811..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_4_bb6972e9870a68a98d90c6847228ab31._comment +++ /dev/null @@ -1,22 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 4" - date="2023-07-12T09:21:55Z" - content=""" -Just for the record: - -- `git annex assist -m MESSAGE` works, you can make own messages. -- `git annex assist` has nothing to do with the assistant and can be used without. It's just `git annex sync --content` plus adding new files. A very important command for newcomers to 'just sync it'. - -To get 'the old sync' back, you have several options: - -- `git config --global annex.synccontent false` for your entire local machine, overrides the next option -- `git annex config --set annex.synccontent false` for all participants of a git annex repo (yes, it syncs between machines), still overrideable by users' local git configs above -- properly set up preferred content expressions (effectively), e.g. `git annex wanted . present`. -- `git annex sync --no-content` - -If you don't use preferred content expressions to decide which repo should get which files and you want only partial checkouts with some files, then I indeed don't see why you would ever need `git annex sync --content` as you probably manually `get` the files you want. In that case one could argue that `sync` is a misleading word, you are actually only looking for a metadata sync (`msync`, `metasync`, ...?) strictly without content. - -[I was also surprised](https://git-annex.branchable.com/todo/Having___39__git_annex_sync__39___optionally_add/#comment-c8c3138128a684080e2aaafc48aedfcf) that joey opted for introducing `assist` instead of teaching `sync` how to (optionally) add new files as well and keeping the rest as it was. But I can understand the direction of his reasoning. -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_5_da9812dd37de7bba4fe2b1785bdca1ae._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_5_da9812dd37de7bba4fe2b1785bdca1ae._comment deleted file mode 100644 index 96df7c5155..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_5_da9812dd37de7bba4fe2b1785bdca1ae._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="ewen" - avatar="http://cdn.libravatar.org/avatar/605b2981cb52b4af268455dee7a4f64e" - subject="History of change" - date="2023-07-12T10:27:43Z" - content=""" -Thanks for the [pointer to the earlier bug discussing changing sync entirely](https://git-annex.branchable.com/todo/Having___39__git_annex_sync__39___optionally_add) @nobodyinperson; it's at least useful to have a reference for where all this breaking change suddenly came from given it's otherwise undocumented. I've added a back pointer comment to that bug pointing at this one, as they're clearly related discussion (and it wasn't obvious the other \"sync\" bug was at all related to the change). - -I still strongly disagree with changing \"sync\" to do something pretty different from what it's done for 10 years being a good idea. But would certainly support there being *another* command that did a \"Dropbox like full sync\". - -(Personally I'd really really prefer that \"sync\" stayed doing the same thing it's done for a decade: meta data sync, only. If we're going to have to run a new different command just to retain core \"meta data sync\" behaviour, I guess \"git annex sync --no-content\" is not much worse than any other command we'd have to remember to suddenly use instead starting version N + 1.) - -Ewen -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_6_bc8154b171c460b4590d2b32401f24a2._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_6_bc8154b171c460b4590d2b32401f24a2._comment deleted file mode 100644 index 5e9f65fbd8..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_6_bc8154b171c460b4590d2b32401f24a2._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="ewen" - avatar="http://cdn.libravatar.org/avatar/605b2981cb52b4af268455dee7a4f64e" - subject="Preferred content settings not really an alternative" - date="2023-07-12T10:53:41Z" - content=""" -To echo @jkniiv's \"preferred content settings for all annexes\", as a git annex user for at least 8 years (the annex I found this issue in dates back to 2015), I've still never figured out how \"preferred content settings\" are supposed to help in git annex -- let alone for my use cases. - -As an example, the git annex I spotted this new warning in is one I use for downloading/collecting podcasts. The downloads happen on my desktop, or sometimes a laptop. All content gets *pushed* to two of my NASes as soon as possible (for an \"archival copy\" -- immediately after download if I'm at home), and then once I've listened to the podcast I drop it from my desktop/laptop, so it's gone from the \"to listen\" queue (and historically, to recover space on smaller laptop drives). I don't want the files suddenly appearing again, unless I need to refer to it for some reason, then I do \"git annex get ...\". - -There's around 300GB of podcasts in the annex (around 7000 files) on my NASes; but the checked out version on my desktop is 9GB (about 200 files). 300GB is more space than some of my laptop internal drives have in total; and definitely more free space than any of them had, which is one of the reasons for using git annex for this purpose. - -The \"preferred content policy\" is \"if I just downloaded it I want it until I drop it, then I don't want it unless I explicitly request it again, but the two NASes should always have copies\". From [setting git annex preferred content policy](https://git-annex.branchable.com/preferred_content/), and the [preferred content policy syntax](https://git-annex.branchable.com/git-annex-preferred-content/), AFAICT \"present\" is the closest thing to the \"content policy\" I want on my desktop/laptops, ie \"trust the user, I know what I'm doing, don't automatically move files\". But up to version \"10.20230626 + N\" that was also the default without a content policy, as file content didn't automagically move around unless some other policy was explicitly set. - -Ewen -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_7_20febad4aa06d5b39668a858d2b118b1._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_7_20febad4aa06d5b39668a858d2b118b1._comment deleted file mode 100644 index d6170ac3e0..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_7_20febad4aa06d5b39668a858d2b118b1._comment +++ /dev/null @@ -1,33 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="There is already something like fullsync: `git annex assist`, suggestion: new `metasync` command" - date="2023-07-12T11:15:26Z" - content=""" -> But would certainly support there being another command that did a \"Dropbox like full sync\". - -You might have missed it above, but `git annex assist` is **exactly** that new command that does a 'Dropbox-like sync' (adding new files, adding changes, metadata sync, content sync) and as OP suggested with `fullsync`. It is there, it works, it's wonderful. I think we can put that part of the discussion to rest. - -> Personally I'd really really prefer that \"sync\" stayed doing the same thing it's done for a decade: meta data sync, only. - -Actually, `git annex sync` isn't always **just** metadata sync. If you're in a repo where someone (anyone!) decided to run `git annex config --set annex.synccontent true`, there flies your Terabytes of content around when you do your `git annex sync`, assuming it's metadata-only. At first I was a fan of teaching `git annex sync` everything with flags and configs, but I think joey is right that this is inconsistent and leads to surprises. - -If we're at it, we could get rid of the ambiguous `sync` command altogether (this is what joey also proposed for the very long run) and introduce a metadata-only sync command (`metasync`?) that **can't** sync content. Not with a setting, not with a flag, it just can't. Then that's perfect for those who only want to sync metadata and `get` files they want manually and without preferred content expressions (I suppose that's how you do it?). - -How about: - -- `git annex assist` (exists, works): - - syncs everything: metadata&content, leaves the repo completely clean and all participating remotes updated to the same state - - for those who don't want to run the assistant, but still have 'one command to sync it all' whenever they feel like it or want to specify a custom commit message. - - Preferred content expressions manage what files land where. - - Very important for newcomers as this will effectively be the only command they'll need to know to get a workflow started. -- `git annex metasync` (new proposal): - - basically `git annex sync --no-content` that **can't** ever sync content. No surprises through configs. - - Leaves the repo in a clean state and all connected remotes at the same state of metadata (the git stuff). Also adds new files! People who want more control should use the manual commands `git annex add|push|pull|etc.`. - - Option-wise as 'dumb' as `git annex assist' - - for those who never want automatic content syncing but `get|copy|move|...` stuff manually and don't want to fiddle with preferred content expressions - - current users of 'old' `git annex sync` without content syncing will eventually switch to this. -- `git annex sync` (exists, has many options and configs): - - roll back to non-content-syncing as before, for those that don't want to change their workflows yet - - deprecated eventually (in many years), with a big warning upon execution now already. -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_8_84d31b8fdde0f0ab0355a5d30760e90d._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_8_84d31b8fdde0f0ab0355a5d30760e90d._comment deleted file mode 100644 index 19b15c0bb8..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_8_84d31b8fdde0f0ab0355a5d30760e90d._comment +++ /dev/null @@ -1,43 +0,0 @@ -[[!comment format=mdwn - username="ewen" - avatar="http://cdn.libravatar.org/avatar/605b2981cb52b4af268455dee7a4f64e" - subject="Alternative proposal for better backwards compatibility" - date="2023-07-12T12:48:03Z" - content=""" -Firstly, if `git annex assist` has been added to create the magical \"Dropbox like experience\", I don't see the need for changing the default behaviour of `git annex sync` at all. (For the record I'm glad `git annex assist` exists; I don't need it, or want it, but it clearly solves a bunch of UI problems for some users, and that's great. This is entirely about backwards incompatible UX changes in existing commands.) - -Secondly, your (@nobodyinperson) suggestion for \"improving\" the current situation seems to be: - -1. Entirely remove `git annex sync` (entirely breaking backwards compatiblity) and/or have it permanently display a warning that cannot be removed (annoying, incompatible with anything looking at output); and -2. Add a new (ie, not in any older version) `git annex metasync` which has some of the existing functionality, and some random other (unwanted by me at least) additional functionality (auto-adding any files in a directory where git annex is run, which I definitely do not want) - -Which, to me, is *even more* breaking the user experience of using `git annex` for the last 8 years, than is currently happening. AFAICT there would then be *no way* at all to get the `git annex` \"just sync metadata without other unwanted behaviour\" that existed before. *And* it'd then require git annex version detection in scripts to tell if `git annex metadata --no-add-really-just-copy-metadata-honest` was available or not, or if one had to fall back to an old version of the command, which is a really user-hostile way to \"retain\" existing behaviour. (FTR in my case at least there is *zero* chance that `git annex sync` would suddenly encounter `annex.synccontent=true`, as it's just me, and I haven't set that; I don't need a \"guarantees\" it won't copy content option, except as a defense against the git annex defaults suddenly changing after 8 years.) - -My suggestion for preserving backwards compatibilty would be: - -1. Move the behaviour change out of `git annex sync` and into `git annex init` (rationale: it's run once per annex/machine pair, not many times per day/week, *and* it makes the change \"new annexes only\" which is much closer to \"opt in\") - -2. From git annex version N + 1, have git annex automatically set `annex.synccontent=true` at `git annex init` of a *new* repo (ie, not just naming the local copy of the annex), and for version N+1 to N+3 (or higher) have `git annex init` issue a warning it has defaulted sync content on now (unlike earlier versions), and describe how to find out how to turn it off (unless a global git setting is making `annex.synccontent=true` the user preferred default anyway, in which case the user doesn't need a warning of a setting matching their preferred option). - -3. For any annex where `annex.synccontent` is *not* set, assume it's an older annex, and use the backwards compatible, historical, default (false) *without* issuing any warnings about \"this is going to change\" (and never change that behaviour for historical annexes) - -4. Retain `git annex sync` forever, with it obeying `annex.synccontent` (default: false, but set to true in newly created annexes by default) - -5. Remove the newly added warning in `git annex sync` entirely, and just keep the functionality of the version before 10.20230623 - -This would: - -1. Allow changing the `git annex sync` default from `metadata only` to `full content` in a \"new usage, opt in only\" way - -2. Provide a clear path to opt out, and a clear path to opt in existing repos to match - -3. Avoid needing to issue warnings to users on every run of a very core command until they set config - -4. Avoid breaking existing usage, and use cases - -The git annex \"meta data sync\" dance (which effectively allows `git push` into a repo that has a working directory attached, something that is normally otherwise difficult) was great, and the thing which made git annex attractive to me over other options. It'd be a shame to render that great functionality unreachable without unwanted \"some users wanted this, so all users must have this\" functionality. - -Ewen - -PS: `git annex push` / `git annex pull` do not appear to be meta data only tools to replace sync; they appear to be very explicitly targetted at copying content, and to be one remote at at a time as well (where as sync is \"all known remotes\"). Historically it looks like my wrapper scripts use `git annex get ...` to retrieve content, and `git annex copy --to=REMOTE ...` to put content onto a specific remote (ie, a `put`, when there is no `put`). -"""]] diff --git a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_9_17a75fa68bf3718230d1d523b7eb0250._comment b/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_9_17a75fa68bf3718230d1d523b7eb0250._comment deleted file mode 100644 index 2c3b4cf6bb..0000000000 --- a/doc/bugs/Changing_sync_to_include_content_breaks_UX/comment_9_17a75fa68bf3718230d1d523b7eb0250._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 9" - date="2023-07-12T13:10:26Z" - content=""" -Fine with me to keep `git annex sync` at its old behaviour. I'm happy about `assist|push|pull` and would also be happy with a `metasync`. - -These are all decisions that joey will have to decide himself. He has written this amazing piece of software that I'll be forever grateful for (Thanks joey!!). If he decides that `git annex sync` syncs content, so be it. - -Your idea with moving the content-syncing into `init` sounds reasonable from a backwards-compatibility point of view. But I doubt joey will want to make `init` (or any other command) accumulate patchy stuff. -"""]] diff --git a/doc/bugs/Debian_and_MacOSX_cannot_read_eachother__39__s_S3_credentials.mdwn b/doc/bugs/Debian_and_MacOSX_cannot_read_eachother__39__s_S3_credentials.mdwn deleted file mode 100644 index b2be7e9fa2..0000000000 --- a/doc/bugs/Debian_and_MacOSX_cannot_read_eachother__39__s_S3_credentials.mdwn +++ /dev/null @@ -1,66 +0,0 @@ -### Please describe the problem. -I seem to be hitting a problem with the way git annex encrypts the credentials for S3 remotes. I am creating an S3 remote with shared encryption and embedded credentials. If I create the remote on MacOSX, enableremote fails on Debian. If I create the remote on Debian, enableremote fails on MacOSX with a similar error. I can enable MacOSX created remotes on other MacOSX systems and Debian created remotes on other Debian systems. -The enableremote creates a .git/annex/creds/uuid file, but the contents are garbled. - -Could this be an issue with different versions of GPG on the systems? - -### What steps will reproduce the problem? -- New git annex remote on Debian. -- Clone and git annex init on MacOSX -- Create S3 remote on MacOSX with encryption=shared embedcreds=yes -- Git annex sync -- On Debian enable remote - -Alternatively, create the S3 remote on Debian, sync and enable remote on MacOSX - -### What version of git-annex are you using? On what operating system? - MacOSX - gpg (GnuPG/MacGPG2) 2.0.30 - libgcrypt 1.6.6 - git-annex version: 6.20170101 - build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV FsEvents ConcurrentOutput TorrentParser MagicMime Feeds Quvi - key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL - remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external - local repository version: 5 - supported repository versions: 3 5 6 - upgrade supported from repository versions: 0 1 2 3 4 5 - operating system: darwin x86_64 - - Debian 7 - gpg (GnuPG) 1.4.12 - git-annex version: 6.20170101+gitg93d69b1-1~ndall+1 - build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Quvi - key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL - remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external - local repository version: 5 - supported repository versions: 3 5 6 - upgrade supported from repository versions: 0 1 2 3 4 5 - operating system: linux x86_64 - -### Please provide any additional information below. - -[[!format sh """ -# Debian 7 - $ git annex enableremote S3 - enableremote S3 gpg: packet(2) with unknown version 13 - gpg error above was caused by an old git-annex bug in credentials storage. Working around it.. - git-annex: bad creds - CallStack (from HasCallStack): - error, called at ./Creds.hs:116:22 in main:Creds - - # MacOSX - git annex enableremote S34 - enableremote S34 gpg: [don't know]: invalid packet (ctb=6b) - gpg error above was caused by an old git-annex bug in credentials storage. Working around it.. - git-annex: bad creds - CallStack (from HasCallStack): - error, called at ./Creds.hs:116:22 in main:Creds - -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) -Yes! git annex has been enormously helpful. Thanks so much for this tool. - - -> I guess this was caused by an old gpg, and so no point in keeping it -> open. [[done]] --[[Joey]] diff --git a/doc/bugs/Debian_and_MacOSX_cannot_read_eachother__39__s_S3_credentials/comment_1_e307e5d0ec02ed76dc7c3f0894770539._comment b/doc/bugs/Debian_and_MacOSX_cannot_read_eachother__39__s_S3_credentials/comment_1_e307e5d0ec02ed76dc7c3f0894770539._comment deleted file mode 100644 index 840bfdaa2e..0000000000 --- a/doc/bugs/Debian_and_MacOSX_cannot_read_eachother__39__s_S3_credentials/comment_1_e307e5d0ec02ed76dc7c3f0894770539._comment +++ /dev/null @@ -1,28 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2017-02-13T18:27:02Z" - content=""" -The .git/annex/creds/uuid files for S3 special remotes are supposed to -contain the S3 access credentials, the same values you set -`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` to. Since it's just some -high-entropy credentials, perhaps that's why you thought it looked garbled? - -I doubt that the error message about the cause of the gpg error is -right; that is a workaround for an old bug in git-annex 5.20140916. Unless -you used such an old version to set up the special remote, it's probably -not related to that problem at all. - -The actual problem seems to be: - - gpg: packet(2) with unknown version 13 - gpg: [don't know]: invalid packet (ctb=6b) - -This probably points to an incompatability between the gpg versions. -The easiest fix is probably to upgrade Debian to gpg 2. - -With embedcreds, the S3 creds are symmetrically encrypted/decrypted -using gpg --symmetric. This is a less common way to use gpg, -and it could be that there's an incompatability with how different -versions of gpg do it. -"""]] diff --git a/doc/bugs/Directory_remotes_with_same_mount_point.mdwn b/doc/bugs/Directory_remotes_with_same_mount_point.mdwn deleted file mode 100644 index 14d422229e..0000000000 --- a/doc/bugs/Directory_remotes_with_same_mount_point.mdwn +++ /dev/null @@ -1,55 +0,0 @@ -### Please describe the problem. - -I have a repository with two directory special remotes that use the same mount point (it's a USB SD card reader using udevil). I use "git-annex-command" with lsblk and grep using the UUID to prevent git-annex from attempting to sync with the remote if it's not mounted. When trying to import from the second remote, I get "file content has changed" for every file, which seems to be related to inode-based caching. - -Each of the remotes uses a subdirectory of the mount, though I did change the directory for the first one (it originally used the mount point directly). - -### What steps will reproduce the problem? - -1. Create a directory special remote on a removable device and import from it -2. Create a second directory special remote on a second removable device at the same mount point and import from it - - -### What version of git-annex are you using? On what operating system? - -8.20201130-g75988a790, Debian Buster - - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - -$ git annex import --from=pi1541 master:c64 -list pi1541 ok -import pi1541 nav96.d64 - - file content has changed -ok -import pi1541 fb64 - - file content has changed -ok -import pi1541 cbmcmd22.d64 - - file content has changed -ok -import pi1541 cbmcmd22.d80 - - file content has changed -ok -... - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -I've used it successfully on Linux and Android. My attempts on Windows have been less successful, though I never tried with core.ignoreStat=true. Now I'm trying to use it to manage the SD cards for various devices: my Pi1541, Odroid Go, and Sansa Clip Zip. - -> Calling this [[done]] because I'm fairly sure if it was some kind of -> inode related bug, it would be fixed now by changes to the directory -> special remote. Please do followup if you still see the bug with the most -> recent git-annex release, or have other information about how to -> reproduce the alleged behavior. --[[Joey]] diff --git a/doc/bugs/Directory_remotes_with_same_mount_point/comment_1_351063bf8fb7779a9a7ecd21bef8b801._comment b/doc/bugs/Directory_remotes_with_same_mount_point/comment_1_351063bf8fb7779a9a7ecd21bef8b801._comment deleted file mode 100644 index 73f88e9dd6..0000000000 --- a/doc/bugs/Directory_remotes_with_same_mount_point/comment_1_351063bf8fb7779a9a7ecd21bef8b801._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="seanl@fe5df935169a5440a52bdbfc5fece85cdd002d68" - nickname="seanl" - avatar="http://cdn.libravatar.org/avatar/082ffc523e71e18c45395e6115b3b373" - subject="Works fine after changing mount point" - date="2021-01-17T07:24:38Z" - content=""" -I uninstalled udevil in favor of udiskie, which uses a mount point name based on the label instead of the device, updated the directory to point at the new mount point with `git annex enableremote pi1541 /1541`, and the problem went away. But it doesn't seem like having different mount points for removable devices should be a requirement. -"""]] diff --git a/doc/bugs/Directory_remotes_with_same_mount_point/comment_2_90351193c14f4c68f9b92640dae82f13._comment b/doc/bugs/Directory_remotes_with_same_mount_point/comment_2_90351193c14f4c68f9b92640dae82f13._comment deleted file mode 100644 index 3e698483dc..0000000000 --- a/doc/bugs/Directory_remotes_with_same_mount_point/comment_2_90351193c14f4c68f9b92640dae82f13._comment +++ /dev/null @@ -1,23 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2021-01-18T16:23:42Z" - content=""" -I don't think this has anything to do with mount points being in common. -Consider that, a single disk with a git-annex repository, and 2 directory -special remotes all on it has everything under a common mount point. And -I'm pretty sure works. Also there's just nothing in the relevant code that -cares what the mount point is. - -It looks to me like the inode was somehow not the same in the "list" stage -of the command as it was in the "import" stage, or the file that was seen -in the first stage was no longer present in the second stage. I don't see -how that could be a git-annex bug. It could, for example, happen if one SD -card was mounted there at the "list" stage, and then a different card got -mounted there at the "import" stage, or even if the SD card just got -unmounted in between the two stages. - -I have not been able to replicate the problem either, using 2 removable -devices on the same mount point, 2 special remotes, etc. So you will have -to show me how to reproduce this if it's actually a bug. -"""]] diff --git a/doc/bugs/Directory_remotes_with_same_mount_point/comment_3_6b7727db8e2487394a80fc061d76868e._comment b/doc/bugs/Directory_remotes_with_same_mount_point/comment_3_6b7727db8e2487394a80fc061d76868e._comment deleted file mode 100644 index 91c72c89f9..0000000000 --- a/doc/bugs/Directory_remotes_with_same_mount_point/comment_3_6b7727db8e2487394a80fc061d76868e._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="seanl@fe5df935169a5440a52bdbfc5fece85cdd002d68" - nickname="seanl" - avatar="http://cdn.libravatar.org/avatar/082ffc523e71e18c45395e6115b3b373" - subject="Went away with fsck?" - date="2021-01-19T01:15:51Z" - content=""" -I tried unmounting the sd card, fscking, and remounting, and then I was able to import without issue. I haven't tried a sync or a second import, though. All of my SD cards use VFAT, and this one has 160k+ files on it. I'm wondering if there are cases where Linux's VFAT implementation will give different inode numbers for the same file during the same session? The only mention of inode numbers in the Linux VFAT documentation is in the NFS section, though. I'll let you know if I run into it again. -"""]] diff --git a/doc/bugs/Directory_remotes_with_same_mount_point/comment_4_5a4b3daa0f8a0d7f330f5a1d922d0f6a._comment b/doc/bugs/Directory_remotes_with_same_mount_point/comment_4_5a4b3daa0f8a0d7f330f5a1d922d0f6a._comment deleted file mode 100644 index ebfd97e4bd..0000000000 --- a/doc/bugs/Directory_remotes_with_same_mount_point/comment_4_5a4b3daa0f8a0d7f330f5a1d922d0f6a._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2021-01-19T15:40:49Z" - content=""" -I don't think linux would change inode numbers while the filesystem was -mounted. But each mount of FAT gets new inode numbers, since FAT doesn't -actually have them and linux makes them up. So the filesystem being -unmounted and remounted for some reason in between the two steps of the -import would explain the behavior. - -Note that I've opened -[[todo/import_tree_from_FAT_does_unnecessary_work_due_to_inode_instability]] -after thinking of some other consequences of this. -"""]] diff --git a/doc/bugs/Directory_remotes_with_same_mount_point/comment_5_fb21865b36819a54a23e1ccb7f377fa1._comment b/doc/bugs/Directory_remotes_with_same_mount_point/comment_5_fb21865b36819a54a23e1ccb7f377fa1._comment deleted file mode 100644 index d9e34d4eb8..0000000000 --- a/doc/bugs/Directory_remotes_with_same_mount_point/comment_5_fb21865b36819a54a23e1ccb7f377fa1._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2021-01-19T16:53:54Z" - content=""" -Well, the changes I made for that todo make changes to inodes due to -remounting in the middle of an import not cause behavior like this. Of -course I don't know that's what caused this behavior, but it does seem -likely those changes would turn out to have fixed this, if we understood -how to reproduce the problem. -"""]] diff --git a/doc/bugs/Disabling_remote_auto-enabling_not_possible.mdwn b/doc/bugs/Disabling_remote_auto-enabling_not_possible.mdwn deleted file mode 100644 index b90623b009..0000000000 --- a/doc/bugs/Disabling_remote_auto-enabling_not_possible.mdwn +++ /dev/null @@ -1,65 +0,0 @@ -### Please describe the problem. - -I have a repository with a special remote that is autoenable=true. The special remote is (no longer) accessible to everyone. I want to disable autoenable, but this fails, because the special remote is not accessible. - -How can autoenable be disabled, once a remote is no longer (or even temporarily) inaccessible to someone? - -### What steps will reproduce the problem? - -Here is a small script that demos this for a type-git special remote. However, the problem has been initially observed with different external special remotes. I only reproduced it with a git-annex built-in to see whether an implementation detail in the external special remote is causing this. - -```sh -# create an annex-repo (using datalad, but this is the standard init+init dance) -$ datalad create originds -# make a clone with datalad (same as clone+init) -$ datalad clone originds dsclone -# add the autoenabling special remote in the clone -$ git -C dsclone annex initremote myremote type=git "location=$(readlink -f originds)" autoenable=true - -# as long as `originds` is accessible everything runs smoothly -$ datalad clone dsclone autoclone1 - -# make the special remote target inaccessible -$ mv originds originds-moved - -# enabling is no longer possible -$ datalad clone dsclone autoclone2 -$ git -C autoclone2 annex enableremote myremote -... -Remote myremote cannot currently be accessed. -... - -# access by special remote name is no possible -$ git -C dsclone annex enableremote myremote autoenable=false -... -Remote myremote cannot currently be accessed. -... -git-annex: That is a normal git remote; passing these parameters does not make sense: autoenable=false - -# determine UUID from -# git -C dsclone annex info | grep myremote -# because `git -C dsclone annex info myremote | grep uuid:` no longer lists it -$ git -C dsclone annex enableremote ba87b7e1-4fec-4826-bd01-302a8e2eda0a autoenable=false -Remote myremote cannot currently be accessed. -... -enableremote ba87b7e1-4fec-4826-bd01-302a8e2eda0a error: remote myremote already exists. - -git-annex: git [Param "remote",Param "add",Param "myremote",Param "/tmp/originds"] failed -failed -enableremote: 1 failed - -# autoenable is still on -$ git -C dsclone cat-file -p git-annex:remote.log -ba87b7e1-4fec-4826-bd01-302a8e2eda0a autoenable=true location=/tmp/originds name=myremote type=git timestamp=1681453180.645186501s -``` - -### What version of git-annex are you using? On what operating system? - -I am on Debian sid. I tried git-annex versions 10.20221003 and 10.20230126 -- both show the same behavior. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Loads! This particular issue started to show up in the datalad support channels due to aging infrastructure setups with still popular datasets tracking them using autoenabling special remotes. - -> [[fixed|done]] --[[Joey]] - diff --git a/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_1_136c0fa106347bd0474f8298fde227f5._comment b/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_1_136c0fa106347bd0474f8298fde227f5._comment deleted file mode 100644 index 3e9a9c8a3b..0000000000 --- a/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_1_136c0fa106347bd0474f8298fde227f5._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-04-17T17:42:31Z" - content=""" -`git-annex dead` will work. - -But it does seem to me that there is perhaps room to have a way -to clear the autoenable setting without killing the remote entirely, -but without needing to be able to enableremote the remote. - -And I think, when you say "The special remote is (no longer) accessible to -everyone" you mean some people still have access, right? But you're not -one of those people and want to disable the autoenable. - -If so, I ought to add something like `git-annex configremote foo -autoenable=false`. -"""]] diff --git a/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_2_f6b1d513e664d1a5ee51b90f49731eeb._comment b/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_2_f6b1d513e664d1a5ee51b90f49731eeb._comment deleted file mode 100644 index 7df5268e80..0000000000 --- a/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_2_f6b1d513e664d1a5ee51b90f49731eeb._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="mih" - avatar="http://cdn.libravatar.org/avatar/f881df265a423e4f24eff27c623148fd" - subject="Confirm" - date="2023-04-18T07:54:46Z" - content=""" -Yes, in this particular case the situation is as you describe it. The remote itself is fine for a select few, but I (and others) do not have the necessary means to access it. I want to disable autoenable in my \"fork\" to streamline usage for myself and people like me. - -I think your proposed solution would work well for this scenario. Thanks! -"""]] diff --git a/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_3_c2c7144fa74bdc455e1bd36b594902c1._comment b/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_3_c2c7144fa74bdc455e1bd36b594902c1._comment deleted file mode 100644 index bbef6139e0..0000000000 --- a/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_3_c2c7144fa74bdc455e1bd36b594902c1._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2023-04-18T16:28:40Z" - content=""" -Hmm, you could dead it in your fork. The issue would be, if they ever merge -from it, their working repo will get marked as dead. - -A `git-annex configremote` would likewise cause them problems if they -merged from your fork, if they rely on autoenabling being enabled. That's -less likely to be a problem to them I suppose. -"""]] diff --git a/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_4_4f3e9f15fcc96cd98c6915ec68cc471f._comment b/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_4_4f3e9f15fcc96cd98c6915ec68cc471f._comment deleted file mode 100644 index 8b148644cf..0000000000 --- a/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_4_4f3e9f15fcc96cd98c6915ec68cc471f._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2023-04-18T16:40:57Z" - content=""" -A sub-bug here is that enableremote of a normal git remote adding autoenable=false -always fails with -"That is a normal git remote; passing these parameters does not make sense: autoenable=false". - -That happens even when the repo is accessible. So there is no way to disable -a normal git remote that has been initremoted with autoenable=true. - -Update: Fixed that. -"""]] diff --git a/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_5_ccb5b6960da43a93ff0b7404190c51de._comment b/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_5_ccb5b6960da43a93ff0b7404190c51de._comment deleted file mode 100644 index c61178fbfe..0000000000 --- a/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_5_ccb5b6960da43a93ff0b7404190c51de._comment +++ /dev/null @@ -1,33 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2023-04-18T16:49:45Z" - content=""" -I retried your example not using a regular git remote, but a special -remote. When the special remote is made inaccessible, it does not fail in -the same way. Actually, the clone doesn't fail at all: - - joey@darkstar:~/tmp/dl>datalad clone dsclone autoclone5 - [INFO ] special remote made inacessible for testing - [INFO ] Fetching updates for Dataset(/home/joey/tmp/dl/autoclone5) - update(ok): . (dataset) - update(ok): . (dataset) - configure-sibling(ok): . (sibling) - install(ok): /home/joey/tmp/dl/autoclone5 (dataset) - action summary: - configure-sibling (ok: 1) - install (ok: 1) - update (ok: 1) - -Likewise `git-annex init` displays the message from the special remote as a warning -but exits successfully and without autoenabling it. - -Compare with datalad's behavior when it is a regular git remote. The failure there -happens because git-annex auto-enables the remote successfully, and then -datalad tries to fetch from it, but the repo is not available. - -So other than the special case of a git remote stored by initremote, this bug's -real impact seems to only be that enableremote can't be used to change the -configuration of a remote that's not accessible and `git-annex init` will -warn about such a remote. -"""]] diff --git a/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_6_5b6359663efb608b9a78ab17c02be9ee._comment b/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_6_5b6359663efb608b9a78ab17c02be9ee._comment deleted file mode 100644 index 7baf64ce95..0000000000 --- a/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_6_5b6359663efb608b9a78ab17c02be9ee._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 6""" - date="2023-04-18T18:11:18Z" - content=""" -Seems that `git-annex configremote` would be very limited in what -configuration it can change. Changing configuration with enableremote -re-runs the `setup` method, which is passed a RemoteConfig with the user's -proposed changes and can either fail if the config is not valid, or return -a RemoteConfig that might not be the same as the input one. Since `setup` also -prepares to use the remote, which is what's failing when the remote is not -accessible, it could not be used by `configremote`. - -So, I think autoenable is about the only config setting that it would make -sense for configremote to be able to change. Changing anything else would -risk storing a configuration that `setup` would not allow. Which could -break the remote in arbitrary ways. -"""]] diff --git a/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_7_995b88efc83f76f14ccb22bc2a6ef4bd._comment b/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_7_995b88efc83f76f14ccb22bc2a6ef4bd._comment deleted file mode 100644 index 1750b0c764..0000000000 --- a/doc/bugs/Disabling_remote_auto-enabling_not_possible/comment_7_995b88efc83f76f14ccb22bc2a6ef4bd._comment +++ /dev/null @@ -1,7 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 7""" - date="2023-04-18T19:29:28Z" - content=""" -Implemented `git-annex configremote` -"""]] diff --git a/doc/bugs/Duplicate_descriptions_cause_confusing_error.mdwn b/doc/bugs/Duplicate_descriptions_cause_confusing_error.mdwn deleted file mode 100644 index ab40b4e527..0000000000 --- a/doc/bugs/Duplicate_descriptions_cause_confusing_error.mdwn +++ /dev/null @@ -1,46 +0,0 @@ -### Please describe the problem. - -Running `git annex info ` on a remote that shares a description with another remote shows a confusing generic error: - -``` -fatal: Not a valid object name testname -info testname (not a directory or an annexed file or a treeish or a remote or a uuid) failed -``` - -It should instead tell the user that there are multiple remotes by that description and unique identification is therefore not possible. - -This is also a thing when all remotes except for one are dead. git-annex should prefer the alive remote here and ignore all the dead ones. - -### What steps will reproduce the problem? - -1. Have a git-annex repo with multiple remotes that share a description -2. Run `git annex info ` - -### What version of git-annex are you using? On what operating system? - -``` -git-annex version: 10.20220822 -build flags: Assistant Webapp Pairing FsEvents TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22.1 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.13.1 persistent-sqlite-2.13.1.0 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2.1 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: darwin aarch64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 10 -``` - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/Enabling_useConfigOnly_not_honored.mdwn b/doc/bugs/Enabling_useConfigOnly_not_honored.mdwn deleted file mode 100644 index 13a117fc07..0000000000 --- a/doc/bugs/Enabling_useConfigOnly_not_honored.mdwn +++ /dev/null @@ -1,67 +0,0 @@ -### Please describe the problem. - -Git's `user.useConfigOnly` is not honored by git-annex commands that create commits (e.g., `init` or `sync`). - -### What steps will reproduce the problem? - -Having not globally set `user.{name,email}`, the following will cause git-annex to ignore said setting and proceed with `$USER` as identity: - - $ git config --global user.useConfigOnly true - $ git init annex && cd $_ - Initialized empty Git repository in /tmp/annex/.git/ - $ git annex init - init Author identity unknown - - *** Please tell me who you are. - - Run - - git config --global user.email "you@example.com" - git config --global user.name "Your Name" - - to set your account's default identity. - Omit --global to set the identity only in this repository. - - fatal: no email was given and auto-detection is disabled - ok - (recording state in git...) - -Looking at the git-annex branch confirms this: - - $ git log git-annex - commit 8eb575828cf52a3c150780d89c22672a51291f46 - Author: $USER <$USER> - Date: 2023-03-29 13:22:44 +0200 - - update - - commit 0b247e1da51e742bcf7b027aa25fc0d61520270d - Author: $USER <$USER> - Date: 2023-03-29 13:22:44 +0200 - - branch created - -Furthermore, the identity is now even set in the local repository: - - $ git config user.name - $USER - -Enabling `user.useConfigOnly` should prevent this, making it easier to work with several different identities. - -### What version of git-annex are you using? On what operating system? - - git-annex version: 10.20230321-gb624394c7 - build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV - dependency versions: aws-0.24 bloomfilter-2.0.1.0 cryptonite-0.30 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.13.1 persistent-sqlite-2.13.1.0 torrent-10000.1.3 uuid-1.3.15 yesod-1.6.2.1 - key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* - remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external - operating system: linux x86_64 - supported repository versions: 8 9 10 - upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 - local repository version: 10 - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Yes, it's great! - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/Enabling_useConfigOnly_not_honored/comment_1_4f0e4da06d8c9a9db771e593be1e2608._comment b/doc/bugs/Enabling_useConfigOnly_not_honored/comment_1_4f0e4da06d8c9a9db771e593be1e2608._comment deleted file mode 100644 index 6644dd83cd..0000000000 --- a/doc/bugs/Enabling_useConfigOnly_not_honored/comment_1_4f0e4da06d8c9a9db771e593be1e2608._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-04-04T18:34:32Z" - content=""" -Yeah, git-annex goes to considerable trouble to make sure it can convince -git to commit. This is because, when I didn't do that, I was *inundated* -with bug reports from users with poorly configured systems where git cannot -guess a default user.name for various reasons. - -I'm willing to support user.useConfigOnly, but only to the extent it -doesn't make me have to deal with all that again. - -And I think it's probably unncessary for git-annex to set user.name and -user.email in the git config ever. It should be able to temporarily override -them when it needs to do so. -"""]] diff --git a/doc/bugs/Enabling_useConfigOnly_not_honored/comment_2_5dede8445ba1c20e9750a30fe29de0b0._comment b/doc/bugs/Enabling_useConfigOnly_not_honored/comment_2_5dede8445ba1c20e9750a30fe29de0b0._comment deleted file mode 100644 index fccb89d2b6..0000000000 --- a/doc/bugs/Enabling_useConfigOnly_not_honored/comment_2_5dede8445ba1c20e9750a30fe29de0b0._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2023-04-04T19:17:56Z" - content=""" -I've made user.useConfigOnly be honored when set. - -Turns out it was not practical for git-annex init to not set user.name and -user.email when that is necessary to get git commit to work. But it now -avoids setting those when user.useConfigOnly is set. -"""]] diff --git a/doc/bugs/FTBFS_for_the_2nd_day.mdwn b/doc/bugs/FTBFS_for_the_2nd_day.mdwn deleted file mode 100644 index b39964410a..0000000000 --- a/doc/bugs/FTBFS_for_the_2nd_day.mdwn +++ /dev/null @@ -1,27 +0,0 @@ -### Please describe the problem. - -Got the report in the past days that failed to build across all platforms. Looked today: - -``` -[274 of 679] Compiling CmdLine.Usage ( CmdLine/Usage.hs, dist/build/git-annex/git-annex-tmp/CmdLine/Usage.o ) - -CmdLine/Usage.hs:63:1: error: - Duplicate type signatures for `paramRemote' - at CmdLine/Usage.hs:61:1-11 - CmdLine/Usage.hs:63:1-11 - | -63 | paramRemote :: String - | ^^^^^^^^^^^ - -CmdLine/Usage.hs:64:1: error: - Multiple declarations of `paramRemote' - Declared at: CmdLine/Usage.hs:62:1 - CmdLine/Usage.hs:64:1 - | -64 | paramRemote = "REPOSITORY" - | ^^^^^^^^^^^ -``` - -on Debian, didn't check if other systems are anyhow different, will bounce report with pointers just in case. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/FTBFS_for_the_2nd_day/comment_1_b7278de8f98c9fd96219281d61284461._comment b/doc/bugs/FTBFS_for_the_2nd_day/comment_1_b7278de8f98c9fd96219281d61284461._comment deleted file mode 100644 index ec15a2ab6f..0000000000 --- a/doc/bugs/FTBFS_for_the_2nd_day/comment_1_b7278de8f98c9fd96219281d61284461._comment +++ /dev/null @@ -1,29 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 1" - date="2022-10-06T06:21:40Z" - content=""" -That seems to be a simple typo where the latter/duplicate definition should be named `paramRepository` instead. -The following fixes the build: - -[[!format diff \"\"\" -diff --git a/CmdLine/Usage.hs b/CmdLine/Usage.hs -index f3434d709..65879f7c3 100644 ---- a/CmdLine/Usage.hs -+++ b/CmdLine/Usage.hs -@@ -60,8 +60,8 @@ paramNumRange :: String - paramNumRange = \"NUM|RANGE\" - paramRemote :: String - paramRemote = \"REMOTE\" --paramRemote :: String --paramRemote = \"REPOSITORY\" -+paramRepository :: String -+paramRepository = \"REPOSITORY\" - paramField :: String - paramField = \"FIELD\" - paramGlob :: String -\"\"\"]] - -Windows tests are A-OK also. -"""]] diff --git a/doc/bugs/FTBFS_for_the_2nd_day/comment_2_05c93396766cb9ac6a596db679519718._comment b/doc/bugs/FTBFS_for_the_2nd_day/comment_2_05c93396766cb9ac6a596db679519718._comment deleted file mode 100644 index afe7d6a7eb..0000000000 --- a/doc/bugs/FTBFS_for_the_2nd_day/comment_2_05c93396766cb9ac6a596db679519718._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="yarikoptic" - avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" - subject="comment 2" - date="2022-10-06T12:50:01Z" - content=""" -Thank you jkniiv! If Joey doesn't fix it soon, I will add that patch. FTR: [an example of a patch PR for our datalad/git-annex build](https://github.com/datalad/git-annex/pull/77). -"""]] diff --git a/doc/bugs/FTBFS_for_the_2nd_day/comment_3_0037f0e43383f32b22ee1ba2c98d78f8._comment b/doc/bugs/FTBFS_for_the_2nd_day/comment_3_0037f0e43383f32b22ee1ba2c98d78f8._comment deleted file mode 100644 index 8e4e5695ac..0000000000 --- a/doc/bugs/FTBFS_for_the_2nd_day/comment_3_0037f0e43383f32b22ee1ba2c98d78f8._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 3" - date="2022-10-06T16:04:41Z" - content=""" -You're welcome, Yarik! I try to do my part for git-annex although for medical reasons my personal -bandwidth is a bit limited (for instance, I don't pretend to grok Haskell much at all except at -a general functional programming level and I have even trouble with some of the intricacies of git, too). -However, I felt your last sentence was a small hint/suggestion, so with a small leap of faith I made my first -contribution to an outside project in the form of a pull request to datalad/git-annex, -[here](https://github.com/datalad/git-annex/pull/134) . :) - - -"""]] diff --git a/doc/bugs/FTBFS_since_yesterday_on_linux.mdwn b/doc/bugs/FTBFS_since_yesterday_on_linux.mdwn deleted file mode 100644 index da7068dbef..0000000000 --- a/doc/bugs/FTBFS_since_yesterday_on_linux.mdwn +++ /dev/null @@ -1,28 +0,0 @@ -### Please describe the problem. - - -thought to check on [another issue test](https://git-annex.branchable.com/todo/standlone__58___growing_number_of_libpcre.so_lookups/) but todays build failed with - -``` -FWIW, thought to look into current number but that test didn't run since build failed today due to - -``` -[34 of 35] Compiling Build.LinuxMkLibs ( Build/LinuxMkLibs.hs, Build/LinuxMkLibs.o ) - -Build/LinuxMkLibs.hs:129:17: error: - parse error (possibly incorrect indentation or mismatched brackets) - | -129 | "arm" -> - | ^ -make[3]: *** [Makefile:150: Build/Standalone] Error 1 -make[3]: Leaving directory '/home/runner/work/git-annex/git-annex/git-annex-source' -make[2]: *** [Makefile:158: linuxstandalone] Error 2 -make[2]: Leaving directory '/home/runner/work/git-annex/git-annex/git-annex-source' -make[1]: *** [debian/rules:24: override_dh_auto_build] Error 2 -make[1]: Leaving directory '/home/runner/work/git-annex/git-annex/git-annex-source' -make: *** [debian/rules:17: build] Error 2 -dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2 -I: FAIL line matches with some context -Error: Process completed with exit code 1. -``` -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop.mdwn b/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop.mdwn deleted file mode 100644 index a190b262e7..0000000000 --- a/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop.mdwn +++ /dev/null @@ -1,56 +0,0 @@ -### Please describe the problem. - -git-annex failed to update the local adjusted branch after `git annex drop` stopped abruptly. In turn, `git annex drop` stopped abruptly because a `git annex copy --from X --to Y` had to be stopped (CTRL-C) because of lack of space. - -(Perhaps there are two issues here: `drop` being unable to cope with the unexpected situation, and the adjustment code being unable to cope with the weird stat left by `drop`'s abrupt interruption.) - -### What steps will reproduce the problem? - -``` -$ git copy -J4 --from X --to Y -[notice that git-annex starts saying "not enough free space, need 27.87 MB more"] -[kill process with CTRL-C] -$ git annex drop dirF/ -drop dirF/a/a1.txt ok -drop dirF/a/a2.txt ok -[...] -error: Your local changes to the following files would be overwritten by checkout: - dirF/a/a1.txt - dirF/a/a2.txt - [...] - -Aborting - - Updating adjusted branch failed. -(recording state in git...) -$ git status -HEAD detached at 9d92415fb -nothing to commit, working tree clean -$ git annex status -$ git branch -* (HEAD detached at 9d92415fb) - adjusted/master(unlocked) - adjusted/master(unlockpresent) - git-annex - master - synced/master -``` - -`dirF/` is the directory whose files were being copied when the process has been stopped with `CTRL-C`. - -### What version of git-annex are you using? On what operating system? - -``` -git-annex version: 10.20230215-gd24914f2a -[...] -operating system: linux x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 8 -``` - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -git-annex is too good. It so rarely causes problems that one does not develop the "git-annex troubleshooting muscle". :) - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_1_08b3eafdabe5f60ec2206584dff5d230._comment b/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_1_08b3eafdabe5f60ec2206584dff5d230._comment deleted file mode 100644 index 1380567474..0000000000 --- a/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_1_08b3eafdabe5f60ec2206584dff5d230._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-03-23T19:26:12Z" - content=""" -I assume this is a --hide-missing adjusted branch? - -Update: Oh, I see from a forum post that it's --unlock-present actually. - -What is the annex.adjustedbranchrefresh git config set to? -"""]] diff --git a/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_2_a947f8afde3d7f63fd33b0b7e5998e43._comment b/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_2_a947f8afde3d7f63fd33b0b7e5998e43._comment deleted file mode 100644 index f5ce8f9aea..0000000000 --- a/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_2_a947f8afde3d7f63fd33b0b7e5998e43._comment +++ /dev/null @@ -1,42 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2023-03-23T19:35:33Z" - content=""" -I was able to reproduce something that looks similar to this without -needing to interrupt any command: - - joey@darkstar:~/tmp/bench>git clone a b - Cloning into 'b'... - done. - joey@darkstar:~/tmp/bench>cd b - joey@darkstar:~/tmp/bench/b>git config annex.adjustedbranchrefresh 1 - joey@darkstar:~/tmp/bench/b>git annex adjust --unlock-present - adjust - Switched to branch 'adjusted/master(unlockpresent)' - ok - joey@darkstar:~/tmp/bench/b#master(unlockpresent)>ls - foo@ - joey@darkstar:~/tmp/bench/b#master(unlockpresent)>git-annex get - get foo (from origin...) - ok - (recording state in git...) - joey@darkstar:~/tmp/bench/b#master(unlockpresent)>ls - foo - joey@darkstar:~/tmp/bench/b#master(unlockpresent)>git-annex drop - drop foo ok - error: Your local changes to the following files would be overwritten by checkout: - foo - Please commit your changes or stash them before you switch branches. - Aborting - - Updating adjusted branch failed. - (recording state in git...) - -And it was left in a similar detached head status: - - HEAD detached at 2aab85d - nothing to commit, working tree clean - -This seems be be a bug with the implementation of annex.adjustedbranchrefresh -"""]] diff --git a/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_3_c8ce6c9fc35fa6ad5165ecf9a3592c9d._comment b/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_3_c8ce6c9fc35fa6ad5165ecf9a3592c9d._comment deleted file mode 100644 index 28215d4c79..0000000000 --- a/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_3_c8ce6c9fc35fa6ad5165ecf9a3592c9d._comment +++ /dev/null @@ -1,23 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2023-03-23T19:50:45Z" - content=""" -git-annex is what writes that detached HEAD, -in updateAdjustedBranch. And I've verified that when it fails like this, -git status shows the file is modified. But of course the modification is -only that the annex content has been replaced by an annex pointer. Running -`git add` on the file makes the modification status go away. - -And this only happens when annex.adjustedbranchrefresh=1. -At higher values, it calls Annex.Queue.flush, but at 1 it does not, and so -restagePointerFiles does not get called before adjustedBranchRefreshFull. -(Or at least may not, they're both running as cleanup actions and order is -not really defined.) - -I wonder if there are other situations where modifications can prevent -checkout of the updated adjusted branch? Eg, what if the user has made some -other modification to an annexed file? It seems wise for git-annex to -defend against it in depth, by making sure no crash can leave it in -detached head state. -"""]] diff --git a/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_4_2a4de8981ced7894148ca9a68e5ef60b._comment b/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_4_2a4de8981ced7894148ca9a68e5ef60b._comment deleted file mode 100644 index 76b2779873..0000000000 --- a/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_4_2a4de8981ced7894148ca9a68e5ef60b._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="gioele@678b7c03f524f2669b179b603f65352fcc16774e" - nickname="gioele" - avatar="http://cdn.libravatar.org/avatar/366dbda84e78aff8a8a070622aeb63ce" - subject="comment 4" - date="2023-03-24T08:13:00Z" - content=""" -Thanks for investigating this! - -These are the local settings: - - thin = true - adjustedbranchrefresh = 1 - -"""]] diff --git a/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_5_7b989f777b211f21ca0e9ca681869bff._comment b/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_5_7b989f777b211f21ca0e9ca681869bff._comment deleted file mode 100644 index 9e7aba1e28..0000000000 --- a/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_5_7b989f777b211f21ca0e9ca681869bff._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="gioele@678b7c03f524f2669b179b603f65352fcc16774e" - nickname="gioele" - avatar="http://cdn.libravatar.org/avatar/366dbda84e78aff8a8a070622aeb63ce" - subject="comment 5" - date="2023-03-25T08:47:01Z" - content=""" -> I wonder if there are other situations where modifications can prevent checkout of the updated adjusted branch? - -I clearly remember seeing something similar when a `sync` failed due a non-recoverable network failure. Unfortunately I have no logs from that day. - -> It seems wise for git-annex to defend against it in depth, by making sure no crash can leave it in detached head state. - -Perhaps also extend `fsck` to automatically recover from such a state? Or at least suggest a couple of possible solutions to the user? -"""]] diff --git a/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_6_f9ef22fcfabc6445060b964f08080c76._comment b/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_6_f9ef22fcfabc6445060b964f08080c76._comment deleted file mode 100644 index b5155705f4..0000000000 --- a/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_6_f9ef22fcfabc6445060b964f08080c76._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 6""" - date="2023-03-28T17:11:53Z" - content=""" -Weird, I don't see how a network failure could lead in that direction. - -Anyway, [[!commit 038a2600f4cf71294976280c5c29f6710359375f]] -avoids it with a detached head. - -git-annex fsck can't do anything about this because the user may have their -own reasons to check out a detached head. -"""]] diff --git a/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_7_f679af07b798cb4bfae75565dd46e8ed._comment b/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_7_f679af07b798cb4bfae75565dd46e8ed._comment deleted file mode 100644 index faff098303..0000000000 --- a/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_7_f679af07b798cb4bfae75565dd46e8ed._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="gioele@678b7c03f524f2669b179b603f65352fcc16774e" - nickname="gioele" - avatar="http://cdn.libravatar.org/avatar/366dbda84e78aff8a8a070622aeb63ce" - subject="comment 7" - date="2023-04-01T11:06:25Z" - content=""" -> I wonder if there are other situations where modifications can prevent checkout of the updated adjusted branch? -> ->> I clearly remember seeing something similar when a sync failed due a non-recoverable network failure. Unfortunately I have no logs from that day. -> ->>> Weird, I don't see how a network failure could lead in that direction. - -All I remember is that the command that was running was `git annex sync --content`. - -Perhaps the fact that `annex.alwayscommit = false` was set made a difference. - - -"""]] diff --git a/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_8_c7c210961b51a87770932f40a9563622._comment b/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_8_c7c210961b51a87770932f40a9563622._comment deleted file mode 100644 index c07046dbbe..0000000000 --- a/doc/bugs/Failed_adjusted_branch_update_after_error_in_drop/comment_8_c7c210961b51a87770932f40a9563622._comment +++ /dev/null @@ -1,22 +0,0 @@ -[[!comment format=mdwn - username="gioele@678b7c03f524f2669b179b603f65352fcc16774e" - nickname="gioele" - avatar="http://cdn.libravatar.org/avatar/366dbda84e78aff8a8a070622aeb63ce" - subject="comment 8" - date="2023-04-01T11:09:49Z" - content=""" -
- -> Anyway, 038a2600f4cf71294976280c5c29f6710359375f avoids it with a detached head. - -With git-annex 10.20230329-g30d7f9ad7 I got the following message after checking out `adjusted/master(unlockpresent)` and running `git annex copy --from foo --to bar -J2`: - -``` -git-annex: .git/HEAD: openFile: resource busy (file is locked)\" -``` - -This message appeared at the very end of the `copy` process. - -Regardless of this, everything seems to work just fine. - -"""]] diff --git a/doc/bugs/Failure_to_get_small_files_over_P2P_protocol.mdwn b/doc/bugs/Failure_to_get_small_files_over_P2P_protocol.mdwn deleted file mode 100644 index cb0a2b663b..0000000000 --- a/doc/bugs/Failure_to_get_small_files_over_P2P_protocol.mdwn +++ /dev/null @@ -1,120 +0,0 @@ -### Please describe the problem. -I'm storing lots of recordings from our satellite receiver into a git annex repo for easy syncing to a backup server that's not always switched on. Now, I'm trying to replace this backup server and pulling all data onto my new node. Most files work fine, but there are a bunch of small files that throw an error. In the debug log the GET command seems to use the file's size as offset, resulting in 0 bytes of data, throwing an error - -### What steps will reproduce the problem? - $ git annex get Documentaires/Oorlogen/20170124_2110_-_CANVAS_HD_-_1945__THE_SAVAGE_PEACE.* --debug - [2021-12-19 17:13:10.308043464] process [4214] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"] - [2021-12-19 17:13:10.309545353] process [4214] done ExitSuccess - [2021-12-19 17:13:10.310048003] process [4215] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"] - [2021-12-19 17:13:10.312009938] process [4215] done ExitSuccess - [2021-12-19 17:13:10.328167499] process [4216] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--stage","-z","--","Documentaires/Oorlogen/20170124_2110_-_CANVAS_HD_-_1945__THE_SAVAGE_PEACE.eit","Documentaires/Oorlogen/20170124_2110_-_CANVAS_HD_-_1945__THE_SAVAGE_PEACE.ts","Documentaires/Oorlogen/20170124_2110_-_CANVAS_HD_-_1945__THE_SAVAGE_PEACE.ts.ap","Documentaires/Oorlogen/20170124_2110_-_CANVAS_HD_-_1945__THE_SAVAGE_PEACE.ts.cuts","Documentaires/Oorlogen/20170124_2110_-_CANVAS_HD_-_1945__THE_SAVAGE_PEACE.ts.meta","Documentaires/Oorlogen/20170124_2110_-_CANVAS_HD_-_1945__THE_SAVAGE_PEACE.ts.sc"] - [2021-12-19 17:13:10.328598306] process [4217] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)","--buffer"] - [2021-12-19 17:13:10.328861515] process [4218] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"] - [2021-12-19 17:13:10.329676569] process [4219] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"] - get Documentaires/Oorlogen/20170124_2110_-_CANVAS_HD_-_1945__THE_SAVAGE_PEACE.ts.meta [2021-12-19 17:13:10.341285273] process [4221] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"] - [2021-12-19 17:13:10.342469002] process [4221] done ExitSuccess - [2021-12-19 17:13:10.342787111] process [4222] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"] - [2021-12-19 17:13:10.343851119] process [4222] done ExitSuccess - [2021-12-19 17:13:10.34416143] process [4223] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/git-annex..33675b567939705c283bd955c2c7de13e15ba28a","--pretty=%H","-n1"] - [2021-12-19 17:13:10.345389634] process [4223] done ExitSuccess - [2021-12-19 17:13:10.345949274] process [4224] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"] - [2021-12-19 17:13:10.346209136] process [4225] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] - (from garage...) - [2021-12-19 17:13:10.349782417] process [4226] read: ssh ["-o","BatchMode=true","-S",".git/annex/ssh/root@10.19.20.147","-o","ControlMaster=auto","-o","ControlPersist=yes","-n","-T","root@10.19.20.147","true"] - [2021-12-19 17:13:11.687712914] process [4226] done ExitSuccess - [2021-12-19 17:13:11.68875019] process [4231] chat: ssh ["root@10.19.20.147","-S",".git/annex/ssh/root@10.19.20.147","-o","ControlMaster=auto","-o","ControlPersist=yes","-T","git-annex-shell 'p2pstdio' '/opt/Sat' '--debug' 'b22315b4-77f0-4f46-aed3-fe376b350ab9' --uuid dd4c0b26-2331-4133-82bf-1888753c3268"] - [2021-12-19 17:13:12.407472022] [ssh connection Just 4231] [ThreadId 4] P2P < AUTH-SUCCESS dd4c0b26-2331-4133-82bf-1888753c3268 - [2021-12-19 17:13:12.40779008] [ssh connection Just 4231] [ThreadId 4] P2P > VERSION 1 - [2021-12-19 17:13:12.423477213] [ssh connection Just 4231] [ThreadId 4] P2P < VERSION 1 - [2021-12-19 17:13:12.423831707] [ssh connection Just 4231] [ThreadId 4] P2P > GET 165 Documentaires/Oorlogen/20170124_2110_-_CANVAS_HD_-_1945__THE_SAVAGE_PEACE.ts.meta SHA256E-s164--41ece560a700c3a1f2c1a28f37399c6cf139dde68f8c97ff9f7e10ac1080c746.ts.meta - [2021-12-19 17:13:28.731615325] [ThreadId 4] P2P < GET 165 Documentaires/Oorlogen/20170124_2110_-_CANVAS_HD_-_1945__THE_SAVAGE_PEACE.ts.meta SHA256E-s164--41ece560a700c3a1f2c1a28f37399c6cf139dde68f8c97ff9f7e10ac1080c746.ts.meta - [2021-12-19 17:13:28.796144027] [ThreadId 4] P2P > DATA 0 - [2021-12-19 17:13:28.796944343] [ThreadId 4] P2P > VALID - [2021-12-19 17:13:12.518895068] [ssh connection Just 4231] [ThreadId 4] P2P < DATA 0 - [2021-12-19 17:13:12.521393464] [ssh connection Just 4231] [ThreadId 4] P2P < VALID - [2021-12-19 17:13:12.522008877] [ssh connection Just 4231] [ThreadId 4] P2P > FAILURE - - transfer failed - [2021-12-19 17:13:12.524417003] process [4224] done ExitSuccess - [2021-12-19 17:13:12.525545594] process [4225] done ExitSuccess - - [2021-12-19 17:13:12.526125714] [ssh connection Just 4231] [ThreadId 4] P2P > GET 165 Documentaires/Oorlogen/20170124_2110_-_CANVAS_HD_-_1945__THE_SAVAGE_PEACE.ts.meta SHA256E-s164--41ece560a700c3a1f2c1a28f37399c6cf139dde68f8c97ff9f7e10ac1080c746.ts.meta - [2021-12-19 17:13:28.807577749] [ThreadId 4] P2P < FAILURE - [2021-12-19 17:13:28.812468221] [ThreadId 4] P2P < GET 165 Documentaires/Oorlogen/20170124_2110_-_CANVAS_HD_-_1945__THE_SAVAGE_PEACE.ts.meta SHA256E-s164--41ece560a700c3a1f2c1a28f37399c6cf139dde68f8c97ff9f7e10ac1080c746.ts.meta - [2021-12-19 17:13:28.812746153] [ThreadId 4] P2P > DATA 0 - [2021-12-19 17:13:28.812854143] [ThreadId 4] P2P > VALID - [2021-12-19 17:13:12.533471116] [ssh connection Just 4231] [ThreadId 4] P2P < DATA 0 - [2021-12-19 17:13:12.534230317] [ssh connection Just 4231] [ThreadId 4] P2P < VALID - [2021-12-19 17:13:12.534505767] [ssh connection Just 4231] [ThreadId 4] P2P > FAILURE - - transfer failed - - Unable to access these remotes: garage - [2021-12-19 17:13:12.537799752] process [4235] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"] - [2021-12-19 17:13:12.538818173] process [4236] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] - [2021-12-19 17:13:28.820206504] [ThreadId 4] P2P < FAILURE - - No other repository is known to contain the file. - failed - [2021-12-19 17:13:12.55009435] process [4235] done ExitSuccess - [2021-12-19 17:13:12.550952683] process [4236] done ExitSuccess - [2021-12-19 17:13:12.551390026] process [4219] done ExitSuccess - [2021-12-19 17:13:12.551709257] process [4218] done ExitSuccess - [2021-12-19 17:13:12.551975888] process [4217] done ExitSuccess - [2021-12-19 17:13:12.552194866] process [4216] done ExitSuccess - [2021-12-19 17:13:12.561312091] process [4238] read: ssh ["-O","stop","-S","root@10.19.20.147","-o","ControlMaster=auto","-o","ControlPersist=yes","localhost"] - [2021-12-19 17:13:12.569198309] process [4238] done ExitSuccess - git-annex: get: 1 failed - -As you can see the offset in the p2p get command is 165, which is exactly the file size of what it's supposed to be getting: - - -r-xr-xr-x 1 root root 165 Apr 29 2020 .git/annex/objects/5V/x8/SHA256E-s164--41ece560a700c3a1f2c1a28f37399c6cf139dde68f8c97ff9f7e10ac1080c746.ts.meta/SHA256E-s164--41ece560a700c3a1f2c1a28f37399c6cf139dde68f8c97ff9f7e10ac1080c746.ts.meta - - - -### What version of git-annex are you using? On what operating system? -On the old server: - - $ git annex version - git-annex version: 8.20200908 - build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV - dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0 - key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* - remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso hook external - operating system: linux x86_64 - supported repository versions: 8 - upgrade supported from repository versions: 0 1 2 3 4 5 6 7 - local repository version: 8 - -On the new server: - - $ git annex version - git-annex version: 8.20210223 - build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV - dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0 - key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* - remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external - operating system: linux x86_64 - supported repository versions: 8 - upgrade supported from repository versions: 0 1 2 3 4 5 6 7 - local repository version: 8 - -Both servers are debian. - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) -I've been using git annex for years now, some periods more intensively than others. The only data I've lost I can only blame on my own stupidity, even after a terminal disk failure I managed to recover almost everything with a reinject. I'm still using it entirely cli though, no assistants or gui for me. - -[[!meta title="corrupted tmp file can prevent getting an object"]] - -> [[fixed|done]] (the related issue that I discovered in comment #1 -> specifically) --[[Joey]] diff --git a/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_1_72a8b9b6ed582dad36301cb9beece2f8._comment b/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_1_72a8b9b6ed582dad36301cb9beece2f8._comment deleted file mode 100644 index f77ba20200..0000000000 --- a/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_1_72a8b9b6ed582dad36301cb9beece2f8._comment +++ /dev/null @@ -1,34 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2021-12-26T16:17:43Z" - content=""" -It looks rather like the full content of the file may already have been -downloaded, in `.git/annex/tmp/` which is why it "resumes" -from the end of the file. - -But this does not necessarily look like a protocol problem. The protocol -handles no more data being sent. The failure is on the receiving side, -after it's received the data it indicates an overall failure. - -One reason this could be happening is if the file in `.git/annex/tmp/` is -corrupted, so verifying the completed download fails. - -I tried two experiements. First, I populated `.git/annex/tmp/` with a copy of -the annexed object file from the other repository. Then `git-annex get` -succeeded, and looked this this: - - [2021-12-26 11:25:32.364107322] (P2P.IO) [ThreadId 4] P2P < GET 30 foo SHA256E-s30--8559e95c5b77e5e577466959d83097039d0d3f2b6a7e18b2fd5df0188ef648c6 - [2021-12-26 11:25:32.365992548] (P2P.IO) [ThreadId 4] P2P > DATA 0 - [2021-12-26 11:25:32.366778465] (P2P.IO) [ThreadId 4] P2P > VALID - [2021-12-26 11:25:32.394964731] (P2P.IO) [ThreadId 4] P2P < SUCCESS - -Next, I did the same, but corrupted the tmp file. With that I reproduced -your problem exactly. - -So, I suggest you try deleting files in `.git/annex/tmp/` and get them again. - -I think that git-annex should probably try to recover from this by, -when verification of a downloaded file fails, deleting the tmp file. -But that will need some more thought. -"""]] diff --git a/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_2_53970994816f1de9816e3496fdf0ef71._comment b/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_2_53970994816f1de9816e3496fdf0ef71._comment deleted file mode 100644 index 2fdb5f6c4c..0000000000 --- a/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_2_53970994816f1de9816e3496fdf0ef71._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="tim@5431dd39464df207b7d46d3cf1bc74c82123ac68" - nickname="tim" - avatar="http://cdn.libravatar.org/avatar/967c2815f001d24b5f9f33300c03ba6f" - subject="Found the issue" - date="2022-01-03T08:25:05Z" - content=""" -Hi Joey, - -Thank you for having a look at this, but we can put this one under \"my own stupidity\". The files that give these errors are apparently only status files which get rewritten every time you play a recording, so they continuously get rewritten, get \"git annex add\"ed with a cron job, but don't always get pushed before they get overwritten, so yeah, it's not unexpected that some of these may get lost at some point. -Still no real data lost though, but errors like these worry me enough to file a bug report. - -regards, -Tim -"""]] diff --git a/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_3_4c359c12d5c61351484dfcb2f5547222._comment b/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_3_4c359c12d5c61351484dfcb2f5547222._comment deleted file mode 100644 index 3ac6afe830..0000000000 --- a/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_3_4c359c12d5c61351484dfcb2f5547222._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2022-01-03T17:25:03Z" - content=""" -Err, unless you have `annex.thin` set, once you `git-annex add` a file, -it should not be possible for it be overwritten in a way that loses the -annexed copy of it. -"""]] diff --git a/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_4_1e286c1506fc542df85c6158a8ec3d1c._comment b/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_4_1e286c1506fc542df85c6158a8ec3d1c._comment deleted file mode 100644 index 3d948e5205..0000000000 --- a/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_4_1e286c1506fc542df85c6158a8ec3d1c._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="tim@5431dd39464df207b7d46d3cf1bc74c82123ac68" - nickname="tim" - avatar="http://cdn.libravatar.org/avatar/967c2815f001d24b5f9f33300c03ba6f" - subject="well, yes" - date="2022-01-04T11:28:33Z" - content=""" -Well, yes, it does, if you start discarding disks after you've done a git annex pull on a temp usb repo and then again to the new server... Hence, my own stupidity. -"""]] diff --git a/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_5_c667e4d4899925a5484c94256044f6ee._comment b/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_5_c667e4d4899925a5484c94256044f6ee._comment deleted file mode 100644 index 260d2f191a..0000000000 --- a/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_5_c667e4d4899925a5484c94256044f6ee._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2022-01-04T15:22:47Z" - content=""" -Ah, I see. :) - -I am going to leave this bug open, since the corrupted tmp file problem I -found is a real problem, though not actually your problem. -"""]] diff --git a/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_6_5ed6591954aeafe6c99ae152f4f4ad67._comment b/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_6_5ed6591954aeafe6c99ae152f4f4ad67._comment deleted file mode 100644 index 08dc1ba5e3..0000000000 --- a/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_6_5ed6591954aeafe6c99ae152f4f4ad67._comment +++ /dev/null @@ -1,24 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 6""" - date="2022-01-07T16:12:20Z" - content=""" -Current thinking on deleting corrupted tmp files: If a download succeeds, -and verification then fails, the whole file content has been downloaded, -and is corrupt. So it would be ok to always delete it then, as far as p2p -transfers goes. - -For other remotes, the same is often true. The only exceptions are like -rsync and bittorrent, which can recover from corruption on retry. But, -I don't think either rsync or bittorrent will usually write corrupt data -to a file anyway. They would catch over-the-wire corruption with rolling -checksums etc. So, it seems like a verification should never fail after -a successful rsync or bittorrent download. Unless the disk corrupted the -data in the meantime. Which is an unlikely situation, and not one that it's -really necessary for git-annex to recover from with optimal efficiency. - -... Oh interesting.. It already is supposed to do that, in -getViaTmpFromDisk. It seems, what is happening is the transfer fails -when all the file content is present, and so it never gets to the point of -verifying it, let alone deleting it. -"""]] diff --git a/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_7_247db3d1c03b207296e97975a3554756._comment b/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_7_247db3d1c03b207296e97975a3554756._comment deleted file mode 100644 index 438e5cd155..0000000000 --- a/doc/bugs/Failure_to_get_small_files_over_P2P_protocol/comment_7_247db3d1c03b207296e97975a3554756._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 7""" - date="2022-01-07T16:34:42Z" - content=""" -This seems to be a reversion caused by incremental verification. In -P2P.Annex it does incremental verification, and so it notices that the temp -file is corrupted. So, the transfer fails, and the temp file is left -behind. - -So I suppose the fix is that, whenever doing incremental verification, -also delete the temp file when it fails. -"""]] diff --git a/doc/bugs/Files_recorded_with_other_file__39__s_checksums.mdwn b/doc/bugs/Files_recorded_with_other_file__39__s_checksums.mdwn deleted file mode 100644 index 55dd7aa5e0..0000000000 --- a/doc/bugs/Files_recorded_with_other_file__39__s_checksums.mdwn +++ /dev/null @@ -1,102 +0,0 @@ -### Please describe the problem. - -I have a special remote "source" set up as type=directory importtree=yes. - -I pulled it into one repo in which none of the files were wanted. So far so good. I cloned that repo to a second, archive, repo. git annex sync worked (but took 2 hours). Then I did `git annex get --auto`, most of the files came through OK. But some said things like this: - -``` -get Pictures/.dtrash/info/IMG_2979_v1-e9dced7b.dtrashinfo (from source...) (checksum...) - verification of content failed - - Unable to access these remotes: source - - No other repository is known to contain the file. -failed -``` - -(Both repos are unlocked via adjust --unlocked) - -Upon looking at the file in the repo where it wasn't wanted, I saw this: - -``` -$ cat IMG_2979_v1-e9dced7b.dtrashinfo -/annex/objects/SHA256E-s144--cec5c7b6a9d97344e374e8395e02b74350678147ff65d6df091f5115cf18bf72 -``` - -Interesting. So, in the source directory: - -``` -$ sha256sum IMG_2979_v1-e9dced7b.dtrashinfo -aca3ed7243def7a0bd5fcad542c66841b8e7d2a670b4cafe749eb27e032d8975 IMG_2979_v1-e9dced7b.dtrashinfo -``` - -That's not a match at all. Well, OK then: - -``` -$ sha256sum * | grep cec5c7 -cec5c7b6a9d97344e374e8395e02b74350678147ff65d6df091f5115cf18bf72 IMG_2981_v1-5fc99c7a.dtrashinfo -``` - -Yikes. So for IMG_2979_v1-e9dced7b.dtrashinfo, git-annex recorded a checksum that belonged to IMG_2981_v1-5fc99c7a.dtrashinfo. Well then, what is this other file recorded as, back in the git-annex repo? - -``` -$ cat IMG_2981_v1-5fc99c7a.dtrashinfo -/annex/objects/SHA256E-s144--cec5c7b6a9d97344e374e8395e02b74350678147ff65d6df091f5115cf18bf72 -``` - -OK, so two files that were not identical in the source directory got recorded with an identical checksum in git-annex somehow. And, when they were attempted to be imported via `git annex get --auto`, this at least was detected there. - -In this .dtrash/info directory, 436 files out of 719 were not loaded by `git annex get`, presumably due to this issue. - -In this directory, the source files were ranging in size from 140 to 227 bytes. - -In a companion directory, .dtrash/files, 24 out of 719 files exhibited this issue. These files tended to be larger, but one that was 495MB triggered it also. - -I have not yet seen it outside .dtrash, but it will be many hours until this get process completes fully, as it needs to copy about 1TB of data. - -In case you are wondering if there is a race condition with .dtrash: no. The only application that writes to it isn't running, and the last time a file was modified in there was over a year ago. Also the content of the .info files is just JSON and a corresponding filename embedded in them, so it is very clear that the files on the filesystem are correct and the calculated checksums at issue here were never correct. - -### What steps will reproduce the problem? - -I have laid that out as best I can above. - -### What version of git-annex are you using? On what operating system? - -8;.20210223 on Debian - -### Please provide any additional information below. - -Assistant is not being used. - -Setup: - -``` -REPO=Pictures -cd /acrypt/git-annex/repos -mkdir $REPO -cd $REPO -git init -git config annex.thin true -git annex init 'local hub' -git annex wanted . "include=* and exclude=$REPO/*" - -# Now initialize things. -touch mtree -git annex add mtree -git annex sync -git annex adjust --unlock - -git annex initremote source type=directory directory=/acrypt/git-annex/bind-ro/$REPO importtree=yes encryption=none - -git annex enableremote source directory=/acrypt/git-annex/bind-ro/$REPO -git config remote.source.annex-readonly true -git config remote.source.annex-tracking-branch main:$REPO -git config annex.securehashesonly true -git config annex.genmetadata true -git config annex.diskreserve 100M -``` - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -> [[done]] since this has pretty strongly been confirmed to be a bug that -> was already fixed in a newer git-annex version. --[[Joey]] diff --git a/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_1_05e3e33e1ca2361546dbe08c6bd476d6._comment b/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_1_05e3e33e1ca2361546dbe08c6bd476d6._comment deleted file mode 100644 index 68ab41bb32..0000000000 --- a/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_1_05e3e33e1ca2361546dbe08c6bd476d6._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="jgoerzen" - avatar="http://cdn.libravatar.org/avatar/090740822c9dcdb39ffe506b890981b4" - subject="comment 1" - date="2022-09-05T01:14:53Z" - content=""" -Update: This also occurred in other directories, with some video files from 2018. One directory contains 1945 files, and another 21 files. I'm not finding an obvious pattern to the issue. -"""]] diff --git a/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_2_0735ed4187e23ddea6bd0aa408451942._comment b/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_2_0735ed4187e23ddea6bd0aa408451942._comment deleted file mode 100644 index b5693f342d..0000000000 --- a/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_2_0735ed4187e23ddea6bd0aa408451942._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Lukey" - avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b" - subject="comment 2" - date="2022-09-05T09:35:18Z" - content=""" -You really should upgrade to the latest version. -"""]] diff --git a/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_3_06b7be9e651e528440a054ef43f7142f._comment b/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_3_06b7be9e651e528440a054ef43f7142f._comment deleted file mode 100644 index 4c6f3bd6bd..0000000000 --- a/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_3_06b7be9e651e528440a054ef43f7142f._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2022-09-05T17:48:49Z" - content=""" -Concur that you need to upgrade. In particular, this seems likely to have -been fixed by [[!commit 3e2f1f73cbc5fc10475745b3c3133267bd1850a7]]. - -If so, the two files in the source directory will turn out to have the -same size and mtime, so were considered to have the same content identifier -when importing from a directory with your version of git-annex. -"""]] diff --git a/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_4_bc9cf721f8087cb91954cc2435376bc2._comment b/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_4_bc9cf721f8087cb91954cc2435376bc2._comment deleted file mode 100644 index c58aceed35..0000000000 --- a/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_4_bc9cf721f8087cb91954cc2435376bc2._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="jgoerzen" - avatar="http://cdn.libravatar.org/avatar/090740822c9dcdb39ffe506b890981b4" - subject="comment 4" - date="2022-09-05T18:29:31Z" - content=""" -Thanks! I noticed some changes in v10 around checksumming, so I got it started with v10 last night. It's still running, but I'll update you when it's done. Appreciate it! -"""]] diff --git a/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_5_8f0bbd819b95dfa97e2f4848e08a787a._comment b/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_5_8f0bbd819b95dfa97e2f4848e08a787a._comment deleted file mode 100644 index a9cf6f5cf1..0000000000 --- a/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_5_8f0bbd819b95dfa97e2f4848e08a787a._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="jgoerzen" - avatar="http://cdn.libravatar.org/avatar/090740822c9dcdb39ffe506b890981b4" - subject="comment 5" - date="2022-09-05T18:44:01Z" - content=""" -Following up a bit - I had blown away the repos that were in use for this, but based on what I logged here, I can confirm that at least the files I mentioned with conflicts had the same size and mtime. - -I may yet want to use ignoreinodes=yes. Eventually I want to be feeding git-annex from zfs clones to get a guaranteed stable source at any given moment, and I'm not really sure what that will do to inodes (it should, at minimum, have a different device number, I would assume). -"""]] diff --git a/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_6_17a0864cec6f28a0445998f3c5fc7f10._comment b/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_6_17a0864cec6f28a0445998f3c5fc7f10._comment deleted file mode 100644 index fde8ede9d7..0000000000 --- a/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_6_17a0864cec6f28a0445998f3c5fc7f10._comment +++ /dev/null @@ -1,35 +0,0 @@ -[[!comment format=mdwn - username="jgoerzen" - avatar="http://cdn.libravatar.org/avatar/090740822c9dcdb39ffe506b890981b4" - subject="comment 6" - date="2022-09-05T22:01:24Z" - content=""" -With the latest v10 git, I'm seeing some things like this: - -``` -get Pictures/0_New/5/20-03-31 21-12-12 0184.mp4 (from source...) file content has changed - - - Unable to access these remotes: source - - No other repository is known to contain the file. -failed -``` - -Now, the file content really had not chnaged. That file looks like this: - -``` -$ cat 20-03-31\ 21-12-12\ 0184.mp4 -/annex/objects/SHA256E-s55936559--0d418e4bb62cfc7ef5b053f8b622dd72794781a49931abc41bb9499acaf51b09.mp4 -``` - -And on-disk, the file is: - -``` -$ sha256sum 20-03-31\ 21-12-12\ 0184.mp4 -0d418e4bb62cfc7ef5b053f8b622dd72794781a49931abc41bb9499acaf51b09 20-03-31 21-12-12 0184.mp4 -``` - -So, this is something different, but it's producing the same result (files left around not checked out). I obvserve that in .dtrash/info, the directory that had such a problem before, zero files now do. But in .dtrash/files, the problem seems to be triggered by a file that has a duplicate somewhere in the source repository (with the same sha256sum). I was able to find another file with the same sha256sum as that one in the same directory. In the origin repo, in which the given files were not present and it's using unlock-present, all instances of the file contained a broken symlink to the appropriate target. In the repo in question here, which uses plain unlock and is on NTFS (though Linux is the OS accessing it), I'm having this problem. - -"""]] diff --git a/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_7_d480e85683f0418441f26c7d61c4af02._comment b/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_7_d480e85683f0418441f26c7d61c4af02._comment deleted file mode 100644 index 0d3ab40e70..0000000000 --- a/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_7_d480e85683f0418441f26c7d61c4af02._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 7""" - date="2022-09-09T18:50:43Z" - content=""" -Well, it seems that you have confirmed that the bug you were seeing is the -known and already fixed problem involving files that had the same size and -mtime. - -I suggest filing a new bug report for the other problem, since it does not -seem like the same problem. I don't like to have to re-read long threads -about unrelated problems to tease out the "current" problem than a bug -report is talking about. 1 bug report per problem saves everyone time and -confusion. -"""]] diff --git a/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_8_80763c1c40acdf0915134182d15e4fd8._comment b/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_8_80763c1c40acdf0915134182d15e4fd8._comment deleted file mode 100644 index 2011fd6c1c..0000000000 --- a/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_8_80763c1c40acdf0915134182d15e4fd8._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="jgoerzen" - avatar="http://cdn.libravatar.org/avatar/090740822c9dcdb39ffe506b890981b4" - subject="comment 8" - date="2022-09-10T00:37:45Z" - content=""" -Sure. Created the new bug at [[/bugs/Files_recorded_with_other_file__39__s_checksums/]]. Thanks Joey! I don't know how to close this one or I would. -"""]] diff --git a/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_9_7c3180616e40b49fc5ddf398d23d0201._comment b/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_9_7c3180616e40b49fc5ddf398d23d0201._comment deleted file mode 100644 index 3cb53063c7..0000000000 --- a/doc/bugs/Files_recorded_with_other_file__39__s_checksums/comment_9_7c3180616e40b49fc5ddf398d23d0201._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="jgoerzen" - avatar="http://cdn.libravatar.org/avatar/090740822c9dcdb39ffe506b890981b4" - subject="comment 9" - date="2022-09-10T00:38:17Z" - content=""" -OK, so that link didn't work. I'll try -"""]] diff --git a/doc/bugs/GIN_is_not_recognized_as_a_git-annex_enabled_repos.mdwn b/doc/bugs/GIN_is_not_recognized_as_a_git-annex_enabled_repos.mdwn deleted file mode 100644 index c882dbe2d1..0000000000 --- a/doc/bugs/GIN_is_not_recognized_as_a_git-annex_enabled_repos.mdwn +++ /dev/null @@ -1,16 +0,0 @@ -Hi, git-annex Team. - -I want to ask you this issue. -GIN is not recognized as a git-annex enabled repository when using the HTTPS protocol. -I cloned a GIN repository with HTTPS protocol and then did `git annex init` on the cloned local repository, but it is not recognized as a git annex-enabled remote. -I checked the .git/config file and it says annex-ignore=true. - -![image](https://user-images.githubusercontent.com/91708725/182578704-c04fdb60-5b6a-4faf-8869-04914723c27f.png)
-Because of this, git annex copy --to [GIN remote repository] is not possible with HTTPS protocol. -Is git-annex capable of copying content over the HTTPS protocol? -If there are any mistakes in my understanding please let me know. - -sincerely yours, -Rino Mizuguchi - -> [[notabug|done]] --[[Joey]] diff --git a/doc/bugs/GIN_is_not_recognized_as_a_git-annex_enabled_repos/comment_1_8a27545e11b3d9e4e45aea4c9a5c9af4._comment b/doc/bugs/GIN_is_not_recognized_as_a_git-annex_enabled_repos/comment_1_8a27545e11b3d9e4e45aea4c9a5c9af4._comment deleted file mode 100644 index 6112e946e2..0000000000 --- a/doc/bugs/GIN_is_not_recognized_as_a_git-annex_enabled_repos/comment_1_8a27545e11b3d9e4e45aea4c9a5c9af4._comment +++ /dev/null @@ -1,42 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-08-04T16:22:09Z" - content=""" -If you're using GIN, perhaps you're also using Datalad. Have you read -[their documentation](http://handbook.datalad.org/en/0.13_a/basics/101-139-gin.html) -on using it with git-annex? - -Part of that documentation that seems likely to be especially -relevant is [here](http://handbook.datalad.org/en/0.13_a/basics/101-139-gin.html#access): - -> Important: Take the URL in the browser, not the copy-paste URL -> Please note that you need to use the browser URL of the repository, -> not the copy-paste URL on the upper right hand side of the repository if you want -> to get anonymous HTTPS access! The two URLs differ only by a .git extension: -> -> Brower bar: `https://gin.g-node.org//` -> -> Copy-paste “HTTPS clone”: `https://gin.g-node.org//.git` -> -> A dataset cloned from `https://gin.g-node.org//.git`, however, can not retrieve annexed files! - -The reason is that git-annex needs to construct urls to download -files from the repo, starting with git's `config` file. -Looking at the sample repo , -an url based on that works , -but its web page suggests cloning the url -, and - does not work. - -How does git manage to clone the latter url then? Well, -the http server is speaking the git smart http protocol. -So git makes requests like `https://gin.g-node.org/adswa/DataLad-101.git/info/refs?service=git-upload-pack` -which do work. But that protocol is not useful for git-annex, so it -needs the url where the actual files in the repository can be downloaded, -not the url where the smart http protocol is spoken. - -There is not generally a way to discover one url from the other url -(removing ".git" might work in this case, but not in other cases). -So unfortunately you will need to remember to use the right url. -"""]] diff --git a/doc/bugs/GIN_is_not_recognized_as_a_git-annex_enabled_repos/comment_2_3e2f01eaaf717710a08ba092856af193._comment b/doc/bugs/GIN_is_not_recognized_as_a_git-annex_enabled_repos/comment_2_3e2f01eaaf717710a08ba092856af193._comment deleted file mode 100644 index 0cc75496da..0000000000 --- a/doc/bugs/GIN_is_not_recognized_as_a_git-annex_enabled_repos/comment_2_3e2f01eaaf717710a08ba092856af193._comment +++ /dev/null @@ -1,75 +0,0 @@ -[[!comment format=mdwn - username="rinomizu5@5ead4c82685c65d7717dbd5591b80425036ae9e3" - nickname="rinomizu5" - avatar="http://cdn.libravatar.org/avatar/62478823018c68821064febcda7e5d4f" - subject="Additional Questions: git annex copy failed" - date="2022-08-08T10:31:04Z" - content=""" -Thanks for the advice, joey. - -I understand that the URL in the browser bar needs to be set to remote. -Then, after modifying the remote URL, I am trying to sync the contents of the git annex file to GIN using the https protocol. -But git annex copy command failed. -
(base) mizuguchi@ubuntu:~/https$ git annex copy --to=origin test2.txt
-copy test2.txt (not found) failed
-copy: 1 failed
-
-I checked the origin remote has annex-uuid, and it's the same as uuid of [annex] in https://gin.g-node.org/ivis-mizuguchi/https/config. -I would like to know why git annex cannot find test2.txt. - -### The version of the tool used in my environment is below. -- git version 2.24.0 -- git-annex version: 8.20211123+git12-g02e3756bd-1~ndall+1 - -### The repository in GIN -[https://gin.g-node.org/ivis-mizuguchi/https](https://gin.g-node.org/ivis-mizuguchi/https) - -### I have done the following process and failed. -1. Create a repository named [https](https://gin.g-node.org/ivis-mizuguchi/https) in GIN with initialization. And set the repository to public. -2. git clone https://gin.g-node.org/ivis-mizuguchi/https -3. cd https -4. git annex init -5. echo \"test\" > test.txt -6. git annex add test.txt -7. git commit -m \"test\" -8. git annex sync origin -9. echo \"test\" > test2.txt -10. git annex add 'test2.txt' -11. git commit -m 'test2' -12. git push origin master -13. git annex copy --to=origin test2.txt - 
(base) mizuguchi@ubuntu:~/https$ git annex copy --to=origin test2.txt
-copy test2.txt (not found) failed
-copy: 1 failed
-
-14. cat .git/config - 
(base) mizuguchi@ubuntu:~/https$ cat .git/config
-[core]
-	repositoryformatversion = 0
-	filemode = true
-	bare = false
-	logallrefupdates = true
-[remote "origin"]
-	url = https://gin.g-node.org/ivis-mizuguchi/https
-	fetch = +refs/heads/*:refs/remotes/origin/*
-	annex-uuid = c7a50190-9146-44de-8f1d-288551fa32e3
-[branch "master"]
-	remote = origin
-	merge = refs/heads/master
-[annex]
-	uuid = 80f0adc3-3144-4bba-a288-09a9ab009cef
-	version = 8
-[filter "annex"]
-	smudge = git-annex smudge -- %f
-	clean = git-annex smudge --clean -- %f
-
- -Any advice would be appreciated. -Sincerely, -Rino Mizuguchi - - - - - -"""]] diff --git a/doc/bugs/GIN_is_not_recognized_as_a_git-annex_enabled_repos/comment_3_07460def4969896c87e8696955740fa9._comment b/doc/bugs/GIN_is_not_recognized_as_a_git-annex_enabled_repos/comment_3_07460def4969896c87e8696955740fa9._comment deleted file mode 100644 index 1bb92766de..0000000000 --- a/doc/bugs/GIN_is_not_recognized_as_a_git-annex_enabled_repos/comment_3_07460def4969896c87e8696955740fa9._comment +++ /dev/null @@ -1,75 +0,0 @@ -[[!comment format=mdwn - username="rinomizu5@5ead4c82685c65d7717dbd5591b80425036ae9e3" - nickname="rinomizu5" - avatar="http://cdn.libravatar.org/avatar/62478823018c68821064febcda7e5d4f" - subject="Additional Questions: git annex copy failed" - date="2022-08-08T10:31:26Z" - content=""" -Thanks for the advice, joey. - -I understand that the URL in the browser bar needs to be set to remote. -Then, after modifying the remote URL, I am trying to sync the contents of the git annex file to GIN using the https protocol. -But git annex copy command failed. -
(base) mizuguchi@ubuntu:~/https$ git annex copy --to=origin test2.txt
-copy test2.txt (not found) failed
-copy: 1 failed
-
-I checked the origin remote has annex-uuid, and it's the same as uuid of [annex] in https://gin.g-node.org/ivis-mizuguchi/https/config. -I would like to know why git annex cannot find test2.txt. - -### The version of the tool used in my environment is below. -- git version 2.24.0 -- git-annex version: 8.20211123+git12-g02e3756bd-1~ndall+1 - -### The repository in GIN -[https://gin.g-node.org/ivis-mizuguchi/https](https://gin.g-node.org/ivis-mizuguchi/https) - -### I have done the following process and failed. -1. Create a repository named [https](https://gin.g-node.org/ivis-mizuguchi/https) in GIN with initialization. And set the repository to public. -2. git clone https://gin.g-node.org/ivis-mizuguchi/https -3. cd https -4. git annex init -5. echo \"test\" > test.txt -6. git annex add test.txt -7. git commit -m \"test\" -8. git annex sync origin -9. echo \"test\" > test2.txt -10. git annex add 'test2.txt' -11. git commit -m 'test2' -12. git push origin master -13. git annex copy --to=origin test2.txt - 
(base) mizuguchi@ubuntu:~/https$ git annex copy --to=origin test2.txt
-copy test2.txt (not found) failed
-copy: 1 failed
-
-14. cat .git/config - 
(base) mizuguchi@ubuntu:~/https$ cat .git/config
-[core]
-	repositoryformatversion = 0
-	filemode = true
-	bare = false
-	logallrefupdates = true
-[remote "origin"]
-	url = https://gin.g-node.org/ivis-mizuguchi/https
-	fetch = +refs/heads/*:refs/remotes/origin/*
-	annex-uuid = c7a50190-9146-44de-8f1d-288551fa32e3
-[branch "master"]
-	remote = origin
-	merge = refs/heads/master
-[annex]
-	uuid = 80f0adc3-3144-4bba-a288-09a9ab009cef
-	version = 8
-[filter "annex"]
-	smudge = git-annex smudge -- %f
-	clean = git-annex smudge --clean -- %f
-
- -Any advice would be appreciated. -Sincerely, -Rino Mizuguchi - - - - - -"""]] diff --git a/doc/bugs/GIN_is_not_recognized_as_a_git-annex_enabled_repos/comment_4_f97de485f97d8016a5ab4e276799d230._comment b/doc/bugs/GIN_is_not_recognized_as_a_git-annex_enabled_repos/comment_4_f97de485f97d8016a5ab4e276799d230._comment deleted file mode 100644 index d2d44cd9cd..0000000000 --- a/doc/bugs/GIN_is_not_recognized_as_a_git-annex_enabled_repos/comment_4_f97de485f97d8016a5ab4e276799d230._comment +++ /dev/null @@ -1,36 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2022-08-08T15:27:21Z" - content=""" -Well, that is a different problem than the problem you reported, isn't it? -You seem to have shown that git-annex does in fact recognize your GIN -remote as a git-annex repo. - -git-annex does not support uploading files to a git repo on GIN using http. - -The datalad documentation says to add a remote using the ssh url of the GIN -repository, which git-annex can store files on. See - - -Without using datalad, the equivilant commands would be something like: - - git remote add gin git@gin.g-node.org:/ivis-mizuguchi/https.git - git-annex copy --to gin - ---- - -So this leaves the question of why git-annex tries to do a http request -when you ask it to copy the content to the http repo. That http request -failing is what results in the "(not found)" message. - -Looks like that is due to `git-annex copy` checking to see if the content -exists on the remote. It does not. So it ought to try to send it, -which would fail with a better "copying to non-ssh repo not supported" error message. -However, intstead it displays the error. - -I've fixed this bug, and now it fails with this message: - - copy test3.txt (to origin...) - copying to non-ssh repo not supported -"""]] diff --git a/doc/bugs/GIN_is_not_recognized_as_a_git-annex_enabled_repos/comment_5_cf4130cb932d3d884e605d7c6447e61e._comment b/doc/bugs/GIN_is_not_recognized_as_a_git-annex_enabled_repos/comment_5_cf4130cb932d3d884e605d7c6447e61e._comment deleted file mode 100644 index dba4af574b..0000000000 --- a/doc/bugs/GIN_is_not_recognized_as_a_git-annex_enabled_repos/comment_5_cf4130cb932d3d884e605d7c6447e61e._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="rinomizu5@5ead4c82685c65d7717dbd5591b80425036ae9e3" - nickname="rinomizu5" - avatar="http://cdn.libravatar.org/avatar/62478823018c68821064febcda7e5d4f" - subject="comment 5" - date="2022-08-09T00:33:55Z" - content=""" -Thank you joey for the additional help. - -I see that I need to use the SSH protocol to upload the file contents to GIN with git annex. -And I also understand why it showed (not found). - -I appreciate your advice. -RIno Mizuguchi -"""]] diff --git a/doc/bugs/Git_repos_corrupt_themselves.mdwn b/doc/bugs/Git_repos_corrupt_themselves.mdwn deleted file mode 100644 index 9bef074686..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves.mdwn +++ /dev/null @@ -1,109 +0,0 @@ -### Please describe the problem. - -This has happened a few times now which is why I'm reporting it. Git-annex seems to do something that causes my git repo to randomly corrupt on their own which is very worrying because I need to trust these repos to keep their content safe. -It has never happened with regular repos and happens on both my Linux machine and my MBP. It hasn't happened on my NAS (SOTERIA) yet though which is exclusively accessed remotely, so I suspect this is an issue with the assistant/daemon. - -### What steps will reproduce the problem? - -Just have the assistant running, committing and syncing. No idea what other factors migh play into this. - -### What version of git-annex are you using? On what operating system? - -``` -git-annex version: 8.20210428 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.28 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.4 http-client-0.6.4.1 persistent-sqlite-2.11.1.0 torrent-10000.1.1 uuid-1.3.14 yesod-1.6.1.1 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 -local repository version: 8 -``` - -https://github.com/Atemu/nixpkgs/tree/498831397e77a265c240cf8f8a7d15e738f2f05b - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log -[2021-05-09 17:55:13.653645489] Committer: Committing changes to git -(recording state in git...) -[2021-05-09 17:55:13.702843262] Pusher: Syncing with SOTERIA -To ssh://192.168.101.24/~/Annex/Documents.git/ - 041f52d19..40cc59126 master -> synced/master -git-annex: internal error: evacuate: strange closure type 4325399 - (GHC version 8.10.4 for x86_64_unknown_linux) - Please report this as a GHC bug: https://www.haskell.org/ghc/reportabug -(recording state in git...) -(scanning...) ControlSocket .git/annex/ssh/atemu@192.168.101.24 already exists, disabling multiplexing -(started...) -(recording state in git...) -Unpacking all pack files. -remote: error: cannot lock ref 'refs/heads/synced/git-annex': is at 9166c83718695df3021d5bbf4fef2297d3f4cc84 but expected 7f761edbbe3c2a4710c629e9e7fdfb730c0639e7 -remote: error: cannot lock ref 'refs/heads/synced/master': is at 42cab466f705c2496fb19249e272487bceb38808 but expected 40cc591261675e872f79b6a9ea966215d3f73581 -To ssh://192.168.101.24/~/Annex/Documents.git/ - ! [remote rejected] git-annex -> synced/git-annex (failed to update ref) - ! [remote rejected] master -> synced/master (failed to update ref) -error: failed to push some refs to 'ssh://192.168.101.24/~/Annex/Documents.git/' -To ssh://192.168.101.24/~/Annex/Documents.git/ - 7f761edbb..9166c8371 git-annex -> synced/git-annex - 40cc59126..42cab466f master -> synced/master -fatal: early EOF -git-annex: internal error: evacuate: strange closure type 4325399 - (GHC version 8.10.4 for x86_64_unknown_linux) - Please report this as a GHC bug: https://www.haskell.org/ghc/reportabug -(recording state in git...) -ControlSocket .git/annex/ssh/atemu@192.168.101.24 already exists, disabling multiplexing -Auto packing the repository in background for optimum performance. -See "git help gc" for manual housekeeping. -error: Could not read 1f2ab777cdd4eb9add96d42de5022bde0a4a9a8b -error: Could not read 0f74453f84ff23d31b00a9bc9c331c00988f465a -error: Could not read 0f74453f84ff23d31b00a9bc9c331c00988f465a -error: Could not read 1f2ab777cdd4eb9add96d42de5022bde0a4a9a8b -error: Could not read 0f74453f84ff23d31b00a9bc9c331c00988f465a -error: Could not read 1f2ab777cdd4eb9add96d42de5022bde0a4a9a8b -error: Could not read 0f74453f84ff23d31b00a9bc9c331c00988f465a -error: Could not read 1f2ab777cdd4eb9add96d42de5022bde0a4a9a8b -error: Could not read 0f74453f84ff23d31b00a9bc9c331c00988f465a -hint: Using 'master' as the name for the initial branch. This default branch name -hint: is subject to change. To configure the initial branch name to use in all -hint: of your new repositories, which will suppress this warning, call: -hint: -hint: git config --global init.defaultBranch -hint: -hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and -hint: 'development'. The just-created branch can be renamed via this command: -hint: -hint: git branch -m -Initialized empty Git repository in /tmp/tmpreporFb6Gq/.git/ -Trying to recover missing objects from remote SOTERIA. -fatal: 'SOTERIA' does not appear to be a git repository -fatal: Could not read from remote repository. - -Please make sure you have the correct access rights -and the repository exists. -Trying to recover missing objects from remote SOTERIA. -fatal: 'SOTERIA' does not appear to be a git repository -fatal: Could not read from remote repository. - -Please make sure you have the correct access rights -and the repository exists. -65 missing objects could not be recovered! -To force a recovery to a usable state, retry with the --force parameter. -# End of transcript or log. -"""]] - -I'm not precisely sure when the actual crash happened (could be that it happened after starting up the machine today) as the log is a bit ambiguous. All I know is that it was corrupted today and that I was making commits till yesterday ~18:00. - -The objects mentioned in the log are both 6 days old; one is a commit on master and one on git-annex. - -Git fsck spat out a huge list of broken blobs and trees. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -There are a surprising amount of weird bugs and quirks like these but it's such an amazing tool, there's nothing like it. - -> [[fixed|done]] although I have opened [[assistant_repair_misfires]] for -> part of the cause of this. --[[Joey]] diff --git a/doc/bugs/Git_repos_corrupt_themselves/comment_10_74e6d17a1b14b2fdc7eddf4ee1895733._comment b/doc/bugs/Git_repos_corrupt_themselves/comment_10_74e6d17a1b14b2fdc7eddf4ee1895733._comment deleted file mode 100644 index 53492231a8..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves/comment_10_74e6d17a1b14b2fdc7eddf4ee1895733._comment +++ /dev/null @@ -1,601 +0,0 @@ -[[!comment format=mdwn - username="Atemu" - avatar="http://cdn.libravatar.org/avatar/d1f0f4275931c552403f4c6707bead7a" - subject="comment 10" - date="2021-05-25T11:00:39Z" - content=""" -A new crash just happened. I haven't upgraded git-annex to master yet but the strange closure bug didn't occur this time. Annotated daemon.log: - -``` -(scanning...) (started...) - -ssh: connect to host 192.168.101.68 port 22: Connection timed out - -add file.etherpad ok -(recording state in git...) - - -(recording state in git...) -(recording state in git...) -git-annex: fsck: 1 failed -(recovering from race...) -ssh: connect to host 192.168.101.68 port 22: Connection timed out -fatal: Could not read from remote repository. - -Please make sure you have the correct access rights -and the repository exists. - - -remote: error: cannot lock ref 'refs/heads/master': is at 6e737fee3f50ee203c9db151e3f9d0308c63df9b but expected fc38ba7550f8dd0d29262bc47ae3085403a55784 To ssh://192.168.101.24/~/Annex/Documents.git/ ! [remote rejected] master -> master (failed to update ref)error: failed to push some refs to 'ssh://192.168.101.24/~/Annex/Documents.git/'ssh: connect to host 192.168.101.68 port 22: No route to host -fatal: Could not read from remote repository.Please make sure you have the correct access rightsand the repository exists.ssh: connect to host 192.168.101.68 port 22: Connection timed out -fatal: Could not read from remote repository.Please make sure you have the correct access rightsand the repository exists.ssh: connect to host 192.168.101.68 port 22: Connection timed out -fatal: Could not read from remote repository.Please make sure you have the correct access rightsand the repository exists.ssh: connect to host 192.168.101.68 port 22: Connection timed out -fatal: Could not read from remote repository.Please make sure you have the correct access rightsand the repository exists.ssh: connect to host 192.168.101.68 port 22: No route to host -fatal: Could not read from remote repository.Please make sure you have the correct access rightsand the repository exists.ssh: connect to host 192.168.101.68 port 22: No route to host -fatal: Could not read from remote repository.Please make sure you have the correct access rightsand the repository exists. Transfer failed -Unpacking all pack files. -hint: Using 'master' as the name for the initial branch. This default branch name -hint: is subject to change. To configure the initial branch name to use in all -hint: of your new repositories, which will suppress this warning, call: -hint: -hint: git config --global init.defaultBranch -hint: -hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and -hint: 'development'. The just-created branch can be renamed via this command: -hint: -hint: git branch -m -Initialized empty Git repository in /tmp/tmprepoZrGQTy/.git/ -Trying to recover missing objects from remote MB-. -fatal: 'MB-' does not appear to be a git repository -fatal: Could not read from remote repository. - -Please make sure you have the correct access rights -and the repository exists. -Trying to recover missing objects from remote SOTERIA. -fatal: 'SOTERIA' does not appear to be a git repository -fatal: Could not read from remote repository. - -Please make sure you have the correct access rights -and the repository exists. -Trying to recover missing objects from remote MB-. -fatal: 'MB-' does not appear to be a git repository -fatal: Could not read from remote repository. - -Please make sure you have the correct access rights -and the repository exists. -Trying to recover missing objects from remote SOTERIA. -fatal: 'SOTERIA' does not appear to be a git repository -fatal: Could not read from remote repository. - -Please make sure you have the correct access rights -and the repository exists. -2 missing objects could not be recovered! -To force a recovery to a usable state, retry with the --force parameter. -error: refs/heads/master does not point to a valid object! -error: refs/heads/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/heads/master does not point to a valid object! -error: refs/heads/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/heads/master does not point to a valid object! -error: refs/heads/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/heads/master does not point to a valid object! -error: refs/heads/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/heads/master does not point to a valid object! -error: refs/heads/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/heads/master does not point to a valid object! -error: refs/heads/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/heads/master does not point to a valid object! -error: refs/heads/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/heads/master does not point to a valid object! -error: refs/heads/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -add <.txt render of the etherpad file> fatal: Invalid revision range refs/heads/master..refs/synced/d7d728f7-891a-4035-a758-c7ee80a8017a/master -error: refs/heads/master does not point to a valid object! -error: refs/heads/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/heads/master does not point to a valid object! -error: refs/heads/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -ok -(recording state in git...) -error: refs/heads/master does not point to a valid object! -error: refs/heads/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! - -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -CONFLICT (add/add): Merge conflict in .org -Auto-merging .org - -Automatic merge failed; fix conflicts and then commit the result. - -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: Merging is not possible because you have unmerged files. -hint: Fix them up in the work tree, and then use 'git add/rm ' -hint: as appropriate to mark resolution and make a commit. -fatal: Exiting because of an unresolved conflict. -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! - - -100% 3.33 KiB 198 MiB/s 0s - -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -ssh: connect to host 192.168.101.68 port 22: Connection timed out -fatal: Could not read from remote repository. - -Please make sure you have the correct access rights -and the repository exists. -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! - -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: Merging is not possible because you have unmerged files. -hint: Fix them up in the work tree, and then use 'git add/rm ' -hint: as appropriate to mark resolution and make a commit. -fatal: Exiting because of an unresolved conflict. - -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: Merging is not possible because you have unmerged files. -hint: Fix them up in the work tree, and then use 'git add/rm ' -hint: as appropriate to mark resolution and make a commit. -fatal: Exiting because of an unresolved conflict. -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -ssh: connect to host 192.168.101.68 port 22: No route to host - - -ssh: connect to host 192.168.101.68 port 22: Connection timed out -fatal: Could not read from remote repository.Please make sure you have the correct access rightsand the repository exists. Transfer failed -fatal: Could not read from remote repository. - -Please make sure you have the correct access rights -and the repository exists. -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -ssh: connect to host 192.168.101.68 port 22: Connection timed out -fatal: Could not read from remote repository. - -Please make sure you have the correct access rights -and the repository exists. -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -ssh: connect to host 192.168.101.68 port 22: Connection timed out -fatal: Could not read from remote repository. - -Please make sure you have the correct access rights -and the repository exists. -To ssh://192.168.101.24/~/Annex/Documents.git/ - 105d6d0..4bd9661 git-annex -> synced/git-annex - ! [rejected] master -> synced/master (fetch first) -error: failed to push some refs to 'ssh://192.168.101.24/~/Annex/Documents.git/' -hint: Updates were rejected because the remote contains work that you do -hint: not have locally. This is usually caused by another repository pushing -hint: to the same ref. You may want to first integrate the remote changes -hint: (e.g., 'git pull ...') before pushing again. -hint: See the 'Note about fast-forwards' in 'git push --help' for details. -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -ssh: connect to host 192.168.101.68 port 22: No route to host -fatal: Could not read from remote repository. - -Please make sure you have the correct access rights -and the repository exists. -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -error: refs/remotes/SOTERIA/master does not point to a valid object! -error: refs/remotes/SOTERIA/synced/master does not point to a valid object! -From ssh://192.168.101.24/~/Annex/Documents - * [new branch] master -> SOTERIA/master - * [new branch] synced/master -> SOTERIA/synced/master -Auto packing the repository in background for optimum performance. -See \"git help gc\" for manual housekeeping. - -error: Merging is not possible because you have unmerged files. -hint: Fix them up in the work tree, and then use 'git add/rm ' -hint: as appropriate to mark resolution and make a commit. -fatal: Exiting because of an unresolved conflict. - -error: Merging is not possible because you have unmerged files. -hint: Fix them up in the work tree, and then use 'git add/rm ' -hint: as appropriate to mark resolution and make a commit. -fatal: Exiting because of an unresolved conflict. - -error: Merging is not possible because you have unmerged files. -hint: Fix them up in the work tree, and then use 'git add/rm ' -hint: as appropriate to mark resolution and make a commit. -fatal: Exiting because of an unresolved conflict. - -error: Merging is not possible because you have unmerged files. -hint: Fix them up in the work tree, and then use 'git add/rm ' -hint: as appropriate to mark resolution and make a commit. -fatal: Exiting because of an unresolved conflict. -To ssh://192.168.101.24/~/Annex/Documents.git/ - ! [rejected] master -> synced/master (non-fast-forward) -error: failed to push some refs to 'ssh://192.168.101.24/~/Annex/Documents.git/' -hint: Updates were rejected because a pushed branch tip is behind its remote -hint: counterpart. Check out this branch and integrate the remote changes -hint: (e.g. 'git pull ...') before pushing again. -hint: See the 'Note about fast-forwards' in 'git push --help' for details. -ssh: connect to host 192.168.101.68 port 22: Connection timed out -fatal: Could not read from remote repository. - -Please make sure you have the correct access rights -and the repository exists. -ssh: connect to host 192.168.101.68 port 22: No route to host -fatal: Could not read from remote repository. - -Please make sure you have the correct access rights -and the repository exists. -To ssh://192.168.101.24/~/Annex/Documents.git/ - 8cd959c..4bd9661 git-annex -> refs/synced/8dc76e63-3dac-4cc3-a752-e73e81d57c1a/git-annex - ! [rejected] master -> refs/synced/8dc76e63-3dac-4cc3-a752-e73e81d57c1a/master (non-fast-forward) -error: failed to push some refs to 'ssh://192.168.101.24/~/Annex/Documents.git/' -hint: Updates were rejected because a pushed branch tip is behind its remote -hint: counterpart. Check out this branch and integrate the remote changes -hint: (e.g. 'git pull ...') before pushing again. -hint: See the 'Note about fast-forwards' in 'git push --help' for details. -ssh: connect to host 192.168.101.68 port 22: Connection timed out -``` - -Git is now trying to merge refs/synced/d7d728f7-891a-4035-a758-c7ee80a8017a/master which points at a commit made on the MBP 4d ago. This commit has previously been **merged** into the original branch on this machine and contains a change to a txt render of a different .etherpad file. This also means that the commit before it contains a change to the according .etherpad file. - -My branch is now a single commit made at 11:49:21 that adds the whole state of the working dir. -However, the diff for the .etherpad file says (without textconv): - -``` -diff --git a/Uni/LAWR/V10/132c6ef4e421c88547d4a228fef3c39491ba0610.etherpad b/Uni/LAWR/V10/132c6ef4e421c88547d4a228fef3c39491ba0610.etherpad -new file mode 100644 -index 0000000..33e0a8f ---- /dev/null -+++ b/Uni/LAWR/V10/132c6ef4e421c88547d4a228fef3c39491ba0610.etherpad -@@ -0,0 +1 @@ -+/annex/objects/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2 -diff --git a/Uni/LAWR/V10/132c6ef4e421c88547d4a228fef3c39491ba0610.txt b/Uni/LAWR/V10/132c6ef4e421c88547d4a228fef3c39491ba0610.txt -new file mode 100644 -index 0000000..f9ce6fe ---- /dev/null -+++ b/Uni/LAWR/V10/132c6ef4e421c88547d4a228fef3c39491ba0610.txt -@@ -0,0 +1 @@ -+/annex/objects/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt -``` - -It added these as a unlocked largefiles despite my largefiles rule excluding them. This has happened before once I think but I manually committed at that time. - -The other .etherpad files I had in the tree were not added as largefiles. - -I remember having issues with textconv and annex before too and disabling a few rules that I didn't really need. - -The git-annex branch seems to be in-tact. However, its commits are a bit weird. Here is `git log synced/git-annex..git-annex --patch`: - -``` -commit 4bd9661256a12d81186420a16d1bc1fb5f5d261a -Author: Atemu -Date: Tue May 25 11:49:22 2021 +0200 - - update (recovery from race #1; expected commit parent Ref \"5edabb3782060a334f966387e03252440ef50b10\" but found [Ref \"c518e6a6cac2b966e757a97dc8d26b64743be377\"] ) - -commit efdaa77dbf258954289366e236f056e50a258e1a -Author: Atemu -Date: Tue May 25 11:49:22 2021 +0200 - - update - -commit c518e6a6cac2b966e757a97dc8d26b64743be377 -Author: Atemu -Date: Tue May 25 11:49:22 2021 +0200 - - update (recovery from race #1; expected commit parent Ref \"5edabb3782060a334f966387e03252440ef50b10\" but found [Ref \"79a6d5f9b7b45ae891529a9bf2ead592f3968faa\"] ) - -commit fd7d4f7a6c875ca43cfe5e2e34b27c4cd6daab0a -Author: Atemu -Date: Tue May 25 11:49:22 2021 +0200 - - update - -commit 79a6d5f9b7b45ae891529a9bf2ead592f3968faa -Author: Atemu -Date: Tue May 25 11:49:22 2021 +0200 - - update (recovery from race #1; expected commit parent Ref \"5edabb3782060a334f966387e03252440ef50b10\" but found [Ref \"79190c0161b13aae6570cb24f232ea6ef9288f66\"] ) - -commit 1276807833ad40c19190049a85ba82b5b171edd2 -Author: Atemu -Date: Tue May 25 11:49:22 2021 +0200 - - update - -commit 79190c0161b13aae6570cb24f232ea6ef9288f66 -Author: Atemu -Date: Tue May 25 11:49:22 2021 +0200 - - update - -diff --git a/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log b/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log -index 92b1312..0ff0604 100644 ---- a/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log -+++ b/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log -@@ -1 +1,2 @@ - 1621936161.088002573s 1 8dc76e63-3dac-4cc3-a752-e73e81d57c1a -+1621936161.329141017s 1 b0512e28-91b3-4792-a4ef-71357fc4b63a - -commit 5edabb3782060a334f966387e03252440ef50b10 -Author: Atemu -Date: Tue May 25 11:49:21 2021 +0200 - - update - -diff --git a/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log b/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log -new file mode 100644 -index 0000000..92b1312 ---- /dev/null -+++ b/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log -@@ -0,0 +1 @@ -+1621936161.088002573s 1 8dc76e63-3dac-4cc3-a752-e73e81d57c1a -diff --git a/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log.met b/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log.met -new file mode 100644 -index 0000000..9972f75 ---- /dev/null -+++ b/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log.met -@@ -0,0 +1 @@ -+1621936161.086396922s day +25 month +5 year +2021 - -commit 6dbb8cc706949f8ef1dbbbd1003dbe728b642f8a -Author: Atemu -Date: Tue May 25 11:45:51 2021 +0200 - - update (recovery from race #1; expected commit parent Ref \"1ccaa3ce85972818d9afee1ff047183d8836c927\" but found [Ref \"260477393b0e18fbf0d4fc77afa10ab0069f2d40\"] ) - -commit 1f035559a42966383b424ebb0dd62509f8259543 -Author: Atemu -Date: Tue May 25 11:45:51 2021 +0200 - - update - -commit 260477393b0e18fbf0d4fc77afa10ab0069f2d40 -Author: Atemu -Date: Tue May 25 11:45:51 2021 +0200 - - update - -diff --git a/activity.log b/activity.log -index e388b9e..890cae5 100644 ---- a/activity.log -+++ b/activity.log -@@ -1,37 +1,8 @@ - 00b14f31-5c38-4eb3-b16f-17140b671470 Fsck timestamp=1610259369.931977s - 1968d496-fca5-4b48-ac56-b4de73d6af82 Fsck timestamp=1609853522.83856s --283a9cfc-cc28-481f-8ab3-28564f61e278 Fsck timestamp=1614085193.126619514s --283a9cfc-cc28-481f-8ab3-28564f61e278 Fsck timestamp=1614155865.00518998s - 283a9cfc-cc28-481f-8ab3-28564f61e278 Fsck timestamp=1614159798.683064383s - 4ea64ba8-3d84-439d-88c9-021380c3b097 Fsck timestamp=1610235815.801036s --8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1614161439.951758036s --8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1614161875.368919604s --8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1614503753.902534237s --8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1618474275.133432744s --8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1619560801.449019468s --8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1620081126.285146124s --8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1620655938.401915298s --8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621159014.044988654s --8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621161156.147989006s --8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621161290.856706788s --8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621161317.85070715s --8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621161413.579088689s --8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621328885.881572492s --8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621422002.80957147s -+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621935951.378910574s - a45a1abf-086e-40ea-8db5-f286a4fb69ef Fsck timestamp=1609849766.99771s --d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1614161148.72413s --d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1614162182.895703s --d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1614162687.460662s --d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1618351280.306775s --d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1619444550.542679s --d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1619733602.694413s --d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1619833893.739576s --d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1620079818.367555s --d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1620513005.967486s --d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1620775822.267693s --d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1621161025.720264s --d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1621239795.218301s - d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1621444868.775013s --f5f5f557-ecf2-4c2e-b704-add87173d58a Fsck timestamp=1612954464.455509s --f5f5f557-ecf2-4c2e-b704-add87173d58a Fsck timestamp=1614085579.354101s - f5f5f557-ecf2-4c2e-b704-add87173d58a Fsck timestamp=1614160037.595461s - -commit 1ccaa3ce85972818d9afee1ff047183d8836c927 -Author: Atemu -Date: Tue May 25 11:45:51 2021 +0200 - - update - -diff --git a/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log b/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log -index 0252166..3150f67 100644 ---- a/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log -+++ b/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log -@@ -1 +1,2 @@ - 1621935950.772957836s 1 8dc76e63-3dac-4cc3-a752-e73e81d57c1a -+1621935951.025150823s 1 b0512e28-91b3-4792-a4ef-71357fc4b63a - -commit 105d6d0ddea616d0f62f5bb78080954545966947 -Author: Atemu -Date: Tue May 25 11:45:50 2021 +0200 - - update - -diff --git a/activity.log b/activity.log -index f925d0a..e388b9e 100644 ---- a/activity.log -+++ b/activity.log -@@ -1,8 +1,37 @@ - 00b14f31-5c38-4eb3-b16f-17140b671470 Fsck timestamp=1610259369.931977s - 1968d496-fca5-4b48-ac56-b4de73d6af82 Fsck timestamp=1609853522.83856s -+283a9cfc-cc28-481f-8ab3-28564f61e278 Fsck timestamp=1614085193.126619514s -+283a9cfc-cc28-481f-8ab3-28564f61e278 Fsck timestamp=1614155865.00518998s - 283a9cfc-cc28-481f-8ab3-28564f61e278 Fsck timestamp=1614159798.683064383s - 4ea64ba8-3d84-439d-88c9-021380c3b097 Fsck timestamp=1610235815.801036s -+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1614161439.951758036s -+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1614161875.368919604s -+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1614503753.902534237s -+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1618474275.133432744s -+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1619560801.449019468s -+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1620081126.285146124s -+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1620655938.401915298s -+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621159014.044988654s -+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621161156.147989006s -+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621161290.856706788s -+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621161317.85070715s -+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621161413.579088689s -+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621328885.881572492s - 8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621422002.80957147s - a45a1abf-086e-40ea-8db5-f286a4fb69ef Fsck timestamp=1609849766.99771s -+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1614161148.72413s -+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1614162182.895703s -+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1614162687.460662s -+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1618351280.306775s -+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1619444550.542679s -+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1619733602.694413s -+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1619833893.739576s -+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1620079818.367555s -+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1620513005.967486s -+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1620775822.267693s -+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1621161025.720264s -+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1621239795.218301s - d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1621444868.775013s -+f5f5f557-ecf2-4c2e-b704-add87173d58a Fsck timestamp=1612954464.455509s -+f5f5f557-ecf2-4c2e-b704-add87173d58a Fsck timestamp=1614085579.354101s - f5f5f557-ecf2-4c2e-b704-add87173d58a Fsck timestamp=1614160037.595461s -diff --git a/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log b/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log -new file mode 100644 -index 0000000..0252166 ---- /dev/null -+++ b/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log -@@ -0,0 +1 @@ -+1621935950.772957836s 1 8dc76e63-3dac-4cc3-a752-e73e81d57c1a -diff --git a/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log.met b/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log.met -new file mode 100644 -index 0000000..2aa827d ---- /dev/null -+++ b/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log.met -@@ -0,0 +1 @@ -+1621935950.772381243s day +25 month +5 year +2021 -diff --git a/efe/f7f/SHA256E-s2496630--0f778a1c32dfddf8098406ff28297d826b28759b4efeb1aa27a1841dd4897248.pdf.log b/efe/f7f/SHA256E-s2496630--0f778a1c32dfddf8098406ff28297d826b28759b4efeb1aa27a1841dd4897248.pdf.log -index 7fd0604..d907850 100644 ---- a/efe/f7f/SHA256E-s2496630--0f778a1c32dfddf8098406ff28297d826b28759b4efeb1aa27a1841dd4897248.pdf.log -+++ b/efe/f7f/SHA256E-s2496630--0f778a1c32dfddf8098406ff28297d826b28759b4efeb1aa27a1841dd4897248.pdf.log -@@ -1,3 +1,4 @@ - 1621350596.596630026s 1 8dc76e63-3dac-4cc3-a752-e73e81d57c1a -+1621350602.99782945s 1 b0512e28-91b3-4792-a4ef-71357fc4b63a - 1621350603.027097277s 1 b0512e28-91b3-4792-a4ef-71357fc4b63a - 1621445067.293418s 1 d7d728f7-891a-4035-a758-c7ee80a8017a -diff --git a/efe/f7f/SHA256E-s2496630--0f778a1c32dfddf8098406ff28297d826b28759b4efeb1aa27a1841dd4897248.pdf.log.met b/efe/f7f/SHA256E-s2496630--0f778a1c32dfddf8098406ff28297d826b28759b4efeb1aa27a1841dd4897248.pdf.log.met -index b7f2ee1..2e12d4b 100644 ---- a/efe/f7f/SHA256E-s2496630--0f778a1c32dfddf8098406ff28297d826b28759b4efeb1aa27a1841dd4897248.pdf.log.met -+++ b/efe/f7f/SHA256E-s2496630--0f778a1c32dfddf8098406ff28297d826b28759b4efeb1aa27a1841dd4897248.pdf.log.met -@@ -1 +1,2 @@ -+1621350596.859331937s day +18 month +5 year +2021 - 1621445071.205197s day -18 +19 month +5 year +2021 -diff --git a/schedule.log b/schedule.log -index 88f8251..f29a67c 100644 ---- a/schedule.log -+++ b/schedule.log -@@ -1,7 +1,16 @@ - 00b14f31-5c38-4eb3-b16f-17140b671470 fsck self 1h every day at any time timestamp=1610259355.086542s - 1968d496-fca5-4b48-ac56-b4de73d6af82 fsck self 1h every day at any time timestamp=1609853466.30146s -+283a9cfc-cc28-481f-8ab3-28564f61e278 fsck self 1h every day at any time timestamp=1609829311.089081351s -+283a9cfc-cc28-481f-8ab3-28564f61e278 fsck self 1h every day at any time timestamp=1609829331.993124414s - 283a9cfc-cc28-481f-8ab3-28564f61e278 fsck self 1h every day at any time timestamp=1609831717.238587482s -+283a9cfc-cc28-481f-8ab3-28564f61e278 fsck self 8d13h every day at any time timestamp=1609829516.974129064s - 4ea64ba8-3d84-439d-88c9-021380c3b097 timestamp=1610030955.40944s -+4ea64ba8-3d84-439d-88c9-021380c3b097 fsck self 1h every day at any time timestamp=1609857462.383747s - 8dc76e63-3dac-4cc3-a752-e73e81d57c1a timestamp=1621328896.270884603s -+8dc76e63-3dac-4cc3-a752-e73e81d57c1a fsck self 1h every day at any time timestamp=1614161325.360455592s -+8dc76e63-3dac-4cc3-a752-e73e81d57c1a fsck self 1h every day at any time timestamp=1616330220.055006692s -+8dc76e63-3dac-4cc3-a752-e73e81d57c1a fsck self 1h every day at any time timestamp=1620642662.423064041s -+8dc76e63-3dac-4cc3-a752-e73e81d57c1a fsck self 1h every day at any time; fsck self 10h1m every day at any time timestamp=1616329120.414652786s - d7d728f7-891a-4035-a758-c7ee80a8017a timestamp=1621444873.058565s -+d7d728f7-891a-4035-a758-c7ee80a8017a fsck self 1h every day at any time timestamp=1616364223.127229s - f5f5f557-ecf2-4c2e-b704-add87173d58a fsck self 1h every day at any time timestamp=1610259937.483933s -``` - -synced/master is in the same state as it was 4d ago, the date of the merge commit git is now trying to merge. -There have only been smallfile changes made to master since. - -I'm especially confused by the changes to schedule.log. - -*** - -Looks like there are some major incompatibilities with textconv and git-annex that need to be documented and/or fixed. - -Smallfiles don't seem to be handled that well either, perhaps my other issues are related to using those almost exclusively in this repo? -Automatic merges have never worked for me for example. (Though I probably wouldn't want them anyways). - -Is anyone else having issues with this combination of features? I can't be the only one, right? - -"""]] diff --git a/doc/bugs/Git_repos_corrupt_themselves/comment_11_d1f454482feef0350a6fa61bdf7673cc._comment b/doc/bugs/Git_repos_corrupt_themselves/comment_11_d1f454482feef0350a6fa61bdf7673cc._comment deleted file mode 100644 index db9b34e25a..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves/comment_11_d1f454482feef0350a6fa61bdf7673cc._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Atemu" - avatar="http://cdn.libravatar.org/avatar/d1f0f4275931c552403f4c6707bead7a" - subject="comment 11" - date="2021-05-31T19:07:31Z" - content=""" -The consistency check somehow enabled itself the other day and sure enough, as soon as I started the webapp, the repo corrupted itself on the MBP. Haven't seen a corruption on the Linux machine since I disabled consistency checks. -"""]] diff --git a/doc/bugs/Git_repos_corrupt_themselves/comment_12_5a13945d6bec6fa5fac84555f7ef8e21._comment b/doc/bugs/Git_repos_corrupt_themselves/comment_12_5a13945d6bec6fa5fac84555f7ef8e21._comment deleted file mode 100644 index 6aa661b5f5..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves/comment_12_5a13945d6bec6fa5fac84555f7ef8e21._comment +++ /dev/null @@ -1,21 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 12""" - date="2021-06-28T18:04:26Z" - content=""" -This seems conclusive that the repair is somehow triggering unncessarily -and also corrupting the repo in this situation. - -The comment #3 log shows that the repair is started, and then 1 minute -later a git object is missing. - -(It's odd that log shows a second fsck run after the repair was already -triggered. I do not see a way that this would happen unless fscks are -scheduled very close together.) - -The automatic repair is supposed to be a non-destructive repair; the -destructive repair only happens after prompting in the UI. - -This also reminds me of a persistent issue with a git-annex repo, using the -assistant, on my sister's laptop corrupting itself. -"""]] diff --git a/doc/bugs/Git_repos_corrupt_themselves/comment_13_6715bc3370f115f90627cfc4b06cf0e8._comment b/doc/bugs/Git_repos_corrupt_themselves/comment_13_6715bc3370f115f90627cfc4b06cf0e8._comment deleted file mode 100644 index 821c937b43..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves/comment_13_6715bc3370f115f90627cfc4b06cf0e8._comment +++ /dev/null @@ -1,20 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 13""" - date="2021-06-28T18:13:40Z" - content=""" -The repair process moves all pack files to a temp dir and then unpacks the -loose objects from them. So, there is a time window, when the repair is -running, where git objects that were present before will be missing. And if -the assistant stops before that is complete, it would leave it in that -state. Unpacking pack files can take a long time, so this might -be a sufficient explanation. - -But then, something must be causing it to incorrectly think it needs -a repair in the first place. Assuming it is incorrect, of course. Either git -fsck is exiting nonzero for some reason, or git-annex is thinking -it sees git fsck complain about a missing object, that is not really -missing. While there are fsck outputs that it can misinterpret, it -double-checks by trying to cat the object, which should avoid the latter -problem. -"""]] diff --git a/doc/bugs/Git_repos_corrupt_themselves/comment_14_6234b04455acc0b2b2f22a1d8c2ab475._comment b/doc/bugs/Git_repos_corrupt_themselves/comment_14_6234b04455acc0b2b2f22a1d8c2ab475._comment deleted file mode 100644 index d8124b248e..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves/comment_14_6234b04455acc0b2b2f22a1d8c2ab475._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 14""" - date="2021-06-28T18:28:06Z" - content=""" -To avoid moving the pack files, repair could set `GIT_OBJECT_DIRECTORY` -to a temp directory, and copy each pack file into it in turn, and unpack. -And after each unpack, move the unpacked objects from the temp directory to -the real object directory, followed by deleting the pack file (in case it's -corrupt). -"""]] diff --git a/doc/bugs/Git_repos_corrupt_themselves/comment_16_586e591a1e3cc53fd0108da91492c5c2._comment b/doc/bugs/Git_repos_corrupt_themselves/comment_16_586e591a1e3cc53fd0108da91492c5c2._comment deleted file mode 100644 index c3f3afa06e..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves/comment_16_586e591a1e3cc53fd0108da91492c5c2._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 16""" - date="2021-06-29T16:11:03Z" - content=""" -I have verified that an interrupted repair results in data loss, when -.git/objects contains pack files. - -Fixed that. Which leaves 2 open questions: - -* Was the assistant interrupted somehow while it was running repair, in - Atemu's case? That could have been anything from the laptop being - powered off, to it being stopped explicitly, to it crashing. -* What caused the assistant to think it needed repair in the first place? - -Although the answer to the second question doesn't matter much if the -data loss bug is fixed -- if there's a problem of some kind causing -unnecessary repairs, it would only be some excess CPU load. -"""]] diff --git a/doc/bugs/Git_repos_corrupt_themselves/comment_16_7a1cdfba17f129f6bf87a71549bbaab6._comment b/doc/bugs/Git_repos_corrupt_themselves/comment_16_7a1cdfba17f129f6bf87a71549bbaab6._comment deleted file mode 100644 index 00374bc0bd..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves/comment_16_7a1cdfba17f129f6bf87a71549bbaab6._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 16""" - date="2021-06-28T18:48:32Z" - content=""" -Removing repair from the assistant (and git-annex repair) should be on the -table as a solution to this. It's a whole lot of complexity that might fix -a few user's repos sometimes, but is outside of git-annex's scope and is -mostly only used by assistant users. -"""]] diff --git a/doc/bugs/Git_repos_corrupt_themselves/comment_17_faaa433ba8df649771469fdbcd9bb69e._comment b/doc/bugs/Git_repos_corrupt_themselves/comment_17_faaa433ba8df649771469fdbcd9bb69e._comment deleted file mode 100644 index e7459a8c71..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves/comment_17_faaa433ba8df649771469fdbcd9bb69e._comment +++ /dev/null @@ -1,52 +0,0 @@ -[[!comment format=mdwn - username="Atemu" - avatar="http://cdn.libravatar.org/avatar/d1f0f4275931c552403f4c6707bead7a" - subject="comment 17" - date="2021-06-29T22:57:39Z" - content=""" -Hi Joey, thank you for looking into this again. - -> It's odd that log shows a second fsck run after the repair was already triggered. I do not see a way that this would happen unless fscks are scheduled very close together.) - -Note that I started the assistant again in that log. The consistency check didn't finish the first time because somehow corruption was detected and the assistant crashed itself in trying to repair (I didn't accept any repair prompts in the GUI that time IIRC). - -AFAICT the consistency check is what triggers the issue because, since turning it off (and creating new repos on my clients), I haven't experienced a single repo corruption. - -The schedule was set to whatever the assistant nudges you to set by default. - -> comment 14 - -That sounds a lot more sane. - -Perhaps the repair functionality could use a complete overhaul. I've also found it to be extremely slow compared to simply copying missing or corrupt objects/packs from another remote. - -Ideally it should be able to restore a git repo to a working state without .git/objects present at all as it should only needs remotes and refs. -Extracting objects from the corrupt objects dir which aren't in any remote should be done if possible though because important local-only objects are a thing (stashes, additional branches, branches that are ahead of the remote's) - -> Removing repair from the assistant (and git-annex repair) should be on the table as a solution to this. - -Manual repair and detection can stay IMO but any *automatic* repair needs to go entirely or be optional. - -> Removing repair from the assistant (and git-annex repair) should be on the table as a solution to this. It's a whole lot of complexity that might fix a few user's repos sometimes, but is outside of git-annex's scope and is mostly only used by assistant users. - -I've actually (tried) using it from the CLI too, having it as an option there would be useful. - -I agree that it's out of scope though. It would probably be better off as a separate project as it's useful in any git repo, not just git-annex ones. Some integration with git-annex would be appreciated though. -Perhaps this functionality could be fully spun out into git-repair which could then become an optional add-on to git-annex. - -> Was the assistant interrupted somehow while it was running repair, in Atemu's case? - -The laptop being powered off etc. was not the case. Though I do shut down the daemon quite often because of https://git-annex.branchable.com/bugs/OSX__58___Pushed_changes_are_autocommited/ (via the GUI or --stop). Do the shutdown and restart procedures check for a running repair? - -I also sometimes had to kill all git processes for one reason or another but I don't think that ever happened after I pressed the repair button (I wait that out for good reason). - -*** - -So current issues distilled from this thread so far: - -* The scheduled consistency check can trigger automatic(!) repairs and even do so erroneously -* This repair procedure does something destructive without a prompt -* Applying the destructive repair to a working repo corrupts it -* Repair can't actually repair the mishap despite all the data being available -* Repair would probably be better off as a dedicated project outside of git-annex -"""]] diff --git a/doc/bugs/Git_repos_corrupt_themselves/comment_18_238bbbcabd626c33acc14f930a7d536c._comment b/doc/bugs/Git_repos_corrupt_themselves/comment_18_238bbbcabd626c33acc14f930a7d536c._comment deleted file mode 100644 index a2ffd63a87..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves/comment_18_238bbbcabd626c33acc14f930a7d536c._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 18""" - date="2021-06-30T15:36:39Z" - content=""" -If the assistant crashed in the middle of a repair that confirms my -analysis, and my fix will avoid both the data loss problem *and* the crash. - -Only remaining question to me is why would it trigger an unnecessary repair. - -But that could be anything that causes git fsck to exit nonzero. Or it -might be that git fsck found an actual problem, but not one that was -preventing the repo from working. Eg, a missing/corrupt object used -somewhere deep in the history. - -(Note that git-repair already exists. git-annex integrates it.) -"""]] diff --git a/doc/bugs/Git_repos_corrupt_themselves/comment_1_c7fbc3da4404fb33a4b0babf068dc4d3._comment b/doc/bugs/Git_repos_corrupt_themselves/comment_1_c7fbc3da4404fb33a4b0babf068dc4d3._comment deleted file mode 100644 index 001f6a85ec..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves/comment_1_c7fbc3da4404fb33a4b0babf068dc4d3._comment +++ /dev/null @@ -1,22 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2021-05-10T14:25:20Z" - content=""" -git-annex does not write directly to your git repository; all writes go via -git. So it is hard to see how any corruption like this could be due to a -bug in git-annex. - -What it usually is caused by is a filesystem losing data, eg because of an -unclean shutdown or a crash, or sometimes because of a disk error. - -Setting the git config core.fsyncObjectFiles may help. Configuring the -filesystem to write data to disk more promptly (eg enabling journalling of -file contents) may help. - -`git-annex repair --force` may be able to recover from this situation. - -The "evacuate: strange closure" error does seem notable, something strange -is going on in the memory of git-annex. Presumably due to either a compiler -bug or some kind of memory problem. -"""]] diff --git a/doc/bugs/Git_repos_corrupt_themselves/comment_2_f2368358a0ccdd36a551a2c833cb9c9a._comment b/doc/bugs/Git_repos_corrupt_themselves/comment_2_f2368358a0ccdd36a551a2c833cb9c9a._comment deleted file mode 100644 index 983c18951f..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves/comment_2_f2368358a0ccdd36a551a2c833cb9c9a._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="Lukey" - avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b" - subject="comment 2" - date="2021-05-11T07:37:07Z" - content=""" -This also looks suspicious: - - Auto packing the repository in background for optimum performance. - See \"git help gc\" for manual housekeeping. - -And after that something (git-annex?) is complaining about missing git objects. So maybe you can try to set `gc.auto` to `0` and see if that works better. -"""]] diff --git a/doc/bugs/Git_repos_corrupt_themselves/comment_3_f7d921256107773dcb294747cdc38ebd._comment b/doc/bugs/Git_repos_corrupt_themselves/comment_3_f7d921256107773dcb294747cdc38ebd._comment deleted file mode 100644 index dec219faf8..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves/comment_3_f7d921256107773dcb294747cdc38ebd._comment +++ /dev/null @@ -1,73 +0,0 @@ -[[!comment format=mdwn - username="Atemu" - avatar="http://cdn.libravatar.org/avatar/d1f0f4275931c552403f4c6707bead7a" - subject="comment 3" - date="2021-05-12T14:17:05Z" - content=""" -The weird thing is, this only ever happens with my Documents git-annex repo, the only one where I use the assistant. It has never happened in 4 years of actively using regular git nor my manually committed and synced git-annex repos. - -Another corruption just occured but this time on the MBP: - -``` -[2021-05-11 08:04:43.038047] main: starting assistant version 8.20210330 -[2021-05-11 08:04:43.169107] Cronner: Consistency check in progress -[2021-05-11 08:04:43.175482] TransferScanner: Syncing with SOTERIA -(scanning...) [2021-05-11 08:04:43.340744] Watcher: Performing startup scan -(recording state in git...) -(recording state in git...) -(recovering from race...) -To ssh://192.168.101.24/~/Annex/Documents.git/ - 7b587c48b..4c7b284c5 git-annex -> synced/git-annex -[2021-05-11 08:04:47.087899] Cronner: Attempting to repair MB- [here] -Watcher crashed: PauseWatcher -[2021-05-11 08:04:47.138205] Watcher: warning Watcher crashed: PauseWatcher -Unpacking all pack files. -fatal: early EOF -[2021-05-11 08:05:27.149344] main: starting assistant version 8.20210330 -[2021-05-11 08:05:27.325057] Cronner: Consistency check in progress -(recording state in git...) -error: invalid object 100644 f30c0963ac4560878ae6803608310c289de30e26 for '319/854/SHA256E-s1400--dd4744948961710e8369365e439bb37013c5664ed6d8e58ef21a855a08ff4108.o.log' -fatal: git-write-tree: error building trees -.git-annex-wrapped: failed to read sha from git write-tree -CallStack (from HasCallStack): - error, called at ./Git/Sha.hs:23:15 in main:Git.Sha -[2021-05-11 08:05:28.719256] TransferScanner: Syncing with SOTERIA -(recording state in git...) -error: invalid object 100644 f30c0963ac4560878ae6803608310c289de30e26 for '319/854/SHA256E-s1400--dd4744948961710e8369365e439bb37013c5664ed6d8e58ef21a855a08ff4108.o.log' -fatal: git-write-tree: error building trees -Merger crashed: failed to read sha from git write-tree -CallStack (from HasCallStack): - error, called at ./Git/Sha.hs:23:15 in main:Git.Sha -[2021-05-11 08:05:28.905248] Merger: warning Merger crashed: failed to read sha from git write-tree -CallStack (from HasCallStack): - error, called at ./Git/Sha.hs:23:15 in main:Git.Sha -Auto packing the repository in background for optimum performance. -See \"git help gc\" for manual housekeeping. -error: Could not read d3720891bb000797d5fba08403540f8db208fd7a -error: Could not read 0fa98c99e87fd34202e68016c39f7ba00b7e3e1a -error: Could not read 0fa98c99e87fd34202e68016c39f7ba00b7e3e1a -error: Could not read 0fa98c99e87fd34202e68016c39f7ba00b7e3e1a -error: Could not read d3720891bb000797d5fba08403540f8db208fd7a -error: Could not read 0fa98c99e87fd34202e68016c39f7ba00b7e3e1a -error: Could not read d3720891bb000797d5fba08403540f8db208fd7a -error: Could not read 0fa98c99e87fd34202e68016c39f7ba00b7e3e1a -error: Could not read d3720891bb000797d5fba08403540f8db208fd7a -(recording state in git...) -error: invalid object 100644 f30c0963ac4560878ae6803608310c289de30e26 for '319/854/SHA256E-s1400--dd4744948961710e8369365e439bb37013c5664ed6d8e58ef21a855a08ff4108.o.log' -fatal: git-write-tree: error building trees -TransferScanner crashed: failed to read sha from git write-tree -CallStack (from HasCallStack): - error, called at ./Git/Sha.hs:23:15 in main:Git.Sha -[2021-05-11 08:05:29.430437] TransferScanner: warning TransferScanner crashed: failed to read sha from git write-tree -CallStack (from HasCallStack): - error, called at ./Git/Sha.hs:23:15 in main:Git.Sha -[2021-05-11 08:05:52.410361] main: warning git-annex has been shut down -``` - -This happened just after I started the assistant/webapp and you can see that I shut it down in the end. - -My filesystems are ZFS on the Linux machine and APFS on the MBP. Being an Apple product, I'd expect APFS to be built to high standards (though I don't fully trust it) but ZFS corrupting anything or losing data is almost unheard of. -I'll try setting `core.fsyncObjectFiles` on this repo and update this issue accordingly when I see another corruption with it on. Shouldn't take too long as this occurs worringly frequently. - -Btw, repair always fails in my setup because git seems to be unable to find the `SOTERIA` remote; is it trying to use the remote name literally as an address perhaps? -"""]] diff --git a/doc/bugs/Git_repos_corrupt_themselves/comment_4_cf152fccbdb281dcbeb02cb398686ac0._comment b/doc/bugs/Git_repos_corrupt_themselves/comment_4_cf152fccbdb281dcbeb02cb398686ac0._comment deleted file mode 100644 index 58dcf16bd5..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves/comment_4_cf152fccbdb281dcbeb02cb398686ac0._comment +++ /dev/null @@ -1,85 +0,0 @@ -[[!comment format=mdwn - username="Atemu" - avatar="http://cdn.libravatar.org/avatar/d1f0f4275931c552403f4c6707bead7a" - subject="comment 4" - date="2021-05-16T10:27:47Z" - content=""" -Something peculiar just happened. I had the MBP with me while I was out and was editing documents with no connection to any of my machines. - -I usually stop the assistant at home so that it only runs on one machine at a time because of https://git-annex.branchable.com/bugs/OSX__58___Pushed_changes_are_autocommited/ but because I was out, I started it. - -On startup, it said that it was trying to repair my repo. I shut it down because it couldn't do any repairs without a connection to my machines (and it wouldn't succeed even if it did unfortunately). - -I ran `git fsck` to see what happened but, to my surprise, it returned no errors besides missing reflog entries from previous corruptions. `git gc` didn't error either. - -I commited my changes manually while I was out. - -That was about two days ago. I synced my changes manually at home where an unrelated and trivial issue happened during which I ran another successful `git fsck` and, after fixing that issue, I started the webapp. -It was attempting to repair again and was complaining about a thread having crashed (logs below). - -Now however, the repo is actually broken with missing blobs, trees and links everywhere. - -daemon.log.1: - -``` -[2021-05-16 11:58:04.848709] main: starting assistant version 8.20210330 -[2021-05-16 11:58:04.946261] Cronner: Consistency check in progress -[2021-05-16 11:58:04.974666] TransferScanner: Syncing with SOTERIA -(scanning...) [2021-05-16 11:58:05.191375] Watcher: Performing startup scan -(recording state in git...) -(recording state in git...) -(recovering from race...) -ControlSocket .git/annex/ssh/atemu@192.168.101.24 already exists, disabling multiplexing -To ssh://192.168.101.24/~/Annex/Documents.git/ - 343d6291c..184ff9756 git-annex -> synced/git-annex -[2021-05-16 11:58:09.404901] Cronner: Attempting to repair MB- [here] -Watcher crashed: PauseWatcher -[2021-05-16 11:58:09.445038] Watcher: warning Watcher crashed: PauseWatcher -Unpacking all pack files. -fatal: early EOF -``` - -daemon.log: - -``` -[2021-05-16 11:58:21.699212] main: starting assistant version 8.20210330 -[2021-05-16 11:58:21.778787] Cronner: You should enable consistency checking to protect your data. -[2021-05-16 11:58:22.249053] TransferScanner: Syncing with SOTERIA -Auto packing the repository in background for optimum performance. -See \"git help gc\" for manual housekeeping. -Everything up-to-date - - - -[2021-05-16 11:58:25.957064] Transferrer: Uploaded <1 pdf file> -[2021-05-16 11:58:25.95937] Pusher: Syncing with SOTERIA -(recording state in git...) -error: invalid object 100644 3d5a69ecabdbf871de3b7c339b16bc9315ae17e4 for '5d3/ae2/SHA256E-s56534--2de587e678bd455f4346a5304e450eafac892f1a389ee4046cd25948a01789a9.pdf.log.met' -fatal: git-write-tree: error building trees -Pusher crashed: failed to read sha from git write-tree -CallStack (from HasCallStack): - error, called at ./Git/Sha.hs:23:15 in main:Git.Sha -[2021-05-16 11:58:26.037969] Pusher: warning Pusher crashed: failed to read sha from git write-tree -CallStack (from HasCallStack): - error, called at ./Git/Sha.hs:23:15 in main:Git.Sha - - -[2021-05-16 11:58:28.564268] Transferrer: Uploaded -[2021-05-16 11:58:51.983601] Pusher: Syncing with SOTERIA -(recording state in git...) -error: invalid object 100644 3b1a2b60bcb28a299c7e6b706ce3de5808909bcc for '031/bc2/SHA256E-s138590--273323cbd322923f9d19280b6b4fa73bfa3638d2982b685b9efcaee43c988a07.pdf.log.met' -fatal: git-write-tree: error building trees -Pusher crashed: failed to read sha from git write-tree -CallStack (from HasCallStack): - error, called at ./Git/Sha.hs:23:15 in main:Git.Sha -[2021-05-16 11:58:52.11129] Pusher: warning Pusher crashed: failed to read sha from git write-tree -CallStack (from HasCallStack): - error, called at ./Git/Sha.hs:23:15 in main:Git.Sha -[2021-05-16 11:59:08.905898] main: warning git-annex has been shut down -``` - -I haven't set the fsync option yet but this system hasn't been restarted in a long time and the corruption happened while the system was on; right in front of my eyes basically. - -I'll now manually fix the corruption again by rsync'ing the `.git/objects/` dir to the state of a working repo's. - -"""]] diff --git a/doc/bugs/Git_repos_corrupt_themselves/comment_5_652928a8f8bd0250973f19fa28cbb029._comment b/doc/bugs/Git_repos_corrupt_themselves/comment_5_652928a8f8bd0250973f19fa28cbb029._comment deleted file mode 100644 index c5219f996a..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves/comment_5_652928a8f8bd0250973f19fa28cbb029._comment +++ /dev/null @@ -1,36 +0,0 @@ -[[!comment format=mdwn - username="Atemu" - avatar="http://cdn.libravatar.org/avatar/d1f0f4275931c552403f4c6707bead7a" - subject="comment 5" - date="2021-05-18T09:13:32Z" - content=""" -Another crash on the Linux machine as I was starting the assistant today. Usually it doesn't happen nearly this frequently though. - -`core.fsyncObjectFiles` was set to `true` this time. - -log.3 (2d ago 19:xx): - -``` -git-annex: internal error: evacuate: strange closure type 4566880 - (GHC version 8.10.4 for x86_64_unknown_linux) - Please report this as a GHC bug: https://www.haskell.org/ghc/reportabug -``` - -log.2 (today 10:16): - -``` - -ControlSocket .git/annex/ssh/e5a88504f7bbb083855b27a1c1ca4a26 already exists, disabling multiplexing -To ssh://192.168.101.24/~/Annex/Documents.git/ - f153f6740..b52bda802 git-annex -> synced/git-annex -To ssh://git-annex-.MacBook.2DPro.2Efritz.2Ebox-TobiasDe_22_Documents/~/Documents/ - bb6821230..b52bda802 git-annex -> synced/git-annex -(started...) -Unpacking all pack files. -fatal: early EOF -``` - -at the time of log.1 (10:55), corruption has already set in. - -I'm sniffing an issue with the auto-repair feature wrongly detecting corruption, trying to fix it and thereby actually corrupting the repo. I'd like to turn it off for investigation, is that possible somehow? By disabling consistency checks perhaps? -"""]] diff --git a/doc/bugs/Git_repos_corrupt_themselves/comment_6_315606cfa07d2c4b20a6c9ec477c83d1._comment b/doc/bugs/Git_repos_corrupt_themselves/comment_6_315606cfa07d2c4b20a6c9ec477c83d1._comment deleted file mode 100644 index f22fc03564..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves/comment_6_315606cfa07d2c4b20a6c9ec477c83d1._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 6""" - date="2021-05-21T16:26:55Z" - content=""" -Note that the strange closure crash is a separate bug, -and has been fixed in git master. I do not think that bug could corrupt git -repos since as I mentioned before git-annex doesn't write directly to the -git repo, so any such bug in it should not be possible to corrupt it. - -Anyway, it would probably be a good idea to upgrade to a current daily -build to get past that bug. -"""]] diff --git a/doc/bugs/Git_repos_corrupt_themselves/comment_7_54aed251ddb9df1c8164a25240e1533f._comment b/doc/bugs/Git_repos_corrupt_themselves/comment_7_54aed251ddb9df1c8164a25240e1533f._comment deleted file mode 100644 index 61a7cdb3c2..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves/comment_7_54aed251ddb9df1c8164a25240e1533f._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Atemu" - avatar="http://cdn.libravatar.org/avatar/d1f0f4275931c552403f4c6707bead7a" - subject="comment 7" - date="2021-05-22T17:02:07Z" - content=""" -What could I do to get to the bottom of this? Is there a verbose logging feature for the assistant that could give more insight? -"""]] diff --git a/doc/bugs/Git_repos_corrupt_themselves/comment_8_a1131917149dd07ec73367e7cbc3b3f2._comment b/doc/bugs/Git_repos_corrupt_themselves/comment_8_a1131917149dd07ec73367e7cbc3b3f2._comment deleted file mode 100644 index c893804877..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves/comment_8_a1131917149dd07ec73367e7cbc3b3f2._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Lukey" - avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b" - subject="comment 8" - date="2021-05-22T17:19:19Z" - content=""" -You could run the equivalent of `strace` for mac os and make it log every syscall of git-annex assistant and it's children. This would show which `git` action corrupts the repo and how. -"""]] diff --git a/doc/bugs/Git_repos_corrupt_themselves/comment_9_033def91d2a30f1cee498fabbfe5d37f._comment b/doc/bugs/Git_repos_corrupt_themselves/comment_9_033def91d2a30f1cee498fabbfe5d37f._comment deleted file mode 100644 index 632afdc356..0000000000 --- a/doc/bugs/Git_repos_corrupt_themselves/comment_9_033def91d2a30f1cee498fabbfe5d37f._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Atemu" - avatar="http://cdn.libravatar.org/avatar/d1f0f4275931c552403f4c6707bead7a" - subject="comment 9" - date="2021-05-22T17:31:01Z" - content=""" -This also happens on Linux, so I'd rather use `strace` itself but I think it'd be extremely hard to spot anything useful in that and probably is not worth the performance cost. -"""]] diff --git a/doc/bugs/Homebrew_build_broken.mdwn b/doc/bugs/Homebrew_build_broken.mdwn deleted file mode 100644 index 20f30d81ff..0000000000 --- a/doc/bugs/Homebrew_build_broken.mdwn +++ /dev/null @@ -1,3 +0,0 @@ -[The PR to upgrade git-annex in Homebrew to 10.20230126](https://github.com/Homebrew/homebrew-core/pull/121594) is currently stalled because of a build failure. The error message seems to imply the problem is due to a faulty `.cabal` file, which would mean it's a problem with the git-annex source itself. - -> [[fixed|done]] in homebrew --[[Joey]] diff --git a/doc/bugs/Homebrew_build_broken/comment_1_c30517144c98faaf592b0e8e2bcc7db4._comment b/doc/bugs/Homebrew_build_broken/comment_1_c30517144c98faaf592b0e8e2bcc7db4._comment deleted file mode 100644 index 0680a12e2d..0000000000 --- a/doc/bugs/Homebrew_build_broken/comment_1_c30517144c98faaf592b0e8e2bcc7db4._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-02-06T16:38:38Z" - content=""" -I don't think this is a problem with the cabal file. It has the necessary -build-depends on async. - -This seems to be some sort of toolchain problem. But not one that affects -other builds on OSX. - -I doubt that there is a change I can make to the source code that will help -them. -"""]] diff --git a/doc/bugs/Homebrew_build_broken/comment_2_d3f253a12a77fe4a354144880d0a4935._comment b/doc/bugs/Homebrew_build_broken/comment_2_d3f253a12a77fe4a354144880d0a4935._comment deleted file mode 100644 index 677a98af02..0000000000 --- a/doc/bugs/Homebrew_build_broken/comment_2_d3f253a12a77fe4a354144880d0a4935._comment +++ /dev/null @@ -1,7 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2023-06-21T17:53:22Z" - content=""" -Looks like homebrew fixed the build on their side, closing this. -"""]] diff --git a/doc/bugs/Hybrid_encryption_can__39__t_generate_the_right_key_after_moving_files.mdwn b/doc/bugs/Hybrid_encryption_can__39__t_generate_the_right_key_after_moving_files.mdwn deleted file mode 100644 index e4b785a01a..0000000000 --- a/doc/bugs/Hybrid_encryption_can__39__t_generate_the_right_key_after_moving_files.mdwn +++ /dev/null @@ -1,118 +0,0 @@ -### Please describe the problem. - -I had to migrate a special remote using hybrid encryption from an Amazon Cloud Drive-hosted rclone remote to a Google Drive-hosted rclone remote. Now I am testing it by trying to fsck a particular file. - -I've left all of the special remote settings on the git annex side the same, and just changed my `.rclone.conf` to have "amazondrive", the rclone remote name, actually point to the new Google Drive location. - -When I try to fsck my file, git annex generates an HMAC key, successfully downloads it from the remote, but then fails to decrypt it. After that, it looks like it retries: it generates a **different HMAC key**, which it can't find the file for. - -I've compared the files in Amazon and in Google Drive and they are the same. - -Using the [independent decryption script](https://git-annex.branchable.com/tips/Decrypting_files_in_special_remotes_without_git-annex/), modified to use the SHA512 MAC that my repo uses, and to not use every matching line from `remote.log`, I can generate the *second* HMAC key that git annex appears to fall back to, but not the first one. I think that this means I can't generate the decryption key either; I've had no luck manually decrypting the file with the characters after the first 256 of the decrypted shared key as the GPG passphrase. - -How is Git Annex generating that first HMAC, if not by the process used in the decryption script? Why is it able to get an HMAC that exists but not a valid decryption key? What is the thinking behind trying multiple HMAC keys for the same file? - -### What steps will reproduce the problem? - -Have an encrypted, hybrid Git Annex repo in an Amazon Drive rclone remote. -Copy all the files over to Google Drive, and make a Google Drive rclone remote in rclone.conf with the same name that the Amazon Drive one had. -Try to retrieve files from the remote with Git Annex. - -### What version of git-annex are you using? On what operating system? - -`6.20170525+gitge1cf095ae-1~ndall+1` on Ubuntu 16.04 - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - -$ git annex fsck info.txt --from=amazon --debug -[2017-06-03 11:24:31.368563623] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"] -[2017-06-03 11:24:31.373358653] process done ExitSuccess -[2017-06-03 11:24:31.373520685] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"] -[2017-06-03 11:24:31.379319755] process done ExitSuccess -[2017-06-03 11:24:31.382954929] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"] -[2017-06-03 11:24:31.386196477] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -[2017-06-03 11:24:31.392852081] read: git ["config","--null","--list"] -[2017-06-03 11:24:31.397686571] process done ExitSuccess -[2017-06-03 11:24:31.400187312] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--cached","-z","--","info.txt"] -[2017-06-03 11:24:31.424951599] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","check-attr","-z","--stdin","annex.backend","annex.numcopies","annex.largefiles","--"] -[2017-06-03 11:24:31.425544622] read: git ["--version"] -[2017-06-03 11:24:31.428009953] process done ExitSuccess -fsck info.txt [2017-06-03 11:24:31.479360829] chat: gpg ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] -[2017-06-03 11:24:31.503040759] process done ExitSuccess -[2017-06-03 11:24:31.506638987] chat: /home/anovak/bin/git-annex-remote-rclone [] -[2017-06-03 11:24:31.510094668] git-annex-remote-rclone[1] --> VERSION 1 -[2017-06-03 11:24:31.510281634] git-annex-remote-rclone[1] <-- PREPARE -[2017-06-03 11:24:31.511006098] git-annex-remote-rclone[1] --> GETCONFIG prefix -[2017-06-03 11:24:31.511148645] git-annex-remote-rclone[1] <-- VALUE data/annex-rclone -[2017-06-03 11:24:31.515487261] git-annex-remote-rclone[1] --> GETCONFIG target -[2017-06-03 11:24:31.515770189] git-annex-remote-rclone[1] <-- VALUE amazondrive -[2017-06-03 11:24:31.519503001] git-annex-remote-rclone[1] --> GETCONFIG rclone_layout -[2017-06-03 11:24:31.519666709] git-annex-remote-rclone[1] <-- VALUE lower -[2017-06-03 11:24:31.523726589] git-annex-remote-rclone[1] --> PREPARE-SUCCESS -[2017-06-03 11:24:31.523902589] git-annex-remote-rclone[1] <-- CHECKPRESENT GPGHMACSHA512--4354571263cd241ac8300e0bd8fc3643c184b922500c3d9318b7ce590684e1de820d41c0b245f1595e0bd38b2844a5151192981b4f9ab41bda135d5f7a184d16 -[2017-06-03 11:24:31.530946791] git-annex-remote-rclone[1] --> DIRHASH-LOWER GPGHMACSHA512--4354571263cd241ac8300e0bd8fc3643c184b922500c3d9318b7ce590684e1de820d41c0b245f1595e0bd38b2844a5151192981b4f9ab41bda135d5f7a184d16 -[2017-06-03 11:24:31.531236912] git-annex-remote-rclone[1] <-- VALUE b95/785/ -Total objects: 1 Total size: 119 Bytes (119 Bytes) 2017/06/03 11:24:42 Transferred: 0 Bytes (0 Bytes/s) Errors: 0 Checks: 0 Transferred: 0 Elapsed time: 10.4s -[2017-06-03 11:24:42.04576211] git-annex-remote-rclone[1] --> CHECKPRESENT-SUCCESS GPGHMACSHA512--4354571263cd241ac8300e0bd8fc3643c184b922500c3d9318b7ce590684e1de820d41c0b245f1595e0bd38b2844a5151192981b4f9ab41bda135d5f7a184d16 - -[2017-06-03 11:24:42.049286934] git-annex-remote-rclone[1] <-- TRANSFER RETRIEVE GPGHMACSHA512--4354571263cd241ac8300e0bd8fc3643c184b922500c3d9318b7ce590684e1de820d41c0b245f1595e0bd38b2844a5151192981b4f9ab41bda135d5f7a184d16 .git/annex/tmp/GPGHMACSHA512--4354571263cd241ac8300e0bd8fc3643c184b922500c3d9318b7ce590684e1de820d41c0b245f1595e0bd38b2844a5151192981b4f9ab41bda135d5f7a184d16 -[2017-06-03 11:24:42.050298028] git-annex-remote-rclone[1] --> DIRHASH-LOWER GPGHMACSHA512--4354571263cd241ac8300e0bd8fc3643c184b922500c3d9318b7ce590684e1de820d41c0b245f1595e0bd38b2844a5151192981b4f9ab41bda135d5f7a184d16 -[2017-06-03 11:24:42.05045573] git-annex-remote-rclone[1] <-- VALUE b95/785/ -2017/06/03 11:24:52 Local file system at /tmp/tmp.ITXaCoES4i: Waiting for checks to finish -2017/06/03 11:24:52 Local file system at /tmp/tmp.ITXaCoES4i: Waiting for transfers to finish -2017/06/03 11:24:54 -Transferred: 119 Bytes (9 Bytes/s) -Errors: 0 -Checks: 0 -Transferred: 1 -Elapsed time: 12s -[2017-06-03 11:24:54.083003717] git-annex-remote-rclone[1] --> TRANSFER-SUCCESS RETRIEVE GPGHMACSHA512--4354571263cd241ac8300e0bd8fc3643c184b922500c3d9318b7ce590684e1de820d41c0b245f1595e0bd38b2844a5151192981b4f9ab41bda135d5f7a184d16 -[2017-06-03 11:24:54.083554696] chat: gpg ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--batch","--passphrase-fd","22","--decrypt"] -gpg: decryption failed: bad key -[2017-06-03 11:24:54.086768586] process done ExitFailure 2 -[2017-06-03 11:24:54.088931923] git-annex-remote-rclone[1] <-- TRANSFER RETRIEVE GPGHMACSHA512--7439fc103f30635cd914af0ce14370aef2f9a81f415ba7a652bc20454c90893bcd5701ebbed855f6a0285fce413e5c039c15a7049738cf519e2719c9d485ce28 .git/annex/tmp/GPGHMACSHA512--7439fc103f30635cd914af0ce14370aef2f9a81f415ba7a652bc20454c90893bcd5701ebbed855f6a0285fce413e5c039c15a7049738cf519e2719c9d485ce28 -[2017-06-03 11:24:54.089806756] git-annex-remote-rclone[1] --> DIRHASH-LOWER GPGHMACSHA512--7439fc103f30635cd914af0ce14370aef2f9a81f415ba7a652bc20454c90893bcd5701ebbed855f6a0285fce413e5c039c15a7049738cf519e2719c9d485ce28 -[2017-06-03 11:24:54.090050153] git-annex-remote-rclone[1] <-- VALUE b47/e54/ -2017/06/03 11:25:03 Local file system at /tmp/tmp.5lapDbIoZy: Waiting for checks to finish -2017/06/03 11:25:03 Local file system at /tmp/tmp.5lapDbIoZy: Waiting for transfers to finish -2017/06/03 11:25:03 Attempt 1/3 failed with 0 errors and: directory not found -2017/06/03 11:25:03 Local file system at /tmp/tmp.5lapDbIoZy: Waiting for checks to finish -2017/06/03 11:25:03 Local file system at /tmp/tmp.5lapDbIoZy: Waiting for transfers to finish -2017/06/03 11:25:03 Attempt 2/3 failed with 0 errors and: directory not found -2017/06/03 11:25:04 Local file system at /tmp/tmp.5lapDbIoZy: Waiting for checks to finish -2017/06/03 11:25:04 Local file system at /tmp/tmp.5lapDbIoZy: Waiting for transfers to finish -2017/06/03 11:25:04 Attempt 3/3 failed with 0 errors and: directory not found -2017/06/03 11:25:04 Failed to copy: directory not found -[2017-06-03 11:25:04.479940212] git-annex-remote-rclone[1] --> TRANSFER-FAILURE RETRIEVE GPGHMACSHA512--7439fc103f30635cd914af0ce14370aef2f9a81f415ba7a652bc20454c90893bcd5701ebbed855f6a0285fce413e5c039c15a7049738cf519e2719c9d485ce28 - - failed to download file from remote -failed -[2017-06-03 11:25:04.492702185] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","hash-object","-w","--stdin-paths","--no-filters"] -[2017-06-03 11:25:04.494054942] feed: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","update-index","-z","--index-info"] -[2017-06-03 11:25:04.672344301] process done ExitSuccess -[2017-06-03 11:25:04.67255166] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"] -[2017-06-03 11:25:04.68070068] process done ExitSuccess -(recording state in git...) -[2017-06-03 11:25:04.680983817] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","write-tree"] -[2017-06-03 11:25:04.900456464] process done ExitSuccess -[2017-06-03 11:25:04.900643429] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","commit-tree","a8fc26b91dab1ffac0499d3e21c0f691858d3a6a","--no-gpg-sign","-p","refs/heads/git-annex"] -[2017-06-03 11:25:04.965122443] process done ExitSuccess -[2017-06-03 11:25:04.965288386] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","update-ref","refs/heads/git-annex","c0b297184826849d8a24c899faa9f4f47c1f8e53"] -[2017-06-03 11:25:04.970235879] process done ExitSuccess -[2017-06-03 11:25:04.971493023] process done ExitSuccess -git-annex: fsck: 1 failed - - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Yeah, it works great most of the time, and it was working great on the amazon remote before I tried to pull these migration shenanigans. - -> Closing this since there has been no response for information for 5 -> years. [[done]] --[[Joey]] diff --git a/doc/bugs/Hybrid_encryption_can__39__t_generate_the_right_key_after_moving_files/comment_1_bbc1e7205d7701afd405c6e62a1c0aa3._comment b/doc/bugs/Hybrid_encryption_can__39__t_generate_the_right_key_after_moving_files/comment_1_bbc1e7205d7701afd405c6e62a1c0aa3._comment deleted file mode 100644 index 7d03495191..0000000000 --- a/doc/bugs/Hybrid_encryption_can__39__t_generate_the_right_key_after_moving_files/comment_1_bbc1e7205d7701afd405c6e62a1c0aa3._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="yibe" - avatar="http://cdn.libravatar.org/avatar/649c9e417b6c509a17f7b48caf065c91" - subject="comment 1" - date="2017-06-04T20:07:31Z" - content=""" -Hello, - -I have no idea why git-annex fails to decrypt your file, but as for the two different HMAC keys, I guess you have chunking enabled on that remote (at least when you uploaded the file) and that first HMAC key is the right key for the 1st chunk of your file. That decryption script does not take chunks into account, so you were only able to generate the second HMAC key, which should be the right key if the file was uploaded without chunking enabled. I've just [posted][1] a modified version of the script that supports chunks. - -[1]: http://git-annex.branchable.com/tips/Decrypting_files_in_special_remotes_without_git-annex/#comment-ea2df7b4739f3d66c169bf297e339e9d - -"""]] diff --git a/doc/bugs/Hybrid_encryption_can__39__t_generate_the_right_key_after_moving_files/comment_2_52a1800537f1244ad6cf417c5f25ebe0._comment b/doc/bugs/Hybrid_encryption_can__39__t_generate_the_right_key_after_moving_files/comment_2_52a1800537f1244ad6cf417c5f25ebe0._comment deleted file mode 100644 index 07fa485449..0000000000 --- a/doc/bugs/Hybrid_encryption_can__39__t_generate_the_right_key_after_moving_files/comment_2_52a1800537f1244ad6cf417c5f25ebe0._comment +++ /dev/null @@ -1,29 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2017-06-06T15:38:24Z" - content=""" -Your remote has chunking enabled, so git-annex first tries generating a -HMAC for a chunked key. When decrypting the content fails for some reason, -it falls back to trying the HMAC for an unchunked key. This is done because -chunking can be enabled after some content has been uploaded to a remote, -so it always tries the unchunked location just in case. - -It looks like gpg is successfully decrypting the hybrid encryption key -that's embedded in your git repository. That is the first, successful -call tp gpg in your log. - -The "bad key" error then comes when gpg is asked to use the hybrid -encryption key to decrypt the content. This seems to indicate it's not -using the same hybrid encryption key that was used to encrypt it. - -The fact that it was able to generate the right HMAC key to download the -content though, indicates that it did get the right hybrid encryption key -(since half of that key is used to generate the HMAC). - -So hmm, I don't understand what is going on. - -Are you able to retrieve the same file successfully when the rclone.conf -is configured to use the Amazon Cloud Drive? (Assuming that the content of -the file is still present over there.) -"""]] diff --git a/doc/bugs/Hybrid_encryption_can__39__t_generate_the_right_key_after_moving_files/comment_3_414bd487619e46b25e8d9a57d9aef1f6._comment b/doc/bugs/Hybrid_encryption_can__39__t_generate_the_right_key_after_moving_files/comment_3_414bd487619e46b25e8d9a57d9aef1f6._comment deleted file mode 100644 index bc136ec452..0000000000 --- a/doc/bugs/Hybrid_encryption_can__39__t_generate_the_right_key_after_moving_files/comment_3_414bd487619e46b25e8d9a57d9aef1f6._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="interfect@b151490178830f44348aa57b77ad58c7d18e8fe7" - nickname="interfect" - avatar="http://cdn.libravatar.org/avatar/3ba541a3204f4ea4b274049a152489c5" - subject="comment 3" - date="2017-06-09T01:40:00Z" - content=""" -Unfortunately even though the content is still over at Amazon, ACD can no longer be accessed through rclone, so I can't get Git Annex to go over there and download it. - -With the new chunk-supporting decryption script (which I further modified to actually use the timestamps on the log entries), I am able to generate the right key for my test file. I was also able to generate the key for and decrypt another test file, and then testing with git annex again shows that I can successfully fsck other file in the remote. - -I think what is happening is that my small file I was testing with somehow became corrupted or was modified while on Amazon's servers. I downloaded from Amazon before the transfer and from Google after and did a diff, and both files are identical, so I think I moved over corrupt data. - -It looks like git annex is just successfully doing its job and identifying some data corruption here. The bug can probably be closed. - - -"""]] diff --git a/doc/bugs/Hybrid_encryption_can__39__t_generate_the_right_key_after_moving_files/comment_4_81abd2627672911ba6367effbf5c487b._comment b/doc/bugs/Hybrid_encryption_can__39__t_generate_the_right_key_after_moving_files/comment_4_81abd2627672911ba6367effbf5c487b._comment deleted file mode 100644 index 66c23ee4ff..0000000000 --- a/doc/bugs/Hybrid_encryption_can__39__t_generate_the_right_key_after_moving_files/comment_4_81abd2627672911ba6367effbf5c487b._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="interfect@b151490178830f44348aa57b77ad58c7d18e8fe7" - nickname="interfect" - avatar="http://cdn.libravatar.org/avatar/3ba541a3204f4ea4b274049a152489c5" - subject="comment 4" - date="2017-06-09T01:48:15Z" - content=""" -I'm actually seeing a lot of files (mostly smaller ones, for some reason) failing to fsck because GPG decryption failed. I can't tell at the moment whether they got corrupted in the transfer, or corrupted in the initial upload somehow. I'm pretty sure the problem here isn't with git annex itself, or more people would have noticed, but I'm definitely going to be fscking my cloud remotes more frequently. -"""]] diff --git a/doc/bugs/Hybrid_encryption_can__39__t_generate_the_right_key_after_moving_files/comment_5_ce0342e38a87017ad58c9a79b17d759a._comment b/doc/bugs/Hybrid_encryption_can__39__t_generate_the_right_key_after_moving_files/comment_5_ce0342e38a87017ad58c9a79b17d759a._comment deleted file mode 100644 index d20a6e6a13..0000000000 --- a/doc/bugs/Hybrid_encryption_can__39__t_generate_the_right_key_after_moving_files/comment_5_ce0342e38a87017ad58c9a79b17d759a._comment +++ /dev/null @@ -1,26 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2017-06-09T17:04:09Z" - content=""" -"I think what is happening is that my small file I was testing with somehow -became corrupted or was modified while on Amazon's servers." - -That was also kind of my guess. It's hard to imagine how -the way that a file is downloaded from The Cloud changes -how git-annex decrypts it. As long as the content is the same, -the decrpytion step should behave identially no matter where -the file is downloaded from. - -But, multiple small files getting corrupted seems like it must -have a cause other than a bit flip. Perhaps something about how -they were transferred between the two clouds corrupted them.. - -I suppose there could also be a bug in git-annex or rclone that somehow -corrupts uploads of small files. Perhaps something to do with chunking.. -What does `git annex info theremote --fast` say about its configuration? - -What is the range of sizes of small files that you've found to be -corrupted? Is there a cut-off point after which all larger files are -not corrupted? Are any small files not corrupted? -"""]] diff --git a/doc/bugs/Incompatibility_with_git_2.37.mdwn b/doc/bugs/Incompatibility_with_git_2.37.mdwn deleted file mode 100644 index 3544c30c37..0000000000 --- a/doc/bugs/Incompatibility_with_git_2.37.mdwn +++ /dev/null @@ -1,55 +0,0 @@ -### Please describe the problem. - -Starting with git 2.37, `git annex init` fails with a lock issue. - -### What steps will reproduce the problem? - -Run `git annex init` with git version > 2.36 (I tested 2.37.0 and 2.37.1) - -### What version of git-annex are you using? On what operating system? - -10.20220624-g17e4081d4 on Arch Linux - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - -arno@genie ⛁ /tmp/annex_test - ★ rm -rf .git - -arno@genie ⛁ /tmp/annex_test - ★ git init -Initialized empty Git repository in /tmp/annex_test/.git/ - -arno@genie ⛁ /tmp/annex_test - ★ git annex init test -init test fatal: Unable to create '/tmp/annex_test/.git/annex/index.lock': File exists. - -Another git process seems to be running in this repository, e.g. -an editor opened by 'git commit'. Please make sure all processes -are terminated then try again. If it still fails, a git process -may have crashed in this repository earlier: -remove the file manually to continue. - -git-annex: failed to read sha from git write-tree -CallStack (from HasCallStack): - error, called at ./Git/Sha.hs:23:15 in main:Git.Sha -failed -init: 1 failed - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Yes ! git-annex is my goto tool for file storage. - - -> [[done]]; this is a bug in git and will have to be fixed there. -> (git-annex could work around it by making sure the index file -> exists before calling write-tree, but I don't think it makes sense to -> work around a segfault bug like that, at least not unless it gets -> into some stable release of an OS...) -> --[[Joey]] diff --git a/doc/bugs/Incompatibility_with_git_2.37/comment_1_c24657bb9d3b3072698572c4e75ccb5d._comment b/doc/bugs/Incompatibility_with_git_2.37/comment_1_c24657bb9d3b3072698572c4e75ccb5d._comment deleted file mode 100644 index a18f60cbec..0000000000 --- a/doc/bugs/Incompatibility_with_git_2.37/comment_1_c24657bb9d3b3072698572c4e75ccb5d._comment +++ /dev/null @@ -1,44 +0,0 @@ -[[!comment format=mdwn - username="akspecs@fae4f3d58a0c6c9d50f01a850ce53d425e1a90ba" - nickname="akspecs" - avatar="http://cdn.libravatar.org/avatar/43d7148aaa9adf3a40c42e3388123326" - subject="cannot reproduce" - date="2022-07-19T12:53:45Z" - content=""" -``` -$ pacman -Q git-annex -git-annex 10.20220624-14 - -$ git annex version -git-annex version: 10.20220624-g17e4081d4 -... - -$ git --version -git version 2.37.1 - -$ mkdir annex_test -$ cd annex_test -$ git init -Initialized empty Git repository in /tmp/annex_test/.git/ - -$ git annex init test -init test ok -(recording state in git...) - -$ git annex info -trusted repositories: 0 -semitrusted repositories: 3 - 00000000-0000-0000-0000-000000000001 -- web - 00000000-0000-0000-0000-000000000002 -- bittorrent - 83b33af6-29d2-4568-aa96-f7ef94667784 -- test [here] -untrusted repositories: 0 -transfers in progress: none -available local disk space: 8.23 gigabytes (+1 megabyte reserved) -local annex keys: 0 -local annex size: 0 bytes -annexed files in working tree: 0 -size of annexed files in working tree: 0 bytes -bloom filter size: 32 mebibytes (0% full) -backend usage: -``` -"""]] diff --git a/doc/bugs/Incompatibility_with_git_2.37/comment_2_4d5eb77736e8512affc25e7bdd5190f1._comment b/doc/bugs/Incompatibility_with_git_2.37/comment_2_4d5eb77736e8512affc25e7bdd5190f1._comment deleted file mode 100644 index b03714d4a2..0000000000 --- a/doc/bugs/Incompatibility_with_git_2.37/comment_2_4d5eb77736e8512affc25e7bdd5190f1._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="arnaud@c1d1cc612a3921dc06a417301be08a3e125478c4" - nickname="arnaud" - avatar="http://cdn.libravatar.org/avatar/c0defbf54541c499f6c0464588fc2ad0" - subject="Happens only when core.untrackedCache=true" - date="2022-07-21T08:23:04Z" - content=""" -I did a little testing with potential triggers in my ~/.gitconfig and traced it down to core.untrackedCache=true - -With core.untrackedCache=true and git 2.36, everything works as expected. With git 2.37, I get the bug. -"""]] diff --git a/doc/bugs/Incompatibility_with_git_2.37/comment_3_889ece8b2168b20019253ab6dd5b0e48._comment b/doc/bugs/Incompatibility_with_git_2.37/comment_3_889ece8b2168b20019253ab6dd5b0e48._comment deleted file mode 100644 index 4bd9542003..0000000000 --- a/doc/bugs/Incompatibility_with_git_2.37/comment_3_889ece8b2168b20019253ab6dd5b0e48._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2022-07-22T15:46:46Z" - content=""" -`git write-tree` has always locked the index, so it seems something else -has changed to also lock the index when core.untrackedCache is set, that -did not before. - -Probably `git update-index`, since that's what looks at the config. -`git-annex init` does run that, before `git write-tree`, but it -waits for it to exit. So I don't understand what would have the index still -locked. - -I've reproduced it, setting core.untrackedCache. -"""]] diff --git a/doc/bugs/Incompatibility_with_git_2.37/comment_4_95b8ea35cd10ac80e1d01b658e24d7a9._comment b/doc/bugs/Incompatibility_with_git_2.37/comment_4_95b8ea35cd10ac80e1d01b658e24d7a9._comment deleted file mode 100644 index b1c285f2ec..0000000000 --- a/doc/bugs/Incompatibility_with_git_2.37/comment_4_95b8ea35cd10ac80e1d01b658e24d7a9._comment +++ /dev/null @@ -1,30 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2022-07-22T16:25:27Z" - content=""" -Seems that git write-tree is segfaulting, which leaves the index locked: - - openat(AT_FDCWD, "/tmp/n/.git/annex/index.lock", O_RDWR|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3 - rt_sigaction(SIGINT, {sa_handler=0x55f953819fb0, sa_mask=[INT], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7fba9b3d2940}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0 - rt_sigaction(SIGHUP, {sa_handler=0x55f953819fb0, sa_mask=[HUP], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7fba9b3d2940}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0 - rt_sigaction(SIGTERM, {sa_handler=0x55f953819fb0, sa_mask=[TERM], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7fba9b3d2940}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0 - rt_sigaction(SIGQUIT, {sa_handler=0x55f953819fb0, sa_mask=[QUIT], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7fba9b3d2940}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0 - rt_sigaction(SIGPIPE, {sa_handler=0x55f953819fb0, sa_mask=[PIPE], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7fba9b3d2940}, {sa_handler=SIG_DFL, sa_mask=[PIPE], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7fba9b3d2940}, 8) = 0 - getpid() = 2873826 - openat(AT_FDCWD, "/tmp/n/.git/annex/index", O_RDONLY) = -1 ENOENT (No such file or directory) - --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0xd0} --- - +++ killed by SIGSEGV +++ - -And this is not due to anything git-annex is doing, git is 100% broken: - - joey@darkstar:/tmp>git init emptyrepo - warning: templates not found in /home/joey/share/git-core/templates - Initialized empty Git repository in /tmp/emptyrepo/.git/ - joey@darkstar:/tmp>cd emptyrepo/ - joey@darkstar:/tmp/emptyrepo>git config core.untrackedCache true - joey@darkstar:/tmp/emptyrepo>git write-tree - Segmentation fault - -git 2.37.1 has the same problem. I have emailed the git list about it. -"""]] diff --git a/doc/bugs/Incompatibility_with_git_2.37/comment_5_d0e60d224dd48d7c31adb6b250f662df._comment b/doc/bugs/Incompatibility_with_git_2.37/comment_5_d0e60d224dd48d7c31adb6b250f662df._comment deleted file mode 100644 index 59a4fb3eca..0000000000 --- a/doc/bugs/Incompatibility_with_git_2.37/comment_5_d0e60d224dd48d7c31adb6b250f662df._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="arnaud@c1d1cc612a3921dc06a417301be08a3e125478c4" - nickname="arnaud" - avatar="http://cdn.libravatar.org/avatar/c0defbf54541c499f6c0464588fc2ad0" - subject="comment 5" - date="2022-07-23T08:48:17Z" - content=""" -Awesome, thanks ! -"""]] diff --git a/doc/bugs/Incompatibility_with_git_2.37/comment_6_3bb6424838d4f8a971f3dce6d1b8dbfb._comment b/doc/bugs/Incompatibility_with_git_2.37/comment_6_3bb6424838d4f8a971f3dce6d1b8dbfb._comment deleted file mode 100644 index 787d17a4ce..0000000000 --- a/doc/bugs/Incompatibility_with_git_2.37/comment_6_3bb6424838d4f8a971f3dce6d1b8dbfb._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 6""" - date="2022-08-04T17:46:32Z" - content=""" -Now 2 weeks since the segfault in git has been known, and they have not -fixed it. - -Now feeling that it's might be worth git-annex init avoiding the -segfault, by making sure to do something that creates the index file before -write-tree. So I don't have to worry about when this gets fixed in git, -or the chances that a buggy release will start being widely used for a long -time. - -Yeah, done so. -"""]] diff --git a/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates.mdwn b/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates.mdwn deleted file mode 100644 index e5411b3d9d..0000000000 --- a/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates.mdwn +++ /dev/null @@ -1,54 +0,0 @@ -### Please describe the problem. - - - -With the latest v10 git, I'm seeing some things like this: - -``` -get Pictures/0_New/5/20-03-31 21-12-12 0184.mp4 (from source...) file content has changed - - - Unable to access these remotes: source - - No other repository is known to contain the file. -failed -``` - -Now, the file content really had not chnaged. That file looks like this: - -``` -$ cat 20-03-31\ 21-12-12\ 0184.mp4 -/annex/objects/SHA256E-s55936559--0d418e4bb62cfc7ef5b053f8b622dd72794781a49931abc41bb9499acaf51b09.mp4 -``` - -And on-disk, the file is: - -``` -$ sha256sum 20-03-31\ 21-12-12\ 0184.mp4 -0d418e4bb62cfc7ef5b053f8b622dd72794781a49931abc41bb9499acaf51b09 20-03-31 21-12-12 0184.mp4 -``` - -The result here is that files are left around not checked out. - -This is in an unlocked, thin repository and it is pulling in data from a directory special remote with importtree=yes. - -The problem seems to be triggered by a file that has a duplicate somewhere in the source repository (with the same sha256sum). I was able to find another file with the same sha256sum as that one in the same directory. In the origin repo, in which the given files were not present and it's using unlock-present, all instances of the file contained a broken symlink to the appropriate target. In the repo in question here, which uses plain unlock and is on NTFS (though Linux is the OS accessing it), I'm having this problem. - - -### What steps will reproduce the problem? - -Any command that causes it to load the file content from the special remote (eg, git annex import, or git annex get) - -### What version of git-annex are you using? On what operating system? - -10.20220823 on Debian bullseye - -### Please provide any additional information below. - -Original conversation was at [[/bugs/Files_recorded_with_other_file__39__s_checksums/]]. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Yes! I used it for awhile with the assistant to sync a lot of files. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_1_b9491e9cb432ca3b8bdb08a00c89b346._comment b/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_1_b9491e9cb432ca3b8bdb08a00c89b346._comment deleted file mode 100644 index f02fd2cd6c..0000000000 --- a/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_1_b9491e9cb432ca3b8bdb08a00c89b346._comment +++ /dev/null @@ -1,37 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-09-16T17:10:24Z" - content=""" -"file content has changed" is a message from the directory special remote -when it sees that a file in the directory has a different mtime,size,inode -tuple (content identifier) than the one that was recorded when importing -the tree. - -Maybe that has something to do with the duplicate files, since they would -have different inodes. Except, it does try to support that; it can keep -track of multiple content identifiers for a key, and when there are -duplicate files, that works in my testing. - -I notice that, if I have a directory special remote that was initially -set up with ignoreinodes=no, and I've imported a tree that way from it, -I can cause what looks like this problem: - - joey@darkstar:~/tmp/bench/x>git-annex enableremote t directory=../x ignoreinodes=yes - enableremote t ok - (recording state in git...) - joey@darkstar:~/tmp/bench/x>git-annex get 3 --from t - get 3 (from t...) - file content has changed - -Which seems like a bug that it allows changing it, but I think that the idea was -that, after changing ignoreinores, the user would re-run git-annex import, -which would re-import all the files since the content identifier has -changed. Still, I think that -[[!commit 3e2f1f73cbc5fc10475745b3c3133267bd1850a7]] didn't consider that -it could cause a get to fail. - -I know you have recently upgraded from a version that defaulted -to ignoring inodes. Maybe you only need to re-run `git-annex import` from the -directory remote to fix the problem? -"""]] diff --git a/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_2_cf9ce09377da2eae63ff41d07cb222d7._comment b/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_2_cf9ce09377da2eae63ff41d07cb222d7._comment deleted file mode 100644 index 03c7024b25..0000000000 --- a/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_2_cf9ce09377da2eae63ff41d07cb222d7._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-09-16T17:32:29Z" - content=""" -If I was on the wrong track with my first comment, it would be helpful -to take a look at the content identifier that is recorded for -the key, by running: - - git show git-annex:$(git-annex examinekey SHA256E-s55936559--0d418e4bb62cfc7ef5b053f8b622dd72794781a49931abc41bb9499acaf51b09.mp4 --format='${hashdirlower}${key}.log.cid') - -The part after the UUID in the output is the content identifier(s). If more -than one, they are separated by colons. The format of a content identifier -is "inode size mtime", and the inode will be 0 if it was generated with -ignoreinodes=yes. So you could then compare the content identifiers -with the stat of the files in the directory special remote and see if -something has actually changed. -"""]] diff --git a/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_3_a99df351ed700f908b6aaf3f52d245b4._comment b/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_3_a99df351ed700f908b6aaf3f52d245b4._comment deleted file mode 100644 index ed50e89846..0000000000 --- a/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_3_a99df351ed700f908b6aaf3f52d245b4._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2022-09-16T17:43:38Z" - content=""" -I've made the directory special remote treat content identifiers -that differ only in one having the inode set to 0. Which will avoid it -failing in the situation I showed. -"""]] diff --git a/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_4_d370abe891d554577a9af7f959c9e3f0._comment b/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_4_d370abe891d554577a9af7f959c9e3f0._comment deleted file mode 100644 index 598a1ac09f..0000000000 --- a/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_4_d370abe891d554577a9af7f959c9e3f0._comment +++ /dev/null @@ -1,47 +0,0 @@ -[[!comment format=mdwn - username="jgoerzen" - avatar="http://cdn.libravatar.org/avatar/090740822c9dcdb39ffe506b890981b4" - subject="comment 4" - date="2022-09-19T13:55:07Z" - content=""" -Examining that `git show` command: - -``` -git show git-annex:$(git-annex examinekey SHA256E-s55936559--0d418e4bb62cfc7ef5b053f8b622dd72794781a49931abc41bb9499acaf51b09.mp4 --format='${hashdirlower}${key}.log.cid') -1662351509.054973915s 58255588-b988-47ba-9a88-7cb1118503ba 164140 55936559 1585707132 :167232 55936559 1585707132 -``` - -From the source material: - -``` -$ stat '0_New/5/20-03-31 21-12-12 0184.mp4' - File: 0_New/5/20-03-31 21-12-12 0184.mp4 - Size: 55936559 Blocks: 109238 IO Block: 131072 regular file -Device: 49h/73d Inode: 167232 Links: 1 -Access: (0644/-rw-r--r--) Uid: ( 1000/jgoerzen) Gid: ( 1000/jgoerzen) -Access: 2020-03-31 21:12:12.000000000 -0500 -Modify: 2020-03-31 21:12:12.000000000 -0500 -Change: 2020-06-07 23:07:21.475946002 -0500 - Birth: - -``` - -So it looks like the inode number changed. But, on a hunch, I thought: this only happens with duplicate files; what if there's a file with that sha256 and that inode number? - -And sure enough: - -``` -stat ./0_New/5/laura-john-20200331.mp4 - File: ./0_New/5/laura-john-20200331.mp4 - Size: 55936559 Blocks: 109238 IO Block: 131072 regular file -Device: 49h/73d Inode: 164140 Links: 1 -Access: (0644/-rw-r--r--) Uid: ( 1000/jgoerzen) Gid: ( 1000/jgoerzen) -Access: 2020-03-31 21:12:12.000000000 -0500 -Modify: 2020-03-31 21:12:12.000000000 -0500 -Change: 2020-03-31 21:21:50.131297383 -0500 - Birth: - -``` - -I didn't set ignoreinodes on this remote. - -Is that helpful? -"""]] diff --git a/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_5_68a3936700a02a17d0379d96880eea15._comment b/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_5_68a3936700a02a17d0379d96880eea15._comment deleted file mode 100644 index a45c9876b9..0000000000 --- a/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_5_68a3936700a02a17d0379d96880eea15._comment +++ /dev/null @@ -1,31 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2022-09-20T16:23:07Z" - content=""" -The two files have the same two inodes that are recorded in the git-annex -branch. The size and mtime also seem to match. Interesting, -it must be another problem than the one I fixed. - -I wonder if it's expecting the first file to have the second one's inode, -or vice-versa? - -I don't know how that would have happened to you, but I was able to force -it to happen, by importing a tree from a directory remote that had 2 -duplicate files. Then I swapped the names of the 2 files in the directory. -This causes git-annex get to fail with the same error message. - -That must be a bug itself, because re-importing from the special remote -generates the same tree (of course) and so get continues failing, there is -no recovery possible except deleting or touching the files. - -Aha: Another indication that handling of duplicate imported files is broken -is to import two files, and then delete the first of them. This also causes -git-annex get to fail, but now it always fails with "no such file or -directory". It is only trying to get the first file, it does not try the -second. Re-importing from the special remote teaches it about the deleted -file, and that leads back to the "file content has changed" problem because -it seems to be looking for the deleted file's inode. So that's one way you -could have gotten into this situation without the unlikely swapping of two -duplicate files. -"""]] diff --git a/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_6_4d05a896fd57b55388fb21638ed02233._comment b/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_6_4d05a896fd57b55388fb21638ed02233._comment deleted file mode 100644 index 848f170302..0000000000 --- a/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_6_4d05a896fd57b55388fb21638ed02233._comment +++ /dev/null @@ -1,24 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 6""" - date="2022-09-20T16:42:18Z" - content=""" -Ah, here's the smoking gun: In Remote.Helper.ExportImport, -it gets the cids for a key. And promptly picks the first one to pass to -retrieveExportWithContentIdentifier, ignoring all the rest. -It also gets the first recorded export location and likewise passes it to -retrieveExportWithContentIdentifier. - -It seems like a fix would be to try retrieveExportWithContentIdentifier -with each combination of cid and export location. But that would cause an -`O(N^2)` explosion and it's possible a remote has say 1000 empty files in -it. - -Maybe instead make retrieveExportWithContentIdentifier take a list of valid -cids, and accept any one of them? Then it would only need to be tried on -each export location in turn until one succeeds. - -(Hm, removeExportWithContentIdentifier and -checkPresentExportWithContentIdentifier already use a list, so similar problems -are avoided with them.) -"""]] diff --git a/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_7_4a1248ea257c04f1afe233891a606eea._comment b/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_7_4a1248ea257c04f1afe233891a606eea._comment deleted file mode 100644 index b023e3c6f6..0000000000 --- a/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_7_4a1248ea257c04f1afe233891a606eea._comment +++ /dev/null @@ -1,7 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 7""" - date="2022-09-20T17:35:10Z" - content=""" -Ok, fixed that. I'm confident this will fix the issue for you, so closing. -"""]] diff --git a/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_8_37eb9d35191bcfc0e7e4496f5720cbed._comment b/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_8_37eb9d35191bcfc0e7e4496f5720cbed._comment deleted file mode 100644 index e80c02e061..0000000000 --- a/doc/bugs/Incorrect___34__file_content_has_changed__34___on_duplicates/comment_8_37eb9d35191bcfc0e7e4496f5720cbed._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="jgoerzen" - avatar="http://cdn.libravatar.org/avatar/090740822c9dcdb39ffe506b890981b4" - subject="comment 8" - date="2022-09-22T03:40:04Z" - content=""" -Thank you! -"""]] diff --git a/doc/bugs/Invalid_argument_saving_FreeSurfer_file_on_NTFS.mdwn b/doc/bugs/Invalid_argument_saving_FreeSurfer_file_on_NTFS.mdwn deleted file mode 100644 index 241ee8ce8c..0000000000 --- a/doc/bugs/Invalid_argument_saving_FreeSurfer_file_on_NTFS.mdwn +++ /dev/null @@ -1,95 +0,0 @@ -### Please describe the problem. - -Annexing files can fail with "Invalid argument" error. It appears to be an interaction of filesystem and file name. The file name is `lh.aparc.DKTatlas.annot`/`rh.aparc.DKTatlas.annot`, and the filesystem is: - -[[!format sh """ -$ mount | grep data -/dev/mapper/bitlocker on /data type fuseblk (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096) -$ ntfs-3g -ntfs-3g: No device is specified. - -ntfs-3g 2021.8.22 integrated FUSE 28 - Third Generation NTFS Driver - Configuration type 7, XATTRS are on, POSIX ACLS are on -"""]] - -However, formatting a file as NTFS and mounting as loop to make a minimal reproduction allowed the operation to succeed. - -### What steps will reproduce the problem? - -I am able to reproduce this problem reliably on the filesystem above: - -[[!format sh """ -$ git init test/ -$ cd test -$ git annex init -$ curl -sSL https://file.io/ZtE5ivep7E9V -o lh.aparc.DKTatlas.annot -$ git annex add lh.aparc.DKTatlas.annot -add lh.aparc.DKTatlas.annot - .git/annex/othertmp/: openTempFile template ingest-lh.: invalid argument (Invalid argument) - -failed -add: 1 failed -"""]] - -However, renaming the file to `L.annot`, annexing it, and `git mv`ing it back works, so I am not stuck adding the file contents directly to git. - -I hope the log below will give you a hint of what's going on, as I regret I am unable to reproduce this on other filesystems. - -### What version of git-annex are you using? On what operating system? - -OS: Ubuntu 22.04.3 LTS - -[[!format sh """ -git-annex version: 10.20230408-g5b1e8ba77 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22.1 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.13.1 persistent-sqlite-2.13.1.0 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2.1 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 10 -"""]] - -### Please provide any additional information below. -[[!format sh """ -$ git annex add -d lh.aparc.DKTatlas.annot -[2023-12-05 09:40:20.256526552] (Utility.Process) process [1162202] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","git-annex"] -[2023-12-05 09:40:20.260613884] (Utility.Process) process [1162202] done ExitSuccess -[2023-12-05 09:40:20.260892906] (Utility.Process) process [1162203] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"] -[2023-12-05 09:40:20.265019085] (Utility.Process) process [1162203] done ExitSuccess -[2023-12-05 09:40:20.265446352] (Utility.Process) process [1162204] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/git-annex..eb539b816feb00e6be3a252d36efaa7426eccca1","--pretty=%H","-n1"] -[2023-12-05 09:40:20.270301207] (Utility.Process) process [1162204] done ExitSuccess -[2023-12-05 09:40:20.271251091] (Utility.Process) process [1162205] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"] -[2023-12-05 09:40:20.275373234] (Utility.Process) process [1162206] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","symbolic-ref","-q","HEAD"] -[2023-12-05 09:40:20.278672692] (Utility.Process) process [1162206] done ExitSuccess -[2023-12-05 09:40:20.278946202] (Utility.Process) process [1162207] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","refs/heads/master"] -[2023-12-05 09:40:20.282475944] (Utility.Process) process [1162207] done ExitFailure 1 -[2023-12-05 09:40:20.282779905] (Utility.Process) process [1162208] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","ls-files","-z","--others","--exclude-standard","--","lh.aparc.DKTatlas.annot"] -[2023-12-05 09:40:20.286209985] (Utility.Process) process [1162209] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","check-attr","-z","--stdin","annex.backend","annex.largefiles","annex.numcopies","annex.mincopies","--"] -[2023-12-05 09:40:20.28954558] (Utility.Process) process [1162210] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -add lh.aparc.DKTatlas.annot [2023-12-05 09:40:20.293105736] (Utility.Process) process [1162211] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","symbolic-ref","-q","HEAD"] -[2023-12-05 09:40:20.296109063] (Utility.Process) process [1162211] done ExitSuccess -[2023-12-05 09:40:20.296268327] (Utility.Process) process [1162212] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","refs/heads/master"] -[2023-12-05 09:40:20.299447211] (Utility.Process) process [1162212] done ExitFailure 1 -[2023-12-05 09:40:20.299472144] (Annex.Perms) freezing content lh.aparc.DKTatlas.annot - - .git/annex/othertmp/: openTempFile template ingest-lh.: invalid argument (Invalid argument) - -failed -[2023-12-05 09:40:20.299683526] (Utility.Process) process [1162208] done ExitSuccess -[2023-12-05 09:40:20.299804817] (Utility.Process) process [1162213] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","ls-files","-z","--modified","--","lh.aparc.DKTatlas.annot"] -[2023-12-05 09:40:20.302582841] (Utility.Process) process [1162213] done ExitSuccess -[2023-12-05 09:40:20.302707305] (Utility.Process) process [1162214] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","diff","--name-only","--diff-filter=T","-z","--cached","--","lh.aparc.DKTatlas.annot"] -[2023-12-05 09:40:20.30608269] (Utility.Process) process [1162214] done ExitSuccess -[2023-12-05 09:40:20.306609675] (Utility.Process) process [1162205] done ExitSuccess -[2023-12-05 09:40:20.306732757] (Utility.Process) process [1162210] done ExitSuccess -[2023-12-05 09:40:20.306873108] (Utility.Process) process [1162209] done ExitSuccess -add: 1 failed -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Yes! I use git annex and datalad all the time for personal and work projects. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/Invalid_argument_saving_FreeSurfer_file_on_NTFS/comment_1_181a319138cc10742bc8676d77e8a614._comment b/doc/bugs/Invalid_argument_saving_FreeSurfer_file_on_NTFS/comment_1_181a319138cc10742bc8676d77e8a614._comment deleted file mode 100644 index 0ab08545f2..0000000000 --- a/doc/bugs/Invalid_argument_saving_FreeSurfer_file_on_NTFS/comment_1_181a319138cc10742bc8676d77e8a614._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-12-05T16:16:35Z" - content=""" -This is EINVAL from probably open(2) or something like that. - - EINVAL The final component ("basename") of pathname is invalid (e.g., - it contains characters not permitted by the underlying filesys‐ - tem). - -The problem is likely the ending ".", since FAT and probably other Microsoft filesystems -don't allow that, and/or have other strange behavior like silently removing that. - -Changed the code to avoid this. -"""]] diff --git a/doc/bugs/J1_display_issue.mdwn b/doc/bugs/J1_display_issue.mdwn deleted file mode 100644 index d0591f16f2..0000000000 --- a/doc/bugs/J1_display_issue.mdwn +++ /dev/null @@ -1,13 +0,0 @@ -With -J1, git-annex get does not display filenames: - - git-annex get -J1 foo - - 23% 166.11 MiB 3 MiB/s 3m18s - -The filename is displayed at end. - -This does not happen at -J2 or without -J. - ---[[Joey]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/No_viable_build_plan_for_ghc_9.2.2.mdwn b/doc/bugs/No_viable_build_plan_for_ghc_9.2.2.mdwn deleted file mode 100644 index e2366a6329..0000000000 --- a/doc/bugs/No_viable_build_plan_for_ghc_9.2.2.mdwn +++ /dev/null @@ -1,86 +0,0 @@ -### Please describe the problem. - -Some dependencies of `git-annex` don't have viable releases for ghc-9.2.2 (aws, bloomfilter). - -``` -cabal v2-build -Resolving dependencies... -cabal: Could not resolve dependencies: -[__0] trying: bloomfilter-2.0.1.0 (user goal) -[__1] trying: deepseq-1.4.6.1/installed-1.4.6.1 (dependency of bloomfilter) -[__2] trying: bytestring-0.11.3.0/installed-0.11.3.0 (dependency of -bloomfilter) -[__3] trying: base-4.16.1.0/installed-4.16.1.0 (dependency of bloomfilter) -[__4] trying: git-annex-10.20220322 (user goal) -[__5] trying: uuid-1.3.15 (dependency of git-annex) -[__6] trying: time-1.11.1.1/installed-1.11.1.1 (dependency of git-annex) -[__7] trying: aws-0.22 (dependency of git-annex) -[__8] next goal: aeson (dependency of git-annex) -[__8] rejecting: aeson-2.0.3.0 (conflict: aws => aeson>=0.6 && <1.6) -[__8] skipping: aeson-2.0.2.0, aeson-2.0.1.0, aeson-2.0.0.0 (has the same -characteristics that caused the previous version to fail: excluded by -constraint '>=0.6 && <1.6' from 'aws') -... -``` - -### What steps will reproduce the problem? - -`cabal v2-build [--allow-newer]` - -### What version of git-annex are you using? On what operating system? - -I checked out HEAD as of `b25bfecb1`. Building it on OpenBSD with ghc-9.2.2. - - -### Please provide any additional information below. - -Building with `--allow-newer` first points at `bloomfilter` which is a [known problem with a patch](https://github.com/bos/bloomfilter/pull/20). The next problem is with `aws` and this appears to [also be a known issue](https://github.com/aristidb/aws/issues/275) which also [has a patch](https://github.com/aristidb/aws/pull/277). - -### Have you had any luck using git-annex before? - -Yes, very much so. It works great as an OpenBSD port when built with ghc-8.10.6. - -### Minor fix is required to git-annex - -Once I apply this little fix to `git-annex` and the two patches referenced above, the code type-checks. - -```patch -diff --git a/Types/Export.hs b/Types/Export.hs -index a5cb173e5..2d5419b91 100644 ---- a/Types/Export.hs -+++ b/Types/Export.hs -@@ -21,7 +21,7 @@ import Git.FilePath - import Utility.Split - import Utility.FileSystemEncoding - --import Data.ByteString.Short as S -+import qualified Data.ByteString.Short as S - import qualified System.FilePath.Posix as Posix - import GHC.Generics - import Control.DeepSeq -``` - -I also see all but one tests pass, but I'm pretty sure the same test used to fail with the older compiler: - -``` -Tests - Tasty - tasty self-check: OK - +++ OK, passed 1 test. - Repo Tests v8 locked - Init Tests - init: OK (0.77s) - add: OK (2.71s) - rsync remote: FAIL (3.58s) - ./Test/Framework.hs:328: - able to modify annexed file's foo content - Use -p '/rsync remote/' to rerun this test only. - conflict resolution (adjusted branch): OK (16.26s) - conversion annexed to git: OK (3.32s) - move (ssh remote): OK (6.11s) - export_import: OK (13.23s) - -1 out of 8 tests failed (46.00s) -``` - -> I took over maintenance of bloomfilter, and this is [[done]] --[[Joey]] diff --git a/doc/bugs/No_viable_build_plan_for_ghc_9.2.2/comment_1_8e7f9605f05880899eba7eae4e6f7ea0._comment b/doc/bugs/No_viable_build_plan_for_ghc_9.2.2/comment_1_8e7f9605f05880899eba7eae4e6f7ea0._comment deleted file mode 100644 index 7c13f68095..0000000000 --- a/doc/bugs/No_viable_build_plan_for_ghc_9.2.2/comment_1_8e7f9605f05880899eba7eae4e6f7ea0._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-05-02T18:13:47Z" - content=""" -I've applied the patch to git-annex to qualify the export. - -That failing test is an unrelated and now fixed bug. - -I can't do much more about the dependencies needing updates. Aside from -maintaining them myself, which I kind of have been with aws to an extent, -but not enough to be able to cut a new release with the fix. -"""]] diff --git a/doc/bugs/No_viable_build_plan_for_ghc_9.2.2/comment_2_e30befdadfd7989de2e214b8e1f6ed07._comment b/doc/bugs/No_viable_build_plan_for_ghc_9.2.2/comment_2_e30befdadfd7989de2e214b8e1f6ed07._comment deleted file mode 100644 index 99c4493b1c..0000000000 --- a/doc/bugs/No_viable_build_plan_for_ghc_9.2.2/comment_2_e30befdadfd7989de2e214b8e1f6ed07._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2023-06-21T18:10:04Z" - content=""" -aws is fixed - -I have emailed bloomfilter's maintainer offering to take on maintenance -"""]] diff --git a/doc/bugs/Redundant_quote_in_the_filepath_with_rsync_remote.mdwn b/doc/bugs/Redundant_quote_in_the_filepath_with_rsync_remote.mdwn deleted file mode 100644 index 71bbb75271..0000000000 --- a/doc/bugs/Redundant_quote_in_the_filepath_with_rsync_remote.mdwn +++ /dev/null @@ -1,49 +0,0 @@ -### Please describe the problem. -Using a rsync remote with hybrid encryption, I encounter the following problem. - - $ git annex get 30.zip - get 30.zip (from to...) - rsync: [sender] change_dir "/home/user/drive/Annex/t/eec/656/'GPGHMACSHA1--addf78b2c59182e951d16ce90e03900cfe15dc7d" failed: No such file or directory (2) - rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1835) [Receiver=3.2.4] - rsync: [Receiver] write error: Broken pipe (32) - rsync exited 23 - rsync: [sender] change_dir "/home/user/drive/Annex/t/Vk/Zj/'GPGHMACSHA1--addf78b2c59182e951d16ce90e03900cfe15dc7d" failed: No such file or directory (2) - rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1835) [Receiver=3.2.4] - rsync: [Receiver] write error: Broken pipe (32) - rsync exited 23 - -Note the extra single quote inserted in the filepath in front of the GPGHMACSHA1... folder name. I have verified that the file exists on the remote drive. - -### What version of git-annex are you using? On what operating system? - -git-annex-8.20211123-3.fc36.x86_64 - -### Have you had any luck using git-annex before? - -I have used git annex with great success for better part of the decade. Thank you for all your hard work! - -### Extra debugging - -To figure out if it was really just the mangled path, I have added a symlink to pretend the variant of the folder name with the leading quote exists: - - lrwxrwxrwx. 1 user user 53 May 31 20:31 "'GPGHMACSHA1--addf78b2c59182e951d16ce90e03900cfe15dc7d" -> GPGHMACSHA1--addf78b2c59182e951d16ce90e03900cfe15dc7d - drwxrwxr-x. 2 user user 4096 Jul 19 2020 GPGHMACSHA1--addf78b2c59182e951d16ce90e03900cfe15dc7d - -Now the git annex get fails with - - rsync: [sender] link_stat "/home/user/drive/Annex/t/eec/656/'GPGHMACSHA1--addf78b2c59182e951d16ce90e03900cfe15dc7d/GPGHMACSHA1--addf78b2c59182e951d16ce90e03900cfe15dc7d'" failed: No such file or directory (2) - rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1835) [Receiver=3.2.4] - rsync: [Receiver] write error: Broken pipe (32) - -Note that we were able to chdir into the folder now, but the file itself is not being found. This time we do not have a leading quote, but a trailing one. Adding another symlink, this time for the file: - - -rw-rw-r--. 1 user user 157865953 Jul 19 2020 GPGHMACSHA1--addf78b2c59182e951d16ce90e03900cfe15dc7d - lrwxrwxrwx. 1 user user 53 May 31 20:42 "GPGHMACSHA1--addf78b2c59182e951d16ce90e03900cfe15dc7d'" -> GPGHMACSHA1--addf78b2c59182e951d16ce90e03900cfe15dc7d - -Makes this particular error go away, only to fail with - - ../../../.git/annex/tmp/GPGHMACSHA1--addf78b2c59182e951d16ce90e03900cfe15dc7d: openBinaryFile: does not exist (No such file or directory) - -This is likely because the file has been downloaded into tmp directory with the trailing quote. To summarize, it seems that file and its parent folder in the rsync remote directory tree are being wrapped by single quotes, resulting into a file lookup failure. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/Redundant_quote_in_the_filepath_with_rsync_remote/comment_1_53d3e78c0fb4d4c398dd47498e6348aa._comment b/doc/bugs/Redundant_quote_in_the_filepath_with_rsync_remote/comment_1_53d3e78c0fb4d4c398dd47498e6348aa._comment deleted file mode 100644 index 8d06f3faae..0000000000 --- a/doc/bugs/Redundant_quote_in_the_filepath_with_rsync_remote/comment_1_53d3e78c0fb4d4c398dd47498e6348aa._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-06-02T15:58:10Z" - content=""" -This is a breakage caused by rsync 3.2.4 which broke backwards -compatability. - -It is fixed in git-annex 10.20220504. -"""]] diff --git a/doc/bugs/Repo_manipulation_breaks_git-annex_drop_--all.mdwn b/doc/bugs/Repo_manipulation_breaks_git-annex_drop_--all.mdwn deleted file mode 100644 index 1c861fd5aa..0000000000 --- a/doc/bugs/Repo_manipulation_breaks_git-annex_drop_--all.mdwn +++ /dev/null @@ -1,58 +0,0 @@ -I am working with a bare repository to transfer two keys from a custom backend to and from a special remote. This seems to be working fine. - -In order to be able to make use of export remotes (exporttree=yes), I need to be able to specific a tree to be exported. For technical reasons, I want to keep using a bare repository, and use a `hash-object`, `update-index`, and `write-tree` manually in order to create a tree. The Python code snippet that does this looks like this: - -``` - for key, prefix, fname in ( - # the prefixes are constant hashdir-lower - (RepoAnnexGitRemote.refs_key, 'a11/1c8', '.datalad/dotgit/refs'), - (RepoAnnexGitRemote.repo_export_key, '6b2/c13', - '.datalad/dotgit/repo-export.zip')): - # create a blob for the annex link - out = repo._git_runner.run( - ['git', 'hash-object', '-w', '--stdin'], - stdin=bytes( - f'../../.git/annex/objects/{prefix}/{key}/{key}', 'utf-8'), - protocol=StdOutCapture) - linkhash = out['stdout'].strip() - # place link into a tree - out = repo._git_runner.run( - ['git', 'update-index', '--add', '--cacheinfo', '120000', - linkhash, fname], - protocol=StdOutCapture) - # write the complete tree, and return ID - out = repo._git_runner.run( - ['git', 'write-tree'], - protocol=StdOutCapture) - exporttree = out['stdout'].strip() - ``` - -It essentially creates the two blobs for the annex links, puts them together in a tree, and writes it to the repo. - -However, after this code ran, git-annex is not longer operating properly in the bare repo: - -``` -% git annex drop --all -fatal: relative path syntax can't be used outside working tree -fatal: relative path syntax can't be used outside working tree -fatal: relative path syntax can't be used outside working tree -fatal: relative path syntax can't be used outside working tree -fatal: relative path syntax can't be used outside working tree -fatal: relative path syntax can't be used outside working tree -fatal: relative path syntax can't be used outside working tree -fatal: relative path syntax can't be used outside working tree -fatal: relative path syntax can't be used outside working tree -fatal: relative path syntax can't be used outside working tree -fatal: relative path syntax can't be used outside working tree -git-annex: fd:21: Data.ByteString.hGetLine: end of file -``` - -(fatal error messages are from cat-file batch calls inside) - -When I comment this code out, everything goes back to normal. It seems to makes no difference whether I follow the problematic code up with a `commit-tree` and `update-ref` to actually have the mainline branch point to a commit with that tree. It also seems to make no difference, when I explicitly `setpresentkey 0`. - -AFAICS this creates the same records as if I would have done this in a regular worktree using high-level git-annex tooling. Other git-annex commands like `fsck` seem to be working fine. If a create a branch with that tree, also `findref` seems to be working properly. - -Is this a bug, or am I doing something wrong? Thanks in advance for your time! - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/Repo_manipulation_breaks_git-annex_drop_--all/comment_1_22fb9214e6e31d04b0ffe3266d26ea69._comment b/doc/bugs/Repo_manipulation_breaks_git-annex_drop_--all/comment_1_22fb9214e6e31d04b0ffe3266d26ea69._comment deleted file mode 100644 index 082b7b7332..0000000000 --- a/doc/bugs/Repo_manipulation_breaks_git-annex_drop_--all/comment_1_22fb9214e6e31d04b0ffe3266d26ea69._comment +++ /dev/null @@ -1,28 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-01-11T16:25:50Z" - content=""" -`git update-index` creates an index file. A repo with an index file is no -longer a bare repo to a certain extent, at least it is very unusual for a -repo to be both bare and contain an index file, since after all an index -file is a record of the files in the working tree. This seems to be the root -of the confusion. - -I was able to reproduce this by simply copying `.git/index` from a non-bare -repo into the bare repo and then running `git-annex drop --all`. - -Removing the index file was not sufficient to fix it. It turned out I also -needed to delete `annex/keydb*`. Then things returned to normal. - -So, it seems that the keys database is getting populated in a bare repo -when there's an index file, and once the keys database is populated, it -runs code paths that will not work in a bare repo, because that database -contains paths (taken from the index) that it treats as being present in a -nonexistant working tree. I've fixed it to both avoid populating the keys -database, and ignore a populated keys database in this situation. - -But.. My suggestion is, if you need to do this kind of thing in a bare repo, -set `GIT_INDEX_FILE` to some other file. That's how git-annex makes similar -tree objects. -"""]] diff --git a/doc/bugs/Repo_manipulation_breaks_git-annex_drop_--all/comment_2_fac648fb7550f0c3c433ce423e3ea7f6._comment b/doc/bugs/Repo_manipulation_breaks_git-annex_drop_--all/comment_2_fac648fb7550f0c3c433ce423e3ea7f6._comment deleted file mode 100644 index ed38f6ac40..0000000000 --- a/doc/bugs/Repo_manipulation_breaks_git-annex_drop_--all/comment_2_fac648fb7550f0c3c433ce423e3ea7f6._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="mih" - avatar="http://cdn.libravatar.org/avatar/f881df265a423e4f24eff27c623148fd" - subject="Thanks!" - date="2022-01-11T18:59:36Z" - content=""" -That was the missing piece for me. Thank you! -"""]] diff --git a/doc/bugs/Resolver_lts-19.16_still_causes_trouble_with_Win32.mdwn b/doc/bugs/Resolver_lts-19.16_still_causes_trouble_with_Win32.mdwn deleted file mode 100644 index 90821b161d..0000000000 --- a/doc/bugs/Resolver_lts-19.16_still_causes_trouble_with_Win32.mdwn +++ /dev/null @@ -1,94 +0,0 @@ -### Please describe the problem. - -The change of resolver to LTS 19 (lts-19.16) still causes stack builds to fail on Windows -with a plan construction problem. Pinning Win32 as `Win32-2.11.1.0` in extra-deps as -suggested by stack is not enough to resolve the issue -- one has to go further and -pin Win32 as `Win32-2.9.0.0` and do similarly with a few other GHC boot packages -including `Cabal`. This setup then builds and tests just fine. I'm aware this needs to -be probably addressed in the cabal file instead but as a user the patch below is my -local fix for the issue. - -### What steps will reproduce the problem? - -`stack setup && stack build`. Observe the following output: - -[[!format sh """ -Warning: C:\Projektit\git-annex.branchable.com\git-annex--BUILD-220730-b5dc04099\stack.yaml: Unrecognized field in ProjectAndConfigMonoid: explicit-setup-deps - -Error: While constructing the build plan, the following exceptions were encountered: - -In the dependencies for git-annex-10.20220724(+assistant -+benchmark --dbus --debuglocks -+gitlfs --magicmime -+pairing -+production -+torrentparser -+webapp): - Win32-2.12.0.1 from stack configuration does not match (>=2.6.1.0 && <2.12.0.0) (latest - matching version is 2.11.1.0) -needed since git-annex is a build target. - -Some different approaches to resolving this: - - * Set 'allow-newer: true' - in C:\hs-stack\config.yaml to ignore all version constraints and build anyway. - - * Recommended action: try adding the following to your extra-deps - in C:\Projektit\git-annex.branchable.com\git-annex--BUILD-220730-b5dc04099\stack.yaml: - -- Win32-2.11.1.0@sha256:f503acb4ea2e7da6433549d5ff7d0851da54f226df032b40ae51559c7914d62f,4387 - -Plan construction failed. -# End of transcript or log. -"""]] - -### What version of git-annex are you using? On what operating system? - -[[!format sh """ -git-annex version: 10.20220725-gb5dc04099 -build flags: Assistant Webapp Pairing TorrentParser Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22.1 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.11 persistent-sqlite-2.13.1.0 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: mingw32 x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 2 3 4 5 6 7 8 9 10 -"""]] - -A successful build that passes the testsuite is achieved with the patch to `stack.yaml` below. - -Windows 10 version 21H2 (build 19044.1826), 64 bit. - -### Please provide any additional information below. - -Apply the following patch to build on Windows: - -[[!format diff """ -diff --git a/stack.yaml b/stack.yaml -index 8ca5d7383..94b724fe2 100644 ---- a/stack.yaml -+++ b/stack.yaml -@@ -22,5 +22,10 @@ extra-deps: - - network-multicast-0.3.2 - - sandi-0.5 - - torrent-10000.1.1 -+- Win32-2.9.0.0 -+- Cabal-3.6.3.0 -+- directory-1.3.7.1 -+- process-1.6.14.0 -+- time-1.11.1.2 - explicit-setup-deps: - git-annex: true -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Git Annex is great. I use it several times a week with my multigigabyte backups, where it gives structure to my image-based backup routines, so you could say I'm a believer. :) - -[[!meta author=jkniiv]] -[[!meta title="windows: Resolver lts-19.16 still causes trouble with Win32 for stack builds"]] - -> reverted, [[done]] --[[Joey]] diff --git a/doc/bugs/Resolver_lts-19.16_still_causes_trouble_with_Win32/comment_1_7fc0ffa7787353cf40fff0bc8da43c1d._comment b/doc/bugs/Resolver_lts-19.16_still_causes_trouble_with_Win32/comment_1_7fc0ffa7787353cf40fff0bc8da43c1d._comment deleted file mode 100644 index 31ef6d2c9f..0000000000 --- a/doc/bugs/Resolver_lts-19.16_still_causes_trouble_with_Win32/comment_1_7fc0ffa7787353cf40fff0bc8da43c1d._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="Ilya_Shlyakhter" - avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" - subject="lts version" - date="2022-07-31T17:25:27Z" - content=""" -Did the resolver change to lts-19.16? [`stack.yaml` on hackage](https://hackage.haskell.org/package/git-annex-10.20220724/src/stack.yaml) still seems to say lts-18. - - -"""]] diff --git a/doc/bugs/Resolver_lts-19.16_still_causes_trouble_with_Win32/comment_2_0b96112c5d57886add62d79e7bf6044e._comment b/doc/bugs/Resolver_lts-19.16_still_causes_trouble_with_Win32/comment_2_0b96112c5d57886add62d79e7bf6044e._comment deleted file mode 100644 index e62aa2a009..0000000000 --- a/doc/bugs/Resolver_lts-19.16_still_causes_trouble_with_Win32/comment_2_0b96112c5d57886add62d79e7bf6044e._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 2" - date="2022-07-31T18:00:59Z" - content=""" -@Ilya: Yes, the change took place in commit [[!commit b5dc04099efd8b1bd2dbfa09a288518eabf8ceec]] which was some 24 hours after -release 10.20220724. :) -"""]] diff --git a/doc/bugs/Resolver_lts-19.16_still_causes_trouble_with_Win32/comment_3_79cafc76eb23e54ff70c72c2f528e724._comment b/doc/bugs/Resolver_lts-19.16_still_causes_trouble_with_Win32/comment_3_79cafc76eb23e54ff70c72c2f528e724._comment deleted file mode 100644 index 98083e6238..0000000000 --- a/doc/bugs/Resolver_lts-19.16_still_causes_trouble_with_Win32/comment_3_79cafc76eb23e54ff70c72c2f528e724._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2022-08-01T17:13:22Z" - content=""" -Hey, thanks and sorry for inflicting that stack change on you a second -time. I hypothesized that the toolchain installation, that failed before, -might have been a transient problem, and it seems it was, but now these other -problems surface.. - -Surprisingly, putting Win32 in the stack file does not break stack build -on linux. But I don't like putting it in there regardless, and needing -to downgrade the other dependencies for it is also suboptimal. - -I'll revert the stack.yaml change again, and it seems we'll need to wait for -[[todo/windows__58___prepare_for_Win32_not_export_c__95___entities]] -to be fixed before updating it. -"""]] diff --git a/doc/bugs/Resolver_lts-19.16_still_causes_trouble_with_Win32/comment_4_07f1b2eacf7e9e58809ee44fc1d6b574._comment b/doc/bugs/Resolver_lts-19.16_still_causes_trouble_with_Win32/comment_4_07f1b2eacf7e9e58809ee44fc1d6b574._comment deleted file mode 100644 index cfe2026977..0000000000 --- a/doc/bugs/Resolver_lts-19.16_still_causes_trouble_with_Win32/comment_4_07f1b2eacf7e9e58809ee44fc1d6b574._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 4" - date="2022-08-03T15:47:07Z" - content=""" -@joey: No problem. Most of my git-annex use is rather vanilla (apart from annexing huge files) so I -don't usually need the very latest in git-annex so I can always fall back to an earlier release/build. -It's just that I like to keep track of git-annex's development and make sure it works on Windows most -of the time which means I feel a need to build fresh versions and then kind of dogfood them as soon -as possible. As a non-developer being able to report build errors and other bugs is my way of taking -part in this project and hopefully in the process help make things smoother for the Windows user constituent -that I represent. - -PS. Ikiwiki broke that link you made to my related todo. What's with that? The backlink appears in the todo, -however. -"""]] diff --git a/doc/bugs/Resolver_lts-19.33_still_causes_trouble_with_Win32.mdwn b/doc/bugs/Resolver_lts-19.33_still_causes_trouble_with_Win32.mdwn deleted file mode 100644 index 97428699b9..0000000000 --- a/doc/bugs/Resolver_lts-19.33_still_causes_trouble_with_Win32.mdwn +++ /dev/null @@ -1,426 +0,0 @@ -### Please describe the problem. - -I'm sorry for barking up the same tree once again but the change of resolver to LTS 19 (now lts-19.33) -still causes stack builds to fail on Windows with a plan construction problem. Joey pinned Win32 as -`Win32-2.11.1.0` in extra-deps and while that might be sufficient on Linux, on Windows stack seems to -be rather picky and wants to have none of it. In fact my bumbling around with editing `stack.yaml` -trying to placate the Haskell Stack gods has only found a single solution to the predicament and that is -to pin Win32 as `Win32-2.9.0.0` (sic! -- due to the dependencies for `haskeline`) and do similarly with -a few other GHC boot packages including `Cabal`. This setup then builds and tests just fine. (There is -a patch to `stack.yaml` below.) - -I know `Win32-2.9.0.0` is a bit long in the tooth (although, who knows, maybe it works equally well -for git-annex' purposes?) but I wonder do we have a choice if we want to keep aws-0.24 in the configuration? - -### What steps will reproduce the problem? - -`stack setup && stack build`. Observe the following output: - -[[!format sh """ -Error: [S-4804] - Stack failed to construct a build plan. - - While constructing the build plan, Stack encountered the following errors: - - In the dependencies for Glob-0.10.2: - directory needed, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.3.8.0) - needed due to git-annex-10.20230214 -> Glob-0.10.2 - - In the dependencies for Win32-notify-0.3.0.3: - directory needed, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.3.8.0) - needed due to git-annex-10.20230214 -> Win32-notify-0.3.0.3 - - In the dependencies for aeson-2.0.3.0: - time must match >=1.6.0.1 && <1.13, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.12.2) - needed due to git-annex-10.20230214 -> aeson-2.0.3.0 - - In the dependencies for alex-3.2.7.1: - directory needed, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.3.8.0) - needed due to git-annex-10.20230214 -> alex-3.2.7.1 - - In the dependencies for attoparsec-iso8601-1.0.2.1: - time must match >=1.6.0.1 && <1.13, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.12.2) - needed due to git-annex-10.20230214 -> attoparsec-iso8601-1.0.2.1 - - In the dependencies for aws-0.24: - directory must match >=1.0 && <2.0, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.3.8.0) - time must match >=1.4.0 && <2.0, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.12.2) - needed due to git-annex-10.20230214 -> aws-0.24 - - In the dependencies for cabal-doctest-1.0.9: - Cabal must match >=1.10 && <3.10, but the Stack configuration has no specified - version (latest matching version is 3.8.1.0) - directory needed, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.3.8.0) - needed due to git-annex-10.20230214 -> cabal-doctest-1.0.9 - - In the dependencies for clientsession-0.9.1.2: - directory must match >=1, but this GHC boot package has been pruned (issue #4510); you - need to add the package explicitly to extra-deps (latest matching version is - 1.3.8.0) - needed due to git-annex-10.20230214 -> clientsession-0.9.1.2 - - In the dependencies for concurrent-output-1.10.16: - directory must match >=1.2.0 && <1.4.0, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.3.8.0) - process must match >=1.6.0 && <1.7.0, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.6.17.0) - needed due to git-annex-10.20230214 -> concurrent-output-1.10.16 - - In the dependencies for conduit-1.3.4.3: - directory needed, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.3.8.0) - needed due to git-annex-10.20230214 -> conduit-1.3.4.3 - - In the dependencies for conduit-extra-1.3.6: - directory needed, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.3.8.0) - process needed, but this GHC boot package has been pruned (issue #4510); you need to add - the package explicitly to extra-deps (latest matching version is 1.6.17.0) - needed due to git-annex-10.20230214 -> conduit-extra-1.3.6 - - In the dependencies for cookie-0.4.5: - time must match >=1.5, but this GHC boot package has been pruned (issue #4510); you need - to add the package explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> cookie-0.4.5 - - In the dependencies for criterion-1.5.13.0: - directory needed, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.3.8.0) - time needed, but this GHC boot package has been pruned (issue #4510); you need to add the - package explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> criterion-1.5.13.0 - - In the dependencies for easy-file-0.2.2: - directory needed, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.3.8.0) - time needed, but this GHC boot package has been pruned (issue #4510); you need to add the - package explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> easy-file-0.2.2 - - In the dependencies for entropy-0.4.1.10: - Cabal must match >=1.10 && <3.9, but the Stack configuration has no specified - version (latest matching version is 3.8.1.0) - directory must match <1.4, but this GHC boot package has been pruned (issue #4510); you - need to add the package explicitly to extra-deps (latest matching version is - 1.3.8.0) - process must match <1.7, but this GHC boot package has been pruned (issue #4510); you - need to add the package explicitly to extra-deps (latest matching version is - 1.6.17.0) - needed due to git-annex-10.20230214 -> entropy-0.4.1.10 - - In the dependencies for fast-logger-3.1.1: - directory needed, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.3.8.0) - needed due to git-annex-10.20230214 -> fast-logger-3.1.1 - - In the dependencies for feed-1.3.2.1: - time must match <1.12, but this GHC boot package has been pruned (issue #4510); you need - to add the package explicitly to extra-deps (latest matching version is 1.11.1.2) - needed due to git-annex-10.20230214 -> feed-1.3.2.1 - - In the dependencies for file-embed-0.0.15.0: - directory must match >=1.0.0.3, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.3.8.0) - needed due to git-annex-10.20230214 -> file-embed-0.0.15.0 - - In the dependencies for git-annex-10.20230214(+assistant - +benchmark - -dbus - -debuglocks - +gitlfs - -magicmime - +pairing - +production - +torrentparser): - Cabal must match <4.0, but the Stack configuration has no specified version (latest - matching version is 3.8.1.0) - directory must match >=1.2.7.0, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.3.8.0) - process must match >=1.6.3, but this GHC boot package has been pruned (issue #4510); you - need to add the package explicitly to extra-deps (latest matching version is - 1.6.17.0) - time must match >=1.5.0, but this GHC boot package has been pruned (issue #4510); you - need to add the package explicitly to extra-deps (latest matching version is 1.12.2) - needed since git-annex is a build target. - - In the dependencies for haskeline-0.8.2: - Win32-2.11.1.0 from Stack configuration does not match >=2.1 && <2.10 || >=2.12 (latest - matching version is 2.13.4.0) - directory must match >=1.1 && <1.4, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.3.8.0) - process must match >=1.0 && <1.7, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.6.17.0) - needed due to git-annex-10.20230214 -> haskeline-0.8.2 - - In the dependencies for http-client-0.7.13.1: - time must match >=1.2, but this GHC boot package has been pruned (issue #4510); you need - to add the package explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> http-client-0.7.13.1 - - In the dependencies for http-date-0.0.11: - time needed, but this GHC boot package has been pruned (issue #4510); you need to add the - package explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> http-date-0.0.11 - - In the dependencies for libyaml-0.1.2: - directory needed, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.3.8.0) - needed due to git-annex-10.20230214 -> libyaml-0.1.2 - - In the dependencies for microstache-1.0.2.2: - directory must match >=1.1.0.2 && <1.4, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.3.8.0) - needed due to git-annex-10.20230214 -> microstache-1.0.2.2 - - In the dependencies for mwc-random-0.15.0.2: - time needed, but this GHC boot package has been pruned (issue #4510); you need to add the - package explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> mwc-random-0.15.0.2 - - In the dependencies for network-3.1.2.7: - directory needed, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.3.8.0) - needed due to git-annex-10.20230214 -> network-3.1.2.7 - - In the dependencies for optparse-applicative-0.16.1.0: - process must match >=1.0 && <1.7, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.6.17.0) - needed due to git-annex-10.20230214 -> optparse-applicative-0.16.1.0 - - In the dependencies for path-pieces-0.2.1: - time needed, but this GHC boot package has been pruned (issue #4510); you need to add the - package explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> path-pieces-0.2.1 - - In the dependencies for persistent-2.13.3.5: - time must match >=1.6, but this GHC boot package has been pruned (issue #4510); you need - to add the package explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> persistent-2.13.3.5 - - In the dependencies for persistent-sqlite-2.13.1.0: - time needed, but this GHC boot package has been pruned (issue #4510); you need to add the - package explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> persistent-sqlite-2.13.1.0 - - In the dependencies for resource-pool-0.2.3.2: - time needed, but this GHC boot package has been pruned (issue #4510); you need to add the - package explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> resource-pool-0.2.3.2 - - In the dependencies for shakespeare-2.0.30: - directory must match >=1.2, but this GHC boot package has been pruned (issue #4510); you - need to add the package explicitly to extra-deps (latest matching version is - 1.3.8.0) - process must match >=1.0, but this GHC boot package has been pruned (issue #4510); you - need to add the package explicitly to extra-deps (latest matching version is - 1.6.17.0) - time must match >=1, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> shakespeare-2.0.30 - - In the dependencies for silently-1.2.5.3: - directory needed, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.3.8.0) - needed due to git-annex-10.20230214 -> silently-1.2.5.3 - - In the dependencies for streaming-commons-0.2.2.5: - directory needed, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.3.8.0) - process needed, but this GHC boot package has been pruned (issue #4510); you need to add - the package explicitly to extra-deps (latest matching version is 1.6.17.0) - needed due to git-annex-10.20230214 -> streaming-commons-0.2.2.5 - - In the dependencies for terminal-size-0.3.3: - process needed, but this GHC boot package has been pruned (issue #4510); you need to add - the package explicitly to extra-deps (latest matching version is 1.6.17.0) - needed due to git-annex-10.20230214 -> terminal-size-0.3.3 - - In the dependencies for th-compat-0.1.4: - directory must match >=1.1.0.0 && <1.4, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.3.8.0) - needed due to git-annex-10.20230214 -> th-compat-0.1.4 - - In the dependencies for time-compat-1.9.6.1: - time must - match >=1.5 && <1.7 || >=1.8 && <1.9 || >=1.9.2 && <1.9.4 || >=1.10 && <1.10.1 || >=1.11 && <1.11.2 || >=1.12 && <1.13, - but this GHC boot package has been pruned (issue #4510); you need to add the package - explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> time-compat-1.9.6.1 - - In the dependencies for time-locale-compat-0.1.1.5(-old-locale): - time must match >=1.5, but this GHC boot package has been pruned (issue #4510); you need - to add the package explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> time-locale-compat-0.1.1.5 - - In the dependencies for typed-process-0.2.10.1: - process must match >=1.2, but this GHC boot package has been pruned (issue #4510); you - need to add the package explicitly to extra-deps (latest matching version is - 1.6.17.0) - needed due to git-annex-10.20230214 -> typed-process-0.2.10.1 - - In the dependencies for unix-compat-0.5.4: - directory must match >=1.2 && <1.4, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.3.8.0) - time must match >=1.0 && <1.13, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.12.2) - needed due to git-annex-10.20230214 -> unix-compat-0.5.4 - - In the dependencies for unliftio-0.2.23.0: - directory needed, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.3.8.0) - process must match >=1.2.0.0, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.6.17.0) - time needed, but this GHC boot package has been pruned (issue #4510); you need to add the - package explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> unliftio-0.2.23.0 - - In the dependencies for uuid-1.3.15: - time must match >=1.4 && <1.13, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.12.2) - needed due to git-annex-10.20230214 -> uuid-1.3.15 - - In the dependencies for wai-app-static-3.1.7.4: - directory must match >=1.0.1, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.3.8.0) - time must match >=1.1.4, but this GHC boot package has been pruned (issue #4510); you - need to add the package explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> wai-app-static-3.1.7.4 - - In the dependencies for wai-extra-3.1.12.1: - directory must match >=1.2.7.0, but this GHC boot package has been pruned (issue - #4510); you need to add the package explicitly to extra-deps (latest matching - version is 1.3.8.0) - time must match >=1.1.4, but this GHC boot package has been pruned (issue #4510); you - need to add the package explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> wai-extra-3.1.12.1 - - In the dependencies for wai-logger-2.4.0: - Cabal needed, but the Stack configuration has no specified version (latest matching - version is 3.8.1.0) - needed due to git-annex-10.20230214 -> wai-logger-2.4.0 - - In the dependencies for warp-3.3.21: - time needed, but this GHC boot package has been pruned (issue #4510); you need to add the - package explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> warp-3.3.21 - - In the dependencies for x509-store-1.6.9: - directory needed, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.3.8.0) - needed due to git-annex-10.20230214 -> x509-store-1.6.9 - - In the dependencies for x509-system-1.6.7: - directory needed, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.3.8.0) - process needed, but this GHC boot package has been pruned (issue #4510); you need to add - the package explicitly to extra-deps (latest matching version is 1.6.17.0) - needed due to git-annex-10.20230214 -> x509-system-1.6.7 - - In the dependencies for xml-conduit-1.9.1.1: - Cabal needed, but the Stack configuration has no specified version (latest matching - version is 3.8.1.0) - needed due to git-annex-10.20230214 -> xml-conduit-1.9.1.1 - - In the dependencies for yaml-0.11.8.0: - directory needed, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.3.8.0) - needed due to git-annex-10.20230214 -> yaml-0.11.8.0 - - In the dependencies for yesod-1.6.2.1: - directory needed, but this GHC boot package has been pruned (issue #4510); you need to - add the package explicitly to extra-deps (latest matching version is 1.3.8.0) - needed due to git-annex-10.20230214 -> yesod-1.6.2.1 - - In the dependencies for yesod-core-1.6.24.0: - time must match >=1.5, but this GHC boot package has been pruned (issue #4510); you need - to add the package explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> yesod-core-1.6.24.0 - - In the dependencies for yesod-form-1.7.3: - time must match >=1.1.4, but this GHC boot package has been pruned (issue #4510); you - need to add the package explicitly to extra-deps (latest matching version is 1.12.2) - needed due to git-annex-10.20230214 -> yesod-form-1.7.3 - - In the dependencies for yesod-static-1.6.1.0: - directory must match >=1.0, but this GHC boot package has been pruned (issue #4510); you - need to add the package explicitly to extra-deps (latest matching version is - 1.3.8.0) - process needed, but this GHC boot package has been pruned (issue #4510); you need to add - the package explicitly to extra-deps (latest matching version is 1.6.17.0) - needed due to git-annex-10.20230214 -> yesod-static-1.6.1.0 - -# End of transcript or log. -"""]] - -### What version of git-annex are you using? On what operating system? - -[[!format sh """ -git-annex version: 10.20230215-g648e59cac -build flags: Assistant Webapp Pairing TorrentParser Benchmark Feeds Testsuite S3 WebDAV -dependency versions: aws-0.24 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.13.1 persistent-sqlite-2.13.1.0 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2.1 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: mingw32 x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 2 3 4 5 6 7 8 9 10 -"""]] - -A successful build that passes the testsuite on Windows is achieved with the patch to `stack.yaml` below. - -Windows 10 version 22H2 (build 19045.2486), 64 bit. - -### Please provide any additional information below. - -Apply the following patch to build on Windows: - -[[!format diff """ -diff --git a/stack.yaml b/stack.yaml -index c7fb4b099..f6a5e06e9 100644 ---- a/stack.yaml -+++ b/stack.yaml -@@ -21,4 +21,8 @@ extra-deps: - - network-multicast-0.3.2 - - sandi-0.5 - - torrent-10000.1.1 --- Win32-2.11.1.0 -+- Win32-2.9.0.0 -+- Cabal-3.6.3.0 -+- directory-1.3.7.1 -+- process-1.6.17.0 -+- time-1.11.1.2 -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Git Annex is great. I use it several times a week with my multigigabyte backups, where it gives structure to my image-based backup routines, so you could say I'm a believer. :) - -[[!meta author=jkniiv]] -[[!meta title="windows: Resolver lts-19.33 still (!) causes trouble with Win32 for stack builds"]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/Resolver_lts-19.33_still_causes_trouble_with_Win32/comment_1_dcc2067ea1bc26510d013ca52096513f._comment b/doc/bugs/Resolver_lts-19.33_still_causes_trouble_with_Win32/comment_1_dcc2067ea1bc26510d013ca52096513f._comment deleted file mode 100644 index d648bcb32a..0000000000 --- a/doc/bugs/Resolver_lts-19.33_still_causes_trouble_with_Win32/comment_1_dcc2067ea1bc26510d013ca52096513f._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-02-20T19:12:19Z" - content=""" -Drat. The problem is haskeline-0.8.2 depending on a different Win32 -version. I'll take your word for it that was the only dependency solution -though. - -Win32-2.9.0.0 is quite old and I don't want to risk reverting to some bug -fixed in the meantime. And I'm doubtful about bumping Cabal and directory -to newer versions too. - -So, I've reverted this attempt, and won't make another one until there's a -ghc release that has the filepath (>=1.4.100.0) that Wind32 needs. -Shouldn't be more than half a year or so.. -"""]] diff --git a/doc/bugs/S3_ACL_deprecation.mdwn b/doc/bugs/S3_ACL_deprecation.mdwn deleted file mode 100644 index b000f20c3c..0000000000 --- a/doc/bugs/S3_ACL_deprecation.mdwn +++ /dev/null @@ -1,158 +0,0 @@ -### Please describe the problem. - -Amazon has [deprecated ACLs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/about-object-ownership.html) - -> A majority of modern use cases in Amazon S3 no longer require the use of ACLs, and we recommend that you disable ACLs except in unusual circumstances where you need to control access for each object individually. With Object Ownership, you can disable ACLs and rely on policies for access control. When you disable ACLs, you can easily maintain a bucket with objects uploaded by different AWS accounts. You, as the bucket owner, own all the objects in the bucket and can manage access to them using policies. - -They are encouraging everyone to [migrate to bucket policies](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-ownership-migrating-acls-prerequisites.html) instead. - -But if such a bucket is public (meaning: write needs credentials, reads are open to the world): - -* `public=yes` causes `git annex copy --to` to always set an ACL, which fails, which fails the entire upload -* But setting `public=no` causes `publicurl` to be ignored by `git annex copy --from`, failing the download - -#### Feature Request - - -* If `public=yes`, instead of trying to set an ACL, first try `HTTP HEAD` on the newly uploaded object without using the AWS credentials. Only if that fails, fall over to trying to set an ACL using credential. And if you get AccessControlListNotSupported (i.e. the error due to BucketOwnerEnforced), then give a warning that the bucket policy is not configured for public access. -* Make `publicurl` orthogonal to `public`: if set, `git annex copy --from` should _always_ use it unconditionally. -* Update [the docs here](https://git-annex.branchable.com/special_remotes/S3/) to explain how to set up a public bucket policy as recommended by Amazon, and that `public=yes` will either try to confirm that the bucket policy is public, or will fallback to using (legacy) ACLs. - - -### What steps will reproduce the problem? - -In a bucket I run, I reset the ACLs on that bucket to Amazon's default permissions: - -* Bucket owner (your AWS account): - * Objects: - * List - * Write - * Bucket ACL (i.e. what ACLs are applied by default to all objects): - * Read - * Write - -and with that set Amazon let me also set - -> Object ownership: Bucket owner enforced - - -This should be the **default configuration** for any new bucket created now, so you only need to do the above if you're migrating an existing bucket like I was; for reproducing, just creating an empty bucket should be enough. - -It should look like this: - -[[!img S3_bucket.png align="right" size="" alt=""]] - -With that in place, I wrote and attached this Bucket Policy to make it public: - -``` -{ - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Principal": "*", - "Action": "s3:GetObject", - "Resource": [ - "arn:aws:s3:::BUCKET-NAME", - "arn:aws:s3:::BUCKET-NAME/*" - ] - } - ] -} -``` - -[[!img S3_bucket-perms.png align="right" size="" alt=""]] - - -I told `git-annex` about it with - -``` -git annex initremote amazon type=S3 bucket=BUCKET_NAME public=yes publicurl=https://BUCKET_NAME.s3.amazonaws.com autoenable=true -``` - -But, attempting to use the new setup failed: - -``` -$ git annex copy --to amazon what.nii.gz -copy what.nii.gz (checking amazon...) (to amazon...) -41% 8.15 MiB 20 MiB/s 0s - S3Error {s3StatusCode = Status {statusCode = 400, statusMessage = "Bad Request"}, s3ErrorCode = "AccessControlListNotSupported", s3ErrorMessage = "The bucket does not allow ACLs", s3ErrorResource = Nothing, s3ErrorHostId = Just "a6+ieujj4z3Z4P8ooA306DdbGAoxWDiXd6O2ZwjdfapGnuOGPyL5/WQ4UBEytR80FG+5b6xdlsM=", s3ErrorAccessKeyId = Nothing, s3ErrorStringToSign = Nothing, s3ErrorBucket = Nothing, s3ErrorEndpointRaw = Nothing, s3ErrorEndpoint = Nothing} - -32% 6.43 MiB 16 MiB/s 0s - S3Error {s3StatusCode = Status {statusCode = 400, statusMessage = "Bad Request"}, s3ErrorCode = "AccessControlListNotSupported", s3ErrorMessage = "The bucket does not allow ACLs", s3ErrorResource = Nothing, s3ErrorHostId = Just "bFOgMomROCOes9yI6HZHysQGoZaTbsPI5b7rHjcTI0wA8Yx5Dm1JOky9BvXvpcXxzY1kVt48FRQ=", s3ErrorAccessKeyId = Nothing, s3ErrorStringToSign = Nothing, s3ErrorBucket = Nothing, s3ErrorEndpointRaw = Nothing, s3ErrorEndpoint = Nothing} - -37% 7.37 MiB 21 MiB/s 0s - S3Error {s3StatusCode = Status {statusCode = 400, statusMessage = "Bad Request"}, s3ErrorCode = "AccessControlListNotSupported", s3ErrorMessage = "The bucket does not allow ACLs", s3ErrorResource = Nothing, s3ErrorHostId = Just "hqd4HRNk5yp3tKJ6yMhcECEpCjBw8qB6oTpKF3PaOsYFeVG0C+dGI06xq3zgmvnPoFUttI040sY=", s3ErrorAccessKeyId = Nothing, s3ErrorStringToSign = Nothing, s3ErrorBucket = Nothing, s3ErrorEndpointRaw = Nothing, s3ErrorEndpoint = Nothing} - -39% 7.81 MiB 21 MiB/s 0s - S3Error {s3StatusCode = Status {statusCode = 400, statusMessage = "Bad Request"}, s3ErrorCode = "AccessControlListNotSupported", s3ErrorMessage = "The bucket does not allow ACLs", s3ErrorResource = Nothing, s3ErrorHostId = Just "7m7wwG5woSPmICIuXr9QnBOEjUikuyzHSebMLuaNyZMc2Ki2vaqKpU9U+GOTYmR/NzFjOeyxngk=", s3ErrorAccessKeyId = Nothing, s3ErrorStringToSign = Nothing, s3ErrorBucket = Nothing, s3ErrorEndpointRaw = Nothing, s3ErrorEndpoint = Nothing} -failed -git-annex: copy: 1 failed -``` - - -### What version of git-annex are you using? On what operating system? - - -``` -$ git annex version -git-annex version: 8.20210223 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 -local repository version: 8 -$ cat /etc/os-release -PRETTY_NAME="Ubuntu 22.04 LTS" -NAME="Ubuntu" -VERSION_ID="22.04" -VERSION="22.04 LTS (Jammy Jellyfish)" -VERSION_CODENAME=jammy -ID=ubuntu -ID_LIKE=debian -HOME_URL="https://www.ubuntu.com/" -SUPPORT_URL="https://help.ubuntu.com/" -BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" -PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" -UBUNTU_CODENAME=jammy -``` - -### Please provide any additional information below. - - -Disabling `public` allows uploads: - -``` -$ git annex enableremote amazon public=no -enableremote amazon ok -(recording state in git...) -$ git annex copy --to amazon what.nii.gz -copy what.nii.gz (checking amazon...) (to amazon...) -ok -``` - -But then causes the new problem that downloads fail when other users try to download the updated dataset: - -``` -$ git annex get what.nii.gz -get what.nii.gz (from amazon...) - - S3 bucket does not allow public access; Set both AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to use S3 - - cannot download content - - Unable to access these remotes: amazon - - (Note that these git remotes have annex-ignore set: origin) -failed -git-annex: get: 1 failed -``` - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -We use git-annex to share large datasets with the scientific community at https://github.com/spine-generic/data-multi-subject ! - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/S3_ACL_deprecation/comment_1_07d0d382b88426180e77bac962306f44._comment b/doc/bugs/S3_ACL_deprecation/comment_1_07d0d382b88426180e77bac962306f44._comment deleted file mode 100644 index f175d1f962..0000000000 --- a/doc/bugs/S3_ACL_deprecation/comment_1_07d0d382b88426180e77bac962306f44._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="nick.guenther@e418ed3c763dff37995c2ed5da4232a7c6cee0a9" - nickname="nick.guenther" - avatar="http://cdn.libravatar.org/avatar/9e85c6ca61c3f877fef4f91c2bf6e278" - subject="comment 1" - date="2022-09-08T00:28:32Z" - content=""" -I'm sorry, I don't know what happened to my screenshot. It didn't upload correctly I guess? I've lost it now. I don't think its loss detracts too much though. -"""]] diff --git a/doc/bugs/S3_ACL_deprecation/comment_2_63b541a937cb34cc4c68504e00dc3fb8._comment b/doc/bugs/S3_ACL_deprecation/comment_2_63b541a937cb34cc4c68504e00dc3fb8._comment deleted file mode 100644 index 5311db3fb3..0000000000 --- a/doc/bugs/S3_ACL_deprecation/comment_2_63b541a937cb34cc4c68504e00dc3fb8._comment +++ /dev/null @@ -1,29 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2023-07-21T17:53:25Z" - content=""" -This only affects new S3 buckets. Existing S3 buckets that were -created before April 2023 and were set up to allow public access should -keep working, including ACL settings when storing new files in them. -Per [Amazon's announcement](https://aws.amazon.com/about-aws/whats-new/2022/12/amazon-s3-automatically-enable-block-public-access-disable-access-control-lists-buckets-april-2023/), -"There is no change for existing buckets." - -I've made `publicurl` orthogonal to `public`. - -As for the idea of `HTTP HEAD` before trying to set the ACL, -the ACL is currently sent at past of the PutObject request. And -either there is not a way to change the ACL later, or the aws haskell library -is missing support for the API to do that. - -While git-annex could HEAD without creds when publicyes=yes to verify that the -user has configured the bucket correctly, and at least warn about a -misconfiguration, that would add some overhead, and I guess if the user has not -configured the bucket correctly, they will notice in some other way eventually -and can fix its bucket policy after the fact. So I'm inclined not to do -that. - -Instead I've simply depredated `public`, noting that it should not be set -on new buckets. The user will have to deal with setting up the Bucket -Policy themselves. -"""]] diff --git a/doc/bugs/S3_remote_errors_with_garage_endpoint.mdwn b/doc/bugs/S3_remote_errors_with_garage_endpoint.mdwn deleted file mode 100644 index a9a7c5a8f0..0000000000 --- a/doc/bugs/S3_remote_errors_with_garage_endpoint.mdwn +++ /dev/null @@ -1,33 +0,0 @@ -### Please describe the problem. - -When I attempt to create a S3 remote against my garage[1] cluster, it errors with the following: - -``` -$ git annex initremote garage type=S3 encryption=none host=my-s3-endpoint.domain.com protocol=https bucket=git-annex requeststyle=path datacenter=garage signature=v4 -initremote garage (checking bucket...) (creating bucket in garage...) -git-annex: S3Error {s3StatusCode = Status {statusCode = 400, statusMessage = "Bad Request"}, s3ErrorCode = "AuthorizationHeaderMalformed", s3ErrorMessage = "Authorization header malformed, expected scope: 20230118/my-s3-endpoint.domain.com/s3/aws4_request", s3ErrorResource = Just "/git-annex/", s3ErrorHostId = Nothing, s3ErrorAccessKeyId = Nothing, s3ErrorStringToSign = Nothing, s3ErrorBucket = Nothing, s3ErrorEndpointRaw = Nothing, s3ErrorEndpoint = Nothing} -failed -initremote: 1 failed - -$ git annex initremote garage type=S3 encryption=none host=my-s3-endpoint.domain.com protocol=https bucket=git-annex requeststyle=path datacenter=garage -initremote garage (checking bucket...) (creating bucket in garage...) -git-annex: S3Error {s3StatusCode = Status {statusCode = 400, statusMessage = "Bad Request"}, s3ErrorCode = "InvalidRequest", s3ErrorMessage = "Bad request: Unsupported authorization method", s3ErrorResource = Just "/git-annex/", s3ErrorHostId = Nothing, s3ErrorAccessKeyId = Nothing, s3ErrorStringToSign = Nothing, s3ErrorBucket = Nothing, s3ErrorEndpointRaw = Nothing, s3ErrorEndpoint = Nothing} -failed -initremote: 1 failed -``` - -Garage appears to support v4 signatures: https://garagehq.deuxfleurs.fr/documentation/reference-manual/s3-compatibility/#high-level-features - and other S3 tooling works against the endpoint. - - -### What version of git-annex are you using? On what operating system? - -Fedora Silverblue 37 / git-annex-10.20221212-1.fc37.x86_64 - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Yes, many years ago - now trying to get it up and running with my self-hosted S3 endpoint. - -[1]: https://garagehq.deuxfleurs.fr/ - -> [[fixed|done]] although it needs git-annex to be built against -> a not yet released version of aws. --[[Joey]] diff --git a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_10_ea7a56193439eb63ad45074bd40f4ff5._comment b/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_10_ea7a56193439eb63ad45074bd40f4ff5._comment deleted file mode 100644 index 8b842d3707..0000000000 --- a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_10_ea7a56193439eb63ad45074bd40f4ff5._comment +++ /dev/null @@ -1,42 +0,0 @@ -[[!comment format=mdwn - username="jpds" - avatar="http://cdn.libravatar.org/avatar/24d746ec6a7726b162c12ecceb3ee267" - subject="comment 10" - date="2023-02-09T09:51:21Z" - content=""" -Thank you for the changes Joey, I can confirm that it's working and I've added this to the garage docs: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/502 - -I did however have to change git-annex's stack.yaml as follows to make this compile: - -``` -diff --git a/stack.yaml b/stack.yaml -index 7dbfb657a..cf2db080b 100644 ---- a/stack.yaml -+++ b/stack.yaml -@@ -11,10 +11,10 @@ flags: - gitlfs: true - packages: - - '.' --resolver: lts-18.13 -+resolver: lts-19.16 - extra-deps: - - IfElse-0.85 --- aws-0.22 -+- aws-0.24 - - bloomfilter-2.0.1.0 - - git-lfs-1.2.0 - - http-client-restricted-0.0.4 -@@ -24,4 +24,4 @@ extra-deps: - - base16-bytestring-0.1.1.7 - - base64-bytestring-1.0.0.3 - - bencode-0.6.1.1 --- http-client-0.7.9 -+- http-client-0.7.11 -``` - -On my machine, it wasn't happy about the aeson version without the lts bump and I also hit the bug at https://github.com/snoyberg/http-client/issues/482 hence the http-client change. - -> Thanks for clarifying that \"scope\" is internal jargon of garage though. - -I believe that this is actually an AWS thing, as it's extensively used in their Python and Rust SDKs: https://github.com/awslabs/aws-sdk-rust/blob/66423e05991ee831696bc32fe3e452694cf0d231/sdk/s3/src/config.rs#L98 -"""]] diff --git a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_1_1d5b499a0cea623aadebf7e3b7fd9752._comment b/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_1_1d5b499a0cea623aadebf7e3b7fd9752._comment deleted file mode 100644 index 94f0e7c110..0000000000 --- a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_1_1d5b499a0cea623aadebf7e3b7fd9752._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="jpds" - avatar="http://cdn.libravatar.org/avatar/24d746ec6a7726b162c12ecceb3ee267" - subject="comment 1" - date="2023-01-18T22:57:58Z" - content=""" -Error on Garage's side is triggered here: https://git.deuxfleurs.fr/Deuxfleurs/garage/src/commit/fcc5033466e58e3beec05ee7748d33522b6b32b0/src/api/signature/payload.rs#L297 -"""]] diff --git a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_2_ca7ffa315cfa49e028fe6ff2d5c3133b._comment b/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_2_ca7ffa315cfa49e028fe6ff2d5c3133b._comment deleted file mode 100644 index c19475c45d..0000000000 --- a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_2_ca7ffa315cfa49e028fe6ff2d5c3133b._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="jpds" - avatar="http://cdn.libravatar.org/avatar/24d746ec6a7726b162c12ecceb3ee267" - subject="comment 2" - date="2023-01-19T15:09:01Z" - content=""" -I took a look at the credentialv4 structure at https://github.com/aristidb/aws/blob/9bdc4ee018d0d9047c0434eeb21e2383afaa9ccf/Aws/Core.hs#L621 and found it curious that it has the region inside the scope (as the garage code) does... however in my error message from git-annex - the hostname of the S3 service is what's inside the scope instead of the 'garage' region name. - -I therefore adjusted the garage API's configuration to have the FQDN as the region and then... git-annex Just Worked. -"""]] diff --git a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_3_ae9308a3bab8904dd0f501cbe2f09de0._comment b/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_3_ae9308a3bab8904dd0f501cbe2f09de0._comment deleted file mode 100644 index c796318031..0000000000 --- a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_3_ae9308a3bab8904dd0f501cbe2f09de0._comment +++ /dev/null @@ -1,43 +0,0 @@ -[[!comment format=mdwn - username="jpds" - avatar="http://cdn.libravatar.org/avatar/24d746ec6a7726b162c12ecceb3ee267" - subject="comment 3" - date="2023-01-19T16:28:19Z" - content=""" -I believe the fix for this is: - -``` -diff --git a/Remote/S3.hs b/Remote/S3.hs -index f5014202e..49f2ebd58 100644 ---- a/Remote/S3.hs -+++ b/Remote/S3.hs -@@ -948,8 +948,8 @@ s3Configuration c = cfg - | otherwise -> AWS.HTTP - cfg = case getRemoteConfigValue signatureField c of - Just (SignatureVersion 4) -> -- S3.s3v4 proto endpoint False S3.SignWithEffort -- _ -> S3.s3 proto endpoint False -+ S3.s3v4 proto datacenter False S3.SignWithEffort -+ _ -> S3.s3 proto datacenter False - - data S3Info = S3Info - { bucket :: S3.Bucket -``` - -...however I cannot test it myself right now as it's failing to compile on another bit of code: - -``` -[452 of 679] Compiling Remote.S3 - -git/joeyh/git-annex.branchable.com/Remote/S3.hs:922:68: error: - • Couldn't match type ‘B8.ByteString’ with ‘[Char]’ - Expected type: String - Actual type: B8.ByteString - • In the first argument of ‘T.pack’, namely ‘datacenter’ - In the second argument of ‘($)’, namely ‘T.pack datacenter’ - In the expression: AWS.s3HostName $ T.pack datacenter - | -922 | | h == AWS.s3DefaultHost = AWS.s3HostName $ T.pack datacenter - | ^^^^^^^^^^ -``` -"""]] diff --git a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_4_f17018fbc7a8205a3ef0bdea74069fb9._comment b/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_4_f17018fbc7a8205a3ef0bdea74069fb9._comment deleted file mode 100644 index 4a3bdc1a7b..0000000000 --- a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_4_f17018fbc7a8205a3ef0bdea74069fb9._comment +++ /dev/null @@ -1,53 +0,0 @@ -[[!comment format=mdwn - username="jpds" - avatar="http://cdn.libravatar.org/avatar/24d746ec6a7726b162c12ecceb3ee267" - subject="comment 4" - date="2023-01-24T15:55:36Z" - content=""" -I think this can all be removed as only the 'region' should be in the `S3.*` calls: - -``` -diff --git a/Remote/S3.hs b/Remote/S3.hs -index f5014202e..d017378a8 100644 ---- a/Remote/S3.hs -+++ b/Remote/S3.hs -@@ -913,14 +913,7 @@ s3Configuration c = cfg - Nothing -> S3.s3RequestStyle cfg - } - where -- h = fromJust $ getRemoteConfigValue hostField c - datacenter = fromJust $ getRemoteConfigValue datacenterField c -- -- When the default S3 host is configured, connect directly to -- -- the S3 endpoint for the configured datacenter. -- -- When another host is configured, it's used as-is. -- endpoint -- | h == AWS.s3DefaultHost = AWS.s3HostName $ T.pack datacenter -- | otherwise = T.encodeUtf8 $ T.pack h - port = case getRemoteConfigValue portField c of - Just s -> - case reads s of -@@ -948,8 +941,8 @@ s3Configuration c = cfg - | otherwise -> AWS.HTTP - cfg = case getRemoteConfigValue signatureField c of - Just (SignatureVersion 4) -> -- S3.s3v4 proto endpoint False S3.SignWithEffort -- _ -> S3.s3 proto endpoint False -+ S3.s3v4 proto datacenter False S3.SignWithEffort -+ _ -> S3.s3 proto datacenter False - - data S3Info = S3Info - { bucket :: S3.Bucket -``` - -But now I get this as an error and I do not know why: - -``` -initremote garage (checking bucket...) (creating bucket in garage...) -git-annex: getRemoteConfigValue datacenter found value of unexpected type [Char]. This is a bug in git-annex! -CallStack (from HasCallStack): - error, called at ./Annex/SpecialRemote/Config.hs:179:28 in main:Annex.SpecialRemote.Config - getRemoteConfigValue, called at ./Remote/S3.hs:916:33 in main:Remote.S3 -failed -initremote: 1 failed -``` -"""]] diff --git a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_5_fb7a448a2a4231b08277741805c12c76._comment b/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_5_fb7a448a2a4231b08277741805c12c76._comment deleted file mode 100644 index 49eb41ff72..0000000000 --- a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_5_fb7a448a2a4231b08277741805c12c76._comment +++ /dev/null @@ -1,42 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2023-01-31T18:45:08Z" - content=""" -Firstly, this code is working, as far as I know, when accessing AWS. -And I want to be very careful to avoid breaking that. So patching all -that out would have to be very carefully examined and/or tested. - -I've verified that the s3Endpoint is supposed to be a hostname. It's -used as such in s3SignQuery: - - (host, path) = case s3RequestStyle of - PathStyle -> ([Just s3Endpoint], [Just "/", fmap (`B8.snoc` '/') s3QBucket, urlEncodedS3QObject]) - -If s3Endpoint were just "garage" that would break. - -s3SignQuery also has `region = s3ExtractRegion s3Endpoint`. -s3ExtractRegion parses a hostname like -"s3-foo.amazonaws.com" to "foo", and that is used as the region -(or "scope" as you've called it). - -git-annex makes sure to set s3Endpoint to a hostname. When the -default AWS hostname is used, it converts the datacenter=foo value -to a hostname like s3-foo.amazonaws.com and sets s3Endpoint to that. -When some other hostname= is provided, it set s3Endpoints to -that hostname, ignoring the datacenter= value. - -This seems like the only thing git-annex *can* do. Your proposed -patch would make a configuration of "datacenter=us-west-1" set -s3Endpoint to "us-west-1" and s3SignQuery would use that as the -host. - -So I think this is probably a bug or shortcoming of the aws library. -It seems that to fix this, the aws library would need to have -a way to specify a "scope" or "region" separate from the s3Endpoint. -And git-annex would then need to expose that as something other -than datacenter= - -I've filed an issue on aws about this: - -"""]] diff --git a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_6_513bdbb6595612432fd4f33bd8e4e737._comment b/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_6_513bdbb6595612432fd4f33bd8e4e737._comment deleted file mode 100644 index b63f80440f..0000000000 --- a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_6_513bdbb6595612432fd4f33bd8e4e737._comment +++ /dev/null @@ -1,45 +0,0 @@ -[[!comment format=mdwn - username="jpds" - avatar="http://cdn.libravatar.org/avatar/24d746ec6a7726b162c12ecceb3ee267" - subject="comment 6" - date="2023-01-31T22:28:16Z" - content=""" -> Firstly, this code is working, as far as I know, when accessing AWS. - -It's working, because presumably AWS accept 's3.amazon.com' as a region setting for the scope in their cloud. - -> I've verified that the s3Endpoint is supposed to be a hostname. It's used as such in s3SignQuery: -> -> (host, path) = case s3RequestStyle of -> PathStyle -> ([Just s3Endpoint], [Just \"/\", fmap (`B8.snoc` '/') s3QBucket, urlEncodedS3QObject]) -> -> If s3Endpoint were just \"garage\" that would break. - -My S3 endpoint is indeed the FQDN of my self-hosted S3 endpoint. My \"AWS region\" is \"garage\". - -To give exact examples, this is what I configure in my environment when interacting with garage with any other S3 utility: - -export AWS_ACCESS_KEY_ID=foo -export AWS_SECRET_ACCESS_KEY=bar -export AWS_DEFAULT_REGION='garage' -export AWS_ENDPOINT='https://my-s3-endpoint.example.com/' - -The region bit is what's added to the credentialV4 in Core.hs (and what AWS appear to also accept as s3.amazon.com(? - I do not have an AWS account to test this). To make git-annex work, I need to do the equivalent of this (after changing garage's configuration): - -export AWS_DEFAULT_REGION='my-s3-endpoint.example.com' - -Once I do this, awscli and every other tool breaks: - -$ aws s3 ls s3:// -Provided region_name 'my-s3-endpoint.example.com' doesn't match a supported format. - -> s3SignQuery also has region = s3ExtractRegion s3Endpoint. s3ExtractRegion parses a hostname like \"s3-foo.amazonaws.com\" to \"foo\", and that is used as the region (or \"scope\" as you've called it). -> -> git-annex makes sure to set s3Endpoint to a hostname. When the default AWS hostname is used, it converts the datacenter=foo value to a hostname like s3-foo.amazonaws.com and sets s3Endpoint to that. When some other hostname= is provided, it set s3Endpoints to that hostname, ignoring the datacenter= value. - -It might be best to deprecate datacenter= and add region= directly to match AWS parlance - -> This seems like the only thing git-annex can do. Your proposed patch would make a configuration of \"datacenter=us-west-1\" set s3Endpoint to \"us-west-1\" and s3SignQuery would use that as the host. - -What I'd really like to test is just being able to set datacenter=garage but with my change it's not accepted as a [Char]. -"""]] diff --git a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_7_131eb46d499d29b124c576653796e270._comment b/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_7_131eb46d499d29b124c576653796e270._comment deleted file mode 100644 index 6be643bda8..0000000000 --- a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_7_131eb46d499d29b124c576653796e270._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="jpds" - avatar="http://cdn.libravatar.org/avatar/24d746ec6a7726b162c12ecceb3ee267" - subject="comment 7" - date="2023-02-02T15:53:34Z" - content=""" -> s3ExtractRegion parses a hostname like \"s3-foo.amazonaws.com\" to \"foo\", and that is used as the region (or \"scope\" as you've called it). - -I'd also like to clarify this bit in that what I mean by \"scope\" is that line in the error message from the backend: - -``` -s3ErrorMessage = \"Authorization header malformed, expected scope: 20230118/my-s3-endpoint.domain.com/s3/aws4_request\" -``` - -git-annex is incorrectly setting the FQDN/endpoint of the service in this, instead of what garage and the linked Haskell library want here after the date which is the region. -"""]] diff --git a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_8_d8cdf46fb8de877ea167d14a055813e0._comment b/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_8_d8cdf46fb8de877ea167d14a055813e0._comment deleted file mode 100644 index ec62c54bbe..0000000000 --- a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_8_d8cdf46fb8de877ea167d14a055813e0._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 8""" - date="2023-02-06T16:51:58Z" - content=""" -> It's working, because presumably AWS accept 's3.amazon.com' as a region setting for the scope in their cloud. - -I doubt it; that's not what aws sends to it. - -Just to be clear, comment #5 is me fully understanding the problem -and escalating the issue to the relevant library. Further speculation -would probably muddy the water. - -Thanks for clarifying that "scope" is internal jargon of garage though. -Also it's good to know that `AWS_DEFAULT_REGION` is a commonly used -environment variable for it. -"""]] diff --git a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_9_3d2033edcd49e62045339e341996fd28._comment b/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_9_3d2033edcd49e62045339e341996fd28._comment deleted file mode 100644 index 046002c5c9..0000000000 --- a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_9_3d2033edcd49e62045339e341996fd28._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 9""" - date="2023-02-06T17:33:25Z" - content=""" -Implemented this in aws: https://github.com/aristidb/aws/pull/284 -Which should be released as version 0.24. - -git-annex will support region= when built with that version of aws. -"""]] diff --git a/doc/bugs/S3_special_remote_support_for_DigitalOcean_Spaces.mdwn b/doc/bugs/S3_special_remote_support_for_DigitalOcean_Spaces.mdwn deleted file mode 100644 index 72e22e7a19..0000000000 --- a/doc/bugs/S3_special_remote_support_for_DigitalOcean_Spaces.mdwn +++ /dev/null @@ -1,33 +0,0 @@ -I tried to use the S3 special remote to access DigitalOcean's Spaces API. [Their docs](https://developers.digitalocean.com/documentation/spaces/) suggest that this should be possible. However, it doesn't work. - -The command I ran, with key removed: - - git annex --debug initremote xamsi-everything type=S3 protocol=https host=sfo2.digitaloceanspaces.com datacenter=sfo2 chunk=64MiB encryption=hybrid keyid=XXX - -The non-debug output, in full, with key removed: - - initremote xamsi-everything (encryption setup) (to gpg keys: XXX) (checking bucket...) (creating bucket in sfo2...) - git-annex: XmlException {xmlErrorMessage = "Missing error Message"} - failed - git-annex: initremote: 1 failed - -The debug output of the part that breaks, again with key material removed: - - (creating bucket in sfo2...) [2019-10-15 08:40:41.119524792] String to sign: "PUT\n\n\nTue, 15 Oct 2019 15:40:41 GMT\n/xamsi-everything-a36e2044-07ac-4d85-8450-e5760c897a9b/" - [2019-10-15 08:40:41.119586065] Host: "xamsi-everything-a36e2044-07ac-4d85-8450-e5760c897a9b.sfo2.digitaloceanspaces.com" - [2019-10-15 08:40:41.119639648] Path: "/" - [2019-10-15 08:40:41.119683721] Query string: "" - [2019-10-15 08:40:41.11972899] Header: [("Date","Tue, 15 Oct 2019 15:40:41 GMT"),("Authorization","AWS XXX")] - [2019-10-15 08:40:41.119846915] Body: "sfo2" - [2019-10-15 08:40:41.174450718] Response status: Status {statusCode = 403, statusMessage = "Forbidden"} - [2019-10-15 08:40:41.174566002] Response header 'Content-Length': '190' - [2019-10-15 08:40:41.174627301] Response header 'x-amz-request-id': 'tx0000000000001c5b175eb-005da5e879-23e283-sfo2a' - [2019-10-15 08:40:41.174685597] Response header 'Accept-Ranges': 'bytes' - [2019-10-15 08:40:41.174730858] Response header 'Content-Type': 'application/xml' - [2019-10-15 08:40:41.174776256] Response header 'Date': 'Tue, 15 Oct 2019 15:40:41 GMT' - [2019-10-15 08:40:41.174821726] Response header 'Strict-Transport-Security': 'max-age=15552000; includeSubDomains; preload' - [2019-10-15 08:40:41.174984394] Response metadata: S3: request ID=tx0000000000001c5b175eb-005da5e879-23e283-sfo2a, x-amz-id-2= - -> This was either fixed by the signature=v4 support, or is a bug on digital -> ocean's side. Anyway, it's very old, and there has been no response, -> so closing. [[done]] --[[Joey]] diff --git a/doc/bugs/S3_special_remote_support_for_DigitalOcean_Spaces/comment_1_12fd9dfb47e157e1d38c5e88a543498b._comment b/doc/bugs/S3_special_remote_support_for_DigitalOcean_Spaces/comment_1_12fd9dfb47e157e1d38c5e88a543498b._comment deleted file mode 100644 index 56718b48e5..0000000000 --- a/doc/bugs/S3_special_remote_support_for_DigitalOcean_Spaces/comment_1_12fd9dfb47e157e1d38c5e88a543498b._comment +++ /dev/null @@ -1,20 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2020-01-06T17:41:21Z" - content=""" -This is either a bug in or -Digital Ocean's implementation of the S3 protocol. I don't know which, -but I do know it has to be fixed in one of those two place, and not -in git-annex. - -The fact that the aws library can't even parse an error message out of -their response kind of says something. - -But the real problem seems to -While they claim to support V2 signatures, -my guess is there's a problem with their support for V2, since they're -mostly dealing with V4. The aws library's support for V4 is experimental -and [apparently buggy](https://github.com/aristidb/aws/issues/262), but -as git-annex uses it, it will only use V2. -"""]] diff --git a/doc/bugs/S3_special_remote_support_for_DigitalOcean_Spaces/comment_2_3bbdf23c8a4a480f4f6b8e8a2f8ddecd._comment b/doc/bugs/S3_special_remote_support_for_DigitalOcean_Spaces/comment_2_3bbdf23c8a4a480f4f6b8e8a2f8ddecd._comment deleted file mode 100644 index 14bce58c32..0000000000 --- a/doc/bugs/S3_special_remote_support_for_DigitalOcean_Spaces/comment_2_3bbdf23c8a4a480f4f6b8e8a2f8ddecd._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2020-05-07T16:46:14Z" - content=""" -I have added to git-annex a way to use v4 authentication signatures. - -You will need a daily build, or the next release of git-annex. - -Give it a try by adding signature=v4 to your initremote -or enableremote, and please let me know if it works or how it fails with -that. -"""]] diff --git a/doc/bugs/SQlite_failed_when_copying_to_remote_repository.mdwn b/doc/bugs/SQlite_failed_when_copying_to_remote_repository.mdwn deleted file mode 100644 index 396c66312a..0000000000 --- a/doc/bugs/SQlite_failed_when_copying_to_remote_repository.mdwn +++ /dev/null @@ -1,82 +0,0 @@ -### Please describe the problem. - -When I copy file from on directory to another, I have the following error - -``` -$ git annex copy . --to another - -... -copy file1 (to another...) -100% 321.93 MiB 115 MiB/s 0ssqlite worker thread crashed: SQLite3 returned ErrorCan'tOpen while attempting to perform open "repo/.git/annex/keysdb/db". - sqlite query crashed: thread blocked indefinitely in an MVar operation - CallStack (from HasCallStack): - error, called at ./Database/Handle.hs:102:40 in main:Database.Handle - - transfer failed - -ok -copy file2 (to another...) -100% 424.24 MiB 109 MiB/s 0ssqlite worker thread crashed: SQLite3 returned ErrorCan'tOpen while attempting to perform open "repo/.git/annex/keysdb/db". - sqlite query crashed: thread blocked indefinitely in an MVar operation - CallStack (from HasCallStack): - error, called at ./Database/Handle.hs:102:40 in main:Database.Handle - - transfer failed - -ok - -... -``` - - -### What steps will reproduce the problem? - -Any time I copy files to this remote - - -### What version of git-annex are you using? On what operating system? - -On local machine: - -``` -$ git-annex version -git-annex version: 8.20210223 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 -local repository version: 8 - -$ uname -a -Linux kiwi 5.10.0-19-amd64 #1 SMP Debian 5.10.149-2 (2022-10-21) x86_64 GNU/Linux -``` - -On remote machine: - -``` -$ git-annex version -git-annex version: 8.20200617 -build flags: Assistant Webapp Pairing S3 WebDAV Kqueue DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.27 DAV-1.3.4 feed-1.3.0.1 ghc-8.10.3 http-client-0.7.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0.1 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs hook external -operating system: freebsd x86_64 -supported repository versions: 8 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 - -$ uname -a -FreeBSD extra-2 13.1-RELEASE-p2 FreeBSD 13.1-RELEASE-p2 n245412-484f039b1d0 TRUENAS amd64 -``` - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -I use git-annex for several years, and I'm very happy with it. I's one of the best files synchronisation tools. - - -[[!meta title="sqlite fails when repository path contains non-unicode"]] - -> [[fixed|done]], when git-annex is built against persistent-sqlite version -> 2.13.3. --[[Joey]] diff --git a/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_1_4578b04c1cfc46849e9440f257f49303._comment b/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_1_4578b04c1cfc46849e9440f257f49303._comment deleted file mode 100644 index 97292cc932..0000000000 --- a/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_1_4578b04c1cfc46849e9440f257f49303._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-01-31T17:22:24Z" - content=""" -Seems to me that you are not able to read the `repo/.git/annex/keysdb/db` file, -likely due to a permissions problem or because it is owned by another user. - -I was able to reproduce this by making the repository itself owned by -me, but making that specific file in it owned by a different user and not -readable by me: - - sqlite query crashed: thread blocked indefinitely in an MVar operation - CallStack (from HasCallStack): - error, called at ./Database/Handle.hs:82:40 in main:Database.Handle - -"""]] diff --git a/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_2_a953d5506f30cf19fe97c6ece155b45e._comment b/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_2_a953d5506f30cf19fe97c6ece155b45e._comment deleted file mode 100644 index 4787543417..0000000000 --- a/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_2_a953d5506f30cf19fe97c6ece155b45e._comment +++ /dev/null @@ -1,22 +0,0 @@ -[[!comment format=mdwn - username="hurlebouc" - avatar="http://cdn.libravatar.org/avatar/bda734a6d937c1fe0c9778a6eaefffbc" - subject="comment 2" - date="2023-02-01T12:26:04Z" - content=""" -Thank you for your anser - -I've checked user right and all seem ok for me: - -``` -$ ls -ln -total 7 --rw-r--r-- 1 1002 1002 28672 Dec 6 08:09 db - -$ id -u -1002 - -``` - -Thus for me the solution is somewhere else. -"""]] diff --git a/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_3_5b2956522ce9c9cdc0c704e63efe8dac._comment b/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_3_5b2956522ce9c9cdc0c704e63efe8dac._comment deleted file mode 100644 index a98ad3ce58..0000000000 --- a/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_3_5b2956522ce9c9cdc0c704e63efe8dac._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="hurlebouc" - avatar="http://cdn.libravatar.org/avatar/bda734a6d937c1fe0c9778a6eaefffbc" - subject="comment 3" - date="2023-02-21T06:49:27Z" - content=""" -Hi, - -Any idea explainig this failure? -"""]] diff --git a/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_4_3966cce3267fe134f5c9c6b05e1149b5._comment b/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_4_3966cce3267fe134f5c9c6b05e1149b5._comment deleted file mode 100644 index b2c5afc5ee..0000000000 --- a/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_4_3966cce3267fe134f5c9c6b05e1149b5._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2023-02-23T18:48:04Z" - content=""" -Version 8.20210223 is quite old, it would be good to try with a newer -version and see if the error message is different. - -You might also try strace and look to see what it says happens when it -tries to open "repo/.git/annex/keysdb/db". - -I don't know if what you showed about file permissions means you can really -read it. There might be ACLs or other complications? The best way to show -you can read it is to actually try to read it, eg `cat db >/dev/null` -"""]] - diff --git a/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_5_0c0aec23302afafffc0f69d1bd89ad8f._comment b/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_5_0c0aec23302afafffc0f69d1bd89ad8f._comment deleted file mode 100644 index ab8b037c00..0000000000 --- a/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_5_0c0aec23302afafffc0f69d1bd89ad8f._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="hurlebouc" - avatar="http://cdn.libravatar.org/avatar/bda734a6d937c1fe0c9778a6eaefffbc" - subject="comment 5" - date="2023-03-08T15:11:26Z" - content=""" -Hi, and thank you for keeping anwsering my question! - -I do succeed in reading this file. And here is the content: - - sqlite> .dump - PRAGMA foreign_keys=OFF; - BEGIN TRANSACTION; - CREATE TABLE IF NOT EXISTS \"associated\"(\"id\" INTEGER PRIMARY KEY,\"key\" BLOB NOT NULL,\"file\" BLOB NOT NULL,CONSTRAINT \"key_file_index\" UNIQUE (\"key\",\"file\"),CONSTRAINT \"file_key_index\" UNIQUE (\"file\",\"key\")); - CREATE TABLE IF NOT EXISTS \"content\"(\"id\" INTEGER PRIMARY KEY,\"key\" BLOB NOT NULL,\"inodecache\" VARCHAR NOT NULL,\"filesize\" VARCHAR NOT NULL,\"mtime\" INTEGER NOT NULL,CONSTRAINT \"key_inode_cache_index\" UNIQUE (\"key\",\"inodecache\"),CONSTRAINT \"inode_cache_key_index\" UNIQUE (\"inodecache\",\"key\")); - COMMIT; - sqlite> - -"""]] diff --git a/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_6_031f049fdc2c284303e9db42071ebda7._comment b/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_6_031f049fdc2c284303e9db42071ebda7._comment deleted file mode 100644 index 8dc022b11d..0000000000 --- a/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_6_031f049fdc2c284303e9db42071ebda7._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="hurlebouc" - avatar="http://cdn.libravatar.org/avatar/bda734a6d937c1fe0c9778a6eaefffbc" - subject="comment 6" - date="2023-03-08T15:16:28Z" - content=""" -Problem identified! - -The directory `repo` was actually containing \"é\" char. When I remove this char, problem disapeared. -"""]] diff --git a/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_7_3d4a7d330d80060275c34d0a13416fb7._comment b/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_7_3d4a7d330d80060275c34d0a13416fb7._comment deleted file mode 100644 index 6b6344a5b0..0000000000 --- a/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_7_3d4a7d330d80060275c34d0a13416fb7._comment +++ /dev/null @@ -1,78 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 7""" - date="2023-03-14T17:00:06Z" - content=""" -Oho, I reproduced it! - -First I made a git-annex repo named "fooé", and added a file "foo" to it, -and that is unlocked. - - joey@darkstar:/tmp>git clone fooé bar - Cloning into 'bar'... - done. - joey@darkstar:/tmp>cd bar - joey@darkstar:/tmp/bar>git-annex move --from origin - move foo (from origin...) - (recording state in git...) - ok - (recording state in git...) - joey@darkstar:/tmp/bar>git-annex copy --to origin - copy foo (to origin...) - (recording state in git...) - ok - (recording state in git...) - joey@darkstar:/tmp/bar>git-annex move --from origin - move foo (from origin...) (recording state in git...) - ok - (recording state in git...) - joey@darkstar:/tmp/bar>LANG=C git-annex copy --to origin - copy foo (to origin...) - - sqlite worker thread crashed: SQLite3 returned ErrorCan'tOpen while attempting to perform open "/tmp/foo\65533\65533/.git/annex/keysdb/db". - CallStack (from HasCallStack): - error, called at ./Database/Handle.hs:87:25 in main:Database.Handle - - ok - -Note that using git-annex in repo fooé with LANG=C works. The problem -seems limited to a remote with a unicode character in its name when not in a -unicode locale. - -In strace I see this: - - 3451696 openat(AT_FDCWD, "/tmp/foo\357\277\275\357\277\275/.git/annex/keysdb/db", O_RDWR|O_CREAT|O_NOFOLLOW|O_CLOEXEC, 0644 - 3451694 <... close resumed>) = 0 - 3451696 <... openat resumed>) = -1 ENOENT (No such file or directory) - -That doesn't look like the correct encoding for "é" does it? -`"/tmp/foo\303\251"` would be correct and is what git-annex otherwise uses -when accessing that repo. - -I think the reason for this is simply that persistent-sqlite uses Text -for the location of the database. And Text is unicode encoded. So when the non -unicode locale results in a FilePath that is encoded using the filesystem encoding, -with surrogate characters, and that gets fed to Data.Text.pack, it replaces the -"invalid scalar values" with "\65533". - -The same thing would happen in a unicode locale if the remote's path was not -valid unicode. - -Filed an issue to get persistent-sqlite to not use Text for the FilePath - - -I don't think that non-unicode FilePaths can be generally squeezed into a Text, -so we may need to wait persistent getting fixed. Although it should be possible, -in a non-unicode locale, to convert a non-unicode FilePath like "fooé" -to a Text. - ----- - -Also notice that git-annex succeeds despite this error. Which is reasonable -since it was only unable to update the remote's keys db, and the remote -can and will just update it itself next time git-annex is used over there. -Which will work since that git-annex will be running in the directory -and will use relative paths. - -So perhaps the error message could just be suppressed? -"""]] diff --git a/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_8_1501223d25a03c27a7ceffc6b0ea32a3._comment b/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_8_1501223d25a03c27a7ceffc6b0ea32a3._comment deleted file mode 100644 index c9fb1de251..0000000000 --- a/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_8_1501223d25a03c27a7ceffc6b0ea32a3._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="hurlebouc" - avatar="http://cdn.libravatar.org/avatar/bda734a6d937c1fe0c9778a6eaefffbc" - subject="comment 8" - date="2023-03-22T05:47:16Z" - content=""" -Thank you very much for the analysis! This reinforces my conviction that git-annex is a very high quality project. -"""]] diff --git a/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_9_192c219b32c95954ec6400367474dd78._comment b/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_9_192c219b32c95954ec6400367474dd78._comment deleted file mode 100644 index 34d7a3170d..0000000000 --- a/doc/bugs/SQlite_failed_when_copying_to_remote_repository/comment_9_192c219b32c95954ec6400367474dd78._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 9""" - date="2023-03-23T19:27:33Z" - content=""" -I sent in a pull request today to persistent-sqlite -that will let git-annex fix this once accepted. - - - -Once/if that is merged, git-annex can be changed to -use the new `open'` -"""]] diff --git a/doc/bugs/Strange_case_of_data_loss__44___possibly_linked_to_git-annex_with_encrypted_rsync_remote.mdwn b/doc/bugs/Strange_case_of_data_loss__44___possibly_linked_to_git-annex_with_encrypted_rsync_remote.mdwn deleted file mode 100644 index ffa9b5d676..0000000000 --- a/doc/bugs/Strange_case_of_data_loss__44___possibly_linked_to_git-annex_with_encrypted_rsync_remote.mdwn +++ /dev/null @@ -1,61 +0,0 @@ -This is not really a proper bug report, but I thought I should post this here -in case someone can find any sane, non-supernatural reason for a strange case -of data loss I have experienced with git-annex. - -Some time ago I cloned a bunch of git-annex repos from an external drive (let's -call it disk1) to a new computer (computer3). On one of my repos git-annex -marked a bunch of files corrupt and moved them to .git/annex/bad. Oops, I -thought, I must have a failing disk. Luckily I had offsite backups -- no less -than two other external hard disks (disk2-3), each having a full copy of the -repo in question. However, **both of these** had the same, corrupt files. The -files have the correct size, but are filled with zeroes. Other files in the -repo are fine, and so are other repos. - -I have been trying to wrap my head around this but I can't think of any reason -how this could occur. However the files have gotten corrupted in the first -place, the corruption should have been picked up when copying the content to -the external drives disk2 and disk3, right? I have to rule out NSA/MIB/aliens -from messing with me because these files are not that valuable or sensitive. - -The files in question were added to git-annex back in 2012, so the trail is -cold on this one. Naturally, I have no idea on how to reproduce this, nor can I -reliably say that git-annex is to blame. I can gather some hints though. The -files were all added on the same commit in 2012, but not all files from that -commit are corrupted. The corrupted files have consecutive file names. The -files were never modified since (except for the corruption), and the content -*may* have been copied via an encrypted rsync transfer repository. I have -always used git-annex on Arch Linux and in indirect more. The files used the -SHA-1 backend. - -All these files have a similar tracking log that looks something like this -(uuids replaced with symbolic names): - - 1356690700.542152s 1 computer1 <- first added - 1356691074.253815s 1 disk1 <- copied to disk1 - 1356719321.145126s 1 rsync <- copied to rsync repo - 1358070999.435676s 1 rsync <- copied to rsync repo (again?) - 1362166895.310332s 1 disk2 <- copied to disk2 - 1362906850.555869s 1 computer2 (dead) <- copied to another computer - 1364926664.362195s 0 computer1 <- dropped from computer1 as enough copies in disks - 1374412057.409496s 0 computer2 (dead) <- dropped from computer2, now dead - 1445691595.764108s 1 disk3 <- copied to disk3 - 1445770764.165792s 0 rsync <- dropped from rsync repo to save space - 1482077052.217353646s 0 disk1 <- first noticed as corrupted on disk1 - 1482741278.318274404s 0 disk3 <- WTF, also corrupted on disk3 - 1482926246.268440532s 0 disk2 <- double-WTF, also corrupted on disk2 - -The only thing that strikes odd to me is the double entry with the rsync -remote. The non-corrupted files from the same commit do not seem to have such a -double entry. - -So my main question is, has there ever been a bug in git-annex that could have -caused this behavior? Or is there any other realistic explanation for this? In -case this is an existing bug, is there any other evidence I can gather? -Needless to say, the lesson here is to run `git annex fsck` regularly even if -you have offsite backups... - -> My diagnosis is that the file got corrupted before it was copied to disk2 -> and disk3. What repository they reached them via does not matter much. -> And indeed, 5 year old git-annex didn't verify the content of -> files it transferred to/from a remote. Current git-annex does, so I guess -> this is [[done]]. --[[Joey]] diff --git a/doc/bugs/Strange_case_of_data_loss__44___possibly_linked_to_git-annex_with_encrypted_rsync_remote/comment_1_4038261a4d49694374c37ff029d3540f._comment b/doc/bugs/Strange_case_of_data_loss__44___possibly_linked_to_git-annex_with_encrypted_rsync_remote/comment_1_4038261a4d49694374c37ff029d3540f._comment deleted file mode 100644 index 7c448d6d7b..0000000000 --- a/doc/bugs/Strange_case_of_data_loss__44___possibly_linked_to_git-annex_with_encrypted_rsync_remote/comment_1_4038261a4d49694374c37ff029d3540f._comment +++ /dev/null @@ -1,20 +0,0 @@ -[[!comment format=mdwn - username="user4" - avatar="http://cdn.libravatar.org/avatar/cd0825fd95c541c0a48c7138b59c240a" - subject="remote.log" - date="2017-06-11T19:53:27Z" - content=""" -Since the most likely culprit (after aliens) is the encrypted rsync remote, here is the remote.log. I've replaced -sensitive information with variables in double curly braces - - {{uuid}} = cipher={{cipher}} cipherkeys={{keyid}} {{url}}= name={{name}} rsyncurl={{url}} type=rsync timestamp=1345706291.523459s - {{uuid}} = cipher={{cipher}} cipherkeys={{keyid}} {{url}}= name={{name}} rsyncurl={{url}} type=rsync timestamp=1482178826.391400069s - {{uuid}} = cipher={{cipher}} cipherkeys={{keyid}} {{url}}= name={{name}} rsyncurl={{url}} type=rsync timestamp=1482742412.058733547s - {{uuid}} = cipher={{cipher}} cipherkeys={{keyid}} {{url}}= name={{name}} rsyncurl={{url}} type=rsync timestamp=1482742686.603751605s - {{uuid}} cipher={{cipher}} cipherkeys={{keyid}} name={{name}} rsyncurl={{url}} type=rsync timestamp=1343199142.018003s - {{uuid}} cipher={{cipher}} cipherkeys={{keyid}} name={{name}} rsyncurl={{url}} type=rsync timestamp=1343199310.53916s - {{uuid}} cipher={{cipher}} cipherkeys={{keyid}} name={{name}} rsyncurl={{url}} type=rsync timestamp=1343486259.34504s - -Here {{uuid}} is the uuid of the encrypted rsync remote. It's odd that there are several lines for the same remote, and not all in the same format. -Could this be a PEBKAC involving a misconfigured rsync remote? -"""]] diff --git a/doc/bugs/Sync_fails_when_borg_remote_is_inaccessible.mdwn b/doc/bugs/Sync_fails_when_borg_remote_is_inaccessible.mdwn deleted file mode 100644 index 46cfc3daa4..0000000000 --- a/doc/bugs/Sync_fails_when_borg_remote_is_inaccessible.mdwn +++ /dev/null @@ -1,83 +0,0 @@ -### Please describe the problem. - -When a Borg special remote becomes inaccessible, git-annex sync interprets it as -a failed sync. This contrasts from the behavior of git-annex sync when a regular -git-annex repository is not available, where it'll be skipped instead. - -My workflow involves backup drives that aren't always connected, which can make -this behavior somewhat confusing to see. - -### What steps will reproduce the problem? - -1. Create a git-annex and BorgBackup repository. - -2. Set up borg repository as a special remote. - -3. Move borg repository to another location. - -4. Attempt to sync, expecting git-annex to skip the borg repository and exit - with a success regardless. Instead, git-annex exits with a failure. - -See below for a script that produces this error for me. - -### What version of git-annex are you using? On what operating system? - -Using the Guix package on Fedora. - -``` -git-annex version: 10.20230626 -build flags: Assistant Webapp Pairing Inotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.23 bloomfilter-2.0.1.0 cryptonite-0.30 DAV-1.3.4 feed-1.3.2.1 ghc-9.2.5 http-client-0.7.13.1 persistent-sqlite-2.13.1.0 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2.1 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 10 -``` - -### Please provide any additional information below. - -[[!format sh """ -#!/usr/bin/env sh - -# Initialize git-annex and borg repository - -git init repo -cd repo -git-annex init 'git-annex repository' -borg init --encryption=none ../repo.borg -git-annex initremote borg-repository type=borg borgrepo=../repo.borg - -# Sync works as expected when repository is found - -echo asdf > testfile -git-annex assist - -# Simulate borg repository becoming inaccessible, which results in a failed sync - -mv ../repo.borg ../repo.borg.tmp -git-annex sync -"""]] - -The following is what `git-annex sync` produces: - -``` -git-annex sync will change default behavior to operate on --content in a future version of git-annex. Recommend you explicitly use --no-content (or -g) to prepare for that change. (Or you can configure annex.synccontent) -commit -On branch master -nothing to commit, working tree clean -ok -list borg-repository Repository /home/alvin/Downloads/test/repo.borg does not exist. - -git-annex: Unable to list contents of borg-repository: user error (borg ["list","--format","{barchive}{NUL}","../repo.borg"] exited 2) -failed -sync: 1 failed -``` - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Been working on an addon command that I'll hopefully get quality enough to be -worth adding to the related software section soon :) - -> [[done]] --[[Joey]] diff --git a/doc/bugs/Sync_fails_when_borg_remote_is_inaccessible/comment_1_5e83630a60aa0242969bd1279a70628a._comment b/doc/bugs/Sync_fails_when_borg_remote_is_inaccessible/comment_1_5e83630a60aa0242969bd1279a70628a._comment deleted file mode 100644 index 70684e3064..0000000000 --- a/doc/bugs/Sync_fails_when_borg_remote_is_inaccessible/comment_1_5e83630a60aa0242969bd1279a70628a._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-08-15T16:59:11Z" - content=""" -Note that same thing happens when a directory special remote is configured -with importtree=yes exporttree=yes and the directory gets moved out of the -way. (Well, it does now.. Previously it failed to fail and imported an empty -tree, which was much worse a bug.) - -Failure to access the borg repo doesn't prevent the rest of the sync from -happening though. It still syncs with other remotes. - -Yes, there is a special case in sync to skip local git repositories that -are not present. Skipping borg and directory special remotes would need -that special case to be improved to an interface. -"""]] diff --git a/doc/bugs/Sync_fails_when_borg_remote_is_inaccessible/comment_2_eff73163e8a7ae09ceeb596fca04c507._comment b/doc/bugs/Sync_fails_when_borg_remote_is_inaccessible/comment_2_eff73163e8a7ae09ceeb596fca04c507._comment deleted file mode 100644 index 106c64be7b..0000000000 --- a/doc/bugs/Sync_fails_when_borg_remote_is_inaccessible/comment_2_eff73163e8a7ae09ceeb596fca04c507._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2023-08-16T20:01:21Z" - content=""" -Made sync skip many kinds of remotes that use a directory when the -directory does not exist. -"""]] diff --git a/doc/bugs/System.PosixCompat.User_removed_in_unix-compat-0.7.mdwn b/doc/bugs/System.PosixCompat.User_removed_in_unix-compat-0.7.mdwn deleted file mode 100644 index 542be432e0..0000000000 --- a/doc/bugs/System.PosixCompat.User_removed_in_unix-compat-0.7.mdwn +++ /dev/null @@ -1,60 +0,0 @@ -### Please describe the problem. - -Module `System.PosixCompat.User` has been removed in `unix-compat-0.7` (see -[changelog](https://hackage.haskell.org/package/unix-compat-0.7/changelog)). As -a result, git-annex failed to build, with the following error: - -``` -Starting git-annex-10.20230329 (all, legacy fallback) -Error: cabal: Failed to build git-annex-10.20230329. The failure occurred -during the configure step. The exception was: -/private/tmp/git-annex-20230329-55610-12n1hf4/git-annex-10.20230329/.brew_home/.cabal/logs/ghc-9.4.4/gt-nnx-10.20230329-579147b2.log: -withFile: user error (Error: cabal: '/opt/homebrew/opt/ghc/bin/ghc' exited -with an error: - -/private/tmp/cabal-install.-55709/dist-newstyle/tmp/src-55709/git-annex-10.20230329/Utility/UserInfo.hs:24:1: -error: -Could not find module ‘System.PosixCompat.User’ -Perhaps you meant -System.PosixCompat.Temp (from unix-compat-0.7) -System.PosixCompat.Time (from unix-compat-0.7) -System.PosixCompat.Files (from unix-compat-0.7) -Use -v (or `:set -v` in ghci) to see a list of the files searched for. -| -24 | import System.PosixCompat.User -| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -) -``` - -### What steps will reproduce the problem? - -``` -cabal v2-update -cabal v2-install --jobs=8 --max-backjumps=100000 --install-method=copy --installdir=/opt/homebrew/Cellar/git-annex/10.20230329/bin --flags=+S3 -``` - -(Note: I omitted some workarounds used to build with GHC >= 9.2. The full -package description for building git-annex can be found -[here](https://github.com/Homebrew/homebrew-core/blob/83f9beeb6ce6d44cd06856f4e9fc513e80cd237d/Formula/git-annex.rb).) - -### What version of git-annex are you using? On what operating system? - -git-annex: 10.20230329 (But it failed with 10.20230321, too.) - -OS: macOS 11, 12, 13 (x86_64 and arm64), Ubuntu 22.04 (x86_64) - -### Please provide any additional information below. - -The error was observed while packaging git-annex for Homebrew -[here](https://github.com/Homebrew/homebrew-core/pull/127002). Currently, that's -being worked around by restricting `unix-compat` version to `>= 0.5 && < 0.7` in -`git-annex.cabal`. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Sorry, I'm not a git-annex user. I'm a maintainer of the Homebrew package -manager, and I help to make the newest git-annex available to our users. - -Thanks for all your work maintaining git-annex! - -> [[fixed|done]] by avoiding the broken version --[[Joey]] diff --git a/doc/bugs/System.PosixCompat.User_removed_in_unix-compat-0.7/comment_1_cd573b785b7d7feec72387cb0dafdcab._comment b/doc/bugs/System.PosixCompat.User_removed_in_unix-compat-0.7/comment_1_cd573b785b7d7feec72387cb0dafdcab._comment deleted file mode 100644 index 9894d24ea6..0000000000 --- a/doc/bugs/System.PosixCompat.User_removed_in_unix-compat-0.7/comment_1_cd573b785b7d7feec72387cb0dafdcab._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-03-31T16:50:28Z" - content=""" -Unfortunately pinning to 0.6 is the only solution, I cannot work around -this ill-considered change in git-annex. I have opened an issue and hope -the maintainers reconsider. - -"""]] diff --git a/doc/bugs/System.PosixCompat.User_removed_in_unix-compat-0.7/comment_2_6b15d11b6e25689e2430663a4aa90168._comment b/doc/bugs/System.PosixCompat.User_removed_in_unix-compat-0.7/comment_2_6b15d11b6e25689e2430663a4aa90168._comment deleted file mode 100644 index 2048e072f1..0000000000 --- a/doc/bugs/System.PosixCompat.User_removed_in_unix-compat-0.7/comment_2_6b15d11b6e25689e2430663a4aa90168._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="zhongruoyu@80ae9772857666624009364c29f07c70beed46ac" - nickname="zhongruoyu" - avatar="http://cdn.libravatar.org/avatar/b1c7aaba6e8b09ef40ca161135d3b14b" - subject="comment 2" - date="2023-03-31T18:32:29Z" - content=""" -Thanks for doing that. I cannot comment on the `unix-compat` change, because I'm not familiar with it. But let me use the fix you've just committed and keep an eye on the updates, if any. - -"""]] diff --git a/doc/bugs/Tests_v8_locked__58___rsync_remote__58___FAIL.mdwn b/doc/bugs/Tests_v8_locked__58___rsync_remote__58___FAIL.mdwn deleted file mode 100644 index d7137d7cce..0000000000 --- a/doc/bugs/Tests_v8_locked__58___rsync_remote__58___FAIL.mdwn +++ /dev/null @@ -1,43 +0,0 @@ -### Please describe the problem. - -``` -2022-04-28T03:40:48.4460611Z Repo Tests v8 locked -2022-04-28T03:40:48.4460898Z Init Tests -2022-04-28T03:40:48.4461239Z init: OK (0.08s) -2022-04-28T03:40:48.4461605Z add: OK (0.31s) -2022-04-28T03:40:48.4461985Z preferred content: OK (1.26s) -2022-04-28T03:40:48.4462414Z rsync remote: FAIL (0.63s) -``` - -across setups: - -``` -(git)smaug:/mnt/datasets/datalad/ci/git-annex/builds/2022/04[master]cron-20220428 -$> git grep -n FAIL| grep test-annex -build-macos.yaml-670-0346631d-failed/1_test-annex (normal, macos-10.15).txt:1580:2022-04-28T02:40:19.9278580Z rsync remote: FAIL (1.44s) -build-macos.yaml-670-0346631d-failed/2_test-annex (crippled-tmp, macos-10.15).txt:1692:2022-04-28T02:44:48.5060190Z rsync remote: FAIL (1.41s) -build-macos.yaml-670-0346631d-failed/3_test-annex (normal, macos-latest).txt:1581:2022-04-28T02:40:39.2774670Z rsync remote: FAIL (1.32s) -build-macos.yaml-670-0346631d-failed/test-annex (crippled-tmp, macos-10.15)/8_Run tests.txt:1425:2022-04-28T02:44:48.5060140Z rsync remote: FAIL (1.41s) -build-macos.yaml-670-0346631d-failed/test-annex (normal, macos-10.15)/8_Run tests.txt:1314:2022-04-28T02:40:19.9278580Z rsync remote: FAIL (1.44s) -build-macos.yaml-670-0346631d-failed/test-annex (normal, macos-latest)/8_Run tests.txt:1314:2022-04-28T02:40:39.2774650Z rsync remote: FAIL (1.32s) -build-ubuntu.yaml-675-0346631d-failed/1_test-annex (normal, ubuntu-latest).txt:1427:2022-04-28T03:40:48.4462418Z rsync remote: FAIL (0.63s) -build-ubuntu.yaml-675-0346631d-failed/2_test-annex (crippled-tmp, ubuntu-latest).txt:1577:2022-04-28T03:41:44.8645593Z rsync remote: FAIL (0.64s) -build-ubuntu.yaml-675-0346631d-failed/4_test-annex (nfs-home, ubuntu-latest).txt:1513:2022-04-28T03:45:52.6285760Z rsync remote: FAIL (1.56s) -build-ubuntu.yaml-675-0346631d-failed/test-annex (crippled-tmp, ubuntu-latest)/7_Run tests.txt:1334:2022-04-28T03:41:44.8645589Z rsync remote: FAIL (0.64s) -build-ubuntu.yaml-675-0346631d-failed/test-annex (nfs-home, ubuntu-latest)/7_Run tests.txt:1270:2022-04-28T03:45:52.6285755Z rsync remote: FAIL (1.56s) -build-ubuntu.yaml-675-0346631d-failed/test-annex (normal, ubuntu-latest)/7_Run tests.txt:1185:2022-04-28T03:40:48.4462414Z rsync remote: FAIL (0.63s) - -``` - -and for some time already (as pinged in an email) -- since 20220420 - - -``` -$> git grep -n rsync.*FAIL| grep test-annex | grep ubuntu-late -cron-20220420/build-ubuntu.yaml-667-0346631d-failed/1_test-annex (normal, ubuntu-latest).txt:1427:2022-04-20T03:30:39.0198567Z rsync remote: FAIL (0.70s) -cron-20220420/build-ubuntu.yaml-667-0346631d-failed/2_test-annex (crippled-tmp, ubuntu-latest).txt:1582:2022-04-20T03:31:17.9571887Z rsync remote: FAIL (0.62s) -cron-20220420/build-ubuntu.yaml-667-0346631d-failed/4_test-annex (nfs-home, ubuntu-latest).txt:1511:2022-04-20T03:34:57.9524586Z rsync remote: FAIL (1.57s) -... -``` - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/Tests_v8_locked__58___rsync_remote__58___FAIL/comment_1_809ca8b986ccadbc3940cc795d1823e8._comment b/doc/bugs/Tests_v8_locked__58___rsync_remote__58___FAIL/comment_1_809ca8b986ccadbc3940cc795d1823e8._comment deleted file mode 100644 index 4ce0ff8c8a..0000000000 --- a/doc/bugs/Tests_v8_locked__58___rsync_remote__58___FAIL/comment_1_809ca8b986ccadbc3940cc795d1823e8._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-05-02T16:53:13Z" - content=""" -The failure message is: - - ./Test/Framework.hs:328: - able to modify annexed file's foo content - -Apparently some change to rsync in 3.2.4 has messed -up something involving permissions. - -See also another bug caused by the new rsync -"""]] diff --git a/doc/bugs/Tests_v8_locked__58___rsync_remote__58___FAIL/comment_2_0327663314d2a8b2f0cab7536fdaa6bd._comment b/doc/bugs/Tests_v8_locked__58___rsync_remote__58___FAIL/comment_2_0327663314d2a8b2f0cab7536fdaa6bd._comment deleted file mode 100644 index c6aa04f30b..0000000000 --- a/doc/bugs/Tests_v8_locked__58___rsync_remote__58___FAIL/comment_2_0327663314d2a8b2f0cab7536fdaa6bd._comment +++ /dev/null @@ -1,32 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-05-02T17:08:15Z" - content=""" -Here is the bug in action: - - -r--r--r-- 1 joey joey 30 May 2 12:42 .git/annex/objects/Gj/8J/SHA256E-s30--3d65cafd9435fde3867a527d75ff8aea05a3632cb60574d45e0fc277f06c8a64/SHA256E-s30--3d65cafd9435fde3867a527d75ff8aea05a3632cb60574d45e0fc277f06c8a64 - joey@darkstar:/tmp/x>git-annex copy --to foo - copy x (to foo...) - ok - joey@darkstar:/tmp/x>ls -l .git/annex/objects/Gj/8J/SHA256E-s30--3d65cafd9435fde3867a527d75ff8aea05a3632cb60574d45e0fc277f06c8a64/SHA256E-s30--3d65cafd9435fde3867a527d75ff8aea05a3632cb60574d45e0fc277f06c8a64 - -rwxr--r-- 1 joey joey 30 May 2 12:42 .git/annex/objects/Gj/8J/SHA256E-s30--3d65cafd9435fde3867a527d75ff8aea05a3632cb60574d45e0fc277f06c8a64/SHA256E-s30--3d65cafd9435fde3867a527d75ff8aea05a3632cb60574d45e0fc277f06c8a64* - -At first I thought this was rsync modifying the permissions of the source file. - -But no... [[!commit 17b20a24502aee3bfc5683146c3899a233295aea]] -changed how temp directories get cleaned up. removePathForcibly -is actually changing the permissions of the object file hard link -in the rsynctmp directory when deleting that directory. Which also -changes the permissions of the object file. - -Filed a bug on removePathForcibly. - -I think this makes removePathForcibly unsuitable for general purpose -use in git-annex, because there are just too many ways for a hard link -to enter the picture. (Eg annex.thin, or even a user making their own -hard link that git-annex does not know about.) - -So, I've reverted that commit, and put in a more -targeted fix for the problem it was addressing. -"""]] diff --git a/doc/bugs/Utility.Url.Parse_fails_to_build_on_Win___40__imports__41__.mdwn b/doc/bugs/Utility.Url.Parse_fails_to_build_on_Win___40__imports__41__.mdwn deleted file mode 100644 index d4dd1b5474..0000000000 --- a/doc/bugs/Utility.Url.Parse_fails_to_build_on_Win___40__imports__41__.mdwn +++ /dev/null @@ -1,96 +0,0 @@ -### Please describe the problem. - -The removal of some imports causes Utility.Url.Parse to fail to build on Windows. - -### What steps will reproduce the problem? - -`stack --stack-yaml stack-lts-18.13.yaml setup && stack --stack-yaml stack-lts-18.13.yaml build` - -Observe the following output: - -[[!format sh """ -[...] -C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-230817-83056e7b5\Utility\Terminal.hs:43:41: warning: [-Wunused-matches] - Defined but not used: b' - | -43 | b' <- isMinTTYHandle h' - | ^^ -[106 of 689] Compiling Utility.ThreadLock -[107 of 689] Compiling Utility.ThreadScheduler -[108 of 689] Compiling Utility.TimeStamp -[109 of 689] Compiling Annex.VectorClock.Utility -[110 of 689] Compiling Utility.Tmp -[111 of 689] Compiling Utility.MoveFile -[112 of 689] Compiling Utility.Tmp.Dir -[113 of 689] Compiling Utility.Touch -[114 of 689] Compiling Utility.Tuple -[115 of 689] Compiling Utility.Scheduled -[116 of 689] Compiling Utility.Scheduled.QuickCheck -[117 of 689] Compiling Utility.Url.Parse - -C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-230817-83056e7b5\Utility\Url\Parse.hs:34:19: error: - Variable not in scope: isPrefixOf :: t0 -> String -> Bool - | -34 | | "file:" `isPrefixOf` s = do - | ^^^^^^^^^^^^ - -Error: [S-7282] - Stack failed to execute the build plan. - - While executing the build plan, Stack encountered the error: - - [S-7011] - While building package git-annex-10.20230802 (scroll up to its section to see the error) - using: - C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-230817-83056e7b5\.stack-work\dist\274b403a\setup\setup --verbose=1 --builddir=.stack-work\dist\274b403a build exe:git-annex --ghc-options "" - Process exited with code: ExitFailure 1 - -# End of transcript or log. -"""]] - -### What version of git-annex are you using? On what operating system? - -[[!format sh """ -git-annex version: 10.20230803-g83056e7b53 -build flags: Assistant Webapp Pairing TorrentParser Benchmark Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.7 http-client-0.7.9 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.1.2 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: mingw32 x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 2 3 4 5 6 7 8 9 10 -"""]] - -A successful build that passes the testsuite on Windows is achieved with the patch below. - -Windows 10 version 22H2 (build 19045.3324), 64 bit. - -### Please provide any additional information below. - -Apply the following patch to build on Windows: - -[[!format diff """ -diff --git a/Utility/Url/Parse.hs b/Utility/Url/Parse.hs -index 7fc952b034..ab456ed31e 100644 ---- a/Utility/Url/Parse.hs -+++ b/Utility/Url/Parse.hs -@@ -18,6 +18,7 @@ module Utility.Url.Parse ( - - import Network.URI - #ifdef mingw32_HOST_OS -+import Common - import qualified System.FilePath.Windows as PW - #endif - -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Sure, Git Annex is great as always. I use it several times a week with my multigigabyte -backups, where it gives structure to my image-based backup routines, so you could -say I'm a believer. :) - -[[!meta title="Utility.Url.Parse fails to build on Windows (missing import)"]] -[[!meta author=jkniiv]] - -> [[fixed|done]], thank you! --[[Joey]] diff --git a/doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects.mdwn b/doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects.mdwn deleted file mode 100644 index f73238565a..0000000000 --- a/doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects.mdwn +++ /dev/null @@ -1,399 +0,0 @@ -### Please describe the problem. - -When exporting commits to S3 with versioning=yes and exporttree=yes, deleted annexed objects are marked as deleted but deleted git objects remain. This prevents a user from retrieving a consistent version of the exporttree commit without using git-annex. - -### What steps will reproduce the problem? - -* Create a repository and add annexed and git objects. -* Add S3 special remote with exporttree=yes and versioning=yes -* Export to the remote. -* Remove a git object and an annexed object. -* Export again and the annexed object will be removed while the git object remains. - -Example repository and publicly readable remote having this issue here: - -https://github.com/openneurodatasets/ds001705 - -Git files in the 1.0.0 tag are still present in the S3 1.0.1 export. sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json is an example file not present in 1.0.1 that is still present on S3. - -### What version of git-annex are you using? On what operating system? - -Debian bullseye - -``` -git-annex version: 10.20220927-ga92546587 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 10 -``` - -### Please provide any additional information below. - -[[!format sh """ -# git annex export 1.0.0 --to=s3-PUBLIC -export s3-PUBLIC .bidsignore -ok -export s3-PUBLIC .datalad/.gitattributes -ok -export s3-PUBLIC .datalad/config -ok -export s3-PUBLIC .gitattributes -ok -export s3-PUBLIC CHANGES -ok -export s3-PUBLIC README -ok -export s3-PUBLIC dataset_description.json -ok -export s3-PUBLIC sub-000101/ses-baseline/anat/sub-000101_ses-baseline_T1w.nii.gz -ok -export s3-PUBLIC sub-000101/ses-baseline/anat/sub-000101_ses-baseline_label-displacementROI_dseg.nii.gz -ok -export s3-PUBLIC sub-000101/ses-baseline/pet/.bidsignore -ok -export s3-PUBLIC sub-000101/ses-baseline/pet/sub-000101_ses-baseline_K1.nii.gz -ok -export s3-PUBLIC sub-000101/ses-baseline/pet/sub-000101_ses-baseline_Vb.nii.gz -ok -export s3-PUBLIC sub-000101/ses-baseline/pet/sub-000101_ses-baseline_k2.nii.gz -ok -export s3-PUBLIC sub-000101/ses-baseline/pet/sub-000101_ses-baseline_k3.nii.gz -ok -export s3-PUBLIC sub-000101/ses-baseline/pet/sub-000101_ses-baseline_k4.nii.gz -ok -export s3-PUBLIC sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json -ok -export s3-PUBLIC sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.nii.gz -ok -export s3-PUBLIC sub-000101/ses-displaced/anat/sub-000101_ses-displaced_T1w.nii.gz -ok -export s3-PUBLIC sub-000101/ses-displaced/anat/sub-000101_ses-displaced_label-displacementROI_dseg.nii.gz -ok -export s3-PUBLIC sub-000101/ses-displaced/pet/.bidsignore -ok -export s3-PUBLIC sub-000101/ses-displaced/pet/sub-000101_ses-displaced_k3.nii.gz -ok -export s3-PUBLIC sub-000101/ses-displaced/pet/sub-000101_ses-displaced_rec-MLEM_pet.json -ok -export s3-PUBLIC sub-000101/ses-displaced/pet/sub-000101_ses-displaced_rec-MLEM_pet.nii.gz -ok -export s3-PUBLIC sub-000102/ses-baseline/anat/sub-000102_ses-baseline_T1w.nii.gz -ok -export s3-PUBLIC sub-000102/ses-baseline/anat/sub-000102_ses-baseline_label-displacementROI_dseg.nii.gz -ok -export s3-PUBLIC sub-000102/ses-baseline/pet/.bidsignore -ok -export s3-PUBLIC sub-000102/ses-baseline/pet/sub-000102_ses-baseline_K1.nii.gz -ok -export s3-PUBLIC sub-000102/ses-baseline/pet/sub-000102_ses-baseline_Vb.nii.gz -ok -export s3-PUBLIC sub-000102/ses-baseline/pet/sub-000102_ses-baseline_k2.nii.gz -ok -export s3-PUBLIC sub-000102/ses-baseline/pet/sub-000102_ses-baseline_k3.nii.gz -ok -export s3-PUBLIC sub-000102/ses-baseline/pet/sub-000102_ses-baseline_k4.nii.gz -ok -export s3-PUBLIC sub-000102/ses-baseline/pet/sub-000102_ses-baseline_rec-MLEM_pet.json -ok -export s3-PUBLIC sub-000102/ses-baseline/pet/sub-000102_ses-baseline_rec-MLEM_pet.nii.gz -ok -export s3-PUBLIC sub-000102/ses-displaced/anat/sub-000102_ses-displaced_T1w.nii.gz -ok -export s3-PUBLIC sub-000102/ses-displaced/anat/sub-000102_ses-displaced_label-displacementROI_dseg.nii.gz -ok -export s3-PUBLIC sub-000102/ses-displaced/pet/.bidsignore -ok -export s3-PUBLIC sub-000102/ses-displaced/pet/sub-000102_ses-displaced_k3.nii.gz -ok -export s3-PUBLIC sub-000102/ses-displaced/pet/sub-000102_ses-displaced_rec-MLEM_pet.json -ok -export s3-PUBLIC sub-000102/ses-displaced/pet/sub-000102_ses-displaced_rec-MLEM_pet.nii.gz -ok -export s3-PUBLIC sub-000103/ses-baseline/anat/sub-000103_ses-baseline_T1w.nii.gz -ok -export s3-PUBLIC sub-000103/ses-baseline/anat/sub-000103_ses-baseline_label-displacementROI_dseg.nii.gz -ok -export s3-PUBLIC sub-000103/ses-baseline/pet/.bidsignore -ok -export s3-PUBLIC sub-000103/ses-baseline/pet/sub-000103_ses-baseline_K1.nii.gz -ok -export s3-PUBLIC sub-000103/ses-baseline/pet/sub-000103_ses-baseline_Vb.nii.gz -ok -export s3-PUBLIC sub-000103/ses-baseline/pet/sub-000103_ses-baseline_k2.nii.gz -ok -export s3-PUBLIC sub-000103/ses-baseline/pet/sub-000103_ses-baseline_k3.nii.gz -ok -export s3-PUBLIC sub-000103/ses-baseline/pet/sub-000103_ses-baseline_k4.nii.gz -ok -export s3-PUBLIC sub-000103/ses-baseline/pet/sub-000103_ses-baseline_rec-MLEM_pet.json -ok -export s3-PUBLIC sub-000103/ses-baseline/pet/sub-000103_ses-baseline_rec-MLEM_pet.nii.gz -ok -export s3-PUBLIC sub-000103/ses-displaced/anat/sub-000103_ses-displaced_T1w.nii.gz -ok -export s3-PUBLIC sub-000103/ses-displaced/anat/sub-000103_ses-displaced_label-displacementROI_dseg.nii.gz -ok -export s3-PUBLIC sub-000103/ses-displaced/pet/.bidsignore -ok -export s3-PUBLIC sub-000103/ses-displaced/pet/sub-000103_ses-displaced_k3.nii.gz -ok -export s3-PUBLIC sub-000103/ses-displaced/pet/sub-000103_ses-displaced_rec-MLEM_pet.json -ok -export s3-PUBLIC sub-000103/ses-displaced/pet/sub-000103_ses-displaced_rec-MLEM_pet.nii.gz -ok -export s3-PUBLIC sub-000104/ses-baseline/anat/sub-000104_ses-baseline_T1w.nii.gz -ok -export s3-PUBLIC sub-000104/ses-baseline/anat/sub-000104_ses-baseline_label-displacementROI_dseg.nii.gz -ok -export s3-PUBLIC sub-000104/ses-baseline/pet/.bidsignore -ok -export s3-PUBLIC sub-000104/ses-baseline/pet/sub-000104_ses-baseline_K1.nii.gz -ok -export s3-PUBLIC sub-000104/ses-baseline/pet/sub-000104_ses-baseline_Vb.nii.gz -ok -export s3-PUBLIC sub-000104/ses-baseline/pet/sub-000104_ses-baseline_k2.nii.gz -ok -export s3-PUBLIC sub-000104/ses-baseline/pet/sub-000104_ses-baseline_k3.nii.gz -ok -export s3-PUBLIC sub-000104/ses-baseline/pet/sub-000104_ses-baseline_k4.nii.gz -ok -export s3-PUBLIC sub-000104/ses-baseline/pet/sub-000104_ses-baseline_rec-MLEM_pet.json -ok -export s3-PUBLIC sub-000104/ses-baseline/pet/sub-000104_ses-baseline_rec-MLEM_pet.nii.gz -ok -export s3-PUBLIC sub-000104/ses-displaced/anat/sub-000104_ses-displaced_T1w.nii.gz -ok -export s3-PUBLIC sub-000104/ses-displaced/anat/sub-000104_ses-displaced_label-displacementROI_dseg.nii.gz -ok -export s3-PUBLIC sub-000104/ses-displaced/pet/.bidsignore -ok -export s3-PUBLIC sub-000104/ses-displaced/pet/sub-000104_ses-displaced_k3.nii.gz -ok -export s3-PUBLIC sub-000104/ses-displaced/pet/sub-000104_ses-displaced_rec-MLEM_pet.json -ok -export s3-PUBLIC sub-000104/ses-displaced/pet/sub-000104_ses-displaced_rec-MLEM_pet.nii.gz -ok -export s3-PUBLIC sub-000105/ses-baseline/anat/sub-000105_ses-baseline_T1w.nii.gz -ok -export s3-PUBLIC sub-000105/ses-baseline/anat/sub-000105_ses-baseline_label-displacementROI_dseg.nii.gz -ok -export s3-PUBLIC sub-000105/ses-baseline/pet/.bidsignore -ok -export s3-PUBLIC sub-000105/ses-baseline/pet/sub-000105_ses-baseline_K1.nii.gz -ok -export s3-PUBLIC sub-000105/ses-baseline/pet/sub-000105_ses-baseline_Vb.nii.gz -ok -export s3-PUBLIC sub-000105/ses-baseline/pet/sub-000105_ses-baseline_k2.nii.gz -ok -export s3-PUBLIC sub-000105/ses-baseline/pet/sub-000105_ses-baseline_k3.nii.gz -ok -export s3-PUBLIC sub-000105/ses-baseline/pet/sub-000105_ses-baseline_k4.nii.gz -ok -export s3-PUBLIC sub-000105/ses-baseline/pet/sub-000105_ses-baseline_rec-MLEM_pet.json -ok -export s3-PUBLIC sub-000105/ses-baseline/pet/sub-000105_ses-baseline_rec-MLEM_pet.nii.gz -ok -export s3-PUBLIC sub-000105/ses-displaced/anat/sub-000105_ses-displaced_T1w.nii.gz -ok -export s3-PUBLIC sub-000105/ses-displaced/anat/sub-000105_ses-displaced_label-displacementROI_dseg.nii.gz -ok -export s3-PUBLIC sub-000105/ses-displaced/pet/.bidsignore -ok -export s3-PUBLIC sub-000105/ses-displaced/pet/sub-000105_ses-displaced_k3.nii.gz -ok -export s3-PUBLIC sub-000105/ses-displaced/pet/sub-000105_ses-displaced_rec-MLEM_pet.json -ok -export s3-PUBLIC sub-000105/ses-displaced/pet/sub-000105_ses-displaced_rec-MLEM_pet.nii.gz -ok -(recording state in git...) - -# git annex export 1.0.1 --to=s3-PUBLIC -unexport s3-PUBLIC sub-000105/ses-baseline/pet/sub-000105_ses-baseline_K1.nii.gz ok -unexport s3-PUBLIC sub-000103/ses-baseline/pet/sub-000103_ses-baseline_k2.nii.gz ok -unexport s3-PUBLIC sub-000104/ses-baseline/pet/sub-000104_ses-baseline_k3.nii.gz ok -unexport s3-PUBLIC .bidsignore ok -unexport s3-PUBLIC sub-000101/ses-baseline/anat/sub-000101_ses-baseline_T1w.nii.gz ok -rename s3-PUBLIC sub-000101/ses-baseline/anat/sub-000101_ses-baseline_label-displacementROI_dseg.nii.gz -> .git-annex-tmp-content-MD5E-s9793--13db6c96808070211b8987c327240ff8.nii.gz ok -unexport s3-PUBLIC sub-000101/ses-baseline/pet/sub-000101_ses-baseline_Vb.nii.gz ok -unexport s3-PUBLIC sub-000102/ses-baseline/pet/sub-000102_ses-baseline_K1.nii.gz ok -unexport s3-PUBLIC sub-000102/ses-baseline/pet/sub-000102_ses-baseline_Vb.nii.gz ok -unexport s3-PUBLIC sub-000104/ses-baseline/anat/sub-000104_ses-baseline_T1w.nii.gz ok -unexport s3-PUBLIC sub-000104/ses-displaced/pet/sub-000104_ses-displaced_rec-MLEM_pet.nii.gz ok -unexport s3-PUBLIC sub-000105/ses-baseline/pet/sub-000105_ses-baseline_k4.nii.gz ok -unexport s3-PUBLIC sub-000102/ses-baseline/pet/sub-000102_ses-baseline_k3.nii.gz ok -unexport s3-PUBLIC sub-000103/ses-displaced/anat/sub-000103_ses-displaced_T1w.nii.gz ok -unexport s3-PUBLIC sub-000105/ses-displaced/pet/sub-000105_ses-displaced_k3.nii.gz ok -rename s3-PUBLIC sub-000102/ses-displaced/anat/sub-000102_ses-displaced_label-displacementROI_dseg.nii.gz -> .git-annex-tmp-content-MD5E-s9794--419a63ecba6de7235554c1f5a7fbc7d8.nii.gz ok -rename s3-PUBLIC sub-000102/ses-baseline/anat/sub-000102_ses-baseline_label-displacementROI_dseg.nii.gz -> .git-annex-tmp-content-MD5E-s9793--45358ccfe2d8a7455c3202691ee632eb.nii.gz ok -unexport s3-PUBLIC sub-000104/ses-baseline/pet/sub-000104_ses-baseline_rec-MLEM_pet.nii.gz ok -rename s3-PUBLIC sub-000105/ses-baseline/anat/sub-000105_ses-baseline_label-displacementROI_dseg.nii.gz -> .git-annex-tmp-content-MD5E-s9793--508a92de80b90e3030d2e46bb920f051.nii.gz ok -rename s3-PUBLIC sub-000101/ses-displaced/anat/sub-000101_ses-displaced_label-displacementROI_dseg.nii.gz -> .git-annex-tmp-content-MD5E-s9794--512f12576f0f99de20f12aa715000251.nii.gz ok -unexport s3-PUBLIC dataset_description.json ok -unexport s3-PUBLIC sub-000105/ses-displaced/pet/sub-000105_ses-displaced_rec-MLEM_pet.nii.gz ok -unexport s3-PUBLIC CHANGES ok -unexport s3-PUBLIC sub-000105/ses-baseline/pet/sub-000105_ses-baseline_Vb.nii.gz ok -unexport s3-PUBLIC sub-000101/ses-displaced/anat/sub-000101_ses-displaced_T1w.nii.gz ok -unexport s3-PUBLIC sub-000103/ses-baseline/pet/sub-000103_ses-baseline_K1.nii.gz ok -unexport s3-PUBLIC sub-000104/ses-baseline/pet/sub-000104_ses-baseline_K1.nii.gz ok -unexport s3-PUBLIC sub-000103/ses-baseline/pet/sub-000103_ses-baseline_rec-MLEM_pet.nii.gz ok -rename s3-PUBLIC sub-000105/ses-displaced/anat/sub-000105_ses-displaced_label-displacementROI_dseg.nii.gz -> .git-annex-tmp-content-MD5E-s9794--726e1c95511fb7b8bc94f275e9d6e9e7.nii.gz ok -unexport s3-PUBLIC sub-000105/ses-baseline/anat/sub-000105_ses-baseline_T1w.nii.gz ok -unexport s3-PUBLIC sub-000105/ses-displaced/pet/.bidsignore ok -unexport s3-PUBLIC sub-000105/ses-displaced/anat/sub-000105_ses-displaced_T1w.nii.gz ok -unexport s3-PUBLIC sub-000104/ses-baseline/pet/sub-000104_ses-baseline_Vb.nii.gz ok -unexport s3-PUBLIC sub-000103/ses-displaced/pet/sub-000103_ses-displaced_rec-MLEM_pet.nii.gz ok -unexport s3-PUBLIC sub-000102/ses-baseline/pet/sub-000102_ses-baseline_rec-MLEM_pet.nii.gz ok -rename s3-PUBLIC sub-000103/ses-displaced/anat/sub-000103_ses-displaced_label-displacementROI_dseg.nii.gz -> .git-annex-tmp-content-MD5E-s9794--887598ee2277aeebae901fd7d97a7f4c.nii.gz ok -unexport s3-PUBLIC sub-000101/ses-baseline/pet/sub-000101_ses-baseline_k4.nii.gz ok -unexport s3-PUBLIC sub-000105/ses-displaced/pet/sub-000105_ses-displaced_rec-MLEM_pet.json ok -unexport s3-PUBLIC sub-000103/ses-baseline/pet/sub-000103_ses-baseline_Vb.nii.gz ok -unexport s3-PUBLIC sub-000103/ses-baseline/pet/sub-000103_ses-baseline_k3.nii.gz ok -unexport s3-PUBLIC sub-000103/ses-displaced/pet/sub-000103_ses-displaced_k3.nii.gz ok -unexport s3-PUBLIC sub-000103/ses-baseline/pet/sub-000103_ses-baseline_k4.nii.gz ok -unexport s3-PUBLIC sub-000105/ses-baseline/pet/sub-000105_ses-baseline_k3.nii.gz ok -unexport s3-PUBLIC sub-000105/ses-baseline/pet/sub-000105_ses-baseline_rec-MLEM_pet.nii.gz ok -unexport s3-PUBLIC sub-000103/ses-baseline/anat/sub-000103_ses-baseline_T1w.nii.gz ok -unexport s3-PUBLIC sub-000102/ses-displaced/anat/sub-000102_ses-displaced_T1w.nii.gz ok -unexport s3-PUBLIC sub-000105/ses-baseline/pet/sub-000105_ses-baseline_k2.nii.gz ok -unexport s3-PUBLIC sub-000101/ses-displaced/pet/sub-000101_ses-displaced_k3.nii.gz ok -unexport s3-PUBLIC README ok -rename s3-PUBLIC sub-000104/ses-displaced/anat/sub-000104_ses-displaced_label-displacementROI_dseg.nii.gz -> .git-annex-tmp-content-MD5E-s9794--bd645a943d74dda136f4e0c421d49c85.nii.gz ok -unexport s3-PUBLIC sub-000104/ses-displaced/pet/sub-000104_ses-displaced_k3.nii.gz ok -unexport s3-PUBLIC sub-000101/ses-baseline/pet/sub-000101_ses-baseline_K1.nii.gz ok -unexport s3-PUBLIC sub-000101/ses-displaced/pet/sub-000101_ses-displaced_rec-MLEM_pet.nii.gz ok -unexport s3-PUBLIC sub-000102/ses-displaced/pet/sub-000102_ses-displaced_k3.nii.gz ok -unexport s3-PUBLIC sub-000102/ses-baseline/anat/sub-000102_ses-baseline_T1w.nii.gz ok -unexport s3-PUBLIC sub-000101/ses-baseline/pet/sub-000101_ses-baseline_k2.nii.gz ok -unexport s3-PUBLIC sub-000102/ses-baseline/pet/sub-000102_ses-baseline_k4.nii.gz ok -unexport s3-PUBLIC sub-000102/ses-baseline/pet/sub-000102_ses-baseline_k2.nii.gz ok -rename s3-PUBLIC sub-000104/ses-baseline/anat/sub-000104_ses-baseline_label-displacementROI_dseg.nii.gz -> .git-annex-tmp-content-MD5E-s9793--de819fa10f3f6253e4eeec3b77158726.nii.gz ok -unexport s3-PUBLIC sub-000102/ses-displaced/pet/sub-000102_ses-displaced_rec-MLEM_pet.nii.gz ok -unexport s3-PUBLIC sub-000101/ses-baseline/pet/sub-000101_ses-baseline_k3.nii.gz ok -rename s3-PUBLIC sub-000103/ses-baseline/anat/sub-000103_ses-baseline_label-displacementROI_dseg.nii.gz -> .git-annex-tmp-content-MD5E-s9793--e6f2a37df1a0a47325fc9cb065117e0d.nii.gz ok -unexport s3-PUBLIC sub-000104/ses-baseline/pet/sub-000104_ses-baseline_k2.nii.gz ok -unexport s3-PUBLIC sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.nii.gz ok -unexport s3-PUBLIC sub-000104/ses-baseline/pet/sub-000104_ses-baseline_k4.nii.gz ok -unexport s3-PUBLIC sub-000104/ses-displaced/anat/sub-000104_ses-displaced_T1w.nii.gz ok -rename s3-PUBLIC .git-annex-tmp-content-MD5E-s9793--13db6c96808070211b8987c327240ff8.nii.gz -> derivatives/masks/sub-000101/ses-baseline/sub-000101_ses-baseline_label-displacementROI_dseg.nii.gz ok -rename s3-PUBLIC .git-annex-tmp-content-MD5E-s9794--419a63ecba6de7235554c1f5a7fbc7d8.nii.gz -> derivatives/masks/sub-000102/ses-displaced/sub-000102_ses-displaced_label-displacementROI_dseg.nii.gz ok -rename s3-PUBLIC .git-annex-tmp-content-MD5E-s9793--45358ccfe2d8a7455c3202691ee632eb.nii.gz -> derivatives/masks/sub-000102/ses-baseline/sub-000102_ses-baseline_label-displacementROI_dseg.nii.gz ok -rename s3-PUBLIC .git-annex-tmp-content-MD5E-s9793--508a92de80b90e3030d2e46bb920f051.nii.gz -> derivatives/masks/sub-000105/ses-baseline/sub-000105_ses-baseline_label-displacementROI_dseg.nii.gz ok -rename s3-PUBLIC .git-annex-tmp-content-MD5E-s9794--512f12576f0f99de20f12aa715000251.nii.gz -> derivatives/masks/sub-000101/ses-displaced/sub-000101_ses-displaced_label-displacementROI_dseg.nii.gz ok -rename s3-PUBLIC .git-annex-tmp-content-MD5E-s9794--726e1c95511fb7b8bc94f275e9d6e9e7.nii.gz -> derivatives/masks/sub-000105/ses-displaced/sub-000105_ses-displaced_label-displacementROI_dseg.nii.gz ok -rename s3-PUBLIC .git-annex-tmp-content-MD5E-s9794--887598ee2277aeebae901fd7d97a7f4c.nii.gz -> derivatives/masks/sub-000103/ses-displaced/sub-000103_ses-displaced_label-displacementROI_dseg.nii.gz ok -rename s3-PUBLIC .git-annex-tmp-content-MD5E-s9794--bd645a943d74dda136f4e0c421d49c85.nii.gz -> derivatives/masks/sub-000104/ses-displaced/sub-000104_ses-displaced_label-displacementROI_dseg.nii.gz ok -rename s3-PUBLIC .git-annex-tmp-content-MD5E-s9793--de819fa10f3f6253e4eeec3b77158726.nii.gz -> derivatives/masks/sub-000104/ses-baseline/sub-000104_ses-baseline_label-displacementROI_dseg.nii.gz ok -rename s3-PUBLIC .git-annex-tmp-content-MD5E-s9793--e6f2a37df1a0a47325fc9cb065117e0d.nii.gz -> derivatives/masks/sub-000103/ses-baseline/sub-000103_ses-baseline_label-displacementROI_dseg.nii.gz ok -export s3-PUBLIC .bidsignore -ok -export s3-PUBLIC CHANGES -ok -export s3-PUBLIC LICENSE -ok -export s3-PUBLIC README -ok -export s3-PUBLIC dataset_description.json -ok -export s3-PUBLIC derivatives/masks/dseg.tsv -ok -export s3-PUBLIC participants.json -ok -export s3-PUBLIC participants.tsv -ok -export s3-PUBLIC sub-000101/ses-baseline/anat/sub-000101_ses-baseline_T1w.json -ok -export s3-PUBLIC sub-000101/ses-baseline/anat/sub-000101_ses-baseline_T1w.nii -ok -export s3-PUBLIC sub-000101/ses-baseline/pet/sub-000101_ses-baseline_pet.json -ok -export s3-PUBLIC sub-000101/ses-baseline/pet/sub-000101_ses-baseline_pet.nii -ok -export s3-PUBLIC sub-000101/ses-displaced/anat/sub-000101_ses-displaced_T1w.json -ok -export s3-PUBLIC sub-000101/ses-displaced/anat/sub-000101_ses-displaced_T1w.nii -ok -export s3-PUBLIC sub-000101/ses-displaced/pet/sub-000101_ses-displaced_pet.json -ok -export s3-PUBLIC sub-000101/ses-displaced/pet/sub-000101_ses-displaced_pet.nii -ok -export s3-PUBLIC sub-000102/ses-baseline/anat/sub-000102_ses-baseline_T1w.json -ok -export s3-PUBLIC sub-000102/ses-baseline/anat/sub-000102_ses-baseline_T1w.nii -ok -export s3-PUBLIC sub-000102/ses-baseline/pet/sub-000102_ses-baseline_pet.json -ok -export s3-PUBLIC sub-000102/ses-baseline/pet/sub-000102_ses-baseline_pet.nii -ok -export s3-PUBLIC sub-000102/ses-displaced/anat/sub-000102_ses-displaced_T1w.json -ok -export s3-PUBLIC sub-000102/ses-displaced/anat/sub-000102_ses-displaced_T1w.nii -ok -export s3-PUBLIC sub-000102/ses-displaced/pet/sub-000102_ses-displaced_pet.json -ok -export s3-PUBLIC sub-000102/ses-displaced/pet/sub-000102_ses-displaced_pet.nii -ok -export s3-PUBLIC sub-000103/ses-baseline/anat/sub-000103_ses-baseline_T1w.json -ok -export s3-PUBLIC sub-000103/ses-baseline/anat/sub-000103_ses-baseline_T1w.nii -ok -export s3-PUBLIC sub-000103/ses-baseline/pet/sub-000103_ses-baseline_pet.json -ok -export s3-PUBLIC sub-000103/ses-baseline/pet/sub-000103_ses-baseline_pet.nii -ok -export s3-PUBLIC sub-000103/ses-displaced/anat/sub-000103_ses-displaced_T1w.json -ok -export s3-PUBLIC sub-000103/ses-displaced/anat/sub-000103_ses-displaced_T1w.nii -ok -export s3-PUBLIC sub-000103/ses-displaced/pet/sub-000103_ses-displaced_pet.json -ok -export s3-PUBLIC sub-000103/ses-displaced/pet/sub-000103_ses-displaced_pet.nii -ok -export s3-PUBLIC sub-000104/ses-baseline/anat/sub-000104_ses-baseline_T1w.json -ok -export s3-PUBLIC sub-000104/ses-baseline/anat/sub-000104_ses-baseline_T1w.nii -ok -export s3-PUBLIC sub-000104/ses-baseline/pet/sub-000104_ses-baseline_pet.json -ok -export s3-PUBLIC sub-000104/ses-baseline/pet/sub-000104_ses-baseline_pet.nii -ok -export s3-PUBLIC sub-000104/ses-displaced/anat/sub-000104_ses-displaced_T1w.json -ok -export s3-PUBLIC sub-000104/ses-displaced/anat/sub-000104_ses-displaced_T1w.nii -ok -export s3-PUBLIC sub-000104/ses-displaced/pet/sub-000104_ses-displaced_pet.json -ok -export s3-PUBLIC sub-000104/ses-displaced/pet/sub-000104_ses-displaced_pet.nii -ok -export s3-PUBLIC sub-000105/ses-baseline/anat/sub-000105_ses-baseline_T1w.json -ok -export s3-PUBLIC sub-000105/ses-baseline/anat/sub-000105_ses-baseline_T1w.nii -ok -export s3-PUBLIC sub-000105/ses-baseline/pet/sub-000105_ses-baseline_pet.json -ok -export s3-PUBLIC sub-000105/ses-baseline/pet/sub-000105_ses-baseline_pet.nii -ok -export s3-PUBLIC sub-000105/ses-displaced/anat/sub-000105_ses-displaced_T1w.json -ok -export s3-PUBLIC sub-000105/ses-displaced/anat/sub-000105_ses-displaced_T1w.nii -ok -export s3-PUBLIC sub-000105/ses-displaced/pet/sub-000105_ses-displaced_pet.json -ok -export s3-PUBLIC sub-000105/ses-displaced/pet/sub-000105_ses-displaced_pet.nii -ok -(recording state in git...) - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Thank you for all your work making this wonderful tool! - -[[!meta title="export tree bug when two files with the same content both should be removed"]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects/comment_1_70428ebc4027538253edc483dc5cb971._comment b/doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects/comment_1_70428ebc4027538253edc483dc5cb971._comment deleted file mode 100644 index 9a016f3498..0000000000 --- a/doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects/comment_1_70428ebc4027538253edc483dc5cb971._comment +++ /dev/null @@ -1,50 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-11-09T18:16:29Z" - content=""" -I tried making a repository with just 2 files, one in git and one in git-annex, -and am unable to reproduce the bug. Here is -what `git-annex export master --to remote --debug` showed when -exporting a tree that deleted file "foo" which was a git object: - - [2022-11-09 14:16:09.513485291] (Remote.S3) String to sign: "DELETE\n/foo\n\nhost:t-a9b9d406-30e5-41cc-a74c-c5d83b2953fb.s3.amazonaws.com\nx-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\nx-amz-date:20221109T181609Z\n\nhost;x-amz-content-sha256;x-amz-date\ne3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" - [2022-11-09 14:16:09.513608493] (Remote.S3) Host: "t-a9b9d406-30e5-41cc-a74c-c5d83b2953fb.s3.amazonaws.com" - [2022-11-09 14:16:09.513697445] (Remote.S3) Path: "/foo" - [2022-11-09 14:16:09.513748086] (Remote.S3) Query string: "" - [2022-11-09 14:16:09.513829584] (Remote.S3) Header: (redacted -- JEH) - [2022-11-09 14:16:09.687814925] (Remote.S3) Response status: Status {statusCode = 204, statusMessage = "No Content"} - -The S3 console showed that the file was deleted from the bucket. -And as far as the S3 remote implementation is concerned, there should -not be anything different between a git object and a git-annex object. -At the level of the S3 remote both have a git-annex key that it deletes -in the same way. - -In your log, the only thing it does with the file is export it, but it does -not later unexport it: - - $ grep baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json Versioned_S3_tree_does_not_unexport_git_objects.mdwn - Git files in the 1.0.0 tag are still present in the S3 1.0.1 export. sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json is an example file not present in 1.0.1 that is still present on S3. - export s3-PUBLIC sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json - $ - -If the bug is that it's somehow failing to try to unexport the file, -that should happen independently of the special remote type, so would also -happen with a directory special remote. So I tried that: - - $ git clone https://github.com/openneurodatasets/ds001705 - $ cd ds001705 - $ git-annex get --branch=tags/1.0.0 - $ git-annex get --branch=tags/1.0.1 - $ mkdir ../d - $ git-annex initremote d type=directory directory=../d encryption=none exporttree=yes - $ git-annex export 1.0.0 --to d - $ ls ../d/sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json - ../d/sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json - $ git-annex export 1.0.1 --to d - $ ls ../d/sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json - ../d/sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json - -Ok, so, nothing to do with S3 or versioning at all. -"""]] diff --git a/doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects/comment_2_059b9beb31d9cbc97ea4a59f47d2e63d._comment b/doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects/comment_2_059b9beb31d9cbc97ea4a59f47d2e63d._comment deleted file mode 100644 index dae5c72f0c..0000000000 --- a/doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects/comment_2_059b9beb31d9cbc97ea4a59f47d2e63d._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-11-09T19:15:59Z" - content=""" -Interestingly, it does not happen in simpler situations: - - $ git checkout 1.0.0 - $ git-annex export 1.0.0 --to d - $ git rm sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json - $ git commit -m removed - $ git-annex export HEAD --to d - unexport d sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json ok - $ ls ../d/sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json - ls: cannot access '../d/sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json': No such file or directory - -So something about the diff between 1.0.0 and 1.0.1 is somehow causing the -bug.. -"""]] diff --git a/doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects/comment_3_176cbc137afb5cf8841ff9114b111fef._comment b/doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects/comment_3_176cbc137afb5cf8841ff9114b111fef._comment deleted file mode 100644 index 0d8f4ce9e4..0000000000 --- a/doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects/comment_3_176cbc137afb5cf8841ff9114b111fef._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2022-11-09T19:41:33Z" - content=""" -Ok, the bug is due to 2 files that have the same content. - - sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json - sub-000101/ses-displaced/pet/sub-000101_ses-displaced_rec-MLEM_pet.json - -Both files get deleted. And the bug makes it only pick one of the two files -to delete, because it's using a map from key to file and the second file -overwrites the first in the map. - -So this would also presumably affect annexed files when two have the same -content and are being deleted. -"""]] diff --git a/doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects/comment_4_a6490af0427bbe4363bea824d55a7593._comment b/doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects/comment_4_a6490af0427bbe4363bea824d55a7593._comment deleted file mode 100644 index 18cb92cde5..0000000000 --- a/doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects/comment_4_a6490af0427bbe4363bea824d55a7593._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="nell@7201fe78ade251118ef3441f4e509b37cd836503" - nickname="nell" - avatar="http://cdn.libravatar.org/avatar/7d9b55f7fec5eaba6fcd3fecc65ccb7d" - subject="comment 4" - date="2022-11-09T19:51:20Z" - content=""" -Nice, thank you for looking into this, Joey! That makes sense why it would appear to be git files, we tend to have identical duplicate metadata files much more often than duplicate image files within datasets. -"""]] diff --git a/doc/bugs/Webapp_won__180__t_start_on_MacOS.mdwn b/doc/bugs/Webapp_won__180__t_start_on_MacOS.mdwn deleted file mode 100644 index 1210fd1c96..0000000000 --- a/doc/bugs/Webapp_won__180__t_start_on_MacOS.mdwn +++ /dev/null @@ -1,315 +0,0 @@ -### Please describe the problem. - -After an update the webapp won´t start. git-annex itself works just fine :) -It doesn´t even try to open the browser. It seems like the command isn't recognised. -(Invalid argument `webapp') - -### What steps will reproduce the problem? - -git annex webapp - - -### What version of git-annex are you using? On what operating system? - -I just updated to version 10.20220822 via homebrew on MacOS 12.5.1 (Monterey) - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - -(scanning...) remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -ControlSocket .git/annex/ssh/git@gitlab.com already exists, disabling multiplexing -To ssh://git-annex-Backupserver.fritz.box-Florian_22_.2Fsrv.2Fmergerfs.2FPOOL01.2FBackupFlosSoundPodcast.2F/srv/mergerfs/POOL01/BackupFlosSoundPodcast/ - d865495..82b25c1 git-annex -> synced/git-annex -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -To gitlab.com:gitannex1/FlosSoundPodcast.git - d865495..82b25c1 git-annex -> synced/git-annex -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -(started...) -(recording state in git...) -git-annex filter-process: git-annex: command not found -fatal: the remote end hung up unexpectedly -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -ControlSocket .git/annex/ssh/24d7fc76050b5b840322d70167a1797d already exists, disabling multiplexing -ControlSocket .git/annex/ssh/24d7fc76050b5b840322d70167a1797d already exists, disabling multiplexing -fatal: Unable to create '/Users/Florian/FlosSoundPodcast/.git/index.lock': File exists. - -Another git process seems to be running in this repository, e.g. -an editor opened by 'git commit'. Please make sure all processes -are terminated then try again. If it still fails, a git process -may have crashed in this repository earlier: -remove the file manually to continue. - - user error (xargs ["-0","git","--git-dir=.git","--work-tree=.","--literal-pathspecs","add","--force","--"] exited 1) -(recording state in git...) -(recording state in git...) -(recording state in git...) -(recording state in git...) -(recording state in git...) -(recording state in git...) -(recording state in git...) -To ssh://git-annex-Backupserver.fritz.box-Florian_22_.2Fsrv.2Fmergerfs.2FPOOL01.2FBackupFlosSoundPodcast.2F/srv/mergerfs/POOL01/BackupFlosSoundPodcast/ - 82b25c1..21d448b git-annex -> synced/git-annex - 0856a3e..961a904 master -> synced/master -(recording state in git...) -(recording state in git...) -(recording state in git...) -(recording state in git...) -remote: -remote: To create a merge request for synced/master, visit: -remote: https://gitlab.com/gitannex1/FlosSoundPodcast/-/merge_requests/new?merge_request%5Bsource_branch%5D=synced%2Fmaster -remote: -To gitlab.com:gitannex1/FlosSoundPodcast.git - 82b25c1..21d448b git-annex -> synced/git-annex - 0856a3e..961a904 master -> synced/master -(recording state in git...) -(recording state in git...) -(recording state in git...) -(recording state in git...) -(recording state in git...) -(recording state in git...) -(recording state in git...) -(recording state in git...) -(recording state in git...) -fatal: Unable to create '.git/index.lock': .git/index.lock: openFd: already exists (File exists) - -If no other git process is currently running, this probably means a -git process crashed in this repository earlier. Make sure no other git -process is running and remove the file manually to continue. - -Committer crashed: .git/index.lock: openFd: already exists (File exists) -(recording state in git...) -(recovering from race...) -(recording state in git...) -(recording state in git...) -(recording state in git...) -(recording state in git...) -(recording state in git...) -(recording state in git...) -(recording state in git...) -remote: error: cannot lock ref 'refs/heads/synced/git-annex': Unable to create '/home/florian/FlosSoundPodcast/.git/./refs/heads/synced/git-annex.lock': File exists. -remote: -remote: Another git process seems to be running in this repository, e.g. -remote: an editor opened by 'git commit'. Please make sure all processes -remote: are terminated then try again. If it still fails, a git process -remote: may have crashed in this repository earlier: -remote: remove the file manually to continue. -(recording state in git...) -To ssh://git-annex-SonyVaioLaptop.local-florian_22_FlosSoundPodcast/~/FlosSoundPodcast/ - ! [remote rejected] git-annex -> synced/git-annex (failed to update ref) -error: failed to push some refs to 'ssh://git-annex-SonyVaioLaptop.local-florian_22_FlosSoundPodcast/~/FlosSoundPodcast/' -(recording state in git...) -(recovering from race...) -To ssh://git-annex-SonyVaioLaptop.local-florian_22_FlosSoundPodcast/~/FlosSoundPodcast/ - 045d2fb..21d448b git-annex -> synced/git-annex - 0856a3e..961a904 master -> synced/master -To ssh://git-annex-SonyVaioLaptop.local-florian_22_FlosSoundPodcast/~/FlosSoundPodcast/ - 21d448b..9aec50f git-annex -> synced/git-annex -Everything up-to-date -To ssh://git-annex-Backupserver.fritz.box-Florian_22_.2Fsrv.2Fmergerfs.2FPOOL01.2FBackupFlosSoundPodcast.2F/srv/mergerfs/POOL01/BackupFlosSoundPodcast/ - 21d448b..9aec50f git-annex -> synced/git-annex -To gitlab.com:gitannex1/FlosSoundPodcast.git - 21d448b..9aec50f git-annex -> synced/git-annex -To ssh://git-annex-Backupserver.fritz.box-Florian_22_.2Fsrv.2Fmergerfs.2FPOOL01.2FBackupFlosSoundPodcast.2F/srv/mergerfs/POOL01/BackupFlosSoundPodcast/ - 961a904..f1e3307 master -> synced/master -To ssh://git-annex-SonyVaioLaptop.local-florian_22_FlosSoundPodcast/~/FlosSoundPodcast/ - 961a904..f1e3307 master -> synced/master -remote: error: cannot lock ref 'refs/heads/synced/master': is at f1e33074064f9c0055c9e1d7e629745136907957 but expected 961a904a20dbc6b246539f295129b1f5cc56cf08 -To gitlab.com:gitannex1/FlosSoundPodcast.git - ! [remote rejected] master -> synced/master (failed to update ref) -error: failed to push some refs to 'gitlab.com:gitannex1/FlosSoundPodcast.git' -Everything up-to-date -Everything up-to-date -Everything up-to-date -Everything up-to-date -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -To ssh://git-annex-SonyVaioLaptop.local-florian_22_FlosSoundPodcast/~/FlosSoundPodcast/ - 9aec50f..04bec1d git-annex -> synced/git-annex -To ssh://git-annex-Backupserver.fritz.box-Florian_22_.2Fsrv.2Fmergerfs.2FPOOL01.2FBackupFlosSoundPodcast.2F/srv/mergerfs/POOL01/BackupFlosSoundPodcast/ - 9aec50f..04bec1d git-annex -> synced/git-annex -To gitlab.com:gitannex1/FlosSoundPodcast.git - 9aec50f..04bec1d git-annex -> synced/git-annex -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -Everything up-to-date -Everything up-to-date -Everything up-to-date -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -From ssh://git-annex-SonyVaioLaptop.local-florian_22_FlosSoundPodcast/~/FlosSoundPodcast - 04bec1d..bdfba09 git-annex -> SonyVaioLaptop/git-annex - 0856a3e..f1e3307 master -> SonyVaioLaptop/master -From ssh://git-annex-SonyVaioLaptop.local-florian_22_FlosSoundPodcast/~/FlosSoundPodcast - bdfba09..45d1d71 git-annex -> SonyVaioLaptop/git-annex -(merging SonyVaioLaptop/git-annex into git-annex...) -Everything up-to-date -To ssh://git-annex-SonyVaioLaptop.local-florian_22_FlosSoundPodcast/~/FlosSoundPodcast/ - 04bec1d..45d1d71 git-annex -> synced/git-annex -Everything up-to-date -Everything up-to-date -Everything up-to-date -Everything up-to-date -Everything up-to-date -Everything up-to-date -Everything up-to-date -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -Everything up-to-date -Everything up-to-date -Everything up-to-date -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -Everything up-to-date -Everything up-to-date -Everything up-to-date -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -Everything up-to-date -Everything up-to-date -Everything up-to-date -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -(recording state in git...) -git-annex-shell: : hGetChar: end of file -git-annex-shell: : hGetChar: end of file - - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) -I had tons of luck using git-annex. I am a non-programmer person and I could easily get the hang of it. At the moment I am using it for my podcast production. -I can cut and export now from every available computer in my home. I have a central server on my LAN and git-annex also helps me to back up every episode. -My next goal is to use it for my personal documents like scans of invoices, letter and so on. I think an archive repo will help me with this. So I can have all the stuff in one big folder. -Thanks for inventing such a nice tool - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/Webapp_won__180__t_start_on_MacOS/comment_1_0a8b46efd3f6b7c9f818fc64af9a5f4f._comment b/doc/bugs/Webapp_won__180__t_start_on_MacOS/comment_1_0a8b46efd3f6b7c9f818fc64af9a5f4f._comment deleted file mode 100644 index ef74d474f7..0000000000 --- a/doc/bugs/Webapp_won__180__t_start_on_MacOS/comment_1_0a8b46efd3f6b7c9f818fc64af9a5f4f._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="Atemu" - avatar="http://cdn.libravatar.org/avatar/d1f0f4275931c552403f4c6707bead7a" - subject="comment 1" - date="2022-08-25T16:18:38Z" - content=""" -First of all, look for and stop background git processes that are holding `/Users/Florian/FlosSoundPodcast/.git/index.lock`. Easiest way would be a restart. If the file persists a reboot, remove it. - -Next you should set `remote..annex-ignore` on the gitlab remote because gitlab does not support git-annex. - -What do the logs say after you've done that? -"""]] diff --git a/doc/bugs/Webapp_won__180__t_start_on_MacOS/comment_2_7eb9cf2b8c7192ae654831cdc0c00253._comment b/doc/bugs/Webapp_won__180__t_start_on_MacOS/comment_2_7eb9cf2b8c7192ae654831cdc0c00253._comment deleted file mode 100644 index e9c02d5ec2..0000000000 --- a/doc/bugs/Webapp_won__180__t_start_on_MacOS/comment_2_7eb9cf2b8c7192ae654831cdc0c00253._comment +++ /dev/null @@ -1,283 +0,0 @@ -[[!comment format=mdwn - username="hoegiflo@befdf51af27be5af346451fe32863dadb1d37ad5" - nickname="hoegiflo" - avatar="http://cdn.libravatar.org/avatar/4ebf46d26c5a42c69b022f8793295991" - subject="comment 2" - date="2022-08-27T21:00:48Z" - content=""" -Hey, thanks a lot. I set: -git config remote.origin.annex-ignore true - -And I deleted the lock file. It was -still there after rebooting. - -What do you think is preventing git-annex from recognizing the webapp command? - -Here is my log: -recording state in git...) -(scanning...) remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -ControlSocket .git/annex/ssh/git@gitlab.com already exists, disabling multiplexing -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -(started...) -To gitlab.com:gitannex1/FlosSoundPodcast.git - 217cb33..aa2388c git-annex -> synced/git-annex -(recording state in git...) -git-annex filter-process: git-annex: command not found -fatal: the remote end hung up unexpectedly -To ssh://git-annex-Backupserver.fritz.box-Florian_22_.2Fsrv.2Fmergerfs.2FPOOL01.2FBackupFlosSoundPodcast.2F/srv/mergerfs/POOL01/BackupFlosSoundPodcast/ - 217cb33..aa2388c git-annex -> synced/git-annex -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -remote: -remote: ======================================================================== -remote: -remote: ERROR: Unknown command: git-annex-shell 'notifychanges' '/~/gitannex1/FlosSoundPodcast.git' - -remote: -remote: ======================================================================== -remote: -git-annex filter-process: git-annex: command not found -fatal: the remote end hung up unexpectedly - - user error (xargs [\"-0\",\"git\",\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"add\",\"--force\",\"--\"] exited 1) -git-annex filter-process: git-annex: command not found -fatal: the remote end hung up unexpectedly - - user error (xargs [\"-0\",\"git\",\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"add\",\"--force\",\"--\"] exited 1) -git-annex filter-process: git-annex: command not found -fatal: the remote end hung up unexpectedly - - user error (xargs [\"-0\",\"git\",\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"add\",\"--force\",\"--\"] exited 1) -git-annex filter-process: git-annex: command not found -fatal: the remote end hung up unexpectedly - - user error (xargs [\"-0\",\"git\",\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"add\",\"--force\",\"--\"] exited 1) -git-annex filter-process: git-annex: command not found -fatal: the remote end hung up unexpectedly - - user error (xargs [\"-0\",\"git\",\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"add\",\"--force\",\"--\"] exited 1) -git-annex filter-process: git-annex: command not found -fatal: the remote end hung up unexpectedly - - user error (xargs [\"-0\",\"git\",\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"add\",\"--force\",\"--\"] exited 1)Der -"""]] diff --git a/doc/bugs/Webapp_won__180__t_start_on_MacOS/comment_3_069b107437b6456d027c8d97bc708d48._comment b/doc/bugs/Webapp_won__180__t_start_on_MacOS/comment_3_069b107437b6456d027c8d97bc708d48._comment deleted file mode 100644 index bf7e39191b..0000000000 --- a/doc/bugs/Webapp_won__180__t_start_on_MacOS/comment_3_069b107437b6456d027c8d97bc708d48._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="Atemu" - avatar="http://cdn.libravatar.org/avatar/d1f0f4275931c552403f4c6707bead7a" - subject="comment 3" - date="2022-08-28T17:56:20Z" - content=""" -There are still errors but those shouldn't prevent git-annex from working. - -Could you run `git annex sync` manually? - -Is the name of the gitlab remote really `origin`? Because it still seems to be trying to sync with it. Could you run `git remote -v`? -"""]] diff --git a/doc/bugs/Webapp_won__180__t_start_on_MacOS/comment_4_1e944c517796ca29f8870fcf76c9a0d4._comment b/doc/bugs/Webapp_won__180__t_start_on_MacOS/comment_4_1e944c517796ca29f8870fcf76c9a0d4._comment deleted file mode 100644 index c6b59c7cbf..0000000000 --- a/doc/bugs/Webapp_won__180__t_start_on_MacOS/comment_4_1e944c517796ca29f8870fcf76c9a0d4._comment +++ /dev/null @@ -1,20 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2022-08-29T17:56:55Z" - content=""" -git-annex can be built without support for `git-annex webapp`, -and it seems that your updated build ended up with that turned off. - -It would be good to contact the homebrew developers and file a bug report -about this. From what I can see, their formula for git-annex does not -explicitly disable the webapp, but some problem may have caused it to get -disabled. - -I suspect what caused this is a problem with a new version of cabal. -When I build git-annex (on linux) with cabal 3.4.1.0 and ghc 9.0.2, -it builds without the webapp. This is even though the webapp is supposed -to be build by default. Even `cabal configure -fWebapp` -does not cause it to build the webapp. Made a bug for that at -[[cabal_builds_without_assistant]]. -"""]] diff --git a/doc/bugs/Webapp_won__180__t_start_on_MacOS/comment_5_2c7d2e81cbe943fe9154830fdf958181._comment b/doc/bugs/Webapp_won__180__t_start_on_MacOS/comment_5_2c7d2e81cbe943fe9154830fdf958181._comment deleted file mode 100644 index d1ec499a92..0000000000 --- a/doc/bugs/Webapp_won__180__t_start_on_MacOS/comment_5_2c7d2e81cbe943fe9154830fdf958181._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2022-08-29T19:21:22Z" - content=""" -I've committed a workaround to that cabal problem, so assuming that was -the cause of your problem, it will be fixed by the next git-annex release. - -If something else caused it, the next git-annex release would -probably fail to build on homebrew. Which the homebrew developers will -follow up on. - -So, I don't think this bug needs to be kept open any longer. Although you -will need to wait, or install git-annex by another means than homebrew, in -the meantime.. -"""]] diff --git a/doc/bugs/When_--git-dir_is_not_in_--work-tree.mdwn b/doc/bugs/When_--git-dir_is_not_in_--work-tree.mdwn deleted file mode 100644 index dbc430b22c..0000000000 --- a/doc/bugs/When_--git-dir_is_not_in_--work-tree.mdwn +++ /dev/null @@ -1,52 +0,0 @@ -Bug report incorrectly originally posted in the forum as -[[/forum/When_--git-dir_is_not_in_--work-tree]]. - -Simple test case: - - joey@darkstar:~/src/git-annex>cd /tmp - joey@darkstar:/tmp>mkdir foo - joey@darkstar:/tmp>cd foo - joey@darkstar:/tmp/foo>git --git-dir=`pwd`/.dotfiles --work-tree=`pwd` init - Initialized empty Git repository in /tmp/foo/.dotfiles/ - joey@darkstar:/tmp/foo>git --git-dir=`pwd`/.dotfiles --work-tree=`pwd` annex init - init - git-annex: Git refuses to operate in this repository, - probably because it is owned by someone else. - - To add an exception for this directory, call: - git config --global --add safe.directory /tmp/foo - - failed - -And --debug shows: - - [2022-09-20 14:17:56.238686901] (Utility.Process) process [1284622] read: git ["config","--local","--list"] - [2022-09-20 14:17:56.240836887] (Utility.Process) process [1284622] done ExitFailure 128 - [2022-09-20 14:17:56.24107763] (Git.Config) config output: fatal: --local can only be used inside a git repository - -So passing --git-dir to that command will make it succeeed. The problem -though is that passing --git-dir to that command also bypasses git's -fix for CVE-2022-24765. The command would even succeed if the directory -were owned by someone else then. - - joey@darkstar:/tmp/foo>sudo chown -R root.root . - [sudo] password for joey: - joey@darkstar:/tmp/foo>git --git-dir=`pwd`/.dotfiles config --local --list - core.repositoryformatversion=0 - core.filemode=true - core.bare=false - core.logallrefupdates=true - core.worktree=/tmp/foo - joey@darkstar:/tmp/foo>git config --local --list - fatal: --local can only be used inside a git repository - -But, if the user runs git-annex with an explicit --git-dir, -it's actually ok for git-annex to bypass the CVE-2022-24765 check. -Because --git-dir actually bypasses that check. - -So, the fix for this seems like it will involve it remembering -when the git repo was originally specified using --git-dir (or `GIT_DIR`), -and if so, guardSafeToUseRepo can skip the check, or pass --git-dir to -`git config --local --list`. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/Windows_build_test_failures.mdwn b/doc/bugs/Windows_build_test_failures.mdwn deleted file mode 100644 index 556f5dd387..0000000000 --- a/doc/bugs/Windows_build_test_failures.mdwn +++ /dev/null @@ -1,1234 +0,0 @@ -Given that others can build+run OK I wonder if this is a build environment setup problem rather than a code issue. -This might be two bugs, but I rather suspect the fix to one is the fix to the other. - -### Please describe the problem. -Some tests fail with: -ssh: Could not resolve hostname C: no address associated with name - -then if you try and do a copy to/from an ssh remote then it fails with simply 'copy: 1 failed' - -### What steps will reproduce the problem? - -For the tests, simply sh standalone/windows/build.sh - -For the other, I'm using a modified version of the other windows user's script, the set -x'd output is: - -
-+ git-annex version
-git-annex version: 4.20130621-g36258de^M
-build flags: Pairing Testsuite S3 WebDAV DNS^M
-+ ssh gitremote sh testrepo.sh
-git-annex version: 4.20130621-g36258de
-build flags: Assistant Webapp Pairing Testsuite S3 WebDAV Inotify DBus XMPP DNS
-Initialized empty Git repository in /media/backup/git/repo.git/
-init origin ok
-(Recording state in git...)
-commit
-ok
-git-annex: no branch is checked out
-+ rm -rf repo
-+ git init repo
-Initialized empty Git repository in c:/Users/Oliver/repo/.git/
-+ cd repo
-+ git-annex init
-init  ^M
-  Detected a crippled filesystem.^M
-^M
-  Enabling direct mode.^M
-^M
-  Detected a filesystem without fifo support.^M
-^M
-  Disabling ssh connection caching.^M
-ok^M
-(Recording state in git...)^M
-+ git remote add origin ssh://git@remote/~/repo.git
-+ echo hello
-+ git-annex add .
-add foo.txt (checksum...) ok^M
-(Recording state in git...)^M
-+ git commit -m .
-[master (root-commit) f34a076] .
- 1 file changed, 1 insertion(+)
- create mode 120000 foo.txt
-+ git-annex sync
-commit  ^M
-ok^M
-pull origin warning: no common commits
-From ssh://remote/~/repo
- * [new branch]      git-annex  -> origin/git-annex
-^M
-ok^M
-(merging origin/git-annex into git-annex...)^M
-(Recording state in git...)^M
-push origin To ssh://git@remote/~/repo.git
- * [new branch]      git-annex -> synced/git-annex
- * [new branch]      master -> synced/master
-^M
-ok^M
-+ git-annex copy --to origin
-copy foo.txt (checking origin...) (to origin...) ^M
-failed^M
-git-annex.exe: copy: 1 failed
-
- - - - - -### What version of git-annex are you using? On what operating system? -Windows (8 x64): -git-annex version: 4.20130621-g36258de -build flags: Pairing Testsuite S3 WebDAV DNS -Linux (debian wheezy i386): -git-annex version: 4.20130621-g36258de -build flags: Assistant Webapp Pairing Testsuite S3 WebDAV Inotify DBus XMPP DNS - -other versions -git-bash: git(1.8.0.msysgit.0),ssh(ssh 4.6p1, ssl 0.9.8e 23 feb 2007) -msys: git(none) ssh(ssh 5.4p1, ssl 1.0.0 29 Mar 2010) -cygwin: git(1.7.9) ssh(none) - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - -build.log: -+ set -e -+ HP='/c/Program Files (x86)/Haskell Platform/2012.4.0.0' -+ FLAGS='-Webapp -Assistant -XMPP' -+ PATH='/c/Program Files (x86)/Haskell Platform/2012.4.0.0/bin:/c/Program Files (x86)/Haskell Platform/2012.4.0.0/lib/extralibs/bin:/c/Program Files (x86)/NSIS:/home/Oliver/bin:.:/usr/local/bin:/mingw/bin:/bin:/c/Program Files (x86)/Haskell/bin:/c/Program Files (x86)/Haskell Platform/2013.2.0.0/lib/extralibs/bin:/c/Program Files (x86)/Haskell Platform/2013.2.0.0/bin:/c/Program Files (x86)/AMD APP/bin/x86_64:/c/Program Files (x86)/AMD APP/bin/x86:/c/Windows/system32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0/:/c/Program Files (x86)/ATI Technologies/ATI.ACE/Core-Static:/c/Program Files (x86)/Windows Kits/8.0/Windows Performance Toolkit/:/c/Program Files/Microsoft/Web Platform Installer/:/c/Program Files (x86)/Haskell Platform/2013.2.0.0/mingw/bin:/c/Users/Oliver/AppData/Roaming/cabal/bin' -+ rm -f git-annex-installer.exe -+ cabal update -Downloading the latest package list from hackage.haskell.org -+ rm -rf MissingH-1.2.0.0 -+ cabal unpack MissingH -Unpacking to MissingH-1.2.0.0\ -+ cd MissingH-1.2.0.0 -+ withcyg patch -p1 -+ PATH='/c/Program Files (x86)/Haskell Platform/2012.4.0.0/bin:/c/Program Files (x86)/Haskell Platform/2012.4.0.0/lib/extralibs/bin:/c/Program Files (x86)/NSIS:/home/Oliver/bin:.:/usr/local/bin:/mingw/bin:/bin:/c/Program Files (x86)/Haskell/bin:/c/Program Files (x86)/Haskell Platform/2013.2.0.0/lib/extralibs/bin:/c/Program Files (x86)/Haskell Platform/2013.2.0.0/bin:/c/Program Files (x86)/AMD APP/bin/x86_64:/c/Program Files (x86)/AMD APP/bin/x86:/c/Windows/system32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0/:/c/Program Files (x86)/ATI Technologies/ATI.ACE/Core-Static:/c/Program Files (x86)/Windows Kits/8.0/Windows Performance Toolkit/:/c/Program Files/Microsoft/Web Platform Installer/:/c/Program Files (x86)/Haskell Platform/2013.2.0.0/mingw/bin:/c/Users/Oliver/AppData/Roaming/cabal/bin:/c/cygwin/bin' -+ patch -p1 -patching file src/System/IO/WindowsCompat.hs -Hunk #1 succeeded at 119 (offset -1 lines). -Hunk #2 succeeded at 132 (offset -1 lines). -+ cabal install -Resolving dependencies... -In order, the following would be installed: -MissingH-1.2.0.0 (reinstall) -cabal.exe: The following packages are likely to be broken by the reinstalls: -hS3-0.5.7 -Use --force-reinstalls if you want to install anyway. -+ true -+ cd .. -+ cabal install --only-dependencies '-f-Webapp -Assistant -XMPP' -Resolving dependencies... -All the requested packages are already installed: -Use --reinstall if you want to reinstall anyway. -+ '[' -e last-incremental-failed ']' -+ touch last-incremental-failed -+ withcyg cabal configure '-f-Webapp -Assistant -XMPP' -+ PATH='/c/Program Files (x86)/Haskell Platform/2012.4.0.0/bin:/c/Program Files (x86)/Haskell Platform/2012.4.0.0/lib/extralibs/bin:/c/Program Files (x86)/NSIS:/home/Oliver/bin:.:/usr/local/bin:/mingw/bin:/bin:/c/Program Files (x86)/Haskell/bin:/c/Program Files (x86)/Haskell Platform/2013.2.0.0/lib/extralibs/bin:/c/Program Files (x86)/Haskell Platform/2013.2.0.0/bin:/c/Program Files (x86)/AMD APP/bin/x86_64:/c/Program Files (x86)/AMD APP/bin/x86:/c/Windows/system32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0/:/c/Program Files (x86)/ATI Technologies/ATI.ACE/Core-Static:/c/Program Files (x86)/Windows Kits/8.0/Windows Performance Toolkit/:/c/Program Files/Microsoft/Web Platform Installer/:/c/Program Files (x86)/Haskell Platform/2013.2.0.0/mingw/bin:/c/Users/Oliver/AppData/Roaming/cabal/bin:/c/cygwin/bin' -+ cabal configure '-f-Webapp -Assistant -XMPP' -Resolving dependencies... -[ 1 of 24] Compiling Utility.Applicative ( Utility\Applicative.hs, dist\setup\Utility\Applicative.o ) -[ 2 of 24] Compiling Utility.PartialPrelude ( Utility\PartialPrelude.hs, dist\setup\Utility\PartialPrelude.o ) -[ 3 of 24] Compiling Utility.FileSystemEncoding ( Utility\FileSystemEncoding.hs, dist\setup\Utility\FileSystemEncoding.o ) -[ 4 of 24] Compiling Utility.Exception ( Utility\Exception.hs, dist\setup\Utility\Exception.o ) -[ 5 of 24] Compiling Utility.Misc ( Utility\Misc.hs, dist\setup\Utility\Misc.o ) -[ 6 of 24] Compiling Utility.Process ( Utility\Process.hs, dist\setup\Utility\Process.o ) -[ 7 of 24] Compiling Utility.Env ( Utility\Env.hs, dist\setup\Utility\Env.o ) -[ 8 of 24] Compiling Utility.UserInfo ( Utility\UserInfo.hs, dist\setup\Utility\UserInfo.o ) -[ 9 of 24] Compiling Utility.OSX ( Utility\OSX.hs, dist\setup\Utility\OSX.o ) -[10 of 24] Compiling Utility.Tmp ( Utility\Tmp.hs, dist\setup\Utility\Tmp.o ) -[11 of 24] Compiling Utility.Monad ( Utility\Monad.hs, dist\setup\Utility\Monad.o ) -[12 of 24] Compiling Utility.Path ( Utility\Path.hs, dist\setup\Utility\Path.o ) -[13 of 24] Compiling Utility.FreeDesktop ( Utility\FreeDesktop.hs, dist\setup\Utility\FreeDesktop.o ) -[16 of 24] Compiling Utility.SafeCommand ( Utility\SafeCommand.hs, dist\setup\Utility\SafeCommand.o ) -[17 of 24] Compiling Utility.ExternalSHA ( Utility\ExternalSHA.hs, dist\setup\Utility\ExternalSHA.o ) -[18 of 24] Compiling Utility.Directory ( Utility\Directory.hs, dist\setup\Utility\Directory.o ) -Linking .\dist\setup\setup.exe ... - checking version... 4.20130621-g36258de - checking git... yes - checking git version... 1.7.9 - checking cp -a... yes - checking cp -p... yes - checking cp --reflink=auto... no - checking xargs -0... yes - checking rsync... yes - checking curl... no - checking wget... yes - checking bup... no - checking gpg... not available - checking lsof... not available - checking ssh connection caching... no - checking sha1... sha1sum - checking sha256... sha256sum - checking sha512... sha512sum - checking sha224... sha224sum - checking sha384... sha384sum -Configuring git-annex-4.20130601... -+ withcyg cabal build -+ PATH='/c/Program Files (x86)/Haskell Platform/2012.4.0.0/bin:/c/Program Files (x86)/Haskell Platform/2012.4.0.0/lib/extralibs/bin:/c/Program Files (x86)/NSIS:/home/Oliver/bin:.:/usr/local/bin:/mingw/bin:/bin:/c/Program Files (x86)/Haskell/bin:/c/Program Files (x86)/Haskell Platform/2013.2.0.0/lib/extralibs/bin:/c/Program Files (x86)/Haskell Platform/2013.2.0.0/bin:/c/Program Files (x86)/AMD APP/bin/x86_64:/c/Program Files (x86)/AMD APP/bin/x86:/c/Windows/system32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0/:/c/Program Files (x86)/ATI Technologies/ATI.ACE/Core-Static:/c/Program Files (x86)/Windows Kits/8.0/Windows Performance Toolkit/:/c/Program Files/Microsoft/Web Platform Installer/:/c/Program Files (x86)/Haskell Platform/2013.2.0.0/mingw/bin:/c/Users/Oliver/AppData/Roaming/cabal/bin:/c/cygwin/bin' -+ cabal build -Building git-annex-4.20130601... -Preprocessing executable 'git-annex' for git-annex-4.20130601... -Touch.hsc:117:2: warning: #warning "utimensat and lutimes not available; building without symlink timestamp preservation support" -Touch.hsc: In function 'main': -Touch.hsc:117:2: warning: #warning "utimensat and lutimes not available; building without symlink timestamp preservation support" -Touch.hsc:117:2: warning: #warning "utimensat and lutimes not available; building without symlink timestamp preservation support" -In file included from Mounts.hsc:23:0: -Utility/libmounts.h:17:3: warning: #warning mounts listing code not available for this OS - -Utility\libdiskfree.c:36:3: - warning: #warning free space checking code not available for this OS - -In file included from Utility\libmounts.c:35:0: - -Utility\libmounts.h:17:3: - warning: #warning mounts listing code not available for this OS -[ 1 of 217] Compiling Utility.Dot ( Utility\Dot.hs, dist\build\git-annex\git-annex-tmp\Utility\Dot.o ) -[ 2 of 217] Compiling BuildFlags ( BuildFlags.hs, dist\build\git-annex\git-annex-tmp\BuildFlags.o ) -[ 3 of 217] Compiling Utility.Percentage ( Utility\Percentage.hs, dist\build\git-annex\git-annex-tmp\Utility\Percentage.o ) -[ 4 of 217] Compiling Utility.Base64 ( Utility\Base64.hs, dist\build\git-annex\git-annex-tmp\Utility\Base64.o ) -[ 5 of 217] Compiling Utility.JSONStream ( Utility\JSONStream.hs, dist\build\git-annex\git-annex-tmp\Utility\JSONStream.o ) -[ 6 of 217] Compiling Git.Types ( Git\Types.hs, dist\build\git-annex\git-annex-tmp\Git\Types.o ) -[ 7 of 217] Compiling Utility.DataUnits ( Utility\DataUnits.hs, dist\build\git-annex\git-annex-tmp\Utility\DataUnits.o ) -[ 8 of 217] Compiling Types.FileMatcher ( Types\FileMatcher.hs, dist\build\git-annex\git-annex-tmp\Types\FileMatcher.o ) -[ 9 of 217] Compiling Types.BranchState ( Types\BranchState.hs, dist\build\git-annex\git-annex-tmp\Types\BranchState.o ) -[ 10 of 217] Compiling Messages.JSON ( Messages\JSON.hs, dist\build\git-annex\git-annex-tmp\Messages\JSON.o ) -[ 11 of 217] Compiling Types.UUID ( Types\UUID.hs, dist\build\git-annex\git-annex-tmp\Types\UUID.o ) -[ 12 of 217] Compiling Types.Group ( Types\Group.hs, dist\build\git-annex\git-annex-tmp\Types\Group.o ) -[ 13 of 217] Compiling Utility.Shell ( Utility\Shell.hs, dist\build\git-annex\git-annex-tmp\Utility\Shell.o ) -[ 14 of 217] Compiling Utility.QuickCheck ( Utility\QuickCheck.hs, dist\build\git-annex\git-annex-tmp\Utility\QuickCheck.o ) -[ 15 of 217] Compiling Utility.PartialPrelude ( Utility\PartialPrelude.hs, dist\build\git-annex\git-annex-tmp\Utility\PartialPrelude.o ) -[ 16 of 217] Compiling Utility.HumanTime ( Utility\HumanTime.hs, dist\build\git-annex\git-annex-tmp\Utility\HumanTime.o ) -[ 17 of 217] Compiling Utility.FileSystemEncoding ( Utility\FileSystemEncoding.hs, dist\build\git-annex\git-annex-tmp\Utility\FileSystemEncoding.o ) -[ 18 of 217] Compiling Utility.Touch ( dist\build\git-annex\git-annex-tmp\Utility\Touch.hs, dist\build\git-annex\git-annex-tmp\Utility\Touch.o ) - -UtilityTouch.hsc:17:1: Warning: - The import of `Utility.FileSystemEncoding' is redundant - except perhaps to import instances from `Utility.FileSystemEncoding' - To import instances alone, use: import Utility.FileSystemEncoding() - -UtilityTouch.hsc:19:1: Warning: - The import of `Foreign' is redundant - except perhaps to import instances from `Foreign' - To import instances alone, use: import Foreign() - -UtilityTouch.hsc:21:1: Warning: - The import of `Control.Monad' is redundant - except perhaps to import instances from `Control.Monad' - To import instances alone, use: import Control.Monad() -[ 19 of 217] Compiling Utility.Applicative ( Utility\Applicative.hs, dist\build\git-annex\git-annex-tmp\Utility\Applicative.o ) -[ 20 of 217] Compiling Utility.Monad ( Utility\Monad.hs, dist\build\git-annex\git-annex-tmp\Utility\Monad.o ) -[ 21 of 217] Compiling Utility.Exception ( Utility\Exception.hs, dist\build\git-annex\git-annex-tmp\Utility\Exception.o ) -[ 22 of 217] Compiling Utility.Tmp ( Utility\Tmp.hs, dist\build\git-annex\git-annex-tmp\Utility\Tmp.o ) -[ 23 of 217] Compiling Utility.Env ( Utility\Env.hs, dist\build\git-annex\git-annex-tmp\Utility\Env.o ) -[ 24 of 217] Compiling Utility.UserInfo ( Utility\UserInfo.hs, dist\build\git-annex\git-annex-tmp\Utility\UserInfo.o ) -[ 25 of 217] Compiling Utility.Misc ( Utility\Misc.hs, dist\build\git-annex\git-annex-tmp\Utility\Misc.o ) - -Utility\Misc.hs:22:1: Warning: - The import of `Utility.Exception' is redundant - except perhaps to import instances from `Utility.Exception' - To import instances alone, use: import Utility.Exception() -[ 26 of 217] Compiling Utility.Process ( Utility\Process.hs, dist\build\git-annex\git-annex-tmp\Utility\Process.o ) - -Utility\Process.hs:44:1: Warning: - The import of `Data.Maybe' is redundant - except perhaps to import instances from `Data.Maybe' - To import instances alone, use: import Data.Maybe() -[ 27 of 217] Compiling Utility.Network ( Utility\Network.hs, dist\build\git-annex\git-annex-tmp\Utility\Network.o ) -[ 28 of 217] Compiling Utility.Verifiable ( Utility\Verifiable.hs, dist\build\git-annex\git-annex-tmp\Utility\Verifiable.o ) -[ 29 of 217] Compiling Utility.Format ( Utility\Format.hs, dist\build\git-annex\git-annex-tmp\Utility\Format.o ) -[ 30 of 217] Compiling Build.SysConfig ( Build\SysConfig.hs, dist\build\git-annex\git-annex-tmp\Build\SysConfig.o ) -[ 31 of 217] Compiling Utility.Path ( Utility\Path.hs, dist\build\git-annex\git-annex-tmp\Utility\Path.o ) -[ 32 of 217] Compiling Config.Cost ( Config\Cost.hs, dist\build\git-annex\git-annex-tmp\Config\Cost.o ) -[ 33 of 217] Compiling Types.Messages ( Types\Messages.hs, dist\build\git-annex\git-annex-tmp\Types\Messages.o ) -[ 34 of 217] Compiling Types.TrustLevel ( Types\TrustLevel.hs, dist\build\git-annex\git-annex-tmp\Types\TrustLevel.o ) -[ 35 of 217] Compiling Utility.SafeCommand ( Utility\SafeCommand.hs, dist\build\git-annex\git-annex-tmp\Utility\SafeCommand.o ) -[ 36 of 217] Compiling Utility.Directory ( Utility\Directory.hs, dist\build\git-annex\git-annex-tmp\Utility\Directory.o ) -[ 37 of 217] Compiling Utility.ExternalSHA ( Utility\ExternalSHA.hs, dist\build\git-annex\git-annex-tmp\Utility\ExternalSHA.o ) -[ 38 of 217] Compiling Common ( Common.hs, dist\build\git-annex\git-annex-tmp\Common.o ) -[ 39 of 217] Compiling Git.Filename ( Git\Filename.hs, dist\build\git-annex\git-annex-tmp\Git\Filename.o ) -[ 40 of 217] Compiling Logs.UUIDBased ( Logs\UUIDBased.hs, dist\build\git-annex\git-annex-tmp\Logs\UUIDBased.o ) -[ 41 of 217] Compiling Types.Key ( Types\Key.hs, dist\build\git-annex\git-annex-tmp\Types\Key.o ) -[ 42 of 217] Compiling Utility.FileMode ( Utility\FileMode.hs, dist\build\git-annex\git-annex-tmp\Utility\FileMode.o ) -[ 43 of 217] Compiling Git ( Git.hs, dist\build\git-annex\git-annex-tmp\Git.o ) - -Git.hs:41:1: Warning: - The import of `Utility.FileMode' is redundant - except perhaps to import instances from `Utility.FileMode' - To import instances alone, use: import Utility.FileMode() -[ 44 of 217] Compiling Utility.InodeCache ( Utility\InodeCache.hs, dist\build\git-annex\git-annex-tmp\Utility\InodeCache.o ) -[ 45 of 217] Compiling Types.KeySource ( Types\KeySource.hs, dist\build\git-annex\git-annex-tmp\Types\KeySource.o ) -[ 46 of 217] Compiling Types.Backend ( Types\Backend.hs, dist\build\git-annex\git-annex-tmp\Types\Backend.o ) -[ 47 of 217] Compiling Utility.Gpg ( Utility\Gpg.hs, dist\build\git-annex\git-annex-tmp\Utility\Gpg.o ) - -Utility\Gpg.hs:12:1: Warning: - The import of `System.Posix.Types' is redundant - except perhaps to import instances from `System.Posix.Types' - To import instances alone, use: import System.Posix.Types() - -Utility\Gpg.hs:14:1: Warning: - The import of `Control.Concurrent' is redundant - except perhaps to import instances from `Control.Concurrent' - To import instances alone, use: import Control.Concurrent() - -Utility\Gpg.hs:15:1: Warning: - The import of `Control.Exception' is redundant - except perhaps to import instances from `Control.Exception' - To import instances alone, use: import Control.Exception() - -Utility\Gpg.hs:16:1: Warning: - The import of `System.Path' is redundant - except perhaps to import instances from `System.Path' - To import instances alone, use: import System.Path() - -Utility\Gpg.hs:19:1: Warning: - The import of `Utility.Env' is redundant - except perhaps to import instances from `Utility.Env' - To import instances alone, use: import Utility.Env() -[ 48 of 217] Compiling Types.Crypto ( Types\Crypto.hs, dist\build\git-annex\git-annex-tmp\Types\Crypto.o ) -[ 49 of 217] Compiling Utility.Matcher ( Utility\Matcher.hs, dist\build\git-annex\git-annex-tmp\Utility\Matcher.o ) -[ 50 of 217] Compiling Utility.Metered ( Utility\Metered.hs, dist\build\git-annex\git-annex-tmp\Utility\Metered.o ) -[ 51 of 217] Compiling Git.FilePath ( Git\FilePath.hs, dist\build\git-annex\git-annex-tmp\Git\FilePath.o ) -[ 52 of 217] Compiling Git.Url ( Git\Url.hs, dist\build\git-annex\git-annex-tmp\Git\Url.o ) -[ 53 of 217] Compiling Git.Construct ( Git\Construct.hs, dist\build\git-annex\git-annex-tmp\Git\Construct.o ) -[ 54 of 217] Compiling Git.Config ( Git\Config.hs, dist\build\git-annex\git-annex-tmp\Git\Config.o ) -[ 55 of 217] Compiling Git.CurrentRepo ( Git\CurrentRepo.hs, dist\build\git-annex\git-annex-tmp\Git\CurrentRepo.o ) - -Git\CurrentRepo.hs:16:1: Warning: - The import of `Utility.Env' is redundant - except perhaps to import instances from `Utility.Env' - To import instances alone, use: import Utility.Env() - -Git\CurrentRepo.hs:43:17: Warning: Defined but not used: `s' -[ 56 of 217] Compiling Git.SharedRepository ( Git\SharedRepository.hs, dist\build\git-annex\git-annex-tmp\Git\SharedRepository.o ) -[ 57 of 217] Compiling Types.GitConfig ( Types\GitConfig.hs, dist\build\git-annex\git-annex-tmp\Types\GitConfig.o ) -[ 58 of 217] Compiling Types.Remote ( Types\Remote.hs, dist\build\git-annex\git-annex-tmp\Types\Remote.o ) -[ 59 of 217] Compiling Types.StandardGroups ( Types\StandardGroups.hs, dist\build\git-annex\git-annex-tmp\Types\StandardGroups.o ) -[ 60 of 217] Compiling Utility.Gpg.Types ( Utility\Gpg\Types.hs, dist\build\git-annex\git-annex-tmp\Utility\Gpg\Types.o ) -[ 61 of 217] Compiling Git.Sha ( Git\Sha.hs, dist\build\git-annex\git-annex-tmp\Git\Sha.o ) -[ 62 of 217] Compiling Utility.CoProcess ( Utility\CoProcess.hs, dist\build\git-annex\git-annex-tmp\Utility\CoProcess.o ) -[ 63 of 217] Compiling Git.Command ( Git\Command.hs, dist\build\git-annex\git-annex-tmp\Git\Command.o ) -[ 64 of 217] Compiling Git.LsFiles ( Git\LsFiles.hs, dist\build\git-annex\git-annex-tmp\Git\LsFiles.o ) -[ 65 of 217] Compiling Git.CatFile ( Git\CatFile.hs, dist\build\git-annex\git-annex-tmp\Git\CatFile.o ) -[ 66 of 217] Compiling Git.UpdateIndex ( Git\UpdateIndex.hs, dist\build\git-annex\git-annex-tmp\Git\UpdateIndex.o ) -[ 67 of 217] Compiling Git.Queue ( Git\Queue.hs, dist\build\git-annex\git-annex-tmp\Git\Queue.o ) -[ 68 of 217] Compiling Git.Version ( Git\Version.hs, dist\build\git-annex\git-annex-tmp\Git\Version.o ) -[ 69 of 217] Compiling Git.CheckAttr ( Git\CheckAttr.hs, dist\build\git-annex\git-annex-tmp\Git\CheckAttr.o ) -[ 70 of 217] Compiling Annex ( Annex.hs, dist\build\git-annex\git-annex-tmp\Annex.o ) -[ 71 of 217] Compiling Types.Option ( Types\Option.hs, dist\build\git-annex\git-annex-tmp\Types\Option.o ) -[ 72 of 217] Compiling Types ( Types.hs, dist\build\git-annex\git-annex-tmp\Types.o ) -[ 73 of 217] Compiling Locations ( Locations.hs, dist\build\git-annex\git-annex-tmp\Locations.o ) -[ 74 of 217] Compiling Messages ( Messages.hs, dist\build\git-annex\git-annex-tmp\Messages.o ) -[ 75 of 217] Compiling Common.Annex ( Common\Annex.hs, dist\build\git-annex\git-annex-tmp\Common\Annex.o ) -[ 76 of 217] Compiling Crypto ( Crypto.hs, dist\build\git-annex\git-annex-tmp\Crypto.o ) -[ 77 of 217] Compiling Annex.CatFile ( Annex\CatFile.hs, dist\build\git-annex\git-annex-tmp\Annex\CatFile.o ) -[ 78 of 217] Compiling Backend.SHA ( Backend\SHA.hs, dist\build\git-annex\git-annex-tmp\Backend\SHA.o ) -[ 79 of 217] Compiling Backend.WORM ( Backend\WORM.hs, dist\build\git-annex\git-annex-tmp\Backend\WORM.o ) -[ 80 of 217] Compiling Backend.URL ( Backend\URL.hs, dist\build\git-annex\git-annex-tmp\Backend\URL.o ) -[ 81 of 217] Compiling Annex.Exception ( Annex\Exception.hs, dist\build\git-annex\git-annex-tmp\Annex\Exception.o ) -[ 82 of 217] Compiling Remote.Helper.Special ( Remote\Helper\Special.hs, dist\build\git-annex\git-annex-tmp\Remote\Helper\Special.o ) -[ 83 of 217] Compiling Remote.Helper.Chunked ( Remote\Helper\Chunked.hs, dist\build\git-annex\git-annex-tmp\Remote\Helper\Chunked.o ) -[ 84 of 217] Compiling Annex.Environment ( Annex\Environment.hs, dist\build\git-annex\git-annex-tmp\Annex\Environment.o ) - -Annex\Environment.hs:13:1: Warning: - The import of `Utility.Env' is redundant - except perhaps to import instances from `Utility.Env' - To import instances alone, use: import Utility.Env() - -Annex\Environment.hs:14:1: Warning: - The import of `Utility.UserInfo' is redundant - except perhaps to import instances from `Utility.UserInfo' - To import instances alone, use: import Utility.UserInfo() -[ 85 of 217] Compiling Types.Command ( Types\Command.hs, dist\build\git-annex\git-annex-tmp\Types\Command.o ) -[ 86 of 217] Compiling Usage ( Usage.hs, dist\build\git-annex\git-annex-tmp\Usage.o ) -[ 87 of 217] Compiling Annex.Queue ( Annex\Queue.hs, dist\build\git-annex\git-annex-tmp\Annex\Queue.o ) -[ 88 of 217] Compiling Annex.BranchState ( Annex\BranchState.hs, dist\build\git-annex\git-annex-tmp\Annex\BranchState.o ) -[ 89 of 217] Compiling Remote.Helper.Encryptable ( Remote\Helper\Encryptable.hs, dist\build\git-annex\git-annex-tmp\Remote\Helper\Encryptable.o ) -[ 90 of 217] Compiling Fields ( Fields.hs, dist\build\git-annex\git-annex-tmp\Fields.o ) -[ 91 of 217] Compiling Annex.CheckAttr ( Annex\CheckAttr.hs, dist\build\git-annex\git-annex-tmp\Annex\CheckAttr.o ) -[ 92 of 217] Compiling Git.HashObject ( Git\HashObject.hs, dist\build\git-annex\git-annex-tmp\Git\HashObject.o ) -[ 93 of 217] Compiling Annex.Link ( Annex\Link.hs, dist\build\git-annex\git-annex-tmp\Annex\Link.o ) -[ 94 of 217] Compiling Utility.CopyFile ( Utility\CopyFile.hs, dist\build\git-annex\git-annex-tmp\Utility\CopyFile.o ) -[ 95 of 217] Compiling Git.Ref ( Git\Ref.hs, dist\build\git-annex\git-annex-tmp\Git\Ref.o ) -[ 96 of 217] Compiling Git.Branch ( Git\Branch.hs, dist\build\git-annex\git-annex-tmp\Git\Branch.o ) -[ 97 of 217] Compiling Git.UnionMerge ( Git\UnionMerge.hs, dist\build\git-annex\git-annex-tmp\Git\UnionMerge.o ) -[ 98 of 217] Compiling Git.Merge ( Git\Merge.hs, dist\build\git-annex\git-annex-tmp\Git\Merge.o ) -[ 99 of 217] Compiling Git.DiffTree ( Git\DiffTree.hs, dist\build\git-annex\git-annex-tmp\Git\DiffTree.o ) -[100 of 217] Compiling Utility.DiskFree ( Utility\DiskFree.hs, dist\build\git-annex\git-annex-tmp\Utility\DiskFree.o ) -[101 of 217] Compiling Utility.Url ( Utility\Url.hs, dist\build\git-annex\git-annex-tmp\Utility\Url.o ) -[102 of 217] Compiling Utility.Rsync ( Utility\Rsync.hs, dist\build\git-annex\git-annex-tmp\Utility\Rsync.o ) -[103 of 217] Compiling Utility.LogFile ( Utility\LogFile.hs, dist\build\git-annex\git-annex-tmp\Utility\LogFile.o ) - -Utility\LogFile.hs:67:1: Warning: - Top-level binding with no type signature: - redir :: forall t t1 t2. t -> t1 -> t2 -[104 of 217] Compiling Utility.Daemon ( Utility\Daemon.hs, dist\build\git-annex\git-annex-tmp\Utility\Daemon.o ) - -Utility\Daemon.hs:13:1: Warning: - The import of `Utility.LogFile' is redundant - except perhaps to import instances from `Utility.LogFile' - To import instances alone, use: import Utility.LogFile() - -Utility\Daemon.hs:19:1: Warning: - The import of `System.Posix.Types' is redundant - except perhaps to import instances from `System.Posix.Types' - To import instances alone, use: import System.Posix.Types() -[105 of 217] Compiling Git.AutoCorrect ( Git\AutoCorrect.hs, dist\build\git-annex\git-annex-tmp\Git\AutoCorrect.o ) -[106 of 217] Compiling Utility.ThreadScheduler ( Utility\ThreadScheduler.hs, dist\build\git-annex\git-annex-tmp\Utility\ThreadScheduler.o ) -[107 of 217] Compiling Git.LsTree ( Git\LsTree.hs, dist\build\git-annex\git-annex-tmp\Git\LsTree.o ) -[108 of 217] Compiling Config ( Config.hs, dist\build\git-annex\git-annex-tmp\Config.o ) -[109 of 217] Compiling Annex.UUID ( Annex\UUID.hs, dist\build\git-annex\git-annex-tmp\Annex\UUID.o ) -[110 of 217] Compiling Backend ( Backend.hs, dist\build\git-annex\git-annex-tmp\Backend.o ) -[111 of 217] Compiling Annex.Version ( Annex\Version.hs, dist\build\git-annex\git-annex-tmp\Annex\Version.o ) -[112 of 217] Compiling Annex.Perms ( Annex\Perms.hs, dist\build\git-annex\git-annex-tmp\Annex\Perms.o ) -[113 of 217] Compiling Logs.Transfer ( Logs\Transfer.hs, dist\build\git-annex\git-annex-tmp\Logs\Transfer.o ) - -Logs\Transfer.hs:126:20: Warning: Defined but not used: `mode' - -Logs\Transfer.hs:146:29: Warning: Defined but not used: `fd' -[114 of 217] Compiling Annex.ReplaceFile ( Annex\ReplaceFile.hs, dist\build\git-annex\git-annex-tmp\Annex\ReplaceFile.o ) -[115 of 217] Compiling Annex.Journal ( Annex\Journal.hs, dist\build\git-annex\git-annex-tmp\Annex\Journal.o ) - -Annex\Journal.hs:89:23: Warning: Defined but not used: `mode' -[116 of 217] Compiling Annex.Branch ( Annex\Branch.hs, dist\build\git-annex\git-annex-tmp\Annex\Branch.o ) -[117 of 217] Compiling Logs.Remote ( Logs\Remote.hs, dist\build\git-annex\git-annex-tmp\Logs\Remote.o ) -[118 of 217] Compiling Logs.Presence ( Logs\Presence.hs, dist\build\git-annex\git-annex-tmp\Logs\Presence.o ) -[119 of 217] Compiling Logs.UUID ( Logs\UUID.hs, dist\build\git-annex\git-annex-tmp\Logs\UUID.o ) -[120 of 217] Compiling Logs.Location ( Logs\Location.hs, dist\build\git-annex\git-annex-tmp\Logs\Location.o ) -[121 of 217] Compiling Annex.Content.Direct ( Annex\Content\Direct.hs, dist\build\git-annex\git-annex-tmp\Annex\Content\Direct.o ) -[122 of 217] Compiling Annex.Content ( Annex\Content.hs, dist\build\git-annex\git-annex-tmp\Annex\Content.o ) - -Annex\Content.hs:50:1: Warning: - The import of `Annex.Exception' is redundant - except perhaps to import instances from `Annex.Exception' - To import instances alone, use: import Annex.Exception() - -Annex\Content.hs:89:21: Warning: Defined but not used: `f' - -Annex\Content.hs:96:21: Warning: Defined but not used: `h' - -Annex\Content.hs:106:9: Warning: Defined but not used: `is_locked' - -Annex\Content.hs:113:13: Warning: Defined but not used: `key' -[123 of 217] Compiling Annex.Direct ( Annex\Direct.hs, dist\build\git-annex\git-annex-tmp\Annex\Direct.o ) -[124 of 217] Compiling Init ( Init.hs, dist\build\git-annex\git-annex-tmp\Init.o ) - -Init.hs:29:1: Warning: - The import of `Utility.UserInfo' is redundant - except perhaps to import instances from `Utility.UserInfo' - To import instances alone, use: import Utility.UserInfo() - -Init.hs:31:1: Warning: - The import of `Utility.FileMode' is redundant - except perhaps to import instances from `Utility.FileMode' - To import instances alone, use: import Utility.FileMode() -[125 of 217] Compiling Logs.Web ( Logs\Web.hs, dist\build\git-annex\git-annex-tmp\Logs\Web.o ) -[126 of 217] Compiling Logs.Group ( Logs\Group.hs, dist\build\git-annex\git-annex-tmp\Logs\Group.o ) -[127 of 217] Compiling Upgrade.V2 ( Upgrade\V2.hs, dist\build\git-annex\git-annex-tmp\Upgrade\V2.o ) -[128 of 217] Compiling Upgrade ( Upgrade.hs, dist\build\git-annex\git-annex-tmp\Upgrade.o ) -[129 of 217] Compiling Creds ( Creds.hs, dist\build\git-annex\git-annex-tmp\Creds.o ) - -Creds.hs:18:1: Warning: - The import of `Utility.Env' is redundant - except perhaps to import instances from `Utility.Env' - To import instances alone, use: import Utility.Env() -[130 of 217] Compiling Remote.Helper.AWS ( Remote\Helper\AWS.hs, dist\build\git-annex\git-annex-tmp\Remote\Helper\AWS.o ) -[131 of 217] Compiling Annex.LockPool ( Annex\LockPool.hs, dist\build\git-annex\git-annex-tmp\Annex\LockPool.o ) - -Annex\LockPool.hs:17:1: Warning: - The import of `Annex.Perms' is redundant - except perhaps to import instances from `Annex.Perms' - To import instances alone, use: import Annex.Perms() - -Annex\LockPool.hs:39:12: Warning: Defined but not used: `fd' -[132 of 217] Compiling Remote.Helper.Hooks ( Remote\Helper\Hooks.hs, dist\build\git-annex\git-annex-tmp\Remote\Helper\Hooks.o ) - -Remote\Helper\Hooks.hs:18:1: Warning: - The import of `Annex.Perms' is redundant - except perhaps to import instances from `Annex.Perms' - To import instances alone, use: import Annex.Perms() - -Remote\Helper\Hooks.hs:74:17: Warning: Defined but not used: `lck' -[133 of 217] Compiling Remote.S3 ( Remote\S3.hs, dist\build\git-annex\git-annex-tmp\Remote\S3.o ) -[134 of 217] Compiling Remote.Directory ( Remote\Directory.hs, dist\build\git-annex\git-annex-tmp\Remote\Directory.o ) -[135 of 217] Compiling Remote.Web ( Remote\Web.hs, dist\build\git-annex\git-annex-tmp\Remote\Web.o ) -[136 of 217] Compiling Remote.WebDAV ( Remote\WebDAV.hs, dist\build\git-annex\git-annex-tmp\Remote\WebDAV.o ) -[137 of 217] Compiling Remote.Glacier ( Remote\Glacier.hs, dist\build\git-annex\git-annex-tmp\Remote\Glacier.o ) -[138 of 217] Compiling Remote.Hook ( Remote\Hook.hs, dist\build\git-annex\git-annex-tmp\Remote\Hook.o ) -[139 of 217] Compiling Annex.Ssh ( Annex\Ssh.hs, dist\build\git-annex\git-annex-tmp\Annex\Ssh.o ) - -Annex\Ssh.hs:21:1: Warning: - The import of `Annex.Perms' is redundant - except perhaps to import instances from `Annex.Perms' - To import instances alone, use: import Annex.Perms() -[140 of 217] Compiling Remote.Rsync ( Remote\Rsync.hs, dist\build\git-annex\git-annex-tmp\Remote\Rsync.o ) -[141 of 217] Compiling Remote.Helper.Ssh ( Remote\Helper\Ssh.hs, dist\build\git-annex\git-annex-tmp\Remote\Helper\Ssh.o ) -[142 of 217] Compiling Remote.Git ( Remote\Git.hs, dist\build\git-annex\git-annex-tmp\Remote\Git.o ) - -Remote\Git.hs:20:1: Warning: - The import of `Utility.CopyFile' is redundant - except perhaps to import instances from `Utility.CopyFile' - To import instances alone, use: import Utility.CopyFile() - -Remote\Git.hs:360:21: Warning: Defined but not used: `r' - -Remote\Git.hs:360:23: Warning: Defined but not used: `key' - -Remote\Git.hs:360:27: Warning: Defined but not used: `file' -[143 of 217] Compiling Remote.Bup ( Remote\Bup.hs, dist\build\git-annex\git-annex-tmp\Remote\Bup.o ) -[144 of 217] Compiling Remote.List ( Remote\List.hs, dist\build\git-annex\git-annex-tmp\Remote\List.o ) -[145 of 217] Compiling Logs.Trust ( Logs\Trust.hs, dist\build\git-annex\git-annex-tmp\Logs\Trust.o ) -[146 of 217] Compiling Remote ( Remote.hs, dist\build\git-annex\git-annex-tmp\Remote.o ) -[147 of 217] Compiling Limit ( Limit.hs, dist\build\git-annex\git-annex-tmp\Limit.o ) -[148 of 217] Compiling Option ( Option.hs, dist\build\git-annex\git-annex-tmp\Option.o ) -[149 of 217] Compiling Annex.FileMatcher ( Annex\FileMatcher.hs, dist\build\git-annex\git-annex-tmp\Annex\FileMatcher.o ) -[150 of 217] Compiling Logs.PreferredContent ( Logs\PreferredContent.hs, dist\build\git-annex\git-annex-tmp\Logs\PreferredContent.o ) -[151 of 217] Compiling Annex.Wanted ( Annex\Wanted.hs, dist\build\git-annex\git-annex-tmp\Annex\Wanted.o ) -[152 of 217] Compiling Seek ( Seek.hs, dist\build\git-annex\git-annex-tmp\Seek.o ) -[153 of 217] Compiling Checks ( Checks.hs, dist\build\git-annex\git-annex-tmp\Checks.o ) -[154 of 217] Compiling Command ( Command.hs, dist\build\git-annex\git-annex-tmp\Command.o ) -[155 of 217] Compiling Logs.Unused ( Logs\Unused.hs, dist\build\git-annex\git-annex-tmp\Logs\Unused.o ) -[156 of 217] Compiling CmdLine ( CmdLine.hs, dist\build\git-annex\git-annex-tmp\CmdLine.o ) -[157 of 217] Compiling Command.ConfigList ( Command\ConfigList.hs, dist\build\git-annex\git-annex-tmp\Command\ConfigList.o ) -[158 of 217] Compiling Command.InAnnex ( Command\InAnnex.hs, dist\build\git-annex\git-annex-tmp\Command\InAnnex.o ) -[159 of 217] Compiling Command.DropKey ( Command\DropKey.hs, dist\build\git-annex\git-annex-tmp\Command\DropKey.o ) -[160 of 217] Compiling Command.SendKey ( Command\SendKey.hs, dist\build\git-annex\git-annex-tmp\Command\SendKey.o ) -[161 of 217] Compiling Command.RecvKey ( Command\RecvKey.hs, dist\build\git-annex\git-annex-tmp\Command\RecvKey.o ) -[162 of 217] Compiling Command.TransferInfo ( Command\TransferInfo.hs, dist\build\git-annex\git-annex-tmp\Command\TransferInfo.o ) -[163 of 217] Compiling Command.Commit ( Command\Commit.hs, dist\build\git-annex\git-annex-tmp\Command\Commit.o ) -[164 of 217] Compiling GitAnnex.Options ( GitAnnex\Options.hs, dist\build\git-annex\git-annex-tmp\GitAnnex\Options.o ) -[165 of 217] Compiling Command.Unannex ( Command\Unannex.hs, dist\build\git-annex\git-annex-tmp\Command\Unannex.o ) -[166 of 217] Compiling Command.FromKey ( Command\FromKey.hs, dist\build\git-annex\git-annex-tmp\Command\FromKey.o ) -[167 of 217] Compiling Command.Fix ( Command\Fix.hs, dist\build\git-annex\git-annex-tmp\Command\Fix.o ) -[168 of 217] Compiling Command.Init ( Command\Init.hs, dist\build\git-annex\git-annex-tmp\Command\Init.o ) -[169 of 217] Compiling Command.Describe ( Command\Describe.hs, dist\build\git-annex\git-annex-tmp\Command\Describe.o ) -[170 of 217] Compiling Command.InitRemote ( Command\InitRemote.hs, dist\build\git-annex\git-annex-tmp\Command\InitRemote.o ) -[171 of 217] Compiling Command.EnableRemote ( Command\EnableRemote.hs, dist\build\git-annex\git-annex-tmp\Command\EnableRemote.o ) -[172 of 217] Compiling Command.Unused ( Command\Unused.hs, dist\build\git-annex\git-annex-tmp\Command\Unused.o ) -[173 of 217] Compiling Command.Unlock ( Command\Unlock.hs, dist\build\git-annex\git-annex-tmp\Command\Unlock.o ) -[174 of 217] Compiling Command.Lock ( Command\Lock.hs, dist\build\git-annex\git-annex-tmp\Command\Lock.o ) -[175 of 217] Compiling Command.Find ( Command\Find.hs, dist\build\git-annex\git-annex-tmp\Command\Find.o ) -[176 of 217] Compiling Command.Whereis ( Command\Whereis.hs, dist\build\git-annex\git-annex-tmp\Command\Whereis.o ) -[177 of 217] Compiling Command.Log ( Command\Log.hs, dist\build\git-annex\git-annex-tmp\Command\Log.o ) -[178 of 217] Compiling Command.Merge ( Command\Merge.hs, dist\build\git-annex\git-annex-tmp\Command\Merge.o ) -[179 of 217] Compiling Command.Uninit ( Command\Uninit.hs, dist\build\git-annex\git-annex-tmp\Command\Uninit.o ) -[180 of 217] Compiling Command.Trust ( Command\Trust.hs, dist\build\git-annex\git-annex-tmp\Command\Trust.o ) -[181 of 217] Compiling Command.Untrust ( Command\Untrust.hs, dist\build\git-annex\git-annex-tmp\Command\Untrust.o ) -[182 of 217] Compiling Command.Semitrust ( Command\Semitrust.hs, dist\build\git-annex\git-annex-tmp\Command\Semitrust.o ) -[183 of 217] Compiling Command.Dead ( Command\Dead.hs, dist\build\git-annex\git-annex-tmp\Command\Dead.o ) -[184 of 217] Compiling Command.Group ( Command\Group.hs, dist\build\git-annex\git-annex-tmp\Command\Group.o ) -[185 of 217] Compiling Command.Content ( Command\Content.hs, dist\build\git-annex\git-annex-tmp\Command\Content.o ) -[186 of 217] Compiling Command.Ungroup ( Command\Ungroup.hs, dist\build\git-annex\git-annex-tmp\Command\Ungroup.o ) -[187 of 217] Compiling Command.Vicfg ( Command\Vicfg.hs, dist\build\git-annex\git-annex-tmp\Command\Vicfg.o ) -[188 of 217] Compiling Command.RmUrl ( Command\RmUrl.hs, dist\build\git-annex\git-annex-tmp\Command\RmUrl.o ) -[189 of 217] Compiling Command.Map ( Command\Map.hs, dist\build\git-annex\git-annex-tmp\Command\Map.o ) -[190 of 217] Compiling Command.Upgrade ( Command\Upgrade.hs, dist\build\git-annex\git-annex-tmp\Command\Upgrade.o ) -[191 of 217] Compiling Command.Version ( Command\Version.hs, dist\build\git-annex\git-annex-tmp\Command\Version.o ) -[192 of 217] Compiling Command.Test ( Command\Test.hs, dist\build\git-annex\git-annex-tmp\Command\Test.o ) -[193 of 217] Compiling Command.Add ( Command\Add.hs, dist\build\git-annex\git-annex-tmp\Command\Add.o ) -[194 of 217] Compiling Command.ReKey ( Command\ReKey.hs, dist\build\git-annex\git-annex-tmp\Command\ReKey.o ) -[195 of 217] Compiling Command.AddUnused ( Command\AddUnused.hs, dist\build\git-annex\git-annex-tmp\Command\AddUnused.o ) -[196 of 217] Compiling Command.PreCommit ( Command\PreCommit.hs, dist\build\git-annex\git-annex-tmp\Command\PreCommit.o ) -[197 of 217] Compiling Command.Import ( Command\Import.hs, dist\build\git-annex\git-annex-tmp\Command\Import.o ) -[198 of 217] Compiling Command.Drop ( Command\Drop.hs, dist\build\git-annex\git-annex-tmp\Command\Drop.o ) -[199 of 217] Compiling Command.Move ( Command\Move.hs, dist\build\git-annex\git-annex-tmp\Command\Move.o ) -[200 of 217] Compiling Command.Copy ( Command\Copy.hs, dist\build\git-annex\git-annex-tmp\Command\Copy.o ) -[201 of 217] Compiling Command.Get ( Command\Get.hs, dist\build\git-annex\git-annex-tmp\Command\Get.o ) -[202 of 217] Compiling Command.TransferKey ( Command\TransferKey.hs, dist\build\git-annex\git-annex-tmp\Command\TransferKey.o ) -[203 of 217] Compiling Command.DropUnused ( Command\DropUnused.hs, dist\build\git-annex\git-annex-tmp\Command\DropUnused.o ) -[204 of 217] Compiling Command.Fsck ( Command\Fsck.hs, dist\build\git-annex\git-annex-tmp\Command\Fsck.o ) -[205 of 217] Compiling Command.Reinject ( Command\Reinject.hs, dist\build\git-annex\git-annex-tmp\Command\Reinject.o ) -[206 of 217] Compiling Command.Migrate ( Command\Migrate.hs, dist\build\git-annex\git-annex-tmp\Command\Migrate.o ) -[207 of 217] Compiling Command.Status ( Command\Status.hs, dist\build\git-annex\git-annex-tmp\Command\Status.o ) -[208 of 217] Compiling Command.Sync ( Command\Sync.hs, dist\build\git-annex\git-annex-tmp\Command\Sync.o ) -[209 of 217] Compiling Command.Help ( Command\Help.hs, dist\build\git-annex\git-annex-tmp\Command\Help.o ) -[210 of 217] Compiling Command.AddUrl ( Command\AddUrl.hs, dist\build\git-annex\git-annex-tmp\Command\AddUrl.o ) -[211 of 217] Compiling Command.Direct ( Command\Direct.hs, dist\build\git-annex\git-annex-tmp\Command\Direct.o ) -[212 of 217] Compiling Command.Indirect ( Command\Indirect.hs, dist\build\git-annex\git-annex-tmp\Command\Indirect.o ) -[213 of 217] Compiling Command.FuzzTest ( Command\FuzzTest.hs, dist\build\git-annex\git-annex-tmp\Command\FuzzTest.o ) -[214 of 217] Compiling Test ( Test.hs, dist\build\git-annex\git-annex-tmp\Test.o ) -[215 of 217] Compiling GitAnnexShell ( GitAnnexShell.hs, dist\build\git-annex\git-annex-tmp\GitAnnexShell.o ) -[216 of 217] Compiling GitAnnex ( GitAnnex.hs, dist\build\git-annex\git-annex-tmp\GitAnnex.o ) -[217 of 217] Compiling Main ( git-annex.hs, dist\build\git-annex\git-annex-tmp\Main.o ) -Linking dist\build\git-annex\git-annex.exe ... -+ cabal install nsis -Resolving dependencies... -All the requested packages are already installed: -nsis-0.2.2 -Use --reinstall if you want to reinstall anyway. -+ ghc --make Build/NullSoftInstaller.hs -[15 of 18] Compiling Build.SysConfig ( Build\SysConfig.hs, Build\SysConfig.o ) -Linking Build\NullSoftInstaller.exe ... -+ withcyg Build/NullSoftInstaller.exe -+ PATH='/c/Program Files (x86)/Haskell Platform/2012.4.0.0/bin:/c/Program Files (x86)/Haskell Platform/2012.4.0.0/lib/extralibs/bin:/c/Program Files (x86)/NSIS:/home/Oliver/bin:.:/usr/local/bin:/mingw/bin:/bin:/c/Program Files (x86)/Haskell/bin:/c/Program Files (x86)/Haskell Platform/2013.2.0.0/lib/extralibs/bin:/c/Program Files (x86)/Haskell Platform/2013.2.0.0/bin:/c/Program Files (x86)/AMD APP/bin/x86_64:/c/Program Files (x86)/AMD APP/bin/x86:/c/Windows/system32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0/:/c/Program Files (x86)/ATI Technologies/ATI.ACE/Core-Static:/c/Program Files (x86)/Windows Kits/8.0/Windows Performance Toolkit/:/c/Program Files/Microsoft/Web Platform Installer/:/c/Program Files (x86)/Haskell Platform/2013.2.0.0/mingw/bin:/c/Users/Oliver/AppData/Roaming/cabal/bin:/c/cygwin/bin' -+ Build/NullSoftInstaller.exe -MakeNSIS v2.46 - Copyright 1995-2009 Contributors -See the file COPYING for license details. -Credits can be found in the Users Manual. - -Processing config: -Processing plugin dlls: "c:\Program Files (x86)\NSIS\Plugins\*.dll" - - AdvSplash::show - - Banner::destroy - - Banner::getWindow - - Banner::show - - BgImage::AddImage - - BgImage::AddText - - BgImage::Clear - - BgImage::Destroy - - BgImage::Redraw - - BgImage::SetBg - - BgImage::SetReturn - - BgImage::Sound - - Dialer::AttemptConnect - - Dialer::AutodialHangup - - Dialer::AutodialOnline - - Dialer::AutodialUnattended - - Dialer::GetConnectedState - - InstallOptions::dialog - - InstallOptions::initDialog - - InstallOptions::show - - LangDLL::LangDialog - - Math::Script - - NSISdl::download - - NSISdl::download_quiet - - Splash::show - - StartMenu::Init - - StartMenu::Select - - StartMenu::Show - - System::Alloc - - System::Call - - System::Copy - - System::Free - - System::Get - - System::Int64Op - - System::Store - - TypeLib::GetLibVersion - - TypeLib::Register - - TypeLib::UnRegister - - UserInfo::GetAccountType - - UserInfo::GetName - - UserInfo::GetOriginalAccountType - - VPatch::GetFileCRC32 - - VPatch::GetFileMD5 - - VPatch::vpatchfile - - nsDialogs::Create - - nsDialogs::CreateControl - - nsDialogs::CreateItem - - nsDialogs::CreateTimer - - nsDialogs::GetUserData - - nsDialogs::KillTimer - - nsDialogs::OnBack - - nsDialogs::OnChange - - nsDialogs::OnClick - - nsDialogs::OnNotify - - nsDialogs::SelectFileDialog - - nsDialogs::SelectFolderDialog - - nsDialogs::SetRTL - - nsDialogs::SetUserData - - nsDialogs::Show - - nsExec::Exec - - nsExec::ExecToLog - - nsExec::ExecToStack - -!define: "MUI_INSERT_NSISCONF"="" - -Changing directory to: "C:\MinGW\msys\1.0\home\Oliver\src\git-annex" - -Processing script file: "git-annex.nsi" -!include: "c:\Program Files (x86)\NSIS\Include\MUI2.nsh" -!include: "c:\Program Files (x86)\NSIS\Contrib\Modern UI 2\MUI2.nsh" -NSIS Modern User Interface version 2.0 - Copyright 2002-2009 Joost Verburg (c:\Program Files (x86)\NSIS\Contrib\Modern UI 2\MUI2.nsh:8) -!define: "MUI_INCLUDED"="" -!define: "MUI_SYSVERSION"="2.0" -!define: "MUI_VERBOSE"="3" -!include: closed: "c:\Program Files (x86)\NSIS\Contrib\Modern UI 2\MUI2.nsh" -!include: closed: "c:\Program Files (x86)\NSIS\Include\MUI2.nsh" -Name: "git-annex" -OutFile: "git-annex-installer.exe" -InstallDir: "$PROGRAMFILES\Git\cmd" -!insertmacro: MUI_PAGE_DIRECTORY -!insertmacro: end of MUI_PAGE_DIRECTORY -!insertmacro: MUI_PAGE_LICENSE -!insertmacro: end of MUI_PAGEDECLARATION_LICENSE -!insertmacro: end of MUI_PAGE_LICENSE -!insertmacro: MUI_PAGE_INSTFILES -!insertmacro: end of MUI_PAGE_INSTFILES -!insertmacro: MUI_LANGUAGE -!insertmacro: end of MUI_LANGUAGE -Section: "main" ->(_sec10) -SetOutPath: "$INSTDIR" -File: "git-annex.exe" [compress] 7664822/31704766 bytes -File: "git-annex-licenses.txt" [compress] 60509/237415 bytes -File: "cp.exe" [compress] 62591/116736 bytes -File: "xargs.exe" [compress] 17002/33280 bytes -File: "rsync.exe" [compress] 188346/359424 bytes -File: "ssh.exe" [compress] 157190/320000 bytes -File: "wget.exe" [compress] 155666/349184 bytes -File: "sha1sum.exe" [compress] 20337/39424 bytes -File: "sha256sum.exe" [compress] 18279/37390 bytes -File: "sha512sum.exe" [compress] 30393/92174 bytes -File: "sha224sum.exe" [compress] 18279/37390 bytes -File: "sha384sum.exe" [compress] 30392/92174 bytes -File: "cygwin1.dll" [compress] 1036075/2874639 bytes -File: "cygasn1-8.dll" [compress] 154881/459293 bytes -File: "cygattr-1.dll" [compress] 5460/13838 bytes -File: "cygheimbase-1.dll" [compress] 4441/10781 bytes -File: "cygroken-18.dll" [compress] 26103/52253 bytes -File: "cygcom_err-2.dll" [compress] 3984/9757 bytes -File: "cygheimntlm-0.dll" [compress] 9018/20509 bytes -File: "cygsqlite3-0.dll" [compress] 334366/601629 bytes -File: "cygcrypt-0.dll" [compress] 3352/7182 bytes -File: "cyghx509-5.dll" [compress] 95839/216093 bytes -File: "cygssp-0.dll" [compress] 3377/8206 bytes -File: "cygcrypto-1.0.0.dll" [compress] 652766/1553920 bytes -File: "cygiconv-2.dll" [compress] 738899/1008654 bytes -File: "cyggcc_s-1.dll" [compress] 40241/80910 bytes -File: "cygintl-8.dll" [compress] 17737/35342 bytes -File: "cygwind-0.dll" [compress] 73172/160797 bytes -File: "cyggssapi-3.dll" [compress] 81662/183837 bytes -File: "cygkrb5-26.dll" [compress] 170204/381469 bytes -File: "cygz.dll" [compress] 42634/74269 bytes -WriteUninstaller: "git-annex-uninstall.exe" -SectionEnd -Section: "Uninstall" ->(_sec11) -Delete: /REBOOTOK "$INSTDIR\git-annex.exe" -Delete: /REBOOTOK "$INSTDIR\git-annex-licenses.txt" -Delete: /REBOOTOK "$INSTDIR\git-annex-uninstall.exe" -Delete: /REBOOTOK "$INSTDIR\cp.exe" -Delete: /REBOOTOK "$INSTDIR\xargs.exe" -Delete: /REBOOTOK "$INSTDIR\rsync.exe" -Delete: /REBOOTOK "$INSTDIR\ssh.exe" -Delete: /REBOOTOK "$INSTDIR\wget.exe" -Delete: /REBOOTOK "$INSTDIR\sha1sum.exe" -Delete: /REBOOTOK "$INSTDIR\sha256sum.exe" -Delete: /REBOOTOK "$INSTDIR\sha512sum.exe" -Delete: /REBOOTOK "$INSTDIR\sha224sum.exe" -Delete: /REBOOTOK "$INSTDIR\sha384sum.exe" -Delete: /REBOOTOK "$INSTDIR\cygwin1.dll" -Delete: /REBOOTOK "$INSTDIR\cygasn1-8.dll" -Delete: /REBOOTOK "$INSTDIR\cygattr-1.dll" -Delete: /REBOOTOK "$INSTDIR\cygheimbase-1.dll" -Delete: /REBOOTOK "$INSTDIR\cygroken-18.dll" -Delete: /REBOOTOK "$INSTDIR\cygcom_err-2.dll" -Delete: /REBOOTOK "$INSTDIR\cygheimntlm-0.dll" -Delete: /REBOOTOK "$INSTDIR\cygsqlite3-0.dll" -Delete: /REBOOTOK "$INSTDIR\cygcrypt-0.dll" -Delete: /REBOOTOK "$INSTDIR\cyghx509-5.dll" -Delete: /REBOOTOK "$INSTDIR\cygssp-0.dll" -Delete: /REBOOTOK "$INSTDIR\cygcrypto-1.0.0.dll" -Delete: /REBOOTOK "$INSTDIR\cygiconv-2.dll" -Delete: /REBOOTOK "$INSTDIR\cyggcc_s-1.dll" -Delete: /REBOOTOK "$INSTDIR\cygintl-8.dll" -Delete: /REBOOTOK "$INSTDIR\cygwind-0.dll" -Delete: /REBOOTOK "$INSTDIR\cyggssapi-3.dll" -Delete: /REBOOTOK "$INSTDIR\cygkrb5-26.dll" -Delete: /REBOOTOK "$INSTDIR\cygz.dll" -SectionEnd -Function: ".onInit" -IfFileExists: "$PROGRAMFILES\Git\cmd" ? _lbl3 : 0 -MessageBox: 48: "You need git installed to use git-annex. Looking at $PROGRAMFILES\Git\cmd , it seems to not be installed, or may be installed in another location. You can install git from http://git-scm.com/" (on IDOK goto 0) -FunctionEnd - -Processed 1 file, writing output: -Processing pages... Done! -Removing unused resources... Done! -Generating language tables... Done! -Generating uninstaller... Done! - -Output: "C:\MinGW\msys\1.0\home\Oliver\src\git-annex\git-annex-installer.exe" -Install: 4 pages (256 bytes), 1 section (1048 bytes), 98 instructions (2744 bytes), 133 strings (239643 bytes), 1 language table (278 bytes). -Uninstall: 2 pages (128 bytes), -1 section (1048 bytes), 33 instructions (924 bytes), 72 strings (1154 bytes), 1 language table (194 bytes). - -Using zlib compression. - -EXE header size: 48640 / 35840 bytes -Install code: 63270 / 244345 bytes -Install data: 11918141 / 41172867 bytes -Uninstall code+data: 10537 / 14897 bytes -CRC (0x090A4FD3): 4 / 4 bytes - -Total size: 12040592 / 41467953 bytes (29.0%) -+ rm -f last-incremental-failed -+ rm -rf .t -+ withcyg dist/build/git-annex/git-annex.exe test -+ PATH='/c/Program Files (x86)/Haskell Platform/2012.4.0.0/bin:/c/Program Files (x86)/Haskell Platform/2012.4.0.0/lib/extralibs/bin:/c/Program Files (x86)/NSIS:/home/Oliver/bin:.:/usr/local/bin:/mingw/bin:/bin:/c/Program Files (x86)/Haskell/bin:/c/Program Files (x86)/Haskell Platform/2013.2.0.0/lib/extralibs/bin:/c/Program Files (x86)/Haskell Platform/2013.2.0.0/bin:/c/Program Files (x86)/AMD APP/bin/x86_64:/c/Program Files (x86)/AMD APP/bin/x86:/c/Windows/system32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0/:/c/Program Files (x86)/ATI Technologies/ATI.ACE/Core-Static:/c/Program Files (x86)/Windows Kits/8.0/Windows Performance Toolkit/:/c/Program Files/Microsoft/Web Platform Installer/:/c/Program Files (x86)/Haskell Platform/2013.2.0.0/mingw/bin:/c/Users/Oliver/AppData/Roaming/cabal/bin:/c/cygwin/bin' -+ dist/build/git-annex/git-annex.exe test -(0 tests) (1 test) (2 tests) (3 tests) (4 tests) (5 tests) (6 tests) (7 tests) (8 tests) (9 tests) (10 tests) (11 tests) (12 tests) (13 tests) (14 tests) (15 tests) (16 tests) (17 tests) (18 tests) (19 tests) (20 tests) (21 tests) (22 tests) (23 tests) (24 tests) (25 tests) (26 tests) (27 tests) (28 tests) (29 tests) (30 tests) (31 tests) (32 tests) (33 tests) (34 tests) (35 tests) (36 tests) (37 tests) (38 tests) (39 tests) (40 tests) (41 tests) (42 tests) (43 tests) (44 tests) (45 tests) (46 tests) (47 tests) (48 tests) (49 tests) (50 tests) (51 tests) (52 tests) (53 tests) (54 tests) (55 tests) (56 tests) (57 tests) (58 tests) (59 tests) (60 tests) (61 tests) (62 tests) (63 tests) (64 tests) (65 tests) (66 tests) (67 tests) (68 tests) (69 tests) (70 tests) (71 tests) (72 tests) (73 tests) (74 tests) (75 tests) (76 tests) (77 tests) (78 tests) (79 tests) (80 tests) (81 tests) (82 tests) (83 tests) (84 tests) (85 tests) (86 tests) (87 tests) (88 tests) (89 tests) (90 tests) (91 tests) (92 tests) (93 tests) (94 tests) (95 tests) (96 tests) (97 tests) (98 tests) (99 tests) ---------------------------------------------------------------------- -First, some automated quick checks of properties ... ----------------------------------------------------------------------- -prop_idempotent_deencode_git -+++ OK, passed 100 tests. -(0 tests) (1 test) (2 tests) (3 tests) (4 tests) (5 tests) (6 tests) (7 tests) (8 tests) (9 tests) (10 tests) (11 tests) (12 tests) (13 tests) (14 tests) (15 tests) (16 tests) (17 tests) (18 tests) (19 tests) (20 tests) (21 tests) (22 tests) (23 tests) (24 tests) (25 tests) (26 tests) (27 tests) (28 tests) (29 tests) (30 tests) (31 tests) (32 tests) (33 tests) (34 tests) (35 tests) (36 tests) (37 tests) (38 tests) (39 tests) (40 tests) (41 tests) (42 tests) (43 tests) (44 tests) (45 tests) (46 tests) (47 tests) (48 tests) (49 tests) (50 tests) (51 tests) (52 tests) (53 tests) (54 tests) (55 tests) (56 tests) (57 tests) (58 tests) (59 tests) (60 tests) (61 tests) (62 tests) (63 tests) (64 tests) (65 tests) (66 tests) (67 tests) (68 tests) (69 tests) (70 tests) (71 tests) (72 tests) (73 tests) (74 tests) (75 tests) (76 tests) (77 tests) (78 tests) (79 tests) (80 tests) (81 tests) (82 tests) (83 tests) (84 tests) (85 tests) (86 tests) (87 tests) (88 tests) (89 tests) (90 tests) (91 tests) (92 tests) (93 tests) (94 tests) (95 tests) (96 tests) (97 tests) (98 tests) (99 tests) prop_idempotent_deencode -+++ OK, passed 100 tests. -(0 tests) (1 test) (2 tests) (3 tests) (4 tests) (5 tests) (6 tests) (7 tests) (8 tests) (9 tests) (10 tests) (11 tests) (12 tests) (13 tests) (14 tests) (15 tests) (16 tests) (17 tests) (18 tests) (19 tests) (20 tests) (21 tests) (22 tests) (23 tests) (24 tests) (25 tests) (26 tests) (27 tests) (28 tests) (29 tests) (30 tests) (31 tests) (32 tests) (33 tests) (34 tests) (35 tests) (36 tests) (37 tests) (38 tests) (39 tests) (40 tests) (41 tests) (42 tests) (43 tests) (44 tests) (45 tests) (46 tests) (47 tests) (48 tests) (49 tests) (50 tests) (51 tests) (52 tests) (53 tests) (54 tests) (55 tests) (56 tests) (57 tests) (58 tests) (59 tests) (60 tests) (61 tests) (62 tests) (63 tests) (64 tests) (65 tests) (66 tests) (67 tests) (68 tests) (69 tests) (70 tests) (71 tests) (72 tests) (73 tests) (74 tests) (75 tests) (76 tests) (77 tests) (78 tests) (79 tests) (80 tests) (81 tests) (82 tests) (83 tests) (84 tests) (85 tests) (86 tests) (87 tests) (88 tests) (89 tests) (90 tests) (91 tests) (92 tests) (93 tests) (94 tests) (95 tests) (96 tests) (97 tests) (98 tests) (99 tests) prop_idempotent_fileKey -+++ OK, passed 100 tests. -(0 tests) (1 test) (2 tests) (3 tests) (4 tests) (5 tests) (6 tests) (7 tests) (8 tests) (9 tests) (10 tests) (11 tests) (12 tests) (13 tests) (14 tests) (15 tests) (16 tests) (17 tests) (18 tests) (19 tests) (20 tests) (21 tests) (22 tests) (23 tests) (24 tests) (25 tests) (26 tests) (27 tests) (28 tests) (29 tests) (30 tests) (31 tests) (32 tests) (33 tests) (34 tests) (35 tests) (36 tests) (37 tests) (38 tests) (39 tests) (40 tests) (41 tests) (42 tests) (43 tests) (44 tests) (45 tests) (46 tests) (47 tests) (48 tests) (49 tests) (50 tests) (51 tests) (52 tests) (53 tests) (54 tests) (55 tests) (56 tests) (57 tests) (58 tests) (59 tests) (60 tests) (61 tests) (62 tests) (63 tests) (64 tests) (65 tests) (66 tests) (67 tests) (68 tests) (69 tests) (70 tests) (71 tests) (72 tests) (73 tests) (74 tests) (75 tests) (76 tests) (77 tests) (78 tests) (79 tests) (80 tests) (81 tests) (82 tests) (83 tests) (84 tests) (85 tests) (86 tests) (87 tests) (88 tests) (89 tests) (90 tests) (91 tests) (92 tests) (93 tests) (94 tests) (95 tests) (96 tests) (97 tests) (98 tests) (99 tests) prop_idempotent_key_encode -+++ OK, passed 100 tests. -(0 tests) (1 test) (2 tests) (3 tests) (4 tests) (5 tests) (6 tests) (7 tests) (8 tests) (9 tests) (10 tests) (11 tests) (12 tests) (13 tests) (14 tests) (15 tests) (16 tests) (17 tests) (18 tests) (19 tests) (20 tests) (21 tests) (22 tests) (23 tests) (24 tests) (25 tests) (26 tests) (27 tests) (28 tests) (29 tests) (30 tests) (31 tests) (32 tests) (33 tests) (34 tests) (35 tests) (36 tests) (37 tests) (38 tests) (39 tests) (40 tests) (41 tests) (42 tests) (43 tests) (44 tests) (45 tests) (46 tests) (47 tests) (48 tests) (49 tests) (50 tests) (51 tests) (52 tests) (53 tests) (54 tests) (55 tests) (56 tests) (57 tests) (58 tests) (59 tests) (60 tests) (61 tests) (62 tests) (63 tests) (64 tests) (65 tests) (66 tests) (67 tests) (68 tests) (69 tests) (70 tests) (71 tests) (72 tests) (73 tests) (74 tests) (75 tests) (76 tests) (77 tests) (78 tests) (79 tests) (80 tests) (81 tests) (82 tests) (83 tests) (84 tests) (85 tests) (86 tests) (87 tests) (88 tests) (89 tests) (90 tests) (91 tests) (92 tests) (93 tests) (94 tests) (95 tests) (96 tests) (97 tests) (98 tests) (99 tests) prop_idempotent_shellEscape -+++ OK, passed 100 tests. -(0 tests) (1 test) (2 tests) (3 tests) (4 tests) (5 tests) (6 tests) (7 tests) (8 tests) (9 tests) (10 tests) (11 tests) (12 tests) (13 tests) (14 tests) (15 tests) (16 tests) (17 tests) (18 tests) (19 tests) (20 tests) (21 tests) (22 tests) (23 tests) (24 tests) (25 tests) (26 tests) (27 tests) (28 tests) (29 tests) (30 tests) (31 tests) (32 tests) (33 tests) (34 tests) (35 tests) (36 tests) (37 tests) (38 tests) (39 tests) (40 tests) (41 tests) (42 tests) (43 tests) (44 tests) (45 tests) (46 tests) (47 tests) (48 tests) (49 tests) (50 tests) (51 tests) (52 tests) (53 tests) (54 tests) (55 tests) (56 tests) (57 tests) (58 tests) (59 tests) (60 tests) (61 tests) (62 tests) (63 tests) (64 tests) (65 tests) (66 tests) (67 tests) (68 tests) (69 tests) (70 tests) (71 tests) (72 tests) (73 tests) (74 tests) (75 tests) (76 tests) (77 tests) (78 tests) (79 tests) (80 tests) (81 tests) (82 tests) (83 tests) (84 tests) (85 tests) (86 tests) (87 tests) (88 tests) (89 tests) (90 tests) (91 tests) (92 tests) (93 tests) (94 tests) (95 tests) (96 tests) (97 tests) (98 tests) (99 tests) prop_idempotent_shellEscape_multiword -+++ OK, passed 100 tests. -(0 tests) (1 test) (2 tests) (3 tests) (4 tests) (5 tests) (6 tests) (7 tests) (8 tests) (9 tests) (10 tests) (11 tests) (12 tests) (13 tests) (14 tests) (15 tests) (16 tests) (17 tests) (18 tests) (19 tests) (20 tests) (21 tests) (22 tests) (23 tests) (24 tests) (25 tests) (26 tests) (27 tests) (28 tests) (29 tests) (30 tests) (31 tests) (32 tests) (33 tests) (34 tests) (35 tests) (36 tests) (37 tests) (38 tests) (39 tests) (40 tests) (41 tests) (42 tests) (43 tests) (44 tests) (45 tests) (46 tests) (47 tests) (48 tests) (49 tests) (50 tests) (51 tests) (52 tests) (53 tests) (54 tests) (55 tests) (56 tests) (57 tests) (58 tests) (59 tests) (60 tests) (61 tests) (62 tests) (63 tests) (64 tests) (65 tests) (66 tests) (67 tests) (68 tests) (69 tests) (70 tests) (71 tests) (72 tests) (73 tests) (74 tests) (75 tests) (76 tests) (77 tests) (78 tests) (79 tests) (80 tests) (81 tests) (82 tests) (83 tests) (84 tests) (85 tests) (86 tests) (87 tests) (88 tests) (89 tests) (90 tests) (91 tests) (92 tests) (93 tests) (94 tests) (95 tests) (96 tests) (97 tests) (98 tests) (99 tests) prop_idempotent_configEscape -+++ OK, passed 100 tests. -(0 tests) (1 test) (2 tests) (3 tests) (4 tests) (5 tests) (6 tests) (7 tests) (8 tests) (9 tests) (10 tests) (11 tests) (12 tests) (13 tests) (14 tests) (15 tests) (16 tests) (17 tests) (18 tests) (19 tests) (20 tests) (21 tests) (22 tests) (23 tests) (24 tests) (25 tests) (26 tests) (27 tests) (28 tests) (29 tests) (30 tests) (31 tests) (32 tests) (33 tests) (34 tests) (35 tests) (36 tests) (37 tests) (38 tests) (39 tests) (40 tests) (41 tests) (42 tests) (43 tests) (44 tests) (45 tests) (46 tests) (47 tests) (48 tests) (49 tests) (50 tests) (51 tests) (52 tests) (53 tests) (54 tests) (55 tests) (56 tests) (57 tests) (58 tests) (59 tests) (60 tests) (61 tests) (62 tests) (63 tests) (64 tests) (65 tests) (66 tests) (67 tests) (68 tests) (69 tests) (70 tests) (71 tests) (72 tests) (73 tests) (74 tests) (75 tests) (76 tests) (77 tests) (78 tests) (79 tests) (80 tests) (81 tests) (82 tests) (83 tests) (84 tests) (85 tests) (86 tests) (87 tests) (88 tests) (89 tests) (90 tests) (91 tests) (92 tests) (93 tests) (94 tests) (95 tests) (96 tests) (97 tests) (98 tests) (99 tests) prop_parse_show_Config -+++ OK, passed 100 tests. -(0 tests) (1 test) (2 tests) (3 tests) (4 tests) (5 tests) (6 tests) (7 tests) (8 tests) (9 tests) (10 tests) (11 tests) (12 tests) (13 tests) (14 tests) (15 tests) (16 tests) (17 tests) (18 tests) (19 tests) (20 tests) (21 tests) (22 tests) (23 tests) (24 tests) (25 tests) (26 tests) (27 tests) (28 tests) (29 tests) (30 tests) (31 tests) (32 tests) (33 tests) (34 tests) (35 tests) (36 tests) (37 tests) (38 tests) (39 tests) (40 tests) (41 tests) (42 tests) (43 tests) (44 tests) (45 tests) (46 tests) (47 tests) (48 tests) (49 tests) (50 tests) (51 tests) (52 tests) (53 tests) (54 tests) (55 tests) (56 tests) (57 tests) (58 tests) (59 tests) (60 tests) (61 tests) (62 tests) (63 tests) (64 tests) (65 tests) (66 tests) (67 tests) (68 tests) (69 tests) (70 tests) (71 tests) (72 tests) (73 tests) (74 tests) (75 tests) (76 tests) (77 tests) (78 tests) (79 tests) (80 tests) (81 tests) (82 tests) (83 tests) (84 tests) (85 tests) (86 tests) (87 tests) (88 tests) (89 tests) (90 tests) (91 tests) (92 tests) (93 tests) (94 tests) (95 tests) (96 tests) (97 tests) (98 tests) (99 tests) prop_parentDir_basics -+++ OK, passed 100 tests. -(0 tests) (1 test) (2 tests) (3 tests) (4 tests) (5 tests) (6 tests) (7 tests) (8 tests) (9 tests) (10 tests) (11 tests) (12 tests) (13 tests) (14 tests) (15 tests) (16 tests) (17 tests) (18 tests) (19 tests) (20 tests) (21 tests) (22 tests) (23 tests) (24 tests) (25 tests) (26 tests) (27 tests) (28 tests) (29 tests) (30 tests) (31 tests) (32 tests) (33 tests) (34 tests) (35 tests) (36 tests) (37 tests) (38 tests) (39 tests) (40 tests) (41 tests) (42 tests) (43 tests) (44 tests) (45 tests) (46 tests) (47 tests) (48 tests) (49 tests) (50 tests) (51 tests) (52 tests) (53 tests) (54 tests) (55 tests) (56 tests) (57 tests) (58 tests) (59 tests) (60 tests) (61 tests) (62 tests) (63 tests) (64 tests) (65 tests) (66 tests) (67 tests) (68 tests) (69 tests) (70 tests) (71 tests) (72 tests) (73 tests) (74 tests) (75 tests) (76 tests) (77 tests) (78 tests) (79 tests) (80 tests) (81 tests) (82 tests) (83 tests) (84 tests) (85 tests) (86 tests) (87 tests) (88 tests) (89 tests) (90 tests) (91 tests) (92 tests) (93 tests) (94 tests) (95 tests) (96 tests) (97 tests) (98 tests) (99 tests) prop_relPathDirToFile_basics -+++ OK, passed 100 tests. -(0 tests) prop_relPathDirToFile_regressionTest -+++ OK, passed 1 tests. -(0 tests) prop_cost_sane -+++ OK, passed 1 tests. -(0 tests) prop_matcher_sane -+++ OK, passed 1 tests. -(0 tests) prop_HmacSha1WithCipher_sane -+++ OK, passed 1 tests. -(0 tests) prop_TimeStamp_sane -+++ OK, passed 1 tests. -(0 tests) prop_addLog_sane -+++ OK, passed 1 tests. -(0 tests) (1 test) (2 tests) (3 tests) (4 tests) (5 tests) (6 tests) (7 tests) (8 tests) (9 tests) (10 tests) (11 tests) (12 tests) (13 tests) (14 tests) (15 tests) (16 tests) (17 tests) (18 tests) (19 tests) (20 tests) (21 tests) (22 tests) (23 tests) (24 tests) (25 tests) (26 tests) (27 tests) (28 tests) (29 tests) (30 tests) (31 tests) (32 tests) (33 tests) (34 tests) (35 tests) (36 tests) (37 tests) (38 tests) (39 tests) (40 tests) (41 tests) (42 tests) (43 tests) (44 tests) (45 tests) (46 tests) (47 tests) (48 tests) (49 tests) (50 tests) (51 tests) (52 tests) (53 tests) (54 tests) (55 tests) (56 tests) (57 tests) (58 tests) (59 tests) (60 tests) (61 tests) (62 tests) (63 tests) (64 tests) (65 tests) (66 tests) (67 tests) (68 tests) (69 tests) (70 tests) (71 tests) (72 tests) (73 tests) (74 tests) (75 tests) (76 tests) (77 tests) (78 tests) (79 tests) (80 tests) (81 tests) (82 tests) (83 tests) (84 tests) (85 tests) (86 tests) (87 tests) (88 tests) (89 tests) (90 tests) (91 tests) (92 tests) (93 tests) (94 tests) (95 tests) (96 tests) (97 tests) (98 tests) (99 tests) prop_verifiable_sane -+++ OK, passed 100 tests. -(0 tests) prop_segment_regressionTest -+++ OK, passed 1 tests. -(0 tests) (1 test) (2 tests) (3 tests) (4 tests) (5 tests) (6 tests) (7 tests) (8 tests) (9 tests) (10 tests) (11 tests) (12 tests) (13 tests) (14 tests) (15 tests) (16 tests) (17 tests) (18 tests) (19 tests) (20 tests) (21 tests) (22 tests) (23 tests) (24 tests) (25 tests) (26 tests) (27 tests) (28 tests) (29 tests) (30 tests) (31 tests) (32 tests) (33 tests) (34 tests) (35 tests) (36 tests) (37 tests) (38 tests) (39 tests) (40 tests) (41 tests) (42 tests) (43 tests) (44 tests) (45 tests) (46 tests) (47 tests) (48 tests) (49 tests) (50 tests) (51 tests) (52 tests) (53 tests) (54 tests) (55 tests) (56 tests) (57 tests) (58 tests) (59 tests) (60 tests) (61 tests) (62 tests) (63 tests) (64 tests) (65 tests) (66 tests) (67 tests) (68 tests) (69 tests) (70 tests) (71 tests) (72 tests) (73 tests) (74 tests) (75 tests) (76 tests) (77 tests) (78 tests) (79 tests) (80 tests) (81 tests) (82 tests) (83 tests) (84 tests) (85 tests) (86 tests) (87 tests) (88 tests) (89 tests) (90 tests) (91 tests) (92 tests) (93 tests) (94 tests) (95 tests) (96 tests) (97 tests) (98 tests) (99 tests) prop_read_write_transferinfo -+++ OK, passed 100 tests. -(0 tests) (1 test) (2 tests) (3 tests) (4 tests) (5 tests) (6 tests) (7 tests) (8 tests) (9 tests) (10 tests) (11 tests) (12 tests) (13 tests) (14 tests) (15 tests) (16 tests) (17 tests) (18 tests) (19 tests) (20 tests) (21 tests) (22 tests) (23 tests) (24 tests) (25 tests) (26 tests) (27 tests) (28 tests) (29 tests) (30 tests) (31 tests) (32 tests) (33 tests) (34 tests) (35 tests) (36 tests) (37 tests) (38 tests) (39 tests) (40 tests) (41 tests) (42 tests) (43 tests) (44 tests) (45 tests) (46 tests) (47 tests) (48 tests) (49 tests) (50 tests) (51 tests) (52 tests) (53 tests) (54 tests) (55 tests) (56 tests) (57 tests) (58 tests) (59 tests) (60 tests) (61 tests) (62 tests) (63 tests) (64 tests) (65 tests) (66 tests) (67 tests) (68 tests) (69 tests) (70 tests) (71 tests) (72 tests) (73 tests) (74 tests) (75 tests) (76 tests) (77 tests) (78 tests) (79 tests) (80 tests) (81 tests) (82 tests) (83 tests) (84 tests) (85 tests) (86 tests) (87 tests) (88 tests) (89 tests) (90 tests) (91 tests) (92 tests) (93 tests) (94 tests) (95 tests) (96 tests) (97 tests) (98 tests) (99 tests) prop_read_show_inodecache -+++ OK, passed 100 tests. -(0 tests) (1 test) (2 tests) (3 tests) (4 tests) (5 tests) (6 tests) (7 tests) (8 tests) (9 tests) (10 tests) (11 tests) (12 tests) (13 tests) (14 tests) (15 tests) (16 tests) (17 tests) (18 tests) (19 tests) (20 tests) (21 tests) (22 tests) (23 tests) (24 tests) (25 tests) (26 tests) (27 tests) (28 tests) (29 tests) (30 tests) (31 tests) (32 tests) (33 tests) (34 tests) (35 tests) (36 tests) (37 tests) (38 tests) (39 tests) (40 tests) (41 tests) (42 tests) (43 tests) (44 tests) (45 tests) (46 tests) (47 tests) (48 tests) (49 tests) (50 tests) (51 tests) (52 tests) (53 tests) (54 tests) (55 tests) (56 tests) (57 tests) (58 tests) (59 tests) (60 tests) (61 tests) (62 tests) (63 tests) (64 tests) (65 tests) (66 tests) (67 tests) (68 tests) (69 tests) (70 tests) (71 tests) (72 tests) (73 tests) (74 tests) (75 tests) (76 tests) (77 tests) (78 tests) (79 tests) (80 tests) (81 tests) (82 tests) (83 tests) (84 tests) (85 tests) (86 tests) (87 tests) (88 tests) (89 tests) (90 tests) (91 tests) (92 tests) (93 tests) (94 tests) (95 tests) (96 tests) (97 tests) (98 tests) (99 tests) prop_parse_show_log -+++ OK, passed 100 tests. -(0 tests) prop_read_show_TrustLevel -+++ OK, passed 1 tests. -(0 tests) prop_parse_show_TrustLog -+++ OK, passed 1 tests. - -Cases: 1 Tried: 0 Errors: 0 Failures: 0init test repo - Detected a crippled filesystem. - - Disabling core.symlinks. - - Enabling direct mode. - - Detected a filesystem without fifo support. - - Disabling ssh connection caching. -ok -(Recording state in git...) - - -Cases: 1 Tried: 1 Errors: 0 Failures: 0 - -Cases: 3 Tried: 0 Errors: 0 Failures: 0add foo (checksum...) ok -(Recording state in git...) -add sha1foo (checksum...) ok -(Recording state in git...) -add apple ok -(Recording state in git...) - -Cases: 3 Tried: 1 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex add:1 -git clone failed - -Cases: 3 Tried: 2 Errors: 0 Failures: 1ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex add:2 -git clone failed -Cases: 3 Tried: 3 Errors: 0 Failures: 2 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex reinject/fromkey -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 2 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex unannex:0:no content -git clone failed - -Cases: 2 Tried: 1 Errors: 0 Failures: 1ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex unannex:1:with content -git clone failed -Cases: 2 Tried: 2 Errors: 0 Failures: 2 - -Cases: 3 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex drop:0:no remotes -git clone failed - -Cases: 3 Tried: 1 Errors: 0 Failures: 1ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex drop:1:with remote -git clone failed - -Cases: 3 Tried: 2 Errors: 0 Failures: 2ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex drop:2:untrusted remote -git clone failed -Cases: 3 Tried: 3 Errors: 0 Failures: 3 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex get -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex move -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex copy -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex unlock/lock -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 2 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex edit/commit:0 -git clone failed - -Cases: 2 Tried: 1 Errors: 0 Failures: 1ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex edit/commit:1 -git clone failed -Cases: 2 Tried: 2 Errors: 0 Failures: 2 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex fix -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex trust/untrust/semitrust/dead -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 4 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex fsck:0 -git clone failed - -Cases: 4 Tried: 1 Errors: 0 Failures: 1ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex fsck:1 -git clone failed - -Cases: 4 Tried: 2 Errors: 0 Failures: 2ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex fsck:2 -git clone failed - -Cases: 4 Tried: 3 Errors: 0 Failures: 3ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex fsck:3 -git clone failed -Cases: 4 Tried: 4 Errors: 0 Failures: 4 - -Cases: 2 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex migrate:0 -git clone failed - -Cases: 2 Tried: 1 Errors: 0 Failures: 1ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex migrate:1 -git clone failed -Cases: 2 Tried: 2 Errors: 0 Failures: 2 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex unused/dropunused -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex describe -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex find -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex merge -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex status -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex version -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex sync -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: union merge regression -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: automatic conflict resolution -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex map -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex uninit -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex upgrade -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex whereis -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex hook remote -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex directory remote -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex rsync remote -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex bup remote -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex crypto -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 - -Cases: 1 Tried: 0 Errors: 0 Failures: 0ssh: Could not resolve hostname C: no address associated with name -fatal: The remote end hung up unexpectedly - - -### Failure in: git-annex preferred-content -git clone failed -Cases: 1 Tried: 1 Errors: 0 Failures: 1 ----------------------------------------------------------------------- -Now, some broader checks ... - (Do not be alarmed by odd output here; it's normal. - wait for the last line to see how it went.) ----------------------------------------------------------------------- -init ----------------------------------------------------------------------- -add ----------------------------------------------------------------------- -reinject ----------------------------------------------------------------------- -unannex ----------------------------------------------------------------------- -drop ----------------------------------------------------------------------- -get ----------------------------------------------------------------------- -move ----------------------------------------------------------------------- -copy ----------------------------------------------------------------------- -lock ----------------------------------------------------------------------- -edit ----------------------------------------------------------------------- -fix ----------------------------------------------------------------------- -trust ----------------------------------------------------------------------- -fsck ----------------------------------------------------------------------- -migrate ----------------------------------------------------------------------- - unused ----------------------------------------------------------------------- -describe ----------------------------------------------------------------------- -find ----------------------------------------------------------------------- -merge ----------------------------------------------------------------------- -status ----------------------------------------------------------------------- -version ----------------------------------------------------------------------- -sync ----------------------------------------------------------------------- -union merge regression ----------------------------------------------------------------------- -conflict resolution ----------------------------------------------------------------------- -map ----------------------------------------------------------------------- -uninit ----------------------------------------------------------------------- -upgrade ----------------------------------------------------------------------- -whereis ----------------------------------------------------------------------- -hook remote ----------------------------------------------------------------------- -directory remote ----------------------------------------------------------------------- -rsync remote ----------------------------------------------------------------------- -bup remote ----------------------------------------------------------------------- -crypto ----------------------------------------------------------------------- -preferred content ----------------------------------------------------------------------- -Some tests failed! - (This could be due to a bug in git-annex, or an incompatibility - with utilities, such as git, installed on this system.) - - -# End of transcript or log. -"""]] - -[[!tag moreinfo]] - -> Closing as this bug report is very old. [[done]] --[[Joey]] diff --git a/doc/bugs/Windows_build_test_failures/comment_1_ea7523fdbafdc8be2971df52d9038826._comment b/doc/bugs/Windows_build_test_failures/comment_1_ea7523fdbafdc8be2971df52d9038826._comment deleted file mode 100644 index dc09fb89b0..0000000000 --- a/doc/bugs/Windows_build_test_failures/comment_1_ea7523fdbafdc8be2971df52d9038826._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="http://joeyh.name/" - ip="4.154.2.134" - subject="comment 1" - date="2013-09-13T19:18:26Z" - content=""" -The Windows port has improved quite a lot since this bug was filed. Can you still reproduce it? - -If you install the pre-built git-annex for Windows, you can run `git annex test` to run the test suite. Does that work? It works for some others on Windows. -"""]] diff --git a/doc/bugs/Wrong_permission_when_using_shared_repository.mdwn b/doc/bugs/Wrong_permission_when_using_shared_repository.mdwn deleted file mode 100644 index bcb217dd01..0000000000 --- a/doc/bugs/Wrong_permission_when_using_shared_repository.mdwn +++ /dev/null @@ -1,68 +0,0 @@ -### Please describe the problem. -I just found that the annexed files are at risk of being altered as far as I understand when the repository is shared with other users. - -### What steps will reproduce the problem? -Go to a temporary location for these steps. -[[!format sh """ -$ git init . -$ git annex init main -$ git config core.sharedRepository group -$ echo foo > foo -$ git annex add foo -$ git annex sync -"""]] - -One can verify the file in the annex (aka in `.git.annex/objects`) is writable by the user and group. - -[[!format sh """ -$ ls -l foo -lrwxrwxrwx 1 christian christian 178 17. Aug 14:50 foo -> .git/annex/objects/91/9x/SHA256E-s4--b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c/SHA256E-s4--b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c -$ ls -lL foo --rw-rw-r-- 1 christian christian 4 17. Aug 16:03 foo -"""]] - -This makes it possible to alter the file in the annex without notification or previous unlocking. - -[[!format sh """ -$ echo bar > foo -$ git status -On branch master -nothing to commit, working tree clean -"""]] - -My understanding is that the files in the annex should be read-only to prevent just this. Only by `git annex unlock` a full copy is generated that can be altered by external programs. - -### What version of git-annex are you using? On what operating system? -``` -$ git annex version -git-annex version: 10.20220624-g17e4081d4 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime DebugLocks Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.30 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.11 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 8 -``` - -### Please provide any additional information below. - -The transcript is interleaved in the steps above. - -When repeating all steps but skipping the step `git config core.sharedRepository group`, the result is as expected: - -``` -$ ls -l foo -lrwxrwxrwx 1 christian christian 178 17. Aug 16:22 foo -> .git/annex/objects/91/9x/SHA256E-s4--b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c/SHA256E-s4--b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c -$ ls -lL foo --r--r--r-- 1 christian christian 4 17. Aug 16:22 foo -``` - -My motivation here is the following: I want to use the git-annex as an (external) storage backend for my Nextcloud. That will allow me to store the files deduplicated and effectively manageable way. Also, I can use it to have multiple views on my files depending on the need. As the Nextcloud server user is different from my personal user, I need a way to allow both users access to the annex. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -I have used git-annex in a single-user scenario yet. So far it works like a charm. I need more time to learn to get a hold of all functions but this is okay. - -> [[done]] in v10 --[[Joey]] diff --git a/doc/bugs/Wrong_permission_when_using_shared_repository/comment_1_bfcb858bfd1c5377c2b15961b59a835a._comment b/doc/bugs/Wrong_permission_when_using_shared_repository/comment_1_bfcb858bfd1c5377c2b15961b59a835a._comment deleted file mode 100644 index 8050f6822c..0000000000 --- a/doc/bugs/Wrong_permission_when_using_shared_repository/comment_1_bfcb858bfd1c5377c2b15961b59a835a._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-08-17T16:38:40Z" - content=""" -This was necessary to allow locking the object files. - -Repository v10 fixed that, so if you upgrade your repository, -you will no longer have this problem. Run `git-annex upgrade`. -"""]] diff --git a/doc/bugs/Wrong_permission_when_using_shared_repository/comment_2_e3c9e3b9004f1fe964b23d955083089b._comment b/doc/bugs/Wrong_permission_when_using_shared_repository/comment_2_e3c9e3b9004f1fe964b23d955083089b._comment deleted file mode 100644 index 16c4ec964c..0000000000 --- a/doc/bugs/Wrong_permission_when_using_shared_repository/comment_2_e3c9e3b9004f1fe964b23d955083089b._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="account+annex@806a7c5f67b75d530bced4ea294c77681ebdf766" - nickname="account+annex" - avatar="http://cdn.libravatar.org/avatar/f325a5448e1b6c79c2998c69ea1435ae" - subject="comment 2" - date="2022-08-17T16:59:16Z" - content=""" -Yes, thank you for the hint. I had to use `--force`. This I found on the help page of the update subprocess. - -Thank you very much! -"""]] diff --git a/doc/bugs/__34__.git__47__HEAD__58___openFile__58___resource_busy__34___after_copy.mdwn b/doc/bugs/__34__.git__47__HEAD__58___openFile__58___resource_busy__34___after_copy.mdwn deleted file mode 100644 index 96b21ca057..0000000000 --- a/doc/bugs/__34__.git__47__HEAD__58___openFile__58___resource_busy__34___after_copy.mdwn +++ /dev/null @@ -1,44 +0,0 @@ -### Please describe the problem. - -Sometimes `git annex copy` complains with `git-annex: .git/HEAD: openFile: resource busy (file is locked)`. For example: - -``` -$ git annex copy --from r1 --to r2 --not --in r2 -copy dir/foo/bar/baz.txt (from r1...) -(checksum...) (to r2...) -ok -copy dir/quux/buz/bez1.bin (from r1...) -(checksum...) (to r2...) -ok -copy dir/quux/buz/bez2.bin (from r1...) -(checksum...) (to r2...) -ok -copy dir/quux/buz/bez3.bin (from r1...) -(checksum...) (to r2...) -ok -git-annex: .git/HEAD: openFile: resource busy (file is locked) -``` - -This happens in a `adjusted/master(unlockpresent)` branch. - -everything seems to work fine regardless of the complaint, so I don't know if this warning message is just a cosmetic issue or if it is the tip of the iceberg, hiding a more serious problem. - -### What version of git-annex are you using? On what operating system? - -``` -git-annex version: 10.20230329-g30d7f9ad7 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22.1 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.13.1 persistent-sqlite-2.13.1.0 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2.1 -key/value backends: SHA256E [...] WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 8 -``` - -### Please provide any additional information below. - -This is probably related to [[bugs/Failed_adjusted_branch_update_after_error_in_drop/]] - -> [[fixed|done]] (I presume) --[[Joey]] diff --git a/doc/bugs/__34__.git__47__HEAD__58___openFile__58___resource_busy__34___after_copy/comment_1_b05e0060d503a1d58c972796c5be151b._comment b/doc/bugs/__34__.git__47__HEAD__58___openFile__58___resource_busy__34___after_copy/comment_1_b05e0060d503a1d58c972796c5be151b._comment deleted file mode 100644 index cfd162c934..0000000000 --- a/doc/bugs/__34__.git__47__HEAD__58___openFile__58___resource_busy__34___after_copy/comment_1_b05e0060d503a1d58c972796c5be151b._comment +++ /dev/null @@ -1,87 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="happens during sync too" - date="2023-04-03T11:48:09Z" - content=""" -I get the same complaint/error while on a hidemissing-unlocked adjusted branch (as a Windows user) -and trying to sync with a regular remote (which happens to be an unlocked adjusted branch). -Manually bisecting I found out that the problem first occurs with [[!commit 038a2600f4cf71294976280c5c29f6710359375f]] -(so \"00f\" indeed) which seems to confirm that the fixes made for the linked bug seem to be -the culprit. - -Here's a simple PowerShell transcript of me committing a disk image backup file to an annex -and then proceeding to `sync` with a remote - trying different versions of git-annex with the one -built from [[!commit cb4d9f7b1f8ce251d86f8f0163c0884cb9454603]] finally working, ie. doing a full -pull and push cycle (here `git annex-sync` is simply an alias for `git annex sync --no-commit`): - -[[!format sh \"\"\" -H:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(hidemissing-unlocked) +1 ~0 -0 !]> git annex version --raw -10.20230329-g000723d96 -H:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(hidemissing-unlocked) +1 ~0 -0 !]> time git annex add -add D2DB92DE496EC33F-06-06.mrimg -ok -(recording state in git...) -00:05:42.5080264 -H:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(hidemissing-unlocked) +1 ~0 -0 ~]> git status -Refresh index: 100% (33/33), done. -On branch adjusted/master(hidemissing-unlocked) -Changes to be committed: - (use \"git restore --staged ...\" to unstage) - new file: D2DB92DE496EC33F-06-06.mrimg - -H:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(hidemissing-unlocked) +1 ~0 -0 ~]> git annex-meta -t differentiaali -s varmistus=differentiaali D2DB92DE496EC33F-06-06.mrimg -metadata D2DB92DE496EC33F-06-06.mrimg - lastchanged=2023-04-03@09-31-18 - tag=differentiaali - tag-lastchanged=2023-04-03@09-31-18 - varmistus=differentiaali - varmistus-lastchanged=2023-04-03@09-31-18 -ok -(recording state in git...) -H:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(hidemissing-unlocked) +1 ~0 -0 ~]> git commit -m 'Lisätty: D2DB92DE496EC33F-06-06.mrimg; differentiaali' -[adjusted/master(hidemissing-unlocked) 6cc1044] Lisätty: D2DB92DE496EC33F-06-06.mrimg; differentiaali - 1 file changed, 1 insertion(+) - create mode 100644 Jarkon ThinkPad T450s (Win10 v21H1) . B/D2DB92DE496EC33F-06-06.mrimg -H:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(hidemissing-unlocked)]> git annex-sync -pull k-levyn-annex2 -From k:\Reflect-varmistukset - 3649f50..407d219 git-annex -> k-levyn-annex2/git-annex -ok -git-annex: .git\HEAD: openFile: resource busy (file is locked) -H:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(hidemissing-unlocked)]> git annex-sync -pull k-levyn-annex2 -ok -git-annex: .git\HEAD: openFile: resource busy (file is locked) -H:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(hidemissing-unlocked)]> git annex version --raw -10.20230329-g000723d96 -H:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(hidemissing-unlocked)]> git annex version --raw -10.20230322-g038a2600f -H:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(hidemissing-unlocked)]> git annex-sync -pull k-levyn-annex2 -ok -git-annex: .git\HEAD: openFile: resource busy (file is locked) -H:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(hidemissing-unlocked)]> git annex version --raw -10.20230322-gcb4d9f7b1 -H:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(hidemissing-unlocked)]> git annex-sync -pull k-levyn-annex2 -ok -push k-levyn-annex2 -Enumerating objects: 17, done. -Counting objects: 100% (17/17), done. -Delta compression using up to 4 threads -Compressing objects: 100% (12/12), done. -Writing objects: 100% (14/14), 1.27 KiB | 432.00 KiB/s, done. -Total 14 (delta 5), reused 0 (delta 0), pack-reused 0 -remote: Checking connectivity: 14, done. -To k:\Reflect-varmistukset - 3649f50..e420747 git-annex -> synced/git-annex - c74664e..6cab55f master -> synced/master -ok -H:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(hidemissing-unlocked)]> git annex version --raw -10.20230322-gcb4d9f7b1 -\"\"\"]] - - -"""]] diff --git a/doc/bugs/__34__.git__47__HEAD__58___openFile__58___resource_busy__34___after_copy/comment_2_399dcbe7f3772526efbe13bd5b13156d._comment b/doc/bugs/__34__.git__47__HEAD__58___openFile__58___resource_busy__34___after_copy/comment_2_399dcbe7f3772526efbe13bd5b13156d._comment deleted file mode 100644 index f102bfa530..0000000000 --- a/doc/bugs/__34__.git__47__HEAD__58___openFile__58___resource_busy__34___after_copy/comment_2_399dcbe7f3772526efbe13bd5b13156d._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2023-04-04T18:13:49Z" - content=""" -Clearly a laziness bug, so the fix seems obvious and I've commited it. - -But I've not managed to reproduce it. It makes sense it would be -intermittent. Please followup if you still see it after -[[!commit 3eb51ee929c75376d004341f41aa38a1488558a3]] -"""]] diff --git a/doc/bugs/__34__.git__47__HEAD__58___openFile__58___resource_busy__34___after_copy/comment_3_201c7dcd0e809cb2c22ab4b621079c71._comment b/doc/bugs/__34__.git__47__HEAD__58___openFile__58___resource_busy__34___after_copy/comment_3_201c7dcd0e809cb2c22ab4b621079c71._comment deleted file mode 100644 index 78815e33ae..0000000000 --- a/doc/bugs/__34__.git__47__HEAD__58___openFile__58___resource_busy__34___after_copy/comment_3_201c7dcd0e809cb2c22ab4b621079c71._comment +++ /dev/null @@ -1,62 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 3" - date="2023-04-05T09:42:11Z" - content=""" -I can confirm that things seem to be AOK now: - -[[!format sh \"\"\" -H:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(hidemissing-unlocked)]> git annex-sync -merge synced/master (Merging into master...) -Merge made by the 'ort' strategy. - Jarkon ThinkPad T450s (Win10 v21H1) . B/arkistoidut/BDF96DE98ECBB6EC-02-02.mrimg | 1 - - Jarkon ThinkPad T450s (Win10 v21H1) . B/arkistoidut/BDF96DE98ECBB6EC-03-03.mrimg | 1 - - Jarkon ThinkPad T450s (Win10 v21H1) . B/arkistoidut/BDF96DE98ECBB6EC-04-04.mrimg | 1 - - Jarkon ThinkPad T450s (Win10 v21H1) . B/arkistoidut/BDF96DE98ECBB6EC-05-05.mrimg | 1 - - Jarkon ThinkPad T450s (Win10 v21H1) . B/arkistoidut/BDF96DE98ECBB6EC-06-06.mrimg | 1 - - Jarkon ThinkPad T450s (Win10 v21H1) . B/arkistoidut/BDF96DE98ECBB6EC-07-07.mrimg | 1 - - Jarkon ThinkPad T450s (Win10 v21H1) . B/arkistoidut/BDF96DE98ECBB6EC-08-08.mrimg | 1 - - 7 files changed, 7 deletions(-) - delete mode 120000 Jarkon ThinkPad T450s (Win10 v21H1) . B/arkistoidut/BDF96DE98ECBB6EC-02-02.mrimg - delete mode 120000 Jarkon ThinkPad T450s (Win10 v21H1) . B/arkistoidut/BDF96DE98ECBB6EC-03-03.mrimg - delete mode 120000 Jarkon ThinkPad T450s (Win10 v21H1) . B/arkistoidut/BDF96DE98ECBB6EC-04-04.mrimg - delete mode 120000 Jarkon ThinkPad T450s (Win10 v21H1) . B/arkistoidut/BDF96DE98ECBB6EC-05-05.mrimg - delete mode 120000 Jarkon ThinkPad T450s (Win10 v21H1) . B/arkistoidut/BDF96DE98ECBB6EC-06-06.mrimg - delete mode 120000 Jarkon ThinkPad T450s (Win10 v21H1) . B/arkistoidut/BDF96DE98ECBB6EC-07-07.mrimg - delete mode 120000 Jarkon ThinkPad T450s (Win10 v21H1) . B/arkistoidut/BDF96DE98ECBB6EC-08-08.mrimg -(Merging into adjusted branch...) -Updating a21ddee..445c0c9 -Fast-forward -ok -pull k-levyn-annex2 -From k:\Reflect-varmistukset - e420747..6621032 git-annex -> k-levyn-annex2/git-annex - 6cab55f..7638c8c master -> k-levyn-annex2/master - 6cab55f..7638c8c synced/master -> k-levyn-annex2/synced/master -ok -git-annex: .git\HEAD: openFile: resource busy (file is locked) -H:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(hidemissing-unlocked)]> git annex version --raw -10.20230329-g000723d96 -H:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(hidemissing-unlocked)]> git annex version --raw -10.20230330-g3eb51ee92 -H:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(hidemissing-unlocked)]> git annex-sync -pull k-levyn-annex2 -ok -push k-levyn-annex2 -Enumerating objects: 11, done. -Counting objects: 100% (11/11), done. -Delta compression using up to 4 threads -Compressing objects: 100% (7/7), done. -Writing objects: 100% (7/7), 810 bytes | 405.00 KiB/s, done. -Total 7 (delta 4), reused 0 (delta 0), pack-reused 0 -To k:\Reflect-varmistukset - e420747..6621032 git-annex -> synced/git-annex - 7638c8c..73de8e2 master -> synced/master -ok -H:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(hidemissing-unlocked)]> git annex version --raw -10.20230330-g3eb51ee92 -\"\"\"]] - -Thank you so much, Joey, well done! 🎉 -"""]] diff --git a/doc/bugs/__34__directory__34___special_remote_leaves_empty_dirs.mdwn b/doc/bugs/__34__directory__34___special_remote_leaves_empty_dirs.mdwn deleted file mode 100644 index 148f57ebed..0000000000 --- a/doc/bugs/__34__directory__34___special_remote_leaves_empty_dirs.mdwn +++ /dev/null @@ -1,116 +0,0 @@ -### Please describe the problem. - -"directory" special remote leaves empty directories after dropping files. I -compared it to rsync, and he cleans empty dirs, so I wonder if this is a bug in -"directory". - -Also I think is it possible to make `testremote` also test for it? First I -found this behavior in rclone special remote and was very confused, then -compared to "directory" and got confused even more. - -### What steps will reproduce the problem? - -I made a little script :D, here it is: - -[[!format sh """ -#!/bin/bash - -test_dir=$(mktemp -d /tmp/test-empties.XXX) -cd "$test_dir" || exit 1 - -# create repo -mkdir rep && cd rep || exit 1 -git init -git annex init - -# add special remotes -mkdir ../sp_directory -git annex initremote sp_directory type=directory directory=../sp_directory encryption=none -git annex initremote sp_rsync type=rsync rsyncurl=../sp_rsync encryption=none - -# add file and copy to remotes -touch a -git annex add a -git commit -m commit -git annex copy a --to sp_directory -git annex copy a --to sp_rsync - - -test_it() { - name="$1" - - echo -e "\n\n" - echo "--------- special remote: $name ---------" - echo "--------- before drop: ---------" && tree "../sp_$name" - echo - git annex drop a --from "sp_$name" --force - echo - echo "--------- after drop: ---------" && tree "../sp_$name" -} - -test_it directory -test_it rsync -"""]] - -### What version of git-annex are you using? On what operating system? - -git-annex version: 10.20230626-g55fef4fb81 - -OS: archlinux - -### Please provide any additional information below. - -Output of my script: - -[[!format sh """ - - ---------- special remote: directory --------- ---------- before drop: --------- -../sp_directory -├── f87 -│   └── 4d5 -│   └── SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 -│   └── SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 -└── tmp - -5 directories, 1 file - -drop sp_directory a ok -(recording state in git...) - ---------- after drop: --------- -../sp_directory -├── f87 -│   └── 4d5 -└── tmp - -4 directories, 0 files - - - ---------- special remote: rsync --------- ---------- before drop: --------- -../sp_rsync -└── f87 - └── 4d5 - └── SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 - └── SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 - -4 directories, 1 file - -drop sp_rsync a ok -(recording state in git...) - ---------- after drop: --------- -../sp_rsync - -0 directories, 0 files -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -I'm tinkering with annex for about 1 week, "doing research" in preparation to -use it as storage for backups. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/__34__directory__34___special_remote_leaves_empty_dirs/comment_1_8cef8650103d118fbbe56d20a4f59f96._comment b/doc/bugs/__34__directory__34___special_remote_leaves_empty_dirs/comment_1_8cef8650103d118fbbe56d20a4f59f96._comment deleted file mode 100644 index 2d75add7f6..0000000000 --- a/doc/bugs/__34__directory__34___special_remote_leaves_empty_dirs/comment_1_8cef8650103d118fbbe56d20a4f59f96._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-07-21T18:08:52Z" - content=""" -Hmm, I guess this wasn't implemented because a generally small number of -hash directories are not a bother. And because implementing it was a bit -annoying. - -Testremote cannot test for this, because remotes do not need to have the -concept of a "directory". For example, a S3 bucket does not contain -directories, only filenames that may contain a "/" in them. Also, I don't -think it's necessarily a bug for a special remote to leave hash directories -in place when removing files. There could be good reasons to do that in -some cases. -"""]] diff --git a/doc/bugs/__34__git_status__34___thread_blocked_indefinitely_in_an_MVar_operation.mdwn b/doc/bugs/__34__git_status__34___thread_blocked_indefinitely_in_an_MVar_operation.mdwn deleted file mode 100644 index d3fae595bc..0000000000 --- a/doc/bugs/__34__git_status__34___thread_blocked_indefinitely_in_an_MVar_operation.mdwn +++ /dev/null @@ -1,44 +0,0 @@ -### Please describe the problem. - -"git status" returns a list of almost all files in the annex (located on a FAT32 device) with these messages: - - git-annex: git status will show music/a.flac to be modified, since content availability has changed and git-annex was unable to update the index. This is only a cosmetic problem affecting git status; git add, git commit, etc won't be affected. To fix the git status display, you can run: git update-index -q --refresh music/a.flac - ... (a lot of similar messages for +30000 files) - -and then a new list of messages: - - ... - git-annex: thread blocked indefinitely in an MVar operation - error: external filter 'git-annex smudge --clean %f' failed 1 - error: external filter 'git-annex smudge --clean %f' failed - git-annex: thread blocked indefinitely in an MVar operation - error: external filter 'git-annex smudge --clean %f' failed 1 - error: external filter 'git-annex smudge --clean %f' failed - ... - refresh index: 81% (30495/37394) - - - -### What steps will reproduce the problem? - - -### What version of git-annex are you using? On what operating system? - -debian sid, git-annex version: 7.20190129 - - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - - -> I think this got fixed by [[!commit 195508fc6541b8f03b78e2ae457422308fc8d9e2]]. -> [[done]] --[[Joey]] diff --git a/doc/bugs/__34__git_status__34___thread_blocked_indefinitely_in_an_MVar_operation/comment_1_a63994b33b651dccfd4d4cf946ab7170._comment b/doc/bugs/__34__git_status__34___thread_blocked_indefinitely_in_an_MVar_operation/comment_1_a63994b33b651dccfd4d4cf946ab7170._comment deleted file mode 100644 index aec5eb15e4..0000000000 --- a/doc/bugs/__34__git_status__34___thread_blocked_indefinitely_in_an_MVar_operation/comment_1_a63994b33b651dccfd4d4cf946ab7170._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2019-03-18T17:59:46Z" - content=""" -Does this still happen with version 7.20190220? - -I take it you're using v7 mode with unlocked files? -"""]] diff --git a/doc/bugs/__34__git_status__34___thread_blocked_indefinitely_in_an_MVar_operation/comment_2_2166a985e48a31c1b86cb77263f72ce2._comment b/doc/bugs/__34__git_status__34___thread_blocked_indefinitely_in_an_MVar_operation/comment_2_2166a985e48a31c1b86cb77263f72ce2._comment deleted file mode 100644 index 2e4e91678b..0000000000 --- a/doc/bugs/__34__git_status__34___thread_blocked_indefinitely_in_an_MVar_operation/comment_2_2166a985e48a31c1b86cb77263f72ce2._comment +++ /dev/null @@ -1,34 +0,0 @@ -[[!comment format=mdwn - username="gueux" - avatar="http://cdn.libravatar.org/avatar/47e44a21505727b2d6bb5d88f0468f34" - subject="comment 2" - date="2019-03-21T20:40:55Z" - content=""" -That's right, I'm using v7 mode with unlocked files. - -``` -[core] - repositoryformatversion = 0 - filemode = false - bare = false - logallrefupdates = true - symlinks = false - ignorecase = true -[annex] - uuid = 25e5f762-db7f-49ad-a504-93e02bba642f - sshcaching = false - crippledfilesystem = true - version = 7 - thin = true -[filter \"annex\"] - smudge = git-annex smudge %f - clean = git-annex smudge --clean %f -``` - - -With 7.20190220-g9d7663432, I now get: - - sqlite worker thread crashed: user error (SQLite3 returned ErrorIO while attempting to perform prepare \"SELECT null from content limit 1\": disk I/O error(while opening database connection)) - sqlite worker thread crashed: user error (SQLite3 returned ErrorIO while attempting to perform prepare \"SELECT null from content limit 1\": disk I/O error(while opening database connection)) - ... -"""]] diff --git a/doc/bugs/__34__git_status__34___thread_blocked_indefinitely_in_an_MVar_operation/comment_3_17b96f95f44a04063f5a52d690d96b32._comment b/doc/bugs/__34__git_status__34___thread_blocked_indefinitely_in_an_MVar_operation/comment_3_17b96f95f44a04063f5a52d690d96b32._comment deleted file mode 100644 index d07500a7c7..0000000000 --- a/doc/bugs/__34__git_status__34___thread_blocked_indefinitely_in_an_MVar_operation/comment_3_17b96f95f44a04063f5a52d690d96b32._comment +++ /dev/null @@ -1,32 +0,0 @@ -[[!comment format=mdwn - username="gueux" - avatar="http://cdn.libravatar.org/avatar/47e44a21505727b2d6bb5d88f0468f34" - subject="comment 3" - date="2019-03-21T22:57:36Z" - content=""" -The command above finally ended. Then `git annex sync` returns: - -``` -commit -fatal: Unable to create '/mnt/usb/audio/.git/index.lock': File exists. - -Another git process seems to be running in this repository, e.g. -an editor opened by 'git commit'. Please make sure all processes -are terminated then try again. If it still fails, a git process -may have crashed in this repository earlier: -remove the file manually to continue. -ok -fatal: Unable to create '.git/index.lock': .git/index.lock: openFd: already exists (File exists) - -If no other git process is currently running, this probably means a -git process crashed in this repository earlier. Make sure no other git -process is running and remove the file manually to continue. - - -git-annex: .git/index.lock: openFd: already exists (File exists) -failed -git-annex: sync: 1 failed -``` - -I think I found the problem: my filesystem is probably full after all, I just noticed that `annex.thin` isn't supported on FAT and that the files got duplicated :-(. Bad news for me and my perfect rockbox player backed up with git-annex! -"""]] diff --git a/doc/bugs/__34__git_status__34___thread_blocked_indefinitely_in_an_MVar_operation/comment_4_df73b69e651c728943a404223f350ebf._comment b/doc/bugs/__34__git_status__34___thread_blocked_indefinitely_in_an_MVar_operation/comment_4_df73b69e651c728943a404223f350ebf._comment deleted file mode 100644 index 92d5c77530..0000000000 --- a/doc/bugs/__34__git_status__34___thread_blocked_indefinitely_in_an_MVar_operation/comment_4_df73b69e651c728943a404223f350ebf._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2019-03-22T13:14:51Z" - content=""" -Ok, it makes sense that sqlite would crash creating a DB if the disk was -full. - -I hope that the earlier Mvar error was the same problem, just a different -handling of the exception in that version. Or something similar failing on -full disk. - -annex.thin needing hard links is a limitation that may one day be lifted, -see [[todo/annex.thin_without_hardlinks]] -"""]] diff --git a/doc/bugs/__39__Production__39___build_doesn__39__t_pass_testsuite_on_Win32.mdwn b/doc/bugs/__39__Production__39___build_doesn__39__t_pass_testsuite_on_Win32.mdwn deleted file mode 100644 index 4965962860..0000000000 --- a/doc/bugs/__39__Production__39___build_doesn__39__t_pass_testsuite_on_Win32.mdwn +++ /dev/null @@ -1,783 +0,0 @@ -### Please describe the problem. - -Building an optimized Windows binary with flag `Production` active doesn't result in one that is able -to pass its test suite. - -### What steps will reproduce the problem? - -`stack --stack-yaml stack-lts-18.13.yaml setup && stack --stack-yaml stack-lts-18.13.yaml build --flag 'git-annex:Production' --flag 'git-annex:-Crypton'` - -(I found that the `Crypton` flag needs to be explicitly negated due to the `Production` flag somehow making it active causing -the build to fail because of a missing dependency `crypton`. A bug in Cabal, perhaps?) - -Then run the test suite in Git Bash in a directory that contains the executable and the requisite DLLs (the latter -obtained by way of `stack exec ldd -- git-annex.exe` and copying over the ones in `/mingw64`). Observe the following -(`just` is a cross-platform command runner that I use instead of GNU make.) - -[[!format sh """ -jkniiv@AINESIS MINGW64 ~/Projektit/git-annex.branchable.com/git-annex--BUILD-231130-10.20231129~P/.annx-test (BUILD-231130-10.20231129) -$ just tests-new -/usr/bin/time ./git-annex.exe test 2>&1 | tee git-annex.test.LOG~200 - -All 0 tests passed (0.00s) -Tests - Repo Tests v10 adjusted unlocked branch - Init Tests - init: OK (9.36s) - add: FAIL - Exception: fd:4: hGetChar: invalid argument (invalid byte sequence) - Use -p '/Init Tests.add/' to rerun this test only. - crypto: SKIP - - Use -p '/crypto/' to rerun this test only. - uninit (in git-annex branch): SKIP - - Use -p '/uninit (in git-annex branch)/' to rerun this test only. - conflict resolution symlink bit: SKIP - - Use -p '/conflict resolution symlink bit/' to rerun this test only. - union merge regression: SKIP - - Use -p '/union merge regression/' to rerun this test only. - unused: SKIP - - Use -p '/unused/' to rerun this test only. - fsck (bare): SKIP - - Use -p '/fsck (bare)/' to rerun this test only. - lock: SKIP - - Use -p '/Repo Tests v10 adjusted unlocked branch.lock/' to rerun this test only. - drop (with remote): SKIP - - Use -p '/drop (with remote)/' to rerun this test only. - log: SKIP - - Use -p '/log/' to rerun this test only. - add extras: SKIP - - Use -p '/add extras/' to rerun this test only. - -11 out of 12 tests failed (11.65s) -[...] -"""]] - - -### What version of git-annex are you using? On what operating system? - -[[!format sh """ -git-annex version: 10.20231129-gbacd781c4fe05126219c4b5f2963677a6de01481 -build flags: Assistant Webapp Pairing TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.7 http-client-0.7.9 pe -rsistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.1.2 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SH -A3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 -BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLA -KE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM UR -L X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg h -ook external -operating system: mingw32 x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 2 3 4 5 6 7 8 9 10 -"""]] - -Windows 10 version 22H2 (build 19045.3693), 64 bit. - -### Please provide any additional information below. - -This is how I build my "production" version of git-annex (in PowerShell): - -[[!format sh """ -C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P [BUILD-231130-10.20231129 +6 ~0 -0 !]> just setup-build -cd 'C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P' && stack --stack-yaml stack-lts-18.13.yaml setup --verbose 2>&1 | & $env:GNU_TEE_CMD stack.setup---verbose.LOG~101 -Version 2.13.1, Git revision 8102bb8afce90fc954f48efae38b87f37cabc988 x86_64 hpack-0.36.0 -2023-12-08 16:24:52.098843: [debug] Loading project config file stack-lts-18.13.yaml -2023-12-08 16:24:52.130837: [debug] Use of Casa server enabled: (CasaRepoPrefix "https://casa.stackage.org", 1280). -2023-12-08 16:24:52.142835: [debug] (SQL) SELECT COUNT(*) FROM "last_performed" WHERE ("action"=?) AND ("timestamp">=?); [PersistInt64 1,PersistUTCTime 2023-12-07 14:24:52.1418345 UTC] -2023-12-08 16:24:52.144833: [debug] Using package location completions from a lock file -2023-12-08 16:24:52.154836: [debug] Loaded snapshot from Pantry database. -2023-12-08 16:24:52.575861: [debug] Prefetching git repos: [] -2023-12-08 16:24:52.576863: [debug] [] -2023-12-08 16:24:52.614841: [debug] Asking for a supported GHC version -2023-12-08 16:24:52.624838: [debug] Installed tools: - - msys2-20200903 - - ghc-9.6.2 - - ghc-9.0.2 - - ghc-8.8.4 - - ghc-8.8.3 - - ghc-8.6.5 - - ghc-8.4.3 - - ghc-8.10.7 - - ghc-8.10.4 -2023-12-08 16:24:52.625841: [debug] Potential GHC builds: standard -2023-12-08 16:24:52.625841: [debug] Found already installed GHC builds: standard -2023-12-08 16:24:52.630839: [debug] Performing a sanity check on: C:\Users\jkniiv\AppData\Local\Programs\stack\x86_64-windows\ghc-8.10.7\bin\ghc-8.10.7.exe -2023-12-08 16:24:52.632844: [debug] Run process within C:\Users\jkniiv\AppData\Local\Temp\stack-sanity-check-2b0507b6fb5a74f8\: C:\Users\jkniiv\AppData\Local\Programs\stack\x86_64-windows\ghc-8.10.7\bin\ghc-8.10.7.exe C:\Users\jkniiv\AppData\Local\Temp\stack-sanity-check-2b0507b6fb5a74f8\Main.hs -no-user-package-db -hide-all-packages "-package base" "-package Cabal" -2023-12-08 16:25:16.713070: [debug] Process finished in 24079ms: C:\Users\jkniiv\AppData\Local\Programs\stack\x86_64-windows\ghc-8.10.7\bin\ghc-8.10.7.exe C:\Users\jkniiv\AppData\Local\Temp\stack-sanity-check-2b0507b6fb5a74f8\Main.hs -no-user-package-db -hide-all-packages "-package base" "-package Cabal" -2023-12-08 16:25:16.721070: [debug] (SQL) SELECT "id","actual_version","arch","ghc_path","ghc_size","ghc_modified","ghc_pkg_path","runghc_path","haddock_path","cabal_version","global_db","global_db_cache_size","global_db_cache_modified","info","global_dump" FROM "compiler_cache" WHERE "ghc_path"=?; [PersistText "C:\\Users\\jkniiv\\AppData\\Local\\Programs\\stack\\x86_64-windows\\ghc-8.10.7\\bin\\ghc-8.10.7.exe"] -2023-12-08 16:25:16.806072: [debug] Loaded compiler information from cache -2023-12-08 16:25:16.806072: [debug] Asking for a supported GHC version -2023-12-08 16:25:16.806072: [info] Stack will use a sandboxed GHC it installed. To use this GHC and packages outside of a project, -consider using: stack ghc, stack ghci, stack runghc, or stack exec. -C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P [BUILD-231130-10.20231129 +6 ~0 -0 !]> just build-production -cd 'C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P' && stack --stack-yaml stack-lts-18.13.yaml build --flag 'git-annex:Production' --flag 'git-annex:-Crypton' 2>&1 | & $env:GNU_TEE_CMD stack.build---flag-git-annex_Production.LOG~102 -Building all executables for git-annex once. After a successful build of all of them, only specified -executables will be rebuilt. -git-annex> configure (exe) -[ 1 of 22] Compiling Author ( Author.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Author.o ) -[ 2 of 22] Compiling Utility.Data ( Utility\Data.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Utility\Data.o ) -[ 3 of 22] Compiling Utility.FileSystemEncoding ( Utility\FileSystemEncoding.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Utility\FileSystemEncoding.o ) -[ 4 of 22] Compiling Utility.Debug ( Utility\Debug.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Utility\Debug.o ) -[ 5 of 22] Compiling Utility.Misc ( Utility\Misc.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Utility\Misc.o ) -[ 6 of 22] Compiling Utility.Monad ( Utility\Monad.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Utility\Monad.o ) -[ 7 of 22] Compiling Utility.Process.Shim ( Utility\Process\Shim.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Utility\Process\Shim.o ) -[ 8 of 22] Compiling Utility.SafeOutput ( Utility\SafeOutput.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Utility\SafeOutput.o ) -[ 9 of 22] Compiling Utility.Exception ( Utility\Exception.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Utility\Exception.o ) -[10 of 22] Compiling Utility.Process ( Utility\Process.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Utility\Process.o ) -[11 of 22] Compiling Utility.SafeCommand ( Utility\SafeCommand.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Utility\SafeCommand.o ) -[12 of 22] Compiling Utility.Env.Basic ( Utility\Env\Basic.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Utility\Env\Basic.o ) -[13 of 22] Compiling Build.Version ( Build\Version.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Build\Version.o ) -[14 of 22] Compiling Utility.Split ( Utility\Split.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Utility\Split.o ) -[15 of 22] Compiling Utility.DottedVersion ( Utility\DottedVersion.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Utility\DottedVersion.o ) -[16 of 22] Compiling Git.Version ( Git\Version.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Git\Version.o ) -[17 of 22] Compiling Utility.SystemDirectory ( Utility\SystemDirectory.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Utility\SystemDirectory.o ) -[18 of 22] Compiling Utility.Path ( Utility\Path.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Utility\Path.o ) -[19 of 22] Compiling Build.TestConfig ( Build\TestConfig.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Build\TestConfig.o ) -[20 of 22] Compiling Build.Configure ( Build\Configure.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Build\Configure.o ) -[21 of 22] Compiling Main ( C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\Setup.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\Main.o ) -[22 of 22] Compiling StackSetupShim ( C:\\hs-stack\setup-exe-src\setup-shim-9p6GVs8J.hs, C:\\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\dist\ed8db9df\setup\StackSetupShim.o ) -Linking C:\\Users\\jkniiv\\Projektit\\git-annex.branchable.com\\git-annex--BUILD-231130-10.20231129~P\\.stack-work\\dist\\ed8db9df\\setup\\setup.exe ... - checking UPGRADE_LOCATION... not available - checking git... yes - checking git version... 2.43.0.windows.1 - checking cp -a... yes - checking cp -p... yes - checking cp --preserve=timestamps... yes - checking cp_reflink_supported... no - checking cp --no-preserve=xattr... yes - checking xargs -0... yes - checking rsync... no - checking curl... yes - checking bup... no - checking borg... no - checking nice... yes - checking ionice... no - checking nocache... no - checking gpg... gpg - checking lsof... not available - checking git-remote-gcrypt... not available - checking ssh connection caching... yes -Configuring git-annex-10.20231129... -git-annex> build (exe) -Preprocessing executable 'git-annex' for git-annex-10.20231129.. -Building executable 'git-annex' for git-annex-10.20231129.. -[ 1 of 693] Compiling Assistant.Types.BranchChange -[ 2 of 693] Compiling Assistant.Types.ThreadName -[ 3 of 693] Compiling Assistant.Types.TransferSlots -[...snip...] -[691 of 693] Compiling Command.Assistant -[692 of 693] Compiling CmdLine.GitAnnex -[693 of 693] Compiling Main -Linking .stack-work\\dist\\ed8db9df\\build\\git-annex\\git-annex.exe ... -git-annex> copy/register -Installing executable git-annex in C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P\.stack-work\install\f241563d\bin -C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P [BUILD-231130-10.20231129 +6 ~0 -0 !]> cp .\.stack-work\install\f241563d\bin\git-annex.exe . -C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P [BUILD-231130-10.20231129 +7 ~0 -0 !]> .\git-annex.exe version > git-annex.version.TXT -C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P [BUILD-231130-10.20231129 +7 ~0 -0 !]> head -1 .\git-annex.version.TXT -git-annex version: 10.20231129-gbacd781c4fe05126219c4b5f2963677a6de01481 -C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P [BUILD-231130-10.20231129 +7 ~0 -0 !]> cat build.TXT -git-annex--BUILD-231130-10.20231129~P -resolver: lts-18.13 -C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P [BUILD-231130-10.20231129 +7 ~0 -0 !]> just copy-dlls-over-to-here -+ PATH='/mingw64/bin:/usr/local/bin:/usr/bin:/bin:/c/Program Files/PowerShell/7:' -+ cd 'C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P' -++ stack exec ldd -- git-annex.exe -++ grep /mingw64 -++ cut '-d ' -f3 -+ dlls_to_copy='/mingw64/bin/libmagic-1.dll -/mingw64/bin/libsystre-0.dll -/mingw64/bin/libtre-5.dll -/mingw64/bin/libintl-8.dll' -+ stack exec cp -- -pv /mingw64/bin/libmagic-1.dll /mingw64/bin/libsystre-0.dll /mingw64/bin/libtre-5.dll /mingw64/bin/libintl-8.dll . -'C:/Users/jkniiv/AppData/Local/Programs/stack/x86_64-windows/msys2-20200903/mingw64/bin/libmagic-1.dll' -> './libmagic-1.dll' -'C:/Users/jkniiv/AppData/Local/Programs/stack/x86_64-windows/msys2-20200903/mingw64/bin/libsystre-0.dll' -> './libsystre-0.dll' -'C:/Users/jkniiv/AppData/Local/Programs/stack/x86_64-windows/msys2-20200903/mingw64/bin/libtre-5.dll' -> './libtre-5.dll' -'C:/Users/jkniiv/AppData/Local/Programs/stack/x86_64-windows/msys2-20200903/mingw64/bin/libintl-8.dll' -> './libintl-8.dll' -C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P [BUILD-231130-10.20231129 +12 ~0 -0 !]> just install-files-to-annx-test-dir -+ PATH='/mingw64/bin:/usr/local/bin:/usr/bin:/bin:/c/Program Files/PowerShell/7:' -+ cd 'C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-231130-10.20231129~P' -++ shopt -s nullglob -++ echo git-annex.exe libintl-8.dll libmagic-1.dll libsystre-0.dll libtre-5.dll build.TXT git-annex.version.TXT stack.build---flag-git-annex_Production.LOG~102 stack.build.LOG~102 stack.setup---verbose.LOG~101 CHANGELOG -+ files_to_copy='git-annex.exe libintl-8.dll libmagic-1.dll libsystre-0.dll libtre-5.dll build.TXT git-annex.version.TXT stack.build---flag-git-annex_Production.LOG~102 stack.build.LOG~102 stack.setup---verbose.LOG~101 CHANGELOG' -+ cp -pv git-annex.exe libintl-8.dll libmagic-1.dll libsystre-0.dll libtre-5.dll build.TXT git-annex.version.TXT stack.build---flag-git-annex_Production.LOG~102 stack.build.LOG~102 stack.setup---verbose.LOG~101 CHANGELOG .annx-test -'git-annex.exe' -> '.annx-test/git-annex.exe' -'libintl-8.dll' -> '.annx-test/libintl-8.dll' -'libmagic-1.dll' -> '.annx-test/libmagic-1.dll' -'libsystre-0.dll' -> '.annx-test/libsystre-0.dll' -'libtre-5.dll' -> '.annx-test/libtre-5.dll' -'build.TXT' -> '.annx-test/build.TXT' -'git-annex.version.TXT' -> '.annx-test/git-annex.version.TXT' -'stack.build---flag-git-annex_Production.LOG~102' -> '.annx-test/stack.build---flag-git-annex_Production.LOG~102' -'stack.build.LOG~102' -> '.annx-test/stack.build.LOG~102' -'stack.setup---verbose.LOG~101' -> '.annx-test/stack.setup---verbose.LOG~101' -'CHANGELOG' -> '.annx-test/CHANGELOG' - -# End of transcript or log. -"""]] - -And this is how I usually run the test suite in Git Bash (with some preliminary actions to record the tests available + make -an SHA256SUMS file of the most important files in this test directory): - -[[!format sh """ -jkniiv@AINESIS MINGW64 ~/Projektit/git-annex.branchable.com/git-annex--BUILD-231130-10.20231129~P (BUILD-231130-10.20231129) -$ cd .annx-test/ - -jkniiv@AINESIS MINGW64 ~/Projektit/git-annex.branchable.com/git-annex--BUILD-231130-10.20231129~P/.annx-test (BUILD-231130-10.20231129) -$ just record-tests -./git-annex.exe test --list-tests > git-annex.test---list-tests.TXT - -jkniiv@AINESIS MINGW64 ~/Projektit/git-annex.branchable.com/git-annex--BUILD-231130-10.20231129~P/.annx-test (BUILD-231130-10.20231129) -$ just sums check-sums -sha256sum git-annex.exe libintl-8.dll libmagic-1.dll libsystre-0.dll libtre-5.dll build.TXT git-annex.test---list-tests. -TXT git-annex.version.TXT > SHA256SUMS -sha256sum -c SHA256SUMS -git-annex.exe: OK -libintl-8.dll: OK -libmagic-1.dll: OK -libsystre-0.dll: OK -libtre-5.dll: OK -build.TXT: OK -git-annex.test---list-tests.TXT: OK -git-annex.version.TXT: OK - -jkniiv@AINESIS MINGW64 ~/Projektit/git-annex.branchable.com/git-annex--BUILD-231130-10.20231129~P/.annx-test (BUILD-231130-10.20231129) -$ ls -l -total 73065 --rw-r--r-- 1 jkniiv 197121 429145 Dec 8 14:28 CHANGELOG --rw-r--r-- 1 jkniiv 197121 1073 Dec 14 2022 Justfile --rw-r--r-- 1 jkniiv 197121 664 Dec 8 16:48 SHA256SUMS --rw-r--r-- 1 jkniiv 197121 60 Dec 8 15:27 build.TXT --rwxr-xr-x 1 jkniiv 197121 73740800 Dec 8 16:41 git-annex.exe* --rw-r--r-- 1 jkniiv 197121 8464 Dec 8 16:48 git-annex.test---list-tests.TXT --rw-r--r-- 1 jkniiv 197121 1112 Dec 8 16:42 git-annex.version.TXT --rwxr-xr-x 1 jkniiv 197121 187145 Nov 23 11:21 libintl-8.dll* --rwxr-xr-x 1 jkniiv 197121 226435 Dec 4 20:11 libmagic-1.dll* --rwxr-xr-x 1 jkniiv 197121 18240 Jan 17 2018 libsystre-0.dll* --rwxr-xr-x 1 jkniiv 197121 92313 Jan 17 2018 libtre-5.dll* --rw-r--r-- 1 jkniiv 197121 37834 Dec 8 16:41 stack.build---flag-git-annex_Production.LOG~102 --rw-r--r-- 1 jkniiv 197121 37834 Dec 8 15:45 stack.build.LOG~102 --rw-r--r-- 1 jkniiv 197121 2757 Dec 8 16:25 stack.setup---verbose.LOG~101 - -jkniiv@AINESIS MINGW64 ~/Projektit/git-annex.branchable.com/git-annex--BUILD-231130-10.20231129~P/.annx-test (BUILD-231130-10.20231129) -$ ldd git-annex.exe - ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ff93f190000) - KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ff93e380000) - KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ff93c9d0000) - ADVAPI32.dll => /c/WINDOWS/System32/ADVAPI32.dll (0x7ff93e870000) - msvcrt.dll => /c/WINDOWS/System32/msvcrt.dll (0x7ff93d2b0000) - libmagic-1.dll => /c/Users/jkniiv/Projektit/git-annex.branchable.com/git-annex--BUILD-231130-10.20231129~P/.annx-test/libmagic-1.dll (0x7ff8fd370000) - sechost.dll => /c/WINDOWS/System32/sechost.dll (0x7ff93e250000) - SHLWAPI.dll => /c/WINDOWS/System32/SHLWAPI.dll (0x7ff93e440000) - libsystre-0.dll => /c/Users/jkniiv/Projektit/git-annex.branchable.com/git-annex--BUILD-231130-10.20231129~P/.annx-test/libsystre-0.dll (0x6bcc0000) - RPCRT4.dll => /c/WINDOWS/System32/RPCRT4.dll (0x7ff93e120000) - CRYPT32.dll => /c/WINDOWS/System32/CRYPT32.dll (0x7ff93d090000) - libtre-5.dll => /c/Users/jkniiv/Projektit/git-annex.branchable.com/git-annex--BUILD-231130-10.20231129~P/.annx-test/libtre-5.dll (0x63bc0000) - ucrtbase.dll => /c/WINDOWS/System32/ucrtbase.dll (0x7ff93cf00000) - GDI32.dll => /c/WINDOWS/System32/GDI32.dll (0x7ff93d1f0000) - win32u.dll => /c/WINDOWS/System32/win32u.dll (0x7ff93ced0000) - libintl-8.dll => /c/Users/jkniiv/Projektit/git-annex.branchable.com/git-annex--BUILD-231130-10.20231129~P/.annx-test/libintl-8.dll (0x7ff8f3d20000) - gdi32full.dll => /c/WINDOWS/System32/gdi32full.dll (0x7ff93ccd0000) - dbghelp.dll => /c/WINDOWS/SYSTEM32/dbghelp.dll (0x7ff93ae50000) - msvcp_win.dll => /c/WINDOWS/System32/msvcp_win.dll (0x7ff93c8c0000) - USER32.dll => /c/WINDOWS/System32/USER32.dll (0x7ff93d6c0000) - ole32.dll => /c/WINDOWS/System32/ole32.dll (0x7ff93d4e0000) - libiconv-2.dll => /c/Users/jkniiv/bin/libiconv-2.dll (0x7ff8a6e00000) - combase.dll => /c/WINDOWS/System32/combase.dll (0x7ff93e510000) - IPHLPAPI.DLL => /c/WINDOWS/SYSTEM32/IPHLPAPI.DLL (0x7ff93bbf0000) - SHELL32.dll => /c/WINDOWS/System32/SHELL32.dll (0x7ff93d900000) - WS2_32.dll => /c/WINDOWS/System32/WS2_32.dll (0x7ff93d860000) - WINMM.dll => /c/WINDOWS/SYSTEM32/WINMM.dll (0x7ff928e00000) - CRYPTBASE.DLL => /c/WINDOWS/SYSTEM32/CRYPTBASE.DLL (0x7ff93c070000) - dbgcore.DLL => /c/WINDOWS/SYSTEM32/dbgcore.DLL (0x7ff928dc0000) - -jkniiv@AINESIS MINGW64 ~/Projektit/git-annex.branchable.com/git-annex--BUILD-231130-10.20231129~P/.annx-test (BUILD-231130-10.20231129) -$ just tests-new -/usr/bin/time ./git-annex.exe test 2>&1 | tee git-annex.test.LOG~200 - -All 0 tests passed (0.00s) -Tests - Repo Tests v10 adjusted unlocked branch - Init Tests - init: OK (9.36s) - add: FAIL - Exception: fd:4: hGetChar: invalid argument (invalid byte sequence) - Use -p '/Init Tests.add/' to rerun this test only. - crypto: SKIP - - Use -p '/crypto/' to rerun this test only. - uninit (in git-annex branch): SKIP - - Use -p '/uninit (in git-annex branch)/' to rerun this test only. - conflict resolution symlink bit: SKIP - - Use -p '/conflict resolution symlink bit/' to rerun this test only. - union merge regression: SKIP - - Use -p '/union merge regression/' to rerun this test only. - unused: SKIP - - Use -p '/unused/' to rerun this test only. - fsck (bare): SKIP - - Use -p '/fsck (bare)/' to rerun this test only. - lock: SKIP - - Use -p '/Repo Tests v10 adjusted unlocked branch.lock/' to rerun this test only. - drop (with remote): SKIP - - Use -p '/drop (with remote)/' to rerun this test only. - log: SKIP - - Use -p '/log/' to rerun this test only. - add extras: SKIP - - Use -p '/add extras/' to rerun this test only. - -11 out of 12 tests failed (11.65s) -Tests - QuickCheck - prop_quote_unquote_roundtrip: OK (0.10s) - +++ OK, passed 1000 tests. - prop_encode_c_decode_c_roundtrip: OK (0.11s) - +++ OK, passed 1000 tests. - prop_isomorphic_key_encode: OK (0.05s) - +++ OK, passed 1000 tests. - prop_isomorphic_shellEscape: OK (0.04s) - +++ OK, passed 1000 tests. - prop_isomorphic_shellEscape_multiword: OK (1.25s) - +++ OK, passed 1000 tests. - prop_isomorphic_configEscape: OK (0.03s) - +++ OK, passed 1000 tests. - prop_parse_show_Config: OK (0.08s) - +++ OK, passed 1000 tests. - prop_upFrom_basics: OK - +++ OK, passed 1000 tests. - prop_relPathDirToFileAbs_basics: OK (0.04s) - +++ OK, passed 1000 tests. - prop_relPathDirToFileAbs_regressionTest: OK - +++ OK, passed 1 test. - prop_dirContains_regressionTest: OK - +++ OK, passed 1 test. - prop_cost_sane: OK - +++ OK, passed 1 test. - prop_matcher_sane: OK - +++ OK, passed 1 test. - prop_HmacSha1WithCipher_sane: OK - +++ OK, passed 1 test. - prop_VectorClock_sane: OK - +++ OK, passed 1 test. - prop_addMapLog_sane: OK - +++ OK, passed 1 test. - prop_verifiable_sane: OK (0.11s) - +++ OK, passed 1000 tests. - prop_segment_regressionTest: OK - +++ OK, passed 1 test. - prop_read_write_transferinfo: OK (0.06s) - +++ OK, passed 1000 tests. - prop_read_show_inodecache: OK (0.04s) - +++ OK, passed 1000 tests. - prop_parse_build_presence_log: OK (1.92s) - +++ OK, passed 1000 tests. - prop_parse_build_contentidentifier_log: OK (1.65s) - +++ OK, passed 1000 tests. - prop_read_show_TrustLevel: OK - +++ OK, passed 1 test. - prop_parse_build_TrustLevelLog: OK - +++ OK, passed 1 test. - prop_schedule_roundtrips: OK (0.01s) - +++ OK, passed 1000 tests. - prop_past_sane: OK - +++ OK, passed 1 test. - prop_duration_roundtrips: OK - +++ OK, passed 1000 tests. - prop_metadata_sane: OK (1.00s) - +++ OK, passed 1000 tests. - prop_metadata_serialize: OK (1.07s) - +++ OK, passed 1000 tests. - prop_branchView_legal: OK (1.10s) - +++ OK, passed 1000 tests. - prop_viewPath_roundtrips: OK (0.02s) - +++ OK, passed 1000 tests. - prop_view_roundtrips: OK (0.55s) - +++ OK, passed 1000 tests. - prop_viewedFile_rountrips: OK (0.04s) - +++ OK, passed 1000 tests. - prop_standardGroups_parse: OK - +++ OK, passed 1 test. - sha1 stable: OK - +++ OK, passed 1 test. - sha2_224 stable: OK - +++ OK, passed 1 test. - sha2_256 stable: OK - +++ OK, passed 1 test. - sha2_384 stable: OK - +++ OK, passed 1 test. - sha2_512 stable: OK - +++ OK, passed 1 test. - skein256 stable: OK - +++ OK, passed 1 test. - skein512 stable: OK - +++ OK, passed 1 test. - sha3_224 stable: OK - +++ OK, passed 1 test. - sha3_256 stable: OK - +++ OK, passed 1 test. - sha3_384 stable: OK - +++ OK, passed 1 test. - sha3_512 stable: OK - +++ OK, passed 1 test. - blake2s_160 stable: OK - +++ OK, passed 1 test. - blake2s_224 stable: OK - +++ OK, passed 1 test. - blake2s_256 stable: OK - +++ OK, passed 1 test. - blake2sp_224 stable: OK - +++ OK, passed 1 test. - blake2sp_256 stable: OK - +++ OK, passed 1 test. - blake2b_160 stable: OK - +++ OK, passed 1 test. - blake2b_224 stable: OK - +++ OK, passed 1 test. - blake2b_256 stable: OK - +++ OK, passed 1 test. - blake2b_384 stable: OK - +++ OK, passed 1 test. - blake2b_512 stable: OK - +++ OK, passed 1 test. - blake2bp_512 stable: OK - +++ OK, passed 1 test. - md5 stable: OK - +++ OK, passed 1 test. - HmacSha1 stable: OK - +++ OK, passed 1 test. - HmacSha224 stable: OK - +++ OK, passed 1 test. - HmacSha256 stable: OK - +++ OK, passed 1 test. - HmacSha384 stable: OK - +++ OK, passed 1 test. - HmacSha512 stable: OK - +++ OK, passed 1 test. - -All 62 tests passed (9.40s) -Tests - Repo Tests v10 adjusted unlocked branch - Init Tests - init: OK (9.46s) - add: FAIL - Exception: fd:4: hGetChar: invalid argument (invalid byte sequence) - Use -p '/add/' to rerun this test only. - bup remote: SKIP - - Use -p '/bup remote/' to rerun this test only. - map: SKIP - - Use -p '/map/' to rerun this test only. - conflict resolution movein regression: SKIP - - Use -p '/conflict resolution movein regression/' to rerun this test only. - sync: SKIP - - Use -p '/sync/' to rerun this test only. - migrate: SKIP - - Use -p '/migrate/' to rerun this test only. - trust: SKIP - - Use -p '/trust/' to rerun this test only. - move (numcopies): SKIP - - Use -p '/move (numcopies)/' to rerun this test only. - unannex (with copy): SKIP - - Use -p '/unannex (with copy)/' to rerun this test only. - export and import of subdir: SKIP - - Use -p '/export and import of subdir/' to rerun this test only. - -10 out of 11 tests failed (11.84s) -Tests - Repo Tests v10 adjusted unlocked branch - Init Tests - init: OK (9.47s) - add: FAIL - Exception: fd:4: hGetChar: invalid argument (invalid byte sequence) - Use -p '/Init Tests.add/' to rerun this test only. - borg remote: SKIP - - Use -p '/borg remote/' to rerun this test only. - uninit: SKIP - - Use -p '/uninit/' to rerun this test only. - conflict resolution (mixed directory and file): SKIP - - Use -p '/conflict resolution (mixed directory and file)/' to rerun this test only. - concurrent get of dup key regression: SKIP - - Use -p '/concurrent get of dup key regression/' to rerun this test only. - migrate (via gitattributes): SKIP - - Use -p '/migrate (via gitattributes)/' to rerun this test only. - fsck (basics): SKIP - - Use -p '/fsck (basics)/' to rerun this test only. - copy: SKIP - - Use -p '/copy/' to rerun this test only. - drop (no remote): SKIP - - Use -p '/drop (no remote)/' to rerun this test only. - shared clone: SKIP - - Use -p '/shared clone/' to rerun this test only. - add dup: SKIP - - Use -p '/add dup/' to rerun this test only. - -11 out of 12 tests failed (11.82s) -Tests - Repo Tests v10 adjusted unlocked branch - Init Tests - init: OK (8.97s) - add: FAIL - Exception: fd:4: hGetChar: invalid argument (invalid byte sequence) - Use -p '/Init Tests.add/' to rerun this test only. - addurl: SKIP - - Use -p '/addurl/' to rerun this test only. - directory remote: SKIP - - Use -p '/directory remote/' to rerun this test only. - conflict resolution (nonannexed symlink): SKIP - - Use -p '/conflict resolution (nonannexed symlink)/' to rerun this test only. - conflict resolution: SKIP - - Use -p '$0=="Tests.Repo Tests v10 adjusted unlocked branch.conflict resolution"' to rerun this test only. - info: SKIP - - Use -p '/info/' to rerun this test only. - conversion git to annexed: SKIP - - Use -p '/conversion git to annexed/' to rerun this test only. - partial commit: SKIP - - Use -p '/partial commit/' to rerun this test only. - move: SKIP - - Use -p '/move/' to rerun this test only. - reinject: SKIP - - Use -p '/reinject/' to rerun this test only. - metadata: SKIP - - Use -p '/metadata/' to rerun this test only. - -11 out of 12 tests failed (11.20s) -Tests - Repo Tests v10 adjusted unlocked branch - Init Tests - init: OK (9.18s) - add: FAIL - Exception: fd:4: hGetChar: invalid argument (invalid byte sequence) - Use -p '/Init Tests.add/' to rerun this test only. - preferred content: SKIP - - Use -p '/preferred content/' to rerun this test only. - upgrade: SKIP - - Use -p '/upgrade/' to rerun this test only. - conflict resolution (uncommitted local file): SKIP - - Use -p '/conflict resolution (uncommitted local file)/' to rerun this test only. - adjusted branch merge regression: SKIP - - Use -p '/adjusted branch merge regression/' to rerun this test only. - describe: SKIP - - Use -p '/describe/' to rerun this test only. - fsck (local untrusted): SKIP - - Use -p '/fsck (local untrusted)/' to rerun this test only. - lock --force: SKIP - - Use -p '/lock --force/' to rerun this test only. - drop (untrusted remote): SKIP - - Use -p '/drop (untrusted remote)/' to rerun this test only. - view: SKIP - - Use -p '/view/' to rerun this test only. - add moved link: SKIP - - Use -p '/add moved link/' to rerun this test only. - -11 out of 12 tests failed (11.64s) -Tests - Repo Tests v10 adjusted unlocked branch - Init Tests - init: OK (9.02s) - add: FAIL - Exception: fd:4: hGetChar: invalid argument (invalid byte sequence) - Use -p '/Init Tests.add/' to rerun this test only. - add subdirs: SKIP - - Use -p '/add subdirs/' to rerun this test only. - hook remote: SKIP - - Use -p '/hook remote/' to rerun this test only. - conflict resolution (nonannexed file): SKIP - - Use -p '/conflict resolution (nonannexed file)/' to rerun this test only. - transition propagation: SKIP - - Use -p '/transition propagation/' to rerun this test only. - merge: SKIP - - Use -p '/merge/' to rerun this test only. - fsck --from remote: SKIP - - Use -p '/fsck --from remote/' to rerun this test only. - edit (pre-commit): SKIP - - Use -p '/edit (pre-commit)/' to rerun this test only. - get (ssh remote): SKIP - - Use -p '/get (ssh remote)/' to rerun this test only. - import: SKIP - - Use -p '/import/' to rerun this test only. - ignore deleted files: SKIP - - Use -p '/ignore deleted files/' to rerun this test only. - -11 out of 12 tests failed (11.22s) -Tests - Repo Tests v10 adjusted unlocked branch - Init Tests - init: OK (8.99s) - add: FAIL - Exception: fd:4: hGetChar: invalid argument (invalid byte sequence) - Use -p '/add/' to rerun this test only. - required_content: SKIP - - Use -p '/required_content/' to rerun this test only. - whereis: SKIP - - Use -p '/whereis/' to rerun this test only. - conflict resolution (removed file): SKIP - - Use -p '/conflict resolution (removed file)/' to rerun this test only. - adjusted branch subtree regression: SKIP - - Use -p '/adjusted branch subtree regression/' to rerun this test only. - find: SKIP - - Use -p '/find/' to rerun this test only. - fsck (remote untrusted): SKIP - - Use -p '/fsck (remote untrusted)/' to rerun this test only. - edit (no pre-commit): SKIP - - Use -p '/edit (no pre-commit)/' to rerun this test only. - get: SKIP - - Use -p '/get/' to rerun this test only. - magic: SKIP - - Use -p '/magic/' to rerun this test only. - readonly remote: SKIP - - Use -p '/readonly remote/' to rerun this test only. - -11 out of 12 tests failed (11.22s) -Tests - Repo Tests v10 adjusted unlocked branch - Init Tests - init: OK (5.91s) - add: FAIL - Exception: fd:4: hGetChar: invalid argument (invalid byte sequence) - Use -p '/add/' to rerun this test only. - repair: SKIP - - Use -p '/repair/' to rerun this test only. - rsync remote: SKIP - - Use -p '/rsync remote/' to rerun this test only. - conflict resolution (mixed locked and unlocked file): SKIP - - Use -p '/conflict resolution (mixed locked and unlocked file)/' to rerun this test only. - conflict resolution (adjusted branch): SKIP - - Use -p '/conflict resolution (adjusted branch)/' to rerun this test only. - version: SKIP - - Use -p '/Repo Tests v10 adjusted unlocked branch.version/' to rerun this test only. - conversion annexed to git: SKIP - - Use -p '/conversion annexed to git/' to rerun this test only. - fix: SKIP - - Use -p '/fix/' to rerun this test only. - move (ssh remote): SKIP - - Use -p '/move (ssh remote)/' to rerun this test only. - unannex (no copy): SKIP - - Use -p '/unannex (no copy)/' to rerun this test only. - export and import: SKIP - - Use -p '/export and import/' to rerun this test only. - -11 out of 12 tests failed (7.23s) - (Failures above could be due to a bug in git-annex, or an incompatibility - with utilities, such as git, installed on this system.) -Command exited with non-zero status 1 -0.00user 0.04system 0:31.25elapsed 0%CPU (0avgtext+0avgdata 5248maxresident)k -0inputs+0outputs (1391major+0minor)pagefaults 0swaps - -jkniiv@AINESIS MINGW64 ~/Projektit/git-annex.branchable.com/git-annex--BUILD-231130-10.20231129~P/.annx-test (BUILD-231130-10.20231129) -$ ./git-annex.exe test -p ' /Init Tests.add/' - -All 0 tests passed (0.00s) -Tests - Repo Tests v10 adjusted unlocked branch - Init Tests - init: OK (5.02s) - add: FAIL - Exception: fd:4: hGetChar: invalid argument (invalid byte sequence) - Use -p '(/Init Tests.add/||/Init Tests/)&&/add/' to rerun this test only. - -1 out of 2 tests failed (6.37s) - -All 0 tests passed (0.00s) - (Failures above could be due to a bug in git-annex, or an incompatibility - with utilities, such as git, installed on this system.) - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Sure, Git Annex is great as always. I use it several times a week with my multigigabyte -backups, where it gives structure to my image-based backup routines, so you could -say I'm a believer. :) - -[[!meta author=jkniiv]] - -### Update 20 Dec 2023 - -P.S. This is a bit embarrasing but I found out this is [[notabug|done]], cf. my comment --[[jkniiv]] diff --git a/doc/bugs/__39__Production__39___build_doesn__39__t_pass_testsuite_on_Win32/comment_1_7ac6938faef9111da701080a62c4d7e9._comment b/doc/bugs/__39__Production__39___build_doesn__39__t_pass_testsuite_on_Win32/comment_1_7ac6938faef9111da701080a62c4d7e9._comment deleted file mode 100644 index 6b70005ccf..0000000000 --- a/doc/bugs/__39__Production__39___build_doesn__39__t_pass_testsuite_on_Win32/comment_1_7ac6938faef9111da701080a62c4d7e9._comment +++ /dev/null @@ -1,67 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="my report was actually a User Failure on my part" - date="2023-12-19T23:02:15Z" - content=""" -Uh-oh, after adding the option `--test-debug` to the `test` subcommand I got a lead on -the real culprit and it wasn't git-annex but libmagic: - -[[!format sh \"\"\" -[...snip...] -ok -[2023-12-19 22:52:17.5370274] (Utility.Process) process [21636] done ExitSuccess -[2023-12-19 22:52:17.5460328] (Utility.Process) process [10460] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-p -athspecs\",\"-c\",\"annex.debug=true\",\"ls-files\",\"-z\",\"--modified\",\"--\",\"foo\"] -[2023-12-19 22:52:17.5970346] (Utility.Process) process [10460] done ExitSuccess -[2023-12-19 22:52:17.6030281] (Utility.Process) process [9208] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pa -thspecs\",\"-c\",\"annex.debug=true\",\"diff\",\"--name-only\",\"--diff-filter=T\",\"-z\",\"--cached\",\"--\",\"foo\"] -[2023-12-19 22:52:17.6550242] (Utility.Process) process [9208] done ExitSuccess -(recording state in git...) -[2023-12-19 22:52:17.6650254] (Utility.Process) process [24064] feed: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-p -athspecs\",\"-c\",\"annex.debug=true\",\"update-index\",\"-z\",\"--index-info\"] -[2023-12-19 22:52:17.7090277] (Utility.Process) process [24064] done ExitSuccess -[2023-12-19 22:52:17.7240248] (Utility.Process) process [22248] feed: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-p -athspecs\",\"-c\",\"annex.debug=true\",\"update-index\",\"-z\",\"--index-info\"] -[2023-12-19 22:52:17.7700259] (Utility.Process) process [22248] done ExitSuccess -[2023-12-19 22:52:17.7780282] (Utility.Process) process [23188] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-p -athspecs\",\"-c\",\"annex.debug=true\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"] -[2023-12-19 22:52:17.8270228] (Utility.Process) process [23188] done ExitSuccess -[2023-12-19 22:52:17.8340274] (Utility.Process) process [17648] feed: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-p -athspecs\",\"-c\",\"annex.debug=true\",\"update-index\",\"-z\",\"--index-info\"] -[2023-12-19 22:52:17.8420331] (Utility.Process) process [21928] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-p -athspecs\",\"-c\",\"annex.debug=true\",\"diff-index\",\"--raw\",\"-z\",\"-r\",\"--no-renames\",\"-l0\",\"--cached\",\"refs/heads/git-annex\", -\"--\"] -[2023-12-19 22:52:17.8980416] (Utility.Process) process [21928] done ExitSuccess -[2023-12-19 22:52:17.9060216] (Utility.Process) process [17648] done ExitSuccess -[2023-12-19 22:52:17.920024] (Utility.Process) process [18680] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pa -thspecs\",\"-c\",\"annex.debug=true\",\"write-tree\"] -[2023-12-19 22:52:17.982027] (Utility.Process) process [18680] done ExitSuccess -[2023-12-19 22:52:17.9890276] (Utility.Process) process [10892] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-p -athspecs\",\"-c\",\"annex.debug=true\",\"commit-tree\",\"bb68ce15d91f27eec51749b9481ede5cc6bcd190\",\"--no-gpg-sign\",\"-p\",\"refs/he -ads/git-annex\"] -[2023-12-19 22:52:18.0490233] (Utility.Process) process [10892] done ExitSuccess -[2023-12-19 22:52:18.0570304] (Utility.Process) process [6552] call: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pa -thspecs\",\"-c\",\"annex.debug=true\",\"update-ref\",\"refs/heads/git-annex\",\"71b9bb13e4244bd2233de7ca7cf7fc01011f1e62\"] -[2023-12-19 22:52:18.1170343] (Utility.Process) process [6552] done ExitSuccess -[2023-12-19 22:52:18.1300416] (Utility.Process) process [10116] done ExitSuccess -[2023-12-19 22:52:18.1350241] (Utility.Process) process [20976] done ExitSuccess -[2023-12-19 22:52:18.1400231] (Utility.Process) process [9880] done ExitSuccess -[2023-12-19 22:52:18.1450249] (Utility.Process) process [872] done ExitSuccess -C:\Users\jkniiv\AppData\Local/.magic/magic.mgc, 1: Warning: offset `∟♦▲FAIL (2.36s) - .\\Test\\Framework.hs:83: - add with SHA1 failed with unexpected exit code - Use -p '(/Init Tests.add/||/Init Tests/)&&/add/' to rerun this test only. - -1 out of 2 tests failed (8.86s) -git-annex: thread blocked indefinitely in an STM transaction - -\"\"\"]] - -So the real error turned out to be a user failure of mine: libmagic (or the msys2 package `mingw-w64-x86_64-file`) -had a recent update and the new library didn't like my previous magic database located in the fallback location -`%localappdata%\.magic\magic.mgc`. By copying the msys2 file `/mingw64/share/misc/magic.mgc` to the aforementioned -location, the whole issue cleared itself and this report became moot. - - -"""]] diff --git a/doc/bugs/__39__git_annex_assist__39___doesn__39__t_honor_--jobs.mdwn b/doc/bugs/__39__git_annex_assist__39___doesn__39__t_honor_--jobs.mdwn deleted file mode 100644 index 8ef89034ef..0000000000 --- a/doc/bugs/__39__git_annex_assist__39___doesn__39__t_honor_--jobs.mdwn +++ /dev/null @@ -1,36 +0,0 @@ -### Please describe the problem. - -It seems that `git annex assist` doesn't actually do multiprocessing despite `--jobs=cpus` or `git config annex.jobs cpus` - -### What steps will reproduce the problem? - -- make an annex repo -- add more than one remote to it -- add some files -- run `git annex assist --jobs` (operates sequentially) -- run `git annex sync --jobs` (operates in parallel) -- set `git config annex.jobs cpus` -- run `git annex assist` (operates sequentially) -- run `git annex sync` (operates in parallel) - -### What version of git-annex are you using? On what operating system? - -[[!format sh """ -❯ git annex version -git-annex version: 10.20230627-g06f734555 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22.1 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.13.1 persistent-sqlite-2.13.1.0 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2.1 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 10 -"""]] - - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -git-annex is **marvellous** and a game-changer for all my workflows 👍 - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/__39__git_annex_assist__39___doesn__39__t_honor_--jobs/comment_1_96b3c705c7c83b153cde2e8bb16ed380._comment b/doc/bugs/__39__git_annex_assist__39___doesn__39__t_honor_--jobs/comment_1_96b3c705c7c83b153cde2e8bb16ed380._comment deleted file mode 100644 index 6717fa0747..0000000000 --- a/doc/bugs/__39__git_annex_assist__39___doesn__39__t_honor_--jobs/comment_1_96b3c705c7c83b153cde2e8bb16ed380._comment +++ /dev/null @@ -1,7 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-07-05T16:00:25Z" - content=""" -Reproduced. The adds do run in parallel, the content transfers do not. -"""]] diff --git a/doc/bugs/__47__exe__47__git-annex.mdwn b/doc/bugs/__47__exe__47__git-annex.mdwn deleted file mode 100644 index 8d7e157f3d..0000000000 --- a/doc/bugs/__47__exe__47__git-annex.mdwn +++ /dev/null @@ -1,51 +0,0 @@ -### Please describe the problem. - - -### What steps will reproduce the problem? -I have system: -Linux RPI-4B 5.15.74-2-MANJARO-ARM-RPI #1 SMP PREEMPT Thu Oct 20 16:43:17 UTC 2022 aarch64 GNU/Linux - -Doesnt't have git-annex in aur and pacman. -So download git-annex-standalone-arm64.tar.gz from web. -./runshell - -```git-annex enable-tor (adjusted/master(unlocked)+2) 10:25:22 -enable-tor - You may be prompted for a password - -git-annex: Failed to run as root: /home/gyurmo/.local/git-annex.linux/bin/git-annex enable-tor 1000 -failed -enable-tor: 1 failed``` - - -sudo /home/gyurmo/.local/git-annex.linux/bin/git-annex enable-tor 1000 -[sudo] gyurmo jelszava: -/home/gyurmo/.local/git-annex.linux/bin/git-annex: sor: 4: /exe/git-annex: No such file or directory - - -### What version of git-annex are you using? On what operating system? - -git-annex version: 10.20220121-g0bcb94487 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux aarch64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 8 - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/__47__exe__47__git-annex/comment_1_8a9e9d83a3dcf07ed76a22f03636f6d1._comment b/doc/bugs/__47__exe__47__git-annex/comment_1_8a9e9d83a3dcf07ed76a22f03636f6d1._comment deleted file mode 100644 index 45c0e01b52..0000000000 --- a/doc/bugs/__47__exe__47__git-annex/comment_1_8a9e9d83a3dcf07ed76a22f03636f6d1._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-10-26T18:53:40Z" - content=""" -Using this command instead should work: - - sudo /home/gyurmo/.local/git-annex.linux/git-annex enable-tor 1000 -"""]] diff --git a/doc/bugs/__47__exe__47__git-annex/comment_2_381ac4e8b27343cdc470584c05edec76._comment b/doc/bugs/__47__exe__47__git-annex/comment_2_381ac4e8b27343cdc470584c05edec76._comment deleted file mode 100644 index 6db2b4d8bf..0000000000 --- a/doc/bugs/__47__exe__47__git-annex/comment_2_381ac4e8b27343cdc470584c05edec76._comment +++ /dev/null @@ -1,7 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-10-26T19:44:24Z" - content=""" -And I've fixed this problem now. -"""]] diff --git a/doc/bugs/__96__git_add__96___adds_files_unlocked_instead_of_locked.mdwn b/doc/bugs/__96__git_add__96___adds_files_unlocked_instead_of_locked.mdwn deleted file mode 100644 index f4b5d71f8d..0000000000 --- a/doc/bugs/__96__git_add__96___adds_files_unlocked_instead_of_locked.mdwn +++ /dev/null @@ -1,55 +0,0 @@ -### Please describe the problem. -If `.gitattributes` contains `* annex.largefiles=anything`, then `git add` will add files in an unlocked state. According to this page (https://git-annex.branchable.com/tips/unlocked_files/) though, git-annex should add files in a locked state by default: -> By default, git-annex commands will add files in locked mode, unless used on a filesystem that does not support symlinks, when unlocked mode is used. To make them always use unlocked mode, run: git config annex.addunlocked true - -### What steps will reproduce the problem? - -``` -cd $(mktemp -d) -git init -echo '* annex.largefiles=anything' > .gitattributes -git add . -git commit -m "Add gitattributes" -git annex init -cat >> annexed.txt < -Initialized empty Git repository in /tmp/tmp.XzlfX1juzC/.git/ -[main (root-commit) 7633536] Add gitattributes - 1 file changed, 1 insertion(+) - create mode 100644 .gitattributes -init ok -(recording state in git...) -(recording state in git...) -[main 60f1b44] Add annexed - 1 file changed, 1 insertion(+) - create mode 100644 annexed.txt -(git-annex) $ cd /tmp/tmp.XzlfX1juzC -(git-annex) /tmp/tmp.XzlfX1juzC$ ls -l -total 4 --rw-rw-r-- 1 27 Okt 23 17:12 annexed.txt -(git-annex) /tmp/tmp.XzlfX1juzC$ git annex find --unlocked -annexed.txt -(git-annex) /tmp/tmp.XzlfX1juzC$ - -# End of transcript or log. -"""]] - -> [[done]] --[[Joey]] diff --git a/doc/bugs/__96__git_add__96___adds_files_unlocked_instead_of_locked/comment_1_58abf8693ed90fc8c6e3f750310c17e4._comment b/doc/bugs/__96__git_add__96___adds_files_unlocked_instead_of_locked/comment_1_58abf8693ed90fc8c6e3f750310c17e4._comment deleted file mode 100644 index 6877505071..0000000000 --- a/doc/bugs/__96__git_add__96___adds_files_unlocked_instead_of_locked/comment_1_58abf8693ed90fc8c6e3f750310c17e4._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-10-23T17:48:54Z" - content=""" -I wish it were possible to make `git add` add files in a locked state. -However, the smudge/clean interface git-annex uses to hook into `git add` -makes it impossible to do that. - -This is why the documentation refers to "git-annex commands", which does -exclude `git add`. I've made that a little bit more explicit. -"""]] diff --git a/doc/bugs/__96__lookupkey__96___unexpectedly_slow.mdwn b/doc/bugs/__96__lookupkey__96___unexpectedly_slow.mdwn deleted file mode 100644 index bac6814d06..0000000000 --- a/doc/bugs/__96__lookupkey__96___unexpectedly_slow.mdwn +++ /dev/null @@ -1,25 +0,0 @@ -### Please describe the problem. - -I need to discover a potentially existing annex-key for any file in the worktree of a Git repository. I assumed that a batch-mode `lookupkey` would be faster than a `find --json --batch`, but that is not the case. - -My test repository has 36k files. - -``` -❯ time git ls-files | git annex lookupkey --batch > /dev/null -git ls-files 0.01s user 0.01s system 0% cpu 2:42.37 total -git annex lookupkey --batch > /dev/null 91.00s user 73.39s system 98% cpu 2:46.25 total - -❯ time git ls-files | git annex find --anything --json --batch > /dev/null -git ls-files 0.01s user 0.00s system 0% cpu 4.093 total -git annex find --anything --json --batch > /dev/null 3.20s user 2.02s system 124% cpu 4.195 total -``` - -`lookupkey` appears to be many times slower than `find`, although the latter reports much more information. - -This surprised me, hence I am reporting it here as a potential bug. - -### What version of git-annex are you using? On what operating system? - -git-annex version: 10.20230126 - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/__96__lookupkey__96___unexpectedly_slow/comment_1_13003e3b46c5cfab60b6e34fb18731d7._comment b/doc/bugs/__96__lookupkey__96___unexpectedly_slow/comment_1_13003e3b46c5cfab60b6e34fb18731d7._comment deleted file mode 100644 index 0fe5dfc5c7..0000000000 --- a/doc/bugs/__96__lookupkey__96___unexpectedly_slow/comment_1_13003e3b46c5cfab60b6e34fb18731d7._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-10-30T18:51:11Z" - content=""" -This is due to lookupkey passing each filename through `git ls-files` -in order to support absolute filepaths input. See -[[!commit cfdfe4df6c8b3fe46bbc7afcc8274237a5b2ce2a]] - -Made it only do that for absolute paths, which should make it at least -marginally faster than git-annex find. I would not expect it to be much -faster though, because git-annex find displaying a little more information -takes negligible CPU time really. -"""]] diff --git a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__.mdwn b/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__.mdwn deleted file mode 100644 index 237f41633c..0000000000 --- a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__.mdwn +++ /dev/null @@ -1,728 +0,0 @@ -### Please describe the problem. - -A `git annex sync --no-commit --content-of X` (or `git annex sync --no-commit --content` when it -wants to get only this one file) between two git remotes takes significantly longer than a simple -`git annex get X`. It turns out that in the latter case git-annex reads the whole copied file -after its been written which to me seems it performs an extra verify / checksum pass to the file, -presumably needlessly. - -### What steps will reproduce the problem? - -1. git annex drop file -2. time git annex get file -3. git annex drop file -4. (time git annex sync --no-commit) -5. time git annex sync --no-commit --content-of file -6. (_rinse and repeat if needed_) - -The latter retrieval takes about double the time of the former. I investigated the reason -to that with [FileSpy](http://www.zezula.net/en/fstools/filespy.html), a program -by Ladislav Zezula for monitoring Windows filesystem requests. - -### What version of git-annex are you using? On what operating system? - -[[!format sh """ -git-annex version: 8.20210804-gab7b5a492 -build flags: Assistant Webapp Pairing TorrentParser Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: mingw32 x86_64 -supported repository versions: 8 -upgrade supported from repository versions: 2 3 4 5 6 7 -local repository version: 8 -"""]] - -Windows 10 version 21H1 (build 19043.1165), 64 bit. - -### Please provide any additional information below. - -First we have the -
-setup (PowerShell). -[[!format sh """ -PS I:\Reflect-varmistukset-test\Jarkon ThinkPad T450s (Win10 v21H1) . B> ls - - Directory: I:\Reflect-varmistukset-test\Jarkon ThinkPad T450s (Win10 v21H1) . B - - -Mode LastWriteTime Length Name ----- ------------- ------ ---- -d----- 25.8.2021 3:36 arkistoidut --a---- 25.8.2021 3:36 75 9BEAE03792B9FAFB-00-00.mrimg --a---- 25.8.2021 6:07 9722174075 9BEAE03792B9FAFB-01-01.mrimg --a---- 25.8.2021 3:36 74 9BEAE03792B9FAFB-02-02.mrimg --a---- 25.8.2021 3:36 617 Justfile - - -PS I:\Reflect-varmistukset-test\Jarkon ThinkPad T450s (Win10 v21H1) . B> git annex info 9BEAE03792B9FAFB-01-01.mrimg -file: 9BEAE03792B9FAFB-01-01.mrimg -size: 9.72 gigabytes -key: MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg -present: true -PS I:\Reflect-varmistukset-test\Jarkon ThinkPad T450s (Win10 v21H1) . B> git annex whereis 9BEAE03792B9FAFB-01-01.mrimg -whereis 9BEAE03792B9FAFB-01-01.mrimg (4 copies) - 3362df51-1789-4471-96a0-d2267ada6aa4 -- Reflect-varmistukset [origin] - 46a41e47-45c8-4a86-b348-db0c4cfb18f3 -- Reflect-varmistukset - e3115644-bf78-420d-89af-7256521d5dda -- Reflect-varmistukset-test [here] - f40fab57-df0e-4297-ba83-76e6ad64f9c1 -- t450s-win10-v21h1-b--chunked -ok -PS I:\Reflect-varmistukset-test\Jarkon ThinkPad T450s (Win10 v21H1) . B> git remote show -n origin -* remote origin - Fetch URL: g:\Reflect-varmistukset - Push URL: g:\Reflect-varmistukset - HEAD branch: (not queried) - Remote branches: (status not queried) - adjusted/master(hidemissing) - adjusted/master(hidemissing-unlocked) - adjusted/master(locked) - adjusted/master(unlocked) - git-annex - master - synced/git-annex - synced/master - Local branch configured for 'git pull': - adjusted/master(hidemissing-unlocked) merges with remote adjusted/master(hidemissing-unlocked) - Local ref configured for 'git push' (status not queried): - (matching) pushes to (matching) - -# End of transcript or log. -"""]] -
- -As we can see from above the file in question is rather large so re-retrieving it from the `origin` -which is on another HDD should take a couple of minutes. (We are talking 2.5" SATA drives in USB -enclosures here.) - -So we have these two retrieval operations, basically `git annex get` and `git annex sync --content`. -To measure these operations, I first `drop` the content of the file mentioned and then continue with -the respective operation in debug mode timing them. The corresponding transcripts are to be found below: - -
-`git annex get --debug X` transcript (PowerShell) -[[!format sh """ -PS I:\Reflect-varmistukset-test\Jarkon ThinkPad T450s (Win10 v21H1) . B> git annex drop 9BEAE03792B9FAFB-01-01.mrimg -drop 9BEAE03792B9FAFB-01-01.mrimg ok -(recording state in git...) -PS I:\Reflect-varmistukset-test\Jarkon ThinkPad T450s (Win10 v21H1) . B> time git annex get --debug 9BEAE03792B9FAFB-01-01.mrimg -[2021-08-27 00:08:56.0972626] (Utility.Process) process [214660] read: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","symbolic-ref","-q","HEAD"] -[2021-08-27 00:08:56.1632595] (Utility.Process) process [214660] done ExitSuccess -[2021-08-27 00:08:56.1732634] (Utility.Process) process [214844] read: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","show-ref","refs/heads/adjusted/master(unlocked)"] -[2021-08-27 00:08:56.2522579] (Utility.Process) process [214844] done ExitSuccess -[2021-08-27 00:08:56.265259] (Utility.Process) process [211408] read: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","ls-files","--stage","-z","--","9BEAE03792B9FAFB-01-01.mrimg"] -[2021-08-27 00:08:56.2793132] (Utility.Process) process [22624] chat: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)","--buffer"] -[2021-08-27 00:08:56.3472587] (Utility.Process) process [214792] chat: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"] -[2021-08-27 00:08:56.3792631] (Utility.Process) process [211324] chat: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"] -get 9BEAE03792B9FAFB-01-01.mrimg [2021-08-27 00:08:56.4872626] (Utility.Process) process [209176] read: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","show-ref","git-annex"] -[2021-08-27 00:08:56.5442548] (Utility.Process) process [209176] done ExitSuccess -[2021-08-27 00:08:56.553261] (Utility.Process) process [214420] read: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"] -[2021-08-27 00:08:56.6162598] (Utility.Process) process [214420] done ExitSuccess -[2021-08-27 00:08:56.6292572] (Utility.Process) process [24416] read: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/git-annex..00be8770deba79dd666680c63e6f5e9d894e6438","--pretty=%H","-n1"] -[2021-08-27 00:08:56.6892571] (Utility.Process) process [24416] done ExitSuccess -[2021-08-27 00:08:56.7002581] (Utility.Process) process [211376] read: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/git-annex..d992de9ad3a7f46b3d532fad356994695e4ea992","--pretty=%H","-n1"] -[2021-08-27 00:08:56.761257] (Utility.Process) process [211376] done ExitSuccess -[2021-08-27 00:08:56.7832583] (Utility.Process) process [214384] chat: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","hash-object","-w","--stdin-paths","--no-filters"] -[2021-08-27 00:08:56.8082631] (Utility.Process) process [214428] chat: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"] -[2021-08-27 00:08:56.8282591] (Utility.Process) process [209492] chat: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -[2021-08-27 00:08:56.8392637] (Utility.Process) process [214696] feed: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","update-index","-z","--index-info"] -[2021-08-27 00:08:56.8512639] (Utility.Process) process [203716] read: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","diff-index","--raw","-z","-r","--no-renames","-l0","--cached","refs/heads/git-annex","--"] -[2021-08-27 00:08:56.9772567] (Utility.Process) process [203716] done ExitSuccess -[2021-08-27 00:08:56.9982588] (Utility.Process) process [214696] done ExitSuccess -[2021-08-27 00:08:57.0332634] (Utility.Process) process [197724] read: git ["config","--null","--list"] -[2021-08-27 00:08:57.0922579] (Utility.Process) process [197724] done ExitSuccess -[2021-08-27 00:08:57.4812578] (Utility.Process) process [212412] read: git ["--git-dir=g:\\Reflect-varmistukset\\.git","--work-tree=g:\\Reflect-varmistukset","--literal-pathspecs","--literal-pathspecs","-c","filter.annex.smudge=","-c","filter.annex.clean=","write-tree"] -[2021-08-27 00:08:57.5552571] (Utility.Process) process [212412] done ExitSuccess -[2021-08-27 00:08:57.5632589] (Utility.Process) process [211664] read: git ["--git-dir=g:\\Reflect-varmistukset\\.git","--work-tree=g:\\Reflect-varmistukset","--literal-pathspecs","--literal-pathspecs","show-ref","--hash","refs/annex/last-index"] -[2021-08-27 00:08:57.6262611] (Utility.Process) process [211664] done ExitSuccess -(from origin...) -[2021-08-27 00:10:45.6276714] (Utility.Process) process [214428] done ExitSuccess -[2021-08-27 00:10:45.6376719] (Utility.Process) process [209492] done ExitSuccess -[2021-08-27 00:10:45.6426688] (Utility.Process) process [214384] done ExitSuccess -[2021-08-27 00:10:45.8886681] (Utility.Process) process [215768] read: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","-c","filter.annex.smudge=","-c","filter.annex.clean=","write-tree"] -[2021-08-27 00:10:46.0606718] (Utility.Process) process [215768] done ExitSuccess -[2021-08-27 00:10:46.0716732] (Utility.Process) process [203184] read: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/annex/last-index"] -[2021-08-27 00:10:46.1826731] (Utility.Process) process [203184] done ExitSuccess -[2021-08-27 00:10:46.7836666] (Annex.Branch) read 8b2\d61\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg.log -[2021-08-27 00:10:46.7936806] (Utility.Process) process [214544] chat: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"] -[2021-08-27 00:10:46.8076747] (Utility.Process) process [194932] chat: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -[2021-08-27 00:10:46.8916778] (Annex.Branch) set 8b2\d61\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg.log -ok -[2021-08-27 00:10:46.9066721] (Utility.Process) process [214544] done ExitSuccess -[2021-08-27 00:10:46.9166731] (Utility.Process) process [194932] done ExitSuccess -[2021-08-27 00:10:46.9406745] (Utility.Process) process [211324] done ExitSuccess -[2021-08-27 00:10:46.9406745] (Utility.Process) process [214792] done ExitSuccess -[2021-08-27 00:10:46.9406745] (Utility.Process) process [22624] done ExitSuccess -[2021-08-27 00:10:46.9416716] (Utility.Process) process [211408] done ExitSuccess -(recording state in git...) -[2021-08-27 00:10:46.9616725] (Utility.Process) process [5096] feed: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","-c","core.safecrlf=false","update-index","-q","--refresh","-z","--stdin"] -[2021-08-27 00:10:47.3986727] (Utility.Process) process [215868] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"] -[2021-08-27 00:10:47.4226935] (Utility.Process) process [210580] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -[2021-08-27 00:10:47.8616724] (Utility.Process) process [212036] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","filter.annex.smudge=","-c","filter.annex.clean=","write-tree"] -[2021-08-27 00:10:47.9116715] (Database.Keys) reconcileStaged start (in conflict) -[2021-08-27 00:10:47.9206678] (Utility.Process) process [212288] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/annex/last-index"] -[2021-08-27 00:10:47.9876758] (Utility.Process) process [210996] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)","--buffer"] -[2021-08-27 00:10:47.9976825] (Utility.Process) process [212180] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"] -[2021-08-27 00:10:48.0136775] (Utility.Process) process [215600] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","filter.annex.smudge=","-c","filter.annex.clean=","-c","diff.external=","diff","8b1d3ee71090ebc5af2edccc01c7a6ae5edb6214","--staged","--raw","-z","--no-abbrev","-G/annex/objects/","--no-renames","--ignore-submodules=all","--no-ext-diff"] -[2021-08-27 00:10:48.1186692] (Utility.Process) process [215600] done ExitSuccess -[2021-08-27 00:10:48.154675] (Utility.Process) process [212180] done ExitSuccess -[2021-08-27 00:10:48.1556741] (Utility.Process) process [210996] done ExitSuccess -[2021-08-27 00:10:48.1556741] (Database.Keys) reconcileStaged end -[2021-08-27 00:10:48.1776731] (Utility.Process) process [215868] done ExitSuccess -[2021-08-27 00:10:48.1916692] (Utility.Process) process [210580] done ExitSuccess -[2021-08-27 00:10:48.2276735] (Utility.Process) process [5096] done ExitSuccess -[2021-08-27 00:10:48.2566706] (Utility.Process) process [214308] feed: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","update-index","-z","--index[2021-08-27 00:10:48.2576698] (Utility.Process) process done ExitSuccess -[2021-08-27 00:10:48.2717046] (Utility.Process) process [214248] chat: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","hash-object","-w","--stdin-paths","--no-filters"] -[2021-08-27 00:10:48.3786694] (Utility.Process) process [214308] done ExitSuccess -[2021-08-27 00:10:48.3946743] (Utility.Process) process [207012] read: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"] -[2021-08-27 00:10:48.4666735] (Utility.Process) process [207012] done ExitSuccess -[2021-08-27 00:10:48.4766715] (Utility.Process) process [215924] chat: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"] -[2021-08-27 00:10:48.4896881] (Utility.Process) process [214828] chat: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -[2021-08-27 00:10:48.5026911] (Utility.Process) process [214220] feed: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","update-index","-z","--index-info"] -[2021-08-27 00:10:48.5786744] (Utility.Process) process [215144] read: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","diff-index","--raw","-z","-r","--no-renames","-l0","--cached","refs/heads/git-annex","--"] -[2021-08-27 00:10:48.8506705] (Utility.Process) process [215144] done ExitSuccess -[2021-08-27 00:10:48.8596735] (Utility.Process) process [214220] done ExitSuccess -[2021-08-27 00:10:48.8766719] (Utility.Process) process [56664] read: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","write-tree"] -[2021-08-27 00:10:49.2976762] (Utility.Process) process [56664] done ExitSuccess -[2021-08-27 00:10:49.3106759] (Utility.Process) process [58928] chat: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","commit-tree","16586410ea49fba2489c28486fed2d6ddf07e73b","--no-gpg-sign","-p","refs/heads/git-annex"] -[2021-08-27 00:10:49.5236701] (Utility.Process) process [58928] done ExitSuccess -[2021-08-27 00:10:49.5346727] (Utility.Process) process [210900] call: git ["--git-dir=..\\.git","--work-tree=..","--literal-pathspecs","-c","annex.debug=true","update-ref","refs/heads/git-annex","2b02572ac2e05fa9729c71fc9e5e79702f42dfa0"] -[2021-08-27 00:10:49.652673] (Utility.Process) process [210900] done ExitSuccess -[2021-08-27 00:10:49.6836729] (Utility.Process) process [215924] done ExitSuccess -[2021-08-27 00:10:49.6936724] (Utility.Process) process [214828] done ExitSuccess -[2021-08-27 00:10:49.7066751] (Utility.Process) process [214248] done ExitSuccess -00:01:53.9447083 - -# End of transcript or log. -"""]] -
- -
-`git annex sync --no-commit` & `git annex sync --no-commit --content-of X --debug` -transcript (PowerShell) -[[!format sh """ -PS I:\Reflect-varmistukset-test\Jarkon ThinkPad T450s (Win10 v21H1) . B> git annex drop 9BEAE03792B9FAFB-01-01.mrimg -drop 9BEAE03792B9FAFB-01-01.mrimg ok -(recording state in git...) -PS I:\Reflect-varmistukset-test\Jarkon ThinkPad T450s (Win10 v21H1) . B> time git annex sync --no-commit -pull origin -ok -push origin -Enumerating objects: 19, done. -Counting objects: 100% (19/19), done. -Delta compression using up to 4 threads -Compressing objects: 100% (12/12), done. -Writing objects: 100% (15/15), 1.09 KiB | 185.00 KiB/s, done. -Total 15 (delta 9), reused 0 (delta 0), pack-reused 0 -remote: Checking connectivity: 15, done. -To g:\Reflect-varmistukset - d992de9..23bccc8 git-annex -> synced/git-annex -ok -00:00:15.2608678 -PS I:\Reflect-varmistukset-test\Jarkon ThinkPad T450s (Win10 v21H1) . B> time git annex sync --no-commit --content-of 9BEAE03792B9FAFB-01-01.mrimg --debug -[2021-08-27 00:13:49.3375398] (Utility.Process) process [214132] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","git-annex"] -[2021-08-27 00:13:49.4105437] (Utility.Process) process [214132] done ExitSuccess -[2021-08-27 00:13:49.4215952] (Utility.Process) process [201560] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"] -[2021-08-27 00:13:49.496541] (Utility.Process) process [201560] done ExitSuccess -[2021-08-27 00:13:49.5115409] (Utility.Process) process [215808] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/git-annex..23bccc89e78eab969cc2241fabbbb0c0d325d857","--pretty=%H","-n1"] -[2021-08-27 00:13:49.6305429] (Utility.Process) process [215808] done ExitSuccess -[2021-08-27 00:13:49.6785395] (Utility.Process) process [216836] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","hash-object","-w","--stdin-paths","--no-filters"] -[2021-08-27 00:13:49.6955432] (Utility.Process) process [216436] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"] -[2021-08-27 00:13:49.7115402] (Utility.Process) process [209236] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -[2021-08-27 00:13:49.7265414] (Utility.Process) process [216160] feed: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","update-index","-z","--index-info"] -[2021-08-27 00:13:49.7395377] (Utility.Process) process [214088] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","diff-index","--raw","-z","-r","--no-renames","-l0","--cached","refs/heads/git-annex","--"] -[2021-08-27 00:13:49.8525381] (Utility.Process) process [214088] done ExitSuccess -[2021-08-27 00:13:49.8635371] (Utility.Process) process [216160] done ExitSuccess -[2021-08-27 00:13:49.8935393] (Utility.Process) process [208424] read: git ["config","--null","--list"] -[2021-08-27 00:13:49.9395358] (Utility.Process) process [208424] done ExitSuccess -[2021-08-27 00:13:49.9555593] (Utility.Process) process [213288] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","symbolic-ref","-q","HEAD"] -[2021-08-27 00:13:50.0065366] (Utility.Process) process [213288] done ExitSuccess -[2021-08-27 00:13:50.0175535] (Utility.Process) process [216916] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","refs/heads/adjusted/master(unlocked)"] -[2021-08-27 00:13:50.1175443] (Utility.Process) process [216916] done ExitSuccess -[2021-08-27 00:13:50.12754] (Utility.Process) process [216968] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--verify","-q","refs/heads/synced/master"] -[2021-08-27 00:13:50.2015442] (Utility.Process) process [216968] done ExitSuccess -[2021-08-27 00:13:50.2135494] (Utility.Process) process [211172] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/adjusted/master(unlocked)..refs/heads/synced/master","--pretty=%H","-n1"] -[2021-08-27 00:13:50.2915401] (Utility.Process) process [211172] done ExitSuccess -pull origin -[2021-08-27 00:13:50.3015454] (Utility.Process) process [198256] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","fetch","origin"] -[2021-08-27 00:13:50.9255402] (Utility.Process) process [198256] done ExitSuccess -[2021-08-27 00:13:50.9375907] (Utility.Process) process [216960] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","branch","-f","synced/master","refs/heads/master"] -[2021-08-27 00:13:51.0135362] (Utility.Process) process [216960] done ExitSuccess -[2021-08-27 00:13:51.0245377] (Utility.Process) process [216716] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--verify","-q","refs/remotes/origin/master"] -[2021-08-27 00:13:51.0825456] (Utility.Process) process [216716] done ExitSuccess -[2021-08-27 00:13:51.0955363] (Utility.Process) process [216888] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/master..refs/remotes/origin/master","--pretty=%H","-n1"] -[2021-08-27 00:13:51.1725363] (Utility.Process) process [216888] done ExitSuccess -[2021-08-27 00:13:51.1825395] (Utility.Process) process [215632] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--verify","-q","refs/remotes/origin/synced/master"] -[2021-08-27 00:13:51.2445413] (Utility.Process) process [215632] done ExitSuccess -[2021-08-27 00:13:51.2575386] (Utility.Process) process [18444] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/synced/master..refs/remotes/origin/synced/master","--pretty=%H","-n1"] -[2021-08-27 00:13:51.3425473] (Utility.Process) process [18444] done ExitSuccess -ok -[2021-08-27 00:13:51.3545543] (Utility.Process) process [214256] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","git-annex"] -[2021-08-27 00:13:51.4295365] (Utility.Process) process [214256] done ExitSuccess -[2021-08-27 00:13:51.4415352] (Utility.Process) process [216864] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"] -[2021-08-27 00:13:51.5255415] (Utility.Process) process [216864] done ExitSuccess -[2021-08-27 00:13:51.5475389] (Utility.Process) process [216180] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/git-annex..23bccc89e78eab969cc2241fabbbb0c0d325d857","--pretty=%H","-n1"] -[2021-08-27 00:13:51.6355408] (Utility.Process) process [216180] done ExitSuccess -[2021-08-27 00:13:51.6485405] (Utility.Process) process [215368] feed: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","update-index","-z","--index-info"] -[2021-08-27 00:13:51.6665404] (Utility.Process) process [217068] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","diff-index","--raw","-z","-r","--no-renames","-l0","--cached","refs/heads/git-annex","--"] -[2021-08-27 00:13:51.7665362] (Utility.Process) process [217068] done ExitSuccess -[2021-08-27 00:13:51.7805369] (Utility.Process) process [215368] done ExitSuccess -[2021-08-27 00:13:51.8005439] (Utility.Process) process [215204] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","ls-files","--stage","-z","--","I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg"] -[2021-08-27 00:13:51.8145422] (Utility.Process) process [214200] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)","--buffer"] -[2021-08-27 00:13:51.8265727] (Utility.Process) process [212624] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"] -[2021-08-27 00:13:51.9115397] (Utility.Process) process [214656] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"] -[2021-08-27 00:13:52.0015476] (Utility.Process) process [206800] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"] -[2021-08-27 00:13:52.0115407] (Utility.Process) process [216304] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -get Jarkon ThinkPad T450s (Win10 v21H1) . B/9BEAE03792B9FAFB-01-01.mrimg [2021-08-27 00:13:52.186541] (Utility.Process) process [215352] read: git ["--git-dir=g:\\Reflect-varmistukset\\.git","--work-tree=g:\\Reflect-varmistukset","--literal-pathspecs","--literal-pathspecs","-c","filter.annex.smudge=","-c","filter.annex.clean=","write-tree"] -[2021-08-27 00:13:52.2955443] (Utility.Process) process [215352] done ExitSuccess -[2021-08-27 00:13:52.3085383] (Utility.Process) process [216744] read: git ["--git-dir=g:\\Reflect-varmistukset\\.git","--work-tree=g:\\Reflect-varmistukset","--literal-pathspecs","--literal-pathspecs","show-ref","--hash","refs/annex/last-index"] -[2021-08-27 00:13:52.3725498] (Utility.Process) process [216744] done ExitSuccess -(from origin...) -[2021-08-27 00:15:36.2939028] (Utility.Process) process [206800] done ExitSuccess -[2021-08-27 00:15:36.3029014] (Utility.Process) process [216304] done ExitSuccess -[2021-08-27 00:15:36.3149024] (Utility.Process) process [216836] done ExitSuccess -[2021-08-27 00:15:36.3499337] (Utility.Process) process [213608] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","-c","filter.annex.smudge=","-c","filter.annex.clean=","write-tree"] -[2021-08-27 00:15:36.4519021] (Utility.Process) process [213608] done ExitSuccess -[2021-08-27 00:15:36.4608984] (Utility.Process) process [215680] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/annex/last-index"] -[2021-08-27 00:15:36.5279003] (Utility.Process) process [215680] done ExitSuccess -[2021-08-27 00:15:36.5779058] (Annex.Branch) read 8b2\d61\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg.log -[2021-08-27 00:15:36.5888998] (Utility.Process) process [215224] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"] -[2021-08-27 00:15:36.6039512] (Utility.Process) process [216116] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -[2021-08-27 00:15:36.6868986] (Annex.Branch) set 8b2\d61\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg.log -ok -[2021-08-27 00:17:46.1533905] (Utility.Process) process [217444] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","check-attr","-z","--stdin","annex.backend","annex.largefiles","annex.numcopies","annex.mincopies","--"] -[2021-08-27 00:17:46.275388] (Annex.Branch) read numcopies.log -[2021-08-27 00:17:46.3353888] (Annex.Branch) read mincopies.log -[2021-08-27 00:17:46.3783936] (Utility.Process) process [215224] done ExitSuccess -[2021-08-27 00:17:46.4003873] (Utility.Process) process [216116] done ExitSuccess -[2021-08-27 00:17:46.4393899] (Utility.Process) process [217444] done ExitSuccess -[2021-08-27 00:17:46.4454328] (Utility.Process) process done ExitSuccess -[2021-08-27 00:17:46.4474155] (Utility.Process) process [214656] done ExitSuccess -[2021-08-27 00:17:46.4524071] (Utility.Process) process done ExitSuccess -[2021-08-27 00:17:46.4553874] (Utility.Process) process done ExitSuccess -[2021-08-27 00:17:46.4553874] (Utility.Process) process [212624] done ExitSuccess -[2021-08-27 00:17:46.4553874] (Utility.Process) process [214200] done ExitSuccess -[2021-08-27 00:17:46.4563875] (Utility.Process) process [215204] done ExitSuccess -pull origin -[2021-08-27 00:17:46.4663927] (Utility.Process) process [217376] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","fetch","origin"] -[2021-08-27 00:17:47.807387] (Utility.Process) process [217376] done ExitSuccess -[2021-08-27 00:17:47.8254033] (Utility.Process) process [216624] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","branch","-f","synced/master","refs/heads/master"] -[2021-08-27 00:17:47.9233892] (Utility.Process) process [216624] done ExitSuccess -[2021-08-27 00:17:47.9403873] (Utility.Process) process [216812] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--verify","-q","refs/remotes/origin/master"] -[2021-08-27 00:17:48.0253856] (Utility.Process) process [216812] done ExitSuccess -[2021-08-27 00:17:48.0453871] (Utility.Process) process [217076] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/master..refs/remotes/origin/master","--pretty=%H","-n1"] -[2021-08-27 00:17:48.1843855] (Utility.Process) process [217076] done ExitSuccess -[2021-08-27 00:17:48.1963916] (Utility.Process) process [213108] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--verify","-q","refs/remotes/origin/synced/master"] -[2021-08-27 00:17:48.2783861] (Utility.Process) process [213108] done ExitSuccess -[2021-08-27 00:17:48.2963914] (Utility.Process) process [217628] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/synced/master..refs/remotes/origin/synced/master","--pretty=%H","-n1"] -[2021-08-27 00:17:48.3833858] (Utility.Process) process [217628] done ExitSuccess -ok -[2021-08-27 00:17:48.3993895] (Utility.Process) process [213996] feed: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","update-index","-z","--index-info"] -[2021-08-27 00:17:48.4083897] (Utility.Process) process done ExitSuccess -[2021-08-27 00:17:48.4303952] (Utility.Process) process [216136] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","hash-object","-w","--stdin-paths","--no-filters"] -[2021-08-27 00:17:48.654389] (Utility.Process) process [213996] done ExitSuccess -[2021-08-27 00:17:48.6703942] (Utility.Process) process [214536] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"] -[2021-08-27 00:17:49.0205141] (Utility.Process) process [214536] done ExitSuccess -(recording state in git...) -[2021-08-27 00:17:49.0415205] (Utility.Process) process [217516] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"] -[2021-08-27 00:17:49.0565146] (Utility.Process) process [217488] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -[2021-08-27 00:17:49.0745205] (Utility.Process) process [217464] feed: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","update-index","-z","--index-info"] -[2021-08-27 00:17:49.0885149] (Utility.Process) process [217664] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","diff-index","--raw","-z","-r","--no-renames","-l0","--cached","refs/heads/git-annex","--"] -[2021-08-27 00:17:49.3175172] (Utility.Process) process [217664] done ExitSuccess -[2021-08-27 00:17:49.3325187] (Utility.Process) process [217464] done ExitSuccess -[2021-08-27 00:17:49.3505184] (Utility.Process) process [217720] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","write-tree"] -[2021-08-27 00:17:49.7095175] (Utility.Process) process [217720] done ExitSuccess -[2021-08-27 00:17:49.7205203] (Utility.Process) process [217760] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","commit-tree","1a868cb2ce3525f2268df3faa4f1784593d6ee5e","--no-gpg-sign","-p","refs/heads/git-annex"] -[2021-08-27 00:17:49.8860794] (Utility.Process) process [217760] done ExitSuccess -[2021-08-27 00:17:49.8990766] (Utility.Process) process [217916] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","update-ref","refs/heads/git-annex","f050618c3839d83854bc5a83bead8d49694fe17e"] -[2021-08-27 00:17:50.0160774] (Utility.Process) process [217916] done ExitSuccess -[2021-08-27 00:17:50.0310721] (Utility.Process) process [217748] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","git-annex"] -[2021-08-27 00:17:50.1360769] (Utility.Process) process [217748] done ExitSuccess -[2021-08-27 00:17:50.148072] (Utility.Process) process [217944] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"] -[2021-08-27 00:17:50.260071] (Utility.Process) process [217944] done ExitSuccess -[2021-08-27 00:17:50.2730764] (Utility.Process) process [213304] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/git-annex..f050618c3839d83854bc5a83bead8d49694fe17e","--pretty=%H","-n1"] -[2021-08-27 00:17:50.3800768] (Utility.Process) process [213304] done ExitSuccess -[2021-08-27 00:17:50.393075] (Utility.Process) process [209548] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/git-annex..23bccc89e78eab969cc2241fabbbb0c0d325d857","--pretty=%H","-n1"] -[2021-08-27 00:17:50.4730747] (Utility.Process) process [209548] done ExitSuccess -[2021-08-27 00:17:50.4880776] (Utility.Process) process [215656] feed: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","update-index","-z","--index-info"] -[2021-08-27 00:17:50.5020794] (Utility.Process) process [218048] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","diff-index","--raw","-z","-r","--no-renames","-l0","--cached","refs/heads/git-annex","--"] -[2021-08-27 00:17:50.5940713] (Utility.Process) process [218048] done ExitSuccess -[2021-08-27 00:17:50.6050732] (Utility.Process) process [215656] done ExitSuccess -[2021-08-27 00:17:50.6260788] (Utility.Process) process [217248] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/basis/adjusted/master(unlocked)"] -[2021-08-27 00:17:50.7000724] (Utility.Process) process [217248] done ExitSuccess -[2021-08-27 00:17:50.7100744] (Utility.Process) process [211624] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"] -[2021-08-27 00:17:50.7250742] (Utility.Process) process [14260] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -[2021-08-27 00:17:50.8180757] (Utility.Process) process [217092] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/basis/adjusted/master(unlocked)..refs/heads/adjusted/master(unlocked)","--pretty=%H","--reverse"] -[2021-08-27 00:17:50.9546056] (Utility.Process) process [217092] done ExitSuccess -[2021-08-27 00:17:50.968617] (Utility.Process) process [12384] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","update-ref","refs/basis/adjusted/master(unlocked)","da6ae1e1f6e3024f0f90079332685bbd4d950068"] -[2021-08-27 00:17:51.0826057] (Utility.Process) process [12384] done ExitSuccess -[2021-08-27 00:17:51.1016108] (Utility.Process) process [217132] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","update-ref","refs/heads/master","da6ae1e1f6e3024f0f90079332685bbd4d950068"] -[2021-08-27 00:17:51.2076769] (Utility.Process) process [217132] done ExitSuccess -[2021-08-27 00:17:51.2306834] (Utility.Process) process [211008] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","commit-tree","8b1d3ee71090ebc5af2edccc01c7a6ae5edb6214","--no-gpg-sign","-p","da6ae1e1f6e3024f0f90079332685bbd4d950068"] -[2021-08-27 00:17:51.3016768] (Utility.Process) process [211008] done ExitSuccess -[2021-08-27 00:17:51.3126762] (Utility.Process) process [216868] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","update-ref","-m","rebasing adjusted branch on top of updated original branch","refs/heads/adjusted/master(unlocked)","12230f33ae02f213ac044680f6e2f44ad504d81b"] -[2021-08-27 00:17:51.4186757] (Utility.Process) process [216868] done ExitSuccess -[2021-08-27 00:17:51.4306819] (Utility.Process) process [213772] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","branch","-f","synced/master","refs/heads/master"] -[2021-08-27 00:17:51.5206755] (Utility.Process) process [213772] done ExitSuccess -[2021-08-27 00:17:51.5306808] (Utility.Process) process [217164] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--verify","-q","refs/remotes/origin/synced/master"] -[2021-08-27 00:17:51.5966814] (Utility.Process) process [217164] done ExitSuccess -[2021-08-27 00:17:51.6106763] (Utility.Process) process [214000] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/remotes/origin/synced/master..refs/heads/synced/master","--pretty=%H","-n1"] -[2021-08-27 00:17:51.7146793] (Utility.Process) process [214000] done ExitSuccess -[2021-08-27 00:17:51.7236796] (Utility.Process) process [216680] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--verify","-q","refs/remotes/origin/git-annex"] -[2021-08-27 00:17:51.8096754] (Utility.Process) process [216680] done ExitSuccess -[2021-08-27 00:17:51.8216807] (Utility.Process) process [217408] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/remotes/origin/git-annex..git-annex","--pretty=%H","-n1"] -[2021-08-27 00:17:51.9116765] (Utility.Process) process [217408] done ExitSuccess -push origin -ster","+git-annex:synced/git-annex"] -Counting objects: 100% (9/9), done. -Delta compression using up to 4 threads -Compressing objects: 100% (4/4), done. -Writing objects: 100% (5/5), 410 bytes | 37.00 KiB/s, done. -Total 5 (delta 3), reused 0 (delta 0), pack-reused 0 -To g:\Reflect-varmistukset - 23bccc8..f050618 git-annex -> synced/git-annex -[2021-08-27 00:17:55.0492122] (Utility.Process) process [216816] done ExitSuccess -[2021-08-27 00:17:55.0622115] (Utility.Process) process [217956] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","push","origin","git-annex"] -[2021-08-27 00:17:56.7432043] (Utility.Process) process [217956] done ExitSuccess -[2021-08-27 00:17:56.7552126] (Utility.Process) process [215284] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","push","origin","master"] -[2021-08-27 00:17:57.1182069] (Utility.Process) process [215284] done ExitSuccess -ok -(recording state in git...) -[2021-08-27 00:17:57.1552103] (Utility.Process) process [217300] feed: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","-c","core.safecrlf=false","update-index","-q","--refresh","-z","--stdin"] -[2021-08-27 00:17:57.6632069] (Utility.Process) process [215568] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"] -[2021-08-27 00:17:57.6752084] (Utility.Process) process [217372] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -[2021-08-27 00:17:57.7862096] (Utility.Process) process [217436] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","filter.annex.smudge=","-c","filter.annex.clean=","write-tree"] -[2021-08-27 00:17:57.8792042] (Utility.Process) process [217436] done ExitFailure 128 -[2021-08-27 00:17:57.8812054] (Database.Keys) reconcileStaged start (in conflict) -[2021-08-27 00:17:57.8932088] (Utility.Process) process [214408] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/annex/last-index"] -[2021-08-27 00:17:57.9732051] (Utility.Process) process [214408] done ExitSuccess -[2021-08-27 00:17:57.9852111] (Utility.Process) process [217984] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)","--buffer"] -[2021-08-27 00:17:58.0202119] (Utility.Process) process [216252] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"] -[2021-08-27 00:17:58.0352085] (Utility.Process) process [205448] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","filter.annex.smudge=","-c","filter.annex.clean=","-c","diff.external=","diff","8b1d3ee71090ebc5af2edccc01c7a6ae5edb6214","--staged","--raw","-z","--no-abbrev","-G/annex/objects/","--no-renames","--ignore-submodules=all","--no-ext-diff"] -[2021-08-27 00:17:58.2442068] (Utility.Process) process [205448] done ExitSuccess -[2021-08-27 00:17:58.2732066] (Utility.Process) process [216252] done ExitSuccess -[2021-08-27 00:17:58.2822569] (Utility.Process) process [217984] done ExitSuccess -[2021-08-27 00:17:58.284208] (Database.Keys) reconcileStaged end -[2021-08-27 00:17:58.3102596] (Utility.Process) process [215568] done ExitSuccess -[2021-08-27 00:17:58.3332062] (Utility.Process) process [217372] done ExitSuccess -[2021-08-27 00:17:58.4222128] (Utility.Process) process [217300] done ExitSuccess -[2021-08-27 00:17:58.4702079] (Utility.Process) process [211624] done ExitSuccess -[2021-08-27 00:17:58.4832068] (Utility.Process) process [14260] done ExitSuccess -[2021-08-27 00:17:58.4992128] (Utility.Process) process [217516] done ExitSuccess -[2021-08-27 00:17:58.5142044] (Utility.Process) process [217488] done ExitSuccess -[2021-08-27 00:17:58.5232096] (Utility.Process) process [216136] done ExitSuccess -00:04:09.6301535 - -# End of transcript or log. -"""]] -
- -Please note that the `time` command output in the powershell transcripts above is non-standard/different -from what we are used to in POSIXland. It's a utility/script from the Scoop installation manager (psutils -package) and simply produces a wall-clock execution time in a hours, minutes, seconds, and microseconds format -(HH:MM:SS.uuuuuu). As you can see, I have also timed a plain `git annex sync --no-commit` in the latter transcript. - -
-FileSpy trace on Windows during `sync --no-commit --content-of X` (the first 160 and the last 120 lines of a converted file [log->csv->markdown]) -[[!format mdwn """ -| # | Time sent | Dur. | Process | Request | IRP Flags | FsContext | Path | Status | More info | -|--------|--------------|------|---------------|---------------------------|-----------|------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| 1 | 00:13:49.235 | | git.exe | IRP_MJ_CREATE | 884 | 0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\.git | STATUS_OBJECT_NAME_NOT_FOUND | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 | -| 2 | 00:13:49.235 | | git.exe | IRP_MJ_NETWORK_QUERY_OPEN | | 0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\HEAD | STATUS_OBJECT_NAME_NOT_FOUND | FileNetworkOpenInformation CreateTime: 0 LastAccTime: 0 LastWrtTime: 0 ChangeTime: 0 AllocationSize: 00000000-00000000 EndOfFile: 00000000-00000000 Attrib: 0 | -| 3 | 00:13:49.277 | | git-annex.exe | IRP_MJ_CREATE | 884 | 0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\.git | STATUS_OBJECT_NAME_NOT_FOUND | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 | -| 4 | 00:13:49.277 | | git-annex.exe | IRP_MJ_CREATE | 884 | 0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\config | STATUS_OBJECT_NAME_NOT_FOUND | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 | -| 5 | 00:13:49.278 | | git-annex.exe | IRP_MJ_NETWORK_QUERY_OPEN | | 0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\git.exe | STATUS_OBJECT_NAME_NOT_FOUND | FileNetworkOpenInformation CreateTime: 0 LastAccTime: 0 LastWrtTime: 0 ChangeTime: 0 AllocationSize: 00000000-00000000 EndOfFile: 00000000-00000000 Attrib: 0 | -| 6 | 00:13:51.788 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 7 | 00:13:51.788 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 8 | 00:13:51.788 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 9 | 00:13:51.789 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 10 | 00:13:51.789 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 11 | 00:13:51.789 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 12 | 00:13:51.789 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 13 | 00:13:51.789 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 14 | 00:13:51.789 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 15 | 00:13:51.789 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 16 | 00:13:51.789 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 17 | 00:13:51.789 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 18 | 00:13:51.789 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00120089 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 19 | 00:13:51.789 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 20 | 00:13:51.789 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 21 | 00:13:51.911 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 22 | 00:13:51.912 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 23 | 00:13:51.912 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 24 | 00:13:51.912 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 25 | 00:13:51.912 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 26 | 00:13:51.912 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 27 | 00:13:51.912 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 28 | 00:13:51.932 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 29 | 00:13:51.932 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 30 | 00:13:51.932 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 31 | 00:13:51.932 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 32 | 00:13:51.932 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 33 | 00:13:51.933 | 4 | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00120089 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 34 | 00:13:51.937 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 35 | 00:13:51.937 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 36 | 00:13:51.991 | | git-annex.exe | IRP_MJ_CREATE | 884 | 0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_OBJECT_NAME_NOT_FOUND | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 | -| 37 | 00:13:51.992 | | git-annex.exe | IRP_MJ_CREATE | 884 | 0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_OBJECT_NAME_NOT_FOUND | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 | -| 38 | 00:14:24.396 | | System | IRP_MJ_CLOSE | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 39 | 00:15:36.317 | | git-annex.exe | IRP_MJ_CREATE | 884 | 0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_OBJECT_NAME_NOT_FOUND | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 | -| 40 | 00:15:36.318 | | git-annex.exe | IRP_MJ_CREATE | 884 | 0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_OBJECT_NAME_NOT_FOUND | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 | -| 41 | 00:15:36.318 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78379036170 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 42 | 00:15:36.318 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78379036170 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 43 | 00:15:36.318 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA78379036170 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 44 | 00:15:36.318 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78379036170 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 0 Access: 00100002 Share: 00000003 Attrib: 0 Result: FILE_OPENED | -| 45 | 00:15:36.319 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78379036170 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 46 | 00:15:36.319 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA78379036170 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 47 | 00:15:36.319 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 48 | 00:15:36.319 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 49 | 00:15:36.528 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 50 | 00:15:36.528 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 51 | 00:15:36.528 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 52 | 00:15:36.530 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 53 | 00:15:36.530 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 54 | 00:15:36.530 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 55 | 00:15:36.531 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 56 | 00:15:36.532 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 57 | 00:15:36.532 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 58 | 00:15:36.532 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 59 | 00:15:36.532 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 60 | 00:15:36.532 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 61 | 00:15:36.533 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00120089 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 62 | 00:15:36.533 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 63 | 00:15:36.533 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 64 | 00:15:36.535 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 65 | 00:15:36.535 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 66 | 00:15:36.535 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 67 | 00:15:36.535 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 68 | 00:15:36.535 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 69 | 00:15:36.535 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 70 | 00:15:36.536 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 72 | 00:15:36.536 | | git-annex.exe | IRP_MJ_READ | 60900 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | Offset: 00000000-00000000 ToRead: 2000 Read: 48 | -| 71 | 00:15:36.536 | | git-annex.exe | IRP_MJ_READ | 60403 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | Offset: 00000000-00000000 ToRead: 1000 Read: 48 | -| 73 | 00:15:36.536 | | git-annex.exe | IRP_MJ_READ | | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_END_OF_FILE | Offset: 00000000-00000048 ToRead: 2000 Read: 0 | -| 74 | 00:15:36.536 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 75 | 00:15:36.536 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 76 | 00:15:36.537 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 77 | 00:15:36.537 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 78 | 00:15:36.537 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 79 | 00:15:36.537 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 80 | 00:15:36.537 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 81 | 00:15:36.537 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 82 | 00:15:36.537 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 83 | 00:15:36.537 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 84 | 00:15:36.537 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 85 | 00:15:36.537 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 86 | 00:15:36.537 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 87 | 00:15:36.538 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00120089 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 88 | 00:15:36.538 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 89 | 00:15:36.538 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 90 | 00:15:36.538 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00200040 Access: 00010080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 91 | 00:15:36.538 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 92 | 00:15:36.538 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 93 | 00:15:36.540 | 1 | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 94 | 00:15:36.541 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 95 | 00:15:36.541 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 96 | 00:15:36.541 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 97 | 00:15:36.541 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 98 | 00:15:36.541 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 99 | 00:15:36.541 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 100 | 00:15:36.541 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 101 | 00:15:36.541 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 102 | 00:15:36.542 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 103 | 00:15:36.542 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 104 | 00:15:36.542 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 105 | 00:15:36.542 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00120089 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 106 | 00:15:36.542 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 107 | 00:15:36.542 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 108 | 00:15:36.543 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00200020 Access: 00100100 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 109 | 00:15:36.543 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 110 | 00:15:36.543 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 111 | 00:15:36.554 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 112 | 00:15:36.554 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 113 | 00:15:36.557 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 114 | 00:15:36.557 | 1 | git-annex.exe | IRP_MJ_READ | 60900 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | Offset: 00000000-00000000 ToRead: 2000 Read: 2000 | -| 115 | 00:15:36.558 | 18 | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 116 | 00:15:36.576 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 117 | 00:15:36.687 | 2 | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 118 | 00:15:36.690 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 119 | 00:15:36.690 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 120 | 00:15:36.690 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 121 | 00:15:36.691 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 122 | 00:15:36.691 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 123 | 00:15:36.691 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 124 | 00:15:36.691 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 125 | 00:15:36.691 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 126 | 00:15:36.691 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 127 | 00:15:36.692 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 128 | 00:15:36.692 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 129 | 00:15:36.692 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 130 | 00:15:36.692 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 131 | 00:15:36.692 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 132 | 00:15:36.692 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 133 | 00:15:36.692 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 134 | 00:15:36.692 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 135 | 00:15:36.693 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00120089 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 136 | 00:15:36.693 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 137 | 00:15:36.693 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 138 | 00:15:36.695 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 139 | 00:15:36.695 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 140 | 00:15:36.695 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 141 | 00:15:36.695 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 142 | 00:15:36.696 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 143 | 00:15:36.696 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 144 | 00:15:36.716 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 145 | 00:15:36.717 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 146 | 00:15:36.717 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 147 | 00:15:36.717 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 148 | 00:15:36.717 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 149 | 00:15:36.717 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 150 | 00:15:36.717 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 151 | 00:15:36.717 | | git-annex.exe | IRP_MJ_READ | 60900 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-00000000 ToRead: 7FF0 Read: 7FF0 | -| 152 | 00:15:36.719 | | System | IRP_MJ_CLOSE | 404 | FFFFA78403C2D7C0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 153 | 00:15:36.720 | | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-00007FF0 ToRead: 7FF0 Read: 7FF0 | -| 154 | 00:15:36.720 | | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-0000FFE0 ToRead: 7FF0 Read: 7FF0 | -| 155 | 00:15:36.720 | | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-00017FD0 ToRead: 7FF0 Read: 7FF0 | -| 156 | 00:15:36.720 | | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-0001FFC0 ToRead: 7FF0 Read: 7FF0 | -| 157 | 00:15:36.721 | | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-00027FB0 ToRead: 7FF0 Read: 7FF0 | -| 158 | 00:15:36.722 | 34 | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-0002FFA0 ToRead: 7FF0 Read: 7FF0 | -| 159 | 00:15:36.757 | 2 | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-00037F90 ToRead: 7FF0 Read: 7FF0 | -| 296985 | 00:17:46.052 | | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000002-437807F0 ToRead: 7FF0 Read: 7FF0 | -| 296986 | 00:17:46.052 | | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000002-437887E0 ToRead: 7FF0 Read: 7FF0 | -| 296987 | 00:17:46.053 | | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000002-437907D0 ToRead: 7FF0 Read: 7FF0 | -| 296988 | 00:17:46.053 | | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000002-437987C0 ToRead: 7FF0 Read: 7FF0 | -| 296989 | 00:17:46.068 | | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000002-437A07B0 ToRead: 7FF0 Read: 7FF0 | -| 296990 | 00:17:46.069 | | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000002-437A87A0 ToRead: 7FF0 Read: 7FF0 | -| 296991 | 00:17:46.069 | 23 | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000002-437B0790 ToRead: 7FF0 Read: 7FF0 | -| 296992 | 00:17:46.096 | | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000002-437B8780 ToRead: 7FF0 Read: 7FF0 | -| 296993 | 00:17:46.099 | | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000002-437C0770 ToRead: 7FF0 Read: 7FF0 | -| 296994 | 00:17:46.099 | | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000002-437C8760 ToRead: 7FF0 Read: 131B | -| 296995 | 00:17:46.099 | 1 | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_END_OF_FILE | Offset: 00000002-437C9A7B ToRead: 6CD5 Read: 0 | -| 296996 | 00:17:46.101 | | git-annex.exe | IRP_MJ_READ | | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_END_OF_FILE | Offset: 00000002-437C9A7B ToRead: 7FF0 Read: 0 | -| 296997 | 00:17:46.102 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 296998 | 00:17:46.102 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 296999 | 00:17:46.102 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297000 | 00:17:46.102 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 297001 | 00:17:46.102 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 297002 | 00:17:46.102 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297003 | 00:17:46.103 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 297004 | 00:17:46.103 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 297005 | 00:17:46.107 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 297006 | 00:17:46.107 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 297007 | 00:17:46.107 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 297008 | 00:17:46.108 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297009 | 00:17:46.108 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 297010 | 00:17:46.108 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 297011 | 00:17:46.109 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00120089 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297012 | 00:17:46.109 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 297013 | 00:17:46.109 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 297014 | 00:17:46.114 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 297015 | 00:17:46.114 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 297016 | 00:17:46.114 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 297017 | 00:17:46.115 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297018 | 00:17:46.115 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 297019 | 00:17:46.115 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | | -| 297020 | 00:17:46.275 | | git.exe | IRP_MJ_NETWORK_QUERY_OPEN | | 0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\.gitattributes | STATUS_OBJECT_NAME_NOT_FOUND | FileNetworkOpenInformation CreateTime: 0 LastAccTime: 0 LastWrtTime: 0 ChangeTime: 0 AllocationSize: 00000000-00000000 EndOfFile: 00000000-00000000 Attrib: 0 | -| 297021 | 00:17:57.163 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297022 | 00:17:57.163 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297023 | 00:17:57.163 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297024 | 00:17:57.164 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297025 | 00:17:57.164 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297026 | 00:17:57.164 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297027 | 00:17:57.164 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 297028 | 00:17:57.164 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297029 | 00:17:57.164 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297030 | 00:17:57.165 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297031 | 00:17:57.165 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297032 | 00:17:57.165 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297033 | 00:17:57.165 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00120089 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297034 | 00:17:57.166 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297035 | 00:17:57.166 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297036 | 00:17:57.170 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 297037 | 00:17:57.170 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297038 | 00:17:57.170 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297039 | 00:17:57.171 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297040 | 00:17:57.171 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297041 | 00:17:57.171 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297042 | 00:17:57.249 | | git.exe | IRP_MJ_CREATE | 884 | FFFFA7841BF14170 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\arkistoidut | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100081 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297043 | 00:17:57.249 | | git.exe | IRP_MJ_CLEANUP | 404 | FFFFA7841BF14170 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\arkistoidut | STATUS_SUCCESS | | -| 297044 | 00:17:57.249 | | git.exe | IRP_MJ_CLOSE | 404 | FFFFA7841BF14170 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\arkistoidut | STATUS_SUCCESS | | -| 297045 | 00:17:57.251 | | git.exe | IRP_MJ_NETWORK_QUERY_OPEN | | 0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FileNetworkOpenInformation CreateTime: 01D79ABF-4571077F LastAccTime: 01D79ABF-D0AAA385 LastWrtTime: 01D79ABF-833DDB03 ChangeTime: 01D79ABF-8373B7ED AllocationSize: 00000002-437CA000 EndOfFile: 00000002-437C9A7B Attrib: 00000020 | -| 297046 | 00:17:57.251 | | git.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000003 Attrib: 00000080 Result: FILE_OPENED | -| 297047 | 00:17:57.253 | | git.exe | IRP_MJ_NETWORK_QUERY_OPEN | | 0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\.gitattributes | STATUS_OBJECT_NAME_NOT_FOUND | FileNetworkOpenInformation CreateTime: 0 LastAccTime: 0 LastWrtTime: 0 ChangeTime: 0 AllocationSize: 00000000-00000000 EndOfFile: 00000000-00000000 Attrib: 0 | -| 297048 | 00:17:57.292 | 8 | git.exe | IRP_MJ_READ | 60043 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-00000000 ToRead: 8000 Read: 8000 | -| 297049 | 00:17:57.300 | 25 | git.exe | IRP_MJ_READ | 60043 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-00008000 ToRead: 8000 Read: 8000 | -| 297050 | 00:17:57.325 | 23 | git.exe | IRP_MJ_READ | 60043 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-00010000 ToRead: 10000 Read: 10000 | -| 297051 | 00:17:57.349 | 10 | git.exe | IRP_MJ_READ | 60043 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-00020000 ToRead: 20000 Read: 20000 | -| 297052 | 00:17:57.359 | 1 | git.exe | IRP_MJ_READ | 60043 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-00040000 ToRead: 40000 Read: 40000 | -| 297053 | 00:17:57.362 | 11 | git.exe | IRP_MJ_READ | 60043 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-00080000 ToRead: 80000 Read: 80000 | -| 297054 | 00:17:57.374 | 9 | git.exe | IRP_MJ_READ | 60043 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-00100000 ToRead: 100000 Read: 100000 | -| 297055 | 00:17:57.384 | 29 | git.exe | IRP_MJ_READ | 60043 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-00200000 ToRead: 100000 Read: 100000 | -| 297056 | 00:17:57.415 | 4 | git.exe | IRP_MJ_READ | 60043 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-00300000 ToRead: 100000 Read: 100000 | -| 297057 | 00:17:57.421 | 11 | git.exe | IRP_MJ_READ | 60043 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-00400000 ToRead: 100000 Read: 100000 | -| 297058 | 00:17:57.435 | 28 | git.exe | IRP_MJ_READ | 60043 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-00500000 ToRead: 100000 Read: 100000 | -| 297059 | 00:17:57.465 | 13 | git.exe | IRP_MJ_READ | 60043 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-00600000 ToRead: 100000 Read: 100000 | -| 297060 | 00:17:57.479 | 27 | git.exe | IRP_MJ_READ | 60043 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\.git\\annex\\objects\\8b2\\d61\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg\\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg | STATUS_SUCCESS | Offset: 00000000-00700000 ToRead: 100000 Read: 100000 | -| 297061 | 00:17:57.756 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297062 | 00:17:57.756 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297063 | 00:17:57.756 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297064 | 00:17:57.756 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297065 | 00:17:57.756 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297066 | 00:17:57.756 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297067 | 00:17:57.757 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 297068 | 00:17:57.757 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297069 | 00:17:57.757 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297070 | 00:17:57.757 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297071 | 00:17:57.757 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297072 | 00:17:57.757 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297073 | 00:17:57.757 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00120089 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297074 | 00:17:57.757 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297075 | 00:17:57.757 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297076 | 00:17:57.759 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 297077 | 00:17:57.759 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297078 | 00:17:57.759 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297079 | 00:17:57.760 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297080 | 00:17:57.760 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297081 | 00:17:57.760 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297082 | 00:17:58.285 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297083 | 00:17:58.285 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297084 | 00:17:58.285 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297085 | 00:17:58.285 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297086 | 00:17:58.285 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297087 | 00:17:58.285 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297088 | 00:17:58.286 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 297089 | 00:17:58.286 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297090 | 00:17:58.286 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297091 | 00:17:58.286 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297092 | 00:17:58.286 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297093 | 00:17:58.286 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297094 | 00:17:58.286 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00120089 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297095 | 00:17:58.286 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297096 | 00:17:58.286 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297097 | 00:17:58.288 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000060 Access: 00120089 Share: 00000007 Attrib: 00000080 Result: FILE_OPENED | -| 297098 | 00:17:58.288 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297099 | 00:17:58.288 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297100 | 00:17:58.289 | | git-annex.exe | IRP_MJ_CREATE | 884 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | FILE_OPEN CreOpts: 00000020 Access: 00100080 Share: 00000007 Attrib: 0 Result: FILE_OPENED | -| 297101 | 00:17:58.289 | | git-annex.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297102 | 00:17:58.289 | | git-annex.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297103 | 00:17:58.410 | | git.exe | IRP_MJ_CLEANUP | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -| 297104 | 00:17:58.410 | | git.exe | IRP_MJ_CLOSE | 404 | FFFFA783ED9DF1B0 | I:\\Reflect-varmistukset-test\\Jarkon ThinkPad T450s (Win10 v21H1) . B\\9BEAE03792B9FAFB-01-01.mrimg | STATUS_SUCCESS | | -"""]] -
- -If you compare the latter transcript and the FileSpy trace (which contains only open, close and read requests), -you notice that around timestamp -"[2021-08-27 00:15:36.6868986]" git-annex says -"(Annex.Branch) set 8b2\d61\MD5E-s9722174075--f7cd18042ba15862c395d98f61eed228.mrimg.log" after which -it stays silent for around two minutes during which the FileSpy trace shows it's reading[^1] the annexed file -from .git/annex/objects presumably for verification / checksumming (I've removed the boring middle section -from the originally huge trace log, so that's why the event number in the first column increases suddenly -from 159 to 296985). - -[^1]: incidentally, 0x7FF0 (32752) bytes at a time - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Git Annex is great. It works quite nicely with my multi-gigabyte backup files (largest around 180GB) via the BLAKE2B160E backend :) - -[[!meta title="windows: sync -C takes longer than get, apparently extra checksum"]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_10_43f5ed4a319fcf63387395f32a345852._comment b/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_10_43f5ed4a319fcf63387395f32a345852._comment deleted file mode 100644 index 1cb822e8e7..0000000000 --- a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_10_43f5ed4a319fcf63387395f32a345852._comment +++ /dev/null @@ -1,38 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 10""" - date="2021-10-01T14:57:11Z" - content=""" -The second hash is done by isUnmodifiedLowLevel, and -is indeed done because the inode cache does not verify. - -The inode cache does not verify because for some reason, -it does not have any inode cached for the object file -when that is called. - -The reason this only happens in sync and not get is that sync also sends -files it received to other remotes, and to do that it needs to check if -their content is present. Which is normally inexpensive, so not a problem. - -It so happens that sync calls inAnnex even when -there are no other remotes to send the content to. Which is unnecessary, and -avoiding that could be used to fix this problem. But there would still be a -problem if there were some other remote that the content ought to be sent to, -if it still called inAnnex in that case. Maybe it's possible to avoid it calling -inAnnex at all though, since it knows when it's gotten the content. - -The inode cache gets written to the database before sync calls -inAnnex. And queued writes get flushed to the database before reading -from it. I've confirmed all this is happening. -So why does getting the inode cache from the db not see the previously -written data? - -I suspect this is where NTFS enters the picture. Perhaps -something about it is causing changes to the sqlite database to not -be visible immediately to a reader. Might involve whatever locking -sqlite is or is not able to do on NTFS, or something like that. - -(Interestingly, at the end of the sync, the smudge clean filter -gets called, and it is able to read the inode cache from the db -at that point. So another process run just a bit later can see the data.) -"""]] diff --git a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_11_d485bbc1c0f2fceaaa2e4142dccdc1f7._comment b/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_11_d485bbc1c0f2fceaaa2e4142dccdc1f7._comment deleted file mode 100644 index 820b3972aa..0000000000 --- a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_11_d485bbc1c0f2fceaaa2e4142dccdc1f7._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 11""" - date="2021-10-01T15:57:12Z" - content=""" -Fixed by avoiding sync calling inAnnex when it knows it has the content, -because it just got it. - -This does leave open the possibility that there are similar situations -elsewhere, that lead to either extra work like this, or to incorrect -behavior. Since sqlite write followed by a read is generally something -git-annex is careful of, and also since it is generally careful to have -reasonable behavior is sqlite somehow loses data, I'm not too worried about -incorrect behavior. I feel comfortable closing this bug with just this fix, -despite not getting to the bottom of the issue of why sqlite writes are -not immediately able to be read back on NTFS. -"""]] diff --git a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_12_b0132b0f846e4882a5511b53694a1fa2._comment b/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_12_b0132b0f846e4882a5511b53694a1fa2._comment deleted file mode 100644 index d5a71432b0..0000000000 --- a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_12_b0132b0f846e4882a5511b53694a1fa2._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 12" - date="2021-10-02T19:58:17Z" - content=""" -Thanks, Joey! 💐 Well done. :) -"""]] diff --git a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_1_fa2fb2493ea1244f334b54e6af2b553b._comment b/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_1_fa2fb2493ea1244f334b54e6af2b553b._comment deleted file mode 100644 index abcd6f1719..0000000000 --- a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_1_fa2fb2493ea1244f334b54e6af2b553b._comment +++ /dev/null @@ -1,73 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2021-09-06T18:52:21Z" - content=""" -I tried to reproduce this on Linux, and cannot: - - joey@darkstar:~/tmp/r>/usr/bin/time git-annex get bigfile - get bigfile (from origin...) - ok - (recording state in git...) - 11.66user 2.08system 0:13.76elapsed 99%CPU (0avgtext+0avgdata 99012maxresident)k - 206440inputs+2048856outputs (46major+21865minor)pagefaults 0swaps - joey@darkstar:~/tmp/r>git annex drop - drop bigfile ok - (recording state in git...) - joey@darkstar:~/tmp/r>/usr/bin/time git annex sync --no-commit --no-push --no-pull --content-of bigfile - get bigfile (from origin...) - ok - (recording state in git...) - 11.43user 1.91system 0:13.43elapsed 99%CPU (0avgtext+0avgdata 97132maxresident)k - 60016inputs+2048312outputs (25major+11680minor)pagefaults 0swaps - -One reason it may do an extra checksum is if for some reason the inode -cache is stale. Since [[!commit 3b5a3e168d8decd196509ad582ad4b8795d979a6]] -it will check that and checksum before starting to send the content. - -In this case, it will display "(checksum...)" before the progress display -has started. - -One way I can reproduce that, on Linux, is to touch the annex object file -in the remote git repository before running git-annex. This is the result: - - joey@darkstar:~/tmp/r>/usr/bin/time git-annex get bigfile - get bigfile (from origin...) (checksum...) - ok - (recording state in git...) - 22.06user 2.22system 0:24.85elapsed 97%CPU (0avgtext+0avgdata 97192maxresident)k - 704inputs+2048432outputs (33major+10701minor)pagefaults 0swaps - -But that shows that git-annex get also behaves that way, which does not -explain why only git-annex sync would have the problem for you. -(And the two run the same code; sync literally runs code from get.) Also -I don't know why the inode cache would be getting stale on windows. - -If it does not display "(checksum...)" before the progress display, -it's certianly not that. And if it does not display "(checksum...)" -at any point at all, that would be good to know, because normally -git-annex does display that when checksumming. -Your transcript seems to show no such output. - -At the "2021-08-27 00:15:36.6868986" timestamp in your log, it's just -completed updating the location log, and you identify the pause after that -as where it's checksumming. So that seems to suggest it's already -transferred the content there, and updated the location log, before -checksumming. But it doesn't do anything normally -after updating the location log, and it won't update the location log -before it's satisfied it transferred the right content, so after any -checksumming. - -Possibly `git annex sync` is trying to drop the content from some -remote due to its preferred content setting no longer wanting the content -to be there. Do you have preferred content expressions configured? -(I'm doubtful about this theaory because I don't think it ever hashes the -content during a drop, but one difference between `get` and `sync` -is that sync drops..) - -Only other thing I can think that it could be is perhaps git is running -the smudge/clean filter on the content, after the transfer, and git-annex -hashes it again when run that way. Hard for me to tell from the filespray -log if that might be the case. Setting `GIT_TRACE=1` would show if this -happens. -"""]] diff --git a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_2_ccd90a2ad9cb9c896a47d62ecdfc5304._comment b/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_2_ccd90a2ad9cb9c896a47d62ecdfc5304._comment deleted file mode 100644 index 449f30f14c..0000000000 --- a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_2_ccd90a2ad9cb9c896a47d62ecdfc5304._comment +++ /dev/null @@ -1,121 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 2" - date="2021-09-25T05:47:15Z" - content=""" -This is really odd. I tried to reproduce the issue in a new git annex repo and if I remember correctly I couldn't --- even on Windows. It must be something that has crept up during the usage of my specific annex meant for -storing whole-system images/backups. Most definitely it does not display \"(checksum...)\" at any point. Nor does -running my git-annex commands while the environment variable `GIT_TRACE=1` is set show any evidence of git running -the smudge/clean filters. - -I prepared a minimal version of my repo by way of the tip [[tips/splitting_a_repository]]. It does not have any -preferred-content (or required) expressions active as far as I know. It includes only one \"small\" (9GB) backup file, -the differential backup named `9BEAE03792B9FAFB-01-01.mrimg`. This repo and its clones exhibit all the same symptoms -reported above and most notably also on Linux with a ntfs-3g mount. In fact -- after some iteration -- I wrote -a bash script to test it out with strace(1): - -[[!format sh \"\"\" -#! /usr/bin/env bash - -set -eu -set -o pipefail - -file_under_test=9BEAE03792B9FAFB-01-01.mrimg - -strace_log_file_pre=strace.git-annex--op- -strace_log_file_post=.LOG -strace_flags='-y -f -qq -tt' - -debug_log_file_pre=git-annex--op- -debug_log_file_post=.LOG - -echo '* Running/strace(1)ing `git annex get` and `git annex sync -C` while targeting the same file ($file_under_test).' -echo '* We also `git annex drop` the file before each operation.' -echo '*' -echo '* Depending on the file this might take a while :)' -echo '*' - -# op: get -## -echo '* op: get' -echo - -git annex drop ${file_under_test} - -strace -e trace=read ${strace_flags} -o ${strace_log_file_pre}get${strace_log_file_post} \ - /bin/time git annex get ${file_under_test} \ - --debug 2> ${debug_log_file_pre}get${debug_log_file_post} - -annexobj=$(git annex find --format='.git/annex/objects/${hashdirlower}${backend}-s${bytesize}--${keyname}/${backend}-s${bytesize}--${keyname}\n' ${file_under_test}) - -echo '* Number of read(2) file operation of annex object during `get` sub-command:' \ - $(fgrep \"${annexobj}\" ${strace_log_file_pre}get${strace_log_file_post} | wc -l) -echo - -# op: sync -C -## -echo '* op: sync -C' -echo - -git annex drop ${file_under_test} - -strace -e trace=read ${strace_flags} -o ${strace_log_file_pre}sync-C${strace_log_file_post} \ - /bin/time git annex sync --no-commit --no-push --no-pull -C ${file_under_test} \ - --debug 2> ${debug_log_file_pre}sync-C${debug_log_file_post} - -annexobj=$(git annex find --format='.git/annex/objects/${hashdirlower}${backend}-s${bytesize}--${keyname}/${backend}-s${bytesize}--${keyname}\n' ${file_under_test}) - -echo '* Number of read(2) file operation of annex object during `sync -C` sub-command:' \ - $(fgrep \"${annexobj}\" ${strace_log_file_pre}sync-C${strace_log_file_post} | wc -l) -echo - -echo '* Finished.' - -# end of script -\"\"\" -]] - -One such test run on Ubuntu 20.04 (git-annex version 8.20210804-gab7b5a492) shows the following: - -[[!format sh \"\"\" -jkniiv@ubuntu:/media/veracrypt1/Reflect-varmistukset-test@issue2$ _scripts/exemplify-issue.git-annex--get-vs-sync-C.sh -* Running/strace(1)ing `git annex get` and `git annex sync -C` whilst targeting the same file ($file_under_test). -* We also `git annex drop` the file before each operation. -* -* Depending on the file this might take a while :) -* -* op: get - -get 9BEAE03792B9FAFB-01-01.mrimg (from origin...) -ok -(recording state in git...) -* Number of read(2) file operation of annex object during `get` sub-command: 296845 - -* op: sync -C - -drop 9BEAE03792B9FAFB-01-01.mrimg ok -(recording state in git...) -get 9BEAE03792B9FAFB-01-01.mrimg (from origin...) -ok -(recording state in git...) -(recording state in git...) -* Number of read(2) file operation of annex object during `sync -C` sub-command: 593690 - -* Finished. -jkniiv@ubuntu:/media/veracrypt1/Reflect-varmistukset-test@issue2$ bc -bc 1.07.1 -Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006, 2008, 2012-2017 Free Software Foundation, Inc. -This is free software with ABSOLUTELY NO WARRANTY. -For details type `warranty'. -296845 * 2 -593690 -\"\"\" -]] - -So the `sync -C` operation above seems to issue exactly double the amounts of reads compared to the `get` operation. -The `get` op doesn't seems to do any reads of the annex object on Windows but I suppose on Linux what it does is -a `tailVerify` instead of passing the file through a pipe or sth. - -"""]] diff --git a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_3_c7e71722379431789f26b5e743e13d9c._comment b/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_3_c7e71722379431789f26b5e743e13d9c._comment deleted file mode 100644 index 31ddf54fc4..0000000000 --- a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_3_c7e71722379431789f26b5e743e13d9c._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 3" - date="2021-09-25T06:05:40Z" - content=""" -Joey, would you be interested in taking a look at my minimal test case if I could send you the repo somehow? -I could just forcefully `drop` the last sensitive file in the annex and add an innocuous, say 100MB random file, -into the annex instead and then zip the repo up (with 7z or fsarchiver) and send it to you (by whatever means) -if that would be fine with you, perhaps? I know this an odd corner case but I would be definitely grateful if -you could oblige this hell-bent Windows user here. :) - -"""]] diff --git a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_4_f730f3593ab5aeac34e74ec48f943949._comment b/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_4_f730f3593ab5aeac34e74ec48f943949._comment deleted file mode 100644 index ca5d6e5da1..0000000000 --- a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_4_f730f3593ab5aeac34e74ec48f943949._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2021-09-27T16:17:19Z" - content=""" -Sure, if you can produce a repository you're willing to share that can -reproduce the problem I am willing to take a look at it. Just email me a -link and instructions for how to use the repository to see the problem. -"""]] diff --git a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_5_934ee22209920f8429c0ac71002bab1e._comment b/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_5_934ee22209920f8429c0ac71002bab1e._comment deleted file mode 100644 index 21bd7c5d4c..0000000000 --- a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_5_934ee22209920f8429c0ac71002bab1e._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 5" - date="2021-09-29T04:38:08Z" - content=""" -Thanks, Joey. Email with a gdrive link and explanation sent your way. -"""]] diff --git a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_6_1611ef71978eeb61da4c42cffcff2411._comment b/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_6_1611ef71978eeb61da4c42cffcff2411._comment deleted file mode 100644 index 70bef48a7e..0000000000 --- a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_6_1611ef71978eeb61da4c42cffcff2411._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 6""" - date="2021-09-30T16:51:42Z" - content=""" -You may need to resend that email, I have not received it. I will reply to -it when I do get it. -"""]] diff --git a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_7_6700aabc3b4cb02a5b370cf4c71c6a0c._comment b/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_7_6700aabc3b4cb02a5b370cf4c71c6a0c._comment deleted file mode 100644 index b311f1cc44..0000000000 --- a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_7_6700aabc3b4cb02a5b370cf4c71c6a0c._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="Email resent from my personal domain jibun.eu" - date="2021-09-30T18:22:44Z" - content=""" -@Joey: It seems Gmail didn't like my email or something didn't like Gmail. :) In any case I resent the message from -my own domain jibun.eu (I usually don't use it cos Gmail has a better spam filter -- which has its pluses and minuses). -I hope your email address is still id ät joeyh.name? - -"""]] diff --git a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_7_a0b287e187396b7072e4d7bab3664621._comment b/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_7_a0b287e187396b7072e4d7bab3664621._comment deleted file mode 100644 index 80a7f63c1d..0000000000 --- a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_7_a0b287e187396b7072e4d7bab3664621._comment +++ /dev/null @@ -1,31 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 7""" - date="2021-09-30T18:44:53Z" - content=""" -I've received the sample repo, and can reproduce the problem using its -script. (After changing a /bin/time to not hardcode the wrong path.) - - get 9BEAE03792B9FAFB-01-01--partial.dat.age (from origin...) - ok - (recording state in git...) - * Number of read(2) file operations of annex object during `get` sub-command: 3205 - - * op: sync -C - - drop 9BEAE03792B9FAFB-01-01--partial.dat.age ok - (recording state in git...) - get 9BEAE03792B9FAFB-01-01--partial.dat.age (from origin...) - ok - (recording state in git...) - (recording state in git...) - * Number of read(2) file operations of annex object during `sync -C` sub-command: 6410 - -Should be possible to track down the extra work from here. - -(Sample repo stored in my big annex as -`git-annex-test-repos/Reflect-varmistukset-test@issue.tar`. -Mail with details of how to use it is -`1321234683.1930009.1633025607916@privateemail.com`. -Note that NTFS filesystem needs to be 1.5 gb or larger in size.) -"""]] diff --git a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_8_0b4e0acecbcc6a8736849801446a189c._comment b/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_8_0b4e0acecbcc6a8736849801446a189c._comment deleted file mode 100644 index 575b876226..0000000000 --- a/doc/bugs/__96__sync_-C__96___takes_longer_to_get_file_than___96__get__96__/comment_8_0b4e0acecbcc6a8736849801446a189c._comment +++ /dev/null @@ -1,52 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 8""" - date="2021-09-30T19:15:14Z" - content=""" -Initial look at the strace of the sync shows it's opening the -object file after it copies it from the remote repo, and reading -it a second time. - - 1654392 14:58:09.293550 openat(AT_FDCWD, "/mnt/Reflect-varmistukset-test@issue/.git/annex/objects/f19/dad/MD5E-s104883384--4187ead516ef69c540aab3d27eba27c5.dat.age/MD5E-s104883384--4187ead516ef69c540aab3d27eba27c5.dat.age", O_RDONLY|O_NOCTTY|O_NONBLOCK - ... - 1654392 14:58:13.572543 openat(AT_FDCWD, ".git/annex/objects/f19/dad/MD5E-s104883384--4187ead516ef69c540aab3d27eba27c5.dat.age/MD5E-s104883384--4187ead516ef69c540aab3d27eba27c5.dat.age", O_RDONLY|O_NOCTTY|O_NONBLOCK) = 15 - 1654392 14:58:13.573055 fstat(15, {st_mode=S_IFREG|0777, st_size=104883384, ...}) = 0 - 1654393 14:58:13.573282 <... read resumed>"\1\0\0\0\0\0\0\0", 8) = 8 - 1654392 14:58:13.573370 ioctl(15, TCGETS - 1654393 14:58:13.573471 read(4, - 1654392 14:58:13.573601 <... ioctl resumed>, 0x7ffd3d3fa9d0) = -1 EINVAL (Invalid argument) - 1654392 14:58:13.573753 read(15, "age-encryption.org/v1\n-> X25519 "..., 32752) = 32752 - 1654393 14:58:13.574238 <... read resumed>"\1\0\0\0\0\0\0\0", 8) = 8 - 1654392 14:58:13.574356 read(15, - -Certianly does look like a checksum operation. - -Just before the second open is this: - - 1654396 14:58:13.570074 pread64(21, "\n\0\0\0\0\20\0\0\17\251\0\0\0\0\0\0\0\0 - \0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096, 20480) = 4096 - 1654393 14:58:13.570353 <... read resumed>"\1\0\0\0\0\0\0\0", 8) = 8 - 1654396 14:58:13.570464 futex(0xb7af808, FUTEX_WAKE_PRIVATE, 1 - 1654393 14:58:13.570562 read(4, - 1654396 14:58:13.570727 <... futex resumed>) = 1 - 1654392 14:58:13.570834 <... futex resumed>) = 0 - 1654396 14:58:13.570948 futex(0xb7afb28, FUTEX_WAIT_PRIVATE, 0, NULL - 1654392 14:58:13.571040 futex(0xb7af810, FUTEX_WAKE_PRIVATE, 1) = 0 - 1654393 14:58:13.571239 <... read resumed>"\1\0\0\0\0\0\0\0", 8) = 8 - 1654393 14:58:13.571356 read(4, - 1654392 14:58:13.571450 stat(".git/annex/objects/f19/dad/MD5E-s104883384--4187ead516ef69c540aab3d27eba27c5.dat.age/MD5E-s104883384--4187e - ad516ef69c540aab3d27eba27c5.dat.age", {st_mode=S_IFREG|0777, st_size=104883384, ...}) = 0 - 1654392 14:58:13.571958 access(".git/annex/objects/f19/dad/MD5E-s104883384--4187ead516ef69c540aab3d27eba27c5.dat.age/MD5E-s104883384--418 - 7ead516ef69c540aab3d27eba27c5.dat.age", F_OK) = 0 - 1654393 14:58:13.572314 <... read resumed>"\1\0\0\0\0\0\0\0", 8) = 8 - 1654393 14:58:13.572425 read(4, - -That looks like a query from the keys db for the inode cache, -followed by checking stat of the object to verify it. -So maybe an inode cache verification issue, which is being -resolved by a checksum.. - -So far, I've ruled out prepSendAnnex's call to sameInodeCache. -The InodeCache of the object verifies there. The second open comes -later. -"""]] diff --git a/doc/bugs/_git-annex__58___bad_url_ssh__58____47____47__git__64__gitlab.com__58____126____47__gitlabname__47__reponame.git.mdwn b/doc/bugs/_git-annex__58___bad_url_ssh__58____47____47__git__64__gitlab.com__58____126____47__gitlabname__47__reponame.git.mdwn deleted file mode 100644 index 2045310559..0000000000 --- a/doc/bugs/_git-annex__58___bad_url_ssh__58____47____47__git__64__gitlab.com__58____126____47__gitlabname__47__reponame.git.mdwn +++ /dev/null @@ -1,36 +0,0 @@ -### Please describe the problem. -An encrypted remote is added to a working git annex repository (mind ":~/" in the remote add command). However, after that any git annex command fails. - -### What steps will reproduce the problem? - > git remote add encrypted gcrypt::ssh://git@gitlab.com:~/gitlabname/reponame.git - > git push encrypted master - gcrypt: Repository not found: ssh://git@gitlab.com:~/gitlabname/reponame.git - gcrypt: Setting up new repository - gcrypt: Remote ID is :id:abcdefghijklmnopqrst - Counting objects: 53, done. - Compressing objects: 100% (52/52), done. - Total 53 (delta 12), reused 0 (delta 0) - gcrypt: Encrypting to: --throw-keyids --default-recipient-self - gcrypt: Requesting manifest signature - ... - To gcrypt::ssh://git@gitlab.com:~/gitlabname/reponame.git - * [new branch] master -> master - > - > git annex sync - git-annex: bad url ssh://git@gitlab.com:~/gitlabname/reponame.git - -### What version of git-annex are you using? On what operating system? -5.20150419-g900e1b6 on Mac OS X - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log -- - -# End of transcript or log. -"""]] - -> Fixed in [[!commit 2aa4fab62a69bb3fb5073d571d8ec110a61a144e]] -> [[done]] --[[Joey]] diff --git a/doc/bugs/_git-annex__58___bad_url_ssh__58____47____47__git__64__gitlab.com__58____126____47__gitlabname__47__reponame.git/comment_1_60db0d6bfc71a62b3c1021527a8d2d60._comment b/doc/bugs/_git-annex__58___bad_url_ssh__58____47____47__git__64__gitlab.com__58____126____47__gitlabname__47__reponame.git/comment_1_60db0d6bfc71a62b3c1021527a8d2d60._comment deleted file mode 100644 index 4b7b0e90a7..0000000000 --- a/doc/bugs/_git-annex__58___bad_url_ssh__58____47____47__git__64__gitlab.com__58____126____47__gitlabname__47__reponame.git/comment_1_60db0d6bfc71a62b3c1021527a8d2d60._comment +++ /dev/null @@ -1,23 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2015-04-29T15:16:31Z" - content=""" -According to the git-fetch man page, the syntax to use -for this kind of url is: - - ssh://[user@]host.xz[:port]/~[user]/path/to/repo.git/ - -Your url is missing the leading slash before the `~`, and has -a : with no port specified. - - ssh://git@gitlab.com:~/gitlabname/reponame.git - -It is in fact, not a legal url. - -Now, git might accept it despite not documenting it as an accepted form, -but why wander into undefined territory when there are legal ways to write -this url that work fine? - -Does GitLab promote using these malformed urls? -"""]] diff --git a/doc/bugs/_git-annex__58___bad_url_ssh__58____47____47__git__64__gitlab.com__58____126____47__gitlabname__47__reponame.git/comment_2_49682a91147990a578929678880bd226._comment b/doc/bugs/_git-annex__58___bad_url_ssh__58____47____47__git__64__gitlab.com__58____126____47__gitlabname__47__reponame.git/comment_2_49682a91147990a578929678880bd226._comment deleted file mode 100644 index e937086445..0000000000 --- a/doc/bugs/_git-annex__58___bad_url_ssh__58____47____47__git__64__gitlab.com__58____126____47__gitlabname__47__reponame.git/comment_2_49682a91147990a578929678880bd226._comment +++ /dev/null @@ -1,26 +0,0 @@ -[[!comment format=mdwn - username="http://rfhbuk.pip.verisignlabs.com.pip.verisignlabs.com/" - nickname="rfhb" - subject="comment 2" - date="2015-04-29T15:45:34Z" - content=""" -Thank you. I was wondering about the URL as flagged above. When I specify the URL as I did, the git commands work but when I change it to ssh://git@gitlab.com:/~/gitlabname/reponame.git, git command fails with - - > git push encrypted master - gcrypt: Development version -- Repository format MAY CHANGE - gcrypt: Repository not found: ssh://git@gitlab.com:/~/gitlabname/reponame.git - gcrypt: Setting up new repository - gcrypt: Remote ID is :id:xxxxxxxxxxxxxxxxx - Counting objects: 3, done. - Total 3 (delta 0), reused 0 (delta 0) - gcrypt: Encrypting to: --throw-keyids --default-recipient-self - gcrypt: Requesting manifest signature - .... passphase entered ... - GitLab: No such project - fatal: Could not read from remote repository. - Please make sure you have the correct access rights - and the repository exists. - error: failed to push some refs to 'gcrypt::ssh://git@gitlab.com:/~/gitlabname/reponame.git' - -So perhaps I need to investigate how to get gcrypt to work with remote git(lab) repositories. Thanks. -"""]] diff --git a/doc/bugs/_git-annex__58___bad_url_ssh__58____47____47__git__64__gitlab.com__58____126____47__gitlabname__47__reponame.git/comment_3_bf0fa06596338444a3772d1865b17e26._comment b/doc/bugs/_git-annex__58___bad_url_ssh__58____47____47__git__64__gitlab.com__58____126____47__gitlabname__47__reponame.git/comment_3_bf0fa06596338444a3772d1865b17e26._comment deleted file mode 100644 index 0d7fb9c2da..0000000000 --- a/doc/bugs/_git-annex__58___bad_url_ssh__58____47____47__git__64__gitlab.com__58____126____47__gitlabname__47__reponame.git/comment_3_bf0fa06596338444a3772d1865b17e26._comment +++ /dev/null @@ -1,7 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2015-04-29T17:09:07Z" - content=""" -This works: ssh://git@gitlab.com/username/reponame.git -"""]] diff --git a/doc/bugs/adb_remote_fails_to_import_treeish.mdwn b/doc/bugs/adb_remote_fails_to_import_treeish.mdwn deleted file mode 100644 index d53bd900d9..0000000000 --- a/doc/bugs/adb_remote_fails_to_import_treeish.mdwn +++ /dev/null @@ -1,75 +0,0 @@ -### Please describe the problem. -i have an adb remote setup to import and export treeishes. Recently when I tried to import from my adb remote i was met with the follwing error - - commit - On branch master - Your branch is ahead of 'seagate5tb/master' by 14 commits. - (use "git push" to publish your local commits) - - nothing to commit, working tree clean - ok - list motoE5 find: bad arg '-printf' - See stat --help - stat: Needs 1 argument - - git-annex: Unable to list contents of motoE5: adb find failed - failed - -### What steps will reproduce the problem? -* setup an adb special remote, enabling importtreeish and exporttreeish -* use the command "git annex sync --content -remoteName- -* it should fail with the above mentioned error - -### What version of git-annex are you using? On what operating system? - -git-annex version: 10.20220504-g4e4c44ed8 - -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV - -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.30 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.11 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2 - -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* - -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external - -operating system: linux x86_64 - -supported repository versions: 8 9 10 - -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 - -local repository version: 8 - - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log -git annex sync --content motoE5 130 ↵ ──(Mon,Jul11)─┘ -commit -On branch master -Your branch is ahead of 'seagate5tb/master' by 14 commits. - (use "git push" to publish your local commits) - -nothing to commit, working tree clean -ok -list motoE5 find: bad arg '-printf' -See stat --help -stat: Needs 1 argument - -git-annex: Unable to list contents of motoE5: adb find failed -failed - -# End of transcript or log. -"""]] -after it fails, i use ctrl-c to break out of the command in fear of it messing up the current state of the tree that is exported to my android device - -also it appears that someone else is having this issue as well, but they posted it in the comment section of the adb remote tutorial page instead of filing a bug report - -http://git-annex.branchable.com/tips/android_sync_with_adb/#comment-88511cef6861a2ab4d2da8b38d52dd64 - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) -git annex has been great so far, aside from some pitfalls on some of the more advanced features, i have been using it to manage my files using all kinds of special and non special remotes very solidly for 2 years now. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/adb_remote_fails_to_import_treeish/comment_1_8012368012cf53e8552d7670b9b8315d._comment b/doc/bugs/adb_remote_fails_to_import_treeish/comment_1_8012368012cf53e8552d7670b9b8315d._comment deleted file mode 100644 index 5349e2e511..0000000000 --- a/doc/bugs/adb_remote_fails_to_import_treeish/comment_1_8012368012cf53e8552d7670b9b8315d._comment +++ /dev/null @@ -1,24 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-07-12T16:15:59Z" - content=""" -How unfortunate that Android on your phone does not support `find -printf`. -What version of Android is that? I've tested a few and they do support it. - -Reverting [[!commit a32ff6cef03ceb695af7bc6415ae7eaac06d3443]] would -certainly avoid this problem. But that commit did solve a bug in supporting -large numbers of files. - -Android's find is (probably) from toybox, and looking at its changelog, -`find -printf` was added in May 2019, version 0.8.1. So versions of Android -before 2020 or so will probably have too old a version. - -Maybe git-annex should have an option for older versions of android that -falls back to using `find -exec stat {} ;`. That would be a lot slower, -since it would run stat once per file, but it would work. Or it could -fall back to the old `find +` behavior that is limited to a certain number -of files. Or, it could consume the `find` output, segment it into groups of -files itself, and run `adb stat` on each group. Which would be somewhat -slower than using xargs, and potentially hard to get right. -"""]] diff --git a/doc/bugs/adb_remote_fails_to_import_treeish/comment_2_54867d63c04ee18c76b1d769db71bc00._comment b/doc/bugs/adb_remote_fails_to_import_treeish/comment_2_54867d63c04ee18c76b1d769db71bc00._comment deleted file mode 100644 index d532e9a53f..0000000000 --- a/doc/bugs/adb_remote_fails_to_import_treeish/comment_2_54867d63c04ee18c76b1d769db71bc00._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="jenkin.schibel@286264d9ceb79998aecff0d5d1a4ffe34f8b8421" - nickname="jenkin.schibel" - avatar="http://cdn.libravatar.org/avatar/692d82fb5c42fc86d97cc44ae0fb61ca" - subject="phone version" - date="2022-07-12T18:37:16Z" - content=""" -to answer your question about what kind of phone i have, i have a 2018 motorola MotoE5. So if what you say is true about the phone possibly being too old, this phone does fit that criteria. I think it would be beneficial to add a configuration option for the adb remote to support older android phones. -"""]] diff --git a/doc/bugs/adb_remote_fails_to_import_treeish/comment_3_5e819d0b5f3101e86f0f95a7ab725ad4._comment b/doc/bugs/adb_remote_fails_to_import_treeish/comment_3_5e819d0b5f3101e86f0f95a7ab725ad4._comment deleted file mode 100644 index 242fa0a53b..0000000000 --- a/doc/bugs/adb_remote_fails_to_import_treeish/comment_3_5e819d0b5f3101e86f0f95a7ab725ad4._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2022-07-13T19:18:13Z" - content=""" -Ok, I've added an oldandroid=yes configuration to fix this. To enable it: - - git-annex enableremote android oldandroid=yes - -(Replace "android" with whatever name you used for the adb remote.) - -I don't much like this, but it's better than git-annex being slow with new -phones, and being slow with old phones is better than failing when there are -too many files to import. I did consider trying to auto-probe it from the -error message, but that seemed likely to be too fragile. -"""]] diff --git a/doc/bugs/adb_replaces_symlinks_by_plain_text_files.mdwn b/doc/bugs/adb_replaces_symlinks_by_plain_text_files.mdwn deleted file mode 100644 index 5e663589fb..0000000000 --- a/doc/bugs/adb_replaces_symlinks_by_plain_text_files.mdwn +++ /dev/null @@ -1,30 +0,0 @@ -### Please describe the problem. -Using adb special remote import/export corrupts all symlinks in main repo (by replacing them by plain text files). - -### What steps will reproduce the problem? -Primary annex repo contains an absolute symlink to another annex repo (or whatever). -Exporting it through adb replaces it by plain text file (as android FS abstraction does not support symlinks). -Importing from adb copies that plain text file and deletes symlink already present in the repo, -treating it as a new file modified by user. - -### What version of git-annex are you using? On what operating system? -git-annex version: 8.20211118-g23ee48898 - - -### Please provide any additional information below. - -Possible solutions: - -1) if PC repo contains symlink -- continue to export it as a plain text file (less changes). -But on "import" compare its size with symlink size, and then compare content. -Don't do anything if they are the same. -Cavet: user won't be able to *intentionally* replace symlinks by plain files on remote. Too esoteric usecase to consider. - -2) resolve symlink on PC and export actual file to android (probably most preferred workflow?). -But on "import" compare its size against followed symlink file size, and then its content. -Don't do anything if they are the same. -Replace symlink by actual file if they differ. - -[[!meta title="export then import replaces non-annex symlinks by plain text files"]] - -> [[fixed|done]] by skipping syminks. --[[Joey]] diff --git a/doc/bugs/adb_replaces_symlinks_by_plain_text_files/comment_1_bcc12b77c5069769ab04e448b23b3d85._comment b/doc/bugs/adb_replaces_symlinks_by_plain_text_files/comment_1_bcc12b77c5069769ab04e448b23b3d85._comment deleted file mode 100644 index cd422c2272..0000000000 --- a/doc/bugs/adb_replaces_symlinks_by_plain_text_files/comment_1_bcc12b77c5069769ab04e448b23b3d85._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-01-03T17:27:03Z" - content=""" -This is not adb specific. No special remotes support preserving symlinks -(except for git-annex object links). Reproduced using a directory special -remote. - -Probably export/import should somehow skip symlinks. -"""]] diff --git a/doc/bugs/adb_replaces_symlinks_by_plain_text_files/comment_2_f5db61aa7efb7a916d0ffebd563cc2c7._comment b/doc/bugs/adb_replaces_symlinks_by_plain_text_files/comment_2_f5db61aa7efb7a916d0ffebd563cc2c7._comment deleted file mode 100644 index d042b41f10..0000000000 --- a/doc/bugs/adb_replaces_symlinks_by_plain_text_files/comment_2_f5db61aa7efb7a916d0ffebd563cc2c7._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="amerlyq+annex@12121d70a602f094228689a0a24d348d478a8af1" - nickname="amerlyq+annex" - avatar="http://cdn.libravatar.org/avatar/3d63c9f436b45570d45bd003e468cbd3" - subject="comment 2" - date="2022-01-04T13:10:13Z" - content=""" -Probably skipping symlinks for both import/export is for the best. - -My personal expectations (pipe dream?) are to export symlinks as real files -- but only compare during import and warn if different. - -Because if I see file/symlink in some repo -- it's placed there for a reason, and after sync I expect to find all the expected files in their folders (even if through multiple copies). -Silently ignoring of symlinks forces me to remember to check that whatever I want to sync does not contain them. -Otherwise I may be in a very delicate situation, if files needed for work were not synced to phone and I have no access to PC at that moment ;) - -On the other hand, I remember literally months I spent debugging and configuring samba, and reading their history of symlinks implementations. -It contains so much frustrating corner cases, that banning symlinks completely isn't a bad idea, actually. -"""]] diff --git a/doc/bugs/adb_replaces_symlinks_by_plain_text_files/comment_3_7346adcffb3d9056dc880dcf1e2ab03f._comment b/doc/bugs/adb_replaces_symlinks_by_plain_text_files/comment_3_7346adcffb3d9056dc880dcf1e2ab03f._comment deleted file mode 100644 index c1f930d9d1..0000000000 --- a/doc/bugs/adb_replaces_symlinks_by_plain_text_files/comment_3_7346adcffb3d9056dc880dcf1e2ab03f._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2022-01-04T15:24:33Z" - content=""" -The only special remotes that could probably even support symlinks -in theory are adb and directory, and those only sometimes. - -Supporting them would complicate the special remote API, and I feel -unncessarily so. -"""]] diff --git a/doc/bugs/add_overwrite_race.mdwn b/doc/bugs/add_overwrite_race.mdwn deleted file mode 100644 index 65f695d69f..0000000000 --- a/doc/bugs/add_overwrite_race.mdwn +++ /dev/null @@ -1,59 +0,0 @@ -I was running `git-annex add` on 5 gb of files, and accidentially overwrote -some of the first ones, which it had already processed, while it was -running. This caused binary files to get staged in git, rather than the -annex pointers. - -Test case: - - echo hi > 1 - dd if=/dev/urandom of=2 bs=1M count=1000 - (sleep 2s; rm 1; echo bye > 1) & - git-annex add - git diff --cached 1 - diff --git a/1 b/1 - new file mode 100644 - index 0000000..b023018 - --- /dev/null - +++ b/1 - @@ -0,0 +1 @@ - +bye - -This happens due to ingestAdd using addLink on the symlink, -which just queues a "git add" of the file for later. In the -meantime, the symlink is replaced with something else, so git -adds that. - -It seems that the solution will be to use update-index rather than git add. -Note that addLink has a comment about why it uses git add, which seems to mostly -be that it's faster. It also sometimes relies on git add to check gitignore, -although sometimes redundandly, some of the callers of it may rely on that -and have to be changed to check it first themselves. - -Since adding a file to the annex also involves locking it down and -detecting modifications made while generating the key, update-index is -sufficient. - -> Update: This is fixed. - -When it's adding a file unlocked, it already stages the pointer file using -update-index instead so there is no overwrite problem there. - -But, there's a similar problem when it decides not to annex a file -and adds it to git. If the file content is overwritten then, it will -git add the new content. Which may be large enough that it should have been -added to the annex after all. Test case for this: - - git config annex.largefiles largerthan=3b - echo hi > 1 - dd if=/dev/urandom of=2 bs=1M count=1000 - (sleep 2s; rm 1; echo bye > 1) & - git-annex add - git diff --cached 1 - -Guess it needs to cache the inode, -hash the file content, then verifiy the inode did not change during -hashing, and then also use update-index. - -> [[done]] - ---[[Joey]] diff --git a/doc/bugs/addurl_+_metadata_on_Windows_doesn__39__t_work.mdwn b/doc/bugs/addurl_+_metadata_on_Windows_doesn__39__t_work.mdwn deleted file mode 100644 index 944d986dbf..0000000000 --- a/doc/bugs/addurl_+_metadata_on_Windows_doesn__39__t_work.mdwn +++ /dev/null @@ -1,39 +0,0 @@ -(Sorry for the uninformative title, but I had to work within the character limit.) - -### Please describe the problem. - -`git-annex metadata` does nothing on Windows if invoked while `git-annex addurl` is in progress on other files. - -### What steps will reproduce the problem? - -On Windows (but not on Linux or macOS, where everything works fine): - -- Start `git-annex addurl` in batch mode -- Feed it two or more URLs -- After reading the completion message for a URL from addurl's stdout, but before reading the remaining output, run `git-annex metadata` in batch mode and try to set the metadata for the file that was just downloaded. -- `git-annex metadata` will output an empty line (i.e., just CR LF), and if nothing further is fed to it, it will exit successfully without printing anything else on stdout or stderr. -- Querying the file's metadata normally after `git-annex addurl` exits will show that no metadata was set for the file. - -The Python script at (Python 3.8+ required) will run the above steps and show the output from `git-annex metadata`. A sample run under GitHub Actions can be seen at ; note the following section of the output under "Run script": - -``` -16:04:04 [DEBUG ] __main__: Opening pipe to: git-annex metadata --batch --json --json-error-messages -16:04:04 [DEBUG ] __main__: Input to metadata: b'{"file": "programming/gameboy.pdf", "fields": {"title": ["GameBoy Programming Manual"]}}\n' -16:04:04 [DEBUG ] __main__: r.returncode=0 -16:04:04 [DEBUG ] __main__: r.stdout=b'\r\n' -16:04:04 [DEBUG ] __main__: r.stderr=b'' -``` - -This problem does not always occur, but it seems to occur most of the time. Using `git-annex registerurl` in place of `git-annex metadata` works fine. - -### What version of git-annex are you using? On what operating system? - -git-annex 10.20221003, provided by datalad/git-annex, on Microsoft Windows Server 2022 - -### Please provide any additional information below. - -This affects a hobby project of mine – "gamdam", implemented in [Python](https://github.com/jwodder/gamdam) and [Rust](https://github.com/jwodder/gamdam-rust) — that interacts with git-annex. - -[[!meta author=jwodder]] - -> [[fixed|done]], see my comments --[[Joey]] diff --git a/doc/bugs/addurl_+_metadata_on_Windows_doesn__39__t_work/comment_1_831dc2185919865d418b29cd06ef42be._comment b/doc/bugs/addurl_+_metadata_on_Windows_doesn__39__t_work/comment_1_831dc2185919865d418b29cd06ef42be._comment deleted file mode 100644 index c1e1d254cb..0000000000 --- a/doc/bugs/addurl_+_metadata_on_Windows_doesn__39__t_work/comment_1_831dc2185919865d418b29cd06ef42be._comment +++ /dev/null @@ -1,42 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-10-26T16:44:21Z" - content=""" -Windows is not needed, this will happen in a -repository where `git annex adjust --unlock` has been run. - -A simpler example: - - joey@darkstar:~/tmp/t2#master(unlocked)>git-annex addurl --batch - http://google.com/ - addurl http://google.com/ - (to google.com_) ok - ^Z - joey@darkstar:~/tmp/t2#master(unlocked)>git-annex metadata --batch --json - {"file":"google.com_","fields":{"author":["bar"]}} - -I'm not sure if this is a bug, because it's documented to output a blank -line when batch mode is provided a file that is not an annexed file, and -the file is not an annexed file yet due to the pointer not yet having been -staged in git. Which is needed, when in an adjusted unlocked branch, for -git-annex to know that this is an annexed file. - -When the file is locked, it just stats the symlink, so the fact that the -symlink is not yet staged in git doesn't matter. - -It does not seem to make sense to have addurl update the index -after each file it adds, because that would make addurl of a lot -of files unncessarily slow. - -So, I think if anything is changed, it would need to be a change to make -the behavior with unlocked files consistent with the behavior with locked -files. Eg, when the symlink is not yet staged in git, treat it as a -non-annexed file. Which is also consistent with other handling of such -files by git-annex when not in batch mode. - -The solution for your program, though, seems like it will be to end the -git-annex addurl process before trying to set metadata on just-added files. -Or, alternatively, to use addurl with --json, extract the key, and set the -metadata of the key. -"""]] diff --git a/doc/bugs/addurl_+_metadata_on_Windows_doesn__39__t_work/comment_2_88b7db5434a56c25c75772caa37bc14a._comment b/doc/bugs/addurl_+_metadata_on_Windows_doesn__39__t_work/comment_2_88b7db5434a56c25c75772caa37bc14a._comment deleted file mode 100644 index 80ac4fc644..0000000000 --- a/doc/bugs/addurl_+_metadata_on_Windows_doesn__39__t_work/comment_2_88b7db5434a56c25c75772caa37bc14a._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-10-26T18:13:42Z" - content=""" -I've made --batch handling of unstaged locked files consistent with the -handling of unstaged unlocked files. -"""]] diff --git a/doc/bugs/addurl_unittest_failing_under_windows.mdwn b/doc/bugs/addurl_unittest_failing_under_windows.mdwn deleted file mode 100644 index 7929112e15..0000000000 --- a/doc/bugs/addurl_unittest_failing_under_windows.mdwn +++ /dev/null @@ -1,56 +0,0 @@ -### Please describe the problem. -git annex works perfectly fine under windows now; awesome work! One of the tests in the suite still fails (which might affect some features, that I don't use (yet?)). - -### What steps will reproduce the problem? -[[!format sh """ -$ git annex test -p addurl -"""]] - -### What version of git-annex are you using? On what operating system? - -[[!format sh """ -$ git --version -git version 2.6.4.windows.1 - -$ git annex version -git-annex version: 5.20151218-g5008846 -build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3(multipartupload) WebDAV ConcurrentOutput DNS Feeds Quvi TDFA TorrentParser Database -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL -remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external -"""]] - -Windows 10 (OS Build 10586.0) - -### Please provide any additional information below. - -[[!format sh """ -$ git annex test -p addurl -Tests - Unit Tests -Init Tests - addurl: init: Detected a filesystem without fifo support. - Disabling ssh connection caching. - Detected a crippled filesystem. - Enabling direct mode. -OK (2.74s) - add: fatal: This operation must be run in a work tree -OK (6.00s) - -All 2 tests passed (8.79s) - Detected a filesystem without fifo support. - Disabling ssh connection caching. - Detected a crippled filesystem. - Enabling direct mode. -removeDirectoryRecursive: permission denied (The process cannot access the file because it is being used by another process.) -sleeping 10 seconds and will retry directory cleanup -FAIL (7.16s) - addurl failed on file:///Users/born/.t/tmprepo0/myurl - -1 out of 1 tests failed (28.13s) - (This could be due to a bug in git-annex, or an incompatibility - with utilities, such as git, installed on this system.) -"""]] - - -> This seems to have been fixed in the meantime, windows test case fully -> passes these days. [[done]] --[[Joey]] diff --git a/doc/bugs/addurl_unittest_failing_under_windows/comment_1_8f723e986d80754a8b68308ab8b15c20._comment b/doc/bugs/addurl_unittest_failing_under_windows/comment_1_8f723e986d80754a8b68308ab8b15c20._comment deleted file mode 100644 index de5b45a1d6..0000000000 --- a/doc/bugs/addurl_unittest_failing_under_windows/comment_1_8f723e986d80754a8b68308ab8b15c20._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2016-01-15T19:17:54Z" - content=""" -That test needs curl to be available. git-annex relies on Git for Windows -containing curl. - -Is the curl command available in the shell where you ran git-annex test? -"""]] diff --git a/doc/bugs/addurl_unittest_failing_under_windows/comment_2_e0bbb72af2852b2210bb1ce520e88f1d._comment b/doc/bugs/addurl_unittest_failing_under_windows/comment_2_e0bbb72af2852b2210bb1ce520e88f1d._comment deleted file mode 100644 index 8f1c3a21a7..0000000000 --- a/doc/bugs/addurl_unittest_failing_under_windows/comment_2_e0bbb72af2852b2210bb1ce520e88f1d._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="yarikoptic" - avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" - subject="comment 2" - date="2018-09-24T19:33:41Z" - content=""" -Just ran into the same situation! Happens in both v6 and direct modes. I have `curl` but not wget available in that shell (Anaconda's shell). -Here are examples: -[v6](http://www.onerussian.com/tmp/gkrellShoot_09-24-18_152906.png), [direct](http://www.onerussian.com/tmp/gkrellShoot_09-24-18_153055.png) -"""]] diff --git a/doc/bugs/adjusted__47__main__40__unlocked__41___not_syncing_with_main.mdwn b/doc/bugs/adjusted__47__main__40__unlocked__41___not_syncing_with_main.mdwn deleted file mode 100644 index 88b0db994d..0000000000 --- a/doc/bugs/adjusted__47__main__40__unlocked__41___not_syncing_with_main.mdwn +++ /dev/null @@ -1,65 +0,0 @@ -### Please describe the problem. - -I'm using Git Annex on Windows and Linux. - -On Windows - Git annex is using `adjusted/main(unlocked)` branch that is syncing to `main`. This setup worked reliably for a few weeks - -A couple days ago `adjusted/main(unlocked)` branch stopped syncing to `main`. Currently log looks like this: - -``` -commit C (HEAD -> adjusted/main(unlocked)) -Date: Wed Feb 8 07:47:49 2023 -0500 - New commit on windows - no longer syncing to main - -commit B -Date: Mon Jan 30 21:09:41 2023 -0500 - git-annex adjusted branch - -commit A (origin/synced/main, origin/main, synced/main, main) -Date: Mon Jan 30 21:09:41 2023 -0500 - Last commit before sync stopped working -``` - -I've tried (several times): - -``` -git annex sync -commit -On branch adjusted/main(unlocked) -nothing to commit, working tree clean -ok -``` - -``` -git annex sync --cleanup -cleanup local ok -``` - -So `sync` is reporting that there is nothing to do, but `main` branch is not syncing. I've tried adding a new commit to `adjusted/main(unlocked)` and the did not help with restoring sync. Not sure what else to try. - -Hoping to figure out what caused this. Help is much appreciated! - -### What steps will reproduce the problem? - -Git annex Windows repo stoped syncing. I don't know what caused this. It worked reliably for a few weeks - -### What version of git-annex are you using? On what operating system? - -git-annex version: 10.20221105-g56f824d21 -Windows 10 -local repository version: 10 - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - - -> Added a warning message in this situation. [[done]] --[[Joey]] diff --git a/doc/bugs/adjusted__47__main__40__unlocked__41___not_syncing_with_main/comment_1_e5ebc1cf8621761b559d53884731d0e3._comment b/doc/bugs/adjusted__47__main__40__unlocked__41___not_syncing_with_main/comment_1_e5ebc1cf8621761b559d53884731d0e3._comment deleted file mode 100644 index 33deefcdf6..0000000000 --- a/doc/bugs/adjusted__47__main__40__unlocked__41___not_syncing_with_main/comment_1_e5ebc1cf8621761b559d53884731d0e3._comment +++ /dev/null @@ -1,23 +0,0 @@ -[[!comment format=mdwn - username="rshalaev@3e2130a1e3cb0aaff7dd80aba7548ad9be0ea2d4" - nickname="rshalaev" - avatar="http://cdn.libravatar.org/avatar/d7f181d338cbcef7418faa01f0441e86" - subject="comment 1" - date="2023-02-09T14:22:19Z" - content=""" -Update: - -I was able to resolve the problem - believe that it was caused by the fact that I changed branch name from `master` to `main` and had leftover `refs/basis/adjusted/master(unlocked)`. - -Resolved following this steps: - -``` -git switch main -git annex merge \"adjusted/main(unlocked)\" -git branch -D \"adjusted/main(unlocked)\" -rm -rf .git/refs/basis/* -git annex adjust --unlocked -``` - -Tested that sync is now working -"""]] diff --git a/doc/bugs/adjusted__47__main__40__unlocked__41___not_syncing_with_main/comment_2_149502c4d8fa2c50507e7e6e312f9703._comment b/doc/bugs/adjusted__47__main__40__unlocked__41___not_syncing_with_main/comment_2_149502c4d8fa2c50507e7e6e312f9703._comment deleted file mode 100644 index 4c3f9e3866..0000000000 --- a/doc/bugs/adjusted__47__main__40__unlocked__41___not_syncing_with_main/comment_2_149502c4d8fa2c50507e7e6e312f9703._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2023-02-10T18:07:47Z" - content=""" -For an adjusted branch, git-annex keeps -a hidden ref, `refs/basis/adjusted/master(unlocked)`. -That ref also needs to be renamed if you're going to -rename the adjusted branch and the original branch. - -It's of course fine to delete the old adjusted branch and -re-adjust the new branch. - -I think that git-annex should warn if it cannot find -the basis ref for an adjusted branch. -"""]] diff --git a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified.mdwn b/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified.mdwn deleted file mode 100644 index f5e51c397c..0000000000 --- a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified.mdwn +++ /dev/null @@ -1,57 +0,0 @@ -### Please describe the problem. - -When checking an adjusted (unlocked) branch, after a while (a few hours), without modifying anything but running several times "git status", a lot of files are reported as "modified" by "git status". - -### What steps will reproduce the problem? - -On a ~1.5To repo with 1Go to 4Go files, running "git adjust --unlock" works, but after a while "git status" reports a lot of files being modified, "git restore myfiles" restores the files, but other files are reported by "git status" as modified... - -### What version of git-annex are you using? On what operating system? - -git-annex version: 8.20201103, Debian sid - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log -$ cat .git/config -[core] - repositoryformatversion = 0 - filemode = true - bare = false - logallrefupdates = true -[annex] - uuid = 56c81321-3ca1-4464-8c42-b4b405a2502d - version = 8 - thin = true - autocommit = false -[receive] - denyCurrentBranch = updateInstead -[filter "annex"] - smudge = git-annex smudge %f - clean = git-annex smudge --clean %f -$ git status -Refresh index: 100% (4703/4703), done. -On branch adjusted/master(unlocked) -Changes not staged for commit: - (use "git add ..." to update what will be committed) - (use "git restore ..." to discard changes in working directory) - modified: file1.avi - modified: file2.avi - ... - - -It took 3.88 seconds to enumerate untracked files. 'status -uno' -may speed it up, but you have to be careful not to forget to add -new files yourself (see 'git help status'). -no changes added to commit (use "git add" and/or "git commit -a") - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -It's almost perfect :-). - -> [[notabug|done]] --[[Joey]] diff --git a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_10_8a355d509fb2005cfd9da8f2d4bb560e._comment b/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_10_8a355d509fb2005cfd9da8f2d4bb560e._comment deleted file mode 100644 index 96a850759b..0000000000 --- a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_10_8a355d509fb2005cfd9da8f2d4bb560e._comment +++ /dev/null @@ -1,23 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 10""" - date="2022-06-02T15:47:14Z" - content=""" -Well, I can confirm that, with .git/info/attributes missing, `git status` -displays unlocked files as modified. - -But, the bug originally reported here had the files only start to show up -as modified after some amount of time, not immediately. And also, it had -`git diff --cached` outputting nothing when run on those files. The -behavior with .git/info/attributes does not match either of those. So I -don't think yours is the same problem as the original bug report. Which -it's pretty clear involved sshfs. - -As to how .git/info/attributes could be missing: -It is created by `git-annex init`. Upgrading a repository from v5 or older -will also create it. So I don't see how it could normally be missing. At the -moment the most likely reason would be if you had deleted it earlier.. - -(One other way it can be missing is if annex.supportunlocked is set to -false.) -"""]] diff --git a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_11_ee707efce9c5537a1f7d0ffded952af8._comment b/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_11_ee707efce9c5537a1f7d0ffded952af8._comment deleted file mode 100644 index 686481aad0..0000000000 --- a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_11_ee707efce9c5537a1f7d0ffded952af8._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 11""" - date="2023-06-21T18:38:29Z" - content=""" -Oh, I think oliv5 and guuex were the same person.. - -If you had .git/info/attributes missing all along, that nicely explains it -all. - -I think this can be closed then. -"""]] diff --git a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_1_0fcdba2f621c8c32f09b2633c11266ee._comment b/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_1_0fcdba2f621c8c32f09b2633c11266ee._comment deleted file mode 100644 index 1e649716df..0000000000 --- a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_1_0fcdba2f621c8c32f09b2633c11266ee._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Lukey" - avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b" - subject="comment 1" - date="2020-11-23T19:06:13Z" - content=""" -Did you change the [[backend|https://git-annex.branchable.com/backends/]] of some files via `.gitattributes` in the meantime? I think git annex's smudge/clean filter will then return the hash of the new backend and that differs from the one of the old backend, so git reports the files as changed. -"""]] diff --git a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_2_4719510b78d347f28a0fe8910c086982._comment b/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_2_4719510b78d347f28a0fe8910c086982._comment deleted file mode 100644 index aea1696764..0000000000 --- a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_2_4719510b78d347f28a0fe8910c086982._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="gueux" - avatar="http://cdn.libravatar.org/avatar/47e44a21505727b2d6bb5d88f0468f34" - subject="comment 2" - date="2020-11-24T21:07:37Z" - content=""" -You mean between the \"git annex adjust\" and the \"git status\"? No, I did nothing at all :(. - -After running \"git restore\" some files (but not all, it seems to be the first N files ordered alphabetically) are still marked as modified, some others get modified again after another \"git restore\"... Weird! -"""]] diff --git a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_3_05f178a19b8949efc5c6e189ff8fa5e0._comment b/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_3_05f178a19b8949efc5c6e189ff8fa5e0._comment deleted file mode 100644 index d37c419fe4..0000000000 --- a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_3_05f178a19b8949efc5c6e189ff8fa5e0._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2021-01-29T16:52:47Z" - content=""" -I've never seen this behavior, but it seems that if git thinks the files -are modified, either `git diff` or `git diff --cached` must show some kind -of change, and seeing what that change might be would be a useful clue to -what happened. -"""]] diff --git a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_4_ad4621b602e7632d74d58f1728b80cda._comment b/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_4_ad4621b602e7632d74d58f1728b80cda._comment deleted file mode 100644 index 250b1f55a6..0000000000 --- a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_4_ad4621b602e7632d74d58f1728b80cda._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="gueux" - avatar="http://cdn.libravatar.org/avatar/47e44a21505727b2d6bb5d88f0468f34" - subject="comment 4" - date="2021-02-01T18:11:40Z" - content=""" -I'd say around 50% of the files are reported as modified by \"git status\". - -`git --no-pager diff --cached --output git-annex-cached-diff.diff` returns -instantly, with no output and nothing in `git-annex-cached-diff.diff`. -`git --no-pager diff --output git-annex-diff.diff` ran for around 5 hours, with no output and nothing in `git-annex-diff.diff`. -"""]] diff --git a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_5_fcb259337b70b0d5c5e50c8facb845e4._comment b/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_5_fcb259337b70b0d5c5e50c8facb845e4._comment deleted file mode 100644 index 34a1d99ee4..0000000000 --- a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_5_fcb259337b70b0d5c5e50c8facb845e4._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="gueux" - avatar="http://cdn.libravatar.org/avatar/47e44a21505727b2d6bb5d88f0468f34" - subject="comment 5" - date="2021-02-01T18:33:48Z" - content=""" -The repo is in a dir mounted with sshfs. -"""]] diff --git a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_6_1df979b8e222dd135a10046c51f0dc0b._comment b/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_6_1df979b8e222dd135a10046c51f0dc0b._comment deleted file mode 100644 index d055b0dae7..0000000000 --- a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_6_1df979b8e222dd135a10046c51f0dc0b._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 6""" - date="2021-02-02T16:28:29Z" - content=""" -Being on sshfs makes me wonder if perhaps something synthesized by that -filesystem is confusing git, so eg it thinks the inode or mtime of a file has -changed and so it's modified. - -However, normally git status does the same work to verify if there's really -a modification as git diff does (running the smudge filter in this case). -So I'm not clear what's going on here, but it's currently looking more like -a problem at the git level, or a filesystem that is too wonky for git to -work well, rather than anything git-annex can do. - -I have to wonder why you'd even use git-annex on sshfs. The more usual way -would be to have a local repository with a remote using ssh to access the -server. -"""]] diff --git a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_7_5e3482024e20759f8d42e14598d4dc26._comment b/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_7_5e3482024e20759f8d42e14598d4dc26._comment deleted file mode 100644 index a97a34b1e0..0000000000 --- a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_7_5e3482024e20759f8d42e14598d4dc26._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="gueux" - avatar="http://cdn.libravatar.org/avatar/47e44a21505727b2d6bb5d88f0468f34" - subject="comment 7" - date="2021-02-03T17:06:24Z" - content=""" -I have a local repo (without most of the files) with Debian sid and a remote using ssh, on a server with Debian stable. The files are big, so it is easier to access them with sshfs, to avoid copying them. I wanted to use adjusted branches for users who access the files with gnome/nautilus (via sshfs integration) which is not clever with copying symlinks. So, I upgraded the repo from my laptop (repo mounted with sshfs). - -Note that this problem only occurs with adjusted branched, I also have other repos accessed and upgraded via sshfs (sometimes because I just ran \"git status\" on a repo mounted with sshfs, which resulted in an upgrade (I think)), and there is no file reported as modified (git status is also slow, though). -"""]] diff --git a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_8_4e9f342281b01ecb9354a08200c04ab0._comment b/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_8_4e9f342281b01ecb9354a08200c04ab0._comment deleted file mode 100644 index 628793ae32..0000000000 --- a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_8_4e9f342281b01ecb9354a08200c04ab0._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="gueux" - avatar="http://cdn.libravatar.org/avatar/47e44a21505727b2d6bb5d88f0468f34" - subject="comment 8" - date="2021-02-03T17:20:24Z" - content=""" -I've just seen it happen on a repo without adjusted branches, sorry! So, it does not seem to be related with them. -"""]] diff --git a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_9_63af8f32512ce8ec07c48361317ce51d._comment b/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_9_63af8f32512ce8ec07c48361317ce51d._comment deleted file mode 100644 index 060eafb9e5..0000000000 --- a/doc/bugs/after_adjust_--unlock_implies__44___a_lot_of_files_are_reported_as_modified/comment_9_63af8f32512ce8ec07c48361317ce51d._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="oliv5" - avatar="http://cdn.libravatar.org/avatar/d7f0d33c51583bbd8578e4f1f9f8cf4b" - subject="comment 9" - date="2022-06-01T11:16:35Z" - content=""" -One note for people with the same issue: on my side, the file .git/info/attributes was missing. - -I created it with the following content, taken from \"git-annex help smudge\" manpage: - - * filter=annex - .* !filter - -And the \"git status\" issue vanished... -"""]] diff --git a/doc/bugs/annex_import_does_not_account_for_versioning_on_S3.mdwn b/doc/bugs/annex_import_does_not_account_for_versioning_on_S3.mdwn deleted file mode 100644 index 4f6229e34c..0000000000 --- a/doc/bugs/annex_import_does_not_account_for_versioning_on_S3.mdwn +++ /dev/null @@ -1,16 +0,0 @@ -### Please describe the problem. -Using the S3 special remote with version support enabled (`versioning=yes`) leads to a bit of a strange effect with multiple `annex import` calls. -First import from an existing bucket is fine. Now, if there were changes done to a file in the bucket and one runs `annex import` again, git-annex will record that the old version of said file is gone from the remote, requiring a `git annex fsck` to be called right after that import to fix it again. That seems a bit strange given that this versioning support comes with a "native" special remote. - -I suppose that's probably a more general issue with how import/export works, since there's no way for a special remote to communicate to annex whether two different versions of the same file (same remote path, but different key) would overwrite each other. Neither an importtree nor an exporttree remote has a way to tell annex whether and how the availability of a previous key associated with the same remote path was affected. - - -### What version of git-annex are you using? On what operating system? -Observed with several versions from 8-10 on linux. As laid out above, I strongly suspect this is true for all versions. - - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Lots. I love git-annex. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/annex_import_does_not_account_for_versioning_on_S3/comment_1_a04fbb936785456ed99512ea4c29fd53._comment b/doc/bugs/annex_import_does_not_account_for_versioning_on_S3/comment_1_a04fbb936785456ed99512ea4c29fd53._comment deleted file mode 100644 index c3d8aa057b..0000000000 --- a/doc/bugs/annex_import_does_not_account_for_versioning_on_S3/comment_1_a04fbb936785456ed99512ea4c29fd53._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-10-11T16:28:39Z" - content=""" -This looks like a simple fix. After importing from a versioned remote, -it can just skip updating the location logs to remove the keys that are not -present in the current tree. The same as is already done when exporting -to a versioned remote. I've made that change. -"""]] diff --git a/doc/bugs/annex_import_does_not_account_for_versioning_on_S3/comment_2_6015c5488e69307424a3c0281b50a49e._comment b/doc/bugs/annex_import_does_not_account_for_versioning_on_S3/comment_2_6015c5488e69307424a3c0281b50a49e._comment deleted file mode 100644 index 134394ae66..0000000000 --- a/doc/bugs/annex_import_does_not_account_for_versioning_on_S3/comment_2_6015c5488e69307424a3c0281b50a49e._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="benjamin.poldrack@d09ccff6d42dd20277610b59867cf7462927b8e3" - nickname="benjamin.poldrack" - avatar="http://cdn.libravatar.org/avatar/5c1a901caa7c2cfeeb7e17e786c5230d" - subject="comment 2" - date="2022-10-12T06:12:17Z" - content=""" -Thank you! -"""]] diff --git a/doc/bugs/assistant_--autostop_stops_after_nonexistant_repo.mdwn b/doc/bugs/assistant_--autostop_stops_after_nonexistant_repo.mdwn deleted file mode 100644 index 712a8057cf..0000000000 --- a/doc/bugs/assistant_--autostop_stops_after_nonexistant_repo.mdwn +++ /dev/null @@ -1,25 +0,0 @@ -### Please describe the problem. - -`git annex assistant --autostop` stops shutting down processes after encountering the first non-existant repo listed in `.config/git-annex/autostart`, leaving other processes running. - -### What steps will reproduce the problem? - -- add several existing repos to `.config/git-annex/autostart` -- somewhere in the middle, add a non-existant path -- run `git annex assistant --autostart` (this just skips the nonexistant repo interestingly) -- run `git annex assistant --autostop` (this fails on the nonexistant repo and doesn't continue with the others) - -### What version of git-annex are you using? On what operating system? - -``` -# on Linux -❯ git annex version -git-annex version: 10.20230329-ge6a337e8e -``` - - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -I love it! 🤩 - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/assistant_committer_crashes_due_to_missing_file.mdwn b/doc/bugs/assistant_committer_crashes_due_to_missing_file.mdwn deleted file mode 100644 index c32d333ce3..0000000000 --- a/doc/bugs/assistant_committer_crashes_due_to_missing_file.mdwn +++ /dev/null @@ -1,71 +0,0 @@ -### Please describe the problem. - -The git annex assistant committer stops working and crashes under certain circumstances when dealing with small files. - -This seems to occur when a small file is in the committer queue but is deleted before the queue is flushed. - -### What steps will reproduce the problem? - -Set the `annex.largefiles` option to exclude certain files from the annex: `git config annex.largefiles 'exclude=*.txt'`. - -Create and immediately delete some file that is excluded from the annex: - -[[!format sh """ -touch somefile.txt && rm somefile.txt -"""]] - -This will cause a "user error" to occur every time the committer thread tries to do a commit, essentially causing it to stop working. - -In addition, if no file activity occurs for 5 minutes, creating another file will cause the committer to crash: - -[[!format sh """ -sleep 310 -touch someotherfile.txt -"""]] - -### What version of git-annex are you using? On what operating system? - -Operating system: Arch Linux x86_64 - -git-annex version: 10.20230329 - -### Please provide any additional information below. - -[[!format sh """ -# .git/annex/daemon.log - -[2023-06-14 17:32:42.652269404] (Watcher) add somefile.txt -[2023-06-14 17:32:42.652761285] (Watcher) add somefile.txt -[2023-06-14 17:32:42.652832039] (Watcher) file deleted somefile.txt -[2023-06-14 17:32:42.663222161] (Utility.Process) process [1178267] read: lsof ["-F0can","+d",".git/annex/watchtmp/"] -[2023-06-14 17:32:42.830892116] (Utility.Process) process [1178267] done ExitFailure 1 -[2023-06-14 17:32:42.831219136] (Utility.Process) process [1178269] feed: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","update-index","-z","--index-info"] -[2023-06-14 17:32:42.832078397] (Utility.Process) process [1178269] done ExitSuccess -[2023-06-14 17:32:42.832140832] (Committer) committing 2 changes -(recording state in git...) -[2023-06-14 17:32:42.832442128] (Utility.Process) process [1178270] feed: xargs ["-0","git","--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","add","--force","--"] -fatal: pathspec 'somefile.txt' did not match any files -[2023-06-14 17:32:42.833714128] (Utility.Process) process [1178270] done ExitFailure 123 -[2023-06-14 17:33:10.166497909] (Utility.Process) process [1178219] done ExitSuccess -[2023-06-14 17:33:10.167361436] (Utility.Process) process [1178207] done ExitSuccess -[2023-06-14 17:33:10.168264148] (Utility.Process) process [1178218] done ExitSuccess -[2023-06-14 17:33:10.16909154] (Utility.Process) process [1178238] done ExitSuccess -[2023-06-14 17:33:10.169892592] (Utility.Process) process [1178240] done ExitSuccess -[2023-06-14 17:33:10.170694357] (Utility.Process) process [1178227] done ExitFailure 1 -[2023-06-14 17:33:40.185689237] (Utility.Process) process [1178330] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","ls-tree","--full-tree","-z","--","refs/heads/git-annex","uuid.log","remote.log","trust.log","group.log","numcopies.log","mincopies.log","schedule.log","preferred-content.log","required-content.log","group-preferred-content.log"] -[2023-06-14 17:33:40.189177926] (Utility.Process) process [1178330] done ExitSuccess -[2023-06-14 17:37:52.65735194] (Utility.Process) process [1178648] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -[2023-06-14 17:37:52.660710625] (Utility.Process) process [1178649] chat: git ["--git-dir=.git","--work-tree=.","-c","annex.debug=true","check-ignore","-z","--stdin","--verbose","--non-matching"] -[2023-06-14 17:37:52.663100743] (Watcher) add someotherfile.txt -[2023-06-14 17:37:52.663984987] (Watcher) add someotherfile.txt -[2023-06-14 17:37:52.674996448] (Utility.Process) process [1178650] read: lsof ["-F0can","+d",".git/annex/watchtmp/"] -[2023-06-14 17:37:52.825736165] (Utility.Process) process [1178650] done ExitFailure 1 -[2023-06-14 17:37:52.826111004] (Utility.Process) process [1178652] feed: xargs ["-0","git","--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","add","--force","--"] -fatal: pathspec 'somefile.txt' did not match any files -[2023-06-14 17:37:52.82750231] (Utility.Process) process [1178652] done ExitFailure 123 -Committer crashed: user error (xargs ["-0","git","--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","add","--force","--"] exited 123) - -# End of transcript or log. -"""]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/backend_option_for_init_is_in_no_effect__63__.mdwn b/doc/bugs/backend_option_for_init_is_in_no_effect__63__.mdwn deleted file mode 100644 index 7f69f9f678..0000000000 --- a/doc/bugs/backend_option_for_init_is_in_no_effect__63__.mdwn +++ /dev/null @@ -1,77 +0,0 @@ -### Please describe the problem. - -`annex init --help` talks about supporting `--backend` so I expected either modification of .gitattributes to add backend, or at least non-persistent setting in .git/config ... but seems nothing is actually done? - - -### What version of git-annex are you using? On what operating system? - -6.20170525+gitge1cf095ae-1~ndall+1 - -### Please provide any additional information below. - -[[!format sh """ - -$> git annex init --backend=MD5E -init ok -(recording state in git...) -(dev) 2 27354 [1].....................................:Wed 12 Jul 2017 07:08:21 PM EDT:. -(git)hopa:/tmp/1233[master] -$> cat .git/config -[core] - repositoryformatversion = 0 - filemode = true - bare = false - logallrefupdates = true -[annex] - uuid = 8a045a11-af4a-43b6-922f-cf402efab619 - version = 5 -(dev) 2 27355 [1].....................................:Wed 12 Jul 2017 07:08:26 PM EDT:. -(git)hopa:/tmp/1233[master] -$> git annex init --help -git-annex init - initialize git-annex - -Usage: git-annex init [DESC] [--version VALUE] - -Available options: - --version VALUE Override default annex.version - --force allow actions that may lose annexed data - -F,--fast avoid slow operations - -q,--quiet avoid verbose output - -v,--verbose allow verbose output (default) - -d,--debug show debug messages - --no-debug don't show debug messages - -b,--backend NAME specify key-value backend to use - -N,--numcopies NUMBER override default number of copies - --trust REMOTE override trust setting - --semitrust REMOTE override trust setting back to default - --untrust REMOTE override trust setting to untrusted - -c,--config NAME=VALUE override git configuration setting - --user-agent NAME override default User-Agent - --trust-glacier Trust Amazon Glacier inventory - --notify-finish show desktop notification after transfer finishes - --notify-start show desktop notification after transfer starts - -h,--help Show this help text - -For details, run: git-annex help init -(dev) 2 27356 [1].....................................:Wed 12 Jul 2017 07:08:31 PM EDT:. -(git)hopa:/tmp/1233[master] -$> ls -(dev) 2 27357 [1].....................................:Wed 12 Jul 2017 07:08:39 PM EDT:. -(git)hopa:/tmp/1233[master] -$> ls -lta -total 312 -drwx------ 9 yoh yoh 4096 Jul 12 19:08 .git/ -drwx------ 3 yoh yoh 4096 Jul 12 19:08 ./ -drwxrwxrwt 143 root root 307200 Jul 12 19:08 ../ -(dev) 2 27358 [1].....................................:Wed 12 Jul 2017 07:08:40 PM EDT:. -(git)hopa:/tmp/1233[master] -$> grep -r MD5E . - - -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -kinda - -> [[fixed|done]], --backend is no longer a global option. --[[Joey]] diff --git a/doc/bugs/backend_option_for_init_is_in_no_effect__63__/comment_1_6b40b5ea277d8e28f0b632701a776587._comment b/doc/bugs/backend_option_for_init_is_in_no_effect__63__/comment_1_6b40b5ea277d8e28f0b632701a776587._comment deleted file mode 100644 index f024c46937..0000000000 --- a/doc/bugs/backend_option_for_init_is_in_no_effect__63__/comment_1_6b40b5ea277d8e28f0b632701a776587._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2017-08-15T16:54:56Z" - content=""" ---backend is a global option, so it appears in every command's help. - -There are several of these global options left over from the previous -option parsing regime. -"""]] diff --git a/doc/bugs/bare_remote_safe_directory.mdwn b/doc/bugs/bare_remote_safe_directory.mdwn deleted file mode 100644 index b05c490ec4..0000000000 --- a/doc/bugs/bare_remote_safe_directory.mdwn +++ /dev/null @@ -1,15 +0,0 @@ - git init foo.git - cd foo.git - git-annex init - cd .. - git clone foo.git bar - sudo chown root.root foo.git - cd bar - git-annex init - init fatal: not a git repository: /tmp/foo.git/.git' - -This is specific to bare git remotes, for non-bare it -detects and warns that safe.directory is needed to use the -remote. --[[Joey]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/batch_commands_miss_journalled_changes_made_while_running.mdwn b/doc/bugs/batch_commands_miss_journalled_changes_made_while_running.mdwn deleted file mode 100644 index 09fc596841..0000000000 --- a/doc/bugs/batch_commands_miss_journalled_changes_made_while_running.mdwn +++ /dev/null @@ -1,12 +0,0 @@ ---batch commands are supposed to notice changes that get made to the -journal while they are running. This way, if two batch commands are run, -one making changes and the other reading the values, the reader notices the -changes. - -That does not currently happen. Looks like runUpdateOnce runs before -enableInteractiveBranchAccess, and so sets journalIgnorable. Only if the -batch command writes to the journal does journalIgnorable get unset, and -then the command will notice journalled changes by itself or others. -But query commands - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/build_for_OpenBSD_is_outdated.mdwn b/doc/bugs/build_for_OpenBSD_is_outdated.mdwn deleted file mode 100644 index c435ba531c..0000000000 --- a/doc/bugs/build_for_OpenBSD_is_outdated.mdwn +++ /dev/null @@ -1,16 +0,0 @@ -### Please describe the problem. -The version of git-annex available on OpenBSD is quite old. -I've tried building a more recent version with cabal install git-annex, but this also failed. I guess these tools are also rather old. -Unfortunately I have zero experience with Haskell and only mediocre OpenBSD skills. -I'm hoping someone can pick this up and build a more up-to-date version. - -- git-annex-8.20210223p1 -- cabal-install-3.4.0.0p2 -- ghc-8.10.6p0 - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) -I've used it extensively during my time at university to keep track of materials and assignments. It ever let me down. -Unfortunately I later lost the data due to multiple corrupted external disks and backup drives with LUKS where I didn't remember the password. Luckily I had no use for it anymore anyway. -I'm hoping to use it again, this time in an effort to disconnect. - -> [[done]] --[[Joey]] diff --git a/doc/bugs/build_for_OpenBSD_is_outdated/comment_1_59d9f99747cd3ba93a87ba634824d4ca._comment b/doc/bugs/build_for_OpenBSD_is_outdated/comment_1_59d9f99747cd3ba93a87ba634824d4ca._comment deleted file mode 100644 index f0d1abb5df..0000000000 --- a/doc/bugs/build_for_OpenBSD_is_outdated/comment_1_59d9f99747cd3ba93a87ba634824d4ca._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-03-21T17:26:23Z" - content=""" -You will need to get in touch with the OpenBSD maintainers to get them to -update the package. Nothing that filing a bug here can do about it. - -Unless there is some problem with the git-annex code that is preventing -them from building a newer version of course. - -ghc 8.10 is not old at all, it's a commonly used version, and the version of -cabal is also recent enough. git-annex should build with those versions. -"""]] diff --git a/doc/bugs/build_for_OpenBSD_is_outdated/comment_2_cb4f3d710b0c46b27bebed48bd2e9a21._comment b/doc/bugs/build_for_OpenBSD_is_outdated/comment_2_cb4f3d710b0c46b27bebed48bd2e9a21._comment deleted file mode 100644 index 6a510d77a2..0000000000 --- a/doc/bugs/build_for_OpenBSD_is_outdated/comment_2_cb4f3d710b0c46b27bebed48bd2e9a21._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="gnezdo" - avatar="http://cdn.libravatar.org/avatar/4176f99bc203b65ebe09a539a3b9c6bb" - subject="Fixed" - date="2022-04-23T21:42:42Z" - content=""" -This has been [fixed in -current on Apr 13](https://github.com/openbsd/ports/commit/a9f826b37e077cd0004f4ef4e7e0ddaa18ffdc4a). -"""]] diff --git a/doc/bugs/build_for_OpenBSD_is_outdated/comment_3_99e6c86785c91b8320ec1e7df1e02e2c._comment b/doc/bugs/build_for_OpenBSD_is_outdated/comment_3_99e6c86785c91b8320ec1e7df1e02e2c._comment deleted file mode 100644 index fd5865bae5..0000000000 --- a/doc/bugs/build_for_OpenBSD_is_outdated/comment_3_99e6c86785c91b8320ec1e7df1e02e2c._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2022-05-02T18:25:12Z" - content=""" -This is not the place to file bug reports about a distribution of git-annex -by eg OpenBSD or a Linux distribution being out of date. In the future, -file the bug with the distributor, which is where the changes need to -happen to update it. - -The exception, of course, is if something about git-annex is preventing the -distributor from upgrading it. -"""]] diff --git a/doc/bugs/bup_often_errors_out_when_-J___62___1.mdwn b/doc/bugs/bup_often_errors_out_when_-J___62___1.mdwn deleted file mode 100644 index f98a123b72..0000000000 --- a/doc/bugs/bup_often_errors_out_when_-J___62___1.mdwn +++ /dev/null @@ -1,44 +0,0 @@ -### Please describe the problem. - -When copying to a bup remote with -J many copies "fail": - -``` -copy samba-2.2.3a/packaging/LSB/lsb-samba.spec (to bup...) - user error (bup ["split","-r","/tmp/silesia_bup","-q","-n","SHA256E-s2260--1e95bcc60c9b332608774459570b80ab9576b5e971013a30ff6da6d93ebcfd9d.spec"] exited 1) -``` - -They actually seem to complete in some sense though as trying to copy them again will print `ok` instantly. - -### What steps will reproduce the problem? - - git annex copy --to bup -J2 - -### What version of git-annex are you using? On what operating system? - -``` -git-annex version: 10.20220624 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.11 persistent-sqlite-2.13.1.0 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 8 -``` - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -It's amazing and I use it every day :) - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/bup_often_errors_out_when_-J___62___1/comment_1_1731cfb2662890299ca6daeacdd4bb47._comment b/doc/bugs/bup_often_errors_out_when_-J___62___1/comment_1_1731cfb2662890299ca6daeacdd4bb47._comment deleted file mode 100644 index 6659b61e8d..0000000000 --- a/doc/bugs/bup_often_errors_out_when_-J___62___1/comment_1_1731cfb2662890299ca6daeacdd4bb47._comment +++ /dev/null @@ -1,22 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-08-05T15:57:26Z" - content=""" -This seems most likely to be a concurrency issue in bup itself, or in how -it uses git. - -However, I have not been able to reproduce it so far. I tried making 8 -different files each 100 mb, and was able to copy them all to a bup repository -concurrently with no failures. - -That was with bup 0.32. What is your bup version? - -It would be good to know what error message bup outputs. -Unfortunately, since `bup split -q` is noisy on stderr, git-annex -sinks it to /dev/null and so you don't see it. - -I've changed git-annex to display the stderr when `bup split` fails. -So if you can get an updated build of git-annex, and reproduce this problem, -we'll be able to see if this is a bug in bup, or what. -"""]] diff --git a/doc/bugs/bup_often_errors_out_when_-J___62___1/comment_2_f0f87b7a2c60afba609072566b0048ac._comment b/doc/bugs/bup_often_errors_out_when_-J___62___1/comment_2_f0f87b7a2c60afba609072566b0048ac._comment deleted file mode 100644 index 1b5e98c3ba..0000000000 --- a/doc/bugs/bup_often_errors_out_when_-J___62___1/comment_2_f0f87b7a2c60afba609072566b0048ac._comment +++ /dev/null @@ -1,190 +0,0 @@ -[[!comment format=mdwn - username="Atemu" - avatar="http://cdn.libravatar.org/avatar/d1f0f4275931c552403f4c6707bead7a" - subject="comment 2" - date="2022-08-07T08:05:23Z" - content=""" -Thanks for looking into this! - -> However, I have not been able to reproduce it so far. I tried making 8 different files each 100 mb, and was able to copy them all to a bup repository concurrently with no failures. - -You need more than that; 8 Files might randomly work without error. - -Perhaps it only really manifests itself on smaller files? That's mainly what I've been copying. - -Here's an example output of git-annex 6d762fafae45cb1f6a6106ac25f347631d6d0ca8 `git-annex copy --not --in bup --to bup -J2 --all`: - -``` -... -copy SHA256E-s227509--23bcff8394a16cb8ae090ac9f67e7cccb8a2c4354788b11426bfc14d01c136c6.csv (to bup...) - bup [\"split\",\"-r\",\"/tmp/bup\",\"-q\",\"-n\",\"SHA256E-s227509--23bcff8394a16cb8ae090ac9f67e7cccb8a2c4354788b11426bfc14d01c136c6.csv\"] exited 1 (stderr output: Traceback (most recent call last): - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup-midx\", line 298, in - do_midx_dir(path, opt.output, byte_stream(sys.stdout)) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup-midx\", line 192, in do_midx_dir - m = git.open_idx(mname) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/git.py\", line 668, in open_idx - return midx.PackMidx(filename) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/midx.py\", line 29, in __init__ - self.map = mmap_read(open(filename)) - FileNotFoundError: [Errno 2] No such file or directory: b'/tmp/bup/index-cache/None__tmp_bup/midx-5bfad00d8c17012ca48231b22dd894649c385b15.midx' - [b'/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup', b'midx', b'--auto', b'--dir', b'/tmp/bup/index-cache/None__tmp_bup']: returned 1 - WARNING: 1 errors encountered while saving. - ) -ok -copy SHA256E-s6061--b57b1e42ee05102f095d700c9bcb12b48142a9c57879573d3fd128d181b60f5f.txt.user (to bup...) ok -copy SHA256E-s1861--86988a46008289c58943f2e59f4e543fdec3fc9853ce8b53375e11f9ccbd4138.cpp (to bup...) - bup [\"split\",\"-r\",\"/tmp/bup\",\"-q\",\"-n\",\"SHA256E-s1861--86988a46008289c58943f2e59f4e543fdec3fc9853ce8b53375e11f9ccbd4138.cpp\"] exited 1 (stderr output: Traceback (most recent call last): - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup-server\", line 320, in - cmd(conn, rest) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup-server\", line 137, in receive_objects_v2 - oldpack = w.exists(shar, want_source=True) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/git.py\", line 804, in exists - self._require_objcache() - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/git.py\", line 797, in _require_objcache - self.objcache = self.objcache_maker() - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/git.py\", line 686, in _make_objcache - return PackIdxList(repo(b'objects/pack')) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/git.py\", line 517, in __init__ - self.refresh() - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/git.py\", line 607, in refresh - midxl.sort(key=lambda ix: - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/git.py\", line 608, in - (-len(ix), -xstat.stat(ix.name).st_mtime)) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/xstat.py\", line 131, in stat - return stat_result.from_xstat_rep(_helpers.stat(path)) - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/bup/objects/pack/midx-13188bb9c81006e5bc683627ee9fe186c09a9104.midx' - Traceback (most recent call last): - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup-split\", line 232, in - pack_writer.close() # must close before we can update the ref - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/client.py\", line 520, in close - id = self._end() - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/client.py\", line 517, in _end - return self.suggest_packs() # Returns last idx received - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/client.py\", line 285, in _suggest_packs - self.check_ok() - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/client.py\", line 155, in check_ok - raise ClientError('server exited unexpectedly with code %r' - bup.client.ClientError: server exited unexpectedly with code 1 - ) -ok -copy SHA256E-s247--598ca11afb6b5a92fb0539d5e51b5933986a02b9af76c306ed0e3d123eba7f73.org (to bup...) ok -copy SHA256E-s17120--e883a44474fb01cb2598a35de65bb8f149e8a86f0edad9e769c29cb47738f1d7.org (to bup...) ok -copy SHA256E-s51256--6a45e5a834b84467ca8730b19d11997b1fda618e8264f3052466b0c0d9df33fb (to bup...) ok -copy SHA256E-s138590--273323cbd322923f9d19280b6b4fa73bfa3638d2982b685b9efcaee43c988a07.pdf (to bup...) ok -copy SHA256E-s1142--68e432e17b706adc8fed0c6e573957ebf3fdb3c2ed52152e8e11ff68742a964c.cpp (to bup...) ok -copy SHA256E-s2357--88f5bd074dce62a99f67a2fcb89f69edc323f5b781c43f75e2266caa75f452a8.nix (to bup...) ok -copy SHA256E-s17167--f1a52bf34dc0c09bd05ea76c129bca1edf01cb1634967facddc87ceed030ac0e.org (to bup...) ok -copy SHA256E-s20152--ec24a0a718ff33a20fd18a672e7f8e8d29358a4930a0cb668fc023c6319acff2.cpp (to bup...) ok -copy SHA256E-s74935712--48a36e91a03d46fe73ec361454b9cd3808ae0a6d5372df089c2400ea135269e1.pptx (to bup...) ok -copy SHA256E-s32801--33568e97fb3c67d5646698a3157d2387ccbd447dbe846604e21f135962b7a82a.org (to bup...) ok -copy SHA256E-s1112--08efad2257400abb2a011b7a2bcee115ad69a7c5f8fe622df94f9252e767f4b1.cpp (to bup...) ok -copy SHA256E-s4--af489f7170eaf213b960628333c4752e3907304303ed0b052b43d864d2653c2e (to bup...) ok -copy SHA256E-s270--fd3d6db55c73971d25b7a12211f618ee0f5b9185329974e15a71b8c7313b71d3.org (to bup...) ok -copy SHA256E-s26781--66ff44f139d2f825d6202437078e2a6652214f2d4841f618268a39cb25919c52.org (to bup...) ok -copy SHA256E-s267--5face6a2e6cf04634542c17626cfb273b5cd6d1ff8cad644bb3dfd50b5b26c10 (to bup...) ok -copy SHA256E-s1117802--d9b03b85205196237df6bdcaaa74642db77da46fbea35d1e0de4034f24387cf4.pdf (to bup...) ok -copy SHA256E-s265--fc2aec4bcb3af72e043a2f892440a1370ed19704cf4fb4a3cc26114441b6f8d0.org (to bup...) ok -copy SHA256E-s34783--ad8860a118f942f58672a95c2df3dcfbfd2cc8ee3f5930f52bfbb88dc0a90132.org (to bup...) ok -copy SHA256E-s56184--30106a0f9dd300f8a88e2eb88b2a184a1342874afa2cb4aaa64404a187f1e8c4 (to bup...) ok -copy SHA256E-s74919040--86dcfa59caa040dcfc512bc41b8fe652abc0814324f8fe18d88043a14ecf4402.pptx (to bup...) ok -copy SHA256E-s26013--0bae4beaba4bdf971bd79061a215e2835953b3bb00e532fb0341e6628b796008.org (to bup...) ok -copy SHA256E-s58752--e03f8dc86ed2dc218612022331323a2f5cac319a250b4a24fc15d7312ee9e607 (to bup...) ok -copy SHA256E-s8847--cc7aa3b84e82f7aa197d653f41b545a1e807c98f16187210ded3d2463b41ea86.csv (to bup...) ok -copy SHA256E-s21189--8ffdc40240a69693c596b70a94dd58258cd753b41f34e0c16c38d8dbd89f698e.org (to bup...) ok -copy SHA256E-s24137--9a5c833a7f70f53ba26aa26e3dd2faf22b16e1c4ce33c5733d0a2109560b7c71.org (to bup...) ok -copy SHA256E-s359450--0d0ec93306f455d231ede0e8f2f6c047dd94e0de9a16639e1a7f0b3d86203ac3.pdf (to bup...) - bup [\"split\",\"-r\",\"/tmp/bup\",\"-q\",\"-n\",\"SHA256E-s359450--0d0ec93306f455d231ede0e8f2f6c047dd94e0de9a16639e1a7f0b3d86203ac3.pdf\"] exited 1 (stderr output: Traceback (most recent call last): - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup-midx\", line 298, in - do_midx_dir(path, opt.output, byte_stream(sys.stdout)) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup-midx\", line 198, in do_midx_dir - midxs.sort(key=lambda ix: (-sizes[ix], -xstat.stat(ix).st_mtime)) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup-midx\", line 198, in - midxs.sort(key=lambda ix: (-sizes[ix], -xstat.stat(ix).st_mtime)) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/xstat.py\", line 131, in stat - return stat_result.from_xstat_rep(_helpers.stat(path)) - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/bup/index-cache/None__tmp_bup/midx-3da3384b3acf53204d7890e1bc78030c8b46b3a9.midx' - [b'/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup', b'midx', b'--auto', b'--dir', b'/tmp/bup/index-cache/None__tmp_bup']: returned 1 - WARNING: 1 errors encountered while saving. - ) -ok -copy SHA256E-s1433--20e8793d98da9272424818b56c28af028c1568ec3e82967d6f07ff81015c4108.cpp (to bup...) ok -copy SHA256E-s91176--46ebe34cba131ea6ced189768edccfbf7eb4b7e33fa0be53c36c7888e2bff4a5 (to bup...) - bup [\"split\",\"-r\",\"/tmp/bup\",\"-q\",\"-n\",\"SHA256E-s91176--46ebe34cba131ea6ced189768edccfbf7eb4b7e33fa0be53c36c7888e2bff4a5\"] exited 1 (stderr output: Traceback (most recent call last): - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup-server\", line 320, in - cmd(conn, rest) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup-server\", line 137, in receive_objects_v2 - oldpack = w.exists(shar, want_source=True) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/git.py\", line 804, in exists - self._require_objcache() - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/git.py\", line 797, in _require_objcache - self.objcache = self.objcache_maker() - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/git.py\", line 686, in _make_objcache - return PackIdxList(repo(b'objects/pack')) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/git.py\", line 517, in __init__ - self.refresh() - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/git.py\", line 607, in refresh - midxl.sort(key=lambda ix: - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/git.py\", line 608, in - (-len(ix), -xstat.stat(ix.name).st_mtime)) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/xstat.py\", line 131, in stat - return stat_result.from_xstat_rep(_helpers.stat(path)) - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/bup/objects/pack/midx-4b7dfbb64132c6a9e65912f5d75a49fcc27dfc14.midx' - Traceback (most recent call last): - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup-split\", line 232, in - pack_writer.close() # must close before we can update the ref - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/client.py\", line 520, in close - id = self._end() - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/client.py\", line 517, in _end - return self.suggest_packs() # Returns last idx received - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/client.py\", line 285, in _suggest_packs - self.check_ok() - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/client.py\", line 155, in check_ok - raise ClientError('server exited unexpectedly with code %r' - bup.client.ClientError: server exited unexpectedly with code 1 - ) -ok -copy SHA256E-s84648--95ced533e6e0117e09eb0c6e887e3258ccd85e7c14b376a48683028d1959e6f4 (to bup...) ok -copy SHA256E-s33116--1c7622be0217bef8b345d7a2d741a1ea7140df79fb69ee966d0e2596aedd9b92.org (to bup...) ok -copy SHA256E-s1405--707992e818ada3dcf8ddee86d0bf01db6206e1cd9354c8b32bd760a842757890.cpp (to bup...) ok -copy SHA256E-s34172--5ae202572227ab6df161298ab61c4bfe5ebca4b710e8646602c910996c9c4229.org (to bup...) ok -copy SHA256E-s1406289--7462f1e3ca02e796090fe2ba18a50168a24f5cabc66bb8bf949836d4c81774cb.csv (to bup...) ok -copy SHA256E-s263520--1ae605c38fb827de59f90fe304ad20c673a2d8fcbf8166a4cf44bfc45320fb43 (to bup...) ok -copy SHA256E-s57968--830ff82c9c697a29a7cdf1f4765aadb532e42c932a01c45eafe3d66b254dc30b (to bup...) ok -copy SHA256E-s16350--45d12a82d07c62c00fbb211b3cfbc7abea2fb8b4b1777140b56ebee9244ef7af.org (to bup...) - bup [\"split\",\"-r\",\"/tmp/bup\",\"-q\",\"-n\",\"SHA256E-s16350--45d12a82d07c62c00fbb211b3cfbc7abea2fb8b4b1777140b56ebee9244ef7af.org\"] exited 1 (stderr output: Traceback (most recent call last): - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup-midx\", line 298, in - do_midx_dir(path, opt.output, byte_stream(sys.stdout)) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup-midx\", line 198, in do_midx_dir - midxs.sort(key=lambda ix: (-sizes[ix], -xstat.stat(ix).st_mtime)) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup-midx\", line 198, in - midxs.sort(key=lambda ix: (-sizes[ix], -xstat.stat(ix).st_mtime)) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/xstat.py\", line 131, in stat - return stat_result.from_xstat_rep(_helpers.stat(path)) - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/bup/index-cache/None__tmp_bup/midx-fef6f07a351680cef8630a1c0ba203f2ed8c0e72.midx' - [b'/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup', b'midx', b'--auto', b'--dir', b'/tmp/bup/index-cache/None__tmp_bup']: returned 1 - WARNING: 1 errors encountered while saving. - ) -ok -copy SHA256E-s2--9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa (to bup...) - bup [\"split\",\"-r\",\"/tmp/bup\",\"-q\",\"-n\",\"SHA256E-s2--9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa\"] exited 1 (stderr output: Traceback (most recent call last): - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup-midx\", line 298, in - do_midx_dir(path, opt.output, byte_stream(sys.stdout)) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup-midx\", line 198, in do_midx_dir - midxs.sort(key=lambda ix: (-sizes[ix], -xstat.stat(ix).st_mtime)) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup-midx\", line 198, in - midxs.sort(key=lambda ix: (-sizes[ix], -xstat.stat(ix).st_mtime)) - File \"/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/../bup/xstat.py\", line 131, in stat - return stat_result.from_xstat_rep(_helpers.stat(path)) - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/bup/index-cache/None__tmp_bup/midx-fef6f07a351680cef8630a1c0ba203f2ed8c0e72.midx' - [b'/nix/store/xqgy6d53wbazvkjcgqwdpbww4s87b69z-bup-0.32/lib/bup/cmd/bup', b'midx', b'--auto', b'--dir', b'/tmp/bup/index-cache/None__tmp_bup']: returned 1 - WARNING: 1 errors encountered while saving. - ) -ok -^C -``` - -(Ran this on aarch64-darwin because that's all I have access to ATM but x86_64-linux will probably have the same errors.) - -Given the error, it indeed looks like a concurrency issue in `bup` itself. Version is 0.32 which is the newest. - -"""]] diff --git a/doc/bugs/bup_often_errors_out_when_-J___62___1/comment_3_ca2b909c54a772e1eea58b7cbc07bf39._comment b/doc/bugs/bup_often_errors_out_when_-J___62___1/comment_3_ca2b909c54a772e1eea58b7cbc07bf39._comment deleted file mode 100644 index d726ca75c2..0000000000 --- a/doc/bugs/bup_often_errors_out_when_-J___62___1/comment_3_ca2b909c54a772e1eea58b7cbc07bf39._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2022-08-08T15:11:53Z" - content=""" -Looks like bup-midx is at fault, and that makes sense; it converts git pack -files into "midx" files, so is reading and deleting pack files. So it seems -likely that two of those running at the same time would fail like this. - -I've posted to bup's mailing list about this: - - -git-annex may need to work around this by only running one bup-split at a -time. -"""]] diff --git a/doc/bugs/bup_often_errors_out_when_-J___62___1/comment_4_d7156224245542f5468783c1b5763154._comment b/doc/bugs/bup_often_errors_out_when_-J___62___1/comment_4_d7156224245542f5468783c1b5763154._comment deleted file mode 100644 index dcae1ea760..0000000000 --- a/doc/bugs/bup_often_errors_out_when_-J___62___1/comment_4_d7156224245542f5468783c1b5763154._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2022-08-08T20:13:24Z" - content=""" -Got confirmation that bup is generally not concurrency safe. - -I've made git-annex limit the number of bup-split it runs to 1. - -It may be that this will also need to be done with bup-join, but I think -probably not since it probably does not write to the repo, and a bup-split -is unlikely to get in its way. -"""]] diff --git a/doc/bugs/bup_often_errors_out_when_-J___62___1/comment_5_90d280582d71f77d75aeedcd30c7c437._comment b/doc/bugs/bup_often_errors_out_when_-J___62___1/comment_5_90d280582d71f77d75aeedcd30c7c437._comment deleted file mode 100644 index 5ccfcb5fae..0000000000 --- a/doc/bugs/bup_often_errors_out_when_-J___62___1/comment_5_90d280582d71f77d75aeedcd30c7c437._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="Atemu" - avatar="http://cdn.libravatar.org/avatar/d1f0f4275931c552403f4c6707bead7a" - subject="comment 5" - date="2022-08-09T06:22:50Z" - content=""" -Thank you! - -That's sad to hear as it means bup will perform even worse where it wasn't performing well to begin with but unsafe is unsafe. - -I'd consider this issue solved as bup performance problems are tracked in https://git-annex.branchable.com/bugs/Copying_many_files_to_bup_remotes_is_very_slow/ already. -"""]] diff --git a/doc/bugs/cabal_builds_without_assistant.mdwn b/doc/bugs/cabal_builds_without_assistant.mdwn deleted file mode 100644 index feab883c71..0000000000 --- a/doc/bugs/cabal_builds_without_assistant.mdwn +++ /dev/null @@ -1,14 +0,0 @@ -With cabal 3.4.1.0, `cabal build` does not build the assistant, even though -all dependencies are there. Even after `cabal configure -fAssistant`, it -silently skips installing the deps for the assistant and does not enable the -CPP for the flag. Same for webapp. - -Earlier versions of cabal did not behave this way. Perhaps it is a cabal -bug? - -I tried moving the build-depends out of the `if flag(Assistant)` section -to the main build-depends section. It was able to install all the build -depends, so it's not some dependency issue making it decide to turn off -that build flag. --[[Joey]] - -> [[fixed|done]] by moving the dependencies into the main block. --[[Joey]] diff --git a/doc/bugs/cannot_ungroup_a_repository.mdwn b/doc/bugs/cannot_ungroup_a_repository.mdwn deleted file mode 100644 index d9dc68e672..0000000000 --- a/doc/bugs/cannot_ungroup_a_repository.mdwn +++ /dev/null @@ -1,55 +0,0 @@ -### Please describe the problem. - -I had a repository in the `transfer` group that I wanted to move out of there and switch to `manual`. - -I can't remove it from the `transfer` group. - -### What steps will reproduce the problem? - -[[!format txt """ -git annex group . transfer -git annex group . manual -git annex ungroup . transfer -"""]] - -### What version of git-annex are you using? On what operating system? - -[[!format txt """ -anarcat@angela:video[1]$ git annex version -git-annex version: 10.20221003 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22.1 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.13.1 persistent-sqlite-2.13.1.0 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2.1 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 9 -"""]] - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log -anarcat@angela:video$ git annex ungroup . transfer -git-annex: Specify a repository and a group. -anarcat@angela:video[1]$ git annex group . -manual transfer -anarcat@angela:video$ git annex ungroup ae3d62e6-49be-4340-ba25-c8736a1637c4 transfer -git-annex: Specify a repository and a group. -anarcat@angela:video[1]$ git annex ungroup . -git-annex: Specify a repository and a group. -anarcat@angela:video[1]$ git annex ungroup -git-annex: Specify a repository and a group. -anarcat@angela:video[1]$ -# End of transcript or log. -"""]] - -I just can't figure out how to use the ungroup command, it seems. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Yes! I could have sworn this worked before too. :) - -> [[unreproducible|done]] by submitter --[[Joey]] diff --git a/doc/bugs/cannot_ungroup_a_repository/comment_1_5476ceb9af2fa2d862a868824e5865e4._comment b/doc/bugs/cannot_ungroup_a_repository/comment_1_5476ceb9af2fa2d862a868824e5865e4._comment deleted file mode 100644 index 017db0a8f5..0000000000 --- a/doc/bugs/cannot_ungroup_a_repository/comment_1_5476ceb9af2fa2d862a868824e5865e4._comment +++ /dev/null @@ -1,39 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-02-07T16:19:21Z" - content=""" -That's weird. It does work for me: - - joey@darkstar:~/tmp/xxx>git annex group . - - joey@darkstar:~/tmp/xxx>git annex group . transfer - group . ok - (recording state in git...) - joey@darkstar:~/tmp/xxx>git annex group . manual - group . ok - (recording state in git...) - joey@darkstar:~/tmp/xxx>git annex ungroup . transfer - ungroup . ok - (recording state in git...) - joey@darkstar:~/tmp/xxx>git annex group . - manual - -That's with exactly the same version of git-annex too (10.20221003). - -The "Specify a repository and a group." error is only -displayed when there are not exactly 2 parameters. You -seem to be providing exactly 2, but it seems we need -to rule out the possibility that something (eg the shell, a -wrapper program, or git) is causing some other number of -parameters to be provided.. - - joey@darkstar:~/tmp/xxx>strace -o log git-annex ungroup . transfer - ungroup . ok - (recording state in git...) - joey@darkstar:~/tmp/xxx>grep execve log - execve("/usr/bin/git-annex", ["git-annex", "ungroup", ".", "transfer"], 0x7ffde3606738 /* 66 vars */) = 0 - joey@darkstar:~/tmp/grep execve log | head -n2 - 2720010 execve("/usr/bin/git", ["git", "annex", "ungroup", ".", "transfer"], 0x7fff42a31338 /* 66 vars */) = 0 - 2720011 execve("/usr/bin/git-annex", ["/usr/bin/git-annex", "ungroup", ".", "transfer"], 0x55e494ddb4f0 /* 67 vars */ -"""]] diff --git a/doc/bugs/cannot_ungroup_a_repository/comment_2_0be32bc80e98c0e10b2eb949b20b18dc._comment b/doc/bugs/cannot_ungroup_a_repository/comment_2_0be32bc80e98c0e10b2eb949b20b18dc._comment deleted file mode 100644 index c20293c4df..0000000000 --- a/doc/bugs/cannot_ungroup_a_repository/comment_2_0be32bc80e98c0e10b2eb949b20b18dc._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="anarcat" - avatar="http://cdn.libravatar.org/avatar/4ad594c1e13211c1ad9edb81ce5110b7" - subject="comment 2" - date="2023-02-08T15:38:03Z" - content=""" -I'm sorry to say that the error went away after attaching a strace to it: - -``` -anarcat@angela:video$ strace -o log -e execve git annex ungroup . transfer -ungroup . ok -(recording state in git...) -``` - -no idea wtf happened here, but i guess we can call this [[done]]? next time i'll try strace, sorry, i should have thought of it the first time. -"""]] diff --git a/doc/bugs/cannot_ungroup_a_repository/comment_3_16f1b5ce3830f9d26df4b14ed666e189._comment b/doc/bugs/cannot_ungroup_a_repository/comment_3_16f1b5ce3830f9d26df4b14ed666e189._comment deleted file mode 100644 index c9ac772ba4..0000000000 --- a/doc/bugs/cannot_ungroup_a_repository/comment_3_16f1b5ce3830f9d26df4b14ed666e189._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2023-02-08T16:23:36Z" - content=""" -Have you confirmed the problem no longer happens when not stracing as well? -If it were say, some shell alias causing a problem, using strace would -bypass it.. -"""]] diff --git a/doc/bugs/cannot_ungroup_a_repository/comment_4_bc93a6cb01e7d5d43cccb0eca81b5f33._comment b/doc/bugs/cannot_ungroup_a_repository/comment_4_bc93a6cb01e7d5d43cccb0eca81b5f33._comment deleted file mode 100644 index b03fc6cc44..0000000000 --- a/doc/bugs/cannot_ungroup_a_repository/comment_4_bc93a6cb01e7d5d43cccb0eca81b5f33._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="anarcat" - avatar="http://cdn.libravatar.org/avatar/4ad594c1e13211c1ad9edb81ce5110b7" - subject="comment 4" - date="2023-02-14T19:03:24Z" - content=""" -Yeah, it's not strace: the problem just disappeared, sorry. :/ If I hit it again, i'll be sure to do my homework. :) -"""]] diff --git a/doc/bugs/chunks_not_deduplicated.mdwn b/doc/bugs/chunks_not_deduplicated.mdwn deleted file mode 100644 index 9a70aae973..0000000000 --- a/doc/bugs/chunks_not_deduplicated.mdwn +++ /dev/null @@ -1,27 +0,0 @@ -### Please describe the problem. -When data is chunked for storage on a remote, identical chunks may be stored repeatedly with different hashes. - -### What steps will reproduce the problem? -``` -git annex init . -dd count=16 bs=1024 if=/dev/urandom of=rand1.bin -dd count=15 skip=1 bs=1024 if=rand1.bin of=rand2.bin -git annex add rand1.bin rand2.bin -mkdir data -git annex initremote data type=directory directory=data encryption=none chunk=1024 -git annex copy --all --to=data -find data -type f | xargs sha1sum | sort # shows doubly-stored identical chunks -``` - -### What version of git-annex are you using? On what operating system? -10.20221004-gbf27a02b0 on redhat 7 - -### Please provide any additional information below. - -I imagine an optional upgrade to a new key format for chunks, such that they had their own hashes, and the larger key they were a part of were identified separately (e.g. in data content or metadata). They would then become deduplicated already using existing means. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Yesterday I recovered a bricked phone by accessing flash partitions I had backed up with git-annex. I deleted them locally by accident, by git-annex cobbled many of them together from remotes. The ones it didn't find still had their hashes preserved in the repository, and I was able to compare with hashes on the device and reconstructed all but one. The one I didn't reconstruct is regenerateable. - -> [[notabug|done]] --[[Joey]] diff --git a/doc/bugs/chunks_not_deduplicated/comment_1_3890af3669ac6b37b88dac938ed79d16._comment b/doc/bugs/chunks_not_deduplicated/comment_1_3890af3669ac6b37b88dac938ed79d16._comment deleted file mode 100644 index 036afc770a..0000000000 --- a/doc/bugs/chunks_not_deduplicated/comment_1_3890af3669ac6b37b88dac938ed79d16._comment +++ /dev/null @@ -1,7 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-11-07T16:25:27Z" - content=""" -See [[todo/option_to_individually_hash_chunks]]. -"""]] diff --git a/doc/bugs/copy+rename_of_file_on_windows_breaks_original.mdwn b/doc/bugs/copy+rename_of_file_on_windows_breaks_original.mdwn deleted file mode 100644 index 538fd45399..0000000000 --- a/doc/bugs/copy+rename_of_file_on_windows_breaks_original.mdwn +++ /dev/null @@ -1,49 +0,0 @@ -### Please describe the problem. -Copying and renaming a file "breaks" the original. I'm not using git-annex long enough to use the correct terminology here. "Broken" means "git annex status" reports "fatal: Not a valid object name". - -### What steps will reproduce the problem? -Using the Assistant on Windows, create a fresh repository. Place a file called starship.jpg in the root. Make two directories, called "pic" and "space". Using Explorer, move the picture into "space". Move "space" into "pic". Go into "pic>space". Copy the image and paste it to the root of the repo. Rename the image "wallpaper.jpg". - - -[[!format sh """ -$ git annex info wallpaper.jpg -file: wallpaper.jpg -size: 225.75 kilobytes -key: SHA256E-s225747--e5cba8967f4bea1a971e7405169056852beb3efbbf7fa2d3758c61ba769296c1.jpg -present: true - -$ git annex info pic/space/starship.jpg -fatal: Not a valid object name pic/space/starship.jpg -info pic/space/starship.jpg (not a directory or an annexed file or a treeish or a remote or a uuid) failed - -$ git annex status -D .\space/starship.jpg -? .\pic/space/starship.jpg -"""]] - -### What version of git-annex are you using? On what operating system? -[[!format sh """ -$ git annex version -git-annex version: 10.20220526-gc6b112108 -build flags: Assistant Webapp Pairing TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.7 http-client-0.7.9 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.1.2 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: mingw32 x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 2 3 4 5 6 7 8 9 10 -local repository version: 8 -"""]] - -OS: Windows 10 Pro, 21H1, 19043.1766 - -### Please provide any additional information below. -There's no .git/annex/daemon.log in my repo. Probably because after encountering this issue and reproducing it for this report, I made a new repo, and it seems that the Assistant's logs ended up in the original repo's logs. - -Test image used in this bug report: https://wallpapercave.com/wp/wp8215486.jpg - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) -I'm really loving git-annex. I've been reading up and I'm more than halfway through the devlog and I'm hoping to use git-annex for a lot and a long while. Mostly, I'm expecting to use it on Linux systems, but I'd love to be able to rely on it on my (Windows) work laptop as well. Thanks for looking at my report! - -[[!meta title="assistant on windows misses copy+rename of file"]] -[[done]] diff --git a/doc/bugs/copy+rename_of_file_on_windows_breaks_original/comment_1_c6e8e3e76185b6ca25067c2a7e5cd8fc._comment b/doc/bugs/copy+rename_of_file_on_windows_breaks_original/comment_1_c6e8e3e76185b6ca25067c2a7e5cd8fc._comment deleted file mode 100644 index 4c25df1a09..0000000000 --- a/doc/bugs/copy+rename_of_file_on_windows_breaks_original/comment_1_c6e8e3e76185b6ca25067c2a7e5cd8fc._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-06-28T16:29:03Z" - content=""" -Well, this appears to be a case where the assistant somehow misses the -rename of the file, so it has not gotten around to annexing it. - -You can get the same error message when not using the assistant, eg: - - # touch foo - # git-annex info foo - fatal: Not a valid object name foo - info foo (not a directory or an annexed file or a treeish or a remote or a uuid) failed - -I guess that `git-annex add` would resolve your problem. -The assistant should also notice the file the next time it starts up, -and add it the same as it would any new file. -"""]] diff --git a/doc/bugs/copy+rename_of_file_on_windows_breaks_original/comment_2_7e2604bad32225682513b51c1be6f571._comment b/doc/bugs/copy+rename_of_file_on_windows_breaks_original/comment_2_7e2604bad32225682513b51c1be6f571._comment deleted file mode 100644 index 6b7f71ed00..0000000000 --- a/doc/bugs/copy+rename_of_file_on_windows_breaks_original/comment_2_7e2604bad32225682513b51c1be6f571._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="contr-error" - avatar="http://cdn.libravatar.org/avatar/d79c2491dfc6947b8084796b06174cdb" - subject="comment 2" - date="2022-06-29T19:13:56Z" - content=""" -Hi Joey, thanks for looking into this and your work on git-annex! You're correct on both counts. After posting the report I tried manually adding the file and, indeed, restarting the assistant also makes it notice the file. I'm still exploring, but what seemed to be inscrutable behaviour at first, now seems reasonable enough. - -I've also encountered some access violations and I'm running into sqlite errors on WSL (1 & 2), both of which probably aren't worth reporting at this time, and the latter of which is probably out of scope of this bug tracker, anyway. -"""]] diff --git a/doc/bugs/copy_-J_--to_local_git_remote_does_not_run_concurrently.mdwn b/doc/bugs/copy_-J_--to_local_git_remote_does_not_run_concurrently.mdwn deleted file mode 100644 index b3ead46927..0000000000 --- a/doc/bugs/copy_-J_--to_local_git_remote_does_not_run_concurrently.mdwn +++ /dev/null @@ -1,36 +0,0 @@ -I am copying files to a remote with -J3 and keep seeing this pattern: - - copy x (to remote..) ok - copy foo (to remote...) - 47% 1.03 GiB 31 MiB/s 37s - copy bar (to remote...) - copy baz (to remote...) - -Ie, there are 3 active jobs, but only one of them, typically the first of -them, is actually copying. The other 2 are blocked on something which must -be a resource the 1 is keeping busy. - -That was with the remote having a path as an url. Changed it to -localhost:path and 3 transfers ran concurrently with progress meters. - -copy --from/get also runs concurrently, only --to has the problem. - -Also, a copy --to a directory special remote runs concurrently. So -only local path git remotes have the problem. - -(Of course, there's often no benefit to parellizing writes to a disk as it -will cause seeking, but -J requests it, and if the disk was a SSD, it -would make sense to.) - -git-annex version: 8.20201103 --[[Joey]] - -> Reproduced with 10.20220625. 3 100 mb files and copy to a local git repo. -> Also, `copy --from` and `get` do have the problem, at least they do now. -> --[[Joey]] - -> The problem is that Remote.Git.onLocal uses a MVar for the remote state, -> and the copy runs in that. So while one copy is running, the rest block. -> So fixing this will need a pool of local remote states. -> --[[Joey]] - ->> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/dead_with_no_params_can_mark_repo_with_empty_description_dead.mdwn b/doc/bugs/dead_with_no_params_can_mark_repo_with_empty_description_dead.mdwn deleted file mode 100644 index d84b000453..0000000000 --- a/doc/bugs/dead_with_no_params_can_mark_repo_with_empty_description_dead.mdwn +++ /dev/null @@ -1,13 +0,0 @@ - joey@darkstar:~/tmp/xxxx>git annex describe here "" - describe here ok - (recording state in git...) - joey@darkstar:~/tmp/xxxx>git-annex dead - dead ok - (recording state in git...) - -I would be ok with `git-annex dead ""` doing that, perhaps, but when -no parameters are given, it constructs the empty string itself, which is a bug. - -Found this in --[[Joey]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/describe_what_git-annex_actually_is.mdwn b/doc/bugs/describe_what_git-annex_actually_is.mdwn deleted file mode 100644 index feee65a0e5..0000000000 --- a/doc/bugs/describe_what_git-annex_actually_is.mdwn +++ /dev/null @@ -1,14 +0,0 @@ -The homepage lists lots of things, but does not succinctly specify the core of git-annex. -It took me multiple sifts through the pages and a failed attempt at setting it up to discover that, and how it does not do what I currently need. - -> git-annex allows managing files with git, without checking the file contents into git. -> While that may seem paradoxical, it is useful when dealing with files larger than git can currently easily handle, whether due to limitations in memory, time, or disk space. - -The second sentence does not really help to explain the tool, for me it sounds a lot like git-lfs. My suggestion: - -> git-annex is a sparse file management tool that tracks files separately from their contents. -> It enables managing files larger than git can easily handle, whether due to limitations in memory, time, or disk space. - -Still not optimal, but feels a little more precise - -> [[done]] --[[Joey]] diff --git a/doc/bugs/describe_what_git-annex_actually_is/comment_1_25fc9ffb24fc4c96b81a6393427bbe95._comment b/doc/bugs/describe_what_git-annex_actually_is/comment_1_25fc9ffb24fc4c96b81a6393427bbe95._comment deleted file mode 100644 index ac81ed6be3..0000000000 --- a/doc/bugs/describe_what_git-annex_actually_is/comment_1_25fc9ffb24fc4c96b81a6393427bbe95._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Lukey" - avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b" - subject="comment 1" - date="2021-11-08T19:38:35Z" - content=""" -Describing what `git-annex` does is pretty hard. So when I talk about it, I usually say what you can do with it: manage, sync, backup and archive your data, offline and online, with everything checksummed. -"""]] diff --git a/doc/bugs/describe_what_git-annex_actually_is/comment_2_1f6ec7cedf56863b261bad0e3f3c80de._comment b/doc/bugs/describe_what_git-annex_actually_is/comment_2_1f6ec7cedf56863b261bad0e3f3c80de._comment deleted file mode 100644 index 70272d5e84..0000000000 --- a/doc/bugs/describe_what_git-annex_actually_is/comment_2_1f6ec7cedf56863b261bad0e3f3c80de._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-01-06T16:23:15Z" - content=""" -Thanks, these are good suggestions. I've made some improvements to the -description. -"""]] diff --git a/doc/bugs/error__58___git-annex_filter-process_died_of_signal_15.mdwn b/doc/bugs/error__58___git-annex_filter-process_died_of_signal_15.mdwn deleted file mode 100644 index 136ae77dbb..0000000000 --- a/doc/bugs/error__58___git-annex_filter-process_died_of_signal_15.mdwn +++ /dev/null @@ -1,101 +0,0 @@ -### Please describe the problem. - -When doing - -git checkout 3b18836 - -on a very large repository, where tens of thousands of files will be changed, I get many - -error: external filter 'git-annex filter-process' failed -error: git-annex filter-process died of signal 15 - -and finally - -git-annex: git: createProcess: runInteractiveProcess: pipe: resource exhausted (Too many open files) -fatal: the remote end hung up unexpectedly - - -### What steps will reproduce the problem? - -git checkout -on a really big commit. - -### What version of git-annex are you using? On what operating system? - -The system and all the SW should be up to date. - -% git --version -git version 2.34.1 - -% git annex version -git-annex version: 10.20220322-1~ndall+1 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 9 - -% uname -a -Linux 5.15.0-33-generic #34-Ubuntu SMP Wed May 18 13:34:26 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux - -% cat /etc/os-release -PRETTY_NAME="Ubuntu 22.04 LTS" -NAME="Ubuntu" -VERSION_ID="22.04" -VERSION="22.04 LTS (Jammy Jellyfish)" -VERSION_CODENAME=jammy -ID=ubuntu -ID_LIKE=debian -HOME_URL="https://www.ubuntu.com/" -SUPPORT_URL="https://help.ubuntu.com/" -BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" -PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" -UBUNTU_CODENAME=jammy - - -The computer has 128GB of main memory, and so is not running out of that. - -This is a large repository: - -du -sh . p -43G . - -(62)% du -sh .git -42G .git - -(63)% du -sh .git/annex -19G .git/annex - -% git gc -Enumerating objects: 375565, done. -Counting objects: 100% (375565/375565), done. -Delta compression using up to 12 threads -Compressing objects: 100% (153675/153675), done. -Writing objects: 100% (375565/375565), done. -Total 375565 (delta 215701), reused 364628 (delta 207259), pack-reused 0 -73.38s real 97.69s user 4.93s system 139% 0,0 socket 13753 mem git gc - - -I'm trying to recover from a mess caused by creating a new remote repository, making it shallow to save space, and updating it by mounting it on a local system with sshfs, etc. That caused an unintended rebase messing things up, e.g., git log shows most of the commits vanished (merged?). So I was trying to go back to an earlier, good, commit. However that requires changing, basically, everything. And that seems to be too big a change for git annex to process. - -FWIW, I'm not using lfs. - - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Otherwise, it's great. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/error__58___git-annex_filter-process_died_of_signal_15/comment_1_02da33e41e4f4471d1ae08e9fede229a._comment b/doc/bugs/error__58___git-annex_filter-process_died_of_signal_15/comment_1_02da33e41e4f4471d1ae08e9fede229a._comment deleted file mode 100644 index df76b6d693..0000000000 --- a/doc/bugs/error__58___git-annex_filter-process_died_of_signal_15/comment_1_02da33e41e4f4471d1ae08e9fede229a._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-06-29T16:13:25Z" - content=""" -How big a commit are we talking here, in terms of number of files -added/changed? - -I've tried with 10000 modified files and not seen a problem IIRC. - -Also, signal 15 is SIGTERM, which would be a strange signal to happen -unless it were ctrl-c'd. - -Final weird thing is "fatal: the remote end hung up unexpectedly", since -that would indicate a git fetch was running, and git-annex certianly does -not fetch.. -"""]] diff --git a/doc/bugs/error__58___git-annex_filter-process_died_of_signal_15/comment_2_5ebfc18008cf4bc21ffce57067314fc0._comment b/doc/bugs/error__58___git-annex_filter-process_died_of_signal_15/comment_2_5ebfc18008cf4bc21ffce57067314fc0._comment deleted file mode 100644 index 4ef52812d5..0000000000 --- a/doc/bugs/error__58___git-annex_filter-process_died_of_signal_15/comment_2_5ebfc18008cf4bc21ffce57067314fc0._comment +++ /dev/null @@ -1,85 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-07-13T19:53:44Z" - content=""" -Just reproduced this. In my big repo, I had commit -eab6ac8b34be1d7c36df2a2f41a9414cf5908041 checked out -and ran `git reset --hard fbbe4045871f` - -Git reported that it updated 112465 files, and near to the beginning, -filter-process died of signal 15. - -Even better, it is reproducible, going back to that -starting commit and running that command again works every time! - -And I got a strace too. Here is the git command: - - write(5, "0013command=smudge\n", 19) = 19 - write(5, "0019pathname=new/cleanup\n", 25) = 25 - write(5, "0035treeish=fbbe4045871fb749299b"..., 53) = 53 - write(5, "0032blob=baad2545defeae59f526ea3"..., 50) = 50 - write(5, "0000", 4) = 4 - write(5, "0322", 4) = 4 - write(5, "#!/bin/sh\n\ngit commit -m 'pre cl"..., 798) = 798 - write(5, "0000", 4) = 4 - read(6, "0000", 4) = 4 - rt_sigaction(SIGPIPE, {sa_handler=0x563cf8debb50, sa_mask=[PIPE], sa_flags=SA_RES - TORER|SA_RESTART, sa_restorer=0x7fab5717c920}, {sa_handler=SIG_IGN, sa_mask=[PIPE - ], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7fab5717c920}, 8) = 0 - write(2, "error: external filter 'git-anne"..., 57) = 57 - kill(1638045, SIGTERM) = 0 - -So git sends SIGTERM when it gets unhappy with the response it got from -git-annex filter-process. - -Here is where git runs git-annex filter-process via the shell: - - execve("/bin/sh", ["/bin/sh", "-c", "git-annex filter-process", "git-annex filter-process"], 0x563cfacbfb40 /* 68 vars */) = 0 - ... - newfstatat(AT_FDCWD, "/home/joey/bin/git-annex", {st_mode=S_IFREG|0755, st_size=72524080, ...}, 0) = 0 - rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], NULL, 8) = 0 - vfork() = 1631132 - rt_sigprocmask(SIG_SETMASK, [], ~[KILL STOP RTMIN RT_1], 8) = 0 - wait4(-1, 0x7fff910d697c, 0, NULL) = ? ERESTARTSYS (To be restarted if SA_RESTART is set) - --- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=1631129, si_uid=1000} --- - +++ killed by SIGTERM +++ - -But the actual git-annex process read the last thing git wrote to it, -replied, and keeps running until it later reads an EOF from git. - - write(1, "0000", 4) = 4 - poll([{fd=0, events=POLLIN}], 1, 0) = 0 (Timeout) - epoll_ctl(4, EPOLL_CTL_MOD, 0, {EPOLLIN|EPOLLONESHOT, {u32=0, u64=287065581021036544}}) = 0 - futex(0x67fe568, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NULL, FUTEX_BITSET_MATCH_ANY) = 0 - futex(0x67fe570, FUTEX_WAKE_PRIVATE, 1) = 0 - futex(0x67fe68c, FUTEX_WAKE_PRIVATE, 1) = 1 - read(0, "0013command=smudge\n0019pathname=new/cleanup\n0035treeish=fbbe4045871fb749299b4484862645bd9a7f175a\n0032blob=baad2545defeae59f526ea3d64174398899cb9cc\n00000322", 8192) = 155 - poll([{fd=0, events=POLLIN}], 1, 0) = 1 ([{fd=0, revents=POLLIN}]) - futex(0x67fe688, FUTEX_WAKE_PRIVATE, 1) = 1 - futex(0x67fe690, FUTEX_WAKE_PRIVATE, 1) = 0 - read(0, "#!/bin/sh....", 8192) = 802 - futex(0x67fe56c, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NULL, FUTEX_BITSET_MATCH_ANY) = -1 EAGAIN (Resource temporarily unavailable) - futex(0x67fe570, FUTEX_WAIT_PRIVATE, 2, NULL) = -1 EAGAIN (Resource temporarily unavailable) - futex(0x67fe570, FUTEX_WAKE_PRIVATE, 1) = 0 - poll([{fd=1, events=POLLOUT}], 1, 0) = 1 ([{fd=1, revents=POLLOUT}]) - futex(0x67fe68c, FUTEX_WAKE_PRIVATE, 1) = 1 - futex(0x67fe690, FUTEX_WAKE_PRIVATE, 1) = 0 - write(1, "0013status=success\n", 19) = 19 - futex(0x67fe568, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NULL, FUTEX_BITSET_MATCH_ANY) = 0 - futex(0x67fe570, FUTEX_WAKE_PRIVATE, 1) = 0 - poll([{fd=1, events=POLLOUT}], 1, 0) = 1 ([{fd=1, revents=POLLOUT}]) - write(1, "0000", 4) = 4 - poll([{fd=1, events=POLLOUT}], 1, 0) = 1 ([{fd=1, revents=POLLOUT}]) - write(1, "0322#!/bin/sh...", 802) = 802 - poll([{fd=1, events=POLLOUT}], 1, 0) = 1 ([{fd=1, revents=POLLOUT}]) - write(1, "0000", 4) = 4 - poll([{fd=1, events=POLLOUT}], 1, 0) = 1 ([{fd=1, revents=POLLOUT}]) - write(1, "0000", 4) = 4 - poll([{fd=0, events=POLLIN}], 1, 0) = 0 (Timeout) - -Git does not see that 0013status=success response though. It sees "0000", -which must be the one that git-annex wrote earlier (top of above strace). - -So, this seems like a protocol problem of some kind. -"""] diff --git a/doc/bugs/error__58___git-annex_filter-process_died_of_signal_15/comment_3_c74018853a4da8e609da1803051a6893._comment b/doc/bugs/error__58___git-annex_filter-process_died_of_signal_15/comment_3_c74018853a4da8e609da1803051a6893._comment deleted file mode 100644 index 93f088f6c0..0000000000 --- a/doc/bugs/error__58___git-annex_filter-process_died_of_signal_15/comment_3_c74018853a4da8e609da1803051a6893._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2022-07-13T20:32:03Z" - content=""" -I also saw this when adding a single file to git: - - joey@darkstar:/tmp/a1>touch .foo.lck - joey@darkstar:/tmp/a1>git add .foo.lck - error: external filter 'git-annex filter-process' failed - error: git-annex filter-process died of signal 15 - -But, I have not succeeded to reproduce that again. This was in a v9 repository BTW. -"""]] diff --git a/doc/bugs/error__58___git-annex_filter-process_died_of_signal_15/comment_4_0a80393f055ead9a3a62da0aa4ec3298._comment b/doc/bugs/error__58___git-annex_filter-process_died_of_signal_15/comment_4_0a80393f055ead9a3a62da0aa4ec3298._comment deleted file mode 100644 index 004b6170d0..0000000000 --- a/doc/bugs/error__58___git-annex_filter-process_died_of_signal_15/comment_4_0a80393f055ead9a3a62da0aa4ec3298._comment +++ /dev/null @@ -1,54 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2022-07-13T20:35:53Z" - content=""" -Feeling like this is most likley a pkt-line protocol bug in git-annex, and -probably one inovlving empty files. Both because of the prior comment with -the empty file and it turns out there is also an empty file in my first -reproducer. - -Here is a `GIT_TRACE_PACKET=1` log when reproducing it. I have omitted -the actual content of my files.. - - 16:30:18.248308 pkt-line.c:80 packet: git> command=smudge - 16:30:18.248335 pkt-line.c:80 packet: git> pathname=new/.updater - 16:30:18.248344 pkt-line.c:80 packet: git> treeish=fbbe4045871fb749299b4484862645bd9a7f175a - 16:30:18.248352 pkt-line.c:80 packet: git> blob=dd75da379e8a2cad5a793d352e7def865d65158d - 16:30:18.248358 pkt-line.c:80 packet: git> 0000 - 16:30:18.248397 pkt-line.c:80 packet: git> #!/bin/sh.... - 16:30:18.248409 pkt-line.c:80 packet: git> 0000 - 16:30:18.248637 pkt-line.c:80 packet: git< status=success - 16:30:18.248728 pkt-line.c:80 packet: git< 0000 - 16:30:18.248806 pkt-line.c:80 packet: git< #!/bin/sh.... - 16:30:18.248818 pkt-line.c:80 packet: git< 0000 - 16:30:18.248826 pkt-line.c:80 packet: git< 0000 - 16:30:18.248985 pkt-line.c:80 packet: git> command=smudge - 16:30:18.249007 pkt-line.c:80 packet: git> pathname=new/.updater.lock - 16:30:18.249017 pkt-line.c:80 packet: git> treeish=fbbe4045871fb749299b4484862645bd9a7f175a - 16:30:18.249024 pkt-line.c:80 packet: git> blob=e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 - 16:30:18.249031 pkt-line.c:80 packet: git> 0000 - 16:30:18.249037 pkt-line.c:80 packet: git> 0000 - 16:30:18.249245 pkt-line.c:80 packet: git< status=success - 16:30:18.249262 pkt-line.c:80 packet: git< 0000 - 16:30:18.249270 pkt-line.c:80 packet: git< 0000 - 16:30:18.249277 pkt-line.c:80 packet: git< 0000 - 16:30:18.280842 pkt-line.c:80 packet: git> command=smudge - 16:30:18.280882 pkt-line.c:80 packet: git> pathname=new/cleanup - 16:30:18.280891 pkt-line.c:80 packet: git> treeish=fbbe4045871fb749299b4484862645bd9a7f175a - 16:30:18.280898 pkt-line.c:80 packet: git> blob=baad2545defeae59f526ea3d64174398899cb9cc - 16:30:18.280904 pkt-line.c:80 packet: git> 0000 - 16:30:18.280932 pkt-line.c:80 packet: git> #!/bin/sh... - 16:30:18.280942 pkt-line.c:80 packet: git> 0000 - 16:30:18.280953 pkt-line.c:80 packet: git< 0000 - error: external filter 'git-annex filter-process' failed - error: git-annex filter-process died of signal 15 - -Now, from git's perspective, the last response of "0000" does look wrong. -It should have been "status=success" followed by "0000" and then the file -content, and then "0000" and "0000" again. - -But as seen in the strace, git-annex sends more than that. - -My guess is that git-annex is sending one 0000 too many for an empty file. -"""]] diff --git a/doc/bugs/error__58___git-annex_filter-process_died_of_signal_15/comment_5_8b27c6099afea7011717492f53fcdac4._comment b/doc/bugs/error__58___git-annex_filter-process_died_of_signal_15/comment_5_8b27c6099afea7011717492f53fcdac4._comment deleted file mode 100644 index 0ba99290ca..0000000000 --- a/doc/bugs/error__58___git-annex_filter-process_died_of_signal_15/comment_5_8b27c6099afea7011717492f53fcdac4._comment +++ /dev/null @@ -1,7 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2022-07-13T21:11:07Z" - content=""" -Indeed, I put in a special case for empty files and solved this! -"""]] diff --git a/doc/bugs/error_out_on_file_size_0_from_external_spec_remote.mdwn b/doc/bugs/error_out_on_file_size_0_from_external_spec_remote.mdwn deleted file mode 100644 index a2ad892628..0000000000 --- a/doc/bugs/error_out_on_file_size_0_from_external_spec_remote.mdwn +++ /dev/null @@ -1,90 +0,0 @@ -### Please describe the problem. - -Here is a reproducer using the place where I encountered it -- using git-annex-remote-rclone (tiny bash script; needs fixing though for recent rclone -- take from [my fork where I am tuning testing etc](https://github.com/yarikoptic/git-annex-remote-rclone/blob/enh-tests/git-annex-remote-rclone)) [https://github.com/DanielDent/git-annex-remote-rclone/](https://github.com/DanielDent/git-annex-remote-rclone/) - -
-this is a reproducer which creates repo and commits to annex of size 0 - -```bash -#!/bin/bash - -cd "$(mktemp -d ${TMPDIR:-/tmp}/dl-XXXXXXX)" - -set -eux - -# provide versioning information to possibly ease troubleshooting -git annex version -rclone --version - -export HOME=$PWD -echo -e '[local]\ntype = local\nnounc =' > ~/.rclone.conf -# to pacify git/git-annex -git config --global user.name Me -git config --global user.email me@example.com -git config --global init.defaultBranch master - -# Prepare rclone remote local store -mkdir rclone-local -export RCLONE_PREFIX=$PWD/rclone-local - -git-annex version -mkdir testrepo -cd testrepo -git init . -git-annex init -git-annex initremote GA-rclone-CI type=external externaltype=rclone target=local prefix=$RCLONE_PREFIX chunk=100MiB encryption=shared mac=HMACSHA512 - -# Rudimentary test, spaces in the filename must be ok, 0 length files should be ok - -touch "test 0" - -#echo 1 > "test 1" - -git-annex add * -git-annex copy * --to GA-rclone-CI -git-annex drop * -git-annex get --debug * - -``` -
- -where it ends with - -``` -+ git-annex get --debug 'test 0' -[2022-06-09 13:16:20.668338168] (Utility.Process) process [1022440] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","ls-files","--stage","-z","--error-unmatch","--","test 0"] -[2022-06-09 13:16:20.668784836] (Utility.Process) process [1022441] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)","--buffer"] -[2022-06-09 13:16:20.669111286] (Utility.Process) process [1022442] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"] -[2022-06-09 13:16:20.66965775] (Utility.Process) process [1022443] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","git-annex"] -[2022-06-09 13:16:20.670318546] (Utility.Process) process [1022443] done ExitSuccess -[2022-06-09 13:16:20.67074703] (Utility.Process) process [1022444] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"] -[2022-06-09 13:16:20.671504449] (Utility.Process) process [1022444] done ExitSuccess -[2022-06-09 13:16:20.672125442] (Utility.Process) process [1022445] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/git-annex..0da4aa78e4397feb9906400d1e08aaebccac1be1","--pretty=%H","-n1"] -[2022-06-09 13:16:20.673019668] (Utility.Process) process [1022445] done ExitSuccess -[2022-06-09 13:16:20.673440353] (Utility.Process) process [1022446] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"] -get test 0 [2022-06-09 13:16:20.674594171] (Utility.Process) process [1022448] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"] -(from GA-rclone-CI...) -[2022-06-09 13:16:20.682205613] (Annex.Perms) freezing content directory .git/annex/objects/pX/ZJ/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 - -git-annex: .git/annex/tmp/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855: rename: does not exist (No such file or directory) -failed -[2022-06-09 13:16:20.682296739] (Utility.Process) process [1022446] done ExitSuccess -[2022-06-09 13:16:20.682333234] (Utility.Process) process [1022442] done ExitSuccess -[2022-06-09 13:16:20.682384256] (Utility.Process) process [1022441] done ExitSuccess -[2022-06-09 13:16:20.682408227] (Utility.Process) process [1022440] done ExitSuccess -[2022-06-09 13:16:20.682701178] (Utility.Process) process [1022448] done ExitSuccess -get: 1 failed - -``` - -- you can see that there is not even an attempt to talk to the remote. git-annex crashes before then with that " rename: does not exist (No such file or directory)" - -so I do not think it is an issue of the remote (which might also have some size 0 issues from looking at some conditions) - - -### What version of git-annex are you using? On what operating system? - - -tried both 8.20211123 and 10.20220504 from debian - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/error_out_on_file_size_0_from_external_spec_remote/comment_1_6af7ba08e93349b2c6176f91ba25a06f._comment b/doc/bugs/error_out_on_file_size_0_from_external_spec_remote/comment_1_6af7ba08e93349b2c6176f91ba25a06f._comment deleted file mode 100644 index a2ac6c2d01..0000000000 --- a/doc/bugs/error_out_on_file_size_0_from_external_spec_remote/comment_1_6af7ba08e93349b2c6176f91ba25a06f._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-06-09T17:34:31Z" - content=""" -I can reproduce this with a directory special remote, initialized with - - git annex initremote d type=directory directory=../d encryption=shared chunk=100MiB - -The chunking is what causes the problem. -"""]] diff --git a/doc/bugs/error_out_on_file_size_0_from_external_spec_remote/comment_2_1d78459554bc12d0423a317bd9f466d1._comment b/doc/bugs/error_out_on_file_size_0_from_external_spec_remote/comment_2_1d78459554bc12d0423a317bd9f466d1._comment deleted file mode 100644 index a37c6f1d06..0000000000 --- a/doc/bugs/error_out_on_file_size_0_from_external_spec_remote/comment_2_1d78459554bc12d0423a317bd9f466d1._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-06-09T18:05:15Z" - content=""" -This was broken by [[!commit dad4be97c2057db1ef3a13bb983d1701a90c9069]]. - -For a key of size zero, `addspeculative` adds on a `[]` to the list of chunk -key. But that causes retrieveChunks to think that it's already retrieved -all the chunks, so it avoids doing any retrieval, so the file is not -written. It needs to retrieve the empty key even though it's empty, -so it can decrypt it when it's encrypted; git-annex does not special case -empty file retrieval. - -Odd that testremote did not detect this. It does test with empty keys, and -with chunking. -"""]] diff --git a/doc/bugs/error_out_on_file_size_0_from_external_spec_remote/comment_3_aa083a54e0c77cdba9939a061b509488._comment b/doc/bugs/error_out_on_file_size_0_from_external_spec_remote/comment_3_aa083a54e0c77cdba9939a061b509488._comment deleted file mode 100644 index af5678741d..0000000000 --- a/doc/bugs/error_out_on_file_size_0_from_external_spec_remote/comment_3_aa083a54e0c77cdba9939a061b509488._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="yarikoptic" - avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" - subject="comment 3" - date="2022-06-09T21:53:05Z" - content=""" - -> fixed in [10.20220525-73-g13fc6a9b6](https://git.kitenet.net/index.cgi/git-annex.git/commit/?id=13fc6a9b6ad4f0ee0a783c91825ab7ad35dd2b64) --[[yarikoptic]] - -PS I wish there was a similar but automated annotation of those `[done]` to know actually \"when/with what\" -"""]] diff --git a/doc/bugs/external_remote_export_sent_to_wrong_process.mdwn b/doc/bugs/external_remote_export_sent_to_wrong_process.mdwn deleted file mode 100644 index 193f5c00f7..0000000000 --- a/doc/bugs/external_remote_export_sent_to_wrong_process.mdwn +++ /dev/null @@ -1,87 +0,0 @@ -### Please describe the problem. - -When getting a lot of objects from an external special remote of the exporttree kind, -sometimes the `EXPORT` command is sent to a different process of the external remote -than the following `TRANSFEREXPORT` command. - -As a result, one process receives two `EXPORT`s in a row, the first of which it ignores, -while some other process receives a `TRANSFEREXPORT` without a prior `EXPORT`, -reusing whatever filename was set in the previous transaction, -and either ovewriting the last accessed remote file with the wrong content, -or retrieving the content of the last accessed remote file instead of the one git-annex wanted. - -Or I misunderstood the protocol - please tell me if that's the case. - -### What steps will reproduce the problem? - -I don't have a minimal reproducing example. - -I was working on adding tree export support to git-annex-remote-rclone, -and its the testsuite on my branch can reproduce the problem around half of the time, but only on MacOS. -It seems like it's a race condition and the interleaving that triggers it doesn't happen on Linux - -Here's an example test run when it fails -[on github](https://github.com/Wolf480pl/git-annex-remote-rclone/actions/runs/4525558194/jobs/7970140554) - -and the test code - [on github](https://github.com/Wolf480pl/git-annex-remote-rclone/blob/bd1a497fca8614286ec290bb557c83442c0e23c9/tests/all-in-one.sh#L224) -(I'd post it as an attachment but I'm getting an error that only admin can upload attachments). - -### What version of git-annex are you using? On what operating system? - -git-annex version: 10.20230227 -OS: macOS 12.6.3 21G419 - -### Please provide any additional information below. - -Full log is quite long so I put it in a [github gist](https://gist.github.com/Wolf480pl/4bdc83e23154827aad46e84bad631419). -I wanted to upload it as an attachment but apparently I'm not allowed. - -Here are the interesting parts: - -First, during the tree export, we see an example of correct upload of the `test 9.dat` and `test 10.dat` files: -[[!format txt """ -2023-03-26T16:34:52.2551390Z [2023-03-26 16:34:52.085377] (Annex.ExternalAddonProcess) /Users/runner/work/git-annex-remote-rclone/git-annex-remote-rclone/git-annex-remote-rclone[3] <-- EXPORT test 9.dat -2023-03-26T16:34:52.2653060Z [2023-03-26 16:34:52.086085] (Annex.ExternalAddonProcess) /Users/runner/work/git-annex-remote-rclone/git-annex-remote-rclone/git-annex-remote-rclone[3] <-- TRANSFEREXPORT STORE SHA256E-s7--a30e9d26ce633b40da3ba8cd8806b9b349cb6c3de6816c6d64f85a571012a973.dat .git/annex/objects/zZ/kw/SHA256E-s7--a30e9d26ce633b40da3ba8cd8806b9b349cb6c3de6816c6d64f85a571012a973.dat/SHA256E-s7--a30e9d26ce633b40da3ba8cd8806b9b349cb6c3de6816c6d64f85a571012a973.dat -[...] -2023-03-26T16:34:49.9382650Z [2023-03-26 16:34:49.629684] (Annex.ExternalAddonProcess) /Users/runner/work/git-annex-remote-rclone/git-annex-remote-rclone/git-annex-remote-rclone[3] <-- EXPORT test 10.dat -2023-03-26T16:34:49.9384770Z [2023-03-26 16:34:49.629761] (Annex.ExternalAddonProcess) /Users/runner/work/git-annex-remote-rclone/git-annex-remote-rclone/git-annex-remote-rclone[3] <-- TRANSFEREXPORT STORE SHA256E-s8--d5624694cf1515bdf8c3a648ae35d64f4bdff934800a22a070c6d0baddc120b0.dat .git/annex/objects/6k/X3/SHA256E-s8--d5624694cf1515bdf8c3a648ae35d64f4bdff934800a22a070c6d0baddc120b0.dat/SHA256E-s8--d5624694cf1515bdf8c3a648ae35d64f4bdff934800a22a070c6d0baddc120b0.dat -"""]] - -so we know that key of `test 9.dat` is `a30e...973.dat`, and for `test 10.dat` it's `d56...0b0.dat` - -Then, during the get, we see the bug happen: -[[!format txt """ -2023-03-26T16:34:55.4392510Z [2023-03-26 16:34:54.534833] (Annex.ExternalAddonProcess) /Users/runner/work/git-annex-remote-rclone/git-annex-remote-rclone/git-annex-remote-rclone[5] --> TRANSFER-SUCCESS RETRIEVE SHA256E-s8--fbb0a327e1528cdb3214abb2ec3fb1dd97cc39dfc12751df102eca019f602e73.dat -2023-03-26T16:34:55.4394480Z [2023-03-26 16:34:54.53532] (Annex.Perms) freezing content .git/annex/objects/45/Gp/SHA256E-s8--91d9183c1a8f61526ed68c5357d52a719481ccbba5039f815cf8f71ae4edbf24.dat/SHA256E-s8--91d9183c1a8f61526ed68c5357d52a719481ccbba5039f815cf8f71ae4edbf24.dat -2023-03-26T16:34:55.4395900Z [2023-03-26 16:34:54.545524] (Annex.Branch) read b51/a3b/SHA256E-s7--a30e9d26ce633b40da3ba8cd8806b9b349cb6c3de6816c6d64f85a571012a973.dat.log -2023-03-26T16:34:55.4397280Z [2023-03-26 16:34:54.545836] (Annex.Perms) freezing content directory .git/annex/objects/45/Gp/SHA256E-s8--91d9183c1a8f61526ed68c5357d52a719481ccbba5039f815cf8f71ae4edbf24.dat -2023-03-26T16:34:55.4398630Z [2023-03-26 16:34:54.54609] (Annex.Branch) read 253/975/SHA256E-s8--91d9183c1a8f61526ed68c5357d52a719481ccbba5039f815cf8f71ae4edbf24.dat.log -2023-03-26T16:34:55.4399950Z [2023-03-26 16:34:54.5477] (Annex.Branch) set 253/975/SHA256E-s8--91d9183c1a8f61526ed68c5357d52a719481ccbba5039f815cf8f71ae4edbf24.dat.log -2023-03-26T16:34:55.4401900Z [2023-03-26 16:34:54.558828] (Annex.ExternalAddonProcess) /Users/runner/work/git-annex-remote-rclone/git-annex-remote-rclone/git-annex-remote-rclone[2] --> TRANSFER-SUCCESS RETRIEVE SHA256E-s7--b163189df1a08d63ba85e1566799cf07ca2de3865ac97a85b2b2d0cbfbd9a2f3.dat -2023-03-26T16:34:55.4403360Z [2023-03-26 16:34:54.559391] (Annex.Branch) read c9a/21e/SHA256E-s8--128888bb8975fc51a7fd410322b088593e158b37a24973483da2ad17fb6d7ff5.dat.log -2023-03-26T16:34:55.4405510Z [2023-03-26 16:34:54.561866] (Annex.ExternalAddonProcess) /Users/runner/work/git-annex-remote-rclone/git-annex-remote-rclone/git-annex-remote-rclone[2] <-- EXPORT test 9.dat -2023-03-26T16:34:55.4407810Z [2023-03-26 16:34:54.561942] (Annex.ExternalAddonProcess) /Users/runner/work/git-annex-remote-rclone/git-annex-remote-rclone/git-annex-remote-rclone[2] <-- EXPORT test 10.dat -2023-03-26T16:34:55.4409480Z [2023-03-26 16:34:54.561946] (Annex.ExternalAddonProcess) /Users/runner/work/git-annex-remote-rclone/git-annex-remote-rclone/git-annex-remote-rclone[5] <-- TRANSFEREXPORT RETRIEVE SHA256E-s7--a30e9d26ce633b40da3ba8cd8806b9b349cb6c3de6816c6d64f85a571012a973.dat .git/annex/tmp/SHA256E-s7--a30e9d26ce633b40da3ba8cd8806b9b349cb6c3de6816c6d64f85a571012a973.dat -2023-03-26T16:34:55.4411390Z [2023-03-26 16:34:54.562032] (Annex.ExternalAddonProcess) /Users/runner/work/git-annex-remote-rclone/git-annex-remote-rclone/git-annex-remote-rclone[2] <-- TRANSFEREXPORT RETRIEVE SHA256E-s8--d5624694cf1515bdf8c3a648ae35d64f4bdff934800a22a070c6d0baddc120b0.dat .git/annex/tmp/SHA256E-s8--d5624694cf1515bdf8c3a648ae35d64f4bdff934800a22a070c6d0baddc120b0.dat -"""]] - -As you can see in the last 4 lines: - - * process 2 got `EXPORT test 9.dat` and then immediately `EXPORT test 10.dat` - * then process 5 got the `TRANSFEREXPORT` command for the `test 9.dat` file (`a30..973.dat`) - * process 5 did not, prior to this transfer, receive any `EXPORT` command since its last transfer of an unrelated file (see `TRANSFER-SUCCESS` in the first quoted line). - * so it was still holding the old filename from that unrelated transfer - * then, process 2 receives a (correct) `TRANSFEREXPORT` for the `test 10.dat` file (`d56...0b0.dat`). - -It appears that the `EXPORT test 9.dat` should've been sent to process 5 instead of process 2. -But since it wasn't, process 5 retrieved the wrong file into what git-annex expected to be `test 9.dat`. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -I really wanted to use git annex for my media storage, it seems like the project I've been looking for for years. - -But I first needed to be able to exporttree over FTP. -(I have a cheap NAS is terribly slow at encryption (SSH, etc) so I'm using FTP to get reasonable speeds with it.) -So I thought I'd implement it and then hit this bug. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/external_remote_export_sent_to_wrong_process/comment_1_831bdc33451de9ef5c54592a7455683f._comment b/doc/bugs/external_remote_export_sent_to_wrong_process/comment_1_831bdc33451de9ef5c54592a7455683f._comment deleted file mode 100644 index f4c5189082..0000000000 --- a/doc/bugs/external_remote_export_sent_to_wrong_process/comment_1_831bdc33451de9ef5c54592a7455683f._comment +++ /dev/null @@ -1,46 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-03-28T18:35:23Z" - content=""" -Looking at the code, handleRequestExport first uses -withExternalState to send EXPORT, and then it calls -handleRequestKey to send the following command. That uses -withExternalState a second time. - -withExternalState operates on a pool of processes, so in a race -(when using -J presumably, as your test case does), the two -calls to it can use different externals. And so the bug. - -Was easy to fix based on that analysis. I have not tried to reproduce it, -but am confident that I fixed it in -[[!commit 02662f52920e84cd9464641ada84f6c3bbe3f86a]] - ----- - -In your implementation, you're keeping EXPORT set after handling TRANSFEREXPORT -and similar commands, and so if you receive a TRANSFEREXPORT not prefixed by an -EXPORT, it can be bad. A natural way to write things, indeed -`doc/special_remotes/external/example.sh` does the same. - -An alternative implementation would be to clear the EXPORT after handling a -TRANSFEREXPORT, CHECKPRESENTEXPORT, REMOVEEXPORT, or RENAMEEXPORT. -And have those error out if no EXPORT was received before them. - -But, that does not fully avoid the data loss problem. Because this might -happen: - - EXPORT foo - EXPORT bar - TRANSFEREXPORT STORE $foo_key $foo_file - -So even ignoring the first EXPORT can result in the external special remote -doing the wrong thing. To fully guard against it, you'd have to error out -if multiple EXPORT were received before a request that uses one, rather -than ignoring the first EXPORT. - -So, fixing this in your remote and others would need significant defensive -coding. Too much to make sense, IMHO. I think git-annex needs to chnge the -protocol in some way instead, to make it easy for you to avoid speaking to -a buggy git-annex. -"""]] diff --git a/doc/bugs/external_remote_export_sent_to_wrong_process/comment_2_8ff0638891c38f423d792f5c3cbfb64f._comment b/doc/bugs/external_remote_export_sent_to_wrong_process/comment_2_8ff0638891c38f423d792f5c3cbfb64f._comment deleted file mode 100644 index ce766cb387..0000000000 --- a/doc/bugs/external_remote_export_sent_to_wrong_process/comment_2_8ff0638891c38f423d792f5c3cbfb64f._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2023-03-28T19:29:01Z" - content=""" -What about simply increasing the protocol version number? If VERSION 2 is -the same as VERSION 1, but only supported by the fixed git-annex, then an -external can just be updated to send VERSION 2, and it does not need to -worry about talking with a buggy version of git-annex. - -git-annex could either continue to also support VERSION 1, or it could -refuse to work with externals that don't use VERSION 2. The latter would -force externals to get updated. But then those externals would have no way to work -with old git-annex even if they wanted to. I think forcing an update is not called -for. So git-annex will keep supporting both versions. -"""]] diff --git a/doc/bugs/external_remote_export_sent_to_wrong_process/comment_3_d774a7319bd1a1778adb2004229f057e._comment b/doc/bugs/external_remote_export_sent_to_wrong_process/comment_3_d774a7319bd1a1778adb2004229f057e._comment deleted file mode 100644 index 5f8d9eef55..0000000000 --- a/doc/bugs/external_remote_export_sent_to_wrong_process/comment_3_d774a7319bd1a1778adb2004229f057e._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2023-03-28T20:51:40Z" - content=""" -I've implemented support for `VERSION 2`. Recommend any external special -remotes that support exporttree=yes use it to avoid talking with a buggy -git-annex version. - -(See [[todo/better_message_for_external_special_remote_protocol_mismatch]] -for related todo.) -"""]] diff --git a/doc/bugs/external_remote_export_sent_to_wrong_process/comment_4_db89603335995755d145141705ac4584._comment b/doc/bugs/external_remote_export_sent_to_wrong_process/comment_4_db89603335995755d145141705ac4584._comment deleted file mode 100644 index fa9eff74b1..0000000000 --- a/doc/bugs/external_remote_export_sent_to_wrong_process/comment_4_db89603335995755d145141705ac4584._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="wolf480@8ad1ccdd08efc303a88f7e88c4e629be6637a44e" - nickname="wolf480" - avatar="http://cdn.libravatar.org/avatar/816b19ee786208f3216fe146d7733086" - subject="comment 4" - date="2023-03-30T17:33:10Z" - content=""" -Hmm but a remote only learns that it's being used with exporttree=yes *after* it has sent a `VERSION`? - -I'm afraid I can't bump the version in case of git-annex-remote-rclone, it's already widely used for non-exporttree scenarios and requiring a git-annex that supports `VERSION 2` for these existing usecases (which aren't affected by this bug) would be a regression... - -I'll ask git-annex-remote-rclone maintainers, but it seems to me that defensive coding around the `EXPORT` command is gonna be a better solution in this remote's case. -"""]] diff --git a/doc/bugs/external_remote_export_sent_to_wrong_process/comment_5_48656a34262fce77ac50836d1d38b9fe._comment b/doc/bugs/external_remote_export_sent_to_wrong_process/comment_5_48656a34262fce77ac50836d1d38b9fe._comment deleted file mode 100644 index e2d96b7ecf..0000000000 --- a/doc/bugs/external_remote_export_sent_to_wrong_process/comment_5_48656a34262fce77ac50836d1d38b9fe._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="wolf480@8ad1ccdd08efc303a88f7e88c4e629be6637a44e" - nickname="wolf480" - avatar="http://cdn.libravatar.org/avatar/816b19ee786208f3216fe146d7733086" - subject="comment 5" - date="2023-03-30T17:33:58Z" - content=""" -Oh, also many thanks for quickly fixing this! -"""]] diff --git a/doc/bugs/external_remote_export_sent_to_wrong_process/comment_6_e38b776c73affdbb06f3debdddb07e59._comment b/doc/bugs/external_remote_export_sent_to_wrong_process/comment_6_e38b776c73affdbb06f3debdddb07e59._comment deleted file mode 100644 index de7cd4053d..0000000000 --- a/doc/bugs/external_remote_export_sent_to_wrong_process/comment_6_e38b776c73affdbb06f3debdddb07e59._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 6""" - date="2023-03-31T16:55:49Z" - content=""" -What if I made git-annex send "EXTENSIONS PROTOCOLVERSION2"? Then -you could reply to EXPORTSUPPORTED with EXPORTSUPPORTED-FAILURE -when used by a buggy git-annex. -"""]] diff --git a/doc/bugs/external_remote_export_sent_to_wrong_process/comment_7_6a85fcfa3eb0b4974733a62bcc0bbab2._comment b/doc/bugs/external_remote_export_sent_to_wrong_process/comment_7_6a85fcfa3eb0b4974733a62bcc0bbab2._comment deleted file mode 100644 index a633b3828a..0000000000 --- a/doc/bugs/external_remote_export_sent_to_wrong_process/comment_7_6a85fcfa3eb0b4974733a62bcc0bbab2._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="wolf480@8ad1ccdd08efc303a88f7e88c4e629be6637a44e" - nickname="wolf480" - avatar="http://cdn.libravatar.org/avatar/816b19ee786208f3216fe146d7733086" - subject="comment 7" - date="2023-04-01T07:23:18Z" - content=""" -So basically I'd reply to it with `EXTENSIONS PROTOCOLVERSION2`, and then set a flag that supporting export is ok? -And `EXTENSIONS` always comes before `EXPORTSUPPORTED` ? -Sounds like a very ad-hoc use of the extension mechanism... I think it'd work but I'd feel bad if you made a release with such a feature and then it turned out it doesn't work... -Do you have some sort of per-releases? -"""]] diff --git a/doc/bugs/external_remote_export_sent_to_wrong_process/comment_8_c4610009f691ce60940979b73bef4d78._comment b/doc/bugs/external_remote_export_sent_to_wrong_process/comment_8_c4610009f691ce60940979b73bef4d78._comment deleted file mode 100644 index 97fc92bed5..0000000000 --- a/doc/bugs/external_remote_export_sent_to_wrong_process/comment_8_c4610009f691ce60940979b73bef4d78._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 8""" - date="2023-04-04T18:26:29Z" - content=""" -You would not need to send back `EXTENSIONS PROTOCOLVERSION2` -yourself, only check if git-annex sends it. A better name might be -EXPORTBUGFIXED, it's only using the existing extensions mechanism as a flag -to communicate that git-annex has the bug fixed. - -Some old versions of git-annex don't send EXTENSIONS at all (and those -also don't have the bug fixed), but if git-annex does send it, it will -always be before EXPORTSUPPORTED. -"""]] diff --git a/doc/bugs/external_remote_protocol_unable_to_parse_command.mdwn b/doc/bugs/external_remote_protocol_unable_to_parse_command.mdwn deleted file mode 100644 index 2b15892c4a..0000000000 --- a/doc/bugs/external_remote_protocol_unable_to_parse_command.mdwn +++ /dev/null @@ -1,33 +0,0 @@ -### Please describe the problem. - -I'm writing an [external special remote](/design/external_special_remote_protocol) and am unable to send GETSTATE in response to CHECKPRESENT, e.g. - -[[!format sh """ -copy foo [2022-03-21 12:31:22.499943048] (Utility.Process) process [3998210] chat: /home/ben/go/bin/git-annex-remote-blah [] -[2022-03-21 12:31:22.503831297] (Annex.ExternalAddonProcess) /home/ben/go/bin/git-annex-remote-blah[1] --> VERSION 1 -[2022-03-21 12:31:22.503974302] (Annex.ExternalAddonProcess) /home/ben/go/bin/git-annex-remote-blah[1] <-- EXTENSIONS INFO GETGITREMOTENAME ASYNC -[2022-03-21 12:31:22.504199734] (Annex.ExternalAddonProcess) /home/ben/go/bin/git-annex-remote-blah[1] --> UNSUPPORTED-REQUEST -[2022-03-21 12:31:22.50428745] (Annex.ExternalAddonProcess) /home/ben/go/bin/git-annex-remote-blah[1] <-- PREPARE -[2022-03-21 12:31:22.504410356] (Annex.ExternalAddonProcess) /home/ben/go/bin/git-annex-remote-blah[1] --> GETCREDS auth_token -[2022-03-21 12:31:22.504586934] (Annex.ExternalAddonProcess) /home/ben/go/bin/git-annex-remote-blah[1] <-- CREDS ellided -[2022-03-21 12:31:22.504764648] (Annex.ExternalAddonProcess) /home/ben/go/bin/git-annex-remote-blah[1] --> PREPARE-SUCCESS -[2022-03-21 12:31:22.504835875] (Annex.ExternalAddonProcess) /home/ben/go/bin/git-annex-remote-blah[1] <-- CHECKPRESENT SHA256E-s19--3b99fd544e5e49915ca060f193c615caf82a045b1145672dd6f2181473fd516d -[2022-03-21 12:31:22.504954457] (Annex.ExternalAddonProcess) /home/ben/go/bin/git-annex-remote-blah[1] --> GETSTATE foo - - external special remote protocol error, unexpectedly received "GETSTATE foo" (unable to parse command) -[2022-03-21 12:31:22.510269196] (Utility.Process) process [3998217] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"] -(unable to use special remote due to protocol error) failed -"""]] - -Have I misunderstood the protocol or is this a bug? - -### What version of git-annex are you using? On what operating system? - -10.20220128-g7e9ebea91 on Debian - - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -I use git-annex successfully constantly and have done for years. :-) First time writing a special external remote, though. - -> [[notabug|done]] --[[Joey]] diff --git a/doc/bugs/external_remote_protocol_unable_to_parse_command/comment_1_a10012aa7df76a2366ea40da8c37b798._comment b/doc/bugs/external_remote_protocol_unable_to_parse_command/comment_1_a10012aa7df76a2366ea40da8c37b798._comment deleted file mode 100644 index 4dd1650617..0000000000 --- a/doc/bugs/external_remote_protocol_unable_to_parse_command/comment_1_a10012aa7df76a2366ea40da8c37b798._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-03-21T17:17:54Z" - content=""" -"foo" is not a valid git-annex key, so "GETSTATE foo" is a protocol error. - -If you had it send "GETSTATE SHA256E-s19--3b99fd544e5e49915ca060f193c615caf82a045b1145672dd6f2181473fd516d", -it would work. -"""]] diff --git a/doc/bugs/external_remote_protocol_unable_to_parse_command/comment_2_a4556ce5c5f40ca022385cfb3a97ddac._comment b/doc/bugs/external_remote_protocol_unable_to_parse_command/comment_2_a4556ce5c5f40ca022385cfb3a97ddac._comment deleted file mode 100644 index 1eeb5366da..0000000000 --- a/doc/bugs/external_remote_protocol_unable_to_parse_command/comment_2_a4556ce5c5f40ca022385cfb3a97ddac._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="bz4" - avatar="http://cdn.libravatar.org/avatar/88de97ce728b76df2a27b496ad27af0c" - subject="comment 2" - date="2022-03-21T19:18:53Z" - content=""" -Ah, I see. Thank you! -"""]] diff --git a/doc/bugs/failure_getting__47__copying_on_git-lfs_remote___40__gcrypt__41__.mdwn b/doc/bugs/failure_getting__47__copying_on_git-lfs_remote___40__gcrypt__41__.mdwn deleted file mode 100644 index fda8ed98a3..0000000000 --- a/doc/bugs/failure_getting__47__copying_on_git-lfs_remote___40__gcrypt__41__.mdwn +++ /dev/null @@ -1,246 +0,0 @@ -### Please describe the problem. - -I'm not sure this is a bug. - -We are trying to share a git-lfs special remote on GitLab, encrypted with gcrypt, but one client is getting "gpg: decryption failed: No secret key" when getting or copying content, but sync works well. - -I followed the [private encrypted git remote on a git-lfs hosting site](https://git-annex.branchable.com/tips/fully_encrypted_git_repositories_with_gcrypt/#index4h2) guide. - -I can sync/add/get git-annex content from the git-lfs special remote (named origin), I can also clone the repository on another machine and get/copy content to origin with no problems. - -My collegue (listed in gcryt-participants) is able to clone the repo and sync the remote but he cannot get/add content to it (see below). - -### What steps will reproduce the problem? - -Fist I created an new empty project on GitLab - -On my machine I created an empty swws-library directory then: - -[[!format sh """ -git init - -# initialize git annex and set the encrypted git-lfs remote -git annex init -git annex initremote origin type=git-lfs url=gcrypt::git@gitlab.com:softwareworkers/swws-library.git keyid=FCE2EDE78BD9B2CB keyid=D37D0EA7CECC3912 -git config remote.origin.gcrypt-participants "D37D0EA7CECC3912 FCE2EDE78BD9B2CB" - -git annex sync -# I had to unprotect the master branch in the GitLab project repository settings and again: -git annex sync - -# Add a couple of media files -git annex sync --content - - -# All went fine -"""]] - -Then, my collegue (the owner of the FCE2EDE78BD9B2CB gpg key) cloned the repository and: - -[[!format sh """ -git annex enableremote origin -git config remote.origin.gcrypt-participants "D37D0EA7CECC3912 FCE2EDE78BD9B2CB" -git push origin master git-annex -git annex sync -# git annex add -git annex sync -# all is going fine -"""]] - -On my machine I can "git annex sync origin" and get the updated repository, but my collegue cannot get or copy from/to that remote: - -[[!format sh """ -a9i@trix:~/Repos/SW/swws-library$ git annex get -smarketing/220517_workshop_testuale_part1.mp3 -get smarketing/220517_workshop_testuale_part1.mp3 (from origin...) -gpg: decryption failed: No secret key - - user error (gpg -["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] -exited 2) - - Unable to access these remotes: origin - - Maybe add some of these git remotes (git remote add ...): - 5e070ec6-adbe-4a60-be06-af60b777d03f -- -g@renaissance:~/{git}/softwareworkers.it/swws-library -failed -get: 1 failed -"""]] - -This is the debug output when copying annexed files to the origin remote: - -[[!format sh """ - a9i@trix:~/Repos/SW/swws-library$ LC_ALL=C git annex copy --debug --to=origin corsi/ - [2022-07-18 15:47:28.478299533] (Utility.Process) process [269581] read: - git - ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","git-annex"] - [2022-07-18 15:47:28.480014105] (Utility.Process) process [269581] done - ExitSuccess - [2022-07-18 15:47:28.480418294] (Utility.Process) process [269582] read: - git - ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"] - [2022-07-18 15:47:28.482213732] (Utility.Process) process [269582] done - ExitSuccess - [2022-07-18 15:47:28.491746214] (Utility.Process) process [269583] read: - git - ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/git-annex..19a97025cc5689891c920766f8fa3ed85fe16229","--pretty=%H","-n1"] - [2022-07-18 15:47:28.494757234] (Utility.Process) process [269583] done - ExitSuccess - [2022-07-18 15:47:28.496856609] (Utility.Process) process [269584] chat: - git - ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"] - [2022-07-18 15:47:28.507245457] (Utility.Process) process [269585] read: - git - ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","ls-files","--stage","-z","--error-unmatch","--","corsi/"] - [2022-07-18 15:47:28.509053804] (Utility.Process) process [269586] chat: - git - ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) - %(objecttype) %(objectsize)","--buffer"] - [2022-07-18 15:47:28.510594336] (Utility.Process) process [269587] chat: - git - ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) - %(objecttype) %(objectsize)","--buffer"] - [2022-07-18 15:47:28.511891608] (Utility.Process) process [269584] done - ExitSuccess - [2022-07-18 15:47:28.513074507] (Utility.Process) process [269588] chat: - git - ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) - %(objecttype) %(objectsize)","--buffer"] - copy corsi/2022-06-09_Landing-Pages/2022-06-09_Landing-Pages.org - [2022-07-18 15:47:28.524203277] (Utility.Process) process [269590] chat: - gpg - ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] - gpg: decryption failed: No secret key - [2022-07-18 15:47:28.532883171] (Utility.Process) process [269590] done - ExitFailure 2 - (user error (gpg - ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] - exited 2)) failed - copy corsi/2022-06-09_Landing-Pages/2022-06-09_Landing-Pages.org~ - [2022-07-18 15:47:28.533500984] (Utility.Process) process [269592] chat: - gpg - ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] - gpg: decryption failed: No secret key - [2022-07-18 15:47:28.540138973] (Utility.Process) process [269592] done - ExitFailure 2 - (user error (gpg - ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] - exited 2)) failed - copy corsi/2022-06-09_Landing-Pages/202206 Formaper - Landing page.pdf - [2022-07-18 15:47:28.540791533] (Utility.Process) process [269594] chat: - gpg - ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] - gpg: decryption failed: No secret key - [2022-07-18 15:47:28.551983197] (Utility.Process) process [269594] done - ExitFailure 2 - (user error (gpg - ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] - exited 2)) failed - copy corsi/2022-06-09_Landing-Pages/3700213292852724482.mp4 [2022-07-18 - 15:47:28.554726904] (Utility.Process) process [269596] chat: gpg - ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] - gpg: decryption failed: No secret key - [2022-07-18 15:47:28.567432702] (Utility.Process) process [269596] done - ExitFailure 2 - (user error (gpg - ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] - exited 2)) failed - copy corsi/2022-06-09_Landing-Pages/6258438201216682752.mp4 [2022-07-18 - 15:47:28.571835037] (Utility.Process) process [269598] chat: gpg - ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] - gpg: decryption failed: No secret key - [2022-07-18 15:47:28.579064056] (Utility.Process) process [269598] done - ExitFailure 2 - (user error (gpg - ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] - exited 2)) failed - copy corsi/2022-06-09_Landing-Pages/8959772244561726210.mp4 [2022-07-18 - 15:47:28.581423831] (Utility.Process) process [269600] chat: gpg - ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] - gpg: decryption failed: No secret key - [2022-07-18 15:47:28.602507936] (Utility.Process) process [269600] done - ExitFailure 2 - (user error (gpg - ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] - exited 2)) failed - copy corsi/2022-06-09_Landing-Pages/Slide - Landing page efficaci.pdf - [2022-07-18 15:47:28.6080323] (Utility.Process) process [269602] chat: - gpg - ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] - gpg: decryption failed: No secret key - [2022-07-18 15:47:28.616162922] (Utility.Process) process [269602] done - ExitFailure 2 - (user error (gpg - ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] - exited 2)) failed - copy corsi/2022-06-09_Landing-Pages/Story_mapping.png [2022-07-18 - 15:47:28.617007178] (Utility.Process) process [269604] chat: gpg - ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] - gpg: decryption failed: No secret key - [2022-07-18 15:47:28.624214363] (Utility.Process) process [269604] done - ExitFailure 2 - (user error (gpg - ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] - exited 2)) failed - copy corsi/2022-06-09_Landing-Pages/canvas.png [2022-07-18 - 15:47:28.624826009] (Utility.Process) process [269606] chat: gpg - ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] - gpg: decryption failed: No secret key - [2022-07-18 15:47:28.634726242] (Utility.Process) process [269606] done - ExitFailure 2 - (user error (gpg - ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"] - exited 2)) failed - [2022-07-18 15:47:28.634918681] (Utility.Process) process [269588] done - ExitSuccess - [2022-07-18 15:47:28.634986759] (Utility.Process) process [269587] done - ExitSuccess - [2022-07-18 15:47:28.635050868] (Utility.Process) process [269586] done - ExitSuccess - [2022-07-18 15:47:28.63509396] (Utility.Process) process [269585] done - ExitSuccess - copy: 9 failed -"""]] - -### What version of git-annex are you using? On what operating system? - -git-annex version: 10.20220624 on Guix on Debian as foreign distro. - -### Please provide any additional information below. - -This is my collegue (the failing part) git repository configuration: - -[[!format sh """ - [core] - repositoryformatversion = 0 - filemode = true - bare = false - logallrefupdates = true - [remote "origin"] - url = gcrypt::git@gitlab.com:softwareworkers/swws-library.git - gcrypt-id = :id:Fb42A4B5ODZW/35Z5ZoI - fetch = +refs/heads/*:refs/remotes/origin/* - annex-git-lfs = true - annex-uuid = 9b43bd79-3d24-4e6f-b196-46f5bc67f214 - gcrypt-participants = D37D0EA7CECC3912 FCE2EDE78BD9B2CB - gcrypt-publish-participants = true - annex-ignore = false - [branch "master"] - remote = origin - merge = refs/heads/master - [annex] - uuid = 6a09ad78-19a1-451a-8ab5-fb51d18966eb - version = 8 - [filter "annex"] - smudge = git-annex smudge -- %f - clean = git-annex smudge --clean -- %f -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Oh yes, I'm using git-annex since many many years ago and it's my preferred storage solution, I really love it! - -**Kudos** Joey! - -> [[fixed|done]], I think, see my comment --[[Joey]] diff --git a/doc/bugs/failure_getting__47__copying_on_git-lfs_remote___40__gcrypt__41__/comment_1_ba5b0ed901c99a23ff72d68b83c0d2fa._comment b/doc/bugs/failure_getting__47__copying_on_git-lfs_remote___40__gcrypt__41__/comment_1_ba5b0ed901c99a23ff72d68b83c0d2fa._comment deleted file mode 100644 index 62bbca6f71..0000000000 --- a/doc/bugs/failure_getting__47__copying_on_git-lfs_remote___40__gcrypt__41__/comment_1_ba5b0ed901c99a23ff72d68b83c0d2fa._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-08-15T17:42:49Z" - content=""" -I think that the problem is that keyid= can only be specified once, -so the last keyid= you specified was used, which was yours, and it ignored -the other one. - -Try `git-annex enableremote origin keyid+=FCE2EDE78BD9B2CB` - -I found some documentation that says that keyid= can be repeated multiple -times, and I think that documentation is wrong, and probably led you -astray. It's also possible that it used to work and was broken in some -option parsing changes, unsure. I'm working to correct the documentation now. - -(Or, encryption=shared seems likely to avoid whatever the problem is.) -"""]] diff --git a/doc/bugs/fsck_--json_incomplete_error_reporting.mdwn b/doc/bugs/fsck_--json_incomplete_error_reporting.mdwn deleted file mode 100644 index d44bf347fa..0000000000 --- a/doc/bugs/fsck_--json_incomplete_error_reporting.mdwn +++ /dev/null @@ -1,53 +0,0 @@ -### Please describe the problem. - -I ran git annex fsck --json. I have some files that are lacking numcopies. I get output in stderr, but no error details in the json body. - -For example: - -{"command":"fsck","success":false,"key":"SHA256E-s165540--ddcf7ce58593667e1b836e2a7f28a9f5227f3d9ba46cf8f98c7ab9dd26ef1896.jpg","error-messages":[],"file":"2022/12/04/chandrian_10:06:41.jpg","dead":[],"untrusted":[],"input":["2022/12/04/chandrian_10:06:41.jpg"]} - Only 2 of 5 trustworthy copies exist of 2022/12/04/chandrian_11:05:20.jpg - Back it up with git-annex copy. - - -It would be great if error-messages contained all the details for failures. Thank you - - -### What steps will reproduce the problem? - -* create a new repo -* annex a file -* set numcopies to 2 or whatever -* git annex fsck --json - - -### What version of git-annex are you using? On what operating system? - -I'm on Fedora 37. - -git annex version -git-annex version: 10.20221103 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.7 http-client-0.6.4.1 persistent-sqlite-2.13.1.0 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 10 - - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Yes! Git annex is amazing and is managing over 10 TB of data across 5 git annexes and around 10 hard drives. No data loss this entire time -- over 8 years. - -> [[notabug|done]]; see my comment --[[Joey]] diff --git a/doc/bugs/fsck_--json_incomplete_error_reporting/comment_1_979d52131fe94a9e800f939da88b369e._comment b/doc/bugs/fsck_--json_incomplete_error_reporting/comment_1_979d52131fe94a9e800f939da88b369e._comment deleted file mode 100644 index 0e33e6f007..0000000000 --- a/doc/bugs/fsck_--json_incomplete_error_reporting/comment_1_979d52131fe94a9e800f939da88b369e._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="kanak@3c4f6e7d832d88751c617b25bdbac896417eb93b" - nickname="kanak" - avatar="http://cdn.libravatar.org/avatar/708121dfec06e554300b2a3a73a26818" - subject="comment 1" - date="2022-12-04T17:40:38Z" - content=""" -Not just limited to numcopies: - -{\"command\":\"fsck\",\"success\":true,\"key\":\"SHA256E-s119046--239da5a85ddf8c4071d8803a864a896d13e2a2fd65fd5684fc2f6dcaf264e875.jpg\",\"error-messages\":[],\"file\":\"12/03/chandrian_22:37:41.jpg\",\"note\":\"checksum...\",\"input -\":[\"12/03/chandrian_22:37:41.jpg\"]} - ** Based on the location log, 12/03/chandrian_23:05:41.jpg - ** was expected to be present, but its content is missing. - - - -"""]] diff --git a/doc/bugs/fsck_--json_incomplete_error_reporting/comment_2_71fd5ede0aa24fcc6522a03010ccec5c._comment b/doc/bugs/fsck_--json_incomplete_error_reporting/comment_2_71fd5ede0aa24fcc6522a03010ccec5c._comment deleted file mode 100644 index f369a27d9e..0000000000 --- a/doc/bugs/fsck_--json_incomplete_error_reporting/comment_2_71fd5ede0aa24fcc6522a03010ccec5c._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-12-26T17:27:54Z" - content=""" -Use the --json-error-messages option to move that into the json. It is -documented on the man page. -"""]] diff --git a/doc/bugs/fsck_ignores_--json-error-messages_when_--quiet.mdwn b/doc/bugs/fsck_ignores_--json-error-messages_when_--quiet.mdwn deleted file mode 100644 index 83c4d44db3..0000000000 --- a/doc/bugs/fsck_ignores_--json-error-messages_when_--quiet.mdwn +++ /dev/null @@ -1,76 +0,0 @@ -### Please describe the problem. - -As I understand the manual: - -- Options `--json --json-error-messages` are provided so that another program can parse the `git annex fsck` results. - -- Option `--quiet` is provided to list only problems (not print anything for OK files). - -However, when options are combined, only plain text error messages are provided, no json output is provided. - -I understand this may be "as designed", quiet is quiet... But then how to log only errors in json? I have +300k files in the annex, and no need to log when everything is fine. - -### What steps will reproduce the problem? - -Create a repo with files b and c - -Corrupt file b - -`git annex fsck --json --json-error-messages --quiet` - -I expected to have a json output with only files that fail the fsck, instead I get only normal stderr, just like with -`git annex fsck --quiet` - -### What version of git-annex are you using? On what operating system? -10.20230926-12 on arch - -### Please provide any additional information below. - -[[!format sh """ - -# Expected plain result -> git annex fsck - -fsck b - ** No known copies exist of b -failed -fsck c (checksum...) ok -(recording state in git...) -fsck: 1 failed - -# Expected json result (error message to stderr, both logs) -> git annex fsck --json - - ** No known copies exist of b -{"command":"fsck","dead":[],"error-messages":[],"file":"b","input":["b"],"key":"SHA256E-s5--f2ca1bb6c7e907d06dafe4687e579fce76b37e4e93b7605022da52e6ccc26fd2","success":false,"untrusted":[]} -{"command":"fsck","error-messages":[],"file":"c","input":["c"],"key":"SHA256E-s4--530a0b93b8c1ea618546d3aaa6ec71f888d2a6095322bfdb1b04c9225e26481e","note":"checksum...","success":true} -fsck: 1 failed - -# Expected json output with error message embedded -> git annex fsck --json --json-error-messages - -{"command":"fsck","dead":[],"error-messages":["** No known copies exist of b"],"file":"b","input":["b"],"key":"SHA256E-s5--f2ca1bb6c7e907d06dafe4687e579fce76b37e4e93b7605022da52e6ccc26fd2","success":false,"untrusted":[]} -{"command":"fsck","error-messages":[],"file":"c","input":["c"],"key":"SHA256E-s4--530a0b93b8c1ea618546d3aaa6ec71f888d2a6095322bfdb1b04c9225e26481e","note":"checksum...","success":true} -fsck: 1 failed - -# Expected only error message -> git annex fsck --quiet - - ** No known copies exist of b -fsck: 1 failed - -# UnExpected result: I expected a json output with the error message embedded "--json --json-error-messages" seem ignored here -> git annex fsck --json --json-error-messages --quiet - - ** No known copies exist of b -fsck: 1 failed - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Yes, great tool ! Thanks ! - - -> [[wontfix|done]] per my comment --[[Joey]] diff --git a/doc/bugs/fsck_ignores_--json-error-messages_when_--quiet/comment_1_a23d96af8c0e418350a73cbce5bc24be._comment b/doc/bugs/fsck_ignores_--json-error-messages_when_--quiet/comment_1_a23d96af8c0e418350a73cbce5bc24be._comment deleted file mode 100644 index f0f8660751..0000000000 --- a/doc/bugs/fsck_ignores_--json-error-messages_when_--quiet/comment_1_a23d96af8c0e418350a73cbce5bc24be._comment +++ /dev/null @@ -1,23 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-11-21T20:03:45Z" - content=""" -Hmm, I've never considered combining --quiet with --json. It's kind of -undefined and really not clear to me what it should do. - -But, --json-error-messages makes the json contain an error-messages field, -and the error message is in there. So you can just extract that and ignore -the other messages in the json output. No need to use --quiet then. - -I suppose there may be someone who uses --json as a matter of course, but -adds --quiet to that when they want to disable the json output. So -changing the current behavior, ill-defined as it is, would be asking for -trouble. - -What actually happens currently is which ever output option comes last -overrides earlier options. So `--json --quiet` is quiet, and `--quiet ---json` outputs json. `--json-error-messages` is like `--json` in this -regard to. Which is just behavior that fell out of the option parser -implementation. -"""]] diff --git a/doc/bugs/fsck_says_ok_for_a_file_that_is_not_in_the_remote.mdwn b/doc/bugs/fsck_says_ok_for_a_file_that_is_not_in_the_remote.mdwn deleted file mode 100644 index a8d249d324..0000000000 --- a/doc/bugs/fsck_says_ok_for_a_file_that_is_not_in_the_remote.mdwn +++ /dev/null @@ -1,47 +0,0 @@ -### Please describe the problem. -fsck says ok while file is not present in the remote - -The remote looks like this: -`apple-slicer gcrypt::rsync://apple-slicer/encr/gits/important_docs` - -``` -> git annex fsck --from apple-slicer Taxes/2018-19/Joint_T5.pdf -fsck Taxes/2018-19/Joint_T5.pdf ok -(recording state in git...) -``` - -This portion in the log does not look good - -``` -[2023-10-20 18:06:23.514382] (Utility.Process) process [63613] done ExitSuccess -[2023-10-20 18:06:23.515339] (Utility.Process) process [63621] read: rsync ["apple-slicer:/encr/gits/important_docs/annex/objects/093/d6e/GPGHMACSHA1--47089ada3aaae768ea5e1d2ccc9284ac83a1d80f/GPGHMACSHA1--47089ada3aaae768ea5e1d2ccc9284ac83a1d80f"] -[2023-10-20 18:06:24.344801] (Utility.Process) process [63621] done ExitFailure 23 -[2023-10-20 18:06:24.346693] (Utility.Process) process [63630] read: rsync ["apple-slicer:/encr/gits/important_docs/annex/objects/f9/V3/GPGHMACSHA1--47089ada3aaae768ea5e1d2ccc9284ac83a1d80f/GPGHMACSHA1--47089ada3aaae768ea5e1d2ccc9284ac83a1d80f"] -[2023-10-20 18:06:25.148852] (Utility.Process) process [63630] done ExitFailure 23 -ok -[2023-10-20 18:06:25.152624] (Utility.Process) process [63599] done ExitSuccess -``` - -Full log: https://privatebin.net/?c70841788ba0b6c1#9aCknSWqLjzQZd3n1hJUCK4GFEcoPFd1cZrfnWWoF5zm - -### What steps will reproduce the problem? -1. Create a special remote of type `gcrypt::rsync` -2. `git annex fsck --from {special remote} --debug` - -### What version of git-annex are you using? On what operating system? -macOS -`git-annex version: 10.20230626` - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -> [[notabug|done]] --[[Joey]] diff --git a/doc/bugs/fsck_says_ok_for_a_file_that_is_not_in_the_remote/comment_1_ce9085e6330625dabf479e6e3beeeb47._comment b/doc/bugs/fsck_says_ok_for_a_file_that_is_not_in_the_remote/comment_1_ce9085e6330625dabf479e6e3beeeb47._comment deleted file mode 100644 index 218cc0050b..0000000000 --- a/doc/bugs/fsck_says_ok_for_a_file_that_is_not_in_the_remote/comment_1_ce9085e6330625dabf479e6e3beeeb47._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-10-21T16:26:06Z" - content=""" -I don't see a bug here. If the file is not present there and was not -expected to be present there, and there are enough copies in other places, -then fsck has nothing to complain about. -"""]] diff --git a/doc/bugs/fsck_says_ok_for_a_file_that_is_not_in_the_remote/comment_2_59e89c065a3cd8f7c80cfdb3c2c2808d._comment b/doc/bugs/fsck_says_ok_for_a_file_that_is_not_in_the_remote/comment_2_59e89c065a3cd8f7c80cfdb3c2c2808d._comment deleted file mode 100644 index dde7cc6805..0000000000 --- a/doc/bugs/fsck_says_ok_for_a_file_that_is_not_in_the_remote/comment_2_59e89c065a3cd8f7c80cfdb3c2c2808d._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Manager2103" - avatar="http://cdn.libravatar.org/avatar/c3e024b0067a77f1ef99c6dd88614060" - subject="comment 2" - date="2023-10-21T17:22:53Z" - content=""" -I see. How could I check the consistency of all the files in a given remote? I thought that was the command -"""]] diff --git a/doc/bugs/fsck_says_ok_for_a_file_that_is_not_in_the_remote/comment_3_264a1b39c1677545af4c02901dd23d22._comment b/doc/bugs/fsck_says_ok_for_a_file_that_is_not_in_the_remote/comment_3_264a1b39c1677545af4c02901dd23d22._comment deleted file mode 100644 index ba110ba0b5..0000000000 --- a/doc/bugs/fsck_says_ok_for_a_file_that_is_not_in_the_remote/comment_3_264a1b39c1677545af4c02901dd23d22._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2023-10-23T17:41:04Z" - content=""" -Well `fsck --from` does check the consistency of what's in the remote. -That entails verifying that the remote contains the data that -git-annex thinks it contains. But it also means verifying that the remote -does not contain any data that git-annex didn't know it contains. - -Checking the file is not present on the remote is part of the latter. - -If you wanted to only verify that files git-annex thinks are on the remote -are present there, you could use eg `git-annex fsck --from theremote --in theremote` -"""]] diff --git a/doc/bugs/fsck_says_ok_for_a_file_that_is_not_in_the_remote/comment_4_f08e305e39953c5c1906df0fb2eb113b._comment b/doc/bugs/fsck_says_ok_for_a_file_that_is_not_in_the_remote/comment_4_f08e305e39953c5c1906df0fb2eb113b._comment deleted file mode 100644 index 2abae48cc5..0000000000 --- a/doc/bugs/fsck_says_ok_for_a_file_that_is_not_in_the_remote/comment_4_f08e305e39953c5c1906df0fb2eb113b._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="Manager2103" - avatar="http://cdn.libravatar.org/avatar/c3e024b0067a77f1ef99c6dd88614060" - subject="comment 4" - date="2023-10-23T17:50:22Z" - content=""" -That makes perfect sense. Thank you so much! - -And thanks for some fantastic software :) -"""]] diff --git a/doc/bugs/fsck_tells___39__ok__39___also_if_no_file_present.mdwn b/doc/bugs/fsck_tells___39__ok__39___also_if_no_file_present.mdwn deleted file mode 100644 index 860a94f3d1..0000000000 --- a/doc/bugs/fsck_tells___39__ok__39___also_if_no_file_present.mdwn +++ /dev/null @@ -1,77 +0,0 @@ -### Please describe the problem. - -git-annex fsck tells ok, if there is no file present - is this behavior correct? I would expect git-annex to tell me that the file is not available. -Well there is a (checksum...) ok output if the file exists, but I find 'ok' to be misleading if there is no file present. You could think that the file was checked and the checksum was correct - but this is not the case. - - -### What steps will reproduce the problem? - -Just drop/move a file in the repo and than fsck it. git-annex will tell you 'ok' but the file dosen't exits. - -```sh -sc@section9 ~/t/repo2> git status -On branch master -Your branch is up to date with 'origin/master'. - -nothing to commit, working tree clean -sc@section9 ~/t/repo2> git annex info XMonad.png -file: XMonad.png -size: 397.17 kilobytes -key: SHA256E-s397167--4114f98837a83581add0f513ac2c1f0257ad8de0c18614ca30a9a5dbfe7a5904.png -present: true -sc@section9 ~/t/repo2> git annex fsck XMonad.png -fsck XMonad.png (checksum...) ok -(recording state in git...) -sc@section9 ~/t/repo2> git annex move --to origin XMonad.png -move XMonad.png ok -(recording state in git...) -sc@section9 ~/t/repo2> git status -On branch master -Your branch is up to date with 'origin/master'. - -nothing to commit, working tree clean -sc@section9 ~/t/repo2> git annex info XMonad.png -file: XMonad.png -size: 397.17 kilobytes -key: SHA256E-s397167--4114f98837a83581add0f513ac2c1f0257ad8de0c18614ca30a9a5dbfe7a5904.png -present: false -sc@section9 ~/t/repo2> git annex fsck XMonad.png -fsck XMonad.png ok # ok? well there is no file, how can git-annex tell me that Xmonad.png checksum is okay? -(recording state in git...) -``` - -### What version of git-annex are you using? On what operating system? - -```sh -sc@section9 ~/t/repo2> git annex version -git-annex version: 8.20200330 -build flags: Assistant Webapp Pairing S3 WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.3 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0.1 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs hook external -operating system: linux x86_64 -supported repository versions: 8 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 -local repository version: 8 - - -sc@section9 ~> cat /etc/issue - -<<< Welcome to NixOS 19.09.2370.e10c65cdb35 (\m) - \l >>> - -``` - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - - -> [[notabug|done]] (I did add some more detail to the man page tho) --[[Joey]] diff --git a/doc/bugs/gcrypt__58___WARNING__58___Remote_ID_has_changed__33__.mdwn b/doc/bugs/gcrypt__58___WARNING__58___Remote_ID_has_changed__33__.mdwn deleted file mode 100644 index 0b15a070fb..0000000000 --- a/doc/bugs/gcrypt__58___WARNING__58___Remote_ID_has_changed__33__.mdwn +++ /dev/null @@ -1,50 +0,0 @@ -### Please describe the problem. - -For some reason, after a while I can't sync to any remotes (removeable drives, servers, etc.) - - -### What steps will reproduce the problem? - -git annex sync, git annex assistant doing its autosync, etc. - - -### What version of git-annex are you using? On what operating system? - -Arch Linux, git annex 6.20160613-g1e4e6f4 - - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. - -Typing git annex synx in a terminal gives me: - -commit ok -pull disk0_annexes_docs -gcrypt: Decrypting manifest -gpg: Signature made Mon 25 Jul 2016 00:13:00 BST using RSA key ID XXXXXXXXXXXXXXX -gpg: Good signature from "XXX " [ultimate] -gcrypt: WARNING: -gcrypt: WARNING: Remote ID has changed! -gcrypt: WARNING: from :id:lczpGq00kb1sF+cxgCNE -gcrypt: WARNING: to :id:2br4BOnJZkrGUsXTlQR8 - - -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - -Not specific to assistant. - - -# End of transcript or log. -"""]] - - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Yes, this worked for ages, a year or more, it's awesome! Haven't plugged the removeable drives in for a while, now when I do this happens... - -> git-annex does not set the gcrypt id. git-remote-gcrypt does that. -> So if it somehow changed, it must be because of a bug in -> git-remote-gcrypt. Or, because of something you did.. -> [[notabug|done]] --[[Joey]] diff --git a/doc/bugs/gcrypt_remotes_using_relative_paths.mdwn b/doc/bugs/gcrypt_remotes_using_relative_paths.mdwn deleted file mode 100644 index fab52069e3..0000000000 --- a/doc/bugs/gcrypt_remotes_using_relative_paths.mdwn +++ /dev/null @@ -1,39 +0,0 @@ -### Please describe the problem. -`git annex sync` is not automatically run for gcrypt remotes using rsync with a relative path - -### What steps will reproduce the problem? -Flow 1 (relative path, broken) - -* `git remote add test gcrypt::rsync://user@user.rsync.net:relative/path/to/repo` -* `git annex sync` -> DOES NOT SYNC to test remote -* Nothing has been synced so I CANNOT successfully clone from the test remote with `git clone gcrypt::rsync://user@user.rsync.net:relative/path/to/repo` -* `git push test git-annex master` -* I can successfully clone from the test remote with `git clone gcrypt::rsync://user@user.rsync.net:relative/path/to/repo` - -Flow 2 (absolute path, working) - -* `git remote add test gcrypt::rsync://user@user.rsync.net/full/path/to/repo` -* `git annex sync` -> DOES SYNC to test remote -* I can successfully clone from the test remote with `git clone gcrypt::rsync://user@user.rsync.net:relative/path/to/repo` - - -### What version of git-annex are you using? On what operating system? -* Debian 11 -* git-annex version: 10.20230227 -* git-remote-gcrypt version 1.5 - - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) -I am VERY happy with git annex and am using it successfully with a gcrypt remote using an absolute path :) - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/gcrypt_remotes_using_relative_paths/comment_1_b4523baf0cdfc2ad70cded93b1629eca._comment b/doc/bugs/gcrypt_remotes_using_relative_paths/comment_1_b4523baf0cdfc2ad70cded93b1629eca._comment deleted file mode 100644 index 7f589384a2..0000000000 --- a/doc/bugs/gcrypt_remotes_using_relative_paths/comment_1_b4523baf0cdfc2ad70cded93b1629eca._comment +++ /dev/null @@ -1,30 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-03-23T17:53:22Z" - content=""" -The lack of a transcript made this bug hard to understand. After about -half an hour of staring at it, I realized that you mean that `git-annex sync` -does not even attempt to sync with the remote when it has the relative url: - - joey@darkstar:~/tmp/bb>git remote add test gcrypt::rsync://user@user.rsync.net:relative/path/to/repo - joey@darkstar:~/tmp/bb>git-annex sync - commit - On branch master - nothing to commit, working tree clean - ok - -It does sync with it when asked to sync with explicitly that remote: - - joey@darkstar:~/tmp/bb>git-annex sync test - commit - On branch master - nothing to commit, working tree clean - ok - pull test - gcrypt: Repository not found: rsync://user@user.rsync.net:relative/path/to/repo - ok - -With the absolute url, it does sync with it when no remote -is explicitly specified. -"""]] diff --git a/doc/bugs/gcrypt_remotes_using_relative_paths/comment_2_45281c947992c2ab124efd0f109255d6._comment b/doc/bugs/gcrypt_remotes_using_relative_paths/comment_2_45281c947992c2ab124efd0f109255d6._comment deleted file mode 100644 index 4175db0e42..0000000000 --- a/doc/bugs/gcrypt_remotes_using_relative_paths/comment_2_45281c947992c2ab124efd0f109255d6._comment +++ /dev/null @@ -1,36 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2023-03-23T18:27:36Z" - content=""" -Analysis: When parsing the relative url, git-annex ends up constructing a -Repo with location `LocalUnknown "./rsync://user@user.rsync.net:relative/path/to/repo.git"` - -So, it thinks it's a local git repository, which of course does not exist, -so it skips syncing with it. - -Why does it do that? Well: - - ghci> import Network.URI - ghci> parseURI "rsync://user@user.rsync.net:relative/path/to/repo.git" - Nothing - -And Git.GCrypt.encryptedRemote strips off the "gcrypt::", -leaving that. - - ghci> r - Repo {location = LocalUnknown "/home/joey/tmp/bb", config = fromList [], fullconfig = fromList [], remoteName = Nothing, gitEnv = Nothing, gitEnvOverridesGitDir = False, gitGlobalOpts = [], gitDirSpecifiedExplicitly = False} - ghci> rr - Repo {location = Url gcrypt::rsync://user@user.rsync.net:relative/path/to/repo, config = fromList [], fullconfig = fromList [], remoteName = Just "test1", gitEnv = Nothing, gitEnvOverridesGitDir = False, gitGlobalOpts = [], gitDirSpecifiedExplicitly = False} - ghci> encryptedRemote r rr - Repo {location = LocalUnknown "/home/joey/tmp/bb/rsync://user@user.rsync.net:relative/path/to/repo.git", config = fromList [], fullconfig = fromList [], remoteName = Nothing, gitEnv = Nothing, gitEnvOverridesGitDir = False, gitGlobalOpts = [], gitDirSpecifiedExplicitly = False} - ghci> parseRemoteLocation "rsync://user@user.rsync.net:relative/path/to/repo" r - RemotePath "rsync://user@user.rsync.net:relative/path/to/repo" - -parseRemoteLocation uses isURI and that does not parse as a valid URI. - -So Git.GCrypt.encryptedRemote will need to force it to parse as an url in this case. - -(I remember that I fixed git-remote-gcrypt to support absolute urls for related -reasons. Those relative nonstandard urls are not a good idea.) -"""]] diff --git a/doc/bugs/git-annex-enableremote_confusing_behavior_when_there_is_a_git_remote_with_same_name_as_special_remote.mdwn b/doc/bugs/git-annex-enableremote_confusing_behavior_when_there_is_a_git_remote_with_same_name_as_special_remote.mdwn deleted file mode 100644 index 84ac671cee..0000000000 --- a/doc/bugs/git-annex-enableremote_confusing_behavior_when_there_is_a_git_remote_with_same_name_as_special_remote.mdwn +++ /dev/null @@ -1,13 +0,0 @@ -If there's a git remote named "foo", and a special remote with the same -name, `git annex enableremote foo` just prints "ok" and silently does -nothing. - -Perhaps worse, passing enableremote the uuid of the special remote causes -it to *modify* the same named git remote's config, adding/replacing -annex-uuid and adding whatever other git configs the special remote uses. ---[[Joey]] - -(initremote does check if there's already a remote with the same name, and -refuses to overwrite it). - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/git-annex-enableremote_confusing_behavior_when_there_is_a_git_remote_with_same_name_as_special_remote/comment_1_7f28aecc6ecefd7d47979f73330759d2._comment b/doc/bugs/git-annex-enableremote_confusing_behavior_when_there_is_a_git_remote_with_same_name_as_special_remote/comment_1_7f28aecc6ecefd7d47979f73330759d2._comment deleted file mode 100644 index d70e00024e..0000000000 --- a/doc/bugs/git-annex-enableremote_confusing_behavior_when_there_is_a_git_remote_with_same_name_as_special_remote/comment_1_7f28aecc6ecefd7d47979f73330759d2._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-09-22T17:05:28Z" - content=""" -The behavior in the first case has changed, it now displays: - - enableremote (normal) foo ok - -And the normal git remote is left as-is. Which seems more or less -reasonable. - -The behavior in the second case (passing the uuid) is still as described. - -I've fixed that to instead fail with: - - git-annex: Not overwriting currently configured git remote named "foo" -"""]] diff --git a/doc/bugs/git-annex-import_imports_outside_of_directory.mdwn b/doc/bugs/git-annex-import_imports_outside_of_directory.mdwn deleted file mode 100644 index 0adc7e253d..0000000000 --- a/doc/bugs/git-annex-import_imports_outside_of_directory.mdwn +++ /dev/null @@ -1,31 +0,0 @@ -### Please describe the problem. -When using `git-annex-import` to import from a directory remote, if the directory tree has directory symlinks pointing to directories outside the directory tree of the directory remote, the targets of these symlinks get imported. This can lead to importing much more than was intended; such symlinks should probably get imported as symlinks by default, with a command-line option to import their targets. There might even be a security issue with unexpectedly importing and sharing content outside the explicitly specified directory tree. - - -### What version of git-annex are you using? On what operating system? - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log -$ git annex version -git-annex version: 10.20220322-g7b64dea -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.7 http-client-0.7.9 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.1.2 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 8 -$ uname -a -Linux ctchpcpx163.merck.com 3.10.0-1160.6.1.el7.x86_64 #1 SMP Tue Nov 17 13:59:11 UTC 2020 x86_64 GNU/Linux -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -That I use it enough to run into corner-case issues shows its continued usefulness :) - -[[!meta title="git-annex import follows symlinks"]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_10_30dab036949bd35e026a31974c4e5164._comment b/doc/bugs/git-annex-import_imports_outside_of_directory/comment_10_30dab036949bd35e026a31974c4e5164._comment deleted file mode 100644 index 02b24b3103..0000000000 --- a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_10_30dab036949bd35e026a31974c4e5164._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="Ilya_Shlyakhter" - avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" - subject="imports and symlinks" - date="2022-08-22T20:40:41Z" - content=""" -> the old git-annex import already skipped symlinks to files - -Then the old import was problematic as well :) - -Sorry for harping on this, but since this affects my use cases (and I suspect others' as well), what's the rationale for not doing what `tar` does? -I assumed that an import is similar to doing a `tar` or a `cp -pr`, except that the files get annexed and a connection to the files' origin is recorded. - -For things like song collections the symlinks might not matter, but for datasets they do. E.g. symlinks are used to ensure tools that expect certain filenames at certain paths can find them. The tools are often written by other people, so as a user you might not know all details of these expectations. Files from related analyses are symlinked to keep the relation explicit. -Absolute symlinks are used to point to large shared resources (e.g. genome files) on a filesystem that all users of the repo share. I can give more detailed illustrations if it would help. But I'm not sure I understand the downside of importing symlinks the way tar does. -"""]] diff --git a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_1_376302b0bac94ee81c0dfdcb647f3b3b._comment b/doc/bugs/git-annex-import_imports_outside_of_directory/comment_1_376302b0bac94ee81c0dfdcb647f3b3b._comment deleted file mode 100644 index 56f6972056..0000000000 --- a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_1_376302b0bac94ee81c0dfdcb647f3b3b._comment +++ /dev/null @@ -1,25 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-05-03T16:15:38Z" - content=""" -`git-annex import $dir` also follows symlinks inside $dir. -So importing has been behaving this way since long before the directory -special remote supported importtree. - -This is not a security hole, because if an attacker wants to make you -import `/foo` when importing `/bar`, and they have write access to bar, -they are not limited to making a `/bar/foo -> /foo` symlink. They can just -`cp -a /foo /bar` instead. - -I don't really think it would make much sense for any import to import -symlinks as symlinks. If the symlink points outside the imported directory, -that would result in a symlink that points outside the git repository, -which is not something one often wants to check into a git repository. - -I don't know if I would really consider this a bug either. It at least -seems plausible that there might be users who import from `~/disk` -which is a symlink to `/media/somethinglong`, and rely on it following -the symlink. I often make symlink aliases for mount points like that, -though I have not imported from them. -"""]] diff --git a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_2_e74c71a5574a8e99503ac83a27d48533._comment b/doc/bugs/git-annex-import_imports_outside_of_directory/comment_2_e74c71a5574a8e99503ac83a27d48533._comment deleted file mode 100644 index b27d15a896..0000000000 --- a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_2_e74c71a5574a8e99503ac83a27d48533._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="Ilya_Shlyakhter" - avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" - subject="should import follow symlinks?" - date="2022-05-03T19:36:10Z" - content=""" -I meant \"security hole\" more in the sense of the user themselves inadvertently importing (and then sharing) more than they meant to. -E.g. I was importing a large subtree created by others, and had no clue it included symlinks outside the subtree; I only noticed this by accident, when the import started taking too long. - ->git-annex import $dir also follows symlinks inside $dir. - -In conjunction with the `mv` semantics, this seems risky... had I been using the original directory import, I'd have inadvertently deleted a large dataset (to which there was a symlink in the imported tree) from a place others expect to find it. The unix `mv` command doesn't even have a flag to follow symlinks (nevermind defaulting to that). - ->It at least seems plausible that there might be users who import from ~/disk which is a symlink to /media/somethinglong, and rely on it following the symlink. - -It's certainly plausible; question is, should it be the default? It's not the default in `cp` (you have to use `-L`) or in `tar` (have to use `-h`). I think most people assume that importing from a directory remote is akin to doing a `cp -r` or `tar cf` from it. -"""]] diff --git a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_3_778e176a44aa57e9223aab1a45c931f3._comment b/doc/bugs/git-annex-import_imports_outside_of_directory/comment_3_778e176a44aa57e9223aab1a45c931f3._comment deleted file mode 100644 index 59910a05ec..0000000000 --- a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_3_778e176a44aa57e9223aab1a45c931f3._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="Ilya_Shlyakhter" - avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" - subject="interaction of out-of-tree symlinks with exporttree" - date="2022-05-09T19:21:15Z" - content=""" -One other scenario where the result might not be what users expect is the following: if a [[directory special remote|/special_remotes/directory]] is configured with both [[`importtree=yes`|git-annex-import]] and [[`exporttree=yes`|git-annex-export]], and the directory contains symlinks pointing outside the tree, then an import followed by an export will replace the symlink in the original directory with a copy of the content. - ->a symlink that points outside the git repository, which is not something one often wants to check into a git repository - -Such a symlink is already not something one often has :) But if one does, then the repo is likely for one's own usage, or for the usage by people with access to the shared filesystem where the link works, so adding the link to git as-is makes sense. Logically, it's likely that the out-of-tree link target represents some separate tree of files that you don't think of as part of the tree (or you'd have put them under the tree); if you did want to import them, you'd make a separate repo for them and import them as a submodule. - -Also, what happens if the target tree of the out-of-tree link has a symlink back to the original tree -- could this cause infinite recursion? - -The [[`git-annex-import` man page|git-annex-import]] says the command imports \"a tree of files\". It seems simplest if this description was always strictly true, regardless of what's in the tree. But if you decide to keep the current default, maybe clarify the web page? - -Thanks again for all your work. - -"""]] diff --git a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_4_3406ba56525cb59b8b1b366e2b3dc291._comment b/doc/bugs/git-annex-import_imports_outside_of_directory/comment_4_3406ba56525cb59b8b1b366e2b3dc291._comment deleted file mode 100644 index 0b95faefc5..0000000000 --- a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_4_3406ba56525cb59b8b1b366e2b3dc291._comment +++ /dev/null @@ -1,26 +0,0 @@ -[[!comment format=mdwn - username="skcin" - avatar="http://cdn.libravatar.org/avatar/11a5a31cf0943808f44edb7154d32bd1" - subject="Option to not follow symlinks?" - date="2022-08-18T15:32:43Z" - content=""" -Hey, -I just got bitten by this. Is there an option to not follow symlinks? -If not(and I couldn't find an option), it would be nice to add that. I have no strong opinion on wether it should be a default or not. - -Maybe it is stupid and there is a much better way, but here is what I did. -I decided to move all my data into git annex. I had a total mess of old hard drives with data, backups or both. Some 10+ years old. -So I started to use the directory import with deduplication to import each Hdd into a separate folder of my annex. That worked very well and left me with a much smaller amount of data to organize and empty Hdds ready to wipe+dispose. - -Where things went wrong was when I imported an old external Hdd with a backup of a home directory from 2011 on it. Apparently I had wine installed at that time which had created symlinks in .wine to the following directories. - -* `/` -* `/home/USERNAME` - -Conveniently I use the same username now as I used back then. ;) -I noticed that something was wrong because the import took very long and the KDE desktop started to misbehave. Probably git annex deleted some files KDE needed. I don't know why but at first glance there is not much data missing from my home directory. But now there are a lot of files in my annex that I don't want. - -I think it is not too painful to sort this out but it would have been easy to avoid. My use case for this will probably go away once all the old Hdds are gone. But scenarios like this might not be that rare for new users coming from a world without numcopies and location tracking. - -I have been eyeing git-annex for a couple of years now and finally got around to actually use it. I am very impressed and a huge fan. Thanks for all the work. -"""]] diff --git a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_5_5b41c061786fd49c75d49aa2cc70bb9f._comment b/doc/bugs/git-annex-import_imports_outside_of_directory/comment_5_5b41c061786fd49c75d49aa2cc70bb9f._comment deleted file mode 100644 index b48f19276a..0000000000 --- a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_5_5b41c061786fd49c75d49aa2cc70bb9f._comment +++ /dev/null @@ -1,21 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2022-08-19T17:01:42Z" - content=""" -@skcin I'm very sorry that happened to you. I suppose it's not data loss, -but it sounds like a mess. You should be able to examine `git log` to find -what got imported, and run `git-annex unannex` on it, and then move it back -to the right place. - -Seems like I underestimated the chance this would be a foot bomb. -I now think that git-annex import and the directory special remote should -skip over symlinks. Probably with an informative message to avoid silently -doing nothing in cases where users had been using them intentionally to -follow symlinks. - -Such a check will be race prone, but that is only likely to matter if an -attacker is racing it to replace a file with a symlink, and as I discussed -in previous comments, such an attacker seems like they would be able to -accomplish the same thing with the write permission they must have. -"""]] diff --git a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_6_6d559ba943ef2a8fed285c463eb2a641._comment b/doc/bugs/git-annex-import_imports_outside_of_directory/comment_6_6d559ba943ef2a8fed285c463eb2a641._comment deleted file mode 100644 index e6108a7083..0000000000 --- a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_6_6d559ba943ef2a8fed285c463eb2a641._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 6""" - date="2022-08-19T17:20:24Z" - content=""" -Ok, I've changed `git-annex import` to skip symbolic links inside the -directory being imported. The directory being imported can still itself be -a symlink and that will be followed. -"""]] diff --git a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_6_d6aaa0519faa596fa42d22fba405cbff._comment b/doc/bugs/git-annex-import_imports_outside_of_directory/comment_6_d6aaa0519faa596fa42d22fba405cbff._comment deleted file mode 100644 index a35b735fdb..0000000000 --- a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_6_d6aaa0519faa596fa42d22fba405cbff._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="Ilya_Shlyakhter" - avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" - subject="git-annex-import and symlinks" - date="2022-08-19T17:18:37Z" - content=""" ->git-annex import and the directory special remote should skip over symlinks - -It should still import them as symlinks into git, just not change them to annexed files. If these are relative symlinks pointing within the tree being imported, they'll still work in the repo. If these are absolute symlinks to common files mounted on a filesystem shared by all users of the repo, you still want them in the imported repo as absolute symlinks. Absolute symlinks that happen to point within the tree being imported should be imported as annexed files. - -"""]] diff --git a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_7_4f9816f7c9a61289bef98820b1ec25cc._comment b/doc/bugs/git-annex-import_imports_outside_of_directory/comment_7_4f9816f7c9a61289bef98820b1ec25cc._comment deleted file mode 100644 index 0bc559ab0c..0000000000 --- a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_7_4f9816f7c9a61289bef98820b1ec25cc._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="Ilya_Shlyakhter" - avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" - subject="imports and symlinks" - date="2022-08-19T20:19:50Z" - content=""" -Re: new manpage description that says - ->symbolic links in the directory being imported are skipped to avoid ->accidentially importing things outside the directory that import was ran ->on - -In my use cases, skipping symlinks (instead of importing them as git symlinks) would make the imported tree unusable by tools that expect specific filenames in specific subdirs of the tree, -when these filenames were symlinks in the original tree. -Could the symlinks be imported as standard git symlinks instead of skipping them? Worst case, a checkout of the repo will have symlinks to non-existing targets -- this can be fixed by mounting volumes with the right paths. OTOH, having filenames missing from the imported tree because they happened to be symlinks in the original tree can cause all sorts of errors. Import is much simpler to think about if it's guaranteed to replicate the full original tree structure, like `tar` does. -"""]] diff --git a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_9_372c44296c1a03778e201a47ec45bee6._comment b/doc/bugs/git-annex-import_imports_outside_of_directory/comment_9_372c44296c1a03778e201a47ec45bee6._comment deleted file mode 100644 index 1394031592..0000000000 --- a/doc/bugs/git-annex-import_imports_outside_of_directory/comment_9_372c44296c1a03778e201a47ec45bee6._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 9""" - date="2022-08-22T15:58:12Z" - content=""" -@Ilya the old git-annex import already skipped symlinks to files. -(Along with not importing sockets, device files, etc.) -It only followed symlinks to directories. -"""]] diff --git a/doc/bugs/git-annex-move_not_using_parallelism__63__.mdwn b/doc/bugs/git-annex-move_not_using_parallelism__63__.mdwn deleted file mode 100644 index ad90d1468d..0000000000 --- a/doc/bugs/git-annex-move_not_using_parallelism__63__.mdwn +++ /dev/null @@ -1,15 +0,0 @@ -### Please describe the problem. -git-annex-move does not seem to be using parallelism. i'm running -(master_env_py27_v28) [02:30 PM ~]$ ps -w -w -w -F 12231 -UID PID PPID C SZ RSS PSR STIME TTY STAT TIME CMD -ilya 12231 12230 99 271873062 1018700 26 14:15 pts/2 Sl+ 506:13 /data/vngs/viral-ngs-etc/mc3/envs/master_env_py27_v28/bin/git-annex move -J48 --all --to ilya-s3 - -and there is only one thread with git-annex. - On what operating system? - -git-annex version: 6.20180926-gc906aaf build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify ConcurrentOutput TorrentParser MagicMime Feed\ s Testsuite dependency versions: aws-0.17.1 bloomfilter-2.0.1.0 cryptonite-0.23 DAV-1.3.1 feed-0.3.12.0 ghc-8.0.2 http-client-0.5.7.0 persistent-s\ qlite-2.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.4.5 key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_2\ 24 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE\ 2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256\ BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external operating system: linux x86_64 supported repository versions: 3 5 6 upgrade supported from repository versions: 0 1 2 3 4 5 local repository version: 5 - -(master_env_py27_v28) [01:05 PM /data/ilya-work]$ uname -a Linux ip-172-31-87-156 4.14.72-68.55.amzn1.x86_64 #1 SMP Fri Sep 28 21:14:54 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux - -> While this might be a bug, there is not enough information to know, and -> so after 3 years, I'm going to close it. [[done]] --[[Joey]] diff --git a/doc/bugs/git-annex-move_not_using_parallelism__63__/comment_1_1080a771213dc524cbd072591030c98c._comment b/doc/bugs/git-annex-move_not_using_parallelism__63__/comment_1_1080a771213dc524cbd072591030c98c._comment deleted file mode 100644 index 6940ad8bb5..0000000000 --- a/doc/bugs/git-annex-move_not_using_parallelism__63__/comment_1_1080a771213dc524cbd072591030c98c._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2018-10-29T19:05:21Z" - content=""" -Do you see git-annex displaying multiple concurrent actions at the same -time? - -Is annex.pidlock set? - -You mention threads, which makes me think you may be examining the -processes's OS threads. But haskell uses green threads. Perhaps I should -not try to guess what you are talking about with the mention of "threads", -but the bug report is very much not clear about what the problem looks -like or how to reproduce it, and I am grasping at straws. -"""]] diff --git a/doc/bugs/git-annex-move_not_using_parallelism__63__/comment_2_a83e833b329df094e1bc7945db9686b0._comment b/doc/bugs/git-annex-move_not_using_parallelism__63__/comment_2_a83e833b329df094e1bc7945db9686b0._comment deleted file mode 100644 index 7bceb5124c..0000000000 --- a/doc/bugs/git-annex-move_not_using_parallelism__63__/comment_2_a83e833b329df094e1bc7945db9686b0._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Ilya_Shlyakhter" - avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" - subject="comment 2" - date="2018-10-29T21:10:29Z" - content=""" -\"Do you see git-annex displaying multiple concurrent actions at the same time?\" -- displaying where? I was just looking at 'top', which seemed to be showing only one git-annex process, and showing that the processors are mostly idle. -"""]] diff --git a/doc/bugs/git-annex-move_not_using_parallelism__63__/comment_3_bdd1aabe7120c4e6b5999aaf396067bc._comment b/doc/bugs/git-annex-move_not_using_parallelism__63__/comment_3_bdd1aabe7120c4e6b5999aaf396067bc._comment deleted file mode 100644 index 483b31558c..0000000000 --- a/doc/bugs/git-annex-move_not_using_parallelism__63__/comment_3_bdd1aabe7120c4e6b5999aaf396067bc._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="andrew" - avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435" - subject="comment 3" - date="2018-11-12T03:17:47Z" - content=""" -I would try adding `--debug`, so `git-annex move -J48 --all --to ilya-s3 --debug` that should give you some insight into the number of simultaneous connections. Seems to be moving concurrently for me, I tried on an s3 remote and a googledrive remote, I saw multiple interleaved uploads in the debug logs as well as multiple simultaneous `git … cat-file` in my processes. -"""]] diff --git a/doc/bugs/git-annex-repair_claims_success_then_failure.mdwn b/doc/bugs/git-annex-repair_claims_success_then_failure.mdwn deleted file mode 100644 index e08d986593..0000000000 --- a/doc/bugs/git-annex-repair_claims_success_then_failure.mdwn +++ /dev/null @@ -1,52 +0,0 @@ -Hello. - -What does this log mean? It seems to tell" "success", then "openDirStream" fails, then "1 failed". What failed? - -Context is in [todo/git annex repair: performance can be abysmal, huge improvements possible](https://git-annex.branchable.com/ikiwiki.cgi?do=goto&page=todo%2Fgit_annex_repair__58___performance_can_be_abysmal__44___huge_improvements_possible) - - fatal: bad object refs/heads/git-annex - fatal: bad object refs/heads/git-annex - fatal: bad object refs/heads/git-annex - error: Could not read somehashA - fatal: Failed to traverse parents of commit somehashB - error: Could not read somehashA - fatal: Failed to traverse parents of commit somehashB - error: Could not read somehashA - fatal: Failed to traverse parents of commit somehashB - error: Could not read somehashA - fatal: Failed to traverse parents of commit somehashB - Deleted these local branches, which could not be recovered due to missing objects: - refs/heads/master - refs/heads/git-annex - You currently have refs/heads/master checked out. You may have staged changes in the index that can be committed to recover the lost state of this branch! - Successfully recovered repository! - Please carefully check that the changes mentioned above are ok.. - - git-annex: .git/annex/journal/: openDirStream: does not exist (No such file or directory) - failed - git-annex: repair: 1 failed - -The fact is: this repo is a plain git clone of a git annex repository. - -There is no `.git/annex` directory there before `git-annex-repair` is run. - -After it ran, there is a `.git/annex` directory with that content: - - total 24 - drwxrwxr-x 3 4096 Jul 22 15:41 . - drwxrwxr-x 9 4096 Jul 23 07:24 .. - -rw-rw-r-- 1 65 Jul 20 11:59 index - -rw-rw-r-- 1 41 Jul 20 11:59 index.lck - -rw-rw-r-- 1 0 Jul 22 15:41 journal.lck - -rw-rw-r-- 1 211 Jul 20 11:59 mergedrefs - drwxrwxr-x 2 4096 Jul 22 15:41 misctmp - -Perhaps git-annex-repair gets confused when recovering a repository that is a plain git clone of a git annex repository? - -I did that because annexed objects are 1.7TB big here, so I wanted a local copy of pure git part only to perform repair of the repo, and propagate things somehow the objects at a later stage. - -I'll keep the repo lying around for a few days, maybe weeks, if some experiment or further feedback is needed. - -Thank you for your attention. - -> [[fixed|done]].. sorry for the delay and thanks for reporting --[[Joey]] diff --git a/doc/bugs/git-annex-repair_claims_success_then_failure/comment_1_de8dfda0bc321e573ab9d17aec1ecb84._comment b/doc/bugs/git-annex-repair_claims_success_then_failure/comment_1_de8dfda0bc321e573ab9d17aec1ecb84._comment deleted file mode 100644 index 1f3c26a3b2..0000000000 --- a/doc/bugs/git-annex-repair_claims_success_then_failure/comment_1_de8dfda0bc321e573ab9d17aec1ecb84._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-06-21T18:53:31Z" - content=""" -I was able to reproduce this bug! - - git-annex init - rm -rf .git/annex - git-annex repair - - git-annex: .git/annex/journal/: openDirStream: does not exist (No such file or directory) - -That's with Annex.Repair modified to always run commitindex. Having a -repository with a git-annex branch that is corrupt, would have the same effect -as that modification. - -Fixed this. -"""]] diff --git a/doc/bugs/git-annex-shell_-c_git-annex-shell_doesn__39__t_work__44___while_git-annex_expects_it_to.mdwn b/doc/bugs/git-annex-shell_-c_git-annex-shell_doesn__39__t_work__44___while_git-annex_expects_it_to.mdwn deleted file mode 100644 index 53124ee040..0000000000 --- a/doc/bugs/git-annex-shell_-c_git-annex-shell_doesn__39__t_work__44___while_git-annex_expects_it_to.mdwn +++ /dev/null @@ -1,29 +0,0 @@ -### Please describe the problem. -git-annex-shell -c git-annex-shell errors, and git-annex interprets this as meaning that git-annex-shell is normal git-shell. - -### What steps will reproduce the problem? -Running git-annex-shell -c git-annex-shell. - -### What version of git-annex are you using? On what operating system? -NixOS, 6.20180529. - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - -[leo60228@nixos:~/test]$ git-annex-shell -c git-annex-shell -fatal: unrecognized command 'git-annex-shell' -git-annex-shell: git-shell failed - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) -Yes. - -[[moreinfo]] - -> Gonna assume this was operator error after 5 years of no information. -> [[done]] --[[Joey]] diff --git a/doc/bugs/git-annex-shell_-c_git-annex-shell_doesn__39__t_work__44___while_git-annex_expects_it_to/comment_1_b0a4f459d40bca7a0b24d1bec1e0f4a8._comment b/doc/bugs/git-annex-shell_-c_git-annex-shell_doesn__39__t_work__44___while_git-annex_expects_it_to/comment_1_b0a4f459d40bca7a0b24d1bec1e0f4a8._comment deleted file mode 100644 index de19505f5e..0000000000 --- a/doc/bugs/git-annex-shell_-c_git-annex-shell_doesn__39__t_work__44___while_git-annex_expects_it_to/comment_1_b0a4f459d40bca7a0b24d1bec1e0f4a8._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2018-08-03T18:18:34Z" - content=""" -Why are you trying to do this? Is there a normal git-annex use case that -involves that happening? -"""]] diff --git a/doc/bugs/git-annex-shell_-c_git-annex-shell_doesn__39__t_work__44___while_git-annex_expects_it_to/comment_3_335fea47cef83b6dfad7ce8c53193f82._comment b/doc/bugs/git-annex-shell_-c_git-annex-shell_doesn__39__t_work__44___while_git-annex_expects_it_to/comment_3_335fea47cef83b6dfad7ce8c53193f82._comment deleted file mode 100644 index 7140359949..0000000000 --- a/doc/bugs/git-annex-shell_-c_git-annex-shell_doesn__39__t_work__44___while_git-annex_expects_it_to/comment_3_335fea47cef83b6dfad7ce8c53193f82._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2018-08-08T15:04:09Z" - content=""" -In what configuration? - -This clearly does not affect the general case, or everyone who is happily -using git-annex would have reported the bug. - -Please provide enough information to reproduce your problem. -"""]] diff --git a/doc/bugs/git-annex-shell_-c_git-annex-shell_doesn__39__t_work__44___while_git-annex_expects_it_to/comment_3_abaeb3d5c0322deeec790d80b2948549._comment b/doc/bugs/git-annex-shell_-c_git-annex-shell_doesn__39__t_work__44___while_git-annex_expects_it_to/comment_3_abaeb3d5c0322deeec790d80b2948549._comment deleted file mode 100644 index a02a758689..0000000000 --- a/doc/bugs/git-annex-shell_-c_git-annex-shell_doesn__39__t_work__44___while_git-annex_expects_it_to/comment_3_abaeb3d5c0322deeec790d80b2948549._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="https://openid-provider.appspot.com/iakornfeld" - nickname="iakornfeld" - avatar="http://cdn.libravatar.org/avatar/c0369f5727cad81d1ecf6c2e657b42a1b756284aad0229351f9027a2cfcb2037" - subject="Yes" - date="2018-08-07T14:07:56Z" - content=""" -git-annex does this when checking remotes for git-annex support. -"""]] diff --git a/doc/bugs/git-annex-shell_fails_to_run_true.mdwn b/doc/bugs/git-annex-shell_fails_to_run_true.mdwn deleted file mode 100644 index 230a94a340..0000000000 --- a/doc/bugs/git-annex-shell_fails_to_run_true.mdwn +++ /dev/null @@ -1,166 +0,0 @@ -### Please describe the problem. - -I have what I speculate is an interoperability problem with two git-annex versions: on one end, it's trying to run `true` over SSH and on the other end `git-annex-shell` is saying that's not a valid command. - -I have a rather gnarly git-remote-gcrypt setup (is there one that isn't?) with a gcrypt remote for the git history and annex content. I honestly barely remember how I set this up, and I'm constantly battling to remember how it works (not great, I know), but here are the notes I have so far: - -https://anarc.at/services/backup/#encrypted-remotes - -Obviously, I have tried to follow the [[tips/fully_encrypted_git_repositories_with_gcrypt/]] tutorial, but I might have drifted off at some point. I often get confused between the builtin [[encryption]] features and the [[special_remotes/gcrypt]] special remote. - -### What steps will reproduce the problem? - -I think the trick is to setup a remote gcrypt repo with a hardcoded `command="GIT_ANNEX_SHELL_APPENDONLY=true git-annex-shell -c \"$SSH_ORIGINAL_COMMAND\""` entry in `authorized_keys`. If I drop the `command` parameter from the line, the command goes through fine. - -### What version of git-annex are you using? On what operating system? - -The "client" end is 8.20210223-2 on Debian bullseye, the remote is 7.20190129-3 on Debian buster. - -### Please provide any additional information below. - - I suspect that something was introduced after the buster release that runs this extra command on start: - - ssh remote true - -Which SSH turns into: - - ssh remote git-annex-shell -c true - -Even with a non-restricted SSH key, the above fails with: - - fatal: unrecognized command 'true' - git-annex-shell: git-shell failed - -... which is, for me, the smoking gun here. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - -anarcat@curie:maison(master)$ LANG=C.UTF-8 git annex sync --content -J2 --debug -[2021-08-19 10:39:04.710287879] process [2036863] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"] -[2021-08-19 10:39:04.71270528] process [2036863] done ExitSuccess -[2021-08-19 10:39:04.713060458] process [2036864] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"] -[2021-08-19 10:39:04.715240868] process [2036864] done ExitSuccess -[2021-08-19 10:39:04.71580494] process [2036865] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/git-annex..0134b426eaca4932d9bcc381bcb2c62609563651","--pretty=%H","-n1"] -[2021-08-19 10:39:04.71856682] process [2036865] done ExitSuccess -[2021-08-19 10:39:04.719607238] process [2036868] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"] -[2021-08-19 10:39:04.720334517] process [2036869] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -[2021-08-19 10:39:04.738790271] process [2036872] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","commit","-a","-m","git-annex in anarcat@curie:/srv/video/maison"] -On branch master -nothing to commit, working tree clean -[2021-08-19 10:39:04.770958097] process [2036872] done ExitFailure 1 -[2021-08-19 10:39:04.771335463] process [2036893] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"] -commit ok -[2021-08-19 10:39:04.773190824] process [2036893] done ExitSuccess -[2021-08-19 10:39:04.773504032] process [2036895] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"] -[2021-08-19 10:39:04.775743665] process [2036895] done ExitSuccess -[2021-08-19 10:39:04.776052102] process [2036896] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--verify","-q","refs/heads/synced/master"] -[2021-08-19 10:39:04.777934271] process [2036896] done ExitSuccess -[2021-08-19 10:39:04.778257724] process [2036897] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/master..refs/heads/synced/master","--pretty=%H","-n1"] -[2021-08-19 10:39:04.781046264] process [2036897] done ExitSuccess -[2021-08-19 10:39:04.781665586] process [2036898] read: ssh ["-o","BatchMode=true","-S",".git/annex/ssh/anarc.at","-o","ControlMaster=auto","-o","ControlPersist=yes","-n","-T","anarc.at","true"] -[2021-08-19 10:39:05.937301502] process [2036898] done ExitSuccess -[2021-08-19 10:39:05.938577543] process [2036926] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","fetch","origin"] -[2021-08-19 10:39:06.108005272] process [2036926] done ExitSuccess -[2021-08-19 10:39:06.108743728] process [2036944] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","branch","-f","synced/master","refs/heads/master"] -[2021-08-19 10:39:06.113172855] process [2036944] done ExitSuccess -[2021-08-19 10:39:06.113913765] process [2036945] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--verify","-q","refs/remotes/origin/master"] -[2021-08-19 10:39:06.118235086] process [2036945] done ExitSuccess -[2021-08-19 10:39:06.118916295] process [2036946] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/master..refs/remotes/origin/master","--pretty=%H","-n1"] -[2021-08-19 10:39:06.124140634] process [2036946] done ExitSuccess -[2021-08-19 10:39:06.124791141] process [2036947] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--verify","-q","refs/remotes/origin/synced/master"] -[2021-08-19 10:39:06.128491407] process [2036947] done ExitSuccess -[2021-08-19 10:39:06.129062268] process [2036948] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/synced/master..refs/remotes/origin/synced/master","--pretty=%H","-n1"] -[2021-08-19 10:39:06.134308196] process [2036948] done ExitSuccess -pull origin ok -[2021-08-19 10:39:06.135431262] process [2036949] read: ssh ["-o","BatchMode=true","-S",".git/annex/ssh/anarcat@remote-annex","-o","ControlMaster=auto","-o","ControlPersist=yes","-n","-T","anarcat@remote-annex","true"] -[2021-08-19 10:39:06.5226911] process [2036949] done ExitFailure 1 -[2021-08-19 10:39:06.524455991] process [2036956] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","fetch","offsite-git"] -git-annex-shell: git-shell failed -rsync: connection unexpectedly closed (0 bytes received so far) [Receiver] -rsync error: error in rsync protocol data stream (code 12) at io.c(228) [Receiver=3.2.3] -gcrypt: Repository not found: rsync://anarcat@remote-annex:/home/anarcat/offsite/Videos.git/ -gcrypt: ..but repository ID is set. Aborting. -[2021-08-19 10:39:07.057455548] process [2036956] done ExitFailure 128 -pull offsite-git ok -[2021-08-19 10:39:07.058197157] process [2037001] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"] -[2021-08-19 10:39:07.063254024] process [2037001] done ExitSuccess -[2021-08-19 10:39:07.064018134] process [2037002] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"] -[2021-08-19 10:39:07.069653083] process [2037002] done ExitSuccess -[2021-08-19 10:39:07.07062786] process [2037003] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/git-annex..0134b426eaca4932d9bcc381bcb2c62609563651","--pretty=%H","-n1"] -[2021-08-19 10:39:07.076284072] process [2037003] done ExitSuccess -[2021-08-19 10:39:07.077172323] process [2037004] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--stage","-z","--"] -[2021-08-19 10:39:07.078079618] process [2037005] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)","--buffer"] -[2021-08-19 10:39:07.078694043] process [2037006] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"] -[2021-08-19 10:39:07.079965189] process [2037007] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"] -[2021-08-19 10:39:07.103420306] process [2037009] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"] -[2021-08-19 10:39:07.107225054] process [2037010] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -[2021-08-19 10:39:07.108049882] process [2037011] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","check-attr","-z","--stdin","annex.backend","annex.largefiles","annex.numcopies","annex.mincopies","--"] -[2021-08-19 10:39:07.110542203] process [2037012] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","check-attr","-z","--stdin","annex.backend","annex.largefiles","annex.numcopies","annex.mincopies","--"] -[2021-08-19 10:39:07.365098056] process [2037013] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"] -[2021-08-19 10:39:07.365245774] process [2037011] done ExitSuccess -[2021-08-19 10:39:07.365363864] process [2037014] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"] -[2021-08-19 10:39:07.36572593] process [2037015] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -[2021-08-19 10:39:07.36730392] process [2037012] done ExitSuccess -[2021-08-19 10:39:07.367432763] process [2037016] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","check-attr","-z","--stdin","annex.backend","annex.largefiles","annex.numcopies","annex.mincopies","--"] -[2021-08-19 10:39:07.367787144] process [2037007] done ExitSuccess -[2021-08-19 10:39:07.367891318] process [2037017] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -[2021-08-19 10:39:07.368118899] process [2037006] done ExitSuccess -[2021-08-19 10:39:07.368213522] process [2037005] done ExitSuccess -[2021-08-19 10:39:07.368287754] process [2037004] done ExitSuccess -[2021-08-19 10:39:07.369558345] process [2037018] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","check-attr","-z","--stdin","annex.backend","annex.largefiles","annex.numcopies","annex.mincopies","--"] -[2021-08-19 10:39:07.370572764] process [2037019] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"] -[2021-08-19 10:39:07.371432055] process [2037020] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -[2021-08-19 10:39:07.374759264] process [2037021] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"] -[2021-08-19 10:39:07.375304372] process [2037022] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -[2021-08-19 10:39:07.378100534] process [2037023] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","branch","-f","synced/master","refs/heads/master"] -[2021-08-19 10:39:07.380269305] process [2037023] done ExitSuccess -[2021-08-19 10:39:07.380584318] process [2037024] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"] -[2021-08-19 10:39:07.38084529] process [2037025] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"] -[2021-08-19 10:39:07.383152906] process [2037025] done ExitSuccess -[2021-08-19 10:39:07.383152909] process [2037024] done ExitSuccess -[2021-08-19 10:39:07.383556344] process [2037026] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"] -[2021-08-19 10:39:07.38382986] process [2037027] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"] -[2021-08-19 10:39:07.385869655] process [2037026] done ExitSuccess -[2021-08-19 10:39:07.385998224] process [2037027] done ExitSuccess -[2021-08-19 10:39:07.386579545] process [2037028] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--verify","-q","refs/remotes/origin/synced/master"] -[2021-08-19 10:39:07.386748615] process [2037029] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--verify","-q","refs/remotes/offsite-git/synced/master"] -[2021-08-19 10:39:07.388481211] process [2037028] done ExitSuccess -[2021-08-19 10:39:07.388627634] process [2037029] done ExitSuccess -[2021-08-19 10:39:07.389087617] process [2037030] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/remotes/origin/synced/master..refs/heads/synced/master","--pretty=%H","-n1"] -[2021-08-19 10:39:07.389447787] process [2037031] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/remotes/offsite-git/synced/master..refs/heads/synced/master","--pretty=%H","-n1"] -[2021-08-19 10:39:07.391475061] process [2037030] done ExitSuccess -[2021-08-19 10:39:07.391885609] process [2037032] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--verify","-q","refs/remotes/origin/git-annex"] -[2021-08-19 10:39:07.392072048] process [2037031] done ExitSuccess -[2021-08-19 10:39:07.392422194] process [2037033] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--verify","-q","refs/remotes/offsite-git/git-annex"] -[2021-08-19 10:39:07.393853689] process [2037032] done ExitSuccess -[2021-08-19 10:39:07.394223125] process [2037034] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/remotes/origin/git-annex..git-annex","--pretty=%H","-n1"] -[2021-08-19 10:39:07.394252126] process [2037033] done ExitSuccess -[2021-08-19 10:39:07.394616572] process [2037035] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/remotes/offsite-git/git-annex..git-annex","--pretty=%H","-n1"] -[2021-08-19 10:39:07.396719833] process [2037034] done ExitSuccess -[2021-08-19 10:39:07.396989352] process [2037035] done ExitSuccess -[2021-08-19 10:39:07.397288729] process [2037019] done ExitSuccess -[2021-08-19 10:39:07.397649558] process [2037020] done ExitSuccess -[2021-08-19 10:39:07.397943289] process [2037021] done ExitSuccess -[2021-08-19 10:39:07.398193463] process [2037022] done ExitSuccess -[2021-08-19 10:39:07.398430683] process [2037018] done ExitSuccess -[2021-08-19 10:39:07.398671489] process [2037016] done ExitSuccess -[2021-08-19 10:39:07.399335705] process [2037036] read: ssh ["-O","stop","-S","anarcat@remote-annex","-o","ControlMaster=auto","-o","ControlPersist=yes","localhost"] -[2021-08-19 10:39:07.405590596] process [2037036] done ExitSuccess -[2021-08-19 10:39:07.406058089] process [2037037] read: ssh ["-O","stop","-S","anarc.at","-o","ControlMaster=auto","-o","ControlPersist=yes","localhost"] -[2021-08-19 10:39:07.412095481] process [2037037] done ExitSuccess - -# End of transcript or log. -"""]] - -Key part above: - - [2021-08-19 10:39:06.135431262] process [2036949] read: ssh ["-o","BatchMode=true","-S",".git/annex/ssh/anarcat@remote-annex","-o","ControlMaster=auto","-o","ControlPersist=yes","-n","-T","anarcat@remote-annex","true"] - [2021-08-19 10:39:06.5226911] process [2036949] done ExitFailure 1 - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -I'm a long time git-annex user (early adopter, even?) and git-annex is generally serving my uses very well, both as a backup and archival system. The encryption features, however, are a bit more obscure than my tolerance level right now. Thankfully, there is the [[special_remotes/borg/]] I'm thinking of using to replace all this complexity in the future... - --- [[done]], problem was with a configuration in the gcrypt remote sandbox. -- [[anarcat]] diff --git a/doc/bugs/git-annex-shell_fails_to_run_true/comment_1_7573eb95bdecc3ecc9b31599bd309d3d._comment b/doc/bugs/git-annex-shell_fails_to_run_true/comment_1_7573eb95bdecc3ecc9b31599bd309d3d._comment deleted file mode 100644 index 431e869990..0000000000 --- a/doc/bugs/git-annex-shell_fails_to_run_true/comment_1_7573eb95bdecc3ecc9b31599bd309d3d._comment +++ /dev/null @@ -1,31 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2021-08-24T16:28:40Z" - content=""" -git-annex-shell is not supposed to run true. That would be a security hole -if it did. - -However, what's actually being done here is that git-annex is probing -to see if it can ssh without a password prompt. It needs to run some -command in the probe, without doing anything, so it uses true. -This code was written with the knowledge that a restricted shell might not -allow true to be run. It doesn't care if ssh exits 1 in such a situation. -The only exit status it cares about is 255, which is what ssh uses if it -was unable to ssh to the host due to needing a password. - -So, your smoking gun is actually a red herring. - -Your actual failure may be here: - - ["--git-dir=.git","--work-tree=.","--literal-pathspecs","fetch","offsite-git"] - git-annex-shell: git-shell failed - rsync: connection unexpectedly closed (0 bytes received so far) [Receiver] - rsync error: error in rsync protocol data stream (code 12) at io.c(228) [Receiver=3.2.3] - gcrypt: Repository not found: rsync://anarcat@remote-annex:/home/anarcat/offsite/Videos.git/ - gcrypt: ..but repository ID is set. Aborting. - -That looks like something is set up incorrectly, but I don't really know what. -It seems to prevent `git fetch` from working though, so currently it does not look -like a git-annex problem. -"""]] diff --git a/doc/bugs/git-annex-shell_fails_to_run_true/comment_2_5be34b5ffaee39013d24daf83a8dd437._comment b/doc/bugs/git-annex-shell_fails_to_run_true/comment_2_5be34b5ffaee39013d24daf83a8dd437._comment deleted file mode 100644 index a9d422e054..0000000000 --- a/doc/bugs/git-annex-shell_fails_to_run_true/comment_2_5be34b5ffaee39013d24daf83a8dd437._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="anarcat" - avatar="http://cdn.libravatar.org/avatar/4ad594c1e13211c1ad9edb81ce5110b7" - subject="indeed." - date="2022-01-12T15:56:25Z" - content=""" -the problem was that i had a `~/git-shell-commands/rsync` overriding the rsync command to try to restrict *that*, since i'm using a gcrypt remote... it was clunky and badly designed and, ultimately, insecure. i replaced it with `rrsync /home/anarcat/offsite/` but even that is not the best either, as the script says it \"assumes someone will not subvert the rsync protocol\" which hardly seems reassuring. - -i also had to change my remotes to follow the \"chroot\" of sorts: - - git config remote.offsite-annex.annex-rsyncurl anarcat@remote-annex:Videos.annex/ - git remote set-url offsite-git gcrypt::rsync://anarcat@remote-annex:Videos.git/ - -Running `git fetch` by hand convinced me the problem wasn't git-annex related... From there I reread the git-annex-shell and git-shell commands and noticed that bit about the obscure `git-shell-commands` directory, and found that jem. I even had an error in `journalctl -f` that confirm the problem, something i overlooked as well... - -so this indeed seems like a problem on my end. sorry for the delay and the noise! -"""]] diff --git a/doc/bugs/git-annex_drop_fails_to_access_file__58____47____47____47___target_URL_on_Windows.mdwn b/doc/bugs/git-annex_drop_fails_to_access_file__58____47____47____47___target_URL_on_Windows.mdwn deleted file mode 100644 index 6f46c6f6ea..0000000000 --- a/doc/bugs/git-annex_drop_fails_to_access_file__58____47____47____47___target_URL_on_Windows.mdwn +++ /dev/null @@ -1,31 +0,0 @@ -### Please describe the problem. - -If I addurl file pointing to file:///C:/... it seems to work just fine, but then refuses to drop it stating that can't verify its presence. - -Please see two screenshots (sorry for not cut/paste here since it was already was painful -- debugging under Virtualbox in remote vnc through debconf internet which for some reason drops for me quite often): - -http://www.onerussian.com/tmp/gkrellShoot_08-19-15_220150.png -http://www.onerussian.com/tmp/gkrellShoot_08-19-15_184052.png - -as screenshot shows, apparently wget is clueless about file:// targets on windows (while curl does fine) -- may be related ;-) - -### What steps will reproduce the problem? - -addurl some file:///C:/ under windows pointing to existing file, and then try to drop that annexed file - -### What version of git-annex are you using? On what operating system? - -windows -20150805 or so - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - - -# End of transcript or log. -"""]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/git-annex_drop_fails_to_access_file__58____47____47____47___target_URL_on_Windows/comment_1_f0d30a953f072f8d9a929a4a6ba69914._comment b/doc/bugs/git-annex_drop_fails_to_access_file__58____47____47____47___target_URL_on_Windows/comment_1_f0d30a953f072f8d9a929a4a6ba69914._comment deleted file mode 100644 index c7de2eb6db..0000000000 --- a/doc/bugs/git-annex_drop_fails_to_access_file__58____47____47____47___target_URL_on_Windows/comment_1_f0d30a953f072f8d9a929a4a6ba69914._comment +++ /dev/null @@ -1,48 +0,0 @@ -[[!comment format=mdwn - username="https://me.yahoo.com/a/EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s-#7c0f4" - subject="seems just ignore errors while adding urls to "unsupported" urls" - date="2015-08-19T21:59:09Z" - content=""" -actually situation is somewhat similar on linux as well in a sense that annex manages to addurl a file using e.g. file:///./data url (not sure even if legit) without puking but contains wrong content (empty): - -[[!format sh \"\"\" - -% mkdir XXX -% cd XXX -% git init; git annex init -Initialized empty Git repository in /tmp/XXX/.git/ -init ok -(recording state in git...) -% echo 123 > data -% git annex addurl --file=annexed file:///./data -addurl annexed (downloading file:///./data ...) - -ok -(recording state in git...) -% cat annexed -% git annex drop annexed -drop annexed (checking file:///./data...) (unsafe) - Could only verify the existence of 0 out of 1 necessary copies - - Rather than dropping this file, try using: git annex move - - (Use --force to override this check, or adjust numcopies.) -failed -git-annex: drop: 1 failed -% git annex addurl --file=annexedfull file://$PWD/data -addurl annexedfull (downloading file:///tmp/XXX/data ...) -######################################################################## 100.0% -ok -(recording state in git...) -% cat annexedfull -123 -% git annex drop annexedfull -drop annexedfull (checking file:///tmp/XXX/data...) ok -(recording state in git...) -% annex version -zsh: command not found: annex -% git annex version -git-annex version: 5.20150812-2 - -\"\"\"]] -"""]] diff --git a/doc/bugs/git-annex_drop_fails_to_access_file__58____47____47____47___target_URL_on_Windows/comment_2_504ea07f798838710cdbf6133135c815._comment b/doc/bugs/git-annex_drop_fails_to_access_file__58____47____47____47___target_URL_on_Windows/comment_2_504ea07f798838710cdbf6133135c815._comment deleted file mode 100644 index 5097e2e16a..0000000000 --- a/doc/bugs/git-annex_drop_fails_to_access_file__58____47____47____47___target_URL_on_Windows/comment_2_504ea07f798838710cdbf6133135c815._comment +++ /dev/null @@ -1,39 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2015-09-09T16:36:13Z" - content=""" -I can't reproduce that behavior on linux. - - joey@darkstar:~/tmp/xx>git annex addurl --file=annexed file:///./data - addurl annexed (downloading file:///./data ...) - curl: (37) Couldn't open file /data - -Here, curl seems to be doing the right thing; the url is not relative; it's -for `/./data`, which doesn't exist. - -Relative `file:` urls shouldn't be valid at all, I think? - ---- - -For checking if a file: url exits, git-annex parses the url and stats -the file itself. The first screenshot -shows this check for file: url existance failing on Windows for -the url `file:///C:/tmp/test/test.dat` - -I guess this might come down to problems with parsing file: urls on -Windows; seems especially complicated by drive letters. git-annex and curl -seem to parse this url in different ways. - -Checking how that url parses, the uriScheme is "file:" and the uriPath is -"/C:/tmp/test/test.dat". So, it seems clear why it fails to stat that file. - -Is there actually a valid way to produce a file: url that refers to a drive -letter? curl seems to think so, since it found the file when `git annex -addurl` ran it. I don't know if the above parse is valid, but it's not -git-annex's code doing the parse, but the URI parsing library. - -(Possibly related bug report: - -) -"""]] diff --git a/doc/bugs/git-annex_drop_fails_to_access_file__58____47____47____47___target_URL_on_Windows/comment_3_a9bd50ebe805afefab103da31b88cf89._comment b/doc/bugs/git-annex_drop_fails_to_access_file__58____47____47____47___target_URL_on_Windows/comment_3_a9bd50ebe805afefab103da31b88cf89._comment deleted file mode 100644 index 3e3f19d19d..0000000000 --- a/doc/bugs/git-annex_drop_fails_to_access_file__58____47____47____47___target_URL_on_Windows/comment_3_a9bd50ebe805afefab103da31b88cf89._comment +++ /dev/null @@ -1,20 +0,0 @@ -[[!comment format=mdwn - username="mih" - avatar="http://cdn.libravatar.org/avatar/f881df265a423e4f24eff27c623148fd" - subject="file:/// must be forced to file:// on windows" - date="2023-01-27T09:19:10Z" - content=""" -I just want to leave a note that the `file:` URL scheme handling on windows is still problematic. A file URL (e.g. generated by Python's `pathlib.as_uri()`) like the following does not work: - -``` -file:///C:/DLTMP/myarchive.zip -``` - -it is parsed to the path `/C:/DLTMP/myarchive.zip` (as reported by OP), which is invalid. - -A workaround is to replace `file:///` with `file://` (remove one slash). This makes git-annex accept the URL and parse it to the correct path `C:/DLTMP/myarchive.zip`. - -However, such a \"double-slash\" URL is invalid. [Wikipedia](https://en.wikipedia.org/wiki/File_URI_scheme) has a dedicated bit on \"How many slashes?\" which states - -> file://path (i.e. two slashes, without a hostname) is never correct, but is often used -"""]] diff --git a/doc/bugs/git-annex_drop_fails_to_access_file__58____47____47____47___target_URL_on_Windows/comment_4_f2bd555ca36b42dac6a213e7c947f1f9._comment b/doc/bugs/git-annex_drop_fails_to_access_file__58____47____47____47___target_URL_on_Windows/comment_4_f2bd555ca36b42dac6a213e7c947f1f9._comment deleted file mode 100644 index df17624c7b..0000000000 --- a/doc/bugs/git-annex_drop_fails_to_access_file__58____47____47____47___target_URL_on_Windows/comment_4_f2bd555ca36b42dac6a213e7c947f1f9._comment +++ /dev/null @@ -1,32 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2023-03-13T18:54:33Z" - content=""" -Wikipedia seems to be wrong about that. -I took a quick look at -and it says that "file:c:/path/to/file" is a valid URI on Windows. And it will be -parsed ok by git-annex. So you could just use those. - -The RFC does say that "file:///c:/path/to/file" should also be supported. -(Though I don't understand its reference to the "path-absolute" rule.) - -I don't know if network-uri could be made to support that, it seems that -it would have to handle windows and non-windows differently. Because on linux, -"file:///c:/path/to/file" should parse to a path "/c:/path/to/file", -which is after all a valid path if you choose to have a `/c:` directory! - -But network-uri is a pure uri parser and it does not seem right for it to parse -the same uri two different ways depending on the OS it's running on. And it doesn't -special-case handling of file urls at all, it only implements RFC3986. -We could try opening an issue at -and find out what its maintainer thinks. - -I suppose that git-annex, when running on windows, every place after it parses an -url could: - -1. Check if it's a file: url -2. If the path starts with "/DRIVE:", remove the leading "/" - -Yugh. -"""]] diff --git a/doc/bugs/git-annex_drop_fails_to_access_file__58____47____47____47___target_URL_on_Windows/comment_5_e00d70ba6b88d9cf60fcb183f7ea4980._comment b/doc/bugs/git-annex_drop_fails_to_access_file__58____47____47____47___target_URL_on_Windows/comment_5_e00d70ba6b88d9cf60fcb183f7ea4980._comment deleted file mode 100644 index 96e0489914..0000000000 --- a/doc/bugs/git-annex_drop_fails_to_access_file__58____47____47____47___target_URL_on_Windows/comment_5_e00d70ba6b88d9cf60fcb183f7ea4980._comment +++ /dev/null @@ -1,7 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2023-03-27T17:57:52Z" - content=""" -Ok, put in that ugly fix. -"""]] diff --git a/doc/bugs/git-annex_findkeys_does_not_know_--largerthan.mdwn b/doc/bugs/git-annex_findkeys_does_not_know_--largerthan.mdwn deleted file mode 100644 index 591564f055..0000000000 --- a/doc/bugs/git-annex_findkeys_does_not_know_--largerthan.mdwn +++ /dev/null @@ -1,50 +0,0 @@ -### Please describe the problem. - -The man page of [git-annex findkeys](https://git-annex.branchable.com/git-annex-findkeys/) says: - -> OPTIONS -> -> * matching options -> The git-annex-matching-options(1) can be used to specify which keys to list. - -However, this is not true for the options that match file size. Being able to do for example `git-annex findkeys --largerthan 100M` -is important for me as it allows me to track down the files that occupy most of my storage, allowing me to move them to some -archive. However, if I try to call the above command, it does not show me a list of keys matching the criterion, but a help page -as if I have misspelled the option: - -> $ git annex findkeys --largerthan 1 -> Invalid option `--largerthan' -> -> Usage: git-annex COMMAND -> git-annex - manage files with git, without checking their contents in -> -> Commonly used commands: -> -> add add files to annex -> [...] - -### What steps will reproduce the problem? - -[[!format sh """ - echo hi > file - git annex init - git annex add file - git commit -m "Test commit" - git annex find --largerthan 1 # << this lists "file" - git annex findkeys --largerthan 1 # << this fails -"""]] - -### What version of git-annex are you using? On what operating system? - -* git-annex version: 10.20230926-g44a7b4c9734adfda5912dd82c1aa97c615689f57 -* Rocky Linux 9.2 -* git 2.40.1 - - - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -I am using it for my research project (data science/predictions in plant breeding) and it allows me to keep track of the -current model iteration and associated results. Thank you for this! - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/git-annex_findkeys_does_not_know_--largerthan/comment_1_f6484f69669439ef49513b84d3bd91cf._comment b/doc/bugs/git-annex_findkeys_does_not_know_--largerthan/comment_1_f6484f69669439ef49513b84d3bd91cf._comment deleted file mode 100644 index 05ea2c90a0..0000000000 --- a/doc/bugs/git-annex_findkeys_does_not_know_--largerthan/comment_1_f6484f69669439ef49513b84d3bd91cf._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-11-28T15:48:09Z" - content=""" -Not all options in git-annex-matching-options can be used by findkeys. It -mentions this when it says "Some of these options can also be used by -commands to specify which keys they act on." - -However in this case, --largerthan and --smallerthan could in fact be made -to operate on keys, and I've done so. -"""]] diff --git a/doc/bugs/git-annex_fsck_in_quiet_mode_don__39__t_report_the_files_it_failed_to_download.mdwn b/doc/bugs/git-annex_fsck_in_quiet_mode_don__39__t_report_the_files_it_failed_to_download.mdwn deleted file mode 100644 index 31da88a6ce..0000000000 --- a/doc/bugs/git-annex_fsck_in_quiet_mode_don__39__t_report_the_files_it_failed_to_download.mdwn +++ /dev/null @@ -1,18 +0,0 @@ -### Please describe the problem. - -When running 'git annex fsck --from $i --quiet' when $i is a special remote, git annex will report failure to download as 'failed to download file from remote' without telling me which file it failed to download. - -### What steps will reproduce the problem? - -Just running 'git annex fsck --from remote --quiet' on some special remote with connexion problem - -### What version of git-annex are you using? On what operating system? - -6.20170101-1+b1 from Debian stretch - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -git annex is very efficient to sync my usb key, laptops and servers together. - -> [[fixed|done]]. Sorry for the 4 year delay, seems I missed this bug -> report. --[[Joey]] diff --git a/doc/bugs/git-annex_fsck_in_quiet_mode_don__39__t_report_the_files_it_failed_to_download/comment_1_074e49431bd18c43164872de3733b0e8._comment b/doc/bugs/git-annex_fsck_in_quiet_mode_don__39__t_report_the_files_it_failed_to_download/comment_1_074e49431bd18c43164872de3733b0e8._comment deleted file mode 100644 index 7465aae6f4..0000000000 --- a/doc/bugs/git-annex_fsck_in_quiet_mode_don__39__t_report_the_files_it_failed_to_download/comment_1_074e49431bd18c43164872de3733b0e8._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Rémi" - avatar="http://cdn.libravatar.org/avatar/0f25950c04685d6c7ccb22215268d6a7" - subject="comment 1" - date="2017-08-21T14:17:14Z" - content=""" -Running 'git annex fsck --more --from remote' allowed me to try again and find which file had the problem. -"""]] diff --git a/doc/bugs/git_annex_assist_adds_gitignored_files.mdwn b/doc/bugs/git_annex_assist_adds_gitignored_files.mdwn deleted file mode 100644 index 30b1e4c213..0000000000 --- a/doc/bugs/git_annex_assist_adds_gitignored_files.mdwn +++ /dev/null @@ -1,39 +0,0 @@ -### Please describe the problem. - -The new `git annex assist` adds `.gitignore`ed files - -### What steps will reproduce the problem? - -``` -git init -git annex init -echo bla > .gitignore -touch bla -git annex assist -git ls-files -# bla # 👈 shouldn't be here! -# .gitignore -``` - -### What version of git-annex are you using? On what operating system? - -Manjaro Linux - -``` -❯ git annex version -git-annex version: 10.20230408-g250194b7d -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22.1 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.13.1 persistent-sqlite-2.13.1.0 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2.1 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 10 -``` - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -git-annex is awesome! - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/git_annex_assistant_adds_dotfiles_to_the_annex.mdwn b/doc/bugs/git_annex_assistant_adds_dotfiles_to_the_annex.mdwn deleted file mode 100644 index 31a2d6ec67..0000000000 --- a/doc/bugs/git_annex_assistant_adds_dotfiles_to_the_annex.mdwn +++ /dev/null @@ -1,35 +0,0 @@ -### Please describe the problem. - -Running `git add` or `git annex add` does not add dotfiles to the annex. However, the git annex assistant appears to add dotfiles to the annex. - -### What steps will reproduce the problem? - -1. Initialize a new repository (`git init` and `git annex init`). - -2. Create a dotfile: `echo "foo" > .somedotfile` - -3. Run `git annex assistant` - -The file will then be added to the annex. - -### What version of git-annex are you using? On what operating system? - -Operating system: Arch Linux x86_64 - -git-annex version: 10.20230329 - -### Please provide any additional information below. - -[[!format sh """ -# Begin .git/annex/daemon.log - - No known volume monitor available through dbus; falling back to mtab polling -(scanning...) (started...) -add .somedotfile ok -(recording state in git...) -(recording state in git...) - -# End of transcript or log. -"""]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/git_annex_connecting_over_tor_does_not_work.mdwn b/doc/bugs/git_annex_connecting_over_tor_does_not_work.mdwn deleted file mode 100644 index 586650e5de..0000000000 --- a/doc/bugs/git_annex_connecting_over_tor_does_not_work.mdwn +++ /dev/null @@ -1,82 +0,0 @@ -### Please describe the problem. -attempting to the use feature which allows me to connect to a friends remote repository results in failure, in which i have tested using multiple methods - -### What steps will reproduce the problem? -#### WEBAPP -* open the webapp -* add repository -* choose "share with a friend" -* click "lets get started" -* encounter the error found in the transcript below - - -#### COMMAND LINE -##### enable tor -* git annex enable tor -* encounter error found in the transcript below - -##### p2p --pair -* git annex p2p --pair -* encounter the error found in the transcript - -### What version of git-annex are you using? On what operating system? - - - -### Please provide any additional information below. - -### external dependency version info -* git annex: using git annex found in nixos repositories -* python version: attempted both python3 and python2.7 -* magic-wormhole version: attempted both 0.12.0 under python3, and version 0.10.0 under python2.7 -* tor version information: Tor version 0.4.7.10. -Tor is running on Linux with Libevent 2.1.12-stable, OpenSSL 1.1.1q, Zlib 1.2.12, Liblzma 5.2.5, Libzstd 1.5.2 and Glibc 2.34 as libc. -Tor compiled with GCC version 11.3.0 - - -### transcript - -[[!format sh """ - -#### WEBAPP -##### error output after following steps as described above - -Failed to enable tor - -enable-tor - You may be prompted for a password -git-annex: This can only be run in a git-annex repository. - -git-annex: Failed to run as root: /nix/store/c42x83x25df9xd053xiii24mawnrlvrd-git-annex-10.20220504/bin/git-annex enable-tor 1000 -failed -enable-tor: 1 failed - -#### COMMAND LINE -##### enable tor -git annex enable-tor -Failed to enable tor - -enable-tor - You may be prompted for a password -git-annex: This can only be run in a git-annex repository. - -git-annex: Failed to run as root: /nix/store/c42x83x25df9xd053xiii24mawnrlvrd-git-annex-10.20220504/bin/git-annex enable-tor 1000 -failed -enable-tor: 1 failed -##### git annex p2p --pair -git annex p2p --pair - -p2p pair peer1 (using Magic Wormhole) - - Failed sending data to pair. -failed -p2p: 1 failed - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -git annex has been awesome so far in helping track and backup my data, also for allowing me to output podcasts/audiobooks to a small mp3 player, as well as ebooks and pdfs to a kindle paper ewhite, which has made using my ereader no longer an inconvinient endeavour - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/git_annex_connecting_over_tor_does_not_work/comment_1_35c33ae1d69382a03c94f4a20ec22f7e._comment b/doc/bugs/git_annex_connecting_over_tor_does_not_work/comment_1_35c33ae1d69382a03c94f4a20ec22f7e._comment deleted file mode 100644 index 289688d5bb..0000000000 --- a/doc/bugs/git_annex_connecting_over_tor_does_not_work/comment_1_35c33ae1d69382a03c94f4a20ec22f7e._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-09-26T17:20:05Z" - content=""" -I can reproduce this by running eg: - - joey@darkstar:~/tmp/torrepo>git-annex init - init ok - joey@darkstar:~/tmp/torrepo>su -c "git-annex --debug enable-tor 1000" - Password: - git-annex: This can only be run in a git-annex repository. - -This was broken by git's security fix for CVE-2022-24765. Now when the root -process tries to run `git config --list`, git does not display the local -.git/config, because it is owned by another user. -"""]] diff --git a/doc/bugs/git_annex_connecting_over_tor_does_not_work/comment_2_5df90e5ffbd3f089a0f640d4a5b85a1c._comment b/doc/bugs/git_annex_connecting_over_tor_does_not_work/comment_2_5df90e5ffbd3f089a0f640d4a5b85a1c._comment deleted file mode 100644 index 6261bfc7f2..0000000000 --- a/doc/bugs/git_annex_connecting_over_tor_does_not_work/comment_2_5df90e5ffbd3f089a0f640d4a5b85a1c._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-09-26T17:24:23Z" - content=""" -Note that when git-annex uses sudo to gain root, git does not behave that -way. But it can also use su or other methods. -"""]] diff --git a/doc/bugs/git_annex_connecting_over_tor_does_not_work/comment_3_9fa2fb82ae3d17a77a9645870e1e12e5._comment b/doc/bugs/git_annex_connecting_over_tor_does_not_work/comment_3_9fa2fb82ae3d17a77a9645870e1e12e5._comment deleted file mode 100644 index 3a688a3f5a..0000000000 --- a/doc/bugs/git_annex_connecting_over_tor_does_not_work/comment_3_9fa2fb82ae3d17a77a9645870e1e12e5._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2022-09-26T18:51:25Z" - content=""" -Fixed `git-annex enable-tor` - -The `git-annex p2p --pair` failing is unrelated. It seems that -`wormhole` has either changed its syntax for --appid, or git-annex has -never passed --appid to it correctly. I've fixed it to use the current -syntax. -"""]] diff --git a/doc/bugs/git_annex_enableremote_locking_issue.mdwn b/doc/bugs/git_annex_enableremote_locking_issue.mdwn deleted file mode 100644 index 0fc2ca3f25..0000000000 --- a/doc/bugs/git_annex_enableremote_locking_issue.mdwn +++ /dev/null @@ -1,56 +0,0 @@ -### Please describe the problem. - -When enabling an S3 remote with hybrid encryption, I get a file locking error: - -``` -$ git annex enableremote s3remote -git-annex: .git/annex/creds/__S3_REMOTE_UUID_REDACTED__: openFile: resource busy (file is locked) -``` - -I can confirm that this happens when no previous process is running (and lsof shows nothing open before running enableremote), for example in the motivating case here of cloning to a new, freshly-mounted HDD. - -If I copy the `.git/annex/creds/__S3_REMOTE_UUID_REDACTED__` file from an existing clone of the repository to this new one, then enableremote works fine. - -``` -$ git annex enableremote s3remote -enableremote s3remote (encryption update) (to gpg keys: HEXREDACTED) ok -(recording state in git...) -``` - -### What steps will reproduce the problem? - -For me, with a source a repository that has a pre-configured S3 special remote: - -``` -$ git show git-annex:remote.log -__UUID__ bucket=bucket-name cipher=__base64_jibberish__ cipherkeys=GPGKEY_HEX datacenter=US host=s3.eu-central-003.backblazeb2.com name=s3remote partsize=512MiB port=80 protocol=https s3creds=__base64_jibberish__ signature=v4 storageclass=STANDARD type=S3 timestamp=168000000.0s -``` - -Then, to reproduce this: - -``` -git clone /path/to/established-repo new_clone -cd new_clone -git annex init new_clone -git annex enableremote s3remote -``` - - - -### What version of git-annex are you using? On what operating system? - -The latest: - -``` -$ git annex version -git-annex version: 10.20230926-g44a7b4c9734adfda5912dd82c1aa97c615689f57 -``` - -### Please provide any additional information below. - - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -A great deal, we use it daily in our research and I also use it on some personal files -- thanks for all your efforts Joey et al.! - -> Reproduced and fixed. [[done]] --[[Joey]] diff --git a/doc/bugs/git_annex_forget_does_not_propagate.mdwn b/doc/bugs/git_annex_forget_does_not_propagate.mdwn deleted file mode 100644 index 82b4c5df53..0000000000 --- a/doc/bugs/git_annex_forget_does_not_propagate.mdwn +++ /dev/null @@ -1,30 +0,0 @@ -### Please describe the problem. -When using git annex forget to clear the git-annex branch, syncing these changes does not result in clearing the git-annex branches of other remotes. Instead, these branches are only appended to with the commit starting the new branch, and when sync is run on the originating repo, the forget is undone. - -### What steps will reproduce the problem? -1. `git annex forget --drop-dead` -2. (on remote) `git annex sync (repo where forget was run)` -3. (on starting repo) `git annex sync *remote*` - -### What version of git-annex are you using? On what operating system? - - git-annex version: 8.20210223 - build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV - dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0 - key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* - remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external - operating system: linux x86_64 - supported repository versions: 8 - upgrade supported from repository versions: 0 1 2 3 4 5 6 7 - local repository version: 8 - -### Please provide any additional information below. - -`git annex sync --cleanup` before sync was ineffective - -There is no error log per se, git annex thinks everything went well. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) -Aside from this, best thing ever. - -> [[fixed|done]] back in 2022 --[[Joey]] diff --git a/doc/bugs/git_annex_forget_does_not_propagate/comment_1_591c3c59d831ed94268f90505a8baaee._comment b/doc/bugs/git_annex_forget_does_not_propagate/comment_1_591c3c59d831ed94268f90505a8baaee._comment deleted file mode 100644 index 7231d606ee..0000000000 --- a/doc/bugs/git_annex_forget_does_not_propagate/comment_1_591c3c59d831ed94268f90505a8baaee._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="Too old?" - date="2023-04-17T01:43:36Z" - content=""" -Have you tried with latest git annex? Yours is two major versions behind and two years old... -"""]] diff --git a/doc/bugs/git_annex_forget_does_not_propagate/comment_2_9026452178c82d0661acc96d009e3764._comment b/doc/bugs/git_annex_forget_does_not_propagate/comment_2_9026452178c82d0661acc96d009e3764._comment deleted file mode 100644 index 8f39274b00..0000000000 --- a/doc/bugs/git_annex_forget_does_not_propagate/comment_2_9026452178c82d0661acc96d009e3764._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2023-04-17T17:34:16Z" - content=""" -Look what I found in the changelog: - - git-annex (10.20220222) upstream; urgency=medium - - * Fix git-annex forget propagation between repositories. - (reversion introduced in version 7.20190122) - - -- Joey Hess Tue, 22 Feb 2022 13:01:20 -0400 -"""]] diff --git a/doc/bugs/git_annex_get_--batch_does_not_work.mdwn b/doc/bugs/git_annex_get_--batch_does_not_work.mdwn deleted file mode 100644 index d6aaea3f2a..0000000000 --- a/doc/bugs/git_annex_get_--batch_does_not_work.mdwn +++ /dev/null @@ -1,27 +0,0 @@ -### Please describe the problem. - -`git annex get foo` does work. - -`echo "foo" | git annex get --batch` does not. - - -### What version of git-annex are you using? On what operating system? - -git-annex version: 8.20210223 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 -local repository version: 8 - - -I use it on Debian 11 - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -YES! Git annex is a wonderful tool! I use it for synchronising TB of data - -> [[notabug|done]] --[[Joey]] diff --git a/doc/bugs/git_annex_get_--batch_does_not_work/comment_1_52311b95bf29706d2b1afb3904428047._comment b/doc/bugs/git_annex_get_--batch_does_not_work/comment_1_52311b95bf29706d2b1afb3904428047._comment deleted file mode 100644 index c81bd49ad6..0000000000 --- a/doc/bugs/git_annex_get_--batch_does_not_work/comment_1_52311b95bf29706d2b1afb3904428047._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="git-annex.visiteur@e9d364191d2ffc1b163c8d9e4c57dbadf58aad8e" - nickname="git-annex.visiteur" - avatar="http://cdn.libravatar.org/avatar/59640df9d44f100f0bf98c1cbb430037" - subject="comment 1" - date="2022-03-06T14:37:42Z" - content=""" -When I say \"It dos not work\", I mean \"It only returns blank line\" -"""]] diff --git a/doc/bugs/git_annex_get_--batch_does_not_work/comment_2_b4fa614626393e01c51b937401bbaa23._comment b/doc/bugs/git_annex_get_--batch_does_not_work/comment_2_b4fa614626393e01c51b937401bbaa23._comment deleted file mode 100644 index 6a20bbf351..0000000000 --- a/doc/bugs/git_annex_get_--batch_does_not_work/comment_2_b4fa614626393e01c51b937401bbaa23._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-03-07T16:01:56Z" - content=""" -A blank line means that the file's content is already -present, or it is not an annexed file. This is documented under --batch in -the man page of git-annex get. -"""]] diff --git a/doc/bugs/git_annex_get_--batch_does_not_work/comment_3_e972b9fb5c6af4ecd295dc954bb2f55a._comment b/doc/bugs/git_annex_get_--batch_does_not_work/comment_3_e972b9fb5c6af4ecd295dc954bb2f55a._comment deleted file mode 100644 index 03451cdfdc..0000000000 --- a/doc/bugs/git_annex_get_--batch_does_not_work/comment_3_e972b9fb5c6af4ecd295dc954bb2f55a._comment +++ /dev/null @@ -1,35 +0,0 @@ -[[!comment format=mdwn - username="git-annex.visiteur@e9d364191d2ffc1b163c8d9e4c57dbadf58aad8e" - nickname="git-annex.visiteur" - avatar="http://cdn.libravatar.org/avatar/59640df9d44f100f0bf98c1cbb430037" - subject="comment 3" - date="2022-03-07T19:22:54Z" - content=""" -I already see in the manual that blank line means the file is not in annex. But it is NOT the case here. - -I do think it's is a bug. See the result of the following commands executed in a row: - - - > echo \"Wasabi\" | git annex info --batch - directory: Wasabi - local annex keys: 0 - local annex size: 0 bytes - annexed files in working tree: 4 - size of annexed files in working tree: 733.1 megabytes - numcopies stats: - numcopies +3: 4 - repositories containing these files: 4 - 733.1 MB: 2a15d9f9-eb82-4b20-8b4d-2ae1e0da4eee -- freenas - 733.1 MB: 7c082253-61af-45e1-b68d-381d2516a626 -- passport - 733.1 MB: c6634036-e251-425e-a075-1ec4e08b0a78 -- pomme - 733.1 MB: d58b0ca5-becd-416e-b1cd-e6fefca3685f -- intenso [osmc] - combined size of repositories containing these files: 2.93 gigabytes - - - > echo \"Wasabi\" | git annex get --batch - (blank line) - - > git annex get Wasabi - get Wasabi/Wasabi-fanart.jpg (from osmc...) - ... -"""]] diff --git a/doc/bugs/git_annex_get_--batch_does_not_work/comment_4_f0c498dad3f4d51e33de863165f29410._comment b/doc/bugs/git_annex_get_--batch_does_not_work/comment_4_f0c498dad3f4d51e33de863165f29410._comment deleted file mode 100644 index e8ad2fb746..0000000000 --- a/doc/bugs/git_annex_get_--batch_does_not_work/comment_4_f0c498dad3f4d51e33de863165f29410._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2022-03-07T19:27:35Z" - content=""" -You have added single quotes around the file name `Wasabi`, -so you have told it to get a file with those quotes in its name. No such -file exists. - -You do not need to single quote anything in the batch interface. -"""]] diff --git a/doc/bugs/git_annex_get_--batch_does_not_work/comment_5_603adcb0b5091ac2fb3c9225d2925dc3._comment b/doc/bugs/git_annex_get_--batch_does_not_work/comment_5_603adcb0b5091ac2fb3c9225d2925dc3._comment deleted file mode 100644 index 7fdf3e69ec..0000000000 --- a/doc/bugs/git_annex_get_--batch_does_not_work/comment_5_603adcb0b5091ac2fb3c9225d2925dc3._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="git-annex.visiteur@e9d364191d2ffc1b163c8d9e4c57dbadf58aad8e" - nickname="git-annex.visiteur" - avatar="http://cdn.libravatar.org/avatar/59640df9d44f100f0bf98c1cbb430037" - subject="comment 5" - date="2022-03-07T19:59:17Z" - content=""" -It is exactely the same with the following - - > echo Wasabi | git annex get --batch - (blank line) - -Moreover, it would be strange that quote works with \"info\" but not with \"get\" -"""]] diff --git a/doc/bugs/git_annex_get_--batch_does_not_work/comment_6_56716644ee1534b8fe90ba502856455c._comment b/doc/bugs/git_annex_get_--batch_does_not_work/comment_6_56716644ee1534b8fe90ba502856455c._comment deleted file mode 100644 index c38749162e..0000000000 --- a/doc/bugs/git_annex_get_--batch_does_not_work/comment_6_56716644ee1534b8fe90ba502856455c._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 6""" - date="2022-03-08T17:56:54Z" - content=""" -According to the git-annex info output, `Wasabi` is a directory. - -Since a directory is not an annexed file, git-annex get --batch will -not get it, so it outputs a blank line. If you tell it to get a specific file -within that directory, it will. - -Since git-annex info --batch can operate on directories as well as on files, -it does. That is the reason for the inconsistency. -"""]] diff --git a/doc/bugs/git_annex_get_--batch_does_not_work/comment_7_b0588132d02f982da7c7208087b0afe2._comment b/doc/bugs/git_annex_get_--batch_does_not_work/comment_7_b0588132d02f982da7c7208087b0afe2._comment deleted file mode 100644 index 1d2b8369c2..0000000000 --- a/doc/bugs/git_annex_get_--batch_does_not_work/comment_7_b0588132d02f982da7c7208087b0afe2._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="git-annex.visiteur@e9d364191d2ffc1b163c8d9e4c57dbadf58aad8e" - nickname="git-annex.visiteur" - avatar="http://cdn.libravatar.org/avatar/59640df9d44f100f0bf98c1cbb430037" - subject="comment 7" - date="2022-03-09T06:33:51Z" - content=""" -Ok, using file name directly instead of parent directory is working. - -Is there a good reason explaining why `git annex get ` works wherease `echo | git annex get --batch` doesn't? -"""]] diff --git a/doc/bugs/git_annex_get_--batch_does_not_work/comment_8_0e776c598c5e0f4f51f5e9ade8c3f0e2._comment b/doc/bugs/git_annex_get_--batch_does_not_work/comment_8_0e776c598c5e0f4f51f5e9ade8c3f0e2._comment deleted file mode 100644 index eba5b62a3a..0000000000 --- a/doc/bugs/git_annex_get_--batch_does_not_work/comment_8_0e776c598c5e0f4f51f5e9ade8c3f0e2._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 8""" - date="2022-03-10T17:09:26Z" - content=""" -Other features like --json when used with --batch would -not work well if it recursed through directories. There would be no way to -know which output corresponded with which input. -"""]] diff --git a/doc/bugs/git_annex_get_--batch_does_not_work/comment_9_0c802c9c5734e7dfc8cd72962fcf2983._comment b/doc/bugs/git_annex_get_--batch_does_not_work/comment_9_0c802c9c5734e7dfc8cd72962fcf2983._comment deleted file mode 100644 index 81d692d0cf..0000000000 --- a/doc/bugs/git_annex_get_--batch_does_not_work/comment_9_0c802c9c5734e7dfc8cd72962fcf2983._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="git-annex.visiteur@e9d364191d2ffc1b163c8d9e4c57dbadf58aad8e" - nickname="git-annex.visiteur" - avatar="http://cdn.libravatar.org/avatar/59640df9d44f100f0bf98c1cbb430037" - subject="comment 9" - date="2022-03-11T18:46:40Z" - content=""" -Ok thank you, i understand the reason. - -Very good job by the way. -"""]] diff --git a/doc/bugs/git_annex_push_fails_on_Windows.mdwn b/doc/bugs/git_annex_push_fails_on_Windows.mdwn deleted file mode 100644 index 21a7d7c50d..0000000000 --- a/doc/bugs/git_annex_push_fails_on_Windows.mdwn +++ /dev/null @@ -1,107 +0,0 @@ -### Please describe the problem. - -After adding a directory special remote to a git annex repository, the git annex push push-win-remote fails with the following error: - -Not updating export to push-win-remote because adjusted/main(unlocked) does not exist. (As configured by remote.push-win-remote.annex-tracking-branch) - -The problem does *not* occur on Linux or macOS. - -### What steps will reproduce the problem? - -[[!format sh """ -mkdir push-win-repo -mkdir push-win-rmt -cd push-win-repo -git init --initial-branch main -git annex init push-win-repository -git annex config --set annex.largefiles include=*.xtx -echo "push test file" >> push-test.xtx -git annex add . -git commit -m "add push test file" -git annex initremote push-win-remote type=directory directory=../push-win-rmt autoenable=true encryption=none exporttree=yes importtree=yes -git config remote.push-win-remote.annex-tracking-branch adjusted/main(unlocked) -git annex group push-win-remote manual -git annex wanted push-win-remote include=*.xtx -git annex push push-win-remote -"""]] - -### What version of git-annex are you using? On what operating system? - -* git-annex version: 10.20230803-gd467c70ef7 -* git version 2.41.0.windows.2 -* OS Name: Microsoft Windows 10 Pro -* OS Version: 10.0.19045 N/A Build 19045 - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - -C:\win-push-bug>mkdir push-win-repo - -C:\win-push-bug>mkdir push-win-rmt - -C:\win-push-bug>cd push-win-repo - -C:\win-push-bug\push-win-repo>git init --initial-branch main -Initialized empty Git repository in C:/win-push-bug/push-win-repo/.git/ - -C:\win-push-bug\push-win-repo>git annex init push-win-repository -init push-win-repository - Detected a filesystem without fifo support. - - Disabling ssh connection caching. - - Detected a crippled filesystem. - - Entering an adjusted branch where files are unlocked as this filesystem does not support locked files. - -Switched to branch 'adjusted/main(unlocked)' -ok -(recording state in git...) - -C:\win-push-bug\push-win-repo>git annex config --set annex.largefiles include=*.xtx -annex.largefiles include=*.xtx ok -(recording state in git...) - -C:\win-push-bug\push-win-repo>echo "push test file" 1>>push-test.xtx - -C:\win-push-bug\push-win-repo>git annex add . -add push-test.xtx -ok -(recording state in git...) - -C:\win-push-bug\push-win-repo>git commit -m "add push test file" -[adjusted/main(unlocked) f3262a2] add push test file - 1 file changed, 1 insertion(+) - create mode 100644 push-test.xtx - -C:\win-push-bug\push-win-repo>git annex initremote push-win-remote type=directory directory=../push-win-rmt autoenable=true encryption=none exporttree=yes importtree=yes -initremote push-win-remote ok -(recording state in git...) - -C:\win-push-bug\push-win-repo>git config remote.push-win-remote.annex-tracking-branch adjusted/main(unlocked) - -C:\win-push-bug\push-win-repo>git annex group push-win-remote manual -group push-win-remote ok -(recording state in git...) - -C:\win-push-bug\push-win-repo>git annex wanted push-win-remote include=*.xtx -wanted push-win-remote ok -(recording state in git...) - -C:\win-push-bug\push-win-repo>git annex push push-win-remote - - Not updating export to push-win-remote because adjusted/main(unlocked) does not exist. (As configured by remote.push-win-remote.annex-tracking-branch) - -C:\win-push-bug\push-win-repo> - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Yes. I am developing the easy-git-annex api. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/git_annex_push_fails_on_Windows/comment_1_a7866c2131df8132e9aecebda085294c._comment b/doc/bugs/git_annex_push_fails_on_Windows/comment_1_a7866c2131df8132e9aecebda085294c._comment deleted file mode 100644 index bf46c03f93..0000000000 --- a/doc/bugs/git_annex_push_fails_on_Windows/comment_1_a7866c2131df8132e9aecebda085294c._comment +++ /dev/null @@ -1,20 +0,0 @@ -[[!comment format=mdwn - username="jstritch" - avatar="http://cdn.libravatar.org/avatar/56756b34ff409071074762951d270002" - subject="git annex pull is also broken" - date="2023-09-09T14:49:35Z" - content=""" -If I put the same file in the remote under the name pull-test.xtx and invoke the command git annex pull push-win-remote, git annex outputs: - -list push-win-remote ok - -import push-win-remote pull-test.xtx - -ok - -update refs/remotes/push-win-remote/adjusted/main(unlocked) ok - -(recording state in git...) - -But the file is not present in the repository. -"""]] diff --git a/doc/bugs/git_annex_push_fails_on_Windows/comment_2_d5c9fd2acdd5b320563a708889e82852._comment b/doc/bugs/git_annex_push_fails_on_Windows/comment_2_d5c9fd2acdd5b320563a708889e82852._comment deleted file mode 100644 index 49c8b772ac..0000000000 --- a/doc/bugs/git_annex_push_fails_on_Windows/comment_2_d5c9fd2acdd5b320563a708889e82852._comment +++ /dev/null @@ -1,27 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2023-09-11T17:23:12Z" - content=""" -The same behavior can be observed on linux. I just ran `git annex adjust ---unlock` after committing to get into the same state as on Windows. - -It's actually not supported to set annex-tracking-branch to an adjusted -branch. Instead, you ought to set: - - git config remote.push-win-remote.annex-tracking-branch main - -When pulling a new file, as in comment #1, that results in the change -getting merged into main, and then the adjusted branch is automatically -updated as well. - -When pushing a newly added file, I first had to run -`git-annex adjust --unlock` in order to update main. That was a -bug in `git-annex push`; it ought to update main for you in this case; -it already does when pushing to a regular git remote. Fixed that. - -The "Not updating export to push-win-remote because adjusted/main(unlocked) -does not exist" message is misleading. I've improved that to be -"Not updating export to push-win-remote because tracking branch name is not valid" -which at least hints in the right direction. -"""]] diff --git a/doc/bugs/git_annex_push_fails_on_Windows/comment_3_fe3818a184e723f86d094253461d8f44._comment b/doc/bugs/git_annex_push_fails_on_Windows/comment_3_fe3818a184e723f86d094253461d8f44._comment deleted file mode 100644 index 5b2387b8c7..0000000000 --- a/doc/bugs/git_annex_push_fails_on_Windows/comment_3_fe3818a184e723f86d094253461d8f44._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="jstritch" - avatar="http://cdn.libravatar.org/avatar/56756b34ff409071074762951d270002" - subject="comment 3" - date="2023-09-12T16:36:26Z" - content=""" -Thanks! -"""]] diff --git a/doc/bugs/git_annex_repair_failed__58___missing_objects.mdwn b/doc/bugs/git_annex_repair_failed__58___missing_objects.mdwn deleted file mode 100644 index 687a81e903..0000000000 --- a/doc/bugs/git_annex_repair_failed__58___missing_objects.mdwn +++ /dev/null @@ -1,48 +0,0 @@ -### Please describe the problem. - -Git annex finds problems with the repository after only adding and syncing a single empty file to the repository. - -### What steps will reproduce the problem? - -* Create one repository each on devices `A` and `B` via webapp -* Pair with a passphrase via webapp -* Wait until full sync -* Make sure `git annex repair` finds no problems -* Create an empty file `test.sh` on device `A` -* Wait until both webapps show sync and transfer has completed - -Both repositories are now in a bad state. - -* `git annex repair` reports error code `repair: 1 failed`, `--force` seems to repair, but the next `git annex repair` always fails again with `repair: 1 failed` -* This is true for both repositories - - -### What version of git-annex are you using? On what operating system? - -`Linux 5.17.3-arch1-1 #1 SMP PREEMPT x86_64 GNU/Linux` device `A` - -`Linux 5.17.4-arch1-1 #1 SMP PREEMPT x86_64 GNU/Linux` device `B` - -Version provided from package manager: - -``` -git-annex version: 10.20220322-g959beeea9 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.30 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.11 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -``` - - -### Please provide any additional information below. - -The bug is very easy to reproduce. This happens in a bigger repository of mine, where I `git annex repair` doesn't seem to do anything. I narrowed it down to this example and I hope it's the same cause. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Using it daily, with small-sized repositories. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/git_annex_repair_failed__58___missing_objects/comment_1_9f2166545159890b05ea061e2ab21a4f._comment b/doc/bugs/git_annex_repair_failed__58___missing_objects/comment_1_9f2166545159890b05ea061e2ab21a4f._comment deleted file mode 100644 index 29fbc25774..0000000000 --- a/doc/bugs/git_annex_repair_failed__58___missing_objects/comment_1_9f2166545159890b05ea061e2ab21a4f._comment +++ /dev/null @@ -1,166 +0,0 @@ -[[!comment format=mdwn - username="Albert" - avatar="http://cdn.libravatar.org/avatar/79aaf48dabedd7cd96a9117685fbe2cc" - subject="Minimal Example" - date="2022-04-29T15:14:20Z" - content=""" -I have managed to further reduce the setup to generate this bug. No connection to another device necessary. - -Steps: - -* Create a fresh repository with `git annex webapp` -* Add an empty file to the repository named `exec.zip` - -Now the repository is broken. - -```bash -❯ ls -exec.zip -❯ git annex repair -repair Running git fsck ... -Fsck found no problems. Checking for broken branches. -Found problems, attempting repair. -Some git branches refer to missing objects: - refs/annex/last-index -To force a recovery to a usable state, retry with the --force parameter. -failed -repair: 1 failed -❯ git annex repair --force -repair Running git fsck ... -Fsck found no problems. Checking for broken branches. -Found problems, attempting repair. -Successfully recovered repository! -You should run \"git fsck\" to make sure, but it looks like everything was recovered ok. -ok -❯ git fsck -Checking object directories: 100% (256/256), done. -❯ git annex fsck -fsck exec.zip (checksum...) ok -(recording state in git...) -❯ git annex repair -repair Running git fsck ... -Fsck found no problems. Checking for broken branches. -Found problems, attempting repair. -Some git branches refer to missing objects: - refs/annex/last-index -To force a recovery to a usable state, retry with the --force parameter. -failed -repair: 1 failed -``` - -The log shows nothing interesting, as far as errors go. - -```bash -[2022-04-29 16:52:08.770880358] (Utility.Process) process [8244] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"cat-file\",\"--batch\"] -[2022-04-29 16:52:08.773934773] (Utility.Process) process [8245] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"ls-files\",\"--stage\",\"-z\",\"--\",\".\"] -[2022-04-29 16:52:08.775028928] (Utility.Process) process [8245] done ExitSuccess -[2022-04-29 16:52:08.775560498] (Utility.Process) process [8246] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"ls-files\",\"--stage\",\"-z\",\"--\",\".\"] -[2022-04-29 16:52:08.77720598] (Utility.Process) process [8246] done ExitSuccess -[2022-04-29 16:52:08.77980743] (Utility.Process) process [8247] chat: nice [\"ionice\",\"-c3\",\"/usr/bin/git-annex\",\"remotedaemon\",\"--foreground\"] -[2022-04-29 16:52:08.831064871] (NetWatcher) Using running DBUS service org.freedesktop.NetworkManager to monitor network connection events. -[2022-04-29 16:52:08.83204466] (MountWatcher) Using running DBUS service org.freedesktop.UDisks2 to monitor mount events. -[2022-04-29 16:52:08.834446186] (Utility.Process) process [8248] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"ls-tree\",\"--full-tree\",\"-z\",\"--\",\"refs/heads/git-annex\",\"uuid.log\",\"remote.log\",\"trust.log\",\"group.log\",\"numcopies.log\",\"mincopies.log\",\"schedule.log\",\"preferred-content.log\",\"required-content.log\",\"group-preferred-content.log\"] -[2022-04-29 16:52:08.835869051] (Utility.Process) process [8248] done ExitSuccess -[2022-04-29 16:52:08.83662762] (Utility.Process) process [8249] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"git-annex\"] -[2022-04-29 16:52:08.836789466] (TransferWatcher) watching for transfers -[2022-04-29 16:52:08.837846991] (Utility.Process) process [8249] done ExitSuccess -[2022-04-29 16:52:08.838365064] (Utility.Process) process [8250] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"] -[2022-04-29 16:52:08.839508977] (Utility.Process) process [8250] done ExitSuccess -[2022-04-29 16:52:08.83982672] (Utility.Process) process [8251] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..8d3e39db152fe946720fc814daaa892ea9b5ae56\",\"--pretty=%H\",\"-n1\"] -[2022-04-29 16:52:08.841105024] (Utility.Process) process [8251] done ExitSuccess -[2022-04-29 16:52:08.841851852] (Utility.Process) process [8252] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"ls-tree\",\"--full-tree\",\"-z\",\"--\",\"refs/heads/git-annex\",\"uuid.log\",\"remote.log\",\"trust.log\",\"group.log\",\"numcopies.log\",\"mincopies.log\",\"schedule.log\",\"preferred-content.log\",\"required-content.log\",\"group-preferred-content.log\"] -[2022-04-29 16:52:08.843105261] (Utility.Process) process [8252] done ExitSuccess -[2022-04-29 16:52:08.843428605] (Utility.Process) process [8253] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"symbolic-ref\",\"-q\",\"HEAD\"] -[2022-04-29 16:52:08.844400262] (Utility.Process) process [8253] done ExitSuccess -[2022-04-29 16:52:08.844653432] (Utility.Process) process [8254] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"refs/heads/adjusted/master(unlocked)\"] -[2022-04-29 16:52:08.845849783] (Utility.Process) process [8254] done ExitSuccess -(scanning...) [2022-04-29 16:52:08.848533041] (Merger) watching .git/refs -[2022-04-29 16:52:08.848937085] (Utility.Process) process [8255] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"ls-files\",\"-z\",\"--deleted\",\"--\",\".\"] -[2022-04-29 16:52:08.850610811] (Utility.Process) process [8255] done ExitSuccess -(started...) -[2022-04-29 16:52:08.85076117] (Watcher) watching . -[2022-04-29 16:52:10.784464555] (TransferScanner) starting scan of [Remote { name =\"bittorrent\" },Remote { name =\"web\" }] -[2022-04-29 16:52:10.786005379] (Utility.Process) process [8266] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"ls-files\",\"-z\",\"--cached\",\"--\"] -[2022-04-29 16:52:10.78835403] (Utility.Process) process [8266] done ExitSuccess -[2022-04-29 16:52:10.788482208] (TransferScanner) finished scan of [Remote { name =\"bittorrent\" },Remote { name =\"web\" }] -[2022-04-29 16:52:38.780947807] (Utility.Process) process [8244] done ExitSuccess -[2022-04-29 16:52:48.957820025] (Utility.Process) process [8373] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"cat-file\",\"--batch-check=%(objectname) %(objecttype) %(objectsize)\"] -[2022-04-29 16:52:48.961668832] (Utility.Process) process [8374] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"check-ignore\",\"-z\",\"--stdin\",\"--verbose\",\"--non-matching\"] -[2022-04-29 16:52:48.96311124] (Watcher) add exec.zip -[2022-04-29 16:52:48.963830706] (Watcher) add exec.zip -[2022-04-29 16:52:48.974374383] (Utility.Process) process [8380] read: lsof [\"-F0can\",\"+d\",\".git/annex/watchtmp/\"] -lsof: WARNING: can't stat() fuse.portal file system /run/user/1000/doc - Output information may be incomplete. -[2022-04-29 16:52:49.063916257] (Utility.Process) process [8380] done ExitFailure 1 -[2022-04-29 16:52:49.0643913] (Utility.Process) process [8391] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"check-attr\",\"-z\",\"--stdin\",\"annex.backend\",\"annex.largefiles\",\"annex.numcopies\",\"annex.mincopies\",\"--\"] -add exec.zip [2022-04-29 16:52:49.067319092] (Utility.Process) process [8392] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"filter.annex.smudge=\",\"-c\",\"filter.annex.clean=\",\"-c\",\"filter.annex.process=\",\"write-tree\"] -[2022-04-29 16:52:49.068674137] (Utility.Process) process [8392] done ExitSuccess -[2022-04-29 16:52:49.069809159] (Utility.Process) process [8393] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"--hash\",\"refs/annex/last-index\"] -[2022-04-29 16:52:49.071868139] (Utility.Process) process [8393] done ExitFailure 1 -[2022-04-29 16:52:49.086327935] (Utility.Process) process [8394] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"cat-file\",\"--batch\"] -[2022-04-29 16:52:49.089453108] (Utility.Process) process [8395] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"hash-object\",\"-w\",\"--stdin-paths\",\"--no-filters\"] -ok -[2022-04-29 16:52:49.091145891] (Committer) committing 1 changes -(recording state in git...) -[2022-04-29 16:52:49.091604661] (Utility.Process) process [8396] feed: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"update-index\",\"-z\",\"--index-info\"] -[2022-04-29 16:52:49.093194636] (Utility.Process) process [8396] done ExitSuccess -[2022-04-29 16:52:49.093559897] (Utility.Process) process [8397] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"symbolic-ref\",\"-q\",\"HEAD\"] -[2022-04-29 16:52:49.094699948] (Utility.Process) process [8397] done ExitSuccess -[2022-04-29 16:52:49.095498501] (Utility.Process) process [8398] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"--hash\",\"refs/heads/adjusted/master(unlocked)\"] -[2022-04-29 16:52:49.096545809] (Utility.Process) process [8398] done ExitSuccess -[2022-04-29 16:52:49.096818906] (Utility.Process) process [8399] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"write-tree\"] -[2022-04-29 16:52:49.098254637] (Utility.Process) process [8399] done ExitSuccess -[2022-04-29 16:52:49.099010966] (Utility.Process) process [8400] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"rev-parse\",\"--verify\",\"--quiet\",\"fe77efb32c94158eff6697b39cb344f45584102a:\"] -[2022-04-29 16:52:49.100061557] (Utility.Process) process [8400] done ExitSuccess -[2022-04-29 16:52:49.100317245] (Utility.Process) process [8401] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"commit-tree\",\"5234be1a3bf0adce43729baa1497a560b7cbebd7\",\"--no-gpg-sign\",\"-p\",\"fe77efb32c94158eff6697b39cb344f45584102a\"] -[2022-04-29 16:52:49.101685356] (Utility.Process) process [8401] done ExitSuccess -[2022-04-29 16:52:49.102371807] (Utility.Process) process [8402] call: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"update-ref\",\"refs/heads/adjusted/master(unlocked)\",\"4be985a06abd216c02f2fdd03ca40a85807bb96b\"] -[2022-04-29 16:52:49.103729957] (Utility.Process) process [8402] done ExitSuccess -[2022-04-29 16:52:49.10456259] (Utility.Process) process [8403] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"--hash\",\"refs/basis/adjusted/master(unlocked)\"] -[2022-04-29 16:52:49.105752856] (Utility.Process) process [8403] done ExitSuccess -[2022-04-29 16:52:49.106040828] (Utility.Process) process [8404] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"cat-file\",\"--batch\"] -[2022-04-29 16:52:49.107590591] (Utility.Process) process [8405] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"log\",\"refs/basis/adjusted/master(unlocked)..refs/heads/adjusted/master(unlocked)\",\"--pretty=%H\",\"--reverse\"] -[2022-04-29 16:52:49.108889132] (Utility.Process) process [8405] done ExitSuccess -[2022-04-29 16:52:49.110083161] (Utility.Process) process [8406] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show\",\"-z\",\"--raw\",\"--no-renames\",\"-l0\",\"--no-abbrev\",\"--pretty=\",\"--raw\",\"4be985a06abd216c02f2fdd03ca40a85807bb96b\"] -[2022-04-29 16:52:49.113120499] (Utility.Process) process [8407] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"mktree\",\"--batch\",\"-z\"] -[2022-04-29 16:52:49.113389197] (Utility.Process) process [8408] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"ls-tree\",\"--full-tree\",\"-z\",\"-r\",\"-t\",\"--\",\"c57819725af0809e4a82dd79efc9cb398e9b5f5e\"] -[2022-04-29 16:52:49.115008909] (Utility.Process) process [8408] done ExitSuccess -[2022-04-29 16:52:49.115220429] (Utility.Process) process [8407] done ExitSuccess -[2022-04-29 16:52:49.115274136] (Utility.Process) process [8406] done ExitSuccess -[2022-04-29 16:52:49.116504243] (Utility.Process) process [8409] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"commit-tree\",\"1ea671eeef97631e93e128bf3ebe8a6d26ea5c26\",\"--no-gpg-sign\",\"-p\",\"c57819725af0809e4a82dd79efc9cb398e9b5f5e\"] -[2022-04-29 16:52:49.118748333] (Utility.Process) process [8409] done ExitSuccess -[2022-04-29 16:52:49.119085462] (Utility.Process) process [8410] call: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"update-ref\",\"refs/basis/adjusted/master(unlocked)\",\"c5dc2761bf626f373d382e2e137a030d54cb830f\"] -[2022-04-29 16:52:49.120787007] (Utility.Process) process [8410] done ExitSuccess -[2022-04-29 16:52:49.121084941] (Utility.Process) process [8411] call: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"update-ref\",\"refs/heads/master\",\"c5dc2761bf626f373d382e2e137a030d54cb830f\"] -[2022-04-29 16:52:49.122396315] (Utility.Process) process [8411] done ExitSuccess -[2022-04-29 16:52:49.123421452] (Utility.Process) process [8412] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"commit-tree\",\"5234be1a3bf0adce43729baa1497a560b7cbebd7\",\"--no-gpg-sign\",\"-p\",\"c5dc2761bf626f373d382e2e137a030d54cb830f\"] -[2022-04-29 16:52:49.125285001] (Utility.Process) process [8412] done ExitSuccess -[2022-04-29 16:52:49.125594743] (Utility.Process) process [8413] call: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"update-ref\",\"-m\",\"rebasing adjusted branch on top of updated original branch\",\"refs/heads/adjusted/master(unlocked)\",\"5e2aa046891e88e6272556602fd783df737dd96b\"] -[2022-04-29 16:52:49.127414941] (Utility.Process) process [8413] done ExitSuccess -[2022-04-29 16:52:49.127755666] (Utility.Process) process [8414] call: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"branch\",\"-f\",\"synced/master\",\"refs/heads/master\"] -[2022-04-29 16:52:49.129567194] (Utility.Process) process [8414] done ExitSuccess -[2022-04-29 16:52:49.130874993] (Utility.Process) process [8415] feed: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"update-index\",\"-z\",\"--index-info\"] -[2022-04-29 16:52:49.13224422] (Utility.Process) process [8415] done ExitSuccess -[2022-04-29 16:52:49.132752938] (Utility.Process) process [8416] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"] -[2022-04-29 16:52:49.134298542] (Utility.Process) process [8416] done ExitSuccess -(recording state in git...) -[2022-04-29 16:52:49.134910162] (Utility.Process) process [8417] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"write-tree\"] -[2022-04-29 16:52:49.136629434] (Utility.Process) process [8417] done ExitSuccess -[2022-04-29 16:52:49.137062879] (Utility.Process) process [8418] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"commit-tree\",\"fdebaeafd3e1ebbc439d1a74184ff97e53071472\",\"--no-gpg-sign\",\"-p\",\"refs/heads/git-annex\"] -[2022-04-29 16:52:49.138527773] (Utility.Process) process [8418] done ExitSuccess -[2022-04-29 16:52:49.13896562] (Utility.Process) process [8419] call: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"update-ref\",\"refs/heads/git-annex\",\"127321afb17f5281528fc71d54c9166ce177596c\"] -[2022-04-29 16:52:49.140314152] (Utility.Process) process [8419] done ExitSuccess -[2022-04-29 16:52:49.143236655] (Utility.Process) process [8420] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"log\",\"refs/heads/master..refs/heads/synced/master\",\"--pretty=%H\",\"-n1\"] -[2022-04-29 16:52:49.144619131] (Utility.Process) process [8420] done ExitSuccess -[2022-04-29 16:52:49.144959688] (Utility.Process) process [8421] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"git-annex\"] -[2022-04-29 16:52:49.146275708] (Utility.Process) process [8421] done ExitSuccess -[2022-04-29 16:52:49.146554643] (Utility.Process) process [8422] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"] -[2022-04-29 16:52:49.147803214] (Utility.Process) process [8422] done ExitSuccess -[2022-04-29 16:52:49.148107921] (Utility.Process) process [8423] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..127321afb17f5281528fc71d54c9166ce177596c\",\"--pretty=%H\",\"-n1\"] -[2022-04-29 16:52:49.149518193] (Utility.Process) process [8423] done ExitSuccess -[2022-04-29 16:53:08.845070487] (Utility.Process) process [8451] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"ls-tree\",\"--full-tree\",\"-z\",\"--\",\"refs/heads/git-annex\",\"uuid.log\",\"remote.log\",\"trust.log\",\"group.log\",\"numcopies.log\",\"mincopies.log\",\"schedule.log\",\"preferred-content.log\",\"required-content.log\",\"group-preferred-content.log\"] -[2022-04-29 16:53:08.848424429] (Utility.Process) process [8451] done ExitSuccess -``` - -Why causes a single file `git annex repair` to fail? -"""]] diff --git a/doc/bugs/git_annex_repair_failed__58___missing_objects/comment_2_7571de4a3ab7030a877fa198fe4810b1._comment b/doc/bugs/git_annex_repair_failed__58___missing_objects/comment_2_7571de4a3ab7030a877fa198fe4810b1._comment deleted file mode 100644 index c9d2685b28..0000000000 --- a/doc/bugs/git_annex_repair_failed__58___missing_objects/comment_2_7571de4a3ab7030a877fa198fe4810b1._comment +++ /dev/null @@ -1,48 +0,0 @@ -[[!comment format=mdwn - username="Albert" - avatar="http://cdn.libravatar.org/avatar/79aaf48dabedd7cd96a9117685fbe2cc" - subject="Reproducible Example" - date="2022-04-29T15:57:34Z" - content=""" -Fast reproducible example: - -```bash -❯ mkdir annextest && cd annextest -❯ git init -Initialized empty Git repository in ./annextest/.git/ -❯ git annex init -init ok -(recording state in git...) -❯ echo \"some data\" > testfile -❯ git status -On branch master - -No commits yet - -Untracked files: - (use \"git add ...\" to include in what will be committed) - testfile - -nothing added to commit but untracked files present (use \"git add\" to track) -❯ git annex add . -add testfile -ok -(recording state in git...) -❯ git annex sync --content -commit -[master (root-commit) d5dfa14] git-annex in user@dev:~/annextest - 1 file changed, 1 insertion(+) - create mode 120000 testfile -ok -❯ git annex repair -repair Running git fsck ... -Fsck found no problems. Checking for broken branches. -Found problems, attempting repair. -Some git branches refer to missing objects: - refs/annex/last-index -To force a recovery to a usable state, retry with the --force parameter. -failed -repair: 1 failed -``` - -"""]] diff --git a/doc/bugs/git_annex_repair_failed__58___missing_objects/comment_3_317bee33fe675a5abb6fe65540a33d25._comment b/doc/bugs/git_annex_repair_failed__58___missing_objects/comment_3_317bee33fe675a5abb6fe65540a33d25._comment deleted file mode 100644 index 3e971faa3b..0000000000 --- a/doc/bugs/git_annex_repair_failed__58___missing_objects/comment_3_317bee33fe675a5abb6fe65540a33d25._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2022-05-04T15:06:31Z" - content=""" -Thank you, your "Reproducible Example" worked for me to reproduce it! -Great work finding this and tracking down an easy way to reproduce it. - -So the root of the problem is refs/annex/last-index, -which contains a tree object, not a commit object. That is unusual for a ref, -but git-annex has a good reason to record a tree's ref there. - -Fixed -"""]] diff --git a/doc/bugs/git_commit_smudges_unncessarily.mdwn b/doc/bugs/git_commit_smudges_unncessarily.mdwn deleted file mode 100644 index 47f990de75..0000000000 --- a/doc/bugs/git_commit_smudges_unncessarily.mdwn +++ /dev/null @@ -1,22 +0,0 @@ -### Please describe the problem. - -git commit smudges unlocked files in the index when they were already smudged with git status. - -### What steps will reproduce the problem? - -``` -for i in {1..200}; do echo $i > $i; done -git init; git annex init; git config annex.thin true; git config annex.crippledfilesystem true; git config annex.addunlocked true; git annex add .; git status; time git commit -m . -``` - -git commit should take less than a second but takes several seconds to smudge the files in the index. - -### What version of git-annex are you using? On what operating system? - -Through bisection, the problem was found to be introduced in [[!commit 428c91606b434512d1986622e751c795edf4df44]]. Problem occurs both in Linux and WSL1. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -git-annex has worked wonderfully for managing my files across different machines and cloud storage services. - -[[done]] diff --git a/doc/bugs/git_commit_smudges_unncessarily/comment_1_202cb8adc7a7b1119f5b4c1f5c6e9c7e._comment b/doc/bugs/git_commit_smudges_unncessarily/comment_1_202cb8adc7a7b1119f5b4c1f5c6e9c7e._comment deleted file mode 100644 index b31e0002b5..0000000000 --- a/doc/bugs/git_commit_smudges_unncessarily/comment_1_202cb8adc7a7b1119f5b4c1f5c6e9c7e._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2021-11-02T19:11:51Z" - content=""" -I doubt this bisection was correct. Whether git commit decides to smudge a -file is entirely an internal decision on the part of git; the only way -git-annex can affect it is by updating the mtime etc of the file. - -Even small things like variations in timing can change git's smudging -behavior. It may unncessarily re-smudge a file when its timestamp is -sufficiently close to the current time. For whatever internal value -of "sufficiently close" it happens to pick for a given -OS/filesystem/whatever. -"""]] diff --git a/doc/bugs/git_commit_smudges_unncessarily/comment_2_5b230c4b74da48099e8be0e264bbf3e6._comment b/doc/bugs/git_commit_smudges_unncessarily/comment_2_5b230c4b74da48099e8be0e264bbf3e6._comment deleted file mode 100644 index 1e61d6251f..0000000000 --- a/doc/bugs/git_commit_smudges_unncessarily/comment_2_5b230c4b74da48099e8be0e264bbf3e6._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2021-11-09T20:03:33Z" - content=""" -While I still need to investigate if there is really a reversion here, -it's worth noting that you can probably avoid the slow speed by running: - -git config filter.annex.process 'git-annex filter-process' - -That needs a current daily build of git-annex, the feature will also be in -the next release. -"""]] diff --git a/doc/bugs/git_commit_smudges_unncessarily/comment_3_c30e84ae438810cbab70c401fd5125c0._comment b/doc/bugs/git_commit_smudges_unncessarily/comment_3_c30e84ae438810cbab70c401fd5125c0._comment deleted file mode 100644 index 3e03ba3035..0000000000 --- a/doc/bugs/git_commit_smudges_unncessarily/comment_3_c30e84ae438810cbab70c401fd5125c0._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="asakurareiko@f3d908c71c009580228b264f63f21c7274df7476" - nickname="asakurareiko" - avatar="http://cdn.libravatar.org/avatar/a865743e357add9d15081840179ce082" - subject="comment 3" - date="2022-09-24T19:43:47Z" - content=""" -This seems to be fixed now, maybe due to the changes made related to [[todo/git_status_smudges_unncessarily_after_unlock]]. -"""]] diff --git a/doc/bugs/git_keeps_refreshing_index.mdwn b/doc/bugs/git_keeps_refreshing_index.mdwn deleted file mode 100644 index c69844ccd1..0000000000 --- a/doc/bugs/git_keeps_refreshing_index.mdwn +++ /dev/null @@ -1,12 +0,0 @@ -Since upgrading to git-annex 7.20190912, when doing `git status` I keep getting the message `Refresh index:`, and there is a delay until it turns into something like `Refresh index: 100% (601422/601422), done.`. I don't recall this happening with earlier `git-annex` versions. Have others seen this? (Not a "bug" in terms of correctness, but posting here as it affects usability and speed.) - -> This was fixed in [[!commit faf84aa5c2b963a41e56d02d6bbd2fbda7da9ceb]], -> as far as handling of unlocking of many files goes. -> -> For other cases where git needs to smudge a lot of files, -> in v9 and above repos, `git-annex filter-process` is used, -> which can speed it up significantly. -> -> While it seems that the bug submitter never followed up with -> what they were doing that caused the problem, this seems likely enough -> fixed to call it [[done]]. --[[Joey]] diff --git a/doc/bugs/git_keeps_refreshing_index/comment_1_bc76561ae4aaf932217c40109a8d04bd._comment b/doc/bugs/git_keeps_refreshing_index/comment_1_bc76561ae4aaf932217c40109a8d04bd._comment deleted file mode 100644 index 3dbdc4bd41..0000000000 --- a/doc/bugs/git_keeps_refreshing_index/comment_1_bc76561ae4aaf932217c40109a8d04bd._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2019-10-08T16:22:16Z" - content=""" -When using v7 repositories, git will do this when there are unlocked files -that have had their content changed. - -git only displays that message when it thinks its going to have to -do a lot of work (re-smudging a lot of files), so you'd mostly see this -happen in a larger repository that has had a lot of (unlocked) files change -in some way. -"""]] diff --git a/doc/bugs/git_keeps_refreshing_index/comment_2_a4c9ccb93a809aff61ae208d31ba0363._comment b/doc/bugs/git_keeps_refreshing_index/comment_2_a4c9ccb93a809aff61ae208d31ba0363._comment deleted file mode 100644 index 0770a76e32..0000000000 --- a/doc/bugs/git_keeps_refreshing_index/comment_2_a4c9ccb93a809aff61ae208d31ba0363._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2019-10-08T16:37:49Z" - content=""" -I noticed that `git annex drop` of an unlocked file causes the next `git -status` to re-smudge the file. That's surprising because git-annex -internally updates the index using git update-index, so git should not see -any need to revisit it. - -So my guess is you were getting or dropping a lot of unlocked files when -you saw that. -"""]] diff --git a/doc/bugs/git_keeps_refreshing_index/comment_3_919fb06b3fc7fe9b54797a805873e3c8._comment b/doc/bugs/git_keeps_refreshing_index/comment_3_919fb06b3fc7fe9b54797a805873e3c8._comment deleted file mode 100644 index 9ef2fd1544..0000000000 --- a/doc/bugs/git_keeps_refreshing_index/comment_3_919fb06b3fc7fe9b54797a805873e3c8._comment +++ /dev/null @@ -1,26 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2019-10-08T16:43:49Z" - content=""" -Dropping a single unlocked file and then running git -status with `GIT_TRACE=1` replicates the additional smudge every time. - -The index before and after git status smudging have identical content, so -git-annex seems to have updated it correctly already. It seems that the -mtime of the index file is causing git to do that one additional smudge. - -Eg, the file git-annex dropped had a mtime of 13:02:43.716964483. The -~/index file that git-annex generated has a mtime of 13:02:43.752964130 -which is newer, but only by a fraction of a second. So, git probably -assumes the mtimes are sufficiently close that it can't trust that the -index file really reflects the current content of the work tree file. And -so it re-smudges the work tree file unncessarily. - -I have not been able to find a number of files to drop that replicates -the bug report. When a lot of files are dropped, it starts taking -sufficiently long to update the index file that it ends up with a newer -timestamp, which avoids the unnecessary additional smudging. The worse -case I have found here is dropping 9 files causes all 9 to get re-smudged, -but that's not enough to get git to use the progress display. -"""]] diff --git a/doc/bugs/git_keeps_refreshing_index/comment_4_b73f9c5603bc69867e1619d871a8abb7._comment b/doc/bugs/git_keeps_refreshing_index/comment_4_b73f9c5603bc69867e1619d871a8abb7._comment deleted file mode 100644 index 7104488322..0000000000 --- a/doc/bugs/git_keeps_refreshing_index/comment_4_b73f9c5603bc69867e1619d871a8abb7._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2019-10-08T17:32:52Z" - content=""" -Anyway, if git-annex could preserve the mtime of an unlocked file when -writing its pointer file or when populating it with content, that would -avoid the unnecessary smudging. (Which seems better than adding a delay when -updating the index file, or setting the index's mtime ahead of the current -time..) - -That's easier done for pointer files than populated files, because -with annex.thin the content is a hard link and it would probably not be -good to change its mtime. - -For now, I didn't do it extensively, but only in depopulatePointerFile. -That was enough to eliminate the unnecessary smudging after drop that I was -seeing. -"""]] diff --git a/doc/bugs/git_keeps_refreshing_index/comment_5_f1b07147627b087231c776522ddc2826._comment b/doc/bugs/git_keeps_refreshing_index/comment_5_f1b07147627b087231c776522ddc2826._comment deleted file mode 100644 index f7e525ea64..0000000000 --- a/doc/bugs/git_keeps_refreshing_index/comment_5_f1b07147627b087231c776522ddc2826._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2019-10-22T17:10:01Z" - content=""" -There is not enough information in this bug report for me to know if the -problem I fixed was the problem that was reported. - -I'm going to assume so, if there is no followup soon. -"""]] diff --git a/doc/bugs/git_keeps_refreshing_index/comment_6_e0bfde0d53042ce8d310f356f88c610b._comment b/doc/bugs/git_keeps_refreshing_index/comment_6_e0bfde0d53042ce8d310f356f88c610b._comment deleted file mode 100644 index b561288e73..0000000000 --- a/doc/bugs/git_keeps_refreshing_index/comment_6_e0bfde0d53042ce8d310f356f88c610b._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="t+gitannex@1d62779e8b54f30a854739f61542a6885167b01f" - nickname="t+gitannex" - avatar="http://cdn.libravatar.org/avatar/87c7f62c00e4a744aa500423e421120f" - subject="comment 6" - date="2020-02-06T11:07:34Z" - content=""" -I'm able to reproduce this with git annex 7.20191230 and git 2.25.0 on Arch Linux, but I've had it on OSX in the past as well. The annex uses a v7 repository. I don't need to do anything besides unlocking some files and running git status. Unlocking 10 files, git status takes 3s and with 85 files it takes 20s, so it seems to scale linearly with the no of files. Happy to share more details about the repository if it's useful. -"""]] diff --git a/doc/bugs/git_keeps_refreshing_index/comment_7_35dff672dfd3c6fdc2b9fcdb9a58e154._comment b/doc/bugs/git_keeps_refreshing_index/comment_7_35dff672dfd3c6fdc2b9fcdb9a58e154._comment deleted file mode 100644 index ccfafef49d..0000000000 --- a/doc/bugs/git_keeps_refreshing_index/comment_7_35dff672dfd3c6fdc2b9fcdb9a58e154._comment +++ /dev/null @@ -1,59 +0,0 @@ -[[!comment format=mdwn - username="jasonb@ab4484d9961a46440958fa1a528e0fc435599057" - nickname="jasonb" - avatar="http://cdn.libravatar.org/avatar/c7330f4da122c671b935fc1d58bb02b1" - subject="I have this behavior consistently on the 2 repos I use" - date="2021-08-11T18:49:41Z" - content=""" -Here's the one. Now that a dozen changed files are committed, it doesn't happen. As long as they were modified, it happened every time. (On the other repo with 100,000+ files, it takes several minutes to complete this check. Absolutely kills my git aware shell prompt.) - -``` -git status -Refresh index: 100% (1453/1453), done. - -git annex info -trusted repositories: 0 -semitrusted repositories: 4 -... -untrusted repositories: 0 -transfers in progress: none -available local disk space: 56.15 gigabytes (+1 megabyte reserved) -local annex keys: 2159 -local annex size: 16.89 gigabytes -annexed files in working tree: 1437 -size of annexed files in working tree: 11.29 gigabytes -bloom filter size: 32 mebibytes (0.4% full) -backend usage: - SHA256E: 1437 - - -cat .git/config -[core] - repositoryformatversion = 0 - filemode = true - bare = false - logallrefupdates = true - ignorecase = true - precomposeunicode = true -[annex] - uuid = abc - version = 8 - thin = true -[filter \"annex\"] - smudge = git-annex smudge %f - clean = git-annex smudge --clean %f -[merge] - renameLimit = 1000 - -git annex version -git-annex version: 8.20210428 -build flags: Assistant Webapp Pairing FsEvents TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.28 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.4 http-client-0.7.8 persistent-sqlite-2.12.0.0 torrent-10000.1.1 uuid-1.3.14 yesod-1.6.1.1 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: darwin x86_64 -supported repository versions: 8 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 -local repository version: 8 -``` -"""]] diff --git a/doc/bugs/git_proxy_uses_ls-files_without_--exclude-standard.mdwn b/doc/bugs/git_proxy_uses_ls-files_without_--exclude-standard.mdwn deleted file mode 100644 index 85bf7089ea..0000000000 --- a/doc/bugs/git_proxy_uses_ls-files_without_--exclude-standard.mdwn +++ /dev/null @@ -1,40 +0,0 @@ -### Please describe the problem. - -I'm starting to slowly migrate my personal data collection (530GB 3.7M files) under git-annex. I'm going piece by piece, and not yet giving up my other synchronization methods (yet); thus I need to stay in direct mode. - -I initially found that `git annex` commands [were quite slow](https://github.com/datalad/datalad/issues/17), but I was able to address that by adding all my "not yet included" directories to `.gitignore` at the root of the working tree. Unfortunately, `git annex proxy` remains super slow, because I notice that it does not include `--exclude-standard` in its calls to ls-files, and thus does not respect `.gitignore`. Here's an example from the `--debug` log: - -``` - read: git ["--git-dir=../../../../../.git","--work-tree=../../../../..","--literal-pathspecs","-c","core.bare=false","ls-files","--others","-z","--","../../../../.."] -``` - -As a result, I was very shocked to learn that (25 min later), `git annex proxy` was still setting up, and had duplicated 140GB of untracked files! - -My end goal is actually just to add files directly to the git repo, bypassing the annex, in spite of being in direct mode. (I can do this with the largefiles attribute, but I'd like to be able to control it directly irrespective of size.) - -### What steps will reproduce the problem? - -``` -git annex proxy --debug -- git commit myfile -m foo -``` - -### What version of git-annex are you using? On what operating system? - -Version 6.20170520 on Mac OS 10.13.4. - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Today is my first day trying it out! It's fabulous so far, but I'm at the beginning of the learning curve. - -> Yay, git-annex proxy is deprecated and just passes through to git, since -> direct mode was eliminated. [[done]] --[[Joey]] diff --git a/doc/bugs/git_proxy_uses_ls-files_without_--exclude-standard/comment_1_81383cf73102346fa8d537e1dde30353._comment b/doc/bugs/git_proxy_uses_ls-files_without_--exclude-standard/comment_1_81383cf73102346fa8d537e1dde30353._comment deleted file mode 100644 index d0bb11b071..0000000000 --- a/doc/bugs/git_proxy_uses_ls-files_without_--exclude-standard/comment_1_81383cf73102346fa8d537e1dde30353._comment +++ /dev/null @@ -1,41 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2018-05-21T15:46:22Z" - content=""" -It wouldn't have actually copied 140 gb of files, unless you're using git-annex -on a filesystem that does not support hard links. If it used hard links, -it would not waste much space while running. - -There may be edge cases where, if git-annex proxy did not copy/hard link -ignored files from the work tree to its temporary directory, the proxied -git command would not behave the same as an unproxed git command. - -Let's see, such edge cases would have to involve a gitignored file that -is still somehow affected by the proxied git command. - -The obvious case is, you have `.*` gitignored, and you run `git annex proxy --- git add .foo --force` to add the ignored file. If git-annex didn't copy -`.foo`, that would fail, albeit in a fairly obvious way. - -Another problem case: You have `.*` gitignored, and you have a local -file `.foo` which is not checked in. You run -`git annex proxy -- git merge branch`, and the branch happens to add -`.foo` with different contents. The merge would normally fail, because -there are conflicting changes in the working tree. If proxy were -changed, the proxied merge would succeed. The local changes in this -case get lost. I've verified that this change causes data loss in this -situation. - -So, the current behavior is the safe and right behavior; git-annex should -not lose data by default to optimise for an unusual edge case. - -It could be an option, but it would have to be flagged as causing data -loss in some situations involving local modifications to gitignored files, -and causing proxied git behavior to differ from non-proxied git behavior -in other situations. I don't know if the potential benefit is worth the -foot-gun potential. - -The code change is very simple if you want to play with it. In -Command/Proxy.hs find the Git.LsFiles line and change "True" to "False". -"""]] diff --git a/doc/bugs/how_to_copy_tp_multiple_remotes_simultaneously__63__.mdwn b/doc/bugs/how_to_copy_tp_multiple_remotes_simultaneously__63__.mdwn deleted file mode 100644 index 555a8bce2d..0000000000 --- a/doc/bugs/how_to_copy_tp_multiple_remotes_simultaneously__63__.mdwn +++ /dev/null @@ -1,43 +0,0 @@ -### Please describe the problem. - -I would like to make multiple copies of precious data and I would like these copies to run simultaneously to minimise the elapsed time taken. Ideally, I would like to specify multiple `--to` options to `git annex copy`. I have tried running multiple processes simultaneously but these start to fail with lock contention on `.git/config`. - -### What steps will reproduce the problem? - -Running four simultaneous `git annex copy --to=` commands. - -### What version of git-annex are you using? On what operating system? - -OS is debian jessie, using `git-annex-standalone` from NeuroDebian. - -``` -git-annex version: 6.20160923+gitgd1dabb3-1~ndall+1 -build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotif -y XMPP ConcurrentOutput TorrentParser MagicMime Feeds Quvi -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_51 -2 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL -remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external -local repository version: 6 -supported repository versions: 5 6 -upgrade supported from repository versions: 0 1 2 3 4 5 -operating system: linux x86_64 -``` - -### Please provide any additional information below. - -What I have is a python script and its output which I can't imagine is that helpful, but if you want it I'll upload it. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Usng it regularly and like it - many thanks. This is a wishlist item, not a bug, as far as I'm concerned. - - -> [[done]], no response to my question... --[[Joey]] diff --git a/doc/bugs/how_to_copy_tp_multiple_remotes_simultaneously__63__/comment_1_311a8a43d360074abd522fff14bfcb45._comment b/doc/bugs/how_to_copy_tp_multiple_remotes_simultaneously__63__/comment_1_311a8a43d360074abd522fff14bfcb45._comment deleted file mode 100644 index 9661ac5f1f..0000000000 --- a/doc/bugs/how_to_copy_tp_multiple_remotes_simultaneously__63__/comment_1_311a8a43d360074abd522fff14bfcb45._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2016-10-26T18:02:43Z" - content=""" -You need to provide more detail about "lock contention on `.git/config`". -Ideally an actual error message. - -Normally it's perfectly fine to run multiple `git annex copy`, or any -other git-annex command for that matter. - -And the only thing I know of that locks .git/config is when the -configuration is being changed, which doesn't normally happen when -copying to a remote, unless perhaps this remote has never been used before. -"""]] diff --git a/doc/bugs/http_remotes_ignore_annex.web-options_--netrc.mdwn b/doc/bugs/http_remotes_ignore_annex.web-options_--netrc.mdwn deleted file mode 100644 index 389a95bd94..0000000000 --- a/doc/bugs/http_remotes_ignore_annex.web-options_--netrc.mdwn +++ /dev/null @@ -1,294 +0,0 @@ -### Please describe the problem. - -The documentation for `git config annex.web-options` says that I should be able to use it to set up HTTP credentials in a ~/.netrc file, but it doesn't work. - -I have been given some repos that are password-protected, I want to be able to download them non-interactively in a CI system. I won't sit there typing in the password 500 times for 500 files, and ideally I don't want to even type it once. - -`git` reads `~/.netrc` if it exists, and does so consistently enough that http://droneci.com/ has built that in as the default way it passes CI credentials to workers. It would be really great if `git-annex` did the same, and did it instead of spawning `curl`. When using an ssh remote, git and git-annex already share the same ssh credentials; it would be awesome if the same could be transparently true for http remotes as well :) - - -### What steps will reproduce the problem? - -1. Set up an HTTP server following https://git-annex.branchable.com/tips/setup_a_public_repository_on_a_web_site/, but password-protect it. - - I set up my server on Arch, but I tested the client from both Arch and Ubuntu. Here's the server set up; it should adapt to Debian or Fedora easily enough: - - 1. `sudo pacman -S --noconfirm apache` - 2. `echo 'Include conf/extra/git-annex.conf' | sudo tee -a /etc/httpd/conf/httpd.conf` - 3. `sudo mkdir -p /srv/http/annex && sudo chown -R http:http /srv/http/annex` - 4. ``` - cat < - AllowOverride All - Options FollowSymlinks Indexes - Require all granted - - ``` - 5. Set up a repo: - - 1. Switch to `http`: `sudo -u http bash`; `cd /srv/http/annex` - 2. `git config user.name httpd; git config user.email httpd@httpd` - 3. `git init; git annex init` - 4. `git config core.sharedrepository world; git config receive.denyCurrentBranch updateInstead` - 5. `mv .git/hooks/post-update.sample .git/hooks/post-update` - 6. `echo Hello > README.md && git add README.md && git commit -m "README.md"` - 7. `dd if=/dev/urandom of=large.bin bs=1M count=1 && git annex add large.bin && git add large.bin && git commit -m "large.bin"` - 5. (optional): verify the repo is functional: - - 1. `git clone http://localhost/.git annex-test; cd annex-test` - 2. `git config annex.security.allowed-ip-addresses all` - 3. `sha256sum large.bin` should fail - 4. `git annex get` - 3. `sha256sum large.bin` should succeed, and match the value shown in the symlink in `ls -l large.bin` - 6. Password protect the repo - - While still in `/srv/http/annex`: - - 1. ``` - cat < Setting this option makes git-annex use curl, but only when annex.security.allowed-ip-addresses is configured in a specific way. - -and I set `allowed-ip-addressess` in the specific way, so why is this no bueno? - -I've searched the wiki and all I've found is: - -* https://git-annex.branchable.com/news/security_fix_release/ -* https://git-annex.branchable.com/devblog/day_494__url_download_changes/ -* https://git-annex.branchable.com/forum/Use_addurl_with_a_file_on_an_HPC_cluster/ - -From these, I understand I need to `git config --global annex.security.allowed-ip-addresses all`, which I did, but otherwise my best guess is that `web-options` only works when [using the web as as _special remote_](https://git-annex.branchable.com/tips/using_the_web_as_a_special_remote/) with `addurl`. But here I'm using the web as a _regular remote_, something which [git-annex has support for](https://git-annex.branchable.com/tips/setup_a_public_repository_on_a_web_site/). But seemingly this corner case isn't working. - -I can work around it by rewriting the contents of `~/.netrc` into `~/.git-credentials` and setting `git config --global credential.helper store`, but I don't want to duplicate the credentials every time I'm in this situation. - - -### What version of git-annex are you using? On what operating system? - -git-annex 10.20220504-g4e4c44ed8 on ArchLinux, and git-annex 8.20210223 on Ubuntu 22.04. - -### Please provide any additional information below. - -[[!format sh """ - -[kousu@nigiri tmp.ztnHTYA3ZC]$ cd $(mktemp -d) -[kousu@nigiri tmp.H5EkrNMUPc]$ git config --global annex.security.allowed-ip-addresses all -[kousu@nigiri tmp.H5EkrNMUPc]$ git config --global annex.web-options --netrc -[kousu@nigiri tmp.H5EkrNMUPc]$ cat < GET / HTTP/1.1 -> Host: localhost -> User-Agent: curl/7.84.0 -> Accept: */* -> -* Mark bundle as not supporting multiuse -< HTTP/1.1 401 Unauthorized -< Date: Thu, 08 Sep 2022 00:49:44 GMT -< Server: Apache/2.4.54 (Unix) -< WWW-Authenticate: Basic realm="gitannex" -< Vary: accept-language,accept-charset -< Accept-Ranges: bytes -< Transfer-Encoding: chunked -< Content-Type: text/html; charset=utf-8 -< Content-Language: en -* The requested URL returned error: 401 -* Closing connection 0 -curl: (22) The requested URL returned error: 401 -[kousu@nigiri tmp.H5EkrNMUPc]$ curl --netrc -v -o /dev/null -f --no-progress-meter http://localhost:80 -* Trying 127.0.0.1:80... -* Connected to localhost (127.0.0.1) port 80 (#0) -* Server auth using Basic with user 'user4' -> GET / HTTP/1.1 -> Host: localhost -> Authorization: Basic dXNlcjQ6cGFzc3dvcmQ= -> User-Agent: curl/7.84.0 -> Accept: */* -> -* Mark bundle as not supporting multiuse -< HTTP/1.1 200 OK -< Date: Thu, 08 Sep 2022 00:49:41 GMT -< Server: Apache/2.4.54 (Unix) -< Content-Length: 693 -< Content-Type: text/html;charset=ISO-8859-1 -< -{ [693 bytes data] -* Connection #0 to host localhost left intact -[kousu@nigiri tmp.H5EkrNMUPc]$ -[kousu@nigiri tmp.H5EkrNMUPc]$ # demonstrate git respects .netrc: -[kousu@nigiri tmp.H5EkrNMUPc]$ git clone http://localhost/.git annex-test -Cloning into 'annex-test'... -[kousu@nigiri tmp.H5EkrNMUPc]$ cd annex-test/ -[kousu@nigiri tmp.H5EkrNMUPc]$ -[kousu@nigiri tmp.H5EkrNMUPc]$ # demonstrate that git-annex *does not* respect .netrc -[kousu@nigiri annex-test]$ git annex get -Username for 'http://localhost': ^C -[kousu@nigiri tmp.H5EkrNMUPc]$ -[kousu@nigiri tmp.H5EkrNMUPc]$ -[kousu@nigiri tmp.H5EkrNMUPc]$ -[kousu@nigiri annex-test]$ git annex version -git-annex version: 10.20220504-g4e4c44ed8 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.30 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.11 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 8 -[kousu@nigiri annex-test]$ cat /etc/os-release -NAME="Arch Linux" -PRETTY_NAME="Arch Linux" -ID=arch -BUILD_ID=rolling -ANSI_COLOR="38;2;23;147;209" -HOME_URL="https://archlinux.org/" -DOCUMENTATION_URL="https://wiki.archlinux.org/" -SUPPORT_URL="https://bbs.archlinux.org/" -BUG_REPORT_URL="https://bugs.archlinux.org/" -LOGO=archlinux-logo - -"""]] - -With the older git-annex, I set up a proxy so I could reuse the same server, which changed the port, but otherwise everything else is the same: - -[[!format sh """ - -$ ssh -R 8080:localhost:80 joplin -p115628@joplin:~$ cd $(mktemp -d) -p115628@joplin:/tmp/tmp.glF9EdYhnR$ git config --global annex.security.allowed-ip-addresses all -p115628@joplin:/tmp/tmp.glF9EdYhnR$ git config --global annex.web-options "--netrc" -p115628@joplin:/tmp/tmp.glF9EdYhnR$ git clone http://localhost:8080/.git annex-test # verify it's password protected -Cloning into 'annex-test'... -Username for 'http://localhost:8080': ^C -p115628@joplin:/tmp/tmp.glF9EdYhnR$ cat < Aside from git-credential-netrc, there is not a single mention of the netrc file in git's documentation. - -I did notice this. It is a surprising behaviour! I only discovered it because DroneCI discovered it. - -If that's your opinion, I would rather make `git config credential.helper store` the canonical solution for noninteractive passworded HTTP, since it works consistently with both `git` and `git-annex`. I think everyone would find it easier to wrap our heads around if the docs dropped the mention of netrc and instead explained that git-annex hooks into git-credential(1) and that everyone should use that. -"""]] diff --git a/doc/bugs/http_remotes_ignore_annex.web-options_--netrc/comment_4_a67253e52fa4510073d7df60d2bd4adb._comment b/doc/bugs/http_remotes_ignore_annex.web-options_--netrc/comment_4_a67253e52fa4510073d7df60d2bd4adb._comment deleted file mode 100644 index 48914d7414..0000000000 --- a/doc/bugs/http_remotes_ignore_annex.web-options_--netrc/comment_4_a67253e52fa4510073d7df60d2bd4adb._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="nick.guenther@e418ed3c763dff37995c2ed5da4232a7c6cee0a9" - nickname="nick.guenther" - avatar="http://cdn.libravatar.org/avatar/9e85c6ca61c3f877fef4f91c2bf6e278" - subject="comment 4" - date="2022-09-09T20:46:31Z" - content=""" -Sorry, this was a typo: - -> or else not[e] the corner cases where it won't invoke curl. -"""]] diff --git a/doc/bugs/http_remotes_ignore_annex.web-options_--netrc/comment_5_d81762737f3854c75e804bad1697b517._comment b/doc/bugs/http_remotes_ignore_annex.web-options_--netrc/comment_5_d81762737f3854c75e804bad1697b517._comment deleted file mode 100644 index d81ec78951..0000000000 --- a/doc/bugs/http_remotes_ignore_annex.web-options_--netrc/comment_5_d81762737f3854c75e804bad1697b517._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2022-09-13T16:47:27Z" - content=""" -I did rework the annex.web-options documentation earlier, -it now has: - - Setting this option prevents git-annex from using git-credential - for prompting for http passwords. Instead, you can include "--netrc" - to make curl use your ~/.netrc file and record the passwords there. -"""]] diff --git a/doc/bugs/http_remotes_ignore_annex.web-options_--netrc/comment_6_e13932ead5920c7599031e85127317bb._comment b/doc/bugs/http_remotes_ignore_annex.web-options_--netrc/comment_6_e13932ead5920c7599031e85127317bb._comment deleted file mode 100644 index 14182c2689..0000000000 --- a/doc/bugs/http_remotes_ignore_annex.web-options_--netrc/comment_6_e13932ead5920c7599031e85127317bb._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="nick.guenther@e418ed3c763dff37995c2ed5da4232a7c6cee0a9" - nickname="nick.guenther" - avatar="http://cdn.libravatar.org/avatar/9e85c6ca61c3f877fef4f91c2bf6e278" - subject="comment 6" - date="2022-09-13T23:32:01Z" - content=""" -That's awesome! Thanks very much joey. - -I'll mark this done now :) -"""]] diff --git a/doc/bugs/import_tree_on_adjusted_unlocked_branch_issues.mdwn b/doc/bugs/import_tree_on_adjusted_unlocked_branch_issues.mdwn deleted file mode 100644 index e668942315..0000000000 --- a/doc/bugs/import_tree_on_adjusted_unlocked_branch_issues.mdwn +++ /dev/null @@ -1,11 +0,0 @@ -The test suite spits this out during the export import test: - - unable to propigate merge commit Ref "4475560d4e3958521ad12e2715aeee244ea0cf78" back to Ref "refs/heads/master" - -It does not fail, but I think this is actually an indication of a bug. - -The ref is a merge commit, because it's imported a tree from a special remote, -and merged it. Which is a reasonable thing to want to do, even when on an -adjusted branch. So, this needs to be dealt with somehow. --[[Joey]] - -> [[done]] I think? --[[Joey]] diff --git a/doc/bugs/import_tree_on_adjusted_unlocked_branch_issues/comment_1_a6aed4a1d1958e4734684b78b0f0a3de._comment b/doc/bugs/import_tree_on_adjusted_unlocked_branch_issues/comment_1_a6aed4a1d1958e4734684b78b0f0a3de._comment deleted file mode 100644 index 5a91c15923..0000000000 --- a/doc/bugs/import_tree_on_adjusted_unlocked_branch_issues/comment_1_a6aed4a1d1958e4734684b78b0f0a3de._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="B" - avatar="http://cdn.libravatar.org/avatar/f9b0137cb32be76e5a171bff7ce98da7" - subject="Workaround" - date="2020-07-02T12:51:41Z" - content=""" -Is there any workaround in the meantime? (I'm seeing this, too). -"""]] diff --git a/doc/bugs/import_tree_on_adjusted_unlocked_branch_issues/comment_2_7fc16d6ff0f1a61907bc34e8f8b71add._comment b/doc/bugs/import_tree_on_adjusted_unlocked_branch_issues/comment_2_7fc16d6ff0f1a61907bc34e8f8b71add._comment deleted file mode 100644 index 08ff5c8e8f..0000000000 --- a/doc/bugs/import_tree_on_adjusted_unlocked_branch_issues/comment_2_7fc16d6ff0f1a61907bc34e8f8b71add._comment +++ /dev/null @@ -1,23 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2020-07-02T18:36:55Z" - content=""" -There's a simple workaround, in fact. Make a git commit, any git commit. -An empty commit will do: - - git commit --allow-empty -m "git-annex is a little bit buggy" - -git annex sync will be able to handle that commit, because it's not a -merge commit. - -It looks to me like this bug does not happen when `git annex merge` or `git -annex sync --content` are used. There needs to have been a manual -`git merge`. - -In the test suite, it happens after a merge conflict occurs, and -git add is run to resolve the conflict. When git-annex sync -is then run, it results in the problem merge commit. - -Do you have another way to reproduce this bug? -"""]] diff --git a/doc/bugs/import_tree_on_adjusted_unlocked_branch_issues/comment_3_c75eee397fc6a53aa0ff55f993b790a7._comment b/doc/bugs/import_tree_on_adjusted_unlocked_branch_issues/comment_3_c75eee397fc6a53aa0ff55f993b790a7._comment deleted file mode 100644 index 304967ade9..0000000000 --- a/doc/bugs/import_tree_on_adjusted_unlocked_branch_issues/comment_3_c75eee397fc6a53aa0ff55f993b790a7._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2023-06-21T18:44:09Z" - content=""" -The test suite doesn't output that any more, so I'm going to assume this is -fixed. -"""]] diff --git a/doc/bugs/importtree_replaces_files_with_identical_size.mdwn b/doc/bugs/importtree_replaces_files_with_identical_size.mdwn deleted file mode 100644 index bf0b78cd29..0000000000 --- a/doc/bugs/importtree_replaces_files_with_identical_size.mdwn +++ /dev/null @@ -1,65 +0,0 @@ -### Please describe the problem. - -I've been managing my about 50k photos (around 72k files) using git-annex. Recently I've decided to add a directory remote using importtree/exporttree in order to synchronize it with Nextcloud. To start slowly, I've first exported the files, then let Nextcloud upload them and then imported the files again. During the import, some files were re-imported, I think Nextcloud re-synced those files due to some errors that occurred during uploading. I've then compared the freshly create git branch to the current branch and found that while those re-imported files were not different, some other files had differences. On closer inspection, I found that these files have been replaced by identically sized files, leading, e.g., to the following diff: - -```diff -diff --git a/Profiles/Sony DSC-RX100M5A Camera Portrait.dcp b/Profiles/Sony DSC-RX100M5A Camera Portrait.dcp -index 918bbfcc8ff..8c163b32ae2 120000 ---- a/Profiles/Sony DSC-RX100M5A Camera Portrait.dcp -+++ b/Profiles/Sony DSC-RX100M5A Camera Portrait.dcp -@@ -1 +1 @@ --../.git/annex/objects/Pj/xF/SHA256E-s278110--e4a0dcd12fa02840605dbc4f7c6e258ff3002faf3fc3dc2ca262e8bbf1cf542f.dcp/SHA256E-s278110--e4a0dcd12fa02840605dbc4f7c6e258ff3002faf3fc3dc2ca262e8bbf1cf542f.dcp -\ No newline at end of file -+../.git/annex/objects/5f/Kg/SHA256E-s278110--82041bcc454f77ee5b4a55522178cdc9d5aa2d78b03176236add7e28dfac2b63.dcp/SHA256E-s278110--82041bcc454f77ee5b4a55522178cdc9d5aa2d78b03176236add7e28dfac2b63.dcp -\ No newline at end of file -``` - -These are camera profiles, and the file it has been replaced with is a different camera profile in the same directory. I've checked in the exported directory and the two camera files have the correct SHA256 hash there. To be precise, the file there has hash `e4a0dcd12fa02840605dbc4f7c6e258ff3002faf3fc3dc2ca262e8bbf1cf542f` while the commit created during import claims it has the hash `82041bcc454f77ee5b4a55522178cdc9d5aa2d78b03176236add7e28dfac2b63`. I have similar changes in Audacity project files that I also have in the same repository where 8 out of 22 files have been replaced by others. Of these 22 files, 20 have the same size. I have seen one case where two files were replaced by the same target file, apart from that from a manual inspection, the hashes of the files in the diff seem to be disjoint. - -The problem remains when running the import command again and when merging the remote, there are actually duplicates in the affected directories. I've rolled back the merge using `git reset --merge`. I assume touching the affected files in the export and re-importing could solve the problem, but I would like to preserve any state that could help with debugging for now as I would really like to know the root cause of this to avoid that this happens again. - -### What steps will reproduce the problem? - -I have tried reproducing the problem using the set of camera profiles in a new git-annex repository, but without any luck. I've also tried generating equally-sized random files, again without any luck. In theory, the steps to reproduce should be the following, but the resulting diff is always empty: - -```sh -mkdir annex-test -cd annex-test/ -git init . -git annex init -mkdir ../d -git annex initremote d type=directory directory=../d exporttree=yes importtree=yes encryption=none -# create some files -for i in $(seq 32); do dd if=/dev/urandom of=file${i}.test bs=200k count=10; done -git annex add . -git commit -m "initial commit" -git annex export master --to d -git annex import master --from d -git diff d/master -``` - -### What version of git-annex are you using? On what operating system? - -``` -git-annex version: 10.20220222-g1c4b0b4c2 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.0 ghc-9.0.2 http-client-0.7.10 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.1.2 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 8 -``` - -I'm using Arch Linux. - -### Please provide any additional information below. - -Unfortunately, I don't have the output of git-annex anymore. However, the used commands, in particular for setting up the remote, exporting and importing were identical to those listed above (apart from the name of the directory and the name of the remote). I'm also quite certain that the files that are now listed as different were not listed in the output of the import command. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -I've been successfully using git-annex to manage the growing collection of photos I take for several years now. Synchronizing the collection across several disks while files are renamed and some disks are only rarely synchronized and some contain only a subset of the photos wouldn't have been possible without git-annex. Also, using git-annex gives me the safety that the original photos will never be modified. There is nothing better than using git-annex to verify that all photos have been copied unmodified from a memory card before wiping it. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/importtree_replaces_files_with_identical_size/comment_1_2742c2cbde12aa85b8254dd6907fcfc1._comment b/doc/bugs/importtree_replaces_files_with_identical_size/comment_1_2742c2cbde12aa85b8254dd6907fcfc1._comment deleted file mode 100644 index b2637e54cc..0000000000 --- a/doc/bugs/importtree_replaces_files_with_identical_size/comment_1_2742c2cbde12aa85b8254dd6907fcfc1._comment +++ /dev/null @@ -1,36 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-03-21T16:20:27Z" - content=""" -The directory special remote use the size and mtime as the -ContentIdentifier of a file. It used to use inode number as well, but -some filesystems make up new inodes on each mount so that was dropped. - -So when two files have the same size and mtime, it will import one of them, -and then when it comes to the other one, see that it already has that -ContentIdentifier and use the content it already imported. - -For example: - - joey@darkstar:/tmp/t>git-annex initremote d type=directory directory=../d importtree=yes exporttree=yes encryption=none - joey@darkstar:/tmp/t>echo hi > ../d/1 - joey@darkstar:/tmp/t>echo no > ../d/2 - joey@darkstar:/tmp/t>touch --reference=../d/1 ../d/2 - joey@darkstar:/tmp/t>git-annex import master --from d - joey@darkstar:/tmp/t>git merge d/master - joey@darkstar:/tmp/t>cmp 1 2 - joey@darkstar:/tmp/t> - -Perhaps NextCloud is for some reason giving multiple files the same mtime? - -This is rather unlikely to happen naturally, especially on modern -filesystems that have high resolution mtimes. Even "echo 1> 1; echo 2>2" -generates files with 2 different mtimes. To get the same mtime, the two -files have to be written by the same process at very close to the same -time. - -Probably that's why NextCloud managed to get the same mtime, although why -it would be rewriting the content of the file when the content had not -changed I don't know and perhaps that's a bug of some kind in it. -"""]] diff --git a/doc/bugs/importtree_replaces_files_with_identical_size/comment_2_46336dc995a7cefb8ebbfe98f098e258._comment b/doc/bugs/importtree_replaces_files_with_identical_size/comment_2_46336dc995a7cefb8ebbfe98f098e258._comment deleted file mode 100644 index 3ce0061a02..0000000000 --- a/doc/bugs/importtree_replaces_files_with_identical_size/comment_2_46336dc995a7cefb8ebbfe98f098e258._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-03-21T16:38:43Z" - content=""" -So this makes me think that [[!73df633a6215faa093e4a7524c6d328aa988aed1]] -needs to be reverted and the inode number used again. But that did fix a -bug of a sort, that importing from a fat filesystem after it was remounted -would re-import everything. So I suppose the best that can be done is to -make it configurable whether inodes should be ignored or not when importing -from a directory special remote. And presumably default to not ignoring -them. - -done -"""]] diff --git a/doc/bugs/importtree_replaces_files_with_identical_size/comment_3_2b9bdb906043aa84ea6cf0d74a685769._comment b/doc/bugs/importtree_replaces_files_with_identical_size/comment_3_2b9bdb906043aa84ea6cf0d74a685769._comment deleted file mode 100644 index 28e0eff04f..0000000000 --- a/doc/bugs/importtree_replaces_files_with_identical_size/comment_3_2b9bdb906043aa84ea6cf0d74a685769._comment +++ /dev/null @@ -1,20 +0,0 @@ -[[!comment format=mdwn - username="michael@ff03af62c7fd492c75066bda2fbf02370f5431f4" - nickname="michael" - avatar="http://cdn.libravatar.org/avatar/125bdfa8a2b91432c072615364bc3fa1" - subject="comment 3" - date="2022-03-21T19:10:35Z" - content=""" -First of all, thank you very much for the quick fix! - -I don't think that Nextcloud rewrote these files as otherwise git-annex would have re-imported them instead of recognizing the exported files. Using stat, I confirm that the replaced files have indeed the exactly same modify time - `Modify: 2021-12-19 22:47:35.988712668 +0100`. I think this is the time when I copied these files from a different directory, as this does not correspond to the modify time of the original files (that I still have). Quite interestingly, the access time of both original files is also the exactly same time, which supports this hypothesis. I can reproduce that copying a couple of small files using cp gives me target files with exactly identical modify times (in fact, all time stamps stat displays are identical). Steps to reproduce: - -``` -for i in $(seq 32); do dd if=/dev/urandom of=file${i}.test bs=20k count=1; done -mkdir d -cp *.test d/ -stat d/*.test -``` - -For me, this reproduces both on ext4 and on tmpfs. -"""]] diff --git a/doc/bugs/importtree_replaces_files_with_identical_size/comment_4_9363ed782b5dc8d70a323fbc2574e7c7._comment b/doc/bugs/importtree_replaces_files_with_identical_size/comment_4_9363ed782b5dc8d70a323fbc2574e7c7._comment deleted file mode 100644 index bdbf6ca127..0000000000 --- a/doc/bugs/importtree_replaces_files_with_identical_size/comment_4_9363ed782b5dc8d70a323fbc2574e7c7._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2022-03-21T19:59:42Z" - content=""" -Makes sense, cp is just the kind of program that can write 2 files at very -close to the same time, and when small enough, get the same mtime. -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb.mdwn b/doc/bugs/importtree_spends_hours_reading_cidsdb.mdwn deleted file mode 100644 index 8a7594c2b9..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb.mdwn +++ /dev/null @@ -1,50 +0,0 @@ -### Please describe the problem. - -Hi! - -I've got a directory special remote set up, with importtree. After scanning through it (import filename... ok), it stops reading from the filesystem and spends hours just reading from cidsdb. Verifying with strace, there is not reading from the filesystem here. - -### What steps will reproduce the problem? - -Here are my prep steps: - -``` -git init -git config annex.thin true -git annex init 'local hub' -git annex wanted . "include=* and exclude=testdata/*" -touch mtree -git annex add mtree -git annex sync -git annex adjust --unlock-present -git annex initremote source type=directory directory=/acrypt/git-annex/bind-ro/testdata importtree=yes encryption=none -git annex enableremote source directory=/acrypt/git-annex/bind-ro/testdata -git config remote.source.annex-readonly true -git config remote.source.annex-tracking-branch main:testdata # (says to put the files in the "$REPO" directory) -git config annex.securehashesonly true -git config annex.genmetadata true -git config annex.diskreserve 100M -git annex sync -``` - -It is that last sync that runs into this problem. - -### What version of git-annex are you using? On what operating system? - -10.20230407 on Debian bullseye - -### Please provide any additional information below. - -There are about 150,000 files in that tree. This problem occurs *after* git-annex is done scanning the tree (verified with strace and lsof). This problem does not occur with the same number of files in a local/standard git-annex repo (as opposed to the directory special remote). - -.git/annex/cidsdb/db is only 51M so it is certainly entirely cached. git-annex is entirely CPU-bound at this point. - -I can rerun the sync with an unchanged import directory. It still takes 107 minutes, the majority of which is spent reading cidsdb. Only the first minute or two are spent scanning the source area. - -I have tested this on a source directory that's 2.2G and another that's 1.1T, both with about 150,000 files. After the first import, the subsequent syncs are similar in performance. In other words, this behavior appears to be related to the number of files, not the size of files. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Yes, and I hope to use it for a project to archive family photos and videos to BD-R (that's what this is about here) - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_10_a2724ab85bc91fba3bb6e55d2b1e3878._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_10_a2724ab85bc91fba3bb6e55d2b1e3878._comment deleted file mode 100644 index d8ca8820a6..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_10_a2724ab85bc91fba3bb6e55d2b1e3878._comment +++ /dev/null @@ -1,38 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""Re: comment 8""" - date="2023-06-02T16:49:35Z" - content=""" -What's the last thing output before it enters this state? I know it's at some -point after the last "import filename... ok" but I haven't a clue what it -could be. - -For what it's worth, I straced the importing stage, and there are no reads -from the database FDs at all. Presumably sqlite realizes its empty and -doesn't need to keep re-reading to respond to the queries done in that -stage. Each file imported took 113 syscalls (there are small files, so only -1 syscall needed to read the content). - -I also attached a strace immediately after it finished the importing stage, -and straced all the way to the end (importing 1000 files). There was not a -single `pread64`. FD #14 was already closed at that point, and the cidsdb -was never accessed again. - -Which matches my reading of the code that it never accesses the cidsdb after -that point. Which makes what you're seeing very strange indeed. - -Something is missing to let me reproduce this. Can you reproduce it with -the script I posted in comment #1? - ----- - -Also straced an entire import of 1000 files, and this is all the preads -from fd 14 (there were a similar amount with 3 other fds) - - joey@darkstar:~/tmp>egrep 'pread64.14' /tmp/strace - pread64(14, "", 8, 512) = 0 - pread64(14, "", 100, 0) = 0 - pread64(14, "", 16, 24) = 0 - pread64(14, "\0\0\0\1\0\0\0\1\0\0\0\0\0\0\0\0", 16, 24) = 16 - pread64(14, "SQLite format 3\0\20\0\2\2\0@ \0\0\0\1\0\0\0\1"..., 4096, 0) = 4096 -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_11_a3fc6b67529ddce9e79c0e72996259ad._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_11_a3fc6b67529ddce9e79c0e72996259ad._comment deleted file mode 100644 index 8f771e2921..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_11_a3fc6b67529ddce9e79c0e72996259ad._comment +++ /dev/null @@ -1,136 +0,0 @@ -[[!comment format=mdwn - username="jgoerzen" - avatar="http://cdn.libravatar.org/avatar/090740822c9dcdb39ffe506b890981b4" - subject="comment 11" - date="2023-06-02T21:44:53Z" - content=""" -So I ran your script on the same dataset that I was using. - -I was rather surprised that it rocketed through in 15 minutes. I then ran a subsequent git annex sync, which after a few minutes started showing the same pread64 behavior as before. - -Perhaps your improvements have been helping here? - -I went back to my original full script and thought: let's time each git-annex call. - -``` -#!/bin/bash - -set -euo pipefail - -if [ -z \"$2\" ]; then - echo \"Syntax: $0 REPO DRIVE\" - exit 5 -fi - -REPO=\"$1\" -DRIVE=\"$2\" -DRIVEBASE=\"/acrypt/no-backup/jgoerzen/git-annex-testing\" - -set -x -cd /acrypt/git-annex/repos -mkdir $REPO -cd $REPO -git init -git config annex.thin true -git annex init 'local hub' -# use newer belowgit annex wanted . \"include=mtree exclude=$REPO/*\" -git annex wanted . \"include=* and exclude=$REPO/*\" - -# Now initialize things. -touch mtree -time git annex add mtree -time git annex sync -time git annex adjust --unlock-present - -time git annex initremote source type=directory directory=/acrypt/git-annex/bind-ro/$REPO importtree=yes encryption=none - -time git annex enableremote source directory=/acrypt/git-annex/bind-ro/$REPO -git config remote.source.annex-readonly true -git config remote.source.annex-tracking-branch main:$REPO # (says to put the files in the \"$REPO\" directory) -git config annex.securehashesonly true -git config annex.genmetadata true -git config annex.diskreserve 100M - -time git annex sync - -# Now set up the clone - -cd $DRIVEBASE/$DRIVE/git-annex -time git clone /acrypt/git-annex/repos/$REPO -cd $REPO -git config annex.thin true -git annex init \"alexandria:Goerzen3\" -time git annex adjust --unlock # NTFS doesn't like symlinks -time git annex sync - -cd /acrypt/git-annex/repos/$REPO -git remote add $DRIVE $DRIVEBASE/$DRIVE/git-annex/$REPO -time git annex sync $DRIVE -git annex group $DRIVE archivedrive -git annex wanted $DRIVE 'include=mtree* or standard' # 'standard and include=mtree*' -time git annex sync - -cd $DRIVEBASE/$DRIVE/git-annex/$REPO -time git annex sync -git annex enableremote source directory=/acrypt/git-annex/bind-ro/$REPO -git config remote.source.annex-readonly true -git config remote.source.annex-tracking-branch main:$REPO # (says to put the files in the \"$REPO\" directory) -git config annex.securehashesonly true -git config annex.genmetadata true -git config annex.diskreserve 100M - -time git annex sync --content -time git annex get --auto - -cd /acrypt/git-annex/repos/$REPO -time git annex sync -``` - -In this script: - -First sync (this is before the initremote so it is doing pretty much nothing): - -real 0m0.075s -user 0m0.027s -sys 0m0.045s - -Second sync (this is after initremote/enableremote): - -real 15m1.417s -user 7m10.770s -sys 5m59.405s - -[ that correlates closely with how long it took to run your script ] - -Third sync (this is after the clone/init/adjust): - -real 0m6.606s -user 0m1.908s -sys 0m4.632s - -Fourth sync (git annex sync $DRIVE): -real 0m1.066s -user 0m0.634s -sys 0m0.647s - -Fifth sync (after the group/wanted): -real 113m32.988s -user 55m21.012s -sys 58m40.752s - -Sixth sync: -real 0m1.180s -user 0m0.819s -sys 0m0.514s - -Seventh sync (git annex sync --content): -real 112m29.594s -user 55m13.650s -sys 57m52.259s - -Eigth sync (back at the original repo): -real 110m3.099s -user 51m32.468s -sys 58m53.885s - -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_12_c1865c88d23873eb4b32489fc387f40c._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_12_c1865c88d23873eb4b32489fc387f40c._comment deleted file mode 100644 index ce05ecb617..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_12_c1865c88d23873eb4b32489fc387f40c._comment +++ /dev/null @@ -1,121 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 12""" - date="2023-06-05T16:28:05Z" - content=""" -That is a significant move of the goal posts. I don't know if that even -involves importtree being slow anymore. - -You've apparently got a lot of stuff going on in the directories used by -that script, NTFS, acrypt, some kind of removable drive, ...? - -I tried to adapt the script to a simple test case that I can run, but had -to leave all that stuff out, since I don't know anything about it. The -script I ran is below. - -I ran it with 10000 files, and the timings of the syncs were: - -1. 0m0.154s -2. 2m5.719s -3. 0m3.088s -4. 0m15.240s -5. 0m34.297s -6. 0m1.244s -7. 2m7.648s -8. 0m35.378s - -So I have not reproduced the (new) problem, #5 and #8 are not slow. - -Your 5th sync taking a long time is particularly strange because the only -difference between it and the 4th sync is that `git annex wanted` is -configured. But, `git-annex sync` does not do anything with that -configuration unless it is syncing content. That made me think you might -have annex.synccontent set in gitconfig globally. It seems to me that would -also make the 3rd sync slow, but I suppose you could have a complex git -config that sets it for some repos but not other ones or something like -that. - -Otherwise I'm out of ideas about how to reproduce your problem. - -I'll reiterate that it would be helpful to see the last output of the sync -command before it does whatever is being slow. - ------ - - #!/bin/bash - - set -euo pipefail - set -x - - NFILES=$1 - - REPO=repo - DRIVE=drive - REPOBASE=`pwd`/bench/repo - DRIVEBASE=`pwd`/bench/drive - SRBASE=`pwd`/bench/sr - - mkdir -p $REPOBASE - mkdir -p $SRBASE/$REPO - perl -e 'for(1..'$NFILES') { open(OUT,">", "'$SRBASE/$REPO'/f$_"); print OUT $_; close OUT }' - - set -x - cd $REPOBASE - mkdir $REPO - cd $REPO - git init - git config annex.thin true - git annex init 'local hub' - # use newer belowgit annex wanted . "include=mtree exclude=$REPO/*" - git annex wanted . "include=* and exclude=$REPO/*" - - # Now initialize things. - touch mtree - git annex add mtree - time git annex sync - git annex adjust --unlock-present - - git annex initremote source type=directory directory=$SRBASE/$REPO importtree=yes encryption=none - - git annex enableremote source directory=$SRBASE/$REPO - git config remote.source.annex-readonly true - git config remote.source.annex-tracking-branch master:$REPO # (says to put the files in the "$REPO" directory) - git config annex.securehashesonly true - git config annex.genmetadata true - git config annex.diskreserve 100M - - time git annex sync - - # Now set up the clone - - mkdir -p $DRIVEBASE/$DRIVE/git-annex - cd $DRIVEBASE/$DRIVE/git-annex - git clone $REPOBASE/$REPO - cd $REPO - git config annex.thin true - git annex init "alexandria:Goerzen3" - git annex adjust --unlock # NTFS doesn't like symlinks - time git annex sync - - cd $REPOBASE/$REPO - git remote add $DRIVE $DRIVEBASE/$DRIVE/git-annex/$REPO - time git annex sync $DRIVE - git annex group $DRIVE archivedrive - git annex wanted $DRIVE 'include=mtree* or standard' # 'standard and include=mtree*' - time git annex sync - - cd $DRIVEBASE/$DRIVE/git-annex/$REPO - time git annex sync - git annex enableremote source directory=$SRBASE/$REPO - git config remote.source.annex-readonly true - git config remote.source.annex-tracking-branch master:$REPO # (says to put the files in the "$REPO" directory) - git config annex.securehashesonly true - git config annex.genmetadata true - git config annex.diskreserve 100M - - time git annex sync --content - git annex get --auto - - cd $REPOBASE/$REPO - time git annex sync -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_13_6043be5152ba2d44fb9aac4063060b56._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_13_6043be5152ba2d44fb9aac4063060b56._comment deleted file mode 100644 index f119c7d78b..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_13_6043be5152ba2d44fb9aac4063060b56._comment +++ /dev/null @@ -1,24 +0,0 @@ -[[!comment format=mdwn - username="jgoerzen" - avatar="http://cdn.libravatar.org/avatar/090740822c9dcdb39ffe506b890981b4" - subject="comment 13" - date="2023-06-05T19:32:28Z" - content=""" -My apologies for forgetting that. - -So initially I thought it was after the last \"import source... ok\". I'm no longer convinced that it happens after the last one. There is clearly a phase before this starts, and a separate phase exhibiting this behavior. - -The last output from one of these runs was: - - -``` -/acrypt/git-annex/bind-ro/testdata/dvd/var..redacted..: openBinaryFile: permission denied (Permission denied) -^Mok - - Failed to import some files from source. Re-run command to resume import. -``` - -And indeed there were about 2 dozen files out of that source set that it didn't have the permission to read. - -I'm sorry it looked like expansion of scope. This is not really on NTFS; it's still on ZFS as it has been from the beginning. I had been testing with one of the repos on NTFS previously but in an effort to rule that out, I've been 100% on ZFS since before reporting this issue. \"acrypt\" is just the name of my ZFS pool and that refers to that it's atop LUKS (I don't believe this to be relevant, since the CPU time is in git-annex, not kcryptd) -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_14_e961d2b508e3260670bb61a5ae51d109._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_14_e961d2b508e3260670bb61a5ae51d109._comment deleted file mode 100644 index 42efb8208e..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_14_e961d2b508e3260670bb61a5ae51d109._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 14""" - date="2023-06-06T17:11:35Z" - content=""" -There's only one import in the sync, and your output shows it completed -(with error). - -The only other phase of sync that could be run after that and take a lot of -time is content syncing. You would have to have annex.synccontent set -somewhere for sync to do that. Do you? -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_15_35d233db5f7410048bd0cf9f5b4fd5cc._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_15_35d233db5f7410048bd0cf9f5b4fd5cc._comment deleted file mode 100644 index 2073dca187..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_15_35d233db5f7410048bd0cf9f5b4fd5cc._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 15""" - date="2023-06-06T17:31:49Z" - content=""" -It would make a lot of sense for --content syncing to be what remains slow. -That has to scan over all the files and when it decides that it does not -need to copy the content anywhere, that's a tight loop with no output. - -In my repo with 10000 files that was set up by the latest test case, -`git-annex sync` takes 13 seconds, and with --content it takes 61 seconds. - -I optimised a numcopies/mincopies lookup away, and that got it -down to 28 seconds. - -The cidsdb does not get accessed by the --content scan -in my testing, although there may be other situations where it does. -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_16_d0a7af6aecf8cb070c34edec1b3e2ecb._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_16_d0a7af6aecf8cb070c34edec1b3e2ecb._comment deleted file mode 100644 index 0973a756c5..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_16_d0a7af6aecf8cb070c34edec1b3e2ecb._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 16""" - date="2023-06-06T18:43:50Z" - content=""" -I did try making the permissions of a file not allow it to be read, and so -reproduced the "Failed to import some files from source." That didn't -change the sync time appreciably (with or without content syncing). -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_17_8fe8c0747a69ea977942a399ac929e77._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_17_8fe8c0747a69ea977942a399ac929e77._comment deleted file mode 100644 index 90fbd5e7a7..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_17_8fe8c0747a69ea977942a399ac929e77._comment +++ /dev/null @@ -1,23 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 17""" - date="2023-06-06T19:07:25Z" - content=""" -Another expensive thing sync has to do, due to the adjusted branch -being checked out, is Command.Sync.updateBranches. In my repo with 10000 -files, when not using --content, that is responsible for most of the run -time of a sync when there is nothing new to import or pull. - -That happens because `git-annex adjust --unlock-present` was used. So sync -needs to update the branch if there have been any changes to the content -that is present. Currently when in such an adjusted branch, it re-builds -the branch every time it's run. - -It would be possible to avoid that when there have been no changes to the -content that is present, but it would need to deal with eg a previous sync -command having transferred some content before it was interrupted. So it -would need some kind of flag file to be written when content changes. - -(With the master branch checked out instead, `git-annex sync --content` is -now as fast as `git-annex find --not --in source`.) -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_18_b5ea8e5a761b254672171c3ab41e09f9._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_18_b5ea8e5a761b254672171c3ab41e09f9._comment deleted file mode 100644 index e49fcdf700..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_18_b5ea8e5a761b254672171c3ab41e09f9._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""Re: comment 17""" - date="2023-06-08T20:56:25Z" - content=""" -I've fixed that, now `git-annex sync` avoids updating the adjusted branch -when there have been no changes to available content. -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_19_238c26c1a3f4ab7dd6ea3b2f26febaef._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_19_238c26c1a3f4ab7dd6ea3b2f26febaef._comment deleted file mode 100644 index fc1804db8a..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_19_238c26c1a3f4ab7dd6ea3b2f26febaef._comment +++ /dev/null @@ -1,52 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 19""" - date="2023-06-08T21:14:35Z" - content=""" -I ran the second test case with 150000 files, and here's how long the syncs -took: - -1. 0m0.170s -2. 33m37.810s -3. 0m36.644s -4. 2m58.773s -5. 13m38.126s -6. 0m3.933s -7. still running after 85 minutes -8. tbd - -Sync 2 took longer than your results in comment #11, but consistent with my -laptop being slower. And I think 33 minutes to import 150k files is fine. - -Still not seeing sync 5 take as long as it did for you. -Nowhere in the same ballpark. 13 minutes seems ok for a sync --content -that has to scan 150000 files. - -The 7th sync is seeming too slow to me. For you it took equally long as the -5th sync, and both are --content syncs. So maybe I'm seeing the same -problem but only on the 7th for some reason? - -For me it seemed to take a long time after outputting "list source ok". At -that point strace showed only a lot of futex(). And the cpu was pegged. And -it had the cidsdb open. Hmmm.. This is feeling a bit like the problem you -originally reported. - -Interrupted the 7th sync and ran again... - -The "list source" takes more than 15 minutes. It's bottlenecked on checking -git ignores. Bottleneck that I didn't notice with a smaller -number of files. Fixed that by making sure the export db was -populated, which it usually is, but not in the 7th sync's situation. -Now "list source" completes in less than 2 minutes. - -And.. after that, it was back to the tight futex() loop.. And this time I -had intrumented the cidsdb, and it was importKeys -calling getContentIdentifierKeys. - -Here's the kicker: It's only running getContentIdentifierKeys 15 times -per second. So that will take 166 minutes for all 150000 files. - -Each call to getContentIdentifierKeys is taking 0.05 seconds. -So, this bug is back to the original problem of being bottlenecked on the -cidsdb. And it is smelling like a lack of indexes. Yay! -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_1_41ca70e65a59d620cedc7cf46a8305fc._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_1_41ca70e65a59d620cedc7cf46a8305fc._comment deleted file mode 100644 index 067697755a..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_1_41ca70e65a59d620cedc7cf46a8305fc._comment +++ /dev/null @@ -1,257 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-05-30T17:53:48Z" - content=""" -Hm. I have tried to reproduce this with a git-annex instrumented to log -each time it accesses the cidsdb. - -I have not tried with the full 150k files, only with 10000, but have no -reason to think it wouldn't have the same cidsbd access pattern with more -files. - -What I found is, there was no access to the cidsdb after it finished the -"import source" part of the sync. - -But, if I then run git-annex sync a second time, it behaves much more -like you describe. It does not display "import source" for any files, -because no files have changed. And it hits the cidsdb twice per file. -Once is the necessary call to getContentIdentifierKeys, which is how it -determines it's seen a file before and doesn't need to import it a second -time. - -The other hit is recordContentIdentifier, which happens for -each recorded cid, due to updateFromLog. That seems unnecessary, because -the previous sync already recorded all the cids. - -So I think a 2x speedup should be possible to repeated syncs when most of -the files have not changed, but either I am missing something to reproduce -your problem, or half the overhead is due to git-annex determining when it's -already imported files before, which is unavoidable. - -Importing trees from special remotes is simply not going to be able to -scale to huge numbers of files. Using a git repository is always gonna be -able to perform better. (But of course hits its own scalability limits -eventually.) This is a more general mechanism, and so many of -the approaches git uses to speed up are not available. - -Here's the script I used. - - #!/bin/sh - NFILES=$1 - mkdir bench - cd bench - mkdir d - perl -e 'for(1..'$NFILES') { open(OUT,">", "d/f$_"); print OUT $_; close OUT }' - git init r - cd r - git config annex.thin true - git annex init 'local hub' - git annex wanted . "include=* and exclude=testdata/*" - touch mtree - git annex add mtree - git annex sync - git annex adjust --unlock-present - git annex initremote source type=directory directory=../d importtree=yes encryption=none - git annex enableremote source directory=../d - git config remote.source.annex-readonly true - git config remote.source.annex-tracking-branch master:testdata - git config annex.securehashesonly true - git config annex.genmetadata true - git config annex.diskreserve 100M - git annex sync - echo sync complete - -Here's the output when run with just 10 files, which shows that cidsdb -access stops after it finishes "import source": - - May 30 13:56:57 Initialized empty Git repository in /home/joey/tmp/bench/r/.git/ - May 30 13:56:57 init local hub ok - May 30 13:56:57 (recording state in git...) - May 30 13:56:57 wanted . ok - May 30 13:56:57 (recording state in git...) - May 30 13:56:57 add mtree - May 30 13:56:57 ok - May 30 13:56:57 (recording state in git...) - May 30 13:56:57 commit - May 30 13:56:57 [master (root-commit) 5ae9d8a] git-annex in local hub - May 30 13:56:57 1 file changed, 1 insertion(+) - May 30 13:56:57 create mode 120000 mtree - May 30 13:56:57 ok - May 30 13:56:57 adjust - May 30 13:56:57 Switched to branch 'adjusted/master(unlockpresent)' - May 30 13:56:57 ok - May 30 13:56:58 initremote source ok - May 30 13:56:58 (recording state in git...) - May 30 13:56:58 enableremote source ok - May 30 13:56:58 (recording state in git...) - May 30 13:56:58 commit - May 30 13:56:58 On branch adjusted/master(unlockpresent) - May 30 13:56:58 nothing to commit, working tree clean - May 30 13:56:58 ok - May 30 13:56:58 list source ok - May 30 13:56:58 "getAnnexBranchTree" - May 30 13:56:58 "updateFromLog" - May 30 13:56:58 "recordAnnexBranchTree" - May 30 13:56:58 ("getContentIdentifierKeys",ContentIdentifier "13243577 1 1685469417 098840021") - May 30 13:56:58 import source f1 - May 30 13:56:58 100% 1 B 2 KiB/s 0s("recordContentIdentifier",MkKey {keyData = Key {keyName = "6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b", keyVariety = SHA2Key (HashSize 256) (HasExt True), keySize = Just 1, keyMtime = Nothing, keyChunkSize = Nothing, keyChunkNum = Nothing}, keySerialization = "SHA256E-s1--6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b"}) - May 30 13:56:58 ok - May 30 13:56:58 ("getContentIdentifierKeys",ContentIdentifier "13285646 1 1685469417 098840021") - May 30 13:56:58 import source f8 - May 30 13:56:58 100% 1 B 2 KiB/s 0s("recordContentIdentifier",MkKey {keyData = Key {keyName = "2c624232cdd221771294dfbb310aca000a0df6ac8b66b696d90ef06fdefb64a3", keyVariety = SHA2Key (HashSize 256) (HasExt True), keySize = Just 1, keyMtime = Nothing, keyChunkSize = Nothing, keyChunkNum = Nothing}, keySerialization = "SHA256E-s1--2c624232cdd221771294dfbb310aca000a0df6ac8b66b696d90ef06fdefb64a3"}) - May 30 13:56:58 ok - May 30 13:56:58 ("getContentIdentifierKeys",ContentIdentifier "13285641 1 1685469417 098840021") - May 30 13:56:58 import source f4 - May 30 13:56:58 100% 1 B 3 KiB/s 0s("recordContentIdentifier",MkKey {keyData = Key {keyName = "4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a", keyVariety = SHA2Key (HashSize 256) (HasExt True), keySize = Just 1, keyMtime = Nothing, keyChunkSize = Nothing, keyChunkNum = Nothing}, keySerialization = "SHA256E-s1--4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a"}) - May 30 13:56:58 ok - May 30 13:56:58 ("getContentIdentifierKeys",ContentIdentifier "13285642 1 1685469417 098840021") - May 30 13:56:58 import source f5 - May 30 13:56:58 100% 1 B 3 KiB/s 0s("recordContentIdentifier",MkKey {keyData = Key {keyName = "ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d", keyVariety = SHA2Key (HashSize 256) (HasExt True), keySize = Just 1, keyMtime = Nothing, keyChunkSize = Nothing, keyChunkNum = Nothing}, keySerialization = "SHA256E-s1--ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d"}) - May 30 13:56:58 ok - May 30 13:56:58 ("getContentIdentifierKeys",ContentIdentifier "13285648 2 1685469417 098840021") - May 30 13:56:58 import source f10 - May 30 13:56:58 100% 2 B 6 KiB/s 0s("recordContentIdentifier",MkKey {keyData = Key {keyName = "4a44dc15364204a80fe80e9039455cc1608281820fe2b24f1e5233ade6af1dd5", keyVariety = SHA2Key (HashSize 256) (HasExt True), keySize = Just 2, keyMtime = Nothing, keyChunkSize = Nothing, keyChunkNum = Nothing}, keySerialization = "SHA256E-s2--4a44dc15364204a80fe80e9039455cc1608281820fe2b24f1e5233ade6af1dd5"}) - May 30 13:56:58 ok - May 30 13:56:58 ("getContentIdentifierKeys",ContentIdentifier "13285647 1 1685469417 098840021") - May 30 13:56:58 import source f9 - May 30 13:56:58 100% 1 B 4 KiB/s 0s("recordContentIdentifier",MkKey {keyData = Key {keyName = "19581e27de7ced00ff1ce50b2047e7a567c76b1cbaebabe5ef03f7c3017bb5b7", keyVariety = SHA2Key (HashSize 256) (HasExt True), keySize = Just 1, keyMtime = Nothing, keyChunkSize = Nothing, keyChunkNum = Nothing}, keySerialization = "SHA256E-s1--19581e27de7ced00ff1ce50b2047e7a567c76b1cbaebabe5ef03f7c3017bb5b7"}) - May 30 13:56:58 ok - May 30 13:56:58 ("getContentIdentifierKeys",ContentIdentifier "13285640 1 1685469417 098840021") - May 30 13:56:58 import source f3 - May 30 13:56:58 100% 1 B 5 KiB/s 0s("recordContentIdentifier",MkKey {keyData = Key {keyName = "4e07408562bedb8b60ce05c1decfe3ad16b72230967de01f640b7e4729b49fce", keyVariety = SHA2Key (HashSize 256) (HasExt True), keySize = Just 1, keyMtime = Nothing, keyChunkSize = Nothing, keyChunkNum = Nothing}, keySerialization = "SHA256E-s1--4e07408562bedb8b60ce05c1decfe3ad16b72230967de01f640b7e4729b49fce"}) - May 30 13:56:58 ok - May 30 13:56:58 ("getContentIdentifierKeys",ContentIdentifier "13285643 1 1685469417 098840021") - May 30 13:56:58 import source f6 - May 30 13:56:58 100% 1 B 4 KiB/s 0s("recordContentIdentifier",MkKey {keyData = Key {keyName = "e7f6c011776e8db7cd330b54174fd76f7d0216b612387a5ffcfb81e6f0919683", keyVariety = SHA2Key (HashSize 256) (HasExt True), keySize = Just 1, keyMtime = Nothing, keyChunkSize = Nothing, keyChunkNum = Nothing}, keySerialization = "SHA256E-s1--e7f6c011776e8db7cd330b54174fd76f7d0216b612387a5ffcfb81e6f0919683"}) - May 30 13:56:58 ok - May 30 13:56:58 ("getContentIdentifierKeys",ContentIdentifier "13285644 1 1685469417 098840021") - May 30 13:56:58 import source f7 - May 30 13:56:58 100% 1 B 4 KiB/s 0s("recordContentIdentifier",MkKey {keyData = Key {keyName = "7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451", keyVariety = SHA2Key (HashSize 256) (HasExt True), keySize = Just 1, keyMtime = Nothing, keyChunkSize = Nothing, keyChunkNum = Nothing}, keySerialization = "SHA256E-s1--7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451"}) - May 30 13:56:58 ok - May 30 13:56:58 ("getContentIdentifierKeys",ContentIdentifier "13285639 1 1685469417 098840021") - May 30 13:56:58 import source f2 - May 30 13:56:58 100% 1 B 3 KiB/s 0s("recordContentIdentifier",MkKey {keyData = Key {keyName = "d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35", keyVariety = SHA2Key (HashSize 256) (HasExt True), keySize = Just 1, keyMtime = Nothing, keyChunkSize = Nothing, keyChunkNum = Nothing}, keySerialization = "SHA256E-s1--d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35"}) - May 30 13:56:58 ok - May 30 13:56:58 update refs/remotes/source/master ok - May 30 13:56:58 (Merging into master...) - May 30 13:56:58 Merge made by the 'ort' strategy. - May 30 13:56:58 testdata/f1 | 1 + - May 30 13:56:58 testdata/f10 | 1 + - May 30 13:56:58 testdata/f2 | 1 + - May 30 13:56:58 testdata/f3 | 1 + - May 30 13:56:58 testdata/f4 | 1 + - May 30 13:56:58 testdata/f5 | 1 + - May 30 13:56:58 testdata/f6 | 1 + - May 30 13:56:58 testdata/f7 | 1 + - May 30 13:56:58 testdata/f8 | 1 + - May 30 13:56:58 testdata/f9 | 1 + - May 30 13:56:58 10 files changed, 10 insertions(+) - May 30 13:56:58 create mode 120000 testdata/f1 - May 30 13:56:58 create mode 120000 testdata/f10 - May 30 13:56:58 create mode 120000 testdata/f2 - May 30 13:56:58 create mode 120000 testdata/f3 - May 30 13:56:58 create mode 120000 testdata/f4 - May 30 13:56:58 create mode 120000 testdata/f5 - May 30 13:56:58 create mode 120000 testdata/f6 - May 30 13:56:58 create mode 120000 testdata/f7 - May 30 13:56:58 create mode 120000 testdata/f8 - May 30 13:56:58 create mode 120000 testdata/f9 - May 30 13:56:58 (Merging into adjusted branch...) - May 30 13:56:58 Updating ce828a9..f420a76 - May 30 13:56:58 Fast-forward - May 30 13:56:58 testdata/f1 | 1 + - May 30 13:56:58 testdata/f10 | 1 + - May 30 13:56:58 testdata/f2 | 1 + - May 30 13:56:58 testdata/f3 | 1 + - May 30 13:56:58 testdata/f4 | 1 + - May 30 13:56:58 testdata/f5 | 1 + - May 30 13:56:58 testdata/f6 | 1 + - May 30 13:56:58 testdata/f7 | 1 + - May 30 13:56:58 testdata/f8 | 1 + - May 30 13:56:58 testdata/f9 | 1 + - May 30 13:56:58 10 files changed, 10 insertions(+) - May 30 13:56:58 create mode 120000 testdata/f1 - May 30 13:56:58 create mode 120000 testdata/f10 - May 30 13:56:58 create mode 120000 testdata/f2 - May 30 13:56:58 create mode 120000 testdata/f3 - May 30 13:56:58 create mode 120000 testdata/f4 - May 30 13:56:58 create mode 120000 testdata/f5 - May 30 13:56:58 create mode 120000 testdata/f6 - May 30 13:56:58 create mode 120000 testdata/f7 - May 30 13:56:58 create mode 120000 testdata/f8 - May 30 13:56:58 create mode 120000 testdata/f9 - May 30 13:56:59 (recording state in git...) - -Here's the patch I used for git-annex to instrument it: - - diff --git a/Database/ContentIdentifier.hs b/Database/ContentIdentifier.hs - index e304dca58f..0d0dcdcdf1 100644 - --- a/Database/ContentIdentifier.hs - +++ b/Database/ContentIdentifier.hs - @@ -109,11 +109,14 @@ flushDbQueue (ContentIdentifierHandle h) = H.flushDbQueue h - - -- Be sure to also update the git-annex branch when using this. - recordContentIdentifier :: ContentIdentifierHandle -> RemoteStateHandle -> ContentIdentifier -> Key -> IO () - -recordContentIdentifier h (RemoteStateHandle u) cid k = queueDb h $ do - - void $ insertUniqueFast $ ContentIdentifiers u cid k - +recordContentIdentifier h (RemoteStateHandle u) cid k = do - + liftIO $ hPutStrLn stderr $ show ("recordContentIdentifier", k) - + queueDb h $ do - + void $ insertUniqueFast $ ContentIdentifiers u cid k - - getContentIdentifiers :: ContentIdentifierHandle -> RemoteStateHandle -> Key -> IO [ContentIdentifier] - -getContentIdentifiers (ContentIdentifierHandle h) (RemoteStateHandle u) k = - +getContentIdentifiers (ContentIdentifierHandle h) (RemoteStateHandle u) k = do - + liftIO $ hPutStrLn stderr $ show ("getContentIdentifiers", k) - H.queryDbQueue h $ do - l <- selectList - [ ContentIdentifiersKey ==. k - @@ -122,7 +125,8 @@ getContentIdentifiers (ContentIdentifierHandle h) (RemoteStateHandle u) k = - return $ map (contentIdentifiersCid . entityVal) l - - getContentIdentifierKeys :: ContentIdentifierHandle -> RemoteStateHandle -> ContentIdentifier -> IO [Key] - -getContentIdentifierKeys (ContentIdentifierHandle h) (RemoteStateHandle u) cid = - +getContentIdentifierKeys (ContentIdentifierHandle h) (RemoteStateHandle u) cid = do - + liftIO $ hPutStrLn stderr $ show ("getContentIdentifierKeys", cid) - H.queryDbQueue h $ do - l <- selectList - [ ContentIdentifiersCid ==. cid - @@ -131,16 +135,20 @@ getContentIdentifierKeys (ContentIdentifierHandle h) (RemoteStateHandle u) cid = - return $ map (contentIdentifiersKey . entityVal) l - - recordAnnexBranchTree :: ContentIdentifierHandle -> Sha -> IO () - -recordAnnexBranchTree h s = queueDb h $ do - - deleteWhere ([] :: [Filter AnnexBranch]) - - void $ insertUniqueFast $ AnnexBranch $ toSSha s - +recordAnnexBranchTree h s = do - + liftIO $ hPutStrLn stderr $ show ("recordAnnexBranchTree") - + queueDb h $ do - + deleteWhere ([] :: [Filter AnnexBranch]) - + void $ insertUniqueFast $ AnnexBranch $ toSSha s - - getAnnexBranchTree :: ContentIdentifierHandle -> IO Sha - -getAnnexBranchTree (ContentIdentifierHandle h) = H.queryDbQueue h $ do - - l <- selectList ([] :: [Filter AnnexBranch]) [] - - case l of - - (s:[]) -> return $ fromSSha $ annexBranchTree $ entityVal s - - _ -> return emptyTree - +getAnnexBranchTree (ContentIdentifierHandle h) = do - + liftIO $ hPutStrLn stderr $ show ("getAnnexBranchTree") - + H.queryDbQueue h $ do - + l <- selectList ([] :: [Filter AnnexBranch]) [] - + case l of - + (s:[]) -> return $ fromSSha $ annexBranchTree $ entityVal s - + _ -> return emptyTree - - {- Check if the git-annex branch has been updated and the database needs - - to be updated with any new content identifiers in it. -} - @@ -155,6 +163,7 @@ needsUpdateFromLog db = do - {- The database should be locked for write when calling this. -} - updateFromLog :: ContentIdentifierHandle -> (Sha, Sha) -> Annex () - updateFromLog db (oldtree, currtree) = do - + liftIO $ hPutStrLn stderr $ show ("updateFromLog") - (l, cleanup) <- inRepo $ - DiffTree.diffTreeRecursive oldtree currtree - mapM_ go l -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_20_d6d3db42bda1fe094140d54a458157b1._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_20_d6d3db42bda1fe094140d54a458157b1._comment deleted file mode 100644 index 15a68b211b..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_20_d6d3db42bda1fe094140d54a458157b1._comment +++ /dev/null @@ -1,67 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 20""" - date="2023-06-09T16:01:06Z" - content=""" -The sqlite schema is: - - CREATE TABLE IF NOT EXISTS "content_identifiers"("id" INTEGER - PRIMARY KEY,"remote" BLOB NOT NULL,"cid" BLOB NOT NULL,"key" BLOB - NOT NULL,CONSTRAINT "content_indentifiers_key_remote_cid_index" - UNIQUE ("key","remote","cid")); - -According to the sqlite docs: - - In most cases, UNIQUE and PRIMARY KEY constraints are implemented by - creating a unique index in the database. (The exceptions are INTEGER - PRIMARY KEY and PRIMARY KEYs on WITHOUT ROWID tables.) - Hence, the following schemas are logically equivalent: - - CREATE TABLE t1(a, b UNIQUE); - - CREATE TABLE t1(a, b PRIMARY KEY); - - CREATE TABLE t1(a, b); - CREATE UNIQUE INDEX t1b ON t1(b); - -But, that index is not used for a cid query: - - sqlite> EXPLAIN QUERY PLAN SELECT key FROM content_identifiers WHERE cid = "foo"; - QUERY PLAN - `--SCAN content_identifiers - -Aha, it does use the UNIQUE index for queries going the other way: - - sqlite> EXPLAIN QUERY PLAN SELECT cid FROM content_identifiers WHERE key = "foo"; - QUERY PLAN - `--SEARCH content_identifiers USING COVERING INDEX sqlite_autoindex_content_identifiers_1 (key=?) - -Even a query using the "remote" column seems to use the UNIQUE index: - - sqlite> EXPLAIN QUERY PLAN SELECT cid FROM content_identifiers WHERE key = "foo" and remote = "bar"; - QUERY PLAN - `--SEARCH content_identifiers USING COVERING INDEX sqlite_autoindex_content_identifiers_1 (key=? AND remote=?) - -Here's a thread on mastodon about this, with some ideas about why sqlite3 behaves this way. -. -Also, see this commit where I added a second index to a database, that was the same columns -as an existing index, only in a different order, and it did speed up a query. -[[!commit ca2a527e93ca22548983a7285fc6e0a892ca44a4]]. -Oh and there was a uniqueness constraint with the cid first, but it was broken and got removed -in [[!commit c6e693b25de49d4d3b2fedb49ffb42f04f5fd544]]. So probably it got slower at that point -(Dec 2020). - -So probably most other tables used by git-annex do have usable indexes, for -most queries. But not this one. Gonna need to manually add an index. - -(Or possibly redo the schema to have `UNIQUE ("key","cid","remote")`, it may be -that sqlite only creates an index for up to 2 columns or something like that.) - - sqlite> CREATE INDEX idx_content_identifiers_cid on content_identifiers (cid); - sqlite> EXPLAIN QUERY PLAN SELECT key FROM content_identifiers WHERE cid = "foo"; - QUERY PLAN - `--SEARCH content_identifiers USING INDEX idx_content_identifiers_cid (cid=?) - -After modifying the database this way, I re-ran the slow sync and it's now able -to run getContentIdentifierKeys 2858 times a second. What an improvement! -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_21_9320fcfb8bd47d4ee547e81fb9e01f38._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_21_9320fcfb8bd47d4ee547e81fb9e01f38._comment deleted file mode 100644 index 4e81590c22..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_21_9320fcfb8bd47d4ee547e81fb9e01f38._comment +++ /dev/null @@ -1,23 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 21""" - date="2023-06-09T17:54:29Z" - content=""" -Here's the final timing from the second test case with -150000 files after all of the above optimisation: - -1. 0m0.131s -2. 33m40.490s -3. 0m41.027s -4. 2m59.070s -5. 14m5.679s -6. 0m3.673s -7. 17m39.271s -8. 11m15.317s - -(git-annex get from an importtree remote was also sped up a lot; -it makes the same cid queries as sync #7) - -I'm satisfied with this, considering this bug done, unless it's -somehow still being slow for you. -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_22_333907125ccf24c2ad8286cd40a35b16._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_22_333907125ccf24c2ad8286cd40a35b16._comment deleted file mode 100644 index 89841e3b6b..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_22_333907125ccf24c2ad8286cd40a35b16._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="jgoerzen" - avatar="http://cdn.libravatar.org/avatar/090740822c9dcdb39ffe506b890981b4" - subject="comment 22" - date="2023-06-11T02:18:22Z" - content=""" -WOW! That was it! - -I ran the index-creation command on my cidsdb/db, then reran git-annex sync - the very long one. Down to 1m48s! A HUGE improvement! - -Thanks Joey! -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_2_8f609e231fdf315d2b658d30d708d471._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_2_8f609e231fdf315d2b658d30d708d471._comment deleted file mode 100644 index 133b943ac0..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_2_8f609e231fdf315d2b658d30d708d471._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2023-05-30T18:49:34Z" - content=""" -> The other hit is recordContentIdentifier, which happens for -> each recorded cid, due to updateFromLog. That seems unnecessary, because -> the previous sync already recorded all the cids. - -I was able to eliminate that extra work. Now the first sync does not write -to the cidsdb but only to the git-annex branch, and the second sync does -the necessary work of updating the cidsdb from the git-annex branch. - -I'm not sure that extra work is what the bug reporter was complaining -about though. -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_3_d54f2c81763b4119c881e283cff80d44._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_3_d54f2c81763b4119c881e283cff80d44._comment deleted file mode 100644 index 4ad5f9c730..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_3_d54f2c81763b4119c881e283cff80d44._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2023-05-30T19:25:11Z" - content=""" -See [[todo/speed_up_import_tree]] for a few ideas of things that would -speed it up. -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_4_76fe32e5ecbf37b0f173de987c59e39f._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_4_76fe32e5ecbf37b0f173de987c59e39f._comment deleted file mode 100644 index 761b2d00fc..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_4_76fe32e5ecbf37b0f173de987c59e39f._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="jgoerzen" - avatar="http://cdn.libravatar.org/avatar/090740822c9dcdb39ffe506b890981b4" - subject="comment 4" - date="2023-05-30T20:58:20Z" - content=""" -Hi Joey, - -Would it be helpful for you if I ran the instrumented version here? - -Also I suspect that the time spent reading cidsdb scales exponentially rather than linearly with the number of files. Or perhaps there's a magic spot somewhere (an in-memory cache limit?) that causes the excessive time. Anyhow, I'm happy to do whatever I can to help. - -Thanks! -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_5_512996afefabc75a0b2258fc05ccbfdd._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_5_512996afefabc75a0b2258fc05ccbfdd._comment deleted file mode 100644 index ef285146db..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_5_512996afefabc75a0b2258fc05ccbfdd._comment +++ /dev/null @@ -1,28 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2023-06-01T17:47:31Z" - content=""" -As far as how reads from the cidsdb scale, I think that sqlite databases -like this do slow down somewhat when tables get massive, but I don't really -know the details. And it's of course possible that something could be -improved in the schema or queries. - -I've been working on that todo I linked above, and the speed gain is -impressive when there are few or no changed files in the remote. With 20,000 -unchanged files, re-running git-annex import[1] sped up from 125.95 to 3.84 -seconds. With 40,000 unchanged files, it sped up from 477 to 8.13 seconds. I -haven't tried with 150000 files yet but the pattern is clear. - -> I can rerun the sync with an unchanged import directory. It still takes -> 107 minutes, the majority of which is spent reading cidsdb. Only the -> first minute or two are spent scanning the source area. - -Well, I think I've certianly solved that problem. But I don't know if there's -something else that is making the initial sync slower than it needs to -be. - -[1] More accurately, re-running it a second time, both to get a warm cache -result, and because the first time, it is busy updating the cidsdb with -the files that were imported earlier, as described in comment #2. -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_6_1b6ff858daf39dd84d4ef39a162ca914._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_6_1b6ff858daf39dd84d4ef39a162ca914._comment deleted file mode 100644 index bfa6fa3625..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_6_1b6ff858daf39dd84d4ef39a162ca914._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="jgoerzen" - avatar="http://cdn.libravatar.org/avatar/090740822c9dcdb39ffe506b890981b4" - subject="comment 6" - date="2023-06-01T21:26:23Z" - content=""" -That's great news! I will rerun my test against master and see what it does. - -If you happen to be able to point me at the sqlite queries that are being run, I could see if I could tease out any optimizations there. -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_6_b8b3eb5fff4ad3478dfbbd2120d4b00a._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_6_b8b3eb5fff4ad3478dfbbd2120d4b00a._comment deleted file mode 100644 index 3543623b2e..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_6_b8b3eb5fff4ad3478dfbbd2120d4b00a._comment +++ /dev/null @@ -1,35 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""Results with 150000 files""" - date="2023-06-01T18:17:34Z" - content=""" -Initial sync: 39:01.37 - -Then `git-annex import master:testdata --from source` -with no changed files: 8:36.30 -(This is writing the cidsdb since the initial sync no -longer does, see comment #2) - -Subsequent imports with no changed files: 0:37.21 - -Subsequent syncs with no changed files: 3:18.61 - -Subsequent syncs with no changed files, when not on an adjusted branch: 0:38.40 - -That 37 second run time for an import with no changed files seems to -indicate that my optimisation so far was successful at eliminating all -cidsdb reading. It's bottlenecked on git operations now. - -I also tried an initial import of 150000 files to see how much of that -initial sync was other work than the import (updating adjusted branch, -checking out files, etc). - -Initial import: 38:24.36 - -So the initial sync is mostly spending time on imports, not other stuff. -It could perhaps be sped up some. Of course it does support -J2 which may -speed it up throuh parallism. - -Seems like a sync with no changed files on an adjusted branch is doing -unncessary work to update the adjusted branch when nothing has changed. -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_8_1389db945973ed42b8ddd0de3cc8889c._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_8_1389db945973ed42b8ddd0de3cc8889c._comment deleted file mode 100644 index e30e0932dc..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_8_1389db945973ed42b8ddd0de3cc8889c._comment +++ /dev/null @@ -1,33 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 8""" - date="2023-06-02T15:25:12Z" - content=""" -@jgoerzen if you want to take a look at the sql, see -`Database/ContentIdentifier.hs`. `getContentIdentifierKeys` is the query -that it's running on each file. I'm not really sure right now if the -persistent schema in there actually creates an index that is used for that -query. persistent's documentation of indexes is lacking and I may have -misunderstood that uniqueness constraints result in indexes being created. - -Dumping the database shows this, which really doesn't seem to have an index -after all: - - CREATE TABLE IF NOT EXISTS "content_identifiers"("id" INTEGER - PRIMARY KEY,"remote" BLOB NOT NULL,"cid" BLOB NOT NULL,"key" BLOB - NOT NULL,CONSTRAINT "content_indentifiers_key_remote_cid_index" - UNIQUE ("key","remote","cid")); - -May need some raw sql to add it, like: - - CREATE INDEX cidindex ON "content_identifiers" ("cid"); - -Also, I re-ran the 150000 file sync benchmark with `getContentIdentifierKeys` -disabled and it took 29:56.78, so 25% faster. - -That gives me the idea for an optimisation -- it could check if the -database is empty at start and if so, avoid calling that at all. (It also -maintains a map in memory which will still allow it to detect duplicate files.) -Speeding up initial imports of a lot of files, but not later imports of a lot -of files is kind of a cop out, but.. -"""]] diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_8_58155dba65976fd92cd212e13f87b888._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_8_58155dba65976fd92cd212e13f87b888._comment deleted file mode 100644 index 9dc1abf12a..0000000000 --- a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_8_58155dba65976fd92cd212e13f87b888._comment +++ /dev/null @@ -1,33 +0,0 @@ -[[!comment format=mdwn - username="jgoerzen" - avatar="http://cdn.libravatar.org/avatar/090740822c9dcdb39ffe506b890981b4" - subject="comment 8" - date="2023-06-02T03:25:27Z" - content=""" -I recompiled git-annex with the latest master, and I'm seeing the same behavior as before. - -I attached strace to the process for 10 seconds and ran some statistics: - -``` -wc -l /tmp/foo -106710 /tmp/foo -egrep 'pread64.14' /tmp/foo | wc -l -103228 -``` - -fd 14 corresponds to the db file. These preads are all 4k, which matches the Sqlite page size on this database. - -There were about a dozen writes; they were to an `[eventfd]` fd. I'm not sure what that is used for here. - -The rest were mainly calles to futex, reads from the timerfd 4, and epolls. - -If it were doing one pread64 per file, at a rate of roughly 10,000 per second, it should query for every file in about 15 seconds. But as I write this, the process has used 84 minutes of CPU time. Either it is making many queries for each file, or there is some pathology in sqlite causing the queries to explode into a vast number of individual reads. - -It seems to me there is some meaningful difference between my situation and yours. - -One thing I have noticed is that it is getting permission denied on about a dozen files out of the test set. However, I don't believe this could have been the case with my original test set (consisting of about 150,000 photos). - -I am happy to do any kind of debugging I can that would be helpful. - -I note that the Sqlite default cache is about 2MB; I don't know if a larger cache may be helpful. I suspect doing fewer syscalls may help, but the underlying problem of numerous queries would still exist. -"""]] diff --git a/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10.mdwn b/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10.mdwn deleted file mode 100644 index e2072e1287..0000000000 --- a/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10.mdwn +++ /dev/null @@ -1,69 +0,0 @@ -### Please describe the problem. - -Our [datalad tests started to fail](https://github.com/datalad/datalad/issues/6369) 2 days ago in [datalad/git-annex](https://github.com/datalad/git-annex/) . - -one relates to the fact that git-annex, if `init` is provided with `--version=6` would auto upgrade all the way to 10, not `8` which is the "default" version of annex: - -``` -lena:/tmp -$> mkdir r && cd r && git init && git annex init --version=6 && git config annex.version -hint: Using 'master' as the name for the initial branch. This default branch name -hint: is subject to change. To configure the initial branch name to use in all -hint: of your new repositories, which will suppress this warning, call: -hint: -hint: git config --global init.defaultBranch -hint: -hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and -hint: 'development'. The just-created branch can be renamed via this command: -hint: -hint: git branch -m -Initialized empty Git repository in /tmp/r/.git/ -init ok -(recording state in git...) -10 -``` - -
-and it would stay at 8 if --version=8 - -``` -$> mkdir r && cd r && git init && git annex init --version=8 && git config annex.version -hint: Using 'master' as the name for the initial branch. This default branch name -hint: is subject to change. To configure the initial branch name to use in all -hint: of your new repositories, which will suppress this warning, call: -hint: -hint: git config --global init.defaultBranch -hint: -hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and -hint: 'development'. The just-created branch can be renamed via this command: -hint: -hint: git branch -m -Initialized empty Git repository in /tmp/r/.git/ -init ok -(recording state in git...) -8 - -``` -
- - -which makes it a bit odd. - -Anyways -- thought to mention this and seek feedback so either git-annex adjusts its ways or we adjust our tests to expect annex auto-upgrade any non-default version to the highest one it supports. - -### What version of git-annex are you using? On what operating system? - -``` -$> git annex version -git-annex version: 8.20211231+git126-gf7a3c1355-1~ndall+1 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 8 -``` - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_1_5b9ba56f682a991a57872d021f2ff08e._comment b/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_1_5b9ba56f682a991a57872d021f2ff08e._comment deleted file mode 100644 index 3d0e73c52d..0000000000 --- a/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_1_5b9ba56f682a991a57872d021f2ff08e._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-01-24T19:11:03Z" - content=""" -Hmm, the git-annex-upgrade man page says "Upgrades the repository to the latest version" -and v10 is the latest version now. --version=8 will do what you want. -I think that people would generally be surprised if `git-annex upgrade` did -not upgrade to the latest supported version. - -As for `git-annex init --version=6`, v6 is not a supported version, so that -was upgrading to v8 before. The git-annex-init man page says -"When the version given is one that automatically upgrades to a newer -version, it will automatically use the newer version instead." -and v6 does automatically upgrade to v10 now, so again this behavior is -following the documentation. Although so I suppose would be using v8 in the -brief amount of time v10 is not upgraded to by default. -"""]] diff --git a/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_2_419bf4e6ebce4bcaae3afc6420da80b8._comment b/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_2_419bf4e6ebce4bcaae3afc6420da80b8._comment deleted file mode 100644 index 47f4635f65..0000000000 --- a/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_2_419bf4e6ebce4bcaae3afc6420da80b8._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-01-25T17:42:32Z" - content=""" -Scratch that about `git-annex init --version`; v8 does not automatically -upgrade to v10 yet, so why should v6? I agree that seems like a bug. - -fixed -"""]] diff --git a/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_3_425e81639f39d723b390d114a6d9e24b._comment b/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_3_425e81639f39d723b390d114a6d9e24b._comment deleted file mode 100644 index 53a77b4c1e..0000000000 --- a/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_3_425e81639f39d723b390d114a6d9e24b._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="yarikoptic" - avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" - subject="comment 3" - date="2022-01-25T18:00:17Z" - content=""" -will next (10.?) release support multiple (8 9 10) versions as it now states in \"supported repository versions:\" or 8 9 will become just \"upgradeable from\"? -"""]] diff --git a/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_4_a24860bb39cf920e34fac9ee6154b69e._comment b/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_4_a24860bb39cf920e34fac9ee6154b69e._comment deleted file mode 100644 index 8bfc9b29bf..0000000000 --- a/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_4_a24860bb39cf920e34fac9ee6154b69e._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2022-01-25T18:02:16Z" - content=""" -v8 will remain supported for at least 1 year, probably a bit longer than -that. And it will still be default in the next release, and probably next -several releases on top of that. -"""]] diff --git a/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_5_3293f29b78d10aaf9254ddba1af6731d._comment b/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_5_3293f29b78d10aaf9254ddba1af6731d._comment deleted file mode 100644 index c4dbf5b443..0000000000 --- a/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_5_3293f29b78d10aaf9254ddba1af6731d._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="yarikoptic" - avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" - subject="comment 5" - date="2022-01-25T18:37:56Z" - content=""" -cool, so I think there is one missing (from documentation) notion of the \"default\" version (in addition to \"supported\", \"upgradeable\" and \"latest\" which is = max(\"supported\")). If so -- I think -- the \"default\" version should be listed in `git annex version` as \"default version:\" in addition to \"supported\" and \"upgradable\" -- `git-annex-init` man page could say \"When the version given is older than default version, it will automatically use the newer default version instead.\" -- may be `git-annex-upgrade` man page would say \"Upgrades the repository to a supported version.\", and allow for upgrade to `default` or `latest` (probably get `--version` option to specify which?) with \"default\" being the default version to upgrade to (thus staying consistent with git-annex-init upgrading to default, not latest)? - -I think that might make present difference between \"default\" and \"latest\" clearer. -"""]] diff --git a/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_6_f09aa0ebe68b5db72d466ae802ce20d5._comment b/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_6_f09aa0ebe68b5db72d466ae802ce20d5._comment deleted file mode 100644 index 20c1f636c8..0000000000 --- a/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_6_f09aa0ebe68b5db72d466ae802ce20d5._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 6""" - date="2022-01-26T17:05:34Z" - content=""" -I think you are making this a lot more complex than it will appear to any -user. These versions are almost indistinguishable by the user; v9 is -literally one git config change from v8. - -And also, I reserve the right to make it do *anything that makes sense* -around upgrades. Overspecifying the behavior will constrain future -upgrades. Repository upgrades often already involve threading a very fine -needle, making the hole artificially smaller does not seem wise. - -I suspect your test suite is unncessarily fragile around git-annex -repository version numbers, perhaps that is the real root of your -problem? -"""]] diff --git a/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_7_2d354e2c1fc2f1f6397fd89f71bf29ca._comment b/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_7_2d354e2c1fc2f1f6397fd89f71bf29ca._comment deleted file mode 100644 index 187116d459..0000000000 --- a/doc/bugs/init__58___autoupgrade___60__8_annex_version_to_8_not_10/comment_7_2d354e2c1fc2f1f6397fd89f71bf29ca._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="yarikoptic" - avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" - subject="comment regarding "default" version of the repo etc" - date="2022-01-26T18:40:50Z" - content=""" -> I think you are making this a lot more complex than it will appear to any user. These versions are almost indistinguishable by the user; v9 is literally one git config change from v8. - -well, to me as a user those upgrades become immediately visible whenever older (not yet supporting v9) version of git-annex tries to operate on v9 repository (regardless how trivial the change between 8 and 9 is). - -I recommended adding explicit \"default\" primarily because you cited man pages as the operational/design guide, and I felt that it is incomplete. - -> I suspect your test suite is unncessarily fragile around git-annex repository version numbers, perhaps that is the real root of your problem? - -might be... although I can proudly state that may be not -- we had only 2 (out of many 100s) tests failing, so at least that check seems to be quite nicely isolated in our tests. And insofar seems have pointed to a defect in git-annex (they are all back to green, woohoo! ;) ). Having stated that -- we might indeed want to adjust them. - -"""]] diff --git a/doc/bugs/introduction_of_aws-0.23_causes_stack_bld_to_fail.mdwn b/doc/bugs/introduction_of_aws-0.23_causes_stack_bld_to_fail.mdwn deleted file mode 100644 index 6f31c3303e..0000000000 --- a/doc/bugs/introduction_of_aws-0.23_causes_stack_bld_to_fail.mdwn +++ /dev/null @@ -1,53 +0,0 @@ -### Please describe the problem. - -After aws-0.23 was introduced into extra-deps of `stack.yaml`, a stack build fails -in the dependency resolution phase (a plan construction problem). Adding `aeson-2.1.1.0` -into extra-deps as advised leads to a dependency hell with conflicting versions of -attoparsec being required amongst other troubles. - -### What steps will reproduce the problem? - -`stack setup && stack build`. Observe the following output: - -[[!format sh """ -Error: While constructing the build plan, the following exceptions were encountered: - -In the dependencies for aws-0.23: - aeson-1.5.6.0 from stack configuration does not match >=2.0.0.0 (latest matching version - is 2.1.1.0) -needed due to git-annex-10.20221103 -> aws-0.23 - -Some different approaches to resolving this: - - * Set 'allow-newer: true' - in C:\hs-stack\config.yaml to ignore all version constraints and build anyway. - - * Recommended action: try adding the following to your extra-deps - in C:\Projektit\git-annex.branchable.com\git-annex--BUILD-221108-4f6c6114f\stack.yaml: - -- aeson-2.1.1.0@sha256:103ceb1421cd0ffa810bfb1acb1261d60addbde1a041fb5cce0056ff7d7dcdc2,5980 - -Plan construction failed. -# End of transcript or log. -"""]] - -### What version of git-annex are you using? On what operating system? - -I was trying to build master or [[!commit 4f6c6114fb487b5f84aa7e6922786b0e19806525]], but alas no luck. - -Windows 10 version 22H2 (build 19045.2130), 64 bit. - -Stack version 2.9.1, Git revision 409d56031b4240221d656db09b2ba476fe6bb5b1 x86_64 hpack-0.35.0 . - -### Please provide any additional information below. - -. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Sure, I use it several times a week with my multigigabyte backups, where it gives structure to my image-based backup routines, so you could say I'm a believer. :) - -[[!meta author=jkniiv]] -[[!meta title="introduction of aws-0.23 causes stack build to fail"]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/long_file_extensions___40__e.g._.numbers__41___dropped.mdwn b/doc/bugs/long_file_extensions___40__e.g._.numbers__41___dropped.mdwn deleted file mode 100644 index fb3f414c04..0000000000 --- a/doc/bugs/long_file_extensions___40__e.g._.numbers__41___dropped.mdwn +++ /dev/null @@ -1,33 +0,0 @@ -### Please describe the problem. - -Using the SHA256E backend should preserve file extensions. This works for most of my files, but not files with a `.numbers` extension. Without the extension in the filename, Numbers (Excel of Apple) can not open the file (as described elsewhere in the forums). Copying the file from the `.git/annex/objects` and re-adding the extension allows me to open the file again, so the file is intact. - -I think it might be that git annex does not recognize `.numbers` as an extension because of its length? - -### What steps will reproduce the problem? - -Adding a file with `.numbers` extension (e.g. `git annex add bids.numbers`) will drop the extension in the `.git/annex/objects` - -[[!format sh """ -bids.numbers -> ../.git/annex/objects/Xp/xZ/SHA256E-s225502--a420fa2986ddd5e4fad39758e91bc753f5f1199a64c47c84b5bd5b9c288b66c2/SHA256E-s225502--a420fa2986ddd5e4fad39758e91bc753f5f1199a64c47c84b5bd5b9c288b66c2 -"""]] - -### What version of git-annex are you using? On what operating system? - -[[!format sh """ -git-annex version: 10.20221103 -build flags: Assistant Webapp Pairing FsEvents TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22.1 bloomfilter-2.0.1.0 cryptonite-0.30 DAV-1.3.4 feed-1.3.2.1 ghc-8.10.7 http-client-0.7.13.1 persistent-sqlite-2.13.1.0 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2.1 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: darwin x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -local repository version: 10 -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -git annex makes me so happy. As I had mentioned in a previous post some years ago, it solves problems that I have not anticipated and teaches me new concepts that I then apply across my work. Thank you so much Joey! :bow: - -> [[notabug|done]] --[[Joey]] diff --git a/doc/bugs/long_file_extensions___40__e.g._.numbers__41___dropped/comment_1_0e3e0966f06ded18ecfd41f8084b4658._comment b/doc/bugs/long_file_extensions___40__e.g._.numbers__41___dropped/comment_1_0e3e0966f06ded18ecfd41f8084b4658._comment deleted file mode 100644 index 78008ecc56..0000000000 --- a/doc/bugs/long_file_extensions___40__e.g._.numbers__41___dropped/comment_1_0e3e0966f06ded18ecfd41f8084b4658._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-11-29T16:44:07Z" - content=""" -What is an extension and what is not is ambiguous -to a computer. Eg, the filename "mr.toadswildride" -does not have an extension. - -So git-annex has a heuristic, that an extension has -a maximum length of 4. Which works for the vast -majority of extensions in use. You can override -that to some other value: - - git config annex.maxextensionlength 7 -"""]] diff --git a/doc/bugs/long_file_extensions___40__e.g._.numbers__41___dropped/comment_2_2836fe9dfa7b5140df1ce893044c254b._comment b/doc/bugs/long_file_extensions___40__e.g._.numbers__41___dropped/comment_2_2836fe9dfa7b5140df1ce893044c254b._comment deleted file mode 100644 index 128336acf6..0000000000 --- a/doc/bugs/long_file_extensions___40__e.g._.numbers__41___dropped/comment_2_2836fe9dfa7b5140df1ce893044c254b._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-11-29T16:47:44Z" - content=""" -Also you might want to consider unlocking files, -either generally or on the OSX systems with this problem. -Only locked files use symlinks that cause problems for these programs. - - git-annex adjust --unlock -"""]] diff --git a/doc/bugs/long_file_extensions___40__e.g._.numbers__41___dropped/comment_3_1f4bc975621d24e6e43d887693ada800._comment b/doc/bugs/long_file_extensions___40__e.g._.numbers__41___dropped/comment_3_1f4bc975621d24e6e43d887693ada800._comment deleted file mode 100644 index a2c7877140..0000000000 --- a/doc/bugs/long_file_extensions___40__e.g._.numbers__41___dropped/comment_3_1f4bc975621d24e6e43d887693ada800._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="cnjr2" - avatar="http://cdn.libravatar.org/avatar/f7e9654cc967c8815947b19829bfd746" - subject="found related bug" - date="2022-11-29T16:51:44Z" - content=""" -I have searched around some more and found a related bug report. As per [your comment](https://git-annex.branchable.com/bugs/Wrong_backend_extension_in_files_with_multiple_dots/#comment-1e0885e62a9c22baae6300d2312c5163) what constitutes a file extension is determined by heuristics. So I guess my question would become: Would it be a bad idea to have long (and ugly) file extensions (i.e. `.numbers`) be recognized as as extensions? What would be situations where that would go wrong? -"""]] diff --git a/doc/bugs/long_file_extensions___40__e.g._.numbers__41___dropped/comment_4_2cded724d1a4368dca475b89010c23e2._comment b/doc/bugs/long_file_extensions___40__e.g._.numbers__41___dropped/comment_4_2cded724d1a4368dca475b89010c23e2._comment deleted file mode 100644 index e2e4b5abfa..0000000000 --- a/doc/bugs/long_file_extensions___40__e.g._.numbers__41___dropped/comment_4_2cded724d1a4368dca475b89010c23e2._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="cnjr2" - avatar="http://cdn.libravatar.org/avatar/f7e9654cc967c8815947b19829bfd746" - subject="comment 4" - date="2022-11-29T16:53:39Z" - content=""" -Thank you Joey - I wrote my previous comment as your answer was coming in. Much obliged! -"""]] diff --git a/doc/bugs/make___39__find__39___clear__40__er__41___on_what_is_unlocked_.mdwn b/doc/bugs/make___39__find__39___clear__40__er__41___on_what_is_unlocked_.mdwn deleted file mode 100644 index 3624584ce9..0000000000 --- a/doc/bugs/make___39__find__39___clear__40__er__41___on_what_is_unlocked_.mdwn +++ /dev/null @@ -1,65 +0,0 @@ -`git annex find --unlocked` reports also not just files which are known to `git-annex` as unlocked, but also files which were regular "locked" (symlinks) but are modified in the tree: - - -
-here is a reproducer script - -```shell -#!/bin/bash - -cd "$(mktemp -d ${TMPDIR:-/tmp}/dl-XXXXXXX)" -set -eu - -git init -git annex init - -echo 123 > 123 -git annex add 123 -git commit -m 'commit 123 locked' -git annex unlock 123 -git commit -m 'commit 123 unlocked' 123 - -set -x -git annex find --unlocked # should show 123 - -git status # should show all clean - -# let's create another file which we just modify - -echo 124 > 124 -git annex add 124 -git commit -m 'committing 124 locked' 124 -rm 124 -touch 124 - -git annex find --unlocked # will report both 123 and 124 -git status - -``` -
- -which with git-annex `8.20211231+git140-gc3817495f-1~ndall+1` would produce at the end - -``` -+ git commit -m 'committing 124 locked' 124 -[master b58274c] committing 124 locked - 1 file changed, 1 insertion(+) - create mode 120000 124 -+ rm 124 -+ touch 124 -+ git annex find --unlocked -123 -124 -+ git status -On branch master -Changes not staged for commit: - (use "git add ..." to update what will be committed) - (use "git restore ..." to discard changes in working directory) - typechange: 124 - -no changes added to commit (use "git add" and/or "git commit -a") -``` - -and I do not think it should report `124` as unlocked. Or is there some `find` way to search only for "truly unlocked" files? - -> [[notabug|done]] --[[Joey]] diff --git a/doc/bugs/make___39__find__39___clear__40__er__41___on_what_is_unlocked_/comment_1_b855c810464bb22bd76681d669313ae5._comment b/doc/bugs/make___39__find__39___clear__40__er__41___on_what_is_unlocked_/comment_1_b855c810464bb22bd76681d669313ae5._comment deleted file mode 100644 index 8ab89d2846..0000000000 --- a/doc/bugs/make___39__find__39___clear__40__er__41___on_what_is_unlocked_/comment_1_b855c810464bb22bd76681d669313ae5._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-02-21T19:16:13Z" - content=""" -The only difference between those two is that git-annex unlock stages the -unlocking, while replacing the file with unlocked content does not stage -it yet. - -Both files are unlocked in the working tree though. One is unlocked in -the index, and the other one is not yet. But git-annex find examines the -working tree. - -What you are looking for is a a way to have `git-annex find` -ignore the working tree and find files that are unlocked in the index. -That is doable: - - git-annex find --branch=$(git write-tree) --unlocked -"""]] diff --git a/doc/bugs/make___39__find__39___clear__40__er__41___on_what_is_unlocked_/comment_2_71b7b4417aa63e469636a03e460c2427._comment b/doc/bugs/make___39__find__39___clear__40__er__41___on_what_is_unlocked_/comment_2_71b7b4417aa63e469636a03e460c2427._comment deleted file mode 100644 index 4c1abdd7f7..0000000000 --- a/doc/bugs/make___39__find__39___clear__40__er__41___on_what_is_unlocked_/comment_2_71b7b4417aa63e469636a03e460c2427._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="yarikoptic" - avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" - subject="comment 2" - date="2022-02-21T21:48:59Z" - content=""" -`--branch` should be the one to remember indeed and would fulfill my desires (I think ;) ). thank you Joey! -"""]] diff --git a/doc/bugs/matching_option_--onlyingroup_works_only_partially.mdwn b/doc/bugs/matching_option_--onlyingroup_works_only_partially.mdwn deleted file mode 100644 index 706c36b6e7..0000000000 --- a/doc/bugs/matching_option_--onlyingroup_works_only_partially.mdwn +++ /dev/null @@ -1,541 +0,0 @@ -### Please describe the problem. - -Thanks for implementing the `--onlyingroup=` matching option and the corresponding -preferred content expression but it seems that at least the matching option does not work -quite all the way how it was intended (as far as I understand). - -### What steps will reproduce the problem? - -Consider the following facts: - - - there is a single file 'a-1' added to the repo annex-a; - - annex-b, annex-c, annex-d are git clones of annex-a; - - the file 'a-1' is not present in annex-d as an annex object; - - annex-a is in group 'main'; - - annex-c and annex-d are in group 'backup'; and also - - annex-b, annex-c, and annex-d are in group 'offline'. - -Also there is no such group as 'foo'. Then the following `list` and `find` commands are issued: - -[[!format sh """ -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex list -here -|annex-b -||annex-c -|||annex-d -||||web -|||||bittorrent -|||||| -XXX___ a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --inallgroup=main -a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --onlyingroup=main -a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --inallgroup=backup -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --onlyingroup=backup -a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --inallgroup=offline -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --onlyingroup=offline -a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --inallgroup=foo -a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --onlyingroup=foo -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex version --raw -10.20230802-gd19139a10d -"""]] - -Using the notation introduced by `--explain`, I would personally evaluate the above propositions as: - - - --inallgroup=main[TRUE] ok - - --onlyingroup=main[FALSE] (is present also in groups 'backup' and 'offline'), yet find considers this TRUE - - --inallgroup=backup[FALSE] (is not present in annex-d which is in 'backup') ok - - --onlyingroup=backup[FALSE] (is also present in the disjoint group 'main'), yet find considers this TRUE - - --inallgroup=offline[FALSE] (is not present in annex-d which is in 'offline') ok - - --onlyingroup=offline[FALSE] (is also present in the disjoint group 'main'), yet find considers this TRUE - - --inallgroup=foo[FALSE] (there is no such group 'foo' so how can the file be present?), yet find considers this TRUE - - --onlyingroup=foo[FALSE] ok - -So, what gives? It seems that `--onlyingroup` only makes sure that the first part of the following specification -matches: - - Matches only when git-annex believes content is present in at least one - repository that is in the specified group, and is not present in any - repositories that are not in the specified group. - -Notice that (in at least the particular case above) it doesn't make sure that -the content should not be present in any repositories that are in disjoint groups. -So the "only" meaning seems to be missing here making this effectively into a (lax) -`--ingroup=` matching option. - -### What version of git-annex are you using? On what operating system? - -[[!format sh """ -git-annex version: 10.20230802-gd19139a10d -build flags: Assistant Webapp Pairing TorrentParser Benchmark Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.7 http-client-0.7.9 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.1.2 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: mingw32 x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 2 3 4 5 6 7 8 9 10 -local repository version: 10 -"""]] - -This is a build that has been attained by using the Windows-specific stack.yaml -(by giving the option `--stack-yaml stack-lts-18.13.yaml` for stack). - -Windows 10 version 22H2 (build 19045.3208), 64 bit. - -### Please provide any additional information below. - -My apologies in advance for the long transcript :) - -[[!format sh """ -PowerShell 7.3.6 -* Istunto aloitettu hetkellä la 05.08.2023 klo 00:28:40 +03:00. Session started on Sat 05-Aug-2023 @ 00:28:41 +03:00. -Loading personal and system profiles took 2630ms. -C:\Users\jkniiv> cd e:\git-annex-tests -E:\git-annex-tests> mkdir test-onlyingroup-matching2 - - Directory: E:\git-annex-tests - -Mode LastWriteTime Length Name ----- ------------- ------ ---- -d---- 5.8.2023 0:29 test-onlyingroup-matching2 - -E:\git-annex-tests> cd .\test-onlyingroup-matching2\ -E:\git-annex-tests\test-onlyingroup-matching2> git init annex-a -Initialized empty Git repository in E:/git-annex-tests/test-onlyingroup-matching2/annex-a/.git/ -E:\git-annex-tests\test-onlyingroup-matching2> cd annex-a -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [master]> git annex init annex-a -init annex-a - Detected a filesystem without fifo support. - - Disabling ssh connection caching. - - Detected a crippled filesystem. - - Entering an adjusted branch where files are unlocked as this filesystem does not support locked files. - -Switched to branch 'adjusted/master(unlocked)' -ok -(recording state in git...) -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> cd .. -E:\git-annex-tests\test-onlyingroup-matching2> git clone annex-a annex-b -Cloning into 'annex-b'... -done. -E:\git-annex-tests\test-onlyingroup-matching2> git clone annex-a annex-c -Cloning into 'annex-c'... -done. -E:\git-annex-tests\test-onlyingroup-matching2> git clone annex-a annex-d -Cloning into 'annex-d'... -done. -E:\git-annex-tests\test-onlyingroup-matching2> ls - - Directory: E:\git-annex-tests\test-onlyingroup-matching2 - -Mode LastWriteTime Length Name ----- ------------- ------ ---- -d---- 5.8.2023 0:29 annex-a -d---- 5.8.2023 0:32 annex-b -d---- 5.8.2023 0:32 annex-c -d---- 5.8.2023 0:32 annex-d - -E:\git-annex-tests\test-onlyingroup-matching2> cd annex-b -E:\git-annex-tests\test-onlyingroup-matching2\annex-b [adjusted/master(unlocked) ≡]> git config --add remote.origin.fetch '^refs/heads/adjusted/*' -E:\git-annex-tests\test-onlyingroup-matching2\annex-b [adjusted/master(unlocked) ≡]> cat .git\config -[core] - repositoryformatversion = 0 - filemode = false - bare = false - logallrefupdates = true - symlinks = false - ignorecase = true -[remote "origin"] - url = E:/git-annex-tests/test-onlyingroup-matching2/annex-a - fetch = +refs/heads/*:refs/remotes/origin/* - fetch = ^refs/heads/adjusted/* -[branch "adjusted/master(unlocked)"] - remote = origin - merge = refs/heads/adjusted/master(unlocked) -E:\git-annex-tests\test-onlyingroup-matching2\annex-b [adjusted/master(unlocked) ≡]> git annex init annex-b -init annex-b - Detected a filesystem without fifo support. - - Disabling ssh connection caching. - - Detected a crippled filesystem. -ok -(recording state in git...) -E:\git-annex-tests\test-onlyingroup-matching2\annex-b [adjusted/master(unlocked) ≡]> cd ..\annex-c -E:\git-annex-tests\test-onlyingroup-matching2\annex-c [adjusted/master(unlocked) ≡]> git config --add remote.origin.fetch '^refs/heads/adjusted/*' -E:\git-annex-tests\test-onlyingroup-matching2\annex-c [adjusted/master(unlocked) ≡]> git annex init annex-c -init annex-c -[...snip...] -(recording state in git...) -E:\git-annex-tests\test-onlyingroup-matching2\annex-c [adjusted/master(unlocked) ≡]> cd ..\annex-d -E:\git-annex-tests\test-onlyingroup-matching2\annex-d [adjusted/master(unlocked) ≡]> git config --add remote.origin.fetch '^refs/heads/adjusted/*' -E:\git-annex-tests\test-onlyingroup-matching2\annex-d [adjusted/master(unlocked) ≡]> git annex init annex-d -init annex-d -[...snip...] -(recording state in git...) -E:\git-annex-tests\test-onlyingroup-matching2\annex-d [adjusted/master(unlocked) ≡]> cd ..\annex-a -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git remote add annex-b ..\annex-b -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git remote add annex-c ..\annex-c -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git remote add annex-d ..\annex-d -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git config --add remote.annex-b.fetch '^refs/heads/adjusted/*' -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git config --add remote.annex-c.fetch '^refs/heads/adjusted/*' -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git config --add remote.annex-d.fetch '^refs/heads/adjusted/*' -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex sync -g -commit -On branch adjusted/master(unlocked) -nothing to commit, working tree clean -ok -pull annex-d -remote: Enumerating objects: 5, done. -remote: Counting objects: 100% (5/5), done. -remote: Compressing objects: 100% (2/2), done. -remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0 -Unpacking objects: 100% (3/3), 332 bytes | 30.00 KiB/s, done. -From ..\annex-d - * [new branch] git-annex -> annex-d/git-annex - * [new branch] master -> annex-d/master -ok -pull annex-c -[...snip...] -pull annex-b -[...snip...] -(merging annex-b/git-annex annex-c/git-annex annex-d/git-annex into git-annex...) -(recording state in git...) -push annex-d -Enumerating objects: 13, done. -Counting objects: 100% (13/13), done. -Delta compression using up to 4 threads -Compressing objects: 100% (6/6), done. -Writing objects: 100% (9/9), 843 bytes | 421.00 KiB/s, done. -Total 9 (delta 4), reused 0 (delta 0), pack-reused 0 -To ..\annex-d - * [new branch] master -> synced/master - * [new branch] git-annex -> synced/git-annex -ok -push annex-c -[...snip...] -push annex-b -[...snip...] -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex wanted . present -wanted . ok -(recording state in git...) -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex group . main -group . ok -(recording state in git...) -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> cd ..\annex-b -E:\git-annex-tests\test-onlyingroup-matching2\annex-b [adjusted/master(unlocked) ≡]> git annex group . offline -group . ok -(recording state in git...) -E:\git-annex-tests\test-onlyingroup-matching2\annex-b [adjusted/master(unlocked) ≡]> cd ..\annex-c -E:\git-annex-tests\test-onlyingroup-matching2\annex-c [adjusted/master(unlocked) ≡]> git annex wanted . standard -wanted . ok -(recording state in git...) -E:\git-annex-tests\test-onlyingroup-matching2\annex-c [adjusted/master(unlocked) ≡]> git annex group . backup -group . ok -(recording state in git...) -E:\git-annex-tests\test-onlyingroup-matching2\annex-c [adjusted/master(unlocked) ≡]> git annex group . offline -group . ok -(recording state in git...) -E:\git-annex-tests\test-onlyingroup-matching2\annex-c [adjusted/master(unlocked) ≡]> git annex group . -backup offline -E:\git-annex-tests\test-onlyingroup-matching2\annex-c [adjusted/master(unlocked) ≡]> cd ..\annex-d -E:\git-annex-tests\test-onlyingroup-matching2\annex-d [adjusted/master(unlocked) ≡]> git annex wanted . standard -wanted . ok -(recording state in git...) -E:\git-annex-tests\test-onlyingroup-matching2\annex-d [adjusted/master(unlocked) ≡]> git annex group . backup -group . ok -(recording state in git...) -E:\git-annex-tests\test-onlyingroup-matching2\annex-d [adjusted/master(unlocked) ≡]> git annex group . offline -group . ok -(recording state in git...) -E:\git-annex-tests\test-onlyingroup-matching2\annex-d [adjusted/master(unlocked) ≡]> git annex group . -backup offline -E:\git-annex-tests\test-onlyingroup-matching2\annex-d [adjusted/master(unlocked) ≡]> cd ..\annex-a -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> ls -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> echo aaa-1 > a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked) +1 ~0 -0 !]> ls - - Directory: E:\git-annex-tests\test-onlyingroup-matching2\annex-a - -Mode LastWriteTime Length Name ----- ------------- ------ ---- --a--- 5.8.2023 0:45 7 a-1 - -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked) +1 ~0 -0 !]> git annex add .\a-1 -add a-1 -ok -(recording state in git...) -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked) +1 ~0 -0 ~]> git status -On branch adjusted/master(unlocked) -Changes to be committed: - (use "git restore --staged ..." to unstage) - new file: a-1 - -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked) +1 ~0 -0 ~]> git commit -m 'add a-1' -[adjusted/master(unlocked) 1fad99e] add a-1 - 1 file changed, 1 insertion(+) - create mode 100644 a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> ls - - Directory: E:\git-annex-tests\test-onlyingroup-matching2\annex-a - -Mode LastWriteTime Length Name ----- ------------- ------ ---- --a--- 5.8.2023 0:45 7 a-1 - -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex sync --content -commit -On branch adjusted/master(unlocked) -nothing to commit, working tree clean -ok -pull annex-d -remote: Enumerating objects: 10, done. -remote: Counting objects: 100% (10/10), done. -remote: Compressing objects: 100% (9/9), done. -remote: Total 9 (delta 1), reused 0 (delta 0), pack-reused 0 -Unpacking objects: 100% (9/9), 1008 bytes | 56.00 KiB/s, done. -From ..\annex-d - ee601cf..9c59e4d git-annex -> annex-d/git-annex -ok -pull annex-c -[...snip...] -pull annex-b -[...snip...] -(merging annex-b/git-annex annex-c/git-annex annex-d/git-annex into git-annex...) -(recording state in git...) -copy a-1 (to annex-d...) -ok -copy a-1 (to annex-c...) -ok -copy a-1 (to annex-b...) -ok -pull annex-d -ok -pull annex-c -ok -pull annex-b -ok -(recording state in git...) -push annex-d -Enumerating objects: 40, done. -Counting objects: 100% (40/40), done. -Delta compression using up to 4 threads -Compressing objects: 100% (32/32), done. -Writing objects: 100% (35/35), 3.65 KiB | 623.00 KiB/s, done. -Total 35 (delta 7), reused 0 (delta 0), pack-reused 0 -To ..\annex-d - ee601cf..7a0c374 git-annex -> synced/git-annex - c9731e7..6e57e3e master -> synced/master -ok -push annex-c -[...snip...] -push annex-b -[...snip...] -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> cd .. -E:\git-annex-tests\test-onlyingroup-matching2> ls - - Directory: E:\git-annex-tests\test-onlyingroup-matching2 - -Mode LastWriteTime Length Name ----- ------------- ------ ---- -d---- 5.8.2023 0:45 annex-a -d---- 5.8.2023 0:32 annex-b -d---- 5.8.2023 0:32 annex-c -d---- 5.8.2023 0:32 annex-d - -E:\git-annex-tests\test-onlyingroup-matching2> cd annex-b -E:\git-annex-tests\test-onlyingroup-matching2\annex-b [adjusted/master(unlocked) ≡]> git annex sync -g -commit -On branch adjusted/master(unlocked) -Your branch is up to date with 'origin/adjusted/master(unlocked)'. - -nothing to commit, working tree clean -ok -merge synced/master master (Merging into master...) -Updating c9731e7..6e57e3e -Fast-forward - a-1 | 1 + - 1 file changed, 1 insertion(+) - create mode 120000 a-1 -(Merging into adjusted branch...) -Updating bffc83c..4daa3a7 -Fast-forward - a-1 | 1 + - 1 file changed, 1 insertion(+) - create mode 100644 a-1 - -Already up to date. -(Merging into adjusted branch...) -Updating 76cec61..68d290a -Fast-forward -ok -pull origin -[...snip...] -push origin -[...snip...] -E:\git-annex-tests\test-onlyingroup-matching2\annex-b [adjusted/master(unlocked) ↓1 ↑2]> ls - - Directory: E:\git-annex-tests\test-onlyingroup-matching2\annex-b - -Mode LastWriteTime Length Name ----- ------------- ------ ---- --a--- 5.8.2023 0:47 7 a-1 - -E:\git-annex-tests\test-onlyingroup-matching2\annex-b [adjusted/master(unlocked) ↓1 ↑2]> cat a-1 -aaa-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-b [adjusted/master(unlocked) ↓1 ↑2]> cd ..\annex-c -E:\git-annex-tests\test-onlyingroup-matching2\annex-c [adjusted/master(unlocked) ≡]> git annex sync -g -commit -[...snip...] -merge synced/master master (Merging into master...) -[...snip...] -(Merging into adjusted branch...) -[...snip...] -Already up to date. -(Merging into adjusted branch...) -[...snip...] -(merging origin/git-annex into git-annex...) -(recording state in git...) -[...snip...] -E:\git-annex-tests\test-onlyingroup-matching2\annex-c [adjusted/master(unlocked) ↓1 ↑2]> ls - - Directory: E:\git-annex-tests\test-onlyingroup-matching2\annex-c - -Mode LastWriteTime Length Name ----- ------------- ------ ---- --a--- 5.8.2023 0:47 7 a-1 - -E:\git-annex-tests\test-onlyingroup-matching2\annex-c [adjusted/master(unlocked) ↓1 ↑2]> cd ..\annex-d -E:\git-annex-tests\test-onlyingroup-matching2\annex-d [adjusted/master(unlocked) ≡]> git annex sync -g -commit -[...snip...] -merge synced/master master (Merging into master...) -[...snip...] -(Merging into adjusted branch...) -[...snip...] -Already up to date. -(Merging into adjusted branch...) -[...snip...] -(merging origin/git-annex into git-annex...) -(recording state in git...) -[...snip...] -E:\git-annex-tests\test-onlyingroup-matching2\annex-d [adjusted/master(unlocked) ↓1 ↑2]> ls - - Directory: E:\git-annex-tests\test-onlyingroup-matching2\annex-d - -Mode LastWriteTime Length Name ----- ------------- ------ ---- --a--- 5.8.2023 0:47 7 a-1 - -E:\git-annex-tests\test-onlyingroup-matching2\annex-d [adjusted/master(unlocked) ↓1 ↑2]> cd ..\annex-a -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex list -here -|annex-b -||annex-c -|||annex-d -||||web -|||||bittorrent -|||||| -XXXX__ a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --inallgroup=main -a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --onlyingroup=main -a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --inallgroup=backup -a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --onlyingroup=backup -a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --inallgroup=offline -a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --onlyingroup=offline -a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --inallgroup=foo -a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --onlyingroup=foo -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --onlyingroups=foo -Invalid option `--onlyingroups=foo' - -Usage: git-annex COMMAND -[...snip...] -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> cd ..\annex-d -E:\git-annex-tests\test-onlyingroup-matching2\annex-d [adjusted/master(unlocked) ↓1 ↑2]> ls - - Directory: E:\git-annex-tests\test-onlyingroup-matching2\annex-d - -Mode LastWriteTime Length Name ----- ------------- ------ ---- --a--- 5.8.2023 0:47 7 a-1 - -E:\git-annex-tests\test-onlyingroup-matching2\annex-d [adjusted/master(unlocked) ↓1 ↑2]> git annex drop a-1 -drop a-1 ok -(recording state in git...) -E:\git-annex-tests\test-onlyingroup-matching2\annex-d [adjusted/master(unlocked) ↓1 ↑2]> git annex sync -g -commit -On branch adjusted/master(unlocked) -Your branch and 'origin/adjusted/master(unlocked)' have diverged, -and have 2 and 1 different commits each, respectively. - (use "git pull" to merge the remote branch into yours) - -nothing to commit, working tree clean -ok -pull origin -ok -push origin -[...snip...] -E:\git-annex-tests\test-onlyingroup-matching2\annex-d [adjusted/master(unlocked) ↓1 ↑2]> cd ..\annex-a -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex sync -g -commit -[...snip...] -pull annex-d -[...snip...] -pull annex-c -[...snip...] -pull annex-b -[...snip...] -push annex-c -[...snip...] -push annex-b -[...snip...] -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex list -here -|annex-b -||annex-c -|||annex-d -||||web -|||||bittorrent -|||||| -XXX___ a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --inallgroup=main -a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --onlyingroup=main -a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --inallgroup=backup -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --onlyingroup=backup -a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --inallgroup=offline -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --onlyingroup=offline -a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --inallgroup=foo -a-1 -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex find --onlyingroup=foo -E:\git-annex-tests\test-onlyingroup-matching2\annex-a [adjusted/master(unlocked)]> git annex version --raw -10.20230802-gd19139a10d - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Sure, Git Annex is great as always. I use it several times a week with my multigigabyte -backups, where it gives structure to my image-based backup routines, so you could -say I'm a believer. :) - -[[!meta author=jkniiv]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/matching_option_--onlyingroup_works_only_partially/comment_1_001927fce7b1c194076f76a092af414e._comment b/doc/bugs/matching_option_--onlyingroup_works_only_partially/comment_1_001927fce7b1c194076f76a092af414e._comment deleted file mode 100644 index de3c4841fe..0000000000 --- a/doc/bugs/matching_option_--onlyingroup_works_only_partially/comment_1_001927fce7b1c194076f76a092af414e._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 1" - date="2023-08-08T16:38:53Z" - content=""" -That repair was quick and to the point, thanks. It seems to work now as specified. - -"""]] diff --git a/doc/bugs/move_that_fails_at_drop_stage_does_not_resume_correctly.mdwn b/doc/bugs/move_that_fails_at_drop_stage_does_not_resume_correctly.mdwn deleted file mode 100644 index 437f140be0..0000000000 --- a/doc/bugs/move_that_fails_at_drop_stage_does_not_resume_correctly.mdwn +++ /dev/null @@ -1,14 +0,0 @@ -git-annex move takes care to handle resuming an interrupted move, where the -dest received the object, but the drop did not happen before it was -interrupted. - -But, the more common case than being interrupted with ctrl-c or power loss -is the drop failing due to a network issue. In this case, re-running move -does not let the drop succeed in cases where the original move would have -let the drop succeed. - -The problem is that logMove runs its cleanup action in this case. -So the move.log gets the record of the move cleaned from it. It seems that -the cleanup action should avoid doing that when the move failed. --[[Joey]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/new_repo_with_git-lfs_as_only_annex_remote.mdwn b/doc/bugs/new_repo_with_git-lfs_as_only_annex_remote.mdwn deleted file mode 100644 index 2336128889..0000000000 --- a/doc/bugs/new_repo_with_git-lfs_as_only_annex_remote.mdwn +++ /dev/null @@ -1,42 +0,0 @@ -### Please describe the problem. - -Setting up a new repo with an existing git-lfs as its only annex remote doesn't find the annex files. - -### What steps will reproduce the problem? - -This is all with encryption=none - -* set up repo w/ lfs-enabled origin. -* annex a file, commit, push. -* set up git-lfs special remote, and git annex copy --to it. - -Now - -* clone origin. -* set up the same git-lfs special remote. -* oops: git annex get says no available copies! - -### What version of git-annex are you using? On what operating system? - -* Debian GNU/Linux (testing) -* git 1:2.27.0-1 -* git-annex 8.20200330-1 - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -I use it all the time for lots of files, git-annex is great! Hoping to migrate off my own ancient server to a gitlab one maintained by somebody else, mainly because it's backed up. But would prefer to stick with git-annex instead of having to migrate to horrible git-lfs. - -[[!tag moreinfo]] - -> [[closing|done]] as the bug reporter did not follow up with information -> about how to reproduce it. --[[Joey]] diff --git a/doc/bugs/new_repo_with_git-lfs_as_only_annex_remote/comment_1_804ecf5218ce147767352fea8ea5eae8._comment b/doc/bugs/new_repo_with_git-lfs_as_only_annex_remote/comment_1_804ecf5218ce147767352fea8ea5eae8._comment deleted file mode 100644 index 67f7252d1e..0000000000 --- a/doc/bugs/new_repo_with_git-lfs_as_only_annex_remote/comment_1_804ecf5218ce147767352fea8ea5eae8._comment +++ /dev/null @@ -1,21 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2020-07-10T20:05:09Z" - content=""" -It seems to me there are two possible mistakes you could have made, both -of which are much more likely than a bug in git-annex. They are: - -1. You forgot to push the git-annex branch to origin, so your clone was not - able to see it. - -2. When you say you "set up the same git-lfs special remote" again, - you mean you used `git-annex initremote` again. Which sets up a - *different* remote with a different uuid. The right thing to do would be - to use `git annex enableremote`. (Or indeed to get git-annex - auto-enable the git-lfs special remote when it sees you cloned the same - url that was used to set it up) - -If you still think you have a bug, you'll need to show me a list of -commands that I can use to reproduce the bug. -"""]] diff --git a/doc/bugs/no_way_to_disable_repository_auto_upgrade.mdwn b/doc/bugs/no_way_to_disable_repository_auto_upgrade.mdwn deleted file mode 100644 index 0e2db6464b..0000000000 --- a/doc/bugs/no_way_to_disable_repository_auto_upgrade.mdwn +++ /dev/null @@ -1,49 +0,0 @@ -### Please describe the problem. - -git-annex upgrades the repository before I could set the annex.autoupgraderepository to false - -### What steps will reproduce the problem? - -Its not trivial to reproduce. After an auto upgrade at one repo from v5 to v8 crashed, I decided to freeze all my v5 repositories first. By requiring the repo version and then immediately setting autoupgrade to false I realized the repo was upgraded during the command `git annex config --set autoupgraderepository false` - -### What version of git-annex are you using? On what operating system? - -git-annex version: 8.20200226 -ubuntu 20.04 - -### Please provide any additional information below. - -[[!format sh """ -$ git annex version -git-annex version: 8.20200226 -build flags: Assistant Webapp Pairing S3 WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite -dependency versions: aws-0.20 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.3 feed-1.0.1.0 ghc-8.6.5 http-client-0.5.14 persistent-sqlite-2.9.3 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs hook external -operating system: linux x86_64 -supported repository versions: 8 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 -local repository version: 5 -$ git annex config --set autoupgraderepository false -autoupgraderepository false ok -(recording state in git...) -$ git annex version -git-annex version: 8.20200226 -build flags: Assistant Webapp Pairing S3 WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite -dependency versions: aws-0.20 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.3 feed-1.0.1.0 ghc-8.6.5 http-client-0.5.14 persistent-sqlite-2.9.3 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs hook external -operating system: linux x86_64 -supported repository versions: 8 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 -local repository version: 8 - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -I am using it since v5 and only v5 for repos like more 100k filecount and size ~800GB. Worked perfectly on Linux desktop, android, phone, tab, raspberry. The greatest sync tool I ever had. I am a little bit worried about the upgrade to v8 from v5 now :) - - -> seems like this was a mistake not a bug, so [[done]] --[[Joey]] diff --git a/doc/bugs/no_way_to_disable_repository_auto_upgrade/comment_1_aab2cb260b68c25facd652946ea2aff7._comment b/doc/bugs/no_way_to_disable_repository_auto_upgrade/comment_1_aab2cb260b68c25facd652946ea2aff7._comment deleted file mode 100644 index fe7ef63172..0000000000 --- a/doc/bugs/no_way_to_disable_repository_auto_upgrade/comment_1_aab2cb260b68c25facd652946ea2aff7._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="zsolt1" - avatar="http://cdn.libravatar.org/avatar/ef26f34d5d953cf139c58e5dc4b5fe73" - subject="reproduce" - date="2021-02-20T19:06:44Z" - content=""" -disabling auto-upgrade was successful by 4 repos before I experienced this problem by the 5th one -"""]] diff --git a/doc/bugs/no_way_to_disable_repository_auto_upgrade/comment_2_dc24c6c0c60bb11d6d871fa62585276d._comment b/doc/bugs/no_way_to_disable_repository_auto_upgrade/comment_2_dc24c6c0c60bb11d6d871fa62585276d._comment deleted file mode 100644 index 221c1c84da..0000000000 --- a/doc/bugs/no_way_to_disable_repository_auto_upgrade/comment_2_dc24c6c0c60bb11d6d871fa62585276d._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Lukey" - avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b" - subject="comment 2" - date="2021-02-20T21:35:05Z" - content=""" -You're supposed to use `git config annex.autoupgraderepository false`. -"""]] diff --git a/doc/bugs/no_way_to_disable_repository_auto_upgrade/comment_3_f3c10f35956dff886d961f1351af71ef._comment b/doc/bugs/no_way_to_disable_repository_auto_upgrade/comment_3_f3c10f35956dff886d961f1351af71ef._comment deleted file mode 100644 index 45e0ee3de5..0000000000 --- a/doc/bugs/no_way_to_disable_repository_auto_upgrade/comment_3_f3c10f35956dff886d961f1351af71ef._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="zsolt1" - avatar="http://cdn.libravatar.org/avatar/ef26f34d5d953cf139c58e5dc4b5fe73" - subject="comment 3" - date="2021-02-21T18:11:33Z" - content=""" -@Lukey , thanks for pointing it out! Layer 8. -"""]] diff --git a/doc/bugs/no_way_to_disable_repository_auto_upgrade/comment_4_7ea295a6c113ea0cc3e4e5ae28769190._comment b/doc/bugs/no_way_to_disable_repository_auto_upgrade/comment_4_7ea295a6c113ea0cc3e4e5ae28769190._comment deleted file mode 100644 index 7945b1f8b4..0000000000 --- a/doc/bugs/no_way_to_disable_repository_auto_upgrade/comment_4_7ea295a6c113ea0cc3e4e5ae28769190._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2021-02-22T16:35:40Z" - content=""" -Yeah, that command you say you ran makes no sense. - -Now, running that command *does* upgrade the repo, if -upgrading has not already been disabled in the git config. -Because git-annex auto-upgrades on the first command run, -and the fact that the command just errors out due to being -used incorrectly doesn't prevent that from happening. - -But I get the feeling that command is not what you actually did? - -You can run `git config --global annex.autoupgraderepository false` -and you only have to set it that one time for that machine and it will -avoid all upgrades. -"""]] diff --git a/doc/bugs/oldkeys_not_in_Windows_release_10.20230803.mdwn b/doc/bugs/oldkeys_not_in_Windows_release_10.20230803.mdwn deleted file mode 100644 index 7292c9f6f5..0000000000 --- a/doc/bugs/oldkeys_not_in_Windows_release_10.20230803.mdwn +++ /dev/null @@ -1,38 +0,0 @@ -### Please describe the problem. - -The new oldkeys command is not included in the latest Windows release. - -The problem does not occur on Linux or macOS. - -Also, the man page summary (on https://git-annex.branchable.com/git-annex/) does not include a link to the documentation page (https://git-annex.branchable.com/git-annex-oldkeys/). - -### What steps will reproduce the problem? - -git annex oldkeys - -### What version of git-annex are you using? On what operating system? - -* git-annex version: 10.20230803-gd467c70ef7 -* git version 2.41.0.windows.2 -* OS Name: Microsoft Windows 10 Pro -* OS Version: 10.0.19045 N/A Build 19045 - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - -git-annex: Unknown command 'oldkeys' - -Did you mean one of these? - findkeys - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Yes. I am developing the easy-git-annex API. - -> Closing as it will be fixed by the next release. [[done]] --[[Joey]] diff --git a/doc/bugs/oldkeys_not_in_Windows_release_10.20230803/comment_1_9969cdc389617f4df2ee286e8d28176e._comment b/doc/bugs/oldkeys_not_in_Windows_release_10.20230803/comment_1_9969cdc389617f4df2ee286e8d28176e._comment deleted file mode 100644 index a8b6e4fbd6..0000000000 --- a/doc/bugs/oldkeys_not_in_Windows_release_10.20230803/comment_1_9969cdc389617f4df2ee286e8d28176e._comment +++ /dev/null @@ -1,7 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-09-11T17:15:45Z" - content=""" -Added the missing link. -"""]] diff --git a/doc/bugs/oldkeys_not_in_Windows_release_10.20230803/comment_2_b265af06b8edda75d305fd836f6c5242._comment b/doc/bugs/oldkeys_not_in_Windows_release_10.20230803/comment_2_b265af06b8edda75d305fd836f6c5242._comment deleted file mode 100644 index b83eaf104e..0000000000 --- a/doc/bugs/oldkeys_not_in_Windows_release_10.20230803/comment_2_b265af06b8edda75d305fd836f6c5242._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2023-09-11T17:17:47Z" - content=""" -d467c70ef7 is before the oldkeys command got added, apparently the windows -build was lagging at the time of release. - -If you need the command, the windows autobuild is up-to-date and includes -it. - -So this will get fixed on the next release. -"""]] diff --git a/doc/bugs/poor_choice_of_name_for_adjusted_branches.mdwn b/doc/bugs/poor_choice_of_name_for_adjusted_branches.mdwn deleted file mode 100644 index 80412fad1f..0000000000 --- a/doc/bugs/poor_choice_of_name_for_adjusted_branches.mdwn +++ /dev/null @@ -1,26 +0,0 @@ -### Please describe the problem. - -The naming conventions for adjusted branches are poorly chosen (adjusted/master(unlocked) or adjusted/master(hidemissing)). The parentheses cause problems when checking out the branch without protecting the name by way of quotation marks. This problem is exacarbated at least on my Ubuntu Focal system by the fact that the provided bash-completion completes the branch name without those marks. - -### What version of git-annex are you using? On what operating system? -8.20200226-1 on Ubuntu Focal - -### What steps will reproduce the problem? - - git annex adjust --hide-missing - git checkout master - #git checkout adjTABusted/master(hidemissing) - git checkout adjusted/master(hidemissing) - -Result: - - $ git checkout adjusted/master(hidemissing) - bash: syntax error near unexpected token `(' - $ git checkout "adjusted/master(hidemissing)" - Switched to branch 'adjusted/master(hidemissing)' - -The problem is easy to fix, I think, by choosing a different naming convention. I'm not sure how to deal with legacy-named branches going forward, though. - -Thank you very much for making git-annex available, much appreciated. - -> I disagree that this needs fixing, so [[wontfix|done]] --[[Joey]] diff --git a/doc/bugs/poor_choice_of_name_for_adjusted_branches/comment_1_adbbc90054eafefa7de58663868986a3._comment b/doc/bugs/poor_choice_of_name_for_adjusted_branches/comment_1_adbbc90054eafefa7de58663868986a3._comment deleted file mode 100644 index 7389cfcd5e..0000000000 --- a/doc/bugs/poor_choice_of_name_for_adjusted_branches/comment_1_adbbc90054eafefa7de58663868986a3._comment +++ /dev/null @@ -1,30 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2021-04-20T15:43:03Z" - content=""" -One of the constraints on naming these is that the name needs to not be one -that is likely to get in the way of something the user is doing with -other branches, eg conflicting with another branch they have for -something unrelated to git-annex. - -Another constraint is that the name of the branch (either the whole thing -or sometimes just the part after the slash) often appears in the -user's prompt and so it would be good if it were reasonably short and also -reasonably human readable and clear about what thing is checked out. - -Those constraints are what led to this name choice. I am well aware there -are reasons people won't typically use () in branch names, which along with -the "adjusted/", helps make a naming conflict unlikely. And it avoids needing -to also put "git-annex" in the branch name to avoid conflicts, so keeps it -short, and the parens imply a relationship to master rather clearly. - -I decided that needing to properly quote a shell parameter in the (somewhat -uncommon) case of manually checking the branch out was a reasonable -tradeoff. - -It does seem like a bug in bash completion that it doesn't tab complete -this correctly. I notice that tab completing a similar filename does escape -the parens, so it may be that the bug can be fixed in the git completion -file somehow. -"""]] diff --git a/doc/bugs/poor_choice_of_name_for_adjusted_branches/comment_2_dc7b2ae4971324d34f6a8d89e888f598._comment b/doc/bugs/poor_choice_of_name_for_adjusted_branches/comment_2_dc7b2ae4971324d34f6a8d89e888f598._comment deleted file mode 100644 index dc269631b0..0000000000 --- a/doc/bugs/poor_choice_of_name_for_adjusted_branches/comment_2_dc7b2ae4971324d34f6a8d89e888f598._comment +++ /dev/null @@ -1,21 +0,0 @@ -[[!comment format=mdwn - username="https://launchpad.net/~r0lf" - nickname="r0lf" - avatar="http://cdn.libravatar.org/avatar/aa82122557e706df7ba83bd1983eb79ef1ba2e51350217850176d4f9a1bb2bc0" - subject="try to find a better solution" - date="2021-04-24T11:03:28Z" - content=""" -Thank you for the your comment. It certainly explains your situation back when you coded this up. I'm not sure I'm convinced it is good justification to keep things the way they are going forward. - -1. The branch names aren't short -2. The branch names aren't clash-free (see my other report) -3. I switch between branches all the time, that's their purpose. - -For #3, I usually rely on either mark-copy'ing the characters off the terminal with the mouse or via bash-completion. Both are broken. - -Isn't it that people aren't using () for branch or filenames precisely because of the problems you will almost certainly run into? I'm certain there is a better way. - -Personally, I see the act of hiding non-local files as a temporary measure. Maybe creating a branch for that isn't even the right thing. Maybe \"git stash\" or a tag is a better solution. Or maybe just remove broken links without committing the result at all. - -To expand on that idea, https://unix.stackexchange.com/a/49470 discusses how to use find and I found \"find . -maxdepth 1 -type l -exec test ! -e {} \; -delete\" to work just fine. \"git checkout .\" then brought back those links for non-local files just fine. How about going without any branch at all? Obviously, in git-annex there are still some corner cases to consider. Like making sure not to continue on master while the links to non-local files are still removed. I haven't considered all potential pitfalls, but I feel this might be a better way to handle this need. -"""]] diff --git a/doc/bugs/poor_choice_of_name_for_adjusted_branches/comment_3_19ce066c0b912398300caf8520f70eb3._comment b/doc/bugs/poor_choice_of_name_for_adjusted_branches/comment_3_19ce066c0b912398300caf8520f70eb3._comment deleted file mode 100644 index 127a097bad..0000000000 --- a/doc/bugs/poor_choice_of_name_for_adjusted_branches/comment_3_19ce066c0b912398300caf8520f70eb3._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Ilya_Shlyakhter" - avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" - subject="sparse checkouts to hide non-present files" - date="2021-04-24T22:00:14Z" - content=""" -Could [sparse checkouts](https://git-scm.com/docs/git-read-tree#_sparse_checkout) be used to hide non-present files, as an alternative to adjusted branches? -"""]] diff --git a/doc/bugs/preferred_content__58___include_statement_does_not_allow_spaces_in_filenames.mdwn b/doc/bugs/preferred_content__58___include_statement_does_not_allow_spaces_in_filenames.mdwn deleted file mode 100644 index 14f007292a..0000000000 --- a/doc/bugs/preferred_content__58___include_statement_does_not_allow_spaces_in_filenames.mdwn +++ /dev/null @@ -1,33 +0,0 @@ -### Please describe the problem. -Filenames for the *include* statement for [preferred content](http://git-annex.branchable.com/preferred_content/) can not contain spaces. - -### What steps will reproduce the problem? - -* Create an annex repo -* Run `git annex vicfg` -* Enter expression *include='pictures/dir with spaces'* -* git annex complains: # ** Parse error in next line: Parse failure: near "with" Parse failure: near "spaces'" - -(The *'* is interpreted as part of the filepath.) - -### What version of git-annex are you using? On what operating system? - -I am using the current binaries from Debian stable amd64. - -
-git-annex version: 5.20140320~bpo70+1
-build flags: Assistant Webapp Pairing S3 Inotify DBus XMPP Feeds Quvi TDFA
-key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SHA256 SHA1 SHA512 SHA224 SHA384 WORM URL
-remote types: git gcrypt S3 bup directory rsync web tahoe glacier hook external
-local repository version: 5
-supported repository version: 5
-upgrade supported from repository versions: 0 1 2 4
-
- -### Please provide any additional information below. - -The only workaround I found is to use a glob for the filepath which only works for the first space: *include='pictures/dir\*'*. - -[[!tag confirmed]] - -> [[fixed|done]] by documenting the `[[:space:]]` workaround. --[[Joey]] diff --git a/doc/bugs/preferred_content__58___include_statement_does_not_allow_spaces_in_filenames/comment_1_ca10638d4b4b178cfd0de8736542c4dc._comment b/doc/bugs/preferred_content__58___include_statement_does_not_allow_spaces_in_filenames/comment_1_ca10638d4b4b178cfd0de8736542c4dc._comment deleted file mode 100644 index a7438c559e..0000000000 --- a/doc/bugs/preferred_content__58___include_statement_does_not_allow_spaces_in_filenames/comment_1_ca10638d4b4b178cfd0de8736542c4dc._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="http://joeyh.name/" - ip="209.250.56.244" - subject="comment 1" - date="2014-04-02T18:45:36Z" - content=""" -A better workaround is: - -include='pictures/dir?with?spaces' - -Tokenizing text with embedded quotes is a bit of a PITA, certianly doable I suppose.. -"""]] diff --git a/doc/bugs/preferred_content__58___include_statement_does_not_allow_spaces_in_filenames/comment_2_986a393a512229d35e529ba242b77b1e._comment b/doc/bugs/preferred_content__58___include_statement_does_not_allow_spaces_in_filenames/comment_2_986a393a512229d35e529ba242b77b1e._comment deleted file mode 100644 index 7dbeb3d088..0000000000 --- a/doc/bugs/preferred_content__58___include_statement_does_not_allow_spaces_in_filenames/comment_2_986a393a512229d35e529ba242b77b1e._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="http://ypid.wordpress.com/" - ip="213.153.84.215" - subject="comment 2" - date="2014-04-02T21:29:54Z" - content=""" -Thanks for your workaround ... Works for me after removing the single quote signs. - -With this workaround on hand one could include/match any filename. Works for me ;) -"""]] diff --git a/doc/bugs/preferred_content__58___include_statement_does_not_allow_spaces_in_filenames/comment_3_a66be1cc2c4ca7e470be1248dcbbe885._comment b/doc/bugs/preferred_content__58___include_statement_does_not_allow_spaces_in_filenames/comment_3_a66be1cc2c4ca7e470be1248dcbbe885._comment deleted file mode 100644 index 038e47386b..0000000000 --- a/doc/bugs/preferred_content__58___include_statement_does_not_allow_spaces_in_filenames/comment_3_a66be1cc2c4ca7e470be1248dcbbe885._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="aurtzy" - avatar="http://cdn.libravatar.org/avatar/7d44f9b718a534ada04551ee8c72e07f" - subject="comment 3" - date="2023-05-14T02:52:38Z" - content=""" -Encountered this issue the other day, and just wanted to mention a more explicit alternative: it looks like matchexpression file globs support POSIX character classes, so one could also use \"\\[[:blank:]]\" which matches a space or tab. -"""]] diff --git a/doc/bugs/preferred_content__58___include_statement_does_not_allow_spaces_in_filenames/comment_4_7a8f1ba849f6e6f6072c39bdbff49405._comment b/doc/bugs/preferred_content__58___include_statement_does_not_allow_spaces_in_filenames/comment_4_7a8f1ba849f6e6f6072c39bdbff49405._comment deleted file mode 100644 index 214c1c8603..0000000000 --- a/doc/bugs/preferred_content__58___include_statement_does_not_allow_spaces_in_filenames/comment_4_7a8f1ba849f6e6f6072c39bdbff49405._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2023-05-15T19:40:59Z" - content=""" -Thanks @aurtzy, that is a great observation! And a feature I was not -actually aware of. - -So, I think that documenting this should be sufficient. - -(The `glob(7)` man page does document POSIX character classes as working in -globs, so this is a fairly standard thing.) -"""]] diff --git a/doc/bugs/proposal_for_timestamp_semantics.mdwn b/doc/bugs/proposal_for_timestamp_semantics.mdwn deleted file mode 100644 index cea7dd1914..0000000000 --- a/doc/bugs/proposal_for_timestamp_semantics.mdwn +++ /dev/null @@ -1,13 +0,0 @@ -# High level overview: -* This only concerns mtime (and its equivalents on other systems, if applicable). atime, ctime and permissions are out of scope. -* An object added to an annex and later retrieved from it via `mv`, `cp -L`, `git annex unannex` and such should always keep its mtime, even if retrieved on an entirely different machine and/or from a backend that doesn't support timestamps natively. -* If an added/reinjected object is already known to the annex, use the older mtime by default, since that's probably the version that's had its metadata preserved better. - * If that's too much of an assumption, provide a switch to use the older/newer/known/unknown mtime, or add a git-annex-touch command. -* symlink and object file mtimes should reflect the mtime tracked by the annex. -* ideally, directory mtimes would also be preserved, or failing that, `git-annex-fix`, `git-annex-add` and `git-checkout` should leave them untouched. - -# open questions/ideas: -* What if the user `touch`es a file/symlink, bypassing git-annex? Should the data be reconciled or ignored? -* preserving directory mtimes looks tricky, but could it maybe be done from a hook or two? - -> [[notabug|done]] --[[Joey]] diff --git a/doc/bugs/proposal_for_timestamp_semantics/comment_1_a54fc1bda24e04b15490d0eaa1838d28._comment b/doc/bugs/proposal_for_timestamp_semantics/comment_1_a54fc1bda24e04b15490d0eaa1838d28._comment deleted file mode 100644 index 8fe2f552b3..0000000000 --- a/doc/bugs/proposal_for_timestamp_semantics/comment_1_a54fc1bda24e04b15490d0eaa1838d28._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2018-04-04T18:01:01Z" - content=""" -This is lacking a lot of detail about how this would be accomplished. - -Assuming it's to be accomplished using git-annex metadata, it seems likely -to signficantly slow down some git-annex operations (which would need to do -an expensive git-annex branch lookup). - -There's also the complication of merging a git-annex -branch that contains changes to the timestamp metadata. Would git-annex -need to look over all the merged changes and go off and frob timestamps? - -[[todo/does_not_preserve_timestamps]] already contains discussion of this -topic. I'm not sure that it's productive to discuss it in two different -places. (Nor does this really seem like a bug report.) -"""]] diff --git a/doc/bugs/proposal_for_timestamp_semantics/comment_2_8ba9e0c35e7761b99feb55bcdd6065cf._comment b/doc/bugs/proposal_for_timestamp_semantics/comment_2_8ba9e0c35e7761b99feb55bcdd6065cf._comment deleted file mode 100644 index b7edd53e40..0000000000 --- a/doc/bugs/proposal_for_timestamp_semantics/comment_2_8ba9e0c35e7761b99feb55bcdd6065cf._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="vrs+annex@ea5fa24dbb279be61a8e50adb638bf8366300717" - nickname="vrs+annex" - avatar="http://cdn.libravatar.org/avatar/74219abcec6eece8e2c9d4351c2c912c" - subject="comment 2" - date="2018-04-05T21:01:51Z" - content=""" -I have no opinion about what backend to use. If doing it via the metadata system significantly slows down things though and is generally awkward, why not build a separate subsystem? - -I don't know what you mean by \"look over all the merged changes and go off and frob timestamps\", but as long as n is on the order of [number of files changed in the commit], updating n files' timestamps sounds reasonable? There's the question of which timestamp has preference in a merge, but that sounds solvable. - -I made this a separate bug because it's a specific design proposal; I consider [[todo/does_not_preserve_timestamps]] a tracking bug/user story. - -proposal re [[/bugs/file_modification_time_should_be_stored_in_exactly_one_metadata_field/#comment-2ea94161228f0653917b91d4f999153f]]: File and symlink timestamps, after `git-annex-get` or `git-checkout`, are set to whatever's in the repo and then considered immutable. The user can of course change them with `touch`, but if the file is locked while that happens, that's considered a corruption like editing an object file and will be caught by `git-annex-fsck`. -"""]] diff --git a/doc/bugs/repair_on_Windows_reports_ambiguous_argument_error.mdwn b/doc/bugs/repair_on_Windows_reports_ambiguous_argument_error.mdwn deleted file mode 100644 index fb4c594f46..0000000000 --- a/doc/bugs/repair_on_Windows_reports_ambiguous_argument_error.mdwn +++ /dev/null @@ -1,77 +0,0 @@ -### Please describe the problem. - -On Windows only the git annex repair command reports an error. - -### What steps will reproduce the problem? - -Following are the minimum steps to reproduce the problem. The problem occurs with any number of files in the repository. The --force option does not correct the problem. The problem does not occur on Linux or macOS. - -[[!format sh """ -mkdir repair-ambiguous-argument-bug -pushd repair-ambiguous-argument-bug -git init -git annex init -git annex repair -popd -"""]] - -### What version of git-annex are you using? On what operating system? - -git-annex version: 10.20230330-g98a3ba0ea5 - -git version 2.40.0.windows.1 - -Microsoft Windows [Version 10.0.19045.2846] - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - -repair Running git fsck ... -Fsck found no problems. Checking for broken branches. -fatal: ambiguous argument 'refs\heads\master': unknown revision or path not in the working tree. -Use '--' to separate paths from revisions, like this: -'git [...] -- [...]' -fatal: ambiguous argument 'refs\heads\git-annex': unknown revision or path not in the working tree. -Use '--' to separate paths from revisions, like this: -'git [...] -- [...]' -fatal: ambiguous argument 'refs\heads\adjusted\master(unlocked)': unknown revision or path not in the working tree. -Use '--' to separate paths from revisions, like this: -'git [...] -- [...]' -fatal: ambiguous argument 'refs\basis\adjusted\master(unlocked)': unknown revision or path not in the working tree. -Use '--' to separate paths from revisions, like this: -'git [...] -- [...]' -Found problems, attempting repair. -fatal: ambiguous argument 'refs\heads\master': unknown revision or path not in the working tree. -Use '--' to separate paths from revisions, like this: -'git [...] -- [...]' -fatal: ambiguous argument 'refs\heads\git-annex': unknown revision or path not in the working tree. -Use '--' to separate paths from revisions, like this: -'git [...] -- [...]' -fatal: ambiguous argument 'refs\heads\adjusted\master(unlocked)': unknown revision or path not in the working tree. -Use '--' to separate paths from revisions, like this: -'git [...] -- [...]' -fatal: ambiguous argument 'refs\basis\adjusted\master(unlocked)': unknown revision or path not in the working tree. -Use '--' to separate paths from revisions, like this: -'git [...] -- [...]' -Some git branches refer to missing objects: - refs\heads\master - refs\heads\git-annex - refs\heads\adjusted\master(unlocked) - refs\basis\adjusted\master(unlocked) -To force a recovery to a usable state, retry with the --force parameter. -failed -repair: 1 failed - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Yes. I am developing the easy-git-annex API for git-annex. - -> Should be [[fixed|done]] now... It was reading the contents of the refs/ -> directory recursively, and treating those names as refs, which is wrong -> on Windows due to path separator being different. --[[Joey]] diff --git a/doc/bugs/s3_remote__58___public__61__yes_sets_disabled_ACLs.mdwn b/doc/bugs/s3_remote__58___public__61__yes_sets_disabled_ACLs.mdwn deleted file mode 100644 index e2e4f98828..0000000000 --- a/doc/bugs/s3_remote__58___public__61__yes_sets_disabled_ACLs.mdwn +++ /dev/null @@ -1,41 +0,0 @@ -### Please describe the problem. - - -### What steps will reproduce the problem? -I followed the [s3 special remote tip](https://git-annex.branchable.com/tips/public_Amazon_S3_remote/) to setup a public S3 special remote: - -``` -export AWS_ACCESS_KEY_ID... -export AWS_SECTRET_ACCESS_KEY... -git annex initremote pubs3 type=S3 encryption=none public=yes -initremote pubs3 (checking bucket...) (creating bucket in US...) -git-annex: S3Error {s3StatusCode = Status {statusCode = 400, statusMessage = "Bad Request"}, s3ErrorCode = "InvalidBucketAclWithObjectOwnership", s3ErrorMessage = "Bucket cannot have ACLs set with ObjectOwnership's BucketOwnerEnforced setting", s3ErrorResource = Nothing, s3ErrorHostId = Just "CvCxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx8ic=", s3ErrorAccessKeyId = Nothing, s3ErrorStringToSign = Nothing, s3ErrorBucket = Nothing, s3ErrorEndpointRaw = Nothing, s3ErrorEndpoint = Nothing} -failed -initremote: 1 failed -``` - -The ``public`` parameter is documented with -> "Set to "yes" to allow public read access to files sent to the S3 remote. This is accomplished by setting an ACL when each file is uploaded to the remote. So, changes to this setting will only affect subseqent uploads." - -This [AWS post](https://aws.amazon.com/about-aws/whats-new/2022/12/amazon-s3-automatically-enable-block-public-access-disable-access-control-lists-buckets-april-2023/) says that it disables setting ACLs for S3 buckets from April 2023 onward following their previous [recommendation to do so](https://docs.aws.amazon.com/AmazonS3/latest/userguide/ensure-object-ownership.html), so this may be the issue. I know too little about AWS S3 to suggest an alternative approach, so this is just a bug report that the "public" parameter causes trouble when set to "yes". -There is no issue if I set ``public=no``. - -### What version of git-annex are you using? On what operating system? -``` -$ git annex version 255 ! -git-annex version: 10.20221003 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22.1 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.13.1 persistent-sqlite-2.13.1.0 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2.1 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10 -``` - - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -I work daily with git-annex and I never fail to be amazed by it. Thank you for your work! - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/s3_remote__58___public__61__yes_sets_disabled_ACLs/comment_1_0bf1f463133476727129ddf15f24983b._comment b/doc/bugs/s3_remote__58___public__61__yes_sets_disabled_ACLs/comment_1_0bf1f463133476727129ddf15f24983b._comment deleted file mode 100644 index 655556998f..0000000000 --- a/doc/bugs/s3_remote__58___public__61__yes_sets_disabled_ACLs/comment_1_0bf1f463133476727129ddf15f24983b._comment +++ /dev/null @@ -1,43 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-07-21T17:04:19Z" - content=""" -This only affects new S3 buckets. Existing S3 buckets that were -created before April 2023 and were set up to allow public access should -keep working, including ACL settings when storing new files in them. -Per Amazon's announcement, "There is no change for existing buckets." - -So users who create new buckets will need to set `public=no` -(the default) and set a bucket policy instread. See -[this comment](https://git-annex.branchable.com/special_remotes/S3/#comment-fcfba0021592de4c1425d3bf3c9563d3) -for an example policy. - -That comment also suggests: - -* If public=yes, instead of trying to set an ACL, first try HEAD on the - newly uploaded object without using the AWS_ACCESS_KEY. Only if that - fails, fall over to trying to set an ACL. And if you get - AccessControlListNotSupported (i.e. the error due to - BucketOwnerEnforced), then give a warning that the bucket policy is not - configured for public access. - -However, the ACL is currently sent at past of the PutObject request. And -either there is not a way to change the ACL later, or the aws haskell library -is missing support for the API to do that. - -I think what needs to be done is discourage initializing new S3 remotes -with public=yes, since it won't work. (Assuming some other S3 -implementation than Amazon doesn't keep on supporting ACLs.) - -And allow setting publicurl=yes without public=yes, so users who create -new buckets and configure a bucket policy to allow public access can tell -git-annex it's set up that way, so it will download from the bucket w/o S3 -credentials. - -While git-annex could HEAD without creds when publicyes=yes to verify that -the user has configured the bucket correctly, that would add some overhead, -and I guess if the user has not configured the bucket correctly, they will -notice in some other way eventually and can fix its bucket policy after the -fact. So I'm inclined not to do that. -"""]] diff --git a/doc/bugs/schedule.log_added_to_annex__63_____58____47____47____47__.mdwn b/doc/bugs/schedule.log_added_to_annex__63_____58____47____47____47__.mdwn deleted file mode 100644 index eaf1157f95..0000000000 --- a/doc/bugs/schedule.log_added_to_annex__63_____58____47____47____47__.mdwn +++ /dev/null @@ -1,58 +0,0 @@ -### Please describe the problem. - -Look at this diff: - -``` -diff --git a/schedule.log b/schedule.log -index 29c727f8c1..2f6d2f0858 100644 ---- a/schedule.log -+++ b/schedule.log -@@ -1,19 +1 @@ --0e6c5057-a323-4bec-b6d2-98bae312e7cd timestamp=1518571009.042438126s --0e6c5057-a323-4bec-b6d2-98bae312e7cd fsck self 15m every day at any time timestamp=1503189066.20635041s --179b6e7f-0b2f-43b9-a95d-39df5b52d2fc timestamp=1518565854.651788575s --179b6e7f-0b2f-43b9-a95d-39df5b52d2fc timestamp=1518565869.769672252s --179b6e7f-0b2f-43b9-a95d-39df5b52d2fc fsck self 15m every day at any time timestamp=1503191937.325891749s --668bdeb3-a3e2-4a9f-ae1c-bae1880c62b8 timestamp=1516100332.848743619s --668bdeb3-a3e2-4a9f-ae1c-bae1880c62b8 timestamp=1518305729.698369069s --668bdeb3-a3e2-4a9f-ae1c-bae1880c62b8 timestamp=1518565670.560184121s --668bdeb3-a3e2-4a9f-ae1c-bae1880c62b8 timestamp=1518868059.554534192s --668bdeb3-a3e2-4a9f-ae1c-bae1880c62b8 fsck self 15m every day at any time timestamp=1504318394.78758178s --7f0bbee7-826f-4021-a774-0569ff58ed54 fsck self 10m every day at any time timestamp=1514213643.848668933s --7f0bbee7-826f-4021-a774-0569ff58ed54 fsck self 15m every day at any time timestamp=1504177427.472846327s --7f0bbee7-826f-4021-a774-0569ff58ed54 fsck self 1h every day at any time timestamp=1496609751.566017331s --83ffd5ab-40b3-4950-b437-87339242be57 fsck self 15m every day at any time timestamp=1503175558.988571178s --ba3a1e0a-9191-43b4-a324-1fee91a9bb8f timestamp=1503184784.521086496s --ba3a1e0a-9191-43b4-a324-1fee91a9bb8f timestamp=1518566966.391960267s --ba3a1e0a-9191-43b4-a324-1fee91a9bb8f fsck self 15m every day at any time timestamp=1503183823.810567346s --ba3a1e0a-9191-43b4-a324-1fee91a9bb8f fsck self 15m every day at any time timestamp=1503185213.589823679s --ba3a1e0a-9191-43b4-a324-1fee91a9bb8f fsck self 5m every day at any time timestamp=1514056713.473067585s -+/annex/objects/SHA256E-s1676--4e76f9f5e9c5eafe53367a6c493d7618366bcfea6ac8a859fcbaba55ed31f349.log -``` - -### What steps will reproduce the problem? - -Absolutely no idea. - - -### What version of git-annex are you using? On what operating system? - -``` -git-annex version: 6.20170818 -operating system: linux x86_64 -``` - - -### Please provide any additional information below. - -None - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Considering my two other issues »WHICH LOSE DATA«, I can only conclude it’s too complicated, cf. - -* https://git-annex.branchable.com/bugs/Data_loss_when_copying_files_with_running_assistant/ -* https://git-annex.branchable.com/bugs/Infinite_loop_when_synchronizing_between_many_machines/ - -> I guess this was fixed in the intervening time. With no way to reproduce -> it, it's hard to tell. [[done]] --[[Joey]] diff --git a/doc/bugs/securehashesonly_conflicts_with_addurl.mdwn b/doc/bugs/securehashesonly_conflicts_with_addurl.mdwn deleted file mode 100644 index 39953a6dae..0000000000 --- a/doc/bugs/securehashesonly_conflicts_with_addurl.mdwn +++ /dev/null @@ -1,17 +0,0 @@ -Turning on `securehashesonly` seems to disable the `addurl` command: - -```console -% git config --get annex.securehashesonly -true -% git annex addurl https://www.gutenberg.org/cache/epub/2591/pg2591-images.html -addurl https://www.gutenberg.org/cache/epub/2591/pg2591-images.html - annex.securehashesonly blocked transfer of URL key -failed -addurl: 1 failed -% git annex addurl https://www.gutenberg.org/cache/epub/2591/pg2591-images.html --relaxed -addurl https://www.gutenberg.org/cache/epub/2591/pg2591-images.html (to www.gutenberg.org_cache_epub_2591_pg2591-images.html) ok -(recording state in git...) -% ls -l www.gutenberg.org_cache_epub_2591_pg2591-images.html -www.gutenberg.org_cache_epub_2591_pg2591-images.html -> .git/annex/objects/gg/kG/URL--https&c%%www.gutenberg.org%cache%epub%2591%pg2591-images.html/URL--https&c%%www.gutenberg.org%cache%epub%2591%pg2591-images.html - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/securehashesonly_conflicts_with_addurl/comment_1_808e276381f98b4ebb7d61628db703d9._comment b/doc/bugs/securehashesonly_conflicts_with_addurl/comment_1_808e276381f98b4ebb7d61628db703d9._comment deleted file mode 100644 index bf5ae10626..0000000000 --- a/doc/bugs/securehashesonly_conflicts_with_addurl/comment_1_808e276381f98b4ebb7d61628db703d9._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-03-27T18:03:11Z" - content=""" -I think that git-annex addurl without --fast or --relaxed should -work with annex.securehashesonly set. (Unless annex.backend is set -to something it prevents adding.) It is currently failing because -of a temporary URL key that never reaches the repository. So the check -should be disabled for that. - -Arguably with --relaxed (or --fast) it should also fail. However, -that does not actually download the content, so it does not technically -add content to the repository that is not using a cryptographically -signed hash. That's how it manages to skate by without failing. -Of course git-annex get will later fail, git-annex fsck -will complain, etc. -"""]] diff --git a/doc/bugs/shared_encryption_bypassed_if_gpg_error_on_init.mdwn b/doc/bugs/shared_encryption_bypassed_if_gpg_error_on_init.mdwn deleted file mode 100644 index 2162f5b3ed..0000000000 --- a/doc/bugs/shared_encryption_bypassed_if_gpg_error_on_init.mdwn +++ /dev/null @@ -1,27 +0,0 @@ -### Please describe the problem. - -When there is a gpg error upon initializing a special remote with shared encryption, the error is effectively skipped and unencrypted content is synced. - -### What steps will reproduce the problem? - -- configure a broken/weird version of gpg (e.g. `git config gpg.program echo` reproduces this but I used the [`ots-git-gpg-wrapper.sh` from OpenTimeStamps](https://github.com/opentimestamps/opentimestamps-client/blob/master/doc/git-integration.md) - that's probably an upstream error but I'd consider a broken gpg not-so-uncommon problem) -- in an annex repo, run `git annex initremote testrepo type=rsync rsyncurl=ssh://... encryption=shared` - - `Not enough bytes returned from gpg [Param "--gen-random",Param "--armor",Param "2",Param "512"] (got 56 ; ex -pected 682 )` - - exits with code 1 - - puts nothing in `git show git-annex:remote.log` but leaves behind a working local git remote that looks to git-annex as an unencrypted remote -- `git annex sync --content` happily syncs unencrypted content - -I guess a sane behaviour would be to not add the local git remote and stop right after encountering the error. - -### What version of git-annex are you using? On what operating system? - -tried with `10.20230329` and `10.20230408+gc70677e31` on Manjaro - - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -It's awesome 👍 - -> Indeed, there was a laziness bug that prevented it from exiting early -> enough. [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/should_error_on_whereis_--batch-keys_--all.mdwn b/doc/bugs/should_error_on_whereis_--batch-keys_--all.mdwn deleted file mode 100644 index 0ee8f21ebb..0000000000 --- a/doc/bugs/should_error_on_whereis_--batch-keys_--all.mdwn +++ /dev/null @@ -1,25 +0,0 @@ -### Please describe the problem. - -AFAIK `--all` option for `whereis` is "incompatible" with operation on keys, so `git-annex` does still error out on `--all --key KEY`: - -``` -$> git annex whereis --json --json-error-messages --key MD5E-s5663237--4608ffbd6b78ce3a325eb338fa556589.nii.gz --all -Invalid option `--all' -... -``` - -but it does not error out if I specify `--all` with `--batch-keys`: - -``` -$> git annex whereis --json --json-error-messages --all --batch-keys -MD5E-s5663237--4608ffbd6b78ce3a325eb338fa556589.nii.gz -{"command":"whereis","note":"2 copies\n\t8d2b6e96-ad81-44a5-99b4-0ec37d6b3800 -- [s3-PUBLIC]\n \tb5dd2e3d-825f-4bc2-b719-cba1059f6bfc -- root@93184394ac19:/datalad/ds000001\n\ns3-PUBLIC: https://s3.amazonaws.com/openneuro.org/ds000001/sub-01/anat/sub-01_T1w.nii.gz?versionId=SHOhSfza0LCFpgctRH3sMZ8iwAGSUwIy\n","success":true,"input":["MD5E-s5663237--4608ffbd6b78ce3a325eb338fa556589.nii.gz"],"untrusted":[],"key":"MD5E-s5663237--4608ffbd6b78ce3a325eb338fa556589.nii.gz","whereis":[{"here":false,"uuid":"8d2b6e96-ad81-44a5-99b4-0ec37d6b3800","urls":["https://s3.amazonaws.com/openneuro.org/ds000001/sub-01/anat/sub-01_T1w.nii.gz?versionId=SHOhSfza0LCFpgctRH3sMZ8iwAGSUwIy"],"description":"[s3-PUBLIC]"},{"here":false,"uuid":"b5dd2e3d-825f-4bc2-b719-cba1059f6bfc","urls":[],"description":"root@93184394ac19:/datalad/ds000001"}],"error-messages":[],"file":null} -^C -``` - - -### What version of git-annex are you using? On what operating system? - -`8.20211231+git140-gc3817495f-1~ndall+1` - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/should_error_on_whereis_--batch-keys_--all/comment_1_724ae0427fb306352ff9b26d9fdef27a._comment b/doc/bugs/should_error_on_whereis_--batch-keys_--all/comment_1_724ae0427fb306352ff9b26d9fdef27a._comment deleted file mode 100644 index 0a5878b4fa..0000000000 --- a/doc/bugs/should_error_on_whereis_--batch-keys_--all/comment_1_724ae0427fb306352ff9b26d9fdef27a._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-01-26T15:58:12Z" - content=""" -More generally, --all and --key are ignored when using --batch or ---batch-keys, by probably all commands that implement them. - -Mixing --batch options with file names also ignores the file names. -"""]] diff --git a/doc/bugs/special_remote_adb_-_Argument_list_too_long.mdwn b/doc/bugs/special_remote_adb_-_Argument_list_too_long.mdwn deleted file mode 100644 index 6ce054d537..0000000000 --- a/doc/bugs/special_remote_adb_-_Argument_list_too_long.mdwn +++ /dev/null @@ -1,56 +0,0 @@ -### Please describe the problem. -When using the adb special remote to sync content from my android phone to the local annex, the command fails with an error of "Argument list too long". - -### What steps will reproduce the problem? - -I'm using Google Pixel 3 with Android 12 on it, installed the `adb` package, and attempted to add the special remote with the path "/sdcard", then checking out the remote tracking branch to "master:android" (since changed to master:android-buggy). The command fails with. - - -### What version of git-annex are you using? On what operating system? -- Debian Linux 11.2 -- git-annex version: 8.20210223 -- Android OS 12 -- Linux strigo 5.10.0-11-amd64 #1 SMP Debian 5.10.92-1 (2022-01-18) x86_64 GNU/Linux - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log -git annex initremote android type=adb androiddirectory=/sdcard encryption=none exporttree=yes importtree=yes -git annex wanted android 'include=*.mp3 or include=*.ogg or include=*.jpg or include=*.png or include=*.mp4 or include=*.pdf' -git annex import --from android master:android - -# on subsequent attempts to sync -git annex sync --content android -commit -On branch master -nothing to commit, working tree clean -ok -list android find: exec stat: Argument list too long -find: exec stat: Argument list too long - -git-annex: Unable to list contents of android: adb find failed -failed - - Not updating export to android to reflect changes to the tree, because export tracking is not enabled. (Set remote.android.annex-tracking-branch to enable it.) - git-annex: sync: 1 failed - -# Version information -git-annex version: 8.20210223 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 -local repository version: 8 -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -This is the first time I've tried it. However, when I restricted the path to /sdcard/DCIM or /sdcard/Music, I didn't have any problems. I could use some assistance regarding how to "copy" a file from one path to another inside an annex. It appears that I could try to resolve and then symlink to the same file, or maybe unlock, copy, and commit both copies of the file. E.g. I'd like to have a copy of the music from my primary "annex/Music" into the "annex/android/Music" tree, then "git rm" the file from "annex/android/Music" when I'm done with it. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/special_remote_adb_-_Argument_list_too_long/comment_1_34f202f91a16dd4d2bb557aee6fd5a3e._comment b/doc/bugs/special_remote_adb_-_Argument_list_too_long/comment_1_34f202f91a16dd4d2bb557aee6fd5a3e._comment deleted file mode 100644 index d8d07e4efd..0000000000 --- a/doc/bugs/special_remote_adb_-_Argument_list_too_long/comment_1_34f202f91a16dd4d2bb557aee6fd5a3e._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-01-31T16:45:55Z" - content=""" -Ah so unfortunately android's `find` does not break the command -runs by arrgument size like xargs does. So the `+` argument -can't be used, instead I'll make it pipe through xargs. - -(To copy a file, you can use `cp -a` or just `cp`, followed by `git-annex -add`. either will work.) -"""]] diff --git a/doc/bugs/special_remote_protocol__58___adding_local_state_variables___40__not_under_git__41__.mdwn b/doc/bugs/special_remote_protocol__58___adding_local_state_variables___40__not_under_git__41__.mdwn deleted file mode 100644 index 9b87d77c7d..0000000000 --- a/doc/bugs/special_remote_protocol__58___adding_local_state_variables___40__not_under_git__41__.mdwn +++ /dev/null @@ -1,17 +0,0 @@ -### Please describe the feature request. -It would be useful in the special remote protocol to add commands to store or retrieve local key / value pairs (not stored under git, but only in local files). At the time of writing it seems to me that there are no ways for a special remote to store key / value pairs that are not so relevant to be stored in the git-annex branch but may be of great relevance for other uses (such as optimization). - -More on my specific use of the feature: Resumable uploads -Use case: while uploading a file to a remote which allow resumable uploads (giving you a token to resume at each uploaded chunk), the network connection may drop. The special remote then may need to store the token value under a local file in order to try resume the upload at a later time. - -Since the token is not relevant to the story of the git-annex branch (not regarding the user or the repository itself) but of great relevance for optimization (and may also expire after some time) the addition of commands such as SETCONFIG and GETCONFIG (maybe with names like SETLOCALCONFIG and GETLOCALCONFIG) seems natural to me. They could be also useful to save other optimization values (settings that may need to vary from a repository to another one). - -### What version of git-annex are you using? On what operating system? -git-annex 6.20170101 on Debian stretch (from debian archives) - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) -I'm using (and enjoying) it a lot :) - -> Seems like my comment is reasonable, so no change to git-annex is needed. -> I would be open to being convinced otherwise though... [[wontfix|done]] -> for now. --[[Joey]] diff --git a/doc/bugs/special_remote_protocol__58___adding_local_state_variables___40__not_under_git__41__/comment_1_3d162dce2af8d84c5582cf63624e9310._comment b/doc/bugs/special_remote_protocol__58___adding_local_state_variables___40__not_under_git__41__/comment_1_3d162dce2af8d84c5582cf63624e9310._comment deleted file mode 100644 index 6fdd1e6ad9..0000000000 --- a/doc/bugs/special_remote_protocol__58___adding_local_state_variables___40__not_under_git__41__/comment_1_3d162dce2af8d84c5582cf63624e9310._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2017-02-07T16:49:40Z" - content=""" -A reasonable use case indeed. - -My initial feeling is, this can be handled well by storing the data -yourself in the filesysystem. You only need to pick a place for your -special remote to store the data. Presumably somewhere under `.git` -or under `.git/annex`. (The protocol's GETGITDIR can be used to -get the path to `.git`.) - -And you need to pick a filename that won't be used by something else. - -You can use the protocol's GETUUID to get the uuid of special remote, -and using that in the path to the data pretty well guarantees nothing -else is going to use that filename. -"""]] diff --git a/doc/bugs/surprising_import_tree_merge_result.mdwn b/doc/bugs/surprising_import_tree_merge_result.mdwn deleted file mode 100644 index 2476944794..0000000000 --- a/doc/bugs/surprising_import_tree_merge_result.mdwn +++ /dev/null @@ -1,18 +0,0 @@ -Had a surprising result of an import tree merge. I am importing photos -from my phone. I had earlier imported some photos and moved the whole tree -containing them from phone/DCIM to raw/photos/something. Then I took a new photo, -and imported again. - -one. In commit cd55a5be377020b7261c3a737b5e32a01e22f4b8 "import from phone" -that added a `phone/DCIM/Camera/IMG_20190516_111640.jpg`. - -But then in the merge commit a548a51de2566c26f74b5791eb0660e4d79e2cb7 -that merged phone/master into master, the new photo moved into -raw/photos/something - -This is quite surprising. It seems somehow when it modified the old -tree, it rooted the new at the location the old had been moved to in -master. - -> Gonna close this old bug since I suspect it's been fixed. [[done]] -> --[[Joey]] diff --git a/doc/bugs/sync_--content_succeeds_w__47__o_fetching_all_files.mdwn b/doc/bugs/sync_--content_succeeds_w__47__o_fetching_all_files.mdwn deleted file mode 100644 index cea39699d2..0000000000 --- a/doc/bugs/sync_--content_succeeds_w__47__o_fetching_all_files.mdwn +++ /dev/null @@ -1,101 +0,0 @@ -### Please describe the problem. -I had assumed that [[`git-annex-sync --content`|git-annex-sync]], if it succeeds, guarantees that there are no broken symlinks to the annex. But there is one circumstance when that's not true: if some of the files exist only on remotes that have not yet been enabled. This still holds under `git annex required here anything`. - -This could of course be avoided with `autoenable=true`, but this doesn't work for [[external special remotes|special_remotes/external]]. [[todo/dockerized_external_special_remotes]], if doable, would address this. - - -### What steps will reproduce the problem? -See below - -### What version of git-annex are you using? On what operating system? -[[!format sh """ -git-annex version: 8.20210224-gf951847c6 -build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: linux x86_64 -supported repository versions: 8 -upgrade supported from repository versions: 0 1 2 3 4 5 6 7 -local repository version: 8 - -Linux New-VirtualBox 5.8.0-48-generic #54~20.04.1-Ubuntu SMP Sat Mar 20 13:40:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux - -"""]] -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log -(ga-env-05) /data/tmp [10:27:08]$ mkdir t -(ga-env-05) /data/tmp [10:27:09]$ cd t -(ga-env-05) /data/tmp/t [10:27:10]$ git init -Initialized empty Git repository in /data/tmp/t/.git/ -(ga-env-05) /data/tmp/t [10:27:54]$ touch README -(ga-env-05) /data/tmp/t [10:27:59]$ git annex init -init (scanning for unlocked files...) -ok -(recording state in git...) -(ga-env-05) /data/tmp/t [10:28:04]$ git annex add README -add README ok -(recording state in git...) -(ga-env-05) /data/tmp/t [10:28:10]$ git commit -m 'checking in README' . -[master (root-commit) e3e6ecf] checking in README - 1 file changed, 1 insertion(+) - create mode 120000 README -(ga-env-05) /data/tmp/t [10:28:19]$ mkdir /data/tmp/dirremote -(ga-env-05) /data/tmp/t [10:29:09]$ git annex initremote dirremote type=directory directory=/data/tmp/dirremote encryption=none -initremote dirremote ok -(recording state in git...) -(ga-env-05) /data/tmp/t [10:30:00]$ git annex required dirremote anything -required dirremote ok -(recording state in git...) -(ga-env-05) /data/tmp/t [10:30:37]$ git annex sync --content -On branch master -nothing to commit, working tree clean -commit ok -copy README (to dirremote...) ok -(recording state in git...) -(ga-env-05) /data/tmp/t [10:31:51]$ git annex drop . -drop README ok -(recording state in git...) -(ga-env-05) /data/tmp/t [10:31:59]$ cd .. -(ga-env-05) /data/tmp [10:32:00]$ git clone t t3 -Cloning into 't3'... -done. -(ga-env-05) /data/tmp [10:32:06]$ cd t3 -(ga-env-05) /data/tmp/t3 [10:32:08]$ git annex init -init (merging origin/git-annex origin/synced/git-annex into git-annex...) -(recording state in git...) -(scanning for unlocked files...) -ok -(recording state in git...) -(ga-env-05) /data/tmp/t3 [10:32:10]$ git annex sync --content -On branch master -Your branch is up to date with 'origin/master'. - -nothing to commit, working tree clean -commit ok -pull origin ok -Enumerating objects: 20, done. -Counting objects: 100% (19/19), done. -Delta compression using up to 8 threads -Compressing objects: 100% (8/8), done. -Writing objects: 100% (10/10), 974 bytes | 974.00 KiB/s, done. -Total 10 (delta 5), reused 0 (delta 0), pack-reused 0 -To /data/tmp/t - 9726f90..f4f2f6c git-annex -> synced/git-annex -push origin ok -(ga-env-05) /data/tmp/t3 [10:32:19]$ echo $? -0 -(ga-env-05) /data/tmp/t3 [10:32:23]$ cat README -cat: README: No such file or directory -(ga-env-05) /data/tmp/t3 [10:32:26]$ - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) -git-annex is versatile and reliable, and reflects a crazy amount of work by @joeyh. Thanks again for it! - -> I think the documentation is reasonably clear now, so [[done]] --[[Joey]] diff --git a/doc/bugs/sync_--content_succeeds_w__47__o_fetching_all_files/comment_1_500bc8f48ae80841e0db21ee0de76c50._comment b/doc/bugs/sync_--content_succeeds_w__47__o_fetching_all_files/comment_1_500bc8f48ae80841e0db21ee0de76c50._comment deleted file mode 100644 index c372a93e52..0000000000 --- a/doc/bugs/sync_--content_succeeds_w__47__o_fetching_all_files/comment_1_500bc8f48ae80841e0db21ee0de76c50._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Lukey" - avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b" - subject="comment 1" - date="2021-03-30T16:51:53Z" - content=""" -This is expected. git annex sync --content only returns an error if copying annexed files to/from a remote failed. If there are no remotes configured or enabled, there is nothing to copy, so it returns success. -"""]] diff --git a/doc/bugs/sync_--content_succeeds_w__47__o_fetching_all_files/comment_1_9581f8135c6eb2ed2135f83a266e5306._comment b/doc/bugs/sync_--content_succeeds_w__47__o_fetching_all_files/comment_1_9581f8135c6eb2ed2135f83a266e5306._comment deleted file mode 100644 index b968ee2376..0000000000 --- a/doc/bugs/sync_--content_succeeds_w__47__o_fetching_all_files/comment_1_9581f8135c6eb2ed2135f83a266e5306._comment +++ /dev/null @@ -1,27 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2021-03-30T16:07:30Z" - content=""" -I don't think the `git-annex sync` documentation says in what situation -it will exit nonzero, so the extent this is really a bug is debatable. - -The general rule is that, if git-annex looks at a file or other item, -decides it needs to take some action, and then that action fails, it will -later exit nonzero. And sync does behave that way: When it determines a -file is not in any of the remotes it's syncing with, it does not try to -download it from anywhere, so no action fails. - -The counterargument to that would be by analogy to `git-annex get` -which does exit nonzero when no available remotes contain -a file (because it acts on every file that's not already present). -We could say that sync is currently behaving more like `git-annex get --from -foo` which skips files not on the specified remote, and that it's -doing it even when no particular remotes are specified, and that it would -be better, when no particular remotes are specified, for it to behave like -`git-annex get`. - -I don't know if most users would come up with this analogy though.. -I also wonder if you would not be better off just using `git-annex get` -if you want to check exit status based on if all files are present. -"""]] diff --git a/doc/bugs/sync_--content_succeeds_w__47__o_fetching_all_files/comment_3_dbc8ad383c4968b1ab78ac86bab06329._comment b/doc/bugs/sync_--content_succeeds_w__47__o_fetching_all_files/comment_3_dbc8ad383c4968b1ab78ac86bab06329._comment deleted file mode 100644 index 00a6af3e01..0000000000 --- a/doc/bugs/sync_--content_succeeds_w__47__o_fetching_all_files/comment_3_dbc8ad383c4968b1ab78ac86bab06329._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Ilya_Shlyakhter" - avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" - subject="comment 3" - date="2021-03-30T19:17:06Z" - content=""" -I guess the \"sync\" in `git-annex-sync` made me think of file synchronizers like `rsync`, which do bring in all remote files not currently local. Maybe just change the manpage to clarify that `git-annex-sync` only synchronizes with [[_enabled_|git-annex-enableremote]] remotes. -"""]] diff --git a/doc/bugs/sync_--content_succeeds_w__47__o_fetching_all_files/comment_4_23a16e94a0f18126caa6cb2f94131277._comment b/doc/bugs/sync_--content_succeeds_w__47__o_fetching_all_files/comment_4_23a16e94a0f18126caa6cb2f94131277._comment deleted file mode 100644 index e8d972db4d..0000000000 --- a/doc/bugs/sync_--content_succeeds_w__47__o_fetching_all_files/comment_4_23a16e94a0f18126caa6cb2f94131277._comment +++ /dev/null @@ -1,20 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2021-04-01T16:36:11Z" - content=""" -It would be impossible to sync with a remote that is not enabled, -so I don't know if that is a useful distinction to draw. - -Also, if a file is not present in any remotes yet either, syncing will -not get it, and not fail, and leave the local repo without the file's -content. And this is not happening because a remote is not enabled, -but because there is no path to get the data from the repo that contains -it. - -I did change the man page to say "The --content option causes the content of annexed files -to also be uploaded and downloaded as necessary, to sync the content between the repository and its remotes." - -Which may be explcit enough to remove the ambiguity that surprised you, -but I'm not sure if I've hit on the right wording. -"""]] diff --git a/doc/bugs/sync_-C_can_export_other_files.mdwn b/doc/bugs/sync_-C_can_export_other_files.mdwn deleted file mode 100644 index db63c9d7c5..0000000000 --- a/doc/bugs/sync_-C_can_export_other_files.mdwn +++ /dev/null @@ -1,17 +0,0 @@ -When an exporttree=yes remote has had a tree exported to it, but eg with -`git-annex export master --to foo --fast`, so not all the files have been -sent to it yet, `git-annex sync -Cfoo` will also export files in other -directories to it. - -This seems like a bug, it ought to only export any files in the specified -directory. - -Another way this can happen is `git-annex sync -Cfoo`, when file bar -has changed. Currently this exports bar to the export remote, but does -not send it to any other remotes, since it's not in directory foo. - -Also this affects `git-annex satisfy`, and other related commands too. ---[[Joey]] - -> Decided to just document this behavior since there does not seem to be -> any good alternative behavior. [[done]] --[[Joey]] diff --git a/doc/bugs/sync_-C_can_export_other_files/comment_1_7a6ab3ee25f0d09df4d7478d26ea2d37._comment b/doc/bugs/sync_-C_can_export_other_files/comment_1_7a6ab3ee25f0d09df4d7478d26ea2d37._comment deleted file mode 100644 index 03193f7e13..0000000000 --- a/doc/bugs/sync_-C_can_export_other_files/comment_1_7a6ab3ee25f0d09df4d7478d26ea2d37._comment +++ /dev/null @@ -1,30 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-09-22T16:26:17Z" - content=""" -If files foo and bar have changed, and `git-annex sync -Cfoo` committed -those changes and only exported foo to an exporttree remote, then that -remote would end up with content that does not correspond to any git tree -that the user has constructed. - -While it's possible for that to also happen when an export of a tree gets -interrupted in the middle, git-annex recovers from that when the export is -re-run. But here the user might have a workflow where they are only running -that command. - -So this would be a new case, and to handle this case it seems it would have -to dummy up a tree representing the changed foo and old bar, in order to -record the export in the git-annex branch and sync it so other repos know -what the content of the export remote is. - -So doing that is not feeling like an entirely good idea.. - -I suppose the alternative would be for `git-annex sync -C` to not send -anything to export remotes. But that also seems potentially confusing. -(Especially as a behavior change.) If I've asked git-annex to sync with an -exporttree remote, shouldn't it do it? Even if it has to do some additional -transferring in the process. - -So it may be that the current behavior of sending additional files is better. -"""]] diff --git a/doc/bugs/sync_claims_data_loss_but_seems_to_just_lose_tracking.mdwn b/doc/bugs/sync_claims_data_loss_but_seems_to_just_lose_tracking.mdwn deleted file mode 100644 index e596e8424b..0000000000 --- a/doc/bugs/sync_claims_data_loss_but_seems_to_just_lose_tracking.mdwn +++ /dev/null @@ -1,218 +0,0 @@ -### Please describe the problem. -Sync / Assistant between a `source` and `transfer` repo results in zero copies for whereis, but data is available. - - -### What steps will reproduce the problem? -See a script below... - -``` sh -#!/bin/sh - -TMP=$(mktemp -d) - - -########################################## -# Set up the repos -########################################## - - -### -# Create working directories -cd ${TMP} -mkdir source transfer - - -### -# Set up the source repo -cd ${TMP}/source -git init -git annex init source -git-annex group here source -git annex wanted here standard - - -### -# Set up the transfer repo -# N.B. It can see the source repo -cd ${TMP}/transfer -git init -git remote add source ${TMP}/source -git annex init transfer -git-annex group here transfer -git annex wanted here \ - 'not (inallgroup=client and copies=client:1) and ((include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1)' - - -########################################## -# Add some files and run the assistant -########################################## - -### -# Make sure it works by adding some files to the source... -cd ${TMP}/source -date > file.1 -date > file.2 -date > file.3 - -# Run the assistant in the source... -(cd ${TMP}/source && \ - git annex assistant && \ - sleep 30 && \ - git annex assistant --stop && \ - git annex whereis file.*) -# PASS: All files are in source. - -# Now run it in the transfer group -(cd ${TMP}/transfer && \ - git annex assistant && \ - sleep 30 && \ - git annex assistant --stop && \ - git annex whereis file.*) -# PASS: All files are in transfer - -# Check the source -(cd ${TMP}/source && \ - git annex whereis file.*) -# FAIL: Only file.1 was updated to be in source. - -# whereis file.1 (1 copy) -# 64b193e0-86f0-4349-b073-70af919ce628 -- transfer -# ok -# whereis file.2 (0 copies) failed -# whereis file.3 (0 copies) failed -# git-annex: whereis: 2 failed - - -########################################## -# Start to scratch head and ask 'Why?' -########################################## - -########## -# What's happening??? - -cd ${TMP}/source && git annex sync -# Source thinks it is updated - working tree clean - -cd ${TMP}/transfer && git annex sync -# Transfer pulls from source, so run again -git annex sync -# OK... working tree clean - - -# Check again -cd ${TMP}/source && git annex whereis file.* -# whereis file.1 (1 copy) -# 64b193e0-86f0-4349-b073-70af919ce628 -- transfer -# ok -# whereis file.2 (0 copies) failed -# whereis file.3 (0 copies) failed -# git-annex: whereis: 2 failed - -cd ${TMP}/transfer && git annex whereis file.* -# whereis file.1 (1 copy) -# 64b193e0-86f0-4349-b073-70af919ce628 -- transfer [here] -# ok -# whereis file.2 (0 copies) failed -# whereis file.3 (0 copies) failed -# git-annex: whereis: 2 failed - - -# HOWEVER... Files are definitely in the transfer repo - -cd ${TMP}/source && cat file.* -# cat: file.1: No such file or directory -# cat: file.2: No such file or directory -# cat: file.3: No such file or directory - - -cd ${TMP}/transfer && cat file.* -# Sun May 14 21:22:02 PDT 2017 -# Sun May 14 21:22:03 PDT 2017 -# Sun May 14 21:22:06 PDT 2017 -``` - - - -### What version of git-annex are you using? On what operating system? - -#### Version: -``` -git-annex version: 6.20170510 -build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV FsEvents ConcurrentOutput TorrentParser MagicMime Feeds Quvi -dependency versions: aws-0.16 bloomfilter-2.0.1.0 cryptonite-0.23 DAV-1.3.1 feed-0.3.12.0 ghc-8.0.2 http-client-0.5.6.1 persistent-sqlite-2.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.4.5 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external -local repository version: 5 -supported repository versions: 3 5 6 -upgrade supported from repository versions: 0 1 2 3 4 5 -operating system: darwin x86_64 -``` - -#### OS: -MacOS 10.12.4, but seems to apply to Debian Jessie too. - - -### Please provide any additional information below. - -Annex assistant daemon.log for `source` repo: - -``` sh -[2017-05-15 14:11:24.150202] main: starting assistant version 6.20170510 -[2017-05-15 14:11:24.196597] Cronner: You should enable consistency checking to protect your data. -(scanning...) [2017-05-15 14:11:24.257123] Watcher: Performing startup scan -(started...) -[2017-05-15 14:11:45.505855] Committer: Adding file.1 file.2 file.3 -add file.1 ok -add file.2 ok -add file.3 ok -[2017-05-15 14:11:45.544277] Committer: Committing changes to git -(recording state in git...) -(recording state in git...) -``` - -Annex assistant daemon.log for `transfer` repo: - -``` sh -[2017-05-15 14:12:09.127406] main: starting assistant version 6.20170510 -[2017-05-15 14:12:09.157611] Cronner: You should enable consistency checking to protect your data. -[2017-05-15 14:12:09.177452] TransferScanner: Syncing with source -(scanning...) [2017-05-15 14:12:09.210605] Watcher: Performing startup scan -(started...) -warning: no common commits -From /Users/olaf/tmp/git-annex/play/bug-play/source - * [new branch] git-annex -> source/git-annex - * [new branch] master -> source/master - * [new branch] synced/master -> source/synced/master -(merging source/git-annex into git-annex...) -(recording state in git...) - - -Already up-to-date. -[2017-05-15 14:12:10.256195] Committer: Committing changes to git -(recording state in git...) -(checksum...) [2017-05-15 14:12:10.425494] Transferrer: Downloaded file.1 -[2017-05-15 14:12:11.17664] Pusher: Syncing with source -(recording state in git...) -[2017-05-15 14:12:11.356447] Committer: Committing changes to git -(recording state in git...) -To /Users/olaf/tmp/git-annex/play/bug-play/source - * [new branch] git-annex -> synced/git-annex -(checksum...) [2017-05-15 14:12:11.428676] Transferrer: Downloaded file.2 -(checksum...) [2017-05-15 14:12:11.451457] Transferrer: Downloaded file.3 -drop source file.1 ok -drop source file.2 ok -drop source file.3 ok -[2017-05-15 14:12:13.477636] Pusher: Syncing with source -(recording state in git...) -To /Users/olaf/tmp/git-annex/play/bug-play/source - 825d3dd..60c96cb git-annex -> synced/git-annex -``` - - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Yes. Love it. Donated. Have been using it for years. Recommend it and get(/force) my collaborators to use it. ;-) - -> Since this bug involved direct mode, and direct mode was known to be -> rather buggy and has been removed, I think it's best to close it. -> [[done]] --[[Joey]] diff --git a/doc/bugs/sync_claims_data_loss_but_seems_to_just_lose_tracking/comment_1_562cf3ab3398c1ef874c0bfe5d89e47b._comment b/doc/bugs/sync_claims_data_loss_but_seems_to_just_lose_tracking/comment_1_562cf3ab3398c1ef874c0bfe5d89e47b._comment deleted file mode 100644 index 7511c36898..0000000000 --- a/doc/bugs/sync_claims_data_loss_but_seems_to_just_lose_tracking/comment_1_562cf3ab3398c1ef874c0bfe5d89e47b._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="leavingchicago@c04c893e78d1c4c76cb3e32b5c227cf42bbf7682" - nickname="leavingchicago" - avatar="http://cdn.libravatar.org/avatar/4ae498d3d6ee558d6b65caa658f72572" - subject="Related..." - date="2017-05-15T04:43:44Z" - content=""" -I think this bug is related to the problems that I'm having with [/forum/Get source group to automatically drop with assistant](https://git-annex.branchable.com/forum/Get___39__source__39___group_to_automatically_drop_with_assistant/) - -"""]] diff --git a/doc/bugs/sync_claims_data_loss_but_seems_to_just_lose_tracking/comment_2_f686273c6be899d2a8bbaabd03a47fbf._comment b/doc/bugs/sync_claims_data_loss_but_seems_to_just_lose_tracking/comment_2_f686273c6be899d2a8bbaabd03a47fbf._comment deleted file mode 100644 index 34ed9f1d7d..0000000000 --- a/doc/bugs/sync_claims_data_loss_but_seems_to_just_lose_tracking/comment_2_f686273c6be899d2a8bbaabd03a47fbf._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2017-06-09T17:25:23Z" - content=""" -I have not had any luck with reproducing the problem, using -your script, on Debian unstable. The transfer repo gets -the file contents and both it and the source repo knows where they are. - -Do you see the same behavior if, rather than running the assistant -for 30 seconds, you run `git annex sync --content` ? - -Are you using git-annex in direct mode and/or on an unusual filesystem? -"""]] diff --git a/doc/bugs/sync_claims_data_loss_but_seems_to_just_lose_tracking/comment_3_fd518d0601587dc5c718944d0b6f51fc._comment b/doc/bugs/sync_claims_data_loss_but_seems_to_just_lose_tracking/comment_3_fd518d0601587dc5c718944d0b6f51fc._comment deleted file mode 100644 index 79e388a4dc..0000000000 --- a/doc/bugs/sync_claims_data_loss_but_seems_to_just_lose_tracking/comment_3_fd518d0601587dc5c718944d0b6f51fc._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2017-06-09T17:33:17Z" - content=""" -I tried the script with `git annex direct` added at initialization, -and didn't see the problem as described, but the assistant -did fail to get the content into the transfer repo at all -when using direct mode. `git annex sync --content` had no difficulty. -"""]] diff --git a/doc/bugs/sync_claims_data_loss_but_seems_to_just_lose_tracking/comment_4_a6074b68754b0d773385a1e406043952._comment b/doc/bugs/sync_claims_data_loss_but_seems_to_just_lose_tracking/comment_4_a6074b68754b0d773385a1e406043952._comment deleted file mode 100644 index 9eb06cf992..0000000000 --- a/doc/bugs/sync_claims_data_loss_but_seems_to_just_lose_tracking/comment_4_a6074b68754b0d773385a1e406043952._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="olaf" - avatar="http://cdn.libravatar.org/avatar/4ae498d3d6ee558d6b65caa658f72572" - subject="comment 4" - date="2017-06-22T04:55:35Z" - content=""" -So, you're suggesting manually running `git annex sync --content`? I can `cron` that up, so I can live with it. - -Do you need any more info to identify the issue you found with the assistant? -"""]] diff --git a/doc/bugs/sync_does_not_propagate_forgetfulness.mdwn b/doc/bugs/sync_does_not_propagate_forgetfulness.mdwn deleted file mode 100644 index ebc018cbcb..0000000000 --- a/doc/bugs/sync_does_not_propagate_forgetfulness.mdwn +++ /dev/null @@ -1,35 +0,0 @@ -After a `git-annex forget`, syncing with a remote that still has the -forgotten data in its git-annex branch merges that data right back in. - -Reproduction script: - - chmod -R +w bench 2>/dev/null - rm -rf bench 2>/dev/null - mkdir bench - cd bench - git init a - cd a - git-annex init - touch foo - git-annex add foo - git commit -m add - cd .. - git clone a b - cd b - git-annex get - git-annex drop - git-annex get - git-annex drop - cd ../a - git-annex sync - git-annex forget --force - git-annex sync - git-annex log foo - cd ../b - git-annex sync - cd ../a - git-annex log foo - ---[[Joey]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/sync_does_not_propagate_forgetfulness/comment_1_acc9744be183c297c6410f9b0a30b46b._comment b/doc/bugs/sync_does_not_propagate_forgetfulness/comment_1_acc9744be183c297c6410f9b0a30b46b._comment deleted file mode 100644 index 2646ff3212..0000000000 --- a/doc/bugs/sync_does_not_propagate_forgetfulness/comment_1_acc9744be183c297c6410f9b0a30b46b._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-02-07T17:49:37Z" - content=""" -Reversion introduced in [[!commit 2eadb6cd68073a905e8ff32cc8093a43d0b05c81]]. -Parsing of the transitions log silently failed, so all propagation of -transitions was broken. -"""]] diff --git a/doc/bugs/sync_with_export_remote_deletes_files_from_it_when_ran_in_other_than_tracking_branch.mdwn b/doc/bugs/sync_with_export_remote_deletes_files_from_it_when_ran_in_other_than_tracking_branch.mdwn deleted file mode 100644 index c5621e674b..0000000000 --- a/doc/bugs/sync_with_export_remote_deletes_files_from_it_when_ran_in_other_than_tracking_branch.mdwn +++ /dev/null @@ -1,18 +0,0 @@ - git-annex initremote d type=directory directory=../d exporttree=yes importtree=yes encryption=none - git config remote.d.annex-tracking-branch master - date > ../d/foo - git checkout -b bar - git-annex sync --content d - -This will import foo from d, but then it unexports foo from d. - -Not quite data loss, because it does honor numcopies and the local copy -remains. But bad. - -The problem seems to be that it assumes it's merged d/master into master, but -that does not happen since master is not the current branch. And so it then -updates d to look like master. --[[Joey]] - -> [[fixed|done]] by avoiding updating export remotes when -> the current branch tree is not the same as the tracking branch tree. -> --[[Joey]] diff --git a/doc/bugs/tasty_test_limiting_broken_by_concurrency.mdwn b/doc/bugs/tasty_test_limiting_broken_by_concurrency.mdwn deleted file mode 100644 index d557dac36e..0000000000 --- a/doc/bugs/tasty_test_limiting_broken_by_concurrency.mdwn +++ /dev/null @@ -1,19 +0,0 @@ -The changes to make `git-annex test` concurrent have -broken using eg `git-annex test -p 'concurrent get of dup key regression'` - -It looks like the repo setup test is not being run, even though it's -supposed to be a dependency of the test it was limited to. ---[[Joey]] - -> Oh, that seems to be a limitation of tasty. From its docs: - - If Test B depends on Test A, remember that either of them may be - filtered out using the --pattern option. Collecting the dependency - info happens after filtering. Therefore, if Test A is filtered out, - Test B will run unconditionally, and if Test B is filtered out, - it simply won't run. - -> This works: `git-annex test -p '/concurrent get of dup key regression/ || /Init Tests/'` -> -> Ok, I was able to work around this by having git-annex test add the latter -> pattern automatically. [[done]] --[[Joey]] diff --git a/doc/bugs/tasty_test_limiting_broken_by_concurrency/comment_1_e043a71b60999cf97bd872d8b32fcac9._comment b/doc/bugs/tasty_test_limiting_broken_by_concurrency/comment_1_e043a71b60999cf97bd872d8b32fcac9._comment deleted file mode 100644 index 592b8a45fa..0000000000 --- a/doc/bugs/tasty_test_limiting_broken_by_concurrency/comment_1_e043a71b60999cf97bd872d8b32fcac9._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 1" - date="2022-05-24T02:46:17Z" - content=""" -Well done. Btw if somebody (like me) wonders why their tasty awk patterns '/like this/' don't work with `git-annex test -p` -in Windows Git Bash, one has to remember that Git Bash is based on Cygwin/Msys and there's a Unix -> Windows -path conversion at work. You can stop that conversion by prepending a space to the awk pattern ' /like this/' -or by setting the env var MSYS_NO_PATHCONV=1. So basically the above should be formulated as follows in this arguably hobbled environment: - -[[!format sh \"\"\" -$ MSYS_NO_PATHCONV=1 git-annex test -p '/concurrent get of dup key regression/ || /Init Tests/' -\"\"\"]] - -"""]] diff --git a/doc/bugs/test_prop__95__relPathDirToFileAbs__95__basics_fails_again.mdwn b/doc/bugs/test_prop__95__relPathDirToFileAbs__95__basics_fails_again.mdwn deleted file mode 100644 index ddf04529cf..0000000000 --- a/doc/bugs/test_prop__95__relPathDirToFileAbs__95__basics_fails_again.mdwn +++ /dev/null @@ -1,55 +0,0 @@ -### Please describe the problem. - -There was this springy little one that sprung -- just like a seedling :) - -Running the test QuickCheck.prop_relPathDirToFileAbs_basics fails (again) in a particular case (but only occasionally). - -Running the following fails: - -[[!format sh """ -jkniiv@AINESIS MINGW64 /c/annx -$ ./git-annex test --jobs=1 -p QuickCheck.prop_relPathDirToFileAbs_basics --quickcheck-replay=586868 2>&1 | tee git-annex.test--p-QuickCheck_prop_relPathDirToFileAbs_basics.LOG~201 -Tests - QuickCheck - prop_relPathDirToFileAbs_basics: FAIL (0.03s) - *** Failed! Falsified (after 711 tests): - TestableFilePath {fromTestableFilePath = ":/"} - Use --quickcheck-replay=586868 to reproduce. - -1 out of 1 tests failed (0.03s) - -All 0 tests passed (0.00s) - -All 0 tests passed (0.01s) - -All 0 tests passed (0.00s) - Test subprocesses exited with unexpected exit codes: [ExitSuccess,ExitSuccess,ExitSuccess] - -# End of transcript. -"""]] - -If you remove the option `--quickcheck-replay=586868`, the test usually passes. - - -### What version of git-annex are you using? On what operating system? - -[[!format sh """ -git-annex version: 10.20220223-g8f6b52b77 -build flags: Assistant Webapp Pairing TorrentParser Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.7 http-client-0.7.9 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.1.2 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: mingw32 x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 2 3 4 5 6 7 8 9 10 -"""]] - -Windows 10 version 21H2 (build 19044.1586), 64 bit. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Git Annex is great. I use it several times a week with my multigigabyte backups, where it gives structure to my image-based backup routines, so you could say I'm a believer. :) - -[[!meta author=jkniiv]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/uftp_5__58___invalid_key_type.mdwn b/doc/bugs/uftp_5__58___invalid_key_type.mdwn deleted file mode 100644 index 440c040133..0000000000 --- a/doc/bugs/uftp_5__58___invalid_key_type.mdwn +++ /dev/null @@ -1,55 +0,0 @@ -### Please describe the problem. - -git annex multicast --send quits with error "Invalid keytype" - - -### What steps will reproduce the problem? - -Follow the example in the multicast command help page, the send operation fails when uftp version is 5.0. - -Probably the reason is the removal of aes256-cbc in version 5.0 where they left aes256-gcm - -### What version of git-annex are you using? On what operating system? - -git-annex version: 8.20211028 -OS is nixos-21.11 with uftp installed -uftp 5.0 - -### Operation log - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log -$ git annex multicast --send --debug -sending files [2022-04-17 00:15:46.699593866] (Utility.Process) process [60540] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","symbolic-ref","-q","HEAD"] -[2022-04-17 00:15:46.701266922] (Utility.Process) process [60540] done ExitSuccess -[2022-04-17 00:15:46.701754626] (Utility.Process) process [60541] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","refs/heads/master"] -[2022-04-17 00:15:46.703889652] (Utility.Process) process [60541] done ExitSuccess -[2022-04-17 00:15:46.70436486] (Utility.Process) process [60542] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","ls-files","-z","--cached","--"] -[2022-04-17 00:15:46.706736386] (Utility.Process) process [60543] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] -[2022-04-17 00:15:46.709237395] (Utility.Process) process [60544] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"] -[2022-04-17 00:15:46.729027634] (Utility.Process) process [60542] done ExitSuccess -[2022-04-17 00:15:46.729292044] (Utility.Process) process [60545] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","git-annex"] -[2022-04-17 00:15:46.730468539] (Utility.Process) process [60545] done ExitSuccess -[2022-04-17 00:15:46.730735082] (Utility.Process) process [60546] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"] -[2022-04-17 00:15:46.73204795] (Utility.Process) process [60546] done ExitSuccess -[2022-04-17 00:15:46.732421746] (Utility.Process) process [60547] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/git-annex..fd32a06095805399cdaf708cd294158b8d05027b","--pretty=%H","-n1"] -[2022-04-17 00:15:46.733706532] (Utility.Process) process [60547] done ExitSuccess -[2022-04-17 00:15:46.735507734] (Utility.Process) process [60548] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"] -[2022-04-17 00:15:46.737226695] (Utility.Process) process [60549] call: uftp ["-c","-Y","aes256-cbc","-h","sha512","-e","ecdh_ecdsa","-k",".git/annex/creds/multicast","-U","0x1de1f677","-H","@/tmp/authlist60534-1","-i","/tmp/send60534-0"] -Invalid keytype -[2022-04-17 00:15:46.748492634] (Utility.Process) process [60549] done ExitFailure 1 -failed -ok - - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Of course! I'm using it to track assets in an infrastructure repository - -> I've changed it to use aes256-gcm. I have not verified if there -> is other breakage from the new version, so file a bug if you find any. -> [[done]] --[[Joey]] diff --git a/doc/bugs/unannex__58___Cannot_proceed_with_uncommitted_changes_staged_in_the_index.mdwn b/doc/bugs/unannex__58___Cannot_proceed_with_uncommitted_changes_staged_in_the_index.mdwn deleted file mode 100644 index e25c72095d..0000000000 --- a/doc/bugs/unannex__58___Cannot_proceed_with_uncommitted_changes_staged_in_the_index.mdwn +++ /dev/null @@ -1,39 +0,0 @@ -### Please describe the problem. - -I understand that `git annex unannex` is essentially there for undoing an accidental `git annex add`. Unfortunately it doesn't do that. - -If I have uncommitted changes, which is the case after a `git annex add`, it tells me: - - git-annex: Cannot proceed with uncommitted changes staged in the index. Recommend you: git commit - CallStack (from HasCallStack): - error, called at ./Command/Unannex.hs:48:19 in main:Command.Unannex - -But I would expect it to `git reset` the file and then replace the symlink by the actual file content. - -### What steps will reproduce the problem? - - > git init - Initialized empty Git repository in /somewhere/.git - > git annex init - init ok - (recording state in git...) - > touch foo - > git annex add foo - add foo ok - (recording state in git...) - > git annex unannex foo - git-annex: Cannot proceed with uncommitted changes staged in the index. Recommend you: git commit - CallStack (from HasCallStack): - error, called at ./Command/Unannex.hs:48:19 in main:Command.Unannex - -### What version of git-annex are you using? On what operating system? - -git-annex version: 6.20160527-gf21a425 - -Installed from the Arch Linux repository. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Sure, I'm using it for photos, music and videos - -> This got fixed in v7 repositories. [[done]] --[[Joey]] diff --git a/doc/bugs/unannex__58___Cannot_proceed_with_uncommitted_changes_staged_in_the_index/comment_1_1c0cad1076d5d4d908b8297e7c13ea33._comment b/doc/bugs/unannex__58___Cannot_proceed_with_uncommitted_changes_staged_in_the_index/comment_1_1c0cad1076d5d4d908b8297e7c13ea33._comment deleted file mode 100644 index 4b54996eb4..0000000000 --- a/doc/bugs/unannex__58___Cannot_proceed_with_uncommitted_changes_staged_in_the_index/comment_1_1c0cad1076d5d4d908b8297e7c13ea33._comment +++ /dev/null @@ -1,32 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2016-07-12T16:23:51Z" - content=""" -In v5 mode, there is a complex interaction between unannex and the -pre-commit hook. An unannexed file looks quite a lot like an unlocked file, -so the pre-commit hook is prone to want to lock it, and so add it back to -the annex as an annexed file. - -To avoid that problem, unannex needs to commit the unannexing of the -files. - -However, if you have other staged changes, they'll also be included in that -commit. Which would be a bug if it were allowed to happen. This is why -it checks for a clean index first. - -It would be possible to improve the behavior by explicitly committing only -the files that got unannexed, rather than all staged changes. Why didn't I -do that? - -Well, `git commit $file` stages any changes to the file's content before -committing. When the file has been unannexed, this stages the entire large -file content into git, and adds it back. Not the desired behavior! - -Git in fact has no interface to make it commit only staged changes to -only specific files. I can't get there from here. It would certianly -be nice if git got the ability to do that, if someone wants a project to -improve git. - -It's very nice that v6 mode avoids this problem entirely! -"""]] diff --git a/doc/bugs/unannex__58___Cannot_proceed_with_uncommitted_changes_staged_in_the_index/comment_2_4609f9a1545b08e08021bf786561f5e5._comment b/doc/bugs/unannex__58___Cannot_proceed_with_uncommitted_changes_staged_in_the_index/comment_2_4609f9a1545b08e08021bf786561f5e5._comment deleted file mode 100644 index 1c3169e70f..0000000000 --- a/doc/bugs/unannex__58___Cannot_proceed_with_uncommitted_changes_staged_in_the_index/comment_2_4609f9a1545b08e08021bf786561f5e5._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="tom.prince@6bf26c878bf6103667f28d70cf49d4fb74d33df7" - nickname="tom.prince" - avatar="http://cdn.libravatar.org/avatar/e81edff3af564b86f4c9d780a8023299" - subject="comment 2" - date="2017-07-21T18:22:28Z" - content=""" -As far as I can tell, from looking at the code, the pre-commit hook only looks at files in the index. Thus, if unannexing an uncommited file removed it from the index, the pre-commit will do the right thing. This would be nice to have, at least for the case where the files have never been committed. -"""]] diff --git a/doc/bugs/unannex__58___Cannot_proceed_with_uncommitted_changes_staged_in_the_index/comment_3_6c99a97d56b1a3b12092c15fafcf8761._comment b/doc/bugs/unannex__58___Cannot_proceed_with_uncommitted_changes_staged_in_the_index/comment_3_6c99a97d56b1a3b12092c15fafcf8761._comment deleted file mode 100644 index 47d417cec0..0000000000 --- a/doc/bugs/unannex__58___Cannot_proceed_with_uncommitted_changes_staged_in_the_index/comment_3_6c99a97d56b1a3b12092c15fafcf8761._comment +++ /dev/null @@ -1,35 +0,0 @@ -[[!comment format=mdwn - username="https://launchpad.net/~stephane-gourichon-lpad" - nickname="stephane-gourichon-lpad" - avatar="http://cdn.libravatar.org/avatar/02d4a0af59175f9123720b4481d55a769ba954e20f6dd9b2792217d9fa0c6089" - subject="User expectations and what git annex unannex does." - date="2017-07-24T08:06:55Z" - content=""" -# Where we are - -@joey thank you for these explanations, more detailed than when I reported the same problem 8 months ago commenting https://git-annex.branchable.com/git-annex-unannex/ (@tom.prince had already written this page but I did not find it). - -Yet all this happens in a git world, where private history can be rewritten, so *there must be a simpler way*. - -# What people expect from \"undo accidental add command\" - -@tom.prince thanks for explaining what you expected `unannex` to do. Looks like I expected exactly the same behavior. - -IMHO current behavior of `git annex unannex` does not match what people expect of \"undo accidental add command\". - -# What current `git-annex unannex` actually does - -If behavior does not match words, perhaps behavior is interesting but should be matched with different words? - -Looking at what `git-annex unannex`, here are the words that came to mind: - -> git-annex unannex - turn a path which points to annexed content into a plain file that is store in regular git. - -Notice that: - -* `git-annex` retains history -* other paths may still refer to the same content, so the annex may still contain a copy of the same data. Else it becomes unused content subject to `git-annex dropunused`. - -Thank you for your attention. - -"""]] diff --git a/doc/bugs/uninit_fails_when_I_symlink_your_symlink.mdwn b/doc/bugs/uninit_fails_when_I_symlink_your_symlink.mdwn deleted file mode 100644 index 1e3506cb40..0000000000 --- a/doc/bugs/uninit_fails_when_I_symlink_your_symlink.mdwn +++ /dev/null @@ -1,39 +0,0 @@ -``` -$ mkdir annex-test -$ cd annex-test/ -$ git init -Initialized empty Git repository in /home/me/annex-test/.git/ -$ git annex init -init (scanning for unlocked files...) -ok -(recording state in git...) -$ cat > .gitignore -* -$ cat > exampyle.txt -hello -$ git annex add --no-check-gitignore -add .gitignore (non-large file; adding content to git repository) ok -add exampyle.txt -ok -(recording state in git...) -$ git commit -m \"added\" -[master (root-commit) 2734615] added - 2 files changed, 2 insertions(+) - create mode 100644 .gitignore - create mode 120000 exampyle.txt -$ ln -rs exampyle.txt what.foo -$ git status -On branch master -nothing to commit, working tree clean -$ git annex uninit -git-annex: what.foo points to annexed content, but is not checked into git. -Perhaps this was left behind by an interrupted git annex add? -Not continuing with uninit; either delete or git annex add the file and retry. -``` -What should I do? - -Honestly, I'm happy with git-annex so far, I'm just thinking that I need to re-init with `annex.tune.objecthashlower=true` because my other computer is windows. - -Thanks! - -> I see this was resolved as ok behavior, so [[done]] --[[Joey]] diff --git a/doc/bugs/uninit_fails_when_I_symlink_your_symlink/comment_5_3fe7520813dee36ffb5419ed70ea43b5._comment b/doc/bugs/uninit_fails_when_I_symlink_your_symlink/comment_5_3fe7520813dee36ffb5419ed70ea43b5._comment deleted file mode 100644 index 7514aa5e68..0000000000 --- a/doc/bugs/uninit_fails_when_I_symlink_your_symlink/comment_5_3fe7520813dee36ffb5419ed70ea43b5._comment +++ /dev/null @@ -1,24 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="forgot to add the new ignored link + shortcoming of 'ln' command" - date="2023-11-20T10:25:52Z" - content=""" -You forgot to add your (gitignore'd) `what.foo` to git. Another `git annex add --no-check-gitignore;git commit -m \"add link\"` will make the subsequent `git annex uninit` work properly. This is also what the error message says, btw. - -But there's another problem underneath: Why is `what.foo` an annex-style symlink anyway? It should just point to `example.txt`, right? - -``` -🐟 ❯ ln -nrs example.txt what.foo -yann in yann-desktop-nixos in …/uninit-test on  main -🐟 ❯ ls -l -lrwxrwxrwx 186 yann users 20 Nov 11:08  example.txt -> .git/annex/objects/mK/4w/SHA256E-s6--5891b5b522d5df086d0ff0b110fbd9d21bb4fc7163af34d08286a2e846f6be03.txt/SHA256E-s6--5891b5b522d5df086d0ff0b110fbd9d21bb4fc7163af34d08286a2e846f6be03.txt -lrwxrwxrwx 186 yann users 20 Nov 11:08  what.foo -> .git/annex/objects/mK/4w/SHA256E-s6--5891b5b522d5df086d0ff0b110fbd9d21bb4fc7163af34d08286a2e846f6be03.txt/SHA256E-s6--5891b5b522d5df086d0ff0b110fbd9d21bb4fc7163af34d08286a2e846f6be03.txt -``` - -For some reason `ln` dereferences `example.txt` before linking, resulting in `what.foo` pointing to the same as `example.txt` -- not just to `example.txt` -- causing `what.foo` to look exactly like an annex link. I thought `-n` could fix this, but it would operate on the target `what.foo`, not the source `example.txt` 🤦. I couldn't make `ln` do this properly, except for renaming/removing `example.txt`, then `ln -rsf example.txt what.foo`, then restoring `example.txt`. Not viable. A better solution is `cp -s example.txt what.foo`. That will make `what.foo` point properly to just `example.txt` and not *its* target. - -BTW you can still just `git add` things like normal non-annex symlinks like `what.foo`, no need for `git annex add` here. - -Also, joey prefers to have bug reports/questions and the like in either [[todo]], [[bugs]] or [[forum]], because it's clear those are the places to look through for issues. Comments below random manpages are quick to be forgotten about. 🙂 -"""]] diff --git a/doc/bugs/uninit_fails_when_I_symlink_your_symlink/comment_6_3afdea4f3705a9cfaa971ed3aa9f114c._comment b/doc/bugs/uninit_fails_when_I_symlink_your_symlink/comment_6_3afdea4f3705a9cfaa971ed3aa9f114c._comment deleted file mode 100644 index d1109eb926..0000000000 --- a/doc/bugs/uninit_fails_when_I_symlink_your_symlink/comment_6_3afdea4f3705a9cfaa971ed3aa9f114c._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="NewUser" - nickname="dont.post.me" - avatar="http://cdn.libravatar.org/avatar/90f59ddc341eaf9b2657422206c06901" - subject="comment 6" - date="2023-11-20T13:17:03Z" - content=""" -Thank you for the illuminating comment! - -> This is also what the error message says, btw. - - Not continuing with uninit; either delete or git annex add the file and retry. - -You’re right! I thought I only had a relative symlink pointing to the other symlink and not pointing into .git/annex/objects. It turns out that this is true for most of my symlinks but, just like you pointed out, some of them were pointing into .git/annex and the right thing to do there is to `add` them. - -Also, thank you for pointing me towards todo/bugs/forum! - -"""]] diff --git a/doc/bugs/unnecessary_work_when_drop_cannot_possibly_succeed.mdwn b/doc/bugs/unnecessary_work_when_drop_cannot_possibly_succeed.mdwn deleted file mode 100644 index d8d5f49cc7..0000000000 --- a/doc/bugs/unnecessary_work_when_drop_cannot_possibly_succeed.mdwn +++ /dev/null @@ -1,30 +0,0 @@ -When annex.numcopies is 4, and there are only 4 copies of a file, -`git-annex drop` behaves like this: - - drop me (locking a2...) (locking a3...) (locking a4...) (unsafe) - Could only verify the existence of 3 out of 4 necessary copies - -That is unnecessary work, because the drop cannot possibly succeed when -there are not more than numcopies. It should be able to skip locking the -content on remotes and immediately fail. - -A user reported a situation where this happened to a lot of files and -caused rate limiting and slowed down git-annex sync. --[[Joey]] - -> I started to implement making `git-annex drop` check if it could -> possibly succeed before starting locking content, and found that -> it was complicating it with a lot of fairly redundant code. -> -> And since this command is run usually by the user when they really -> want to drop something, not in a more general situation like git-annex -> sync is, it seems ok for it to do a little bit of extra work, -> showing what remotes it's able to lock, before failing, even though it -> was always going to fail in the end. And if the user wants it to avoid -> that, they can use `--auto`, which already avoids doing anything when it -> cannot succeed. -> -> So, I've decided to leave git-annex drop as it is. But what about -> git-annex sync? I tried it, and when there are not enough copies it does -> not appear to try to drop at all. -> -> [[done]] --[[Joey]] diff --git a/doc/bugs/unusable_with_ghc_8.10.7_on_DragonFlyBSD.mdwn b/doc/bugs/unusable_with_ghc_8.10.7_on_DragonFlyBSD.mdwn deleted file mode 100644 index 7059e00c06..0000000000 --- a/doc/bugs/unusable_with_ghc_8.10.7_on_DragonFlyBSD.mdwn +++ /dev/null @@ -1,46 +0,0 @@ -On DragonflyBSD with git-annex built with ghc 8.10.7, a simple `git annex add` puts strange entries in the index, as shown: - -``` -copter-dfly tmp $ mkdir d -copter-dfly tmp $ cd d -copter-dfly d $ git init -Initialized empty Git repository in /home/falsifian/tmp/d/.git/ -copter-dfly d $ git annex init -init ok -(recording state in git...) -copter-dfly d $ echo test>f -copter-dfly d $ git annex add f -add f -ok -(recording state in git...) -copter-dfly d $ git status -On branch master - -No commits yet - -Changes to be committed: - (use "git rm --cached ..." to unstage) - new file: 30d/f3a/SHA256-s5--f2ca1bb6c7e907d06dafe4687e579fce76b37e4e93b7605022da52e6ccc26fd2.log - new file: f - new file: uuid.log - -Changes not staged for commit: - (use "git add/rm ..." to update what will be committed) - (use "git restore ..." to discard changes in working directory) - deleted: 30d/f3a/SHA256-s5--f2ca1bb6c7e907d06dafe4687e579fce76b37e4e93b7605022da52e6ccc26fd2.log - deleted: uuid.log - -copter-dfly d $ -``` - -Digging into it further with dfly people over IRC, it's been discovered: - -* The bug does not appear with ghc 8.10.5, with git-annex 20210903 or 20210223. -* But it consistently happens with ghc 8.10.7, with all the versions I tried. -* I also noticed that `git annex init` does not create the file `.git/annex/index` when the bug is present. - -Has anyone tried git-annex with ghc 8.10.7 on a different OS? - -I tried to figure out why `.git/annex/index` was not being created, but I had some trouble figuring out which line of code is supposed to create it. If someone could point me there I could do some further debugging to see why it isn't created. - -> [[done]] --[[Joey]] diff --git a/doc/bugs/unusable_with_ghc_8.10.7_on_DragonFlyBSD/comment_1_70b807c69be263fd53c578fe6ec84561._comment b/doc/bugs/unusable_with_ghc_8.10.7_on_DragonFlyBSD/comment_1_70b807c69be263fd53c578fe6ec84561._comment deleted file mode 100644 index 2481394326..0000000000 --- a/doc/bugs/unusable_with_ghc_8.10.7_on_DragonFlyBSD/comment_1_70b807c69be263fd53c578fe6ec84561._comment +++ /dev/null @@ -1,26 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2021-12-31T15:05:40Z" - content=""" -It seems that somehow it is failing to set environment variables. Eg, -`GIT_INDEX_FILE` is what makes git use `.git/annex/index` and if it was not -set it would behave as you describe. - -This strongly suggests that there is a bug in the compiler/libraries. - -Here is a small test program: - - import System.Environment - import System.Process - - main = do - let p = (proc "sh" ["-c", "echo testing: $TEST"]) - let p' = p { env = Just [("TEST", "foo")] } - withCreateProcess p' $ \_ _ _ pid -> waitForProcess pid - setEnv "TEST" "bar" - withCreateProcess p $ \_ _ _ pid -> waitForProcess pid - -If run with "runghc foo.hs", that should output "testing: foo" and then -"testing: bar". -"""]] diff --git a/doc/bugs/unusable_with_ghc_8.10.7_on_DragonFlyBSD/comment_2_eeff3cb51df849b783446e238de002a8._comment b/doc/bugs/unusable_with_ghc_8.10.7_on_DragonFlyBSD/comment_2_eeff3cb51df849b783446e238de002a8._comment deleted file mode 100644 index da0d68bd57..0000000000 --- a/doc/bugs/unusable_with_ghc_8.10.7_on_DragonFlyBSD/comment_2_eeff3cb51df849b783446e238de002a8._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2021-12-31T15:54:25Z" - content=""" -On Linux this problem does not occur. - -I built git-annex with ghc-8.10.7 by changing the -stack.yaml to use "resolver: lts-18.20" and running -"stack build". -"""]] diff --git a/doc/bugs/unusable_with_ghc_8.10.7_on_DragonFlyBSD/comment_3_641473e43c8da7e7be02658cf8d3bd3e._comment b/doc/bugs/unusable_with_ghc_8.10.7_on_DragonFlyBSD/comment_3_641473e43c8da7e7be02658cf8d3bd3e._comment deleted file mode 100644 index 0e689741e4..0000000000 --- a/doc/bugs/unusable_with_ghc_8.10.7_on_DragonFlyBSD/comment_3_641473e43c8da7e7be02658cf8d3bd3e._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="falsifian" - avatar="http://cdn.libravatar.org/avatar/59c3c23c500d20d83ecb9d1f149be9ae" - subject="comment 3" - date="2021-12-31T16:03:33Z" - content=""" -Looks like you are right. Thanks for the help! On DragonFlyBSD with ghc 8.10.7 I see: - -``` -copter-dfly tmp $ runghc foo.hs -testing: -testing: bar -``` - -I guess it is time to look at the `System.Process` module. -"""]] diff --git a/doc/bugs/unusable_with_ghc_8.10.7_on_DragonFlyBSD/comment_4_59e2ea4d59d5f4a8472cc8c945b8a419._comment b/doc/bugs/unusable_with_ghc_8.10.7_on_DragonFlyBSD/comment_4_59e2ea4d59d5f4a8472cc8c945b8a419._comment deleted file mode 100644 index 75d4e2c10c..0000000000 --- a/doc/bugs/unusable_with_ghc_8.10.7_on_DragonFlyBSD/comment_4_59e2ea4d59d5f4a8472cc8c945b8a419._comment +++ /dev/null @@ -1,7 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2021-12-31T19:00:10Z" - content=""" -Upstream bug report: -"""]] diff --git a/doc/bugs/unusable_with_ghc_8.10.7_on_DragonFlyBSD/comment_5_27a017a942f805e9b0216e77d4890c86._comment b/doc/bugs/unusable_with_ghc_8.10.7_on_DragonFlyBSD/comment_5_27a017a942f805e9b0216e77d4890c86._comment deleted file mode 100644 index 3b9278918a..0000000000 --- a/doc/bugs/unusable_with_ghc_8.10.7_on_DragonFlyBSD/comment_5_27a017a942f805e9b0216e77d4890c86._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="falsifian" - avatar="http://cdn.libravatar.org/avatar/59c3c23c500d20d83ecb9d1f149be9ae" - subject="comment 5" - date="2022-01-03T19:06:11Z" - content=""" -This turned out to a bug in DragonFlyBSD. It is [now fixed](https://gitweb.dragonflybsd.org/dragonfly.git/commit/8202da125af7f265528f7844bdc98d544a320bd1). Thanks for your help finding the problem! -"""]] diff --git a/doc/bugs/use_reflink_to_export_to_directory_special_remote.mdwn b/doc/bugs/use_reflink_to_export_to_directory_special_remote.mdwn deleted file mode 100644 index e716fceb22..0000000000 --- a/doc/bugs/use_reflink_to_export_to_directory_special_remote.mdwn +++ /dev/null @@ -1,44 +0,0 @@ -I'm using version `10.20230321-gb624394c7` on Arch Linux and BTRFS. -I'd love to have the directory special remote using `cp --reflink=auto` to export to a directory on the same file system. - -My use case is using syncthing as a way to export (parts of my repo) to my phone. -I'm deliberately not using `adb` as this needs my phone to be connected to the computer somehow. -I basically want to do what's described on [syncthing special remote](/todo/syncthing_special_remote/#index2h2) under “Copying objects”. - -The main drawback, as also noted in the linked document, is that all files are duplicated. -As I'm using BTRFS it would really help if the `export` were to use `cp --reflink=auto` instead of its own copying mechanism. -I've read [here](/forum/Use_reflinks_on_BTRFS_instead_of_symlinks___63__/) that you suggested (albeit 7 years ago) using a shared clone instead. -However, this does not work for me as syncthing does not synchronize symlinks to Android (they are ignored even if they point to something). -(Trying with a shared repo and `adjust --unlock-present` gives merge problems when dropping files from the directory.) - -Nearly two years ago, you wrote [here](/todo/import_from_directory_does_not_use_cp_--reflink__63___/) that you “Implemented CoW for directory special remote, comprehensively”. -It seems to me that this is only true for importing files which, as I checked, actually uses cow. - -Since I want to manage relatively many files of decent size (music, audio books, pictures, videos / movies) it would be much better if the tree export would also use reflink. -Maybe having this as an option, for the cost of not having a nice progress bar, would be something worth considering. - -A test script to show that an export does indeed use cow is the following: - -```bash -mkdir export import -git init repo -cd repo -git annex init -git annex initremote import type=directory directory=../import encryption=none importtree=yes -git annex initremote export type=directory directory=../export encryption=none exporttree=yes -git config remote.import.annex-tracking-branch main -git config remote.export.annex-tracking-branch main - -for i in {1..100}; do - dd if=/dev/urandom bs=1M count=10 of=../import/file$i -done - -git annex sync --content -``` -After running, a `btrfs filesystem du ...` tells me that the `repo` and `import` files point to the same file, whereas `export` does not. - ---- - -In general: Thank you so much for this wonderful piece of software, I'm using it for years now and manage virtually everything with it (audio, video, pictures, important documents, papers, …). - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/v8_repo_auto_upgrades_to_v10.mdwn b/doc/bugs/v8_repo_auto_upgrades_to_v10.mdwn deleted file mode 100644 index c9f294ed92..0000000000 --- a/doc/bugs/v8_repo_auto_upgrades_to_v10.mdwn +++ /dev/null @@ -1,19 +0,0 @@ -### Please describe the problem. - -v8 repo automatically upgrades to v10 which is contrary to what the changelog states here [[news/version_10.20220822]]. - -### What steps will reproduce the problem? - -``` -git init -git annex init --version 8 -git config annex.version -git annex info > /dev/null -git config annex.version -``` - -### What version of git-annex are you using? On what operating system? - -10.20220822 Linux - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/v8_repo_auto_upgrades_to_v10/comment_1_446aa74f0b54918dc8737483e3e8564a._comment b/doc/bugs/v8_repo_auto_upgrades_to_v10/comment_1_446aa74f0b54918dc8737483e3e8564a._comment deleted file mode 100644 index c9f786c0bd..0000000000 --- a/doc/bugs/v8_repo_auto_upgrades_to_v10/comment_1_446aa74f0b54918dc8737483e3e8564a._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-09-26T16:15:41Z" - content=""" -This was broken in [[!commit 0d2e3058ee01d55dc3b929ddf8e0573a95a2ca85]] -which was included in version 10.20220822. - -I've reverted the commit as a first step. This means repos initialized -at v9 will never autoupgrade to v10, which will need to be fixed -somehow. (Update: Fixed that now.) - -Gonna need to make a git-annex release ASAP to get this fix out there. -"""]] diff --git a/doc/bugs/view_branch_on_top_of_adjusted_branch_doesn__39__t_sync.mdwn b/doc/bugs/view_branch_on_top_of_adjusted_branch_doesn__39__t_sync.mdwn deleted file mode 100644 index bda167a806..0000000000 --- a/doc/bugs/view_branch_on_top_of_adjusted_branch_doesn__39__t_sync.mdwn +++ /dev/null @@ -1,149 +0,0 @@ -### Please describe the problem. - -Release 10.20230214 was supposed to give the ability to refresh the view branch with `sync` to -reflect any changes made to metadata or parent branch, yet for users of crippled filesystems -(for whom the view branches are a bit marginal anyway considering that symbolic or hard links -are not employed) who are dependent on adjusted branches this does not work. It seems git annex -tries to reach for the parent branch in the form of `adjusted/master` but normally it should be -similar to `adjusted/master(unlocked)` with the parenthesized portion at the end. - -[[!format sh """ -X:\git-annex-tests\annex-view-test-a [views/adjusted/master(unlocked)(subdir_=_)]> git status -On branch views/adjusted/master(unlocked)(subdir_=_) -nothing to commit, working tree clean -X:\git-annex-tests\annex-view-test-a [views/adjusted/master(unlocked)(subdir_=_)]> git annex sync --no-commit -fatal: not a valid object name: 'adjusted/master' -git-annex: failed to update refs/heads/synced/adjusted/master -"""]] - -### What steps will reproduce the problem? - -Use a crippled filesystem on Windows with adjusted unlocked branches and enter a view branch -and then try to `git annex sync --no-commit` -- the sync at least partially fails. - -### What version of git-annex are you using? On what operating system? - -[[!format sh """ -git-annex version: 10.20230214-gf3019d7e2 -build flags: Assistant Webapp Pairing TorrentParser Benchmark Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.7 http-client-0.7.9 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.1.2 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: mingw32 x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 2 3 4 5 6 7 8 9 10 -local repository version: 10 -"""]] - -Windows 10 version 22H2 (build 19045.2486), 64 bit. - -### Please provide any additional information below. - -[[!format sh """ -X:\> cd git-annex-tests -X:\git-annex-tests> git init annex-view-test-a -X:\git-annex-tests> cd .\annex-view-test-a\ -X:\git-annex-tests\annex-view-test-a [master]> git config annex.thin true -X:\git-annex-tests\annex-view-test-a [master]> git annex init Annex-view-test-a -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked)]> echo Aaa > a -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked) +1 ~0 -0 !]> mkdir subdir -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked) +1 ~0 -0 !]> echo Bbb > subdir\b -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked) +2 ~0 -0 !]> echo Ccc > subdir\c -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked) +2 ~0 -0 !]> mkdir subdir\subsubdir -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked) +2 ~0 -0 !]> echo Ddd > .\subdir\subsubdir\d -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked) +2 ~0 -0 !]> git annex add . -add a -ok -add subdir/b -ok -add subdir/c -ok -add subdir/subsubdir/d -ok -(recording state in git...) -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked) +4 ~0 -0 ~]> ls -r - - Directory: X:\git-annex-tests\annex-view-test-a - -Mode LastWriteTime Length Name ----- ------------- ------ ---- -d---- 15.2.2023 12:37 subdir --a--- 15.2.2023 12:31 5 a - - Directory: X:\git-annex-tests\annex-view-test-a\subdir - -Mode LastWriteTime Length Name ----- ------------- ------ ---- -d---- 15.2.2023 12:38 subsubdir --a--- 15.2.2023 12:34 5 b --a--- 15.2.2023 12:37 5 c - - Directory: X:\git-annex-tests\annex-view-test-a\subdir\subsubdir - -Mode LastWriteTime Length Name ----- ------------- ------ ---- --a--- 15.2.2023 12:38 5 d - -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked) +4 ~0 -0 ~]> fsutil hardlink list a -\git-annex-tests\annex-view-test-a\.git\annex\objects\b29\a20\SHA256E-s5--e3c555b2205407e76d3a460513cf44c1c62e48ba83ec4948ed05d0f87b957051\SHA256E-s5--e3c555b2205407e76d3a460513cf44c1c62e48ba83ec4948ed05d0f87b957051 -\git-annex-tests\annex-view-test-a\a -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked) +4 ~0 -0 ~]> git commit -m 'Add test files' -[adjusted/master(unlocked) 56b11c9] Add test files - 4 files changed, 4 insertions(+) - create mode 100644 a - create mode 100644 subdir/b - create mode 100644 subdir/c - create mode 100644 subdir/subsubdir/d -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked)]> git annex metadata -t test subdir\b -metadata subdir/b - lastchanged=2023-02-15@10-52-31 - tag=test - tag-lastchanged=2023-02-15@10-52-31 -ok -(recording state in git...) -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked)]> git annex metadata -t test subdir\subsubdir\d -metadata subdir/subsubdir/d - lastchanged=2023-02-15@10-53-00 - tag=test - tag-lastchanged=2023-02-15@10-53-00 -ok -(recording state in git...) -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked)]> git annex metadata subdir\c -metadata subdir/c - -ok -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked)]> git annex view 'subdir\=*' -view (searching...) -Switched to branch 'views/adjusted/master(unlocked)(subdir_=_)' -ok -X:\git-annex-tests\annex-view-test-a [views/adjusted/master(unlocked)(subdir_=_)]> ls -r - - Directory: X:\git-annex-tests\annex-view-test-a - -Mode LastWriteTime Length Name ----- ------------- ------ ---- -d---- 15.2.2023 12:55 subsubdir --a--- 15.2.2023 12:55 178 b_%subdir% - - Directory: X:\git-annex-tests\annex-view-test-a\subsubdir - -Mode LastWriteTime Length Name ----- ------------- ------ ---- --a--- 15.2.2023 12:55 181 d_%subdir%subsubdir% - -X:\git-annex-tests\annex-view-test-a [views/adjusted/master(unlocked)(subdir_=_)]> git annex sync --no-commit -fatal: not a valid object name: 'adjusted/master' -git-annex: failed to update refs/heads/synced/adjusted/master - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Git Annex is great. I use it several times a week with my multigigabyte backups, where it gives structure to my image-based backup routines, so you could say I'm a believer. :) - -Big thanks, Joey! - -[[!meta author=jkniiv]] - -> Closing as a dup with a todo item;[[done]] --[[Joey]] diff --git a/doc/bugs/view_branch_on_top_of_adjusted_branch_doesn__39__t_sync/comment_1_3c0be3ff163aae411f323e3445af94b1._comment b/doc/bugs/view_branch_on_top_of_adjusted_branch_doesn__39__t_sync/comment_1_3c0be3ff163aae411f323e3445af94b1._comment deleted file mode 100644 index 2fdf2b2ea6..0000000000 --- a/doc/bugs/view_branch_on_top_of_adjusted_branch_doesn__39__t_sync/comment_1_3c0be3ff163aae411f323e3445af94b1._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-02-16T18:17:33Z" - content=""" -This is kind of a duplicate of[[todo/unify_adjust_with_view]]. - -I think it's best to keep it as a todo item, not a bug, since this is a -desired new behavior. -"""]] diff --git a/doc/bugs/views_do_use_utf8_for_chars_that_does_not_need_to.mdwn b/doc/bugs/views_do_use_utf8_for_chars_that_does_not_need_to.mdwn deleted file mode 100644 index 815d5f569b..0000000000 --- a/doc/bugs/views_do_use_utf8_for_chars_that_does_not_need_to.mdwn +++ /dev/null @@ -1,37 +0,0 @@ -### Please describe the problem. -My whole setup on all of my systems is with iso-8859-1. There is no use of UFT-8 (simply as my brain is not compatible to it). - -I have metadata with `:` in it. That works perfect and ad it is a low char 0x3a, it should be displayed correctly. Unfortunately, when I create a view, that char gets displayed as `ï¹?`, which is fully unlogical. - -### What steps will reproduce the problem? -* Create metadata with : in it -* create a view with that metadata - -### What version of git-annex are you using? On what operating system? -10.20230126, Devuan - -### Please provide any additional information below. - -[[!format sh """ -~> locale -LANG=de_DE -LANGUAGE=de_DE:de:en -LC_CTYPE=de_DE -LC_NUMERIC=de_DE -LC_TIME=de_DE -LC_COLLATE=de_DE -LC_MONETARY=de_CH -LC_MESSAGES=de_DE -LC_PAPER=de_DE -LC_NAME=de_DE -LC_ADDRESS=de_CH -LC_TELEPHONE=de_DE -LC_MEASUREMENT=de_DE -LC_IDENTIFICATION=de_DE -LC_ALL= -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) -yes, that worked in the past (I think) - -> [[wontfix|done]] --[[Joey]] diff --git a/doc/bugs/views_do_use_utf8_for_chars_that_does_not_need_to/comment_1_a6014d95b0ac0dc20e9495a717d6d615._comment b/doc/bugs/views_do_use_utf8_for_chars_that_does_not_need_to/comment_1_a6014d95b0ac0dc20e9495a717d6d615._comment deleted file mode 100644 index f025d1b2b7..0000000000 --- a/doc/bugs/views_do_use_utf8_for_chars_that_does_not_need_to/comment_1_a6014d95b0ac0dc20e9495a717d6d615._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-05-23T15:50:51Z" - content=""" -The character is actually `﹕` - -That is used in views because on windows colon is a special character -and putting it in the name of a viewed file would prevent checking out the -view. - -It would be possible to skip that on linux, but note that it also -escapes `/` on linux with a unicode equivilant (for similar reasons). - -Also, the linux executable can sometimes be run on a windows system through -WSL. I'm not sure how a `:` in a filename would be handled in that situation. -"""]] diff --git a/doc/bugs/views_do_use_utf8_for_chars_that_does_not_need_to/comment_2_323f1be80e2422e73ebd99b702a28e80._comment b/doc/bugs/views_do_use_utf8_for_chars_that_does_not_need_to/comment_2_323f1be80e2422e73ebd99b702a28e80._comment deleted file mode 100644 index 345414088e..0000000000 --- a/doc/bugs/views_do_use_utf8_for_chars_that_does_not_need_to/comment_2_323f1be80e2422e73ebd99b702a28e80._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="Mowgli" - avatar="http://cdn.libravatar.org/avatar/17ab194dddf7b7da59ec039cbb3ac252" - subject="Use locales for that porpose" - date="2023-05-29T22:42:13Z" - content=""" -That is what locales are for. - -Why not use them just correctly? It is just wrong to use utf8 characters in non-utf8 environments. - -I think, the correct handling would be to stay with the user choice (locales and the raw text, a user used for the metadata) and mangle that chars just on such broken systems like windows. -"""]] diff --git a/doc/bugs/views_do_use_utf8_for_chars_that_does_not_need_to/comment_3_576c31b2cc5031cfb64ff58a1866b3b5._comment b/doc/bugs/views_do_use_utf8_for_chars_that_does_not_need_to/comment_3_576c31b2cc5031cfb64ff58a1866b3b5._comment deleted file mode 100644 index c522b41dad..0000000000 --- a/doc/bugs/views_do_use_utf8_for_chars_that_does_not_need_to/comment_3_576c31b2cc5031cfb64ff58a1866b3b5._comment +++ /dev/null @@ -1,34 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2023-06-05T18:10:23Z" - content=""" -> That is what locales are for. -> -> Why not use them just correctly? It is just wrong to use utf8 characters in non-utf8 environments. - -It's perfectly find to use unicode in filenames at any time. Files have the -name they have no matter how you configure your locale. - -If git renamed unicode files when cloning a repository, just because the -current locale did not support unicode, it would be broken. - -If git-annex metadata contains unicode and you enter a view, git-annex is -operating acceptably when it preserves that unicode in the viewed filename. - -Maybe git-annex could try to transliterate unicode in viewed filenames -in some way to work better non-unicode locales. But the locale can change. -And git-annex needs to be able to reverse view filenames back to the -filename used on the viewed branch. So it's not practical to vary the view -filenames to fit the locale, because that would prevent that reversing -from working unless it had a way to determine that locale that was in use -when the view was generated. - -git-annex has to replace the `/` character with *something* when generating -a viewed file from metadata that contains that character. It used to -use `%`, since that at least contains a slash, but I didn't think that was -very readable. The unicode slash character it uses is very readable for the -vast percentage of users who are not stuck with 1980's era displays. - -Sorry, it's simply a tradeoff between you and everyone else. -"""]] diff --git a/doc/bugs/views_do_use_utf8_for_chars_that_does_not_need_to/comment_4_4097e8dde9d9fe29b540112f7400689a._comment b/doc/bugs/views_do_use_utf8_for_chars_that_does_not_need_to/comment_4_4097e8dde9d9fe29b540112f7400689a._comment deleted file mode 100644 index c8547ce29e..0000000000 --- a/doc/bugs/views_do_use_utf8_for_chars_that_does_not_need_to/comment_4_4097e8dde9d9fe29b540112f7400689a._comment +++ /dev/null @@ -1,20 +0,0 @@ -[[!comment format=mdwn - username="Mowgli" - avatar="http://cdn.libravatar.org/avatar/17ab194dddf7b7da59ec039cbb3ac252" - subject="comment igendwas" - date="2023-06-05T20:33:42Z" - content=""" -> If git-annex metadata contains unicode and you enter a view, git-annex is operating acceptably when it preserves that unicode in the viewed filename. - -But in this case, the metadata does NOT contain unicode/utf-8/whatever exotic charset. It contains ASCII, plain ASCII and git-annex is doing something with it that is not expetable. - -> Maybe git-annex could try to transliterate unicode in viewed filenames in some way to work better non-unicode locales. But the locale can change. And git-annex needs to be able to reverse view filenames back to the filename used on the viewed branch. So it's not practical to vary the view filenames to fit the locale, because that would prevent that reversing from working unless it had a way to determine that locale that was in use when the view was generated. - -It gets even more strange. I tried to rename the files, replacing the strange chars with just a `:`. Git was happy with it, my eyes was happy with it, my filesystem was happy with it but git-annex added now a second tag, additional to the content `:`, that is already there, it created the same key with a value of `ï¹?`. So it is even inconsistent in that case of renaming. - -> git-annex has to replace the / character with something when generating a viewed file from metadata that contains that character. It used to use %, since that at least contains a slash, but I didn't think that was very readable. The unicode slash character it uses is very readable for the vast percentage of users who are not stuck with 1980's era displays. - -Another opportunity is to abort and tell the user that this char is not allowed in current filesystem. Or give the user a opportunity to replace it with his own choice. - -But I did not speak of the slash, I spoke about a double point (`:`). That is a fully legal character in mature filesystems (the most). With that windows filesystems, you mentioned, you have even other troubles that is more important like symlinks... -"""]] diff --git a/doc/bugs/views_do_use_utf8_for_chars_that_does_not_need_to/comment_5_5e5441cca2f3bbc252029aca358e9a47._comment b/doc/bugs/views_do_use_utf8_for_chars_that_does_not_need_to/comment_5_5e5441cca2f3bbc252029aca358e9a47._comment deleted file mode 100644 index 39fd026c01..0000000000 --- a/doc/bugs/views_do_use_utf8_for_chars_that_does_not_need_to/comment_5_5e5441cca2f3bbc252029aca358e9a47._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="Mowgli" - avatar="http://cdn.libravatar.org/avatar/17ab194dddf7b7da59ec039cbb3ac252" - subject="comment 5" - date="2023-06-05T20:35:19Z" - content=""" -> Sorry, it's simply a tradeoff between you and everyone else. - -Well, the majority would use git-annex on proper filesystems. It is only a minority that still uses such broken filesystems that do not allow a double point. -"""]] diff --git a/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_.mdwn b/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_.mdwn deleted file mode 100644 index ec55838884..0000000000 --- a/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_.mdwn +++ /dev/null @@ -1,41 +0,0 @@ -### Please describe the problem. -I have a git-annex repo as a submodule. When adding a file using git-annex, I get the error message: - -warning: unable to convert submodule to form that will work with git-annex - -This I seem to get for every file, i.e. thousands of error messages sometimes. But the repo in the submodule seems to work fine, -the .git symlink is there. - -I am not able to spot the problem in Annex/Fixup.hs, maybe you have an idea what could go wrong? - -### What steps will reproduce the problem? - -mike@leguin experiments % date > test2 -mike@leguin experiments % git annex add test2 -add test2 (non-large file; adding content to git repository) ok -(recording state in git...) -warning: unable to convert submodule to form that will work with git-annex -mike@leguin experiments % ls -l ../.git -lrwxrwxrwx. 1 mike mike 53 Nov 16 17:31 ../.git -> ../../../.git/modules/experiments/ocr-evaluation/data - -### What version of git-annex are you using? On what operating system? -git-annex version: 8.20201127 -Fedora 32 - -Also the same on CentOS 7, I was just able to reproduce it on my F32 workstation. ext4 filesystem. - -### Please provide any additional information below. - -[[!format sh """ -# If you can, paste a complete transcript of the problem occurring here. -# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log - - -# End of transcript or log. -"""]] - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) -Besides this problem, I am super happy with git-annex and use it at work and at home daily :-) - -> [[fixed|done]] although w/o a way to reproduce it I can't be 100% sure -> --[[Joey]] diff --git a/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_2_1d3214382c2bb604101a835c0bb7f146._comment b/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_2_1d3214382c2bb604101a835c0bb7f146._comment deleted file mode 100644 index 6262203742..0000000000 --- a/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_2_1d3214382c2bb604101a835c0bb7f146._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="mike@2d6d71f56ce2a992244350475251df87c26fe351" - nickname="mike" - avatar="http://cdn.libravatar.org/avatar/183fa439752e2f0c6f39ede658d81050" - subject="comment 2" - date="2021-01-19T12:21:07Z" - content=""" - -In the submodule: - -``` -% git rev-parse --git-dir -.git -``` - -If I understand needsSubmoduleFixup correctly, this should not need fixing anymore, is that correct? - -"""]] diff --git a/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_2_7f4800ee68c98606bc8a3ce959646014._comment b/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_2_7f4800ee68c98606bc8a3ce959646014._comment deleted file mode 100644 index 723169a9b4..0000000000 --- a/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_2_7f4800ee68c98606bc8a3ce959646014._comment +++ /dev/null @@ -1,20 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2021-01-19T16:14:38Z" - content=""" -Hmm, that happens if it somehow gets an exception while fixing up the .git -file and unsetting core.worktree in the git config. I've modified the code -so it now will display the exception as part of that message. - -If by "get for every file" you mean the same run of git-annex is displaying -it repeatedly, that would be weird -- it's run in early startup not later. - -One way I can see this failing is if core.worktree is not actually set. -Then unsetting it would fail and it would display this message. But it -would not even try to unset core.worktree unless something tells -git-annex that the git directory is located under the parent repo's -.git/modules/ Despite the `git rev-parse` you show, something is -making git-annex still think that is the case. Likely because .git -is not a symlink. -"""]] diff --git a/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_3_54c9716cd5cab7b196549af2f90bed0a._comment b/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_3_54c9716cd5cab7b196549af2f90bed0a._comment deleted file mode 100644 index 7097de329d..0000000000 --- a/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_3_54c9716cd5cab7b196549af2f90bed0a._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="mike@2d6d71f56ce2a992244350475251df87c26fe351" - nickname="mike" - avatar="http://cdn.libravatar.org/avatar/183fa439752e2f0c6f39ede658d81050" - subject="comment 3" - date="2021-01-19T18:16:27Z" - content=""" -> I've modified the code so it now will display the exception as part of that message. - -I'll try to compile from source and get the exact exception then! I'll also check if I can come up with a minimal repo that reproduces the problem. - -> If by \"get for every file\" you mean the same run of git-annex is displaying it repeatedly, that would be weird -- it's run in early startup not later. - -I do get thousands of those error messages and on the occasion that I noticed this problem I was checking in thousands of files, and I also get the message once(!) when I `git annex add` a single test file, so this only my suspicion. - -> One way I can see this failing is if core.worktree is not actually set. Then unsetting it would fail and it would display this message. But it would not even try to unset core.worktree unless something tells git-annex that the git directory is located under the parent repo's .git/modules/ Despite the git rev-parse you show, something is making git-annex still think that is the case. Likely because .git is not a symlink. - -See above, it's a symlink to the directory under the parent repo's ./git/modules. Is this how it is intended? -"""]] diff --git a/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_4_864988b3ebc11ea9e92c55ee765d6d5c._comment b/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_4_864988b3ebc11ea9e92c55ee765d6d5c._comment deleted file mode 100644 index 3f403db0dd..0000000000 --- a/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_4_864988b3ebc11ea9e92c55ee765d6d5c._comment +++ /dev/null @@ -1,27 +0,0 @@ -[[!comment format=mdwn - username="mike@2d6d71f56ce2a992244350475251df87c26fe351" - nickname="mike" - avatar="http://cdn.libravatar.org/avatar/183fa439752e2f0c6f39ede658d81050" - subject="comment 4" - date="2021-01-20T15:41:50Z" - content=""" -For some reason, I couldn't reproduce it easily using `git annex add` on my F32 workstation, but I got it on a `git annex get`: - -``` -% git annex get model_strukturerkennung.h5 -get model_strukturerkennung.h5 (from qurator-box...) -(checksum...) ok -(recording state in git...) -warning: unable to convert submodule to form that will work with git-annex: unset core.worktree failed -CallStack (from HasCallStack): - error, called at ./Annex/Fixup.hs:119:24 in main:Annex.Fixup -% ls -ld ../.git -lrwxrwxrwx. 1 mike mike 53 Nov 16 17:31 ../.git -> ../../../.git/modules/experiments/ocr-evaluation/data - - -% git config --unset core.worktree; echo $? -5 -% git config --get core.worktree; echo $? -1 -``` -"""]] diff --git a/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_5_06d31ed5a8c187cec7bc90339550b5e8._comment b/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_5_06d31ed5a8c187cec7bc90339550b5e8._comment deleted file mode 100644 index 8655215084..0000000000 --- a/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_5_06d31ed5a8c187cec7bc90339550b5e8._comment +++ /dev/null @@ -1,24 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2021-01-21T18:14:09Z" - content=""" -Thanks Mike, that confirms it's failing to unset the config that's not set. - -The .git symlink is what git-annex is supposed to set up, so it should not -need to do anything else and the question is, why does it still think it -needs to change something? - -I was able to reproduce the warning, just run `git submodule add foo -/path/to/annexrepo` and then in foo, `git-annex init` and then it -happens part-way through. Due to two different git-annex processes both -trying to do the same conversion, with the second one failing. -I've fixed that. - -My fix will also fix your problem. But, I don't actually -understand how to reproduce your problem. I was only able to make it happen -when running `git-annex init` (or letting it auto-init), -not running other git-annex commands in a repo it's already initialized. - -Maybe you have `GIT_DIR` set to "../../../.git/modules/..."? -"""]] diff --git a/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_6_5e7007182372d28ceb11a8692e2cf801._comment b/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_6_5e7007182372d28ceb11a8692e2cf801._comment deleted file mode 100644 index 1b434ce221..0000000000 --- a/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_6_5e7007182372d28ceb11a8692e2cf801._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="mike@2d6d71f56ce2a992244350475251df87c26fe351" - nickname="mike" - avatar="http://cdn.libravatar.org/avatar/183fa439752e2f0c6f39ede658d81050" - subject="comment 6" - date="2021-01-22T11:08:38Z" - content=""" -First a side note about the thousands of error messages: This seems to be happening when I change thousands of files and a subsequent `git annex add` runs a `git ls-files` which in turn runs `git-annex smudge` for every file, which probably each outputs the error message. -"""]] diff --git a/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_7_4de04de71792801f50256214f4b5fe9f._comment b/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_7_4de04de71792801f50256214f4b5fe9f._comment deleted file mode 100644 index eed218c7b9..0000000000 --- a/doc/bugs/warning__58___unable_to_convert_submodule_to_form_that_/comment_7_4de04de71792801f50256214f4b5fe9f._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 7""" - date="2021-01-22T18:33:01Z" - content=""" -Makes sense that `git-annex smudge` being called repeatedly would output -the message like that. And unfortunately, yes, when a file is unlocked and -gets modified, git ls-files --modified does need to smudge it. Thanks -for clearing that part up. -"""]] diff --git a/doc/bugs/windows__58___needlessly_reminded_to_run___96__restage__96__.mdwn b/doc/bugs/windows__58___needlessly_reminded_to_run___96__restage__96__.mdwn deleted file mode 100644 index 4b4c10d4d4..0000000000 --- a/doc/bugs/windows__58___needlessly_reminded_to_run___96__restage__96__.mdwn +++ /dev/null @@ -1,182 +0,0 @@ -### Please describe the problem. - -After release 10.20220927 any get or drop command that changes availability of content causes git-annex -to issue the following notice: - -``` - git status will show some files to be modified, since content availability has changed and git-annex was unable to update the index. This is only a cosmetic problem affecting git status; git add, git commit, etc won't be affected. To fix the git status display, you can run: git-annex restage -``` - -Yet git status is clean and no `git-annex restage` is needed. - -### What steps will reproduce the problem? - -Use a crippled filesystem on Windows with adjusted unlocked branches and try to get content from a remote or drop content locally. - -### What version of git-annex are you using? On what operating system? - -[[!format sh """ -git-annex version: 10.20220927-g26dea5641 -build flags: Assistant Webapp Pairing TorrentParser Benchmark Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.7 http-client-0.7.9 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.1.2 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: mingw32 x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 2 3 4 5 6 7 8 9 10 -"""]] - -The above is for all practical purposes the released version 10.20220927, just a few commits before the release with no code changes. - -Windows 10 version 21H2 (build 19044.2006), 64 bit. - -### Please provide any additional information below. - -[[!format sh """ -C:\Users\jkniiv> k: -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B\arkistoidut [adjusted/master(unlocked)]> cd .. -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> duf . -╭─────────────────────────────────────────────────────────────────────╮ -│ 1 local device │ -├────────────┬──────┬──────┬───────┬────────┬──────┬──────────────────┤ -│ MOUNTED ON │ SIZE │ USED │ AVAIL │ USE% │ TYPE │ FILESYSTEM │ -├────────────┼──────┼──────┼───────┼────────┼──────┼──────────────────┤ -│ K:\ │ 3.6T │ 3.6T │ 34.9G │ 99.1% │ NTFS │ Jibun.Tila2109kc │ -╰────────────┴──────┴──────┴───────┴────────┴──────┴──────────────────╯ -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> git annex-sync --content -merge synced/master (Merging into master...) -Updating 6d4e731..05224ad -Fast-forward - Jarkon ThinkPad T450s (Win10 v21H1) . B/79698DAC29A079D3-06-06.mrimg | 1 + - 1 file changed, 1 insertion(+) - create mode 120000 Jarkon ThinkPad T450s (Win10 v21H1) . B/79698DAC29A079D3-06-06.mrimg -(Merging into adjusted branch...) -Updating 51cb822..f175b90 -Fast-forward - Jarkon ThinkPad T450s (Win10 v21H1) . B/79698DAC29A079D3-06-06.mrimg | 1 + - 1 file changed, 1 insertion(+) - create mode 100644 Jarkon ThinkPad T450s (Win10 v21H1) . B/79698DAC29A079D3-06-06.mrimg -ok -pull origin -From G:\Reflect-varmistukset - f984af1..013ea2d git-annex -> origin/git-annex - 6d4e731..05224ad master -> origin/master - 6d4e731..05224ad synced/master -> origin/synced/master -ok -get Jarkon ThinkPad T450s (Win10 v21H1) . B/79698DAC29A079D3-06-06.mrimg (from origin...) -ok -pull origin -ok -(recording state in git...) -push origin -Enumerating objects: 9, done. -Counting objects: 100% (9/9), done. -Delta compression using up to 4 threads -Compressing objects: 100% (5/5), done. -Writing objects: 100% (5/5), 476 bytes | 238.00 KiB/s, done. -Total 5 (delta 2), reused 0 (delta 0), pack-reused 0 -To G:\Reflect-varmistukset - f984af1..3ebe493 git-annex -> synced/git-annex -ok -(recording state in git...) - - git status will show some files to be modified, since content availability has changed and git-annex was unable to update the index. This is only a cosmetic problem affecting git status; git add, git commit, etc won't be affected. To fix the git status display, you can run: git-annex restage -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> git annex version --raw -10.20220927-g26dea5641 -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> git status -On branch adjusted/master(unlocked) -nothing to commit, working tree clean -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> git annex drop 79698DAC29A079D3-06-06.mrimg -drop 79698DAC29A079D3-06-06.mrimg (unsafe) - Could only verify the existence of 1 out of 2 necessary copies - - Rather than dropping this file, try using: git annex move - - (Use --force to override this check, or adjust numcopies.) -failed -drop: 1 failed -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> git annex drop --force 79698DAC29A079D3-06-06.mrimg -drop 79698DAC29A079D3-06-06.mrimg ok -(recording state in git...) - - git status will show some files to be modified, since content availability has changed and git-annex was unable to update the index. This is only a cosmetic problem affecting git status; git add, git commit, etc won't be affected. To fix the git status display, you can run: git-annex restage -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> git status -On branch adjusted/master(unlocked) -nothing to commit, working tree clean -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> echo "Installing version 10.20220823-g34e313f78 in the background" -Installing version 10.20220823-g34e313f78 in the background -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> git annex version --raw -10.20220823-g34e313f78 -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> ls 79698DAC29A079D3-06-06.mrimg - - Directory: K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B - -Mode LastWriteTime Length Name ----- ------------- ------ ---- --a--- 30.9.2022 14:53 87 79698DAC29A079D3-06-06.mrimg - -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> git annex get 79698DAC29A079D3-06-06.mrimg -get 79698DAC29A079D3-06-06.mrimg (from origin...) -ok -(recording state in git...) -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> git annex drop --force 79698DAC29A079D3-06-06.mrimg -drop 79698DAC29A079D3-06-06.mrimg ok -(recording state in git...) -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> git status -On branch adjusted/master(unlocked) -nothing to commit, working tree clean -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> ls 79698DAC29A079D3-06-06.mrimg - - Directory: K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B - -Mode LastWriteTime Length Name ----- ------------- ------ ---- --a--- 30.9.2022 15:01 87 79698DAC29A079D3-06-06.mrimg - -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> head 79698DAC29A079D3-06-06.mrimg -/annex/objects/BLAKE2B160E-s8305341589--ab8ca27391c2f68c1c4d677942deaf5022498752.mrimg -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> cat ..\.git\config -[core] - repositoryformatversion = 0 - filemode = false - bare = false - logallrefupdates = true - symlinks = false - ignorecase = true -[interactive] - diffFilter = delta --color-only -[remote "origin"] - url = G:\\Reflect-varmistukset - fetch = +refs/heads/*:refs/remotes/origin/* - fetch = ^refs/heads/adjusted/* - annex-uuid = redacted1-1789-4471-96a0-redactedabcd -[annex] - thin = true - backend = MD5E - maxextensionlength = 8 - uuid = redacted2-45c8-4a86-b348-redactedabcd - sshcaching = false - crippledfilesystem = true - version = 10 - diskreserve = 1000M -[filter "annex"] - smudge = git-annex smudge -- %f - clean = git-annex smudge --clean -- %f -[merge] - renames = true - directoryRenames = false - -# End of transcript or log. -"""]] - -Btw. the git `annex-sync` alias above is defined to be `annex sync --no-commit`. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Git Annex is great. I use it several times a week with my multigigabyte backups, where it gives structure to my image-based backup routines, so you could say I'm a believer. :) - -Big thanks, Joey! - -[[!meta author=jkniiv]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/windows__58___needlessly_reminded_to_run___96__restage__96__/comment_1_e738eb3d45bb7c4baa0a5c88cdf7a40a._comment b/doc/bugs/windows__58___needlessly_reminded_to_run___96__restage__96__/comment_1_e738eb3d45bb7c4baa0a5c88cdf7a40a._comment deleted file mode 100644 index a5594e67c0..0000000000 --- a/doc/bugs/windows__58___needlessly_reminded_to_run___96__restage__96__/comment_1_e738eb3d45bb7c4baa0a5c88cdf7a40a._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1t""" - date="2022-09-30T17:24:27Z" - content=""" -Not sure this is quite the same situation, but on linux, I am seeing -`git-annex unlock` of a locked file display the warning. - -Also, .git/annex/restage.log is left empty after the command completes. -Which seems to indicate it managed to restage the file after all. Is that -file empty for you on windows after it displays the warning and before -running other commands? -"""]] diff --git a/doc/bugs/windows__58___needlessly_reminded_to_run___96__restage__96__/comment_2_8ecb958a1a2c8cd9fd37798c0d4eb0cf._comment b/doc/bugs/windows__58___needlessly_reminded_to_run___96__restage__96__/comment_2_8ecb958a1a2c8cd9fd37798c0d4eb0cf._comment deleted file mode 100644 index c371e292bc..0000000000 --- a/doc/bugs/windows__58___needlessly_reminded_to_run___96__restage__96__/comment_2_8ecb958a1a2c8cd9fd37798c0d4eb0cf._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-09-30T17:43:13Z" - content=""" -Found the bug: "fd:16: hFlush: illegal operation (handle is closed)" -This exception gets caught after it's actually finished restaging -everything, so it displays the warning unncessarily. - -Pretty sure the problem on Windows will have the same cause. -"""]] diff --git a/doc/bugs/windows__58___needlessly_reminded_to_run___96__restage__96__/comment_3_aed127be0616464fda064f00de954fc2._comment b/doc/bugs/windows__58___needlessly_reminded_to_run___96__restage__96__/comment_3_aed127be0616464fda064f00de954fc2._comment deleted file mode 100644 index 30745fff7d..0000000000 --- a/doc/bugs/windows__58___needlessly_reminded_to_run___96__restage__96__/comment_3_aed127be0616464fda064f00de954fc2._comment +++ /dev/null @@ -1,7 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2022-09-30T17:55:20Z" - content=""" -I've fixed this, and I'm planning to make a minor release on Monday for it. -"""]] diff --git a/doc/bugs/windows__58___needlessly_reminded_to_run___96__restage__96__/comment_4_42ef3d3d157a97a202718f3b768916f8._comment b/doc/bugs/windows__58___needlessly_reminded_to_run___96__restage__96__/comment_4_42ef3d3d157a97a202718f3b768916f8._comment deleted file mode 100644 index ac5bc2b181..0000000000 --- a/doc/bugs/windows__58___needlessly_reminded_to_run___96__restage__96__/comment_4_42ef3d3d157a97a202718f3b768916f8._comment +++ /dev/null @@ -1,45 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="restage.log was empty indeed" - date="2022-10-01T05:14:17Z" - content=""" -Just for completeness sake: - -[[!format sh \"\"\" -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> git-annex version --raw -10.20220927-g26dea5641 -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> git annex drop --force 79698DAC29A079D3-06-06.mrimg -drop 79698DAC29A079D3-06-06.mrimg ok -(recording state in git...) - - git status will show some files to be modified, since content availability has changed and git-annex was unable to update the index. This is only a cosmetic problem affecting git status; git add, git commit, etc won't be affected. To fix the git status display, you can run: git-annex restage -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> ls ..\.git\annex\restage.log - - Directory: K:\Reflect-varmistukset\.git\annex - -Mode LastWriteTime Length Name ----- ------------- ------ ---- --a--- 1.10.2022 7:47 0 restage.log - -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> time git annex get 79698DAC29A079D3-06-06.mrimg -get 79698DAC29A079D3-06-06.mrimg (from origin...) -ok -(recording state in git...) - - git status will show some files to be modified, since content availability has changed and git-annex was unable to update the index. This is only a cosmetic problem affecting git status; git add, git commit, etc won't be affected. To fix the git status display, you can run: git-annex restage -00:01:53.0001475 -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> ls ..\.git\annex\restage.log - - Directory: K:\Reflect-varmistukset\.git\annex - -Mode LastWriteTime Length Name ----- ------------- ------ ---- --a--- 1.10.2022 8:02 0 restage.log - -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> git status -On branch adjusted/master(unlocked) -nothing to commit, working tree clean -\"\"\"]] - -"""]] diff --git a/doc/bugs/windows__58___needlessly_reminded_to_run___96__restage__96__/comment_5_8f71510a7547db8d0c63e4a4e3ddbd65._comment b/doc/bugs/windows__58___needlessly_reminded_to_run___96__restage__96__/comment_5_8f71510a7547db8d0c63e4a4e3ddbd65._comment deleted file mode 100644 index d81bf9b141..0000000000 --- a/doc/bugs/windows__58___needlessly_reminded_to_run___96__restage__96__/comment_5_8f71510a7547db8d0c63e4a4e3ddbd65._comment +++ /dev/null @@ -1,47 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 5" - date="2022-10-01T08:23:43Z" - content=""" -Yes, thank you, Joey! The annoying (but sometimes necessary) reminder is gone now in regular get/drop -circumstances: - -[[!format sh \"\"\" -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> git annex drop --force 79698DAC29A079D3-06-06.mrimg -drop 79698DAC29A079D3-06-06.mrimg ok -(recording state in git...) -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> ls ..\.git\annex\restage.log - - Directory: K:\Reflect-varmistukset\.git\annex - -Mode LastWriteTime Length Name ----- ------------- ------ ---- --a--- 1.10.2022 11:08 0 restage.log - -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> git status -On branch adjusted/master(unlocked) -nothing to commit, working tree clean -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> time git annex get 79698DAC29A079D3-06-06.mrimg -get 79698DAC29A079D3-06-06.mrimg (from origin...) -ok -(recording state in git...) -00:01:50.7710391 -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> ls ..\.git\annex\restage.log - - Directory: K:\Reflect-varmistukset\.git\annex - -Mode LastWriteTime Length Name ----- ------------- ------ ---- --a--- 1.10.2022 11:11 0 restage.log - -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> git status -On branch adjusted/master(unlocked) -nothing to commit, working tree clean -K:\Reflect-varmistukset\Jarkon ThinkPad T450s (Win10 v21H1) . B [adjusted/master(unlocked)]> git annex version --raw -10.20220928-g49ee07f93 -\"\"\"]] - -:) - -"""]] diff --git a/doc/bugs/windows__58___two_repo_tests_fail_after_commit_5a98f2d5.mdwn b/doc/bugs/windows__58___two_repo_tests_fail_after_commit_5a98f2d5.mdwn deleted file mode 100644 index a359ca8bee..0000000000 --- a/doc/bugs/windows__58___two_repo_tests_fail_after_commit_5a98f2d5.mdwn +++ /dev/null @@ -1,217 +0,0 @@ -### Please describe the problem. - -Recently I found that two repo tests, namely 'concurrent get of dup key regression' -and 'import', fail due to locking problems. I bisected this on my Windows laptop and -found that [[!commit 5a98f2d50913682c4ebe0e0c4ce695c450a96091]] is the first bad commit. - -### What steps will reproduce the problem? - -`stack setup && stack build`. Then copy git-annex.exe to `C:\annx` and in (Git) Bash say: - -[[!format sh """ -$ ./git-annex test -p 'Repo Tests' 2>&1 | tee git-annex.test--p-Repo_Tests.LOG~202 -$ grep -E '(concurrent get of dup key regression|import):' git-annex.test--p-Repo_Tests.LOG~202 -"""]] - -Observe two FAILs. - -(It is unfortunate that you can't call for instance `./git-annex.exe test -p 'concurrent get of dup key regression'` -directly because of an unrelated setup issue with Tasty. I guess this would warrant a separate bug report.) - -### Please provide any additional information below. - -_Under Windows with 10.20220505-g5a98f2d50:_ - -[[!format sh """ -jkniiv@AINESIS MINGW64 /c/annx -$ cat git-annex.test--p-Repo_Tests.LOG~202 - -All 0 tests passed (0.00s) -Tests - Repo Tests v8 adjusted unlocked branch - Init Tests - init: OK (10.30s) - add: OK (14.20s) - borg remote: OK - uninit: OK (15.54s) - conflict resolution (mixed directory and file): OK (154.59s) - concurrent get of dup key regression: FAIL (18.41s) - .\\Test\\Framework.hs:70: - drop with dup failed (transcript follows) - drop foo ok - drop foo2 - failed - (recording state in git...) - git-annex: content is locked - drop: 1 failed - - Use -p '/Repo Tests/&&/concurrent get of dup key regression/' to rerun this test only. - migrate (via gitattributes): OK (35.88s) - fsck (basics): OK (33.18s) - copy: OK (29.05s) - drop (no remote): OK (18.30s) - shared clone: OK (8.98s) - -1 out of 11 tests failed (338.52s) - -All 0 tests passed (0.03s) -Tests - Repo Tests v8 adjusted unlocked branch - Init Tests - init: OK (10.26s) - add: gpg testing not implemented on Windows -OK (13.95s) - crypto: OK - uninit (in git-annex branch): OK (12.13s) - conflict resolution symlink bit: OK - union merge regression: OK (148.89s) - unused: OK (59.95s) - fsck (bare): OK (5.63s) - lock: OK (29.89s) - drop (with remote): OK (22.18s) - log: OK (10.16s) - add dup: OK (15.61s) - -All 12 tests passed (328.74s) -Tests - Repo Tests v8 adjusted unlocked branch - Init Tests - init: OK (10.27s) - add: OK (14.07s) - rsync remote: OK (8.80s) - conflict resolution (mixed locked and unlocked file): OK (70.62s) - conflict resolution (adjusted branch): OK (92.89s) - version: OK (10.99s) - conversion annexed to git: OK (18.28s) - fix: OK (9.54s) - move (ssh remote): OK - unannex (no copy): OK (12.49s) - export_import: OK (88.90s) - -All 11 tests passed (336.95s) -Tests - Repo Tests v8 adjusted unlocked branch - Init Tests - init: OK (9.80s) - add: OK (13.22s) - add subdirs: OK (21.85s) - hook remote: OK (9.82s) - conflict resolution (nonannexed file): OK (133.39s) - transition propagation: OK (139.19s) - merge: OK (11.89s) - fsck --from remote: OK (10.01s) - edit (pre-commit): OK (15.37s) - get (ssh remote): OK - import: FAIL (20.87s) - .\\Test\\Framework.hs:70: - drop failed (transcript follows) - drop import1/f ok - drop import2/f - failed - drop import5/f - failed - (recording state in git...) - git-annex: content is locked - git-annex: content is locked - drop: 2 failed - - Use -p '/Repo Tests/&&/import/' to rerun this test only. - ignore deleted files: OK (8.43s) - -1 out of 12 tests failed (393.92s) -Tests - Repo Tests v8 adjusted unlocked branch - Init Tests - init: OK (10.27s) - add: OK (14.14s) - bup remote: OK (8.85s) - map: OK (12.69s) - conflict resolution movein regression: OK (92.81s) - sync: OK (58.83s) - migrate: OK (35.32s) - trust: OK (30.37s) - move (numcopies): OK (40.73s) - unannex (with copy): OK (14.55s) - export_import_subdir: OK (56.87s) - -All 11 tests passed (375.49s) -Tests - Repo Tests v8 adjusted unlocked branch - Init Tests - init: OK (8.97s) - add: OK (13.79s) - preferred content: OK (63.02s) - upgrade: OK (10.77s) - conflict resolution (uncommitted local file): OK (101.39s) - adjusted branch merge regression: OK (19.12s) - describe: OK (12.57s) - fsck (local untrusted): OK (26.13s) - lock --force: OK (16.35s) - drop (untrusted remote): OK (24.97s) - view: OK (15.62s) - add extras: OK (13.99s) - -All 12 tests passed (326.79s) -Tests - Repo Tests v8 adjusted unlocked branch - Init Tests - init: OK (10.67s) - add: OK (13.44s) - addurl: OK (16.86s) - directory remote: OK (32.54s) - conflict resolution (nonannexed symlink): OK (40.41s) - conflict resolution: OK (86.32s) - info: OK (13.57s) - conversion git to annexed: OK (19.97s) - partial commit: OK (20.86s) - move: OK (41.33s) - reinject: OK (16.08s) - metadata: OK (23.76s) - -All 12 tests passed (335.87s) -Tests - Repo Tests v8 adjusted unlocked branch - Init Tests - init: OK (9.19s) - add: OK (13.38s) - required_content: OK (31.54s) - whereis: OK (25.92s) - conflict resolution (removed file): OK (203.83s) - adjusted branch subtree regression: OK (30.38s) - find: OK (26.15s) - fsck (remote untrusted): OK (18.86s) - edit (no pre-commit): OK (17.01s) - get: OK (13.68s) - magic: OK (5.86s) - readonly remote: OK - -All 12 tests passed (395.88s) - (Failures above could be due to a bug in git-annex, or an incompatibility - with utilities, such as git, installed on this system.) - -# End of transcript. -"""]] - - -### What version of git-annex are you using? On what operating system? - -[[!format sh """ -git-annex version: 10.20220505-g5a98f2d50 -build flags: Assistant Webapp Pairing TorrentParser Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.7 http-client-0.7.9 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.1.2 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: mingw32 x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 2 3 4 5 6 7 8 9 10 -"""]] - -Windows 10 version 21H2 (build 19044.1706), 64 bit. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Git Annex is great. I use it several times a week with my multigigabyte backups, where it gives structure to my image-based backup routines, so you could say I'm a believer. :) - -[[!meta author=jkniiv]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/windows__58___two_repo_tests_fail_after_commit_5a98f2d5/comment_1_af091a5799ac0445033c05173e25a4af._comment b/doc/bugs/windows__58___two_repo_tests_fail_after_commit_5a98f2d5/comment_1_af091a5799ac0445033c05173e25a4af._comment deleted file mode 100644 index 0328e72957..0000000000 --- a/doc/bugs/windows__58___two_repo_tests_fail_after_commit_5a98f2d5/comment_1_af091a5799ac0445033c05173e25a4af._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-05-23T16:57:34Z" - content=""" -Thanks for another fine bug report. - -I took a look at that commit, and it seems that reverting it for Windows -makes the most sense. There were two places where it changed behavior on -Windows, that were both separated from the code paths for other OS's. - -I've made that change, which should fix it. I'm going to close this bug -report, but please follow up if you still see the issue. - -(Also, I've opened [[tasty_test_limiting_broken_by_concurrency]] -"""]] diff --git a/doc/bugs/windows__58___two_repo_tests_fail_after_commit_5a98f2d5/comment_2_5c7ec0d8520f957ec4f6a4df30941cd8._comment b/doc/bugs/windows__58___two_repo_tests_fail_after_commit_5a98f2d5/comment_2_5c7ec0d8520f957ec4f6a4df30941cd8._comment deleted file mode 100644 index 3d80885926..0000000000 --- a/doc/bugs/windows__58___two_repo_tests_fail_after_commit_5a98f2d5/comment_2_5c7ec0d8520f957ec4f6a4df30941cd8._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 2" - date="2022-05-23T19:41:27Z" - content=""" -That did seem to do the trick as the testsuite passes fully again. Thanks for the quick fix, Joey! -"""]] diff --git a/doc/bugs/windows_build_fails_starting_from_commit_54ad1b4cf.mdwn b/doc/bugs/windows_build_fails_starting_from_commit_54ad1b4cf.mdwn deleted file mode 100644 index 3a84a49494..0000000000 --- a/doc/bugs/windows_build_fails_starting_from_commit_54ad1b4cf.mdwn +++ /dev/null @@ -1,122 +0,0 @@ -### Please describe the problem. - -Thanks for the improvements in Windows long file support but it turns out that Git-annex -commit 54ad1b4cf doesn't quite compile on Windows without a small fix to Remote.Git and -Utility.DirWatcher.Win32Notify regarding imports. - -### What steps will reproduce the problem? - -`stack setup && stack build` - -### What version of git-annex are you using? On what operating system? - -[[!format sh """ -git-annex version: 10.20230228-g54ad1b4cf -build flags: Assistant Webapp Pairing TorrentParser Benchmark Feeds Testsuite S3 WebDAV -dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.7 http-client-0.7.9 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.1.2 -key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X* -remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external -operating system: mingw32 x86_64 -supported repository versions: 8 9 10 -upgrade supported from repository versions: 2 3 4 5 6 7 8 9 10 -"""]] - -A successful build that passes the testsuite on Windows is achieved with the patch below. - -Windows 10 version 22H2 (build 19045.2486), 64 bit. - -### Please provide any additional information below. - -Relevant parts of the build log: - -[[!format sh """ -jkniiv@AINESIS MINGW64 /c/annx -$ tail -n 35 stack.build.LOG~102 -[143 of 676] Compiling Types.Backend -[144 of 676] Compiling Database.Handle -[145 of 676] Compiling Database.Queue -[146 of 676] Compiling Database.Keys.Handle -[147 of 676] Compiling Utility.Gpg -[148 of 676] Compiling Types.Crypto -[149 of 676] Compiling Utility.DirWatcher.Types -[150 of 676] Compiling Utility.DirWatcher.Win32Notify - -Utility\DirWatcher\Win32Notify.hs:54:51: error: - * Variable not in scope: isRegularFile :: FileStatus -> Bool - * Perhaps you meant `System.PosixCompat.Files.isRegularFile' (imported from System.PosixCompat.Files) - | -54 | | isRegularFile s -> - - | ^^^^^^^^^^^^^ - -Utility\DirWatcher\Win32Notify.hs:62:36: error: - Variable not in scope: getFileStatus :: a -> m a1 - | -62 | getstatus = catchMaybeIO . getFileStatus - - | ^^^^^^^^^^^^^ - - -Error: [S-7282] - Stack failed to execute the build plan. - - While executing the build plan, Stack encountered the following errors: - - [S-7011] - While building package git-annex-10.20230227 (scroll up to its section to see the error) - using: - C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-230302-54ad1b4cf\.stack-work\dist\274b403a\setup\setup --verbose=1 --builddir=.stack-work\dist\274b403a build exe:git-annex --ghc-options " -fdiagnostics-color=always" - Process exited with code: ExitFailure 1 - -# End of transcript. -"""]] - -The change I made was the following: - -[[!format diff """ -diff --git a/Remote/Git.hs b/Remote/Git.hs -index c65d1388d..0e5fbe8df 100644 ---- a/Remote/Git.hs -+++ b/Remote/Git.hs -@@ -65,9 +65,7 @@ import Types.ProposedAccepted - import Annex.Action - import Messages.Progress - --#ifndef mingw32_HOST_OS - import qualified Utility.RawFilePath as R --#endif - - import Control.Concurrent - import qualified Data.Map as M -diff --git a/Utility/DirWatcher/Win32Notify.hs b/Utility/DirWatcher/Win32Notify.hs -index 7eba40de9..acc6bd3af 100644 ---- a/Utility/DirWatcher/Win32Notify.hs -+++ b/Utility/DirWatcher/Win32Notify.hs -@@ -11,7 +11,7 @@ import Common hiding (isDirectory) - import Utility.DirWatcher.Types - - import System.Win32.Notify --import qualified System.PosixCompat.Files (isRegularFile) -+import System.PosixCompat.Files (isRegularFile, getFileStatus) - - watchDir :: FilePath -> (FilePath -> Bool) -> Bool -> WatchHooks -> IO WatchManager - watchDir dir ignored scanevents hooks = do - """]] - -This then compiled cleanly and completed the test suite ok. - -### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) - -Git Annex is great. I use it several times a week with my multigigabyte backups, where it gives structure to my image-based backup routines, so you could say I'm a believer. :) - -[[!meta author=jkniiv]] -[[!meta title="windows: build fails starting from commits 54ad1b4cf, 398633c12"]] - -> Kind of expected there would be something! The patch is almost right, -> but we're avoiding using System.PosixCompat.Files.getFileStatus -> since it doesn't support long paths, so have to use -> Utility.RawFilePath.getFileStatus instead. \[\[done\]\] --[[Joey]] - ->> Sorry, I had to reopen the bug as Joey's commit didn't quite compile. See page comments for details. --[[jkniiv]] - ->>> Should be fully fixed now. [[done]] --[[Joey]] diff --git a/doc/bugs/windows_build_fails_starting_from_commit_54ad1b4cf/comment_1_c353c67f06f40c84b956990160205e0a._comment b/doc/bugs/windows_build_fails_starting_from_commit_54ad1b4cf/comment_1_c353c67f06f40c84b956990160205e0a._comment deleted file mode 100644 index 34fcf6b8a5..0000000000 --- a/doc/bugs/windows_build_fails_starting_from_commit_54ad1b4cf/comment_1_c353c67f06f40c84b956990160205e0a._comment +++ /dev/null @@ -1,84 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="ok, that didn't quite resolve it" - date="2023-03-04T19:15:31Z" - content=""" -@Joey, unfortunately commit [[!commit 398633c12bfb6201ba1fcd9b50e5e8055bfff89e]] wasn't quite enough to -resolve my issue. :) First of all I wondered why you would employ `fromRawFilePath` instead of `toRawFilePath` -considering that the argument of `getstatus` seems to be a FilePath, and indeed Utility.DirWatcher.Win32Notify -didn't compile without replacing the former with the latter. Also you didn't notice that I had changed a qualified -import into a regular one because otherwise `isRegularFile` wouldn't be available. - -Here's the tail end of my build log trying to build commit 398633c12: - -[[!format sh \"\"\" -jkniiv@AINESIS MINGW64 /c/annx -$ tail -n 35 stack.build.LOG~102 -[150 of 676] Compiling Utility.DirWatcher.Win32Notify - -Utility\DirWatcher\Win32Notify.hs:55:51: error: - * Variable not in scope: isRegularFile :: FileStatus -> Bool - * Perhaps you meant `System.PosixCompat.Files.isRegularFile' (imported from System.PosixCompat.Files) - | -55 | | isRegularFile s -> - - | ^^^^^^^^^^^^^ - -Utility\DirWatcher\Win32Notify.hs:63:54: error: - * Couldn't match type `[Char]' - with `Data.ByteString.Internal.ByteString' - Expected type: RawFilePath -> RawFilePath - Actual type: RawFilePath -> FilePath - * In the second argument of `(.)', namely `fromRawFilePath' - In the second argument of `(.)', namely - `R.getFileStatus . fromRawFilePath' - In the expression: catchMaybeIO . R.getFileStatus . fromRawFilePath - | -63 | getstatus = catchMaybeIO . R.getFileStatus . fromRawFilePath - - | ^^^^^^^^^^^^^^^ - - -Error: [S-7282] - Stack failed to execute the build plan. - - While executing the build plan, Stack encountered the following errors: - - [S-7011] - While building package git-annex-10.20230227 (scroll up to its section to see the error) - using: - C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-230304-398633c12\.stack-work\dist\274b403a\setup\setup --verbose=1 --builddir=.stack-work\dist\274b403a build exe:git-annex --ghc-options \" -fdiagnostics-color=always\" - Process exited with code: ExitFailure 1 - -\"\"\"]] - -So, I ended up with the following simple fix (although this situation left me wondering if `watchDir` ought to -be rewritten to deal with RawFilePaths directly but then that's beyond my abilities): - -[[!format diff \"\"\" -diff --git a/Utility/DirWatcher/Win32Notify.hs b/Utility/DirWatcher/Win32Notify.hs -index a6a7e4ce9..0b02053c9 100644 ---- a/Utility/DirWatcher/Win32Notify.hs -+++ b/Utility/DirWatcher/Win32Notify.hs -@@ -12,7 +12,7 @@ import Utility.DirWatcher.Types - import qualified Utility.RawFilePath as R - - import System.Win32.Notify --import qualified System.PosixCompat.Files (isRegularFile) -+import System.PosixCompat.Files (isRegularFile) - - watchDir :: FilePath -> (FilePath -> Bool) -> Bool -> WatchHooks -> IO WatchManager - watchDir dir ignored scanevents hooks = do -@@ -60,7 +60,7 @@ watchDir dir ignored scanevents hooks = do - where - runhook h s = maybe noop (\a -> a f s) (h hooks) - -- getstatus = catchMaybeIO . R.getFileStatus . fromRawFilePath -+ getstatus = catchMaybeIO . R.getFileStatus . toRawFilePath - - {- Check each component of the path to see if it's ignored. -} - ignoredPath :: (FilePath -> Bool) -> FilePath -> Bool -\"\"\"]] - -"""]] diff --git a/doc/bugs/windows_build_fails_starting_from_commit_54ad1b4cf/comment_2_bd9a77da095bffdcd3ed30e08192b9d2._comment b/doc/bugs/windows_build_fails_starting_from_commit_54ad1b4cf/comment_2_bd9a77da095bffdcd3ed30e08192b9d2._comment deleted file mode 100644 index 1def6c13e9..0000000000 --- a/doc/bugs/windows_build_fails_starting_from_commit_54ad1b4cf/comment_2_bd9a77da095bffdcd3ed30e08192b9d2._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 2" - date="2023-03-05T20:18:01Z" - content=""" -On a positive note I can confirm that my compiled versions of commits 54ad1b4cf and 398633c12 -pass their test suite even in a subdirectory some 64 characters deep (for instance in a directory -named like `c:/Users/jkniiv/annx/annx.10.20230228-g505f1a654--test-test-test`) whereas the commit -just prior to \"Windows: Support long filenames in more (possibly all) of the code\" (54ad1b4cf), or -505f1a654, fails some one third to one half of its tests in said directory. Shortening the test -directory name to say `c:/Users/jkniiv/annx/annx` allows the test suite to pass in older versions -of git-annex all other things considered equal. So far I have been running the built-in tests -always in 'c:/annx' (/c/annx in Git Bash) but here on forward I'll probably start using a deeper -path somewhere in my user folder / home directory just to catch possible future regressions in long -path handling. In any case, @Joey, congratulations 🎉 are in order for your fine achievement from -a Windows user point of view. :) - -"""]] diff --git a/doc/bugs/windows_build_fails_starting_from_commit_54ad1b4cf/comment_3_c46e7c7f3870b54563eb8e94d451fded._comment b/doc/bugs/windows_build_fails_starting_from_commit_54ad1b4cf/comment_3_c46e7c7f3870b54563eb8e94d451fded._comment deleted file mode 100644 index efc9c195c9..0000000000 --- a/doc/bugs/windows_build_fails_starting_from_commit_54ad1b4cf/comment_3_c46e7c7f3870b54563eb8e94d451fded._comment +++ /dev/null @@ -1,22 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2023-03-06T16:17:37Z" - content=""" -That's great news about the test suite in a deep directory. That was a -longstanding problem indeed and a good indication it's fully supporting -long filenames now. - -I often mix up fromRawFilePath/toRawFilePath. Luckily the compiler will -notice. Should be fixed now. - -As for converting the whole `watchDir` to RawFilePath, there is probably -not much benefit in doing that in such windows-specific code, but the goal -is to do such a conversion in as much of git-annex as possible, since it is -faster on unix to use RawFilePath end-to-end. - -Doing it is as simple as juggling the use of fromRawFilePath/toRawFilePath -as the compiler allows, while converting code to use functions from -Utility.RawFilePath or System.FilePath.ByteString or that otherwise -use RawFilePath. I think you would be capable of that for sure! -"""]] diff --git a/doc/bugs/windows_ftbfs.mdwn b/doc/bugs/windows_ftbfs.mdwn deleted file mode 100644 index 77d3a8c926..0000000000 --- a/doc/bugs/windows_ftbfs.mdwn +++ /dev/null @@ -1,44 +0,0 @@ -### Please describe the problem. - -Started to happen recently (0924): - -```shell -(git)smaug:/mnt/datasets/datalad/ci/git-annex/builds/2022/09[master]git -$> git grep -l 'Couldn.t match expected type ' -cron-20220924/build-windows.yaml-788-e26581b6-failed/1_build-package (1).txt -cron-20220924/build-windows.yaml-788-e26581b6-failed/build-package/18_Build git-annex.txt -cron-20220925/build-windows.yaml-790-e26581b6-failed/1_build-package (1).txt -cron-20220925/build-windows.yaml-790-e26581b6-failed/build-package/18_Build git-annex.txt -cron-20220926/build-windows.yaml-791-40917e42-failed/1_build-package (1).txt -cron-20220926/build-windows.yaml-791-40917e42-failed/build-package/18_Build git-annex.txt -pr-133/build-windows.yaml-789-06e70ac7-failed/1_build-package (1).txt -pr-133/build-windows.yaml-789-06e70ac7-failed/build-package/18_Build git-annex.txt -``` - - - -``` -Annex\PidLock.hs:130:48: error: - * Couldn't match expected type `IO a' with actual type `Annex a' - * In the second argument of `($)', namely `a r' - In the expression: liftIO $ a r - In an equation for runsGitAnnexChildProcessViaGit': - runsGitAnnexChildProcessViaGit' r a = liftIO $ a r - * Relevant bindings include - a :: Repo -> Annex a (bound at Annex\PidLock.hs:130:35) - runsGitAnnexChildProcessViaGit' :: Repo - -> (Repo -> Annex a) -> Annex a - (bound at Annex\PidLock.hs:130:1) - | -130 | runsGitAnnexChildProcessViaGit' r a = liftIO $ a r - - | ^^^ - - -Warning: Failed to decode module interface: - D:\a\git-annex\git-annex\.stack-work\dist\274b403a\build\git-annex\git-annex-tmp\Annex.hi - Decoding failure: Invalid magic: e49ceb0f -... -``` - -> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/wrong_modes_for_some_lock_files_withcoresharedrepository.mdwn b/doc/bugs/wrong_modes_for_some_lock_files_withcoresharedrepository.mdwn deleted file mode 100644 index 1067470b4c..0000000000 --- a/doc/bugs/wrong_modes_for_some_lock_files_withcoresharedrepository.mdwn +++ /dev/null @@ -1,24 +0,0 @@ -With core.sharedRepository=0666, some lock files get created mode 644 -(with umask 0022). - -With core.sharedRepository=group, some lock files get created mode 660, -rather than 664 (with umask 0022). - -Root of the problem is uses of annexFileMode. - -Some callers use noUmask with it, which works in cases other than these (at -least with umask 0022). But in the core.sharedRepository=group case, the -umask is cleared by noUmask, which is why the o+r bit is not set. - -Some callers don't use noUmask with it, and when -core.sharedRepository=0666, that results in the umask being applied -to the mode. Which it's not supposed to with an octal mode configured. - -The affected files are all lock files. - -Fix will probably involve getting rid of annexFileMode, and noUmask, and -creating the lock file with default umask, then fixing up the mode if necessary, -before using it. Ie, the same pattern used everywhere else in git-annex. ---[[Joey]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/todo/--get_option_for_diffdriver.mdwn b/doc/todo/--get_option_for_diffdriver.mdwn deleted file mode 100644 index 6e424d0293..0000000000 --- a/doc/todo/--get_option_for_diffdriver.mdwn +++ /dev/null @@ -1,5 +0,0 @@ -since there is no generic 'fuse' mode, I would like to request to have `--get` (or `--auto-get`) option for diffdriver. I am trying to compare files across two branches on a repo I just cloned. I cannot download all the files and downloading differing keys across branches for the same file is a bit painful. So I felt that it would be super nice if git annex could auto get those files from somewhere (well -- original clone) - -[[!tag confirmed]] - -> [[done]] --[[Joey]] diff --git a/doc/todo/--get_option_for_diffdriver/comment_1_6b2501b2dc0318242e444e8d4b8b314f._comment b/doc/todo/--get_option_for_diffdriver/comment_1_6b2501b2dc0318242e444e8d4b8b314f._comment deleted file mode 100644 index afaa14e986..0000000000 --- a/doc/todo/--get_option_for_diffdriver/comment_1_6b2501b2dc0318242e444e8d4b8b314f._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2018-02-08T17:28:21Z" - content=""" -This is a good idea. I wonder what to do about the objects it downloads for -a diff; should they be left in the annex for later use/dropunused, or -immediately deleted after the diff completes. - -My inclination is to keep -them around, for one thing when I'm diffing stuff I often run diff more -than once, perhaps to widen the diff or because I want to take a second -look at it, and re-downloading a bunch of big files would be painful then. - -By the way, what diffdriver program are you using? I've had a hard time -finding any real-life examples of git diffdrivers. -"""]] diff --git a/doc/todo/--get_option_for_diffdriver/comment_2_cb4eb05f447a214353a5bd5c29d207d0._comment b/doc/todo/--get_option_for_diffdriver/comment_2_cb4eb05f447a214353a5bd5c29d207d0._comment deleted file mode 100644 index 8f2f810c0d..0000000000 --- a/doc/todo/--get_option_for_diffdriver/comment_2_cb4eb05f447a214353a5bd5c29d207d0._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 2" - date="2023-08-09T05:31:26Z" - content=""" -How about handling them as if they were manually fetched with `git annex get`? The next run of `satisfy`, `sync`, etc. will then take care of dropping. -"""]] diff --git a/doc/todo/--metadata_fieldname__62____61__VALUE_string_comparison.mdwn b/doc/todo/--metadata_fieldname__62____61__VALUE_string_comparison.mdwn deleted file mode 100644 index a7eea78b13..0000000000 --- a/doc/todo/--metadata_fieldname__62____61__VALUE_string_comparison.mdwn +++ /dev/null @@ -1,95 +0,0 @@ -Thank you for `git-annex`, it's awesome! - -I recently figured I could add `git-annex metadata` to my research data files that contains the start and end date of timeseries data inside the files so a quick lookup by date range (”which files contain data in that time range”) is possible. - -This is possible when using numeric timestamps (e.g. unix timestamp like `1669981463`) but not with stringy dates (e.g. `2022-11-12T20:10:14+0200`) as `--metadata fieldname>=VALUE` does _numeric_ comparison. - -## Proposal: How about when `--metadata fieldname>=VALUE` falls back to string comparison when `VALUE` can't be parsed as a number? - -## Test case - -Consider this script `make-git-annex-dir-with-timestamps.sh`: - -```sh -#/bin/sh -fmt="$1";test -n "$fmt" || fmt="%FT%T%z" -# make a new git annex repository -d=git-annex-with-times-"$fmt";chmod +w -R "$d";rm -rf "$d";mkdir "$d";cd "$d" -git init -git annex init -# create some files -for i in `seq 1 9`;do echo "File $i" > "file$i";done -git annex add . -git commit -m "Add files" -# add metadata to files -for i in `seq 1 9`;do - time_start="$(date -d"$((-20 + $i)) hours" +"$fmt")" - (set -x;git annex metadata --set time-start="$time_start" "file$i") - time_end="$(date -d"$((-10 + $i)) hours" +"$fmt")" - (set -x;git annex metadata --set time-end="$time_end" "file$i") -done -timerange_start="$(date -d "-16 hours -5 minutes" +"$fmt")" -timerange_end="$(date -d "-12 hours +5 minutes" +"$fmt")" -( -set -x -git annex find \ - "-(" --metadata "time-start>=$timerange_start" --and --metadata "time-start<=$timerange_end" "-)" \ - --or \ - "-(" --metadata "time-end>=$timerange_start" --and --metadata "time-end<=$timerange_end" "-)" -) -echo "⬆⬆⬆ This should only output file4 through file8 ⬆⬆⬆" -``` - -Invoked with unix timestamps time format, it works as expected: - -```sh -> ./make-git-annex-dir-with-timestamps.sh '%s' -# ... -+ git annex find '-(' --metadata 'time-start>=1669923315' --and --metadata 'time-start<=1669938315' '-)' --or '-(' --metadata 'time-end>=1669923315' --and --metadata 'time-end<=1669938315' '-)' -file4 -file5 -file6 -file7 -file8 -⬆⬆⬆ This should only output file4 through file8 ⬆⬆⬆ -``` - -However, other stringy date formats match all files: - -```bash -# typical ISO-ish time format -> ./make-git-annex-dir-with-timestamps.sh "%FT%T%z" -# ... -+ git annex find '-(' --metadata 'time-start>=2022-12-01T20:49:37+0100' --and --metadata 'time-start<=2022-12-02T00:59:37+0100' '-)' --or '-(' --metadata 'time-end>=2022-12-01T20:49:37+0100' --and --metadata 'time-end<=2022-12-02T00:59:37+0100' '-)' -file1 -file2 -file3 -file4 -file5 -file6 -file7 -file8 -file9 -⬆⬆⬆ This should only output file4 through file8 ⬆⬆⬆ -``` - -```sh -# git-annex's own time format for 'FIELDNAME-lastchanged' -> ./make-git-annex-dir-with-timestamps.sh "%Y-%m-%d@%H-%M-%S" -# ... -+ git annex find '-(' --metadata 'time-start>=2022-12-01@20-38-04' --and --metadata 'time-start<=2022-12-02@00-38-04' '-)' --or '-(' --metadata 'time-end>=2022-12-01@20-38-04' --and --metadata 'time-end<=2022-12-02@00-38-04' '-)' -file1 -file2 -file3 -file4 -file5 -file6 -file7 -file8 -file9 -⬆⬆⬆ This should only output file4 through file8 ⬆⬆⬆ -``` - -Yann / @nobodyinperson - -> [[fixed|done]] --[[Joey]] diff --git a/doc/todo/--metadata_fieldname__62____61__VALUE_string_comparison/comment_1_e2da4d1fceb5f7c37e3c19d42256c8d5._comment b/doc/todo/--metadata_fieldname__62____61__VALUE_string_comparison/comment_1_e2da4d1fceb5f7c37e3c19d42256c8d5._comment deleted file mode 100644 index 32799822fa..0000000000 --- a/doc/todo/--metadata_fieldname__62____61__VALUE_string_comparison/comment_1_e2da4d1fceb5f7c37e3c19d42256c8d5._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-12-12T17:06:40Z" - content=""" -I think this is a good idea. Particularly there was no good reason for -it to default to false when either value is not a number. - -I've implemented this. -"""]] diff --git a/doc/todo/--metadata_fieldname__62____61__VALUE_string_comparison/comment_2_38c46dee95ef2f246cd403717faaaa09._comment b/doc/todo/--metadata_fieldname__62____61__VALUE_string_comparison/comment_2_38c46dee95ef2f246cd403717faaaa09._comment deleted file mode 100644 index 9e80fe0aa5..0000000000 --- a/doc/todo/--metadata_fieldname__62____61__VALUE_string_comparison/comment_2_38c46dee95ef2f246cd403717faaaa09._comment +++ /dev/null @@ -1,37 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 2" - date="2022-12-22T11:39:41Z" - content=""" -Thanks joey! However I don't think it works quite now. Slightly modifying the script above I made a [testsuite](https://gitlab.com/-/snippets/2475815) and it seems metadata fields are still interpreted as numbers somehow: - -```bash -# old version without lexicographical ordering -❯ git annex version --raw -10.20221212-gab11fd70e -❯ bash make-git-annex-dir-with-timestamps.sh %s %Y%m%d%H%M%S %Y.%m%d%H%M%S %Y:%m%d%H%M%S %Y-%m-%dT%H%:M:%S %Y-%m-%dT%H%:M:%S%z %FT%T %FT%T%z - %s ok - %Y%m%d%H%M%S ok - %Y.%m%d%H%M%S ok - %Y:%m%d%H%M%S fail - %Y-%m-%dT%H%:M:%S fail - %Y-%m-%dT%H%:M:%S%z fail - %FT%T fail - %FT%T%z fail - -# new version apparently with lexicographical ordering but no change -❯ git annex version --raw -10.20221213-gac6ccb513 -❯ bash make-git-annex-dir-with-timestamps.sh %s %Y%m%d%H%M%S %Y.%m%d%H%M%S %Y:%m%d%H%M%S %Y-%m-%dT%H%:M:%S %Y-%m-%dT%H%:M:%S%z %FT%T %FT%T%z - %s ok - %Y%m%d%H%M%S ok - %Y.%m%d%H%M%S ok - %Y:%m%d%H%M%S fail - %Y-%m-%dT%H%:M:%S fail - %Y-%m-%dT%H%:M:%S%z fail - %FT%T fail - %FT%T%z fail - -``` -"""]] diff --git a/doc/todo/--metadata_fieldname__62____61__VALUE_string_comparison/comment_3_9c298fbda8d395fe9b8395105708c3bd._comment b/doc/todo/--metadata_fieldname__62____61__VALUE_string_comparison/comment_3_9c298fbda8d395fe9b8395105708c3bd._comment deleted file mode 100644 index d27c4641b5..0000000000 --- a/doc/todo/--metadata_fieldname__62____61__VALUE_string_comparison/comment_3_9c298fbda8d395fe9b8395105708c3bd._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2022-12-22T18:30:46Z" - content=""" -Thanks for double-checking. I have not tried your test but I think I see -what is going on from the pattern. Strings starting with a number but then -subsequently not a number were still being parsed to numbers and compared -numerically. I've fixed that now. -"""]] diff --git a/doc/todo/--metadata_fieldname__62____61__VALUE_string_comparison/comment_4_b8f2602e633dae4c85f89c97c8f03d81._comment b/doc/todo/--metadata_fieldname__62____61__VALUE_string_comparison/comment_4_b8f2602e633dae4c85f89c97c8f03d81._comment deleted file mode 100644 index a20013aa3c..0000000000 --- a/doc/todo/--metadata_fieldname__62____61__VALUE_string_comparison/comment_4_b8f2602e633dae4c85f89c97c8f03d81._comment +++ /dev/null @@ -1,24 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 4" - date="2022-12-23T09:39:57Z" - content=""" -Now it works, thank you very much joey! 🎉 - -```bash -❯ ./make-git-annex-dir-with-timestamps.sh -+ git annex version --raw -10.20221213-g29176f131 - %s ok - %Y%m%d%H%M%S ok - %Y.%m%d%H%M%S ok - %Y:%m%d%H%M%S ok - %Y-%m-%dT%H:%M:%S ok - %Y-%m-%dT%H:%M:%S%z ok - %FT%T ok - %FT%T%z ok -``` - -(I had a typo in the above tests - a `%` out of place - but it didn't change the picture, now everything works as expected anyway.) -"""]] diff --git a/doc/todo/--metadata_fieldname__62____61__VALUE_string_comparison/comment_5_afba02fb24eddc6569dce63ffe606dcf._comment b/doc/todo/--metadata_fieldname__62____61__VALUE_string_comparison/comment_5_afba02fb24eddc6569dce63ffe606dcf._comment deleted file mode 100644 index ffa3fb6893..0000000000 --- a/doc/todo/--metadata_fieldname__62____61__VALUE_string_comparison/comment_5_afba02fb24eddc6569dce63ffe606dcf._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 5" - date="2022-12-23T10:05:26Z" - content=""" -A very nice benefit of this feature is that we can now `git annex find` files depending on when their metadata was changed: - -```sh -# annexed files which have recently had their metadata changed -git annex find --metadata \"lastchanged>=$(date -d'2 days ago' +%Y-%m-%d@%H-%M-%S)\" - -# annexed files which have their tag changed in a specific week -git annex find --metadata \"tag-lastchanged>=2022-12-18\" --and --metadata \"tag-lastchanged<=2022-12-25\" -``` -"""]] diff --git a/doc/todo/Allow_globally_limiting_filename_length.mdwn b/doc/todo/Allow_globally_limiting_filename_length.mdwn deleted file mode 100644 index 18e87c32a5..0000000000 --- a/doc/todo/Allow_globally_limiting_filename_length.mdwn +++ /dev/null @@ -1,11 +0,0 @@ -I have some Git Annex repos that I keep copies of on both NTFS on Linux and on ecryptfs (which Ubuntu uses for home directory encryption) on Linux. Now, ecryptfs allows each path component of a filename to be only up to 140-ish characters, because it has to encrypt that filename, add some encryption info to it, and store it inside another filename on a backing ext4 filesystem (which limits path components to 255 characters). - -Several times now I've added a bunch of stuff to my annex on the NTFS checkout, where path components are allowed to be longer than 140 characters, synced it over to my other annex checkout on ecryptfs, and then had Git Annex fail during the sync, trying to create these empty symlinks with path components too long for the filesystem it is on. When in this state, I don't really know how to fix it. I can't just "git mv" the offending file to a valid name, both because "git mv" needs the source file to be on disk in the first place and because the failed "git checkout" leaves my repo thinking it has thousands of untracked files (because some stuff did get created, but git refused to officially move to the commit it was trying to check out, because the checkout failed). - -I am looking for a solution for this inside Git Annex. The simplest thing, I think, would be to set a max path component length for the whole set of repos, so I could get an error when I go to "git annex add" on the NTFS checkout that the filenames being added are too long for some of the repos that will eventually want to check them out. Is it possible to do this with a pre-commit hook somehow? - -The next simplest thing would be for Git Annex to look at the filesystem it is running on and do something smarter than exploding and leaving my repo in a weird out-of-sync state if some of the filenames it wants to create can't be created. Maybe it should fail the sync earlier, in Git Annex itself rather than in git checkout. Maybe it should just leave those files out of the checkout, or force/allow me to rename them right then. - -The most complex thing would be to somehow make it work anyway and check out the symlinks under different, valid names. Perhaps it could just truncate those path components in the symlink view? There's already support for different metadata views; this would be sort of like that. You get a special view of the repo subject to the constraints of your filesystem. - -> [[rejected|done]] --[[Joey]] diff --git a/doc/todo/Copy-on-Write__47__reflink__47__clonefile_support_on_macOS.mdwn b/doc/todo/Copy-on-Write__47__reflink__47__clonefile_support_on_macOS.mdwn deleted file mode 100644 index cc35f33847..0000000000 --- a/doc/todo/Copy-on-Write__47__reflink__47__clonefile_support_on_macOS.mdwn +++ /dev/null @@ -1,11 +0,0 @@ -Thank you for your previous message here: [[/todo/Copy-on-Write__47__reflink_support_for_unlocked_files/]]. - -I indeed discovered it was done for Linux after I posted it, but macOS with APFS also supports reflinks, under the `-c` (clonefile) option, which does the same as the `--reflink=always` option on Linux. - -The biggest problem is that there is no auto option, but in theory it’s possible to first try the clone option, and then fall back if it doesn’t work, although that would be inefficient. - -I can try to code something, although I am not that proficient in Haskell, or run test code if needed on macOS 12. - --- Lena Wildervanck - -> [[dup|done]] --[[Joey]] diff --git a/doc/todo/Copy-on-Write__47__reflink__47__clonefile_support_on_macOS/comment_1_81bf4ac2fe810cc4e7a21c8bb4d0f15a._comment b/doc/todo/Copy-on-Write__47__reflink__47__clonefile_support_on_macOS/comment_1_81bf4ac2fe810cc4e7a21c8bb4d0f15a._comment deleted file mode 100644 index cf619af016..0000000000 --- a/doc/todo/Copy-on-Write__47__reflink__47__clonefile_support_on_macOS/comment_1_81bf4ac2fe810cc4e7a21c8bb4d0f15a._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-11-21T19:11:52Z" - content=""" -This todo aleady exists for the same thing: - -"""]] diff --git a/doc/todo/Copy-on-Write__47__reflink_support_for_unlocked_files.mdwn b/doc/todo/Copy-on-Write__47__reflink_support_for_unlocked_files.mdwn deleted file mode 100644 index bc49a7512f..0000000000 --- a/doc/todo/Copy-on-Write__47__reflink_support_for_unlocked_files.mdwn +++ /dev/null @@ -1,3 +0,0 @@ -Some filesystems, like BTRFS or ZFS have copy on write support, which means that if files are copied in a special way they share sectors until one of the files is edited. Implementing support would make files take up less space if they are in unlocked mode and not significantly edited. - -> Already [[done]]. I did add a mention of it to the tip. --[[Joey]] diff --git a/doc/todo/Copy-on-Write__47__reflink_support_for_unlocked_files/comment_1_52db92711e28abb0eb7a346646b91573._comment b/doc/todo/Copy-on-Write__47__reflink_support_for_unlocked_files/comment_1_52db92711e28abb0eb7a346646b91573._comment deleted file mode 100644 index 15052ef9d7..0000000000 --- a/doc/todo/Copy-on-Write__47__reflink_support_for_unlocked_files/comment_1_52db92711e28abb0eb7a346646b91573._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="lena.wildervanck@4b6aac156870f72a36b090e210e4747f702b69cb" - nickname="lena.wildervanck" - avatar="http://cdn.libravatar.org/avatar/8a423528476e6f6dc6a06588b5bbf457" - subject="comment 1" - date="2022-11-13T19:03:48Z" - content=""" -Oh, nvm, there already seem to be some references in the code to it. - -Maybe it should be added that it does that under [[https://git-annex.branchable.com/tips/unlocked_files/]] under using less disk space? - -Also, macOS uses a `-c` flag instead of `reflink`, according to [the cp man page](https://www.unix.com/man-page/mojave/1/cp/). -"""]] diff --git a/doc/todo/Extend_merge_conflict_resolution_to_git_conflicts.mdwn b/doc/todo/Extend_merge_conflict_resolution_to_git_conflicts.mdwn deleted file mode 100644 index 67e95edaa7..0000000000 --- a/doc/todo/Extend_merge_conflict_resolution_to_git_conflicts.mdwn +++ /dev/null @@ -1,13 +0,0 @@ -Currently, git-annex handles merge conflicts of annexed files well. It makes two differently named versions of the conflicting files (A more descriptive naming scheme including date and commit hash would be nice, but that's a different topic). - -However, a conflict of git-tracked files is not resolved, leading to merge conflicts being left open after a `git annex sync` (thus leaving file contents with the weird merge markers `<<<<<` etc., breaking file contents.) - -Could `git annex [resolve]merge` be extended to also handle those git conflicts and leave the repo in a clean, merged state after `git annex sync|pull|assist`? - -To not confuse users, it could be opt-in (`git annex --set annex.resolvegitmerge true`), but maybe the default for `git annex assist`? - -On conflict, it would remove the conflicting file and instead create two versions with suffixes either like annexed files or better the commit hash and/or date. - -Submodule conflicts can't be resolved like this, in that case I would use the most recent commit of the two in question. - -> [[rejected|done]] --[[Joey]] diff --git a/doc/todo/Extend_merge_conflict_resolution_to_git_conflicts/comment_1_c08e7f51503f103c447064a01b085783._comment b/doc/todo/Extend_merge_conflict_resolution_to_git_conflicts/comment_1_c08e7f51503f103c447064a01b085783._comment deleted file mode 100644 index 887b4d7111..0000000000 --- a/doc/todo/Extend_merge_conflict_resolution_to_git_conflicts/comment_1_c08e7f51503f103c447064a01b085783._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-06-05T18:49:43Z" - content=""" -I think this would violate least surprise. Users expect git-annex -to behave like git, and that means merge conflicts like git handles them, -when the files are text files tracked by git. - -Consider that, if the file is a config file, applying git-annex's -resolvemerge to it would make 2 files with different names be in the -repository. So the config file would stop taking effect for whatever it was -supposed to configure. It would be easy for the user to miss that, because -the repository would not be left in a conflicted state. - -When you check a file into git rather than git-annex, you are -choosing to have git manage that file normally. If you want the git-annex -behavior, you can check the file into git-annex. -"""]] diff --git a/doc/todo/Fsck_remote_files_in-flight.mdwn b/doc/todo/Fsck_remote_files_in-flight.mdwn deleted file mode 100644 index 93bd87f2a0..0000000000 --- a/doc/todo/Fsck_remote_files_in-flight.mdwn +++ /dev/null @@ -1,7 +0,0 @@ -When fsck'ing a remote repo, files seem to be copied from the remote to a local dir (thus written to disk), read back again for checksumming and then deleted. - -This is very time-inefficient and wastes precious SSD erase cycles which is especially problematic in the case of special remotes because they can only be fsck'd "remotely" (AFAIK). - -Instead, remote files should be directly piped into an in-memory checksum function and never written to disk on the machine performing the fsck. - -> [[done]] per my comments --[[Joey]] diff --git a/doc/todo/Fsck_remote_files_in-flight/comment_1_1a70ae7c9821d664ddf72fd4c431be29._comment b/doc/todo/Fsck_remote_files_in-flight/comment_1_1a70ae7c9821d664ddf72fd4c431be29._comment deleted file mode 100644 index 7c6f486e88..0000000000 --- a/doc/todo/Fsck_remote_files_in-flight/comment_1_1a70ae7c9821d664ddf72fd4c431be29._comment +++ /dev/null @@ -1,28 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2021-04-14T17:07:50Z" - content=""" -Only some remotes support checksums in-flight; this recently includes -downloads from other git-annex repositories over ssh. Progress -on that front is being tracked at - -Most special remotes can't yet, but that should change eventually -for at least some of them. - -I've made fsck notice when content was able to be verified as part of a -transfer, and avoid a redundant checksum of them. - -What I've not done, and don't think I will be able to, is make the file -not be written to disk by fsck in that case. Since the `retrieveKeyFile` -interface is explicitly about writing to a file on disk, it would take ether -a whole separate interface being implemented for all remotes that avoids -writing to the file when they can checksum in flight, or it would need -some change to the `retrieveKeyFile` interface to do the same. - -Neither seems worth the complication to implement just to reduce disk IO in -this particular case. And it seems likely that, for files that fit in -memory, it never actually reaches disk before it's deleted. Also if this is -a concern for you, you can I guess avoid fscking remotes too frequently or -use a less fragile medium? -"""]] diff --git a/doc/todo/Fsck_remote_files_in-flight/comment_2_3baff888fbc3068571cdc9fee73fbe36._comment b/doc/todo/Fsck_remote_files_in-flight/comment_2_3baff888fbc3068571cdc9fee73fbe36._comment deleted file mode 100644 index 35c53cd76c..0000000000 --- a/doc/todo/Fsck_remote_files_in-flight/comment_2_3baff888fbc3068571cdc9fee73fbe36._comment +++ /dev/null @@ -1,29 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2021-10-14T15:55:14Z" - content=""" -Checksum during transfer is now implemented for as many remotes as it -reasonably can be, which is almost all of them. But not 100% of all -remotes in all circumstances. And there's no way to know if a remote -will support it before doing the transfer. - -To avoid changing the API, it occurs to me that retrieveKeyFile could be -passed `/dev/null`. But any remote that does not support resuming and tries -to overwrite the existing destination file would fail. - -Also some kinds of remotes download to the file in one process or thread -and while the download is happening, git-annex checksums the file as new -data appears in it. External special remotes in particular do this. -That would break with `/dev/null` too. - -Putting the temp file on some other medium seems like the only way to -address this. If there were a config of a directory to use, you could point -it at a disk rather than the SSD, or even at a ram disk, if you have -sufficient memory. Unsure if it's worth adding such an option though, -probably few people would use it. And cloning the repository onto the other -medium and running the remote fsck from there would have the same result -without needing an option. - -I'm inclined to close this, since I don't think it can be addressed. -"""]] diff --git a/doc/todo/Having___39__git_annex_sync__39___optionally_add.mdwn b/doc/todo/Having___39__git_annex_sync__39___optionally_add.mdwn deleted file mode 100644 index 5a357bb233..0000000000 --- a/doc/todo/Having___39__git_annex_sync__39___optionally_add.mdwn +++ /dev/null @@ -1,38 +0,0 @@ -Hey joey, - -Currently, there is no one-command-to-rule-them-all to actually 'get the repo synced entirely'. Current workarounds: - -[[!format bash """ -# plain git-annex -git annex add -(git add -A) # for the weird filter issues, see https://github.com/datalad/datalad/issues/7268 -git annex sync --content - -# with DataLad -datalad save # no way around the weird filter issues currently -git annex sync --content -"""]] - -Whenever I introduce new people to git annex, it's always a pain point to explain these multiple steps. It would be awesome to have just one command that syncs everything. Especially for beginners. - -`git annex sync` already does quite a bit including `git add`ing and `git commit`ting changes to tracked files. However it doesn't add new files. It would be awesome if `git annex sync` could optionally also do `git annex add`: - -[[!format bash """ -# configure 'git annex sync' to run 'git annex add' before as well (opt-in) -git annex config --set annex.syncadd true -# have it sync the content as well -git annex config --set annex.synccontent true - -# From then on, 'git annex sync' does a *real* sync: everything that's new/changed here gets pushed elsewhere and new changes get pulled as well. -git annex sync - -# This would then basically be a poor-mans-git-annex-assistant 😛 -while true;do git annex sync --content;done -"""]] - - -What do you think? - -Cheers, Yann - -> [[done]] as `git-annex assist` --[[Joey]] diff --git a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_10_6d9ef10f9534c7b95ca58913cfe3b5fd._comment b/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_10_6d9ef10f9534c7b95ca58913cfe3b5fd._comment deleted file mode 100644 index be5c98392b..0000000000 --- a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_10_6d9ef10f9534c7b95ca58913cfe3b5fd._comment +++ /dev/null @@ -1,31 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 10""" - date="2023-05-18T17:45:44Z" - content=""" -Implementing `git-annex assist` now... - -A tricky point is, should it add files only in the current directory and -below, or all files in the repository? Note that the assistant can be run -in a directory and it will only add changed/new files in that directory, -although it can receive pulls that change files in other directories -(and will then download those files content). - -OTOH, `git-annex sync` commits all changes, not only those in the -current directory. (The assistant does in some circumstanges commit -changes made outside the current directory. Its behavior is a bit -inconsistent in this area.) - -So I think it makes sense for `git-annex assist` to only add files in the -current directory by default. (Of course an option like -A could be added -later.) - -And while I'm a bit ambivilant about it, I'm making it commit all staged -changes, not only those in the current directory. As well as following the -behavior of `git-annex sync` and to an extent the assistant, it seems to -me that if you run `git-annex add ../foo; git-annex assist`, you are -intentionally building up a commit that includes file "foo". The same -as if you ran `git-annex add ../foo; git-annex add .` ... If you're not, -and you care about what files get added in what commit, you can of course -commit manually. -"""]] diff --git a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_11_25be855ca688e37644fa86357b96b3d2._comment b/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_11_25be855ca688e37644fa86357b96b3d2._comment deleted file mode 100644 index 898518f353..0000000000 --- a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_11_25be855ca688e37644fa86357b96b3d2._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="👍 git annex assist" - date="2023-05-19T05:54:43Z" - content=""" -Cool, joey! - -I personally would like `assist` to add all changes in the entire repo by default. For beginners it's always a hassle to know what directory one is in and if one opens a terminal in a subfolder via a file manager, then run `git annex assist`, it won't sync all changes. - -But I see that this would be inconsistent with `git annex add`... On the other hand, `assist` should help you easily sync the repo state without typing too much. One can always do `git annex assist .` for only this directory. A `-A` option is also an option, but effectively one would need to use it every time. As you said in the other comment - if you want more control, use the lower level commands. The assistant makes syncing a no-brainer, `git annex assist` should do so, too. - -That the assistant can operate on only a subdir (do people do this?) itself is again an inconsistency then... A config like `annex.assistaddall` could be introduced, but meh... -"""]] diff --git a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_12_8327e0a3f3d4bf35c52b1f2f4089dcbc._comment b/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_12_8327e0a3f3d4bf35c52b1f2f4089dcbc._comment deleted file mode 100644 index 5a2a2033d8..0000000000 --- a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_12_8327e0a3f3d4bf35c52b1f2f4089dcbc._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 12""" - date="2023-05-19T18:37:28Z" - content=""" -I'm also not too happy with the inconsistency of assist committing all staged -changes and syncing all file contents, but only adding files in the cwd. - -I suppose that consistency with the assistant doesn't really matter. The -assistant's behavior when ran in a subdirectory is surprising, -inconsistent, and undocumented. - -So I'm going to change assist to add all files. Except when -C is used, -then only add files in the specified directory. -"""]] diff --git a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_13_8b150a2a4b9b0de8039ee8995c01bf8c._comment b/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_13_8b150a2a4b9b0de8039ee8995c01bf8c._comment deleted file mode 100644 index ad680ade0b..0000000000 --- a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_13_8b150a2a4b9b0de8039ee8995c01bf8c._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 13" - date="2023-05-20T06:03:30Z" - content=""" -Sounds good! Maybe also a `-m` short form of `--message` for less typing. Consistent with sync and datalad save. - -"""]] diff --git a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_14_0156a70246f19beeee6ce5cfb510de86._comment b/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_14_0156a70246f19beeee6ce5cfb510de86._comment deleted file mode 100644 index dc07b3495d..0000000000 --- a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_14_0156a70246f19beeee6ce5cfb510de86._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 14""" - date="2023-05-23T15:46:27Z" - content=""" -Oh, this new command and sync always supported -m. I've added it to the man -page. -"""]] diff --git a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_15_b0f6d52ef7a94badd572c6cfa759efb3._comment b/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_15_b0f6d52ef7a94badd572c6cfa759efb3._comment deleted file mode 100644 index 8972a3b1b7..0000000000 --- a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_15_b0f6d52ef7a94badd572c6cfa759efb3._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="ewen" - avatar="http://cdn.libravatar.org/avatar/605b2981cb52b4af268455dee7a4f64e" - subject="Breaking change to "sync"" - date="2023-07-12T10:21:03Z" - content=""" -Via a comment on [my bug about the new `sync` warning suddenly appearing in 10.20230626](https://git-annex.branchable.com/bugs/Changing_sync_to_include_content_breaks_UX/?updated) I see that this fairly hidden discussion seems to have been the rationale for completely changing what \"sync\" does \"because some users expect it to do everything\". - -At the beginning of the design I might have agreed with you about \"what the sync command should do\" (and having another, eg, \"metasync\" command for the smaller version). But changing a fundamental command, a decade later, to do something different, based entirely on a \"wouldn't it have been nice (for some use cases) if...\" seems quite a stretch. - -As per the other bug, if you want to implement new behaviour for such a fundamental command (\"git annex sync\" is something one runs pretty constantly if using git annex actively from the command line) then it'd be best to implement it in another command name, instead of dramatically changing an existing one. (The other thread has a few more suggestions; I personally still like \"fullsync\" for the expanded version.) - -Either way, such a major -- behaviour breaking -- change needs to be much better documented than \"discussion in a bug that a few people saw\", and a note in passing in a changelog (which people have to find by noticing strange new output in probably scripted git annex usage). - -Ewen - -PS: I too think it's a terrible idea to have git annex default to auto-adding any files it can find. Maybe that makes some \"Dropbox-like\" use cases nicer, but it also entirely breaks other long standing use cases. -"""]] diff --git a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_16_6c4c1541a146c66c11ebe6abaf7f1810._comment b/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_16_6c4c1541a146c66c11ebe6abaf7f1810._comment deleted file mode 100644 index 5b4329eae5..0000000000 --- a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_16_6c4c1541a146c66c11ebe6abaf7f1810._comment +++ /dev/null @@ -1,24 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="Clarification" - date="2023-07-12T11:29:41Z" - content=""" -@ewen for the record: - -I never suggested that `git annex sync` auto-adds new files by default, see [my comment above](http://git-annex.branchable.com/todo/Having___39__git_annex_sync__39___optionally_add/#comment-37c3eb24df11d85c07b78c0297447c45): - -> Exactly, I also would never want git annex sync to do the adding by default - strictly as an opt-in configuration. - -As `git annex sync` already had so many options for configuring its behaviour, I thought having one more that runs `git annex add` in the beginning wouldn't hurt. I never suggested changing `git annex sync`'s default behaviour. Purely opt-in. Same for the existing options like `annex.synccontent` etc. - -Changing `git annex sync`'s default behaviour of now syncing content was joey's idea, not mine. - -> if you want to implement new behaviour for such a fundamental command - -There is already `git annex assist` to do exactly that. See above or the changelogs. - -Just to clarify. Please read the discussions you're referring to thoroughly before claiming things between the lines. 🙂 - -Now back to constructive discussion. -"""]] diff --git a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_1_2160800299fff5dc920847bec1ab0220._comment b/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_1_2160800299fff5dc920847bec1ab0220._comment deleted file mode 100644 index 5fdfb6bce5..0000000000 --- a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_1_2160800299fff5dc920847bec1ab0220._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="adding all files or only in current directory?" - date="2023-05-03T14:23:04Z" - content=""" -`git annex add` only adds files in the current directory and below and has no flag to do a `git add -A` equivalent. For the purpose of making `git annex sync` also adding all files (e.g. with `git annex sync --add` or with prior `git annex config --set annex.syncadd true`), such an `git annex add --all|-A` option would be handy (which would also be used by `git annex sync --add`). It would also make `git annex add` more consistent with `git add`. -"""]] diff --git a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_2_eeb1bb0b1a02a74291daa41e9a8eb725._comment b/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_2_eeb1bb0b1a02a74291daa41e9a8eb725._comment deleted file mode 100644 index 0e1d8ee110..0000000000 --- a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_2_eeb1bb0b1a02a74291daa41e9a8eb725._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="Git Alias for a 'full sync'" - date="2023-05-15T07:31:41Z" - content=""" -This alias pretty much does what I mean: - -[[!format perl \"\"\" -# set the alias -git config --global alias.sync '!sh -xc '\"'\"'cd \"$(git rev-parse --show-toplevel)\";git annex add;git add -A;git annex sync'\"'\"'' -# This then goes into the repo root, adds stuff with git annex, adds the rest git-annex didn't add and syncs with git annex -git sync -\"\"\"]] - - -"""]] diff --git a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_3_a49dcb3963a986c530571f69fa9326b4._comment b/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_3_a49dcb3963a986c530571f69fa9326b4._comment deleted file mode 100644 index 62b9ccbd36..0000000000 --- a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_3_a49dcb3963a986c530571f69fa9326b4._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2023-05-15T20:09:30Z" - content=""" -I don't think this can possibly become the default behavior, because plenty -of users will have temporary files that they don't want to add (and haven't -gitignored), or even sensitive files that happen to be in the same -repository that it would be a security hole level surprise for `git-annex -sync` to start adding, when it has not before. - -For the same reason, `git-annex config` couldn't be used to make this a -default `git-annex sync` behavior. A local git config to enable it would be -ok. - -It would would need to look at annex.largefiles, and add small files to git -directly. -"""]] diff --git a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_4_cf4ed68e05fd7d553a314746b444b7c0._comment b/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_4_cf4ed68e05fd7d553a314746b444b7c0._comment deleted file mode 100644 index dbe999bfe8..0000000000 --- a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_4_cf4ed68e05fd7d553a314746b444b7c0._comment +++ /dev/null @@ -1,37 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 4" - date="2023-05-15T21:13:34Z" - content=""" -Exactly, I also would never want `git annex sync` to do the adding by default - strictly as an opt-in configuration. - -I see your point that people might have sensitive files in a repo that they don't have `.gitignore`d and would not want to get pushed. However, I argue: - -- If you clone a git annex repo and do `git annex sync` on it, you're pretty much already trusting that repo and the remotes. You either already have push access (that doesn't happen easily unless you're in control of the repo) or you don't, but then it's not a problem when you add your sensitive files because your changes won't ever get synced back. Even if they do at some later point: You `git annex drop --force` them (or just `git rebase -i`) to remove them from the repo entirely and worst case is that the file metadata is leaked back into the repo collection. Again, something I'd consider OK if you trust the repo. -- When you trust a git annex repo, you also trust its `git annex config` settings like `synccontent` and consent with it. -- Thus, if that repo has a `git annex config --set annex.syncadd true` setting that has `git annex sync` also do a prior `git annex add`, that's fine. - -I think that only having a local `git config annex.syncadd true` setting is basically the same as having a customized alias like mine above, so that doesn't really improve the situation: in both cases users of the repo need to do local configs. A `git annex config` will be stored in the repo and be immediately effective for everybody using it - a major benefit for collaboration, for which git annex is a golden tool. If I understand correctly, any `annex.*` config can be overridden locally with a `git config annex.*` setting (right?). So cautious users can have `git config --global annex.syncadd false` to prevent `git annex sync` from ever adding files, even if the repo is configured to do so. - -If the `git config annex.syncadd true` setting would just literally run `git annex add`, it automatically obeys the largefiles settings (also from the .gitattributes file). - -All in all I'd consider the `syncadd` a matching feature considering how much `git annex sync` is already doing: committing, pulling, merging, pushing all over the place - only the adding it doesn't do. - -The more behaviour can be configured in the repo directly, the better the out-of-the-box experience for (inexperienced) end users of a repo will be. I maintain many git annex repos and collaborate with different students like this on scientific projects. Together with [default preferred content expressions](https://git-annex.branchable.com/todo/Setting_default_preferred_content_expressions/) the workflow would be extremely straight-forward: - -[[!format bash \"\"\" -# get the repo -git clone URL/repo -cd repo -# ”whenever you're done, run”: -git annex sync -# - auto-enables common remotes -# - auto-sets preferred content expressions (to prevent the repos from uselessly downloading EVERYTHING - just what's sensible by default) -# - auto-adds/commits/pulls/merges/pushes changes -# - auto-syncs content around -\"\"\"]] - -Sensible settings for largefiles in `.gitattributes` or the `git annex config` can be set by those who understand it, others just do `git annex sync` and are done with it. - -"""]] diff --git a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_5_3a2fe8d0c0dd6f5066370086c0f03300._comment b/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_5_3a2fe8d0c0dd6f5066370086c0f03300._comment deleted file mode 100644 index 559780738f..0000000000 --- a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_5_3a2fe8d0c0dd6f5066370086c0f03300._comment +++ /dev/null @@ -1,32 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2023-05-16T16:32:51Z" - content=""" -sync's intent is to replicate as close as possible the following common -git workflow: - - first stage some changes, and then run: - git commit -m foo - git pull - git push - -Since git-annex has some complications involving pulling and pushing -the git-annex branch, and transferring the content, it adds a learning -curve that was too high. Giving users one new command to learn minimizes -the learning curve. (It's unfortunate it didn't originally send content, -and annex.synccontent aims to fix that oversight and I hope it may eventually -become default.) - -For git-annex sync to default to doing something not in the above -pull+commit+push workflow would be surprising, because that's the workflow -users have been told it handles. - -Notice that the git-annex assistant will happily add all files to the -repository and send their content. That's because it's *not* targeting -users who expect to use that git workflow. So it's not surprising that it -does what it does. - -In conclusion, if having this behavior enabled by git config is not useful, -then it does not belong in `git-annex sync`. It could go in a new command. -"""]] diff --git a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_6_2e0ad0cdb9d813d0fbc514c1642669ca._comment b/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_6_2e0ad0cdb9d813d0fbc514c1642669ca._comment deleted file mode 100644 index e05874f65b..0000000000 --- a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_6_2e0ad0cdb9d813d0fbc514c1642669ca._comment +++ /dev/null @@ -1,21 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 6""" - date="2023-05-16T17:05:35Z" - content=""" -I also generally feel like `git-annex sync` was probably a bad conflation -of 3 git commands into 1 command when there could have just been `git-annex -pull` and `git-annex push`. The fact that it conflates several things makes -users think of it as just a big "does all the things" command, which makes -users want it to do more things. - -It is not too late to split it up, and eventually deprecating it would be a -good path to making annex.synccontent default. - -Update: Implemented `git-annex pull` and `git-annex push`, although I have -no plans yet to deprecate `git-annex sync`. - -Update: I found a better path to transition to sync --content by -default, and have started that transition. `git-annex pull/push` sync -content by default already. -"""]] diff --git a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_7_bb6b5503ec5f699749fa8e3ffce59aab._comment b/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_7_bb6b5503ec5f699749fa8e3ffce59aab._comment deleted file mode 100644 index df6d21fe52..0000000000 --- a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_7_bb6b5503ec5f699749fa8e3ffce59aab._comment +++ /dev/null @@ -1,23 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 7""" - date="2023-05-17T17:24:03Z" - content=""" -Brainstorming some names for a new command that is `git add -A` + sync... - -`git-annex publish` kind of implies more exposure of local files to others -than just syncing. But it more implies sending data, not also receiving it. - -`git-annex addsync` is clear what it does, but not very memorable. - -`git-annex assist` has a nice analogy to the `git-annex assistant`, -which is close in behavior. Users who don't know about the assistant -will miss the analogy though. "assist" also suggests git-annex will take -care of everything, even more broadly than sync. - -`git-annex share` also implies more exposure of local files than git-annex -sync, and sharing goes both ways so it's better than `git-annex publish`. - -I'd be inclined toward `git-annex share` or `git-annex assist`, but welcome -better ideas. -"""]] diff --git a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_7_e6e6b344d533de57ce7661b5f1ada027._comment b/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_7_e6e6b344d533de57ce7661b5f1ada027._comment deleted file mode 100644 index 4941288362..0000000000 --- a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_7_e6e6b344d533de57ce7661b5f1ada027._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 7" - date="2023-05-17T10:41:18Z" - content=""" -Cool, thanks joey! - -If you want to go as far as deprecating `git annex sync` (a huge change in habits for git-annex users if that one goes away), I wonder how adding the `annex.syncadd` config (a strictly opt-in consensus for users of a repo) is too much of a disruption 😅 But okay, you're the maintainer. - -I still think that having **one command** that does what the assistant does, but as a one-shot manual step, would benefit git-annex immensely. I recently tried running the assistant on my SailfishOS phone (https://fosstodon.org/@nobodyinperson/110203412944552425) and realised that having the assistant running in the background all the time is not ideal due to e.g. the power consumption - maybe due to the permanent SSH connection to the remotes. I am now using a systemd service that basically does `git annex add;git annex sync`. Would be awesome and certainly good for the understanding of new users if there was a command that basically one-steps the assistant - This is something I as a previous Syncthing user was dearly missing: one-stepping the syncing to see what's the bottleneck/problem/what it does exactly, etc. If you'd rather put this into a new command, I'm fine with that, though I still think `git annex sync` is a perfect candidate also due to the name. -"""]] diff --git a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_9_e128886c09b9ca2127ff540c2af0beda._comment b/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_9_e128886c09b9ca2127ff540c2af0beda._comment deleted file mode 100644 index 3e523b0482..0000000000 --- a/doc/todo/Having___39__git_annex_sync__39___optionally_add/comment_9_e128886c09b9ca2127ff540c2af0beda._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 9" - date="2023-05-18T08:40:37Z" - content=""" -Good suggestions, I'd prefer `assist`, as it'll be one step of what the assistant does. I'll also through `git annex update` into the ring. `update`ing goes both ways, `share`ing also sounds more like you getting your stuff out there, not necessarily others' stuff to you. -"""]] diff --git a/doc/todo/Improving_diffdriver_--text.mdwn b/doc/todo/Improving_diffdriver_--text.mdwn deleted file mode 100644 index 4ad12ff20c..0000000000 --- a/doc/todo/Improving_diffdriver_--text.mdwn +++ /dev/null @@ -1,32 +0,0 @@ -Thank you joey for `git annex diffdriver --text`, that is a big step towards easier diffing of annexed files. The following is now a copy-paste solution to 'make git diff work with git annex': - -``` -echo '* diff=annextextdiff' >> .git/info/attributes && git config diff.annextextdiff.command "git annex diffdriver --text" -``` - -This however then has `git diff` use git-annex' diffing mechanism for *all* files, including normal git-tracked files. There probably is no gitattributes-way of applying a diff command only to annexed files, right? - -## Customizing `diff` options - -Apparently, `git annex diffdriver --text` uses the system's `diff` command and doesn't (allow to) give it any specific options. The below points could be worked around by having something like `git annex diffdriver --text --diffopts='--color=auto'` so that the user can customize the `diff` invocation. Alternatively you could introduce an environment variable like `GIT_ANNEX_DIFFDRIVER_DIFF_OPTIONS` or shorter `GIT_ANNEX_DIFF_FLAGS` that could also be used to temporarily diff *that one file* with specific options. - -## Coloring the output - -How about passing the (sanitized, `false`→`never` and `true`→`auto`) `git config color.diff` (and fallback `color.ui`) setting as the `diff --color=...` option? The experience would then match the users configured expectation. - - -## Handling all files as text - -`diff` detects some files as binary, although it can make sense to text-diff them (e.g. PDFs), just to get an impression of the changes. `diff` (and `git diff` as well, AFAIK without any good workaround without an external driver again) then just displays the unhelpful message that the 'binary files differ' (you don't say...🙄). - -How about having `git annex diffdriver --text` always using `diff --text`? That would deviate from the usual `git diff` behaviour, but I argue that: - -- a) People *explicitly* configure (a subset of) files in `.gitattributes` to be diffed with `git annex diffdriver --text`. -- b) If they wouldn't care about the diff between files, then they wouldn't configure it. -- c) Tracking binary(-like) files is something git-annex is explicitly designed for, it makes sense that git just skips over those, but git-annex could add value here - -Thanks again a ton for git-annex, the Tübix2023-Workshop was well appreciated and lots of fun. 👍 - -Yann - -> [[done]] --[[Joey]] diff --git a/doc/todo/Improving_diffdriver_--text/comment_1_a38b0f30f9b4011d703e70c093f139ab._comment b/doc/todo/Improving_diffdriver_--text/comment_1_a38b0f30f9b4011d703e70c093f139ab._comment deleted file mode 100644 index a116744826..0000000000 --- a/doc/todo/Improving_diffdriver_--text/comment_1_a38b0f30f9b4011d703e70c093f139ab._comment +++ /dev/null @@ -1,26 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-07-05T18:55:33Z" - content=""" -If your annexed files have a common extension you can match them in -.gitattributes that way. - -I've added support for passing options for diff. Eg: - - git annex diffdriver --text -- --color --text -- - -As for passing --color automatically, I don't know if every diff(1) supports ---color, and this would be more complexity. - -I doubt that most users would appreciate diff --text by default for large -binary files. That is really getting into territory that an external -diff driver program can handle much better. Eg, if you're diffing jpegs and -also zip files, you can write a diff driver program that lists the contents of -zip files and passes that to diff, and that uses imagemagick -`compare -compose` or -to show a visual diff between two jpegs. - -The point off the --text option is that sometimes you know you have plain -text and so writing such a program is not necessary. -"""]] diff --git a/doc/todo/Improving_diffdriver_--text/comment_2_2a59862281c1be5ee92912912c2045e9._comment b/doc/todo/Improving_diffdriver_--text/comment_2_2a59862281c1be5ee92912912c2045e9._comment deleted file mode 100644 index 98c5696b48..0000000000 --- a/doc/todo/Improving_diffdriver_--text/comment_2_2a59862281c1be5ee92912912c2045e9._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 2" - date="2023-07-06T05:44:57Z" - content=""" -Wonderful, thank you joey! -"""]] diff --git a/doc/todo/Long_Running_Filter_Process.mdwn b/doc/todo/Long_Running_Filter_Process.mdwn deleted file mode 100644 index 2f42c09724..0000000000 --- a/doc/todo/Long_Running_Filter_Process.mdwn +++ /dev/null @@ -1,24 +0,0 @@ -Hello, - -while reading the release notes of git 2.11 I noticed a cool new feature has been merged: - -> If the filter command (a string value) is defined via -> `filter..process` then Git can process all blobs with a -> single filter invocation for the entire life of a single Git -> command. - -see the [git documentation][1]. - -This has been developed in the context of git-lfs (see [PR 1382] [2]). - -If I understand correctly how it works this could speed up v6 repos. Looking at the history/website -of git-annex there doesn't seem to be yet any work on this so I though it was worth calling the -attention on the feature. - -Thanks a lot for all the work on git-annex, it's a really amazing project! The more I study it the more cool features I discover :) - - -[1]: https://github.com/git/git/blob/v2.11.0/Documentation/gitattributes.txt#L384 -[2]: https://github.com/git-lfs/git-lfs/pull/1382 - -> [[done]] --[[Joey]] diff --git a/doc/todo/Long_Running_Filter_Process/comment_1_f155ffc7dbd074964dd53165274ec8a0._comment b/doc/todo/Long_Running_Filter_Process/comment_1_f155ffc7dbd074964dd53165274ec8a0._comment deleted file mode 100644 index 34d05d7711..0000000000 --- a/doc/todo/Long_Running_Filter_Process/comment_1_f155ffc7dbd074964dd53165274ec8a0._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2016-12-13T15:57:05Z" - content=""" -Yes, this will make [[smudge]] faster when eg checking out a lot of working -tree changes. I will need to add support for it. -"""]] diff --git a/doc/todo/Long_Running_Filter_Process/comment_3_24d89d0e8eb2da6e43d107caa71e042b._comment b/doc/todo/Long_Running_Filter_Process/comment_3_24d89d0e8eb2da6e43d107caa71e042b._comment deleted file mode 100644 index f37fdf21f3..0000000000 --- a/doc/todo/Long_Running_Filter_Process/comment_3_24d89d0e8eb2da6e43d107caa71e042b._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2018-08-13T20:24:02Z" - content=""" -The "filterdriver" branch implements support for these. - -However, it's actually slower than the old interface, because the new -interface requires git-annex read the whole file content from git when -adding a file, and the old interface let it not read any content. - -Since the new interface does have capabilities, a new capability could -prevent schepping the content over the pipe, and let the filter driver -refer to the worktree file instead, and respond with the path of a file. -This would be similar to my old patch set for the old interface. -"""]] diff --git a/doc/todo/Long_Running_Filter_Process/comment_4_96c20518a6bd2806fed3646331b3ca1c._comment b/doc/todo/Long_Running_Filter_Process/comment_4_96c20518a6bd2806fed3646331b3ca1c._comment deleted file mode 100644 index 401d81ed54..0000000000 --- a/doc/todo/Long_Running_Filter_Process/comment_4_96c20518a6bd2806fed3646331b3ca1c._comment +++ /dev/null @@ -1,21 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2018-10-26T16:21:28Z" - content=""" -While `git add` would be a lot slower when using this interface to add -large files, it would make `git checkout` and other commands that update -the work tree a lot faster. - -Since the smudge filter is not providing git with the file content any more, -using filterdriver would avoid git running many git-annex smudge processes, -greatly speeding up large checkouts. - -Unfortunately, `git annex smudge --update` ends up running the smudge filter -on all files that the clean filter earlier acted on, so even if filterdriver were -used to speed up the clean filter, there would still be one process spawned per -file for the smudge filter. - -So some interface improvement is needed before git-annex can usefully use -this. -"""]] diff --git a/doc/todo/Make_addunlocked_configurable_like_largefiles.mdwn b/doc/todo/Make_addunlocked_configurable_like_largefiles.mdwn deleted file mode 100644 index a9719ec87d..0000000000 --- a/doc/todo/Make_addunlocked_configurable_like_largefiles.mdwn +++ /dev/null @@ -1,45 +0,0 @@ -Hi joey, - -How about making `annex.addunlocked` configurable similar to `annex.largefiles` so different file types can be configured to be added unlocked and others locked? - -The following illustrates a use case: - -[[!format bash """ -#!/bin/sh -x -# make a repo -export LC_ALL=C.UTF-8 -chmod -R +w addunlocked;rm -rf addunlocked -git init addunlocked -cd addunlocked -git annex init -# configure *.txt files to be added unlocked so they stay editable -echo '*.txt annex.addunlocked=true' > .gitattributes -git add .gitattributes -git commit -m "Add .gitattributes" -# make a note -echo bla > notes.txt -git annex add -git annex sync -m "add notes.txt" -# However editing doesn't work as it was added locked -echo blubb >> notes.txt # fails as it was not added unlocked -"""]] - -Probably not all matching options (like `copies` for example) make sense or work at all in this context, but at least `include=` would be useful („all files in that dir should be added unlocked always”). For example, in a synced folder for collaboration, PDFs could stay locked to save space, but Spreadsheets and ODT documents for example make sense to be added unlocked by default to stay editable. This could be achieved with something like this: - -[[!format bash """ -# repo-wide stored in git-annex branch -git annex config --set annex.addunlocked 'include=*.odt or include=**.pdf' - -# folder-specific via .gitattributes -echo '*.pdf annex.addunlocked=true' >> .gitattributes # (or 'anything') -echo '*.odt annex.addunlocked=true' >> .gitattributes # (or 'anything') - -# repo-specific -git config annex.addunlocked 'include=**/*.odt or include=**/*.pdf' -"""]] - -What do you think? - -Cheers, Yann - -> [[notabug|done]] --[[Joey]] diff --git a/doc/todo/Make_addunlocked_configurable_like_largefiles/comment_1_823bddc3b3824419f1123748f163e0ff._comment b/doc/todo/Make_addunlocked_configurable_like_largefiles/comment_1_823bddc3b3824419f1123748f163e0ff._comment deleted file mode 100644 index 1ec57c8d68..0000000000 --- a/doc/todo/Make_addunlocked_configurable_like_largefiles/comment_1_823bddc3b3824419f1123748f163e0ff._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 1" - date="2023-06-06T12:54:35Z" - content=""" -[git annex matching options](https://git-annex.branchable.com/git-annex-matching-options/) that make sense for `annex.addunlocked`: - -- include -- exclude -- mimetype -- mimeencoding -- largerthan -- smallerthan -- (securehash) -- (inbackend) - -All the others won't work AFAICS because location tracking hasn't yet happened at adding time. -"""]] diff --git a/doc/todo/Make_addunlocked_configurable_like_largefiles/comment_2_89ea6c764e18ed4335733f979fb31140._comment b/doc/todo/Make_addunlocked_configurable_like_largefiles/comment_2_89ea6c764e18ed4335733f979fb31140._comment deleted file mode 100644 index a8e6eb4799..0000000000 --- a/doc/todo/Make_addunlocked_configurable_like_largefiles/comment_2_89ea6c764e18ed4335733f979fb31140._comment +++ /dev/null @@ -1,22 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2023-06-06T16:50:16Z" - content=""" -annex.addunlocked can already be set to a [[git-annex-matching-expression]] -and this allows configuring it differently for different filenames, etc. -This has been supported since 7.20191230. - -You can already configure annex.addunlocked with `git-annex config`. - -Adding gitattributes for configs is problimatic because querying -for them is not fast. They have to be queried for every file. Also the -gitattributes format is not well suited to complex expressions since the -value cannot contain spaces. - -The only benefit to supporting it in gitattributes would be to configure it -differently in different branches of the repository. - -There was a todo already suggesting this, -[[annex.addunlocked_in_gitattributes]] and I decided to close it. -"""]] diff --git a/doc/todo/Make_addunlocked_configurable_like_largefiles/comment_3_8f32ef554635d852383eeaf3f0629d45._comment b/doc/todo/Make_addunlocked_configurable_like_largefiles/comment_3_8f32ef554635d852383eeaf3f0629d45._comment deleted file mode 100644 index 90265dab09..0000000000 --- a/doc/todo/Make_addunlocked_configurable_like_largefiles/comment_3_8f32ef554635d852383eeaf3f0629d45._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 3" - date="2023-06-07T04:49:04Z" - content=""" -Heh, didn't check that as I assumed when gitattributes does't work, git annex config won't either. Different branch config is a point but I guess not a huge one, right. - -Maybe a note about addunlocked would fit into the docs: https://git-annex.branchable.com/git-annex-add/ -"""]] diff --git a/doc/todo/Make_addunlocked_configurable_like_largefiles/comment_4_17d12c5d9f5aa40aa8c20ae478b324b2._comment b/doc/todo/Make_addunlocked_configurable_like_largefiles/comment_4_17d12c5d9f5aa40aa8c20ae478b324b2._comment deleted file mode 100644 index 1b37740122..0000000000 --- a/doc/todo/Make_addunlocked_configurable_like_largefiles/comment_4_17d12c5d9f5aa40aa8c20ae478b324b2._comment +++ /dev/null @@ -1,7 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2023-06-12T20:29:05Z" - content=""" -I've updated the man page to document annex.addunlocked. -"""]] diff --git a/doc/todo/Metadata_changes_are_not_reflected_in_a_view.mdwn b/doc/todo/Metadata_changes_are_not_reflected_in_a_view.mdwn deleted file mode 100644 index 7b698bc3a0..0000000000 --- a/doc/todo/Metadata_changes_are_not_reflected_in_a_view.mdwn +++ /dev/null @@ -1,33 +0,0 @@ -### Please describe the problem. -Changing metadata while being in an active view will not update the view. - -### What steps will reproduce the problem? -(inside a repository) - -1. Create a file - - $ uuidgen >file - -2. switch into a view - - $ git annex view !blah - $ ls - file - -3. changed the metadata the view is based upon - - $ git annex metadata -t blah file - $ ls - file - - It would be nice/expected that the view gets updated when the metadata changes, hiding 'file' now - - -### What version of git-annex are you using? On what operating system? - -git-annex version: 5.20141024~bpo70+1 -on debian wheezy with backports - -### Please provide any additional information below. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_1_5c3a0d8ec7b1da69b0d81aa4acc0c75b._comment b/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_1_5c3a0d8ec7b1da69b0d81aa4acc0c75b._comment deleted file mode 100644 index 9b079791b1..0000000000 --- a/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_1_5c3a0d8ec7b1da69b0d81aa4acc0c75b._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2015-07-20T17:45:13Z" - content=""" -This might be worth adding. But, one of the ways views can be used -is to use file operations to adjust the metadata that is being viewed. - -So, when you delete a file from a view, and git commit, git-annex -will remove the metadata that had made the file be in the view. - -And, if you copy or move a file in a view to a different subdirectory, -and add and git commit the change, the metadata will be updated to -update the metadata to reflect the files new location. - -As such, I see this as a wishlist todo item at best, and will move it from -the bugs list to the todo list. -"""]] diff --git a/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_2_74597dc57b426adccfa5aab456e499d2._comment b/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_2_74597dc57b426adccfa5aab456e499d2._comment deleted file mode 100644 index 4b0e8aa0f3..0000000000 --- a/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_2_74597dc57b426adccfa5aab456e499d2._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="👍 +1 for updating the view when committing" - date="2023-02-03T13:25:21Z" - content=""" -I'd also love to see the current view being updated to reflect the new metadata after committing. Workaround is currently to `git switch` back to the main branch and re-make the view with `git annex view bla='*'`. - -BTW views are **really** cool, what an awesome idea, joey! -"""]] diff --git a/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_3_39d7b5588595067332909c53b232c413._comment b/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_3_39d7b5588595067332909c53b232c413._comment deleted file mode 100644 index 73785d7e55..0000000000 --- a/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_3_39d7b5588595067332909c53b232c413._comment +++ /dev/null @@ -1,27 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2023-02-08T16:29:36Z" - content=""" -Another time it would make sense to update the view is after git pull from -elsewhere, which could change the metadata, or change the files present in -the parent branch. I think it would make sense for `git-annex sync` run in -a view to update the view after pulling. - -Updating the branch after `git-annex metadata` is run locally is of course -also possible. There is a similarity to updating an adjusted branch after -get/drop. Which has a config annex.adjustedbranchrefresh to tune how -frequently to update the branch. Or the user could just run -`git-annex sync --no-pull --no-push` themselves. - -There is probably a lot of scope for optimisation in updating the view -branch, that might be able to get it reasonably quick. -I have not fully thought through it, but basically diffing from the old -parent branch to the new parent to find files that have changed, and -adding/removing those from the view. And also diffing from the old -git-annex branch tree to the new one to get changes to metadata logs, -parsing those and using changes to metadata to also move/delete/add -files to the view branch. - -But it would be ok to start with a simple, slow implementation. -"""]] diff --git a/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_4_6ab0e38d8be8a70d501108f207d41d82._comment b/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_4_6ab0e38d8be8a70d501108f207d41d82._comment deleted file mode 100644 index 75af01c6c8..0000000000 --- a/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_4_6ab0e38d8be8a70d501108f207d41d82._comment +++ /dev/null @@ -1,29 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2023-02-08T19:32:34Z" - content=""" -`git-annex sync` when in a view branch will now update it. - -Leaving this open for optimising it. - -Also because of this problem: - - joey@darkstar:~/tmp/m#master(author=_)>git-annex sync - commit - On branch views/master(author=_) - nothing to commit, working tree clean - ok - merge synced/master - fatal: refusing to merge unrelated histories - failed - -Looks like it should not be trying to merge the synced/master -branch into the view branch. But, this makes me wonder, does the master branch -get updated with new files pulled from remotes? If not, the view branch -won't be updated to have them either. - -Also, I think that it may try to import trees from importree special remotes, -into the view branch. Perhaps those should also get imported but merged into -the master branch... -"""]] diff --git a/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_5_b1b381c560fd81f372fc393ec5d63d8d._comment b/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_5_b1b381c560fd81f372fc393ec5d63d8d._comment deleted file mode 100644 index 7d50e72b75..0000000000 --- a/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_5_b1b381c560fd81f372fc393ec5d63d8d._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="`git annex sync` when in view" - date="2023-02-08T20:17:12Z" - content=""" -Right, git refusing to merge unrelated histories is something I also bumped into while developing `thunar-plugins`. There I worked around it by having the *Synchronize* menu item do `git add -A;git commit -m ...;git annex sync --only-annex`, which works fine. -"""]] diff --git a/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_6_9d3d9877e8581f986257dd3d495a8eab._comment b/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_6_9d3d9877e8581f986257dd3d495a8eab._comment deleted file mode 100644 index 426bbd21e6..0000000000 --- a/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_6_9d3d9877e8581f986257dd3d495a8eab._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 6""" - date="2023-02-10T19:49:51Z" - content=""" -I've now fixed `git-annex sync`'s other behavior when ran in a view branch -to be acceptable. - -I'm going to close this as fixed. But have opened -[[faster_incremental_update_of_view_branch_by_git-annex_sync]] as the -performance could be improved. -"""]] diff --git a/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_7_f97d171fdd715d984e43c1a8139bb8ae._comment b/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_7_f97d171fdd715d984e43c1a8139bb8ae._comment deleted file mode 100644 index 0dfbb93e00..0000000000 --- a/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_7_f97d171fdd715d984e43c1a8139bb8ae._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="👍 This made it easier for the Thunar plugin!" - date="2023-02-14T09:35:32Z" - content=""" -[`thunar-plugins`](https://gitlab.com/nobodyinperson/thunar-plugins) now detect git annex' support for this (checking if `view` appears in `git annex help sync`, which it didn't before) and does `git annex sync` instead of the `git add -A;git commit;git annex sync --only-annex` workaround in that case. Very nice, joey! -"""]] diff --git a/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_8_e10e8e6daf155fc493454f1d0d8595a8._comment b/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_8_e10e8e6daf155fc493454f1d0d8595a8._comment deleted file mode 100644 index 0690c708a3..0000000000 --- a/doc/todo/Metadata_changes_are_not_reflected_in_a_view/comment_8_e10e8e6daf155fc493454f1d0d8595a8._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 8""" - date="2023-02-14T16:17:05Z" - content=""" -I'd recommend not relying on the `git-annex help sync` to detect this, -since man pages may not be available. Look for '?=' in -`git-annex view --help` -"""]] diff --git a/doc/todo/Package_for_Lacie_NasOS__63__.mdwn b/doc/todo/Package_for_Lacie_NasOS__63__.mdwn deleted file mode 100644 index 635a9fcb82..0000000000 --- a/doc/todo/Package_for_Lacie_NasOS__63__.mdwn +++ /dev/null @@ -1,3 +0,0 @@ -Would it be possible to create a package for Lacie's NacOS? - -> [[closing|done]] --[[Joey]] diff --git a/doc/todo/Package_for_Lacie_NasOS__63__/comment_1_1b1b23b95b1fa29a8d71848a5cec9008._comment b/doc/todo/Package_for_Lacie_NasOS__63__/comment_1_1b1b23b95b1fa29a8d71848a5cec9008._comment deleted file mode 100644 index 4fffad28c5..0000000000 --- a/doc/todo/Package_for_Lacie_NasOS__63__/comment_1_1b1b23b95b1fa29a8d71848a5cec9008._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2016-01-20T18:56:36Z" - content=""" -If that is a Linux-based NAS, there's a reasonably good chance that one -of the [Linux standalone builds](https://git-annex.branchable.com/install/Linux_standalone/) -of git-annex will run on it. - -Actually, from what I can see, it's based on Debian, so you might be able -to just apt-get install git-annex -"""]] diff --git a/doc/todo/Package_for_Lacie_NasOS__63__/comment_2_81b181781da615c56aa30eab131cec53._comment b/doc/todo/Package_for_Lacie_NasOS__63__/comment_2_81b181781da615c56aa30eab131cec53._comment deleted file mode 100644 index 7795a7ca4d..0000000000 --- a/doc/todo/Package_for_Lacie_NasOS__63__/comment_2_81b181781da615c56aa30eab131cec53._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="db" - subject="comment 2" - date="2016-01-22T19:00:57Z" - content=""" -Yes it is Linux: - -$ uname -a -Linux nas 3.10.59-svn15282 #1 SMP Fri Jul 31 13:36:10 UTC 2015 x86_64 GNU/Linux - -But it does not have apt. And probably not a compiler either. - -Is it complicated to install a compiled version manually? Or is it \"self-containing\"? Any instructions for cross compiling on a Mac? -"""]] diff --git a/doc/todo/Package_for_Lacie_NasOS__63__/comment_3_fd7eeee6a86101867f0ab17f883f0c6a._comment b/doc/todo/Package_for_Lacie_NasOS__63__/comment_3_fd7eeee6a86101867f0ab17f883f0c6a._comment deleted file mode 100644 index b69146873c..0000000000 --- a/doc/todo/Package_for_Lacie_NasOS__63__/comment_3_fd7eeee6a86101867f0ab17f883f0c6a._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2016-01-22T19:39:17Z" - content=""" -Get a build from - -"""]] diff --git a/doc/todo/Set_total_storage_limit_for_special_remotes.mdwn b/doc/todo/Set_total_storage_limit_for_special_remotes.mdwn deleted file mode 100644 index 59c5794a87..0000000000 --- a/doc/todo/Set_total_storage_limit_for_special_remotes.mdwn +++ /dev/null @@ -1,3 +0,0 @@ -I'd like to be able to set a fixed limit on how much storage can be uploaded to a special remote. A use case for this may be that I want to spend no more than Y dollars, on a storage service that charges $X per gigabyte. I would thus set a limit where I a upload would be interrupted with a warning about the limit, and to continue I would need to use a --force option. - -> dup of [[Specify_maximum_usable_space_per_remote]]; [[done]] --[[Joey]] diff --git a/doc/todo/Set_total_storage_limit_for_special_remotes/comment_1_8945025ad54e28f48474f8931746a775._comment b/doc/todo/Set_total_storage_limit_for_special_remotes/comment_1_8945025ad54e28f48474f8931746a775._comment deleted file mode 100644 index 0df5669d66..0000000000 --- a/doc/todo/Set_total_storage_limit_for_special_remotes/comment_1_8945025ad54e28f48474f8931746a775._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2015-07-02T20:01:42Z" - content=""" -This totally makes sense to want, but it's rather difficult to implement since -multiple git-annex repos can all be uploading content to the same special -remote, while disconnected from each other. -"""]] diff --git a/doc/todo/Set_total_storage_limit_for_special_remotes/comment_2_86aa368db46dfedb065a18edfa7c9ad4._comment b/doc/todo/Set_total_storage_limit_for_special_remotes/comment_2_86aa368db46dfedb065a18edfa7c9ad4._comment deleted file mode 100644 index 818b7f3862..0000000000 --- a/doc/todo/Set_total_storage_limit_for_special_remotes/comment_2_86aa368db46dfedb065a18edfa7c9ad4._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="mawillcockson" - subject="comment 2" - date="2015-07-18T19:13:03Z" - content=""" -Perhaps server-side [git hooks](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks)? - -Maybe a git hook script that uses [inotify](https://en.wikipedia.org/wiki/Inotify) for inode size notifications. - -I would have no idea how to implement this, though. -"""]] diff --git a/doc/todo/Should_git_annex_diffdriver_fetch_non-present_file.mdwn b/doc/todo/Should_git_annex_diffdriver_fetch_non-present_file.mdwn deleted file mode 100644 index ac67e8406b..0000000000 --- a/doc/todo/Should_git_annex_diffdriver_fetch_non-present_file.mdwn +++ /dev/null @@ -1,9 +0,0 @@ -I noticed that `git annex diffdriver` doesn't make sure that files it tries to diff are actually present. Would it make sense that `git annex diffdriver ...` runs `git annex get` to ensure files are present? 🤔 A `git diff` could then result in a lot of traffic if the user configured the `diffdriver`. But I'd say that users who took their time to configure would be are aware of this or even expect that to happen - after all they want to see the differences to a past version. - -A manual workaround is to handle the `get`ting in a custom supplied diffing program. - -Context: - -- [recently added `git annex diffdriver --text`](https://git-annex.branchable.com/todo/Improving_diffdriver_--text/) - -> Duplicate of [[--get_option_for_diffdriver]], so closing [[done]]. --[[Joey]] diff --git a/doc/todo/Store_git_pack_files_on_special_remotes.mdwn b/doc/todo/Store_git_pack_files_on_special_remotes.mdwn deleted file mode 100644 index 0a9938c04a..0000000000 --- a/doc/todo/Store_git_pack_files_on_special_remotes.mdwn +++ /dev/null @@ -1,5 +0,0 @@ -It would be nice to be able to upload and download git history with special remotes. This could be a move towards full special remote syncing. - -> I feel this is out of scope. git has its own interface to let a program -> be registered as performing a transport, to store a git repository -> anywhere. [[wontfix|done]] --[[Joey]] diff --git a/doc/todo/Straight-forward_diffing_with___39__git_annex_diff__39__.mdwn b/doc/todo/Straight-forward_diffing_with___39__git_annex_diff__39__.mdwn deleted file mode 100644 index 243697db7a..0000000000 --- a/doc/todo/Straight-forward_diffing_with___39__git_annex_diff__39__.mdwn +++ /dev/null @@ -1,29 +0,0 @@ -Hi joey, - -Currently, getting a useful diff between annexed file versions is quite involved [(setting up git-annex diffdriver)](https://git-annex.branchable.com/forum/git-like_git-annex_diff/). - -It would be very nice if showing changes between annexed files was a little more straight-forward and ideally without any user config needed. UI suggestions: - -- `git annex diff`: would behave exactly like `git diff`, but operatign on both unannexed and annexed contents - - ideally re-implementing all its options (e.g. `--word-diff`, `--word-diff-regex`, etc.) - - would need a diff implementation in Haskell (surely there is one) - - sounds complicated to do TBH -- Teaching `git diff` to use the annexed content instead of the pointer links/files - - software like [`nbstripout`](https://github.com/kynan/nbstripout) passes the git-tracked contents through a filter before diffing. This sounds like git-annex could do the same to add straight-forward `git diff` support without user configuration. - - git-annex already has a `* filter=annex` attribute in place, for text diffing there apparently needs to be a `* diff=annex` attribute and a `[diff "annex"] textconv=git-annex-output-content-instead-of-pointer` config. - - even if the above works, I don't know how to temporarily switch this off without uncommenting the `textconv` e.g. with `git config --edit`. Sometimes you just want to see the actual hashes of old and new file. - -Maybe `git annex diffdriver` kind of does part of this, but I don't really understand what it actually does. - -Here other posts related to diffing: - -- https://git-annex.branchable.com/forum/enabling_git-annex-diffdriver_for_gitk/ -- https://git-annex.branchable.com/todo/--get_option_for_diffdriver/ - -What do you think? - -Cheers, Yann - -PS: Thank you very much for git-annex, it's awesome! I'm giving a git-annex workshop next weekend [@Tuebix](https://cfp.tuebix.org/tuebix-2023/talk/review/GWRP3UKE3VFKVDG8RNQ8ZZPCZPNZYYWM), really looking forward to it. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/todo/Straight-forward_diffing_with___39__git_annex_diff__39__/comment_1_2f5ea091b7f6defa397d15055d04a67d._comment b/doc/todo/Straight-forward_diffing_with___39__git_annex_diff__39__/comment_1_2f5ea091b7f6defa397d15055d04a67d._comment deleted file mode 100644 index 618195c7c7..0000000000 --- a/doc/todo/Straight-forward_diffing_with___39__git_annex_diff__39__/comment_1_2f5ea091b7f6defa397d15055d04a67d._comment +++ /dev/null @@ -1,28 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-06-28T17:43:19Z" - content=""" -This works: - - echo * diff=text >>.gitattributes - git config diff.text.command "git-annex diffdriver -- sh -c 'echo diff a/\$0 b/\$0; echo \$0; diff -u \$1 \$4 || true' --" - -Unfortunatly, the textconv approach will not work for locked files, because -git does not apply the textconv to symlinks. It could be made to work for -unlocked files but the above works for all annexed files. - -diffing like this is only useful on text files, and most annexed files -are not text files. That's why the diffdriver command focuses on using some -external diff driver that knows how to diff whatever type of binary file is -being stored in git-annex. So this should certianly not be enabled by -default. It makes sense to enable it in .gitattributes when you have -annexed some textual files. - -That command is a bit hard to come up with, with the complex shell quoting. -So rather than try to document it, I made an easier way to do the same -thing: - - echo * diff=text >>.gitattributes - git config diff.text.command "git annex diffdriver --text" -"""]] diff --git a/doc/todo/Straight-forward_diffing_with___39__git_annex_diff__39__/comment_2_cae9e0be287bd42498caf1f61c518f0e._comment b/doc/todo/Straight-forward_diffing_with___39__git_annex_diff__39__/comment_2_cae9e0be287bd42498caf1f61c518f0e._comment deleted file mode 100644 index 286e43b453..0000000000 --- a/doc/todo/Straight-forward_diffing_with___39__git_annex_diff__39__/comment_2_cae9e0be287bd42498caf1f61c518f0e._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="Awesome, thanks!" - date="2023-06-28T21:09:50Z" - content=""" -Awesome, thanks joey! -"""]] diff --git a/doc/todo/Usability_improvements_for_new_users.mdwn b/doc/todo/Usability_improvements_for_new_users.mdwn deleted file mode 100644 index 6fad95e82d..0000000000 --- a/doc/todo/Usability_improvements_for_new_users.mdwn +++ /dev/null @@ -1,21 +0,0 @@ -Hi there, - -I'm a relatively new to `git-annex` and quite impressed so far. - -I've found the command to be somewhat less self-documenting than I'd like it to be for a new user. -Particularly, since it's a `git` subcommand, I'd wish for annex to 'mirror' as many subcommands as possible -with `annex`-enabled equivalents. My chief complaint at the moment is that it is a bit hard to remember the -specific commands for showing remotes (`git annex info` has no equivalent `git info`) and `git annex remote` -should ideally return a list of remotes for parallelism. - -Similarly, it'd be nice if some of the commands were a bit nested to match `git` semantics. This would increase discoverability: -For instance, `git annex enableremote` -> `git annex remote enable` or `git annex renameremote -> git annex remote rename`. Then subcommands for dealing with remotes -could be more easily summarized on a single page in the `git-annex-remote` section. - -It's worth noting that such changes could happen alongside the current commands without a loss of functionality. Git core -has recently gone through a similar process to simplify the semantics of some of its commands. I'd ask for such changes to be considered. - -Daven - -> [[wontfix|done]] at least for the listed things, for reasons explained -> in my comment below. --[[Joey]] diff --git a/doc/todo/Usability_improvements_for_new_users/comment_1_70ecef547308f9b049bd67f3d2f14117._comment b/doc/todo/Usability_improvements_for_new_users/comment_1_70ecef547308f9b049bd67f3d2f14117._comment deleted file mode 100644 index 096e25a0cf..0000000000 --- a/doc/todo/Usability_improvements_for_new_users/comment_1_70ecef547308f9b049bd67f3d2f14117._comment +++ /dev/null @@ -1,37 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-01-16T17:16:54Z" - content=""" -> `git annex info` has no equivalent `git info` - -git-annex can't be limited to commands that happen to have equivilant git -commands. `git annex drop` also has no equivilant in git, but is -fundamental to git-annex. - -If you're thinking of `git annex info` as equivilant to `git remote` in -listing remotes, that does not consider everything the info command does. - -> git annex remote should ideally return a list of remotes for parallelism - -`git remote` lists all remotes including git-annex special remotes, -so `git annex remote` would be unnecessary duplication. - -> git annex renameremote -> git annex remote rename - -This would encourage drawing a false equivilance with `git remote rename`. - -I'm also not convinced that stacking subcommands 3 levels deep is a good -idea. The user has to dig through every level to find something then. - -There's also an intentional similarity in naming between `git annex init` -and `git annex initremote`, which I think makes sense and helps users -learn and remember the latter command after having first learned the former. -And `enableremote` and `renameremote` then flow naturally from that. -This is, IMHO, a more natural learning flow than searching for strained -equivilances to `git remote` commands. - -I do think that `git annex add` has a very good reason to parallel `git -add`, and if there are other git-annnex commands that could directly -parallel a git command like that and don't, that would be worth addressing. -"""]] diff --git a/doc/todo/Usability_improvements_for_new_users/comment_2_ea77f0894802a17854aeedeceababf8a._comment b/doc/todo/Usability_improvements_for_new_users/comment_2_ea77f0894802a17854aeedeceababf8a._comment deleted file mode 100644 index d7c2398dd7..0000000000 --- a/doc/todo/Usability_improvements_for_new_users/comment_2_ea77f0894802a17854aeedeceababf8a._comment +++ /dev/null @@ -1,29 +0,0 @@ -[[!comment format=mdwn - username="daven.quinn@d0ed4e0e5e4462d9a74a5d5a8fbd1b17f85db13e" - nickname="daven.quinn" - avatar="http://cdn.libravatar.org/avatar/4811a26fff610f0c9668d183481a7158" - subject="comment 1 response" - date="2023-01-16T21:45:35Z" - content=""" -Fair enough, I suppose. However, I think that feedback from early-stage users should be weighted highly, as we interact with \"onboarding\"/\"discoverability\" problems that eventually fade from being users' primary impediments to use. - -> git-annex can't be limited to commands that happen to have equivalant git commands. - -I'm not suggesting that all commands should have `git` equivalents. Instead, when a commonly used equivalent (or close relative) is available, the name should be mirrored to increase discoverability of common idioms. - - -> This would encourage drawing a false equivilance with git remote rename. - -I'm not quite sure why the equivalence would be false — yes, the commands do somewhat different things. But they both fundamentally rename a remote. - -> searching for strained equivilances to git remote commands. - -Again, I am not sure the equivalences are so strained. Users should not (and I think broadly won't) expect a git-annex subcommand to do exactly the same thing as a git equivalent. But semantic-level parallelism with git top-level commands, and adopting git's pattern of grouping like functionality into subcommands, could greatly improve discoverability. As it stands, the goals of *preventing confusion between similarly named functionality* and *allowing functionality to be easily found* are in tension. - -Another way to frame my suggestion is to consider the layout of the `git-annex` command line help output. Right now, when seeking assistance, the user is presented with a long list of commands and has to hunt for related functionality (`renameremote`, `enableremote`, etc.) within it. In contrast, when I type `git remote --help`, I see all of the commands for managing remotes in a single, shorter output. It is another nesting level, true, but I think a help page entry of `remote - manages git-annex special remotes` would be easier to parse than the current output. - -Anyway, I appreciate your work on this system and consideration of user feedback. - -Regards, -Daven -"""]] diff --git a/doc/todo/When_autostarting_the_assistant_on_boot_delay_the_execution_2min.mdwn b/doc/todo/When_autostarting_the_assistant_on_boot_delay_the_execution_2min.mdwn deleted file mode 100644 index 88df27a2a3..0000000000 --- a/doc/todo/When_autostarting_the_assistant_on_boot_delay_the_execution_2min.mdwn +++ /dev/null @@ -1,36 +0,0 @@ -I think the assistant should have a delay before starting so it doesn't impact on the system boot performance. - -I know this can be configured per user (I have it set to 5min), but I think 2min is a sane default. - -I don't really know how to submit patches (I can't attach it), so I simply copy it here. It's trivial enough. - -[[!format txt """ -From bc6b90ce333659bd2511eedd4ab9067241f73780 Mon Sep 17 00:00:00 2001 -From: =?UTF-8?q?Jon=20Ander=20Pe=C3=B1alba?= -Date: Wed, 28 Oct 2015 12:00:51 +0100 -Subject: [PATCH] When autostarting the assistant on boot delay the execution - 2min - ---- - Assistant/Install/AutoStart.hs | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) - -diff --git a/Assistant/Install/AutoStart.hs b/Assistant/Install/AutoStart.hs -index 5745030..6a09c3a 100644 ---- a/Assistant/Install/AutoStart.hs -+++ b/Assistant/Install/AutoStart.hs -@@ -35,6 +35,6 @@ fdoAutostart command = genDesktopEntry - "Git Annex Assistant" - "Autostart" - False -- (command ++ " assistant --autostart") -+ (command ++ " assistant --autostart --startdelay=120") - Nothing - [] --- -2.6.2 - - -"""]] - -> [[done]] --[[Joey]] diff --git a/doc/todo/When_autostarting_the_assistant_on_boot_delay_the_execution_2min/comment_1_49990eaf4a3d6ee89f668dfc159f5eae._comment b/doc/todo/When_autostarting_the_assistant_on_boot_delay_the_execution_2min/comment_1_49990eaf4a3d6ee89f668dfc159f5eae._comment deleted file mode 100644 index 129eeac977..0000000000 --- a/doc/todo/When_autostarting_the_assistant_on_boot_delay_the_execution_2min/comment_1_49990eaf4a3d6ee89f668dfc159f5eae._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2015-11-02T15:14:20Z" - content=""" -There's actually an undocumented default startdelay of 5 seconds when using -autostart. I've now documented it. - -Maybe 5 seconds is too little. OTOH, 2 minutes may be a bit too much; -if the user is booting up their computer to access a new file in their -annex they'd expect it to start syncing ASAP. - -I think my justification for the 5 seconds default was that about how long -it seems to take a typical desktop system to get loaded (after the login -screen) and for the user to open a web browser, on reasonably modern -hardware. Even windows in an emulator doesn't take much longer than that. -"""]] diff --git a/doc/todo/Wishlist__58___additional_environment_variables_for_hooks.mdwn b/doc/todo/Wishlist__58___additional_environment_variables_for_hooks.mdwn deleted file mode 100644 index 720db90099..0000000000 --- a/doc/todo/Wishlist__58___additional_environment_variables_for_hooks.mdwn +++ /dev/null @@ -1,19 +0,0 @@ -It would be nice if a couple of additional environment variables to be set for hook uses. - -In particular: - - GIT_ANNEX_DIRECT=`git config annex.direct` - -and - - GIT_TOP_LEVEL=`git rev-parse --show-toplevel` - - -I've made some changes to flickrannex to allow the sub-directories above the uploaded image to be added as tags. This change has been merged into trunk: [[https://github.com/TobiasTheViking/flickrannex]] - -What I needed was both the environment variables mentioned above. One is set as part of the annex-hook and the other I guestimate from the file path. If it was set in git-annex it would be much cleaner (and accurate). So...I think this info would be useful for other hook. - -> I think this was referring to the old hook special remote. External -> special remotes have supplanted that. And when using an external special -> remote you can query for such data once when starting your special remote -> program, so no need for git-annex to do it. [[wontfix|done]] --[[Joey]] diff --git a/doc/todo/Wishlist__58___additional_environment_variables_for_hooks/comment_1_d82cbbb478a81a651fbe6cb8b71c1192._comment b/doc/todo/Wishlist__58___additional_environment_variables_for_hooks/comment_1_d82cbbb478a81a651fbe6cb8b71c1192._comment deleted file mode 100644 index 4d0409d58e..0000000000 --- a/doc/todo/Wishlist__58___additional_environment_variables_for_hooks/comment_1_d82cbbb478a81a651fbe6cb8b71c1192._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="http://joeyh.name/" - ip="209.250.56.154" - subject="comment 1" - date="2014-03-18T19:36:08Z" - content=""" -Is this still relevant? -"""]] diff --git a/doc/todo/Wishlist__58___sanitychecker_fix_wrong_UUID__47__duplicate_remote.mdwn b/doc/todo/Wishlist__58___sanitychecker_fix_wrong_UUID__47__duplicate_remote.mdwn deleted file mode 100644 index 0f01ae680b..0000000000 --- a/doc/todo/Wishlist__58___sanitychecker_fix_wrong_UUID__47__duplicate_remote.mdwn +++ /dev/null @@ -1,9 +0,0 @@ -In certain situations different client annexes might get the same remote repository added, but before being synced. - -Once the two clients sync they will both have two remotes with the same name. But only one UUID will have any content(Assuming only one client pushed). - -It would be nice to have some (automatic?) way to resolve this conflict. - -Not sure if anything sane can be done if both clients have pushed? - -> [[done]] see comment --[[Joey]] diff --git a/doc/todo/Wishlist__58___sanitychecker_fix_wrong_UUID__47__duplicate_remote/comment_1_a9e54e4bc540cbab7cdbbfdeaa973f4a._comment b/doc/todo/Wishlist__58___sanitychecker_fix_wrong_UUID__47__duplicate_remote/comment_1_a9e54e4bc540cbab7cdbbfdeaa973f4a._comment deleted file mode 100644 index 57083fb239..0000000000 --- a/doc/todo/Wishlist__58___sanitychecker_fix_wrong_UUID__47__duplicate_remote/comment_1_a9e54e4bc540cbab7cdbbfdeaa973f4a._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-01-16T17:54:51Z" - content=""" -This does not make sense to me. If the remote is a git repo, -it will be initialized by git-annex only once, so there will only be one -UUID. The second "client" that uses that remote will see the earlier UUID. - -Unless there were some kind of a race in initializing the same repo -concurrently, but that seems unlikely and if so this is lacking in any -details about it. - -I'm going to close this since it does not make sense, but do followup if -you remember what this was supposed to be about. -"""]] diff --git a/doc/todo/__91__PATCH__93___Fix_annex.adviceNoSshCaching_no_effect.mdwn b/doc/todo/__91__PATCH__93___Fix_annex.adviceNoSshCaching_no_effect.mdwn deleted file mode 100644 index 082a73997b..0000000000 --- a/doc/todo/__91__PATCH__93___Fix_annex.adviceNoSshCaching_no_effect.mdwn +++ /dev/null @@ -1,30 +0,0 @@ -``` -From 1a67bc14aa74262a304e7b6bd6372b0eca40486e Mon Sep 17 00:00:00 2001 -From: Reiko Asakura -Date: Fri, 30 Sep 2022 10:56:17 -0400 -Subject: [PATCH] Fix annex.adviceNoSshCaching having no effect - -git will always return option names in lowercase ---- - Types/GitConfig.hs | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) - -diff --git a/Types/GitConfig.hs b/Types/GitConfig.hs -index 44629d412..4b892bd5f 100644 ---- a/Types/GitConfig.hs -+++ b/Types/GitConfig.hs -@@ -265,7 +265,7 @@ extractGitConfig configsource r = GitConfig - | otherwise = Nothing - in mapMaybe get (M.toList (Git.config r)) - ] -- , annexAdviceNoSshCaching = getbool (annexConfig "adviceNoSshCaching") True -+ , annexAdviceNoSshCaching = getbool (annexConfig "advicenosshcaching") True - } - where - getbool k d = fromMaybe d $ getmaybebool k --- -2.30.2 - -``` - -> Thanks for the fix! [[done]] --[[Joey]] diff --git a/doc/todo/__91__PATCH__93___Run_freeze__47__thaw_hooks_on_crippled_fs.mdwn b/doc/todo/__91__PATCH__93___Run_freeze__47__thaw_hooks_on_crippled_fs.mdwn deleted file mode 100644 index ea6ffb1a7b..0000000000 --- a/doc/todo/__91__PATCH__93___Run_freeze__47__thaw_hooks_on_crippled_fs.mdwn +++ /dev/null @@ -1,76 +0,0 @@ -This patch allows hooks to be run in WSL1 for a repo created on an NTFS volume but I think it also applies in general. - -``` -From d4587806d8fd1ea767a8effc06edc1a4e10f5bca Mon Sep 17 00:00:00 2001 -From: Reiko Asakura -Date: Sun, 25 Sep 2022 15:21:24 -0400 -Subject: [PATCH] Run freeze and thaw hooks on crippled filesystems - -The user sets these hooks deliberately so they should always be run. For -example this allows hooks to be used to manage file permissions on NTFS -volumes in WSL1. ---- - Annex/Perms.hs | 14 +++++++------- - 1 file changed, 7 insertions(+), 7 deletions(-) - -diff --git a/Annex/Perms.hs b/Annex/Perms.hs -index 6681da7e0..69e344452 100644 ---- a/Annex/Perms.hs -+++ b/Annex/Perms.hs -@@ -154,7 +154,7 @@ createWorkTreeDirectory dir = do - - that happens with write permissions. - -} - freezeContent :: RawFilePath -> Annex () --freezeContent file = unlessM crippledFileSystem $ -+freezeContent file = - withShared $ \sr -> freezeContent' sr file - - freezeContent' :: SharedRepository -> RawFilePath -> Annex () -@@ -163,7 +163,7 @@ freezeContent' sr file = freezeContent'' sr file =<< getVersion - freezeContent'' :: SharedRepository -> RawFilePath -> Maybe RepoVersion -> Annex () - freezeContent'' sr file rv = do - fastDebug "Annex.Perms" ("freezing content " ++ fromRawFilePath file) -- go sr -+ unlessM crippledFileSystem $ go sr - freezeHook file - where - go GroupShared = if versionNeedsWritableContentFiles rv -@@ -253,7 +253,7 @@ thawContent' sr file = do - - permissions. -} - thawPerms :: Annex () -> Annex () -> Annex () - thawPerms a hook = ifM crippledFileSystem -- ( void (tryNonAsync a) -+ ( hook >> void (tryNonAsync a) - , hook >> a - ) - -@@ -263,9 +263,9 @@ thawPerms a hook = ifM crippledFileSystem - - file. - -} - freezeContentDir :: RawFilePath -> Annex () --freezeContentDir file = unlessM crippledFileSystem $ do -+freezeContentDir file = do - fastDebug "Annex.Perms" ("freezing content directory " ++ fromRawFilePath dir) -- withShared go -+ unlessM crippledFileSystem $ withShared go - freezeHook dir - where - dir = parentDir file -@@ -287,9 +287,9 @@ createContentDir dest = do - unlessM (liftIO $ R.doesPathExist dir) $ - createAnnexDirectory dir - -- might have already existed with restricted perms -- unlessM crippledFileSystem $ do -+ do - thawHook dir -- liftIO $ allowWrite dir -+ unlessM crippledFileSystem $ liftIO $ allowWrite dir - where - dir = parentDir dest - --- -2.30.2 - -``` - -> [[done]] --[[Joey]] diff --git a/doc/todo/__91__PATCH__93___Run_freeze__47__thaw_hooks_on_crippled_fs/comment_1_b943081483e4f1f9897b214004d48493._comment b/doc/todo/__91__PATCH__93___Run_freeze__47__thaw_hooks_on_crippled_fs/comment_1_b943081483e4f1f9897b214004d48493._comment deleted file mode 100644 index c1abde42f3..0000000000 --- a/doc/todo/__91__PATCH__93___Run_freeze__47__thaw_hooks_on_crippled_fs/comment_1_b943081483e4f1f9897b214004d48493._comment +++ /dev/null @@ -1,7 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-09-26T17:01:33Z" - content=""" -Thank you for this patch. I've applied it. -"""]] diff --git a/doc/todo/add_--jobs_for_dropunused_command.mdwn b/doc/todo/add_--jobs_for_dropunused_command.mdwn deleted file mode 100644 index 6128ff2f58..0000000000 --- a/doc/todo/add_--jobs_for_dropunused_command.mdwn +++ /dev/null @@ -1,3 +0,0 @@ -Simple "drop" have it, but no for "dropunused". Missed? - -> [[fixed|done]] --[[Joey]] diff --git a/doc/todo/add_--jobs_for_dropunused_command/comment_1_bcd06faab6b5f6394333e74a17249d08._comment b/doc/todo/add_--jobs_for_dropunused_command/comment_1_bcd06faab6b5f6394333e74a17249d08._comment deleted file mode 100644 index 4f643b2874..0000000000 --- a/doc/todo/add_--jobs_for_dropunused_command/comment_1_bcd06faab6b5f6394333e74a17249d08._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="git annex drop --unused --jobs=cpus works" - date="2023-07-15T05:53:43Z" - content=""" -If you want to drop all unused files, `git annex unused;git annex drop --unused --jobs=cpus` works. -"""]] diff --git a/doc/todo/add_--json-progress_to_fsck_--json.mdwn b/doc/todo/add_--json-progress_to_fsck_--json.mdwn deleted file mode 100644 index c228b321d9..0000000000 --- a/doc/todo/add_--json-progress_to_fsck_--json.mdwn +++ /dev/null @@ -1,17 +0,0 @@ -Please add the feature/switch `--json-progress` to `git annex fsck --json`. - -For a basic version, it would be nice to show the percentage using the number of processed chunks / total number of chunks for a given file while calculating checksum. - -Let's not forget that a file on a remote can be fsck-ed when using `--from `. In this case if the file is not present locally, the file must be downloaded first (`--json-progress` is already supported for `get`). Usually this takes much-much longer as the checksum calculation. When checking a large file (~1Gb) on a not too fast remote, it's quite a time for the first "response" that something is happening when using the `--json` switch. To be exact, the first and only response is that the check has finished (happy way). It would be nice to see the json progress of the download+checksum calculation. - - -Test environment: - -Docker Community 20.10.5 - -Ubuntu 20.04.2 LTS (Focal Fossa) - -git-annex version: 8.20210224-gf951847c6 (kitenet: SHA256E-s51232145--1e57d2e334cffdb3c3874785cfba4f6d5fc521808d0b1deef4fb68160dd88d48.tar.gz) - -> [[done]] via incremental checksum while downloading, for nearly all -> remotes. --[[Joey]] diff --git a/doc/todo/add_--json-progress_to_fsck_--json/comment_1_170acd9fc5b5a40ae22de0bf418b424c._comment b/doc/todo/add_--json-progress_to_fsck_--json/comment_1_170acd9fc5b5a40ae22de0bf418b424c._comment deleted file mode 100644 index adb705703f..0000000000 --- a/doc/todo/add_--json-progress_to_fsck_--json/comment_1_170acd9fc5b5a40ae22de0bf418b424c._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="parhuzamos" - avatar="http://cdn.libravatar.org/avatar/79ed445f40e89bd8ba696bab3942ec56" - subject="comment 1" - date="2021-05-24T09:33:50Z" - content=""" -Is this possible? Or makes no sense? Do you plan implementing this? - -Thank you, -Bence -"""]] diff --git a/doc/todo/add_--json-progress_to_fsck_--json/comment_2_2ce9c3c35ea5c4033bfa42e2f3365a64._comment b/doc/todo/add_--json-progress_to_fsck_--json/comment_2_2ce9c3c35ea5c4033bfa42e2f3365a64._comment deleted file mode 100644 index 7fb6967d69..0000000000 --- a/doc/todo/add_--json-progress_to_fsck_--json/comment_2_2ce9c3c35ea5c4033bfa42e2f3365a64._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-06-28T17:00:33Z" - content=""" -It seems problimatic to have json progress for both downloading the file -and for checksumming it. I don't know how that json would look. - -In the meantime, though, git-annex has changed to checksum files while they -are being downloaded from most remotes, and when it can do that, fsck ---from avoids the separate checksumming step. - -While some special remotes do not support that, it's a very limited list: -only bittorrent on linux, and adb, external, gcrypt, hook, and rsync on -other OSs. - -This makes me think that complicating the json output to display two -different kinds of progress is probably not worth the complication. -"""]] diff --git a/doc/todo/add_--json-progress_to_fsck_--json/comment_3_5c0c3bdf1e5cf68f2dcebdd85a8253ac._comment b/doc/todo/add_--json-progress_to_fsck_--json/comment_3_5c0c3bdf1e5cf68f2dcebdd85a8253ac._comment deleted file mode 100644 index a54b4bee12..0000000000 --- a/doc/todo/add_--json-progress_to_fsck_--json/comment_3_5c0c3bdf1e5cf68f2dcebdd85a8253ac._comment +++ /dev/null @@ -1,20 +0,0 @@ -[[!comment format=mdwn - username="jstritch" - avatar="http://cdn.libravatar.org/avatar/56756b34ff409071074762951d270002" - subject="comment 3" - date="2023-07-11T18:56:39Z" - content=""" -The --json-progress flag should produce progress. I do not understand why it would be a problem to intermix types of 'action' in the output. If file A is uploaded and file B is downloaded and checksummed, that's (at least) three different progress objects to produce. This is exactly the type of information I want to see on the screen instead of wondering if it's stuck. I would also apply this pattern to sync and friends. - -Upload A 35% - -Upload A 100% - -Download B 100% - -Checksum B 50% - -Checksum B 100% - -Go ahead and make per-file json error results available and describe in the documentation. -"""]] diff --git a/doc/todo/add_ancient_armel_build.mdwn b/doc/todo/add_ancient_armel_build.mdwn deleted file mode 100644 index 602cc76568..0000000000 --- a/doc/todo/add_ancient_armel_build.mdwn +++ /dev/null @@ -1,3 +0,0 @@ -Add an armel build like the i386ancient build. - -> [[wontfix|done]] --[[Joey]] diff --git a/doc/todo/add_ancient_armel_build/comment_1_704f1a854e5ef62edbca60ab209d2123._comment b/doc/todo/add_ancient_armel_build/comment_1_704f1a854e5ef62edbca60ab209d2123._comment deleted file mode 100644 index 28842b5c08..0000000000 --- a/doc/todo/add_ancient_armel_build/comment_1_704f1a854e5ef62edbca60ab209d2123._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="rustikus@9db90d0c115a1825e2f1e5f15257ec1298a6c7b6" - nickname="rustikus" - avatar="http://cdn.libravatar.org/avatar/6fe372a7a46198829723c408af75c8cc" - subject="Any update or help needed?" - date="2017-01-12T13:40:38Z" - content=""" -Hi, - -is there any update? I do have an older version of Synology DiskStation (DS211+) which does have this CPU. Anything I can do to help? -Disclaimer: I am quite familiar with Linux but I am no expert in programming with haskel. - -Thanks! -"""]] diff --git a/doc/todo/add_ancient_armel_build/comment_2_0b02f96699e9ec9447e336b74b255801._comment b/doc/todo/add_ancient_armel_build/comment_2_0b02f96699e9ec9447e336b74b255801._comment deleted file mode 100644 index 5964c7a486..0000000000 --- a/doc/todo/add_ancient_armel_build/comment_2_0b02f96699e9ec9447e336b74b255801._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="http://schnouki.net/" - nickname="Schnouki" - avatar="http://cdn.libravatar.org/avatar/5f6406e9db28564121169f0051645b8c30a12a20ca7bc40287ac9bf2cd3ad283" - subject="comment 2" - date="2017-02-02T22:05:23Z" - content=""" -+1. Can't upgrade the kernel on my NAS (Synology DS413j, running Linux 2.6.32...), so I'm stuck with git-annex 5.20150731... -"""]] diff --git a/doc/todo/add_ancient_armel_build/comment_3_eaeb9aec42fde6b8a25199b1e1e6a78c._comment b/doc/todo/add_ancient_armel_build/comment_3_eaeb9aec42fde6b8a25199b1e1e6a78c._comment deleted file mode 100644 index b645d7b6c1..0000000000 --- a/doc/todo/add_ancient_armel_build/comment_3_eaeb9aec42fde6b8a25199b1e1e6a78c._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2017-02-20T16:22:58Z" - content=""" -The build is set up; it's not run to completion yet, but fingers crossed it -will eventually. -"""]] diff --git a/doc/todo/add_ancient_armel_build/comment_4_da51390e7b67cd1dc075c793e99d4367._comment b/doc/todo/add_ancient_armel_build/comment_4_da51390e7b67cd1dc075c793e99d4367._comment deleted file mode 100644 index 7ef936ff96..0000000000 --- a/doc/todo/add_ancient_armel_build/comment_4_da51390e7b67cd1dc075c793e99d4367._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2018-10-03T16:58:26Z" - content=""" -Unfortunately the build failed in an obscure way while installing the deps -of git-annex back when I tried to set it up. - -I tried it again today, and there's no longer a binary of stack for armel, -only for armhf, so I don't think this build can be provided, at least not -without quite a lot of backporting. -"""]] diff --git a/doc/todo/add_import_--to_command.mdwn b/doc/todo/add_import_--to_command.mdwn deleted file mode 100644 index 1b2683932a..0000000000 --- a/doc/todo/add_import_--to_command.mdwn +++ /dev/null @@ -1,8 +0,0 @@ -I have a bunch of files I want to track with `git-annex` that are sitting in an external drive. I have a repository sitting on my laptop, but I don't actually have enough free disk space on my laptop to import the files. Really, I just want the file content to be sitting in a special remote. I was thinking the following workflow could be useful: - - cd ~/my-laptop-repo - git-annex import --to=s3-remote /mnt/usb-drive/myfiles - -The proposed `--to=remote` option would add the files to my repo as `import` normally does, but it wouldn't every keep the content in the repo, the only copy would now sit in `s3-remote`. As little disk space as possible would be staged temporarily in `~/my-laptop-repo`. Perhaps the easiest option would be to import a file normally, but them immediately do a `move` to `s3-remote`? But, ideally for larger files, we would want to stream them directly from `/mnt/usb-drive/myfiles` to `s3-remote` without ever staging them at `~/my-laptop-repo`. - -> [[done]] --[[Joey]] diff --git a/doc/todo/add_import_--to_command/comment_1_b70d10190722c0d21c2343fe51e38414._comment b/doc/todo/add_import_--to_command/comment_1_b70d10190722c0d21c2343fe51e38414._comment deleted file mode 100644 index 684f3aff00..0000000000 --- a/doc/todo/add_import_--to_command/comment_1_b70d10190722c0d21c2343fe51e38414._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="Ilya_Shlyakhter" - avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" - subject="comment 1" - date="2019-01-14T16:11:39Z" - content=""" -Cf. [[todo/import_tree]]. - -"""]] diff --git a/doc/todo/add_import_--to_command/comment_2_c7a95793a46ed948301b01a1f29bfac1._comment b/doc/todo/add_import_--to_command/comment_2_c7a95793a46ed948301b01a1f29bfac1._comment deleted file mode 100644 index 11ac58a048..0000000000 --- a/doc/todo/add_import_--to_command/comment_2_c7a95793a46ed948301b01a1f29bfac1._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="lykos@d125a37d89b1cfac20829f12911656c40cb70018" - nickname="lykos" - avatar="http://cdn.libravatar.org/avatar/085df7b04d3408ba23c19f9c49be9ea2" - subject="comment 2" - date="2019-01-21T11:57:07Z" - content=""" -Import tree won't help here. Actually I've got the same problem on one of my machines. I'm currently using a script that imports a bunch of files and moves them to the remote, then imports some more files and so on. It's quite cumbersome, to be honest. So yes, I'd appreciate `git annex import --to ` as well. (Same with `git annex move --from remote-a --to remote-b` actually.) -"""]] diff --git a/doc/todo/add_import_--to_command/comment_3_f393450ae2daa14e1e7008db2b3c7dc8._comment b/doc/todo/add_import_--to_command/comment_3_f393450ae2daa14e1e7008db2b3c7dc8._comment deleted file mode 100644 index fcc2eddd14..0000000000 --- a/doc/todo/add_import_--to_command/comment_3_f393450ae2daa14e1e7008db2b3c7dc8._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2019-01-22T15:39:49Z" - content=""" -Import tree + move --from --to would be equivilant functionality to import --to. -The directory you want to import "to" would be set up as a directory -special remote and import tree used on it. - -However, it seems to me that if I wanted to accomplish this today, I'd -simply make a clone of my git-annex repository onto the external drive, -move the files I want into that clone (a cheap operation as it's the same -drive), and `git annex copy --to s3-remote` -"""]] diff --git a/doc/todo/add_import_--to_command/comment_4_4a30627ac78b32911604c3377b958cd0._comment b/doc/todo/add_import_--to_command/comment_4_4a30627ac78b32911604c3377b958cd0._comment deleted file mode 100644 index 20d03ae2c1..0000000000 --- a/doc/todo/add_import_--to_command/comment_4_4a30627ac78b32911604c3377b958cd0._comment +++ /dev/null @@ -1,40 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2019-11-19T16:58:06Z" - content=""" -Since git-annex does now support importtree from directory special remotes, -you can almost get what you said you want by: - - git annex initremote usb-drive type=directory directory=/mnt/usb-drive/myfiles \ - exporttree=yes importtree=yes encryption=none - -Then `git annex import master --from usb-drive` will import the files -into a usb-drive/master branch that you can merge. And you can run it -repeatedly to import new and changed files from the directory. - -So then you have the files sitting in a special remote like you wanted. -Namely the directory special remote on the USB drive. Only problem is that -importing the files does also copy them into the git-annex repo. So you'd -have to drop the files again, assuming you had disk space for them all -to begin with. - -I wonder, if it were possible to import the files without add their content -to the repo you ran the import from, leaving them on the special remote, -would that meet your use case? That seems like something it would be -possible to add. - -It would still probably have to copy the file into the local repo, in order -to hash it, and then just delete the content from the local repo. Of course -when the file is in a directory on the local system, that's not strictly -necessary; it could do the hashing of the file in place. But that would -need an extension to the special remote API to hash a file. - -But like I said in my other comment, I'd just clone my git-annex repo onto the -drive and add the files to the repo there. Avoids all this complication. -You'd need to provide a good justification for why you can't do that for -me to pursue this any further. - -(As far as adding a --to switch to import, [[transitive_transfers]] -discusses this kind of thing, and some issues with implementing that.) -"""]] diff --git a/doc/todo/add_import_--to_command/comment_5_549edb19b222bf4bd1b87aceba007005._comment b/doc/todo/add_import_--to_command/comment_5_549edb19b222bf4bd1b87aceba007005._comment deleted file mode 100644 index ce52fec86e..0000000000 --- a/doc/todo/add_import_--to_command/comment_5_549edb19b222bf4bd1b87aceba007005._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2023-06-23T16:38:50Z" - content=""" -It's now possible to `git-annex import --no-content --from -a-directory-special-remote`. So then you have the files sitting on the -external drive and not filling up the git repository. - -And `git-annex copy --from foo --to bar` is also supported, so you can -use that to send the files over the S3 or wherever. - -So, I think this todo can be closed. -"""]] diff --git a/doc/todo/additional_git-annex-config_settings__63__.mdwn b/doc/todo/additional_git-annex-config_settings__63__.mdwn deleted file mode 100644 index b31427ebee..0000000000 --- a/doc/todo/additional_git-annex-config_settings__63__.mdwn +++ /dev/null @@ -1,5 +0,0 @@ -Should `annex.gitaddtoannex` and `annex.addsmallfiles` be [[`git-annex-config`|git-annex-config]] settings, given that `annex.largefiles`, `annex.dotfiles` and `annex.addunlocked` already are? -Also, [maybe](https://git-annex.branchable.com/todo/Avoid_lengthy___34__Scanning_for_unlocked_files_...__34__/#comment-85cb4d3eac345df7b08a31ea4bd810f6) make `annex.supportunlocked` a `git-annex-config` setting as well. - - -> [[rejected|done]] --[[Joey]] diff --git a/doc/todo/additional_git-annex-config_settings__63__/comment_1_74d61d6a4412223092f468628d3f2f45._comment b/doc/todo/additional_git-annex-config_settings__63__/comment_1_74d61d6a4412223092f468628d3f2f45._comment deleted file mode 100644 index a110770fd8..0000000000 --- a/doc/todo/additional_git-annex-config_settings__63__/comment_1_74d61d6a4412223092f468628d3f2f45._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="https://christian.amsuess.com/chrysn" - nickname="chrysn" - avatar="http://christian.amsuess.com/avatar/c6c0d57d63ac88f3541522c4b21198c3c7169a665a2f2d733b4f78670322ffdc" - subject="+1 on supportunlocked in git-annex config" - date="2021-07-09T15:25:53Z" - content=""" -supportunlocked would definitely make sense; it can help protect against users accidentally checking in a file in the wrong mode even though it doesn't fit the repo's workflows. - -The alternative to setting this across all repos using git-annex config, at least in my workflows, would be to set a git hook to reject pushes (to master or synced/master) that contain unlocked files, but that sounds way more complex. -"""]] diff --git a/doc/todo/additional_git-annex-config_settings__63__/comment_2_2973d438e8f3fda50cfb151534a8ec73._comment b/doc/todo/additional_git-annex-config_settings__63__/comment_2_2973d438e8f3fda50cfb151534a8ec73._comment deleted file mode 100644 index b9c837ba3e..0000000000 --- a/doc/todo/additional_git-annex-config_settings__63__/comment_2_2973d438e8f3fda50cfb151534a8ec73._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="https://christian.amsuess.com/chrysn" - nickname="chrysn" - avatar="http://christian.amsuess.com/avatar/c6c0d57d63ac88f3541522c4b21198c3c7169a665a2f2d733b4f78670322ffdc" - subject="+1 on supportunlocked in git-annex config" - date="2021-07-09T15:26:13Z" - content=""" -supportunlocked would definitely make sense; it can help protect against users accidentally checking in a file in the wrong mode even though it doesn't fit the repo's workflows. - -The alternative to setting this across all repos using git-annex config, at least in my workflows, would be to set a git hook to reject pushes (to master or synced/master) that contain unlocked files, but that sounds way more complex. -"""]] diff --git a/doc/todo/additional_git-annex-config_settings__63__/comment_3_3abeddccdf5c742962b25b7dbe8fb69e._comment b/doc/todo/additional_git-annex-config_settings__63__/comment_3_3abeddccdf5c742962b25b7dbe8fb69e._comment deleted file mode 100644 index 6891dd4e89..0000000000 --- a/doc/todo/additional_git-annex-config_settings__63__/comment_3_3abeddccdf5c742962b25b7dbe8fb69e._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Ilya_Shlyakhter" - avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" - subject="comment 3" - date="2021-07-09T16:04:47Z" - content=""" -Related: [[todo/add_maxextensionlength_to_git-annex-config]] -"""]] diff --git a/doc/todo/additional_git-annex-config_settings__63__/comment_4_1d777383fda7c50d630e19ec1d1ade52._comment b/doc/todo/additional_git-annex-config_settings__63__/comment_4_1d777383fda7c50d630e19ec1d1ade52._comment deleted file mode 100644 index 11ad13656e..0000000000 --- a/doc/todo/additional_git-annex-config_settings__63__/comment_4_1d777383fda7c50d630e19ec1d1ade52._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2021-07-12T14:49:12Z" - content=""" -annex.supportunlocked is documented to be about avoiding -extra work needed to support unlocked files, and not -about preventing them from being added to a branch. - -In general, I do not want to add global config settings for every single -git config setting. So if your only rartionalle for adding global configs -is that other global configs that are somewhat related exist, that is not a -compelling reason to me. -"""]] diff --git a/doc/todo/additional_git-annex-config_settings__63__/comment_5_ba2cbf8dcf14a1d0b20621b48b8e8600._comment b/doc/todo/additional_git-annex-config_settings__63__/comment_5_ba2cbf8dcf14a1d0b20621b48b8e8600._comment deleted file mode 100644 index 4bfab326ab..0000000000 --- a/doc/todo/additional_git-annex-config_settings__63__/comment_5_ba2cbf8dcf14a1d0b20621b48b8e8600._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="Ilya_Shlyakhter" - avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" - subject="comment 5" - date="2021-07-12T15:30:25Z" - content=""" ->I do not want to add global config settings for every single git config setting - -Of course. Just the settings that reflect the nature of the repo data and the intended usage processes. -E.g. I want to be able to enforce that a repo be locked-files-only. Or to set `maxextensionlength` based on the -fact that the repo stores `.fastq.gz` files. - ->annex.supportunlocked is documented to be about avoiding extra work needed to support unlocked files, and not about preventing them from being added to a branch - -To me it seems more natural to have it prevent the addition of unlocked files, and be a global setting. -"""]] diff --git a/doc/todo/additional_git-annex-config_settings__63__/comment_6_4ec0261f3b8bc6d2492fa3dabb0121e5._comment b/doc/todo/additional_git-annex-config_settings__63__/comment_6_4ec0261f3b8bc6d2492fa3dabb0121e5._comment deleted file mode 100644 index 820de1eddb..0000000000 --- a/doc/todo/additional_git-annex-config_settings__63__/comment_6_4ec0261f3b8bc6d2492fa3dabb0121e5._comment +++ /dev/null @@ -1,35 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 6""" - date="2021-07-12T15:42:52Z" - content=""" -Almost anything can be argued to reflect the nature of some repo in some way. -That's not a useful criteria. - -And the same could be argued about many git configs, and of course they -cannot be set globally, and noone is bothered much by this, because we can -all arrange for git configs to be set after cloning a repo. - -I don't know what the right criteria is, but I do know I don't want to -force users to have to worry about overriding every possible config locally -because it's been set globally. git-annex should not behave in a near infinity -of different ways in clones of different repos, because that would make its -starting behavior impossible to understand. - -So the more there are requests for more global configs, the more it seems -like adding any global configs, without a strong criteria, is not a good -idea. - -Some global configs that do make sense are numcopies and required copies -settings, because those values need to be coordinated globally to make sure -enough copies are preserved. Similarly, preferred content because one -repo needs to know what is preferred content of another repo. And I think -annex.securehashesonly makes sense as a global, to avoid adding files -with insecure hashes that would then not be accessible in repos with -that config set. annex.largefiles mostly makes sense because .gitattributes -has a whitespace problem which it avoids, and it's similar to using -.gitattributes (although not identical). It feels on the edge. - -annex.synccontent etc are explicitly about changing the default behavior -of a command. At the moment I feel like they were a bad idea. -"""]] diff --git a/doc/todo/additional_git-annex-config_settings__63__/comment_7_c1768ab8568c8e58525ed40e76cc13f2._comment b/doc/todo/additional_git-annex-config_settings__63__/comment_7_c1768ab8568c8e58525ed40e76cc13f2._comment deleted file mode 100644 index 10d506ec08..0000000000 --- a/doc/todo/additional_git-annex-config_settings__63__/comment_7_c1768ab8568c8e58525ed40e76cc13f2._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Ilya_Shlyakhter" - avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" - subject="comment 7" - date="2021-07-13T03:05:45Z" - content=""" -Realized that for the use case I was thinking of -- setting repo-wide policies such as locked-files-only -- git-annex-config isn't right, since the value can be changed by any clone. For that use case, have to configure hooks/triggers on the central repo to reject pull requests that break the policy. -"""]] diff --git a/doc/todo/additional_git-annex-config_settings__63__/comment_8_845a504ea9f038a19a21d8a985585b2b._comment b/doc/todo/additional_git-annex-config_settings__63__/comment_8_845a504ea9f038a19a21d8a985585b2b._comment deleted file mode 100644 index 777399c181..0000000000 --- a/doc/todo/additional_git-annex-config_settings__63__/comment_8_845a504ea9f038a19a21d8a985585b2b._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 8""" - date="2022-04-04T17:54:33Z" - content=""" -I have opened a new todo, -[[need_a_clear_criteria_for_adding_git-annex-config_settings]]. - -I am going to close this one, because I'm certianly not adding a bunch of -configs listed here without that crieria. -"""]] diff --git a/doc/todo/aeson_2.0.mdwn b/doc/todo/aeson_2.0.mdwn deleted file mode 100644 index 9a30ac968d..0000000000 --- a/doc/todo/aeson_2.0.mdwn +++ /dev/null @@ -1,7 +0,0 @@ -Need to update to allow building with aeson 2.0. - -I have a patch in my mailbox from sternenseemann that -mostly gets there, but it needs some revision to fix a problem I spotted. ---[[Joey]] - -[[done]] diff --git a/doc/todo/annex.addunlocked_in_gitattributes.mdwn b/doc/todo/annex.addunlocked_in_gitattributes.mdwn deleted file mode 100644 index 22ff605a53..0000000000 --- a/doc/todo/annex.addunlocked_in_gitattributes.mdwn +++ /dev/null @@ -1,13 +0,0 @@ -Make annex.addunlocked be able to be configured in .gitattributes, -the same way as annex.largefiles can be. - -This would be useful if certain filename extensions need to be unlocked to -be used, and others are desired to be kept locked. - -The annex.addunlocked git config is a boolean, but this gitattributes one -would effectively build up a file match expression. So it might then follow -that the git config should also be a file match expression, with "true" -being the same as "anything" and "false" the same as "nothing" for -back-compat. --[[Joey]] - -> This got accomplished by other means, [[done]] --[[Joey]] diff --git a/doc/todo/annex.addunlocked_in_gitattributes/comment_1_1d21cde3d21a9b5bb129b17b9e0fc95d._comment b/doc/todo/annex.addunlocked_in_gitattributes/comment_1_1d21cde3d21a9b5bb129b17b9e0fc95d._comment deleted file mode 100644 index b657a5043b..0000000000 --- a/doc/todo/annex.addunlocked_in_gitattributes/comment_1_1d21cde3d21a9b5bb129b17b9e0fc95d._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="yarikoptic" - avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" - subject="comment 1" - date="2022-02-18T20:18:04Z" - content=""" -Dear Joey, could you please elaborate on what \"other means\" it was accomplished through? we are on the expedition ATM to discover. -"""]] diff --git a/doc/todo/annex.addunlocked_in_gitattributes/comment_2_38f248f7596646dfc11248f0149c0beb._comment b/doc/todo/annex.addunlocked_in_gitattributes/comment_2_38f248f7596646dfc11248f0149c0beb._comment deleted file mode 100644 index f3b93cba26..0000000000 --- a/doc/todo/annex.addunlocked_in_gitattributes/comment_2_38f248f7596646dfc11248f0149c0beb._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="yarikoptic" - avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" - subject="comment 2" - date="2022-02-18T21:56:19Z" - content=""" -I guess it is the fact that this particular config variable is also supported by `git annex config` so could be set persistently (across clones) this way. -"""]] diff --git a/doc/todo/annex.addunlocked_in_gitattributes/comment_3_270c5156c8adce0aff86eda5f2746703._comment b/doc/todo/annex.addunlocked_in_gitattributes/comment_3_270c5156c8adce0aff86eda5f2746703._comment deleted file mode 100644 index bfbc9afeab..0000000000 --- a/doc/todo/annex.addunlocked_in_gitattributes/comment_3_270c5156c8adce0aff86eda5f2746703._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2022-02-21T19:40:50Z" - content=""" -Yes, annex.addunlocked with a matching expression was -implemented in 2019 and accomplishes the same thing -as this would have, without the problems of using git attributes. -"""]] diff --git a/doc/todo/annex.addunlocked_in_gitattributes/comment_4_d87e08ccdec98867dd52b629e6fad330._comment b/doc/todo/annex.addunlocked_in_gitattributes/comment_4_d87e08ccdec98867dd52b629e6fad330._comment deleted file mode 100644 index 67d8b41c87..0000000000 --- a/doc/todo/annex.addunlocked_in_gitattributes/comment_4_d87e08ccdec98867dd52b629e6fad330._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="yarikoptic" - avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" - subject="comment 4" - date="2022-02-21T21:53:03Z" - content=""" -just it is getting a bit \"confusing\" to use `.gitattributes` for some types of annotations (e.g., `largefiles`) for files patterns and then `git annex config` for others. Ideally there should either be a singular or equivalently expressive (not complimentary) multiple ways. - -> without the problems of using git attributes - -could you please remind (or point to prior composed list) of those? I personally just keep forgetting which one (first or last match) applies ;) -"""]] diff --git a/doc/todo/annex.addunlocked_in_gitattributes/comment_5_be25f3b3c2279eb3fd3ef69c4d1cb52d._comment b/doc/todo/annex.addunlocked_in_gitattributes/comment_5_be25f3b3c2279eb3fd3ef69c4d1cb52d._comment deleted file mode 100644 index d4436a15ca..0000000000 --- a/doc/todo/annex.addunlocked_in_gitattributes/comment_5_be25f3b3c2279eb3fd3ef69c4d1cb52d._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2022-02-25T17:28:20Z" - content=""" -git-annex config does work for both largefiles and addunlocked. - -gitattributes does not allow setting an attribute containing a space -and so complex expressions, which either of these can have, become very -annoying to shoehorn in. gitattributes also adds a round-trip overhead -to query it for every file. -"""]] diff --git a/doc/todo/annex.addunlocked_in_gitattributes/comment_6_f672dee4c1712e0d20dd820999598d87._comment b/doc/todo/annex.addunlocked_in_gitattributes/comment_6_f672dee4c1712e0d20dd820999598d87._comment deleted file mode 100644 index 09941c90b5..0000000000 --- a/doc/todo/annex.addunlocked_in_gitattributes/comment_6_f672dee4c1712e0d20dd820999598d87._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="yarikoptic" - avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" - subject="question about backend" - date="2022-02-28T22:42:33Z" - content=""" -ATM we rely on .gitattributes to set default (to DataLad datasets) backend to be MD5E, so every dataset then is guaranteed to have `.gitattributes`. We also rely in many configurations on setting `largefiles` for different extensions within `.gitattributes`. I think this is two primary target use cases. You say that having `.gitattributes` slows down `git` (and `git-annex`) operations -- so would you recommend to switch to specifying those (backend, largefiles) via `git annex config` instead? - -In [datalad/issues/5383 (Stop using .gitattributes for annex.largefiles config )](https://github.com/datalad/datalad/issues/5383) note was that default backend cannot be specified via `git-annex config` -- is that still the case? -"""]] diff --git a/doc/todo/annex.addunlocked_in_gitattributes/comment_7_d06e3412f07d98c784b420d82d2d8d49._comment b/doc/todo/annex.addunlocked_in_gitattributes/comment_7_d06e3412f07d98c784b420d82d2d8d49._comment deleted file mode 100644 index dc4eea27e7..0000000000 --- a/doc/todo/annex.addunlocked_in_gitattributes/comment_7_d06e3412f07d98c784b420d82d2d8d49._comment +++ /dev/null @@ -1,34 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 7""" - date="2022-08-01T16:10:31Z" - content=""" -Well, moving your annex.largefiles settings from gitattributes to `git-annex -config` won't speed up queries for it, because the gitattribute overrides -the `git-annex config` setting. And so git-annex still has to do the work -of querying for the gitattribute anyway, even when it's not set. - -In [[!commit 4acbb40112aa73dcde63841d8d8c04c433f6a806]] I benchmarked that -as making `git-annex add` 2% slower than it would be otherwise (excluding -hashing). We will just have to live with that, unless the gitattribute -can eventually somehow be deprecated. -That is a good lesson about the risks of adding more gitattributes. - -annex.backend is not currently configurable by `git-annex config`. -It would be listed in its man page if it were. - -I'd support adding that, but annex.backend is currently the name of a -single backend, so this would not allow setting the backend differently for -different filenames. Which is something that gitattributes can do. So it -would need annex.backend to be expanded, so it can specify different -backends for different filenames or other properties. I don't know how that -syntax would look; the syntax git-annex currently uses for annex.largefiles -etc is not suitable here. It would certianly be an added complication. - -Also, it seems that the reasoning that made the annex.largefiles -gitattributes override `git-annex config` would also make sense for -annex.backend, and if so there would be no performance benefit to moving -it. I'm not sure what that reasoning was. Possibly that there -might be cases where the desired value depends on the branch that's checked -out. -"""]] diff --git a/doc/todo/annex.addunlocked_in_gitattributes/comment_8_73a402df13f30b279dfd2914adec5f35._comment b/doc/todo/annex.addunlocked_in_gitattributes/comment_8_73a402df13f30b279dfd2914adec5f35._comment deleted file mode 100644 index a688bb6937..0000000000 --- a/doc/todo/annex.addunlocked_in_gitattributes/comment_8_73a402df13f30b279dfd2914adec5f35._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 8""" - date="2022-08-01T16:37:20Z" - content=""" -I would support adding annex.backend to `git-annex config` using the -current simple value though. There is benefit to doing it for consistencty, -surely. If a more complicated syntax is needed someone -can still use gitattributes or git-annex could be changed later. - -Please open a new todo if this would be useful to you.. -"""]] diff --git a/doc/todo/annex.addunlocked_in_gitattributes/comment_9_02866c504617c2914eff01d3529d5cc7._comment b/doc/todo/annex.addunlocked_in_gitattributes/comment_9_02866c504617c2914eff01d3529d5cc7._comment deleted file mode 100644 index ea7d96fc27..0000000000 --- a/doc/todo/annex.addunlocked_in_gitattributes/comment_9_02866c504617c2914eff01d3529d5cc7._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 9""" - date="2022-08-01T16:39:40Z" - content=""" -I also think that the thinking in this comment is worth considering: - - -Those are valid reasons to prefer to be able to set things like this -in gitattributes rather than the global config. Those kinds of -considerations are why the global config always has a local way to override -it. Sometimes that is necessarily .git/config, not .gitattributes though. -"""]] diff --git a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed.mdwn b/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed.mdwn deleted file mode 100644 index ffc25229e6..0000000000 --- a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed.mdwn +++ /dev/null @@ -1,29 +0,0 @@ -As seen in datalad -is having to parse error messages to determine when nonexistant -filenames were passed to git-annex. It would be better if there was some -kind of API that it could use, since error messages can (and have) changed. - -What would this API look like? It seems it could not be a special -exit status of git-annex, because exit status 101 is already allocated -for indicating when things like --size-limit and --time-limit have caused -it to skip processing certian files. - -Perhaps something like `--json-error-unmatch` which would make it output -an additional line with a JSON record indicating that git ls-files ---error-unmatch had errored out. - -Or `--json` could enable this additional JSON record output. -The advantage would be that datalad would not need to check the git-annex -version to see if the option is supported. It would need to output a -JSON record even when git-ls-files did not fail. Perhaps something like -this: - - {"git-ls-files exit status": 0} - {"git-ls-files exit-status": 1} - -This does risk breaking things that parse the existing JSON and fall -over on the new record, but I think git-annex should be free to add new -records and fields to its JSON output in general, and it has probably -at least added new fields before. --[[Joey]] - -> [[done]] --[[Joey]] diff --git a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_10_32ebb1ea1278b891b58dd3c4a8546453._comment b/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_10_32ebb1ea1278b891b58dd3c4a8546453._comment deleted file mode 100644 index 1c4470eba6..0000000000 --- a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_10_32ebb1ea1278b891b58dd3c4a8546453._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 10""" - date="2023-04-26T16:42:24Z" - content=""" -I decided to rename it "message-id" since it could go on warning or other -messages too conceivably. -"""]] diff --git a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_1_672457817d5dca898f5aefbe97da9b23._comment b/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_1_672457817d5dca898f5aefbe97da9b23._comment deleted file mode 100644 index d37afaf7d7..0000000000 --- a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_1_672457817d5dca898f5aefbe97da9b23._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="yarikoptic" - avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" - subject="comment 1" - date="2022-02-28T18:48:50Z" - content=""" -> This does risk breaking things that parse the existing JSON and fall over on the new record, but I think git-annex should be free to add new records and fields to its JSON output in general, and it has probably at least added new fields before. - -Although sounds logical, FWIW, not sure if DataLad would be robust to such an assumption since I don't think it was exercised before and indeed all returned records were typically associated with some particular `\"path\"` so our code might rely on that. Also it makes it a bit unclear on what datalad code should do with \"unrecognized\" outputs -- are they sign of a problem, or could be safely ignored? - -Ideally, IMHO and IIRC, DataLad should not concern itself with exit codes of `git` commands which `git-annex` ran -- it should be up to `git-annex` to decide on what any particular underlying `git` invocation supposed to do etc. I guess only in the cases of some faulty, unknown to git-annex how to handle, behavior where git-annex itself reports an error, might be worth including details on underlying failed `git` invocation in the output, so datalad also could channel it to the user. -"""]] diff --git a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_2_b6a76bfc063dcba439b2ead004caabc9._comment b/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_2_b6a76bfc063dcba439b2ead004caabc9._comment deleted file mode 100644 index 4212e006f7..0000000000 --- a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_2_b6a76bfc063dcba439b2ead004caabc9._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-02-28T19:40:13Z" - content=""" -But datalad is apparently concerning itself with just that, since -there was a bug when git-annex's "not found" stderr changed. - -I do think this needs to have a good motivation before it's implemented, -and that's currently all I have, that datalad for some reason cares about -it. Maybe there is not a good reason for datalad to care though.. - -You may well be right about adding new JSON objects breaking parsers. -"""]] diff --git a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_3_3f0872f13160a77a82bd5a95fc3f397d._comment b/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_3_3f0872f13160a77a82bd5a95fc3f397d._comment deleted file mode 100644 index b48dd361f7..0000000000 --- a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_3_3f0872f13160a77a82bd5a95fc3f397d._comment +++ /dev/null @@ -1,38 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2023-04-25T16:57:58Z" - content=""" -I see that datalad parses the error messages to get the filename that -caused it, and so an exit status is not enough information. - -(See `./datalad/support/annex_utils.py` -`_get_non_existing_from_annex_output`) - -So the additional json would need to include the filename that didn't -exist. - -[this closed todo](https://git-annex.branchable.com/projects/datalad/bugs-done/copy_does_not_reflect_some_failed_copies_in_--json_output/) -argues persuasively that --json-error-messages cannot be extended to -include these errors as-is. In particular, putting them in the current -json would mean outputting a record without a "key" field in situations -where that field is always present, so json consumers might -misbehave without it. - -bpoldrack makes a good point -[here](https://github.com/datalad/datalad/pull/6510#issuecomment-1057320621): - -> Ideally, there'd be some indication of the problem that isn't meant to -> be a message to the user and not (so easily) subject to change - pretty -> much something that's akin to an exception name. That's because there -> are possible errors that we can deal with from within datalad if we can -> "understand" the problem - not everything needs to be passed onto the -> user. - -So eg: - - {"exception":"UNIQUEID", "file":"foo", "error-messages":["foo not found"]} - -That seems about right to me, and it future proofs git-annex to be able to -report other exceptions in json output later on. -"""]] diff --git a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_4_0217c8acd2a9eb3bba1fc497856cd96a._comment b/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_4_0217c8acd2a9eb3bba1fc497856cd96a._comment deleted file mode 100644 index d977b33910..0000000000 --- a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_4_0217c8acd2a9eb3bba1fc497856cd96a._comment +++ /dev/null @@ -1,27 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2023-04-25T17:49:23Z" - content=""" -What should enable that new json output for exceptions, ---json-error-messages or a new option like --json-exceptions? - -The risk is that, since this is a new object type, it breaks a json -consumer that expects to find a `{"command":...}` object. - -I do think that git-annex should be free to add new fields to existing json -objects. But adding a new object type seems considerably more risky. -git-annex has never done that before (eg --json-progress outputs the -"command" object with updated values). - -I took a look at datalad's parsing of json from git-annex, to see what it might -do if it got a new object type unexpectedly. I'm not convinced it wouldn't -misbehave. For example, in `datalad/interface/results.py` `annexjson2result`: - - res['status'] = 'ok' if d.get('success', False) is True else 'error' - -Without a "status" field this will yield "error" which could go on to -cause unexpected behavior - -So, I think this would need to be a new --json-exceptions option. -"""]] diff --git a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_5_7501eed2e6cb732966277ef2f1ff28e9._comment b/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_5_7501eed2e6cb732966277ef2f1ff28e9._comment deleted file mode 100644 index 3bf70bb5fb..0000000000 --- a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_5_7501eed2e6cb732966277ef2f1ff28e9._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="yarikoptic" - avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" - subject="comment 5" - date="2023-04-25T18:31:20Z" - content=""" -Yes -- having some UNIQUEIDs for errors might be quite great! - -As for specific field to identify what that error applicable to: the other changed behavior which broke our code/tests was the change to behavior of `annex info` to no longer list unknown to annex path in `\"file\"` and rather list it in `\"input\"` for which the [PR](https://github.com/datalad/datalad/pull/7372/files) was cooked up already. So may be here it also just should be `\"input\"` field if `annex` desires to not make claims on either it is a `\"file\"` or may be some \"pathspec\" (well -- in DataLad nobody was brave enough to open that -- path-vs-pathspec -- pandora box [see issues/6933](https://github.com/datalad/datalad/issues/6933))? -"""]] diff --git a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_6_7a5b3d2694a48e4e44436f6bf7ed6642._comment b/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_6_7a5b3d2694a48e4e44436f6bf7ed6642._comment deleted file mode 100644 index d60f16611f..0000000000 --- a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_6_7a5b3d2694a48e4e44436f6bf7ed6642._comment +++ /dev/null @@ -1,37 +0,0 @@ -[[!comment format=mdwn - username="yarikoptic" - avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" - subject="comment 6" - date="2023-04-25T18:45:58Z" - content=""" -Just for completeness -- do we really need a new type of record for this? In immediate use case which is [in the heart of the path escaping change discussion](https://git-annex.branchable.com/bugs/started_to_escape_characters_in_the_output/) let's consider other commands. E.g. `addurl` already reports `--json-errors` just fine in a similar case - -```shell -❯ git annex addurl --json --json-error-messages '\e[31mfo\o\e[0m' -{\"command\":\"addurl\",\"error-messages\":[\"git-annex: bad url \\e[31mfo\\o\\e[0m\"],\"file\":null,\"input\":[\"\\e[31mfo\\o\\e[0m\"],\"success\":false} -addurl: 1 failed -``` - -so why just not to make `add`'s - -```shell -❯ git annex add --json --json-error-messages '\e[31mfo\o\e[0m' -git-annex: \e[31mfo\o\e[0m not found -add: 1 failed -``` - -into - -```shell -{\"command\":\"add\",\"error-messages\":[\"git-annex: \\e[31mfo\\o\\e[0m not found\"],\"file\":null,\"input\":[\"\\e[31mfo\\o\\e[0m\"],\"success\":false} -add: 1 failed -``` - -or if you want to expand with unambigous ID: - -``` -{\"command\":\"add\",\"error-messages\":[\"git-annex: \\e[31mfo\\o\\e[0m not found\"],\"file\":null,\"input\":[\"\\e[31mfo\\o\\e[0m\"],\"success\":false, \"errorid\": \"NOT_FOUND\"} -add: 1 failed -``` -or alike without requiring some new CLI option? -"""]] diff --git a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_8_2550e1760dcfb90c9c2ca1ee145adcf1._comment b/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_8_2550e1760dcfb90c9c2ca1ee145adcf1._comment deleted file mode 100644 index a72b9b3286..0000000000 --- a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_8_2550e1760dcfb90c9c2ca1ee145adcf1._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 8""" - date="2023-04-25T22:45:24Z" - content=""" -Ok, implemented the simple alternative. Here's how it looks: - - joey@darkstar:~/tmp/xxx>git-annex add 'dne' --json --json-error-messages - {"command":"add","error-messages":["git-annex: dne not found"],"errorid":"FileNotFound","file":"dne","input":["dne"],"success":false} - add: 1 failed - -The errorid will remain stable. I can add those to other error messages -now, on request BTW. - -Note that when git-annex relies on `git ls-files --error-unmatch` to -complain about nonexistant or non-git files, the error messages from -git will still be displayed to stderr, not this nice json. So -datalad will need to keep its parser for that part. -"""]] diff --git a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_8_f34293aa30b8b9abc71fac4e713aded3._comment b/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_8_f34293aa30b8b9abc71fac4e713aded3._comment deleted file mode 100644 index 1bec987de0..0000000000 --- a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_8_f34293aa30b8b9abc71fac4e713aded3._comment +++ /dev/null @@ -1,34 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""Re: comment 6""" - date="2023-04-25T21:10:21Z" - content=""" -Oh well spotted yarikoptic! I wish I had noticed your comment 2 hours ago, -but I was head down implementing --json-exceptions. - -Ok, so `addurl` does `giveup "bad url"` and that does indeed result in json -output that lacks a `key` and has `"file":null` as well. - -I'm sure that somewhere in `git-annex add`, it's possible for it to -`giveup` with an error too. Oh of course... a device file! - - joey@darkstar:/home/joey/tmp/xxx>git-annex add --force-small null --json --json-error-messages - {"command":"add","error-messages":["git-annex: null is not a regular file"],"file":"null","input":["null"],"note":"adding content to git repository","success":false} - add: 1 failed - -Other perhaps more likely cases where add can `giveup` include when it's unable -to remove all write perms due to an xattr, and probably some permissions -problems. - -So, json consumers of add already have to deal with the `key` being missing. - -Now, it might be that some git-annex commands don't `giveup`, so this would be -a new complication for consumers of their json. But so would adding a `giveup` -for any reason to a command, and I don't worry about that. - -While I've implemented --json-exceptions, I don't like the complexity, so this new -information inclines me to rip it back out and instead handle the case of the -nonexistant file like: - - {"command":"add","exception":"UNIQUEID","file":"foo","input":["foo"],"error-messages":["foo not found"],"success":false} -"""]] diff --git a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_9_d7e879ae1e7ae5e27e15dab9d5b99f16._comment b/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_9_d7e879ae1e7ae5e27e15dab9d5b99f16._comment deleted file mode 100644 index d3b130a16a..0000000000 --- a/doc/todo/api_for_telling_when_nonexistant_or_non_git_files_passed/comment_9_d7e879ae1e7ae5e27e15dab9d5b99f16._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="yarikoptic" - avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" - subject="comment 9" - date="2023-04-26T16:38:26Z" - content=""" -quick/small one -- given that there is `\"error-messages\"`, would it may be make sense to name new one `\"error-id\"` and not `\"errorid\"` for consistency? -"""]] diff --git a/doc/todo/arm64_autobuild_broken_since_2022.mdwn b/doc/todo/arm64_autobuild_broken_since_2022.mdwn deleted file mode 100644 index fcaff06c3b..0000000000 --- a/doc/todo/arm64_autobuild_broken_since_2022.mdwn +++ /dev/null @@ -1,10 +0,0 @@ -Hi joey, - -Could you have a look at the arm64 autobuilds? The last successful build is from January 2022, the recent ones (which still run) fail: https://downloads.kitenet.net/git-annex/autobuild/arm64/ - -Would be awesome to use latest git-annex on my Raspberry PIs etc. 🙂 - -Cheers, -Yann - -> [[fixed_in_april_already|done]] --[[Joey]] diff --git a/doc/todo/arm64_autobuild_broken_since_2022/comment_1_0642002d8ad1eb318de5fdb63956a2fc._comment b/doc/todo/arm64_autobuild_broken_since_2022/comment_1_0642002d8ad1eb318de5fdb63956a2fc._comment deleted file mode 100644 index 2d51d5f855..0000000000 --- a/doc/todo/arm64_autobuild_broken_since_2022/comment_1_0642002d8ad1eb318de5fdb63956a2fc._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-06-12T16:44:41Z" - content=""" -Um, the autobuild you linked to is a successful build, and the last release of -git-annex included a current arm64 build. - -I stood up a new arm64 build box in April... -"""]] diff --git a/doc/todo/arm64_autobuild_broken_since_2022/comment_2_48e40f76e0480adc965505543a4f25e5._comment b/doc/todo/arm64_autobuild_broken_since_2022/comment_2_48e40f76e0480adc965505543a4f25e5._comment deleted file mode 100644 index 355879eeb8..0000000000 --- a/doc/todo/arm64_autobuild_broken_since_2022/comment_2_48e40f76e0480adc965505543a4f25e5._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 2" - date="2023-06-12T21:12:29Z" - content=""" -Huh, this is very weird. I could've sworn that the date 202201 was all over the place this morning and was also the version one got when downloading the standalone arm64 version or the autobuild version. But now everything seems to be fine. Sorry for the confusion, then. -"""]] diff --git a/doc/todo/command_to_list_keys_for_old_versions_of_files.mdwn b/doc/todo/command_to_list_keys_for_old_versions_of_files.mdwn deleted file mode 100644 index 17fe085417..0000000000 --- a/doc/todo/command_to_list_keys_for_old_versions_of_files.mdwn +++ /dev/null @@ -1,46 +0,0 @@ -A command like `git-annex oldkeys $file` that lists the keys used for all -previous versions of the file. - -It would allow a workflow like eg: - - edit foo - git-annex add foo - git commit -m new\ foo - git-annex oldkeys foo | git-annex move --batch-keys --to bar - -Or this, to send someone the current version of a file, and also all old -versions: - - git-annex copy foo --to baz - git-annex oldkeys foo | git-annex copy --to baz - -Or like this: - - git-annex oldkeys foo | git-annex get --batch-keys - # proceed to diff between old versions of the file - # (although git-annex-diffdriver --get is another way to do this) - -Or this to make every old version visible as files to flip through in a -slideshow: - - n=0 - for k in $(git-annex oldkeys my.gif); do - $n=$(expr $n + 1) - ln -s $(git-annex examinkey $k --format='${objectpath}') my.$n.gif - done - ----- - -Is oldkeys the best name for this? `git-annex log` is already taken. - -Since this would be implemented on top of `git log --raw`, it would -be possible to support multiple files at once, or whole directories. - -If an old key is the same as the current key, should it list the old key or -not? If it did, then the move example above would move the current -version of the file away. And there are tricky cases involving renames -and reverts. So it seems that it ought to avoid -ever listing a key currently used by the file(s) it is run on -as an old key. - -> [[done]] --[[Joey]] diff --git a/doc/todo/command_to_list_keys_for_old_versions_of_files/comment_1_43c11d9467b4d4460aa97bfb88b4896f._comment b/doc/todo/command_to_list_keys_for_old_versions_of_files/comment_1_43c11d9467b4d4460aa97bfb88b4896f._comment deleted file mode 100644 index 34183374ed..0000000000 --- a/doc/todo/command_to_list_keys_for_old_versions_of_files/comment_1_43c11d9467b4d4460aa97bfb88b4896f._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="Oh yes please 🤩" - date="2023-08-22T16:48:58Z" - content=""" -`unusedkeys` would be consistently named with `git annex unused`. Would it make sense to teach `git annex unused` that? It could be taught to accept paths as arguments (it doesn't currently, right?) and then only operate on those to find old versions. Then no new command is necessary and people already know `unused`. The workflow would be: - -``` -# two-step -git annex unused file.txt -git annex move --unused --to there - -# new option to only list keys for further scripting -git annex unused --onlykeys file.txt | xargs ... -# maybe --batch is a better name 🤷 -``` -"""]] diff --git a/doc/todo/command_to_list_keys_for_old_versions_of_files/comment_2_5062a0f8b57fdebf74fef8a65ac0a0b2._comment b/doc/todo/command_to_list_keys_for_old_versions_of_files/comment_2_5062a0f8b57fdebf74fef8a65ac0a0b2._comment deleted file mode 100644 index 152a91ecc0..0000000000 --- a/doc/todo/command_to_list_keys_for_old_versions_of_files/comment_2_5062a0f8b57fdebf74fef8a65ac0a0b2._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 2" - date="2023-08-22T17:57:45Z" - content=""" -Or better: make `git annex unused` accept the common matching options. Then `git annex unused --include=myfile.txt` can be used for the same effect. -"""]] diff --git a/doc/todo/command_to_list_keys_for_old_versions_of_files/comment_3_3c605fbd4c20522e265ef080675c3669._comment b/doc/todo/command_to_list_keys_for_old_versions_of_files/comment_3_3c605fbd4c20522e265ef080675c3669._comment deleted file mode 100644 index e5f0ba4be4..0000000000 --- a/doc/todo/command_to_list_keys_for_old_versions_of_files/comment_3_3c605fbd4c20522e265ef080675c3669._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2023-08-22T19:02:26Z" - content=""" -This is certainly adjacent to unused, but it's not quite the same. - -I think users will generally want this to list keys even if they are -still used by some other files in the tree besides the files that they -are querying. (There may be cases where that is not desirable of course, -but at least 3 of the 4 examples I came up with seem desirable to operate -that way.) - -And a key listed by this command might be used by other branches, so unused -would not list it. -"""]] diff --git a/doc/todo/complete_annex.skipunknown_transition_in_2022.mdwn b/doc/todo/complete_annex.skipunknown_transition_in_2022.mdwn deleted file mode 100644 index a8de0c61b6..0000000000 --- a/doc/todo/complete_annex.skipunknown_transition_in_2022.mdwn +++ /dev/null @@ -1,6 +0,0 @@ -annex.skipunknown should be changed to default to false in early 2022. -This transition was started in mid 2020 and there should have been plenty -of time for people to learn about it and either set their preferred -behavior or change workflows for the new behavior. - -> [[done]] --[[Joey]] diff --git a/doc/todo/create_debug_logs_but_erase_them_on_success.mdwn b/doc/todo/create_debug_logs_but_erase_them_on_success.mdwn deleted file mode 100644 index 28e1ebc482..0000000000 --- a/doc/todo/create_debug_logs_but_erase_them_on_success.mdwn +++ /dev/null @@ -1,5 +0,0 @@ -To help understand hard-to-replicate failures, add an option to always generate a debug log but to erase it as a final step if an operation succeeds. If an operation fails, keep the log and print a message pointing to it. - -Maybe, save such logs somewhere under .git/annex and have a command to upload them (over https and encrypted with @joeyh's key?) to some server where they can be examined. - -> [[rejected|done]] --[[Joey]] diff --git a/doc/todo/create_debug_logs_but_erase_them_on_success/comment_1_72b1e3b25a221f42ea7919aaecd6012d._comment b/doc/todo/create_debug_logs_but_erase_them_on_success/comment_1_72b1e3b25a221f42ea7919aaecd6012d._comment deleted file mode 100644 index 5dcf61d1ff..0000000000 --- a/doc/todo/create_debug_logs_but_erase_them_on_success/comment_1_72b1e3b25a221f42ea7919aaecd6012d._comment +++ /dev/null @@ -1,21 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2020-03-16T18:09:30Z" - content=""" -I'm doubtful about this: - -* It's not super likely that a debug log will have enough information in it - to replicate a one-off problem. At best they generally provide some clues - that need to be confirmed with experiments, so the problem needs to be - reproduced before I can fix it. -* This risks accumulating a bunch of debug logs if the user is doing things - that fail. Things can fail for a wide array or reasons that do not need - to be debugged, eg git-annex get fails all the time for valid reasons - such as a host not being accessible or drive not being mounted. -* A debug log might contain some sensitive information, git-annex should - certianly not be sending them around without the user's eyes on them. -* If the user noticed git-annex is accumulating a bunch of debug logs, - and does feel they are sensitive information, they may lose trust in - git-annex. -"""]] diff --git a/doc/todo/create_debug_logs_but_erase_them_on_success/comment_2_8809f216aac308962917053ea24b9022._comment b/doc/todo/create_debug_logs_but_erase_them_on_success/comment_2_8809f216aac308962917053ea24b9022._comment deleted file mode 100644 index e28490bc0a..0000000000 --- a/doc/todo/create_debug_logs_but_erase_them_on_success/comment_2_8809f216aac308962917053ea24b9022._comment +++ /dev/null @@ -1,36 +0,0 @@ -[[!comment format=mdwn - username="lykos@d125a37d89b1cfac20829f12911656c40cb70018" - nickname="lykos" - avatar="http://cdn.libravatar.org/avatar/085df7b04d3408ba23c19f9c49be9ea2" - subject="comment 2" - date="2020-03-26T20:36:28Z" - content=""" -When testing a new version of git-annex-remote-googledrive, I often use a wrapper script like this: - -``` -# .zshrc.local - -f_git_annex () { - log_file=$(mktemp \"$(git rev-parse --git-dir 2>/dev/null)/annex/debug.XXX.log\" 2>/dev/null) - if [ $? -ne 0 ]; then - echo \"Error creating log file. Proceeding without logging.\" - git annex $@ - return $? - fi - - git annex $@ --debug 2> $log_file - if [ $? -ne 0 ]; then - echo \"Errors occurred. Debug log in $log_file\" - return $? - else - rm $log_file - fi - } -alias ga='f_git_annex' -``` -It's good enough for me in those cases. Changing the first line of the function to something like this -``` -log_file=$(umask 077; mktemp /tmp/annex.debug.XXX.log\" 2>/dev/null) -``` -would help against accumulating logs. -"""]] diff --git a/doc/todo/deferred_update_mode.mdwn b/doc/todo/deferred_update_mode.mdwn deleted file mode 100644 index 84193fd92e..0000000000 --- a/doc/todo/deferred_update_mode.mdwn +++ /dev/null @@ -1,42 +0,0 @@ -`git annex sync` and the assistant do a merge of new revs, and then -download the content of files. However, this means that broken links can -show up, when a file has changed, or a new file was added. In some -workflows, the user would prefer not to ever see such broken links -(or at least never for files that are in the repo's preferred content). - -So, how about a new mode, that defers updating the work tree until -the content of everything wanted is available? - -This could be a annex.merge=downloadfirst setting; it would make sync/assistant -look at the diff between HEAD and the new rev, and try to get all annexed files -added in that diff, before merging it. - -Of course, it could take a long time to get to see a new work tree. -Might have to download a lot of content. - -What to do if it fails to download a file's content? Could either -abort, leaving the current work tree as-is, or could go ahead and merge, -letting broken links show up in this case. I kind of prefer the abort -option. But, if the content never reaches any remote, or has gone missing -entirely, that would make sync never succeed. That could be surprising -behavior. - -Probably best to let the user pick either behavior, so -annex.merge=trydownloadfirst and annex.merge=reqdownloadfirst - -Let `git annex merge` be used to force a merge, even when content is not -available. - -## alternatively - -What about just making `git-annex sync --content` try to get the content of -all files before updating the work tree? (The assistant would need changes -too; it would need to queue all the downloads and trigger a work tree -update once all the downloads have been tried.) - -> `git-annex adjust --hide-missing` implements this. The assistant -> does not support it yet, and there's a todo for that, -> [[todo/assistant_support_hide-missing]] -> -> So, closing this as there's a path forward in that other todo. [[done]] -> --[[Joey]] diff --git a/doc/todo/distributed_migration.mdwn b/doc/todo/distributed_migration.mdwn deleted file mode 100644 index 45a6d14398..0000000000 --- a/doc/todo/distributed_migration.mdwn +++ /dev/null @@ -1,71 +0,0 @@ -Currently `git-annex migrate` only hard links the objects in the local -repo. This leaves other clones without the new keys' objects unless -they re-download them, or unless the same migrate command is -re-run, in the same tree, on each clone. - -It would be good to support distributed migration, so that whatever -migration is done in one repo is reflected in the other repos. - -This needs some way to store, in the git repo, a mapping between the old -key and the new key it has been migrated to. (I investigated -how much space that would need in the git repo, in -[this comment](https://git-annex.branchable.com/todo/alternate_keys_for_same_content/#comment-917eba0b2d1637236c5d900ecb5d8da0).) -The mapping might be communicated via the git branch but be locally stored -in a sqlite database to make querying it fast. - -Once that mapping is available, one simple way to use it would be a -git-annex command that updates the local repo to reflect migrations that -have happened elsewhere. It would not touch the HEAD branch, but would -just hard link object files from the old to new key, and update the location -log for the new key to indicate the content is present in the repo. -This command could be something like `git-annex migrate --update`. - -Rather than a dedicated command that users need to remember to run, -distributed migration could be done automatically when merging a git-annex -branch that adds migration information. Just hardlink object files and -update the location log. - -It would be possible to avoid updating the location log, but then all -location log queries would have to check the migration mapping. It would be -hard to make that fast enough. Consider `git-annex find --in foo`, which -queries the location log for each file. - ---[[Joey]] - -> [[done]] --[[Joey]] - -# security - -It is possible for bad migration information to be recorded in the -git-annex branch by someone malicious. To avoid bad or insecure behavior -when bad migration information is recorded: - -* When updating the local repository with a migration, verify that - the object file hashes to the new key before hardlinking. - -> This was done. - -That leaves at these possible security problems: - -* DOS by flooding the git-annex branch with migrations, resulting in - lots of hard links (or copies on filesystems not supporting hard links) - and hashing of large files. - -Note that a malicious person who can write to the git-annex branch -can already set their own repo as trusted, wait for someone -to drop their local copy, and then demand a ransom for the content. -For that matter, someone hosting a git-annex remote on a server can wait -for someone to rely on it to contain the only copy of content and ransom -it then. - -git-annex is probably not normally used in situations where we -need to worry about this kind of attack; if we don't trust someone we -shouldn't pull the git-annex branch from them, and should not trust their -remote to contain the only copy. - -If we pull a git-annex branch from someone, they can already DOS disk space -and CPU by checking a lot of junk into git. So maybe a DOS by migration is -not really a concern. - -> If people are worried about this kind of thing, they can avoid using the -> feature. --[[Joey]] diff --git a/doc/todo/distributed_migration/comment_1_8734d30aa0c1cb27dce81a0277d24948._comment b/doc/todo/distributed_migration/comment_1_8734d30aa0c1cb27dce81a0277d24948._comment deleted file mode 100644 index 180e937c2d..0000000000 --- a/doc/todo/distributed_migration/comment_1_8734d30aa0c1cb27dce81a0277d24948._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-12-01T19:00:41Z" - content=""" -Even if the stuff with special remotes turned out to be too complicated to -implement, `git-annex migrate --update` would be useful for some users. -So it's worth implementing the mapping and then incrementally implementing -these ideas. -"""]] diff --git a/doc/todo/distributed_migration/comment_2_fdaef5d870221d44c57fc3bd1501e7ee._comment b/doc/todo/distributed_migration/comment_2_fdaef5d870221d44c57fc3bd1501e7ee._comment deleted file mode 100644 index 40276efe9d..0000000000 --- a/doc/todo/distributed_migration/comment_2_fdaef5d870221d44c57fc3bd1501e7ee._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2023-12-06T19:31:42Z" - content=""" -On the `distributedmigration` branch I have `git-annex migrate` recording -migrations on the git-annex branch. - -Its method of grafting in 2 trees, one with the old keys and one with the -new is quite efficient. In a migration of 1000 files from SHA256E to SHA1, -the git objects only needs 52kb to record the migration trees. -Compared with 424 kb needed to update the location logs. -The total git repo grew from 508kb to 984k. - -Next up: Make `git-annex migrate --update` find new migrations started -elsewhere and apply them to the local annex objects. -"""]] diff --git a/doc/todo/does_not_preserve_timestamps.mdwn b/doc/todo/does_not_preserve_timestamps.mdwn deleted file mode 100644 index c0b3667b03..0000000000 --- a/doc/todo/does_not_preserve_timestamps.mdwn +++ /dev/null @@ -1,18 +0,0 @@ -### Please describe the problem. -I see that files are synced between my computers with git-annex but the timestamps do not match. The one that receives files always puts the current time of file creation on the file. - -### What steps will reproduce the problem? -Install git-annex on two computers. Connect with XMPP. Then add cloud storage with shared encryption for transferring files. Since you want also backup, choose "full backup" as the type of cloud storage. - - -### What version of git-annex are you using? On what operating system? -Downloaded binary package dated 13/09/2014 amd64 Ubuntu 14.04. - - -### Please provide any additional information below. - -Files are in sync. For example, I move a file from a directory to my synced annex directory. It contains timestamp of 01/01/2010 for example. Once the file gets transferred to the remote computer, it gets current time, for example 20/09/2014 rather than keeping 01/01/2010. - -All computers are linux based, ext4 filesystems. File transfers are done through shared encryption rsync remote. - -> [[wontfix|done]] --[[Joey]] diff --git a/doc/todo/does_not_preserve_timestamps/comment_10_8cce043ff86fefc3dcd97cf5c4428786._comment b/doc/todo/does_not_preserve_timestamps/comment_10_8cce043ff86fefc3dcd97cf5c4428786._comment deleted file mode 100644 index 7080a5877e..0000000000 --- a/doc/todo/does_not_preserve_timestamps/comment_10_8cce043ff86fefc3dcd97cf5c4428786._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="vrs+annex@ea5fa24dbb279be61a8e50adb638bf8366300717" - nickname="vrs+annex" - avatar="http://cdn.libravatar.org/avatar/74219abcec6eece8e2c9d4351c2c912c" - subject="related bugs" - date="2018-03-17T19:06:46Z" - content=""" -I've created a number of timestamp-related bugs that will hopefully result in better timestamp support: - -* [annex.genmetadata should default to true](/bugs/annex.genmetadata_should_default_to_true/) -* [file modification time should be stored in exactly one metadata field](/bugs/file_modification_time_should_be_stored_in_exactly_one_metadata_field/) -* [if annex.genmetadata is true, modification metadata is imported from older file versions after unlock+add](/bugs/if_annex.genmetadata_is_true__44___modification_metadata_is_imported_from_older_file_versions_after_unlock+add/) -* [proposal for timestamp semantics](/bugs/proposal_for_timestamp_semantics/) -"""]] diff --git a/doc/todo/does_not_preserve_timestamps/comment_11_eea115becce4ac79932d8cfa2903f20c._comment b/doc/todo/does_not_preserve_timestamps/comment_11_eea115becce4ac79932d8cfa2903f20c._comment deleted file mode 100644 index 6ef3a96290..0000000000 --- a/doc/todo/does_not_preserve_timestamps/comment_11_eea115becce4ac79932d8cfa2903f20c._comment +++ /dev/null @@ -1,21 +0,0 @@ -[[!comment format=mdwn - username="https://christian.amsuess.com/chrysn" - nickname="chrysn" - avatar="http://christian.amsuess.com/avatar/c6c0d57d63ac88f3541522c4b21198c3c7169a665a2f2d733b4f78670322ffdc" - subject="Summary; Application: shared thumbnails" - date="2020-01-10T08:41:18Z" - content=""" -There are two conflicting approaches to mtimes: - -* Treat them as local artifacts - - This works great with Make, and generally with any software that works on \"is newer than\" properties. - -* Treat them as preservation-worthy file attributes - - This is generally required by tools that compare time stamps by identity. - -Both approaches break tools that expect the other, and no single out-of-the-box choice will make all users happy. Tools like metastore, a bespoke solution like etckeeper's generated mkdir/chmod file or a git-annex solution like [[storing the full mtime at genmetadata time|bugs/file_modification_time_should_be_stored_in_exactly_one_metadata_field/]] with a (local or repository-wide) option to set the mtime at annex-get time would be convenient. - -One more application where this would be relevant is sharing generated thumbnails among clones of repositories (to eventually maybe even have them available when the full files are not present) following the [XDG specification on shared thumnail repositories](https://specifications.freedesktop.org/thumbnail-spec/thumbnail-spec-latest.html#SHARED). Not only does that design rely on the mtimes of the thumbnail and the file to match, it even encodes the mtime again inside the thumbnail, practically requiring all checkouts to not only have consistent mtimes between thumbnails and files, but identical ones. -"""]] diff --git a/doc/todo/does_not_preserve_timestamps/comment_12_a8b69299c9278c16c6c5061d88bc7eca._comment b/doc/todo/does_not_preserve_timestamps/comment_12_a8b69299c9278c16c6c5061d88bc7eca._comment deleted file mode 100644 index afe36e709a..0000000000 --- a/doc/todo/does_not_preserve_timestamps/comment_12_a8b69299c9278c16c6c5061d88bc7eca._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Cebtenzzre" - avatar="http://cdn.libravatar.org/avatar/238202b8214db86e24636e11f4452a12" - subject="comment 12" - date="2020-11-22T16:07:15Z" - content=""" -The semantics described in [proposal for timestamp semantics](https://git-annex.branchable.com/bugs/proposal_for_timestamp_semantics/) would be all I need to stop wishing I just used rsync instead of git-annex. It seems easier to track history with rsync (snapshots and a CHANGELOG?) than to synchronize timestamps (or xattrs for comments) with git-annex. I don't care if it impacts performance, or if I have to set flags to non-default values, or how it resolves conflicts. I know git doesn't do it natively, but I thought the point of git-annex was to be better than native git with data files. Metadata like this is very useful to me, and often I want to sort by *some* kind of meaningful timestamp. ls can't read Exif or ID3 tags, and sometimes they aren't available or even relevant. -"""]] diff --git a/doc/todo/does_not_preserve_timestamps/comment_13_e37ea5188bd48817ccac82141906ee83._comment b/doc/todo/does_not_preserve_timestamps/comment_13_e37ea5188bd48817ccac82141906ee83._comment deleted file mode 100644 index 834513db0e..0000000000 --- a/doc/todo/does_not_preserve_timestamps/comment_13_e37ea5188bd48817ccac82141906ee83._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="jgoerzen" - avatar="http://cdn.libravatar.org/avatar/090740822c9dcdb39ffe506b890981b4" - subject="mtree can help" - date="2022-09-03T17:30:57Z" - content=""" -You can see my workaround for this using mtree at [[/forum/Use_on_large_media_collection_without_modifying_it]]. -"""]] diff --git a/doc/todo/does_not_preserve_timestamps/comment_14_9a3f50f1b8cfd1cc0da83937a36e1369._comment b/doc/todo/does_not_preserve_timestamps/comment_14_9a3f50f1b8cfd1cc0da83937a36e1369._comment deleted file mode 100644 index a48afdc01d..0000000000 --- a/doc/todo/does_not_preserve_timestamps/comment_14_9a3f50f1b8cfd1cc0da83937a36e1369._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 14""" - date="2022-09-20T17:55:45Z" - content=""" -Note that `git-annex add` does preserve the timestamp of the file -while adding it to the annex. Much the same as a file's timestamp is -the same after `git add` as it was before running that command. -When `git-annex add` replaces a file with a symlink, it even makes the -symlink's timestamp be the same as the original file. - -What git-annex does not do is try to store the timestamp in git and arrange -for clones that receive the file to get the same timestamp. There are lots -of things like this that someone *might* want git to preserve, but git -doesn't, and it's out of scope for git-annex to try to work around git's -limitations in these areas. Plenty of room for other tools that add -git tracked timestamps, etc. Many such tools exist. -"""]] diff --git a/doc/todo/does_not_preserve_timestamps/comment_1_caf5e5cb17f4d05fff8c2fab661cd93f._comment b/doc/todo/does_not_preserve_timestamps/comment_1_caf5e5cb17f4d05fff8c2fab661cd93f._comment deleted file mode 100644 index 48ec44d2b1..0000000000 --- a/doc/todo/does_not_preserve_timestamps/comment_1_caf5e5cb17f4d05fff8c2fab661cd93f._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="http://joeyh.name/" - ip="209.250.56.144" - subject="comment 1" - date="2014-09-23T20:27:25Z" - content=""" -The closest git comes to storing a timestamp is the date of the last commit of a file for mtime, and first commit for ctime. However, those are pretty expensive to look up for a given file. And git doesn't try to preserve timestamps in checkouts at all, which argues that git-annex, at least at the command line, should not either. -"""]] diff --git a/doc/todo/does_not_preserve_timestamps/comment_2_c337fca1474b5b78f61ad6f421138ae4._comment b/doc/todo/does_not_preserve_timestamps/comment_2_c337fca1474b5b78f61ad6f421138ae4._comment deleted file mode 100644 index 4b5a750cf2..0000000000 --- a/doc/todo/does_not_preserve_timestamps/comment_2_c337fca1474b5b78f61ad6f421138ae4._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="https://www.google.com/accounts/o8/id?id=AItOawmK0703vNSIQsP1mGf-4MAPnsBZiSc6yVo" - nickname="Emre" - subject="comment 2" - date="2014-09-23T20:58:10Z" - content=""" -Thanks Joey for the comment. - -But when syncing two repos, timestamps are critical at least for my use case. I can't lose this info. Even if it's expensive. - -Appreciate if you can consider to add it for direct mode repos, ie when a file is synced to another repo and created there, it shall carry at least the mtime of the file in source repo. Owncloud sync does it, btsync does it, although I know git-annex is different than those. -"""]] diff --git a/doc/todo/does_not_preserve_timestamps/comment_3_9a3eeddc46e5a420575f00cb47caf703._comment b/doc/todo/does_not_preserve_timestamps/comment_3_9a3eeddc46e5a420575f00cb47caf703._comment deleted file mode 100644 index ba34823d45..0000000000 --- a/doc/todo/does_not_preserve_timestamps/comment_3_9a3eeddc46e5a420575f00cb47caf703._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="https://www.google.com/accounts/o8/id?id=AItOawmK0703vNSIQsP1mGf-4MAPnsBZiSc6yVo" - nickname="Emre" - subject="comment 3" - date="2014-09-23T21:15:29Z" - content=""" -Btw, git storing the last commit time as Mtime is not enough, it shall store the original timestamp of the file, not the date of commit. Hope I could explain and hope this is something doable. -"""]] diff --git a/doc/todo/does_not_preserve_timestamps/comment_4_99b064259fc2e3c6eb83c3da3b2d3bac._comment b/doc/todo/does_not_preserve_timestamps/comment_4_99b064259fc2e3c6eb83c3da3b2d3bac._comment deleted file mode 100644 index 08de756190..0000000000 --- a/doc/todo/does_not_preserve_timestamps/comment_4_99b064259fc2e3c6eb83c3da3b2d3bac._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="http://svario.it/gioele" - nickname="gioele" - subject="comment 4" - date="2014-09-24T07:15:09Z" - content=""" -You can try to store the timestamps just before commit and restore them on checkout. - -Have a look at [metastore](https://github.com/przemoc/metastore): it is a ready-made solution for plain git. Maybe you can adapt it to git-annex. -"""]] diff --git a/doc/todo/does_not_preserve_timestamps/comment_5_c95c8b9bd617830604500213c962fc7a._comment b/doc/todo/does_not_preserve_timestamps/comment_5_c95c8b9bd617830604500213c962fc7a._comment deleted file mode 100644 index 7632548cdf..0000000000 --- a/doc/todo/does_not_preserve_timestamps/comment_5_c95c8b9bd617830604500213c962fc7a._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="https://www.google.com/accounts/o8/id?id=AItOawlM_DRhi_5pJrTA0HbApHR25iAgy-NBXTY" - nickname="Tor Arne" - subject="comment 5" - date="2014-10-01T22:02:43Z" - content=""" -I was just about to start using git-annex for my data when I saw this bug report, which unfortunately is a blocker. Persisting basic metadata of files that are synced seems like a core feature of a file sync/transfer tool, so I'm really hoping this can be solved somehow :/ - -"""]] diff --git a/doc/todo/does_not_preserve_timestamps/comment_6_b99e00d0bc4258c4cb28b544b19ea3b8._comment b/doc/todo/does_not_preserve_timestamps/comment_6_b99e00d0bc4258c4cb28b544b19ea3b8._comment deleted file mode 100644 index 67fe255950..0000000000 --- a/doc/todo/does_not_preserve_timestamps/comment_6_b99e00d0bc4258c4cb28b544b19ea3b8._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="https://www.google.com/accounts/o8/id?id=AItOawlM_DRhi_5pJrTA0HbApHR25iAgy-NBXTY" - nickname="Tor Arne" - subject="comment 6" - date="2014-10-01T22:39:01Z" - content=""" -Isn't this what the metadata feature does though? http://git-annex.branchable.com/design/metadata/ - -With annex.genmetadata true set, it should store year and mont (but not day/time? if so why not? - -Is the missing piece of the puzzle to apply the metadata again on checkout? -"""]] diff --git a/doc/todo/does_not_preserve_timestamps/comment_7_eefa6afce824f1069bb5216b392405b7._comment b/doc/todo/does_not_preserve_timestamps/comment_7_eefa6afce824f1069bb5216b392405b7._comment deleted file mode 100644 index 19c508c508..0000000000 --- a/doc/todo/does_not_preserve_timestamps/comment_7_eefa6afce824f1069bb5216b392405b7._comment +++ /dev/null @@ -1,7 +0,0 @@ -[[!comment format=mdwn - username="fastguy" - subject="Any updates?" - date="2015-09-25T19:35:18Z" - content=""" -Hi, would there be any updates on this issue or do we still not preserve timestamps when syncing between locations? -"""]] diff --git a/doc/todo/does_not_preserve_timestamps/comment_8_f2a32e39356f7b772ab9cf3136436700._comment b/doc/todo/does_not_preserve_timestamps/comment_8_f2a32e39356f7b772ab9cf3136436700._comment deleted file mode 100644 index 6085171f25..0000000000 --- a/doc/todo/does_not_preserve_timestamps/comment_8_f2a32e39356f7b772ab9cf3136436700._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="graboluk@f6de53961ab0f884e203f602f65eb5cdc0fb7513" - nickname="graboluk" - subject="timestamps are wrong as of 5.20150731" - date="2015-09-26T18:31:46Z" - content=""" -I confirm this issue is still present in the debian testing version 5.20150731. This really makes git-annex quite unusable for me, as part of my setup is that my server downloads various podcasts (think news videos from euronews, german tagesschau, some youtube podcasts), which then get distributed to my other computers. This way I don't have to use youtube etc., which makes a huge difference in terms of speed (my main home computer is from 2006 and it plays all hte videos just fine, but a website likie youtube.com is too much for it). At a speed of single shortcut I see a list of newest podcasts and I can choose one without bothering to open the webbrowser. - -The point is that I really like to have all those podcasts to be sorted by their modified date, for rather obvious reasons. (with dropbox it works fine.) -"""]] diff --git a/doc/todo/does_not_preserve_timestamps/comment_9_62736ee2f299d62f685800353fc36ccf._comment b/doc/todo/does_not_preserve_timestamps/comment_9_62736ee2f299d62f685800353fc36ccf._comment deleted file mode 100644 index ed9c0e41ea..0000000000 --- a/doc/todo/does_not_preserve_timestamps/comment_9_62736ee2f299d62f685800353fc36ccf._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="kalle@bdf75651b439b088e51f28f10f5a46ffcd2a704d" - nickname="kalle" - subject="importfeed template" - date="2015-09-28T19:52:16Z" - content=""" -You can use the `--template` flag and prefix the filename with the date. See [[/tips/downloading_podcasts]]. If you need very accurate modification times that won't work but should work in this case? -"""]] diff --git a/doc/todo/dynamic_stall_detection.mdwn b/doc/todo/dynamic_stall_detection.mdwn deleted file mode 100644 index a7db6ae90f..0000000000 --- a/doc/todo/dynamic_stall_detection.mdwn +++ /dev/null @@ -1,24 +0,0 @@ -annex.stalldetection lets remotes be configured with a minimum throughput -to detect and retry stalls. But most users are not going to configure this. -Could something be done to dynamically detect a stall, without configuration? - -Eg, wait until data starts to flow, and then check if there's at least some -data being sent each half minute. If so, the progress display is being updated -at least every minute. So then if 1 minute goes by without more data -flowing, it's almost certainly stalled. And if the progress display is -updated less frequently, see if it's updated every 2 minutes, etc. Although -realistically, progress displays are updated every chunk, and there's -typically more than 1 chunk per minute. So longer durations than 1 minute -may be an unnecessary complication. And a minute to detect a stall is fine. - -> Implemented this, annex.stalldetection = true enables automatic. - -It may still need a config to turn it on, because running -transfers in separate processes can lead to more resource use, or even -password prompting, which could be annoying to existing users. Also, if it -gets it wrong and the remote does not support resuming transfers, -defaulting to on could lead to bad waste of resources. It could -detect stalls even when not turned on, but only display a message -suggesting enabling the config. --[[Joey]] - -> [[done]] --[[Joey]] diff --git a/doc/todo/encrypted_keys_in_git_repository.mdwn b/doc/todo/encrypted_keys_in_git_repository.mdwn deleted file mode 100644 index 38dd27ddcc..0000000000 --- a/doc/todo/encrypted_keys_in_git_repository.mdwn +++ /dev/null @@ -1,61 +0,0 @@ -The idea is to have a type of key that is based on a hash of the annexed -file, but with some form of encryption or password protection. - -So someone who can access the git repository is prevented from checking -known hashes of files, and learning about content in the repository that is -not available to them. (However, they would be able to see filenames -and commit metadata, which might also expose information about what the -files are. This is not a fully encrypted git repository like can be made -with gcrypt.) - -This might be an added layer of security, or the git repository might be -made public, including perhaps some of the annexed files, while this is -used to obscure the hashes of some other annexed files. For example, a -scientific dataset might make public files that are derived from patient -data, and use this for the patient data files. - ----- - -If two such repositories were merged, git-annex would need to somehow -be able to tell how to decrypt a key, which could have come from either -repository. So it seems that the key needs to include within it some -identifier of the secret that is used to encrypt it. For example: - - ENC--ident--foo - -Where ident would be something like a UUID of the secret, and foo is the -file's hash (and perhaps size, or indeed a whole regular key) that is -encrypted by the secret. - ----- - -Should the key's size be included only in encrypted form, or in plaintext? - -Since git-annex constructs a Key without using the associated Backend, -it currently parses the size field the same way for each type of key. -So, it does not seem to be possible to encrypt the size field and -use that encrypted size to populate keySize. What would be -doable though, is to replace uses of keySize with a new Backend method that -returns the size. - ----- - -Should the encryption method be reversible? - -If the encryption method is not reversible, the key's size would need to be -included in plaintext, or left out entirely. - -But it does not seem necessary for the encryption method to be reversible -otherwise. Consider if scrypt was used. When adding a file, git-annex would -first hash it, and then run it through scrypt. That is not reversible, -so when fscking a file, just repeat the same process and compare the -resulting scrypt keys. - -Not being reversible is a nice benefit, because it makes it much harder -for an attacker to brute-force. If it's reversible the attacker can -brute-force the user's password, looking for a password that decrypts -to something that looks right. - ---[[Joey]] - -> [[wontfix|done]]; see description --[[Joey]] diff --git a/doc/todo/encrypted_keys_in_git_repository/comment_1_4d17c427756127f22dc472984259f8d7._comment b/doc/todo/encrypted_keys_in_git_repository/comment_1_4d17c427756127f22dc472984259f8d7._comment deleted file mode 100644 index 58c4f3366d..0000000000 --- a/doc/todo/encrypted_keys_in_git_repository/comment_1_4d17c427756127f22dc472984259f8d7._comment +++ /dev/null @@ -1,23 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="Maybe add a very slow KDF first?" - date="2022-12-27T20:26:45Z" - content=""" -Hey joey, - -As a simple-to-implement yet quite effective approach to the problem of storing some secrets in a public git-annex repo, wouldn't a very slow hash/key derivation function (like scrypt) as keys for those specific files be enough? The hash can be public when brute-forcing is infeasible. So for git-annex: - -- add an scrypt key backend -- makes the params n,r,p configurable (slowness of hashing) and put them i to the key name (e.g. `SCRYPT-n10-r100-p1--...`) -- the salt could also be configurable or generated randomly per file but needs to be present in the key name (base64-encoded?) and uses either the file content or as such or some cryptographically secure hash of it as scrypt 'password' -- file size shouldn't be included in the key - -With sane defaults (maybe settings that make hashing take several seconds?), this would make git-annex a very nice way of hiding some files' content in public repositories while still tracking it. - -Some resources: - -- https://stackoverflow.com/questions/23985540/whats-the-is-maximum-length-of-scrypt-output#23985541 -- https://words.filippo.io/the-scrypt-parameters/ - -"""]] diff --git a/doc/todo/encrypted_keys_in_git_repository/comment_2_6679caba100bee0a0fe32c8eef50758e._comment b/doc/todo/encrypted_keys_in_git_repository/comment_2_6679caba100bee0a0fe32c8eef50758e._comment deleted file mode 100644 index 6bc238b1f6..0000000000 --- a/doc/todo/encrypted_keys_in_git_repository/comment_2_6679caba100bee0a0fe32c8eef50758e._comment +++ /dev/null @@ -1,21 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-12-27T20:43:35Z" - content=""" -Including the salt in the key is an interesting idea. I am not sure I buy -that it would be secure. - -Normally with scrypt (or argon2 etc), and a strong password, the attacker has -to make a huge number of guesses, and so the comparatively modest amount of -work per guess is enough to make it infeasible for them to succeed. - -Here though, the attacker will only be interested in guessing the hashes of -known files that they care about. That might be millions of files and so be -a reasonable amount of work to try them all. Probably less work than a good -password, so the hash difficulty parameters would need to be turned up to -secure against that attacker. - -But... If the attacker only cares about a single file, they only have to run -scrypt once. -"""]] diff --git a/doc/todo/encrypted_keys_in_git_repository/comment_3_a79ea55e2f8a1095f0110f2f0853dee1._comment b/doc/todo/encrypted_keys_in_git_repository/comment_3_a79ea55e2f8a1095f0110f2f0853dee1._comment deleted file mode 100644 index 2455fcf1b0..0000000000 --- a/doc/todo/encrypted_keys_in_git_repository/comment_3_a79ea55e2f8a1095f0110f2f0853dee1._comment +++ /dev/null @@ -1,21 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 3" - date="2022-12-29T10:49:27Z" - content=""" -> But... If the attacker only cares about a single file, they only have to run scrypt **once.** - -Well, the attacker would still need to run scrypt many times to brute-force the actual content, IIUC. - -I understand it like this (might be wrong, I'm no security expert): - -- scrypt is an arbitrarily-slow key derivation function taking an input (the „password” - or for us: the file content) and some parameters (salt, n, r, p) -- knowing the parameters (salt, n, r, p) is necessary to verify file integrity so storing those in an easily accessible way (e.g. directly in the key) would simplify things and we wouldn't need to teach git-annex much more to handle this. -- A (public) git-annex repo that has some files „scrypt”-ed like this then contains the scrypt hash and the parameters used to generate that hash. So this is equivalent to a database breach where user credentials were secured with scrypt. (right?) -- An attacker thus now still has to brute-force the content of the desired file. With little information about how that file looks in detail („how exactly is that YAML containing the password structured? How big is it even? What key order? Are there comments? That comment could even contain a random salt itself...”), this brute-forcing should be pretty infeasible if the scrypt parameters are tuned to hash *ridiculously* slow (like multiple seconds even on a beefy machine with much RAM - doesn't bother us if `git annex fsck` is slow on that one file. Big files hash slowly anyway as well...). It should definitely be harder than cracking a breached password database where it is clear that the passwords don't contain newlines and there most likely is a size limit, etc. - -If my above assumptions are correct, an `scrypt` key backend for git-annex should make for a nice way of hiding the content of *some* files in a public repo, right? - -P.S: Thinking about the 'that secured file could contain potentially large random comments acting as a salt next to the actual secret' together with removing the size from the key ([`git annex migrate --remove-size`](https://git-annex.branchable.com/git-annex-migrate/)) should already be pretty safe with the default SHA256-backend, right? 🤔 -"""]] diff --git a/doc/todo/encrypted_keys_in_git_repository/comment_4_e88664a0939b7bca730a61efb47d34ab._comment b/doc/todo/encrypted_keys_in_git_repository/comment_4_e88664a0939b7bca730a61efb47d34ab._comment deleted file mode 100644 index df0611bf0f..0000000000 --- a/doc/todo/encrypted_keys_in_git_repository/comment_4_e88664a0939b7bca730a61efb47d34ab._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2023-01-02T19:03:34Z" - content=""" -I'm talking about a single file that the attacker already knows the content -of, and wishes to determine if it's present in the repository. That takes a -single scrypt operation, so scrypt's added difficulty is not relevant. - -If the attacker doesn't know the file content, and is trying to hash random -values until they find one that matches the git-annex key, then a sha2 is -equally good protection as scrypt, because the amount of work is -computationally infeasible in both cases. -"""]] diff --git a/doc/todo/encrypted_keys_in_git_repository/comment_5_05b0e2e88b233691b403d4c908b50c52._comment b/doc/todo/encrypted_keys_in_git_repository/comment_5_05b0e2e88b233691b403d4c908b50c52._comment deleted file mode 100644 index d0b388b8f1..0000000000 --- a/doc/todo/encrypted_keys_in_git_repository/comment_5_05b0e2e88b233691b403d4c908b50c52._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 5" - date="2023-01-02T22:58:08Z" - content=""" -> I'm talking about a single file that the attacker already knows the content of, and wishes to determine if it's present in the repository. - -Ah alright, in that case scrypt doesn't help, I agree. Among the large userbase for such a versatile software as git-annex there surely are people that could make use of hiding the presence of some 'common' files from their public repos. Though it rather seems like quite an edge case to me 😅 Better go full-gcrypt in that case I'd say... - -> sha2 is equally good protection as scrypt - -In that case, this'll be enough for my application, thanks! 🙏 -"""]] diff --git a/doc/todo/encrypted_keys_in_git_repository/comment_6_9e25b4d05d61fc3a3cca4f7111681a5f._comment b/doc/todo/encrypted_keys_in_git_repository/comment_6_9e25b4d05d61fc3a3cca4f7111681a5f._comment deleted file mode 100644 index 59c3692e7a..0000000000 --- a/doc/todo/encrypted_keys_in_git_repository/comment_6_9e25b4d05d61fc3a3cca4f7111681a5f._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 6""" - date="2023-01-16T18:39:38Z" - content=""" -Ok, I think we've concluded there is nothing to do here! I'll close this -todo then. -"""]] diff --git a/doc/todo/explicit_metadata_needed_for_view_inclusion_or_not.mdwn b/doc/todo/explicit_metadata_needed_for_view_inclusion_or_not.mdwn deleted file mode 100644 index 6ec6332a4a..0000000000 --- a/doc/todo/explicit_metadata_needed_for_view_inclusion_or_not.mdwn +++ /dev/null @@ -1,116 +0,0 @@ -I'm not sure if this is a bug or a feature but the question is should files have (some/any) -explicit metadata set with `git-annex metadata` for them to be included in a view branch? -I mean if you create a view with just path-based fields (say, restrict to a subdirectory -with a criterion like 'subdir/=*'), do you expect files to match even if they don't have -any metadata fields manually set? In release version 10.20230214 the behavior has changed -in that files that don't have manual metadata set don't seem to be included in -(directory-based) view branches. The inverse seems to be true of the unreleased version -10.20230127-gc209e0f64 committed a week before. - -In my opinion, this *could* be a feature if it's documented but it could also be construed -as violating the principle of least surprise as not all users routinely tag all annexed files -with metadata. - -The following is a transcript that illustrates how the recent release version behaves. See -how the file `subdir/c` is excluded from the view branch (my apologies for this being a -PowerShell session on Windows): - -[[!format sh """ -X:\> cd git-annex-tests -X:\git-annex-tests> git init annex-view-test-a -X:\git-annex-tests> cd .\annex-view-test-a\ -X:\git-annex-tests\annex-view-test-a [master]> git config annex.thin true -X:\git-annex-tests\annex-view-test-a [master]> git annex init Annex-view-test-a -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked)]> echo Aaa > a -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked) +1 ~0 -0 !]> mkdir subdir -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked) +1 ~0 -0 !]> echo Bbb > subdir\b -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked) +2 ~0 -0 !]> echo Ccc > subdir\c -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked) +2 ~0 -0 !]> mkdir subdir\subsubdir -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked) +2 ~0 -0 !]> echo Ddd > .\subdir\subsubdir\d -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked) +2 ~0 -0 !]> git annex add . -add a -ok -add subdir/b -ok -add subdir/c -ok -add subdir/subsubdir/d -ok -(recording state in git...) -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked) +4 ~0 -0 ~]> ls -r - - Directory: X:\git-annex-tests\annex-view-test-a - -Mode LastWriteTime Length Name ----- ------------- ------ ---- -d---- 15.2.2023 12:37 subdir --a--- 15.2.2023 12:31 5 a - - Directory: X:\git-annex-tests\annex-view-test-a\subdir - -Mode LastWriteTime Length Name ----- ------------- ------ ---- -d---- 15.2.2023 12:38 subsubdir --a--- 15.2.2023 12:34 5 b --a--- 15.2.2023 12:37 5 c - - Directory: X:\git-annex-tests\annex-view-test-a\subdir\subsubdir - -Mode LastWriteTime Length Name ----- ------------- ------ ---- --a--- 15.2.2023 12:38 5 d - -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked) +4 ~0 -0 ~]> fsutil hardlink list a -\git-annex-tests\annex-view-test-a\.git\annex\objects\b29\a20\SHA256E-s5--e3c555b2205407e76d3a460513cf44c1c62e48ba83ec4948ed05d0f87b957051\SHA256E-s5--e3c555b2205407e76d3a460513cf44c1c62e48ba83ec4948ed05d0f87b957051 -\git-annex-tests\annex-view-test-a\a -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked) +4 ~0 -0 ~]> git commit -m 'Add test files' -[adjusted/master(unlocked) 56b11c9] Add test files - 4 files changed, 4 insertions(+) - create mode 100644 a - create mode 100644 subdir/b - create mode 100644 subdir/c - create mode 100644 subdir/subsubdir/d -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked)]> git annex metadata -t test subdir\b -metadata subdir/b - lastchanged=2023-02-15@10-52-31 - tag=test - tag-lastchanged=2023-02-15@10-52-31 -ok -(recording state in git...) -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked)]> git annex metadata -t test subdir\subsubdir\d -metadata subdir/subsubdir/d - lastchanged=2023-02-15@10-53-00 - tag=test - tag-lastchanged=2023-02-15@10-53-00 -ok -(recording state in git...) -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked)]> git annex metadata subdir\c -metadata subdir/c - -ok -X:\git-annex-tests\annex-view-test-a [adjusted/master(unlocked)]> git annex view 'subdir\=*' -view (searching...) -Switched to branch 'views/adjusted/master(unlocked)(subdir_=_)' -ok -X:\git-annex-tests\annex-view-test-a [views/adjusted/master(unlocked)(subdir_=_)]> ls -r - - Directory: X:\git-annex-tests\annex-view-test-a - -Mode LastWriteTime Length Name ----- ------------- ------ ---- -d---- 15.2.2023 12:55 subsubdir --a--- 15.2.2023 12:55 178 b_%subdir% - - Directory: X:\git-annex-tests\annex-view-test-a\subsubdir - -Mode LastWriteTime Length Name ----- ------------- ------ ---- --a--- 15.2.2023 12:55 181 d_%subdir%subsubdir% - -# End of transcript or log. -"""]] - - -[[!meta author=jkniiv]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/todo/explicit_metadata_needed_for_view_inclusion_or_not/comment_1_17f8adc8555ea0a2716308e3f5188a25._comment b/doc/todo/explicit_metadata_needed_for_view_inclusion_or_not/comment_1_17f8adc8555ea0a2716308e3f5188a25._comment deleted file mode 100644 index 00d42ef5a6..0000000000 --- a/doc/todo/explicit_metadata_needed_for_view_inclusion_or_not/comment_1_17f8adc8555ea0a2716308e3f5188a25._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-02-16T18:55:06Z" - content=""" -I'm surprised by this behavior change! (And I replicated it on linux, no -apologies necessary for also testing Windows support..) - -[[!commit 826b225ca81d469640e51628865dbd64a1d37773]] is the buggy commit. -When there is no metadata log, it accidentially skips calling -getfilemetadata. - -I've fixed that. I wonder if I should push up the shedule of the next -release because of this reversion? Currently I'm inclined to treat it as -medium urgency for release (1-2 weeks). -"""]] diff --git a/doc/todo/faster_incremental_update_of_view_branch_by_git-annex_sync.mdwn b/doc/todo/faster_incremental_update_of_view_branch_by_git-annex_sync.mdwn deleted file mode 100644 index 7745bb7af3..0000000000 --- a/doc/todo/faster_incremental_update_of_view_branch_by_git-annex_sync.mdwn +++ /dev/null @@ -1,27 +0,0 @@ -git-annex sync, when run in a view branch, regenerates the view branch from -scratch. That is as expensive as entering the view branch in the first -place, which can take quite a long time when there are a lot of files. - -And users with a lot of files are just the kind of user a view branch -appeals to.. - -There is probably a lot of scope for optimisation in updating the view -branch, that might be able to get it reasonably quick. -I have not fully thought through it, but basically diffing from the old -parent branch to the new parent to find files that have changed, and -adding/removing those from the view. And also diffing from the old -git-annex branch tree to the new one to get changes to metadata logs, -parsing those and using changes to metadata to also move/delete/add -files to the view branch. - -It may also be that just improving the precaching of metadata logs -would improve the speed a lot. The streaming precaching of location logs -sped up some commands around 2x before IIRC. - -> catObjectStream of metadata logs sped view constriction up by ~50%. -> More streaming should speed it up more; it still uses -> lookupKey/catKey once per file. --[[Joey]] - -> Seems like this was implemented in -> [[!commit 5f9bf51438a16b73d6fcf30f3ca43fbb894ea5df]] -> ... so [[done]]! --[[Joey]] diff --git a/doc/todo/fsverify.mdwn b/doc/todo/fsverify.mdwn deleted file mode 100644 index 4c16044172..0000000000 --- a/doc/todo/fsverify.mdwn +++ /dev/null @@ -1,53 +0,0 @@ -git-annex could use linux's [fsverify](https://www.kernel.org/doc/html/latest/filesystems/fsverity.html) -feature as an alternative to hashing and verifying hashes of files itself. - -Benefits would include: - -* Any read of an annexed file that uses fsverify would check the blocks - that are read, and the read would fail if the file had gotten corrupted. -* Avoiding any theoretical cases where `git-annex add` is hashing a file - and something modifies it, causing the file to be added with the wrong - hash (which `git-annex fsck` will later detect). The - `FS_IOC_ENABLE_VERITY` ioctl prevents anything else from possibly - modifying the file while it's hashing it. -* Slightly faster git-annex fsck, because it would not need to hash - verified files. It would suffice to read the file, and if it all read - successfully, it's valid! - -Since fsverify uses a merkle tree, its hashes are not the same as simply -using SHA on the whole file. So for git-annex to use the fsverify hash as -the key for the file, it would need to be a separate type of key. That's a -bit problimatic because then git-annex would need a way to verify that -merkle hash itself on systems that do not support fsverify. Also, for large -files, the merkle tree can get relatively large (1/127th the size of the -file the docs say). So with a terabyte of annexed files, that's gigabytes -of merkle hashes, which seems too large to want to stote them in git. - -Alternatively, git-annex could hash as usual for the key. This would mean -that `git-annex add` would hash a file twice, once for the git-annex key -and the second time calling the `FS_IOC_ENABLE_VERITY` ioctl. Slower, but -perhaps these could parallelize and only use 2x the CPU or so. - -Since fsverified files are readonly, this would only be useful for locked -files. Unlocking a file would need to either remove the fsverify from it -(if possible?) or copy it. - -Using fsverify in this way would not work if the sysctl -`fs.verity.require_signatures` is set, because the annexed files would -not have signatures. - ---- - -Putting all this together, fsverify is not too compelling for use by -git-annex. A user who wants the verification on all reads of a file can -just call `FS_IOC_ENABLE_VERITY` on it themselves after git-annex add. -The annex.freezecontent-command hook could be used to to that. - -Then the only benefit of supporting it in git-annex is that perhaps `git-annex -add` could parallize enabling verification with checksumming, or avoid its -own checksumming, and so run faster than if a hook were used to enable -fsverify. And fsck would use less CPU. Is that worth complicating git-annex for? ---[[Joey]] - -> After investigating that, I currently don't think it's compelling, so I'm -> gonna close this. [[done]] --[[Joey]] diff --git a/doc/todo/git_status_smudges_unncessarily_after_unlock.mdwn b/doc/todo/git_status_smudges_unncessarily_after_unlock.mdwn deleted file mode 100644 index 2cd819e742..0000000000 --- a/doc/todo/git_status_smudges_unncessarily_after_unlock.mdwn +++ /dev/null @@ -1,39 +0,0 @@ -After unlocking a file, `git status` runs the smudge filter. That is -unnecessary, and when many files were unlocked, it can take a long time -because [[git_smudge_clean_interface_suboptiomal]] means it runs git-annex -once per file. - -It should be possible to avoid that, as was done with git drop in [[!commit -1113caa53efedbe7ab1d98b74010160f20473e8d]]. I tried making Command.Unlock -use restagePointerFile, but that did not help; git update-index does then -smudge it during the `git annex unlock`, which is no faster (but at least -doing it then would avoid the surprise of a slow `git status` or `git -commit -a`). Afterwards, `git status` then smudged it again, unsure why! ---[[Joey]] - -[[!tag confirmed]] - -> I wondered if this was still a problem in a v9 repository with -> filter-process used instead of smudge. It's not really -- after unlocking -> 1000 files, git status did need to refresh all 1000, but it ran -> relatively quickly because it was able to use filter-process. -> -> But, those were small files. Large files would make it slower as it pipes -> their content though. It would be better for Command.Unlock to use -> restagePointerFile, so whatever price there is is paid during unlocking -> and not unexpectedly later on. -> -> I tried again making Command.Unlock use restagePointerFile, and this -> slowed git-annex unlock. But git status did then avoid doing any more -> smudgeing. It seems that each call to restagePointerFile is running -> git update-index, so still one git-annex smudge per file, rather -> than combining several together. In a v8 repo, the same amount of work -> is done either way. In v9+, this is actually slower than before when the -> files are small. -> -> restagePointerFile was running git update-index once per file -> because restagePointerFile uses the git queue, and unlock -> also queues a git add or something, so the queue isn't able to built -> up because two dissimilar things are being queued. This seems an -> unnecessary behavior; it could queue up all the git adds and then -> run restagePointerFile after them all. Implemented that, and [[done]]! --[[Joey]] diff --git a/doc/todo/hidden_files.mdwn b/doc/todo/hidden_files.mdwn deleted file mode 100644 index 28832b28a2..0000000000 --- a/doc/todo/hidden_files.mdwn +++ /dev/null @@ -1,38 +0,0 @@ -Add a `git annex hide $file` that behaves like drop, checking counter info -and updating location log to say the current repo no longer has a file -- -but does not actually remove the content. - -Then `git annex unused` can be used to clean it up later. And in the -meantime, it's still locally accessible. This can be useful if you're -planning to need to free up space later, but want to hold onto the content -for a while. Possibly you'll be disconnected later, so it's easier to push -out that intent now. - --- - -TODO: - -* Make 100% sure this is safe. Drop, etc should never check content files - are present on other repos if the location log doesn't say the repo - has the content. - -* What will `git annex get` do if it's asked to get a file that has been - hidden? - -> Unless I am missing something: Make sure the data is correct (for SHA1 or other tracking) and restore locally. If that's not the case, delete and restore from remote. -- RichiH - ----- - -Is 'unused' a good name? 'clean' and 'autoclean' would make more sense, imo. 'clean' deletes everything, whereas an optional 'autoclean' could try to be smart based on disk usage and/or SHA1, etc. -- RichiH - -> Nah, `git annex unused/dropunused` already exist. --[[Joey]] - ->> OK, in that case forget what I said. No idea about your internal policy, but feel free to delete this part of the page, then. -- RichiH - - ---- - -Revisiting old todos, I noticed this and -[[todo/wishlist: `git annex drop --relaxed`]] -are basically the same idea. So gonna -close this as a dup. [[done]] --[[Joey]] diff --git a/doc/todo/import_tree_preferred_content_expansions.mdwn b/doc/todo/import_tree_preferred_content_expansions.mdwn deleted file mode 100644 index 4ff04ffc82..0000000000 --- a/doc/todo/import_tree_preferred_content_expansions.mdwn +++ /dev/null @@ -1,17 +0,0 @@ -Importing a tree from a special remote -when it has a preferred content configured currently only works when the -expression does not use any terms that operate on keys. -(Eg things like copies=). - -[[!commit e06feb7316af35b1277b7159a74c216c8f2e7422]] -implemented that. Could it be extended to also support -using it with an expression that does operate on keys? - -This would probably entail making any term of the expression that needs a -key evaluate to true. Then it would import all files -(that match also terms that don't need a key). After the import, it may -turn out that the special remote doesn't want to contain particular content -that was imported from it, and it would make sense that an export to the -special remote would remove those files. --[[Joey]] - -> [[done]] --[[Joey]] diff --git a/doc/todo/import_tree_preferred_content_expansions/comment_1_df8ca8665e1dfc530a832b6d24d60ea4._comment b/doc/todo/import_tree_preferred_content_expansions/comment_1_df8ca8665e1dfc530a832b6d24d60ea4._comment deleted file mode 100644 index 7e9b4507a4..0000000000 --- a/doc/todo/import_tree_preferred_content_expansions/comment_1_df8ca8665e1dfc530a832b6d24d60ea4._comment +++ /dev/null @@ -1,29 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2023-12-18T18:02:40Z" - content=""" -Hmm, consider for example a camera. If the user wants to import all jpeg -files from it, and not export to it jpeg files that have a copy in an -archive, they might use: - - not copies=archive:1 and include=*.jpeg - -But on import, if "copies=archive:1" were made to evaluate to true -as suggested here, then this expression would not match, and so -nothing would be imported. - -Seems that the approach needs to be instead to traverse the expression -and prune terms that operate on keys. So convert the example -above to "include=*.jpeg". - -How to prune in some other cases: - - not (copies=archive:1 and copies=backup:1) => anything - - not (copies=archive:1 and include=*.jpeg) => not (include=*.jpeg) - - not (copies=archive:1 or include=*.jpeg) => not (include=*.jpeg) - - not ((not copies=archive:1) or include=*.jpeg) => not (include=*.jpeg) -"""]] diff --git a/doc/todo/importfeed_needs_more_memory_the_more_urls_there_are.mdwn b/doc/todo/importfeed_needs_more_memory_the_more_urls_there_are.mdwn deleted file mode 100644 index eaca6becb1..0000000000 --- a/doc/todo/importfeed_needs_more_memory_the_more_urls_there_are.mdwn +++ /dev/null @@ -1,19 +0,0 @@ -git-annex tries to run in a constant amount of memory, however `knownUrls` -loads all urls ever seen into a list, so the more urls there are, the more -memory `git annex importfeed` will need. - -This is probably not a big problem in practice, but seems worth doing -something about if somehow possible. - -Unfortunately, can't use a bloom filter, because a false positive would -prevent importing an url that has not been imported before. A sqlite -database would work, but would need to be updated whenever the git-annex -branch is changed. --[[Joey]] - -> A sqlite database that is updated by diffing would probably also speed up -> the stage of importfeed where it gathers known urls. That can be -> significantly slow in large repos. So I think worth doing. - -[[!tag confirmed]] - -> [[done]] --[[Joey]] diff --git a/doc/todo/importfeed_parallell.mdwn b/doc/todo/importfeed_parallell.mdwn deleted file mode 100644 index 832a2a217a..0000000000 --- a/doc/todo/importfeed_parallell.mdwn +++ /dev/null @@ -1,5 +0,0 @@ -Make git-annex importfeed support -J. Ideally, when run with multiple feed -utls, it would check them in parallel, and also parallelize download of new -items found in the feeds. --[[Joey]] - -> [[done]] --[[Joey]] diff --git a/doc/todo/info_--size-history.mdwn b/doc/todo/info_--size-history.mdwn deleted file mode 100644 index 15c464ace8..0000000000 --- a/doc/todo/info_--size-history.mdwn +++ /dev/null @@ -1,62 +0,0 @@ -Support eg `git-annex info --size-history=30d` which would display -the combined size of all repositories every 30 days throughout the history -of the git-annex branch. This would allow graphing, analysis, etc of repo -growth patterns. - -Also, `git-annex info somerepo --size-history=30d` would display the size -of only the selected repository. - -Maybe also a way to get the size of each repository plus total size in a -single line of output? - ----- - -Implementation of this is fairly subtle. My abandoned first try just went -through `git log` and updated counters as the location logs were updated. -That resulted in bad numbers. (The size went negative eventually in fact!) -The problem is that the git-annex branch is often updated both locally and -on a remote, eg when copying a file to a remote. And that results in 2 -changes to the git-annex branch that both record the same data. So it gets -counted twice by my naive implementation. - -I think it is not possible for an accumulation based approach to work in -constant memory and fast. In the worst case, there is a fork of the branch -that diverges hugely over a long period of time. So that divergence either -needs to be buffered in memory, or recalculated repeatedly. - -What I think needs to be done is use `git log --reverse --date-order git-annex`. -Feed the changed annex log file refs into catObjectStream to get the log -files. (Or use --patch and parse the diff to extract log file lines, -might be faster?) Parse the log files, and update a simple data structure: - - Map Key [UUIDPointer] - -Where UUIDPointer is a number that points to the UUID in a Map. This -avoids storing copies of the uuids in the map. - -This is essentially union merging all forks of the git-annex branch at -each commit, but far faster and in memory. Since union merging a git-annex -branch can be done at any point and always results in a consistent view of -the data, this will be consistent as well. - -And when updating the data structure, then it can update a counter when -something changed, and avoid updating it when a redundant log was logged. - -This approach will use an amount of memory that scales with -the number of keys and numbers of copies. I mocked it up using my big -repository. Storing every key in it in such a map, with 64 UUIDPointers -in the list (many more than the usual number of copies) took 2 gb of -memory. Which is a lot but also most users have that much if necessary. -With a more usual 5 copies, memory use was only 0.5 gb. So I think this is -an acceptable exception to git-annex's desire to use a constant amount of -memory. - -(I considered a bloom filter, but a false positive would wreck the -statistics. An in-memory sqlite db might be more efficient, but probably -not worth the bother.) - -[[!tag confirmed]] - ---[[Joey]] - -> [[done]] diff --git a/doc/todo/info__58___allow_file_matching_options_for_all_keys.mdwn b/doc/todo/info__58___allow_file_matching_options_for_all_keys.mdwn deleted file mode 100644 index 365c6e0047..0000000000 --- a/doc/todo/info__58___allow_file_matching_options_for_all_keys.mdwn +++ /dev/null @@ -1,13 +0,0 @@ -I want to see aggregate stats on all keys known to git-annex while using matching options like `--in here`, `--copies` etc. - -The obvious thing I tried was to use `git annex info --in here` but that complains: - -``` -git-annex: File matching options can only be used when getting info on a directory. -``` - -There should be a way to use `info` to query aggregate information properties of all keys instead of directories. - -I have used `git annex info .` in the repos I used up until now because every key was in the tree. Though I also have a feeling that operating on all keys could be significantly faster than filtering them to match some directory. - -> [[done]] --[[Joey]] diff --git a/doc/todo/info__58___allow_file_matching_options_for_all_keys/comment_1_3b96becda1db6ae23adb5077b6332718._comment b/doc/todo/info__58___allow_file_matching_options_for_all_keys/comment_1_3b96becda1db6ae23adb5077b6332718._comment deleted file mode 100644 index 45cbb1e736..0000000000 --- a/doc/todo/info__58___allow_file_matching_options_for_all_keys/comment_1_3b96becda1db6ae23adb5077b6332718._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-02-21T16:19:27Z" - content=""" -You can use: `git-annex info . --in here` - -But, it should be possible when getting info for all keys -to use the matching options like --in that do not match on a filename. -It used to be that there was no way to tell which kind of matching options -where used, but now `matchNeedsFileName` is available and it could only -reject those. - -So this can be implemented by making cachedPresentData -and cachedRemoteData (etc) get the matcher, check if it's -the right kind and apply it to the keys. - -done -"""]] diff --git a/doc/todo/info__58___allow_file_matching_options_for_all_keys/comment_2_2d0a6abd5be8afce677eeea575aae548._comment b/doc/todo/info__58___allow_file_matching_options_for_all_keys/comment_2_2d0a6abd5be8afce677eeea575aae548._comment deleted file mode 100644 index 5b91f004fa..0000000000 --- a/doc/todo/info__58___allow_file_matching_options_for_all_keys/comment_2_2d0a6abd5be8afce677eeea575aae548._comment +++ /dev/null @@ -1,25 +0,0 @@ -[[!comment format=mdwn - username="Atemu" - avatar="http://cdn.libravatar.org/avatar/86b8c2d893dfdf2146e1bbb8ac4165fb" - subject="comment 2" - date="2023-12-25T13:37:58Z" - content=""" -Sorry for the late reply but this somehow didn't reach me. - -I just tried querying some basic stats such as `--not --copies 2` and running that with `git annex info .` works as expected: - -``` -directory: . -local annex keys: 0 -local annex size: 0 bytes -annexed files in working tree: 0 -size of annexed files in working tree: 0 bytes -``` - - -`git annex info --not --copies 2` confusingly displays the repo overview and then starts calculating first local and then \"annexed files in working tree\" keys. It does not show the useful repo overview which displays how many matched keys are in each repo that has at least one however. - -With that filter, `git annex info .` takes 15s in my case while `git annex info` takes 20s, likely due to the local key lookup. Trying to avoid the local lookup using `--fast` unfortunately also avoids the working tree lookup; outputting the same info as `git annex info --fast` without any filters which doesn't seem very useful. - -Does the latter really query all keys however? It appears to me that it's the same as querying `.`. I have nearly 100 unused keys in that repo but both info commands show the same amount. -"""]] diff --git a/doc/todo/info__58___allow_file_matching_options_for_all_keys/comment_3_aae866dc71811074d69b91c2390ccb01._comment b/doc/todo/info__58___allow_file_matching_options_for_all_keys/comment_3_aae866dc71811074d69b91c2390ccb01._comment deleted file mode 100644 index 940d1e2801..0000000000 --- a/doc/todo/info__58___allow_file_matching_options_for_all_keys/comment_3_aae866dc71811074d69b91c2390ccb01._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2023-12-29T14:24:45Z" - content=""" -`git annex info --not --copies 2` displays the global overview (because you -have not specified a directory), and limits it to keys that have the -specified number of copies. - -Unused keys are included in the "local annexed keys" count when using -`git-annex info` without a directory. - -I agree it would make sense for `git-annex info` to display something -similar to the "repositories containing these files". Although in -the global overview it should show the total annex size of each repository. -[[todo/info_show_total_annex_sizes_of_repositories]] -"""]] diff --git a/doc/todo/info__58___allow_file_matching_options_for_all_keys/comment_4_371d53b4539bee6e3d595154033a2471._comment b/doc/todo/info__58___allow_file_matching_options_for_all_keys/comment_4_371d53b4539bee6e3d595154033a2471._comment deleted file mode 100644 index 3dbdaf3a62..0000000000 --- a/doc/todo/info__58___allow_file_matching_options_for_all_keys/comment_4_371d53b4539bee6e3d595154033a2471._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="Atemu" - avatar="http://cdn.libravatar.org/avatar/86b8c2d893dfdf2146e1bbb8ac4165fb" - subject="comment 4" - date="2023-12-29T17:06:37Z" - content=""" -I may have been unclear in my wording but what I really want is `info` to be able to show me numbers on *all* keys of the git-annex branch, including keys not in the current working tree or repo. -The intent is that I want to preserve past versions and also want to be able to query copies, where they're located etc. of those together with the \"used\" keys. I don't care whether they're referenced in the current revision and want to be able to tell `git-annex info` to not care too. - -Ideally, this should skip any unused check; operating only on the git-annex branch's data and none of master's. (That might also yield a much needed performance benefit.) - -Total size of annex sounds amazing; I've been wanting that ever since I set up distributed cold storage. I'll give it a spin :) -"""]] diff --git a/doc/todo/info_show_total_annex_sizes_of_repositories.mdwn b/doc/todo/info_show_total_annex_sizes_of_repositories.mdwn deleted file mode 100644 index d96e7e9b51..0000000000 --- a/doc/todo/info_show_total_annex_sizes_of_repositories.mdwn +++ /dev/null @@ -1,10 +0,0 @@ -`git-annex info` when run with a path displays "repositories containing these files" -with a list of repositories and sizes. It would be good for it, when not -run with a path to display a corresponding "annex size of repositories". - -I think that this recently became possible to inplement cheaply. -cachedAllRepoData is collected now, used for "combined annex size of all repositories". -For this, it also needs to accumulate the size of each separate repository. ---[[Joey]] - -> [[done]] --[[Joey]] diff --git a/doc/todo/key_checksum_from_chunk_checksums.mdwn b/doc/todo/key_checksum_from_chunk_checksums.mdwn deleted file mode 100644 index e6c28f58ab..0000000000 --- a/doc/todo/key_checksum_from_chunk_checksums.mdwn +++ /dev/null @@ -1,4 +0,0 @@ -Would it be hard to add a variantion to checksumming [[backends]], that would change how the checksum is computed: instead of computing it on the whole file, it would first be computed on file chunks of given size, and then the final checksum computed on the concatenation of the chunk checksums? You'd add a new [[key field|internals/key_format]], say cNNNNN, specifying the chunking size (the last chunk might be shorter). Then (1) for large files, checksum computation could be parallelized (there could be a config option specifying the default chunk size for newly added files); (2) I often have large files on a remote, for which I have md5 for each chunk, but not for the full file; this would enable me to register the location of these fies with git-annex without downloading them, while still using a checksum-based key. - -> Closing, because [[external_backends]] is implemented, so you should be -> able to roll your own backend for your use case here. [[done]] --[[Joey]] diff --git a/doc/todo/key_checksum_from_chunk_checksums/comment_1_946e2679cafd2c74a3b0b408e39f2d8d._comment b/doc/todo/key_checksum_from_chunk_checksums/comment_1_946e2679cafd2c74a3b0b408e39f2d8d._comment deleted file mode 100644 index c112bc9c26..0000000000 --- a/doc/todo/key_checksum_from_chunk_checksums/comment_1_946e2679cafd2c74a3b0b408e39f2d8d._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2019-05-06T16:10:17Z" - content=""" -Parallizable checksums exist, I would rather add support for a standard one -to git-annex than roll my own. In fact, I already did when I added -BLAKE2SP224 etc which are 8-way parallel and also can be optimised by SIMD -instructions on less parallel systems. -"""]] diff --git a/doc/todo/key_checksum_from_chunk_checksums/comment_2_aaff896ff99366d3b96c523284c1248e._comment b/doc/todo/key_checksum_from_chunk_checksums/comment_2_aaff896ff99366d3b96c523284c1248e._comment deleted file mode 100644 index 9a02bff2de..0000000000 --- a/doc/todo/key_checksum_from_chunk_checksums/comment_2_aaff896ff99366d3b96c523284c1248e._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Ilya_Shlyakhter" - avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" - subject="comment 2" - date="2019-05-07T00:59:31Z" - content=""" -Thanks for the BLAKE2SP224 pointer; it does solve (1). I'm still looking for the best way to solve (2): registering large remote files with git-annex, without downloading them. `addurl --fast` does that, but creates a non-checksum key. If I can get an MD5 without downloading, I can use `setpresentkey`. But often I only have the MD5 for the fixed-size chunks of the file, not for the whole. Adding a backend variant computable from MD5s of the chunks would solve the problem. Maybe, there are other solutions? -"""]] diff --git a/doc/todo/key_checksum_from_chunk_checksums/comment_3_352bc3eb419184a4ec70ddaffd78968e._comment b/doc/todo/key_checksum_from_chunk_checksums/comment_3_352bc3eb419184a4ec70ddaffd78968e._comment deleted file mode 100644 index fc9baaea47..0000000000 --- a/doc/todo/key_checksum_from_chunk_checksums/comment_3_352bc3eb419184a4ec70ddaffd78968e._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2019-06-26T14:51:32Z" - content=""" -That seems like an unusual use case that would be unnecessary complication -to add to git-annex, but that [[external_backends]] could be used to -implenent as needed. -"""]] diff --git a/doc/todo/key_checksum_from_chunk_checksums/comment_4_e766f725e22ea0a38ac2277e08fd3839._comment b/doc/todo/key_checksum_from_chunk_checksums/comment_4_e766f725e22ea0a38ac2277e08fd3839._comment deleted file mode 100644 index 82e760c8cf..0000000000 --- a/doc/todo/key_checksum_from_chunk_checksums/comment_4_e766f725e22ea0a38ac2277e08fd3839._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="Chel" - avatar="http://cdn.libravatar.org/avatar/a42feb5169f70b3edf7f7611f7e3640c" - subject="comment 4" - date="2020-01-26T22:48:07Z" - content=""" -Another theoretical use case (not available for now, but maybe for the future): -verify with checksums parts of the file and re-download only those parts/chunks, that are bad. -For this you need a checksum for each chunk and a \"global\" checksum in key, that somehow incorporates all these chunk checksums. -An example of this is Tiger Tree Hash in file sharing. - -When I used the SHA256 backend in my downloads, I often wondered that the long process of checksumming a movie -or an OS installation .iso is not ideal. Because if the file download is not finished, I get the wrong checksum, -and the whole process needs to be repeated. - -And in the future git-annex can integrate a FUSE filesystem and literally store just chunks of files, -but represent files as a whole in this virtual filesystem view. -"""]] diff --git a/doc/todo/key_checksum_from_chunk_checksums/comment_5_561b9bb28c5d375334ce915da75d5ce6._comment b/doc/todo/key_checksum_from_chunk_checksums/comment_5_561b9bb28c5d375334ce915da75d5ce6._comment deleted file mode 100644 index afd7e7e4ef..0000000000 --- a/doc/todo/key_checksum_from_chunk_checksums/comment_5_561b9bb28c5d375334ce915da75d5ce6._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="Ilya_Shlyakhter" - avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" - subject="chunks and checksums" - date="2020-01-28T18:20:34Z" - content=""" -\"verify with checksums parts of the file and re-download only those parts/chunks, that are bad.\" -- if I understand correctly, git-annex doesn't checksum [[chunks|chunking]], but can tell incompletely downloaded chunks based on size. - -My original use case (registering the presence of a chunked file in a remote without downloading it) might be implementable with [[todo/setpresentkey_option_to_record_chunked_state/]]. The checksums of the chunks would not be used though. - -"""]] diff --git a/doc/todo/more_efficient_resolution_of_trivial_export_conflicts.mdwn b/doc/todo/more_efficient_resolution_of_trivial_export_conflicts.mdwn deleted file mode 100644 index 5a6fe0d1e0..0000000000 --- a/doc/todo/more_efficient_resolution_of_trivial_export_conflicts.mdwn +++ /dev/null @@ -1,20 +0,0 @@ -Export conflicts happen whenever two repos export different trees -to the same special remote. - -Some conflicts that seem trivial currently involve unnecessary unexporting -and re-uploading in their resolution. - -For example, if A exports a tree containing `[foo]`, and B exports a tree -containing `[foo, bar]`, bar gets unexported when resolving the conflict. - -Bear in mind that A could have first exported a tree containing `[foo, bar]` -then B exported a tree containing `[foo, bar]`, and then A exported -`[foo]`, so unexported bar. - -So, if one exported tree is a subset of the other, it's not necessary to -unexport files added by the other tree. It's sufficient to check that files -are present in the export and upload any that are missing. --[[Joey]] - -[[!tag confirmed]] - -> I proved this is not a safe optimisation, so [[wontfix|done]] --[[Joey]] diff --git a/doc/todo/more_efficient_resolution_of_trivial_export_conflicts/comment_1_5a2d7ea060f5e1be11c39521c0a5631d._comment b/doc/todo/more_efficient_resolution_of_trivial_export_conflicts/comment_1_5a2d7ea060f5e1be11c39521c0a5631d._comment deleted file mode 100644 index 2db042e03c..0000000000 --- a/doc/todo/more_efficient_resolution_of_trivial_export_conflicts/comment_1_5a2d7ea060f5e1be11c39521c0a5631d._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="thk" - avatar="http://cdn.libravatar.org/avatar/bfef10a428769701aeee1db978951461" - subject="sponsoring?" - date="2020-03-19T08:40:31Z" - content=""" -I am running into this problem right now. I exported a movie collection from laptop A and then a huge collection of photos from laptop B to an USB drive. Both exports ran many hours. -Now when I want to export the next collection of files, I see: - -> git annex export master --to usb-sg2 -> -> Resolving export conflict.. -> -> unexport usb-sg2 path/to/some/picture/file.jpg ok - -Do you already have a solution for this in mind? Could I speed up the implementation with sponsorship? This is a hard blocker now for me in my consolidation of our data with git-annex. I would rather not spend many nights to keep exporting and unexporting the same files over and over. - -Maybe I could hack this as a temporary workaround by manually changing the exported trees in the exports.log file? The \"conflicting\" exports are in two separate sub-trees. -"""]] diff --git a/doc/todo/more_efficient_resolution_of_trivial_export_conflicts/comment_2_70c3cfa9b95f3aa6c2e2e2f6dbbc50d0._comment b/doc/todo/more_efficient_resolution_of_trivial_export_conflicts/comment_2_70c3cfa9b95f3aa6c2e2e2f6dbbc50d0._comment deleted file mode 100644 index 6f8c923df9..0000000000 --- a/doc/todo/more_efficient_resolution_of_trivial_export_conflicts/comment_2_70c3cfa9b95f3aa6c2e2e2f6dbbc50d0._comment +++ /dev/null @@ -1,66 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2020-05-04T17:30:59Z" - content=""" -I took a look at this, the relevant code is here: - - warning "Resolving export conflict.." - forM_ ts $ \oldtreesha -> do - -- Unexport both the srcsha and the dstsha, - -- because the wrong content may have - -- been renamed to the dstsha due to the - -- export conflict. - let unexportboth d = - [ Git.DiffTree.srcsha d - , Git.DiffTree.dstsha d - ] - -- Don't rename to temp, because the - -- content is unknown; delete instead. - mapdiff - (\diff -> commandAction $ startUnexport r db (Git.DiffTree.file diff) (unexportboth diff)) - oldtreesha new - -So, it diffs from the tree in the export conflict to the new, resolved tree. -Any file that differs -- eg, any file that was involved in the conflict -- -gets removed from the export. - -> In the todo, I said: -> -> For example, if A exports a tree containing `[foo]`, and B exports a tree -> containing `[foo, bar]`, bar gets unexported when resolving the conflict. - -Let's be more clear about the content of the trees, and say A exports -`[(foo, 1)]` and B exports `[(foo, 1), (bar, 1)]`. - -If the export is then resolved to `[(foo, 1), (bar, 1)]`, -we can see nothing needs to be done. But what it currently does is -diff from `[(foo, 1)]` to the resolution and so unexports bar. - -If B had instead exported `[(foo, 1), (bar, 2)]`, then -it would still need to diff from that the the resolution, and so would -unexport bar, and so it should. - -But.. but.. ugh. Consider an export conflict that started with `[(bar, 1)]` -exported. A exported `[(bar, 2)]` and B exported `[(baz, 1)]` (by renaming bar -to baz). So the export remote might contain `[(baz, 2)]` (A uploaded 2 to bar, -and then B renamed bar to baz) or it might contain `[(bar, 2), (baz, 1)]`; -we do not know ordering between A and B. - -If the export conflict resolution is `[(bar, 2), (baz 1)]` then the tree -exported by B is a subset, so it skips that one. And the tree -exported by A is a subset, so ummm... it skips that one. And so nothing gets -unexported. Then, it proceeds to try to upload any missing files -to the export. If the export remote contains `[(bar, 2), (baz, 1)]` nothing is -missing, nothing gets uploaded, all is well. But if the export remote -contains `[(baz, 2)]`, it will upload `(bar, 2)`, resulting in -`[(bar, 2), (baz 2)]`. That is not what it's supposed to contain. - -So, no, this optimisation will not work! - -The only way to make this optimisation work, I think, is to not do renaming -when updating export remotes. But file renaming is more common than export -conflicts; you can always adjust your workflow to avoid export conflicts, -by pulling from the remote you tend to conflict with, before performing an -export. -"""]] diff --git a/doc/todo/more_efficient_resolution_of_trivial_export_conflicts/comment_3_d0549f8a07032ce61e88b4ccb2c6ef3b._comment b/doc/todo/more_efficient_resolution_of_trivial_export_conflicts/comment_3_d0549f8a07032ce61e88b4ccb2c6ef3b._comment deleted file mode 100644 index 2b2b8314fc..0000000000 --- a/doc/todo/more_efficient_resolution_of_trivial_export_conflicts/comment_3_d0549f8a07032ce61e88b4ccb2c6ef3b._comment +++ /dev/null @@ -1,31 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2020-05-04T18:36:30Z" - content=""" -Hmm, but.. What if both A and B's trees are subsets -of the resolved tree? Safe then? - -A exports `[(foo, 1)]`, while B exports `[(bar, 2)]` -the resolved tree is `[(foo, 1), (bar, 2)]`. - -Well, what was in the export before? Suppose it was `[(foo, 2)]`.. -Then B would have renamed foo to bar, and A exported 1 to foo. -Order is unknown, so the export has either of `[(foo, 1), (bar, 2)]` -or `[(bar, 1)]` - -Yeah, still not safe even when both trees are subsets. - ----- - -An optimisation like this needs some way to detect if there's been a rename -like B keeps doing in these examples. If there has not been any rename, -the optimisation is safe. - -export.log contains only the sha of the tree that has been exported -by each repo to the export remote. It might contains some trees that -were exported before, but when it gets compacted, that information is -lost, and anyway there's no way to know if B exported some old tree before -or after A exported its most recently exported tree. So, I don't think -retrospective rename detection is possible. -"""]] diff --git a/doc/todo/move_readonly_values_to_AnnexRead.mdwn b/doc/todo/move_readonly_values_to_AnnexRead.mdwn deleted file mode 100644 index 8aabd70b37..0000000000 --- a/doc/todo/move_readonly_values_to_AnnexRead.mdwn +++ /dev/null @@ -1,8 +0,0 @@ -AnnexRead has recently been split out of AnnexState, -and reading values from it is faster (no MVar). So, -anything that never needs to be modified while git-annex is running can be -moved to AnnexRead for a performance win and also to make clean how it's -used. --[[Joey]] - -> [[done]]; all AnnexState fields have been checked and all that can be -> made readonly are. --[[Joey]] diff --git a/doc/todo/new_command_for_syncing_content_only.mdwn b/doc/todo/new_command_for_syncing_content_only.mdwn deleted file mode 100644 index 834a9808c7..0000000000 --- a/doc/todo/new_command_for_syncing_content_only.mdwn +++ /dev/null @@ -1,23 +0,0 @@ -`git-annex sync` has been split into push/pull, but there is another split -possible: `git-annex sync --no-commit --no-pull --no-push --content`. - -That is a mouthful, so maybe it could be a separate command? - -> Actually, that used to sync content with older git-annex at least some -> of the time, but [[!commit 2e984c51b65165e79c2fcd5212712ac4e73850de]] -> changed/fixed --no-pull and --no-push to also prevent syncing content. -> This new command would be the content-syncing part of sync only. - -Also, such a command could be perhaps be more like `git-annex get/drop/copy` -in operating on the listed files or current directory and below by default. -Although that might make it difficult for it to handle sending files to -exporttree remotes, which sync does with `seekExportContent` which operates -on the whole exported tree. - -What would be a good name for such a command? --[[Joey]] - -> How about `git-annex port` as it carries content over like a luggage carrier aka. porter? -> Another one that comes to mind is `git-annex shuttle` but that could be semantically too -> similar to `git-annex assist`, perhaps. --[[jkniiv]] - ->> `git-annex satisfy` [[done]] --[[Joey]] diff --git a/doc/todo/new_command_for_syncing_content_only/comment_1_7f983de645430d970c75f25fee27b2ed._comment b/doc/todo/new_command_for_syncing_content_only/comment_1_7f983de645430d970c75f25fee27b2ed._comment deleted file mode 100644 index 234a8d3120..0000000000 --- a/doc/todo/new_command_for_syncing_content_only/comment_1_7f983de645430d970c75f25fee27b2ed._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 1" - date="2023-06-29T05:20:56Z" - content=""" -It would also be awesome if there was something like `git annex synconlycontent --recent` or `--since=HEAD~10` or `--since=\"1 week ago\"` to just operate on the latest changed files. -In contrast to `--accessedwithin` this would look at the git history to check when files were touched. This is currently possible with something like `git diff --name-only | sort -u | xargs git annex ...` or `git log --since=... --format... | ... | xargs git annex` but it's far from convenient. - -This might be a general new matching option for other subcommands as well. -"""]] diff --git a/doc/todo/new_command_for_syncing_content_only/comment_2_311cd71d61bea5782c71ee33282d152e._comment b/doc/todo/new_command_for_syncing_content_only/comment_2_311cd71d61bea5782c71ee33282d152e._comment deleted file mode 100644 index ad4e368e0a..0000000000 --- a/doc/todo/new_command_for_syncing_content_only/comment_2_311cd71d61bea5782c71ee33282d152e._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="How about `git annex transfer`?" - date="2023-06-29T05:25:50Z" - content=""" -Throwing `git annex transfer` into the ring for an alias for `git annex sync --content --no-commit --no-push --no-pull`. -"""]] diff --git a/doc/todo/new_command_for_syncing_content_only/comment_3_19398beb81c56b3b7cc3aff7081a2886._comment b/doc/todo/new_command_for_syncing_content_only/comment_3_19398beb81c56b3b7cc3aff7081a2886._comment deleted file mode 100644 index 5d238a7a99..0000000000 --- a/doc/todo/new_command_for_syncing_content_only/comment_3_19398beb81c56b3b7cc3aff7081a2886._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2023-06-29T17:49:32Z" - content=""" -@nobodyinperson open another todo for the --recent or similar idea. -Something like that would be just another matching option. -(However, getting and parsing `git log --since` seems like it would be hard -to do efficiently..) -"""]] diff --git a/doc/todo/new_command_for_syncing_content_only/comment_4_c72cbc566af9585b4701f833883f878e._comment b/doc/todo/new_command_for_syncing_content_only/comment_4_c72cbc566af9585b4701f833883f878e._comment deleted file mode 100644 index ed40e20c9f..0000000000 --- a/doc/todo/new_command_for_syncing_content_only/comment_4_c72cbc566af9585b4701f833883f878e._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2023-06-29T17:51:30Z" - content=""" -@nobodyinperson `git-annex transfer` is not bad. Although I was envisioning -this also handling dropping content from repos that don't want it, same as -sync does. - -Well, I suppose getting a file from a remote and dropping it from the -remote can also be seen as a "transfer". Just because git-annex internally -uses "transfer" specifically for copying an object around doesn't mean the -command line interface has to. -"""]] diff --git a/doc/todo/new_command_for_syncing_content_only/comment_5_857157a775711330df39b57828cf9bd4._comment b/doc/todo/new_command_for_syncing_content_only/comment_5_857157a775711330df39b57828cf9bd4._comment deleted file mode 100644 index 09317dc85e..0000000000 --- a/doc/todo/new_command_for_syncing_content_only/comment_5_857157a775711330df39b57828cf9bd4._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2023-06-29T17:58:01Z" - content=""" -How about `satisfy`? It satisfies preferred content configuration. - -(Not numcopies configuration though, as implemented.) -"""]] diff --git a/doc/todo/new_command_for_syncing_content_only/comment_6_3ca0ef151b73781d2fe168c56d54faad._comment b/doc/todo/new_command_for_syncing_content_only/comment_6_3ca0ef151b73781d2fe168c56d54faad._comment deleted file mode 100644 index 3134158dfc..0000000000 --- a/doc/todo/new_command_for_syncing_content_only/comment_6_3ca0ef151b73781d2fe168c56d54faad._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="Thanks for git annex satisfy, numcopies question" - date="2023-06-30T05:37:06Z" - content=""" -Marvellous! I was wondering, if `git annex sync` doesn't satisfy numcopies, what does? Is numcopies just for drop/move? `sync`/`satisfy` won't override the preferred content to satisfy numcopies, right? Then my mental model ”git annex ensures there's always at least `numcopies` copies of your files” wasn't really true. It just ”won't by itself reduce the amount of copies of a file below numcopies/mincopies”, right? If that's true, would a command or option to sync/satisfy make sense that enforces numcopies and potentionally overrides the preferred content? -"""]] diff --git a/doc/todo/new_command_for_syncing_content_only/comment_7_084e139c8325bb8607f6c0a9f3873afe._comment b/doc/todo/new_command_for_syncing_content_only/comment_7_084e139c8325bb8607f6c0a9f3873afe._comment deleted file mode 100644 index 36ebe85763..0000000000 --- a/doc/todo/new_command_for_syncing_content_only/comment_7_084e139c8325bb8607f6c0a9f3873afe._comment +++ /dev/null @@ -1,26 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""Re: numcopies question""" - date="2023-07-05T21:22:10Z" - content=""" -It won't send content that is otherwise not wanted just to satisfy -numcopies. Of course, like all commands, it refuses to drop a file if -numcopies is not satisfied. - -Comsider a remote foo that only wants `include=foo/*`. And another remote -bar that wants `include=bar/*`. And numcopies=2. If `git-annex satisfy` -was only able to access foo, and worked to satisfy numcopies, foo would end -up with files in bar/* that it doesn't want. But there's a better -place for those files to go, namly bar, which wants them. So over time it -would do a large amount of unncessary work, sending files to foo, only -to later drop those same files from foo once it was able to populate bar. - -If the user wants to, they can configure foo to want -`include=foo/* or lackingcopies=1` and then `git-annex satisfy` will know -this is a repository that wants any files that don't have enough copies, -and when bar is not accessible, it will send its files to foo. - -I think this could stand to be better documented. Especially since -things like `git-annex copy --auto` *do* work to satisfy numcopies in -addition to preferred content. -"""]] diff --git a/doc/todo/new_command_for_syncing_content_only/comment_8_c666e3ef20e823230392a70d6b588e7d._comment b/doc/todo/new_command_for_syncing_content_only/comment_8_c666e3ef20e823230392a70d6b588e7d._comment deleted file mode 100644 index b2fd3654f6..0000000000 --- a/doc/todo/new_command_for_syncing_content_only/comment_8_c666e3ef20e823230392a70d6b588e7d._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="comment 8" - date="2023-07-06T05:52:50Z" - content=""" -Aah okay, I forgot there's `lackingcopies`. So if one adds `or lackingcopies=1` to every (sensible) remote's preferred content, then git annex is allowed to put \"fragile\" files there which haven't been copied enough yet (until there eventually are and they're removed again). I was missing that link, now everything makes sense. - -And now I see that this is exactly what the `client` group does 😅 -"""]] diff --git a/doc/todo/onlyingroup_preferred_content_expression.mdwn b/doc/todo/onlyingroup_preferred_content_expression.mdwn deleted file mode 100644 index 54abb4db0e..0000000000 --- a/doc/todo/onlyingroup_preferred_content_expression.mdwn +++ /dev/null @@ -1,24 +0,0 @@ -Hi joey, - -While writing a more complex preferred content expression today I noticed that a `onlyingroup=GROUPNAME` condition would be handy. - -Consider a sneakernet where clumps of nodes can communicate quickly with each other but the inter-clump communication is not so great. This is the case for my setup where a raspberry pi in the field has a USB stick attached (so two repos in `field` group), but their uplink is slow mobile data. Then I have an `offsite-backup` group for other disks/servers that I have access to. - -When I sync content from within an `offsite-backup` repo, I don't want it to copy data via the slow mobile data - this I do via sneakernet by swapping the USB drive in the field from time to time. Not setting up the `field` repos here is possible but prevents syncing of metadata, which is not ideal. - -If I could do `git annex groupwanted offsite-backup 'anything and not onlyingroup=field'`, the preferred content expression would be quite consise and short. `onlyingroup` would match if no other group has the file in question, only the given group. This would extend the functionality already provided by `inallgroup`. - -Without `onlyingroup`, one has to basically hard-code the amount of repos in the group with a rather complex expression ( 2 in this case, the Pi and the USB drive): - -``` -(anything and not copies=field:1 or copies=untrusted+:2) or (anything and not copies=field:2 or copies=untrusted+:3) -``` - -One also has to specifically catch each combination case where files between the `field` group are (only on the Pi? Only on the USB drive? on both?). The `copies=untrusted+:2` is then needed to allow copying other files that are around (e.g. because they were indeed copied over mobile data) but still in the `field` group. - -Maybe you see an easier way, but this is what I came up with. I think an `onlyingroup` expression would be very helpful. - -Cheers, -Yann - -> [[done]] --[[Joey]] diff --git a/doc/todo/onlyingroup_preferred_content_expression/comment_1_eddf28997789400398fbdd58ec64861d._comment b/doc/todo/onlyingroup_preferred_content_expression/comment_1_eddf28997789400398fbdd58ec64861d._comment deleted file mode 100644 index dee6f17dc3..0000000000 --- a/doc/todo/onlyingroup_preferred_content_expression/comment_1_eddf28997789400398fbdd58ec64861d._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="onlyingroup also useful for offline backup drives" - date="2023-07-31T04:40:42Z" - content=""" -Such an `onlyingroup` group would also be useful to match (and ignore from standard sync operations) files that are only available on hard-to-reach `offline-backup` repos, say offline backup drives or those in a safe. Normal repos can want `include=* and not onlyingroup=offline-backup` and then won't bother with unreachable files. This is an alternative to the `archive/` directory, which might be undesired or impractical to use (original file location is 'lost' when moving files into the archive). -"""]] diff --git a/doc/todo/onlyingroup_preferred_content_expression/comment_2_b1a22e4fbc594c3085e7fbc6fbaab69a._comment b/doc/todo/onlyingroup_preferred_content_expression/comment_2_b1a22e4fbc594c3085e7fbc6fbaab69a._comment deleted file mode 100644 index 1f5658d5fb..0000000000 --- a/doc/todo/onlyingroup_preferred_content_expression/comment_2_b1a22e4fbc594c3085e7fbc6fbaab69a._comment +++ /dev/null @@ -1,34 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2023-07-31T17:12:52Z" - content=""" -This seems like a pretty good idea. In particular, I think that the -`not onlyingroup=offline-backup` example is persuasive. The -`field` group example might be able to be dealt with by having -a `collected` group and using `copies=collected:1` in repositories -that only want a copy of data once it's been collected from the field. - -Hmm, one problem is that if you have 2 groups that you are using -onlyingroup with, eg -`(not onlyingroup=offline-backup) and (not onlyingroup=offline-archive)`. -A file that is in two repositories, one for each group, will not match. - -One way to address that would be something like -`onlyingroups=offline-backup,offline-archive` which would match -on presence in any listed groups. I don't really like that this has -its own ANDing happening inside a term of a preferred content expression -though. - -What is onlyingroup supposed to do when a repository is in the -specified group, but also in another group? My first reaction was that -it should never match when content is in such a repository, since the -content is indeed present in another group. -But that doesn't really seem useful. - -If it treated a repository that's in the specified group, but also is -in other groups, as matching, then it would be possible to put a repository -in groups offline and backup and match on `not onlyingroup=offline`. -I like this better than the `onlyingroups=offline-backup,offline-archive` -because it's simpler and more composable. -"""]] diff --git a/doc/todo/onlyingroup_preferred_content_expression/comment_3_8f205160e62aa1312b8e2ef4cafe8c50._comment b/doc/todo/onlyingroup_preferred_content_expression/comment_3_8f205160e62aa1312b8e2ef4cafe8c50._comment deleted file mode 100644 index e59f4a66ff..0000000000 --- a/doc/todo/onlyingroup_preferred_content_expression/comment_3_8f205160e62aa1312b8e2ef4cafe8c50._comment +++ /dev/null @@ -1,22 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2023-07-31T17:51:00Z" - content=""" -Another way to look at this is that preffered content allows expressing -`copies=offline:1`, but there's no way to express a match on the -number of copies that are in repositories that are not in the offline group. - -That might be expressed as say, `copies=!offline:1` - -Then `not copies=!offline:0` would be useful to only want files that are -available in some repository that is not offline. - -This would avoid the question of what to do with onlyingroup when a -repository is in the specified group but also another one. - -I really don't like the `copies=!offline:1` syntax though. (I already -disliked copies=group:n` and this takes it in an even worse direction.) - -So while this is more generic, I kind of prefer the onlyingroup name.. -"""]] diff --git a/doc/todo/sharedRepository_mode_not_supported_by_git-annex.mdwn b/doc/todo/sharedRepository_mode_not_supported_by_git-annex.mdwn deleted file mode 100644 index 14ab5fd529..0000000000 --- a/doc/todo/sharedRepository_mode_not_supported_by_git-annex.mdwn +++ /dev/null @@ -1,9 +0,0 @@ -git's core.SharedRepository is supported by git-annex, but only -with the group/all/world/everybody settings. core.SharedRepository=0644 -etc is not supported. - -There's no insormountable reason why not, Joey just hates umask mode math -stuff and nobody has sent a patch. Note that Annex.Content.freezeContent -should remove the write bit from files, no matter what. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/todo/sharedRepository_mode_not_supported_by_git-annex/comment_1_b219d62cd7d21002ae73d0dbd4b792be._comment b/doc/todo/sharedRepository_mode_not_supported_by_git-annex/comment_1_b219d62cd7d21002ae73d0dbd4b792be._comment deleted file mode 100644 index 8584a60423..0000000000 --- a/doc/todo/sharedRepository_mode_not_supported_by_git-annex/comment_1_b219d62cd7d21002ae73d0dbd4b792be._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="yarikoptic" - avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" - subject="comment 1" - date="2023-04-26T14:34:31Z" - content=""" -Hi Joey, We just [now (re)discovered for ourselves in DataLad](https://github.com/psychoinformatics-de/knowledge-base/issues/15) that octal modes are not supported by git-annex **at all**. We wonder if you are planing to add support for them, or should we may be just start [warning users](https://github.com/datalad/datalad/pull/7375) if there is an unsupported configuration of shared setup is encountered. -"""]] diff --git a/doc/todo/sharedRepository_mode_not_supported_by_git-annex/comment_2_7140ad0abbb54f5a4e20b165a08793e2._comment b/doc/todo/sharedRepository_mode_not_supported_by_git-annex/comment_2_7140ad0abbb54f5a4e20b165a08793e2._comment deleted file mode 100644 index e9b4680cd8..0000000000 --- a/doc/todo/sharedRepository_mode_not_supported_by_git-annex/comment_2_7140ad0abbb54f5a4e20b165a08793e2._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2023-04-26T16:50:23Z" - content=""" -I don't think it would be that hard to implement, I count 6 places, all in -Annex.Perms that don't handle UmaskShared. - -For now I've made all of them display a warning. - -(Note that your warning patch to datalad is probably buggy, -since core.sharedrepository=1 and =2 are supported by git-annex. -(As legacy alternatives to "group" and "everybody".)) -"""]] diff --git a/doc/todo/sharedRepository_mode_not_supported_by_git-annex/comment_3_c3dff87829e3f055881829eb4598f903._comment b/doc/todo/sharedRepository_mode_not_supported_by_git-annex/comment_3_c3dff87829e3f055881829eb4598f903._comment deleted file mode 100644 index 3ea11b49c1..0000000000 --- a/doc/todo/sharedRepository_mode_not_supported_by_git-annex/comment_3_c3dff87829e3f055881829eb4598f903._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2023-04-26T17:39:21Z" - content=""" -I've implemented this today! - -(Did notice a bug, -[[wrong_modes_for_some_lock_files_withcoresharedrepository]]) -"""]] diff --git a/doc/todo/show_dead_in_info___40__behind_option__63____41__.mdwn b/doc/todo/show_dead_in_info___40__behind_option__63____41__.mdwn deleted file mode 100644 index ebb40443e3..0000000000 --- a/doc/todo/show_dead_in_info___40__behind_option__63____41__.mdwn +++ /dev/null @@ -1,3 +0,0 @@ -Would it be possible for `git annex info` to also show dead remotes (behind an option is fine), to make it easy to find what dead remotes there are? I needed to check what I had done for a service that was going away (AARNET Cloudstor, which used webdav), and was confused as to why it wasn't appearing anywhere. - -> Implemented `git-annex info --dead-repositories` [[done]] --[[Joey]] diff --git a/doc/todo/show_dead_in_info___40__behind_option__63____41__/comment_1_74c3bbb7819368a894ff1189988e4cd3._comment b/doc/todo/show_dead_in_info___40__behind_option__63____41__/comment_1_74c3bbb7819368a894ff1189988e4cd3._comment deleted file mode 100644 index de77f57b16..0000000000 --- a/doc/todo/show_dead_in_info___40__behind_option__63____41__/comment_1_74c3bbb7819368a894ff1189988e4cd3._comment +++ /dev/null @@ -1,44 +0,0 @@ -[[!comment format=mdwn - username="nobodyinperson" - avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5" - subject="Workaround for listing dead remotes" - date="2023-07-26T17:40:56Z" - content=""" -Current workaround is to do some shell magic: - -[[!format bash \"\"\" -❯ git show git-annex:trust.log | grep ' X ' | sort -r -k3,3 | sort -u -k1,1 | sort -k3,3 -00000000-0000-0000-0000-000000000001 X timestamp=1665735093.590311882s -00000000-0000-0000-0000-000000000002 X timestamp=1665735096.60698876s -bccdc354-3e91-434d-b558-92bda037262f X timestamp=1665756359.672362517s -07e1a2c4-6a48-4149-8c1d-578d01d101ca X timestamp=1666190795.982309234s -6f2f88b1-75b0-4628-aca0-b7e404881787 X timestamp=1666190981.07615739s -c15fa203-2c18-4925-ba00-bf2896c837b8 X timestamp=1666191690.101011744s -9b0e8fa2-6516-489a-a765-535185fd13da X timestamp=1666252027.044902201s -6eece6e9-af12-4493-812b-a93ef37d1c97 X timestamp=1666773713.866014699s -556463c4-498b-4c48-86cf-891d5d350d92 X timestamp=1666773716.613288244s -699550a9-8927-4e39-860f-9391dadf658a X timestamp=1666773720.004014314s - -\"\"\"]] - -This should be a time-sorted overview of all dead git annex remote IDs (read: get all trust level info from `git-annex`-branch, keep only dead ones, sort reversely by time to then keep only the most up-to-date entry per remote, finally sorting by time again). Unfortunately, `git annex info DEADREMOTEID` doesn't work (Temporarily setting the trustlevel back to something undead e.g. with `git -c remote.DEAD.annex-trustlevel=untrust -c remote.DEAD.annex-uuid=6f2f88b1-75b0-4628-aca0-b7e404881787 annex info` or putting the equivalent in `.git/config` also doesn't work.), so to get the description we need to read it manually from the `git-annex`-branch (provided you didn't run `git annex forget --drop-dead` of course...): - -[[!format bash \"\"\" -$ git show git-annex:uuid.log | grep -f <(git show git-annex:trust.log | grep ' X ' | cut -d' ' -f1) -12239c61-893b-409b-8593-242b3bd06d59 hetzner timestamp=1681837014.563205098s -1f010ad0-66f5-4b4d-abb6-c995639ddb8d also broken on Yann IdeaPad timestamp=1674395012.267824079s -1f010ad0-66f5-4b4d-abb6-c995639ddb8d yann@yann-ideapad:~/Sync-broken timestamp=1674392076.605762565s -6726366a-3417-4d87-a5e0-e6c6e174611e broken on Yann IdeaPad timestamp=1674395003.447162736s -6726366a-3417-4d87-a5e0-e6c6e174611e yann@yann-ideapad:~/Sync timestamp=1674392895.243361127s -7eacb3f6-668f-4c76-8a93-e60bb83195e9 yann@yann-ideapad:~/Sync2 timestamp=1672691029.638810716s -92f1c925-5360-4d35-8876-97724e5e8d68 hetzner timestamp=1681841459.689233275s -a295b60b-440e-43be-8d63-a1251b939039 hetzner timestamp=1681837210.139095335s -cc849692-1dca-4d07-889a-2c0b37fef154 hetzner timestamp=1681836822.12092766s -\"\"\"]] - - -This is as far as it gets in terms of prettiness I'm afraid. - -I'd also like an option to show dead remotes, e.g. with `git annex info --all` or `git annex info --show-dead`. And having `git annex info DEADREMOTEID` work would also be nice 🙂 - -"""]] diff --git a/doc/todo/show_dead_in_info___40__behind_option__63____41__/comment_2_0cc7a3c53ffbabcb13807956541d83ec._comment b/doc/todo/show_dead_in_info___40__behind_option__63____41__/comment_2_0cc7a3c53ffbabcb13807956541d83ec._comment deleted file mode 100644 index 09e8ebdb85..0000000000 --- a/doc/todo/show_dead_in_info___40__behind_option__63____41__/comment_2_0cc7a3c53ffbabcb13807956541d83ec._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="aragilar" - avatar="http://cdn.libravatar.org/avatar/7390f7c0dd07d43ca827a5446b65c102" - subject="comment 2" - date="2023-07-28T02:47:42Z" - content=""" -Yep, I realised I could grep trust.log to see that I'd marked it as dead, but it did take me a bit of time to realise this (I was thinking \"did I enable cloudstor on this machine, maybe it only shows up on machines you enable it on, but I though I did that previously...\"). - -I think even a count of dead remotes (so you know what you need to look for) would be helpful, if minimal changes are required. -"""]] diff --git a/doc/todo/show_me_where_unused_file_was__44___i_can_wait.mdwn b/doc/todo/show_me_where_unused_file_was__44___i_can_wait.mdwn deleted file mode 100644 index e4b34aa223..0000000000 --- a/doc/todo/show_me_where_unused_file_was__44___i_can_wait.mdwn +++ /dev/null @@ -1,9 +0,0 @@ -i know that `git annex unused` would be slower if, instead of just showing the hash, it would also show the pathname where the file was. it does tell me that i can use `git log --stat -SKEY` to find that out myself, but then i would need to make some silly shell script to loop over multiple files. i'm hoping that git-annex has more efficient and clever ways of doing that, and even if it's slower, i'd be ready to wait if there was an extra flag to show me where it was... - -i have used this oneliner so far, but it's ugly and painful, especially since `git annex unused` doesn't have a very parseable output format... - - git annex unused 2>&1 | grep '^ *[0-9][0-9]*' | sed 's/^ *[0-9][0-9]* *//' | xargs -I'{}' git log --oneline --stat -S'{}' -1 - -any way to do this more easily? --[[anarcat]] - -> [[done]]; use `git-annex whereused --unused` --[[Joey]] diff --git a/doc/todo/show_me_where_unused_file_was__44___i_can_wait/comment_1_732f726f72f2540372d434974fda60f2._comment b/doc/todo/show_me_where_unused_file_was__44___i_can_wait/comment_1_732f726f72f2540372d434974fda60f2._comment deleted file mode 100644 index 522696b18e..0000000000 --- a/doc/todo/show_me_where_unused_file_was__44___i_can_wait/comment_1_732f726f72f2540372d434974fda60f2._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2015-10-26T19:29:12Z" - content=""" -git-annex has no better way to find files that linked to that key than the -given log -S. - -However, your script can be improved by reading the list of unused keys -from `.git/annex/unused` -"""]] diff --git a/doc/todo/specify_gitconfig_for_test_suite.mdwn b/doc/todo/specify_gitconfig_for_test_suite.mdwn deleted file mode 100644 index 78cf7d78f5..0000000000 --- a/doc/todo/specify_gitconfig_for_test_suite.mdwn +++ /dev/null @@ -1,8 +0,0 @@ -`git-annex test` prevents ~/.gitconfig or /etc/gitconfig from being read. -The `-c` option also doesn't propagate into the test suite. - -It would sometimes be useful to test git-annex with a given git config set. -Although some might break the test suite, others might expose actual bugs -in git-annex. --[[Joey]] - -> Added "--test-git-config" option, [[done]] --[[Joey]] diff --git a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests.mdwn b/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests.mdwn deleted file mode 100644 index 3476c8e73e..0000000000 --- a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests.mdwn +++ /dev/null @@ -1,41 +0,0 @@ -As part of the [datalad/git-annex](https://github.com/datalad/git-annex) builds setup there is work to enable testing on some target deployments, but there is problem (see [git-annex/pull/102#issuecomment-1054561966](https://github.com/datalad/git-annex/pull/102#issuecomment-1054561966)) that overall sweep of `git annex test` of the standalone build on a target system takes over 5,000 seconds (over an hour!). - -Clearly that system is "unique" in its "performance" (most likely due to nfs4'ed filesystem tests go under, not due to "Standalone" but I can't change the issue title at this moment) and here is the timing spread we see from other runs on CI: - -```shell -(git)smaug:/mnt/datasets/datalad/ci/git-annex/builds/2022/02[master]git -$> git grep 'All 9.. tests passed (' | sort -t '(' -n -k 3 | head -n 5 -cron-20220222/build-ubuntu.yaml-586-85dd9355-failed/1_test-annex (normal, ubuntu-latest).txt:2022-02-22T02:57:08.4002055Z All 994 tests passed (239.44s) -cron-20220222/build-ubuntu.yaml-586-85dd9355-failed/test-annex (normal, ubuntu-latest)/7_Run tests.txt:2022-02-22T02:57:08.4002052Z All 994 tests passed (239.44s) -pr-102/build-ubuntu.yaml-600-12c9bb39-failed/1_test-annex (normal, ubuntu-latest).txt:2022-02-25T18:47:54.5577129Z All 994 tests passed (240.31s) -pr-102/build-ubuntu.yaml-600-12c9bb39-failed/test-annex (normal, ubuntu-latest)/7_Run tests.txt:2022-02-25T18:47:54.5577125Z All 994 tests passed (240.31s) -cron-20220225/build-ubuntu.yaml-595-85dd9355-failed/1_test-annex (normal, ubuntu-latest).txt:2022-02-25T03:01:16.8138190Z All 994 tests passed (240.87s) - -$> git grep 'All 9.. tests passed (' | sort -t '(' -n -k 3 | tail -n 5 -cron-20220227/build-macos.yaml-604-85dd9355-failed/test-annex (normal, macos-latest)/8_Run tests.txt:2022-02-27T02:32:31.2444240Z All 994 tests passed (1284.60s) -cron-20220203/build-macos.yaml-578-85dd9355-success/3_test-annex (normal, macos-latest).txt:2022-02-03T02:27:38.5945050Z All 991 tests passed (1292.09s) -cron-20220203/build-macos.yaml-578-85dd9355-success/test-annex (normal, macos-latest)/8_Run tests.txt:2022-02-03T02:27:38.5945040Z All 991 tests passed (1292.09s) -manual-20220222/build-macos.yaml-599-85dd9355-failed/3_test-annex (normal, macos-latest).txt:2022-02-22T18:44:42.5220730Z All 994 tests passed (1340.62s) -manual-20220222/build-macos.yaml-599-85dd9355-failed/test-annex (normal, macos-latest)/8_Run tests.txt:2022-02-22T18:44:42.5220720Z All 994 tests passed (1340.62s) -``` - -so we have runs as short as 4 minutes (`cron-20220222/build-ubuntu.yaml-586-85dd9355-failed`) and as "long" as 20 minutes (`manual-20220222/build-macos.yaml-599-85dd9355-failed`) . - -but at >5k seconds it made me wonder if worth looking into making operation on such system a bit more "snappy". FWIW, here are the slowest tests: - -``` -[d31548v@discovery7 git-annex]$ grep 's)' tests.out | sort -t '(' -n -k 2 | tail -n 10 - conflict resolution: OK (55.24s) - adjusted branch merge regression: OK (56.63s) - conflict resolution movein regression: OK (60.02s) - transition propagation: OK (60.61s) -OK (61.00s) -OK (70.30s) -OK (78.37s) - union merge regression: OK (91.15s) - transition propagation: OK (97.68s) -All 994 tests passed (5576.99s) - -``` - -> [[done]] --[[Joey]] diff --git a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_10_0e070bc39eb0b8d9cf307af8a06504dc._comment b/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_10_0e070bc39eb0b8d9cf307af8a06504dc._comment deleted file mode 100644 index 0c2d092239..0000000000 --- a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_10_0e070bc39eb0b8d9cf307af8a06504dc._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="yarikoptic" - avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" - subject="comment 10" - date="2022-03-16T20:32:57Z" - content=""" -Awesome -- thank you ! I will give it a test use after the next daily build. -"""]] diff --git a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_1_41749c4c9b317a58e02df99f748844a1._comment b/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_1_41749c4c9b317a58e02df99f748844a1._comment deleted file mode 100644 index bcf8307548..0000000000 --- a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_1_41749c4c9b317a58e02df99f748844a1._comment +++ /dev/null @@ -1,23 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-03-01T18:06:32Z" - content=""" -Since you mentioned standalone builds, I suspect you are running -something like: "cd git-annex.linux; ./git-annex test" - -I have verified that is much much slower than running git-annex not -from the standalone build. On the order of 1+ hours vs 5 minutes. - -There is a simple workaround, run "git-annex.linux/git-annex" -from somewhere outside the standalone build. That will make it much faster. -Here it takes about 5 minutes. - -With that said, there's still a lot of room to speed up git-annex -test, and the main thing would probably be to parallelize its tests. -Which can be done, but needs tasty-1.2. That finally made it into debian -stable, so it should be able to depend on it now. I would not be surprised -if it can be sped up 10x that way, because tests often have to wait one or -more seconds after writing a file due to time stamp issues etc. -I have created a `paralleltest` branch with a start on that, not yet working. -"""]] diff --git a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_2_f462ee4719b4c459c12e36ce29ab82b5._comment b/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_2_f462ee4719b4c459c12e36ce29ab82b5._comment deleted file mode 100644 index 03a1db490a..0000000000 --- a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_2_f462ee4719b4c459c12e36ce29ab82b5._comment +++ /dev/null @@ -1,13 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2022-03-01T19:58:37Z" - content=""" -I have fixed the problem I identified, which was due to git-annex test adding -the cwd to PATH, which caused it to run git-annex.linux/git, so runshell -was being run repeatedly and unncessarily. - -Now it will run git-annex.linux/bin/git and avoid the repeated runshell -overhead, so will be about as fast as git-annex not run from the standalone -tarball. -"""]] diff --git a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_3_dfd20b6c6a49c99243945a267e041d58._comment b/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_3_dfd20b6c6a49c99243945a267e041d58._comment deleted file mode 100644 index 82b3acb149..0000000000 --- a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_3_dfd20b6c6a49c99243945a267e041d58._comment +++ /dev/null @@ -1,38 +0,0 @@ -[[!comment format=mdwn - username="yarikoptic" - avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" - subject="still slow" - date="2022-03-08T00:27:59Z" - content=""" -I have ran tests with \"bleeding edge\" build of `git-annex` on ndoli (less busy node) of discovery and unfortunately it is still quite slow -- took 50 minutes - -
-more details - -```shell -> time git annex test 2>&1 | tee 10.20220222+git62-gce523f756-1~ndall+1-tests.log -... -OK (71.94s) - preferred content: OK (34.75s) - required_content: OK (17.34s) - add subdirs: OK (12.63s) - addurl: OK (10.71s) - -All 840 tests passed (2670.47s) - -real 48m52.802s -user 3m23.073s -sys 4m53.776s - -[d31548v@ndoli tmp]$ git annex version | head -git-annex version: 10.20220222+git62-gce523f756-1~ndall+1 -... - -[d31548v@ndoli tmp]$ pwd -/dartfs-hpc/rc/lab/C/CANlab/labdata/data/tmp - -``` -
- -so there is probably more to the story (NFS), or probably needing nfs4_*etfacl? well -- I ran under a folder which should be proper POSIX -- took 5075.48s (the same 10.20220222+git62-gce523f756-1~ndall+1) -"""]] diff --git a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_4_650e5dd2471cdfd0223d712771067d55._comment b/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_4_650e5dd2471cdfd0223d712771067d55._comment deleted file mode 100644 index 678bd3cd5a..0000000000 --- a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_4_650e5dd2471cdfd0223d712771067d55._comment +++ /dev/null @@ -1,24 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2022-03-08T18:16:40Z" - content=""" -One thing I notice is that the test suite reports -it took 44 minutes (2670 seconds), but time reports -49 minutes. Those additional 5 minutes must be the test -suite cleaning up the test directories. Which fits with NFS. -That is 5 minutes to effectively `rm -rf` maybe 20k directories/files. - -Also, you originally said it took more than 1 hour (or perhaps more -than 5000 seconds, which would be 1.4 hours). So it seems that my fix did -have a significant impact on speed. - -What I see benchmarking locally is that the standalone tarball takes 1016 -seconds (down from 3600+), while a bare git-annex binary takes 614 seconds. -That is probably due to the small overhead (100 failed opens) discussed in -[[this_old_todo|todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__]], -multiplied by the thousands of times the test suite runs git-annex. - -Parallelizing the test suite seems like the only way to get a substantial -speedup. -"""]] diff --git a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_5_0ee51fffee1f591173418212919bcd30._comment b/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_5_0ee51fffee1f591173418212919bcd30._comment deleted file mode 100644 index 388e95c0b5..0000000000 --- a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_5_0ee51fffee1f591173418212919bcd30._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2022-03-08T19:43:01Z" - content=""" -Re that 5 minutes to clean up, I tried making it clean up test directories -in the background while running other tests. Interestingly, that slowed -it down here by 33 seconds (5%). Due to disk IO contention I suppose? -(SSD) - -Makes me wonder how much benefit parallelism would be.. -"""]] diff --git a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_6_b4aecfd9f1b7f0e16d5f1dcb3c16af3e._comment b/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_6_b4aecfd9f1b7f0e16d5f1dcb3c16af3e._comment deleted file mode 100644 index 5bdd08053d..0000000000 --- a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_6_b4aecfd9f1b7f0e16d5f1dcb3c16af3e._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 6""" - date="2022-03-08T19:47:12Z" - content=""" -@yoh It would be interesting if you could check the speed on NFS when bypassing -the standalone build's overhead by running -`git-annex.linux/shimmed/git-annex/git-annex test` - -That will not use the bundled libraries and programs, but if the test -system is reasonably similar to the build system it would still work. - -It's good you're testing the standalone build works, but I think this -test is about testing NFS really, so you could leave the standalone build -testing to other test runs than this one, if that is a significant speedup. -"""]] diff --git a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_7_e11ae8ba6dd6e6c53891db954d28bfff._comment b/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_7_e11ae8ba6dd6e6c53891db954d28bfff._comment deleted file mode 100644 index 517af1ead8..0000000000 --- a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_7_e11ae8ba6dd6e6c53891db954d28bfff._comment +++ /dev/null @@ -1,45 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 7""" - date="2022-03-10T17:58:07Z" - content=""" -I tried as an experiment, opening 5 terminals, and running each of the 5 -main groups of tests in parallel manually, each command in a different -directory: - - git-annex test -p Tests.QuickCheck - git-annex test -p Tests.Remote - git-annex test -p 'Tests.Unit Tests v8 locked' - git-annex test -p 'Tests.Unit Tests v8 unlocked' - git-annex test -p 'Tests.Unit Tests v8 adjusted unlocked branch' - -They took, respectively, 34, 58, 159, 154, and 220 seconds. Compared to a -sequential runtime of 444 seconds, this shows it can be sped up well by -parallelism at least in some cases. Seems likely that splitting up the -slower blocks further and having 8 groups of tests could make it faster yet. - -@yoh, it would be interesting if you could try this on the NFS system and -see if it speeds it up enough. - ---- - -Tasty does not seem to have a way for parallel forks of the test program -to report back their status in a way that will be combined together. That -does seem like something that could perhaps be added to it in a nice clean -way. - -But, a quick hack is also possible: Have git-annex test fork off one child -process per each of these groups, and serialize the output. Using ---color=always when at the console and using concurrent-output to stream -one of the currently running tests while buffering the rest for later -display should make this almost indistinguishable from the "right" way. -Would also need to detect some tasty options and fall back to running it -normally. - -This approach would avoid the problem I hit on the `paralleltest` branch of -needing to rewrite all the testing code to not run chdired into the repo -in order to make it able to be run in parallel. That would be a lot of -work, and would also make it harder to write new tests, since it would be -easy to make a mistake that caused a test to write outside the test -repo. -"""]] diff --git a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_8_b06360071e05df18b895322382357b61._comment b/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_8_b06360071e05df18b895322382357b61._comment deleted file mode 100644 index da6246634c..0000000000 --- a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_8_b06360071e05df18b895322382357b61._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 8""" - date="2022-03-14T18:52:09Z" - content=""" -I have implemented parallelism as described in comment 7. - -Currently there are 5 child processes, and the test runtime -dropped from 444 to 334 seconds on my laptop. -Splitting up the test groups further, so there are more child -processes will probably improve that more. -Remains to be seen if it helps on NFS much.. - -The `git-annex test` output is currently a mess, it needs to be serialized. -Ran out of time to do that today, but the speed improvement is worth -temporarily ugly output. -"""]] diff --git a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_9_2253c6916a1c6e4f1393b8656617aa10._comment b/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_9_2253c6916a1c6e4f1393b8656617aa10._comment deleted file mode 100644 index 3050c2dd35..0000000000 --- a/doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_9_2253c6916a1c6e4f1393b8656617aa10._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 9""" - date="2022-03-16T17:55:52Z" - content=""" -I've finished up parallelizing git-annex test. - -Splitting up the test groups further and improved scheduling sped it up more. -On my laptop, it's dropped from 444 to 334 to now 289 seconds. - -Also, the `-J` option is now supported by git-annex test, so you can experiment -to find the number of jobs where it runs fastest in your particular situation. -The default is one job per CPU core. - -My guess is that on NFS, it's not CPU bound but is network latency bound, -and so a rather high -J value like -J10 may behave better. -"""]] diff --git a/doc/todo/terminal_escapes_in_filenames.mdwn b/doc/todo/terminal_escapes_in_filenames.mdwn deleted file mode 100644 index ed56ea37b0..0000000000 --- a/doc/todo/terminal_escapes_in_filenames.mdwn +++ /dev/null @@ -1,70 +0,0 @@ - touch $(echo -e "\e[31mfoo\e[0m") - git-annex add - git-annex whereis - -That displays "foo" in red twice. Compare with behavior of git commands that -display that filename, which display it escaped. - -git-annex should probably do the same, when displaying filenames that it's -working on or in messages. - -`git-annex find` is an interesting case because it's expected to be -pipeable, and so should have raw filenames. Note that `find` actually -escapes such filenames when outputting to a terminal, but not a pipe. - -git porcelain also accepts the escaped form of files as input, necessary for -round-tripping though. git-annex currently does not. (git plumbing doesn't -either) - -While terminals mostly protect against escape sequences doing very bad -things, there are security holes in terminals still being found. - -Of course, such files in git repos can also be exploited by other commands -eg `echo *`. - -So this does not seem like a security hole in git-annex, but it would be -useful defense in depth against terminal security holes, and also good to -behave more like git. - ---[[Joey]] - -> Git.Filename.encode is implemented, and only needs to be used. -> Note that core.quotePath controls whether git quotes unicode characters -> (by default it does), so once this gets implemented, some users may want -> to set that config to false. --[[Joey]] - -> Update: Messages now handles quoting of filenames, and also filtering -> out any escape sequences in other things that get displayed (like Keys..) -> -> Update: git-annex find and others that output directly all do -> escaping/filtering. - ----- - -Also: -It's possible that keys can also contain an escape sequence, eg in the -extension of a SHA-E key. So commands like `git-annex lookupkey` -and `git-annex find` that output keys might need to handle -that, when outputting to a terminal? - -> Fixed this with filtering of control characters in output. - -Also: git-annex initremote with autoenable may be able to cause a remote -with a malicious name to be set up? - -> Fixed this by silently skipping autoenable, which seems fine since only -> an attacker would ever try this. - -Also: Any place that an exception is thrown with an attacker-controlled value. -`giveup` has been made to filter out control characters, but that leaves -other exceptions, including ones thrown by libraries. Catch all exceptions -at top-level (of program and/or worker threads) and filter out control -characters? - -> Fixed with a top-level exception catcher; assuming all worker threads -> have something waiting on them that displays or propagates their -> exceptions. - ----- - -> all [[fixed|done]]! --[[Joey]] diff --git a/doc/todo/test_suite_unsafe_use_of_setEnv.mdwn b/doc/todo/test_suite_unsafe_use_of_setEnv.mdwn deleted file mode 100644 index 859d9ea428..0000000000 --- a/doc/todo/test_suite_unsafe_use_of_setEnv.mdwn +++ /dev/null @@ -1,22 +0,0 @@ -The test suite uses setEnv in several places. (No other part of git-annex -does, basically.) However, setEnv is not thread safe, if another thread -runs getEnv at the same time it can crash. Or another setEnv, probably. - -This actually does segfault the test suite runners from time to time. -The test suite currently notices if there was a segfault and re-runs it. -While that works well enough, it's always possible there could be some -other behavior than just a segfault. - -See for a test case and -some analysis. See also . -Tasty uses getEnv, and that can apparently run while a test case is using setEnv. - -The main places that setEnv is used are in setTestMode. Fixing it will mean -plumbing a value all through the test suite. Or perhaps, using a single -toplevel MVar would be ok, since tests don't run concurrently? - -There is also Utility.Gpg.testHarness, which sets GNUPGHOME. It seems that -instead, every place that git-annex is run inside the gpg test harness -would need to add GNUPGHOME to the environment of the git-annex process. - -> [[fixed|done]] --[[Joey]] diff --git a/doc/todo/tor.mdwn b/doc/todo/tor.mdwn deleted file mode 100644 index b1197643e7..0000000000 --- a/doc/todo/tor.mdwn +++ /dev/null @@ -1,28 +0,0 @@ -git-annex sync over tor - -Mostly working! - -Current todo list: - -* When a transfer can't be done because another transfer of the same - object is already in progress, the message about this is output by the - remotedaemon --debug, but not forwarded to the peer, which shows - "Connection reset by peer" -* Think about locking some more. What happens if the connection to the peer - is dropped while we think we're locking content there from being dropped? - -Eventually: - -* Windows and Android support. -* Limiting authtokens to read-only access. -* Revoking authtokens. (This and read-only need a name associated with an - authtoken, so the user can adjust its configuration after creating it.) -* friend-of-a-friend peer discovery to build more interconnected networks - of nodes -* Discovery of nodes on same LAN, and direct connection to them. -* Make `git annex map` show a peer's remotes? - Would it be surprising if peers can learn this information? - -> While the above "eventually" todo list is not implemented, the main -> todo has been implemented for a long time. I think if users need -> such things, they should be in separate todos. So, [[done]] --[[Joey]] diff --git a/doc/todo/unify_adjust_with_view.mdwn b/doc/todo/unify_adjust_with_view.mdwn deleted file mode 100644 index d9e9b46570..0000000000 --- a/doc/todo/unify_adjust_with_view.mdwn +++ /dev/null @@ -1,73 +0,0 @@ -`git annex adjust` and `git annex view` (et all) both derive a branch from -the main branch and enter it. They have different capabilies. It would be -useful to be able to compose them. For example, to enter a view based on -metadata that also has all files unlocked. ---[[Joey]] - -> Now that `git-annex sync` supports view branches, this is more apparently -> a problem. -> -> It's possible to check out a view branch, and then -> `git-annex adjust --unlock`, and the result will be a view branch with -> unlocked files. But then `git-annex sync` doesn't work; it neither syncs -> back to the parent view branch, nor all the way back to the parent master -> branch. It seems that git-annex gets confused about what this strangely -> named branch is (`"adjusted/views/master(author=_)(unlocked)"`), -> and does not treat it as either kind of branch. -> -> Conversely, first doing `git-annex adjust --unlock`, followed by checking -> out a view branch from there results in a view branch that does not have -> the files unlocked. (`"views/adjusted/master(unlocked)(author=_)"`). And -> syncing fails from there: -> -> fatal: not a valid object name: 'adjusted/master' -> git-annex: failed to update refs/heads/synced/adjusted/master ---- - -There's also probably a fair amount of overlap in their implementations. - -> I now think not really so much. View branches are constructed using a -> temporary index file (and need it), while adjusted branches are able to -> stream to `git mktree`. -> -> The main overlap is that there is a basis branch that gets transformed -> to a new branch. And a way for `git-annex sync` to update the branch when -> the basis branch (or other info) changes. - ----- - -Consider a branch that is a view branch, where `git-annex adjust --hide-missing` -was then run. After `git-annex drop`, updating the adjusted branch -should cause the dropped file to be removed. But removing a file -from a view branch means removing that metadata from the object. - -So, it seems that composing that kind of adjusted branch with -view branches is unlikely to work. - ----- - -Considering all the the above, I think this would be a good plan: - -Implemented adjusted view branches. Not views of adjusted branches. -Although running `git-annex adjust` with a view branch checked out could -be one way to get into an adjusted view branch. Or running `git-annex view` -with an adjusted branch checked out. - -Prohibit entering a view branch with the --hide-missing adjustment. -Update: Actually, I was able to implement support for that adjustment that -seems to work ok in a view branch. - -Building the branch seems simple: Construct the view branch like it does -now, using the master branch as the basis. And apply the adjustment to each -file that gets added to it. - -It seems it would make sense for an adjusted view branch to -have a name like `"views/adjusted/master(unlocked)(author=_)"` if -that can be parsed unambiguously. Update: Actually -`"adjusted/views/master(author=_)(unlocked)"` worked better. - -When `git-annex sync` runs in an adjusted view branch, it does not need to -do the usual adjusted branch propagation at all. Because the only change -that gets synced from a view branch is changes to metadata. - -> [[done]]! --[[Joey]] diff --git a/doc/todo/v9_changes.mdwn b/doc/todo/v9_changes.mdwn deleted file mode 100644 index 4b242359f1..0000000000 --- a/doc/todo/v9_changes.mdwn +++ /dev/null @@ -1,22 +0,0 @@ -This is a todo for collecting changes that could lead to a v9 repository -version. - -Currently, there does not seem to be enough reason to warrant one, but that -could change and if it does, these things could be included. - -* Change locking of annexed files to use a separate lock file - rather than posix locking the file itself. - - This would let write bits be removed from the file when - core.sharedRepository is set. See - - Note that windows already uses a separate lock file. - - This is being implemented in git branch `v9-locking`. - -* Possibly enable `git-annex filter-process` by default. If the tradeoffs - seem worth it. - - May want to implement [[incremental_hashing_for_add]] first. - -[[done]] --[[Joey]] diff --git a/doc/todo/whishlist__58___kde-connect_as_a_transport.mdwn b/doc/todo/whishlist__58___kde-connect_as_a_transport.mdwn deleted file mode 100644 index 003d79b62d..0000000000 --- a/doc/todo/whishlist__58___kde-connect_as_a_transport.mdwn +++ /dev/null @@ -1,9 +0,0 @@ -Running an SSH server on a phone is quite some setup and could perhaps even be considered insecure. - -KDE-connect provides a an easy to set up method of creating a transport tunnel between phone and computer. Many users might actually have it set up already. - -Apparently, it can even be used to connect multiple desktops with another but I haven't tested that. This could be an alternative (or perhaps even replacement?) to git-annex' current pairing mechanism. - -KDE-connect offers remote command execution and file sharing, so it should be possible to use it for git-annex' purposes. - -> [[rejected|done]] --[[Joey]] diff --git a/doc/todo/whishlist__58___kde-connect_as_a_transport/comment_1_14426ad1fa5dfc32d290da484c1e4c7b._comment b/doc/todo/whishlist__58___kde-connect_as_a_transport/comment_1_14426ad1fa5dfc32d290da484c1e4c7b._comment deleted file mode 100644 index a6a412057a..0000000000 --- a/doc/todo/whishlist__58___kde-connect_as_a_transport/comment_1_14426ad1fa5dfc32d290da484c1e4c7b._comment +++ /dev/null @@ -1,21 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2021-06-04T17:23:06Z" - content=""" -From what I can see, kde-connect lets the phone be used to run specific, -pre-selected commands on the linux computer it's paired with. Not the other -way around. -Also being able to run a command on a phone is rather a long way from what -git-annex actually needs. - -It seems like it would be possible to use the remote filesystem via sftp part -of it. -That would be a useful alternative to adb. I don't think git-annex needs -any changes to use it that way, just mount it with sshfs and -initremote a directory special remote with exporttree=yes and -importtree=yes. - -See also [[add_sftp_special_remote]] which if it were implemented would -avoid needing to use sshfs. -"""]] diff --git a/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities.mdwn b/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities.mdwn deleted file mode 100644 index 9442fa50ac..0000000000 --- a/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities.mdwn +++ /dev/null @@ -1,57 +0,0 @@ -I was going to report a bug with the transition to lts-19.13 in stack.yaml as I couldn't -get for instance commit [[!commit d2bc2683173ca1ade437aa294b26fd5e54353a55]] "lts-19.13 (ghc 9.0.2)" -that introduced it to compile on windows (with stack). But then I procrastinated too long and commit -[[!commit d01530ac21aead6607085b5cc90b62a52acd03a9]] came along which reverted the transition and -went back to lts-18.13 due to Joey having trouble getting ghc 9.0.2 to install. That was kind of fortunate -to me because I didn't want to be the bringer of bad news regarding Windows in a bug report setting. - -The problem with compiling with LTS 19 on Windows is the following: - -[[!format sh """ -[...] -[109 of 670] Compiling Utility.LockFile.Windows - -Utility\LockFile\Windows.hs:64:17: error: - * Variable not in scope: - c_CreateFile - :: LPTSTR - -> AccessMode - -> ShareMode - -> GHC.Ptr.Ptr a0 - -> CreateMode - -> FileAttributeOrFlag - -> GHC.Ptr.Ptr a1 - -> IO HANDLE - * Perhaps you meant `createFile' (imported from System.Win32.File) - | -64 | c_CreateFile c_f gENERIC_READ sharemode security_attributes - - | ^^^^^^^^^^^^ -Completed 229 action(s). - --- While building package git-annex-10.20220624 (scroll up to its section to see the error) using: - C:\Users\jkniiv\Projektit\git-annex.branchable.com\git-annex--BUILD-220630-d2bc26831\.stack-work\dist\d53b6a14\setup\setup --builddir=.stack-work\dist\d53b6a14 build exe:git-annex --ghc-options " -fdiagnostics-color=always" - Process exited with code: ExitFailure 1 -"""]] - -I digged into this and it turned out that as of Win32-2.12.0.0 it does not include those bindings -anymore that begin with `c_` in its exports and considers them as internal details. Unfortunately in -System.Win32.File the alternative, `createFile`, is a "cooked" API in the sense that if there are -intermittent errors opening the file there is some Haskell code that busy-waits on behalf of the user. -But according to Joey that is undesirable for a lockfile implementation (.\Utility\LockFile\Windows.hs:53): - -``` - - Note that createFile busy-waits to try to avoid failing when some other - - process briefly has a file open. But that would make checking locks - - much more expensive, so is not done here. Thus, the use of c_CreateFile. - -``` - -So what to do when the authors of package Win32 pull the rug from under you? Should we perhaps weave -our own tiny c_CreateFile API with FFI, or, could we do without it and use one of the implementations -using Windows LockFileEx, for instance the one by my compatriot: - -[[!meta title="windows: prepare for System.Win32.File not exporting c_ entities"]] -[[!meta author=jkniiv]] - -> [[fixed|done]] --[[Joey]] diff --git a/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_1_40985c5919f20fd47fa57caec09bfa3c._comment b/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_1_40985c5919f20fd47fa57caec09bfa3c._comment deleted file mode 100644 index a034ce08f5..0000000000 --- a/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_1_40985c5919f20fd47fa57caec09bfa3c._comment +++ /dev/null @@ -1,19 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2022-07-12T19:59:54Z" - content=""" -We could go argue with the Win32 maintainer to put it back in. -I have: - -But it is a simple FFI implementation so I would not mind just including it -in git-annex. - -... But also git-annex can depend on a version of Win32 before this -breakage. I've made that change. It's perhaps only a stopgap, if something -else that git-annex uses begins depending on the newer version of Win32 -later. - -Switching to LockFileEx seems like it might break interoperation with older -git-annex versions? -"""]] diff --git a/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_2_1d9c331e774138ceb89b1f7920bb76d4._comment b/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_2_1d9c331e774138ceb89b1f7920bb76d4._comment deleted file mode 100644 index c0323240a2..0000000000 --- a/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_2_1d9c331e774138ceb89b1f7920bb76d4._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 2" - date="2022-07-14T20:32:06Z" - content=""" -> We could go argue with the Win32 maintainer to put it back in. I have: https://github.com/haskell/win32/issues/208 - -Thanks. I see they seem be rather receptive to your suggestion. (I put in a thumbs up, too) The only issue I see is deciding on -how to name the raw version of createFile. Is there some precedent in Haskell land as to naming conventions? C and Python -would just prepend an underscore or two to the name without necessarily indicating that they would be thereby off limits (\"internal\"). -Maybe something like `_raw_CreateFile` with or without the underscore in the beginning? - -"""]] diff --git a/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_3_88b58dd27cbff3f046b6f8d1c612f87f._comment b/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_3_88b58dd27cbff3f046b6f8d1c612f87f._comment deleted file mode 100644 index 00662ed0d9..0000000000 --- a/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_3_88b58dd27cbff3f046b6f8d1c612f87f._comment +++ /dev/null @@ -1,50 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 3" - date="2022-07-21T20:08:54Z" - content=""" -I see that things are slowly moving ahead in that github issue. On a somewhat related note -I was able to compile with stack on Windows with lts-19.13 by adding the following extra-deps -to stack.yaml (starting from Win32 onward in the following patch): - -[[!format diff \"\"\" -diff --git a/stack.yaml b/stack.yaml -index 9b067c82f..27b0fe53c 100644 ---- a/stack.yaml -+++ b/stack.yaml -@@ -12,19 +12,21 @@ flags: - gitlfs: true - packages: - - '.' --resolver: lts-18.13 -+resolver: lts-19.13 - extra-deps: - - IfElse-0.85 - - aws-0.22 - - bloomfilter-2.0.1.0 - - git-lfs-1.2.0 --- http-client-restricted-0.0.4 -+- http-client-restricted-0.0.5 - - network-multicast-0.3.2 - - sandi-0.5 - - torrent-10000.1.1 --- base16-bytestring-0.1.1.7 --- base64-bytestring-1.0.0.3 --- bencode-0.6.1.1 --- http-client-0.7.9 -+- aeson-1.5.6.0 -+- Win32-2.9.0.0 -+- Cabal-3.6.3.0 -+- directory-1.3.7.0 -+- process-1.6.14.0 -+- time-1.11.1.2 - explicit-setup-deps: - git-annex: true -\"\"\"]] - -Yes, that was the newest Win32 I was able to make things work with as stack wasn't able -to resolve the cross-dependencies involving `Win32`, `directory`, `process` and `time` packages otherwise. -And if I remember correctly there was something called `haskeline` that also had specific -requirements. -"""]] diff --git a/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_4_28e5593e0a58fb7d7a2a297fa63e8208._comment b/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_4_28e5593e0a58fb7d7a2a297fa63e8208._comment deleted file mode 100644 index 5365531c7f..0000000000 --- a/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_4_28e5593e0a58fb7d7a2a297fa63e8208._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2022-08-02T14:43:47Z" - content=""" -The necessary change will be in win32 2.13.3.1, which should be released -in due course. And I've made a (so far untested) change to git-annex to -support that version. -"""]] diff --git a/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_5_43603d84801cfe0ecfcdc408db97ae22._comment b/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_5_43603d84801cfe0ecfcdc408db97ae22._comment deleted file mode 100644 index 8021baf121..0000000000 --- a/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_5_43603d84801cfe0ecfcdc408db97ae22._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="comment 5" - date="2022-08-03T15:14:22Z" - content=""" -Thanks, Joey, for making the pull request in Win32 with the necessary changes to support git-annex's -use case. I will be on the lookout for its next release and then as soon as possible try building -git-annex with LTS 19 via stack. I'll then report my findings here if you have not already done -something to that effect before me. -"""]] diff --git a/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_6_a623b2e131dd85f50e04483d262abc9a._comment b/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_6_a623b2e131dd85f50e04483d262abc9a._comment deleted file mode 100644 index 59f1c522a8..0000000000 --- a/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_6_a623b2e131dd85f50e04483d262abc9a._comment +++ /dev/null @@ -1,26 +0,0 @@ -[[!comment format=mdwn - username="jkniiv" - avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d" - subject="typo in Utility/LockFile/Windows.hs" - date="2022-08-03T20:46:14Z" - content=""" -It seems there was a tiny typo in commit [[!commit 472f5c142b063053828358a3031a2ca09a373127]], hence -the following simple correction is needed for the build to complete: - -[[!format diff \"\"\" -diff --git a/Utility/LockFile/Windows.hs b/Utility/LockFile/Windows.hs -index 9ec34c9a7..6985f408c 100644 ---- a/Utility/LockFile/Windows.hs -+++ b/Utility/LockFile/Windows.hs -@@ -61,7 +61,7 @@ lockExclusive = openLock fILE_SHARE_NONE - openLock :: ShareMode -> LockFile -> IO (Maybe LockHandle) - openLock sharemode f = do - f' <- convertToNativeNamespace f --#if MIN_VERSION_win32(2,13,3) -+#if MIN_VERSION_Win32(2,13,3) - r <- tryNonAsync $ createFile_NoRetry f' gENERIC_READ sharemode - security_attributes oPEN_ALWAYS fILE_ATTRIBUTE_NORMAL - (maybePtr Nothing) -\"\"\"]] - -"""]] diff --git a/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_7_91c17601eed0936bab17cdf9ec5f3ecb._comment b/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_7_91c17601eed0936bab17cdf9ec5f3ecb._comment deleted file mode 100644 index 24bf69ed23..0000000000 --- a/doc/todo/windows__58___prepare_for_Win32_not_export_c__95___entities/comment_7_91c17601eed0936bab17cdf9ec5f3ecb._comment +++ /dev/null @@ -1,7 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 7""" - date="2022-08-04T16:16:59Z" - content=""" -Ah yes, case-sensativity urk. Thanks, fixed. -"""]] diff --git a/doc/todo/wishlist__58___--dry-run_option_for_all_commands.mdwn b/doc/todo/wishlist__58___--dry-run_option_for_all_commands.mdwn deleted file mode 100644 index dd2a6263e5..0000000000 --- a/doc/todo/wishlist__58___--dry-run_option_for_all_commands.mdwn +++ /dev/null @@ -1,6 +0,0 @@ -Could a --dry-run option be added to the git annex commands? Or, at least, to the most common ones like `git annex add`. - -Given that there is no undo command, it would be nice to have the ability to simulate what git annex will do. - -> [[wontfix|done]], see comments. Instead, options like --want-get and -> --want-drop can be used to simulate things. --[[Joey]] diff --git a/doc/todo/wishlist__58___--dry-run_option_for_all_commands/comment_1_03bf493d5a7f957339f9aa388ba85ef8._comment b/doc/todo/wishlist__58___--dry-run_option_for_all_commands/comment_1_03bf493d5a7f957339f9aa388ba85ef8._comment deleted file mode 100644 index 7725747388..0000000000 --- a/doc/todo/wishlist__58___--dry-run_option_for_all_commands/comment_1_03bf493d5a7f957339f9aa388ba85ef8._comment +++ /dev/null @@ -1,20 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2014-11-03T16:44:45Z" - content=""" -This would add a lot of complexity; it's not like I could switch off -running all external commands, since many external commands are run to -query state to decide what to do. And then there's large chunks of code -that actually do stuff and would have to all be guarded to not run. - -I don't see the benefit to justify this work. `git annex add` is entirely -predictable; it's very similar to `git add`. Which itself lacks a dry-run -option. And like `git add`, you can certianly undo the effects of `git -annex add`. - -Matching options can make commands like `git annex find` list the same -set of files that are acted on by commands like `git annex drop` -(`git annex find --in here`) -and `git annex get` (`git annex find --not --in here`). -"""]] diff --git a/doc/todo/wishlist__58___--dry-run_option_for_all_commands/comment_2_8ac7ae6937d219854bc60b56b8646d8b._comment b/doc/todo/wishlist__58___--dry-run_option_for_all_commands/comment_2_8ac7ae6937d219854bc60b56b8646d8b._comment deleted file mode 100644 index d484d5559f..0000000000 --- a/doc/todo/wishlist__58___--dry-run_option_for_all_commands/comment_2_8ac7ae6937d219854bc60b56b8646d8b._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="Chris" - avatar="http://cdn.libravatar.org/avatar/02b67b7ebace8cb6433504ba9a17d4a4" - subject="comment 2" - date="2018-10-20T08:09:40Z" - content=""" -Hm, IMHO such a “dry-run” flag would be most beneficial for better understanding the preferred/required content settings and try their effect while working on finding the right expression. Therefore, the “drop”, “get” and “sync --content” commands would benefit from such a flag, especially in combination with flags such as “--auto”. Maybe it's just me, but I am not able to utilize the “find” command in a way that it respects preferred content and numcopies and further settings in the same way as the three commands above would do, and testing preferred content expressions is therefore a blind flight. -"""]] diff --git a/doc/todo/wishlist__58___--dry-run_option_for_all_commands/comment_3_411cabdfebe353d35f8dc78364f4a507._comment b/doc/todo/wishlist__58___--dry-run_option_for_all_commands/comment_3_411cabdfebe353d35f8dc78364f4a507._comment deleted file mode 100644 index a3d41b7244..0000000000 --- a/doc/todo/wishlist__58___--dry-run_option_for_all_commands/comment_3_411cabdfebe353d35f8dc78364f4a507._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="git-annex.branchable.com@1c3a8a83c15a19620a0a1a2e653d7c662fc8fe50" - nickname="git-annex.branchable.com" - avatar="http://cdn.libravatar.org/avatar/fd46557ea8b9b21ae4ce66517076fbcb" - subject="get dry-run-ish option" - date="2019-01-11T16:18:13Z" - content=""" -You can find out what would be done by a get (ish), but running something like this: - - git annex find --not --in here - -..which will show files which are known about but aren't in this annexed copy. -I've been using this to see what needs to be 'got' to a location. - -It does, however, need an annex repo to have a remote set up if you wish to check against it, as they aren't automatically bi-directional (which makes sense as firewalls and similar may not make this possible). -"""]] diff --git a/doc/todo/wishlist__58___--dry-run_option_for_all_commands/comment_4_2824eb6fd3fe9a44d48639c38d478351._comment b/doc/todo/wishlist__58___--dry-run_option_for_all_commands/comment_4_2824eb6fd3fe9a44d48639c38d478351._comment deleted file mode 100644 index 51b7f316ca..0000000000 --- a/doc/todo/wishlist__58___--dry-run_option_for_all_commands/comment_4_2824eb6fd3fe9a44d48639c38d478351._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2020-04-23T20:13:39Z" - content=""" -`git annex find --want-get --not --in here` should simulate get --auto/sync well - -`git annex find --want-drop --in here` will simulate what drop --auto/sync would -try to drop - -What would be nice to have is a way to simulate `copy --auto --to foo` -and `drop --auto --from foo` which are of course also things sync does. -That could be something like: - - git annex find --want-get-by remote --not --in remote - git annex find --want-drop-by remote --in remote -"""]] diff --git a/doc/todo/wishlist__58___--dry-run_option_for_all_commands/comment_5_e139434d88638d05c84b721abdbb406c._comment b/doc/todo/wishlist__58___--dry-run_option_for_all_commands/comment_5_e139434d88638d05c84b721abdbb406c._comment deleted file mode 100644 index 9f2dc9aee4..0000000000 --- a/doc/todo/wishlist__58___--dry-run_option_for_all_commands/comment_5_e139434d88638d05c84b721abdbb406c._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="kdm9" - avatar="http://cdn.libravatar.org/avatar/b7b736335a0e9944a8169a582eb4c43d" - subject="Bump for --want-get/drop-by" - date="2022-07-26T18:19:36Z" - content=""" -Hi Joey, - -A million thanks for a fantastic tool. - -I came across the need today for the aforementioned --want-get-by/ --want-drop-by REMOTE options. Is this something that has since been developed/worked around, or is it still in the infinite TODO? - -In the mean time, some more explicit documentation that --want-get/want-drop only ever applies to . would be good. I'll see if I can make a PR for this since I can write docs (but alas Haskell not so much). - -best, -Kevin -"""]] diff --git a/doc/todo/wishlist__58___--dry-run_option_for_all_commands/comment_6_cd430ea90286759eb5311ef376f63929._comment b/doc/todo/wishlist__58___--dry-run_option_for_all_commands/comment_6_cd430ea90286759eb5311ef376f63929._comment deleted file mode 100644 index 0d21988e6d..0000000000 --- a/doc/todo/wishlist__58___--dry-run_option_for_all_commands/comment_6_cd430ea90286759eb5311ef376f63929._comment +++ /dev/null @@ -1,7 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 6""" - date="2022-07-28T16:39:58Z" - content=""" -Thanks for the bump. I've implemented both. -"""]] diff --git a/doc/todo/wishlist__58___--maxdepth_option_for_git_annex_find.mdwn b/doc/todo/wishlist__58___--maxdepth_option_for_git_annex_find.mdwn deleted file mode 100644 index d997ca6e2c..0000000000 --- a/doc/todo/wishlist__58___--maxdepth_option_for_git_annex_find.mdwn +++ /dev/null @@ -1,24 +0,0 @@ -`git annex find` currently makes for a great way to find which files are already local, and don't need to get `git annex get` gotten; obviously `ls` just shows me all the files in a given directory, disregarding git-annex (and without recursing to subdirectories). I think that adding a '--maxdepth' option to `git annex find` would make it much easier to use at directories high up in the directory structure, since currently `git annex find` recurses all subdirectories necessarily, when I really just want to see whether or not there are git-annex files present from a given directory. - -Obviously, since directories themselves are not git-annex objects, there is no way to say whether or not they are "present", but perhaps the most intuitive would be to say whether or not any git-annex files under a given directory are present. - -For example, if I have: -./ -+-- subdir0/ -| +-- file0 (present in local git-annex repo) -| +-- file1 (present in local git-annex repo) -+-- subdir1/ -| +-- file0 (not present in local git-annex repo) -| +-- file1 (not present in local git-annex repo) -+-- file2 (present in local git-annex repo) - -and I type `git annex find --maxdepth 1 .`, the output might look something like: -subdir0/ -file2 - -rather than: -subdir0/file0 -subdir0/file1 -file2 - -> [[wontfix|done]] --[[Joey]] diff --git a/doc/todo/wishlist__58___--maxdepth_option_for_git_annex_find/comment_1_c355878ac49bbb23a4cf82fe685da9ee._comment b/doc/todo/wishlist__58___--maxdepth_option_for_git_annex_find/comment_1_c355878ac49bbb23a4cf82fe685da9ee._comment deleted file mode 100644 index 73d4cba5f8..0000000000 --- a/doc/todo/wishlist__58___--maxdepth_option_for_git_annex_find/comment_1_c355878ac49bbb23a4cf82fe685da9ee._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="http://joeyh.name/" - ip="108.236.230.124" - subject="comment 1" - date="2014-07-21T17:28:53Z" - content=""" -`find --maxdepth` is a nice optimisation because it can short-circuit when it gets deep in the tree. However, `git annex find` is built on top of `git ls-files --cached`, which has no equivilant way to short-circuit. I am not sure if the format of the index makes it practical for it to get a --maxdepth option (it may need to traverse the whole index, or might be able to short-circuit). - -I don't see any point in adding a --matdepth to git-annex if it doesn't actually make it any faster, so getting such a thing into `git ls-files` would be the first step. So, suggest filing a feature request on git. -"""]] diff --git a/doc/todo/wishlist__58___--maxdepth_option_for_git_annex_find/comment_2_da30a066c4de465fe172ad01057e2380._comment b/doc/todo/wishlist__58___--maxdepth_option_for_git_annex_find/comment_2_da30a066c4de465fe172ad01057e2380._comment deleted file mode 100644 index 944f36088e..0000000000 --- a/doc/todo/wishlist__58___--maxdepth_option_for_git_annex_find/comment_2_da30a066c4de465fe172ad01057e2380._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="https://www.google.com/accounts/o8/id?id=AItOawlScsufvQF7s8TVTwPd-h_QiP5Hn_i-hrs" - nickname="Jason" - subject="comment 2" - date="2014-07-21T18:37:19Z" - content=""" -I see your point, `git ls-files` may still have to walk the whole tree, precluding a speed advantage. -But I guess the point of what I was saying was more that a way summarize from a high level what is here and what is not would be nice. -I certainly understand if this is not something you see as worthwhile, but if someone were inclined to write a patch (if ever I find the time) that would add a `--maxdepth` option that would merely summarize the results of `git annex find` would it be something you would be inclined to include in the main repo (providing, of course, that you find the behavior sensible)? -"""]] diff --git a/doc/todo/wishlist__58___--maxdepth_option_for_git_annex_find/comment_3_f3eadd6241f5cc2886515b2826dc5cf9._comment b/doc/todo/wishlist__58___--maxdepth_option_for_git_annex_find/comment_3_f3eadd6241f5cc2886515b2826dc5cf9._comment deleted file mode 100644 index 8da436e09e..0000000000 --- a/doc/todo/wishlist__58___--maxdepth_option_for_git_annex_find/comment_3_f3eadd6241f5cc2886515b2826dc5cf9._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="http://joeyh.name/" - ip="108.236.230.124" - subject="comment 3" - date="2014-07-21T18:40:00Z" - content=""" -I think that --maxdepth has a well-defined meaning and this summary option would need to be named something else. - -I don't object to the idea of implementing it. However, I don't know that it would be very easy to implement either. -"""]] diff --git a/doc/todo/wishlist__58___--maxdepth_option_for_git_annex_find/comment_4_c766c1465407324fc933db78be325b33._comment b/doc/todo/wishlist__58___--maxdepth_option_for_git_annex_find/comment_4_c766c1465407324fc933db78be325b33._comment deleted file mode 100644 index 6025101e2e..0000000000 --- a/doc/todo/wishlist__58___--maxdepth_option_for_git_annex_find/comment_4_c766c1465407324fc933db78be325b33._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="https://www.google.com/accounts/o8/id?id=AItOawlScsufvQF7s8TVTwPd-h_QiP5Hn_i-hrs" - nickname="Jason" - subject="good point" - date="2014-07-21T19:22:30Z" - content=""" -You make another good point `--maxdepth` is vague in this context... -I guess if we were to decide to come up with a summary option, it will have be named something else, like `--summary-depth`, where the default would be to represent all files of whatever depth, and specifying the option would take the output that would otherwise get from `git annex find `, truncate the paths to a certain depth, and then make a set thereof (to remove the many dups), that way any directory that had any files that would have been output by `git annex find `, that would also be at or above a certain depth, would be listed. - -I think if I get a chance I'll try to implement something like this. - -"""]] diff --git a/doc/todo/wishlist__58___--maxdepth_option_for_git_annex_find/comment_5_dcda795ff29de150ced0c839c85f7837._comment b/doc/todo/wishlist__58___--maxdepth_option_for_git_annex_find/comment_5_dcda795ff29de150ced0c839c85f7837._comment deleted file mode 100644 index 20d2c3fd23..0000000000 --- a/doc/todo/wishlist__58___--maxdepth_option_for_git_annex_find/comment_5_dcda795ff29de150ced0c839c85f7837._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2023-01-16T18:03:26Z" - content=""" -Occurs to me that you can do this with existing options, eg to filter -out directories that are 3 or more deep: - - git-annex find --exclude='*/*/*' - -It won't display subdirectories that contain filtered out files of course. - -It would also be easy enough to write a wrapper around `git-annex find` -that processed its output and generated output like that. - -So I'm going to close this. -"""]] diff --git a/doc/todo/wishlist__58___Add_--byte-limit_option.mdwn b/doc/todo/wishlist__58___Add_--byte-limit_option.mdwn deleted file mode 100644 index 5c2c5fd50e..0000000000 --- a/doc/todo/wishlist__58___Add_--byte-limit_option.mdwn +++ /dev/null @@ -1,16 +0,0 @@ -This option is related to `--time-limit`, but stops after processing X -bytes. For example, I often need to free up a certain amount of space on -a disk and want `git-annex move` to stop after it has copied a specified -amount of data: - - git annex move --to otherdisk --byte-limit 3G - git annex drop --auto --byte-limit 500m - git annex get --byte-limit 500m - -I've been using some `annex.diskreserve` trickery now and then to -accomplish this, but it's a bit cumbersome and also not a very precise -way to specify the amount I want to copy/move/get. The last example -would also be a useful command to limit the traffic when I'm connecting -via mobile – get as much as possible, but don't blow the mobile quota. - -> [[done]] as --size-limit --[[Joey]] diff --git a/doc/todo/wishlist__58___Freeing_X_space_on_remote_Y.mdwn b/doc/todo/wishlist__58___Freeing_X_space_on_remote_Y.mdwn deleted file mode 100644 index b577c2ae69..0000000000 --- a/doc/todo/wishlist__58___Freeing_X_space_on_remote_Y.mdwn +++ /dev/null @@ -1,5 +0,0 @@ -As suggested during the first Gitify BoF during DebConf13: Adding a way to have on-demand dropping of content in a given remote would allow a user to quickly free up disk space on demand while still heeding numcopies etc. - -> [[done]] as --size-limit. This does affect dropping, so eg -> `git-annex drom --from foo --size-limit=10gb` frees up 10gb from remote -> foo. --[[Joey]] diff --git a/doc/todo/wishlist__58___Option_to_specify_max_transfer_rate.mdwn b/doc/todo/wishlist__58___Option_to_specify_max_transfer_rate.mdwn deleted file mode 100644 index 127e4832e5..0000000000 --- a/doc/todo/wishlist__58___Option_to_specify_max_transfer_rate.mdwn +++ /dev/null @@ -1,5 +0,0 @@ -A big part of my online use is done via a low-speed connection over my mobile phone, this is limited to 16KB/sec because I always use up my 500MB quota the very first day of the month. `;-/` So when I need to download big files, I first download them to my online server, then transfer the files to my laptop with git-annex. If I'm connected via GSM, this occupies all the bandwidth and everything else moves like a heavily sedated slug. So if I want to work via VNC or SSH, I have to terminate ongoing transfers with Ctrl-C and then hopefully remember to restart it when I work locally. I know git-annex is robust enough to handle this gracefully, but it would be really nice to have a continuous connection going on in the background, limited to a value I choose. - -rsync(1) has a `--bwlimit` (bandwidth limit) where you can specify max download/upload speed in kilobytes/sec. It would be great if a similar option was integrated into git-annex. Thanks in advance. - -> [[done]] generally as annex.bwlimit and related options. --[[Joey]] diff --git a/doc/todo/wishlist__58___Option_to_specify_max_transfer_rate/comment_1_4fd870e14b5b70c8a6ade41406294387._comment b/doc/todo/wishlist__58___Option_to_specify_max_transfer_rate/comment_1_4fd870e14b5b70c8a6ade41406294387._comment deleted file mode 100644 index 78ca76939a..0000000000 --- a/doc/todo/wishlist__58___Option_to_specify_max_transfer_rate/comment_1_4fd870e14b5b70c8a6ade41406294387._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="https://www.google.com/accounts/o8/id?id=AItOawmBUR4O9mofxVbpb8JV9mEbVfIYv670uJo" - nickname="Justin" - subject="trickle" - date="2013-01-22T22:44:52Z" - content=""" -not exactly integrated, but you can easily use trickle for this. - - trickle -d 50 git annex ... -"""]] diff --git a/doc/todo/wishlist__58___Option_to_specify_max_transfer_rate/comment_2_dd854f297ad6a94b54be9f3edfd0f766._comment b/doc/todo/wishlist__58___Option_to_specify_max_transfer_rate/comment_2_dd854f297ad6a94b54be9f3edfd0f766._comment deleted file mode 100644 index 70f04c6168..0000000000 --- a/doc/todo/wishlist__58___Option_to_specify_max_transfer_rate/comment_2_dd854f297ad6a94b54be9f3edfd0f766._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="http://sunny256.sunbase.org/" - nickname="sunny256" - subject="Yay, trickle works" - date="2013-01-23T01:36:21Z" - content=""" -Justin, thanks a lot! trickle(1) works great. I didn't know about this program, but I'm not surprised that such a program is available. It never ceases to amaze me what's possible in a *NIX environment. -"""]] diff --git a/doc/todo/wishlist__58___Option_to_specify_max_transfer_rate/comment_3_a8b7e90a473d5937807cc7eb456efe33._comment b/doc/todo/wishlist__58___Option_to_specify_max_transfer_rate/comment_3_a8b7e90a473d5937807cc7eb456efe33._comment deleted file mode 100644 index a5f8f6a1bf..0000000000 --- a/doc/todo/wishlist__58___Option_to_specify_max_transfer_rate/comment_3_a8b7e90a473d5937807cc7eb456efe33._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="http://joeyh.name/" - ip="2001:4978:f:21a::2" - subject="comment 3" - date="2013-01-24T01:55:10Z" - content=""" -In addition to trickle, the git-annex man page has examples of how to make rsync use --bwlimit - -Something like trickle is needed to limit rates for remotes not using rsync, however. -"""]] diff --git a/doc/todo/wishlist__58___Option_to_specify_max_transfer_rate/comment_4_53c43e480a37b20bc8fed9d38527651b._comment b/doc/todo/wishlist__58___Option_to_specify_max_transfer_rate/comment_4_53c43e480a37b20bc8fed9d38527651b._comment deleted file mode 100644 index 9b2cb64bf7..0000000000 --- a/doc/todo/wishlist__58___Option_to_specify_max_transfer_rate/comment_4_53c43e480a37b20bc8fed9d38527651b._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="gerzoyayde@85d55f6dec266753698f42d1c8e06917ef6674a3" - nickname="gerzoyayde" - avatar="http://cdn.libravatar.org/avatar/0866019bf4d6741b2f4a2c288356513f" - subject="annex.bwlimit and jobs" - date="2023-03-28T00:00:43Z" - content=""" -Does annex.bwlimit limit across multiple jobs? E.g. I have a annex.bwlimit of \"30MiB\" and run a copy with --jobs 3, will the maximum possible bandwidth be 30MB/s or 90MB/s? -"""]] diff --git a/doc/todo/wishlist__58___Option_to_specify_max_transfer_rate/comment_5_428d638e1f41d84df7e6c51675fe65da._comment b/doc/todo/wishlist__58___Option_to_specify_max_transfer_rate/comment_5_428d638e1f41d84df7e6c51675fe65da._comment deleted file mode 100644 index 232eda99e0..0000000000 --- a/doc/todo/wishlist__58___Option_to_specify_max_transfer_rate/comment_5_428d638e1f41d84df7e6c51675fe65da._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2023-03-28T16:10:43Z" - content=""" -annex.bwlimit is the limit for each transfer, it doesn't take into account -how many transfers might be running whether by --jobs or multiple git-annex -processes. -"""]] diff --git a/doc/todo/wishlist__58_____96__git_annex_optimize__96__.mdwn b/doc/todo/wishlist__58_____96__git_annex_optimize__96__.mdwn deleted file mode 100644 index 3f7fc0edfc..0000000000 --- a/doc/todo/wishlist__58_____96__git_annex_optimize__96__.mdwn +++ /dev/null @@ -1,12 +0,0 @@ -Given [this tip](https://git-annex.branchable.com/forum/__34__git_annex_sync__34___synced_after_8_hours/#comment-890ca1381d800ac833ccbb8c5db175ea), [this comment](https://git-annex.branchable.com/todo/wishlist:_rsync_efficiency/#comment-870ae805efd35343edefdbed792dac04) and possibly others, it would be nice if git-annex could look at any given repo and make suggestions for improvements. As a second step, it could look at remotes as well. And as a third, maybe even change repo settings and not just make suggestions. - -Having a few old repos with terabytes of data on various disks, I would just toss this stanza into my mr templates to eventually optimize all my repos to current best practices. - - --- Richard - -> One of those was fixed by improvements to git-annex, the other -> is not something that a command like the mooted one would be able to -> safely do since it has to happen at repository init time. -> -> [[wontfix|done]] --[[Joey]] diff --git a/doc/todo/wishlist__58___annex.largefiles_configuration_in_webapp_and_sync.mdwn b/doc/todo/wishlist__58___annex.largefiles_configuration_in_webapp_and_sync.mdwn deleted file mode 100644 index ce0a6161af..0000000000 --- a/doc/todo/wishlist__58___annex.largefiles_configuration_in_webapp_and_sync.mdwn +++ /dev/null @@ -1,4 +0,0 @@ -The `annex.largefiles` feature is very nice to mix annexed files with normal git managed files. I'd like to be able to configure this setting on the webapp and that the configuration directive would be synchronized accross all remotes. - -> annex.largefiles can be configured in .gitattributes or by `git-annex -> config` and will sync across remotes either way. [[done]] diff --git a/doc/todo/wishlist__58___annex.largefiles_configuration_in_webapp_and_sync/comment_1_db632de391ce9fce42af51a770ed3aeb._comment b/doc/todo/wishlist__58___annex.largefiles_configuration_in_webapp_and_sync/comment_1_db632de391ce9fce42af51a770ed3aeb._comment deleted file mode 100644 index e402d26c3b..0000000000 --- a/doc/todo/wishlist__58___annex.largefiles_configuration_in_webapp_and_sync/comment_1_db632de391ce9fce42af51a770ed3aeb._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="http://joeyh.name/" - ip="209.250.56.47" - subject="comment 1" - date="2013-11-05T16:46:28Z" - content=""" -It might make sense to sync this across remotes and have it edited with `git annex vicfg` - -Putting it in the webapp would need some nice interface for constructing the underlying expression. Might be doable for at least simple filtering (ie, files larger than XX or with extensions .A, .B, .C). I tend to think of this as a setting for people comfortable with the command line though. -"""]] diff --git a/doc/todo/wishlist__58___annex.largefiles_configuration_in_webapp_and_sync/comment_2_4a0931d9884054d764fde315d4fe4851._comment b/doc/todo/wishlist__58___annex.largefiles_configuration_in_webapp_and_sync/comment_2_4a0931d9884054d764fde315d4fe4851._comment deleted file mode 100644 index 0e24014d21..0000000000 --- a/doc/todo/wishlist__58___annex.largefiles_configuration_in_webapp_and_sync/comment_2_4a0931d9884054d764fde315d4fe4851._comment +++ /dev/null @@ -1,14 +0,0 @@ -[[!comment format=mdwn - username="https://www.google.com/accounts/o8/id?id=AItOawne_amN4fko4p5cRY_9EYwaYuJKNn7LRio" - nickname="Tobias" - subject="feedback" - date="2013-11-05T21:23:11Z" - content=""" -> It might make sense to sync this across remotes and have it edited with git annex vicfg - -That would be great! - -> Putting it in the webapp would need some nice interface for constructing the underlying expression. Might be doable for at least simple filtering (ie, files larger than XX or with extensions .A, .B, .C). I tend to think of this as a setting for people comfortable with the command line though. - -I could live with a simple filtering interface without too many fancy stuff. The fancy stuff could be done on the command line if needed... -"""]] diff --git a/doc/todo/wishlist__58___provide_a_config_option_for_using_new_hashing_scheme_in_non-bare_remotes.mdwn b/doc/todo/wishlist__58___provide_a_config_option_for_using_new_hashing_scheme_in_non-bare_remotes.mdwn deleted file mode 100644 index d987d03e15..0000000000 --- a/doc/todo/wishlist__58___provide_a_config_option_for_using_new_hashing_scheme_in_non-bare_remotes.mdwn +++ /dev/null @@ -1,8 +0,0 @@ -I understand that for backwards compatibility the non-bare remotes use the old "mixed" case scheme. However, for new annexes, it'd make sense to be able to use the new one so the scheme matches in all remotes. - -> If this option existed then every clone of a repository would need to set -> it, or files would be hashed into the wrong location and would appear not -> visible. Sounds like a bug magnet to me; not attractive. --[[Joey]] - -> > Actually, `git-anex init -c annex.tune.objecthashlower=true` -> > does just this. So, [[done]] --[[Joey]] diff --git a/doc/todo/wishlist__58___recursive_directory_remote_setup__47__addurl.mdwn b/doc/todo/wishlist__58___recursive_directory_remote_setup__47__addurl.mdwn deleted file mode 100644 index 9c9d7ce75d..0000000000 --- a/doc/todo/wishlist__58___recursive_directory_remote_setup__47__addurl.mdwn +++ /dev/null @@ -1,10 +0,0 @@ -I think it would be interesting to have a way to recursively import a local directory without actually moving files around. And to be able to checksum these files as well (without moving them into the annex). - -This would work somewhat similar to looping over a directory and adding file:// remotes for each file. - -A use case is importing optical media (read-only), whilst keeping that media as a remote, and being able to calculate checksums directly without moving any files around. - -For single files, it would also be interesting if addurl had a "--localchecksum" option that would only work for file:// urls, and make it checksum files directly from their source location?) - -> A directory special remote with importtree=yes and `git-annex import`--no-content` -> can be used to do this. [[done]] --[[Joey]] diff --git a/doc/todo/wishlist__58___recursive_directory_remote_setup__47__addurl/comment_1_b79976afc2242791523e63831f30af71._comment b/doc/todo/wishlist__58___recursive_directory_remote_setup__47__addurl/comment_1_b79976afc2242791523e63831f30af71._comment deleted file mode 100644 index caefee9a82..0000000000 --- a/doc/todo/wishlist__58___recursive_directory_remote_setup__47__addurl/comment_1_b79976afc2242791523e63831f30af71._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="https://launchpad.net/~arand" - nickname="arand" - subject="comment 1" - date="2013-03-10T23:29:55Z" - content=""" -Recursively adding urls is already feasiable with some simple scripting: - -[https://gitorious.org/arand-scripts/arand-scripts/blobs/master/annex-importdir](https://gitorious.org/arand-scripts/arand-scripts/blobs/master/annex-importdir) - -but this is obviously missing the checksumming bit though. -"""]] diff --git a/doc/todo/wishlist__58___recursive_directory_remote_setup__47__addurl/comment_2_1741d2392006a9af9cfd1f3b847600b9._comment b/doc/todo/wishlist__58___recursive_directory_remote_setup__47__addurl/comment_2_1741d2392006a9af9cfd1f3b847600b9._comment deleted file mode 100644 index 64d5924795..0000000000 --- a/doc/todo/wishlist__58___recursive_directory_remote_setup__47__addurl/comment_2_1741d2392006a9af9cfd1f3b847600b9._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="http://joeyh.name/" - nickname="joey" - subject="comment 2" - date="2013-03-11T03:04:44Z" - content=""" -See also: [[forum/Managing a large number of files archived on many pieces of read-only medium (E.G. DVDs)]] -particularly [[this comment|forum/Managing a large number of files archived on many pieces of read-only medium (E.G. DVDs)#comment-908dbe02f29e011f030bba4ab5ef73d1]] -"""]] diff --git a/doc/todo/wishlist__58___recursive_directory_remote_setup__47__addurl/comment_3_b6fb39030f98bbc9915712e3d35d1838._comment b/doc/todo/wishlist__58___recursive_directory_remote_setup__47__addurl/comment_3_b6fb39030f98bbc9915712e3d35d1838._comment deleted file mode 100644 index fd5dc9e051..0000000000 --- a/doc/todo/wishlist__58___recursive_directory_remote_setup__47__addurl/comment_3_b6fb39030f98bbc9915712e3d35d1838._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="0xloem@0bd8a79a57e4f0dcade8fc81d162c37eae4d6730" - nickname="0xloem" - subject="Assistant support" - date="2016-04-16T11:52:14Z" - content=""" -This would be really augmented if the assistant and webapp could watch external folders and automatically add new files that appeared. Then e.g. your system logs and family photos could be archived together without maintenance needed. -I'm running into this on android, wherw I'd like to store files my phone produces alongside my other files. -"""]]