diff --git a/doc/bugs/A_case_where_file_tracked_by_git_unexpectedly_becomes_annex_pointer_file.mdwn b/doc/bugs/A_case_where_file_tracked_by_git_unexpectedly_becomes_annex_pointer_file.mdwn new file mode 100644 index 0000000000..03a7ba579c --- /dev/null +++ b/doc/bugs/A_case_where_file_tracked_by_git_unexpectedly_becomes_annex_pointer_file.mdwn @@ -0,0 +1,76 @@ +In datalad test builds with git-annex 7.20191114+git43-ge29663773, one +of the new test failures is due to an unexpectedly dirty repository +([related datalad issue][0]). The dirty status comes from a file that +was tracked in Git switching over to an annex pointer file. Here's a +script that distills enough of the test to trigger the failure on my +end. + +[[!format sh """ +#!/bin/sh + +set -eu + +assert_clean () { + if test -n "$(git status --porcelain)" + then + printf "\n\nUnexpectedly dirty:\n" >&2 + git status >&2 + git diff >&2 + exit 1 + fi +} + +cd "$(mktemp -d --tmpdir gx-pointer-dirty-XXXXXXX)" +git init && git annex init + +printf content-git >file-git +git -c annex.largefiles=nothing annex add -- file-git +git commit -m'file-git added' +assert_clean + +printf content-annex >file-annex +git -c annex.largefiles=anything annex add -- file-annex +git commit -m'file-annex annexed' +assert_clean +"""]] + +On Travis as well as my local machine, the failure is intermittent, +but seems to happen much more often than not. In the failing case, +the last assert_clean call shows: + +``` +Unexpectedly dirty: +On branch master +Changes not staged for commit: + modified: file-git + +no changes added to commit +diff --git a/file-git b/file-git +index d1c416a..b41ca32 100644 +--- a/file-git ++++ b/file-git +@@ -1 +1 @@ +-content-git +\ No newline at end of file ++/annex/objects/SHA256E-s11--726732d25826965592478fcc7c145d5a10fa1aa70c49fe3a4f847174b6d8889c +``` + +I see the failure with git-annex built from the latest master +b962471c2 (2019-12-12). Bisecting against the git-annex repo (with a +commit being marked "bad" if there was a failure within ten runs of the +above script), points to ec08b66bd (shouldAnnex: check isInodeKnown, +2019-10-23) as the first bad commit. Just looking at the topic of +the commit, that result seems plausible to me. + +### Other details + +My git version 2.24.1 and locally I'm building git-annex through guix. +On the failing Travis run, git-annex 7.20191114+git43-ge29663773 came +from neurodebian, and the git version was 2.24.0. + +Hopefully the script above is sufficient to trigger the issue on your end. +Thanks for having a look. + +[0]: https://github.com/datalad/datalad/issues/3890 + +[[!meta author=kyle]] diff --git a/doc/bugs/addurl_fails_because_special_remote_is_not_available/comment_3_b0e9f373c3439b15a28fd20e9b0e24a6._comment b/doc/bugs/addurl_fails_because_special_remote_is_not_available/comment_3_b0e9f373c3439b15a28fd20e9b0e24a6._comment new file mode 100644 index 0000000000..1c699cc1aa --- /dev/null +++ b/doc/bugs/addurl_fails_because_special_remote_is_not_available/comment_3_b0e9f373c3439b15a28fd20e9b0e24a6._comment @@ -0,0 +1,51 @@ +[[!comment format=mdwn + username="lykos@d125a37d89b1cfac20829f12911656c40cb70018" + nickname="lykos" + avatar="http://cdn.libravatar.org/avatar/085df7b04d3408ba23c19f9c49be9ea2" + subject="comment 3" + date="2019-12-12T21:11:58Z" + content=""" +Apparently it happens when the remote returns PREPARE-FAILURE (in this case due to insufficient internet connection). So we get back to +[PREPARE-LOCAL](http://git-annex.branchable.com/design/external_special_remote_protocol/#comment-2eec51c9f774f577b8634e9cdc86cde3) and [external_remote_querying_transition](https://git-annex.branchable.com/todo/external_remote_querying_transition). + +I'm going to change git-annex-remote-googledrive to defer network connection to when when it's needed. But as external remotes are encouraged to establish network connections in PREPARE, something should be done on git-annex's side, too. + + + % git annex addurl --batch --with-files --debug + [url] [filename] + [2019-12-12 21:41:14.540686936] read: git [\"--git-dir=../../../.git\",\"--work-tree=../../..\",\"--literal-pathspecs\",\"show-ref\",\"git-annex\"] + [2019-12-12 21:41:14.546632041] process done ExitSuccess + [2019-12-12 21:41:14.546795605] read: git [\"--git-dir=../../../.git\",\"--work-tree=../../..\",\"--literal-pathspecs\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"] + [2019-12-12 21:41:14.551391222] process done ExitSuccess + [2019-12-12 21:41:14.551593051] read: git [\"--git-dir=../../../.git\",\"--work-tree=../../..\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..802429a871b754d61047457aa9936bde404e3172\",\"--pretty=%H\",\"-n1\"] + [2019-12-12 21:41:14.553261192] process done ExitSuccess + [2019-12-12 21:41:14.553318623] read: git [\"--git-dir=../../../.git\",\"--work-tree=../../..\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..0e8f6eb6cc45d9bb11f42fcf8590b3473e6f2623\",\"--pretty=%H\",\"-n1\"] + [2019-12-12 21:41:14.555647103] process done ExitSuccess + [2019-12-12 21:41:14.555710805] read: git [\"--git-dir=../../../.git\",\"--work-tree=../../..\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..7271aaaa110b0f84b5e730673c66b176789ddcaf\",\"--pretty=%H\",\"-n1\"] + [2019-12-12 21:41:14.557235619] process done ExitSuccess + [2019-12-12 21:41:14.557298991] read: git [\"--git-dir=../../../.git\",\"--work-tree=../../..\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..b9112bcb862041cf96f00e12c6dd33dc31fda5fa\",\"--pretty=%H\",\"-n1\"] + [2019-12-12 21:41:14.560071044] process done ExitSuccess + [2019-12-12 21:41:14.560126818] read: git [\"--git-dir=../../../.git\",\"--work-tree=../../..\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..1598b00a78067edc438652f401498a25635cd5a9\",\"--pretty=%H\",\"-n1\"] + [2019-12-12 21:41:14.562966332] process done ExitSuccess + [2019-12-12 21:41:14.563218447] chat: git [\"--git-dir=../../../.git\",\"--work-tree=../../..\",\"--literal-pathspecs\",\"cat-file\",\"--batch\"] + [2019-12-12 21:41:14.563535554] chat: git [\"--git-dir=../../../.git\",\"--work-tree=../../..\",\"--literal-pathspecs\",\"cat-file\",\"--batch-check=%(objectname) %(objecttype) %(objectsize)\"] + [2019-12-12 21:41:14.56562967] read: git [\"config\",\"--null\",\"--list\"] + [2019-12-12 21:41:14.566788099] process done ExitSuccess + [2019-12-12 21:41:14.567203332] read: git [\"config\",\"--null\",\"--list\"] + [2019-12-12 21:41:14.568286923] read: git [\"config\",\"--null\",\"--list\"] + [2019-12-12 21:41:14.569077497] chat: /home/silvio/.local/bin/git-annex-remote-googledrive [] + [2019-12-12 21:41:14.944193013] git-annex-remote-googledrive[1] --> VERSION 1 + [2019-12-12 21:41:14.944312387] git-annex-remote-googledrive[1] <-- EXTENSIONS INFO + [2019-12-12 21:41:14.944486597] git-annex-remote-googledrive[1] --> EXTENSIONS + [2019-12-12 21:41:14.944557495] git-annex-remote-googledrive[1] <-- EXPORTSUPPORTED + [2019-12-12 21:41:14.94467435] git-annex-remote-googledrive[1] --> EXPORTSUPPORTED-SUCCESS + [2019-12-12 21:41:14.944906493] chat: /home/silvio/.local/bin/git-annex-remote-googledrive [] + [2019-12-12 21:41:15.333971094] git-annex-remote-googledrive[1] --> VERSION 1 + [2019-12-12 21:41:15.334086558] git-annex-remote-googledrive[1] <-- EXTENSIONS INFO + [2019-12-12 21:41:15.334273359] git-annex-remote-googledrive[1] --> EXTENSIONS + [2019-12-12 21:41:15.334334558] git-annex-remote-googledrive[1] <-- PREPARE + [...] + [2019-12-12 21:41:43.372727829] git-annex-remote-googledrive[1] --> PREPARE-FAILURE ('Failed to connect with Google. Please check your internet connection.', ServerNotFoundError('Unable to find the server at www.googleapis.com')) + git-annex: ('Failed to connect with Google. Please check your internet connection.', ServerNotFoundError('Unable to find the server at www.googleapis.com')) + +"""]] diff --git a/doc/install/rpm_standalone/comment_1_7d234e64f3b6e97cd6d91c339ebb0025._comment b/doc/install/rpm_standalone/comment_1_7d234e64f3b6e97cd6d91c339ebb0025._comment new file mode 100644 index 0000000000..19722c779f --- /dev/null +++ b/doc/install/rpm_standalone/comment_1_7d234e64f3b6e97cd6d91c339ebb0025._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="mike@4e7a118bd37129091199ae0fb28184999c5f4725" + nickname="mike" + avatar="http://cdn.libravatar.org/avatar/9751260f8a13fc27ad837eaf66fac5d7" + subject="repodata is outdated" + date="2019-12-18T10:06:08Z" + content=""" +The repodata under https://downloads.kitenet.net/git-annex/linux/current/rpms/repodata/ is outdated (from September), latest RPM in https://downloads.kitenet.net/git-annex/linux/current/rpms/ is from November. I (think) I had an issue using git-annex-shell using the 201909 version, so this is somewhat relevant. +"""]] diff --git a/doc/todo/git-annex-cat/comment_1_a5da8264c2d662e96802535820cb2249._comment b/doc/todo/git-annex-cat/comment_1_a5da8264c2d662e96802535820cb2249._comment new file mode 100644 index 0000000000..03d33376d6 --- /dev/null +++ b/doc/todo/git-annex-cat/comment_1_a5da8264c2d662e96802535820cb2249._comment @@ -0,0 +1,15 @@ +[[!comment format=mdwn + username="https://christian.amsuess.com/chrysn" + nickname="chrysn" + avatar="http://christian.amsuess.com/avatar/c6c0d57d63ac88f3541522c4b21198c3c7169a665a2f2d733b4f78670322ffdc" + subject="Would be useful" + date="2019-12-17T09:08:08Z" + content=""" +A `git annex cat` would be useful for the very web server purpose you describe (WIP at https://gitlab.com/chrysn/annex-to-web, though I'm not sure it's going anywhere). + +Unlike `git annex inprogress` that I (will) use for a workaround, this could take a `--skip` argument that usually just seeks into the file. +If the data is served from a remote that allows seeking access (eg. IPFS), +then that access could be priorized and that part downloaded first. +(Implementing this would require another tmp pool for sparse files as they couldn't go with the `git annex inprogress` files for there is the expectation that those would grow to completion, +but anyway this would be an entry point for such a feature if it is ever added). +"""]] diff --git a/doc/todo/git-annex-inprogress_--key.mdwn b/doc/todo/git-annex-inprogress_--key.mdwn new file mode 100644 index 0000000000..f96d381e64 --- /dev/null +++ b/doc/todo/git-annex-inprogress_--key.mdwn @@ -0,0 +1,8 @@ +Unlike `whereis` and other subcommands, `inprogress` does not offer a `--key` argument to select files by key rather than checked-out name, +making it unusable in bare repositories. + +Please consider adding a `--key` option there, which would display the single incomplete file corresponding to the key if one is in progress. + +My use case is serving git-annexed files to the web from a bare repository (, see also [[todo/git-annex-cat]]), which would be especially useful with gitolite repositories as they are by design bare, and on devices where checkouts are cumbersome (cf. [[forum/Dealing_with_crippled_Android_file_system]]). + +A workaround is running `git annex inprogress --all | grep $KEY`, but that's probably relying on an implementation detail that could be changed at any time (though it probably won't as to avoid race conditions as in `tail -f $(git annex inprogress file-thats-almost.done)`). diff --git a/doc/todo/making_it_easier_to_smudge_dotfiles.mdwn b/doc/todo/making_it_easier_to_smudge_dotfiles.mdwn new file mode 100644 index 0000000000..0e392c48ac --- /dev/null +++ b/doc/todo/making_it_easier_to_smudge_dotfiles.mdwn @@ -0,0 +1,5 @@ +I want to add some dotfiles in the root of my repository to git-annex as unlocked annexed files. So I edited `.git/info/attributes` to remove the line `.* !filter`, such that it only contains the line `* filter=annex`. This seems to be working fine. + +I was thinking that it might make sense to have a `git annex config` option to tell git-annex not to add the `.* !filter` line to `.git/info/attributes` when initialising other clones of this repo. In the meantime, I've worked around it using a `post_checkout` hook in my `~/.mrconfig` which edits `.git/info/attributes`. + +--spwhitton