From c85165ece527614e4db2eb1ad7797c12b7333fcc Mon Sep 17 00:00:00 2001 From: "amindfv@6f75355c5dad3450ed73d1f01715be90dfdd6cd6" Date: Sat, 12 Oct 2019 21:12:52 +0000 Subject: [PATCH 01/12] Added a comment --- ..._d7dd41721dd86ede3746e60e3b7aa2b2._comment | 141 ++++++++++++++++++ 1 file changed, 141 insertions(+) create mode 100644 doc/forum/lets_discuss_git_add_behavior/comment_11_d7dd41721dd86ede3746e60e3b7aa2b2._comment diff --git a/doc/forum/lets_discuss_git_add_behavior/comment_11_d7dd41721dd86ede3746e60e3b7aa2b2._comment b/doc/forum/lets_discuss_git_add_behavior/comment_11_d7dd41721dd86ede3746e60e3b7aa2b2._comment new file mode 100644 index 0000000000..71fac9a60f --- /dev/null +++ b/doc/forum/lets_discuss_git_add_behavior/comment_11_d7dd41721dd86ede3746e60e3b7aa2b2._comment @@ -0,0 +1,141 @@ +[[!comment format=mdwn + username="amindfv@6f75355c5dad3450ed73d1f01715be90dfdd6cd6" + nickname="amindfv" + avatar="http://cdn.libravatar.org/avatar/9cdda587f634ea9a85b34b25be421676" + subject="comment 11" + date="2019-10-12T21:12:51Z" + content=""" +First off, I love git annex and I truly appreciate all the hard work that's gone into it, so I hope you'll take my frustration as constructive when I say: + +Making \"git add\" mean \"git annex add\" is a **terrible** default and it should be reverted **ASAP**. + +## v7 is an entirely different tool than v5 + + mkdir foo && cd foo + git init && git annex init + echo 'one' > a.txt + git add a.txt + git commit -m '+' + echo 'two' >> a.txt + git diff + +I don't get a diff!! What??! Except for after \"git annex init\", git-annex has kept completely quiet (not warning me about any of this), and yet it's hijacked the whole repo. +\"git show\" is also broken, \"git add -p\", \"git log -p\", etc etc. + +As others have mentioned, \"git clone\" may cause people to lose their data as well. + +In other words, \"git annex\" is no longer a couple of additional commands to use within git - it's something closer to a replacement of git. It feels like a takeover of my git repos. + +## It forces a workflow on users + +One of the beautiful things about git-annex is it adds a few simple concepts to git, and allows us to use those new primitives in any way we like. This has allowed users to invent lots of different workflows that meet their needs. + +I've seen lots of different types of repo configurations and workflows, but for discussion here I think we only need to talk about two: + + 1. \"Big bunch-o-binaries\" (BBOB): user wants to keep their photo collection/scientific data/etc. in a big repo and they've got some way (like the assistant) to sync it. This could be considered to be a \"dropbox replacement\". In a BBOB nobody ever wants to look at a diff between versions of a file, do line-by-line merges, or use most of the other features of git. + + 2. All uses that aren't that one! But let's be specific and describe a couple: + * I've got a repo (dozens, actually) which is just code, but somewhere along the line I had a large data file I wanted to add and didn't want to slow git down. I \"git annex init && git annex add\"ed and was done with it. Back to writing code. + * I've got a repo which is a true mixture of large binaries and small text files. E.g. for a video editing project I've got raw video files as well as various configuration scripts, notes, the .kdenlive file, etc. + +This change puts a giant wedge between use case #1 and #2. + +As an example, the org-mode people suggest using git-annex: . I can't imagine they'd be happy to accidentally lose the ability to get diffs between versions of their .org files. + +## We don't know how big of a breaking change this is... + +...and it may be very large. + +How many people use \"git add\" to mean \"git add\" in at least one of their annex repos? We don't know! Compatibility with a big breaking change like this wasn't asked in the last user survey (which I happily took part in): + +But we can look at what _was_ asked to try and estimate the extent of the breakage: + +As a proxy, we could examine how many people use the assistant (more likely to simply have a big-bunch-o-binaries), vs. number of people who use it on the command line (more likely to be carefully managing their repos, including which files are in \"vanilla\" git vs annex). The numbers in the survey indicate there are 14 people using it on the command line for every 1 person using the assistant (85% to 6%). + +We can also look at how many people were using any v7 features: in the most recent survey 75% of people say they're not using any v7 features, and another 7% say they don't know, which I read as not following this discussion very closely. This suggests to me most people (at least 82%) were happy using git annex basically as it was. + +That idea (\"we're basically happy how it is already\") is borne out by other questions: 83% of users rate themselves between \"happy\" and \"one of my favorite applications of all time\" (FWIW, I fall into the \"one of my favorie applications of all time\" camp!) + +In addition, the \"blocking problems,\" \"focus,\" and \"roadmap\" sections of the survey don't provide compelling evidence that changing fundamentally how git-annex interacts with git is something anyone's clamoring for. + +85% of people mostly use `git annex` from the command line. How many of those people have used \"git add\" in an annex repo at least once, and (now incorrectly) believe they know what it's going to do to their repo? + +## More broken workflows + +Even repos which mainly *are* BBOB (big-bunch-o-binaries) may have a README file or other files like them. I note that most (all?) repos here have text files that are in \"plain git\": + +Now when I \"apt upgrade\" and add a new source file to one of my code repos, it's going to be `git-annex-add`ed? Does that mean I can't (easily, sensibly) push it to a code hosting platform (GitLab, Github)? + +Forcing people to this behavior by default reifies one workflow (BBOB) over all others, and basically replaces one tool (basically a few added primitives for git) with another that's also called git annex (basically a replacement for content tracking in git - and isn't that basically a replacement for git?). + +I realize every user's going to have slightly different preferences, but I truly think this is the rare case where simply saying \"if you don't like it, set these flags in your repos' configs\" is not nearly good enough. +I realize it would be a pain to roll this change back, but the benefits still far outweigh the costs. It's going to be a much bigger pain point for all users who are suddenly confused and have put their history in a broken state, for all the tutorials that are now giving users inaccurate information, etc. + +## Worse experience for new/inexperienced users + +Anecdotally, I've turned 5-10 people on to the beauty of git annex, and in every case the reason I told them about it was they mentioned to me they needed some way to store a large data file in their existing code repository. + +Now when I tell people about git annex, I have to also tell them to be super careful to set up a set of configs in orer for it not to fundamentally redefine the meaning of their repo? + +The common new-user experience for me has been: + +> Friend (from across the room): \"Dang, this file's too big for git\" + +> Me: \"Have you tried git annex?\" + +> [...talking about the benefits, installation, etc...] + +> Friend: \"Ok now how do I use it?\" + +> Me (still across the room): \"Just 'git annex init' in the repo and then 'git annex add' the file\" + +It seems almost comical that I'd memorize a line so I could instead say: + +> Me: \"Just 'git annex init' in the repo and then 'git annex add' the file. But first be SUPER careful to type 'git config annex.largefiles 'largerthan=100kb and not (include=*.c or include=*.h)' and don't forget to keep that list of file extensions up to date\" + +\"git annex init\" should be set-it-and-forget-it. I shouldn't have to worry about what parts of my git repo it's messing with because I'm not being vigilant enough. + +## Should have been discussed and then announced far more widely + +I go to the git annex homepage every couple of weeks, which I imagine is on the high end for a user of a command line tool. Even I was caught completely by surprise with this change, and only saw it when I \"apt-upgrade\"d. + + +## (Subjective:) Worse even if it didn't break (most?) users' expectations + +IMO, even if this were a new tool with no existing users or workflows or repos, this would not be the best default and instead should be behind a flag. I know I'd have been less enthusiastic to start using it if it were nudging me to basically use it as a replacement for Dropbox, instead of an unobtrusive set of additional tools for git. + +I'd also be less enthusiastic to know that if I weren't vigilant I'd get totally wrong behavior (e.g. I say 'git config annex.largefiles 'largerthan=100kb and not (include=*.c or include=*.h)' but then I \"git add\" a .hsc file and it \"git-annex-add\"s it \"behind my back\") + +## In summary: + +Issues as I see it are (and there may be more): + + * It breaks users' workflows. Potentially a huge number of them. This alone should be a big reason to be careful when making a change like this. + I - one single user - have already spent hours assessing how this will affect me and my repos. It requires me to be very careful and I'm far from sure I won't slip up somewhere. Hopefully if/when I do, I'll notice the mistake. + I had other things planned for my Saturday, but some of them involved using \"git annex\" and so now I have to halt everything to make sure I'm not screwing anything up now that I \"apt upgrade\"d and got a new version of git-annex this morning. + * Uncomfortable new-user experience + * (Subjective:) not a good default even if didn't break expectations + * Should have been announced and discussed MUCH more widely and extensively + + +## A response to a few issues already raised: + +> \"Suppose you have an unlocked file in your repo, and you rename it (not using git move), and then git add it. Oops, now you've added to git a large file that you wanted to be annexed. \" + +Why not simply provide a configurable warning about attempting to \"git add\" a file above a certain size? A \"did you mean 'git annex add'?\" type warning would be helpful for everyone. It'd catch all mistakes, not just ones caused by mv. + +> \"But mv foo bar; git add bar is normally identical to git mv foo bar. Why should using git-annex break that identity?\" + +This to me feels like killing a mosquito with a sledgehammer. This change breaks myriad other identities, including a very simple one: + - \"git add\"ing a new file should behave the same whether or not we've at some point typed \"git annex init\" + +The \"mv ; add\" identity has never been an issue for me in years of using git annex. By contrast, this change has already eaten up half of my Saturday. (Admittedly some of it writing this up. And again I should mention: I still hugely appreciate all the work on git-annex!) + +## What to do about it? + +I'm 100% behind Dwk's 4-point plan. +My one clarification is potentially to take \"only if largefiles is set\" to mean \"only if largefiles is set to a value other than 'nothing'.\" Not sure about this one. + + + +"""]] From 8348e7534da4c20b8930151c51103adf7b36f9e4 Mon Sep 17 00:00:00 2001 From: Ilya_Shlyakhter Date: Sat, 12 Oct 2019 21:39:03 +0000 Subject: [PATCH 02/12] Added a comment: experience with v7 semantics --- .../comment_12_ea51470a483da0ca38156d68df30184b._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/forum/lets_discuss_git_add_behavior/comment_12_ea51470a483da0ca38156d68df30184b._comment diff --git a/doc/forum/lets_discuss_git_add_behavior/comment_12_ea51470a483da0ca38156d68df30184b._comment b/doc/forum/lets_discuss_git_add_behavior/comment_12_ea51470a483da0ca38156d68df30184b._comment new file mode 100644 index 0000000000..a2c9aec76a --- /dev/null +++ b/doc/forum/lets_discuss_git_add_behavior/comment_12_ea51470a483da0ca38156d68df30184b._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="Ilya_Shlyakhter" + avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" + subject="experience with v7 semantics" + date="2019-10-12T21:39:03Z" + content=""" +To add to the anecdotal reports of user experience, I find myself periodically running `git annex lock` just to make sure nothing got inadvertently unlocked, or added as unlocked. The main benefit of using `git-annex` to version data analysis results, besides avoiding git's choking on large files or breaking github size limits, is the certainty that the result files are exactly as originally output, and haven't been accidentally changed (e.g. by re-running the analysis with different parameters but same output file name). With locked files, I have that guarantee -- once added and committed, the files won't change unless explicitly unlocked. With unlocked files, there's a chance of changing the file and then not noticing the change and committing it. The git log will of course reflect the change, but I might miss it, unless I inspect the log. So it's important to have a foolproof way to prevent files from being added as unlocked, and that's hard to do with the current `git-annex` version. I can set `annex.largefiles=nothing` at the repo root, but then `git annex add` won't annex anything, either. One solution is [[todo/separate_annex.largefiles.git-add_and_annex.largefiles.git-annex-add_settings]]; there may be others. +"""]] From 3b86bfdb79151cda24def98cdef551df93d4e577 Mon Sep 17 00:00:00 2001 From: Lukey Date: Sat, 12 Oct 2019 21:48:57 +0000 Subject: [PATCH 03/12] Added a comment: v5 mode --- .../comment_13_81373a9181bcf7c80f06936e62742678._comment | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 doc/forum/lets_discuss_git_add_behavior/comment_13_81373a9181bcf7c80f06936e62742678._comment diff --git a/doc/forum/lets_discuss_git_add_behavior/comment_13_81373a9181bcf7c80f06936e62742678._comment b/doc/forum/lets_discuss_git_add_behavior/comment_13_81373a9181bcf7c80f06936e62742678._comment new file mode 100644 index 0000000000..73706daa35 --- /dev/null +++ b/doc/forum/lets_discuss_git_add_behavior/comment_13_81373a9181bcf7c80f06936e62742678._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="Lukey" + avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b" + subject="v5 mode" + date="2019-10-12T21:48:56Z" + content=""" +Maybe something like \"git annex v5mode true\" is needed so we get v5 semantics in v7 mode. + +"""]] From 302e43f8527b61c58574b4dfe081e30712d1ab61 Mon Sep 17 00:00:00 2001 From: TroisSinges Date: Sun, 13 Oct 2019 23:55:20 +0000 Subject: [PATCH 04/12] --- ..._copies__58___unsafe_to_drop_from_another_repo__63__.mdwn | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 doc/forum/S3_copies__58___unsafe_to_drop_from_another_repo__63__.mdwn diff --git a/doc/forum/S3_copies__58___unsafe_to_drop_from_another_repo__63__.mdwn b/doc/forum/S3_copies__58___unsafe_to_drop_from_another_repo__63__.mdwn new file mode 100644 index 0000000000..11ca875974 --- /dev/null +++ b/doc/forum/S3_copies__58___unsafe_to_drop_from_another_repo__63__.mdwn @@ -0,0 +1,5 @@ +Hi! + +A file is present in my local repository (v7), and in a special encrypted repository using S3. Numcopies is set to 1. Whenever I try to drop the file, git-annex says it's "Unable to lock down 1 copy of file that is required to safely drop it" after having been "checking repository-s3... (unsafe)". + +What "unsafe" means? When I fsck the file in repository-s3, everything's fine and the file is present. From bc4a194c6dba4116c1b60c16903d1c326645ee12 Mon Sep 17 00:00:00 2001 From: TroisSinges Date: Sun, 13 Oct 2019 23:59:23 +0000 Subject: [PATCH 05/12] Added a comment --- .../comment_1_536a526a78a775142ad66c0689411c6a._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/forum/S3_copies__58___unsafe_to_drop_from_another_repo__63__/comment_1_536a526a78a775142ad66c0689411c6a._comment diff --git a/doc/forum/S3_copies__58___unsafe_to_drop_from_another_repo__63__/comment_1_536a526a78a775142ad66c0689411c6a._comment b/doc/forum/S3_copies__58___unsafe_to_drop_from_another_repo__63__/comment_1_536a526a78a775142ad66c0689411c6a._comment new file mode 100644 index 0000000000..a5841f99de --- /dev/null +++ b/doc/forum/S3_copies__58___unsafe_to_drop_from_another_repo__63__/comment_1_536a526a78a775142ad66c0689411c6a._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="TroisSinges" + avatar="http://cdn.libravatar.org/avatar/f4c62f9ebe09016b8852b04f1dd6612f" + subject="comment 1" + date="2019-10-13T23:59:23Z" + content=""" +git-annex version is 7.20190912-gab739242a3 (on Linux x86_64) +"""]] From 7a950e3088820e1be33c251dd473a2d152d6831d Mon Sep 17 00:00:00 2001 From: "amindfv@6f75355c5dad3450ed73d1f01715be90dfdd6cd6" Date: Mon, 14 Oct 2019 00:20:25 +0000 Subject: [PATCH 06/12] Added a comment --- ..._7966d8b5346de2718feccb2d246baf2e._comment | 65 +++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 doc/forum/lets_discuss_git_add_behavior/comment_14_7966d8b5346de2718feccb2d246baf2e._comment diff --git a/doc/forum/lets_discuss_git_add_behavior/comment_14_7966d8b5346de2718feccb2d246baf2e._comment b/doc/forum/lets_discuss_git_add_behavior/comment_14_7966d8b5346de2718feccb2d246baf2e._comment new file mode 100644 index 0000000000..9ff500e1ca --- /dev/null +++ b/doc/forum/lets_discuss_git_add_behavior/comment_14_7966d8b5346de2718feccb2d246baf2e._comment @@ -0,0 +1,65 @@ +[[!comment format=mdwn + username="amindfv@6f75355c5dad3450ed73d1f01715be90dfdd6cd6" + nickname="amindfv" + avatar="http://cdn.libravatar.org/avatar/9cdda587f634ea9a85b34b25be421676" + subject="comment 14" + date="2019-10-14T00:20:25Z" + content=""" +I'm again trying to do my best at expressing what I perceive as the seriousness of this change without being too critical of the dev who made it all possible. Sincere apologies in advance if I haven't hit the right balance. Git Annex really is one of my favorite tools! :) + + +I missed the fact that (if I understand correctly) regardless of what flags you set, \"git add\" really is just \"git annex add\" with the newest version. Setting e.g. 'largefile = nothing' just means you can't add anything to git annex at all! + +(Side note: if I after hours of research missed that, mow much confusion can we expect for the average user without that much time to spend?) + +After realizing this, I created a couple of aliases to - I thought - get back control by being super explicit. Aliases for + + git annex add -c annex.largefiles=anything + +and + + git annex add -c annex.largefiles=nothing + +But then, without thinking, I ran a couple of scripts, one of which calls \"git annex add\" and the other of which calls \"git add\" an an \"annex-init\"ed repo. Now I've got to fix another mess. + +I can't assume there's any part of my digital life involving git that this doesn't impact! + +So my only option, it seems, to wrest back control of my repos is to define extensive rules: + + largerthan=100kb and not (include=*.c or include=*.h or include=*.hs or include=*.lhs or include=*.hsc or include=*.pl or include=*.py or include= ...etc etc etc) + +What happens when I forget a file extension? If I remember to include '*.hs' but forget '*.lhs'? And then I push code to a team repo? + +And even still: I've gone through my git annex repos and found that I have greater than 100 files each of the following categories: + + * Files managed by \"vanilla\" git which are greater than 100kb in size, which have no file extension + * Files managed by git annex which are smaller than 100kb, which have no file extension + +(I stopped searching when I passed the 100 threshold on all these, but I could get more complete data if it's useful) + +Closer to what I want is the `\"mimeencoding=binary\"` criterion, but: + + * It's just a more accurate rule, not a completely accurate rule, and I don't want to fight my tools (or not notice till it's too late) when it incorrectly guesses. I don't want it guessing at all! + * As the manual notes, \"This is only available to use when git-annex was built with the MagicMime build flag.\" + +I've decided the only safe thing to do is to downgrade to an earlier git-annex version until something's sorted out. + +It's not an exaggeration to say that this change redefines what git annex is. Previously if someone asked \"what's git annex?\" my answer has been: + +\"Oh yeah, it's super cool! It allows you to add large files to git without keeping their actual contents in git, which means [... etc]\" + +Now I'd have to phrase it differently: + +\"It redefines 'git add' to try to correctly guess which files should be tracked in git vs. a separate store, which has benefit A, B, and C. It's cool but it makes me nervous.\" + +I worry \"git add\" will suffer from the problem of the old Microsoft Word, where it's decided what you're writing is a resume and you have to fight the f***ing thing to convince it it's not a resume and to stop \"fixing\" your work. + +@Lukey writes: + +> Maybe something like \"git annex v5mode true\" is needed so we get v5 semantics in v7 mode. + +I _really_ don't think this is good enough. \"git add\" has a meaning, and \"everyone knows\" what it means. This change unnecessarily breaks all users' understanding of what \"git add\" does. It may cause them to lose data, etc. +I'd be fine with a flag to do the inverse (explicitly set \"git add\" to mean \"git annex add\"). + + +"""]] From 52b0a6530d68a3667649b1c26484b2c6f59ebdfe Mon Sep 17 00:00:00 2001 From: Ilya_Shlyakhter Date: Mon, 14 Oct 2019 03:42:05 +0000 Subject: [PATCH 07/12] Added a comment: getting v5 semantics --- .../comment_15_3725ed643660d0e1b8fadd17705aebf1._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/forum/lets_discuss_git_add_behavior/comment_15_3725ed643660d0e1b8fadd17705aebf1._comment diff --git a/doc/forum/lets_discuss_git_add_behavior/comment_15_3725ed643660d0e1b8fadd17705aebf1._comment b/doc/forum/lets_discuss_git_add_behavior/comment_15_3725ed643660d0e1b8fadd17705aebf1._comment new file mode 100644 index 0000000000..6ae8603b5d --- /dev/null +++ b/doc/forum/lets_discuss_git_add_behavior/comment_15_3725ed643660d0e1b8fadd17705aebf1._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="Ilya_Shlyakhter" + avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" + subject="getting v5 semantics" + date="2019-10-14T03:42:05Z" + content=""" +The best way to restore v5 semantics right now seems to be: (1) set `annex.largefiles=nothing` to prevent `git add` from annexing; (2) make an alias to use instead of `git annex add`, which temporarily sets `annex.largefiles` to whatever it normally would be and then calls `git-annex-add`. But old scripts that call `git-annex-add` would need to be changed to call the alias. +"""]] From 6dd6f3ccf297d3757302e19a22e1c45009c8581d Mon Sep 17 00:00:00 2001 From: TroisSinges Date: Mon, 14 Oct 2019 09:25:18 +0000 Subject: [PATCH 08/12] --- .../S3_copies__58___unsafe_to_drop_from_another_repo__63__.mdwn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/forum/S3_copies__58___unsafe_to_drop_from_another_repo__63__.mdwn b/doc/forum/S3_copies__58___unsafe_to_drop_from_another_repo__63__.mdwn index 11ca875974..35c363d8a4 100644 --- a/doc/forum/S3_copies__58___unsafe_to_drop_from_another_repo__63__.mdwn +++ b/doc/forum/S3_copies__58___unsafe_to_drop_from_another_repo__63__.mdwn @@ -2,4 +2,4 @@ Hi! A file is present in my local repository (v7), and in a special encrypted repository using S3. Numcopies is set to 1. Whenever I try to drop the file, git-annex says it's "Unable to lock down 1 copy of file that is required to safely drop it" after having been "checking repository-s3... (unsafe)". -What "unsafe" means? When I fsck the file in repository-s3, everything's fine and the file is present. +What does "unsafe" means? When I fsck the file in repository-s3, everything's fine and the file is present. From 5f36424eaa28fda29e2c8cee9ebc370e75e3a819 Mon Sep 17 00:00:00 2001 From: "https://tribut.de/" Date: Mon, 14 Oct 2019 09:33:34 +0000 Subject: [PATCH 09/12] Added a comment --- .../comment_3_63ae1e0e0437c83516d167be142a8179._comment | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 doc/bugs/git-annex-sync_sometimes_fails_in_submodule_in_V6_adjusted_branch/comment_3_63ae1e0e0437c83516d167be142a8179._comment diff --git a/doc/bugs/git-annex-sync_sometimes_fails_in_submodule_in_V6_adjusted_branch/comment_3_63ae1e0e0437c83516d167be142a8179._comment b/doc/bugs/git-annex-sync_sometimes_fails_in_submodule_in_V6_adjusted_branch/comment_3_63ae1e0e0437c83516d167be142a8179._comment new file mode 100644 index 0000000000..e15ab3380e --- /dev/null +++ b/doc/bugs/git-annex-sync_sometimes_fails_in_submodule_in_V6_adjusted_branch/comment_3_63ae1e0e0437c83516d167be142a8179._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="https://tribut.de/" + nickname="Felix" + avatar="http://cdn.libravatar.org/avatar/d7508a030fd9902a76b02f7feff1327e80400a1317ee98e463c68ef4c9c40bb8" + subject="comment 3" + date="2019-10-14T09:33:34Z" + content=""" +`git annex sync` fails even when `commit.gpgSign` is `false` and I use a repository without signatures as submodule. So my hunch about signatures was probably not correct. +"""]] From 25e78d5f22bc621a7f2a7a7003c79d0bd548c0f0 Mon Sep 17 00:00:00 2001 From: yarikoptic Date: Mon, 14 Oct 2019 14:56:19 +0000 Subject: [PATCH 10/12] initial complaint that cannot use --duplicate and --reinject-duplicates --- ...while_keeping_original_files_in_place.mdwn | 47 +++++++++++++++++++ 1 file changed, 47 insertions(+) create mode 100644 doc/bugs/impossible__40____63____41___to_continuously_re-import_a_directory_while_keeping_original_files_in_place.mdwn diff --git a/doc/bugs/impossible__40____63____41___to_continuously_re-import_a_directory_while_keeping_original_files_in_place.mdwn b/doc/bugs/impossible__40____63____41___to_continuously_re-import_a_directory_while_keeping_original_files_in_place.mdwn new file mode 100644 index 0000000000..c7ebc3f433 --- /dev/null +++ b/doc/bugs/impossible__40____63____41___to_continuously_re-import_a_directory_while_keeping_original_files_in_place.mdwn @@ -0,0 +1,47 @@ +### Please describe the problem. + +I am trying to import (and then reimport) a directory which I sync to from box.com shared with me folder. +I have used `--duplicate` option to not delete original files upon `import`. But then upon-rerunning `import` command git-annex would error out if file already exists. `--reinject-duplicates` seems to be the option to use, but all those modes are "exclusive" so I cannot use `--duplicate --reinject-duplicates`, and using `--reinject-duplicates` alone would result in removing original files (as without `--duplicates`) + +### What version of git-annex are you using? On what operating system? + +7.20190819+git2-g908476a9b-1~ndall+1 + +### Please provide any additional information below. + +my little demo snippet for import with using --duplicate and then both options at the same time: + +[[!format sh """ +$> mkdir /tmp/d-in /tmp/d-repo && touch /tmp/d-in/file && ( cd /tmp/d-repo && git init && git annex init && for r in 1 2; do echo "Run $r"; ls -l ../d-in && git annex import --duplicate ../d-in/.; done ) +Initialized empty Git repository in /tmp/d-repo/.git/ +init ok +(recording state in git...) +Run 1 +total 0 +-rw------- 1 yoh yoh 0 Oct 14 10:51 file +import ./file ok +(recording state in git...) +Run 2 +total 0 +-rw------- 1 yoh yoh 0 Oct 14 10:51 file +import ./file + not overwriting existing ./file (is a symlink) +failed +git-annex: import: 1 failed + + +$> cd d-repo +$> git annex import ../d-in/. --reinject-duplicates --duplicate 2>&1 | head -n 3 +Invalid option `--duplicate' + +Usage: git-annex COMMAND + +"""]] + + +Or may be there is a better way to establish re-runnable import from a directory workflow? + +[[!meta author=yoh]] +[[!tag projects/dandi]] + + From d671481ed7d9e3c3bec7007f8cbec40e6873d274 Mon Sep 17 00:00:00 2001 From: strmd Date: Mon, 14 Oct 2019 18:23:59 +0000 Subject: [PATCH 11/12] Added a comment: A new user chiming in --- ...ment_16_7ba4c08b3ab5753907e87781e15ba4e4._comment | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 doc/forum/lets_discuss_git_add_behavior/comment_16_7ba4c08b3ab5753907e87781e15ba4e4._comment diff --git a/doc/forum/lets_discuss_git_add_behavior/comment_16_7ba4c08b3ab5753907e87781e15ba4e4._comment b/doc/forum/lets_discuss_git_add_behavior/comment_16_7ba4c08b3ab5753907e87781e15ba4e4._comment new file mode 100644 index 0000000000..20bc15f85c --- /dev/null +++ b/doc/forum/lets_discuss_git_add_behavior/comment_16_7ba4c08b3ab5753907e87781e15ba4e4._comment @@ -0,0 +1,12 @@ +[[!comment format=mdwn + username="strmd" + avatar="http://cdn.libravatar.org/avatar/035707b9756129bbdea6b36a7f7b38d3" + subject="A new user chiming in" + date="2019-10-14T18:23:59Z" + content=""" +I, too, am bewildered by how this major, breaking change was implemented and communicated so casually. Having basically been drawn in by the default v5 workflow and just barely finished migrating all of my archives/data sets and many of my current projects, I was initially convinced I must have not understood something right. But here we are, and at least I'm glad I'm not alone. + +To a certain extent, I get that git-annex users are expected to be pretty savvy, but the auto-upgrade caught me completely by suprise. I'm on a Mac and ran `brew upgrade ` as part of everyday maintenance, made some changes in my repo, synced it to a remote and to my complete surprise was the repo now behaving the exact opposite way as the day before, i.e. `git add` annexed the file. + +I am super happy about git-annex and your work on it Joey, but this has been a scary experience for me. And I'd very much like the default v5 workflow back. +"""]] From a3fdc23828e1f1dfdbf37ef8b1c01eef47a80757 Mon Sep 17 00:00:00 2001 From: "nrg@bd619d1ebf16e6324c546adea8be8fe1cc2b4325" Date: Mon, 14 Oct 2019 19:14:02 +0000 Subject: [PATCH 12/12] --- doc/bugs/build_failure_on_macOS_10.14.6.mdwn | 73 ++++++++++++++++++++ 1 file changed, 73 insertions(+) create mode 100644 doc/bugs/build_failure_on_macOS_10.14.6.mdwn diff --git a/doc/bugs/build_failure_on_macOS_10.14.6.mdwn b/doc/bugs/build_failure_on_macOS_10.14.6.mdwn new file mode 100644 index 0000000000..9e7d3a193c --- /dev/null +++ b/doc/bugs/build_failure_on_macOS_10.14.6.mdwn @@ -0,0 +1,73 @@ +### Please describe the problem. + + +Working to update the macOS Homebrew Formula for git-annex 7.20191009 and I'm receiving a build error using the Homebrew build system to build via ghc 8.6.5 on macOS 10.14.6: + +[[!format sh """ +[ 70 of 624] Compiling Utility.Shell ( Utility/Shell.hs, dist/dist-sandbox-11319d3/build/git-annex/git-annex-tmp/Utility/Shell.o ) +[ 71 of 624] Compiling Git.Types ( Git/Types.hs, dist/dist-sandbox-11319d3/build/git-annex/git-annex-tmp/Git/Types.o ) +[ 72 of 624] Compiling Utility.FileSystemEncoding ( Utility/FileSystemEncoding.hs, dist/dist-sandbox-11319d3/build/git-annex/git-annex-tmp/Utility/FileSystemEncoding.o ) +[ 73 of 624] Compiling Utility.Base64 ( Utility/Base64.hs, dist/dist-sandbox-11319d3/build/git-annex/git-annex-tmp/Utility/Base64.o ) +[ 74 of 624] Compiling Utility.Aeson ( Utility/Aeson.hs, dist/dist-sandbox-11319d3/build/git-annex/git-annex-tmp/Utility/Aeson.o ) +[ 75 of 624] Compiling Types.Messages ( Types/Messages.hs, dist/dist-sandbox-11319d3/build/git-annex/git-annex-tmp/Types/Messages.o ) +[ 76 of 624] Compiling Database.Handle ( Database/Handle.hs, dist/dist-sandbox-b74b6b12/build/git-annex/git-annex-tmp/Database/Handle.o ) + +Database/Handle.hs:273:28: error: + • Could not deduce (BackendCompatible SqlBackend backend) + arising from a use of ‘close'’ + from the context: (IsPersistBackend backend, + BaseBackend backend ~ SqlBackend) + bound by the type signature for: + closeRobustly :: forall backend. + (IsPersistBackend backend, BaseBackend backend ~ SqlBackend) => + backend -> IO () + at Database/Handle.hs:(260,1)-(263,16) + • In the second argument of ‘($)’, namely ‘close' conn’ + In a stmt of a 'do' block: r <- try $ close' conn + In the expression: + do r <- try $ close' conn + case r of + Right () -> return () + Left ex@(Sqlite.SqliteException {Sqlite.seError = e}) + | e == Sqlite.ErrorBusy -> do ... + | otherwise -> rethrow "while closing database connection" ex + | +273 | r <- try $ close' conn + | ^^^^^^^^^^^ +cabal: Leaving directory '.' +cabal: Error: some packages failed to install: +git-annex-7.20191009-KZnzbjdUc2rE3oBBwl75Ho failed during the building phase. +The exception was: +ExitFailure 1 +"""]] + +Is this an artifact of ghc 8.6.5 vs 8.8.1? + +### What steps will reproduce the problem? + +Updated the Homebrew Formula with the appropriate changes. + +[[!format sh """ +# brew install ghc@8.6 +# brew install --verbose --build-bottle git-annex` +"""]] + +### What version of git-annex are you using? On what operating system? + +git-annex 7.20191009 +macOS 10.14.6 +The Glorious Glasgow Haskell Compilation System, version 8.6.5 + +### Please provide any additional information below. + +[[!format sh """ +# If you can, paste a complete transcript of the problem occurring here. +# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log + + +# End of transcript or log. +"""]] + +### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) + +git-annex has been awesome for managing a massive collection of raw images and video clips across several storage targets and editing systems.