From fe518e13d51940d172299fe729456584f6900926 Mon Sep 17 00:00:00 2001 From: "jenkin.schibel@286264d9ceb79998aecff0d5d1a4ffe34f8b8421" Date: Sun, 6 Jun 2021 14:43:40 +0000 Subject: [PATCH 01/11] Added a comment: using import tree and export tree --- .../comment_23_d1b44f0cf171fb8cab85add778f2949b._comment | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 doc/special_remotes/webdav/comment_23_d1b44f0cf171fb8cab85add778f2949b._comment diff --git a/doc/special_remotes/webdav/comment_23_d1b44f0cf171fb8cab85add778f2949b._comment b/doc/special_remotes/webdav/comment_23_d1b44f0cf171fb8cab85add778f2949b._comment new file mode 100644 index 0000000000..c375606c37 --- /dev/null +++ b/doc/special_remotes/webdav/comment_23_d1b44f0cf171fb8cab85add778f2949b._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="jenkin.schibel@286264d9ceb79998aecff0d5d1a4ffe34f8b8421" + nickname="jenkin.schibel" + avatar="http://cdn.libravatar.org/avatar/692d82fb5c42fc86d97cc44ae0fb61ca" + subject="using import tree and export tree" + date="2021-06-06T14:43:39Z" + content=""" +hey will being able to import a treeish from a webdav remote ever be supported? my use case is that i have a nextcloud instance where i store photo backups for all the smart devices in my family which all get backed up to a single shared directory. since this tree would be ever changing due to the many smart phones connecting to it, and storing data in it, i figured a push and pull method similar to what can be done with the adb special remote could be useful to keep all the files tracked in my annex. +"""]] From 234d235a0323ccd3a9b626f7ef07be81bfad8e99 Mon Sep 17 00:00:00 2001 From: Atemu Date: Sun, 6 Jun 2021 16:50:40 +0000 Subject: [PATCH 02/11] rename bugs/delayadd_doesn__39__t_work.mdwn to bugs/delayadd_doesn__39__t_work_with_smallfiles.mdwn --- ..._work.mdwn => delayadd_doesn__39__t_work_with_smallfiles.mdwn} | 0 .../comment_1_1501fc7de682c0f2920c6c592204268c._comment | 0 .../comment_2_3aa7b34ff3d0606f97fb9e80ece34255._comment | 0 .../comment_3_f0b5e6f0554eb43f55bfc99d178c506d._comment | 0 .../comment_4_dbe41188bc6650418b68f52ec479fc11._comment | 0 .../comment_5_31a194407e433b17450725170552b8f7._comment | 0 6 files changed, 0 insertions(+), 0 deletions(-) rename doc/bugs/{delayadd_doesn__39__t_work.mdwn => delayadd_doesn__39__t_work_with_smallfiles.mdwn} (100%) rename doc/bugs/{delayadd_doesn__39__t_work => delayadd_doesn__39__t_work_with_smallfiles}/comment_1_1501fc7de682c0f2920c6c592204268c._comment (100%) rename doc/bugs/{delayadd_doesn__39__t_work => delayadd_doesn__39__t_work_with_smallfiles}/comment_2_3aa7b34ff3d0606f97fb9e80ece34255._comment (100%) rename doc/bugs/{delayadd_doesn__39__t_work => delayadd_doesn__39__t_work_with_smallfiles}/comment_3_f0b5e6f0554eb43f55bfc99d178c506d._comment (100%) rename doc/bugs/{delayadd_doesn__39__t_work => delayadd_doesn__39__t_work_with_smallfiles}/comment_4_dbe41188bc6650418b68f52ec479fc11._comment (100%) rename doc/bugs/{delayadd_doesn__39__t_work => delayadd_doesn__39__t_work_with_smallfiles}/comment_5_31a194407e433b17450725170552b8f7._comment (100%) diff --git a/doc/bugs/delayadd_doesn__39__t_work.mdwn b/doc/bugs/delayadd_doesn__39__t_work_with_smallfiles.mdwn similarity index 100% rename from doc/bugs/delayadd_doesn__39__t_work.mdwn rename to doc/bugs/delayadd_doesn__39__t_work_with_smallfiles.mdwn diff --git a/doc/bugs/delayadd_doesn__39__t_work/comment_1_1501fc7de682c0f2920c6c592204268c._comment b/doc/bugs/delayadd_doesn__39__t_work_with_smallfiles/comment_1_1501fc7de682c0f2920c6c592204268c._comment similarity index 100% rename from doc/bugs/delayadd_doesn__39__t_work/comment_1_1501fc7de682c0f2920c6c592204268c._comment rename to doc/bugs/delayadd_doesn__39__t_work_with_smallfiles/comment_1_1501fc7de682c0f2920c6c592204268c._comment diff --git a/doc/bugs/delayadd_doesn__39__t_work/comment_2_3aa7b34ff3d0606f97fb9e80ece34255._comment b/doc/bugs/delayadd_doesn__39__t_work_with_smallfiles/comment_2_3aa7b34ff3d0606f97fb9e80ece34255._comment similarity index 100% rename from doc/bugs/delayadd_doesn__39__t_work/comment_2_3aa7b34ff3d0606f97fb9e80ece34255._comment rename to doc/bugs/delayadd_doesn__39__t_work_with_smallfiles/comment_2_3aa7b34ff3d0606f97fb9e80ece34255._comment diff --git a/doc/bugs/delayadd_doesn__39__t_work/comment_3_f0b5e6f0554eb43f55bfc99d178c506d._comment b/doc/bugs/delayadd_doesn__39__t_work_with_smallfiles/comment_3_f0b5e6f0554eb43f55bfc99d178c506d._comment similarity index 100% rename from doc/bugs/delayadd_doesn__39__t_work/comment_3_f0b5e6f0554eb43f55bfc99d178c506d._comment rename to doc/bugs/delayadd_doesn__39__t_work_with_smallfiles/comment_3_f0b5e6f0554eb43f55bfc99d178c506d._comment diff --git a/doc/bugs/delayadd_doesn__39__t_work/comment_4_dbe41188bc6650418b68f52ec479fc11._comment b/doc/bugs/delayadd_doesn__39__t_work_with_smallfiles/comment_4_dbe41188bc6650418b68f52ec479fc11._comment similarity index 100% rename from doc/bugs/delayadd_doesn__39__t_work/comment_4_dbe41188bc6650418b68f52ec479fc11._comment rename to doc/bugs/delayadd_doesn__39__t_work_with_smallfiles/comment_4_dbe41188bc6650418b68f52ec479fc11._comment diff --git a/doc/bugs/delayadd_doesn__39__t_work/comment_5_31a194407e433b17450725170552b8f7._comment b/doc/bugs/delayadd_doesn__39__t_work_with_smallfiles/comment_5_31a194407e433b17450725170552b8f7._comment similarity index 100% rename from doc/bugs/delayadd_doesn__39__t_work/comment_5_31a194407e433b17450725170552b8f7._comment rename to doc/bugs/delayadd_doesn__39__t_work_with_smallfiles/comment_5_31a194407e433b17450725170552b8f7._comment From 550f67d8b48e406511b42b28dcf56de37987e1b6 Mon Sep 17 00:00:00 2001 From: Atemu Date: Sun, 6 Jun 2021 17:18:14 +0000 Subject: [PATCH 03/11] --- .../How_to_keep_connected_files_with_another__63__.mdwn | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 doc/forum/How_to_keep_connected_files_with_another__63__.mdwn diff --git a/doc/forum/How_to_keep_connected_files_with_another__63__.mdwn b/doc/forum/How_to_keep_connected_files_with_another__63__.mdwn new file mode 100644 index 0000000000..c1af975d9b --- /dev/null +++ b/doc/forum/How_to_keep_connected_files_with_another__63__.mdwn @@ -0,0 +1,5 @@ +I've got a repo full of (legally aquired) Movies and series, most of which have corresponding metadata JSON and ASS subtitle files. When distributing them over many cold storage drives, I've noticed that git-annex would try to fill them up with many of the (much smaller) text files when there isn't enough space for another video file, leaving video and subtitles files on separate drives. + +This isn't a critical issue since there are still enough copies and everything but it'd be annoying to have to search for and connect two or more drives to get videos + subtitles for a single series. + +I was wondering if there was perhaps a clever solution to prevent this from happening. Everything is organised into subfolders, so *ideally* I'd prefer if git-annex could be made to operate on full series (as defined by subdirectories or metadata perhaps?) instead of context-less files somehow. From cf9a93e90120f34741773f51a0007259d16e97b9 Mon Sep 17 00:00:00 2001 From: Lukey Date: Sun, 6 Jun 2021 18:01:08 +0000 Subject: [PATCH 04/11] Added a comment --- ...ent_1_89e04a7dc25a3df849aa2ada4dbc8439._comment | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 doc/forum/How_to_keep_connected_files_with_another__63__/comment_1_89e04a7dc25a3df849aa2ada4dbc8439._comment diff --git a/doc/forum/How_to_keep_connected_files_with_another__63__/comment_1_89e04a7dc25a3df849aa2ada4dbc8439._comment b/doc/forum/How_to_keep_connected_files_with_another__63__/comment_1_89e04a7dc25a3df849aa2ada4dbc8439._comment new file mode 100644 index 0000000000..e5c379bf61 --- /dev/null +++ b/doc/forum/How_to_keep_connected_files_with_another__63__/comment_1_89e04a7dc25a3df849aa2ada4dbc8439._comment @@ -0,0 +1,14 @@ +[[!comment format=mdwn + username="Lukey" + avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b" + subject="comment 1" + date="2021-06-06T18:01:08Z" + content=""" +You could make the subtitles wanted in every repo so that all subtitles are present in every repo. Since they are small, the overhead shouldn't be large. + +Or you could directly add them to git (\"small files\") so they are also present everywhere. On a fresh repo, this would help a bit with speed too since git-annex then doesn't need to keep track of the location of these small files. + +Or (depending on how you configured your preferred content) you could increase numcopies just for the small files. See [[walkthrough/backups/]]. + +Unfortunately, preferred-content can't directly relate multiple files with each other. git-annex iterates over each file in the tree and checks if preferred-content matches for that particular file. +"""]] From a706708d17708c385d7aec996ceb8c3a1a77f5f7 Mon Sep 17 00:00:00 2001 From: Atemu Date: Sun, 6 Jun 2021 20:47:31 +0000 Subject: [PATCH 05/11] Added a comment --- ..._2_20e8dba904eff7cc8f72ee12b8119632._comment | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 doc/forum/How_to_keep_connected_files_with_another__63__/comment_2_20e8dba904eff7cc8f72ee12b8119632._comment diff --git a/doc/forum/How_to_keep_connected_files_with_another__63__/comment_2_20e8dba904eff7cc8f72ee12b8119632._comment b/doc/forum/How_to_keep_connected_files_with_another__63__/comment_2_20e8dba904eff7cc8f72ee12b8119632._comment new file mode 100644 index 0000000000..f20e1db040 --- /dev/null +++ b/doc/forum/How_to_keep_connected_files_with_another__63__/comment_2_20e8dba904eff7cc8f72ee12b8119632._comment @@ -0,0 +1,17 @@ +[[!comment format=mdwn + username="Atemu" + avatar="http://cdn.libravatar.org/avatar/d1f0f4275931c552403f4c6707bead7a" + subject="comment 2" + date="2021-06-06T20:47:31Z" + content=""" +> You could make the subtitles wanted in every repo so that all subtitles are present in every repo. Since they are small, the overhead shouldn't be large. + +This is probably what I'll end up doing. + +> Or you could directly add them to git (\"small files\") so they are also present everywhere. + +I do that for one type of metadata file that isn't important for consumption but I want everything else to be annex files so that I can assign metadata etc. to them. + + + +"""]] From 3d8cd26497da59b1403a3a7eecbdd2544c5a733d Mon Sep 17 00:00:00 2001 From: Ilya_Shlyakhter Date: Mon, 7 Jun 2021 14:26:27 +0000 Subject: [PATCH 06/11] Added a comment: specifying preferred content by metadata --- ...comment_3_7b14aa71a166cf14e852ae896b05ee74._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/forum/How_to_keep_connected_files_with_another__63__/comment_3_7b14aa71a166cf14e852ae896b05ee74._comment diff --git a/doc/forum/How_to_keep_connected_files_with_another__63__/comment_3_7b14aa71a166cf14e852ae896b05ee74._comment b/doc/forum/How_to_keep_connected_files_with_another__63__/comment_3_7b14aa71a166cf14e852ae896b05ee74._comment new file mode 100644 index 0000000000..2a7a65aebd --- /dev/null +++ b/doc/forum/How_to_keep_connected_files_with_another__63__/comment_3_7b14aa71a166cf14e852ae896b05ee74._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="Ilya_Shlyakhter" + avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" + subject="specifying preferred content by metadata" + date="2021-06-07T14:26:27Z" + content=""" +>I'd prefer if git-annex could be made to operate on full series (as defined by subdirectories or metadata + +Maybe I'm missing something, but doesn't [[git-annex-preferred-content]] support `metadata=field=glob`? +"""]] From da98ede56a9d00e791861da7dafbf970941f89ad Mon Sep 17 00:00:00 2001 From: Ilya_Shlyakhter Date: Mon, 7 Jun 2021 14:26:57 +0000 Subject: [PATCH 07/11] Added a comment: specifying preferred content by metadata --- ...comment_4_1d73e3bcbda6c28556e2d5a6473de245._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/forum/How_to_keep_connected_files_with_another__63__/comment_4_1d73e3bcbda6c28556e2d5a6473de245._comment diff --git a/doc/forum/How_to_keep_connected_files_with_another__63__/comment_4_1d73e3bcbda6c28556e2d5a6473de245._comment b/doc/forum/How_to_keep_connected_files_with_another__63__/comment_4_1d73e3bcbda6c28556e2d5a6473de245._comment new file mode 100644 index 0000000000..dfcb6f8de2 --- /dev/null +++ b/doc/forum/How_to_keep_connected_files_with_another__63__/comment_4_1d73e3bcbda6c28556e2d5a6473de245._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="Ilya_Shlyakhter" + avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" + subject="specifying preferred content by metadata" + date="2021-06-07T14:26:57Z" + content=""" +>I'd prefer if git-annex could be made to operate on full series (as defined by subdirectories or metadata + +Maybe I'm missing something, but doesn't [[git-annex-preferred-content]] support `metadata=field=glob`? +"""]] From 58a07c6a42c549b49232e31e40e064a8e120b79e Mon Sep 17 00:00:00 2001 From: Ilya_Shlyakhter Date: Mon, 7 Jun 2021 14:31:52 +0000 Subject: [PATCH 08/11] removed --- ...comment_4_1d73e3bcbda6c28556e2d5a6473de245._comment | 10 ---------- 1 file changed, 10 deletions(-) delete mode 100644 doc/forum/How_to_keep_connected_files_with_another__63__/comment_4_1d73e3bcbda6c28556e2d5a6473de245._comment diff --git a/doc/forum/How_to_keep_connected_files_with_another__63__/comment_4_1d73e3bcbda6c28556e2d5a6473de245._comment b/doc/forum/How_to_keep_connected_files_with_another__63__/comment_4_1d73e3bcbda6c28556e2d5a6473de245._comment deleted file mode 100644 index dfcb6f8de2..0000000000 --- a/doc/forum/How_to_keep_connected_files_with_another__63__/comment_4_1d73e3bcbda6c28556e2d5a6473de245._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="Ilya_Shlyakhter" - avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" - subject="specifying preferred content by metadata" - date="2021-06-07T14:26:57Z" - content=""" ->I'd prefer if git-annex could be made to operate on full series (as defined by subdirectories or metadata - -Maybe I'm missing something, but doesn't [[git-annex-preferred-content]] support `metadata=field=glob`? -"""]] From 8822bf0803759dafa052a64871f9ce6f6bee00ae Mon Sep 17 00:00:00 2001 From: jwodder Date: Mon, 7 Jun 2021 14:32:25 +0000 Subject: [PATCH 09/11] --- doc/bugs/Build_fails_on_Windows_as_of_commit_a706708d1.mdwn | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 doc/bugs/Build_fails_on_Windows_as_of_commit_a706708d1.mdwn diff --git a/doc/bugs/Build_fails_on_Windows_as_of_commit_a706708d1.mdwn b/doc/bugs/Build_fails_on_Windows_as_of_commit_a706708d1.mdwn new file mode 100644 index 0000000000..8c0e362934 --- /dev/null +++ b/doc/bugs/Build_fails_on_Windows_as_of_commit_a706708d1.mdwn @@ -0,0 +1,4 @@ +As of commit a706708d1, trying to build git-annex on Windows fails because the import of `oneSecond` from `Utility.ThreadScheduler` is not available. [This patch](https://raw.githubusercontent.com/datalad/git-annex/master/patches/20210607-a706708d1-fix-oneSecond.patch) fixes that. + +[[!meta author=jwodder]] +[[!tag projects/datalad]] From def7e001c64301d305907016c9a132d828d5f7b7 Mon Sep 17 00:00:00 2001 From: Ilya_Shlyakhter Date: Mon, 7 Jun 2021 14:45:35 +0000 Subject: [PATCH 10/11] Added a comment: keeping connected files together --- ...comment_4_a33437b28ea7fc74eb221d90efaec487._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/forum/How_to_keep_connected_files_with_another__63__/comment_4_a33437b28ea7fc74eb221d90efaec487._comment diff --git a/doc/forum/How_to_keep_connected_files_with_another__63__/comment_4_a33437b28ea7fc74eb221d90efaec487._comment b/doc/forum/How_to_keep_connected_files_with_another__63__/comment_4_a33437b28ea7fc74eb221d90efaec487._comment new file mode 100644 index 0000000000..73139ec723 --- /dev/null +++ b/doc/forum/How_to_keep_connected_files_with_another__63__/comment_4_a33437b28ea7fc74eb221d90efaec487._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="Ilya_Shlyakhter" + avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" + subject="keeping connected files together" + date="2021-06-07T14:45:35Z" + content=""" +One other option is to `tar` up each movie and all associated files into one archive, and annex that. + +There's a [special remote in DataLad](https://github.com/datalad/datalad/blob/master/datalad/customremotes/archives.py) for accessing individual files inside annexed archives, though I guess in your case you'd normally want all files anyway. +"""]] From 5359f8bc14e7a4c131c8793555789b0fc5643e9e Mon Sep 17 00:00:00 2001 From: Ilya_Shlyakhter Date: Mon, 7 Jun 2021 15:08:51 +0000 Subject: [PATCH 11/11] added suggestion to match keys by file extension in the key --- doc/todo/find__47__prefer_keys_by_file_extension_in_key.mdwn | 1 + 1 file changed, 1 insertion(+) create mode 100644 doc/todo/find__47__prefer_keys_by_file_extension_in_key.mdwn diff --git a/doc/todo/find__47__prefer_keys_by_file_extension_in_key.mdwn b/doc/todo/find__47__prefer_keys_by_file_extension_in_key.mdwn new file mode 100644 index 0000000000..635f6a7ba4 --- /dev/null +++ b/doc/todo/find__47__prefer_keys_by_file_extension_in_key.mdwn @@ -0,0 +1 @@ +Add [[preferred content expression|git-annex-preferred-content]] and [[matching option|git-annex-matching-options]] to match the file extension incorporated into a `*E` [[key|backends]], e.g. `keyext=.mp3` . This would help address the limitation that `include=*.mp3` does not work with `--all` or `--unused`.