From f02545b482fe042b2916aefd53b27cadfc275b22 Mon Sep 17 00:00:00 2001 From: "https://me.yahoo.com/a/5j.FKrMpxZS.luSB.5ahyosMU6RcaYq2#74c60" Date: Sat, 25 Nov 2017 00:03:24 +0000 Subject: [PATCH 1/8] --- ...Adjust_--unlock_not_using_--reflink__63__.mdwn | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 doc/bugs/Adjust_--unlock_not_using_--reflink__63__.mdwn diff --git a/doc/bugs/Adjust_--unlock_not_using_--reflink__63__.mdwn b/doc/bugs/Adjust_--unlock_not_using_--reflink__63__.mdwn new file mode 100644 index 0000000000..91a69a4ea3 --- /dev/null +++ b/doc/bugs/Adjust_--unlock_not_using_--reflink__63__.mdwn @@ -0,0 +1,15 @@ +### Please describe the problem. + +Running adjust --unlock is unexpectedly slow and seems to use a lot of space, even on BTRFS, suggesting it probably does not use --reflink=auto like most other commands. + +### What steps will reproduce the problem? + +Run adjust --unlock with very large files. + +### What version of git-annex are you using? On what operating system? + +6.20170101-1+deb9u1 on Debian Stretch + +### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) + +Yes I have! I've used it manage lots of video editing disks before, and am now migrating several slightly different copies of 15TB sized documentary footage from random USB3 disks and LTO tapes to a RAID server with BTRFS. From bbcad2b0af8cd9264d0cb86e6ca126ae626171f3 Mon Sep 17 00:00:00 2001 From: "s998758@d9eef097c6baf74cc86fd8dbb003c33be91873b8" Date: Sun, 26 Nov 2017 01:01:18 +0000 Subject: [PATCH 2/8] --- ...rs_to_start_playing_with_metadata_cache_plz.mdwn | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100644 doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz.mdwn diff --git a/doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz.mdwn b/doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz.mdwn new file mode 100644 index 0000000000..da8c030702 --- /dev/null +++ b/doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz.mdwn @@ -0,0 +1,13 @@ +Hello. Am a newbie to Git Annex(ga), but love it already. I kept trying to index own important files for the past long time, but ended up all tangled up. With ga I now see a light at the end of the tunnel! (Hope it's not a train heading my way :) + +So thanks a bucket for writing Git Annex! + +I am an "archiver": Every file I add to ga repo is a never-to-be-changed file (it's checksum stays same throughout eternity, only metadata keeps changin). All I need ga for atm is to tag all files. Unfortunately we are talking about few hundred thousand files and the performance with the master git-annex-6.20170519 is not quite what one might hope for. + +From your design/caching_database doc I gather that the outlook with metadata is positive ( "For metadata, the story is much nicer. Querying for 30000 keys that all have a particular tag in their metadata takes 0.65s. So fast enough to be used in views." ), but is not in a db (sqlite) yet in the master (git-annex-6.20170519) . I tried to dig through some of the Links there to find out which commit could I checkout and build to try out a cached metadata, but no avail. + +Since I don't ever change any file once it gets checked into the ga repo, does that simplify my possible use of current metadata cache code, or will I have to try to learn haskell and will I need to code stuff to get performance (creating views and such). + +TIA for any pointers, tips and cavats and THANKS AGAIN FOR WRITING GIT-ANNEX. + + ganewbie01 From f31469e6d23e700545fa79688e99aeda2ba6cdce Mon Sep 17 00:00:00 2001 From: ganewbie01 Date: Sun, 26 Nov 2017 13:12:16 +0000 Subject: [PATCH 3/8] Added a comment: development branches inaccessible? --- ...comment_1_0547fcabefe0b8e99a43977ffec3e28c._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_1_0547fcabefe0b8e99a43977ffec3e28c._comment diff --git a/doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_1_0547fcabefe0b8e99a43977ffec3e28c._comment b/doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_1_0547fcabefe0b8e99a43977ffec3e28c._comment new file mode 100644 index 0000000000..db608ffb8e --- /dev/null +++ b/doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_1_0547fcabefe0b8e99a43977ffec3e28c._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="ganewbie01" + avatar="http://cdn.libravatar.org/avatar/a3b7d6e560486cb87c51cb0cf3328c8e" + subject="development branches inaccessible?" + date="2017-11-26T13:12:16Z" + content=""" +To not sit idle, I've been looking for development branches (specifically the one containing code that gave the rise to Joey's claim \"Querying for 30000 keys that all have a particular tag in their metadata takes 0.65s.\"), but could find only repos with the one branch - the master branch, which doesn't (naturally seem to) include the code for SQLite metadata tinkering. + +Is there someplace I could find such development branches please? +"""]] From 544f999e74c43746dc8a35937a9c3c27b6224569 Mon Sep 17 00:00:00 2001 From: olaf Date: Mon, 27 Nov 2017 05:30:06 +0000 Subject: [PATCH 4/8] Added a comment --- ..._407d0585331805d1506609f0bd753762._comment | 41 +++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_2_407d0585331805d1506609f0bd753762._comment diff --git a/doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_2_407d0585331805d1506609f0bd753762._comment b/doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_2_407d0585331805d1506609f0bd753762._comment new file mode 100644 index 0000000000..03bc2f1783 --- /dev/null +++ b/doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_2_407d0585331805d1506609f0bd753762._comment @@ -0,0 +1,41 @@ +[[!comment format=mdwn + username="olaf" + avatar="http://cdn.libravatar.org/avatar/4ae498d3d6ee558d6b65caa658f72572" + subject="comment 2" + date="2017-11-27T05:30:06Z" + content=""" +Did you clone the git repository? + + $ git clone git://git-annex.branchable.com/ + +I see a whole lot of branches (they're *remote* branches initially, so don't forget the `-a`): + + $ git branch -a + * master + remotes/origin/HEAD -> origin/master + remotes/origin/atomic-store-test + remotes/origin/debian + remotes/origin/debian-jessie-backport + remotes/origin/debian-squeeze-backport + remotes/origin/debian-stable-security-fix + remotes/origin/debian-wheezy-backport + remotes/origin/ghc7.0 + remotes/origin/improved-smudge-filters + remotes/origin/master + remotes/origin/newwinrelease + remotes/origin/no-direct-mode + remotes/origin/p2p-map + remotes/origin/setup + remotes/origin/smudge + remotes/origin/tweak-fetch + remotes/origin/uuid-type-rework + remotes/origin/winsplicehack + +You can checkout one of the branches like: + + $ git checkout remotes/origin/ghc7.0 + + +Does that help? + +"""]] From 008443581aa35bc1747eed9bd570777db7d64c95 Mon Sep 17 00:00:00 2001 From: olaf Date: Mon, 27 Nov 2017 05:31:16 +0000 Subject: [PATCH 5/8] removed --- ..._407d0585331805d1506609f0bd753762._comment | 41 ------------------- 1 file changed, 41 deletions(-) delete mode 100644 doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_2_407d0585331805d1506609f0bd753762._comment diff --git a/doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_2_407d0585331805d1506609f0bd753762._comment b/doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_2_407d0585331805d1506609f0bd753762._comment deleted file mode 100644 index 03bc2f1783..0000000000 --- a/doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_2_407d0585331805d1506609f0bd753762._comment +++ /dev/null @@ -1,41 +0,0 @@ -[[!comment format=mdwn - username="olaf" - avatar="http://cdn.libravatar.org/avatar/4ae498d3d6ee558d6b65caa658f72572" - subject="comment 2" - date="2017-11-27T05:30:06Z" - content=""" -Did you clone the git repository? - - $ git clone git://git-annex.branchable.com/ - -I see a whole lot of branches (they're *remote* branches initially, so don't forget the `-a`): - - $ git branch -a - * master - remotes/origin/HEAD -> origin/master - remotes/origin/atomic-store-test - remotes/origin/debian - remotes/origin/debian-jessie-backport - remotes/origin/debian-squeeze-backport - remotes/origin/debian-stable-security-fix - remotes/origin/debian-wheezy-backport - remotes/origin/ghc7.0 - remotes/origin/improved-smudge-filters - remotes/origin/master - remotes/origin/newwinrelease - remotes/origin/no-direct-mode - remotes/origin/p2p-map - remotes/origin/setup - remotes/origin/smudge - remotes/origin/tweak-fetch - remotes/origin/uuid-type-rework - remotes/origin/winsplicehack - -You can checkout one of the branches like: - - $ git checkout remotes/origin/ghc7.0 - - -Does that help? - -"""]] From 72a9df1caeddcd62b248267ad1b5411fa55ddcaa Mon Sep 17 00:00:00 2001 From: olaf Date: Mon, 27 Nov 2017 05:39:04 +0000 Subject: [PATCH 6/8] Added a comment --- ..._606281f102d8c3168cbc55142a996e05._comment | 39 +++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100644 doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_2_606281f102d8c3168cbc55142a996e05._comment diff --git a/doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_2_606281f102d8c3168cbc55142a996e05._comment b/doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_2_606281f102d8c3168cbc55142a996e05._comment new file mode 100644 index 0000000000..f503270d17 --- /dev/null +++ b/doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_2_606281f102d8c3168cbc55142a996e05._comment @@ -0,0 +1,39 @@ +[[!comment format=mdwn + username="olaf" + avatar="http://cdn.libravatar.org/avatar/4ae498d3d6ee558d6b65caa658f72572" + subject="comment 2" + date="2017-11-27T05:39:04Z" + content=""" +Did you clone the repository? + + $ git clone git://git-annex.branchable.com/ git-annex + +I see lots of branches (remember they are *remote* branches so you will need the `-a` flag): + + $ git branch -a + * master + remotes/origin/HEAD -> origin/master + remotes/origin/atomic-store-test + remotes/origin/debian + remotes/origin/debian-jessie-backport + remotes/origin/debian-squeeze-backport + remotes/origin/debian-stable-security-fix + remotes/origin/debian-wheezy-backport + remotes/origin/ghc7.0 + remotes/origin/improved-smudge-filters + remotes/origin/master + remotes/origin/newwinrelease + remotes/origin/no-direct-mode + remotes/origin/p2p-map + remotes/origin/setup + remotes/origin/smudge + remotes/origin/tweak-fetch + remotes/origin/uuid-type-rework + remotes/origin/winsplicehack + +You can checkout one of the branches like: + + $ git checkout remotes/origin/setup + +Does that help? +"""]] From d8d04defa23d59b29d59eb8fd2c70dda1cb07abb Mon Sep 17 00:00:00 2001 From: ynikitenko Date: Mon, 27 Nov 2017 11:22:04 +0000 Subject: [PATCH 7/8] --- ...e_local_repository_for_git-annex-info.mdwn | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+) create mode 100644 doc/forum/Rename_local_repository_for_git-annex-info.mdwn diff --git a/doc/forum/Rename_local_repository_for_git-annex-info.mdwn b/doc/forum/Rename_local_repository_for_git-annex-info.mdwn new file mode 100644 index 0000000000..1c26974314 --- /dev/null +++ b/doc/forum/Rename_local_repository_for_git-annex-info.mdwn @@ -0,0 +1,32 @@ +I initialized local repository with + + git-annex init $HOSTNAME --version=6 + +unfortunately I didn't change HOSTNAME on a new machine and it was 'localhost.localdomain'. I didn't notice that before I cloned a git-annex repository. + +Now in the remote repository when I run `git-annex info` (and same in local repository), I see + + $ git-annex info + ... + semitrusted repositories: 6 + 00000000-0000-0000-0000-000000000001 -- web + 00000000-0000-0000-0000-000000000002 -- bittorrent + 0085(maybe it's not secure to write it on the internet)-e8f803a -- localhost.localdomain + ... + +(and other repositories. By the way, I never initialized 'web' and 'bittorent', where did they get from?) + +I would like 'localhost.localdomain' to become my real $HOSTNAME, so that I would distinguish that machine. How could I do that? + +I found [How to rename a remote](https://git-annex.branchable.com/forum/How_to_rename_a_remote__63__/), but my 'localhost' is not listed in git-remotes. + +I grep-ed .git for 'localhost.localdomain', and changed `.git/COMMIT_EDITMSG`. However, after running git-annex sync it returns to 'localhost.localdomain'. + + $ more .git/COMMIT_EDITMSG + git-annex in Acer + $ git-annex sync + ... + $ more .git/COMMIT_EDITMSG + git-annex in localhost.localdomain + +I would like to change 'localhost' to my real machine name both on the remote repository from which I cloned and on local repository. Thank you. From 391619f6de00a1f0cea6383a4a70ce58063fa4ce Mon Sep 17 00:00:00 2001 From: ganewbie01 Date: Tue, 28 Nov 2017 01:03:06 +0000 Subject: [PATCH 8/8] Added a comment: found it! ( I think ... or should I be still looking for "database" branch? ) --- ...nt_3_ca138f01184631135d07835d79a95112._comment | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_3_ca138f01184631135d07835d79a95112._comment diff --git a/doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_3_ca138f01184631135d07835d79a95112._comment b/doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_3_ca138f01184631135d07835d79a95112._comment new file mode 100644 index 0000000000..a8933e1b4c --- /dev/null +++ b/doc/forum/ga_dev_newbie_Q__58___pointers_to_start_playing_with_metadata_cache_plz/comment_3_ca138f01184631135d07835d79a95112._comment @@ -0,0 +1,15 @@ +[[!comment format=mdwn + username="ganewbie01" + avatar="http://cdn.libravatar.org/avatar/a3b7d6e560486cb87c51cb0cf3328c8e" + subject="found it! ( I think ... or should I be still looking for "database" branch? )" + date="2017-11-28T01:03:05Z" + content=""" +hi, thanks for your reply; +I've spent several hours today looking through the git-annex repo. I think it was a great idea to place the forums and everything in one repo! It provides sort of a \"running commentary\" on what was going on and why ... + +After a couple of hours looking through the repo using tig, I checked out the key commit \"bb242bdd82a438ebfc937609d8d13b512cb49943\" and found the foo.hs and fooes.hs files which are most likely the ones that Joey was writing about when he expressed hopes for metadata in an sqlite file. ( I didn't find a way to see \"old branches\" though, e.g. the one named `database`. Maybe if I study git more ... ) + +Thanks for your reply to a silly newbie question anyway! I'll study this some more and see if I have some on-topic questions (hopefully they will be more educated by then :) ) + +g'day! +"""]]