mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

Merge branch 'master' into hiddenannex

Browse source
This commit is contained in:
Joey Hess 2021-04-21 13:04:40 -04:00
commit 9b870e29fd
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
13 changed files with 205 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

23
doc/todo/hiding_a_repository.mdwn
View file

@ -145,6 +145,29 @@ all reads followed by writes do go via Annex.Branch.change, so Annex.Branch.get
can just concacenate the two without worrying about it leaking back out in a
later write.
> Implementing this is in progress, in the `hiddenannex` branch.
>
> Got the separate journal mostly working. No separate index yet.
> No way to configure what repo is hidden yet. --[[Joey]]
>
> Implementation notes:
>
> * CmdLine.Seek precaches git-annex branch
> location logs, but that does not include private ones. Since they're
> cached, the private ones don't get read. Result is eg, whereis finds no
> copies. Either need to disable CmdLine.Seek precaching when there's
> hidden repos, or could make the cache indicate it's only of public
> info, so private info still gets read.
> * CmdLine.Seek contains a LsTreeRecursive over the branch to handle
> --all, and again that won't see private information, including even
> annexed files that are only present in the hidden repo.
> * (And I wonder, don't both the caches above already miss things in
> the journal?)
> * Any other direct accesses of the branch, not going through
> Annex.Branch, also need to be fixed (and may be missing journal files
> already?) Command.ImportFeed.knownItems is one. Command.Log behavior
> needs to be investigated, may be ok. And Logs.Web.withKnownUrls is another.
## networks of hidden repos
There are a lot of complications involving using hidden repos as remotes.

8
doc/todo/hiding_a_repository/comment_3_64ba3f1d5099e066d554566ca818bdd6._comment Normal file
View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="Ilya_Shlyakhter"
avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
subject="auto-expire temp repos"
date="2021-04-21T15:37:37Z"
content="""
As a possible simpler alternative, maybe add an option to [[git-annex-dead]] to mark a repo dead from a future time onwards? I often have temp repos created on temp cloud instances. I mark them untrusted right after cloning, and then manually mark them dead after the cloud instance is gone. If the latter part were automated, would that cover most of what hidden repos do?
"""]]

15
doc/todo/option_for___40__fast__41___compression_on_special_remotes_like___34__directory__34__/comment_3_d4672f72a00509186cfc5dd85e1da140._comment Normal file
View file

@ -0,0 +1,15 @@
[[!comment format=mdwn
username="Atemu"
avatar="http://cdn.libravatar.org/avatar/d1f0f4275931c552403f4c6707bead7a"
subject="comment 3"
date="2021-04-20T18:05:27Z"
content="""
Would it perhaps be possible to set the compression using filters like file name/extension?
For example, I wouldn't want GA to waste time on compressing multimedia files that are already at entropy and, since they make up the majority of my special remote's content, re-writing them would be very time intensive (even more so when remote solutions are involved).
Certain compressors might also work better on some files types compared to others.
This could be very important to scientists using datalad as they are likely to A. be working very specific kinds of data where certain compressors might significantly outperform others and B. have large quantities of data where compression is essential.
If compressors are going to be limited to a known-safe selection, an important aspect to keep in mind would be compression levels as some compressors like zstd can range from lzo-like performance characteristics to lzma ones.
Definitely a +1 on this one though, it would be very useful for my use-case aswell.
"""]]