Merge branch 'master' of ssh://git-annex.branchable.com

This commit is contained in:
Joey Hess 2019-09-19 10:47:58 -04:00
commit 133aba5729
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
8 changed files with 71 additions and 0 deletions

View file

@ -0,0 +1,28 @@
### Please describe the problem.
I just upgraded one of my repositories to v7. Now, git status takes ages. The "Refresh index" count increments every second, so it will take ~12 hours to complete. With such slow operations I really fear the moment when v7 becomes default.
I've got ~1000 unlocked files.
### What steps will reproduce the problem?
[[!format sh """
$ git status
Refresh index: 0% (49/43974)
"""]]
### What version of git-annex are you using? On what operating system?
[[!format sh """
$ git annex version
git-annex version: 7.20190819-g972fd11f4e
build flags: Assistant Webapp Pairing S3 WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite
dependency versions: aws-0.21.1 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.3 feed-1.2.0.0 ghc-8.6.5 http-client-0.6.4 persistent-sqlite-2.10.5 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs hook external
operating system: linux x86_64
supported repository versions: 5 7
upgrade supported from repository versions: 0 1 2 3 4 5 6
local repository version: 7
"""]]

View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="lykos@d125a37d89b1cfac20829f12911656c40cb70018"
nickname="lykos"
avatar="http://cdn.libravatar.org/avatar/085df7b04d3408ba23c19f9c49be9ea2"
subject="comment 1"
date="2019-09-18T20:20:33Z"
content="""
OK so after processing the unlocked files, it got a lot faster. Will it be like that on every new clone and with every new unlocked file?
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="Ilya_Shlyakhter"
avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
subject="comment 2"
date="2019-09-18T20:16:26Z"
content="""
That's what I thought, just checking. The [[conda install|install/conda]] builds with these flags on.
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="Ilya_Shlyakhter"
avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
subject="comment 2"
date="2019-09-18T20:33:04Z"
content="""
\"The assistant handles copying files to other remotes, so use it instead of watch if you want that\" -- but the assistant is a GUI, right? My use case has to run in batch mode. I just run [[`git-annex-sync`|git-annex-sync]] periodically. I assume it's ok to both do that and have [[`git-annex-watch`|git-annex-watch]] running in the background?
"""]]

View file

@ -0,0 +1 @@
As suggested at [[forum/Find_unlocked__47__locked_files]], can the file matching conditions be extended to be able to find unlocked files?

View file

@ -0,0 +1 @@
If running the clean/smudge filter once per file is a [[bottleneck|forum/Adding_files_to_git__58___Very_long___34__recording_state_in_git__34___phase]], might it speed things up to split them off into something more lightweight than the full git-annex binary?

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="Ilya_Shlyakhter"
avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
subject="git-annex REST API"
date="2019-09-18T22:55:06Z"
content="""
Even better would be if git-annex had a \"server\" mode where one git-annex process runs and exposes a REST API for invoking git-annex commands (cf. [[todo/universal_batch_mode]]; something like [this](https://cromwell.readthedocs.io/en/stable/api/RESTAPI/)). Then the clean and smudge filters could be lightweight scripts that just call curl.
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="Ilya_Shlyakhter"
avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
subject="comment 5"
date="2019-09-18T21:30:46Z"
content="""
One more option is to add the paths for missing unlocked files to .git/info/exclude .
"""]]