Merge branch 'master' of ssh://git-annex.branchable.com

This commit is contained in:
Joey Hess 2018-10-22 13:01:06 -04:00
commit d6b2468b4c
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
4 changed files with 146 additions and 0 deletions

View file

@ -0,0 +1,90 @@
### Please describe the problem.
After following the (new) Android installation instructions, git-annex is not on `$PATH`, therefore literal `git annex` commands fail.
I'm sure this is trivially fixed by enhancing the install script so that it modifies `$PATH` in the termux profile, or similar.
### What steps will reproduce the problem?
1. Follow the instructions at https://git-annex.branchable.com/Android/ up to `sh git-annex-install`
2. Observe that the install succeeds
3. Try the next step: `git annex webapp`
4. Observe the error `git: 'annex' is not a git command. See 'git --help'.`
5. Observe that it works when using `./git-annex.linux/git-annex` instead
### What version of git-annex are you using? On what operating system?
6.20180927-gc5b6c55af on Android 8.1.0 (OxygenOS 5.1.5) on a OnePlus 5T (A5010).
### Please provide any additional information below.
[[!format sh """
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
% sh -x git-annex-install [20/4719]
+ set -e
+ uname -m
+ arch=arm64
+ url=https://downloads.kitenet.net/git-annex/linux/current/git-annex-standalone-arm64.tar.gz
+ echo Installing dependencies with termux pkg manager...
Installing dependencies with termux pkg manager...
+ pkg install git wget tar coreutils proot
Hit:1 https://termux.net stable InRelease
Reading package lists... Done
Building dependency tree
Reading state information... Done
19 packages can be upgraded. Run 'apt list --upgradable' to see them.
Reading package lists... Done
Building dependency tree
Reading state information... Done
coreutils is already the newest version (8.30-1).
git is already the newest version (2.19.1).
proot is already the newest version (5.1.107-18).
tar is already the newest version (1.30-1).
wget is already the newest version (1.19.5-1).
0 upgraded, 0 newly installed, 0 to remove and 19 not upgraded.
+ echo Downloading git-annex...
Downloading git-annex...
+ cd
+ wget -O- https://downloads.kitenet.net/git-annex/linux/current/git-annex-standalone-arm64.tar.gz
+ tar zx
--2018-10-21 23:41:37-- https://downloads.kitenet.net/git-annex/linux/current/git-annex-standalone-arm64.tar.gz
Resolving downloads.kitenet.net... 66.228.36.95, 2600:3c03::f03c:91ff:fe73:b0d2
Connecting to downloads.kitenet.net|66.228.36.95|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 64590742 (62M) [application/x-gzip]
Saving to: STDOUT
- 100%[==================================================================>] 61.60M 2.51MB/s in 81s
2018-10-21 23:42:59 (778 KB/s) - written to stdout [64590742/64590742]
+ git-annex.linux/git-annex version
Running on Android.. Adding git-annex to PATH for you, and tuning for optimal behavior.
git-annex version: 6.20180927-gc5b6c55af
build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser Mag$
cMime Feeds Testsuite
dependency versions: aws-0.19 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.2 feed-1.0.0.0 ghc-8.2.2 http-client-0.5.12 persistent-sqlite-2.8.1.$
torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_$
84E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224
BLAKE2B384E BLAKE2B384 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 $
HA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external
operating system: linux aarch64
supported repository versions: 3 5 6
upgrade supported from repository versions: 0 1 2 3 4 5
+ echo git-annex is successfully installed.
git-annex is successfully installed.
+ echo Now running termux-setup-storage, to let git-annex access system storage.
Now running termux-setup-storage, to let git-annex access system storage.
+ termux-setup-storage
+ echo Installation complete.
Installation complete.
% git annex webapp
git: 'annex' is not a git command. See 'git --help'.
# End of transcript or log.
"""]]
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
Yes, lots of luck over the years :-) This is the final little tweak required to get it working for me on Android.

View file

@ -0,0 +1,16 @@
This is a bug report about the "Starting at power on" section of the documentation in <http://git-annex.branchable.com/Android/>, which says:
>If you install the Termux:Boot app, git-annex will automatically be automatically started when your Android device powers on. It will run in the background in whatever repositories you have set up in the webapp.
>(Or have listed in (or listed in ~/.config/git-annex/autostart)
This gives rise to several questions:
Will it really be automatically started without any additional configuration from the user? That's what the first sentence appears to claim, but it appears to contradict <https://wiki.termux.com/wiki/Termux:Boot#Usage> which suggests that manual configuration of `~/.termux/boot` is required.
But if it's really true, exactly what `git-annex` command will be automatically started? Since running with no arguments simply outputs usage text.
Or if manual configuration of `~/.termux/boot` *is* required, what exactly should that look like? Should it be `git annex assistant --autostart`, as per [[forum/Autostart_the_assistant]]?
What's the format of the contents of `~/.config/git-annex/autostart`?
Finally, the last sentence has unbalanced parentheses, and doesn't parse correctly.

View file

@ -0,0 +1,35 @@
I recently discovered (thanks to Paul Wise) the [Meow hash][]. The
TL;DR: is that it's a fast non-crypto hash which might be useful for
git-annex. Here's their intro, quoted from the website:
[Meow hash]: https://mollyrocket.com/meowhash
> The Meow hash is a high-speed hash function named after the character
> Meow in [Meow the Infinite][]. We developed the hash function at
> [Molly Rocket][] for use in the asset pipeline of [1935][].
>
> Because we have to process hundreds of gigabytes of art assets to build
> game packages, we wanted a fast, non-cryptographic hash for use in
> change detection and deduplication. We had been using a cryptographic
> hash ([SHA-1][]), but it was
> unnecessarily slowing things down.
>
> To our surprise, we found a lack of published, well-optimized,
> large-data hash functions. Most hash work seems to focus on small input
> sizes (for things like dictionary lookup) or on cryptographic quality.
> We wanted the fastest possible hash that would be collision-free in
> practice (like SHA-1 was), and we didn't need any cryptograhic security.
>
> We ended up creating Meow to fill this niche.
[1935]: https://molly1935.com/
[Molly Rocket]: https://mollyrocket.com/
[Meow the Infinite]: https://meowtheinfinite.com/
[SHA-1]: https://en.m.wikipedia.org/wiki/SHA-1
I don't an immediate use case for this right now, but I think it could
be useful to speed up checks on larger files. The license is a
*little* weird but seems close enough to a BSD to be acceptable.
I know it might sound like a conflict of interest, but I *swear* I am
not bringing this up only as a oblique feline reference. ;) -- [[anarcat]]

View file

@ -0,0 +1,5 @@
It would be good if one could define custom external [[backends]], the way one can define external custom remotes. This would solve [[todo/consider_meow_backend]] but also have other uses. For instance, sometimes files contain details irrelevant to the file's semantics (e.g. comments), but that change the file's checksum; with a custom backend, one could "canonicalize" a file before computing the checksum.
@joey pointed out a potential problem: "needing to deal with the backend being missing or failing to work could have wide repurcussions in the code base." I wonder if there are ways around that. Suppose you specified a default backend to use in case a custom one was unavailable? Then you could always compute a key from a file, even if it's not in the right backend. And once a key is stored in git-annex, most of git-annex treats the key as just a string. If the custom backend supports checksum verification, without the backend's implementation, keys from that backend would be treated like WORM/URL keys that do not support checksum checking.
Thoughts?