CI to automate build of cargo lockfiles on different Alpine releases for git-annex aports https://gitlab.alpinelinux.org/alpine/aports/-/tree/master/community/git-annex
Find a file
Joey Hess 1be4d281d6 Better sanitization of problem characters when generating URL and WORM keys.
FAT has a lot of characters it does not allow in filenames, like ? and *
It's probably the worst offender, but other filesystems also have
limitiations.

In 2011, I made keyFile escape : to handle FAT, but missed the other
characters. It also turns out that when I did that, I was also living
dangerously; any existing keys that contained a : had their object
location change. Oops.

So, adding new characters to escape to keyFile is out. Well, it would be
possible to make keyFile behave differently on a per-filesystem basis, but
this would be a real nightmare to get right. Consider that a rsync special
remote uses keyFile to determine the filenames to use, and we don't know
the underlying filesystem on the rsync server..

Instead, I have gone for a solution that is backwards compatable and
simple. Its only downside is that already generated URL and WORM keys
might not be able to be stored on FAT or some other filesystem that
dislikes a character used in the key. (In this case, the user can just
migrate the problem keys to a checksumming backend. If this became a big
problem, fsck could be made to detect these and suggest a migration.)

Going forward, new keys that are created will escape all characters that
are likely to cause problems. And if some filesystem comes along that's
even worse than FAT (seems unlikely, but here it is 2013, and people are
still using FAT!), additional characters can be added to the set that are
escaped without difficulty.

(Also, made WORM limit the part of the filename that is embedded in the key,
to deal with filesystem filename length limits. This could have already
been a problem, but is more likely now, since the escaping of the filename
can make it longer.)

This commit was sponsored by Ian Downes
2013-10-05 15:01:49 -04:00
Annex Automatically and safely detect and recover from dangling .git/annex/index.lock files, which would prevent git from committing to the git-annex branch, eg after a crash. 2013-10-03 15:43:08 -04:00
Assistant watcher: Detect at startup time when there is a stale .git/lock, and remove it so it does not interfere with the automatic commits of changed files. 2013-10-03 16:57:21 -04:00
Backend Better sanitization of problem characters when generating URL and WORM keys. 2013-10-05 15:01:49 -04:00
Build oops 2013-09-22 21:47:24 -04:00
Command honor fileNameLengthLimit for quvi 2013-10-05 13:32:42 -04:00
Common Use lower case hash directories for storing files on crippled filesystems, same as is already done for bare repositories. 2013-04-04 15:46:33 -04:00
Config hlint 2013-09-25 03:09:06 -04:00
debian Better sanitization of problem characters when generating URL and WORM keys. 2013-10-05 15:01:49 -04:00
doc Better sanitization of problem characters when generating URL and WORM keys. 2013-10-05 15:01:49 -04:00
Git fix windows build 2013-10-02 20:26:00 -04:00
GitAnnex Send a git-annex user-agent when downloading urls. 2013-09-28 14:35:21 -04:00
Logs remove *>=> and >=*> ; use <$$> instead 2013-09-27 19:58:48 -04:00
Messages hlint 2013-04-03 03:52:41 -04:00
Remote typoe and debug info 2013-10-01 19:10:45 -04:00
standalone future work 2013-09-23 17:39:53 -04:00
static animate syncing and other activity icons 2013-04-08 22:54:02 -04:00
templates blind enabling of existing ssh and ssh gcrypt repos 2013-10-02 15:54:32 -04:00
Types enabling rsync.net gcrypt repos 2013-09-26 23:47:30 -04:00
Upgrade hlint 2013-09-25 23:19:01 -04:00
Utility watcher: Detect at startup time when there is a stale .git/lock, and remove it so it does not interfere with the automatic commits of changed files. 2013-10-03 16:57:21 -04:00
.ghci workaround for ghci fragility in reusing objects compiled with ghc 2012-10-20 16:20:42 -04:00
.gitattributes decruft 2011-06-22 18:56:08 -04:00
.gitignore re-enable .o and .hi exclusion 2013-06-26 11:43:45 -04:00
Annex.hs Send a git-annex user-agent when downloading urls. 2013-09-28 14:35:21 -04:00
Assistant.hs assistant: Work around git-cat-file's not reloading the index after files are staged. 2013-05-25 00:37:41 -04:00
Backend.hs Added SKEIN256 and SKEIN512 backends 2013-10-01 20:34:36 -04:00
BuildFlags.hs allow building w/o cryptohash 2013-10-03 12:33:38 -04:00
CHANGELOG symlinks 2010-10-27 15:14:59 -04:00
Checks.hs direct, indirect: Refuse to do anything when the assistant or git-annex watch daemon is running. 2013-04-22 16:41:46 -04:00
CmdLine.hs missed some __WINDOWS__ defines 2013-08-04 13:07:55 -04:00
Command.hs status: In local mode, displays information about variance from configured numcopies levels. 2013-09-15 19:10:38 -04:00
Common.hs clean up some ugly code 2013-09-27 19:52:36 -04:00
Config.hs Support hot-swapping of removable drives containing gcrypt repositories. 2013-09-12 15:54:35 -04:00
configure.hs Rewrote free disk space checking code 2012-03-22 17:32:47 -04:00
COPYRIGHT Add link COPYRIGHT -> debian/copyright; add COPYRIGHT to sdist. 2012-06-11 23:27:00 -07:00
Creds.hs fix windows build 2013-10-02 20:33:13 -04:00
Crypto.hs remove *>=> and >=*> ; use <$$> instead 2013-09-27 19:58:48 -04:00
Fields.hs safe recv-key in direct mode 2013-01-11 16:03:45 -04:00
ghci brought back fast ghci 2013-02-27 21:38:01 -04:00
git-annex.cabal increase base upper bound 2013-10-05 13:15:00 -04:00
git-annex.hs try building test suite on windows 2013-05-17 16:45:44 -04:00
git-union-merge.hs factor out nukeFile 2012-06-06 13:13:13 -04:00
Git.hs squash compiler warnings on Windows 2013-08-04 13:18:05 -04:00
GitAnnex.hs rename remotes to list 2013-09-19 14:16:55 -04:00
GitAnnexShell.hs git-annex-shell gcryptsetup command 2013-10-01 17:20:51 -04:00
Init.hs missed some __WINDOWS__ defines 2013-08-04 13:07:55 -04:00
INSTALL symlinks 2010-10-27 15:14:59 -04:00
Limit.hs squash compiler warnings on Windows 2013-08-04 13:18:05 -04:00
Locations.hs Better sanitization of problem characters when generating URL and WORM keys. 2013-10-05 15:01:49 -04:00
Logs.hs refactor git-annex branch log filename code into central location 2013-08-29 19:13:00 -04:00
Makefile successfully builds (except XMPP) 2013-09-22 19:13:31 +01:00
Messages.hs annex.debug can now be set to enable debug logging by default. The webapp's debugging check box does this. 2013-06-17 20:41:27 -04:00
NEWS update NEWS 2012-01-19 15:27:39 -04:00
Option.hs annex.debug can now be set to enable debug logging by default. The webapp's debugging check box does this. 2013-06-17 20:41:27 -04:00
README add README 2011-05-13 19:20:54 -04:00
Remote.hs webapp: Fixed a bug where when a new remote is added, one file may fail to sync to or from it 2013-09-29 14:51:49 -04:00
Seek.hs import: Preserve top-level directory structure. 2013-09-25 13:16:55 -04:00
Setup.hs split out library 2013-04-17 12:02:44 -04:00
Test.hs ensure that hash representations don't change in future 2013-10-01 21:11:47 -04:00
Types.hs moved AssociatedFile definition 2013-07-04 02:36:02 -04:00
Upgrade.hs missed some __WINDOWS__ defines 2013-08-04 13:07:55 -04:00
Usage.hs content: New command line way to view and configure a repository's preferred content settings. 2013-05-25 12:44:58 -04:00

git-annex allows managing files with git, without checking the file
contents into git. While that may seem paradoxical, it is useful when
dealing with files larger than git can currently easily handle, whether due
to limitations in memory, checksumming time, or disk space.

For documentation, see doc/ or <http://git-annex.branchable.com/>