Commit graph

33799 commits

Author SHA1 Message Date
guzik.sergey@9391b6c15e4938a539e36fbe5bab71df07111d2e
92013d86f6 2019-01-11 10:40:22 +00:00
6yearold@36d59212c29d2959d6532d6db7928f01541bcf83
a472cd3488 Added a comment 2019-01-11 06:28:32 +00:00
alogic0@d8fbd9f5b547237a650aa1d5605c2d3592496916
d2dd0a2b52 Added a comment: it's fixed 2019-01-11 00:54:02 +00:00
Joey Hess
c7333be02d
devblog 2019-01-10 17:24:51 -04:00
Joey Hess
ed8d9a29fe
add missing case 2019-01-10 17:17:37 -04:00
Joey Hess
2eadb6cd68
convert transitions.log to attoparsec and bytestring-builder
Not likely to be any speed gain here, but this completes porting every
log file over.

And, it let me get rid of code copied from ghc and modified, so
simplifying the licensing.
2019-01-10 17:13:30 -04:00
Joey Hess
591e4b145f
convert old uuid-based log parsers to attoparsec
This preserves the workaround for the old bug that caused NoUUID items
to be stored in the log, prefixing log lines with " ". It's now handled
implicitly, by using takeWhile1 (/= ' ') to get the uuid.

There is a behavior change from the old parser, which split the value
into words and then recombined it. That meant that "foo  bar" and "foo\tbar"
came out as "foo bar". That behavior was not documented, and seems
surprising; it meant that after a git-annex describe here "foo  bar",
you wouldn't get that same string back out when git-annex displayed repo
descriptions.

Otoh, some other parsers relied on the old behavior, and the attoparsec
rewrites had to deal with the issue themselves...

For group.log, there are some edge cases around the user providing a
group name with a leading or trailing space. The old parser would ignore
such excess whitespace. The new parser does too, because the alternative
is to refuse to parse something like " group1  group2 " due to excess
whitespace, which would be even more confusing behavior.

The only git-annex branch log file that is not converted to attoparsec
and bytestring-builder now is transitions.log.
2019-01-10 16:34:20 -04:00
Joey Hess
66603d6f75
attoparsec parsers for all new-format uuid-based logs
There should be some speed gains here, especially for chunk and remote
state logs, which are queried once per key.

Now only old-format uuid-based logs still need to be converted to attoparsec.
2019-01-10 13:30:36 -04:00
Joey Hess
7e54c215b4
Remove redundant endOfInput
parseLogLines consumes till endOfInput
2019-01-10 12:42:59 -04:00
Joey Hess
6f66b53a30
newtype Group to ByteString
This may speed up queries for things in groups, due to Eq and Ord being faster.
2019-01-09 15:05:49 -04:00
Joey Hess
3f7fe1d325
Merge branch 'master' of ssh://git-annex.branchable.com 2019-01-09 14:22:35 -04:00
Joey Hess
1928b82867
marginally faster VectorClock Builder
show of a POSIXTime is 7-bit ascii, so no need to use the filesystem
encoding on it
2019-01-09 14:17:00 -04:00
Joey Hess
232b1a08f3
simplification now that all logs use Builder 2019-01-09 14:10:05 -04:00
Joey Hess
2fef43dd71
convert all per-uuid log files to use Builder
Mostly didn't push the ByteStrings down very deep, but all of these log
files are not written to frequently at all, so slight remaining
innefficiency doesn't matter.

In Logs.UUID, removed the fixBadUUID code that cleaned up after a bug in
git-annex versions 3.20111105-3.20111110. In the unlikely event that a repo was
last touched by that ancient git-annex version, the descriptions of remotes
would appear missing when used with this version of git-annex. That is such minor
breakage, and so unlikely to still be a problem for any repos, that it was not
worth forward-porting that code to ByteString.
2019-01-09 14:00:35 -04:00
Joey Hess
de4980ef85
simplify Show instance by deriving 2019-01-09 13:13:31 -04:00
Joey Hess
11f4d993b5
fix format of show instance
It's only used to show test suite failures..
2019-01-09 13:12:25 -04:00
Joey Hess
2d46038754
converting more log files to use Builder
Probably not any particular speedup in this, since most of these logs
are not written to often. Possibly chunk log writing is sped up, but
writes to chunk logs are interleaved with expensive data transfers to
remotes, so unlikely to be a noticiable speedup.
2019-01-09 13:06:37 -04:00
duncan_bayne
74682ffaad Added a comment: Known issue 2019-01-09 09:17:20 +00:00
duncan_bayne
3126dfd84d Added a comment: Bug raised 2019-01-08 21:13:38 +00:00
Ilya_Shlyakhter
15fddad749 added suggestion to support MD5E keys that omit file size, and/or support attaching checksums to URL/WORM keys in metadata 2019-01-08 20:12:38 +00:00
Ilya_Shlyakhter
921c9f4e72 posted a question about the "here" remote 2019-01-08 19:47:05 +00:00
aaron@784fd85903b38afc8a44d117ba95e3dcce0bd230
42b8a23be4 Format post with code blocks for readability 2019-01-07 21:54:10 +00:00
Joey Hess
5500cbbc30
Merge branch 'master' of ssh://git-annex.branchable.com 2019-01-07 16:36:53 -04:00
Joey Hess
95a966666a
devblog 2019-01-07 16:36:34 -04:00
aaron@784fd85903b38afc8a44d117ba95e3dcce0bd230
aed4ceda8c sign my name at the end 2019-01-07 20:19:55 +00:00
aaron@784fd85903b38afc8a44d117ba95e3dcce0bd230
a826c97d52 Ask question about progress info 2019-01-07 20:17:39 +00:00
Joey Hess
4e89f008fa
remove unused import 2019-01-07 16:15:13 -04:00
https://christian.amsuess.com/chrysn
be273881f3 new forum entry 2019-01-07 19:51:29 +00:00
Joey Hess
cb375977a6
follow-on changes from MetaData type changes
Including writing and parsing the metadata log files with
bytestring-builder and attoparsec.
2019-01-07 15:51:05 -04:00
Joey Hess
16c798b5ef
switch MetaValue to ByteString and MetaField to Text
MetaField was already limited to alphanumerics, so it makes sense to use
Text for it.

Note that technically a UUID can contain invalid UTF-8, and so
remoteMetaDataPrefix's use of T.pack . fromUUID could replace non-UTF8
values with '?' or whatever. In practice, a UUID is usually also text,
I only kept open the possibility of it containing invalid UTF-8 to avoid
breaking parsing of strange UUIDs in git-annex branch files. So, I
decided to let this edge case slip by.

Have not updated the rest of the code base yet for this change, as the
change took 2.5 hours longer than I expected to get working properly.
2019-01-07 14:18:24 -04:00
Joey Hess
a80922a594
support for ByteStrings 2019-01-07 12:29:25 -04:00
anarcat
2f3559d970 one more SNAFU with webdav 2019-01-06 19:35:15 +00:00
anarcat
0a90a48612 fix typo and triage 2019-01-06 19:23:36 +00:00
timeless-ventricle
842e20a141 Added a comment 2019-01-06 12:24:50 +00:00
anarcat
cf04151ab6 Added a comment: thanks! 2019-01-06 04:12:03 +00:00
Chymera
5fe6f33b19 Added a comment 2019-01-06 01:25:10 +00:00
anarcat
9c6a644896 Added a comment: bump 2019-01-05 23:06:05 +00:00
Joey Hess
ccd75c60d2
correct ghc version number 2019-01-05 16:07:53 -04:00
Joey Hess
2e0e557e75
Support being built with ghc 8.0.1 (MonadFail)
Tested on an older ghc by enabling MonadFailDesugaring globally.

In TransferQueue, the lack of a MonadFail for STM exposed what would
normally be a bug in the pattern matching, although in this case an
earlier check that the queue was not empty avoided a pattern match
failure.
2019-01-05 11:55:15 -04:00
Joey Hess
6ec993252e
Merge branch 'master' of ssh://git-annex.branchable.com 2019-01-05 11:27:49 -04:00
Joey Hess
7a74f2497b
update 2019-01-05 11:27:04 -04:00
Joey Hess
5ba14b5095
build cleanrly when benchmark flag is not enabled 2019-01-05 08:09:28 -04:00
Joey Hess
fc3fd0cfe0
do union merge on bytestrings
My concern with using bytestring for this is the file needs to be split
into lines, and the encoding is not known. It's safe to split a utf-8
encoded file on the \n byte; only newlines get encoded to that byte in utf-8.
And this code already assumes utf-8 or ascii encoding, because it used
the filesystem encoding.
2019-01-05 08:06:47 -04:00
Chymera
ff668bba28 Added a comment 2019-01-04 23:05:43 +00:00
Chymera
7b933afadb Added a comment 2019-01-04 21:08:54 +00:00
Chymera
efe755d4ae 2019-01-04 21:02:16 +00:00
Joey Hess
9cf9ef5077
Merge branch 'master' of ssh://git-annex.branchable.com 2019-01-04 15:12:02 -04:00
Joey Hess
c73698fb33
devblog 2019-01-04 15:11:16 -04:00
pthomasdelaney@9b04608ad7e837fde64ab60a285a7b7254b5bb26
d5c4100da3 2019-01-04 18:23:45 +00:00
geoffrey.jost
fbad26da1e Updated Densho description , formerly "Japanese American Legacy Project", and total collection size. 2019-01-04 17:55:54 +00:00