Support eg `git-annex info --size-history=30d` which would display
the combined size of all repositories every 30 days throughout the history
of the git-annex branch. This would allow graphing, analysis, etc of repo
growth patterns.

Also, `git-annex info somerepo --size-history=30d` would display the size
of only the selected repository.

Maybe also a way to get the size of each repository plus total size in a
single line of output?

----

Implementation of this is fairly subtle. My abandoned first try just went
through `git log` and updated counters as the location logs were updated.
That resulted in bad numbers. (The size went negative eventually in fact!)
The problem is that the git-annex branch is often updated both locally and
on a remote, eg when copying a file to a remote. And that results in 2
changes to the git-annex branch that both record the same data. So it gets
counted twice by my naive implementation.

I think it is not possible for an accumulation based approach to work in
constant memory and fast. In the worst case, there is a fork of the branch
that diverges hugely over a long period of time. So that divergence either
needs to be buffered in memory, or recalculated repeatedly.

What I think needs to be done is use `git log --reverse --date-order git-annex`.
Feed the changed annex log file refs into catObjectStream to get the log
files. (Or use --patch and parse the diff to extract log file lines, 
might be faster?) Parse the log files, and update a simple data structure:

    Map Key [UUIDPointer]

Where UUIDPointer is a number that points to the UUID in a Map. This
avoids storing copies of the uuids in the map.

This is essentially union merging all forks of the git-annex branch at
each commit, but far faster and in memory. Since union merging a git-annex
branch can be done at any point and always results in a consistent view of
the data, this will be consistent as well.

And when updating the data structure, then it can update a counter when
something changed, and avoid updating it when a redundant log was logged.

This approach will use an amount of memory that scales with
the number of keys and numbers of copies. I mocked it up using my big
repository. Storing every key in it in such a map, with 64 UUIDPointers
in the list (many more than the usual number of copies) took 2 gb of
memory. Which is a lot but also most users have that much if necessary.
With a more usual 5 copies, memory use was only 0.5 gb. So I think this is
an acceptable exception to git-annex's desire to use a constant amount of
memory.

(I considered a bloom filter, but a false positive would wreck the
statistics. An in-memory sqlite db might be more efficient, but probably
not worth the bother.)

[[!tag confirmed]]

--[[Joey]] 

> [[done]]