Commit graph

17322 commits

Author SHA1 Message Date
https://www.google.com/accounts/o8/id?id=AItOawkLdR1fuu5aEz3s9VKTBKVMize_SmeNRJM
f11ccc16a6 Added a comment: Past the SHA issues 2014-01-20 23:14:53 +00:00
Joey Hess
df66e15555 devblog 2014-01-20 17:47:17 -04:00
Joey Hess
3159da2693 Add and use numcopiesneeded preferred content expression.
* Add numcopiesneeded preferred content expression.
* Client, transfer, incremental backup, and archive repositories
  now want to get content that does not yet have enough copies.

This means the asssistant will make copies of files that don't yet
meet the configured numcopies, even to places that would not normally want
the file.

For example, if numcopies is 4, and there are 2 client repos and
2 transfer repos, and 2 removable backup drives, the file will be sent
to both transfer repos in order to make 4 copies. Once a removable drive
get a copy of the file, it will be dropped from one transfer repo or the
other (but not both).

Another example, numcopies is 3 and there is a client that has a backup
removable drive and two small archive repos. Normally once one of the small
archives has a file, it will not be put into the other one. But, to satisfy
numcopies, the assistant will duplicate it into the other small archive
too, if the backup repo is not available to receive the file.

I notice that these examples are fairly unlikely setups .. the old behavior
was not too bad, but it's nice to finally have it really correct.

.. Almost. I have skipped checking the annex.numcopies .gitattributes
out of fear it will be too slow.

This commit was sponsored by Florian Schlegel.
2014-01-20 17:35:29 -04:00
Joey Hess
5ddbd24a1c stability analysis 2014-01-20 17:11:02 -04:00
Joey Hess
b7828ec22e Merge branch 'master' of ssh://git-annex.branchable.com 2014-01-20 16:48:31 -04:00
Joey Hess
d66535f065 global numcopies setting
* numcopies: New command, sets global numcopies value that is seen by all
  clones of a repository.
* The annex.numcopies git config setting is deprecated. Once the numcopies
  command is used to set the global number of copies, any annex.numcopies
  git configs will be ignored.
* assistant: Make the prefs page set the global numcopies.

This global numcopies setting is needed to let preferred content
expressions operate on numcopies.

It's also convenient, because typically if you want git-annex to preserve N
copies of files in a repo, you want it to do that no matter which repo it's
running in. Making it global avoids needing to warn the user about gotchas
involving inconsistent annex.numcopies settings.
(See changes to doc/numcopies.mdwn.)

Added a new variety of git-annex branch log file, that holds only 1 value.
Will probably be useful for other stuff later.

This commit was sponsored by Nicolas Pouillard.
2014-01-20 16:47:56 -04:00
https://id.koumbit.net/anarcat
d979f2fbdf a separate sync daemon has many advantages indeed 2014-01-20 18:41:59 +00:00
Joey Hess
ecd4c35d7e promote stm dependency, since Remote.External needs it 2014-01-20 14:32:06 -04:00
Joey Hess
514a4efb7a Merge branch 'master' of ssh://git-annex.branchable.com 2014-01-20 14:30:39 -04:00
Joey Hess
e7f8c1911a design for preferred content numcopies check 2014-01-20 14:28:33 -04:00
Joey Hess
5130bfdff3 export cleanup 2014-01-20 13:37:13 -04:00
Joey Hess
73c420ffcf much better command action handling for sync --content 2014-01-20 13:31:03 -04:00
Joey Hess
cfa6865056 wontfix 2014-01-20 12:36:19 -04:00
http://joeyh.name/
dde13f775a Added a comment 2014-01-20 16:33:28 +00:00
http://joeyh.name/
4a0fbbd472 Added a comment 2014-01-20 16:28:50 +00:00
https://www.google.com/accounts/o8/id?id=AItOawm_YXzEdPHzbSGVwtmTR7g1BqDtTnIBB5s
1ba8ea0c8a Added a comment: Chunk it 2014-01-20 16:22:21 +00:00
https://www.google.com/accounts/o8/id?id=AItOawkqUg2sSO_pOdi6M7GU86Hsyv5-r33baLs
92071e2170 2014-01-20 10:42:06 +00:00
Joey Hess
34c8af74ba fix inversion of control in CommandSeek (no behavior changes)
I've been disliking how the command seek actions were written for some
time, with their inversion of control and ugly workarounds.

The last straw to fix it was sync --content, which didn't fit the
Annex [CommandStart] interface well at all. I have not yet made it take
advantage of the changed interface though.

The crucial change, and probably why I didn't do it this way from the
beginning, is to make each CommandStart action be run with exceptions
caught, and if it fails, increment a failure counter in annex state.
So I finally remove the very first code I wrote for git-annex, which
was before I had exception handling in the Annex monad, and so ran outside
that monad, passing state explicitly as it ran each CommandStart action.

This was a real slog from 1 to 5 am.

Test suite passes.

Memory usage is lower than before, sometimes by a couple of megabytes, and
remains constant, even when running in a large repo, and even when
repeatedly failing and incrementing the error counter. So no accidental
laziness space leaks.

Wall clock speed is identical, even in large repos.

This commit was sponsored by an anonymous bitcoiner.
2014-01-20 04:57:36 -04:00
Richard Hartmann
7845e09320 Fix misleading description 2014-01-20 01:13:41 +01:00
https://me.yahoo.com/a/FHnTlSBo1eCGJRwueeKeB6.RCaPbGMPr5jxx8A--#ce0d8
6633af6736 Added a comment 2014-01-19 23:56:18 +00:00
Joey Hess
df5e2e3d65 gathd 2014-01-19 19:19:19 -04:00
Joey Hess
12302cd05d Merge branch 'master' of ssh://git-annex.branchable.com 2014-01-19 18:48:17 -04:00
http://joeyh.name/
348a270cfe Added a comment 2014-01-19 22:43:11 +00:00
Joey Hess
b0044a4a0c devblog 2014-01-19 18:12:24 -04:00
Joey Hess
e3625e3d89 include information about remotes just uloaded to when calling handleDropsFrom 2014-01-19 18:11:47 -04:00
Joey Hess
7e6e018408 followups and closures 2014-01-19 17:56:24 -04:00
Joey Hess
b6ba0bd556 sync --content: New option that makes the content of annexed files be transferred.
Similar to the assistant, this honors any configured preferred content
expressions.

I am not entirely happpy with the implementation. It would be nicer if
the seek function returned a list of actions which included the individual
file gets and copies and drops, rather than the current list of calls to
syncContent. This would allow getting rid of the somewhat reundant display
of "sync file [ok|failed]" after the get/put display.

But, do that, withFilesInGit would need to somehow be able to construct
such a mixed action list. And it would be less efficient than the current
implementation, which is able to reuse several values between eg get and
drop.

Note that currently this does not try to satisfy numcopies when
getting/putting files (numcopies are of course checked when dropping
files!) This makes it like the assistant, and unlike get --auto
and copy --auto, which do duplicate files when numcopies is not yet
satisfied. I don't know if this is the right decision; it only seemed to
make sense to have this parallel the assistant as far as possible to start
with, since I know the assistant works.

This commit was sponsored by Øyvind Andersen Holm.
2014-01-19 17:49:54 -04:00
Richard Hartmann
83562fccbc Typo 2014-01-19 22:41:36 +01:00
https://www.google.com/accounts/o8/id?id=AItOawniCRkhl_W87gOK5eElfsef3FoUsUFpAr4
eee3540e2d create 2014-01-19 19:17:55 +00:00
https://www.google.com/accounts/o8/id?id=AItOawkq2cjugiSvKWiWmcah3CPBqviQV_cin9I
f6c45f8ba4 Added a comment: The world of Mac 2014-01-19 14:13:55 +00:00
https://arthur.lutz.im/
8c027c5ced added bug about .local adresses 2014-01-19 10:53:10 +00:00
http://joeyh.name/
c5c339e87e Added a comment 2014-01-18 22:39:37 +00:00
https://me.yahoo.com/a/FHnTlSBo1eCGJRwueeKeB6.RCaPbGMPr5jxx8A--#ce0d8
215c02f0a5 2014-01-18 22:08:15 +00:00
Joey Hess
1729205dba devblog 2014-01-18 17:26:34 -04:00
Yaroslav Halchenko
bae8392ea6 seed .mailmap file for more sensible git shortlog 2014-01-18 15:50:57 -04:00
Joey Hess
85185b8f50 Allow --all to be mixed with matching options like --copies and --in (but not --include and --exclude). 2014-01-18 14:58:56 -04:00
Joey Hess
8ce515ffe4 improve matcher data type to allow matching Keys, instead of just files (no behavior changes) 2014-01-18 14:51:55 -04:00
Joey Hess
a135bbd5a2 note that --all can't be mixed with eg --copies 2014-01-18 13:52:35 -04:00
Joey Hess
939eb666fe clarify sync 2014-01-18 13:26:47 -04:00
Joey Hess
2b1a7895fc list: Fix specifying of files to list. 2014-01-18 13:25:06 -04:00
Joey Hess
163aee374a improve output when there are no known special remotes 2014-01-18 13:21:12 -04:00
Yaroslav Halchenko
0bf41b335b Minor git-annex.mdwn tune ups (trailing spaces, typos, more consistency in tense)
Conflicts:
	doc/git-annex.mdwn -- I have managed to work on an old copy, so overlapped a bit
2014-01-18 13:06:15 -04:00
Joey Hess
d793f20886 whereis supports --json, add option 2014-01-18 13:05:56 -04:00
Joey Hess
456c9eedb0 status: Support --json. 2014-01-18 12:05:10 -04:00
Joey Hess
babd03d252 Merge branch 'master' of ssh://git-annex.branchable.com 2014-01-18 12:01:40 -04:00
Joey Hess
1d90e72e91 Remove --json option from commands not supporting it. 2014-01-18 11:54:43 -04:00
Joey Hess
dce9faf6e4 close 2014-01-18 11:44:58 -04:00
http://joeyh.name/
386ce9395d Added a comment 2014-01-18 15:43:00 +00:00
Joey Hess
099c650292 avoid trying to find minimum element of empty map 2014-01-18 11:38:01 -04:00
https://www.google.com/accounts/o8/id?id=AItOawnx8kHW66N3BqmkVpgtXDlYMvr8TJ5VvfY
9135cb7b7c 2014-01-18 06:13:20 +00:00