Commit graph

334 commits

Author SHA1 Message Date
Joey Hess
d41849bc23
support commit.gpgsign
Support users who have set commit.gpgsign, by disabling gpg signatures for
git-annex branch commits and commits made by the assistant.

The thinking here is that a user sets commit.gpgsign intending the commits
that they manually initiate to be gpg signed. But not commits made in the
background, whether by a deamon or implicitly to the git-annex branch.
gpg signing those would be at best a waste of CPU and at worst would fail,
or flood the user with gpg passphrase prompts, or put their signature on
changes they did not directly do.

See Debian bug #753720.

Also makes all commits done by git-annex go through a few central control
points, to make such changes easier in future.

Also disables commit.gpgsign in the test suite.

This commit was sponsored by Antoine Boegli.
2014-07-04 11:53:51 -04:00
Joey Hess
501cc8623a assistant: Fix one-way assistant->assistant sync in direct mode.
When in direct mode, update the master branch after committing to the
annex/direct/master branch. Also, update the synced/master branch.

This fixes a topology A->B where both A and B are in direct mode and
running the assistant, and a change is made to B. Before this fix, A pulled
the changes from B, but since they were only on the annex/direct/master
branch, it did not merge them.

Note that I considered making the assistant merge the
remotes/B/annex/direct/master, but decided to keep it simple and only merge
the sync branches as before.
2014-06-16 11:32:13 -04:00
Joey Hess
e880d0d22c replace (Key, Backend) with Key
Only fsck and reinject and the test suite used the Backend, and they can
look it up as needed from the Key. This simplifies the code and also speeds
it up.

There is a small behavior change here. Before, all commands would warn when
acting on an annexed file with an unknown backend. Now, only fsck and
reinject show that warning.
2014-04-17 18:03:39 -04:00
Joey Hess
15917ec1a8 sync, assistant, remotedaemon: Use ssh connection caching for git pushes and pulls.
For sync, saves 1 ssh connection per remote. For remotedaemon, the same
ssh connection that is already open to run git-annex-shell notifychanges
is reused to pull from the remote.

Only potential problem is that this also enables connection caching
when the assistant syncs with a ssh remote. Including the sync it does
when a network connection has just come up. In that case, cached ssh
connections are likely to be stale, and so using them would hang.
Until I'm sure such problems have been dealt with, this commit needs to
stay on the remotecontrol branch, and not be merged to master.

This commit was sponsored by Alexandre Dupas.
2014-04-12 15:59:34 -04:00
Joey Hess
4cc34973d9 copy --fast --to remote: Avoid printing anything for files that are already believed to be present on the remote. 2014-03-13 14:51:22 -04:00
Joey Hess
14d1e878ab sync: Automatically resolve merge conflict between and annexed file and a regular git file.
This is a new feature, it was not handled before, since it's a bit of an
edge case. However, it can be handled exactly the same as a file/dir
conflict, just leave the non-annexed item alone.

While implementing this, the core resolveMerge' function got a lot simpler
and clearer. Note especially that where before there was an asymetric call to
stagefromdirectmergedir, now graftin is called symmetrically in both cases.

And, in order to add that `graftin us`, the current branch needed to be
known (if there is no current branch, there cannot be a merge conflict).
This led to some cleanups of how autoMergeFrom behaved when there is no
current branch.

This commit was sponsored by Philippe Gauthier.
2014-03-04 19:35:55 -04:00
Joey Hess
99295f2c1d factor out Annex.AutoMerge from Command.Sync 2014-03-04 16:26:15 -04:00
Joey Hess
8496d8aa63
improved direct mode dir/file conflicted merge resultion, using tree grafting 2014-03-04 15:00:19 -04:00
Joey Hess
0afa7ae261 much improved test and real fix for FAT symlink loss on conflicted merge
I think that 751f496c11 didn't quite manage
to actually fix the bug, although I have not checked since its "fix" got
redone.

The test suite now actually checks the file staged in git is a symlink,
rather than relying on the bug casing a later sync failure. This seems a
more reliable way to detect it, and probably avoids a heisenbug in the test
suite.
2014-03-04 14:31:26 -04:00
Joey Hess
4a847cdc08 finish fixing direct mode merge bug involving unstaged local files
Added test cases for both ways this can happen, with a conflict involving a
file, or a directory.

Cleaned up resolveMerge to not touch the work tree in direct mode, which
turned out to be the only way to handle things.. And makes it much nicer.

Still need to run test suite on windows.
2014-03-04 02:03:15 -04:00
Joey Hess
6dc8bfad0b rename for clarity 2014-03-03 16:33:21 -04:00
Joey Hess
d6696dbc98 simplfy 2014-03-03 16:21:36 -04:00
Joey Hess
1192d98721 sync: Fix bug in direct mode that caused a file not checked into git to be deleted when merging with a remote that added a file by the same name. (Thanks, jkt) 2014-03-03 14:57:16 -04:00
Joey Hess
d0fce426c4 pre-commit-annex hook script to automatically extract metadata from lots of types of files
Using the extract(1) program to do the heavy lifting.

Decided to make git-annex run pre-commit-annex when committing. Since
git-annex pre-commit also runs it, it'll be run when git commit is run too,
via the pre-commit hook. This basically gives back the pre-commit hook
that git-annex took away. The implementation avoids repeatedly looking
for the hook script when the assistant is running and committing
repeatedly; only checks if the hook is available once.

To make the script simpler, made git-annex metadata -s field?=value
only set a field when it's not already got a value.

This commit was sponsored by bak.
2014-03-02 20:11:58 -04:00
Joey Hess
4e0be2792b remove Read instance for Ref
Removed instance, got it all to build using fromRef. (With a few things
that really need to show something using a ref for debugging stubbed out.)

Then added back Read instance, and made Logs.View use it for serialization.
This changes the view log format.
2014-02-19 01:19:57 -04:00
Joey Hess
556cfeb8f0
hlint to give autobuilder something to do 2014-02-11 13:11:49 -04:00
Joey Hess
751f496c11 add test case & fix conflict resolution bug on Windows & FAT
Fix bug in automatic merge conflict resolution code when used
on a filesystem not supporting symlinks, which resulted in it losing
track of the symlink bit of annexed files.

This was the underlying bug that was causing another test to fail,
which got worked around in 1c997fd08c.
I've chosen to keep 2 separate test cases since the old test case only
detected the problem accidentially.

Test suite passes on FAT & in windows, as well as on proper unix systems.

This commit was sponsored by Ellis Whitehead.
2014-02-04 17:24:12 -04:00
Joey Hess
fded408b44 sync --content: Drop files from remotes that don't have them after getting them.
Need to include the uuid of the local repo in the list of belived locations
of a key after getting it, in order for the drop from remote to include it
in the numcopies calculation.
2014-02-02 22:48:45 -04:00
Joey Hess
0de879e264 update function name in comment 2014-02-02 22:37:09 -04:00
Joey Hess
4676706756 sync --content: Reuse smart copy code from copy command, including handling and repairing out of date location tracking info. Closes: #737480 2014-02-02 19:57:22 -04:00
Joey Hess
a542781f6a remove some monkey faces 2014-02-01 17:14:38 -04:00
Joey Hess
15ac2ea4de sync --content: Re-pull from remotes after downloading content, since that can take a while and other changes may be pushed in the meantime. 2014-02-01 10:49:50 -04:00
Joey Hess
bbef0cddfd improve sync with xmpp and annex-ignore
* sync --content: Honor annex-ignore configuration.
* sync: Don't try to sync with xmpp remotes, which are only currently
  supported when using the assistant.
2014-02-01 10:33:55 -04:00
Joey Hess
cce69eee4d avoid using function named that conflicts with name used in newer version of process library 2014-01-29 13:44:53 -04:00
Joey Hess
86ffeb73d1 reorganize some files and imports 2014-01-26 16:25:55 -04:00
Joey Hess
3518c586cf fix transfers of key with no associated file
Several places assumed this would not happen, and when the AssociatedFile
was Nothing, did nothing.

As part of this, preferred content checks pass the Key around.

Note that checkMatcher is sometimes now called with Just Key and Just File.
It currently constructs a FileMatcher, ignoring the Key. However, if it
constructed a FileKeyMatcher, which contained both, then it might be
possible to speed up parts of Limit, which currently call the somewhat
expensive lookupFileKey to get the Key.

I have not made this optimisation yet, because I am not sure if the key is
always the same. Will need some significant checking to satisfy myself
that's the case..
2014-01-23 16:44:02 -04:00
Joey Hess
73c420ffcf much better command action handling for sync --content 2014-01-20 13:31:03 -04:00
Joey Hess
34c8af74ba fix inversion of control in CommandSeek (no behavior changes)
I've been disliking how the command seek actions were written for some
time, with their inversion of control and ugly workarounds.

The last straw to fix it was sync --content, which didn't fit the
Annex [CommandStart] interface well at all. I have not yet made it take
advantage of the changed interface though.

The crucial change, and probably why I didn't do it this way from the
beginning, is to make each CommandStart action be run with exceptions
caught, and if it fails, increment a failure counter in annex state.
So I finally remove the very first code I wrote for git-annex, which
was before I had exception handling in the Annex monad, and so ran outside
that monad, passing state explicitly as it ran each CommandStart action.

This was a real slog from 1 to 5 am.

Test suite passes.

Memory usage is lower than before, sometimes by a couple of megabytes, and
remains constant, even when running in a large repo, and even when
repeatedly failing and incrementing the error counter. So no accidental
laziness space leaks.

Wall clock speed is identical, even in large repos.

This commit was sponsored by an anonymous bitcoiner.
2014-01-20 04:57:36 -04:00
Joey Hess
e3625e3d89 include information about remotes just uloaded to when calling handleDropsFrom 2014-01-19 18:11:47 -04:00
Joey Hess
b6ba0bd556 sync --content: New option that makes the content of annexed files be transferred.
Similar to the assistant, this honors any configured preferred content
expressions.

I am not entirely happpy with the implementation. It would be nicer if
the seek function returned a list of actions which included the individual
file gets and copies and drops, rather than the current list of calls to
syncContent. This would allow getting rid of the somewhat reundant display
of "sync file [ok|failed]" after the get/put display.

But, do that, withFilesInGit would need to somehow be able to construct
such a mixed action list. And it would be less efficient than the current
implementation, which is able to reuse several values between eg get and
drop.

Note that currently this does not try to satisfy numcopies when
getting/putting files (numcopies are of course checked when dropping
files!) This makes it like the assistant, and unlike get --auto
and copy --auto, which do duplicate files when numcopies is not yet
satisfied. I don't know if this is the right decision; it only seemed to
make sense to have this parallel the assistant as far as possible to start
with, since I know the assistant works.

This commit was sponsored by Øyvind Andersen Holm.
2014-01-19 17:49:54 -04:00
Joey Hess
f7727d2df1 Remotes can now be made read-only, by setting remote.<name>.annex-readonly 2014-01-02 13:12:32 -04:00
Joey Hess
03932212ec Avoid using git commit in direct mode, since in some situations it will read the full contents of files in the tree.
The assistant's commit code also always avoids git commit, for simplicity.
Indirect mode sync still does a git commit -a to catch unstaged changes.

Note that this means that direct mode sync no longer runs the pre-commit
hook or any other hooks git commit might call. The git annex pre-commit
hook action for direct mode is however explicitly run. (The assistant
already ran git commit with hooks disabled, so no change there.)
2013-12-01 13:59:45 -04:00
Joey Hess
2a297dd6d1 on second thought, don't change sync to exit nonzero on successful conflicted merge resolution
For one thing, that breaks the test suite. May revisit later.
2013-11-14 14:36:48 -04:00
Joey Hess
2d0a05c0fe add a note after successful conflicted merge resolution 2013-11-14 14:18:15 -04:00
Joey Hess
4f373503f2 sync: When there are merge conflicts that get automatically resolved, exit nonzero at the end. 2013-11-07 15:02:11 -04:00
Joey Hess
3802f2f270 work around lack of receive.denyCurrentBranch in direct mode
Now that direct mode sets core.bare=true, git's normal prohibition about
pushing into the currently checked out branch doesn't work.

A simple fix for this would be an update hook which blocks the pushes..
but git hooks must be executable, and git-annex needs to be usable on eg,
FAT, which lacks x bits.

Instead, enabling direct mode switches the branch (eg master) to a special
purpose branch (eg annex/direct/master). This branch is not pushed when
syncing; instead any changes that git annex sync commits get written to
master, and it's pushed (along with synced/master) to the remote.

Note that initialization has been changed to always call setDirect,
even if it's just setDirect False for indirect mode. This is needed because
if the user has just cloned a direct mode repo, that nothing has synced
with before, it may have no master branch, and only a annex/direct/master.
Resulting in that branch being checked out locally too. Calling setDirect False
for indirect mode moves back out of this branch, to a new master branch,
and ensures that a manual "git push" doesn't push changes directly to
the annex/direct/master of the remote. (It's possible that the user
makes a commit w/o using git-annex and pushes it, but nothing I can do
about that really.)

This commit was sponsored by Jonathan Harrington.
2013-11-05 21:08:31 -04:00
Joey Hess
bc18cde325 sync: Work even when the local git repository is new and empty, with no master branch. 2013-11-02 15:29:38 -04:00
Joey Hess
35676feef6 use a long note here 2013-11-01 11:04:19 -04:00
Joey Hess
2af652e1b8 sync: print a hint about receive.denyNonFastForwards when a push fails 2013-10-26 15:53:05 -04:00
Joey Hess
2ddcba7a15 sync: fix crash on first sync when no branch exists yet 2013-10-17 13:34:27 -04:00
Joey Hess
f6560ffcb7 fix indirect mode conflict merge when only one side is annexed file
git-merge's creation of file~HEAD type files did not make this especially
nice to do.
2013-10-16 15:37:06 -04:00
Joey Hess
e227e8f683 sync: Fix automatic resolution of merge conflicts where one side is an annexed file, and the other side is a non-annexed file, or a directory.
Note that this case is only fully automatically resolved in direct mode.
In indirect mode, git merge moves the file to file~HEAD, and replaces it
with the directory, and leaves the file in unmerged state, and sync doesn't
yet change that.
2013-10-16 14:56:40 -04:00
Joey Hess
b405295aee hlint
test suite still passes
2013-09-25 03:09:06 -04:00
Joey Hess
006cf7976f more completely solve catKey memory leak
Done using a mode witness, which ensures it's fixed everywhere.

Fixing catFileKey was a bear, because git cat-file does not provide a
nice way to query for the mode of a file and there is no other efficient
way to do it. Oh, for libgit2..

Note that I am looking at tree objects from HEAD, rather than the index.
Because I cat-file cannot show a tree object for the index.
So this fix is technically incomplete. The only cases where it matters
are:

1. A new large file has been directly staged in git, but not committed.
2. A file that was committed to HEAD as a symlink has been staged
   directly in the index.

This could be fixed a lot better using libgit2.
2013-09-19 16:41:21 -04:00
Joey Hess
2407170eaf sync: Don't fail if the directory it is run in gets removed by the sync. 2013-09-13 14:55:55 -04:00
Joey Hess
b64f5baf2d sync: support gcrypt 2013-09-09 10:02:15 -04:00
Joey Hess
6cdac3a003 sync, assistant: Force push of the git-annex branch.
Necessary to ensure it gets pushed to remotes after being rewritten by forget.
See inline rationalles for why I think this is safe!
2013-08-29 14:27:53 -04:00
Joey Hess
a6a047192e sync, merge: Bug fix: Don't try to merge into master when in a bare repo. 2013-08-17 21:29:44 +02:00
Joey Hess
9666addfaa sync: Better support for bare git remotes. Now pushes directly to the master branch on such a remote, instead of to synced/master. This makes it easier to clone from a bare git remote that has been populated with git annex sync or by the assistant. 2013-06-12 14:54:23 -04:00
Joey Hess
f1cce62283 fix merge conflict resolution when both sides have the same key
Still need to git rm the old file so git accepts the merge is resolved.
2013-05-26 18:32:11 -04:00
Joey Hess
2180068e30 correct recent fix
fc37456d0fe1fb0fd3e33338223977b3e7a940bb's fix caused it to try to stage a
symlink in .git/annex/tmp, oops
2013-05-26 18:10:07 -04:00
Joey Hess
919a7d7316 sync: Fix double merge conflict resolution handling.
Ie, when there'a a conflicted merge we may get foo.variant-xxxx
created in a merge. If a second merge conflict occurs on that new file,
it was not falling back to putting in the whole key (which should stop
the merge conflicts happening for good, but is ugly).
2013-05-26 17:42:15 -04:00
Joey Hess
fc37456d0f fuzz test discovered a way automatic merge resultion can fail; fix
It may be that the directory we need to make a symlink in, to resolve the
merge, doesn't exist locally.
2013-05-25 18:42:43 -04:00
Joey Hess
16503f5692 sync: Commit in direct mode even if no changes were staged
There may be already staged changes from a prior `git annex add`,
so always commit.

Also, suppressed the commit output, since it contains noise due to
typechanged files in direct mode.
2013-04-23 17:14:28 -04:00
Joey Hess
8861e270be sync, assistant: Sync with remotes that have annex-ignore set
This is so git remotes on servers without git-annex installed can be used
to keep clients' git repos in sync.

This is a behavior change, but since annex-sync can be set to disable
syncing with a remote, I think it's acceptable.
2013-04-22 14:57:09 -04:00
Joey Hess
f1b0a4b404 Use lower case hash directories for storing files on crippled filesystems, same as is already done for bare repositories.
* since this is a crippled filesystem anyway, git-annex doesn't use
  symlinks on it
* so there's no reason to use the mixed case hash directories that we're
  stuck using to avoid breaking everyone's symlinks to the content
* so we can do what is already done for all bare repos, and make non-bare
  repos on crippled filesystems use the all-lower case hash directories
* which are, happily, all 3 letters long, so they cannot conflict with
  mixed case hash directories
* so I was able to 100% fix this and even resuming `git annex add` in the
  test case will recover and it will all just work.
2013-04-04 15:46:33 -04:00
Joey Hess
cfd3b16fe1 add section metadata to all commands
Not yet used .. mindless train work.
2013-03-24 18:28:21 -04:00
Joey Hess
0c13d3065e git subcommand cleanup
Pass subcommand as a regular param, which allows passing git parameters
like -c before it. This was already done in the pipeing set of functions,
but not the command running set.
2013-03-03 13:39:07 -04:00
Joey Hess
d7c93b8913 fully support core.symlinks=false in all relevant symlink handling code
Refactored annex link code into nice clean new library.

Audited and dealt with calls to createSymbolicLink.
Remaining calls are all safe, because:

Annex/Link.hs:  ( liftIO $ createSymbolicLink linktarget file
  only when core.symlinks=true
Assistant/WebApp/Configurators/Local.hs:                createSymbolicLink link link
  test if symlinks can be made
Command/Fix.hs: liftIO $ createSymbolicLink link file
  command only works in indirect mode
Command/FromKey.hs:     liftIO $ createSymbolicLink link file
  command only works in indirect mode
Command/Indirect.hs:                    liftIO $ createSymbolicLink l f
  refuses to run if core.symlinks=false
Init.hs:                createSymbolicLink f f2
  test if symlinks can be made
Remote/Directory.hs:    go [file] = catchBoolIO $ createSymbolicLink file f >> return True
  fast key linking; catches failure to make symlink and falls back to copy
Remote/Git.hs:          liftIO $ catchBoolIO $ createSymbolicLink loc file >> return True
  ditto
Upgrade/V1.hs:                          liftIO $ createSymbolicLink link f
  v1 repos could not be on a filesystem w/o symlinks

Audited and dealt with calls to readSymbolicLink.
Remaining calls are all safe, because:

Annex/Link.hs:		( liftIO $ catchMaybeIO $ readSymbolicLink file
  only when core.symlinks=true
Assistant/Threads/Watcher.hs:		ifM ((==) (Just link) <$> liftIO (catchMaybeIO $ readSymbolicLink file))
  code that fixes real symlinks when inotify sees them
  It's ok to not fix psdueo-symlinks.
Assistant/Threads/Watcher.hs:		mlink <- liftIO (catchMaybeIO $ readSymbolicLink file)
  ditto
Command/Fix.hs:	stopUnless ((/=) (Just link) <$> liftIO (catchMaybeIO $ readSymbolicLink file)) $ do
  command only works in indirect mode
Upgrade/V1.hs:	getsymlink = takeFileName <$> readSymbolicLink file
  v1 repos could not be on a filesystem w/o symlinks

Audited and dealt with calls to isSymbolicLink.
(Typically used with getSymbolicLinkStatus, but that is just used because
getFileStatus is not as robust; it also works on pseudolinks.)
Remaining calls are all safe, because:

Assistant/Threads/SanityChecker.hs:                             | isSymbolicLink s -> addsymlink file ms
  only handles staging of symlinks that were somehow not staged
  (might need to be updated to support pseudolinks, but this is
  only a belt-and-suspenders check anyway, and I've never seen the code run)
Command/Add.hs:         if isSymbolicLink s || not (isRegularFile s)
  avoids adding symlinks to the annex, so not relevant
Command/Indirect.hs:                            | isSymbolicLink s -> void $ flip whenAnnexed f $
  only allowed on systems that support symlinks
Command/Indirect.hs:            whenM (liftIO $ not . isSymbolicLink <$> getSymbolicLinkStatus f) $ do
  ditto
Seek.hs:notSymlink f = liftIO $ not . isSymbolicLink <$> getSymbolicLinkStatus f
  used to find unlocked files, only relevant in indirect mode
Utility/FSEvents.hs:                    | Files.isSymbolicLink s = runhook addSymlinkHook $ Just s
Utility/FSEvents.hs:                                            | Files.isSymbolicLink s ->
Utility/INotify.hs:                             | Files.isSymbolicLink s ->
Utility/INotify.hs:                     checkfiletype Files.isSymbolicLink addSymlinkHook f
Utility/Kqueue.hs:              | Files.isSymbolicLink s = callhook addSymlinkHook (Just s) change
  all above are lower-level, not relevant

Audited and dealt with calls to isSymLink.
Remaining calls are all safe, because:

Annex/Direct.hs:			| isSymLink (getmode item) =
  This is looking at git diff-tree objects, not files on disk
Command/Unused.hs:		| isSymLink (LsTree.mode l) = do
  This is looking at git ls-tree, not file on disk
Utility/FileMode.hs:isSymLink :: FileMode -> Bool
Utility/FileMode.hs:isSymLink = checkMode symbolicLinkMode
  low-level

Done!!
2013-02-17 16:43:14 -04:00
Joey Hess
39d5f3f11c avoid queueing rm of no files 2013-02-05 15:11:05 -04:00
Joey Hess
49f4ba297c sync: Automatic merge conflict resolution now stages deleted files. 2013-01-17 21:19:00 -04:00
Joey Hess
4008590c68 type based git config handling for remotes
Still a couple of places that use git config ad-hoc, but this is most of it
done.
2013-01-01 13:58:14 -04:00
Joey Hess
8a8380f1b7 use sync command merge engine in assistant
To handle direct mode merging.
2012-12-25 14:10:07 -04:00
Joey Hess
ddb0adb998 more quickcheck fun 2012-12-19 16:36:19 -04:00
Joey Hess
05ec4587dd partial and incomplete automatic merging in direct mode
Handles our file right, but not theirs.
2012-12-18 17:15:16 -04:00
Joey Hess
53dbcce645 direct mode merging works!
Automatic merge resoltion code needs to be fixed to preserve objects from
direct mode files.
2012-12-18 15:04:44 -04:00
Joey Hess
2cfda59174 reorder for better display 2012-12-13 15:58:38 -04:00
Joey Hess
e7b8cb0063 direct mode committing 2012-12-12 19:20:38 -04:00
Joey Hess
676c78436d also update direct mode associated files in local merge 2012-12-12 13:06:03 -04:00
Joey Hess
514957914d direct mode mappings now updated by git annex sync
Still lots to do to make sync handle direct mode, but this is a good first
step.
2012-12-10 14:37:24 -04:00
Joey Hess
ebd576ebcb where indentation 2012-11-12 01:05:04 -04:00
Joey Hess
589d1711f2 git config remote.name.annex-sync can be used to control whether a remote gets synced. 2012-10-11 18:39:21 -04:00
Joey Hess
47314c0fad fix last zombies in the assistant
Made Git.LsFiles return cleanup actions, and everything waits on
processes now, except of course for Seek.
2012-10-04 19:56:32 -04:00
Joey Hess
601ee470af sync: Pushes the git-annex branch to remote/synced/git-annex, rather than directly to remote/git-annex.
This fixes a problem I was seeing in the assistant where two remotes would
attempt to sync with one another at the same time, and both failed pushing
the diverged git-annex branch. Then when both tried to resolve the failed
push, they each modified their git-annex branch, which again each blocked
the other from pushing into it. The result was that the git-annex
branches were perpetually diverged (despite having the same content!) and
once the assistant fell into this trap, it couldn't get out and always
had to do the slow push/fail/pull/merge/push/fail cycle.
2012-09-16 17:54:12 -04:00
Joey Hess
60c31afc38 add decodeW8 2012-09-13 19:14:29 -04:00
Joey Hess
c58d553265 refactor 2012-08-26 14:45:47 -04:00
Joey Hess
f4ca592cd0 refactor 2012-08-26 14:34:30 -04:00
Joey Hess
5c3e14649e avoid unnecessary transfer scans when syncing a disconnected remote
Found a very cheap way to determine when a disconnected remote has
diverged, and has new content that needs to be transferred: Piggyback on
the git-annex branch update, which already checks for divergence.

However, this does not check if new content has appeared locally while
disconnected, that should be transferred to the remote.

Also, this does not handle cases where the two git repos are in sync,
but their content syncing has not caught up yet.

This code could have its efficiency improved:

* When multiple remotes are synced, if any one has diverged, they're
  all queued for transfer scans.
* The transfer scanner could be told whether the remote has new content,
  the local repo has new content, or both, and could optimise its scan
  accordingly.
2012-08-22 15:05:57 -04:00
Joey Hess
94fcd0cf59 add routes to pause/start/cancel transfers
This commit includes a paydown on technical debt incurred two years ago,
when I didn't know that it was bad to make custom Read and Show instances
for types. As the routes need Read and Show for Transfer, which includes a
Key, and deriving my own Read instance of key was not practical,
I had to finally clean that up.

So the compact Key read and show functions are now file2key and key2file,
and Read and Show are now derived instances.

Changed all code that used the old instances, compiler checked.
(There were a few places, particularly in Command.Unused, and the test
suite where the Show instance continue to be used for legitimate
comparisons; ie show key_x == show key_y (though really in a bloom filter))
2012-08-08 16:20:24 -04:00
Joey Hess
5ae1f75a39 handle case of adding populated drive to just created repo
The just created repo has no master branch commits yet. This is now
handled, merging in the master branch from the populated drive.
2012-08-05 16:35:30 -04:00
Joey Hess
34fc0d358e fix crashes when run in a git repo that has been initted but has no master branch yet 2012-08-05 15:53:47 -04:00
Joey Hess
2e501364d4 Merge branch 'master' into assistant 2012-06-27 18:09:11 -04:00
Joey Hess
2d7ebc0582 typo 2012-06-27 18:08:52 -04:00
Joey Hess
8baff14054 Merge branch 'master' into assistant 2012-06-27 16:14:33 -04:00
Joey Hess
36ddb81df6 use "variant" rather than "version"
While this word may be less familiar to some users, it avoids the
connotation that version 2 is better than version 1, which is wrong
when the two variants were conflicting.
2012-06-27 16:09:17 -04:00
Joey Hess
054ddda18a better filenames for conflict resolution files 2012-06-27 16:03:42 -04:00
Joey Hess
9147ad7493 commit merge resolution
this is necessary so the sync can continue successfully with its push phase
2012-06-27 15:06:47 -04:00
Joey Hess
8810e57995 fix file name 2012-06-27 15:00:26 -04:00
Joey Hess
abd36ed336 don't automerge when the symlinks cannot be parsed as keys 2012-06-27 13:35:02 -04:00
Joey Hess
048b64024a sync: Automatically resolves merge conflicts.
untested, but it compiles :)
2012-06-27 13:08:32 -04:00
Joey Hess
051c68041b properly handle deleted files when processing ls-files --unmerged 2012-06-27 12:11:03 -04:00
Joey Hess
d88ee75a2d Merge branch 'master' into assistant 2012-06-23 10:27:12 -04:00
Joey Hess
c79e3b67e9 sync: Avoid recent git's interactive merge. 2012-06-23 10:22:56 -04:00
Joey Hess
e9630e90de the syncer now pushes out changes to remotes, in parallel
Note that, since this always pushes branch synced/master to the remote, it
assumes that master has already gotten all the commits that are on the
remote merged in. Otherwise, fast-forward prevention may prevent the push.

That's probably ok, because the next stage is to automatically detect
incoming pushes and merge.
2012-06-22 15:49:48 -04:00
Joey Hess
6e213d04f1 sync: Show a nicer message if a user tries to sync to a special remote. 2012-05-27 20:55:56 -04:00
Joey Hess
60ab3d84e1 added ifM and nuked 11 lines of code
no behavior changes
2012-03-14 17:43:34 -04:00
Joey Hess
997e29f294 sync: Sync to lower cost remotes first.
This has two benefits.

1. When a lot of refs are going to be received, get them via lower cost
   connection when possible.
2. Allows ctrl-c of sync after the cheaper remotes have been pulled from
   (or pushed to).
2012-03-10 15:37:38 -04:00
Joey Hess
1f73db3469 improve alwayscommit=false mode
Now changes are staged into the branch's index, but not committed,
which avoids growing a large journal. And sync and merge always
explicitly commit, ensuring that even when they do nothing else,
they commit the staged changes.

Added a flag file to indicate that the branch's journal contains
uncommitted changes. (Could use git ls-files, but don't want to run
that every time.)

In the future, this ability to have uncommitted changes staged in the
journal might be used on remotes after a series of oneshot commands.
2012-02-25 16:18:55 -04:00
Joey Hess
a1e52f0ce5 hlint 2012-02-16 00:44:51 -04:00
Joey Hess
df21cbfdd2 look up --to and --from remote names only once
This will speed up commands like move and drop.
2012-01-06 04:06:13 -04:00
Joey Hess
508b427c7b tweak 2012-01-02 11:57:02 -04:00
Joey Hess
f0957426c5 skip local remotes that are not available (ie, not mounted)
With --fast, unavailable local remotes are filtered out of the fast set.
This way, if there are local remotes, --fast always acts only on them,
and if none are mounted, acts on nothing. This consistency is better
than --fast acting on different remotes depending on what's mounted.
2011-12-31 04:50:39 -04:00
Joey Hess
4a02c2ea62 type alias cleanup 2011-12-31 04:11:58 -04:00
Joey Hess
a2ec2d3760 refactor and check for a detached HEAD 2011-12-31 03:38:58 -04:00
Joey Hess
8a33573caf better filtering out of special remotes 2011-12-31 03:27:37 -04:00
Joey Hess
6cd4c7efcd never pick special remotes in --fast
even if they have the lowest cost, we cannot use them
2011-12-31 03:14:05 -04:00
Joey Hess
c61642ef0c remove unnecessary check
mergeLocal always creates the local sync branch, so no need to check that
it exists later.
2011-12-31 03:08:44 -04:00
Joey Hess
aa64b8ceaf refactor 2011-12-31 03:01:18 -04:00
Joey Hess
2998340abb really fix check that remote needs merged 2011-12-31 02:45:12 -04:00
Joey Hess
9a7a77488e tweak 2011-12-31 02:18:16 -04:00
Joey Hess
0396f9c795 tweak 2011-12-31 02:15:13 -04:00
Joey Hess
f2b584ad74 fix check that remote branch needs merged 2011-12-31 02:03:39 -04:00
Joey Hess
79231bcff0 minor cleanups
mergeFrom is never called on branches that don't exist anymore
2011-12-31 01:51:39 -04:00
Joey Hess
015a497914 avoid syncing remotes configured annex-ignore, unless explicitly specified 2011-12-31 01:42:42 -04:00
Joey Hess
e7d3e546c2 sync --fast: Selects some of the remotes with the lowest annex.cost and syncs those, in addition to any specified at the command line. 2011-12-30 21:17:36 -04:00
Joey Hess
a31b7d93c8 push when git-annex branch changed
I was too heavy-handed in optimising away pushes
2011-12-30 19:38:46 -04:00
Joey Hess
79872e360e automated syncing
Some changes to make automated syncing nicer. Merge from both the remote's
$branch and its synced/$branch; either could have new changes. Create
synced/$branch on the remote when pushing.
2011-12-30 19:24:57 -04:00
Joey Hess
f6f7ee7131 automatically create the syncbranch 2011-12-30 18:52:24 -04:00
Joey Hess
14d16b77b3 refactor 2011-12-30 18:37:55 -04:00
Joey Hess
52104dae6f refactor 2011-12-30 18:36:40 -04:00
Joey Hess
56488e807b check that synced/master exists before trying to use it
and a nice error message if syncing is not set up yet
2011-12-30 18:19:45 -04:00
Joey Hess
f2fa29bf3b check if branches are up-to-date before merging, pushing
This optimises away the need to run anything in some common cases.
It's particularly useful on push; no need to push if the tracking branch
we just pulled is the same as the branch we're going to push.
2011-12-30 18:04:01 -04:00
Joey Hess
9d85baa314 improve wording 2011-12-30 17:54:09 -04:00
Joey Hess
4400f65967 message cleanup 2011-12-30 17:38:38 -04:00
Joey Hess
556618a3ec avoid using Git.Ref.describe except for when generating user messages
The other uses of it can all be simplified using Git.Ref.base,
Git.Ref.under, and show.

In some cases, describe was being used to shorten the branch name
unnecessarily, and I instead pass the fully qualified name to git.
2011-12-30 17:01:03 -04:00
Joey Hess
5d17da5eb3 update to my indentation style 2011-12-30 16:24:30 -04:00
Joey Hess
5728bb58e0 force git-annex branch update after fetching remotes
git-annex normally only runs the branch update once per run, for speed, but
since this fetches new remote git-annex tracking branches, they need to be
merged in after that fetch. An earlier call to Remote.byName was causing
the update to run before the fetch sometimes, but it could have been
anything. Just force the update to happen in the right place.
2011-12-30 16:03:41 -04:00
Joachim Breitner
b6e7b40be4 By default, sync with all remotes having the synced/ branch 2011-12-29 20:50:57 +01:00
Joachim Breitner
0ee1141f30 Implement branch-syncing in Command.Sync
as described in the previous commit to the documentation. The loggin UI
is not great yet.
2011-12-29 18:37:30 +01:00
Joey Hess
95d2391f58 more partial function removal
Left a few Prelude.head's in where it was checked not null and too hard to
remove, etc.
2011-12-15 18:19:36 -04:00
Joey Hess
ef28b3fef7 split out Git/Command.hs 2011-12-14 15:56:11 -04:00
Joey Hess
13fff71f20 split out three modules from Git
Constructors and configuration make sense in separate modules.
A separate Git.Types is needed to avoid cycles.
2011-12-13 15:06:49 -04:00
Joey Hess
4200b8038a separate operations 2011-12-10 12:21:22 -04:00
Joey Hess
fb8231f3a1 sync: New command that synchronises the local repository and default remote, by running git commit, pull, and push for you. 2011-12-09 20:27:22 -04:00