Commit graph

559 commits

Author SHA1 Message Date
Joey Hess
d8a2f658dd direct mode merge relative path trickiness
This fixes 9 test suite failures. There are some tricky things going on
with the paths to the index file, and git's working directory, which
are hard to get right with relative paths. So, I switched back to absolute
here, at least for now.

Only 2 test suite failures remain on this branch, but there are other
potential problems the test suite doesn't catch. Including some calls to
setCurrentDirectory -- I was wrong and git-annex does do that in a few
places, like when generating a view.
2015-01-06 17:18:12 -04:00
Joey Hess
cd865c3b8f Switch to using relative paths to the git repository.
This allows the git repository to be moved while git-annex is running in
it, with fewer problems.

On Windows, this avoids some of the problems with the absurdly small
MAX_PATH of 260 bytes. In particular, git-annex repositories should
work in deeper/longer directory structures than before. See
http://git-annex.branchable.com/bugs/__34__git-annex:_direct:_1_failed__34___on_Windows/

There are several possible ways this change could break git-annex:

1. If it changes its working directory while it's running, that would
   be Bad News. Good news everyone! git-annex never does so. It would also
   break thread safety, so all such things were stomped out long ago.

2. parentDir "." -> "" which is not a valid path. I had to fix one
   instace of this, and I should probably wipe all calls to parentDir out
   of the git-annex code base; it was never a good idea.

3. Things like relPathDirToFile require absolute input paths,
   and code assumes that the git repo path is absolute and passes it to it
   as-is. In the case of relPathDirToFile, I converted it to not make
   this assumption.

Currently, the test suite has 16 failures.
2015-01-06 16:19:41 -04:00
Joey Hess
a4cf80f460 Windows: Fix handling of views of filenames containing '%' 2014-12-30 17:48:04 -04:00
Joey Hess
402bfff665 fix test case on windows
"a:" is an absolute path, so viewedfile test cannot be run on it.
2014-12-30 16:04:06 -04:00
Joey Hess
33f1062bc3 Revert "temporary debugging code for windows autobuilder test suite failure"
This reverts commit 0d9fbd18c1.
2014-12-30 15:18:38 -04:00
Joey Hess
0d9fbd18c1 temporary debugging code for windows autobuilder test suite failure 2014-12-30 15:17:51 -04:00
Joey Hess
c9a3e80d32 fixed all remaining build warnings on Windows 2014-12-29 17:30:20 -04:00
Joey Hess
7e422269a6 move dummy uuids to Annex.UUID 2014-12-17 13:57:52 -04:00
Joey Hess
7ae16bb6f7 Revert "let url claims optionally include a suggested filename"
This reverts commit 85df9c30e9.

Putting filename in the claim was a bad idea.
2014-12-11 14:09:57 -04:00
Joey Hess
85df9c30e9 let url claims optionally include a suggested filename 2014-12-11 12:47:57 -04:00
Joey Hess
6ecd3ff421 diffdriver: New git-annex command, to make git external diff drivers work with annexed files.
Closes https://github.com/datalad/datalad/issues/18
2014-11-24 16:14:06 -04:00
Joey Hess
864086a956 proxy: for all your direct mode repository munging needs
This allows bypassing the direct mode guard in a safe way to do all sorts
of things including git revert, git mv, git checkout ...

This commit was sponsored by the WikiMedia Foundation.
2014-11-12 15:51:46 -04:00
Joey Hess
5ccc2a2d7c no longer used imports 2014-11-06 14:18:38 -04:00
Joey Hess
334f366979 Remove fixup code for bad bare repositories created by versions 5.20131118 through 5.20131127. That fixup code would accidentially fire when --git-dir was incorrectly pointed at the working tree of a git-annex repository, resulting in data loss. Closes: #768093 2014-11-04 18:04:19 -04:00
Joey Hess
0f6aaf8012 Windows: Fix crash when user.name is not set in git config. 2014-10-31 16:14:12 -04:00
Joey Hess
4edfda59c0 fix windows build 2014-10-16 15:48:30 -04:00
Joey Hess
1e59df083d Use haskell setenv library to clean up several ugly workarounds for inability to manipulate the environment on windows.
Didn't know that this library existed!

This includes making git-annex not re-exec itself on start on windows, and
making the test suite on Windows run tests without forking.
2014-10-15 20:33:52 -04:00
Joey Hess
db9121ecee vicfg: Deleting configurations now resets to the default, where before it has no effect.
Added a Default instance for TrustLevel, and was able to use that to clear
up several other parts of the code too.

This commit was sponsored by Stephan Schulz
2014-10-14 14:15:07 -04:00
Joey Hess
9fd95d9025 indent with tabs not spaces
Found these with:
git grep "^  " $(find -type  f -name \*.hs) |grep -v ':  where'

Unfortunately there is some inline hamlet that cannot use tabs for
indentation.

Also, Assistant/WebApp/Bootstrap3.hs is a copy of a module and so I'm
leaving it as-is.
2014-10-09 15:09:26 -04:00
Joey Hess
7b50b3c057 fix some mixed space+tab indentation
This fixes all instances of " \t" in the code base. Most common case
seems to be after a "where" line; probably vim copied the two space layout
of that line.

Done as a background task while listening to episode 2 of the Type Theory
podcast.
2014-10-09 15:09:11 -04:00
Joey Hess
0598412e5c Fix transfer lock file FD leak that could occur when two separate git-annex processes were both working to perform the same set of transfers. 2014-09-11 13:53:26 -04:00
Joey Hess
b874f84086 New annex.hardlink setting. Closes: #758593
* New annex.hardlink setting. Closes: #758593
* init: Automatically detect when a repository was cloned with --shared,
  and set annex.hardlink=true, as well as marking the repository as
  untrusted.

Had to reorganize Logs.Trust a bit to avoid a cycle between it and
Annex.Init.
2014-09-05 13:44:09 -04:00
Joey Hess
6eb5c3f479 Do not preserve permissions and acls when copying files from one local git repository to another. Timestamps are still preserved as long as cp --preserve=timestamps is supported.
This avoids cp -a overriding the default mode acls that the user might have
set in a git repository.

With GNU cp, this behavior change should not be a breaking change, because
git-anex also uses rsync sometimes in the same situation, and has only ever
preserved timestamps when using rsync.

Systems without GNU cp will no longer use cp -a, but instead just cp.
So, timestamps will no longer be preserved. Preserving timestamps when
copying between repos is not guaranteed anyway.

Closes: #729757
2014-08-26 17:10:25 -07:00
Joey Hess
2b234634f6 fix imports for windows 2014-08-23 16:27:24 -07:00
Joey Hess
aebcc395ff use types to enforce that removeAnnex can only be called inside lockContent
This fixed one bug where it needed to be and wasn't (in Assistant.Unused).
And also found one place where lockContent was used unnecessarily (by
drop --from remote).

A few other places like uninit probably don't really need to lockContent,
but it doesn't hurt to do call it anyway.

This commit was sponsored by David Wagner.
2014-08-20 20:13:47 -04:00
Joey Hess
1994771215 more lock file refactoring
Also fixes a test suite failures introduced in recent commits, where
inAnnexSafe failed in indirect mode, since it tried to open the lock file
ReadWrite. This is why the new checkLocked opens it ReadOnly.

This commit was sponsored by Chad Horohoe.
2014-08-20 18:58:14 -04:00
Joey Hess
e386e26ef2 avoid trying to create a content file in order to lock it
The nice refactoring in ec7dd0446a
highlighted a bug in lockContent -- when the content is not present,
this incorrectly created an empty lock file, using the same filename
as the content file.

This seems like it could result in empty objects, which fsck would detect
and complain about. Both drop and move --to call lockContent, as does
Remote.Git.dropKey -- I think we got lucky and this bug didn't show up
because both all of those only operate on files that are present. So
this bug could only manifest if there was a race, and a file's content
was dropped at just the wrong time, just as another process was about to
drop it. (And then only if the other process's dropping failed, otherwise
it'd delete the empty object file.)

Hmm, move --from also called lockContent. Unnecessarily, since the content
is not being removed from the local annex. In this case, the combination of
the 2 bugs could result in an empty lock file being written, and then if
the download of the content failed, left in the object directory as the
content.

This commit also optimises lockContent, avoiding an unncessary
doesFileExist test and instead just catching the exception that's thrown
when the file doesn't exist.

This commit was sponsored by Justine Lam.
2014-08-20 17:25:30 -04:00
Joey Hess
ec7dd0446a more lock file refactoring 2014-08-20 17:03:04 -04:00
Joey Hess
d279180266 reorganize and refactor lock code
Added a convenience Utility.LockFile that is not a windows/posix
portability shim, but still manages to cut down on the boilerplate around
locking.

This commit was sponsored by Johan Herland.
2014-08-20 16:45:58 -04:00
Joey Hess
0a4d301051 fix lockFileShared to actually create lock file
This was a bug, but it was only used for ssh locks and by the hook special
remote locking. At least in the case of ssh locks, the lock files happened
to already exist before this tried to use them, so the bug didn't cause
anything to break.
2014-08-20 15:49:49 -04:00
Joey Hess
bf3133ebb0 whoops, I the debug prints 2014-08-20 12:14:56 -04:00
Joey Hess
96dc423e39 When accessing a local remote, shut down git-cat-file processes afterwards, to ensure that remotes on removable media can be unmounted. Closes: #758630
This does mean that eg, copying multiple files to a local remote will
become slightly slower, since it now restarts git-cat-file after each copy.
Should not be significant slowdown.

The reason git-cat-file is run on the remote at all is to update its
location log. In order to add an item to it, it needs to get the current
content of the log. Finding a way to avoid needing to do that would be a
good path to avoiding this slowdown if it does become a problem somehow.

This commit was sponsored by Evan Deaubl.
2014-08-20 12:07:57 -04:00
Joey Hess
83dc82c232 forgot some lifts 2014-08-20 11:51:47 -04:00
Joey Hess
092041fab0 Ensure that all lock fds are close-on-exec, fixing various problems with them being inherited by child processes such as git commands.
(With the exception of daemon pid locking.)

This fixes at part of #758630. I reproduced the assistant locking eg, a
removable drive's annex journal lock file and forking a long-running
git-cat-file process that inherited that lock.

This did not affect Windows.

Considered doing a portable Utility.LockFile layer, but git-annex uses
posix locks in several special ways that have no direct Windows equivilant,
and it seems like it would mostly be a complication.

This commit was sponsored by Protonet.
2014-08-20 11:37:02 -04:00
Joey Hess
e0227dfedf memoize construction of the Request -> Request function to apply the UrlOptions 2014-08-15 17:47:21 -04:00
Joey Hess
852185c242 git-annex-shell sendkey: Don't fail if a remote asks for a key to be sent that already has a transfer lock file indicating it's being sent to that remote. The remote may have moved between networks, or reconnected. 2014-08-15 14:17:05 -04:00
Joey Hess
bb6cec3461 direct: Avoid leaving file content in misctemp if interrupted. 2014-08-15 13:38:05 -04:00
Joey Hess
d8be828734 direct: Fix ugly warning messages.
replaceFileOr was broken and ran the rollback action always.
Luckily, for replaceFile, the rollback action was safe to run, since it
just nuked a temp file that had already been moved into place.

However, when `git annex direct` used replaeFileOr, its rollback printed a
scary message:

  /home/joey/tmp/rrrr/.git/annex/misctmp/tmp32268: rename: does not exist (No such file or directory)

There was actually no bad result though.
2014-08-12 13:00:08 -04:00
Joey Hess
c784ef4586 unify exception handling into Utility.Exception
Removed old extensible-exceptions, only needed for very old ghc.

Made webdav use Utility.Exception, to work after some changes in DAV's
exception handling.

Removed Annex.Exception. Mostly this was trivial, but note that
tryAnnex is replaced with tryNonAsync and catchAnnex replaced with
catchNonAsync. In theory that could be a behavior change, since the former
caught all exceptions, and the latter don't catch async exceptions.

However, in practice, nothing in the Annex monad uses async exceptions.
Grepping for throwTo and killThread only find stuff in the assistant,
which does not seem related.

Command.Add.undo is changed to accept a SomeException, and things
that use it for rollback now catch non-async exceptions, rather than
only IOExceptions.
2014-08-07 22:03:29 -04:00
Joey Hess
5aa2286e7b Merge branch 'newchunks'
I am happy enough with this to make it live!
2014-08-01 18:00:47 -04:00
Joey Hess
76d894f2e5 Display exception message when a transfer fails due to an exception.
For example, I had a copy to a remote that was failing for an unknown
reason. This let me see the exception was createDirectory: permission
denied; the underlying problem being a permissions issue.
2014-07-30 15:57:19 -04:00
Joey Hess
f3e457a195 add missing Ord constraint (fixes android build)
Probably the new ghc used on android is the root cause of needing this
constraint.
2014-07-30 11:57:40 -04:00
Joey Hess
bc9e4697b9 better type for Retriever
Putting a callback in the Retriever type allows for the callback to
remove the retrieved file when it's done with it.

I did not really want to make Retriever be fixed to Annex Bool,
but when I tried to use Annex a, I got into some type of type mess.
2014-07-29 18:41:41 -04:00
Joey Hess
47e522979c allow Retriever action to update the progress meter
Needed for eg, Remote.External.

Generally, any Retriever that stores content in a file is responsible for
updating the meter, while ones that procude a lazy bytestring cannot update
the meter, so are not asked to.
2014-07-29 17:18:49 -04:00
Joey Hess
7496355031 add some more exception handling primitives 2014-07-26 23:24:27 -04:00
Joey Hess
e2c44bf656 implement chunk logs
Slightly tricky as they are not normal UUIDBased logs, but are instead maps
from (uuid, chunksize) to chunkcount.

This commit was sponsored by Frank Thomas.
2014-07-24 16:23:36 -04:00
Joey Hess
3e4cb0e7f9 fix windows build 2014-07-14 15:55:48 -04:00
Joey Hess
822f4619ae resolvemerge: finish up by committing 2014-07-11 16:59:49 -04:00
Joey Hess
61a35de433 Deal with change in git 2.0 that made indirect mode merge conflict resolution leave behind old files.
I think this is a git behavior change, but have not checked to be sure.
Conflict cruft used to look like $foo~HEAD, but now just $foo is left
behind as conflict cruft.

With test case.
2014-07-11 16:56:19 -04:00
Joey Hess
cb66ca3a76 resolvemerge: New plumbing command that runs the automatic merge conflict resolver. 2014-07-11 16:45:18 -04:00