Commit graph

1084 commits

Author SHA1 Message Date
Joey Hess
64f8819ae4 fix build 2013-06-17 21:30:52 -04:00
Joey Hess
9ef09587dc fsck: Avoid getting confused by Windows path separators 2013-06-17 21:18:43 -04:00
Joey Hess
98be446d02 remove workaround for old bug that was only in one release
It's causing some problem on windows, see
http://git-annex.branchable.com/bugs/windows_port_-_repo_can__39__t_pull_newly_added_files_/#comment-45df9748bba687d95e3c96b3877ea925
And only affected WORM backend, and for one release well over a year ago,
so could well be bitrotted.
2013-06-17 20:51:36 -04:00
Joey Hess
2844e7175e status: No longer shows dead repositories.
This is because people continually whine about it.  Seemingly not aware
that data generally cannot be deleted from git anyway.
2013-06-17 12:35:33 -04:00
Joey Hess
9666addfaa sync: Better support for bare git remotes. Now pushes directly to the master branch on such a remote, instead of to synced/master. This makes it easier to clone from a bare git remote that has been populated with git annex sync or by the assistant. 2013-06-12 14:54:23 -04:00
Joey Hess
6dcf21db93 Direct mode: No longer temporarily remove write permission bit of files when adding them.
This write permission frobbing is very appropriate in indirect mode,
since annexed objects are stored as immutably as can be managed. But not
in direct mode, where files should be able to be modified at any time.

There are already sufficient guards that there's no need to prevent a file
being written to while it's being ingested, in direct mode. The inode cache
will detect (most) types of modifications, and the add will fail. Then a
re-add should be done. The assistant should get another inotify change
event, and automatically add the new version of the file.
2013-06-12 14:02:31 -04:00
Joey Hess
c46b263fde Android: Make the "Open webapp" menu item open the just created repository when a new repo is made. 2013-06-10 23:55:53 -04:00
Joey Hess
a64106dcef Supports indirect mode on encfs in paranoia mode, and other filesystems that do not support hard links, but do support symlinks and other POSIX filesystem features. 2013-06-10 13:11:33 -04:00
Joey Hess
92f036fcb4 avoid warnings when built with ghc 7.6 2013-06-02 15:01:58 -04:00
Joey Hess
91c9ae83f1 squash warning 2013-06-02 14:06:17 -04:00
Joey Hess
a48d340abd Android: Work around Android devices where the am command doesn't work. 2013-05-31 21:30:21 -04:00
Joey Hess
cba2942cda Revert "android dupped stderr workaround"
This reverts commit 4cc803c733.

The stderr fd is also trashed after `am` fails to open the web browser.
2013-05-30 16:27:10 -04:00
Joey Hess
4cc803c733 android dupped stderr workaround
Avoid using dupped stderr, since
http://git-annex.branchable.com/bugs/warning_-_WebApp_crashed:___60__file_descriptor_15__62__:_hPutStr:_illegal_operation___40__handle_is_closed__41___on_Android/#comment-a24c73803fb10bd35afdc10d50e071c8
seems to involve that handle not being dupped originally, or perhaps
getting closed when the web browser is started on Android.

Using the dupped stdout is known to work before starting the web browser,
so it should work after -- unless perhaps starting it closes both handles.

In any case, there's no real need to write to stderr here.
2013-05-30 13:55:22 -04:00
Joey Hess
3e2d50a336 Android: Added an "Open WebApp" item to the terminal's menu. Should work for Android devices that cannot auto-open the webapp on start. 2013-05-28 18:25:27 -04:00
Joey Hess
f1cce62283 fix merge conflict resolution when both sides have the same key
Still need to git rm the old file so git accepts the merge is resolved.
2013-05-26 18:32:11 -04:00
Joey Hess
2180068e30 correct recent fix
fc37456d0fe1fb0fd3e33338223977b3e7a940bb's fix caused it to try to stage a
symlink in .git/annex/tmp, oops
2013-05-26 18:10:07 -04:00
Joey Hess
919a7d7316 sync: Fix double merge conflict resolution handling.
Ie, when there'a a conflicted merge we may get foo.variant-xxxx
created in a merge. If a second merge conflict occurs on that new file,
it was not falling back to putting in the whole key (which should stop
the merge conflicts happening for good, but is ugly).
2013-05-26 17:42:15 -04:00
Joey Hess
469b3859fc reduce the amount of subdirectories created by the fuzz tester to saner limit 2013-05-26 16:15:25 -04:00
Joey Hess
9978269b55 make fuzztest honor annex.diskreserve 2013-05-26 16:04:52 -04:00
Joey Hess
fc37456d0f fuzz test discovered a way automatic merge resultion can fail; fix
It may be that the directory we need to make a symlink in, to resolve the
merge, doesn't exist locally.
2013-05-25 18:42:43 -04:00
Joey Hess
33fe3dac9e fuzz tester: avoid deleting entire repository (had to happen eventually, right?) 2013-05-25 18:15:34 -04:00
Joey Hess
377bc7bbb7 fix 2013-05-25 17:55:31 -04:00
Joey Hess
ce0ee2aa44 fuzz improvements 2013-05-25 17:52:33 -04:00
Joey Hess
4dc61d74c9 tune frequencies to avoid enormous trees being created 2013-05-25 17:18:37 -04:00
Joey Hess
8b5b4522d1 make fuzz test also output its actions to stdout 2013-05-25 15:52:28 -04:00
Joey Hess
b276857a7a content: New command line way to view and configure a repository's preferred content settings. 2013-05-25 12:44:58 -04:00
Joey Hess
2b14fe2c98 refactor 2013-05-24 23:07:26 -04:00
Joey Hess
a96e982bd3 fuzz tester 2013-05-23 19:00:46 -04:00
Joey Hess
cf7fec4b46 list standard groups in vicfg 2013-05-23 14:54:59 -04:00
Joey Hess
21d5489bd3 typo 2013-05-19 14:46:48 -04:00
Joey Hess
345ee4f37c Switch to MonadCatchIO-transformers for better handling of state while catching exceptions.
As seen in this bug report, the lifted exception handling using the StateT
monad throws away state changes when an action throws an exception.
http://git-annex.branchable.com/bugs/git_annex_fork_bombs_on_gpg_file/
  .. Which can result in cached values being redundantly calculated, or other
     possibly worse bugs when the annex state gets out of sync with reality.

This switches from a StateT AnnexState to a ReaderT (MVar AnnexState).
All changes to the state go via the MVar. So when an Annex action is
running inside an exception handler, and it makes some changes, they
immediately go into affect in the MVar. If it then throws an exception
(or even crashes its thread!), the state changes are still in effect.

The MonadCatchIO-transformers change is actually only incidental.
I could have kept on using lifted-base for the exception handling.
However, I'd have needed to write a new instance of MonadBaseControl
for the new monad.. and I didn't write the old instance.. I begged Bas
and he kindly sent it to me. Happily, MonadCatchIO-transformers is
able to derive a MonadCatchIO instance for my monad.

This is a deep level change. It passes the test suite! What could it break?

Well.. The most likely breakage would be to code that runs an Annex action
in an exception handler, and *wants* state changes to be thrown away.
Perhaps the state changes leaves the state inconsistent, or wrong. Since
there are relatively few places in git-annex that catch exceptions in the
Annex monad, and the AnnexState is generally just used to cache calculated
data, this is unlikely to be a problem.

Oh yeah, this change also makes Assistant.Types.ThreadedMonad a bit
redundant. It's now entirely possible to run concurrent Annex actions in
different threads, all sharing access to the same state! The ThreadedMonad
just adds some extra work on top of that, with its own MVar, and avoids
such actions possibly stepping on one-another's toes. I have not gotten
rid of it, but might try that later. Being able to run concurrent Annex
actions would simplify parts of the Assistant code.
2013-05-19 14:16:36 -04:00
Joey Hess
b8e5b9c645 test suite passes in direct mode
This fixes a bug with git annex add in direct mode. If some files already
existed in the tree pointing at the same key as a file that was just added,
and their content was not present, add neglected to copy the content to
those files.

I also changed the behavior of moveAnnex slightly: When content is moved
into the annex in direct mode, it does not overwrite any content already
present in direct mode files. That content may be modified after all.
2013-05-17 15:59:37 -04:00
Joey Hess
dc66b1f27d Merge branch 'master' into windows
Conflicts:
	Annex/Environment.hs
	Build/Configure.hs
	Git/Construct.hs
	Utility/FileMode.hs
2013-05-14 15:37:24 -04:00
Joey Hess
c553f56634 migrate: Detect if a file gets corrupted while it's being migrated. 2013-05-13 14:27:39 -04:00
Joey Hess
abe8d549df fix permission damage (thanks, Windows) 2013-05-11 23:54:25 -04:00
Joey Hess
1e2ddcb68a use setCurrentDirectory
On POSIX, this just calls changeWorkingDirectory.
2013-05-11 19:14:30 -04:00
Joey Hess
18bdff3fae clean up from windows porting 2013-05-11 18:23:41 -04:00
Joey Hess
3c7e30a295 git-annex now builds on Windows (doesn't work) 2013-05-11 15:03:00 -05:00
Joey Hess
763cbda14f fixup #if 0 stubs to use #ifndef mingw32_HOST_OS
That's needed in files used to build the configure program.
For the other files, I'm keeping my __WINDOWS__ define, as I find that much easier to type.
I may search and replace it to use the mingw32_HOST_OS thing later.
2013-05-10 16:57:21 -05:00
Joey Hess
6c74a42cc6 stub out POSIX stuff 2013-05-10 16:29:59 -05:00
Joey Hess
adde00f4f3 git-annex-shell: Ensure that received files can be read. Files transferred from some Android devices may have very broken permissions as received. 2013-05-06 17:30:57 -04:00
Joey Hess
886897f627 fix build 2013-05-02 20:38:57 -04:00
Joey Hess
29df1a6bec fix build 2013-05-02 19:10:30 -04:00
Joey Hess
6dca92defe need cpp 2013-05-02 19:00:50 -04:00
Joey Hess
1e9b8e1467 cannot open file:// url on Android, so don't use the htmlshim there 2013-05-02 16:46:47 -04:00
Joey Hess
906258ca5e include build flags in version info when not in a repo 2013-05-01 23:11:13 -04:00
Bruno Bigras
0fe8eb0953 Fix the URL not showing until the app closes 2013-05-01 15:48:16 +00:00
Joey Hess
0807211a67 thaw content directory in direct mode too
A content directory can be frozen in direct mode. One way this can happen
is if the content is transferred before direct mode has a mapping for it,
so it's stored in the content directory.

So, we need to thaw the content directory before doing things with it.
2013-04-30 19:33:43 -04:00
Joey Hess
85d83e7756 To enable an existing special remote, the new enableremote command must be used. The initremote command now is used only to create new special remotes. 2013-04-26 18:22:52 -04:00
Joey Hess
49547ad32d initremote: If two existing remotes have the same name, prefer the one with a higher trust level. 2013-04-24 21:53:58 -04:00
Joey Hess
68f38a7ae6 show a message to tell why adding a file failed 2013-04-23 18:09:00 -04:00
Joey Hess
8b0dcb3136 add: avoid ugly error message when adding a deleted file in direct mode
Due to add using withFilesMaybeModified, it will get files that have been
deleted but are still in the index. So catch the IO error that results when
trying to stat such a file.
2013-04-23 17:22:56 -04:00
Joey Hess
16503f5692 sync: Commit in direct mode even if no changes were staged
There may be already staged changes from a prior `git annex add`,
so always commit.

Also, suppressed the commit output, since it contains noise due to
typechanged files in direct mode.
2013-04-23 17:14:28 -04:00
Joey Hess
2c42e70f6c rename module 2013-04-23 11:38:52 -04:00
Joey Hess
8a2d1988d3 expose Control.Monad.join
I think I've been looking for that function for some time.
Ie, I remember wanting to collapse Just Nothing to Nothing.
2013-04-22 20:24:53 -04:00
Joey Hess
6be815a30c rmurl: New command, removes one of the recorded urls for a file. 2013-04-22 17:18:53 -04:00
Joey Hess
dc39da52f8 direct, indirect: Refuse to do anything when the assistant or git-annex watch daemon is running. 2013-04-22 16:41:46 -04:00
Joey Hess
9cb223a8b3 Detect systems that have no user name set in GECOS, and also don't have user.name set in git config, and put in a workaround so that commits to the git-annex branch (and the assistant) will still succeed despite git not liking the system configuration. 2013-04-22 15:36:34 -04:00
Joey Hess
8861e270be sync, assistant: Sync with remotes that have annex-ignore set
This is so git remotes on servers without git-annex installed can be used
to keep clients' git repos in sync.

This is a behavior change, but since annex-sync can be set to disable
syncing with a remote, I think it's acceptable.
2013-04-22 14:57:09 -04:00
Joey Hess
accf949f9c change section 2013-04-20 20:24:57 -04:00
Joey Hess
984bc8570d improve usage messages 2013-04-20 20:23:11 -04:00
Joey Hess
911506bc93 untested browser opening on Android
I've found multiple references to using the `am` command from the adb shell
to open a browser. So I assume it also works in a terminal emulator.
2013-04-18 12:52:55 -04:00
Joey Hess
22afcdf2a5 fsck: Check content of direct mode files (only when the inode cache thinks they are unmodified).
I wrote this earlier, but it never worked because it was looking at the
.git/annex/object content, which is not there..
2013-04-16 16:20:30 -04:00
Joey Hess
11d106a18c turn on PackageImports globally
This will make it easier to use the Evil Splicer, when it needs to add
package qualified imports

And there's no real downside.
2013-04-13 18:12:44 -04:00
Joey Hess
9e11699c76 connect existing meters to the transfer log for downloads
Most remotes have meters in their implementations of retrieveKeyFile
already. Simply hooking these up to the transfer log makes that information
available. Easy peasy.

This is particularly valuable information for encrypted remotes, which
otherwise bypass the assistant's polling of temp files, and so don't have
good progress bars yet.

Still some work to do here (see progressbars.mdwn changes), but this
is entirely an improvement from the lack of progress bars for encrypted
downloads.
2013-04-11 17:32:31 -04:00
Joey Hess
1eb3fff787 addurl: Register transfer so the webapp can see it.
* addurl: Register transfer so the webapp can see it.
* addurl: Automatically retry downloads that fail, as long as some
  additional content was downloaded.
2013-04-11 16:14:17 -04:00
Joey Hess
7b4733f0e8 addurl: Bugfix: Did not properly add file in direct mode. 2013-04-11 13:35:52 -04:00
Joey Hess
5e2e4347a3 webapp: New --listen= option allows running the webapp on one computer and connecting to it from another.
Does not yet use HTTPS. I'd need to generate a certificate, and I'm not
sure what's the best way to do that.
2013-04-08 15:04:35 -04:00
Joey Hess
602baae12e Bugfix: Direct mode no longer repeatedly checksums duplicated files.
Fixed by storing a list of cached inodes for a key, instead of just one.

Backwards compatability note: An old git-annex version will fail to parse
an inode cache file that has been written by a new version, and has
multiple items. It will succees if just one. So old git-annexes will have
even worse behavior when there are duplicated files, if that is possible.
I don't think it will be a problem. (Famous last words.)

Also, note that it doesn't expire old and unused inode caches for a key.
It would be possible to add this if needed; just look through the
associated files for a key and if there are more cached inodes, throw out
any not corresponding to associated files. Unless a file is being copied
repeatedly and the old copy deleted, this lack of expiry should not be a
problem.
2013-04-06 16:07:25 -04:00
Joey Hess
f1b0a4b404 Use lower case hash directories for storing files on crippled filesystems, same as is already done for bare repositories.
* since this is a crippled filesystem anyway, git-annex doesn't use
  symlinks on it
* so there's no reason to use the mixed case hash directories that we're
  stuck using to avoid breaking everyone's symlinks to the content
* so we can do what is already done for all bare repos, and make non-bare
  repos on crippled filesystems use the all-lower case hash directories
* which are, happily, all 3 letters long, so they cannot conflict with
  mixed case hash directories
* so I was able to 100% fix this and even resuming `git annex add` in the
  test case will recover and it will all just work.
2013-04-04 15:46:33 -04:00
Joey Hess
021c564319 clean up urlrenderer handling when the webapp is not built 2013-04-03 17:48:54 -04:00
Joey Hess
0ba205f125 squelch protocol error on shutdown 2013-04-02 15:18:03 -04:00
Joey Hess
38d61f934d Update working tree files fully atomically
This avoids commit churn by the assistant when eg,
replacing a file with a symlink.

But, just as importantly, it prevents the working tree being left with a
deleted file if git-annex, or perhaps the whole system, crashes at the
wrong time.

(It also probably avoids confusing displays in file managers.)
2013-04-02 15:02:00 -04:00
Joey Hess
d87af82d97 drop --auto: Fix bug that prevented dropping files from untrusted repositories.
This is a corresponding bug to the one I fixed yesterday in the assistant.
2013-04-01 15:20:42 -04:00
Joey Hess
c57baaaa30 webapp: Added UI to delete repositories. Closes: #689847 2013-03-31 16:38:05 -04:00
Joey Hess
acd6a150e5 minor refactoring 2013-03-30 19:05:51 -04:00
Joey Hess
553e9dc736 let's use words and unwords rather than split and intercalate " " 2013-03-30 18:56:01 -04:00
guilhem
3bfe011867 Make git-annex-shell call the command with its (safe) options. 2013-03-30 18:49:43 -04:00
Joey Hess
b89efc79f6 add --force overrides annex.largefiles 2013-03-29 16:20:15 -04:00
Joey Hess
67e817c6a1 New annex.largefiles setting, which configures which files git annex add and the assistant add to the annex.
I would have sort of liked to put this in .gitattributes, but it seems
it does not support multi-word attribute values. Also, making this a single
config setting makes it easy to only parse the expression once.

A natural next step would be to make the assistant `git add` files that
are not annex.largefiles. OTOH, I don't think `git annex add` should
`git add` such files, because git-annex command line tools are
not in the business of wrapping git command line tools.
2013-03-29 16:17:13 -04:00
Joey Hess
cf07a2c412 webapp: Progess bar fixes for many types of special remotes.
There was confusion in different parts of the progress bar code about
whether an update contained the total number of bytes transferred, or the
number of bytes transferred since the last update. One way this bug
showed up was progress bars that seemed to stick at zero for a long time.
In order to fix it comprehensively, I add a new BytesProcessed data type,
that is explicitly a total quantity of bytes, not a delta.

Note that this doesn't necessarily fix every problem with progress bars.
Particularly, buffering can now cause progress bars to seem to run ahead
of transfers, reaching 100% when data is still being uploaded.
2013-03-28 17:04:37 -04:00
Joey Hess
50e2ea3825 Per-command usage messages. 2013-03-27 13:51:24 -04:00
Joey Hess
cfd3b16fe1 add section metadata to all commands
Not yet used .. mindless train work.
2013-03-24 18:28:21 -04:00
Joey Hess
2c26bd1c65 2 minor fixes to transferkeys, otherwise it was perfect 1st time!
Needed to send a trailing NUL to end a request, and set the read handle
non-blocking.

Also, set fileSystemEncoding on all handles, since there's a filename in
there.
2013-03-20 13:24:35 -04:00
Joey Hess
b6d691aff7 maintain pools of running transferkeys processes (untested) 2013-03-19 18:46:29 -04:00
Joey Hess
ef3221181d implement transferkeys plumbing command 2013-03-19 16:58:36 -04:00
Joey Hess
6314be1e3c map: Combine duplicate repositories, for a nicer looking map. 2013-03-16 16:31:59 -04:00
Joey Hess
70b7555eaf fix relaxed with existing file 2013-03-12 15:58:36 -04:00
Joey Hess
de6f74ac88 addurl: Add --relaxed option. 2013-03-11 19:55:01 -04:00
Joey Hess
06046a0d2b finish fast direct mode rename handling. wow, it's fast 2013-03-11 14:14:45 -04:00
Joey Hess
40df015d90 remove Eq instance for InodeCache
There are two types of equality here, and which one is right varies,
so this forces me to consider and choose between them.

Based on this, I learned that the commit in git anex sync was
always doing a strong comparison, even when in a repository where
the inodes had changed. Fixed that.
2013-03-11 02:57:48 -04:00
Joey Hess
0deb0a9658 skip non-directories in status 2013-03-11 01:37:26 -04:00
Joey Hess
baf226e313 status: Can now be run with a directory path to show only the status of that directory, rather than the whole annex. 2013-03-11 01:22:56 -04:00
Joey Hess
a3eac50fe9 bugfix: drop --from an unavailable remote no longer updates the location log, incorrectly, to say the remote does not have the key.
The comments correctly noted that the remote could drop the key and
yet False be returned due to some problem that occurred afterwards.
For example, if it's a network remote, it could drop the key just
as the network goes down, and so things timeout and a nonzero exit
from ssh is propigated through and False returned.

However... Most of the time, this scenario will not have happened.
False will mean the remote was not available or could not drop the key
at all.

So, instead of assuming the worst, just trust the status we have.

If we get it wrong, and the scenario above happened, our location
log will think the remote has the key. But the remote's location
log (assuming it has one) will know it dropped it, and the next sync
will regain consistency.

For a special remote, with no location log, our location log will be wrong,
but this is no different than the situation where someone else dropped
the key from the remote and we've not synced with them. The standard
paranoia about not trusting the location log to be the last word about
whether a remote has a key will save us from these situations. Ie,
if we try to drop the file, we'll actively check the remote,
and determine the inconsistency then.
2013-03-10 19:15:53 -04:00
Joey Hess
0155f1effa add build flags to version output 2013-03-09 14:57:23 -04:00
Joey Hess
921f29c004 two types of byName
Clean up from 9769235d6b.
In some cases, looking up a remote by name even though it has no UUID is
desirable. This includes git annex sync, which can operate on remotes
without an annex, and XMPP pairing, which runs addRemote (with calls
byName) before the UUID of the XMPP remote has been configured in git.
2013-03-05 15:43:56 -04:00
Joey Hess
08bdea7e52 webapp: New preferences page allows enabling/disabling debug logging at runtime, as well as configuring numcopies and diskreserve. 2013-03-03 17:07:27 -04:00
Joey Hess
0c13d3065e git subcommand cleanup
Pass subcommand as a regular param, which allows passing git parameters
like -c before it. This was already done in the pipeing set of functions,
but not the command running set.
2013-03-03 13:39:07 -04:00
Joey Hess
3a32454b13 break dependency cycle by special casing running of test command 2013-02-27 15:51:28 -04:00
Joey Hess
d1dbf266f6 embed test suite into git annex; available by running: git annex test
I have seen some other programs do this, and think it's pretty cool. Means
you can test wherever it's deployed, as well as at build time.

My other reason for doing it is less happy. Cabal's handling of test suites
sucks, requiring duplicated info, and even when that's done, it fails to
preprocess hsc files here. Building it in avoids that and avoids having
to explicitly tell cabal to enable test suites, which would then make it
link the test executable every time, which is unnecessarily slow.

This also has the benefit that now "make fast test" does a max speed build
and tests it.
2013-02-27 15:38:21 -04:00
Joey Hess
cbd53b4a8c Makefile now builds using cabal, taking advantage of cabal's automatic detection of appropriate build flags.
The only thing lost is ./ghci

Speed: make fast used to take 20 seconds here, when rebuilding from
touching Command/Unused.hs. With cabal, it's 29 seconds.
2013-02-27 02:39:22 -04:00
Joey Hess
2d9c046dea annex.version is now set to 4 for direct mode repositories
To avoid old versions of git-annex getting confused.

There is no upgrade required though.
We switch back to 3 when going from direct to indirect.
2013-02-26 15:13:10 -04:00
Joey Hess
3b92c279e8 copy: Update location log when no copy was performed, if the location log was out of date. 2013-02-26 14:39:37 -04:00
Joey Hess
52902c0945 make adding modified files work on crippled filesystems 2013-02-20 14:12:55 -04:00
Joey Hess
af1da07302 Direct mode: Fix support for adding a modified file.
Adding a file that is already annexed, but has been modified, was broken in
direct mode.

This fix makes the new content be added. It does have the problem that
re-running `git annex add` will checksum and re-add the content repeatedly,
until it's committed. This happens because the key associated with the file
does not change until the new one gets committed, so it keeps thinking the
file has changed.
2013-02-20 13:37:46 -04:00
Joey Hess
d7c93b8913 fully support core.symlinks=false in all relevant symlink handling code
Refactored annex link code into nice clean new library.

Audited and dealt with calls to createSymbolicLink.
Remaining calls are all safe, because:

Annex/Link.hs:  ( liftIO $ createSymbolicLink linktarget file
  only when core.symlinks=true
Assistant/WebApp/Configurators/Local.hs:                createSymbolicLink link link
  test if symlinks can be made
Command/Fix.hs: liftIO $ createSymbolicLink link file
  command only works in indirect mode
Command/FromKey.hs:     liftIO $ createSymbolicLink link file
  command only works in indirect mode
Command/Indirect.hs:                    liftIO $ createSymbolicLink l f
  refuses to run if core.symlinks=false
Init.hs:                createSymbolicLink f f2
  test if symlinks can be made
Remote/Directory.hs:    go [file] = catchBoolIO $ createSymbolicLink file f >> return True
  fast key linking; catches failure to make symlink and falls back to copy
Remote/Git.hs:          liftIO $ catchBoolIO $ createSymbolicLink loc file >> return True
  ditto
Upgrade/V1.hs:                          liftIO $ createSymbolicLink link f
  v1 repos could not be on a filesystem w/o symlinks

Audited and dealt with calls to readSymbolicLink.
Remaining calls are all safe, because:

Annex/Link.hs:		( liftIO $ catchMaybeIO $ readSymbolicLink file
  only when core.symlinks=true
Assistant/Threads/Watcher.hs:		ifM ((==) (Just link) <$> liftIO (catchMaybeIO $ readSymbolicLink file))
  code that fixes real symlinks when inotify sees them
  It's ok to not fix psdueo-symlinks.
Assistant/Threads/Watcher.hs:		mlink <- liftIO (catchMaybeIO $ readSymbolicLink file)
  ditto
Command/Fix.hs:	stopUnless ((/=) (Just link) <$> liftIO (catchMaybeIO $ readSymbolicLink file)) $ do
  command only works in indirect mode
Upgrade/V1.hs:	getsymlink = takeFileName <$> readSymbolicLink file
  v1 repos could not be on a filesystem w/o symlinks

Audited and dealt with calls to isSymbolicLink.
(Typically used with getSymbolicLinkStatus, but that is just used because
getFileStatus is not as robust; it also works on pseudolinks.)
Remaining calls are all safe, because:

Assistant/Threads/SanityChecker.hs:                             | isSymbolicLink s -> addsymlink file ms
  only handles staging of symlinks that were somehow not staged
  (might need to be updated to support pseudolinks, but this is
  only a belt-and-suspenders check anyway, and I've never seen the code run)
Command/Add.hs:         if isSymbolicLink s || not (isRegularFile s)
  avoids adding symlinks to the annex, so not relevant
Command/Indirect.hs:                            | isSymbolicLink s -> void $ flip whenAnnexed f $
  only allowed on systems that support symlinks
Command/Indirect.hs:            whenM (liftIO $ not . isSymbolicLink <$> getSymbolicLinkStatus f) $ do
  ditto
Seek.hs:notSymlink f = liftIO $ not . isSymbolicLink <$> getSymbolicLinkStatus f
  used to find unlocked files, only relevant in indirect mode
Utility/FSEvents.hs:                    | Files.isSymbolicLink s = runhook addSymlinkHook $ Just s
Utility/FSEvents.hs:                                            | Files.isSymbolicLink s ->
Utility/INotify.hs:                             | Files.isSymbolicLink s ->
Utility/INotify.hs:                     checkfiletype Files.isSymbolicLink addSymlinkHook f
Utility/Kqueue.hs:              | Files.isSymbolicLink s = callhook addSymlinkHook (Just s) change
  all above are lower-level, not relevant

Audited and dealt with calls to isSymLink.
Remaining calls are all safe, because:

Annex/Direct.hs:			| isSymLink (getmode item) =
  This is looking at git diff-tree objects, not files on disk
Command/Unused.hs:		| isSymLink (LsTree.mode l) = do
  This is looking at git ls-tree, not file on disk
Utility/FileMode.hs:isSymLink :: FileMode -> Bool
Utility/FileMode.hs:isSymLink = checkMode symbolicLinkMode
  low-level

Done!!
2013-02-17 16:43:14 -04:00
Joey Hess
397082013a proper fix for dropunused
Now getKeysPresent checks that the key's content, not only its directory,
exists. In direct mode, the inode cache file is used as a standin for the
content.

removeAnnex always removes the inode cache file, and drop and move --from
always call removeAnnex, even if the object does not seem to be inAnnex,
to ensure it's always deleted.
2013-02-15 17:58:49 -04:00
Joey Hess
5a8fb26d0a Revert "Clean up direct mode cache and mapping info when dropping keys."
This reverts commit 57780cb3a4.

This was buggy, it caused the direct mode cache to be lost when dropping
keys, so when the file is gotten back, it's stored in indirect mode.

Note to self: Do not attempt bug fixes at 6 am!
2013-02-15 16:37:57 -04:00
Joey Hess
2e49a7e729 don't allow setting indirect mode on a crippled filesystem 2013-02-15 14:17:31 -04:00
Joey Hess
5e6a60c17d migrate, rekey: copy rather than hard linking in crippled filesystem mode 2013-02-15 13:51:50 -04:00
Joey Hess
7ce30b534f add: Improved detection of files that are modified while being added.
In indirect mode, now checks the inode cache to detect changes to a file.
Note that a file can still be changed if a process has it open for write,
after landing in the annex.

In direct mode, some checking of the inode cache was done before, but
from a much later point, so fewer modifications could be detected. Now it's
as good as indirect mode.

On crippled filesystems, no lock down is done before starting to add a
file, so checking the inode cache is the only protection we have.
2013-02-14 16:54:36 -04:00
Joey Hess
a52f8f382b split out Utility.InodeCache 2013-02-14 16:17:40 -04:00
Joey Hess
47477b2807 crippled filesystem support, probing and initial support
git annex init probes for crippled filesystems, and sets direct mode, as
well as `annex.crippledfilesystem`.

Avoid manipulating permissions of files on crippled filesystems.
That would likely cause an exception to be thrown.

Very basic support in Command.Add for cripped filesystems; avoids the lock
down entirely since doing it needs both permissions and hard links.
Will make this better soon.
2013-02-14 14:15:26 -04:00
Joey Hess
43b4b7d43a can now build Android targeted binary
Various things that don't work on Android are just ifdefed out.

* the webapp (needs template haskell for arm)
* --include and --exclude globbing (needs libpcre, which is not ported;
  probably I'll make it use the pure haskell glob library instead)
* annex.diskreserve checking (missing sys/statvfs.h)
* timestamp preservation support (yawn)
* S3
* WebDAV
* XMPP

The resulting 17mb binary has been tested on Android, and it is able to,
at least, print its usage message.
2013-02-10 15:48:38 -04:00
Joey Hess
57780cb3a4 Clean up direct mode cache and mapping info when dropping keys.
These files were left behind, and made getKeysPresent find keys that were
not present. It would be expensive to make getKeysPresent check that the
actual key files are present (it just lists the directories). But that's not
needed if we just clean up the stale cache and mapping files.

To handle systems that were in direct mode and got switched back with stale
direct mode files, made cleanObjectLoc remove all files in the key's directory.

git annex unused will still list keys that are gone but for which the stale
direct mode files exists. To deal with that, made dropunused remove the key's
directory even if the key does not seem to be present.
2013-02-07 08:28:40 -04:00
Joey Hess
b1de99c1d4 uninit, unannex --fast: If hard link creation fails, fall back to slow mode. 2013-02-06 14:02:18 -04:00
Joey Hess
547d7745fb pre-commit: Update direct mode mappings.
Making the pre-commit hook look at git diff-index to find changed direct
mode files and update the mappings works pretty well.

One case where it does not work is when a file is git annex added, and then
git rmed, and then this is committed. That's a no-op commit, so the hook
probably doesn't even run, and it certianly never notices that the file
was deleted, so the mapping will still have the original filename in it.

For this and other reasons, it's important that the mappings still be
treated as possibly inconsistent.

Also, the assistant now allows the pre-commit hook to run when in direct
mode, so the mappings also get updated there.
2013-02-06 12:44:19 -04:00
Joey Hess
39d5f3f11c avoid queueing rm of no files 2013-02-05 15:11:05 -04:00
Joey Hess
b19c2e6122 assistant: Fix location log when adding new file in direct mode. 2013-02-05 13:41:48 -04:00
Joey Hess
76ddf9b6d3 webapp: Now allows restarting any threads that crash. 2013-01-26 17:09:33 +11:00
Joey Hess
1713ed95f7 use async to track and manage threads 2013-01-26 14:14:32 +11:00
Joey Hess
672f8b5b83 fsck: Detect and fix consistency errors in direct mode mapping files. 2013-01-19 14:11:23 -04:00
Joey Hess
49f4ba297c sync: Automatic merge conflict resolution now stages deleted files. 2013-01-17 21:19:00 -04:00
Joey Hess
7272179979 avoid running pre-commit hook in direct mode
The code that handles committing unlocked files in indirect mode did
something unexpected and data lossy.
2013-01-17 14:11:01 -04:00
Joey Hess
52e6eeaf06 drop: fix misleading message 2013-01-16 21:44:42 -04:00
Joey Hess
d7ca6fb856 webapp: Now always logs to .git/annex/daemon.log
It used to not log to daemon.log when a repository was first created, and
when starting the webapp. Now both do. Redirecting stdout and stderr to the
log is tricky when starting the webapp, because the web browser may want to
communicate with the user. (Either a console web browser, or web.browser = echo)
This is handled by restoring the original fds when running the browser.
2013-01-15 13:34:59 -04:00
Joey Hess
f51ad2a00c assistant: Avoid committer crashing if a file is deleted at the wrong instant. 2013-01-14 15:02:13 -04:00
Joey Hess
18a6935e42 safe recv-key in direct mode
Checks the key's size and checksum. This is sorta expensive, but it avoids
needing to add another round-trip to the protocol.
2013-01-11 16:03:45 -04:00
Joey Hess
2e11a6013b drop: Suggest using git annex move when numcopies prevents dropping a file. 2013-01-09 18:53:59 -04:00
Joey Hess
1bc49b7158 Special remotes now all rollback storage of keys that get modified during the transfer, which can happen in direct mode. 2013-01-09 18:42:29 -04:00
Joey Hess
0da2507fd6 improve direct mode fsck
An earlier commit (mislabeled) made direct mode fsck check file checksums.
While it's expected for files to change at any time in direct mode, and so
fsck cannot complain every time there's a checksum mismatch, it is possible
for it to detect when a file does not *seem* to have changed, then check
its checksum, and so detect disk corruption or other problems.

This commit improves that, by checking a second time, if the checksum
fails, that the file is still not modified, before taking action. This way,
a direct mode file can be modified while being fscked.
2013-01-08 15:07:00 -04:00
Joey Hess
174867b846 blog for yesterday 2013-01-08 12:41:09 -04:00
Joey Hess
fc63e3b660 fix a stupid typo that made fsck loop when it found bad content
Thank goodness for test suites!
2013-01-07 13:01:53 -04:00
Joey Hess
248090064d addurl in direct mode 2013-01-06 17:34:44 -04:00
Joey Hess
858ad6783b add works in direct mode
Also, changed sync to no longer automatically add files in direct mode.
That was only necessary before because add didn't work.
2013-01-06 17:24:22 -04:00
Joey Hess
f12202f771 optimize pre-commit in direct mode 2013-01-06 16:56:55 -04:00
Joey Hess
9d3e571f77 support fsck in direct mode 2013-01-06 15:42:49 -04:00
Joey Hess
b68eee625f More commands work in direct mode repositories: find, whereis, move, copy, drop, log.
These started working, for free, once lookupFile supported direct mode.

yay!!
2013-01-05 17:17:04 -04:00
Joey Hess
aedfcde969 guard readSymbolicLink
throws an exception if the file is not a symlink
2013-01-05 16:07:27 -04:00
Joey Hess
20fafc6a2d avoid pre-commit in direct mode
It was a no-op until my recent change that made lookupFile work in direct
mode.
2013-01-05 16:06:20 -04:00
Joey Hess
15ecce2bfd squelch warning 2013-01-05 15:09:43 -04:00
Joey Hess
bf1981f60e committer: Fix a file handle leak. 2013-01-05 13:42:31 -04:00
Joey Hess
4008590c68 type based git config handling for remotes
Still a couple of places that use git config ad-hoc, but this is most of it
done.
2013-01-01 13:58:14 -04:00
Joey Hess
7f7c31df1c type based git config handling
Now there's a Config type, that's extracted from the git config at startup.
Note that laziness means that individual config values are only looked up
and parsed on demand, and so we get implicit memoization for all of them.
So this is not only prettier and more type safe, it optimises several
places that didn't have explicit memoization before. As well as getting rid
of the ugly explicit memoization code.

Not yet done for annex.<remote>.* configuration settings.
2012-12-29 23:10:18 -04:00
Joey Hess
92287f6905 ensure that direct mode file is not modified while generating its key 2012-12-29 15:32:29 -04:00
Joey Hess
e872c3f648 convert notBareRepo to a CommandCheck
This avoids some small overhead by only running the check once per command;
it also ensures that, even if the command doesn't find anything to run on,
it still fails to run when in a bare repo.
2012-12-29 14:45:19 -04:00
Joey Hess
2ce736ac50 block all commands that don't work in direct mode
I left status working in direct mode, although it doesn't show correct
stats for known annex keys.
2012-12-29 14:28:19 -04:00
Joey Hess
8a8380f1b7 use sync command merge engine in assistant
To handle direct mode merging.
2012-12-25 14:10:07 -04:00
Joey Hess
c3a35eb857 add a guard against using git annex add in direct mode repo
Currently, it deletes files when run in one, so until I get a chance to fix
it, block foot shooting.
2012-12-24 14:54:36 -04:00
Joey Hess
c6d2bbe402 assistant adding of files in direct mode 2012-12-24 13:37:29 -04:00
Joey Hess
e71f85645e handle sha*sum's leading \ in checksum with certian unsual filenames
* Bugfix: Remove leading \ from checksums output by sha*sum commands,
  when the filename contains \ or a newline. Closes: #696384
* fsck: Still accept checksums with a leading \ as valid, now that
  above bug is fixed.
* migrate: Remove leading \ in checksums
2012-12-20 17:07:10 -04:00
Joey Hess
ddb0adb998 more quickcheck fun 2012-12-19 16:36:19 -04:00
Joey Hess
915cd7f676 comment 2012-12-19 12:50:24 -04:00
Joey Hess
05ec4587dd partial and incomplete automatic merging in direct mode
Handles our file right, but not theirs.
2012-12-18 17:15:16 -04:00
Joey Hess
53dbcce645 direct mode merging works!
Automatic merge resoltion code needs to be fixed to preserve objects from
direct mode files.
2012-12-18 15:04:44 -04:00
Joey Hess
d62a58b9c8 Merge branch 'master' into desymlink 2012-12-18 12:36:29 -04:00
Joey Hess
77931c1e92 vicfg: Quote filename. Closes: #696193 2012-12-18 12:19:24 -04:00
Joey Hess
44402159dd add ok's 2012-12-13 16:02:10 -04:00
Joey Hess
2cfda59174 reorder for better display 2012-12-13 15:58:38 -04:00
Joey Hess
5df3c66a85 added direct and indirect commands 2012-12-13 15:44:56 -04:00
Joey Hess
cf129c2545 show direct/indirect mode 2012-12-13 13:48:07 -04:00
Joey Hess
ffdd08fd2e Merge branch 'master' into desymlink 2012-12-13 00:46:10 -04:00
Joey Hess
0d50a6105b whitespace fixes 2012-12-13 00:45:27 -04:00
Joey Hess
e7b8cb0063 direct mode committing 2012-12-12 19:20:38 -04:00
Joey Hess
676c78436d also update direct mode associated files in local merge 2012-12-12 13:06:03 -04:00
Joey Hess
514957914d direct mode mappings now updated by git annex sync
Still lots to do to make sync handle direct mode, but this is a good first
step.
2012-12-10 14:37:24 -04:00
Joey Hess
b4c6da9cbd Got object sending working in direct mode.
However, I don't yet have a reliable way to deal with files being modified
while they're being transferred. I have code that detects it on the sending
side, but the receiver is still free to move the wrong content into its
annex, and record that it has the content. So that's not acceptable, and
I'll need to work on it some more.

However, at this point I can use a direct mode repository as a remote and
transfer files from and to it.
2012-12-08 17:03:39 -04:00
Joey Hess
99a8a5297c --auto fixes
* get/copy --auto: Transfer data even if it would exceed numcopies,
  when preferred content settings want it.
* drop --auto: Fix dropping content when there are no preferred content
  settings.
2012-12-06 13:22:16 -04:00
Joey Hess
2525fefbb9 The standalone builds now unset their special path and library path variables before running the system web browser.
Should fix a crash reported on OSX.
2012-11-27 17:05:29 -04:00
Joey Hess
e80ca5f43d formatting 2012-11-25 15:52:35 -04:00
Joey Hess
0ab05c32c8 avoid commits when running fix and find 2012-11-24 17:58:16 -04:00
Joey Hess
463cf58140 webapp and assistant glacier support 2012-11-24 16:30:15 -04:00
Joey Hess
5f977cc725 directory special remote: Made more efficient and robust.
Files are now written to a tmp directory in the remote, and once all
chunks are written, etc, it's moved into the final place atomically.

For now, checkpresent still checks every single chunk of a file, because
the old method could leave partially transferred files with some chunks
present and others not.
2012-11-19 13:18:23 -04:00
Joey Hess
83993a2ba0 remove showOutput; git is run in quiet mode 2012-11-15 15:19:02 -04:00
Joey Hess
ebd576ebcb where indentation 2012-11-12 01:05:04 -04:00
Joey Hess
887fe1714b flush stdout
It's block-buffered here.
2012-11-09 14:33:34 -04:00
Joey Hess
82ccb385e3 use xmpp::user@host for xmpp remotes
Inject the required git-remote-xmpp into PATH when running xmpp git push.
Rest of the time it will not be in PATH, and git won't be able to talk to
xmpp remotes.
2012-11-09 13:35:23 -04:00
Joey Hess
cb7523b9e8 add xmppgit command; roughed out xmpp push protocol and design 2012-11-06 00:59:20 -04:00
Joey Hess
8f08aa3f45 better handling of lifting from XMPP -> Assistant 2012-11-05 19:39:08 -04:00
Joey Hess
68118b8986 split remaining assistant types 2012-10-30 14:34:48 -04:00
Joey Hess
f78ca9bc58 split out daemonstatus types 2012-10-30 14:11:14 -04:00
Joey Hess
4e765327ca Assistant monad, stage 1
This adds the Assistant monad, and an AssistantData structure.
So far, none of the assistant's threads run in the monad yet.
2012-10-29 00:15:43 -04:00
Joey Hess
4ac2fd0a22 ensure that git-annex branch is pushed after a successful transfer
I now have this topology working:

   assistant ---> {bare repo, special remote} <--- assistant

And, I think, also this one:

        +----------- bare repo --------+
        v                              v
  assistant ---> special remote <--- assistant

While before with assistant <---> assistant connections, both sides got
location info updated after a transfer, in this topology, the bare repo
*might* get its location info updated, but the other assistant has no way to
know that it did. And a special remote doesn't record location info,
so transfers to it won't propigate out location log changes at all.

So, for these to work, after a transfer succeeds, the git-annex branch
needs to be pushed. This is done by recording a synthetic commit has
occurred, which lets the pusher handle pushing out the change (which will
include actually committing any still journalled changes to the git-annex
branch).

Of course, this means rather a lot more syncing action than happened
before. At least the pusher bundles together very close together pushes,
somewhat. Currently it just waits 2 seconds between each push.
2012-10-28 16:05:34 -04:00
Joey Hess
c71836269b (re)start XMPP when it's configured in the webapp 2012-10-27 00:50:14 -04:00
Joey Hess
9856641ef1 deal with mtl/monads-tf conflict
I had been using -ignore-package monads-tf to deal with this, but
the XMPP library uses monads-tf, so that also ignores it. Instead,
use PackageImports to force use of mtl in my own code.
2012-10-24 14:43:32 -04:00
Joey Hess
b05981d973 uninit: Check and abort if there are symlinks to annexed content that are not checked into git. 2012-10-22 11:54:50 -04:00
Joey Hess
b281584422 remove some more !! 2012-10-20 16:21:43 -04:00
Joey Hess
f7f34d2072 drop unwanted content in the transfer scan
This was complicated quite a bit by needing to check numcopies. I optimised
that, so it only looks up numcopies once per file, no matter how many
remotes it checks to drop from. Although it did just occur to me that
it might be better to first check if it wants to drop content, and only
then check numcopies..
2012-10-18 15:07:11 -04:00
Joey Hess
919fec85cd better fix for zombie problem, which turns out to be a zombie ssh started by rsync
When rsyncProgress pipes rsync's stdout, this turns out to cause a ssh
process started by rsync to be left behind as a zombie. I don't know why,
but my recent zombie reaping cleanup was correct, it's just that this other
zombie, that's not directly started by git-annex, was no longer reaped
due to changes in the cleanup. Make rsyncProgress reap the zombie started
by rsync, as a workaround.

FWIW, the process tree looks like this. It seems like the rsync child
is for some reason starting but not waiting on this extra ssh process.
Ssh connection caching may be involved -- disabling it seemed to change
the shape of the tree, but did not eliminate the zombie.

 9378 pts/14   S+     0:00  |           \_ rsync -p --progress --inplace -4 -e 'ssh' '-S' ...
 9379 pts/14   S+     0:00  |           |   \_ ssh ...
 9380 pts/14   S+     0:00  |           |   \_ rsync -p --progress --inplace -4 -e 'ssh' '-S' ...
 9381 pts/14   Z+     0:00  |           \_ [ssh] <defunct>
2012-10-17 00:47:52 -04:00
Joey Hess
51ef707a59 nub the autostart file
It's possible for the file to get duplicate lines in it, and if so, we want
to ignore the dups.
2012-10-14 15:19:34 -04:00
Joey Hess
4571ad9590 add help command 2012-10-13 19:07:56 -04:00
Joey Hess
9c3e1ca3c9 full analysis of ways content could stop being preferred and need to be dropped 2012-10-13 13:21:43 -04:00
Joey Hess
e52fc5ba89 vicfg: New file format, avoids ambiguity with repos that have the same description, or no description.
This is also nice in that uuids are all the same length, so the values
of each line, line up.

Also a great deal of boilerplate elimination.
2012-10-12 23:11:26 -04:00
Joey Hess
589d1711f2 git config remote.name.annex-sync can be used to control whether a remote gets synced. 2012-10-11 18:39:21 -04:00
Joey Hess
80b3952930 webapp: display message about starting web browser
One reason to do this is that on OSX, it doesn't jump to the web browser
when opening a new page. Linux seems ahead in usability here... :P
2012-10-11 15:19:48 -04:00
Joey Hess
bbf2c31aa7 better message 2012-10-11 12:14:23 -04:00
Joey Hess
c0aec874a2 webapp: avoid infinite loop on start
If the autostart file lists a repository, for which a directory exists,
but there's not actually a valid git repo in there, the web app used to
try to use it, and see it wasn't valid, and then try to autostart again.

The ensuing runaway loop also ate memory, although not as fast as I was led
to belive was happening to someone on IRC yesterday. So that guy may have
had a different problem. But this seems otherwise a reasonable fit for the
circumstances described, if git-annex was started before something that
occurred during desktop login that made the repository available.
2012-10-11 12:08:11 -04:00
Joey Hess
bf72760af2 dead: Remove dead repository from all groups.
This is less expensive than having inallgroup weed out dead repositories.
2012-10-10 15:39:13 -04:00
Joey Hess
5ac15149cc assistant: Now honors preferred content settings when deciding what to transfer.
Both when queueing downloads, and uploads, consults the preferred content
settings.

I didn't make it check yet when requeing failed transfers or queuing
deferred downloads; dealing with the preferred content settings (or indeed,
other settings) changing while the assistant is running still needs work.
2012-10-09 12:18:41 -04:00
Joey Hess
fee40dd374 generalized Annex.Wanted
this should make it easy to use from inside the assistant, where
everything is an AssociatedFile.
2012-10-08 17:14:01 -04:00
Joey Hess
1eedf495c3 make copy --to check preferred content of the remote 2012-10-08 16:06:56 -04:00
Joey Hess
17543f6e80 drop --auto --from with preferred content
With --from, it needs to examine the preferred content of the repository
being dropped from, instead of the local repository.
2012-10-08 15:34:44 -04:00
Joey Hess
71fd18a97f wired preferred content up to get, copy, and drop --auto 2012-10-08 13:16:53 -04:00
Joey Hess
47314c0fad fix last zombies in the assistant
Made Git.LsFiles return cleanup actions, and everything waits on
processes now, except of course for Seek.
2012-10-04 19:56:32 -04:00
Joey Hess
de3ea4adb6 remove now-unnecessary manual reaps 2012-10-04 18:58:57 -04:00
Joey Hess
5594bf0643 more zombie fighting
I'm down to 9 places in the code that can produce unwaited for zombies.

Most of these are pretty innocuous, at least for now, are only
used in short-running commands, or commands that run a set of
actions and explicitly reap zombies after each one.

The one from Annex.Branch.files could be trouble later,
since both Command.Fsck and Command.Unused can trigger it,
and the assistant will be doing those eventally. Ditto the one in
Git.LsTree.lsTree, which Command.Unused uses.

The only ones currently affecting the assistant though, are
in Git.LsFiles. Several threads use several of those.

(And yeah, using pipes or ResourceT would be a less ad-hoc approach,
but I don't really feel like ripping my entire code base apart right
now to change a foundation monad. Maybe one of these days..)
2012-10-04 18:47:31 -04:00
Joey Hess
f67b54e5e3 make a pipeReadStrict, that properly waits on the process
Nearly everything that's reading from git is operating on a small
amount of output and has been switched to use that. Only pipeNullSplit
stuff continues using the lazy version that yields zombies.
2012-10-04 18:04:09 -04:00
Joey Hess
bc649a35ba added preferred-content log, and allow editing it with vicfg
This includes a full parser for the boolean expressions in the log,
that compiles them into Matchers. Those matchers are not used yet.

A complication is that matching against an expression should never
crash git-annex with an error. Instead, vicfg checks that the expressions
parse. If a bad expression (or an expression understood by some future
git-annex version) gets into the log, it'll be ignored.

Most of the code in Limit couldn't fail anyway, but I did have to make
limitCopies check its parameter first, and return an error if it's bad,
rather than erroring at runtime.
2012-10-04 16:00:19 -04:00
Joey Hess
b67525ce65 tweak 2012-10-03 23:29:37 -04:00
Joey Hess
b793b77d85 finished vicfg
One note: Deleted lines are not currently parsed as config changes.
That makes sense for trust settings. It may make sense to support deleted
lines as a way to clear group settings.
2012-10-03 19:37:39 -04:00
Joey Hess
dda953bcce wrote parser 2012-10-03 19:13:21 -04:00
Joey Hess
7a7f63182c vicfg: New command, allows editing (or simply viewing) most of the repository configuration settings stored in the git-annex branch.
Incomplete; I need to finish parsing and saving. This will also be used
for editing transfer control expresssions.

Removed the group display from the status output, I didn't really
like that format, and vicfg can be used to see as well as edit rempository
group membership.
2012-10-03 17:04:52 -04:00
Joey Hess
717e008390 status: display repository groups 2012-10-02 13:45:30 -04:00
Joey Hess
5bd5bc094a simplify 2012-10-01 15:17:21 -04:00
Joey Hess
2a96b1aab3 group, ungroup: New commands to indicate groups of repositories. 2012-10-01 15:12:04 -04:00
Joey Hess
3da78cc241 make the standalone OSX app automatically install itself when run 2012-09-26 16:50:04 -04:00
Joey Hess
84d431a679 rename option 2012-09-25 19:43:33 -04:00
Joey Hess
3e297e99a3 fsck: New --incremental-restart option which is nice for scheduling eg, monthly incremental fsck runs in cron jobs. 2012-09-25 19:37:34 -04:00
Joey Hess
995b04d36f only read/set the incremental timestamp file once 2012-09-25 15:45:17 -04:00
Joey Hess
ed93d662d3 use --more rather than --new to continue incremental fsck 2012-09-25 15:07:20 -04:00
Joey Hess
e855cffa1b basic incremental fsck now working 2012-09-25 15:06:33 -04:00
Joey Hess
6885b2deda add recordStartTime and getStartTime 2012-09-25 14:16:34 -04:00
Joey Hess
1233417698 move sticky bit code into Utility.FileMode
Simplified it using existing functions.

I doubt setSticky needs to return the FileMode; if it does for some
reason, it can be changed to use modifyFileMode'

Converted isSticky to a pure function for consistency with isSymlink.
Note that the sticky bit of a file can be tested thus:
  isSticky . fileMode <$> getFileStatus file
2012-09-25 13:33:13 -04:00
Joey Hess
39d60f3570 Justin's metadata code 2012-09-25 13:22:12 -04:00
Joey Hess
9e54a29546 add --incremental and --new options
Not yet used..
2012-09-25 13:19:05 -04:00
Joey Hess
40df26757a copy: avoid updating location log when no copy is performed
git annex copy --to remote often does not need to copy a file,
but it was still updating the location log in this case.
2012-09-24 19:58:34 -04:00
Joey Hess
96b0e3d6e9 quiet calls to transferkey 2012-09-24 13:36:05 -04:00
Joey Hess
3887432c54 fixes for transfer resume
Fix resuming of downloads, which do not have a transfer info file to read.

When checking upload progress, use the MVar, rather than re-reading
the info file.

Catch exceptions in the transfer action. Required a tryAnnex.
2012-09-24 13:18:16 -04:00
Joey Hess
df07ccf404 make the assistant retry failed transfers
When a transfer fails, the progress info can be used to intelligently
retry it. If the transfer managed to make some progress, but did not
fully complete, then there's a good chance that a retry will finish it
(or at least make more progress).
2012-09-23 13:27:13 -04:00
Joey Hess
582316f66f avoid webapp crash on startup when there's no ~/.gitconfig
git config --list --global exits nonzero when there's no global config
2012-09-23 12:43:14 -04:00
Joey Hess
c048add74d hooked up git-annex-shell transferinfo
Finally done with progressbars!
2012-09-21 23:25:06 -04:00
Joey Hess
77af38ec6c git-annex-shell transferinfo command
TODO: Use this when running sendkey, to feed back transfer info from the
client side rsync.
2012-09-21 16:23:25 -04:00
Joey Hess
226781c047 unify types 2012-09-21 14:50:14 -04:00
Joey Hess
084aba8aaf fix breakage caught by test suite
Can't use show-ref --tags --branches, as that omits remote branches.
Instead, filter out the synced refs directly.
2012-09-21 00:49:48 -04:00
Joey Hess
0cb4382880 typo 2012-09-19 17:10:22 -04:00
Joey Hess
f542701620 recvkey is not an upload, so ignore the progress callback 2012-09-19 17:02:33 -04:00
Joey Hess
aff09a1f33 add a progress callback to storeKey, and threaded it all the way through
Transfer info files are updated when the callback is called, updating
the number of bytes transferred.

Left unused p variables at every place the callback should be used.
Which is rather a lot..
2012-09-19 16:08:37 -04:00
Joey Hess
45a26175d6 renamed RsyncFile -> Rsync 2012-09-19 14:28:32 -04:00
Joey Hess
ba744c84a4 better name for fallback sync refs
Don't expose these as branches in refs/heads/. Instead hide them away in
refs/synced/ where only show-ref will find them.

Make unused only look at branches and tags, not these other things,
so it won't care if some stale sync ref used to use a file.

This means they don't need to be deleted, which could have
led to an incoming sync being missed.
2012-09-16 23:09:08 -04:00
Joey Hess
601ee470af sync: Pushes the git-annex branch to remote/synced/git-annex, rather than directly to remote/git-annex.
This fixes a problem I was seeing in the assistant where two remotes would
attempt to sync with one another at the same time, and both failed pushing
the diverged git-annex branch. Then when both tried to resolve the failed
push, they each modified their git-annex branch, which again each blocked
the other from pushing into it. The result was that the git-annex
branches were perpetually diverged (despite having the same content!) and
once the assistant fell into this trap, it couldn't get out and always
had to do the slow push/fail/pull/merge/push/fail cycle.
2012-09-16 17:54:12 -04:00
Joey Hess
48fd1e629c reinject: When the provided file doesn't match, leave it where it is, rather than moving to .git/annex/bad/ 2012-09-16 01:17:48 -04:00
Joey Hess
ba0334116c more descriptive name for oneshot 2012-09-15 20:46:38 -04:00
Joey Hess
7f45baee5e migrate: Check content before generating the new key, to avoid generating a key for corrupt data. 2012-09-14 00:18:18 -04:00
Joey Hess
60c31afc38 add decodeW8 2012-09-13 19:14:29 -04:00
Joey Hess
df337bb63b hlint 2012-09-13 00:57:52 -04:00
Joey Hess
e59b0a1c88 first pass at alert buttons
They work fine. But I had to go to a lot of trouble to get Yesod to render
routes in a pure function. It may instead make more sense to have each
alert have an assocated IO action, and a single route that runs the IO
action of a given alert id. I just wish I'd realized that before the past
several hours of struggling with something Yesod really doesn't want to
allow.
2012-09-08 19:57:15 -04:00
Joey Hess
a00f1d26bc display errors when any named thread crashes 2012-09-06 14:56:04 -04:00
Joey Hess
2433f6ca5a use the ~/.config/git-annex/program file to find command when running transfers 2012-08-27 13:43:03 -04:00
Joey Hess
c58d553265 refactor 2012-08-26 14:45:47 -04:00
Joey Hess
f4ca592cd0 refactor 2012-08-26 14:34:30 -04:00