Commit graph

1966 commits

Author SHA1 Message Date
Joey Hess
9c4650358c
add KeyVariety type
Where before the "name" of a key and a backend was a string, this makes
it a concrete data type.

This is groundwork for allowing some varieties of keys to be disabled
in file2key, so git-annex won't use them at all.

Benchmarks ran in my big repo:

old git-annex info:

real	0m3.338s
user	0m3.124s
sys	0m0.244s

new git-annex info:

real	0m3.216s
user	0m3.024s
sys	0m0.220s

new git-annex find:

real	0m7.138s
user	0m6.924s
sys	0m0.252s

old git-annex find:

real	0m7.433s
user	0m7.240s
sys	0m0.232s

Surprising result; I'd have expected it to be slower since it now parses
all the key varieties. But, the parser is very simple and perhaps
sharing KeyVarieties uses less memory or something like that.

This commit was supported by the NSF-funded DataLad project.
2017-02-24 15:16:56 -04:00
Joey Hess
3afc7d83f2
noCommit for PostReceive
This was noticed because it broke the datalad test suite, which pushed
to the remote and then fetched to check if it had received the expected
branches. Auto-init caused the git-annex branch on the remote to
diverge, breaking that test.

https://github.com/datalad/datalad/issues/1319#issuecomment-281649518

The auto-init still happens, it's staged in the journal, and will be
commited by some later git-annex command when it runs. Which is fine,
it's the same as that later command doing the auto-init.

This commit was supported by the NSF-funded DataLad project
2017-02-23 18:37:02 -04:00
Joey Hess
75a15e1ad7
status: Pass --ignore-submodules=when option on to git status.
Didn't make --ignore-submodules without a value be handled because I can't
see a way to make optparse-applicative parse that. I've opened a bug
requesting a way to do that:
https://github.com/pcapriotti/optparse-applicative/issues/243
2017-02-20 17:01:24 -04:00
Joey Hess
e6857e75a6
sync hack to make updateInstead work on eg FAT
sync: When syncing with a local repository located on a crippled
filesystem, run the post-receive hook there, since it wouldn't get run
otherwise. This makes pushing to repos on FAT-formatted removable drives
update them when receive.denyCurrentBranch=updateInstead.

Made Remote.Git export onLocal, which was cleaned up to not have so many
caveats about its use.

This commit was sponsored by Jeff Goeke-Smith on Patreon.
2017-02-17 15:21:52 -04:00
Joey Hess
d074532aff
post-recive hook to make updateInstead work in direct mode and adjusted branches
* Added post-recieve hook, which makes updateInstead work with direct
  mode and adjusted branches.
* init: Set up the post-receive hook.

This commit was sponsored by Fernando Jimenez on Patreon.
2017-02-17 14:04:43 -04:00
Joey Hess
4594bece40
make git-annex:git-annex push quiet again
Recent changes had a side effect of displaying errors in the fairly
common case when this push fails. Since the synced/git-annex push
is always forced, those errors are noise, so hide again.

This means 3 separate pushes are done now, where before it only made 2.
A bit more expensive, but ssh connection caching eliminates most of
the costs.

This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2017-02-17 14:03:17 -04:00
Joey Hess
d0651bb567
make query commands not output extraneous messages
config group groupwanted numcopies schedule wanted required:  Avoid
displaying extraneous messages about repository auto-init, git-annex branch
merging, etc, when being used to get information.
2017-02-16 13:24:35 -04:00
Joey Hess
a73c8ce4a1
sync: Improve integration with receive.denyCurrentBranch=updateInstead
By displaying error messages from the remote then it fails to update
its checked out branch.

Error messages in the default receive.denyCurrentBranch are still
suppressed, which matches user expectations.

This commit was sponsored by Nick Daly on Patreon.
2017-02-15 16:13:30 -04:00
Joey Hess
f07af03018
Run ssh with -n whenever input is not being piped into it
... to avoid it consuming stdin that it shouldn't.

This fixes git-annex-checkpresentkey --batch remote, which didn't output
results for all keys passed into it.

Other git-annex commands that communicate with a remote over ssh may also
have been consuming stdin that they shouldn't have, which could have
impacted using them in eg, shell scripts. For example, a shell script
reading files from stdin and passing them to git annex drop would be
impacted by this bug, whenever git annex drop ran git-annex-shell
checkpresent, it would consume part/all of the stdin that the shell script
was supposed to consume.

Fixed by adding a ConsumeStdin parameter to Annex.Ssh.sshOptions, which
is used throughout git-annex to run ssh (in order for ssh connection
caching to work). Every call site was checked to see if it used
CreatePipe for stdin, and if not was marked NoConsumeStdin.
2017-02-15 15:08:46 -04:00
Joey Hess
2af5f727a9
forgot to compile last commit; fix mistakes 2017-02-15 13:55:06 -04:00
Joey Hess
69baa45f14
sync, merge: Fail when the current branch has no commits yet, instead of not merging in anything from remotes and appearing to succeed.
At first I wanted to make it go ahead and merge into the newborn branch,
so made it use Git.Branch.currentUnsafe to get the current branch. But that
failed:

fatal: ambiguous argument 'refs/heads/master..refs/heads/synced/master':
unknown revision or path not in the working tree.

A whole nother code path to handle merging into newborn branches seemed
excessive, so went with displaying a warning and propigating failure
status.

This commit was sponsored by Brock Spratlen on Patreon.
2017-02-14 16:09:55 -04:00
Edward Betts
0750913136
correct spelling mistakes 2017-02-12 17:30:23 -04:00
Joey Hess
c1ece47ea0
import --reinject-duplicates
This is the same as running git annex reinject --known, followed by
git-annex import. The advantage to having it in one command is that it
only has to hash each file once; the two commands have to
hash the imported files a second time.

This commit was sponsored by Shane-o on Patreon.
2017-02-09 15:41:00 -04:00
Joey Hess
f617988a29
Make import --deduplicate and --skip-duplicates only hash once, not twice
import: --deduplicate and --skip-duplicates were implemented inneficiently;
they unncessarily hashed each file twice. They have been improved to only
hash once.

The new approach is to lock down (minimally) and hash files, and then
reuse that information when importing them.

This was rather tricky, especially in detecting changes to files while
they are being imported.

The output of import changed slightly. While before it silently skipped
over files with eg --skip-duplicates, now it shows each file as it starts
to act on it. Since every file is hashed first thing, it would otherwise
not be clear what file import is chewing on. (Actually, it wasn't clear
before when any of the duplicates switches were used.)

This commit was sponsored by Alexander Thompson on Patreon.
2017-02-09 15:32:22 -04:00
Joey Hess
e7e36b6e72
import: Changed how --deduplicate, --skip-duplicates, and --clean-duplicates determine if a file is a duplicate
Before, only content known to be present somewhere was considered a
duplicate. Now, any content that has been annexed before will be considered
a duplicate, even if all annexed copies of the data have been lost.

Note that --clean-duplicates and --deduplicate still check numcopies,
so won't delete duplicate files unless there's an annexed copy.

This makes import use the same method as reinject --known.

The man page already said that duplicate meant "its content is either
present in the local repository already, or git-annex knows of another
repository that contains it, or it was present in the annex before but has
been removed now". So, this is really only bringing the implementation into
line with the man page.

This commit was sponsored by Jochen Bartl on Patreon.
2017-02-07 17:41:58 -04:00
Joey Hess
27e89aeffc
initremote: When a uuid= parameter is passed, use the specified UUID for the new special remote, instead of generating a UUID.
This can be useful in some situations, eg when the same data can be
accessed via two different special remote backends.
2017-02-07 15:10:41 -04:00
Joey Hess
5c804cf42e
add SetupStage parameter to RemoteType.setup
Most remotes have an idempotent setup that can be reused for
enableremote, but in a few cases, it needs to tell which, and whether
a UUID was provided to setup was used.

This is groundwork for making initremote be able to provide a UUID.
It should not change any behavior.

Note that it would be nice to make the UUID always be provided to setup,
and make setup not need to generate and return a UUID. What prevented
this simplification is Remote.Git.gitSetup, which needs to reuse the
UUID of the git remote when setting it up, and so has to return that
UUID.

This commit was sponsored by Thom May on Patreon.
2017-02-07 14:55:58 -04:00
Joey Hess
3439f3cc87
assistant: Make --autostart --foreground wait for the children it starts.
Before, the --foreground was ignored when autostarting.

This commit was sponsored by Denis Dzyubenko on Patreon.
2017-02-07 13:31:45 -04:00
Joey Hess
3fe9d99f24
wormhole pairing appid flag day 2021-12-31
Wormhole pairing will start to provide an appid to wormhole on 2021-12-31.
An appid can't be provided now because Debian stable is going to ship a
older version of git-annex that does not provide an appid. Assumption is
that by 2021-12-31, this version of git-annex will be shipped in a Debian
stable release. If that turns out to not be the case, this change will need
to be cherry-picked into the git-annex in Debian stable, or its wormhole
pairing will break.

This commit was sponsored by Thomas Hochstein on Patreon.
2017-02-03 15:06:40 -04:00
Joey Hess
c545701224
make sync --no-commit override annex.annex.autocommit 2017-02-03 14:36:14 -04:00
Joey Hess
b77903af48
New annex.synccontent config setting
.. which can be set to true to make git annex sync default to --content.

This may become the default at some point in the future.

As well as being configuable by git config, it can be configured by
git-annex config to control the default behavior in all clones of a
repository.

Had to add a separate --no-content switch to we can tell if it's been
explicitly set, and should override annex.synccontent. If --content was the
default, this complication would not be necessary.

This commit was sponsored by Jake Vosloo on Patreon.
2017-02-03 14:31:17 -04:00
Joey Hess
ed56dba868
annex.autocommit can be configured via git-annex config
... to control the default behavior in all clones of a repository.

This includes a new Configurable data type, so the GitConfig type indicates
which values can be configured this way.

The implementation should be quite efficient; the config log is only read
once, and only when a Configurable value has not already been set by
git-config.

Indeed, it would be nice in the future to extend this, so that git-config
is itself only read on demand. Some commands may not need to look at the
git configuration at all.

This commit was sponsored by Trenton Cronholm on Patreon.
2017-02-03 13:58:53 -04:00
Joey Hess
062286135c
unused: When large files are checked right into git, avoid buffering their contents in memory.
This makes it a little bit slower since it has to check file size,
but worth it to fix a potential memory use problem.

This commit was sponsored by Fernando Jimenez on Patreon.
2017-01-31 19:09:37 -04:00
Joey Hess
9eb10caa27
Some optimisations to string splitting code.
Turns out that Data.List.Utils.split is slow and makes a lot of
allocations. Here's a much simpler single character splitter that behaves
the same (even in wacky corner cases) while running in half the time and
75% the allocations.

As well as being an optimisation, this helps move toward eliminating use of
missingh.

(Data.List.Split.splitOn is nearly as slow as Data.List.Utils.split and
allocates even more.)

I have not benchmarked the effect on git-annex, but would not be surprised
to see some parsing of eg, large streams from git commands run twice as
fast, and possibly in less memory.

This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2017-01-31 19:06:22 -04:00
Joey Hess
183f3f7a9c
make git annex config settings editable in vicfg
This commit was sponsored by Shane-o on Patreon.
2017-01-30 17:08:05 -04:00
Joey Hess
339464e847
config: New command for storing configuration in the git-annex branch.
Any config names can be set using this; git-annex commands will only look
at specific ones that make sense and are worth the overhead of querying the
branch.

This might also be useful for storing whatever other config-type stuff the
user might want to shove into the git-annex branch.

This commit was sponsored by Jochen Bartl on Patreon.
2017-01-30 16:46:38 -04:00
Joey Hess
26d23e38f1
vicfg: Include the numcopies configuation.
Docs say vicfg can configure everything from git-annex branch,
so it ought to configure numcopies.

Note that commenting out existing numcopies does not unset it.

This commit was sponsored by Thom May on Patreon.
2017-01-30 15:27:25 -04:00
Joey Hess
b219be5100
refactor 2016-12-30 12:31:17 -04:00
Joey Hess
e92f2d1080
improve description of password prompting
Since the user does not know whether it will run su or sudo, indicate
whether the password prompt will be for root or the user's password,
when possible.

I assume that programs like gksu that can prompt for either depending on
system setup will make clear in their prompt what they're asking for.
2016-12-28 16:07:49 -04:00
Joey Hess
b68d2a4b68
webapp: full wormhole pairing UI (untested)
This commit was sponsored by Riku Voipio.
2016-12-27 16:41:35 -04:00
Joey Hess
de79be2ba6
wording 2016-12-24 16:56:56 -04:00
Joey Hess
ab66bbfeb6
Merge branch 'master' into no-xmpp 2016-12-24 15:01:55 -04:00
Joey Hess
8484c0c197
Always use filesystem encoding for all file and handle reads and writes.
This is a big scary change. I have convinced myself it should be safe. I
hope!
2016-12-24 14:46:31 -04:00
Joey Hess
e08691b393
enable-tor: When run as a regular user, test a connection back to the hidden service over tor.
This way we know that after enable-tor, the tor hidden service is fully
published and working, and so there should be no problems with it at
pairing time.

It has to start up its own temporary listener on the hidden service. It
would be nice to have it start the remotedaemon running, so that extra
step is not needed afterwards. But, there may already be a remotedaemon
running, in communication with the assistant and we don't want to start
another one. I thought about trying to HUP any running remotedaemon, but
Windows does not make it easy to do that. In any case, having the user
start the remotedaemon themselves lets them know it needs to be running
to serve the hidden service.

This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2016-12-24 12:50:23 -04:00
Joey Hess
22252e8e4c
Revert "close"
This reverts commit 3aaabc906b.

Commit contained incomplete work.
2016-12-24 12:07:15 -04:00
Joey Hess
3aaabc906b
close 2016-12-22 13:59:21 -04:00
Joey Hess
405fbd25e1
include tor-annex in hidden service directory names
To make it easier to manage/delete them etc.

Backwards compatablity is preserved for existing tor configs.
2016-12-21 14:39:32 -04:00
Joey Hess
86401f84e2
fail before suing when not in a git-annex repo 2016-12-20 17:46:14 -04:00
Joey Hess
f7ca2b92fb
enable-tor: No longer needs to be run as root.
When run by not root, su's to root automatically.

This commit was sponsored by Brock Spratlen on Patreon.
2016-12-20 17:40:36 -04:00
Joey Hess
944a6503b9
relocate tor socket out of /etc
weasel explained that apparmor limits on what files tor can read do not
apply to sockets (because they're not files). And apparently the
problems I was seeing with hidden services not being accessible had to
do with onion address propigation and not the location of the socket
file.

remotedaemon looks up the HiddenServicePort in torrc, so if it was
previously configured with the socket in /etc, that will still work.

This commit was sponsored by Denis Dzyubenko on Patreon.
2016-12-20 16:24:46 -04:00
Joey Hess
a171e576b2
rekey --force: Incorrectly marked the new key's content as being present in the local repo even when it was not. 2016-12-19 18:18:57 -04:00
Joey Hess
df5a0059ca
analysis 2016-12-19 16:28:12 -04:00
Joey Hess
50e2d97847
shorten note 2016-12-18 17:31:02 -04:00
Joey Hess
a1d6f4f9a2
improve note display 2016-12-18 17:23:20 -04:00
Joey Hess
249ddb5953
typo 2016-12-18 17:16:53 -04:00
Joey Hess
7f2e7fa271
check if wormhole is installed 2016-12-18 17:11:13 -04:00
Joey Hess
ccde0932a5
p2p --pair with magic wormhole (untested)
It builds. I have not tried to run it yet. :)

This commit was sponsored by Jake Vosloo on Patreon.
2016-12-18 16:51:41 -04:00
Joey Hess
38f9337e16
Revert "p2p --link now defaults to setting up a bi-directional link"
This reverts commit 3037feb1bf.

On second thought, this was an overcomplication of what should be the
lowest-level primitive. Let's build bi-directional links at the pairing
level with eg magic wormhole.
2016-12-16 18:26:07 -04:00
Joey Hess
bd811d3853
p2p: Added --one-way option.
This commit was sponsored by Fernando Jimenez on Patreon.
2016-12-16 16:43:37 -04:00
Joey Hess
3037feb1bf
p2p --link now defaults to setting up a bi-directional link
Both the local and remote git repositories get remotes added
pointing at one-another.

Makes pairing twice as easy!

Security: The new LINK command in the protocol can be sent repeatedly,
but only by a peer who has authenticated with us. So, it's entirely safe to
add a link back to that peer, or to some other peer it knows about.
Anything we receive over such a link, the peer could send us over the
current connection.

There is some risk of being flooded with LINKs, and adding too many
remotes. To guard against that, there's a hard cap on the number of remotes
that can be set up this way. This will only be a problem if setting up
large p2p networks that have exceptional interconnectedness.

A new, dedicated authtoken is created when sending LINK.

This also allows, in theory, using a p2p network like tor, to learn about
links on other networks, like telehash.

This commit was sponsored by Bruno BEAUFILS on Patreon.
2016-12-16 16:38:06 -04:00
Joey Hess
e67a310da1
p2p: --link no longer takes a remote name, instead the --name option can be used. 2016-12-16 15:37:50 -04:00
Joey Hess
469bfa7ff3
Make all --batch input, as well as fromkey and registerurl stdin be processed without requiring it to be in the current encoding. 2016-12-13 15:35:04 -04:00
Joey Hess
d9490685fd
metadata --batch: Fix bug when conflicting metadata changes were made in the same batch run.
1 microsecond delay is ugly.. but, maintaining an queue of a list of timestamps
and taking a new one from the queue each time around, or maintaining a timestamp
counter, would probably be slower.
2016-12-13 11:07:49 -04:00
Joey Hess
9dd510bf29
make tor hidden service work when directory watching is not available
Avoid crashing when built w/o inotify..
2016-12-09 16:40:47 -04:00
Joey Hess
8ac4126bd2
cleanup 2016-12-09 16:22:06 -04:00
Joey Hess
e152c322f8
refactor ref change watching
Added to change notification to P2P protocol.

Switched to a TBChan so that a single long-running thread can be
started, and serve perhaps intermittent requests for change
notifications, without buffering all changes in memory.

The P2P runner currently starts up a new thread each times it waits
for a change, but that should allow later reusing a thread. Although
each connection from a peer will still need a new watcher thread to run.

The dependency on stm-chans is more or less free; some stuff in yesod
uses it, so it was already indirectly pulled in when building with the
webapp.

This commit was sponsored by Francois Marier on Patreon.
2016-12-09 15:01:09 -04:00
Joey Hess
15be5c04a6
git-annex-shell, remotedaemon, git remote: Fix some memory DOS attacks.
The attacker could just send a very lot of data, with no \n and it would
all be buffered in memory until the kernel killed git-annex or perhaps OOM
killed some other more valuable process.

This is a low impact security hole, only affecting communication between
local git-annex and git-annex-shell on the remote system. (With either
able to be the attacker). Only those with the right ssh key can do it. And,
there are probably lots of ways to construct git repositories that make git
use a lot of memory in various ways, which would have similar impact as
this attack.

The fix in P2P/IO.hs would have been higher impact, if it had made it to a
released version, since it would have allowed DOSing the tor hidden
service without needing to authenticate.

(The LockContent and NotifyChanges instances may not be really
exploitable; since the line is read and ignored, it probably gets read
lazily and does not end up staying buffered in memory.)
2016-12-09 13:34:32 -04:00
Joey Hess
8e00efb938
didn't mean to commit this change yet 2016-12-08 17:10:48 -04:00
Joey Hess
43e7044b43
comment 2016-12-08 17:10:24 -04:00
Joey Hess
af41519126
convert P2P runners from Maybe to Either String
So we get some useful error messages when things fail.

This commit was sponsored by Peter Hogg on Patreon.
2016-12-08 15:47:49 -04:00
Joey Hess
e56506d83c
include error message when unable to connect to peer 2016-12-08 14:14:08 -04:00
Joey Hess
2fb6fd7434
Merge branch 'master' into tor 2016-12-07 14:32:25 -04:00
Joey Hess
0d9a11625c
remote uuid discovery in p2p --link
This also tests that we can connect to the peer.

This commit was sponsored by Jeff Goeke-Smith on Patreon.
2016-12-07 12:38:21 -04:00
Joey Hess
f61508aed4
add: Stage modified non-large files when running in indirect mode.
(This was already done in v6 mode and direct mode.)
2016-12-05 14:10:21 -04:00
Joey Hess
82d01f5619
rekey: Added --batch mode.
Would have liked to make the Parser parse the file and key pairs, but it
seems that optparse-applicative is unable to handle eg:

	many ((,) <$> argument <*> argument)

This commit was sponsored by Thomas Hochstein on Patreon.
2016-12-05 12:55:50 -04:00
Joey Hess
6246c4a6db
minor style 2016-12-05 12:16:07 -04:00
Joey Hess
b0978b0196
Merge kite:tmp/git-annex 2016-12-05 12:15:48 -04:00
Joey Hess
93852dd7e8
rmurl: --batch
* rmurl: Multiple pairs of files and urls can be provided on the
  command line.
* rmurl: Added --batch mode.

This commit was sponsored by Trenton Cronholm on Patreon.
2016-12-05 12:10:07 -04:00
Daniel Brooks
24317be646 git-annex fromkey now takes multiple pairs of keys and filenames
It also still reads from stdin when none are specified.
2016-12-05 09:59:20 -05:00
Joey Hess
3ab12ba923
implement p2p --link
This commit was sponsored by Riku Voipio.
2016-11-30 15:16:25 -04:00
Joey Hess
bfc8305814
implement p2p command 2016-11-30 14:35:24 -04:00
Joey Hess
568d81944a
avoid too-long command synopsis
It was making git-annex usage output columns far too wide
2016-11-30 14:16:57 -04:00
Joey Hess
24593aaa32
Merge branch 'master' into tor 2016-11-30 14:16:36 -04:00
Joey Hess
8354612131
prefer xdot over dot
* map: Run xdot if it's available in PATH. On OSX, the dot command
  does not support graphical display, while xdot does.
* Debian: xdot is a better interactive viewer than dot, so Suggest
  xdot, rather than graphviz.
2016-11-30 12:50:49 -04:00
Joey Hess
38425fdc39
finish git-annex enable-tor
Make it stash the address away for git-annex p2p to use later, rather
than outputting it. And, look up the UUID itself.
2016-11-29 17:30:27 -04:00
Joey Hess
398345cb26
Merge branch 'master' into tor 2016-11-29 15:45:29 -04:00
Markus Hauru
9e2073f331
Fixed typo in Schedule.hs. 2016-11-24 07:37:33 -04:00
Joey Hess
9f179ae8b9
fix regression
The file matcher needs to be run on the destination file not the tmp
file, in order for filename matches to work properly. However, it also
needs to be able to probe the file for size and mime type.

This is a quick fix to a regression. The double rename is not pretty.
It would be good to either have a way to run the largeFileMatcher
such that it is matching on the final filename but looks at the temp
file, or to make addAnnexedFile not need the temp file in a different
location.
2016-11-22 11:18:41 -04:00
Joey Hess
48d8c175f8
avoid backtrace when rekey cntent verification fails 2016-11-22 01:16:18 -04:00
Joey Hess
070fb9e624
Added git-remote-tor-annex, which allows git pull and push to the tor hidden service.
Almost working, but there's a bug in the relaying.

Also, made tor hidden service setup pick a random port, to make it harder
to port scan.

This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2016-11-21 17:27:38 -04:00
Joey Hess
6e6d1a8c15
addurl: Fix bug in checking annex.largefiles expressions using largerthan, mimetype, and smallerthan; the first two always failed to match, and the latter always matched. 2016-11-21 11:30:53 -04:00
Joey Hess
74691ddf0e
remotedaemon: serve tor hidden service 2016-11-20 15:48:12 -04:00
Joey Hess
a101b8de37
remotedaemon: Fork to background by default. Added --foreground switch to enable old behavior.
Groundwork for tor hidden services, which the remotedaemon will serve.
2016-11-20 14:50:36 -04:00
Joey Hess
95916b2ecf
Merge branch 'master' into tor 2016-11-17 12:56:27 -04:00
Joey Hess
10703dc817
improve comment 2016-11-16 16:03:23 -04:00
Joey Hess
2577f1c0a2
fsck --all --from was checking the content of files in the local repository, rather than on the special remote.
Straight up forgot to handle this case!

This commit was sponsored by Fernando Jimenez on Patreon.
2016-11-16 15:33:57 -04:00
Joey Hess
0a4479b8ec
Avoid backtraces on expected failures when built with ghc 8; only use backtraces for unexpected errors.
ghc 8 added backtraces on uncaught errors. This is great, but git-annex was
using error in many places for a error message targeted at the user, in
some known problem case. A backtrace only confuses such a message, so omit it.

Notably, commands like git annex drop that failed due to eg, numcopies,
used to use error, so had a backtrace.

This commit was sponsored by Ethan Aubin.
2016-11-15 21:29:54 -04:00
Joey Hess
556b2ded2b
sync: Pass --allow-unrelated-histories to git merge when used with git git 2.9.0 or newer.
This makes merging a remote into a freshly created direct mode repository
work the same as it works in indirect mode.

The git-annex branches would get merged in any case by a sync,
since that doesn't use git merge.

This might need to be revisited later to better mirror git's behavior.
2016-11-15 18:26:17 -04:00
Joey Hess
57d33f7923
use socket for tor hidden service
This avoids needing to bind to the right port before something else
does.

The socket is in /var/run/user/$uid/ which ought to be writable by only
that uid. At least it is on linux systems using systemd.

For Windows, may need to revisit this and use ports or something.

The first version of tor to support sockets for hidden services
was 0.2.6.3. That is not in Debian stable, but is available in
backports.

This commit was sponsored by andrea rota.
2016-11-14 16:47:56 -04:00
Joey Hess
d58148031b
remove xmpp support
I've long considered the XMPP support in git-annex a wart.
It's nice to remove it.

(This also removes the NetMessager, which was only used for XMPP, and the
daemonstatus's desynced list (likewise).)

Existing XMPP remotes should be ignored by git-annex.

This commit was sponsored by Brock Spratlen on Patreon.
2016-11-14 14:53:08 -04:00
Joey Hess
07ad19f421
git-annex enable-tor command
Tor unfortunately does not come out of the box configured to let hidden
services register themselves on the fly via the ControlPort.

And, changing the config to enable the ControlPort and a particular type
of auth for it may break something already using the ControlPort, or
lessen the security of the system.

So, this leaves only one option to us: Add a hidden service to the
torrc. git-annex enable-tor does so, and picks an unused high port for
tor to listen on for connections to the hidden service.

It's up to the caller to somehow pick a local port to listen on
that won't be used by something else. That may be difficult to do..

This commit was sponsored by Jochen Bartl on Patreon.
2016-11-14 13:48:35 -04:00
Joey Hess
5afc2eaa54
reinject --known: Avoid second, unncessary checksum of file. 2016-11-07 12:07:36 -04:00
Joey Hess
8dcf79694d
enable forwardRetry for command-line transfers
If a transfer fails for some reason, but some data managed to be sent, the
transfer will be retried. (The assistant already did this.)

Possible impacts:

* More ssh prompts if ssh needs to prompt for a password to connect to a
  host, or is prompting about some other problem like a ssh key mismatch.

* More data transfer due to retrying, epecially when a remote does not
  support resuming a transfer.

  In the worst case, a lot of data will be transferred but it fails before
  the end, and then all that data gets transferred again plus one byte more;
  repeat until it manages to get the whole file.
2016-10-26 15:38:27 -04:00
Joey Hess
0b1c061382
importfeed: Drop URL parameters from file extension.
Thanks, James MacMahon.
2016-10-17 16:02:05 -04:00
Joey Hess
8e22114735
upgrade: Handle upgrade to v6 when the repository already contains v6 unlocked files whose content is already present.
Closes https://github.com/datalad/datalad/issues/1020

The use of runWriter in scanUnlockedFiles broke due to this change;
it failed with blocked indefinitely in mvar, because the database write
handle was taken while linkFromAnnex needed to also write to it (to update
the inode cache). So, switched to using a separate runWriter for each call
to addAssociatedFileFast. A little less efficient, but not greatly; the
writes should all still be cached.
2016-10-17 15:19:47 -04:00
Joey Hess
ee309d6941
lock: Fix edge cases where data loss could occur in v6 mode.
In the case where the pointer file is in place, and not the content
of the object, lock's  performNew was called with filemodified=True,
which caused it to try to repopulate the object from an unmodified
associated file, of which there were none. So, the content of the object
got thrown away incorrectly. This was the cause (although not the root
cause) of data loss in https://github.com/datalad/datalad/issues/1020

The same problem could also occur when the work tree file is modified,
but the object is not, and lock is called with --force. Added a test case
for this, since it's excercising the same code path and is easier to set up
than the problem above.

Note that this only occurred when the keys database did not have an inode
cache recorded for the annex object. Normally, the annex object would be in
there, but there are of course circumstances where the inode cache is out
of sync with reality, since it's only a cache.

Fixed by checking if the object is unmodified; if so we don't need to
try to repopulate it. This does add an additional checksum to the unlock
path, but it's already checksumming the worktree file in another case,
so it doesn't slow it down overall.

Further investigation found a similar problem occurred when smudge --clean
is called on a file and the inode cache is not populated. cleanOldKeys
deleted the unmodified old object file in this case. This was also
fixed by checking if the object is unmodified.

In general, use of getInodeCaches and sameInodeCache is potentially
dangerous if the inode cache has not gotten populated for some reason.
Better to use isUnmodified. I breifly auited other places that check the
inode cache, and did not see any immediate problems, but it would be easy
to miss this kind of problem.
2016-10-17 13:58:43 -04:00
Joey Hess
f867fc157f
When auto-upgrading a v3 remote, avoid upgrading to version 6, instead keep it at version 5.
Fixes a bug introduced with v6 mode that I didn't notice until now.
Probably not many v3 repos left out there, and upgrading them to v6 mode
is not disastrous, only a little premature.

This commit was sponsored by Riku Voipio
2016-10-05 16:23:09 -04:00
Joey Hess
166d70db77
convert TMVars that are never left empty into TVars
This is probably more efficient, and it avoids mistakenly leaving them
empty.
2016-09-30 19:51:16 -04:00
Joey Hess
c910004d50
addurl, importfeed: Improve behavior when file being added is gitignored. 2016-09-21 17:21:48 -04:00
Joey Hess
a569f195b7
fix bugs in handing of deep branches with sync and adjusted branches
* sync: Previously, when run in a branch with a slash in its name,
  such as "foo/bar", the sync branch was "synced/bar". That conflicted
  with the sync branch used for branch "bar", so has been changed to
  "synced/foo/bar".
* adjust: Previously, when adjusting a branch with a slash in its name,
  such as "foo/bar", the adjusted branch was "adjusted/bar(unlocked)".
  That conflicted with the adjusted branch used for branch "bar",
  so has been changed to "adjusted/foo/bar(unlocked)"
* Also, running sync in an adjusted branch did not correctly sync
  changes back to the parent branch when it had a slash in its name.
  This bug has been fixed.

Eliminate use of Git.Ref.under and Git.Ref.basename; using
Git.Ref.underBase and Git.Ref.base make everything handle deep branches
correctly.

Probably noone was adjusting deep branches, and v6 is still experimental
anyway, so I'm not going to worry about the mess that was left by that bug.

In the case of git-annex sync, using a fixed git-annex with an old unfixed
one will mean they use different sync branches for a deep branch, and so
they may stop syncing until the old one is upgraded. However, that's only
a problem when syncing between repositories without going via a central
bare repository. Added a warning about this to the CHANGELOG, but it's
probably not going to affect many people at all.

This commit was sponsored by Riku Voipio.
2016-09-21 15:23:47 -04:00
Joey Hess
0e30e71e9c
info: Support being passed a treeish, and show info about the annexed files in it similar to how a directory is handled. 2016-09-15 12:51:00 -04:00
Joey Hess
3e22d60549
copy, move, mirror: Support --json and --json-progress. 2016-09-09 16:24:26 -04:00
Joey Hess
a108235565
better locking for json with -J
Avoid threads emitting json at the same time and scrambling, which was
still possible even with the buffering, just less likely.

Converted json IO actions to JSONChunk data too.
2016-09-09 15:51:34 -04:00
Joey Hess
05d4438383
addurl, get: Added --json-progress option, which adds progress objects to the json output.
This doesn't work right when used with -J yet, and there is some really
ugly hand-crafting of part of the json output.
2016-09-09 15:06:54 -04:00
Joey Hess
8ef494a833
disentangle concurrency and message type
This makes -Jn work with --json and --quiet, where before
setting -Jn disabled those options.

Concurrent json output is currently a mess though since threads output
chunks over top of one-another.
2016-09-09 12:57:42 -04:00
Joey Hess
31289da691
get -J: Download different files from different remotes when the remotes have the same costs.
Only done in -J mode because only if there's concurrency can downloading
from two remotes be faster. Without concurrency, it's likely the case that
sequential downloads from the same remote are faster than switching back
and forth between two remotes.

There is some hairy MVar code here, but basically it just keeps
the activeremotes MVar full except when deciding which remote to assign
to a thread.

Also affects gets by sync --content -J

This commit was sponsored by Jochen Bartl.
2016-09-06 12:45:21 -04:00
Joey Hess
eb469bd139
use keyLocations not loggedLocations
Skip dead remotes.
2016-09-06 11:57:45 -04:00
Joey Hess
5d70eaacaf
examimekey: Allow being run in a git repo that is not initialized by git-annex yet.
No reason not to; indeed there's no real reason to need a git repository
at all except the implementation uses the Annex monad.
2016-09-05 12:26:59 -04:00
Joey Hess
10ddf2c3bd
remove TransferObserver
unused after last commit
2016-08-03 13:46:20 -04:00
Joey Hess
f461bcae4b
Re-enable accumulating transfer failure log files for command-line actions
This was disabled in commit 61ccf95004,
because only the assistant used them, and they were clutter. But, now
--failed also uses them.

Remove the failure log files after successful transfers. Should avoid
most of the clutter problems.

Commit 61ccf95004 mentions a subtle behavior
change, which has now been reverted:

    There is one behavior change from this. If glacier is being used, and a
    manual git annex get --from glacier fails because the file isn't available
    yet, the assistant will no longer later see that failed transfer file and
    retry the get.
2016-08-03 13:41:07 -04:00
Joey Hess
1a0e2c9901
get, move, copy, mirror: Added --failed switch which retries failed copies/moves
Note that get --from foo --failed will get things that a previous get --from bar
tried and failed to get, etc. I considered making --failed only retry
transfers from the same remote, but it was easier, and seems more useful,
to not have the same remote requirement.

Noisy due to some refactoring into Types/
2016-08-03 12:37:12 -04:00
Joey Hess
f0886a1bdd
info: When run on a file now includes an indication of whether the content is present locally. 2016-07-30 12:29:59 -04:00
Joey Hess
bf3327ff25
Added metadata --batch option, which allows getting, setting, deleting, and modifying metadata for multiple files/keys. 2016-07-27 10:46:25 -04:00
Joey Hess
928fbb162d
improved use of Aeson for JSONActionItem 2016-07-26 19:50:02 -04:00
Joey Hess
870873bdaa
Removed dependency on json library; all JSON is now handled by aeson.
I've eyeballed all --json commands, and the only difference should be
that some fields are re-ordered.
2016-07-26 19:15:34 -04:00
Joey Hess
8bc8469c38
saner format for metadata --json
metadata --json output format has changed, adding a inner json object
named "fields" which contains only the fields and their values.

This should be easier to parse than the old format, which mixed up
metadata fields with other keys in the json object.

Any consumers of the old format will need to be updated.

This adds a dependency on unordered-containers for parsing MetaData
from JSON, but it's a free dependency; aeson pulls in that library.
2016-07-26 15:41:04 -04:00
Joey Hess
a030d0a8b7
allow using Aeson for streaming JSON output
Keeping Text.JSON use for now, because it seems a better fit for most of
the commands, which don't use very structured JSON objects, but just output
whatever fields suites them. But this lets Aeson be used when a more
structured data type is available to serialize to JSON.
2016-07-26 13:30:07 -04:00
Joey Hess
d13194b230
--branch, stage 2
Show branch:file that is being operated on.

I had to make ActionItem a type and not a type class because
withKeyOptions' passed two different types of values when using the type
class, and I could not get the type checker to accept that.
2016-07-20 15:23:43 -04:00
Joey Hess
847944e6b1
more generic showStart' 2016-07-20 14:03:54 -04:00
Joey Hess
bf8bf14e8e
--branch, stage 1
Added --branch option to copy, drop, fsck, get, metadata, mirror, move, and
whereis commands. This option makes git-annex operate on files that are
included in a specified branch (or other treeish).

The names of the files from the branch that are being operated on are not
displayed yet; only the keys. Displaying the filenames will need changes
to every affected command.

Also, note that --branch can be specified repeatedly. This is not really
documented, but seemed worth supporting, especially since we may later want
the ability to operate on all branches matching a refspec. However, when
operating on two branches that contain the same key, that key will be
operated on twice.
2016-07-20 12:05:26 -04:00
Joey Hess
c4d011bf3e
log: Added --all option. 2016-07-17 15:15:08 -04:00
Joey Hess
0c713a94bd
uninit: Fix crash due to trying to write to deleted keys db.
Reversion introduced by v6 mode support, affects v5 too.

Also fix a similar crash when the webapp is used to delete a repository.
2016-07-12 14:18:35 -04:00
Joey Hess
5642daa651
fsck: Fix a reversion in direct mode fsck of a file that is present when the location log thinks it is not. Reversion introduced in version 5.20151208. 2016-07-12 13:41:03 -04:00
Joey Hess
5171e98988
drop: Add --batch and --json options. 2016-07-06 11:56:11 -04:00
Joey Hess
ed8ecbea0c
get: Add --batch and --json options. 2016-07-05 08:57:45 -04:00
Joey Hess
b6b5a11601
Make git clean filter preserve the backend that was used for a file. 2016-06-09 15:17:08 -04:00
Joey Hess
7fe2ecff91
Fix update of associated files db when unlocking a file in a v6 repo. 2016-06-09 14:45:00 -04:00
Joey Hess
0bc7fee660
Make lock and unlock work in v6 repos on files whose content is not present. 2016-06-09 14:40:44 -04:00
Joey Hess
74e01a2d01
move --to: Better behavior when system is completely out of disk space; drop content from disk before writing location log.
I noticed move --to failing when there was no disk space. The file was sent
to the remote, but it crashed before it could be dropped locally. This
could fix that.
2016-06-05 13:51:22 -04:00
Joey Hess
9996e04f41
list: Do not include dead repositories. 2016-06-04 14:33:31 -04:00
Joey Hess
26887745a0
refactor isBareRepo 2016-06-02 16:59:47 -04:00
Joey Hess
fbf5045d4f
sync --content: Fix bug that caused transfers of files to be made to a git remote that does not have a UUID. This particularly impacted clones from gcrypt repositories.
Added guard in Annex.Transfer to prevent this problem at a deeper level.

I'm unhappy ith NoUUID, but having Maybe UUID instead wouldn't help either
if nothing checked that there was a UUID. Since there legitimately need to
be Remotes that do not have a UUID, I can't see a way to fix it at the type
level, short making there be two separate types of Remotes.
2016-06-02 13:50:43 -04:00
Joey Hess
1b3bde0625
enableremote: Remove annex-ignore configuration from a remote. 2016-05-24 15:58:27 -04:00
Joey Hess
b33a649a25
enableremote: Can now be used to explicitly enable git-annex to use git remotes. Using the command this way prevents other git-annex commands from probing new git remotes to auto-enable them. 2016-05-24 15:24:38 -04:00
Joey Hess
91df4c6b53
Pass the various gnupg-options configs to gpg in several cases where they were not before.
Removed the instance LensGpgEncParams RemoteConfig because it encouraged
code that does not take the RemoteGitConfig into account.

RemoteType's setup was changed to take a RemoteGitConfig,
although the only place that is able to provide a non-empty one is
enableremote, when it's changing an existing remote. This led to several
folow-on changes, and got RemoteGitConfig plumbed through.
2016-05-23 17:03:20 -04:00
Joey Hess
16efe45a35
remove unused 2016-05-23 16:46:43 -04:00
Joey Hess
eda5d9cc74
adjust: Add --fix adjustment, which is useful when the git directory is in a nonstandard place. 2016-05-16 17:18:33 -04:00
Joey Hess
76170b0457
add: Adding a v6 pointer file used to annex it; now the pointer file is added to git as-is.
(git add of a pointer file already did the right thing)
2016-05-16 15:30:40 -04:00
Joey Hess
0860731760
reorder associated file db update
There's a potential race where the smudge filter is run at the same time an
object is being downloaded. If the download finished after the inAnnex
check, and before the keys db was updated, the associated file would not
get updated with the downloaded content.

I'm not sure this closes the race; it may only narrow the window. Problem
is, the keys database needs to communicate between two different processes.
In the case of the assistant, the transferkeys command is the other
process, and it closes the db handle after getting the file. So, it should
re-open the database and so see the update that the smudge filter has
written to it. But, what if the smudge filter takes a while to update the
database?
2016-05-16 14:55:05 -04:00
Joey Hess
5f0b551c0c
assistant: Fix race in v6 mode that caused downloaded file content to sometimes not replace pointer files.
The keys database handle needs to be closed after merging, because the
smudge filter, in another process, updates the database. Old cached info
can be read for a while from the open database handle; closing it ensures
that the info written by the smudge filter is available.

This is pretty horribly ad-hoc, and it's especially nasty that the
transferrer closes the database every time.
2016-05-16 14:49:12 -04:00
Joey Hess
9f05be393e
adjust: If the adjusted branch already exists, avoid overwriting it, since it might contain changes that have not yet been propigated to the original branch.
Could not think of a foolproof way to detect if the old adjusted branch was
just behind the current branch. It's possible that the user amended the
adjusting commit at the head of the adjusted branch, for example.

I decided to bail in this situation, instead of just entering the old
branch, so that if git annex adjust succeeds the user is always in a
*current* adjusted branch, not some old and out of date one.

What could perhaps be done is enter the old branch and then update it. But
that seems too magical; the user may have rebased master or something or
may not want to propigate the changes from the old branch. Best to error
out.
2016-05-13 14:04:22 -04:00
Joey Hess
ae65aecb0b
fsck: When a key is not previously known in the location log, record something so that reinject --known will work. 2016-05-10 13:20:45 -04:00
Joey Hess
9169234c34
fix overindent 2016-05-10 13:08:24 -04:00
Joey Hess
0897da391e
version: Display OS version and architecture too. 2016-05-05 16:06:01 -04:00
Joey Hess
93192b9279
fix build warning on windows and android 2016-05-05 15:49:56 -04:00
Joey Hess
c608df5802
map: Hide dead repositories that are not connected to the graph.
* map: Hide dead repositories that are not connected to the graph.
* map: Changed colors; red is used for untrusted repositories and grey
  for dead.
2016-05-04 14:12:41 -04:00
Joey Hess
617f2834fc
Fix bug that sometimes prevented git-annex smudge --clean from consuming all its input, which resulted in git add bypassing git-annex. 2016-05-02 10:53:24 -04:00
Joey Hess
7945dd3c3e
refactor 2016-04-22 14:35:48 -04:00
Joey Hess
46e3319995
assistant: Deal with upcoming git's refusal to merge unrelated histories by default
git 2.8.1 (or perhaps 2.9.0) is going to prevent git merge from merging in
unrelated branches. Since the webapp's pairing etc features often combine
together repositories with unrelated histories, work around this behavior
change by setting GIT_MERGE_ALLOW_UNRELATED_HISTORIES when the assistant
merges.

Note though that this is not done for git annex sync's merges, so
it will follow git's default or configured behavior.
2016-04-22 14:26:44 -04:00
Joey Hess
8ab27235ea
reinject: Added new mode which can reinject known files into the annex.
For example: git-annex reinject --known /mnt/backup/*
2016-04-22 13:49:32 -04:00
Joey Hess
0273cd5005
adjusted branches need git 2.2.0 or newer
When git-annex is used with a git version older than 2.2.0, disable support for
adjusted branches, since GIT_COMMON_DIR is needed to update them and was first
added in that version of git.
2016-04-22 12:29:32 -04:00
Joey Hess
1bfea9b3e5
calckey: New plumbing command, calculates the key that would be used to refer to a file 2016-04-20 13:50:26 -04:00
Joey Hess
9d952fe9d1
reinject: When src file's content cannot be verified, leave it alone, instead of deleting it. 2016-04-20 13:21:56 -04:00
Joey Hess
bd516af734
fsck: Warn when core.sharedRepository is set and an annex object file's write bit is not set and cannot be set due to the file being owned by a different user.
Made all Annex.Perms file mode changing functions ignore errors when
core.sharedRepository is set, because the file might be owned by someone
else. I don't fancy getting bug reports about crashes due to set modes in
this configuration, which is a very foot-shooty configuration in the first
place.

The fsck warning is necessary because old repos kept files mode 444, which
doesn't allow locking them, and so if the mode remains 444 due to the file
being owned by someone else, the user should be told about it.
2016-04-14 15:36:53 -04:00
Joey Hess
b7c8bf5274
Preserve execute bits of unlocked files in v6 mode.
When annex.thin is set, adding an object will add the execute bits to the
work tree file, and this does mean that the annex object file ends up
executable.

This doesn't add any complexity that wasn't already present, because git
annex add of an executable file has always ingested it so that the annex
object ends up executable.

But, since an annex object file can be executable or not, when populating
an unlocked file from one, the executable bit is always added or removed
to match the mode of the pointer file.
2016-04-14 14:47:08 -04:00
Joey Hess
1010482bc7
webapp: When $HOME is a git repository, and has been initialized for use by git-annex, opening the webapp went ahead and ran the assistant there, annexing all files. Since this is almost certianly not desirable, especially when the user is just opening the webapp from a dekstop menu which happens to run it in $HOME, the webapp will now not treat such a $HOME git repository as a git-annex repository. 2016-04-13 14:07:50 -04:00
Joey Hess
402e5adab8
smudge: Print a warning when annex.thin is set, as git's smudge interface does not allow honoring that configuration. 2016-04-13 13:34:24 -04:00
Joey Hess
c47d7d0f7e
correct comment 2016-04-13 13:04:38 -04:00
Joey Hess
5e190913a4
add AdjBranch newtype; some simplications 2016-04-09 15:10:26 -04:00
Joey Hess
cf06dac2b8
hard links on windows
* annex.thin and annex.hardlink are now supported on Windows.
* unannex --fast now makes hard links on Windows.
2016-04-08 15:25:32 -04:00
Joey Hess
887ef93a7f
run out of tree merge with --no-ff
This is how direct mode does it too, and somehow, for reasons that
currently escape me, this makes git merge not care if it's run with an
empty work tree.
2016-04-06 18:40:28 -04:00
Joey Hess
60bdffe43e
fix auto merge conflict resolution when doing out of tree merge for adjusted branch 2016-04-06 17:32:04 -04:00
Joey Hess
eb9ac8d6d7
sync: Show output of git commit.
Rationalle: User might have hook scripts whose output they want to see.
Also, git commit output may tell the user they forgot to add a file.
The output is not too ugly when there's nothing to commit.
2016-04-05 16:22:21 -04:00
Joey Hess
ed3e8e1886
Merge branch 'adjustedbranch' 2016-03-31 19:05:47 -04:00
Joey Hess
12ddb6e8b2
fixed merging of changes from adjusted branch + a remote 2016-03-31 18:54:35 -04:00
Joey Hess
f08149207c
autoinit on upgrade 2016-03-31 17:20:43 -04:00
Joey Hess
860602a1e6
made some progress on syncing adjusted branches, but still buggy 2016-03-31 14:56:10 -04:00
Joey Hess
fb656b9048
log --raw-date: Use to display seconds from unix epoch. 2016-03-29 14:39:16 -04:00
Joey Hess
1d848a45d2
log: Display time with time zone.
Also cleaned up ugliness with zombies.
2016-03-29 14:27:49 -04:00
Joey Hess
6301543c00
prevent git-annex adjust changing things out from under the daemon 2016-03-29 13:57:48 -04:00
Joey Hess
8a69298bf2
init: Automatically enter the adjusted unlocked branch when in a v6 repo on a filesystem not supporting symlinks. 2016-03-29 13:54:42 -04:00
Joey Hess
5e1d7bbc00
limit git annex adjust to v6 mode
doesn't work in v5
2016-03-29 12:05:02 -04:00
Joey Hess
f4dd3fbb68
option parser for adjust command 2016-03-29 11:54:18 -04:00
Joey Hess
70e8d6860e
Merge branch 'master' into adjustedbranch 2016-03-29 11:07:40 -04:00
Joey Hess
1df49506c4
Correct git-annex info to include unlocked files in v6 repository.
An unlocked present file does not have a pointer file in the worktree, so
info skipped counting it.

It may be that unused was also affected by the problem, but it seemed not
to be in my tests. I think because of the use of the associatedFilesFilter.

This fix slows down both info and unused a little bit, since they have to
query the contents of files from git, but only when handling unlocked files.
2016-03-14 13:14:01 -04:00
Joey Hess
41b7c5f6aa
implement another adjustment -- easy to do now! 2016-03-11 19:54:10 -04:00
Joey Hess
97e97dccda
Merge branch 'master' into adjustedbranch 2016-03-11 12:21:26 -04:00
Joey Hess
cbd38ef367
remove redundant start message 2016-03-09 15:04:00 -04:00
Joey Hess
4b3355cf3c
refactor 2016-03-09 13:43:22 -04:00
Joey Hess
a07fd19ce9
dropkey --json 2016-03-06 12:58:36 -04:00
Joey Hess
78fa865721
dropkey: Add --batch. 2016-03-06 12:45:57 -04:00
Joey Hess
be80c29dbc
Merge branch 'no-cbits' 2016-03-05 11:22:32 -04:00
Joey Hess
cf24e9b892
working toward adjusted commit propigation 2016-03-03 16:19:09 -04:00
Joey Hess
6024108ab2
push original branch, not adjusted branch 2016-03-03 14:13:54 -04:00
Joey Hess
9e6839fd3f
fix bug introduced in recent commit 2016-02-29 17:12:55 -04:00
Joey Hess
9731514c75
update sync branch to the orig branch when in adjusted branch 2016-02-29 16:57:42 -04:00
Joey Hess
048d513233
make assistant aware of adjusted branches when merging 2016-02-29 15:57:47 -04:00
Joey Hess
7c20bf6e7a
make sync aware of adjusted branches
So, it will pull and push the original branch, not the adjusted one.

And, for merging, it will use updateAdjustedBranch (not implemented yet).

Note that remaining uses of Git.Branch.current need to be checked too;
for things that should act on the original branch, and not the adjusted
branch.
2016-02-29 15:23:08 -04:00
Joey Hess
3b4557c754
Merge branch 'master' into adjustedbranch 2016-02-29 14:05:10 -04:00
Joey Hess
e520366c4d
metadata: Added -r to remove all current values of a field. 2016-02-29 13:00:46 -04:00
Joey Hess
98fcc23e4f
use DelMeta to unset tag
No behavior change, just simplification.
2016-02-29 13:00:23 -04:00
Joey Hess
0a1b02ce04
adjusted branches, proof of concept
"git annex adjust" may be a temporary interface, but works for a proof of
concept.

It is pretty fast at creating the adjusted branch. The main overhead is
injecting pointer files. It might be worth optimising that by reusing the
symlink target as the pointer file content. When I tried to do that,
the problem was that the clean filter doesn't use that same format, and so
git thought files had changed. Could be dealt with, perhaps make the clean
filter use symlink format for pointer files when on an adjusted branch?

But the real overhead is in checking out the branch, when git runs the
smudge filter once per file. That is perhaps too slow to be usable,
although it may only affect initial checkout of the branch, and not
updates. TBD.
2016-02-25 16:23:24 -04:00
Joey Hess
8bb80748ef
info: Mention when run in a dead repository. 2016-02-19 15:16:52 -04:00
Joey Hess
ec198fec83
fsck: When the only copy of a file is in a dead repository, mention the repository. 2016-02-19 15:12:11 -04:00
Joey Hess
15148ee9eb
annex.addunlocked
* add, addurl, import, importfeed: When in a v6 repository on a crippled
  filesystem, add files unlocked.
* annex.addunlocked: New configuration setting, makes files always be
  added unlocked. (v6 only)
2016-02-16 14:43:43 -04:00
Joey Hess
40207b26ea
move old ghc compat code into separate module; eliminate WITH_CLIBS
This avoids hsc2hs being run except when building for the old version of ghc.
Should speed up builds.
2016-02-15 11:47:33 -04:00
Joey Hess
885e54df0a
fsck: Populate unlocked files in v6 repositories whose content is present in annex/objects but didn't reach the work tree.
This also handles fixing up after cf260d9a15
2016-02-14 17:27:50 -04:00
Joey Hess
675321264f
fsck: Detect and fix missing associated file mappings in v6 repositories.
This also handles fixing up after the bad data written by
cf260d9a15.
2016-02-14 17:09:54 -04:00
Joey Hess
74bbdfa888
files with only 1 linkCount may still be unlocked
When on crippled filesystem, or without annex.thin set.
2016-02-14 17:04:09 -04:00
Joey Hess
5b51db7645
clean up 2016-02-14 16:52:43 -04:00