Commit graph

2529 commits

Author SHA1 Message Date
Joey Hess
bfb4095c13
Improve behavior when a just added http remote is not available during uuid probe. Do not mark it as annex-ignore, so it will be tried again later. 2016-05-03 12:53:42 -04:00
Joey Hess
617f2834fc
Fix bug that sometimes prevented git-annex smudge --clean from consuming all its input, which resulted in git add bypassing git-annex. 2016-05-02 10:53:24 -04:00
Joey Hess
9c7e46c9c5
correction of scope of security problem
AFAICS, it's not only affecting resumes, but any upload to a special remote
with chunking enabled.
2016-04-28 16:07:10 -04:00
Joey Hess
5fe450514b
Fix build with directory-1.2.6.2.
It started exporting a isSymbolicLink which supports windows. But,
git-annex does no use symlinks on windows yet and this conflicts with the
function by the same name from unix-compat, so hide it.
2016-04-28 13:18:44 -04:00
Joey Hess
21118084db
releasing package git-annex version 6.20160419 2016-04-28 09:48:08 -04:00
Joey Hess
325e21d8bb
add missing dep on transformers 2016-04-27 14:16:45 -04:00
Joey Hess
a66b265b5c
wording 2016-04-27 14:12:43 -04:00
Joey Hess
a3a0ab77df
--allow-unrelated-histories will be in git 2.9.0, not 2.8.2
Also, I had the logic backwards in the version comparison.
2016-04-27 13:54:46 -04:00
Joey Hess
7f3137f408
reorder 2016-04-27 13:50:01 -04:00
Joey Hess
b890f3a53d
Fix bug that prevented resuming of uploads to encrypted special remotes that used chunking. This bug could also expose the names of keys to such remotes.
This is a low-severity security hole.
2016-04-27 12:54:43 -04:00
Joey Hess
6ec154d70c
use --allow-unrelated-histories for now
I'd prefer to use the env var, but let's use what git currently supports.
Revert this when the env var gets supported.

Note that the version checking assumes git 2.8.2 will get support for the
switch.
2016-04-22 15:57:28 -04:00
Joey Hess
46e3319995
assistant: Deal with upcoming git's refusal to merge unrelated histories by default
git 2.8.1 (or perhaps 2.9.0) is going to prevent git merge from merging in
unrelated branches. Since the webapp's pairing etc features often combine
together repositories with unrelated histories, work around this behavior
change by setting GIT_MERGE_ALLOW_UNRELATED_HISTORIES when the assistant
merges.

Note though that this is not done for git annex sync's merges, so
it will follow git's default or configured behavior.
2016-04-22 14:26:44 -04:00
Joey Hess
0273cd5005
adjusted branches need git 2.2.0 or newer
When git-annex is used with a git version older than 2.2.0, disable support for
adjusted branches, since GIT_COMMON_DIR is needed to update them and was first
added in that version of git.
2016-04-22 12:29:32 -04:00
Joey Hess
c3fdaf764d
Isolate test suite from global git config settings. 2016-04-20 15:04:38 -04:00
Joey Hess
b56218f0c2
Fix bug that prevented annex.sshcaching=false configuration from taking effect when on a crippled filesystem. Thanks, divergentdave. 2016-04-20 14:43:43 -04:00
Joey Hess
1bfea9b3e5
calckey: New plumbing command, calculates the key that would be used to refer to a file 2016-04-20 13:50:26 -04:00
Joey Hess
9d952fe9d1
reinject: When src file's content cannot be verified, leave it alone, instead of deleting it. 2016-04-20 13:21:56 -04:00
Joey Hess
f947781fa0
Avoid setting LOCPATH in linux standalone builds now that ghc has been fixed to not hang when it cannot find locale files. 2016-04-20 12:40:11 -04:00
Joey Hess
850d0da699
Fix duplicate progress meter display when downloading from a git remote over http with -J. 2016-04-19 13:10:56 -04:00
Joey Hess
283b126a87
prep release 2016-04-18 18:34:32 -04:00
Joey Hess
2d7e46ea98
fix drop hang reported by musicmatze
Fix hang when dropping content needs to lock the content on a ssh remote,
which occurred when the remote has git-annex version 5.20151019 or newer.

Analysis: `race` runs 2 threads at once, and the hGetLine finishes first.
So, it tries to cancel the waitForProcess, but unfortunately that is making
a foreign call and so cannot be canceled. The remote git-annex-shell
is waiting for a line on stdin before it will exit. Deadlock.

This only occurred sometimes; I reproduced it going from darkstar to
elephant, but not from darkstar to darkstar. Not sure how that fits into
the above analysis -- perhaps a race condition is also involved?

Fixed by not using `race`; now the hGetLine will fail with an exception
if the remote git-annex-shell exits without any output.
2016-04-18 14:04:50 -04:00
Joey Hess
bd516af734
fsck: Warn when core.sharedRepository is set and an annex object file's write bit is not set and cannot be set due to the file being owned by a different user.
Made all Annex.Perms file mode changing functions ignore errors when
core.sharedRepository is set, because the file might be owned by someone
else. I don't fancy getting bug reports about crashes due to set modes in
this configuration, which is a very foot-shooty configuration in the first
place.

The fsck warning is necessary because old repos kept files mode 444, which
doesn't allow locking them, and so if the mode remains 444 due to the file
being owned by someone else, the user should be told about it.
2016-04-14 15:36:53 -04:00
Joey Hess
b7c8bf5274
Preserve execute bits of unlocked files in v6 mode.
When annex.thin is set, adding an object will add the execute bits to the
work tree file, and this does mean that the annex object file ends up
executable.

This doesn't add any complexity that wasn't already present, because git
annex add of an executable file has always ingested it so that the annex
object ends up executable.

But, since an annex object file can be executable or not, when populating
an unlocked file from one, the executable bit is always added or removed
to match the mode of the pointer file.
2016-04-14 14:47:08 -04:00
Joey Hess
d05a75e45a
fix bug in unlocked file scanner that skipped over executable unlocked files 2016-04-14 13:07:46 -04:00
Joey Hess
ed68f0c0a0
webapp: Update url to add gitlab.com ssh key. (Psst GitLab.. cool urls don't change!) 2016-04-13 14:49:01 -04:00
Joey Hess
1010482bc7
webapp: When $HOME is a git repository, and has been initialized for use by git-annex, opening the webapp went ahead and ran the assistant there, annexing all files. Since this is almost certianly not desirable, especially when the user is just opening the webapp from a dekstop menu which happens to run it in $HOME, the webapp will now not treat such a $HOME git repository as a git-annex repository. 2016-04-13 14:07:50 -04:00
Joey Hess
402e5adab8
smudge: Print a warning when annex.thin is set, as git's smudge interface does not allow honoring that configuration. 2016-04-13 13:34:24 -04:00
Joey Hess
bbe994295f
Fix build with ghc v7.11. Thanks, Gabor Greif. 2016-04-12 15:27:51 -04:00
Joey Hess
0ec7c281dd
prep release 2016-04-12 14:55:43 -04:00
Joey Hess
eac26f13db
Fix bug in annex.largefiles mimetype= matching when git-annex is run in a subdirectory of the repository. 2016-04-12 14:19:34 -04:00
Joey Hess
cf06dac2b8
hard links on windows
* annex.thin and annex.hardlink are now supported on Windows.
* unannex --fast now makes hard links on Windows.
2016-04-08 15:25:32 -04:00
Joey Hess
eb9ac8d6d7
sync: Show output of git commit.
Rationalle: User might have hook scripts whose output they want to see.
Also, git commit output may tell the user they forgot to add a file.
The output is not too ugly when there's nothing to commit.
2016-04-05 16:22:21 -04:00
Joey Hess
2046502407
v6: Close pointer file handles more quickly, to avoid problems on Windows.
Was using L.readFile, so the Handle would remain open until the garbage
collector got around to it. Changed to explicit open and close, so we know
it's always closed when the function returns.
2016-04-04 15:42:33 -04:00
Joey Hess
c3e0859846
Upgrading a direct mode repository to v6 has changed to enter an adjusted unlocked branch.
This makes the direct mode to v6 upgrade able to be performed in one clone
of a repository without affecting other clones, which can continue using v5
and direct mode.
2016-04-04 13:17:24 -04:00
Joey Hess
ed3e8e1886
Merge branch 'adjustedbranch' 2016-03-31 19:05:47 -04:00
Joey Hess
fb656b9048
log --raw-date: Use to display seconds from unix epoch. 2016-03-29 14:39:16 -04:00
Joey Hess
1d848a45d2
log: Display time with time zone.
Also cleaned up ugliness with zombies.
2016-03-29 14:27:49 -04:00
Joey Hess
8a69298bf2
init: Automatically enter the adjusted unlocked branch when in a v6 repo on a filesystem not supporting symlinks. 2016-03-29 13:54:42 -04:00
Joey Hess
4c9ca98769
docs for git-annex adjust 2016-03-29 11:39:43 -04:00
Joey Hess
589ff4ebbb
changelog 2016-03-23 11:42:49 -04:00
Joey Hess
43713b1972
fix typo 2016-03-21 10:43:24 -04:00
Joey Hess
d8b8984b9b
prep release 2016-03-18 11:30:46 -04:00
Joey Hess
204dbf597d
well, that broke the i386ancient autobuilder. revert 2016-03-17 23:42:45 -04:00
Joey Hess
b482cb9d90
Fix build with stack 0.1.6 which needs explicit-setup-deps. 2016-03-17 23:14:37 -04:00
Joey Hess
13f4af5d33
typo 2016-03-14 16:46:00 -04:00
Joey Hess
2d234de781
Sped up git-annex merge by using git hash-object --batch.
This does mean that it has to write out temp files containing updated
objects for the merge. So may use more disk space, and disk IO, but that
should generally win out over needing to launch N separate
git hash-object processes.
2016-03-14 16:23:22 -04:00
Joey Hess
88a4a6f396
Sped up git-annex add in direct mode and v6 by using git hash-object --batch.
Speeds up hashSymlink and hashPointerFile.
2016-03-14 15:58:46 -04:00
Joey Hess
1df49506c4
Correct git-annex info to include unlocked files in v6 repository.
An unlocked present file does not have a pointer file in the worktree, so
info skipped counting it.

It may be that unused was also affected by the problem, but it seemed not
to be in my tests. I think because of the use of the associatedFilesFilter.

This fix slows down both info and unused a little bit, since they have to
query the contents of files from git, but only when handling unlocked files.
2016-03-14 13:14:01 -04:00
Richard Hartmann
c483e11cb3
Bump standards version
(cherry picked from commit 6efa2492bb)
2016-03-12 13:02:37 -04:00
Joey Hess
9039bdb4ea
Always try to thaw content, even when annex.crippledfilesystem is set. 2016-03-09 13:33:13 -04:00
Joey Hess
ec376cf1cb
adjust architecture lists 2016-03-09 00:34:18 -04:00
Joey Hess
ab5f7b05f5
Fix OSX dmg to include libraries needed by bundled gpg, lost in last release. 2016-03-07 12:55:01 -04:00
Joey Hess
a07fd19ce9
dropkey --json 2016-03-06 12:58:36 -04:00
Joey Hess
78fa865721
dropkey: Add --batch. 2016-03-06 12:45:57 -04:00
Joey Hess
38d7df3374
add new deps 2016-03-05 11:29:14 -04:00
Joey Hess
9eecb4084e
changelog 2016-03-05 11:27:33 -04:00
Joey Hess
be80c29dbc
Merge branch 'no-cbits' 2016-03-05 11:22:32 -04:00
Joey Hess
baa9954e06
Fix metadata hook behavior when multiple files are added at once. Thanks, Klaus Ethgen. 2016-03-02 12:25:41 -04:00
Joey Hess
3334130368
Fix shared lock file FD leak.
This fixes behavior in this situation:

	l1 <- lockShared Nothing "lck"
	l2 <- lockShared Nothing "lck"
	dropLock l1
	dropLock l2

Before, the lock was dropped upon the second dropLock call, but the fd
remained open, and would never be closed while the program was running.

Fixed by a rather round-about method, but it should work well enough.

It would have been simpler to open open the shared lock once, and not open
it again in the second call to lockShared. But, that's difficult to do
atomically.

This also affects Windows and PID locks, not just posix locks.

In the case of pid locks, multiple calls to waitLock within the same
process are allowed because the side lock is locked using a posix lock,
and so multiple exclusive locks can be taken in the same process. So,
this change fixes a similar problem with pid locks.

	l1 <- waitLock (Seconds 1) "lck"
	l2 <- waitLock (Seconds 1) "lck"
	dropLock l1
	dropLock l2

Here the l2 side lock fd remained open but not locked,
although the pid lock file was removed. After this change, the second
dropLock will close both fds to the side lock, and delete the pidlock.
2016-03-01 15:31:39 -04:00
Joey Hess
ad888a6b76
Fix bug preventing moving files to/from a repository with annex.pidlock set. 2016-03-01 12:51:54 -04:00
Joey Hess
3e91cd13ba
Fix data loss that can occur when annex.pidlock is set in a repository. 2016-03-01 12:12:57 -04:00
Joey Hess
e520366c4d
metadata: Added -r to remove all current values of a field. 2016-02-29 13:00:46 -04:00
Joey Hess
c4494f1abe
leap year release 2016-02-29 12:41:59 -04:00
Joey Hess
764e57f1f1
The pre-commit-annex hook script that automatically extracts metadata has been updated to also use exiftool. Thanks, Klaus Ethgen. 2016-02-28 11:10:55 -04:00
Joey Hess
b946ca44c3
Support --metadata field<number, --metadata field>number etc to match ranges of numeric values.
Similarly (well, for free), support preferred content expressions like
metadata=field<number and metadata=field>number
2016-02-27 10:55:02 -04:00
Joey Hess
01f1de0c3c
close bug and changelog
finished testing recent commits, and I did solve it
2016-02-26 16:51:28 -04:00
Joey Hess
471a211d21
Include magic database in the linux and OSX standalone builds. 2016-02-26 11:54:15 -04:00
Joey Hess
a5bf674bec
Avoid crashing when built with MagicMime support, but when the magic database cannot be loaded. 2016-02-23 14:39:56 -04:00
Joey Hess
398d93d4d2
Linux and OSX standalone builds put the bundled gpg last in PATH, so any system gpg will be preferred over it. 2016-02-19 16:19:19 -04:00
Joey Hess
8bb80748ef
info: Mention when run in a dead repository. 2016-02-19 15:16:52 -04:00
Joey Hess
ec198fec83
fsck: When the only copy of a file is in a dead repository, mention the repository. 2016-02-19 15:12:11 -04:00
Joey Hess
b0081598c7
Fix memory leak in last release, which affected commands like git-annex status when a large non-annexed file is present in the work tree.
The whole file was strictly read, and so buffered in memory, and remained
buffered for some time when running git-annex status.
2016-02-19 14:45:26 -04:00
Joey Hess
bcda35c6b4
Update perlmagick build dependency. Closes: #789225 2016-02-18 13:09:43 -04:00
Joey Hess
610632b9ad
prep release 2016-02-17 14:49:05 -04:00
Joey Hess
15148ee9eb
annex.addunlocked
* add, addurl, import, importfeed: When in a v6 repository on a crippled
  filesystem, add files unlocked.
* annex.addunlocked: New configuration setting, makes files always be
  added unlocked. (v6 only)
2016-02-16 14:43:43 -04:00
Joey Hess
f9d79d194b
Windows: Fix v6 unlocked files to actually work.
Pointer files were not being treated as annex content, so "git annex get"
didn't replace them with the object.
2016-02-15 16:12:18 -04:00
Joey Hess
2e3b5e645f
When initializing a v6 repo on a crippled filesystem, don't force it into direct mode. 2016-02-15 15:41:49 -04:00
Joey Hess
46fe686ba0
remove Utility.Mounts et al; moved to mountpoints package 2016-02-15 11:14:37 -04:00
Joey Hess
885e54df0a
fsck: Populate unlocked files in v6 repositories whose content is present in annex/objects but didn't reach the work tree.
This also handles fixing up after cf260d9a15
2016-02-14 17:27:50 -04:00
Joey Hess
675321264f
fsck: Detect and fix missing associated file mappings in v6 repositories.
This also handles fixing up after the bad data written by
cf260d9a15.
2016-02-14 17:09:54 -04:00
Joey Hess
cf260d9a15
Fix storing of filenames of v6 unlocked files when the filename is not representable in the current locale.
This is a mostly backwards compatable change. I broke backwards
compatability in the case where a filename starts with double-quote.
That seems likely to be very rare, and v6 unlocked files are a new feature
anyway, and fsck needs to fix missing associated file mappings anyway. So,
I decided that is good enough.

The encoding used is to just show the String when it contains a problem
character. While that adds some overhead to addAssociatedFile and
removeAssociatedFile, those are not called very often. This approach has
minimal decode overhead, because most filenames won't be encoded that way,
and it only has to look for the leading double-quote to skip the expensive
read. So, getAssociatedFiles remains fast.

I did consider using ByteString instead, but getting a FilePath converted
with all chars intact, even surrigates, is difficult, and it looks like
instance PersistField ByteString uses Text, which I don't trust for problem
encoded data. It would probably be slower too, and it would make the
database less easy to inspect manually.
2016-02-14 16:37:25 -04:00
Joey Hess
0f18636c8a
Work around problem with concurrent-output when in a non-unicode locale by avoiding use of it in such a locale.
Instead -J will behave as if it was built without concurrent-output support
in this situation. Ie, it will be mostly quiet, except when there's an
error.

Note that it's not a problem for a filename to contain invalid utf-8 when
in a utf-8 locale. That is handled ok by concurrent-output. It's only
displaying unicode characters in a non-unicode locale that doesn't work.
2016-02-14 15:02:42 -04:00
Joey Hess
cc4d3e3d45
checkpresentkey: Allow to be run without an explicit remote and add --batch
* checkpresentkey: Allow to be run without an explicit remote.
* checkpresentkey: Added --batch.
2016-02-12 16:43:51 -04:00
Joey Hess
9df13e73ae
if keys database cannot be opened due to permissions, ignore
This lets readonly repos be used. If a repo is readonly, we can ignore the
keys database, because nothing that we can do will change the state of the
repo anyway.
2016-02-12 14:16:35 -04:00
Joey Hess
c8b201ad87
releasing package git-annex version 6.20160211 2016-02-11 12:02:38 -04:00
Joey Hess
5d05aad74c
S3: Allow configuring with requeststyle=path to use path-style bucket access instead of the default DNS-style access.
untested
2016-02-09 15:36:36 -04:00
Joey Hess
c40d14a37d
WebDAV: Remove a bogus trailing slash from the end of the url to the temporary store location for a key. Thanks, wzhd.
That trailing slash is needed for legacy chunked mode, because it puts the
chunks in a subdir under the key. But, outside legacy chunked mode, it's BS
and it's amazing it worked at all with some webdav servers.
2016-02-09 11:50:40 -04:00
Joey Hess
850a645233
WebDAV: Set depth 1 in PROPFIND request, for better compatability with some servers. Thanks, wzhd. 2016-02-09 11:47:35 -04:00
Joey Hess
17c97434f2
init: Fix bugs in submodule .git symlink fixup, that occurred when initializing in a subdirectory of a submodule and a submodule of a submodule. 2016-02-08 15:41:27 -04:00
Joey Hess
0226122842
Brought back the dbus and xmpp build flags, so build from source can be done without C libraries that may be hard to install. 2016-02-05 18:00:20 -04:00
Joey Hess
246d0d45d7
move -j1 setting to BUILDEROPTIONS, set in debian/rules file
I needed BUILDEROPTIONS to allow passing flags to stack build, but it also
lets me move the -j1 out of the normal build path, and to debian/rules
which has the goal of having a reproducible build
2016-02-05 13:56:27 -04:00
Joey Hess
23cc315c38
matchexpression: Added --largefiles option to parse an annex.largefiles expression. 2016-02-03 16:58:36 -04:00
Joey Hess
5127cb59cc
annex.largefiles: Add support for mimetype=text/* etc, when git-annex is linked with libmagic. 2016-02-03 16:29:34 -04:00
Joey Hess
403b56fb91
Limit annex.largefiles parsing to the subset of preferred content expressions that make sense in its context.
So, not "standard" or "lackingcopies", etc.
2016-02-03 15:04:42 -04:00
Joey Hess
d37fe6a547
annex.largefiles can be configured in .gitattributes too
This is particulary useful for v6 repositories, since the .gitattributes
configuration will apply in all clones of the repository.
2016-02-02 15:18:17 -04:00
Joey Hess
e8fc2ff27c
add "nothing" to preferred content DSL
Same as "not anything"; will be particularly useful in annex.largefiles
gitattributes.
2016-02-02 14:42:13 -04:00
Joey Hess
ef4b4c3856
webapp: Fix deletion of current repository directory.
removeDirectoryRecursive "." fails; need full path.

I think this is a reversion introduced in the relative path conversion, in
january of *last* year!
2016-01-29 14:04:39 -04:00
Joey Hess
9a4af2324e
Fix reversion in lookupkey, contentlocation, and examinekey which caused them to sometimes output side messages. 2016-01-29 13:20:24 -04:00
Joey Hess
7c1df36d63
annex.addsmallfiles: New option controlling what is done when adding files not matching annex.largefiles. 2016-01-28 14:04:32 -04:00
Joey Hess
540acd32bf
Debian: Avoid building debug package, since gdb is not often useful to debug haskell programs. 2016-01-26 15:39:37 -04:00