Commit graph

32579 commits

Author SHA1 Message Date
yarikoptic
dae76d8ef3 fixup the trailer 2018-08-21 16:02:25 +00:00
Joey Hess
056b081280
followup 2018-08-21 11:56:51 -04:00
yarikoptic
e009c0efed original filing about web remote listing 2018-08-21 15:09:10 +00:00
Joey Hess
7ee3b02d49
replace stack trace with an explanation 2018-08-20 21:26:07 -04:00
Joey Hess
b8cd5fde17
idea 2018-08-20 16:13:46 -04:00
Joey Hess
c7331eebbf
Merge branch 'master' of ssh://git-annex.branchable.com 2018-08-20 12:21:11 -04:00
Joey Hess
fa44bca8b3
linux standalone: When LOCPATH is already set, use it instead of the bundled locales.
It can be set to an empty string to use the system locales too. Of course
whether that will work depends on the amount of divergence.

This commit was supported by the NSF-funded DataLad project.
2018-08-20 12:20:54 -04:00
emanuele.olivetti@47d88ed185b03191e25329caa6fabc2efb3118b2
f8b4a1f70c 2018-08-20 15:54:11 +00:00
Gus
48fb6f9886 2018-08-17 22:16:45 +00:00
https://launchpad.net/~liori
5d0c336994 Added a comment 2018-08-17 21:58:03 +00:00
Gus
d1eabddee6 2018-08-17 21:44:05 +00:00
Joey Hess
085338a29d
response 2018-08-17 16:18:50 -04:00
Joey Hess
9091d52a1f
close old openbsd build bugs 2018-08-17 16:18:40 -04:00
Joey Hess
59b2ebbaeb
Merge branch 'master' of ssh://git-annex.branchable.com 2018-08-17 16:04:55 -04:00
Joey Hess
ab14d7cd9f
devblog 2018-08-17 16:04:30 -04:00
Joey Hess
54d49eeac8
avoid update-index race
This commit was supported by the NSF-funded DataLad project.
2018-08-17 16:03:40 -04:00
Joey Hess
82c5dd8a01
queueing of internal IO actions on files
This would be better if getInternalFiles were
more polymorphic, but I can't see a good
way to accomplish that without messing with Data.Typeable,
which seemed like overkill.

Reverted CommandAction back to the simpler version.

This commit was sponsored by Eric Drechsel on Patreon.
2018-08-17 13:28:21 -04:00
Joey Hess
c5a8abb130
comment 2018-08-17 13:19:37 -04:00
Joey Hess
ec91b6e4b2
plan to fix race 2018-08-17 11:18:53 -04:00
Joey Hess
35e1697377
expand 2018-08-17 10:48:18 -04:00
spwhitton
8a15ee0303 Added a comment 2018-08-17 14:44:13 +00:00
bnyn
127dc50898 2018-08-17 01:18:51 +00:00
Joey Hess
2e153de34a
close 2018-08-16 16:41:08 -04:00
Joey Hess
9eb708a11c
Merge branch 'master' of ssh://git-annex.branchable.com 2018-08-16 16:39:08 -04:00
Joey Hess
b187bc9cc6
devblog 2018-08-16 16:38:01 -04:00
Joey Hess
5799d325f0
update todo categories 2018-08-16 16:36:47 -04:00
Joey Hess
0f25d48639
pass absolute path to update-index
Test suite found a case where this is necessary.

And the man page says this, although current behavior is not as
documented..

           Note that files beginning with .  are discarded.
           This includes ./file and dir/./file. If you don’t want
           this, then use cleaner names.

This may hit path length limits on Windows. shrug

This commit was supported by the NSF-funded DataLad project.
2018-08-16 16:00:29 -04:00
Joey Hess
82a239675f
narrow the race where a file gets modified before update-index
Check just before running update-index if the worktree file's content is
still the same, don't update it when it's been modified. This narrows
the race window a lot, from possibly minutes or hours, to seconds or
less.

(Use replaceFile so that the worktree update happens atomically,
allowing the InodeCache of the new worktree file to itself be gathered
w/o any other race.)

This doesn't eliminate the race; it can still occur in the window before
update-index runs. When annex.queue is large, a lot of files will be
statted by the checks, and so the window may still be large enough to be a
problem.

When only a few files are being processed, the window is as small as it
is in the race where a modification gets overwritten by git-annex when
it updates the worktree. Or maybe as small as whatever race git
checkout/pull/merge may have when the worktree gets modified during it.
Still, I've kept a todo about this race.

This commit was supported by the NSF-funded DataLad project.
2018-08-16 15:56:43 -04:00
Joey Hess
6a445dc086
support conditionally excluding queued files
Switched code to use a for loop to avoid a filterM that would have
doubled the memory used.

This commit was supported by the NSF-funded DataLad project.
2018-08-16 14:38:37 -04:00
Joey Hess
82cfcfc838
better index file refresh method
Use git update-index --refresh, since it's a little bit more
efficient and the user can be told to run it if a locked index prevents
git-annex from running it.

This also fixes the problem where an annexed file was deleted in the index
and a get of another file that uses the same key caused the index update to
add back the deleted file. update-index will not add back the deleted file.

Documented in tips/unlocked_files.mdwn the gotcha that the index update
may conflict with other operations. I can't see any way to possibly avoid
that conflict.

One new todo about a race that causes a modification to be accidentially
staged.

Note that the assistant only flushes the git command queue when it
commits a modification. I have not tested the assistant with v6 unlocked
files, but assume most users of the assistant won't care if the index
shows a file as modified for a while.

This commit was supported by the NSF-funded DataLad project.
2018-08-16 14:16:24 -04:00
duncan_bayne
7e9d6c4ef1 Added a comment: RTFM 2018-08-16 04:25:27 +00:00
Joey Hess
5e87389f40
refactor 2018-08-15 13:46:28 -04:00
Joey Hess
4c5a9965c1
remove invalid todo item
I tested it, and it's ok. I think I was adding it under a filename that
produced a different key.
2018-08-15 13:34:48 -04:00
Joey Hess
24ed323208
response 2018-08-15 11:43:50 -04:00
https://launchpad.net/~liori
a6927da198 2018-08-15 14:07:31 +00:00
Joey Hess
e891e78032
devblog 2018-08-14 16:24:13 -04:00
Joey Hess
48e9e12961
finally fixed v6 get/drop git status
After updating the worktree for an add/drop, update git's index, so git
status will not show the files as modified.

What actually happens is that the index update removes the inode
information from the index. The next git status (or similar) run
then has to do some work. It runs the clean filter.

So, this depends on the clean filter being reasonably fast and on git
not leaking memory when running it. Both problems were fixed in
a96972015d, but only for git 2.5. Anyone
using an older git will see very expensive git status after an add/drop.

This uses the same git update-index queue as other parts of git-annex, so
the actual index update is fairly efficient. Of course, updating the index
does still have some overhead. The annex.queuesize config will control how
often the index gets updated when working on a lot of files.

This is an imperfect workaround... Added several todos about new
problems this workaround causes. Still, this seems a lot better than the
old behavior.

This commit was supported by the NSF-funded DataLad project.
2018-08-14 16:23:58 -04:00
Joey Hess
06fd4657db
response 2018-08-14 13:41:02 -04:00
Joey Hess
b3178547f1
Merge branch 'master' of ssh://git-annex.branchable.com 2018-08-14 13:39:51 -04:00
Joey Hess
42c5d7c64f
moving to filterdriver branch
Not used in master, so remove until/unless filterdriver branch is
merged.
2018-08-14 13:37:17 -04:00
jarno
3b155dd32d Added a comment 2018-08-14 16:03:39 +00:00
Joey Hess
63cfc1a615
Merge branch 'master' of ssh://git-annex.branchable.com 2018-08-14 11:08:44 -04:00
Joey Hess
66a4483dfa
response 2018-08-14 11:02:55 -04:00
hoabg102@4a7a5918bc078f792113da142cf8c6d825a1825c
f0779bc9f1 2018-08-14 10:22:06 +00:00
Joey Hess
d8a8f2df70
full plan 2018-08-13 17:51:02 -04:00
Joey Hess
86df0d6e1b
even better idea 2018-08-13 17:43:16 -04:00
Joey Hess
df5823cea0
update 2018-08-13 17:29:33 -04:00
Joey Hess
bc7d431a6a
status 2018-08-13 16:37:23 -04:00
Joey Hess
7a1a3ef11f
add missing flush-pkt after version 2018-08-13 16:22:39 -04:00
Joey Hess
d963d40815
use String not Text
On second thought, git passes filepaths, which may not be valid utf8, so
can't use Text here.

String will be a little bit slower, but not enough to worry about.
2018-08-13 16:22:31 -04:00