Commit graph

928 commits

Author SHA1 Message Date
Joey Hess
097605e2e9
git's handing of relative GIT_INDEX_FILE is more insane than I thought; always make absolute
This is actually worse than I thought; when git is being run with a
detached work tree, GIT_INDEX_FILE is treated as a path relative to CWD,
instead of the normal behavior of relative the top of the work tree.

This seems to make it basically impossible for any program that wants to
use GIT_INDEX_FILE to use anything other than an absolute path to it; there
are too many configurations to keep straight that can change how git
interprets what should be a simple relative path to a file.

(I have complained to the git developers.)
2016-05-22 15:02:55 -04:00
Joey Hess
823c28d2dc
nub transitionList to avoid ugly message after repeated transitions, and avoid redundant work for repeated ForgetDeadRemotes transitions 2016-05-18 12:26:38 -04:00
Joey Hess
766728c8cf
unify handling of unusual GIT_INDEX_FILE relative path
This is probably a git bug that stuck in its interface.
2016-05-17 14:42:06 -04:00
Joey Hess
b4ab1fb093
Fix crash when entering/changing view in a subdirectory of a repo that has a dotfile in its root. 2016-05-17 13:49:10 -04:00
Joey Hess
e91037a38b
use indexEnv 2016-05-17 13:38:04 -04:00
Joey Hess
93c03b5dd5
Work around git bug in handling of relative path to GIT_INDEX_FILE when in a subdirectory of the repository.
This affected git annex view. It turns out that some other places
that use GIT_INDEX_FILE were already working around the bug. I removed the
workaround from Annex.Branch since the new workaround will do.
2016-05-17 13:29:51 -04:00
Joey Hess
d56175164b
avoid checking locations in regular repo
In commit 2d00523609 I accidentially
made gitAnnexLocation do more work, checking content locations,
when used in a regular repo.
2016-05-16 17:19:07 -04:00
Joey Hess
eda5d9cc74
adjust: Add --fix adjustment, which is useful when the git directory is in a nonstandard place. 2016-05-16 17:18:33 -04:00
Joey Hess
4efc26ca6c
move keys db closure to AutoMerge
This makes git-annex sync also do it, which makes sure that the keys db
info is fresh when doing a sync --content.
2016-05-16 15:11:14 -04:00
Joey Hess
9f05be393e
adjust: If the adjusted branch already exists, avoid overwriting it, since it might contain changes that have not yet been propigated to the original branch.
Could not think of a foolproof way to detect if the old adjusted branch was
just behind the current branch. It's possible that the user amended the
adjusting commit at the head of the adjusted branch, for example.

I decided to bail in this situation, instead of just entering the old
branch, so that if git annex adjust succeeds the user is always in a
*current* adjusted branch, not some old and out of date one.

What could perhaps be done is enter the old branch and then update it. But
that seems too magical; the user may have rebased master or something or
may not want to propigate the changes from the old branch. Best to error
out.
2016-05-13 14:04:22 -04:00
Joey Hess
2d00523609
In the unusual configuration where annex.crippledfilesystem=true but core.symlinks=true, store object contents in mixed case hash directories so that symlinks will point to them.
Contents are searched for in both locations, same as before, so this
does not add any overhead.
2016-05-10 15:00:22 -04:00
Joey Hess
8a81ddb448
improve comment 2016-05-10 14:42:57 -04:00
Joey Hess
c456833179
Windows: Fix an over-long temp directory name. 2016-05-06 12:49:41 -04:00
Joey Hess
6cf9dbb564
fix build warning on windows 2016-05-05 15:48:58 -04:00
Joey Hess
a9e8cf42d6
more windows path fixes
normalize filepaths in the map because it may be constructed with
windows-style paths and then queried for git-style
2016-05-04 13:00:02 -04:00
Joey Hess
b22409db38
avoid warnings about not exported System.Directory.isSymbolicLink 2016-04-28 15:18:11 -04:00
Joey Hess
5fe450514b
Fix build with directory-1.2.6.2.
It started exporting a isSymbolicLink which supports windows. But,
git-annex does no use symlinks on windows yet and this conflicts with the
function by the same name from unix-compat, so hide it.
2016-04-28 13:18:44 -04:00
Joey Hess
46e3319995
assistant: Deal with upcoming git's refusal to merge unrelated histories by default
git 2.8.1 (or perhaps 2.9.0) is going to prevent git merge from merging in
unrelated branches. Since the webapp's pairing etc features often combine
together repositories with unrelated histories, work around this behavior
change by setting GIT_MERGE_ALLOW_UNRELATED_HISTORIES when the assistant
merges.

Note though that this is not done for git annex sync's merges, so
it will follow git's default or configured behavior.
2016-04-22 14:26:44 -04:00
Joey Hess
0273cd5005
adjusted branches need git 2.2.0 or newer
When git-annex is used with a git version older than 2.2.0, disable support for
adjusted branches, since GIT_COMMON_DIR is needed to update them and was first
added in that version of git.
2016-04-22 12:29:32 -04:00
Joey Hess
b56218f0c2
Fix bug that prevented annex.sshcaching=false configuration from taking effect when on a crippled filesystem. Thanks, divergentdave. 2016-04-20 14:43:43 -04:00
Joey Hess
9d952fe9d1
reinject: When src file's content cannot be verified, leave it alone, instead of deleting it. 2016-04-20 13:21:56 -04:00
Joey Hess
bd516af734
fsck: Warn when core.sharedRepository is set and an annex object file's write bit is not set and cannot be set due to the file being owned by a different user.
Made all Annex.Perms file mode changing functions ignore errors when
core.sharedRepository is set, because the file might be owned by someone
else. I don't fancy getting bug reports about crashes due to set modes in
this configuration, which is a very foot-shooty configuration in the first
place.

The fsck warning is necessary because old repos kept files mode 444, which
doesn't allow locking them, and so if the mode remains 444 due to the file
being owned by someone else, the user should be told about it.
2016-04-14 15:36:53 -04:00
Joey Hess
b7c8bf5274
Preserve execute bits of unlocked files in v6 mode.
When annex.thin is set, adding an object will add the execute bits to the
work tree file, and this does mean that the annex object file ends up
executable.

This doesn't add any complexity that wasn't already present, because git
annex add of an executable file has always ingested it so that the annex
object ends up executable.

But, since an annex object file can be executable or not, when populating
an unlocked file from one, the executable bit is always added or removed
to match the mode of the pointer file.
2016-04-14 14:47:08 -04:00
Joey Hess
5e190913a4
add AdjBranch newtype; some simplications 2016-04-09 15:10:26 -04:00
Joey Hess
b5be04027c
change name of basis branch
Making the name look too much like the adjusted branch was ambiguous.
2016-04-09 14:17:20 -04:00
Joey Hess
7d28110c68
fix master push overwrite race when updating adjusted branch, by maintaining basis ref 2016-04-09 14:12:25 -04:00
Joey Hess
cf06dac2b8
hard links on windows
* annex.thin and annex.hardlink are now supported on Windows.
* unannex --fast now makes hard links on Windows.
2016-04-08 15:25:32 -04:00
Joey Hess
251405eca2
avoid withWorkTreeRelated affecting annex symlink calculation 2016-04-08 14:24:00 -04:00
Joey Hess
6549049142
fix commit tree after merge into adjusted branch 2016-04-06 19:22:15 -04:00
Joey Hess
887ef93a7f
run out of tree merge with --no-ff
This is how direct mode does it too, and somehow, for reasons that
currently escape me, this makes git merge not care if it's run with an
empty work tree.
2016-04-06 18:40:28 -04:00
Joey Hess
60bdffe43e
fix auto merge conflict resolution when doing out of tree merge for adjusted branch 2016-04-06 17:32:04 -04:00
Joey Hess
b9e4e2ba84
new method for merging changes into adjusted branch that avoids unncessary merge conflicts
Still needs work when there are actual merge conflicts.
2016-04-06 15:36:18 -04:00
Joey Hess
2046502407
v6: Close pointer file handles more quickly, to avoid problems on Windows.
Was using L.readFile, so the Handle would remain open until the garbage
collector got around to it. Changed to explicit open and close, so we know
it's always closed when the function returns.
2016-04-04 15:42:33 -04:00
Joey Hess
f78cbd9f0d
todo 2016-04-04 14:38:29 -04:00
Joey Hess
7c7f3a0f76
deal with cloning a repo that has an ajdusted branch checked out 2016-04-04 13:51:42 -04:00
Joey Hess
7ba836eec3
make way for git checkout output 2016-04-04 13:25:30 -04:00
Joey Hess
c3e0859846
Upgrading a direct mode repository to v6 has changed to enter an adjusted unlocked branch.
This makes the direct mode to v6 upgrade able to be performed in one clone
of a repository without affecting other clones, which can continue using v5
and direct mode.
2016-04-04 13:17:24 -04:00
Joey Hess
12ddb6e8b2
fixed merging of changes from adjusted branch + a remote 2016-03-31 18:54:35 -04:00
Joey Hess
860602a1e6
made some progress on syncing adjusted branches, but still buggy 2016-03-31 14:56:10 -04:00
Joey Hess
a585731935
add reflog messages 2016-03-31 12:27:48 -04:00
Joey Hess
02ce75c87d
clean up handling of commit lock
Closing the lock manually caused a later exception when the bracket tried
to close it again.
2016-03-31 12:04:05 -04:00
Joey Hess
8a69298bf2
init: Automatically enter the adjusted unlocked branch when in a v6 repo on a filesystem not supporting symlinks. 2016-03-29 13:54:42 -04:00
Joey Hess
42b7ccc89f
git annex add in adjusted unlocked branch
Cached the current branch lookup just because it seems unnecessary overhead
to run an extra git command per add to query the current branch.
2016-03-29 13:26:06 -04:00
Joey Hess
5e1d7bbc00
limit git annex adjust to v6 mode
doesn't work in v5
2016-03-29 12:05:02 -04:00
Joey Hess
1df62b43d1
remove hashPointerFile'
no longer needed now that hashPointerFile uses a long-running git
hash-object handle
2016-03-29 11:15:21 -04:00
Joey Hess
70e8d6860e
Merge branch 'master' into adjustedbranch 2016-03-29 11:07:40 -04:00
Joey Hess
a2b668a8f6
reuse annex's HashObjectHandle 2016-03-14 16:29:59 -04:00
Joey Hess
2d234de781
Sped up git-annex merge by using git hash-object --batch.
This does mean that it has to write out temp files containing updated
objects for the merge. So may use more disk space, and disk IO, but that
should generally win out over needing to launch N separate
git hash-object processes.
2016-03-14 16:23:22 -04:00
Joey Hess
00d9da3534
use hash-object --batch
Handle was plumbed through, but not used.
2016-03-14 16:12:55 -04:00
Joey Hess
88a4a6f396
Sped up git-annex add in direct mode and v6 by using git hash-object --batch.
Speeds up hashSymlink and hashPointerFile.
2016-03-14 15:58:46 -04:00
Joey Hess
f2772f469a
followup 2016-03-14 15:54:46 -04:00
Joey Hess
1df49506c4
Correct git-annex info to include unlocked files in v6 repository.
An unlocked present file does not have a pointer file in the worktree, so
info skipped counting it.

It may be that unused was also affected by the problem, but it seemed not
to be in my tests. I think because of the use of the associatedFilesFilter.

This fix slows down both info and unused a little bit, since they have to
query the contents of files from git, but only when handling unlocked files.
2016-03-14 13:14:01 -04:00
Joey Hess
41b7c5f6aa
implement another adjustment -- easy to do now! 2016-03-11 19:54:10 -04:00
Joey Hess
a85196bd4e
simplify adjustment reversal 2016-03-11 19:41:11 -04:00
Joey Hess
ba1ef156a2
fix deletion of files in adjustTree 2016-03-11 16:30:06 -04:00
Joey Hess
b9184f69a7
improve propigation of commits from adjusted branches
Only reverse adjust the changes in the commit, which means that adjustments
do not need to be generally cleanly reversable.

For example, an adjustment can unlock all locked files, but does not need
to worry about files that were originally unlocked when reversing, because
it will only ever be run on files that have been changed. So, it's ok
if it locks all files when reversed, or even leaves all files as-is when
reversed.
2016-03-11 16:05:06 -04:00
Joey Hess
97e97dccda
Merge branch 'master' into adjustedbranch 2016-03-11 12:21:26 -04:00
Joey Hess
4b3355cf3c
refactor 2016-03-09 13:43:22 -04:00
Joey Hess
9039bdb4ea
Always try to thaw content, even when annex.crippledfilesystem is set. 2016-03-09 13:33:13 -04:00
Joey Hess
be80c29dbc
Merge branch 'no-cbits' 2016-03-05 11:22:32 -04:00
Joey Hess
5e3f707c34
rebase on top of updated original branch 2016-03-03 17:00:48 -04:00
Joey Hess
ac08f6580e
fix abs filepath generation 2016-03-03 16:47:51 -04:00
Joey Hess
40509e20e5
change name of adjusted branches to eg adjusted/master(unlocked)
Using adjusted/unlocked/master made lots of git stuff dealing with "master"
complain that it was ambiguous. This new appoach is more like view branch
names, and shows the adjustment right there in the branch display even if
only the basename of the branch is shown.
2016-03-03 16:38:56 -04:00
Joey Hess
cf24e9b892
working toward adjusted commit propigation 2016-03-03 16:19:09 -04:00
Joey Hess
7811556a5b
Merge branch 'master' into adjustedbranch 2016-03-03 15:40:25 -04:00
Joey Hess
91f37673df
can't checkout adjusted branch while index is still locked
There's a race here, but entering an adjusted branch for the first time is
not something to do when a commit is being made at the same time. Although,
may want to prevent the assistant from committing while entering the
adjusted branch.
2016-03-03 14:20:39 -04:00
Joey Hess
6024108ab2
push original branch, not adjusted branch 2016-03-03 14:13:54 -04:00
Joey Hess
ef1abda78b
lock index while making index-less commits
Avoids race with another git commit at the same time adjusted branch is
being updated.
2016-03-03 12:55:00 -04:00
Joey Hess
84de8bd2d0
clarify 2016-03-01 16:22:47 -04:00
Joey Hess
3e91cd13ba
Fix data loss that can occur when annex.pidlock is set in a repository. 2016-03-01 12:12:57 -04:00
Joey Hess
a97a9aaaee
remove debug 2016-02-29 17:36:20 -04:00
Joey Hess
70e78cc53e
update keys database when adjusting branches 2016-02-29 17:27:19 -04:00
Joey Hess
d7bd4d971d
implement updateAdjustedBranch 2016-02-29 17:16:56 -04:00
Joey Hess
048d513233
make assistant aware of adjusted branches when merging 2016-02-29 15:57:47 -04:00
Joey Hess
7c20bf6e7a
make sync aware of adjusted branches
So, it will pull and push the original branch, not the adjusted one.

And, for merging, it will use updateAdjustedBranch (not implemented yet).

Note that remaining uses of Git.Branch.current need to be checked too;
for things that should act on the original branch, and not the adjusted
branch.
2016-02-29 15:23:08 -04:00
Joey Hess
9e1ebc2336
include adjustment in the adjusted branch name
Allows it to be recovered easily.
2016-02-29 15:04:58 -04:00
Joey Hess
3b4557c754
Merge branch 'master' into adjustedbranch 2016-02-29 14:05:10 -04:00
Joey Hess
e520366c4d
metadata: Added -r to remove all current values of a field. 2016-02-29 13:00:46 -04:00
Joey Hess
b946ca44c3
Support --metadata field<number, --metadata field>number etc to match ranges of numeric values.
Similarly (well, for free), support preferred content expressions like
metadata=field<number and metadata=field>number
2016-02-27 10:55:02 -04:00
Joey Hess
471a211d21
Include magic database in the linux and OSX standalone builds. 2016-02-26 11:54:15 -04:00
Joey Hess
0a1b02ce04
adjusted branches, proof of concept
"git annex adjust" may be a temporary interface, but works for a proof of
concept.

It is pretty fast at creating the adjusted branch. The main overhead is
injecting pointer files. It might be worth optimising that by reusing the
symlink target as the pointer file content. When I tried to do that,
the problem was that the clean filter doesn't use that same format, and so
git thought files had changed. Could be dealt with, perhaps make the clean
filter use symlink format for pointer files when on an adjusted branch?

But the real overhead is in checking out the branch, when git runs the
smudge filter once per file. That is perhaps too slow to be usable,
although it may only affect initial checkout of the branch, and not
updates. TBD.
2016-02-25 16:23:24 -04:00
Joey Hess
4712882776
add hashPointerFile' 2016-02-25 16:10:54 -04:00
Joey Hess
04d4830ac3
add catCommit 2016-02-25 15:34:46 -04:00
Joey Hess
be2e9427ad
refactor 2016-02-25 13:46:31 -04:00
Joey Hess
a5bf674bec
Avoid crashing when built with MagicMime support, but when the magic database cannot be loaded. 2016-02-23 14:39:56 -04:00
Joey Hess
b0081598c7
Fix memory leak in last release, which affected commands like git-annex status when a large non-annexed file is present in the work tree.
The whole file was strictly read, and so buffered in memory, and remained
buffered for some time when running git-annex status.
2016-02-19 14:45:26 -04:00
Joey Hess
3fba4f83ed
fix windows build 2016-02-16 16:15:32 -04:00
Joey Hess
aa569500d5
fix numerous problem with test suite on crippled filesystems etc 2016-02-16 15:30:59 -04:00
Joey Hess
15148ee9eb
annex.addunlocked
* add, addurl, import, importfeed: When in a v6 repository on a crippled
  filesystem, add files unlocked.
* annex.addunlocked: New configuration setting, makes files always be
  added unlocked. (v6 only)
2016-02-16 14:43:43 -04:00
Joey Hess
adc27f081a
escape slashes in annex pointer files
The problem with having the slashes unescaped is, it broke parsing, since
the parser takes the filename to get the part containing the key.
That particularly affected URL keys.

This makes the format be the same as symlinks point to, which keeps things
simple.

Existing pointer files will continue to work ok.
2016-02-16 14:10:08 -04:00
Joey Hess
7899f7248a
force strict file read
Avoid possibly having the file open still when it gets deleted.

Needed on Windows, particularly.
2016-02-15 16:47:34 -04:00
Joey Hess
4d89a1ffd1
allow \r in pointer files
git-annex doesn't write \r, but it can be present due to line ending
conversions or perhaps user edits.
2016-02-15 16:37:40 -04:00
Joey Hess
f9d79d194b
Windows: Fix v6 unlocked files to actually work.
Pointer files were not being treated as annex content, so "git annex get"
didn't replace them with the object.
2016-02-15 16:12:18 -04:00
Joey Hess
2e3b5e645f
When initializing a v6 repo on a crippled filesystem, don't force it into direct mode. 2016-02-15 15:41:49 -04:00
Joey Hess
540a0343ba
more windows build fix 2016-02-15 15:03:44 -04:00
Joey Hess
f55c576923
fix windows build 2016-02-15 14:58:45 -04:00
Joey Hess
40207b26ea
move old ghc compat code into separate module; eliminate WITH_CLIBS
This avoids hsc2hs being run except when building for the old version of ghc.
Should speed up builds.
2016-02-15 11:47:33 -04:00
Joey Hess
0983f136b8
create directory for transfer lock file, and catch perm error
Before, the call to mkProgressUpdater created the directory as a
side-effect, but since that ignored failure to create it, this led to
a "does not exist" exception when the transfer lock file was created,
rather than a permissions error.

So, make sure the directory exists before trying to lock the file in it.

When a PermissionDenied exception is caught, skip making the transfer lock.
This lets downloads from readonly remotes happen.

If an upload is being tried, and the lock file can't be written due to
permissions, then probably the actual transfer will fail for the same
reason, so I think it's ok that it continues w/o taking the lock in that
case.
2016-02-12 14:11:25 -04:00
Joey Hess
17c97434f2
init: Fix bugs in submodule .git symlink fixup, that occurred when initializing in a subdirectory of a submodule and a submodule of a submodule. 2016-02-08 15:41:27 -04:00
Joey Hess
23cc315c38
matchexpression: Added --largefiles option to parse an annex.largefiles expression. 2016-02-03 16:58:36 -04:00