82cfcfc838
Use git update-index --refresh, since it's a little bit more efficient and the user can be told to run it if a locked index prevents git-annex from running it. This also fixes the problem where an annexed file was deleted in the index and a get of another file that uses the same key caused the index update to add back the deleted file. update-index will not add back the deleted file. Documented in tips/unlocked_files.mdwn the gotcha that the index update may conflict with other operations. I can't see any way to possibly avoid that conflict. One new todo about a race that causes a modification to be accidentially staged. Note that the assistant only flushes the git command queue when it commits a modification. I have not tested the assistant with v6 unlocked files, but assume most users of the assistant won't care if the index shows a file as modified for a while. This commit was supported by the NSF-funded DataLad project.
187 lines
7.1 KiB
Markdown
187 lines
7.1 KiB
Markdown
Normally, git-annex stores annexed files in the repository, locked down,
|
|
which prevents the content of the file from being modified.
|
|
That's a good thing, because it might be the only copy, you wouldn't
|
|
want to lose it in a fumblefingered mistake.
|
|
|
|
# git annex add some_file
|
|
add some_file
|
|
# echo oops > some_file
|
|
bash: some_file: Permission denied
|
|
|
|
Sometimes though you want to modify a file. Maybe once, or maybe
|
|
repeatedly. To modify a locked file, you have to first unlock it,
|
|
by running `git annex unlock`.
|
|
|
|
# git annex unlock some_file
|
|
# echo "new content" > some_file
|
|
|
|
Back before git-annex version 6, and its v6 repository mode, unlocking a file
|
|
like this was a transient thing. You'd modify it and then `git annex add` the
|
|
modified version to the annex, and finally `git commit`. The new version of
|
|
the file was then back to being locked.
|
|
|
|
# git annex add some_file
|
|
add some_file
|
|
# git commit
|
|
|
|
But, that had some problems. The main one is that some users want to be able
|
|
to edit files repeatedly, without manually having to unlock them every time.
|
|
The [[direct_mode]] made all files be unlocked all the time, but it
|
|
had many problems of its own.
|
|
|
|
## enter v6 mode
|
|
|
|
/!\ This is a new feature; see its [[todo_list|todo/smudge]]
|
|
for known issues.
|
|
|
|
This led to the v6 repository mode, which makes unlocked files remain
|
|
unlocked after they're committed, so you can keep changing them and
|
|
committing the changes whenever you'd like. It also lets you use more
|
|
normal git commands (or even interfaces on top of git) for handling
|
|
annexed files.
|
|
|
|
To get a repository into v6 mode, you can [[upgrade|upgrades]] it.
|
|
This will eventually happen automatically, but for now it's a manual process
|
|
(be sure to read [[upgrades]] before doing this):
|
|
|
|
# git annex upgrade
|
|
|
|
Or, you can init a new repository in v6 mode.
|
|
|
|
# git init
|
|
# git annex init --version=6
|
|
|
|
## using it
|
|
|
|
Using a v6 repository is easy! Simply use regular git commands to add
|
|
and commit files. In a git-annex repository, git will use git-annex
|
|
to store the file contents, and the files will be left unlocked.
|
|
|
|
[[!template id=note text="""
|
|
Want `git add` to add some file contents to the annex, but store the contents of
|
|
smaller files in git itself? Configure annex.largefiles to match the former.
|
|
See [[largefiles]].
|
|
"""]]
|
|
|
|
# cp ~/my_cool_big_file .
|
|
# git add my_cool_big_file
|
|
# git commit -m "added my_cool_big_file to the annex"
|
|
[master (root-commit) 92f2725] added my_cool_big_file to the annex
|
|
1 file changed, 1 insertion(+)
|
|
create mode 100644 my_cool_big_file
|
|
# git annex find
|
|
my_cool_big_file
|
|
|
|
You can make whatever modifications you want to unlocked files, and commit
|
|
your changes.
|
|
|
|
# echo more stuff >> my_cool_big_file
|
|
# git mv my_cool_big_file my_cool_bigger_file
|
|
# git commit -a -m "some changes"
|
|
[master 196c0e2] some changes
|
|
2 files changed, 1 insertion(+), 1 deletion(-)
|
|
delete mode 100644 my_cool_big_file
|
|
create mode 100644 my_cool_bigger_file
|
|
|
|
Under the hood, this uses git's [[todo/smudge]] filter interface, and
|
|
git-annex converts between the content of the big file and a pointer file,
|
|
which is what gets committed to git. All the regular git-annex commands
|
|
(get, drop, etc) can be used on unlocked files too.
|
|
|
|
[[!template id=note text="""
|
|
By default, git-annex commands will add files in locked mode,
|
|
unless used on a filesystem that does not support symlinks, when unlocked
|
|
mode is used. To make them always use unlocked mode, run:
|
|
`git config annex.addunlocked true`
|
|
"""]]
|
|
|
|
## mixing locked and unlocked files
|
|
|
|
A v6 repository can contain both locked and unlocked files. You can switch
|
|
a file back and forth using the `git annex lock` and `git annex unlock`
|
|
commands. This changes what's stored in git between a git-annex symlink
|
|
(locked) and a git-annex pointer file (unlocked). To add a file to
|
|
the repository in locked mode, use `git annex add`; to add a file in
|
|
unlocked mode, use `git add`.
|
|
|
|
If you want to mostly keep files locked, but be able to locally switch
|
|
to having them all unlocked, you can do so using `git annex adjust
|
|
--unlock`. See [[git-annex-adjust]] for details. This is particularly
|
|
useful when using filesystems like FAT, and OS's like Windows that don't
|
|
support symlinks.
|
|
|
|
## index gotchas
|
|
|
|
When git-annex gets or drops the content of an unlocked file, it updates
|
|
the file in git's worktree accordingly. Then it needs to update the index
|
|
file to reflect the change. Otherwise, `git status` would show the file
|
|
as modified, even though there are no changes to commit.
|
|
|
|
This means that when git-annex is running a command that gets or drops the
|
|
content of an unlocked file, the index will sometimes be locked. This might
|
|
prevent you from `git commit` at the same time. Or, if you have a git
|
|
commit in progress, or are running multiple git-annex processes, git-annex
|
|
may complain that the index is locked.
|
|
|
|
To manually update the index when git-annex was not able to, you can run:
|
|
|
|
git update-index -q --refresh $file
|
|
|
|
## using less disk space
|
|
|
|
Unlocked files are handy, but they have one significant disadvantage
|
|
compared with locked files: They use more disk space.
|
|
|
|
While only one copy of a locked file has to be stored, often
|
|
two copies of an unlocked file are stored on disk. One copy is in
|
|
the git work tree, where you can use and modify it,
|
|
and the other is stashed away in `.git/annex/objects` (see [[internals]]).
|
|
|
|
The reason for that second copy is to preserve the old version of the file,
|
|
when you modify the unlocked file in the work tree. Being able to access
|
|
old versions of files is an important part of git after all!
|
|
|
|
That's a good safe default. But there are ways to use git-annex that
|
|
make the second copy not be worth keeping:
|
|
|
|
* When you're using git-annex to sync the current version of files across
|
|
devices, and don't care much about previous versions.
|
|
* When you have set up a backup repository, and use git-annex to copy
|
|
your files to the backup.
|
|
|
|
In situations like these, you may want to avoid the overhead of the second
|
|
local copy of unlocked files. There's a config setting for that.
|
|
|
|
[[!template id=note text="""
|
|
Note that setting annex.thin only has any effect on systems that support
|
|
hard links. It is supported on Windows, but not on FAT filesystems.
|
|
"""]]
|
|
|
|
git config annex.thin true
|
|
|
|
After changing annex.thin, you'll want to fix up the work tree to
|
|
match the new setting:
|
|
|
|
git annex fix
|
|
|
|
Unfortunately, git's smudge interface does not let git-annex honor
|
|
the annex.thin configuration when git is checking out a file.
|
|
So, using `git checkout` to check out a different branch, or even
|
|
`git merge` can result in some non-thin files making their way into the
|
|
working tree, and using more disk space. A warning will be printed out in
|
|
this situation. You can always run `git annex fix` to re-thin such files.
|
|
|
|
## annex.thin tradeoffs
|
|
|
|
[[!template id=note text="""
|
|
When a [[direct_mode]] repository is upgraded, annex.thin is automatically
|
|
set, because direct mode made the same single-copy tradeoff.
|
|
"""]]
|
|
|
|
Setting annex.thin can save a lot of disk space, but it's a tradeoff
|
|
between disk usage and safety.
|
|
|
|
Keeping files locked is safer and also avoids using unnecessary
|
|
disk space, but trades off easy modification of files.
|
|
|
|
Pick the tradeoff that's right for you.
|