Merge branch 'master' of ssh://git-annex.branchable.com
This commit is contained in:
Joey Hess
commit
15ba668a20
14 changed files with 149 additions and 0 deletions
|
|
@ -0,0 +1,12 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="4.154.0.63"
|
||||
subject="comment 1"
|
||||
date="2013-08-24T18:49:18Z"
|
||||
content="""
|
||||
git-annex assumes that it can make a stable symlink from a file in the working tree to a file in the .git directory. There are several ways to break this. One, as noted, is sometimes using a repository as a submodule, and sometimes not. Another would be to play around with `GIT_DIR`.
|
||||
|
||||
I don't see a way git-annex can support those use cases, at least in indirect mode.
|
||||
|
||||
It does seem like, in direct mode, it should just work. git-annex will commit various symlinks to git, but these symlinks will never be followed to get at the content of a file, since direct mode arranges for the content to be directly present in the working tree.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="4.154.0.63"
|
||||
subject="comment 2"
|
||||
date="2013-08-24T18:52:11Z"
|
||||
content="""
|
||||
This also affects creating a second repository in the webapp, not just the repository creation at first startup.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,12 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="4.154.0.63"
|
||||
subject="comment 3"
|
||||
date="2013-08-24T16:04:21Z"
|
||||
content="""
|
||||
Unfortunately, the git error message you pasted suggests very strongly that your git repository has gotten corrupted. You can probably verify that by running `git annex fsck`. Assuming it is corrupted, the best thing to do is to make a new clone and move .git/annex and .git/config over from the corrupted repository to it, and finally run `git annex fsck`
|
||||
|
||||
You then seem to have some reason decided to go delete git-annex's .git/annex/journal/*.log files, which are just files that have not yet been committed to the git-annex branch. The only reason this \"restored some sanity\" is that git-annex was trying to commit that data to git, and failing because your git repository is corrupted.
|
||||
|
||||
(This has nothing at all to do with direct mode.)
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="4.154.0.63"
|
||||
subject="comment 5"
|
||||
date="2013-08-24T17:09:43Z"
|
||||
content="""
|
||||
Thanks for an excellent amount of debug information.
|
||||
|
||||
I can see what's leaking is UDP connections to 224.0.0.251. That address is used for the \"local pairing\" option in the webapp.
|
||||
|
||||
I was able to reproduce the problem by disabling all network interfaces except `lo`. The PairListener then failed to open a multicast listening socket. When it fails that way, it retried every second, as you noticed. And there is a socket leak in that failure mode.
|
||||
|
||||
I wonder if you're seeing this even when on the network? If so, perhaps your Ubuntu system has something going on that prevents opening a multicast listening socket on even `eth0` or `wlan0` or whatever.
|
||||
|
||||
Unfortunately, the actual socket leak bug is in the [network-multicast](http://hackage.haskell.org/package/network-multicast) library, and not in git-annex. I have filed an upstream bug report: <https://github.com/audreyt/network-multicast/issues/4>
|
||||
|
||||
Hopefully that will be dealt with soon. There is a workaround I could do in git-annex: If it fails (leaking one socket), it could wait until the NetworkListener indicated a new network interface was opened, before trying again (possibly leaking one socket again). This would change it from a 1 per second leak to a 1 per change of network leak at worst, which is probably much less likely to cause problems.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="4.154.0.63"
|
||||
subject="comment 6"
|
||||
date="2013-08-24T18:38:56Z"
|
||||
content="""
|
||||
My NetListener workaround turned out to not be portable enough.
|
||||
|
||||
However, I have sent a patch to fix the FD leak: <https://github.com/audreyt/network-multicast/pull/6>
|
||||
Hopefully it gets applied soon.
|
||||
|
||||
I have also made the PairListener only retry every 60 seconds. Which makes the leak 1/60th as bad, for whatever that's worth.
|
||||
|
||||
Once a fix for this gets into Debian, I need to remember to backport it to stable, and update the autobuilders to use it. Also need to remember to update the Android autobuilder. Leaving this bug report open until that happens.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="4.154.0.63"
|
||||
subject="comment 1"
|
||||
date="2013-08-24T16:41:04Z"
|
||||
content="""
|
||||
You are running the git annex assistant, which takes care of getting and dropping files as it's configured to do, and you are then going in and manually running `git annex get`. If the assistant sees a file's content has appeared, and that file is in a directory that it has been configured to not want the content of file in (the `archive` directory by default), it will immediately try to drop it.
|
||||
|
||||
The only thing I don't understand is why the number of `..` in the symlink would change.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="4.154.0.63"
|
||||
subject="comment 2"
|
||||
date="2013-08-24T16:41:47Z"
|
||||
content="""
|
||||
BTW if you don't want the assistant to automatically get and drop files in your repository, you can edit the repository in the webapp and select \"manual mode\".
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="4.154.0.63"
|
||||
subject="comment 15"
|
||||
date="2013-08-24T15:56:47Z"
|
||||
content="""
|
||||
No, you can sync before you copy, get, or whatever. git-annex will replace the symlinks with the actual files when they arrive at the repository.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="4.154.0.63"
|
||||
subject="comment 1"
|
||||
date="2013-08-24T18:43:08Z"
|
||||
content="""
|
||||
It should be sufficient to delete `/sdcard/git-annex.home` and delete the .git directory inside whatever directory you set up as the repository.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="4.154.0.63"
|
||||
subject="comment 1"
|
||||
date="2013-08-24T16:20:29Z"
|
||||
content="""
|
||||
In the assistant, putting a repository in manual mode prevents the assistant from automatically downloading every file that is added to other clones of the repository. As you've noticed, it does not prevent the assistant from automatically adding new files that are put into the local repository, or from uploading those files to other clients that want them (ie, clients not themselves set to use manual mode).
|
||||
|
||||
You can prevent the assistant from noticing when you add new files to a repository by clicking on \"syncing enabled\" in the first repository in the list. (The repository labeled as \"here\".) You can then `git annex add` the files you want to add by hand, and manually `copy annex copy` them to other repositories, and manually `git annex sync`.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="4.154.0.63"
|
||||
subject="comment 2"
|
||||
date="2013-08-24T15:54:02Z"
|
||||
content="""
|
||||
git-annex has always used cp --reflink=auto, including when it cp's the file in `git annex unlock`.
|
||||
|
||||
You can verify this with --debug.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="4.154.0.63"
|
||||
subject="comment 6"
|
||||
date="2013-08-24T15:58:28Z"
|
||||
content="""
|
||||
You can run any of these operations in any order and it will work. Ie, this is far more complicated and detailed than it needs to be. Just run git-annex commands in any order to do what you want to do. If they don't, file a (detailed) bug report.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="4.154.0.63"
|
||||
subject="comment 3"
|
||||
date="2013-08-24T16:35:33Z"
|
||||
content="""
|
||||
Seems to me that this could easily be dealt with by installing git-annex on the webserver, making the directory there a git repository, and using either a cron job or `git annex watch` to commit files as they were changed there.
|
||||
|
||||
Then you can make a direct mode, indirect mode, or even a bare clone on your local machine and use git-annex to get the files.
|
||||
|
||||
Maybe you have good reasons for not wanting to go that route. And rsync on a direct mode repository should work, provided to tell it to not delete `.git`. :P I don't see any way to make rsync work in an indirect mode repository. As for trying to make git-annex handle this import over rsync itself in a way that would work in an indirect mode repository, let alone a bare repository -- I don't see a good way to do it and it seems quite special case and likely to get quite complicated to implement.
|
||||
|
||||
In the meantime, I did implement `git annex mirror`, which I think is a much more interesting and generally useful tool to have. And could even be used in my recommended solution above.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://rmunn.myopenid.com/"
|
||||
nickname="rmunn"
|
||||
subject="comment 4"
|
||||
date="2013-08-24T15:31:36Z"
|
||||
content="""
|
||||
Either quvi or youtube-dl might be a good possibility: youtube-dl has the --get-url option (or -g for short) that outputs just the download URL (and nothing else) to stdout. So if for some reason quvi turned out not to be suitable, it wouldn't necessarily mean the idea would have to be abandoned.
|
||||
"""]]
|
||||
Loading…
Reference in a new issue