Merge branch 'master' into assistant

This commit is contained in:
Joey Hess 2012-08-07 13:31:43 -04:00
commit 7e2d07484f
15 changed files with 238 additions and 30 deletions

View file

@ -65,7 +65,8 @@ getKeyFile key file dest = dispatch =<< Remote.keyPossibilities key
| Remote.hasKeyCheap r =
either (const False) id <$> Remote.hasKey r key
| otherwise = return True
docopy r continue = download (Remote.uuid r) key (Just file) $ do
showAction $ "from " ++ Remote.name r
ifM (Remote.retrieveKeyFile r key (Just file) dest)
( return True , continue)
docopy r continue = do
ok <- download (Remote.uuid r) key (Just file) $ do
showAction $ "from " ++ Remote.name r
Remote.retrieveKeyFile r key (Just file) dest
if ok then return ok else continue

View file

@ -105,8 +105,8 @@ runTransfer t file a = do
writeFile tfile $ writeTransferInfo info
return fd
cleanup tfile fd = do
removeFile tfile
removeFile $ transferLockFile tfile
void $ tryIO $ removeFile tfile
void $ tryIO $ removeFile $ transferLockFile tfile
closeFd fd
{- If a transfer is still running, returns its TransferInfo. -}

View file

@ -3,30 +3,9 @@
- Copyright 2011 Joey Hess <joey@kitenet.net>
-
- Licensed under the GNU GPL version 3 or higher.
-}
module Utility.DataUnits (
dataUnits,
storageUnits,
memoryUnits,
bandwidthUnits,
oldSchoolUnits,
roughSize,
compareSizes,
readSize
) where
import Data.List
import Data.Char
type ByteSize = Integer
type Name = String
type Abbrev = String
data Unit = Unit ByteSize Abbrev Name
deriving (Ord, Show, Eq)
{- And now a rant:
-
-
- And now a rant:
-
- In the beginning, we had powers of two, and they were good.
-
@ -56,6 +35,27 @@ data Unit = Unit ByteSize Abbrev Name
- progress?
-}
module Utility.DataUnits (
dataUnits,
storageUnits,
memoryUnits,
bandwidthUnits,
oldSchoolUnits,
roughSize,
compareSizes,
readSize
) where
import Data.List
import Data.Char
type ByteSize = Integer
type Name = String
type Abbrev = String
data Unit = Unit ByteSize Abbrev Name
deriving (Ord, Show, Eq)
dataUnits :: [Unit]
dataUnits = storageUnits ++ memoryUnits

2
debian/changelog vendored
View file

@ -5,6 +5,8 @@ git-annex (3.20120722) UNRELEASED; urgency=low
* init: If no description is provided for a new repository, one will
automatically be generated, like "joey@gnu:~/foo"
* unused, status: Avoid crashing when ran in bare repo.
* Avoid crashing when "git annex get" fails to download from one
location, and falls back to downloading from a second location.
-- Joey Hess <joeyh@debian.org> Fri, 27 Jul 2012 21:04:47 -0400

View file

@ -0,0 +1,99 @@
Spent yesterday and today making the WebApp handle adding removable drives.
While it needs more testing, I think that it's now possible to use the WebApp
for a complete sneakernet usage scenario.
* Start up the webapp, let it make a local repo.
* Add some files, by clicking to open the file manager, and dragging them in.
* Plug in a drive, and tell the webapp to add it.
* Wait while files sync..
* Take the drive to another computer, and repeat the process there.
No command-line needed, and files will automatically be synced between
two or more computers using the drive.
Sneakernet is only one usage scenario for the git-annex assistant, but I'm
really happy to have one scenario 100% working!
Indeed, since the assistant and webapp can now actually do something
useful, I'll probably be merging them into `master` soon.
Details follow..
---
So, yesterday's part of this was building the configuration page to add
a removable drive. That needs to be as simple as possible, and it currently
consists of a list of things git-annex thinks might be mount points of
removable drives, along with how much free space they have. Pick a drive,
click the pretty button, and away it goes..
(I decided to make the page so simple it doesn't even ask where you want
to put the directory on the removable drive. It always puts it in
a "annex" directory. I might add an expert screen later, but experts can
always set this up themselves at the command line too.)
I also fought with Yesod and Bootstrap rather a lot to make the form look good.
Didn't entirely succeed, and had to file a bug on Yesod about its handling of
check boxes. (Bootstrap also has a bug, IMHO; its drop down lists are not
always sized wide enough for their contents.)
Ideally this configuration page would listen for mount events, and refresh
its list. I may add that eventually; I didn't have a handy channel it
could use to do that, so defferred it. Another idea is to have the mount
event listener detect removable drives that don't have an annex on them yet,
and pop up an alert with a link to this configuration page.
----
Making the form led to a somewhat interesting problem: How to tell if a mounted
filesystem is a removable drive, or some random thing like `/proc` or
a fuse filesystem. My answer, besides checking that the user can
write to it, was various heuristics, which seem to work ok, at least here..
[[!format haskell """
sane Mntent { mnt_dir = dir, mnt_fsname = dev }
{- We want real disks like /dev/foo, not
- dummy mount points like proc or tmpfs or
- gvfs-fuse-daemon. -}
| not ('/' `elem` dev) = False
{- Just in case: These mount points are surely not
- removable disks. -}
| dir == "/" = False
| dir == "/tmp" = False
| dir == "/run/shm" = False
| dir == "/run/lock" = False
"""]]
----
Today I did all the gritty coding to make it create a git repository on the
removable drive, and tell the Annex monad about it, and ensure it gets synced.
As part of that, it detects when the removable drive's filesystem doesn't
support symlinks, and makes a bare repository in that case. Another expert
level config option that's left out for now is to always make a bare
repository, or even to make a directory special remote rather than a git
repository at all. (But directory special remotes cannot support the
sneakernet use case by themselves...)
----
Another somewhat interesting problem was what to call the git remotes
that it sets up on the removable drive and the local repository.
Again this could have an expert-level configuration, but the defaults
I chose are to use the hostname as the remote name on the removable drive,
and to use the basename of the mount point of the removable drive as the
remote name in the local annex.
----
Originally, I had thought of this as cloning the repository to the drive.
But, partly due to luck, I started out just doing a `git init` to make
the repository (I had a function lying around to do that..).
And as I worked on it some more, I realized this is not as simple as a
clone. It's a bi-directional sync/merge, and indeed the removable drive may
have all the data already in it, and the local repository have just been
created. Handling all the edge cases of that (like, the local repository
may not have a "master" branch yet..) was fun!

View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawl9sYlePmv1xK-VvjBdN-5doOa_Xw-jH4U"
nickname="Richard"
subject="comment 1"
date="2012-08-05T23:50:57Z"
content="""
Using `annex` as default is fine, but even non-technical users will need a way to create different annexes for different usage. Or if two users share one thumb drive and each has their own annex. Long story short, I don't think this is an expert option, but something every user should be able to change immediately.
A pre-populated text area makes most sense, imo.
"""]]

View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="http://dieter-be.myopenid.com/"
nickname="dieter"
subject="comment 2"
date="2012-08-06T08:30:47Z"
content="""
+1 on what Richard said
about the correctly figuring out what are valid mountpoints / devices that can be used, your current code doesn't seem very robust.
check this out https://github.com/Dieterbe/aif/blob/develop/src/core/libs/lib-blockdevices-filesystems.sh#L213
"""]]

View file

@ -0,0 +1,10 @@
Nothing flashy today; I was up all night trying to download photos taken
by a robot lowered onto Mars by a skycrane.
Some work on alerts. Added an alert when a file transfer succeeds or fails.
Improved the alert combining code so it handles those alerts, and
simplified it a lot, and made it more efficient.
Also made the text of action alerts change from present to past tense when
the action finishes. To support that I wrote a fun data type, a `TenseString`
that can be rendered in either tense.

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawlJ2utMQgMEYAOs3Dfc6eZRyUzt4acNXUU"
nickname="David"
subject="comment 1"
date="2012-08-06T23:48:28Z"
content="""
I see you are a regular XKCD reader
"""]]

View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawlCGROoy62svBUy6P24x1KoGoDWrBq2ErA"
nickname="Steve"
subject="comment 1"
date="2012-08-07T04:15:43Z"
content="""
NTFS symbolic links should do exactly what you would expect them to do. They can point to files or directories. Junction points are legacy NTFS functionality and reparse points are more like the POSIX mount functionality.
NTFS symbolic links should work for you, junction point should be avoided, and reparse points would be like using a nuke to kill a fly. The only hang up you might have is that I think all three features require elevated privileges to manage.
"""]]

View file

@ -0,0 +1,18 @@
I have a remote rsync with gpg encryption and can restore without problems on my thinkpad (FreeBSD) - but not on my MacOSX:
$ git annex whereis DSC_7615.JPG
whereis DSC_7615.JPG (2 copies)
6855de17-c8fb-11e1-9948-f0def1c18073 -- thinkpad
e388bcf6-c8fc-11e1-a96f-6ffcbceb4af4 -- backup (rsync xxxx)
ok
$ git annex get --from backup DSC_7615.JPG
fatal: Could not switch to '../../../.git/annex/objects/Pw/XP/SHA256-s1028494--SHA256': No such file or directory
git-annex: <file descriptor: 6>: hGetLine: end of file
failed
git-annex: get: 1 failed
$ git annex version
git-annex version: 3.20120629
local repository version: 3

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawmBUR4O9mofxVbpb8JV9mEbVfIYv670uJo"
nickname="Justin"
subject="comment 1"
date="2012-08-07T13:11:48Z"
content="""
Do the remotes in `.git/config` look the same?
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="hannes"
ip="130.226.133.37"
subject="they do"
date="2012-08-07T14:06:55Z"
content="""
I was also able to copy from the macosx to the rsync backup...
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="hannes"
ip="130.226.133.37"
subject="also, fsck works"
date="2012-08-07T14:12:20Z"
content="""
git annex fsck -fbackup some-local-and-remote-file works fine.
"""]]

View file

@ -0,0 +1,16 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawnnIQkoUQo4RYzjUNyiB3v6yJ5aR41WG8k"
nickname="Markus"
subject="Updated install instructions with homebrew"
date="2012-08-07T06:46:47Z"
content="""
To install git annex with homebrew simply do:
brew update
brew install haskell-platform git ossp-uuid md5sha1sum coreutils pcre
cabal install git-annex
Then link the binary to your `PATH` e.g. with
ln -s ~/.cabal/bin/git-annex* /usr/local/bin/
"""]]