From cb91f3c86ad829eb07e8906d63e5d0ff9bea277d Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawnJTqmRu1YCKS2Hsm4vtOflLhP4fU-k98w" Date: Tue, 25 Feb 2014 11:42:08 +0000 Subject: [PATCH 01/17] Added a comment: Customise conflict resolution behaviour --- ...comment_1_307898855f91a2a189d4fa5eae62cce1._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/automatic_conflict_resolution/comment_1_307898855f91a2a189d4fa5eae62cce1._comment diff --git a/doc/automatic_conflict_resolution/comment_1_307898855f91a2a189d4fa5eae62cce1._comment b/doc/automatic_conflict_resolution/comment_1_307898855f91a2a189d4fa5eae62cce1._comment new file mode 100644 index 0000000000..69e136b44f --- /dev/null +++ b/doc/automatic_conflict_resolution/comment_1_307898855f91a2a189d4fa5eae62cce1._comment @@ -0,0 +1,10 @@ +[[!comment format=txt + username="https://www.google.com/accounts/o8/id?id=AItOawnJTqmRu1YCKS2Hsm4vtOflLhP4fU-k98w" + nickname="Ahmed" + subject="Customise conflict resolution behaviour" + date="2014-02-25T11:42:08Z" + content=""" +How to customise git-annex conflict resolution behaviour, such that for example: change naming convention of conflicted files with suffix or prefix, move conflicted files to another directory structure, overwrite conflicted files from preferred content ... + + +"""]] From d201d7378c3e843563e512cf689114f11ea8a2ca Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawm78jq1Uo-ZbyOPG3diJUWVvEiM0kyAcvk" Date: Tue, 25 Feb 2014 13:41:16 +0000 Subject: [PATCH 02/17] Added a comment: any ideas or questions? --- ...t_2_2e1ae66bac4f55b74074b09e22ab270d._comment | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 doc/bugs/problems_with_android_and_gpg/comment_2_2e1ae66bac4f55b74074b09e22ab270d._comment diff --git a/doc/bugs/problems_with_android_and_gpg/comment_2_2e1ae66bac4f55b74074b09e22ab270d._comment b/doc/bugs/problems_with_android_and_gpg/comment_2_2e1ae66bac4f55b74074b09e22ab270d._comment new file mode 100644 index 0000000000..c7709f0fc9 --- /dev/null +++ b/doc/bugs/problems_with_android_and_gpg/comment_2_2e1ae66bac4f55b74074b09e22ab270d._comment @@ -0,0 +1,16 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawm78jq1Uo-ZbyOPG3diJUWVvEiM0kyAcvk" + nickname="Dorian" + subject="any ideas or questions?" + date="2014-02-25T13:41:16Z" + content=""" +Hey Joey, + +I was wondering if you had any idea how we could fix this problem or if you need further information on this. +Any response would be appreciated. + +Thanks for your great work on git-annex! + +Cheers, +Dorian +"""]] From 7af533db34870dd35d78d6435e3b1eee81388d3d Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawmmA6YpvP2dFDs5rMlhBWEWWgo2IZjOQGo" Date: Tue, 25 Feb 2014 13:53:16 +0000 Subject: [PATCH 03/17] --- ...st_crashes__47__closes__47__doesn__39__t_run_on_launch.mdwn | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 doc/forum/git-annex_on_osx_10.9.1_just_crashes__47__closes__47__doesn__39__t_run_on_launch.mdwn diff --git a/doc/forum/git-annex_on_osx_10.9.1_just_crashes__47__closes__47__doesn__39__t_run_on_launch.mdwn b/doc/forum/git-annex_on_osx_10.9.1_just_crashes__47__closes__47__doesn__39__t_run_on_launch.mdwn new file mode 100644 index 0000000000..5104e7712e --- /dev/null +++ b/doc/forum/git-annex_on_osx_10.9.1_just_crashes__47__closes__47__doesn__39__t_run_on_launch.mdwn @@ -0,0 +1,3 @@ +on a Mac OSX 10.9.1 client I would like to try out git-annex. I download the latest Mavericks build, extract from the .dmg and it just does not run. I open the program, one "bounce" on the dock and it is gone, nothing happens. I sadly do not have another OSX client to try it on. But nothing shows in the Console.app or anything. + +Am I missing something? From 827d6916ba04317892e9072007098aff37865042 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawmmA6YpvP2dFDs5rMlhBWEWWgo2IZjOQGo" Date: Tue, 25 Feb 2014 13:53:56 +0000 Subject: [PATCH 04/17] --- ...crashes__47__closes__47__doesn__39__t_run_on_launch.mdwn | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/doc/forum/git-annex_on_osx_10.9.1_just_crashes__47__closes__47__doesn__39__t_run_on_launch.mdwn b/doc/forum/git-annex_on_osx_10.9.1_just_crashes__47__closes__47__doesn__39__t_run_on_launch.mdwn index 5104e7712e..c5d4c40bcf 100644 --- a/doc/forum/git-annex_on_osx_10.9.1_just_crashes__47__closes__47__doesn__39__t_run_on_launch.mdwn +++ b/doc/forum/git-annex_on_osx_10.9.1_just_crashes__47__closes__47__doesn__39__t_run_on_launch.mdwn @@ -1,3 +1,7 @@ -on a Mac OSX 10.9.1 client I would like to try out git-annex. I download the latest Mavericks build, extract from the .dmg and it just does not run. I open the program, one "bounce" on the dock and it is gone, nothing happens. I sadly do not have another OSX client to try it on. But nothing shows in the Console.app or anything. +on a Mac OSX 10.9.1 client I would like to try out git-annex. + +I download the latest Mavericks build, extract from the .dmg and it just does not run. I open the program, one "bounce" on the dock and it is gone, nothing happens. + +I sadly do not have another OSX client to try it on. But nothing shows in the Console.app or anything. Am I missing something? From 46b028d3cd293f228e227fd25f23b200ab917f6d Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawlm_3m5gLhML9bHbZ8FpJ-HBZhWaRfFeO8" Date: Tue, 25 Feb 2014 17:23:34 +0000 Subject: [PATCH 05/17] Added a comment: I got this error too. --- .../comment_5_9bfd8df548d5866599dfc69fb3aaf94a._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/bugs/Crash_when_adding_jabber_account_/comment_5_9bfd8df548d5866599dfc69fb3aaf94a._comment diff --git a/doc/bugs/Crash_when_adding_jabber_account_/comment_5_9bfd8df548d5866599dfc69fb3aaf94a._comment b/doc/bugs/Crash_when_adding_jabber_account_/comment_5_9bfd8df548d5866599dfc69fb3aaf94a._comment new file mode 100644 index 0000000000..40bc68dcb0 --- /dev/null +++ b/doc/bugs/Crash_when_adding_jabber_account_/comment_5_9bfd8df548d5866599dfc69fb3aaf94a._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawlm_3m5gLhML9bHbZ8FpJ-HBZhWaRfFeO8" + nickname="Corey" + subject="I got this error too." + date="2014-02-25T17:23:33Z" + content=""" +With the armel tarbell downloaded 2014-02-21 on an Ubuntu (precise) chroot running on a Chromebook. +"""]] From 360ecb9f3503b2c41b77e71426978216f7f7d504 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 25 Feb 2014 13:47:09 -0400 Subject: [PATCH 06/17] fix bare repo optimisation on Windows --- Remote/Git.hs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Remote/Git.hs b/Remote/Git.hs index d7385ef31b..4508d45554 100644 --- a/Remote/Git.hs +++ b/Remote/Git.hs @@ -285,7 +285,7 @@ keyUrls r key = map tourl locs' #ifndef mingw32_HOST_OS locs' = locs #else - locs' = map (replace "\\" "/") (annexLocations key) + locs' = map (replace "\\" "/") locs #endif dropKey :: Remote -> Key -> Annex Bool From 3f6e4b8c7c950770aa4f29c2f4634954dbed4420 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 25 Feb 2014 14:09:39 -0400 Subject: [PATCH 07/17] fix all remaining -Wall warnings on Windows --- Assistant.hs | 19 +++++++++++-------- Assistant/Threads/SanityChecker.hs | 10 ++++++---- Assistant/Threads/Watcher.hs | 4 +++- Assistant/WebApp/Configurators/Local.hs | 8 +++++--- Assistant/WebApp/Configurators/Pairing.hs | 4 +++- Command/Add.hs | 8 ++++---- Command/Fsck.hs | 22 +++++++++++++--------- Git/Command.hs | 3 --- Git/Queue.hs | 11 ++++++----- Utility/LogFile.hs | 2 ++ Utility/Path.hs | 1 - Utility/WebApp.hs | 5 ++++- Utility/WinProcess.hs | 4 ---- 13 files changed, 57 insertions(+), 44 deletions(-) diff --git a/Assistant.hs b/Assistant.hs index c66a1b73b1..db6fc850ce 100644 --- a/Assistant.hs +++ b/Assistant.hs @@ -49,11 +49,13 @@ import Assistant.Threads.XMPPPusher import Assistant.Types.UrlRenderer #endif import qualified Utility.Daemon -import Utility.LogFile import Utility.ThreadScheduler import Utility.HumanTime -import Annex.Perms import qualified Build.SysConfig as SysConfig +#ifndef mingw32_HOST_OS +import Utility.LogFile +import Annex.Perms +#endif import System.Log.Logger import Network.Socket (HostName) @@ -70,8 +72,8 @@ startDaemon :: Bool -> Bool -> Maybe Duration -> Maybe String -> Maybe HostName startDaemon assistant foreground startdelay cannotrun listenhost startbrowser = do Annex.changeState $ \s -> s { Annex.daemon = True } pidfile <- fromRepo gitAnnexPidFile - logfile <- fromRepo gitAnnexLogFile #ifndef mingw32_HOST_OS + logfile <- fromRepo gitAnnexLogFile createAnnexDirectory (parentDir logfile) logfd <- liftIO $ openLog logfile if foreground @@ -93,11 +95,12 @@ startDaemon assistant foreground startdelay cannotrun listenhost startbrowser = start (Utility.Daemon.daemonize logfd (Just pidfile) False) Nothing #else -- Windows is always foreground, and has no log file. - liftIO $ Utility.Daemon.lockPidFile pidfile - start id $ do - case startbrowser of - Nothing -> Nothing - Just a -> Just $ a Nothing Nothing + when (foreground || not foreground) $ do + liftIO $ Utility.Daemon.lockPidFile pidfile + start id $ do + case startbrowser of + Nothing -> Nothing + Just a -> Just $ a Nothing Nothing #endif where desc diff --git a/Assistant/Threads/SanityChecker.hs b/Assistant/Threads/SanityChecker.hs index b94d010e3d..d7fe366e2a 100644 --- a/Assistant/Threads/SanityChecker.hs +++ b/Assistant/Threads/SanityChecker.hs @@ -27,7 +27,6 @@ import qualified Git.Command import qualified Git.Config import Utility.ThreadScheduler import qualified Assistant.Threads.Watcher as Watcher -import Utility.LogFile import Utility.Batch import Utility.NotificationBroadcaster import Config @@ -43,6 +42,9 @@ import qualified Annex #ifdef WITH_WEBAPP import Assistant.WebApp.Types #endif +#ifndef mingw32_HOST_OS +import Utility.LogFile +#endif import Data.Time.Clock.POSIX import qualified Data.Text as T @@ -214,10 +216,10 @@ checkLogSize n = do checkLogSize $ n + 1 where filesize f = fromIntegral . fileSize <$> liftIO (getFileStatus f) -#endif -oneMegabyte :: Int -oneMegabyte = 1000000 + oneMegabyte :: Int + oneMegabyte = 1000000 +#endif oneHour :: Int oneHour = 60 * 60 diff --git a/Assistant/Threads/Watcher.hs b/Assistant/Threads/Watcher.hs index 50a0efdd54..611eca1a74 100644 --- a/Assistant/Threads/Watcher.hs +++ b/Assistant/Threads/Watcher.hs @@ -23,7 +23,6 @@ import Assistant.Types.Changes import Assistant.Alert import Utility.DirWatcher import Utility.DirWatcher.Types -import qualified Utility.Lsof as Lsof import qualified Annex import qualified Annex.Queue import qualified Git @@ -40,6 +39,9 @@ import Annex.ReplaceFile import Git.Types import Config import Utility.ThreadScheduler +#ifndef mingw32_HOST_OS +import qualified Utility.Lsof as Lsof +#endif import Data.Bits.Utils import Data.Typeable diff --git a/Assistant/WebApp/Configurators/Local.hs b/Assistant/WebApp/Configurators/Local.hs index 8bbdadacf1..99592f6c39 100644 --- a/Assistant/WebApp/Configurators/Local.hs +++ b/Assistant/WebApp/Configurators/Local.hs @@ -114,9 +114,9 @@ checkRepositoryPath p = do - browsed to a directory with git-annex and run it from there. -} defaultRepositoryPath :: Bool -> IO FilePath defaultRepositoryPath firstrun = do - cwd <- liftIO getCurrentDirectory - home <- myHomeDir #ifndef mingw32_HOST_OS + home <- myHomeDir + cwd <- liftIO getCurrentDirectory if home == cwd && firstrun then inhome else ifM (legit cwd <&&> canWrite cwd) @@ -127,7 +127,7 @@ defaultRepositoryPath firstrun = do -- On Windows, always default to ~/Desktop/annex or ~/annex, -- no cwd handling because the user might be able to write -- to the entire drive. - inhome + if firstrun then inhome else inhome #endif where inhome = do @@ -136,9 +136,11 @@ defaultRepositoryPath firstrun = do ( relHome $ desktop gitAnnexAssistantDefaultDir , return $ "~" gitAnnexAssistantDefaultDir ) +#ifndef mingw32_HOST_OS -- Avoid using eg, standalone build's git-annex.linux/ directory -- when run from there. legit d = not <$> doesFileExist (d "git-annex") +#endif newRepositoryForm :: FilePath -> Hamlet.Html -> MkMForm RepositoryPath newRepositoryForm defpath msg = do diff --git a/Assistant/WebApp/Configurators/Pairing.hs b/Assistant/WebApp/Configurators/Pairing.hs index 788b5f637d..d2d63eb114 100644 --- a/Assistant/WebApp/Configurators/Pairing.hs +++ b/Assistant/WebApp/Configurators/Pairing.hs @@ -12,7 +12,6 @@ module Assistant.WebApp.Configurators.Pairing where import Assistant.Pairing import Assistant.WebApp.Common -import Assistant.WebApp.Configurators import Assistant.Types.Buddies import Annex.UUID #ifdef WITH_PAIRING @@ -34,6 +33,9 @@ import Assistant.NetMessager import Assistant.WebApp.RepoList import Assistant.WebApp.Configurators.XMPP #endif +#if defined(WITH_PAIRING) || defined(WITH_XMP) +import Assistant.WebApp.Configurators +#endif import Utility.UserInfo import Git diff --git a/Command/Add.hs b/Command/Add.hs index 662ce4242c..ea4933748b 100644 --- a/Command/Add.hs +++ b/Command/Add.hs @@ -218,15 +218,15 @@ link file key mcache = flip catchAnnex (undo file key) $ do l <- inRepo $ gitAnnexLink file key replaceFile file $ makeAnnexLink l -#ifdef WITH_CLIBS -#ifndef __ANDROID__ -- touch symlink to have same time as the original file, -- as provided in the InodeCache case mcache of +#if defined(WITH_CLIBS) && ! defined(__ANDROID__) Just c -> liftIO $ touch file (TimeSpec $ inodeCacheToMtime c) False +#else + Just _ -> noop +#endif Nothing -> noop -#endif -#endif return l diff --git a/Command/Fsck.hs b/Command/Fsck.hs index 6cf444967c..a30e895d0b 100644 --- a/Command/Fsck.hs +++ b/Command/Fsck.hs @@ -466,7 +466,8 @@ getFsckTime key = do - To guard against time stamp damange (for example, if an annex directory - is copied without -a), the fsckstate file contains a time that should - be identical to its modification time. - - (This is not possible to do on Windows.) + - (This is not possible to do on Windows, and so the timestamp in + - the file will only be equal or greater than the modification time.) -} recordStartTime :: Annex () recordStartTime = do @@ -477,10 +478,10 @@ recordStartTime = do withFile f WriteMode $ \h -> do #ifndef mingw32_HOST_OS t <- modificationTime <$> getFileStatus f - hPutStr h $ showTime $ realToFrac t #else - noop + t <- getPOSIXTime #endif + hPutStr h $ showTime $ realToFrac t where showTime :: POSIXTime -> String showTime = show @@ -494,15 +495,18 @@ getStartTime = do f <- fromRepo gitAnnexFsckState liftIO $ catchDefaultIO Nothing $ do timestamp <- modificationTime <$> getFileStatus f -#ifndef mingw32_HOST_OS - t <- readishTime <$> readFile f - return $ if Just (realToFrac timestamp) == t + let fromstatus = Just (realToFrac timestamp) + fromfile <- readishTime <$> readFile f + return $ if matchingtimestamp fromfile fromstatus then Just timestamp else Nothing -#else - return $ Just timestamp -#endif where readishTime :: String -> Maybe POSIXTime readishTime s = utcTimeToPOSIXSeconds <$> parseTime defaultTimeLocale "%s%Qs" s + matchingtimestamp fromfile fromstatus = +#ifndef mingw32_HOST_OS + fromfile == fromstatus +#else + fromfile >= fromstatus +#endif diff --git a/Git/Command.hs b/Git/Command.hs index 0fa3d1b3b6..a0c7c4b2aa 100644 --- a/Git/Command.hs +++ b/Git/Command.hs @@ -15,9 +15,6 @@ import Common import Git import Git.Types import qualified Utility.CoProcess as CoProcess -#ifdef mingw32_HOST_OS -import Git.FilePath -#endif import Utility.Batch {- Constructs a git command line operating on the specified repo. -} diff --git a/Git/Queue.hs b/Git/Queue.hs index 9bb7f77d1f..7f8724c8d5 100644 --- a/Git/Queue.hs +++ b/Git/Queue.hs @@ -17,16 +17,17 @@ module Git.Queue ( flush, ) where -import qualified Data.Map as M -import System.IO -import System.Process - import Utility.SafeCommand import Common import Git import Git.Command import qualified Git.UpdateIndex +import qualified Data.Map as M +#ifndef mingw32_HOST_OS +import System.Process +#endif + {- Queable actions that can be performed in a git repository. -} data Action @@ -149,6 +150,7 @@ runAction repo (UpdateIndexAction streamers) = Git.UpdateIndex.streamUpdateIndex repo $ reverse streamers runAction repo action@(CommandAction {}) = #ifndef mingw32_HOST_OS + let p = (proc "xargs" $ "-0":"git":toCommand gitparams) { env = gitEnv repo } withHandle StdinHandle createProcessSuccess p $ \h -> do fileEncoding h hPutStr h $ intercalate "\0" $ toCommand $ getFiles action @@ -162,6 +164,5 @@ runAction repo action@(CommandAction {}) = void $ boolSystem "git" (gitparams ++ [f]) #endif where - p = (proc "xargs" $ "-0":"git":toCommand gitparams) { env = gitEnv repo } gitparams = gitCommandLine (Param (getSubcommand action):getParams action) repo diff --git a/Utility/LogFile.hs b/Utility/LogFile.hs index 4e76116ba8..73fbf820e3 100644 --- a/Utility/LogFile.hs +++ b/Utility/LogFile.hs @@ -11,7 +11,9 @@ module Utility.LogFile where import Common +#ifndef mingw32_HOST_OS import System.Posix.Types +#endif #ifndef mingw32_HOST_OS openLog :: FilePath -> IO Fd diff --git a/Utility/Path.hs b/Utility/Path.hs index e22d0c3f7e..570350d61f 100644 --- a/Utility/Path.hs +++ b/Utility/Path.hs @@ -18,7 +18,6 @@ import Data.Char import Control.Applicative #ifdef mingw32_HOST_OS -import Data.Char import qualified System.FilePath.Posix as Posix #else import System.Posix.Files diff --git a/Utility/WebApp.hs b/Utility/WebApp.hs index f92c21e4e8..6318abdcf1 100644 --- a/Utility/WebApp.hs +++ b/Utility/WebApp.hs @@ -23,7 +23,6 @@ import Network.HTTP.Types import System.Log.Logger import qualified Data.CaseInsensitive as CI import Network.Socket -import Control.Exception import "crypto-api" Crypto.Random import qualified Web.ClientSession as CS import qualified Data.ByteString.Lazy as L @@ -39,6 +38,10 @@ import Control.Concurrent #ifdef __ANDROID__ import Data.Endian #endif +#if defined(__ANDROID__) || defined (mingw32_HOST_OS) +#else +import Control.Exception (bracketOnError) +#endif localhost :: HostName localhost = "localhost" diff --git a/Utility/WinProcess.hs b/Utility/WinProcess.hs index 5c6d4cfce7..7a566dcba9 100644 --- a/Utility/WinProcess.hs +++ b/Utility/WinProcess.hs @@ -11,9 +11,5 @@ module Utility.WinProcess where import Utility.PID -import System.Win32.Process -import Foreign.C -import Control.Exception - foreign import ccall unsafe "terminatepid" terminatePID :: PID -> IO () From e1cf4e5e4e0ec23a2507269518edc2e2f174f5f2 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 25 Feb 2014 14:49:27 -0400 Subject: [PATCH 08/17] only warn about no dbus when building on linux --- Assistant/Threads/NetWatcher.hs | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Assistant/Threads/NetWatcher.hs b/Assistant/Threads/NetWatcher.hs index a7124fa01c..0b009647c1 100644 --- a/Assistant/Threads/NetWatcher.hs +++ b/Assistant/Threads/NetWatcher.hs @@ -24,8 +24,10 @@ import DBus import Data.Word (Word32) import Assistant.NetMessager #else +#ifdef linux_HOST_OS #warning Building without dbus support; will poll for network connection changes #endif +#endif netWatcherThread :: NamedThread #if WITH_DBUS From f8cfcd4e445fbcde071ef3abd5d8b675061f53a7 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 25 Feb 2014 14:53:43 -0400 Subject: [PATCH 09/17] couple more warning fixes --- Assistant/WebApp/Configurators/Pairing.hs | 4 +--- Git/Queue.hs | 2 +- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/Assistant/WebApp/Configurators/Pairing.hs b/Assistant/WebApp/Configurators/Pairing.hs index d2d63eb114..3dd10c1012 100644 --- a/Assistant/WebApp/Configurators/Pairing.hs +++ b/Assistant/WebApp/Configurators/Pairing.hs @@ -31,10 +31,8 @@ import Network.Protocol.XMPP import Assistant.Types.NetMessager import Assistant.NetMessager import Assistant.WebApp.RepoList -import Assistant.WebApp.Configurators.XMPP -#endif -#if defined(WITH_PAIRING) || defined(WITH_XMP) import Assistant.WebApp.Configurators +import Assistant.WebApp.Configurators.XMPP #endif import Utility.UserInfo import Git diff --git a/Git/Queue.hs b/Git/Queue.hs index 7f8724c8d5..5f7b142c0a 100644 --- a/Git/Queue.hs +++ b/Git/Queue.hs @@ -148,7 +148,7 @@ runAction :: Repo -> Action -> IO () runAction repo (UpdateIndexAction streamers) = -- list is stored in reverse order Git.UpdateIndex.streamUpdateIndex repo $ reverse streamers -runAction repo action@(CommandAction {}) = +runAction repo action@(CommandAction {}) = do #ifndef mingw32_HOST_OS let p = (proc "xargs" $ "-0":"git":toCommand gitparams) { env = gitEnv repo } withHandle StdinHandle createProcessSuccess p $ \h -> do From 2b66aaa763051ef777e69ecca631d9bd91562023 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 25 Feb 2014 15:13:27 -0400 Subject: [PATCH 10/17] Windows webdav: Fix DOS path separator bug. Use posix etc for urls. --- Remote/WebDAV.hs | 3 ++- debian/changelog | 1 + doc/todo/windows_support.mdwn | 2 ++ 3 files changed, 5 insertions(+), 1 deletion(-) diff --git a/Remote/WebDAV.hs b/Remote/WebDAV.hs index 8ac9c2c79a..301ec537c1 100644 --- a/Remote/WebDAV.hs +++ b/Remote/WebDAV.hs @@ -24,8 +24,9 @@ import Network.HTTP.Conduit (HttpException(..)) #endif import Network.HTTP.Types import System.IO.Error +import System.FilePath.Posix ((), addTrailingPathSeparator) -import Common.Annex +import Common.Annex hiding ((), addTrailingPathSeparator) import Types.Remote import qualified Git import Config diff --git a/debian/changelog b/debian/changelog index 1e260a4242..1bce6a26ce 100644 --- a/debian/changelog +++ b/debian/changelog @@ -23,6 +23,7 @@ git-annex (5.20140222) UNRELEASED; urgency=medium * repair: Optimise unpacking of pack files, and avoid repeated error messages about corrupt pack files. * Make annex.web-options be used in several places that call curl. + * Windows webdav: Fix DOS path separator bug. -- Joey Hess Fri, 21 Feb 2014 13:03:04 -0400 diff --git a/doc/todo/windows_support.mdwn b/doc/todo/windows_support.mdwn index ea532dfc1c..75c576401a 100644 --- a/doc/todo/windows_support.mdwn +++ b/doc/todo/windows_support.mdwn @@ -30,6 +30,8 @@ now! --[[Joey]] (and possibly gpg) are not prompted there anymore. * Local pairing seems to fail, after acking on Linux box, it stalls. + (Also, of course, the Windows box is unlikely to have a ssh server, + so only pairing with a !Windows box will work.) * gcrypt is not ported to windows (and as a shell script, may need to be rewritten) From b1931d1cc18860e790246cc64401127c0edf03cb Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 25 Feb 2014 15:35:45 -0400 Subject: [PATCH 11/17] add protocol-level debugging for dav --- Remote/WebDAV.hs | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/Remote/WebDAV.hs b/Remote/WebDAV.hs index 301ec537c1..737b98fa70 100644 --- a/Remote/WebDAV.hs +++ b/Remote/WebDAV.hs @@ -23,6 +23,7 @@ import Network.HTTP.Client (HttpException(..)) import Network.HTTP.Conduit (HttpException(..)) #endif import Network.HTTP.Types +import System.Log.Logger (debugM) import System.IO.Error import System.FilePath.Posix ((), addTrailingPathSeparator) @@ -312,12 +313,16 @@ contentType = Just $ B8.fromString "application/octet-stream" throwIO :: String -> IO a throwIO msg = ioError $ mkIOError userErrorType msg Nothing Nothing +debugDAV :: DavUrl -> String -> IO () +debugDAV msg url = debugM "DAV" $ msg ++ " " ++ url + {--------------------------------------------------------------------- - Low-level DAV operations, using the new DAV monad when available. ---------------------------------------------------------------------} putDAV :: DavUrl -> DavUser -> DavPass -> L.ByteString -> IO () -putDAV url user pass b = +putDAV url user pass b = do + debugDAV "PUT" url #if MIN_VERSION_DAV(0,6,0) goDAV url user pass $ putContentM (contentType, b) #else @@ -325,7 +330,9 @@ putDAV url user pass b = #endif getDAV :: DavUrl -> DavUser -> DavPass -> IO (Maybe L.ByteString) -getDAV url user pass = eitherToMaybe <$> tryNonAsync go +getDAV url user pass = do + debugDAV "GET" url + eitherToMaybe <$> tryNonAsync go where #if MIN_VERSION_DAV(0,6,0) go = goDAV url user pass $ snd <$> getContentM @@ -334,7 +341,8 @@ getDAV url user pass = eitherToMaybe <$> tryNonAsync go #endif deleteDAV :: DavUrl -> DavUser -> DavPass -> IO () -deleteDAV url user pass = +deleteDAV url user pass = do + debugDAV "DELETE" url #if MIN_VERSION_DAV(0,6,0) goDAV url user pass delContentM #else @@ -342,7 +350,8 @@ deleteDAV url user pass = #endif moveDAV :: DavUrl -> DavUrl -> DavUser -> DavPass -> IO () -moveDAV url newurl user pass = +moveDAV url newurl user pass = do + debugDAV ("MOVE to " ++ newurl ++ " from ") url #if MIN_VERSION_DAV(0,6,0) goDAV url user pass $ moveContentM newurl' #else @@ -352,7 +361,8 @@ moveDAV url newurl user pass = newurl' = B8.fromString newurl mkdirDAV :: DavUrl -> DavUser -> DavPass -> IO Bool -mkdirDAV url user pass = +mkdirDAV url user pass = do + debugDAV "MKDIR" url #if MIN_VERSION_DAV(0,6,0) goDAV url user pass mkCol #else @@ -360,7 +370,9 @@ mkdirDAV url user pass = #endif existsDAV :: DavUrl -> DavUser -> DavPass -> IO (Either String Bool) -existsDAV url user pass = either (Left . show) id <$> tryNonAsync check +existsDAV url user pass = do + debugDAV "EXISTS" url + either (Left . show) id <$> tryNonAsync check where ispresent = return . Right #if MIN_VERSION_DAV(0,6,0) From 2aeb0750f9742d1a13d958652d68264de825c4f7 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 25 Feb 2014 16:09:50 -0400 Subject: [PATCH 12/17] more DAV url fixes for windows --- Remote/WebDAV.hs | 24 ++-------------------- Remote/WebDAV/DavUrl.hs | 44 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 46 insertions(+), 22 deletions(-) create mode 100644 Remote/WebDAV/DavUrl.hs diff --git a/Remote/WebDAV.hs b/Remote/WebDAV.hs index 737b98fa70..2cfe2f6d2d 100644 --- a/Remote/WebDAV.hs +++ b/Remote/WebDAV.hs @@ -14,7 +14,6 @@ import qualified Data.Map as M import qualified Data.ByteString.UTF8 as B8 import qualified Data.ByteString.Lazy.UTF8 as L8 import qualified Data.ByteString.Lazy as L -import Network.URI (normalizePathSegments) import qualified Control.Exception as E import qualified Control.Exception.Lifted as EL #if MIN_VERSION_DAV(0,6,0) @@ -25,9 +24,8 @@ import Network.HTTP.Conduit (HttpException(..)) import Network.HTTP.Types import System.Log.Logger (debugM) import System.IO.Error -import System.FilePath.Posix ((), addTrailingPathSeparator) -import Common.Annex hiding ((), addTrailingPathSeparator) +import Common.Annex import Types.Remote import qualified Git import Config @@ -40,8 +38,8 @@ import Creds import Utility.Metered import Annex.Content import Annex.UUID +import Remote.WebDAV.DavUrl -type DavUrl = String type DavUser = B8.ByteString type DavPass = B8.ByteString @@ -237,19 +235,6 @@ toDavUser = B8.fromString toDavPass :: String -> DavPass toDavPass = B8.fromString -{- The directory where files(s) for a key are stored. -} -davLocation :: DavUrl -> Key -> DavUrl -davLocation baseurl k = addTrailingPathSeparator $ - davUrl baseurl $ hashDirLower k keyFile k - -{- Where we store temporary data for a key as it's being uploaded. -} -tmpLocation :: DavUrl -> Key -> DavUrl -tmpLocation baseurl k = addTrailingPathSeparator $ - davUrl baseurl $ "tmp" keyFile k - -davUrl :: DavUrl -> FilePath -> DavUrl -davUrl baseurl file = baseurl file - {- Creates a directory in WebDAV, if not already present; also creating - any missing parent directories. -} mkdirRecursiveDAV :: DavUrl -> DavUser -> DavPass -> IO () @@ -272,11 +257,6 @@ mkdirRecursiveDAV url user pass = go url - to use this directory will fail. -} Left _ -> return () -urlParent :: DavUrl -> DavUrl -urlParent url = dropTrailingPathSeparator $ - normalizePathSegments (dropTrailingPathSeparator url ++ "/..") - where - {- Test if a WebDAV store is usable, by writing to a test file, and then - deleting the file. Exits with an IO error if not. -} testDav :: String -> Maybe CredPair -> Annex () diff --git a/Remote/WebDAV/DavUrl.hs b/Remote/WebDAV/DavUrl.hs new file mode 100644 index 0000000000..4862c4f37d --- /dev/null +++ b/Remote/WebDAV/DavUrl.hs @@ -0,0 +1,44 @@ +{- WebDAV urls. + - + - Copyright 2014 Joey Hess + - + - Licensed under the GNU GPL version 3 or higher. + -} + +{-# LANGUAGE CPP #-} + +module Remote.WebDAV.DavUrl where + +import Types +import Locations + +import Network.URI (normalizePathSegments) +import System.FilePath.Posix +#ifdef mingw32_HOST_OS +import Data.String.Utils +#endif + +type DavUrl = String + +{- The directory where files(s) for a key are stored. -} +davLocation :: DavUrl -> Key -> DavUrl +davLocation baseurl k = addTrailingPathSeparator $ + davUrl baseurl $ hashdir keyFile k + where +#ifndef mingw32_HOST_OS + hashdir = hashDirLower k +#else + hashdir = replace "\\" "/" (hashDirLower k) +#endif + +{- Where we store temporary data for a key as it's being uploaded. -} +tmpLocation :: DavUrl -> Key -> DavUrl +tmpLocation baseurl k = addTrailingPathSeparator $ + davUrl baseurl $ "tmp" keyFile k + +davUrl :: DavUrl -> FilePath -> DavUrl +davUrl baseurl file = baseurl file + +urlParent :: DavUrl -> DavUrl +urlParent url = dropTrailingPathSeparator $ + normalizePathSegments (dropTrailingPathSeparator url ++ "/..") From 326aa6c604256740892d2a1dc3e34af0b074978c Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 25 Feb 2014 16:41:44 -0400 Subject: [PATCH 13/17] webdav and S3 tested and working on windows, for real now --- doc/todo/windows_support.mdwn | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/todo/windows_support.mdwn b/doc/todo/windows_support.mdwn index 75c576401a..bddd920986 100644 --- a/doc/todo/windows_support.mdwn +++ b/doc/todo/windows_support.mdwn @@ -47,7 +47,8 @@ now! --[[Joey]] * Does not work with Cygwin's build of git (that git does not consistently support use of DOS style paths, which git-annex uses on Windows). - Must use Msysgit. + Must use Msysgit. (Actually, this may no longer be the case; git-annex's + handling of paths has improved significantly on windows..) * rsync special remotes with a rsyncurl of a local directory are known buggy. (git-annex tells rsync C:foo and it thinks it means a remote host named C...) @@ -60,7 +61,6 @@ now! --[[Joey]] ## stuff needing testing -* test S3 and box.com setup in webapp now that they should work.. * test that adding a repo on a removable drive works; that git is synced to it and files can be transferred to it and back * Does stopping in progress transfers work in the webapp? From 2dc336b1b8cf186994814bb2ee34b4f39072c394 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 25 Feb 2014 17:12:21 -0400 Subject: [PATCH 14/17] remove todo item about gcywin git, seems no longer a problem! --- doc/todo/windows_support.mdwn | 4 ---- 1 file changed, 4 deletions(-) diff --git a/doc/todo/windows_support.mdwn b/doc/todo/windows_support.mdwn index bddd920986..599fbbbbf9 100644 --- a/doc/todo/windows_support.mdwn +++ b/doc/todo/windows_support.mdwn @@ -45,10 +45,6 @@ now! --[[Joey]] ## minor problems -* Does not work with Cygwin's build of git (that git does not consistently - support use of DOS style paths, which git-annex uses on Windows). - Must use Msysgit. (Actually, this may no longer be the case; git-annex's - handling of paths has improved significantly on windows..) * rsync special remotes with a rsyncurl of a local directory are known buggy. (git-annex tells rsync C:foo and it thinks it means a remote host named C...) From 3bf518313673ff814d5b11b79c87fd21921bbeda Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 25 Feb 2014 17:25:53 -0400 Subject: [PATCH 15/17] reorg --- debian/changelog | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/debian/changelog b/debian/changelog index 1bce6a26ce..e0dac361f7 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,29 +1,29 @@ git-annex (5.20140222) UNRELEASED; urgency=medium - * Fix handling of rsync remote urls containing a username, - including rsync.net. - * --metadata field=value can now use globs to match, and matches - case insensatively, the same as git annex view field=value does. + * metadata: Field names limited to alphanumerics and a few whitelisted + punctuation characters to avoid issues with views, etc. * When constructing views, metadata is available about the location of the file in the view's reference branch. Allows incorporating parts of the directory hierarchy in a view. For example `git annex view tag=* podcasts/=*` makes a view in the form tag/showname. + * --metadata field=value can now use globs to match, and matches + case insensatively, the same as git annex view field=value does. * annex.genmetadata can be set to make git-annex automatically set metadata (year and month) when adding files. + * Make annex.web-options be used in several places that call curl. + * Fix handling of rsync remote urls containing a username, + including rsync.net. * Preserve metadata when staging a new version of an annexed file. - * metadata: Field names limited to alphanumerics and a few whitelisted - punctuation characters to avoid issues with views, etc. * metadata: Support --json * webapp: Fix creation of box.com and Amazon S3 and Glacier repositories, broken in 5.20140221. * webdav: When built with DAV 0.6.0, use the new DAV monad to avoid locking files, which is not needed by git-annex's use of webdav, and does not work on Box.com. + * webdav: Fix path separator bug when used on Windows. * repair: Optimise unpacking of pack files, and avoid repeated error messages about corrupt pack files. - * Make annex.web-options be used in several places that call curl. - * Windows webdav: Fix DOS path separator bug. -- Joey Hess Fri, 21 Feb 2014 13:03:04 -0400 From aa39457f5faa6709dcc7d0caf3647c9322200845 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 25 Feb 2014 17:26:04 -0400 Subject: [PATCH 16/17] update --- doc/git-annex.mdwn | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/doc/git-annex.mdwn b/doc/git-annex.mdwn index 441da7b98d..9b9b53902f 100644 --- a/doc/git-annex.mdwn +++ b/doc/git-annex.mdwn @@ -1525,8 +1525,7 @@ Here are all the supported configuration settings. * `annex.web-options` - Options to use when using wget or curl to download a file from the web. - (wget is always used in preference to curl if available.) + Options to pass when running wget or curl. For example, to force ipv4 only, set it to "-4" * `annex.quvi-options` From 923aa48bf4eb0a6c337f938b742d7029d95f6ace Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 25 Feb 2014 17:26:13 -0400 Subject: [PATCH 17/17] devblog --- doc/devblog/day_122_more_windows_porting.mdwn | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 doc/devblog/day_122_more_windows_porting.mdwn diff --git a/doc/devblog/day_122_more_windows_porting.mdwn b/doc/devblog/day_122_more_windows_porting.mdwn new file mode 100644 index 0000000000..6f87a0bc09 --- /dev/null +++ b/doc/devblog/day_122_more_windows_porting.mdwn @@ -0,0 +1,4 @@ +More windows porting. Made the build completely -Wall safe on Windows. +Fixed some DOS path separator bugs that were preventing WebDav from +working. Have now tested both box.com and Amazon S3 to be completely +working in the webapp on Windows.