mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
git-annex/Types/GitConfig.hs

353 lines
13 KiB
Haskell
Raw Normal View History

type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
{- git-annex configuration
-
pid locking configuration and abstraction layer for git-annex (not actually used anywhere yet)
2015-11-12 21:47:31 +00:00
- Copyright 2012-2015 Joey Hess <id@joeyh.name>
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
-
update licenses from GPL to AGPL This does not change the overall license of the git-annex program, which was already AGPL due to a number of sources files being AGPL already. Legally speaking, I'm adding a new license under which these files are now available; I already released their current contents under the GPL license. Now they're dual licensed GPL and AGPL. However, I intend for all my future changes to these files to only be released under the AGPL license, and I won't be tracking the dual licensing status, so I'm simply changing the license statement to say it's AGPL. (In some cases, others wrote parts of the code of a file and released it under the GPL; but in all cases I have contributed a significant portion of the code in each file and it's that code that is getting the AGPL license; the GPL license of other contributors allows combining with AGPL code.)
2019-03-13 19:48:14 +00:00
- Licensed under the GNU AGPL version 3 or higher.
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
-}
module Types.GitConfig (
annex.autocommit can be configured via git-annex config ... to control the default behavior in all clones of a repository. This includes a new Configurable data type, so the GitConfig type indicates which values can be configured this way. The implementation should be quite efficient; the config log is only read once, and only when a Configurable value has not already been set by git-config. Indeed, it would be nice in the future to extend this, so that git-config is itself only read on demand. Some commands may not need to look at the git configuration at all. This commit was sponsored by Trenton Cronholm on Patreon.
2017-02-03 17:40:14 +00:00
Configurable(..),
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
GitConfig(..),
extractGitConfig,
annex.autocommit can be configured via git-annex config ... to control the default behavior in all clones of a repository. This includes a new Configurable data type, so the GitConfig type indicates which values can be configured this way. The implementation should be quite efficient; the config log is only read once, and only when a Configurable value has not already been set by git-config. Indeed, it would be nice in the future to extend this, so that git-config is itself only read on demand. Some commands may not need to look at the git configuration at all. This commit was sponsored by Trenton Cronholm on Patreon.
2017-02-03 17:40:14 +00:00
mergeGitConfig,
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
RemoteGitConfig(..),
extractRemoteGitConfig,
add annex-ignore-command and annex-sync-command configs Added remote configuration settings annex-ignore-command and annex-sync-command, which are dynamic equivilants of the annex-ignore and annex-sync configurations. For this I needed a new DynamicConfig infrastructure. Its implementation should be as fast as before when there is no dynamic config, and it caches so shell commands are only run once. Note that annex-ignore-command exits nonzero when the remote should be ignored. While that may seem backwards, it allows using the same command for it as for annex-sync-command when you want to disable both. This commit was sponsored by Trenton Cronholm on Patreon.
2017-08-17 16:26:14 +00:00
dummyRemoteGitConfig,
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
) where
import Common
import qualified Git
import qualified Git.Config
The ssh-options git config is now used by gcrypt, rsync, and ddar special remotes that use ssh as a transport.
2015-02-12 19:44:10 +00:00
import qualified Git.Construct
removal of the rest of remoteGitConfig In keyUrls, the GitConfig is used only by annexLocations to support configured Differences. Since such configurations affect all clones of a repository, the local repo's GitConfig must have the same information as the remote's GitConfig would have. So, used getGitConfig to get the local GitConfig, which is cached and so available cheaply. That actually fixed a bug noone had ever noticed: keyUrls is used for remotes accessed over http. The full git config of such a remote is normally not available, so the remoteGitConfig that keyUrls used would not have the necessary information in it. In copyFromRemoteCheap', it uses gitAnnexLocation, which does need the GitConfig of the remote repo itself in order to check if it's crippled, supports symlinks, etc. So, made the State include that GitConfig, cached. The use of gitAnnexLocation is within a (not $ Git.repoIsUrl repo) guard, so it's local, and so its git config will always be read and available. (Note that gitAnnexLocation in turn calls annexLocations, so the Differences config it uses in this case comes from the remote repo's GitConfig and not from the local repo's GitConfig. As explained above this is ok since they must have the same value.) Not very happy with this mess of different GitConfigs not type-safe and some read only sometimes etc. Very hairy. Think I got it this change right. Test suite passes.. This commit was sponsored by Ethan Aubin.
2018-06-05 18:23:34 +00:00
import Git.Types
post-recive hook to make updateInstead work in direct mode and adjusted branches * Added post-recieve hook, which makes updateInstead work with direct mode and adjusted branches. * init: Set up the post-receive hook. This commit was sponsored by Fernando Jimenez on Patreon.
2017-02-17 18:04:43 +00:00
import Git.ConfigTypes
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
import Utility.DataUnits
split cost out into its own module Added a function to insert a new cost into a list, which could be used to asjust costs after a drag and drop.
2013-03-13 20:16:01 +00:00
import Config.Cost
optimise getUUID This avoids a Map lookup each time it's called, instead the GitConfig field lazily looks it up once and then caches.
2016-01-20 20:55:06 +00:00
import Types.UUID
annex.autoupgrade setting
2013-11-22 20:04:20 +00:00
import Types.Distribution
add GETAVAILABILITY to external special remote protocol And some reworking of types, and added an annex-availability git config setting.
2014-01-13 18:41:10 +00:00
import Types.Availability
annex.jobs Added annex.jobs setting, which is like using the -J option. Of course, -J overrides annex.jobs. This commit was sponsored by Trenton Cronholm on Patreon.
2018-10-04 16:47:27 +00:00
import Types.Concurrency
numcopies cleanup, part 2 This includes several bug fixes.
2014-01-21 21:08:49 +00:00
import Types.NumCopies
Repository tuning parameters can now be passed when initializing a repository for the first time. * init: Repository tuning parameters can now be passed when initializing a repository for the first time. For details, see http://git-annex.branchable.com/tuning/ * merge: Refuse to merge changes from a git-annex branch of a repo that has been tuned in incompatable ways.
2015-01-27 21:38:06 +00:00
import Types.Difference
add annex.used-refspec
2015-05-14 19:44:08 +00:00
import Types.RefSpec
v7 Install new git hooks in this version. This does beg the question of what to do if git later gets eg a post-smudge hook, that could run git-annex smudge --update. I think the thing to do in that case would be to make git-annex smudge --update install the new hooks. That way, as the user uses git-annex, the hook would be created pretty quickly and without needing any extra syscalls except for when git-annex smudge --update is called. I considered doing something like that for installation of the post-checkout and post-merge hooks, which would have avoided the need for v7. But the only place it was cheap to do it would be in git-annex smudge which could cheaply notice that smudge.log didn't exist yet and so know the hooks needed to be installed. But since smudge used to populate pointer files, it would be quite surprising if a single git checkout/merge failed to update the work tree, and so that idea didn't work out. The other reason for v7 is psychological -- users don't need to worry about whether they might be running an old version of git-annex that doesn't support their v7 repository very well. And bug reports about "v6" have gotten a bit of a bad association in my head since they often hit one of the known limitations and didn't realize it was experimental. newtyped RepoVersion Int to avoid needing 2 comparisons in versionSupportsUnlockedPointers etc. Also it's just nicer. This commit was sponsored by John Pellman on Patreon.
2018-10-25 21:23:53 +00:00
import Types.RepoVersion
add annex-ignore-command and annex-sync-command configs Added remote configuration settings annex-ignore-command and annex-sync-command, which are dynamic equivilants of the annex-ignore and annex-sync configurations. For this I needed a new DynamicConfig infrastructure. Its implementation should be as fast as before when there is no dynamic config, and it caches so shell commands are only run once. Note that annex-ignore-command exits nonzero when the remote should be ignored. While that may seem backwards, it allows using the same command for it as for annex-sync-command when you want to disable both. This commit was sponsored by Trenton Cronholm on Patreon.
2017-08-17 16:26:14 +00:00
import Config.DynamicConfig
assistant unused file handling Make sanity checker run git annex unused daily, and queue up transfers of unused files to any remotes that will have them. The transfer retrying code works for us here, so eg when a backup disk remote is plugged in, any transfers to it are done. Once the unused files reach a remote, they'll be removed locally as unwanted. If the setup does not cause unused files to go to a remote, they'll pile up, and the sanity checker detects this using some heuristics that are pretty good -- 1000 unused files, or 10% of disk used by unused files, or more disk wasted by unused files than is left free. Once it detects this, it pops up an alert in the webapp, with a button to take action. TODO: Webapp UI to configure this, and also the ability to launch an immediate cleanup of all unused files. This commit was sponsored by Simon Michael.
2014-01-23 02:48:56 +00:00
import Utility.HumanTime
support gpg.program When gpg.program is configured, it's used to get the command to run for gpg. Useful on systems that have only a gpg2 command or want to use it instead of the gpg command.
2015-09-09 22:06:49 +00:00
import Utility.Gpg (GpgCmd, mkGpgCmd)
pid locking configuration and abstraction layer for git-annex (not actually used anywhere yet)
2015-11-12 21:47:31 +00:00
import Utility.ThreadScheduler (Seconds(..))
limit url downloads to whitelisted schemes Security fix! Allowing any schemes, particularly file: and possibly others like scp: allowed file exfiltration by anyone who had write access to the git repository, since they could add an annexed file using such an url, or using an url that redirected to such an url, and wait for the victim to get it into their repository and send them a copy. * Added annex.security.allowed-url-schemes setting, which defaults to only allowing http and https URLs. Note especially that file:/ is no longer enabled by default. * Removed annex.web-download-command, since its interface does not allow supporting annex.security.allowed-url-schemes across redirects. If you used this setting, you may want to instead use annex.web-options to pass options to curl. With annex.web-download-command removed, nearly all url accesses in git-annex are made via Utility.Url via http-client or curl. http-client only supports http and https, so no problem there. (Disabling one and not the other is not implemented.) Used curl --proto to limit the allowed url schemes. Note that this will cause git annex fsck --from web to mark files using a disallowed url scheme as not being present in the web. That seems acceptable; fsck --from web also does that when a web server is not available. youtube-dl already disabled file: itself (probably for similar reasons). The scheme check was also added to youtube-dl urls for completeness, although that check won't catch any redirects it might follow. But youtube-dl goes off and does its own thing with other protocols anyway, so that's fine. Special remotes that support other domain-specific url schemes are not affected by this change. In the bittorrent remote, aria2c can still download magnet: links. The download of the .torrent file is otherwise now limited by annex.security.allowed-url-schemes. This does not address any external special remotes that might download an url themselves. Current thinking is all external special remotes will need to be audited for this problem, although many of them will use http libraries that only support http and not curl's menagarie. The related problem of accessing private localhost and LAN urls is not addressed by this commit. This commit was sponsored by Brett Eisenberg on Patreon.
2018-06-15 20:52:24 +00:00
import Utility.Url (Scheme, mkScheme)
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
add annex-ignore-command and annex-sync-command configs Added remote configuration settings annex-ignore-command and annex-sync-command, which are dynamic equivilants of the annex-ignore and annex-sync configurations. For this I needed a new DynamicConfig infrastructure. Its implementation should be as fast as before when there is no dynamic config, and it caches so shell commands are only run once. Note that annex-ignore-command exits nonzero when the remote should be ignored. While that may seem backwards, it allows using the same command for it as for annex-sync-command when you want to disable both. This commit was sponsored by Trenton Cronholm on Patreon.
2017-08-17 16:26:14 +00:00
import Control.Concurrent.STM
limit url downloads to whitelisted schemes Security fix! Allowing any schemes, particularly file: and possibly others like scp: allowed file exfiltration by anyone who had write access to the git repository, since they could add an annexed file using such an url, or using an url that redirected to such an url, and wait for the victim to get it into their repository and send them a copy. * Added annex.security.allowed-url-schemes setting, which defaults to only allowing http and https URLs. Note especially that file:/ is no longer enabled by default. * Removed annex.web-download-command, since its interface does not allow supporting annex.security.allowed-url-schemes across redirects. If you used this setting, you may want to instead use annex.web-options to pass options to curl. With annex.web-download-command removed, nearly all url accesses in git-annex are made via Utility.Url via http-client or curl. http-client only supports http and https, so no problem there. (Disabling one and not the other is not implemented.) Used curl --proto to limit the allowed url schemes. Note that this will cause git annex fsck --from web to mark files using a disallowed url scheme as not being present in the web. That seems acceptable; fsck --from web also does that when a web server is not available. youtube-dl already disabled file: itself (probably for similar reasons). The scheme check was also added to youtube-dl urls for completeness, although that check won't catch any redirects it might follow. But youtube-dl goes off and does its own thing with other protocols anyway, so that's fine. Special remotes that support other domain-specific url schemes are not affected by this change. In the bittorrent remote, aria2c can still download magnet: links. The download of the .torrent file is otherwise now limited by annex.security.allowed-url-schemes. This does not address any external special remotes that might download an url themselves. Current thinking is all external special remotes will need to be audited for this problem, although many of them will use http libraries that only support http and not curl's menagarie. The related problem of accessing private localhost and LAN urls is not addressed by this commit. This commit was sponsored by Brett Eisenberg on Patreon.
2018-06-15 20:52:24 +00:00
import qualified Data.Set as S
add annex-ignore-command and annex-sync-command configs Added remote configuration settings annex-ignore-command and annex-sync-command, which are dynamic equivilants of the annex-ignore and annex-sync configurations. For this I needed a new DynamicConfig infrastructure. Its implementation should be as fast as before when there is no dynamic config, and it caches so shell commands are only run once. Note that annex-ignore-command exits nonzero when the remote should be ignored. While that may seem backwards, it allows using the same command for it as for annex-sync-command when you want to disable both. This commit was sponsored by Trenton Cronholm on Patreon.
2017-08-17 16:26:14 +00:00
-- | A configurable value, that may not be fully determined yet because
-- the global git config has not yet been loaded.
annex.autocommit can be configured via git-annex config ... to control the default behavior in all clones of a repository. This includes a new Configurable data type, so the GitConfig type indicates which values can be configured this way. The implementation should be quite efficient; the config log is only read once, and only when a Configurable value has not already been set by git-config. Indeed, it would be nice in the future to extend this, so that git-config is itself only read on demand. Some commands may not need to look at the git configuration at all. This commit was sponsored by Trenton Cronholm on Patreon.
2017-02-03 17:40:14 +00:00
data Configurable a
= HasConfig a
-- ^ Value is fully determined.
| DefaultConfig a
-- ^ A default value is known, but not all config sources
-- have been read yet.
New annex.synccontent config setting .. which can be set to true to make git annex sync default to --content. This may become the default at some point in the future. As well as being configuable by git config, it can be configured by git-annex config to control the default behavior in all clones of a repository. Had to add a separate --no-content switch to we can tell if it's been explicitly set, and should override annex.synccontent. If --content was the default, this complication would not be necessary. This commit was sponsored by Jake Vosloo on Patreon.
2017-02-03 18:31:17 +00:00
deriving (Show)
annex.autocommit can be configured via git-annex config ... to control the default behavior in all clones of a repository. This includes a new Configurable data type, so the GitConfig type indicates which values can be configured this way. The implementation should be quite efficient; the config log is only read once, and only when a Configurable value has not already been set by git-config. Indeed, it would be nice in the future to extend this, so that git-config is itself only read on demand. Some commands may not need to look at the git configuration at all. This commit was sponsored by Trenton Cronholm on Patreon.
2017-02-03 17:40:14 +00:00
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
{- Main git-annex settings. Each setting corresponds to a git-config key
- such as annex.foo -}
data GitConfig = GitConfig
v7 Install new git hooks in this version. This does beg the question of what to do if git later gets eg a post-smudge hook, that could run git-annex smudge --update. I think the thing to do in that case would be to make git-annex smudge --update install the new hooks. That way, as the user uses git-annex, the hook would be created pretty quickly and without needing any extra syscalls except for when git-annex smudge --update is called. I considered doing something like that for installation of the post-checkout and post-merge hooks, which would have avoided the need for v7. But the only place it was cheap to do it would be in git-annex smudge which could cheaply notice that smudge.log didn't exist yet and so know the hooks needed to be installed. But since smudge used to populate pointer files, it would be quite surprising if a single git checkout/merge failed to update the work tree, and so that idea didn't work out. The other reason for v7 is psychological -- users don't need to worry about whether they might be running an old version of git-annex that doesn't support their v7 repository very well. And bug reports about "v6" have gotten a bit of a bad association in my head since they often hit one of the known limitations and didn't realize it was experimental. newtyped RepoVersion Int to avoid needing 2 comparisons in versionSupportsUnlockedPointers etc. Also it's just nicer. This commit was sponsored by John Pellman on Patreon.
2018-10-25 21:23:53 +00:00
{ annexVersion :: Maybe RepoVersion
optimise getUUID This avoids a Map lookup each time it's called, instead the GitConfig field lazily looks it up once and then caches.
2016-01-20 20:55:06 +00:00
, annexUUID :: UUID
numcopies cleanup, part 2 This includes several bug fixes.
2014-01-21 21:08:49 +00:00
, annexNumCopies :: Maybe NumCopies
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
, annexDiskReserve :: Integer
, annexDirect :: Bool
annex.backend is the new name for what was annex.backends It takes a single key-value backend, rather than the unncessary and confusing list. The old option still works if set. Simplified some old old code too. This commit was sponsored by Thomas Hochstein on Patreon.
2017-05-09 19:04:07 +00:00
, annexBackend :: Maybe String
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
, annexQueueSize :: Maybe Int
, annexBloomCapacity :: Maybe Int
, annexBloomAccuracy :: Maybe Int
, annexSshCaching :: Maybe Bool
, annexAlwaysCommit :: Bool
added annex.commitmessage Added annex.commitmessage config that can specify a commit message for the git-annex branch instead of the usual "update". This commit was supported by the NSF-funded DataLad project.
2018-08-02 18:06:06 +00:00
, annexCommitMessage :: Maybe String
annex.merge-annex-branches Added annex.merge-annex-branches config setting which can be used to disable automatic merge of git-annex branches. I wonder if git-annex merge/sync/assistant should disable this setting? Not sure yet, so have not done so. May be that users will not set it in git config, but pass it via -c to commands that need it. Checking the config setting adds a very small overhead, but it's only checked once per command so should be insignificant. This commit was supported by the NSF-funded DataLad project.
2018-02-22 18:25:32 +00:00
, annexMergeAnnexBranches :: Bool
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
, annexDelayAdd :: Maybe Int
, annexHttpHeaders :: [String]
, annexHttpHeadersCommand :: Maybe String
annex.autocommit can be configured via git-annex config ... to control the default behavior in all clones of a repository. This includes a new Configurable data type, so the GitConfig type indicates which values can be configured this way. The implementation should be quite efficient; the config log is only read once, and only when a Configurable value has not already been set by git-config. Indeed, it would be nice in the future to extend this, so that git-config is itself only read on demand. Some commands may not need to look at the git configuration at all. This commit was sponsored by Trenton Cronholm on Patreon.
2017-02-03 17:40:14 +00:00
, annexAutoCommit :: Configurable Bool
configuration to disable automatic merge conflict resolution * Added annex.resolvemerge configuration, which can be set to false to disable the usual automatic merge conflict resolution done by git-annex sync and the assistant. * sync: Added --no-resolvemerge option. Note that disabling merge conflict resolution is probably not a good idea in a direct mode repo or adjusted branch. Since updates to both are done outside the usual work tree, if it fails the tree is not left in a conflicted state, and it would be hard to manually resolve the conflict. Still, made annex.resolvemerge be supported in those cases for consistency. This commit was sponsored by Riku Voipio.
2017-06-01 16:46:36 +00:00
, annexResolveMerge :: Configurable Bool
New annex.synccontent config setting .. which can be set to true to make git annex sync default to --content. This may become the default at some point in the future. As well as being configuable by git config, it can be configured by git-annex config to control the default behavior in all clones of a repository. Had to add a separate --no-content switch to we can tell if it's been explicitly set, and should override annex.synccontent. If --content was the default, this complication would not be necessary. This commit was sponsored by Jake Vosloo on Patreon.
2017-02-03 18:31:17 +00:00
, annexSyncContent :: Configurable Bool
annex.debug can now be set to enable debug logging by default. The webapp's debugging check box does this.
2013-06-18 00:41:17 +00:00
, annexDebug :: Bool
add another setting to GitConfig
2013-01-27 13:33:19 +00:00
, annexWebOptions :: [String]
youtube-dl working Including resuming and cleanup of incomplete downloads. Still todo: --fast, --relaxed, importfeed, disk reserve checking, quvi code cleanup. This commit was sponsored by Anthony DeRobertis on Patreon.
2017-11-29 19:49:05 +00:00
, annexYoutubeDlOptions :: [String]
Added bittorrent special remote addurl behavior change: When downloading an url ending in .torrent, it will download files from bittorrent, instead of the old behavior of adding the torrent file to the repository. Added Recommends on aria2 and bittornado | bittorrent. This commit was sponsored by Asbjørn Sloth Tønnesen.
2014-12-17 03:22:46 +00:00
, annexAriaTorrentOptions :: [String]
crippled filesystem support, probing and initial support git annex init probes for crippled filesystems, and sets direct mode, as well as `annex.crippledfilesystem`. Avoid manipulating permissions of files on crippled filesystems. That would likely cause an exception to be thrown. Very basic support in Command.Add for cripped filesystems; avoids the lock down entirely since doing it needs both permissions and hard links. Will make this better soon.
2013-02-14 18:10:36 +00:00
, annexCrippledFileSystem :: Bool
New annex.largefiles setting, which configures which files `git annex add` and the assistant add to the annex. I would have sort of liked to put this in .gitattributes, but it seems it does not support multi-word attribute values. Also, making this a single config setting makes it easy to only parse the expression once. A natural next step would be to make the assistant `git add` files that are not annex.largefiles. OTOH, I don't think `git annex add` should `git add` such files, because git-annex command line tools are not in the business of wrapping git command line tools.
2013-03-29 20:17:13 +00:00
, annexLargeFiles :: Maybe String
annex.addsmallfiles: New option controlling what is done when adding files not matching annex.largefiles.
2016-01-28 18:04:32 +00:00
, annexAddSmallFiles :: Bool
webapp: remind user when using repositories that lack consistency checks When starting up the assistant, it'll remind about the current repository, if it doesn't have checks. And when a removable drive is plugged in, it will remind if a repository on it lacks checks. Since that might be annoying, the reminders can be turned off. This commit was sponsored by Nedialko Andreev.
2013-10-29 20:48:06 +00:00
, annexFsckNudge :: Bool
annex.autoupgrade setting
2013-11-22 20:04:20 +00:00
, annexAutoUpgrade :: AutoUpgrade
allow annex.expireunused to be set to false, as well as to a duration
2014-01-23 03:10:51 +00:00
, annexExpireUnused :: Maybe (Maybe Duration)
added annex.secure-erase-command config option.
2014-01-24 16:58:52 +00:00
, annexSecureEraseCommand :: Maybe String
annex.genmetadata can be set to make git-annex automatically set metadata (year and month) when adding files
2014-02-23 04:08:29 +00:00
, annexGenMetaData :: Bool
annex.listen can be configured, instead of using --listen
2014-03-01 04:31:17 +00:00
, annexListen :: Maybe String
annex.startupscan can be set to false to disable the assistant's startup scan.
2014-03-05 21:44:14 +00:00
, annexStartupScan :: Bool
New annex.hardlink setting. Closes: #758593 * New annex.hardlink setting. Closes: #758593 * init: Automatically detect when a repository was cloned with --shared, and set annex.hardlink=true, as well as marking the repository as untrusted. Had to reorganize Logs.Trust a bit to avoid a cycle between it and Annex.Init.
2014-09-05 17:44:09 +00:00
, annexHardLink :: Bool
annex.thin Decided it's too scary to make v6 unlocked files have 1 copy by default, but that should be available to those who need it. This is consistent with git-annex not dropping unused content without --force, etc. * Added annex.thin setting, which makes unlocked files in v6 repositories be hard linked to their content, instead of a copy. This saves disk space but means any modification of an unlocked file will lose the local (and possibly only) copy of the old version. * Enable annex.thin by default on upgrade from direct mode to v6, since direct mode made the same tradeoff. * fix: Adjusts unlocked files as configured by annex.thin.
2015-12-27 19:59:59 +00:00
, annexThin :: Bool
support gpg.program When gpg.program is configured, it's used to get the command to run for gpg. Useful on systems that have only a gpg2 command or want to use it instead of the gpg command.
2015-09-09 22:06:49 +00:00
, annexDifferences :: Differences
, annexUsedRefSpec :: Maybe RefSpec
Do verification of checksums of annex objects downloaded from remotes. * When annex objects are received into git repositories, their checksums are verified then too. * To get the old, faster, behavior of not verifying checksums, set annex.verify=false, or remote.<name>.annex-verify=false. * setkey, rekey: These commands also now verify that the provided file matches the key, unless annex.verify=false. * reinject: Already verified content; this can now be disabled by setting annex.verify=false. recvkey and reinject already did verification, so removed now duplicate code from them. fsck still does its own verification, which is ok since it does not use getViaTmp, so verification doesn't happen twice when using fsck --from.
2015-10-01 19:54:37 +00:00
, annexVerify :: Bool
pid locking configuration and abstraction layer for git-annex (not actually used anywhere yet)
2015-11-12 21:47:31 +00:00
, annexPidLock :: Bool
, annexPidLockTimeout :: Seconds
annex.addunlocked * add, addurl, import, importfeed: When in a v6 repository on a crippled filesystem, add files unlocked. * annex.addunlocked: New configuration setting, makes files always be added unlocked. (v6 only)
2016-02-16 18:43:43 +00:00
, annexAddUnlocked :: Bool
annex.securehashesonly Cryptographically secure hashes can be forced to be used in a repository, by setting annex.securehashesonly. This does not prevent the git repository from containing files with insecure hashes, but it does prevent the content of such files from being pulled into .git/annex/objects from another repository. We want to make sure that at no point does git-annex accept content into .git/annex/objects that is hashed with an insecure key. Here's how it was done: * .git/annex/objects/xx/yy/KEY/ is kept frozen, so nothing can be written to it normally * So every place that writes content must call, thawContent or modifyContent. We can audit for these, and be sure we've considered all cases. * The main functions are moveAnnex, and linkToAnnex; these were made to check annex.securehashesonly, and are the main security boundary for annex.securehashesonly. * Most other calls to modifyContent deal with other files in the KEY directory (inode cache etc). The other ones that mess with the content are: - Annex.Direct.toDirectGen, in which content already in the annex directory is moved to the direct mode file, so not relevant. - fix and lock, which don't add new content - Command.ReKey.linkKey, which manually unlocks it to make a copy. * All other calls to thawContent appear safe. Made moveAnnex return a Bool, so checked all callsites and made them deal with a failure in appropriate ways. linkToAnnex simply returns LinkAnnexFailed; all callsites already deal with it failing in appropriate ways. This commit was sponsored by Riku Voipio.
2017-02-27 17:01:32 +00:00
, annexSecureHashesOnly :: Bool
add retry configuration (not used yet)
2018-03-24 14:37:25 +00:00
, annexRetry :: Maybe Integer
, annexRetryDelay :: Maybe Seconds
limit url downloads to whitelisted schemes Security fix! Allowing any schemes, particularly file: and possibly others like scp: allowed file exfiltration by anyone who had write access to the git repository, since they could add an annexed file using such an url, or using an url that redirected to such an url, and wait for the victim to get it into their repository and send them a copy. * Added annex.security.allowed-url-schemes setting, which defaults to only allowing http and https URLs. Note especially that file:/ is no longer enabled by default. * Removed annex.web-download-command, since its interface does not allow supporting annex.security.allowed-url-schemes across redirects. If you used this setting, you may want to instead use annex.web-options to pass options to curl. With annex.web-download-command removed, nearly all url accesses in git-annex are made via Utility.Url via http-client or curl. http-client only supports http and https, so no problem there. (Disabling one and not the other is not implemented.) Used curl --proto to limit the allowed url schemes. Note that this will cause git annex fsck --from web to mark files using a disallowed url scheme as not being present in the web. That seems acceptable; fsck --from web also does that when a web server is not available. youtube-dl already disabled file: itself (probably for similar reasons). The scheme check was also added to youtube-dl urls for completeness, although that check won't catch any redirects it might follow. But youtube-dl goes off and does its own thing with other protocols anyway, so that's fine. Special remotes that support other domain-specific url schemes are not affected by this change. In the bittorrent remote, aria2c can still download magnet: links. The download of the .torrent file is otherwise now limited by annex.security.allowed-url-schemes. This does not address any external special remotes that might download an url themselves. Current thinking is all external special remotes will need to be audited for this problem, although many of them will use http libraries that only support http and not curl's menagarie. The related problem of accessing private localhost and LAN urls is not addressed by this commit. This commit was sponsored by Brett Eisenberg on Patreon.
2018-06-15 20:52:24 +00:00
, annexAllowedUrlSchemes :: S.Set Scheme
rename annex.security.allowed-http-addresses Renamed annex.security.allowed-http-addresses to annex.security.allowed-ip-addresses because it is not really specific to the http protocol, also limiting eg, git-annex's use of ftp and via youtube-dl, several other protocols. The old name for the config will still work. If both old and new name are set, the new name will win.
2019-05-30 16:43:40 +00:00
, annexAllowedIPAddresses :: String
enforce retrievalSecurityPolicy Leveraged the existing verification code by making it also check the retrievalSecurityPolicy. Also, prevented getViaTmp from running the download action at all when the retrievalSecurityPolicy is going to prevent verifying and so storing it. Added annex.security.allow-unverified-downloads. A per-remote version would be nice to have too, but would need more plumbing, so KISS. (Bill the Cat reference not too over the top I hope. The point is to make this something the user reads the documentation for before using.) A few calls to verifyKeyContent and getViaTmp, that don't involve downloads from remotes, have RetrievalAllKeysSecure hard-coded. It was also hard-coded for P2P.Annex and Command.RecvKey, to match the values of the corresponding remotes. A few things use retrieveKeyFile/retrieveKeyFileCheap without going through getViaTmp. * Command.Fsck when downloading content from a remote to verify it. That content does not get into the annex, so this is ok. * Command.AddUrl when using a remote to download an url; this is new content being added, so this is ok. This commit was sponsored by Fernando Jimenez on Patreon.
2018-06-21 17:34:11 +00:00
, annexAllowUnverifiedDownloads :: Bool
annex.maxextensionlength Added annex.maxextensionlength for use cases where extensions longer than 4 characters are needed. This commit was sponsored by Henrik Riomar on Patreon.
2018-09-24 16:07:46 +00:00
, annexMaxExtensionLength :: Maybe Int
annex.jobs Added annex.jobs setting, which is like using the -J option. Of course, -J overrides annex.jobs. This commit was sponsored by Trenton Cronholm on Patreon.
2018-10-04 16:47:27 +00:00
, annexJobs :: Concurrency
annex.cachecreds: New config to allow disabling of credentials caching for special remotes. Note that it does not prevent storing p2p access tokens or multicast encryption keys, since those are not cached; the previous commit established the distinction. How well this works depends on how often getRemoteCredPair is called and how expensive it is. In some cases setting this will result in an annoying number of gpg password prompts and/or slowdowns due to reading creds from the git-annex branch and decrypting, which could be improved by calling getRemoteCredPair less often. This commit was sponsored by Ilya Shlyakhter on Patreon.
2018-12-04 18:16:56 +00:00
, annexCacheCreds :: Bool
Added annex.autoupgraderepository configuration Can be set to false to prevent any automatic repository upgrades. Also, removed direct mode specific upgrade code in Annex.Init, and made needsUpgrade always include the name/path of the repo, so if there's a problem it's clear what repo has the problem. And, made needsUpgrade catch any exceptions that might occur during the upgrade, so it can display a more useful error message than just the exception.
2019-09-01 17:29:55 +00:00
, annexAutoUpgradeRepository :: Bool
start to support core.symlinks=false Utility functions to handle no symlink mode, and converted Annex.Content to use them; still many other places to convert.
2013-02-15 20:02:35 +00:00
, coreSymlinks :: Bool
better memoize core.sharedrepository handling It was memoized, but that was not used consistently. Move it to Types.GitConfig so it will auto-memoize.
2015-05-19 19:04:24 +00:00
, coreSharedRepository :: SharedRepository
post-recive hook to make updateInstead work in direct mode and adjusted branches * Added post-recieve hook, which makes updateInstead work with direct mode and adjusted branches. * init: Set up the post-receive hook. This commit was sponsored by Fernando Jimenez on Patreon.
2017-02-17 18:04:43 +00:00
, receiveDenyCurrentBranch :: DenyCurrentBranch
git-annex-shell: Added support for operating inside gcrypt repositories. * Note that the layout of gcrypt repositories has changed, and if you created one you must manually upgrade it. See http://git-annex.branchable.com/upgrades/gcrypt/
2013-09-24 21:25:47 +00:00
, gcryptId :: Maybe String
support gpg.program When gpg.program is configured, it's used to get the command to run for gpg. Useful on systems that have only a gpg2 command or want to use it instead of the gpg command.
2015-09-09 22:06:49 +00:00
, gpgCmd :: GpgCmd
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
}
extractGitConfig :: Git.Repo -> GitConfig
extractGitConfig r = GitConfig
v7 Install new git hooks in this version. This does beg the question of what to do if git later gets eg a post-smudge hook, that could run git-annex smudge --update. I think the thing to do in that case would be to make git-annex smudge --update install the new hooks. That way, as the user uses git-annex, the hook would be created pretty quickly and without needing any extra syscalls except for when git-annex smudge --update is called. I considered doing something like that for installation of the post-checkout and post-merge hooks, which would have avoided the need for v7. But the only place it was cheap to do it would be in git-annex smudge which could cheaply notice that smudge.log didn't exist yet and so know the hooks needed to be installed. But since smudge used to populate pointer files, it would be quite surprising if a single git checkout/merge failed to update the work tree, and so that idea didn't work out. The other reason for v7 is psychological -- users don't need to worry about whether they might be running an old version of git-annex that doesn't support their v7 repository very well. And bug reports about "v6" have gotten a bit of a bad association in my head since they often hit one of the known limitations and didn't realize it was experimental. newtyped RepoVersion Int to avoid needing 2 comparisons in versionSupportsUnlockedPointers etc. Also it's just nicer. This commit was sponsored by John Pellman on Patreon.
2018-10-25 21:23:53 +00:00
{ annexVersion = RepoVersion <$> getmayberead (annex "version")
optimise getUUID This avoids a Map lookup each time it's called, instead the GitConfig field lazily looks it up once and then caches.
2016-01-20 20:55:06 +00:00
, annexUUID = maybe NoUUID toUUID $ getmaybe (annex "uuid")
numcopies cleanup, part 2 This includes several bug fixes.
2014-01-21 21:08:49 +00:00
, annexNumCopies = NumCopies <$> getmayberead (annex "numcopies")
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
, annexDiskReserve = fromMaybe onemegabyte $
start to support core.symlinks=false Utility functions to handle no symlink mode, and converted Annex.Content to use them; still many other places to convert.
2013-02-15 20:02:35 +00:00
readSize dataUnits =<< getmaybe (annex "diskreserve")
, annexDirect = getbool (annex "direct") False
annex.backend is the new name for what was annex.backends It takes a single key-value backend, rather than the unncessary and confusing list. The old option still works if set. Simplified some old old code too. This commit was sponsored by Thomas Hochstein on Patreon.
2017-05-09 19:04:07 +00:00
, annexBackend = maybe
-- annex.backends is the old name of the option, still used
-- when annex.backend is not set.
(headMaybe $ getwords (annex "backends"))
Just
(getmaybe (annex "backend"))
start to support core.symlinks=false Utility functions to handle no symlink mode, and converted Annex.Content to use them; still many other places to convert.
2013-02-15 20:02:35 +00:00
, annexQueueSize = getmayberead (annex "queuesize")
, annexBloomCapacity = getmayberead (annex "bloomcapacity")
, annexBloomAccuracy = getmayberead (annex "bloomaccuracy")
, annexSshCaching = getmaybebool (annex "sshcaching")
, annexAlwaysCommit = getbool (annex "alwayscommit") True
added annex.commitmessage Added annex.commitmessage config that can specify a commit message for the git-annex branch instead of the usual "update". This commit was supported by the NSF-funded DataLad project.
2018-08-02 18:06:06 +00:00
, annexCommitMessage = getmaybe (annex "commitmessage")
annex.merge-annex-branches Added annex.merge-annex-branches config setting which can be used to disable automatic merge of git-annex branches. I wonder if git-annex merge/sync/assistant should disable this setting? Not sure yet, so have not done so. May be that users will not set it in git config, but pass it via -c to commands that need it. Checking the config setting adds a very small overhead, but it's only checked once per command so should be insignificant. This commit was supported by the NSF-funded DataLad project.
2018-02-22 18:25:32 +00:00
, annexMergeAnnexBranches = getbool (annex "merge-annex-branches") True
start to support core.symlinks=false Utility functions to handle no symlink mode, and converted Annex.Content to use them; still many other places to convert.
2013-02-15 20:02:35 +00:00
, annexDelayAdd = getmayberead (annex "delayadd")
, annexHttpHeaders = getlist (annex "http-headers")
, annexHttpHeadersCommand = getmaybe (annex "http-headers-command")
annex.autocommit can be configured via git-annex config ... to control the default behavior in all clones of a repository. This includes a new Configurable data type, so the GitConfig type indicates which values can be configured this way. The implementation should be quite efficient; the config log is only read once, and only when a Configurable value has not already been set by git-config. Indeed, it would be nice in the future to extend this, so that git-config is itself only read on demand. Some commands may not need to look at the git configuration at all. This commit was sponsored by Trenton Cronholm on Patreon.
2017-02-03 17:40:14 +00:00
, annexAutoCommit = configurable True $
getmaybebool (annex "autocommit")
configuration to disable automatic merge conflict resolution * Added annex.resolvemerge configuration, which can be set to false to disable the usual automatic merge conflict resolution done by git-annex sync and the assistant. * sync: Added --no-resolvemerge option. Note that disabling merge conflict resolution is probably not a good idea in a direct mode repo or adjusted branch. Since updates to both are done outside the usual work tree, if it fails the tree is not left in a conflicted state, and it would be hard to manually resolve the conflict. Still, made annex.resolvemerge be supported in those cases for consistency. This commit was sponsored by Riku Voipio.
2017-06-01 16:46:36 +00:00
, annexResolveMerge = configurable True $
getmaybebool (annex "resolvemerge")
New annex.synccontent config setting .. which can be set to true to make git annex sync default to --content. This may become the default at some point in the future. As well as being configuable by git config, it can be configured by git-annex config to control the default behavior in all clones of a repository. Had to add a separate --no-content switch to we can tell if it's been explicitly set, and should override annex.synccontent. If --content was the default, this complication would not be necessary. This commit was sponsored by Jake Vosloo on Patreon.
2017-02-03 18:31:17 +00:00
, annexSyncContent = configurable False $
getmaybebool (annex "synccontent")
annex.debug can now be set to enable debug logging by default. The webapp's debugging check box does this.
2013-06-18 00:41:17 +00:00
, annexDebug = getbool (annex "debug") False
start to support core.symlinks=false Utility functions to handle no symlink mode, and converted Annex.Content to use them; still many other places to convert.
2013-02-15 20:02:35 +00:00
, annexWebOptions = getwords (annex "web-options")
youtube-dl working Including resuming and cleanup of incomplete downloads. Still todo: --fast, --relaxed, importfeed, disk reserve checking, quvi code cleanup. This commit was sponsored by Anthony DeRobertis on Patreon.
2017-11-29 19:49:05 +00:00
, annexYoutubeDlOptions = getwords (annex "youtube-dl-options")
Added bittorrent special remote addurl behavior change: When downloading an url ending in .torrent, it will download files from bittorrent, instead of the old behavior of adding the torrent file to the repository. Added Recommends on aria2 and bittornado | bittorrent. This commit was sponsored by Asbjørn Sloth Tønnesen.
2014-12-17 03:22:46 +00:00
, annexAriaTorrentOptions = getwords (annex "aria-torrent-options")
start to support core.symlinks=false Utility functions to handle no symlink mode, and converted Annex.Content to use them; still many other places to convert.
2013-02-15 20:02:35 +00:00
, annexCrippledFileSystem = getbool (annex "crippledfilesystem") False
New annex.largefiles setting, which configures which files `git annex add` and the assistant add to the annex. I would have sort of liked to put this in .gitattributes, but it seems it does not support multi-word attribute values. Also, making this a single config setting makes it easy to only parse the expression once. A natural next step would be to make the assistant `git add` files that are not annex.largefiles. OTOH, I don't think `git annex add` should `git add` such files, because git-annex command line tools are not in the business of wrapping git command line tools.
2013-03-29 20:17:13 +00:00
, annexLargeFiles = getmaybe (annex "largefiles")
annex.addsmallfiles: New option controlling what is done when adding files not matching annex.largefiles.
2016-01-28 18:04:32 +00:00
, annexAddSmallFiles = getbool (annex "addsmallfiles") True
webapp: remind user when using repositories that lack consistency checks When starting up the assistant, it'll remind about the current repository, if it doesn't have checks. And when a removable drive is plugged in, it will remind if a repository on it lacks checks. Since that might be annoying, the reminders can be turned off. This commit was sponsored by Nedialko Andreev.
2013-10-29 20:48:06 +00:00
, annexFsckNudge = getbool (annex "fscknudge") True
annex.autoupgrade setting
2013-11-22 20:04:20 +00:00
, annexAutoUpgrade = toAutoUpgrade $ getmaybe (annex "autoupgrade")
allow annex.expireunused to be set to false, as well as to a duration
2014-01-23 03:10:51 +00:00
, annexExpireUnused = maybe Nothing Just . parseDuration
<$> getmaybe (annex "expireunused")
added annex.secure-erase-command config option.
2014-01-24 16:58:52 +00:00
, annexSecureEraseCommand = getmaybe (annex "secure-erase-command")
annex.genmetadata can be set to make git-annex automatically set metadata (year and month) when adding files
2014-02-23 04:08:29 +00:00
, annexGenMetaData = getbool (annex "genmetadata") False
annex.listen can be configured, instead of using --listen
2014-03-01 04:31:17 +00:00
, annexListen = getmaybe (annex "listen")
annex.startupscan can be set to false to disable the assistant's startup scan.
2014-03-05 21:44:14 +00:00
, annexStartupScan = getbool (annex "startupscan") True
New annex.hardlink setting. Closes: #758593 * New annex.hardlink setting. Closes: #758593 * init: Automatically detect when a repository was cloned with --shared, and set annex.hardlink=true, as well as marking the repository as untrusted. Had to reorganize Logs.Trust a bit to avoid a cycle between it and Annex.Init.
2014-09-05 17:44:09 +00:00
, annexHardLink = getbool (annex "hardlink") False
annex.thin Decided it's too scary to make v6 unlocked files have 1 copy by default, but that should be available to those who need it. This is consistent with git-annex not dropping unused content without --force, etc. * Added annex.thin setting, which makes unlocked files in v6 repositories be hard linked to their content, instead of a copy. This saves disk space but means any modification of an unlocked file will lose the local (and possibly only) copy of the old version. * Enable annex.thin by default on upgrade from direct mode to v6, since direct mode made the same tradeoff. * fix: Adjusts unlocked files as configured by annex.thin.
2015-12-27 19:59:59 +00:00
, annexThin = getbool (annex "thin") False
Repository tuning parameters can now be passed when initializing a repository for the first time. * init: Repository tuning parameters can now be passed when initializing a repository for the first time. For details, see http://git-annex.branchable.com/tuning/ * merge: Refuse to merge changes from a git-annex branch of a repo that has been tuned in incompatable ways.
2015-01-27 21:38:06 +00:00
, annexDifferences = getDifferences r
add annex.used-refspec
2015-05-14 19:44:08 +00:00
, annexUsedRefSpec = either (const Nothing) Just . parseRefSpec
=<< getmaybe (annex "used-refspec")
Do verification of checksums of annex objects downloaded from remotes. * When annex objects are received into git repositories, their checksums are verified then too. * To get the old, faster, behavior of not verifying checksums, set annex.verify=false, or remote.<name>.annex-verify=false. * setkey, rekey: These commands also now verify that the provided file matches the key, unless annex.verify=false. * reinject: Already verified content; this can now be disabled by setting annex.verify=false. recvkey and reinject already did verification, so removed now duplicate code from them. fsck still does its own verification, which is ok since it does not use getViaTmp, so verification doesn't happen twice when using fsck --from.
2015-10-01 19:54:37 +00:00
, annexVerify = getbool (annex "verify") True
pid locking configuration and abstraction layer for git-annex (not actually used anywhere yet)
2015-11-12 21:47:31 +00:00
, annexPidLock = getbool (annex "pidlock") False
, annexPidLockTimeout = Seconds $ fromMaybe 300 $
getmayberead (annex "pidlocktimeout")
annex.addunlocked * add, addurl, import, importfeed: When in a v6 repository on a crippled filesystem, add files unlocked. * annex.addunlocked: New configuration setting, makes files always be added unlocked. (v6 only)
2016-02-16 18:43:43 +00:00
, annexAddUnlocked = getbool (annex "addunlocked") False
annex.securehashesonly Cryptographically secure hashes can be forced to be used in a repository, by setting annex.securehashesonly. This does not prevent the git repository from containing files with insecure hashes, but it does prevent the content of such files from being pulled into .git/annex/objects from another repository. We want to make sure that at no point does git-annex accept content into .git/annex/objects that is hashed with an insecure key. Here's how it was done: * .git/annex/objects/xx/yy/KEY/ is kept frozen, so nothing can be written to it normally * So every place that writes content must call, thawContent or modifyContent. We can audit for these, and be sure we've considered all cases. * The main functions are moveAnnex, and linkToAnnex; these were made to check annex.securehashesonly, and are the main security boundary for annex.securehashesonly. * Most other calls to modifyContent deal with other files in the KEY directory (inode cache etc). The other ones that mess with the content are: - Annex.Direct.toDirectGen, in which content already in the annex directory is moved to the direct mode file, so not relevant. - fix and lock, which don't add new content - Command.ReKey.linkKey, which manually unlocks it to make a copy. * All other calls to thawContent appear safe. Made moveAnnex return a Bool, so checked all callsites and made them deal with a failure in appropriate ways. linkToAnnex simply returns LinkAnnexFailed; all callsites already deal with it failing in appropriate ways. This commit was sponsored by Riku Voipio.
2017-02-27 17:01:32 +00:00
, annexSecureHashesOnly = getbool (annex "securehashesonly") False
add retry configuration (not used yet)
2018-03-24 14:37:25 +00:00
, annexRetry = getmayberead (annex "retry")
, annexRetryDelay = Seconds
<$> getmayberead (annex "retrydelay")
limit url downloads to whitelisted schemes Security fix! Allowing any schemes, particularly file: and possibly others like scp: allowed file exfiltration by anyone who had write access to the git repository, since they could add an annexed file using such an url, or using an url that redirected to such an url, and wait for the victim to get it into their repository and send them a copy. * Added annex.security.allowed-url-schemes setting, which defaults to only allowing http and https URLs. Note especially that file:/ is no longer enabled by default. * Removed annex.web-download-command, since its interface does not allow supporting annex.security.allowed-url-schemes across redirects. If you used this setting, you may want to instead use annex.web-options to pass options to curl. With annex.web-download-command removed, nearly all url accesses in git-annex are made via Utility.Url via http-client or curl. http-client only supports http and https, so no problem there. (Disabling one and not the other is not implemented.) Used curl --proto to limit the allowed url schemes. Note that this will cause git annex fsck --from web to mark files using a disallowed url scheme as not being present in the web. That seems acceptable; fsck --from web also does that when a web server is not available. youtube-dl already disabled file: itself (probably for similar reasons). The scheme check was also added to youtube-dl urls for completeness, although that check won't catch any redirects it might follow. But youtube-dl goes off and does its own thing with other protocols anyway, so that's fine. Special remotes that support other domain-specific url schemes are not affected by this change. In the bittorrent remote, aria2c can still download magnet: links. The download of the .torrent file is otherwise now limited by annex.security.allowed-url-schemes. This does not address any external special remotes that might download an url themselves. Current thinking is all external special remotes will need to be audited for this problem, although many of them will use http libraries that only support http and not curl's menagarie. The related problem of accessing private localhost and LAN urls is not addressed by this commit. This commit was sponsored by Brett Eisenberg on Patreon.
2018-06-15 20:52:24 +00:00
, annexAllowedUrlSchemes = S.fromList $ map mkScheme $
allow ftp urls by default They're no worse than http certianly. And, the backport of these security fixes has to deal with wget, which supports http https and ftp and has no way to turn off individual schemes, so this will make that easier.
2018-06-18 19:36:12 +00:00
maybe ["http", "https", "ftp"] words $
limit url downloads to whitelisted schemes Security fix! Allowing any schemes, particularly file: and possibly others like scp: allowed file exfiltration by anyone who had write access to the git repository, since they could add an annexed file using such an url, or using an url that redirected to such an url, and wait for the victim to get it into their repository and send them a copy. * Added annex.security.allowed-url-schemes setting, which defaults to only allowing http and https URLs. Note especially that file:/ is no longer enabled by default. * Removed annex.web-download-command, since its interface does not allow supporting annex.security.allowed-url-schemes across redirects. If you used this setting, you may want to instead use annex.web-options to pass options to curl. With annex.web-download-command removed, nearly all url accesses in git-annex are made via Utility.Url via http-client or curl. http-client only supports http and https, so no problem there. (Disabling one and not the other is not implemented.) Used curl --proto to limit the allowed url schemes. Note that this will cause git annex fsck --from web to mark files using a disallowed url scheme as not being present in the web. That seems acceptable; fsck --from web also does that when a web server is not available. youtube-dl already disabled file: itself (probably for similar reasons). The scheme check was also added to youtube-dl urls for completeness, although that check won't catch any redirects it might follow. But youtube-dl goes off and does its own thing with other protocols anyway, so that's fine. Special remotes that support other domain-specific url schemes are not affected by this change. In the bittorrent remote, aria2c can still download magnet: links. The download of the .torrent file is otherwise now limited by annex.security.allowed-url-schemes. This does not address any external special remotes that might download an url themselves. Current thinking is all external special remotes will need to be audited for this problem, although many of them will use http libraries that only support http and not curl's menagarie. The related problem of accessing private localhost and LAN urls is not addressed by this commit. This commit was sponsored by Brett Eisenberg on Patreon.
2018-06-15 20:52:24 +00:00
getmaybe (annex "security.allowed-url-schemes")
rename annex.security.allowed-http-addresses Renamed annex.security.allowed-http-addresses to annex.security.allowed-ip-addresses because it is not really specific to the http protocol, also limiting eg, git-annex's use of ftp and via youtube-dl, several other protocols. The old name for the config will still work. If both old and new name are set, the new name will win.
2019-05-30 16:43:40 +00:00
, annexAllowedIPAddresses = fromMaybe "" $
getmaybe (annex "security.allowed-ip-addresses")
<|>
getmaybe (annex "security.allowed-http-addresses") -- old name
enforce retrievalSecurityPolicy Leveraged the existing verification code by making it also check the retrievalSecurityPolicy. Also, prevented getViaTmp from running the download action at all when the retrievalSecurityPolicy is going to prevent verifying and so storing it. Added annex.security.allow-unverified-downloads. A per-remote version would be nice to have too, but would need more plumbing, so KISS. (Bill the Cat reference not too over the top I hope. The point is to make this something the user reads the documentation for before using.) A few calls to verifyKeyContent and getViaTmp, that don't involve downloads from remotes, have RetrievalAllKeysSecure hard-coded. It was also hard-coded for P2P.Annex and Command.RecvKey, to match the values of the corresponding remotes. A few things use retrieveKeyFile/retrieveKeyFileCheap without going through getViaTmp. * Command.Fsck when downloading content from a remote to verify it. That content does not get into the annex, so this is ok. * Command.AddUrl when using a remote to download an url; this is new content being added, so this is ok. This commit was sponsored by Fernando Jimenez on Patreon.
2018-06-21 17:34:11 +00:00
, annexAllowUnverifiedDownloads = (== Just "ACKTHPPT") $
getmaybe (annex "security.allow-unverified-downloads")
annex.maxextensionlength Added annex.maxextensionlength for use cases where extensions longer than 4 characters are needed. This commit was sponsored by Henrik Riomar on Patreon.
2018-09-24 16:07:46 +00:00
, annexMaxExtensionLength = getmayberead (annex "maxextensionlength")
annex.jobs=cpus etc Added the ability to run one job per CPU (core), by setting annex.jobs=cpus, or using option --jobs=cpus or -Jcpus. Built with future expansion in mind, including not defaulting matching on Concurrency so more constructors can later be added, and using "cpu" instead of "0".
2019-05-10 17:24:31 +00:00
, annexJobs = fromMaybe NonConcurrent $
parseConcurrency =<< getmaybe (annex "jobs")
annex.cachecreds: New config to allow disabling of credentials caching for special remotes. Note that it does not prevent storing p2p access tokens or multicast encryption keys, since those are not cached; the previous commit established the distinction. How well this works depends on how often getRemoteCredPair is called and how expensive it is. In some cases setting this will result in an annoying number of gpg password prompts and/or slowdowns due to reading creds from the git-annex branch and decrypting, which could be improved by calling getRemoteCredPair less often. This commit was sponsored by Ilya Shlyakhter on Patreon.
2018-12-04 18:16:56 +00:00
, annexCacheCreds = getbool (annex "cachecreds") True
Added annex.autoupgraderepository configuration Can be set to false to prevent any automatic repository upgrades. Also, removed direct mode specific upgrade code in Annex.Init, and made needsUpgrade always include the name/path of the repo, so if there's a problem it's clear what repo has the problem. And, made needsUpgrade catch any exceptions that might occur during the upgrade, so it can display a more useful error message than just the exception.
2019-09-01 17:29:55 +00:00
, annexAutoUpgradeRepository = getbool (annex "autoupgraderepository") True
support gpg.program When gpg.program is configured, it's used to get the command to run for gpg. Useful on systems that have only a gpg2 command or want to use it instead of the gpg command.
2015-09-09 22:06:49 +00:00
, coreSymlinks = getbool "core.symlinks" True
, coreSharedRepository = getSharedRepository r
post-recive hook to make updateInstead work in direct mode and adjusted branches * Added post-recieve hook, which makes updateInstead work with direct mode and adjusted branches. * init: Set up the post-receive hook. This commit was sponsored by Fernando Jimenez on Patreon.
2017-02-17 18:04:43 +00:00
, receiveDenyCurrentBranch = getDenyCurrentBranch r
support gpg.program When gpg.program is configured, it's used to get the command to run for gpg. Useful on systems that have only a gpg2 command or want to use it instead of the gpg command.
2015-09-09 22:06:49 +00:00
, gcryptId = getmaybe "core.gcrypt-id"
, gpgCmd = mkGpgCmd (getmaybe "gpg.program")
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
}
where
import Data.Default in Common
2015-01-28 20:11:28 +00:00
getbool k d = fromMaybe d $ getmaybebool k
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
getmaybebool k = Git.Config.isTrue =<< getmaybe k
getmayberead k = readish =<< getmaybe k
start to support core.symlinks=false Utility functions to handle no symlink mode, and converted Annex.Content to use them; still many other places to convert.
2013-02-15 20:02:35 +00:00
getmaybe k = Git.Config.getMaybe k r
getlist k = Git.Config.getList k r
add another setting to GitConfig
2013-01-27 13:33:19 +00:00
getwords k = fromMaybe [] $ words <$> getmaybe k
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
annex.autocommit can be configured via git-annex config ... to control the default behavior in all clones of a repository. This includes a new Configurable data type, so the GitConfig type indicates which values can be configured this way. The implementation should be quite efficient; the config log is only read once, and only when a Configurable value has not already been set by git-config. Indeed, it would be nice in the future to extend this, so that git-config is itself only read on demand. Some commands may not need to look at the git configuration at all. This commit was sponsored by Trenton Cronholm on Patreon.
2017-02-03 17:40:14 +00:00
configurable d Nothing = DefaultConfig d
configurable _ (Just v) = HasConfig v
start to support core.symlinks=false Utility functions to handle no symlink mode, and converted Annex.Content to use them; still many other places to convert.
2013-02-15 20:02:35 +00:00
annex k = "annex." ++ k
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
onemegabyte = 1000000
annex.autocommit can be configured via git-annex config ... to control the default behavior in all clones of a repository. This includes a new Configurable data type, so the GitConfig type indicates which values can be configured this way. The implementation should be quite efficient; the config log is only read once, and only when a Configurable value has not already been set by git-config. Indeed, it would be nice in the future to extend this, so that git-config is itself only read on demand. Some commands may not need to look at the git configuration at all. This commit was sponsored by Trenton Cronholm on Patreon.
2017-02-03 17:40:14 +00:00
{- Merge a GitConfig that comes from git-config with one containing
- repository-global defaults. -}
mergeGitConfig :: GitConfig -> GitConfig -> GitConfig
mergeGitConfig gitconfig repoglobals = gitconfig
{ annexAutoCommit = merge annexAutoCommit
New annex.synccontent config setting .. which can be set to true to make git annex sync default to --content. This may become the default at some point in the future. As well as being configuable by git config, it can be configured by git-annex config to control the default behavior in all clones of a repository. Had to add a separate --no-content switch to we can tell if it's been explicitly set, and should override annex.synccontent. If --content was the default, this complication would not be necessary. This commit was sponsored by Jake Vosloo on Patreon.
2017-02-03 18:31:17 +00:00
, annexSyncContent = merge annexSyncContent
annex.autocommit can be configured via git-annex config ... to control the default behavior in all clones of a repository. This includes a new Configurable data type, so the GitConfig type indicates which values can be configured this way. The implementation should be quite efficient; the config log is only read once, and only when a Configurable value has not already been set by git-config. Indeed, it would be nice in the future to extend this, so that git-config is itself only read on demand. Some commands may not need to look at the git configuration at all. This commit was sponsored by Trenton Cronholm on Patreon.
2017-02-03 17:40:14 +00:00
}
where
merge f = case f gitconfig of
HasConfig v -> HasConfig v
DefaultConfig d -> case f repoglobals of
HasConfig v -> HasConfig v
DefaultConfig _ -> HasConfig d
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
{- Per-remote git-annex settings. Each setting corresponds to a git-config
- key such as <remote>.annex-foo, or if that is not set, a default from
removal of the rest of remoteGitConfig In keyUrls, the GitConfig is used only by annexLocations to support configured Differences. Since such configurations affect all clones of a repository, the local repo's GitConfig must have the same information as the remote's GitConfig would have. So, used getGitConfig to get the local GitConfig, which is cached and so available cheaply. That actually fixed a bug noone had ever noticed: keyUrls is used for remotes accessed over http. The full git config of such a remote is normally not available, so the remoteGitConfig that keyUrls used would not have the necessary information in it. In copyFromRemoteCheap', it uses gitAnnexLocation, which does need the GitConfig of the remote repo itself in order to check if it's crippled, supports symlinks, etc. So, made the State include that GitConfig, cached. The use of gitAnnexLocation is within a (not $ Git.repoIsUrl repo) guard, so it's local, and so its git config will always be read and available. (Note that gitAnnexLocation in turn calls annexLocations, so the Differences config it uses in this case comes from the remote repo's GitConfig and not from the local repo's GitConfig. As explained above this is ok since they must have the same value.) Not very happy with this mess of different GitConfigs not type-safe and some read only sometimes etc. Very hairy. Think I got it this change right. Test suite passes.. This commit was sponsored by Ethan Aubin.
2018-06-05 18:23:34 +00:00
- annex.foo.
-
- Note that this is from the perspective of the local repository,
- it is not influenced in any way by the contents of the remote
- repository's git config.
-}
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
data RemoteGitConfig = RemoteGitConfig
use DynamicConfig to handle cost-command This commit was sponsored by Jake Vosloo on Patreon.
2017-08-17 18:04:29 +00:00
{ remoteAnnexCost :: DynamicConfig (Maybe Cost)
add annex-ignore-command and annex-sync-command configs Added remote configuration settings annex-ignore-command and annex-sync-command, which are dynamic equivilants of the annex-ignore and annex-sync configurations. For this I needed a new DynamicConfig infrastructure. Its implementation should be as fast as before when there is no dynamic config, and it caches so shell commands are only run once. Note that annex-ignore-command exits nonzero when the remote should be ignored. While that may seem backwards, it allows using the same command for it as for annex-sync-command when you want to disable both. This commit was sponsored by Trenton Cronholm on Patreon.
2017-08-17 16:26:14 +00:00
, remoteAnnexIgnore :: DynamicConfig Bool
, remoteAnnexSync :: DynamicConfig Bool
Added remote.<name>.annex-push and remote.<name>.annex-pull The former can be useful to make remotes that don't get fully synced with local changes, which comes up in a lot of situations. The latter was mostly added for symmetry, but could be useful (though less likely to be). Implementing `remote.<name>.annex-pull` was a bit tricky, as there's no one place where git-annex pulls/fetches from remotes. I audited all instances of "fetch" and "pull". A few cases were left not checking this config: * Git.Repair can try to pull missing refs from a remote, and if the local repo is corrupted, that seems a reasonable thing to do even though the config would normally prevent it. * Assistant.WebApp.Gpg and Remote.Gcrypt and Remote.Git do fetches as part of the setup process of a remote. The config would probably not be set then, and having the setup fail seems worse than honoring it if it is already set. I have not prevented all the code that does a "merge" from merging branches from remotes with remote.<name>.annex-pull=false. That could perhaps be done, but it would need a way to map from branch name to remote name, and the way refspecs work makes that hard to get really correct. So if the user fetches manually, the git-annex branch will get merged, for example. Anther way of looking at/justifying this is that the setting is called "annex-pull", not "annex-merge". This commit was supported by the NSF-funded DataLad project.
2017-04-05 17:04:02 +00:00
, remoteAnnexPull :: Bool
, remoteAnnexPush :: Bool
Remotes can now be made read-only, by setting remote.<name>.annex-readonly
2014-01-02 17:12:32 +00:00
, remoteAnnexReadOnly :: Bool
Do verification of checksums of annex objects downloaded from remotes. * When annex objects are received into git repositories, their checksums are verified then too. * To get the old, faster, behavior of not verifying checksums, set annex.verify=false, or remote.<name>.annex-verify=false. * setkey, rekey: These commands also now verify that the provided file matches the key, unless annex.verify=false. * reinject: Already verified content; this can now be disabled by setting annex.verify=false. recvkey and reinject already did verification, so removed now duplicate code from them. fsck still does its own verification, which is ok since it does not use getViaTmp, so verification doesn't happen twice when using fsck --from.
2015-10-01 19:54:37 +00:00
, remoteAnnexVerify :: Bool
add remote.<name>.annex-checkuuid Added remote.<name>.annex-checkuuid config, which can be set to false to disable the default checking of the uuid of remotes that point to directories. This can be useful to avoid unncessary drive spin-ups and automounting. Note that the UUID check is still done before writing to the repository, to avoid writing to the wrong repository if it got relocated. Check is also done before checkPresent to avoid getting confused about what is in which repo. This is effectively the same as the use of git-annex-shell with a uuid to check that the remote repository is the expected one. Did not bother with the check for retrieveKeyFile because it doesn't matter if the wrong repo is used then. This commit was sponsored by Trenton Cronholm on Patreon.
2018-01-10 18:21:18 +00:00
, remoteAnnexCheckUUID :: Bool
export: Deprecated the --tracking option. Instead, users can configure remote.<name>.annex-tracking-branch themselves.
2019-02-23 19:48:25 +00:00
, remoteAnnexTrackingBranch :: Maybe Git.Ref
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
, remoteAnnexTrustLevel :: Maybe String
, remoteAnnexStartCommand :: Maybe String
, remoteAnnexStopCommand :: Maybe String
add GETAVAILABILITY to external special remote protocol And some reworking of types, and added an annex-availability git config setting.
2014-01-13 18:41:10 +00:00
, remoteAnnexAvailability :: Maybe Availability
cache remotes via annex-speculate-present Added remote.name.annex-speculate-present config that can be used to make cache remotes. Implemented it in Remote.keyPossibilities, which is used by the get/move/copy/mirror commands, and nothing else. This way, things like whereis will not show content that's speculatively present. The assistant and sync --content were not using Remote.keyPossibilities, and were changed to use it. The efficiency hit should be small; Remote.keyPossibilities is only used before transferring a file, which is the expensive operation. And, it's only doing one lookup of the remoteList and a very cheap filter over it. Note that, git-annex still updates the location log when copying content to a remote with annex-speculate-present set. In this case, the location tracking will indicate that content is present in the remote. This may not be wanted for caches, or may not be a real problem for them. TBD. This commit was supported by the NSF-funded DataLad project.
2018-08-01 18:22:52 +00:00
, remoteAnnexSpeculatePresent :: Bool
Optimise non-bare http remotes; no longer does a 404 to the wrong url every time before trying the right url. Needs annex-bare to be set to false, which is done when initially probing the uuid of a http remote.
2014-01-26 17:03:25 +00:00
, remoteAnnexBare :: Maybe Bool
add retry configuration (not used yet)
2018-03-24 14:37:25 +00:00
, remoteAnnexRetry :: Maybe Integer
, remoteAnnexRetryDelay :: Maybe Seconds
remote.name.annex-security-allow-unverified-downloads Added remote.name.annex-security-allow-unverified-downloads, a per-remote setting for annex.security.allow-unverified-downloads. This commit was sponsored by Brock Spratlen on Patreon.
2018-09-25 19:32:50 +00:00
, remoteAnnexAllowUnverifiedDownloads :: Bool
add remote.<name>.annex-config-uuid for use by sameas remotes
2019-10-10 16:12:22 +00:00
, remoteAnnexConfigUUID :: Maybe UUID
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
Use lower case hash directories for storing files on crippled filesystems, same as is already done for bare repositories. * since this is a crippled filesystem anyway, git-annex doesn't use symlinks on it * so there's no reason to use the mixed case hash directories that we're stuck using to avoid breaking everyone's symlinks to the content * so we can do what is already done for all bare repos, and make non-bare repos on crippled filesystems use the all-lower case hash directories * which are, happily, all 3 letters long, so they cannot conflict with mixed case hash directories * so I was able to 100% fix this and even resuming `git annex add` in the test case will recover and it will all just work.
2013-04-04 19:46:33 +00:00
{- These settings are specific to particular types of remotes
- including special remotes. -}
execute remote.<name>.annex-shell on remote, if set It is useful to be able to specify an alternative git-annex-shell program to execute on the remote, e.g., to run a version not on the PATH. Use remote.<name>.annex-shell if specified, instead of the default "git-annex-shell" i.e., first so-named executable on the PATH.
2014-05-16 11:34:43 +00:00
, remoteAnnexShell :: Maybe String
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
, remoteAnnexSshOptions :: [String]
, remoteAnnexRsyncOptions :: [String]
Added ways to configure rsync options to be used only when uploading or downloading from a remote. Useful to eg limit upload bandwidth.
2014-02-02 20:06:34 +00:00
, remoteAnnexRsyncUploadOptions :: [String]
, remoteAnnexRsyncDownloadOptions :: [String]
Allow rsync to use other remote shells. Introduced a new per-remote option 'annex-rsync-transport' to specify the remote shell that it to be used with rsync. In case the value is 'ssh', connections are cached unless 'sshcaching' is unset.
2013-04-13 22:10:49 +00:00
, remoteAnnexRsyncTransport :: [String]
GnuPG options for symmetric encryption.
2013-03-11 01:33:13 +00:00
, remoteAnnexGnupgOptions :: [String]
Added annex.gnupg-decrypt-options and remote.<name>.annex-gnupg-decrypt-options, which are passed to gpg when it's decrypting data. The naming is unofrtunately not consistent, but the gnupg-options were only used for encrypting, and it's too late to change that. It would be nice to have a third setting that is always passed to gnupg, but ~/.gnupg/options can be used to specify such global options when really needed.
2016-05-10 17:03:56 +00:00
, remoteAnnexGnupgDecryptOptions :: [String]
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
, remoteAnnexRsyncUrl :: Maybe String
, remoteAnnexBupRepo :: Maybe String
Added tahoe special remote. Known problems: 1. Tries to tahoe start when daemon is already running. 2. If multiple tahoe remotes are set up on the same computer, they will have the same node.url configured by default, and this confuses tahoe commands. This commit was sponsored by LeastAuthority.com
2014-01-08 20:14:37 +00:00
, remoteAnnexTahoe :: Maybe FilePath
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
, remoteAnnexBupSplitOptions :: [String]
, remoteAnnexDirectory :: Maybe FilePath
Added adb special remote which allows exporting files to Android devices. git annex testremote passes. exportree not implemented yet, although the documentation talks about it, since it will be the main way this remote will be used. The adb push/pull progress is displayed for now; it would be better to consume it and use it to update the git-annex progress bar. This commit was sponsored by andrea rota.
2018-03-27 16:41:57 +00:00
, remoteAnnexAndroidDirectory :: Maybe FilePath
, remoteAnnexAndroidSerial :: Maybe String
git-annex-shell: Added support for operating inside gcrypt repositories. * Note that the layout of gcrypt repositories has changed, and if you created one you must manually upgrade it. See http://git-annex.branchable.com/upgrades/gcrypt/
2013-09-24 21:25:47 +00:00
, remoteAnnexGCrypt :: Maybe String
skeleton git-lfs special remote This is a special remote and a git remote at the same time; git can pull and push to it and git-annex can use it as a special remote. Remote.Git has to check if it's configured as a git-lfs special remote and sets it up as one if so. Object methods not implemented yet.
2019-08-01 19:11:45 +00:00
, remoteAnnexGitLFS :: Bool
ddar special remote
2014-05-15 18:44:00 +00:00
, remoteAnnexDdarRepo :: Maybe String
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
, remoteAnnexHookType :: Maybe String
external special remotes mostly implemented (untested) This has not been tested at all. It compiles! The only known missing things are support for encryption, and for get/set of special remote configuration, and of key state. (The latter needs separate work to add a new per-key log file to store that state.) Only thing I don't much like is that initremote needs to be passed both type=external and externaltype=foo. It would be better to have just type=foo Most of this is quite straightforward code, that largely wrote itself given the types. The only tricky parts were: * Need to lock the remote when using it to eg make a request, because in theory git-annex could have multiple threads that each try to use a remote at the same time. I don't think that git-annex ever does that currently, but better safe than sorry. * Rather than starting up every external special remote program when git-annex starts, they are started only on demand, when first used. This will avoid slowdown, especially when running fast git-annex query commands. Once started, they keep running until git-annex stops, currently, which may not be ideal, but it's hard to know a better time to stop them. * Bit of a chicken and egg problem with caching the cost of the remote, because setting annex-cost in the git config needs the remote to already be set up. Managed to finesse that. This commit was sponsored by Lukas Anzinger.
2013-12-26 22:23:13 +00:00
, remoteAnnexExternalType :: Maybe String
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
}
removal of the rest of remoteGitConfig In keyUrls, the GitConfig is used only by annexLocations to support configured Differences. Since such configurations affect all clones of a repository, the local repo's GitConfig must have the same information as the remote's GitConfig would have. So, used getGitConfig to get the local GitConfig, which is cached and so available cheaply. That actually fixed a bug noone had ever noticed: keyUrls is used for remotes accessed over http. The full git config of such a remote is normally not available, so the remoteGitConfig that keyUrls used would not have the necessary information in it. In copyFromRemoteCheap', it uses gitAnnexLocation, which does need the GitConfig of the remote repo itself in order to check if it's crippled, supports symlinks, etc. So, made the State include that GitConfig, cached. The use of gitAnnexLocation is within a (not $ Git.repoIsUrl repo) guard, so it's local, and so its git config will always be read and available. (Note that gitAnnexLocation in turn calls annexLocations, so the Differences config it uses in this case comes from the remote repo's GitConfig and not from the local repo's GitConfig. As explained above this is ok since they must have the same value.) Not very happy with this mess of different GitConfigs not type-safe and some read only sometimes etc. Very hairy. Think I got it this change right. Test suite passes.. This commit was sponsored by Ethan Aubin.
2018-06-05 18:23:34 +00:00
{- The Git.Repo is the local repository, which has the remote with the
- given RemoteName. -}
extractRemoteGitConfig :: Git.Repo -> RemoteName -> STM RemoteGitConfig
add annex-ignore-command and annex-sync-command configs Added remote configuration settings annex-ignore-command and annex-sync-command, which are dynamic equivilants of the annex-ignore and annex-sync configurations. For this I needed a new DynamicConfig infrastructure. Its implementation should be as fast as before when there is no dynamic config, and it caches so shell commands are only run once. Note that annex-ignore-command exits nonzero when the remote should be ignored. While that may seem backwards, it allows using the same command for it as for annex-sync-command when you want to disable both. This commit was sponsored by Trenton Cronholm on Patreon.
2017-08-17 16:26:14 +00:00
extractRemoteGitConfig r remotename = do
use DynamicConfig to handle cost-command This commit was sponsored by Jake Vosloo on Patreon.
2017-08-17 18:04:29 +00:00
annexcost <- mkDynamicConfig readCommandRunner
(notempty $ getmaybe "cost-command")
(getmayberead "cost")
add annex-ignore-command and annex-sync-command configs Added remote configuration settings annex-ignore-command and annex-sync-command, which are dynamic equivilants of the annex-ignore and annex-sync configurations. For this I needed a new DynamicConfig infrastructure. Its implementation should be as fast as before when there is no dynamic config, and it caches so shell commands are only run once. Note that annex-ignore-command exits nonzero when the remote should be ignored. While that may seem backwards, it allows using the same command for it as for annex-sync-command when you want to disable both. This commit was sponsored by Trenton Cronholm on Patreon.
2017-08-17 16:26:14 +00:00
annexignore <- mkDynamicConfig unsuccessfullCommandRunner
(notempty $ getmaybe "ignore-command")
(getbool "ignore" False)
annexsync <- mkDynamicConfig successfullCommandRunner
(notempty $ getmaybe "sync-command")
(getbool "sync" True)
return $ RemoteGitConfig
use DynamicConfig to handle cost-command This commit was sponsored by Jake Vosloo on Patreon.
2017-08-17 18:04:29 +00:00
{ remoteAnnexCost = annexcost
add annex-ignore-command and annex-sync-command configs Added remote configuration settings annex-ignore-command and annex-sync-command, which are dynamic equivilants of the annex-ignore and annex-sync configurations. For this I needed a new DynamicConfig infrastructure. Its implementation should be as fast as before when there is no dynamic config, and it caches so shell commands are only run once. Note that annex-ignore-command exits nonzero when the remote should be ignored. While that may seem backwards, it allows using the same command for it as for annex-sync-command when you want to disable both. This commit was sponsored by Trenton Cronholm on Patreon.
2017-08-17 16:26:14 +00:00
, remoteAnnexIgnore = annexignore
, remoteAnnexSync = annexsync
, remoteAnnexPull = getbool "pull" True
, remoteAnnexPush = getbool "push" True
, remoteAnnexReadOnly = getbool "readonly" False
add remote.<name>.annex-checkuuid Added remote.<name>.annex-checkuuid config, which can be set to false to disable the default checking of the uuid of remotes that point to directories. This can be useful to avoid unncessary drive spin-ups and automounting. Note that the UUID check is still done before writing to the repository, to avoid writing to the wrong repository if it got relocated. Check is also done before checkPresent to avoid getting confused about what is in which repo. This is effectively the same as the use of git-annex-shell with a uuid to check that the remote repository is the expected one. Did not bother with the check for retrieveKeyFile because it doesn't matter if the wrong repo is used then. This commit was sponsored by Trenton Cronholm on Patreon.
2018-01-10 18:21:18 +00:00
, remoteAnnexCheckUUID = getbool "checkuuid" True
add annex-ignore-command and annex-sync-command configs Added remote configuration settings annex-ignore-command and annex-sync-command, which are dynamic equivilants of the annex-ignore and annex-sync configurations. For this I needed a new DynamicConfig infrastructure. Its implementation should be as fast as before when there is no dynamic config, and it caches so shell commands are only run once. Note that annex-ignore-command exits nonzero when the remote should be ignored. While that may seem backwards, it allows using the same command for it as for annex-sync-command when you want to disable both. This commit was sponsored by Trenton Cronholm on Patreon.
2017-08-17 16:26:14 +00:00
, remoteAnnexVerify = getbool "verify" True
export: Deprecated the --tracking option. Instead, users can configure remote.<name>.annex-tracking-branch themselves.
2019-02-23 19:48:25 +00:00
, remoteAnnexTrackingBranch = Git.Ref <$>
fix name of annex-tracking-branch config
2019-03-11 17:56:59 +00:00
( notempty (getmaybe "tracking-branch")
export: Deprecated the --tracking option. Instead, users can configure remote.<name>.annex-tracking-branch themselves.
2019-02-23 19:48:25 +00:00
<|> notempty (getmaybe "export-tracking") -- old name
)
add annex-ignore-command and annex-sync-command configs Added remote configuration settings annex-ignore-command and annex-sync-command, which are dynamic equivilants of the annex-ignore and annex-sync configurations. For this I needed a new DynamicConfig infrastructure. Its implementation should be as fast as before when there is no dynamic config, and it caches so shell commands are only run once. Note that annex-ignore-command exits nonzero when the remote should be ignored. While that may seem backwards, it allows using the same command for it as for annex-sync-command when you want to disable both. This commit was sponsored by Trenton Cronholm on Patreon.
2017-08-17 16:26:14 +00:00
, remoteAnnexTrustLevel = notempty $ getmaybe "trustlevel"
, remoteAnnexStartCommand = notempty $ getmaybe "start-command"
, remoteAnnexStopCommand = notempty $ getmaybe "stop-command"
, remoteAnnexAvailability = getmayberead "availability"
cache remotes via annex-speculate-present Added remote.name.annex-speculate-present config that can be used to make cache remotes. Implemented it in Remote.keyPossibilities, which is used by the get/move/copy/mirror commands, and nothing else. This way, things like whereis will not show content that's speculatively present. The assistant and sync --content were not using Remote.keyPossibilities, and were changed to use it. The efficiency hit should be small; Remote.keyPossibilities is only used before transferring a file, which is the expensive operation. And, it's only doing one lookup of the remoteList and a very cheap filter over it. Note that, git-annex still updates the location log when copying content to a remote with annex-speculate-present set. In this case, the location tracking will indicate that content is present in the remote. This may not be wanted for caches, or may not be a real problem for them. TBD. This commit was supported by the NSF-funded DataLad project.
2018-08-01 18:22:52 +00:00
, remoteAnnexSpeculatePresent = getbool "speculate-present" False
add annex-ignore-command and annex-sync-command configs Added remote configuration settings annex-ignore-command and annex-sync-command, which are dynamic equivilants of the annex-ignore and annex-sync configurations. For this I needed a new DynamicConfig infrastructure. Its implementation should be as fast as before when there is no dynamic config, and it caches so shell commands are only run once. Note that annex-ignore-command exits nonzero when the remote should be ignored. While that may seem backwards, it allows using the same command for it as for annex-sync-command when you want to disable both. This commit was sponsored by Trenton Cronholm on Patreon.
2017-08-17 16:26:14 +00:00
, remoteAnnexBare = getmaybebool "bare"
add retry configuration (not used yet)
2018-03-24 14:37:25 +00:00
, remoteAnnexRetry = getmayberead "retry"
, remoteAnnexRetryDelay = Seconds
<$> getmayberead "retrydelay"
remote.name.annex-security-allow-unverified-downloads Added remote.name.annex-security-allow-unverified-downloads, a per-remote setting for annex.security.allow-unverified-downloads. This commit was sponsored by Brock Spratlen on Patreon.
2018-09-25 19:32:50 +00:00
, remoteAnnexAllowUnverifiedDownloads = (== Just "ACKTHPPT") $
getmaybe ("security-allow-unverified-downloads")
add remote.<name>.annex-config-uuid for use by sameas remotes
2019-10-10 16:12:22 +00:00
, remoteAnnexConfigUUID = toUUID <$> getmaybe "config-uuid"
add annex-ignore-command and annex-sync-command configs Added remote configuration settings annex-ignore-command and annex-sync-command, which are dynamic equivilants of the annex-ignore and annex-sync configurations. For this I needed a new DynamicConfig infrastructure. Its implementation should be as fast as before when there is no dynamic config, and it caches so shell commands are only run once. Note that annex-ignore-command exits nonzero when the remote should be ignored. While that may seem backwards, it allows using the same command for it as for annex-sync-command when you want to disable both. This commit was sponsored by Trenton Cronholm on Patreon.
2017-08-17 16:26:14 +00:00
, remoteAnnexShell = getmaybe "shell"
, remoteAnnexSshOptions = getoptions "ssh-options"
, remoteAnnexRsyncOptions = getoptions "rsync-options"
, remoteAnnexRsyncDownloadOptions = getoptions "rsync-download-options"
, remoteAnnexRsyncUploadOptions = getoptions "rsync-upload-options"
, remoteAnnexRsyncTransport = getoptions "rsync-transport"
, remoteAnnexGnupgOptions = getoptions "gnupg-options"
, remoteAnnexGnupgDecryptOptions = getoptions "gnupg-decrypt-options"
, remoteAnnexRsyncUrl = notempty $ getmaybe "rsyncurl"
, remoteAnnexBupRepo = getmaybe "buprepo"
, remoteAnnexTahoe = getmaybe "tahoe"
, remoteAnnexBupSplitOptions = getoptions "bup-split-options"
, remoteAnnexDirectory = notempty $ getmaybe "directory"
Added adb special remote which allows exporting files to Android devices. git annex testremote passes. exportree not implemented yet, although the documentation talks about it, since it will be the main way this remote will be used. The adb push/pull progress is displayed for now; it would be better to consume it and use it to update the git-annex progress bar. This commit was sponsored by andrea rota.
2018-03-27 16:41:57 +00:00
, remoteAnnexAndroidDirectory = notempty $ getmaybe "androiddirectory"
, remoteAnnexAndroidSerial = notempty $ getmaybe "androidserial"
add annex-ignore-command and annex-sync-command configs Added remote configuration settings annex-ignore-command and annex-sync-command, which are dynamic equivilants of the annex-ignore and annex-sync configurations. For this I needed a new DynamicConfig infrastructure. Its implementation should be as fast as before when there is no dynamic config, and it caches so shell commands are only run once. Note that annex-ignore-command exits nonzero when the remote should be ignored. While that may seem backwards, it allows using the same command for it as for annex-sync-command when you want to disable both. This commit was sponsored by Trenton Cronholm on Patreon.
2017-08-17 16:26:14 +00:00
, remoteAnnexGCrypt = notempty $ getmaybe "gcrypt"
skeleton git-lfs special remote This is a special remote and a git remote at the same time; git can pull and push to it and git-annex can use it as a special remote. Remote.Git has to check if it's configured as a git-lfs special remote and sets it up as one if so. Object methods not implemented yet.
2019-08-01 19:11:45 +00:00
, remoteAnnexGitLFS = getbool "git-lfs" False
add annex-ignore-command and annex-sync-command configs Added remote configuration settings annex-ignore-command and annex-sync-command, which are dynamic equivilants of the annex-ignore and annex-sync configurations. For this I needed a new DynamicConfig infrastructure. Its implementation should be as fast as before when there is no dynamic config, and it caches so shell commands are only run once. Note that annex-ignore-command exits nonzero when the remote should be ignored. While that may seem backwards, it allows using the same command for it as for annex-sync-command when you want to disable both. This commit was sponsored by Trenton Cronholm on Patreon.
2017-08-17 16:26:14 +00:00
, remoteAnnexDdarRepo = getmaybe "ddarrepo"
, remoteAnnexHookType = notempty $ getmaybe "hooktype"
, remoteAnnexExternalType = notempty $ getmaybe "externaltype"
}
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
where
import Data.Default in Common
2015-01-28 20:11:28 +00:00
getbool k d = fromMaybe d $ getmaybebool k
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
getmaybebool k = Git.Config.isTrue =<< getmaybe k
getmayberead k = readish =<< getmaybe k
hlint
2013-04-03 07:52:41 +00:00
getmaybe k = mplus (Git.Config.getMaybe (key k) r)
(Git.Config.getMaybe (remotekey k) r)
type based git config handling for remotes Still a couple of places that use git config ad-hoc, but this is most of it done.
2013-01-01 17:52:47 +00:00
getoptions k = fromMaybe [] $ words <$> getmaybe k
key k = "annex." ++ k
remotekey k = "remote." ++ remotename ++ ".annex-" ++ k
notempty :: Maybe String -> Maybe String
notempty Nothing = Nothing
notempty (Just "") = Nothing
notempty (Just s) = Just s
add annex-ignore-command and annex-sync-command configs Added remote configuration settings annex-ignore-command and annex-sync-command, which are dynamic equivilants of the annex-ignore and annex-sync configurations. For this I needed a new DynamicConfig infrastructure. Its implementation should be as fast as before when there is no dynamic config, and it caches so shell commands are only run once. Note that annex-ignore-command exits nonzero when the remote should be ignored. While that may seem backwards, it allows using the same command for it as for annex-sync-command when you want to disable both. This commit was sponsored by Trenton Cronholm on Patreon.
2017-08-17 16:26:14 +00:00
dummyRemoteGitConfig :: IO RemoteGitConfig
dummyRemoteGitConfig = atomically $
extractRemoteGitConfig Git.Construct.fromUnknown "dummy"
Reference in a new issue Copy permalink