mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
git-annex/Types/StandardGroups.hs

110 lines
4.7 KiB
Haskell
Raw Normal View History

refactor
2012-10-10 19:15:56 +00:00
{- git-annex standard repository groups
-
update my email address and homepage url
2015-01-21 16:50:09 +00:00
- Copyright 2012 Joey Hess <id@joeyh.name>
refactor
2012-10-10 19:15:56 +00:00
-
update licenses from GPL to AGPL This does not change the overall license of the git-annex program, which was already AGPL due to a number of sources files being AGPL already. Legally speaking, I'm adding a new license under which these files are now available; I already released their current contents under the GPL license. Now they're dual licensed GPL and AGPL. However, I intend for all my future changes to these files to only be released under the AGPL license, and I won't be tracking the dual licensing status, so I'm simply changing the license statement to say it's AGPL. (In some cases, others wrote parts of the code of a file and released it under the GPL; but in all cases I have contributed a significant portion of the code in each file and it's that code that is getting the AGPL license; the GPL license of other contributors allows combining with AGPL code.)
2019-03-13 19:48:14 +00:00
- Licensed under the GNU AGPL version 3 or higher.
refactor
2012-10-10 19:15:56 +00:00
-}
newtype Group to ByteString This may speed up queries for things in groups, due to Eq and Ord being faster.
2019-01-09 19:00:43 +00:00
{-# LANGUAGE OverloadedStrings #-}
add standard group selector to repo edit form
2012-10-10 20:04:28 +00:00
module Types.StandardGroups where
webapp: put new repos in standard groups I'm using transfer for most things, both removable drives and cloud storage, because it's the safest choice. We'll see if it makes sense to prompt for the group when setting this up, or let the user pick something else after the fact.
2012-10-10 19:27:25 +00:00
per-IA-item content directories
2013-04-26 03:44:55 +00:00
import Types.Remote (RemoteConfig)
vicfg: Allows editing preferred content expressions for groups. This is stored in the git-annex branch, but not yet actually hooked up and used.
2014-03-15 20:17:01 +00:00
import Types.Group
be stricter about rejecting invalid configurations for remotes This is a first step toward that goal, using the ProposedAccepted type in RemoteConfig lets initremote/enableremote reject bad parameters that were passed in a remote's configuration, while avoiding enableremote rejecting bad parameters that have already been stored in remote.log This does not eliminate every place where a remote config is parsed and a default value is used if the parse false. But, I did fix several things that expected foo=yes/no and so confusingly accepted foo=true but treated it like foo=no. There are still some fields that are parsed with yesNo but not not checked when initializing a remote, and there are other fields that are parsed in other ways and not checked when initializing a remote. This also lays groundwork for rejecting unknown/typoed config keys.
2020-01-10 18:10:20 +00:00
import Types.ProposedAccepted
a few forgotten remote config fields preferreddir can be used with any special remote, so its parser needs to be included in the commonFieldParsers. initremote with uuid= changed to delete that field, so it does not need to be included in commonFieldParsers. Note that, existing remotes initialized before this change will have the field in remote.log. This will not cause problems parsing, because the value will be Accepted. Grepping for 'Accepted "' found these, and I'm pretty sure this is all of them.
2020-01-15 15:22:36 +00:00
import Annex.SpecialRemote.Config (preferreddirField)
per-IA-item content directories
2013-04-26 03:44:55 +00:00
import qualified Data.Map as M
add PreferredContentExpression type
2014-01-01 23:58:02 +00:00
type PreferredContentExpression = String
manual and source repository groups
2013-01-21 08:18:05 +00:00
data StandardGroup
= ClientGroup
| TransferGroup
| BackupGroup
forgot to add to sum type
2013-03-16 16:09:22 +00:00
| IncrementalBackupGroup
manual and source repository groups
2013-01-21 08:18:05 +00:00
| SmallArchiveGroup
| FullArchiveGroup
| SourceGroup
| ManualGroup
Internet Archive! * Add public repository group. * webapp: Can now set up Internet Archive repositories. TODO: Enabling IA repositories.
2013-04-25 16:23:36 +00:00
| PublicGroup
Adjust built-in preferred content expressions to make most types of repositories want content that is only located on untrusted or dead repositories.
2013-03-31 21:10:25 +00:00
| UnwantedGroup
add standard group selector to repo edit form
2012-10-10 20:04:28 +00:00
deriving (Eq, Ord, Enum, Bounded, Show)
refactor
2012-10-10 19:15:56 +00:00
vicfg: Allows editing preferred content expressions for groups. This is stored in the git-annex branch, but not yet actually hooked up and used.
2014-03-15 20:17:01 +00:00
fromStandardGroup :: StandardGroup -> Group
newtype Group to ByteString This may speed up queries for things in groups, due to Eq and Ord being faster.
2019-01-09 19:00:43 +00:00
fromStandardGroup ClientGroup = Group "client"
fromStandardGroup TransferGroup = Group "transfer"
fromStandardGroup BackupGroup = Group "backup"
fromStandardGroup IncrementalBackupGroup = Group "incrementalbackup"
fromStandardGroup SmallArchiveGroup = Group "smallarchive"
fromStandardGroup FullArchiveGroup = Group "archive"
fromStandardGroup SourceGroup = Group "source"
fromStandardGroup ManualGroup = Group "manual"
fromStandardGroup PublicGroup = Group "public"
fromStandardGroup UnwantedGroup = Group "unwanted"
refactor
2012-10-10 19:15:56 +00:00
vicfg: Allows editing preferred content expressions for groups. This is stored in the git-annex branch, but not yet actually hooked up and used.
2014-03-15 20:17:01 +00:00
toStandardGroup :: Group -> Maybe StandardGroup
newtype Group to ByteString This may speed up queries for things in groups, due to Eq and Ord being faster.
2019-01-09 19:00:43 +00:00
toStandardGroup (Group "client") = Just ClientGroup
toStandardGroup (Group "transfer") = Just TransferGroup
toStandardGroup (Group "backup") = Just BackupGroup
toStandardGroup (Group "incrementalbackup") = Just IncrementalBackupGroup
toStandardGroup (Group "smallarchive") = Just SmallArchiveGroup
toStandardGroup (Group "archive") = Just FullArchiveGroup
toStandardGroup (Group "source") = Just SourceGroup
toStandardGroup (Group "manual") = Just ManualGroup
toStandardGroup (Group "public") = Just PublicGroup
toStandardGroup (Group "unwanted") = Just UnwantedGroup
refactor
2012-10-10 19:15:56 +00:00
toStandardGroup _ = Nothing
allow configuring the preferreddir
2013-04-26 17:00:14 +00:00
descStandardGroup :: StandardGroup -> String
descStandardGroup ClientGroup = "client: a repository on your computer"
descStandardGroup TransferGroup = "transfer: distributes files to clients"
descStandardGroup BackupGroup = "full backup: backs up all files"
descStandardGroup IncrementalBackupGroup = "incremental backup: backs up files not backed up elsewhere"
descStandardGroup SmallArchiveGroup = "small archive: archives files located in \"archive\" directories"
descStandardGroup FullArchiveGroup = "full archive: archives all files not archived elsewhere"
descStandardGroup SourceGroup = "file source: moves files on to other repositories"
descStandardGroup ManualGroup = "manual mode: only stores files you manually choose"
descStandardGroup UnwantedGroup = "unwanted: remove content from this repository"
descStandardGroup PublicGroup = "public: publishes files located in an associated directory"
per-IA-item content directories
2013-04-26 03:44:55 +00:00
allow configuring the preferreddir
2013-04-26 17:00:14 +00:00
associatedDirectory :: Maybe RemoteConfig -> StandardGroup -> Maybe FilePath
associatedDirectory _ SmallArchiveGroup = Just "archive"
associatedDirectory _ FullArchiveGroup = Just "archive"
associatedDirectory (Just c) PublicGroup = Just $
be stricter about rejecting invalid configurations for remotes This is a first step toward that goal, using the ProposedAccepted type in RemoteConfig lets initremote/enableremote reject bad parameters that were passed in a remote's configuration, while avoiding enableremote rejecting bad parameters that have already been stored in remote.log This does not eliminate every place where a remote config is parsed and a default value is used if the parse false. But, I did fix several things that expected foo=yes/no and so confusingly accepted foo=true but treated it like foo=no. There are still some fields that are parsed with yesNo but not not checked when initializing a remote, and there are other fields that are parsed in other ways and not checked when initializing a remote. This also lays groundwork for rejecting unknown/typoed config keys.
2020-01-10 18:10:20 +00:00
maybe "public" fromProposedAccepted $
a few forgotten remote config fields preferreddir can be used with any special remote, so its parser needs to be included in the commonFieldParsers. initremote with uuid= changed to delete that field, so it does not need to be included in commonFieldParsers. Note that, existing remotes initialized before this change will have the field in remote.log. This will not cause problems parsing, because the value will be Accepted. Grepping for 'Accepted "' found these, and I'm pretty sure this is all of them.
2020-01-15 15:22:36 +00:00
M.lookup preferreddirField c
allow configuring the preferreddir
2013-04-26 17:00:14 +00:00
associatedDirectory Nothing PublicGroup = Just "public"
associatedDirectory _ _ = Nothing
add standard group selector to repo edit form
2012-10-10 20:04:28 +00:00
webapp: Don't list the public repository group when editing a git repository; it only makes sense for special remotes.
2014-03-01 00:37:03 +00:00
specialRemoteOnly :: StandardGroup -> Bool
specialRemoteOnly PublicGroup = True
specialRemoteOnly _ = False
refactor
2012-10-10 19:15:56 +00:00
{- See doc/preferred_content.mdwn for explanations of these expressions. -}
finish wiring up groupwanted
2014-03-15 21:08:55 +00:00
standardPreferredContent :: StandardGroup -> PreferredContentExpression
standardPreferredContent ClientGroup = lastResort $
adjust standard preferred content to work better with git annex sync --all --content backup: Use new "anything" terminal. This means that content that is not unused, but has no associated file will be wanted by backup repos. unwanted: "not anything" will result in any and all content moving off of these repos. incremental backup: Remove the "(include=* or unused)", so it matches content that has no associated files but is not unused. client: Add a include=* to the expression. This limits it to matching only files in the work tree. Without this change, sync --all --content would match a key against the expression, and since it matches exclude=archive/*, the client repo would have wanted the file content. The "and not unused" would have kept unused objects out, but not objects that were not known to be unused, or objects that another branch referred to. In practice, everything would have flooded into client repos without this change.
2015-06-16 21:17:51 +00:00
"include=* and ((exclude=*/archive/* and exclude=archive/*) or (" ++ notArchived ++ "))"
finish wiring up groupwanted
2014-03-15 21:08:55 +00:00
standardPreferredContent TransferGroup = lastResort $
"not (inallgroup=client and copies=client:2) and (" ++ standardPreferredContent ClientGroup ++ ")"
adjust standard preferred content to work better with git annex sync --all --content backup: Use new "anything" terminal. This means that content that is not unused, but has no associated file will be wanted by backup repos. unwanted: "not anything" will result in any and all content moving off of these repos. incremental backup: Remove the "(include=* or unused)", so it matches content that has no associated files but is not unused. client: Add a include=* to the expression. This limits it to matching only files in the work tree. Without this change, sync --all --content would match a key against the expression, and since it matches exclude=archive/*, the client repo would have wanted the file content. The "and not unused" would have kept unused objects out, but not objects that were not known to be unused, or objects that another branch referred to. In practice, everything would have flooded into client repos without this change.
2015-06-16 21:17:51 +00:00
standardPreferredContent BackupGroup = "anything"
finish wiring up groupwanted
2014-03-15 21:08:55 +00:00
standardPreferredContent IncrementalBackupGroup = lastResort
adjust standard preferred content to work better with git annex sync --all --content backup: Use new "anything" terminal. This means that content that is not unused, but has no associated file will be wanted by backup repos. unwanted: "not anything" will result in any and all content moving off of these repos. incremental backup: Remove the "(include=* or unused)", so it matches content that has no associated files but is not unused. client: Add a include=* to the expression. This limits it to matching only files in the work tree. Without this change, sync --all --content would match a key against the expression, and since it matches exclude=archive/*, the client repo would have wanted the file content. The "and not unused" would have kept unused objects out, but not objects that were not known to be unused, or objects that another branch referred to. In practice, everything would have flooded into client repos without this change.
2015-06-16 21:17:51 +00:00
"(not copies=backup:1) and (not copies=incrementalbackup:1)"
finish wiring up groupwanted
2014-03-15 21:08:55 +00:00
standardPreferredContent SmallArchiveGroup = lastResort $
"(include=*/archive/* or include=archive/*) and (" ++ standardPreferredContent FullArchiveGroup ++ ")"
standardPreferredContent FullArchiveGroup = lastResort notArchived
standardPreferredContent SourceGroup = "not (copies=1)"
standardPreferredContent ManualGroup = "present and (" ++ standardPreferredContent ClientGroup ++ ")"
standardPreferredContent PublicGroup = "inpreferreddir"
adjust standard preferred content to work better with git annex sync --all --content backup: Use new "anything" terminal. This means that content that is not unused, but has no associated file will be wanted by backup repos. unwanted: "not anything" will result in any and all content moving off of these repos. incremental backup: Remove the "(include=* or unused)", so it matches content that has no associated files but is not unused. client: Add a include=* to the expression. This limits it to matching only files in the work tree. Without this change, sync --all --content would match a key against the expression, and since it matches exclude=archive/*, the client repo would have wanted the file content. The "and not unused" would have kept unused objects out, but not objects that were not known to be unused, or objects that another branch referred to. In practice, everything would have flooded into client repos without this change.
2015-06-16 21:17:51 +00:00
standardPreferredContent UnwantedGroup = "not anything"
better archive directory handling Adjust preferred content expressions so that content in archive directories is preferred until it has reached an archive or smallarchive repository.
2013-04-06 22:29:52 +00:00
notArchived :: String
notArchived = "not (copies=archive:1 or copies=smallarchive:1)"
fix some mixed space+tab indentation This fixes all instances of " \t" in the code base. Most common case seems to be after a "where" line; probably vim copied the two space layout of that line. Done as a background task while listening to episode 2 of the Type Theory podcast.
2014-10-09 18:53:13 +00:00
move content from remote when user asks to delete it
2013-03-31 23:00:43 +00:00
{- Most repositories want any content that is only on untrusted
benchmarked numcopies .gitattributes in preferred content Checking .gitattributes adds a full minute to a git annex find looking for files that don't have enough copies. 2:25 increasts to 3:27. I feel this is too much of a slowdown to justify making it the default. So, exposed two versions of the preferred content expression, a slow one and a fast but approximate one. I'm using the approximate one in the default preferred content expressions to avoid slowing down the assistant.
2014-01-21 22:46:39 +00:00
- or dead repositories, or that otherwise does not have enough copies.
- Does not look at .gitattributes since that is quite a lot slower.
-}
add PreferredContentExpression type
2014-01-01 23:58:02 +00:00
lastResort :: String -> PreferredContentExpression
benchmarked numcopies .gitattributes in preferred content Checking .gitattributes adds a full minute to a git annex find looking for files that don't have enough copies. 2:25 increasts to 3:27. I feel this is too much of a slowdown to justify making it the default. So, exposed two versions of the preferred content expression, a slow one and a fast but approximate one. I'm using the approximate one in the default preferred content expressions to avoid slowing down the assistant.
2014-01-21 22:46:39 +00:00
lastResort s = "(" ++ s ++ ") or approxlackingcopies=1"
Reference in a new issue Copy permalink