Merge branch 'master' into tor
This commit is contained in:
commit
398345cb26
35 changed files with 697 additions and 20 deletions
|
@ -2,7 +2,7 @@
|
|||
-
|
||||
- Copyright 2013 Joey Hess <id@joeyh.name>
|
||||
-
|
||||
- Licensed under the GNU AGPL version 3 or higher.
|
||||
- Licensed under the GNU GPL version 3 or higher.
|
||||
-}
|
||||
|
||||
module Assistant.Fsck where
|
||||
|
|
|
@ -2,7 +2,7 @@
|
|||
-
|
||||
- Copyright 2013 Joey Hess <id@joeyh.name>
|
||||
-
|
||||
- Licensed under the GNU AGPL version 3 or higher.
|
||||
- Licensed under the GNU GPL version 3 or higher.
|
||||
-}
|
||||
|
||||
module Assistant.Gpg where
|
||||
|
|
|
@ -2,7 +2,7 @@
|
|||
-
|
||||
- Copyright 2013 Joey Hess <id@joeyh.name>
|
||||
-
|
||||
- Licensed under the GNU AGPL version 3 or higher.
|
||||
- Licensed under the GNU GPL version 3 or higher.
|
||||
-}
|
||||
|
||||
{-# LANGUAGE CPP #-}
|
||||
|
|
|
@ -2,7 +2,7 @@
|
|||
-
|
||||
- Copyright 2013 Joey Hess <id@joeyh.name>
|
||||
-
|
||||
- Licensed under the GNU AGPL version 3 or higher.
|
||||
- Licensed under the GNU GPL version 3 or higher.
|
||||
-}
|
||||
|
||||
{-# LANGUAGE CPP #-}
|
||||
|
|
|
@ -2,7 +2,7 @@
|
|||
-
|
||||
- Copyright 2013 Joey Hess <id@joeyh.name>
|
||||
-
|
||||
- Licensed under the GNU AGPL version 3 or higher.
|
||||
- Licensed under the GNU GPL version 3 or higher.
|
||||
-}
|
||||
|
||||
{-# LANGUAGE CPP #-}
|
||||
|
|
|
@ -6,8 +6,12 @@ git-annex (6.20161119) UNRELEASED; urgency=medium
|
|||
hidden service.
|
||||
* remotedaemon: Fork to background by default. Added --foreground switch
|
||||
to enable old behavior.
|
||||
* addurl: Fix bug in checking annex.largefiles expressions using
|
||||
largerthan, mimetype, and smallerthan; the first two always failed
|
||||
to match, and the latter always matched.
|
||||
* Relicense 5 source files that are not part of the webapp from AGPL to GPL.
|
||||
|
||||
-- Joey Hess <id@joeyh.name> Sun, 20 Nov 2016 14:10:15 -0400
|
||||
-- Joey Hess <id@joeyh.name> Mon, 21 Nov 2016 11:27:50 -0400
|
||||
|
||||
git-annex (6.20161118) unstable; urgency=medium
|
||||
|
||||
|
|
|
@ -340,13 +340,18 @@ cleanup :: UUID -> URLString -> FilePath -> Key -> Maybe FilePath -> Annex ()
|
|||
cleanup u url file key mtmp = case mtmp of
|
||||
Nothing -> go
|
||||
Just tmp -> do
|
||||
-- Move to final location for large file check.
|
||||
liftIO $ renameFile tmp file
|
||||
largematcher <- largeFilesMatcher
|
||||
ifM (checkFileMatcher largematcher file)
|
||||
( go
|
||||
, do
|
||||
liftIO $ renameFile tmp file
|
||||
void $ Command.Add.addSmall file
|
||||
)
|
||||
large <- checkFileMatcher largematcher file
|
||||
if large
|
||||
then do
|
||||
-- Move back to tmp because addAnnexedFile
|
||||
-- needs the file in a different location
|
||||
-- than the work tree file.
|
||||
liftIO $ renameFile file tmp
|
||||
go
|
||||
else void $ Command.Add.addSmall file
|
||||
where
|
||||
go = do
|
||||
maybeShowJSON $ JSONChunk [("key", key2file key)]
|
||||
|
|
|
@ -44,7 +44,7 @@ perform :: FilePath -> Key -> Key -> CommandPerform
|
|||
perform file oldkey newkey = do
|
||||
ifM (inAnnex oldkey)
|
||||
( unlessM (linkKey file oldkey newkey) $
|
||||
error "failed"
|
||||
giveup "failed"
|
||||
, unlessM (Annex.getState Annex.force) $
|
||||
giveup $ file ++ " is not available (use --force to override)"
|
||||
)
|
||||
|
|
|
@ -29,7 +29,7 @@ start = parse
|
|||
where
|
||||
parse (name:[]) = go name performGet
|
||||
parse (name:expr:[]) = go name $ \uuid -> do
|
||||
showStart "schedile" name
|
||||
showStart "schedule" name
|
||||
performSet expr uuid
|
||||
parse _ = giveup "Specify a repository."
|
||||
|
||||
|
|
|
@ -0,0 +1,34 @@
|
|||
[[!comment format=mdwn
|
||||
username="boh"
|
||||
avatar="http://cdn.libravatar.org/avatar/e7fa2d1c5d95e323fe48887f7f827b1f"
|
||||
subject="comment 9"
|
||||
date="2016-11-27T12:23:20Z"
|
||||
content="""
|
||||
Seems as if the problem still exists in 6.20161118 (Debian).
|
||||
|
||||
I have three repositories (among others), `jolla`, `sts-3xx`, and `here`. `jolla` and `here` are in group `manual`, `sts-3xx` is `backup`; `here` and `sts-3xx` have assistants running, `jolla` not. `jolla` and `sts-3xx` have slightly older versions of git-annex installed.
|
||||
|
||||
Now, when I copy a file from `here` to `jolla` like this
|
||||
|
||||
git annex copy real_programmers.png -t jolla
|
||||
|
||||
the file is subsequently dropped by the assistant:
|
||||
|
||||
```
|
||||
drop real_programmers.png (locking jolla...) [2016-11-27 13:00:02.667376556] chat: ssh [\"-S\",\".git/annex/ssh/jolla\",\"-o\",\"ControlMaster
|
||||
=auto\",\"-o\",\"ControlPersist=yes\",\"-F\",\".git/annex/ssh.config\",\"-T\",\"jolla\",\"git-annex-shell 'lockcontent' '/~/Music/media/' '--debug' '
|
||||
SHA256E-s84499--ff98a733cc0122858fb11433c720e2d038fec190a3d36380d0e7e8dab468f883.png' --uuid 5298e3ce-1106-4d5e-b052-0aee4b27a344\"]
|
||||
(locking sts-3xx...) [2016-11-27 13:00:03.252473676] chat: ssh [..., \"git-annex-shell 'lockcontent' '/backups/exot/media/' '--debug' 'SHA256E-s84499--ff98a733cc0122858fb11433c720e2d038fec190a3d 36380d0e7e8dab468f883.png' --uuid 1fec6253-171d-4f86-885b-e233be2d65ec\"]
|
||||
(lockcontent failed) [2016-11-27 13:00:03.486158016] process done ExitFailure 1
|
||||
(checking sts-3xx...) [2016-11-27 13:00:03.487047149] call: ssh [..., \"git-annex-shell 'inannex' '/backups/exot/media/' '--debug' 'SHA256E-s84499--ff98a733cc0122858fb11433c720e2d038fec190a3d363 80d0e7e8dab468f883.png' --uuid 1fec6253-171d-4f86-885b-e233be2d65ec\"]
|
||||
[2016-11-27 13:00:03.76435136] process done ExitSuccess
|
||||
[2016-11-27 13:00:03.764705754] Dropping from here proof: Just (SafeDropProof (NumCopies 2) [RecentlyVerifiedCopy UUID \"1fec6253-171d-4 f86-885b-e233be2d65ec\",LockedCopy UUID \"5298e3ce-1106-4d5e-b052-0aee4b27a344\"] (Just (ContentRemovalLock (Key {keyName = \"ff98a733cc012 2858fb11433c720e2d038fec190a3d36380d0e7e8dab468f883.png\", keyBackendName = \"SHA256E\", keySize = Just 84499, keyMtime = Nothing, keyChun kSize = Nothing, keyChunkNum = Nothing}))))
|
||||
[2016-11-27 13:00:04.24333081] process done ExitFailure 1
|
||||
ok
|
||||
[2016-11-27 13:00:04.251232455] dropped real_programmers.png (from here) (copies now 4) : drop wanted after Upload UUID \"5298e3ce-1106- 4d5e-b052-0aee4b27a344\" real_programmers.png Just 84499
|
||||
```
|
||||
|
||||
However, I failed to reproduce the problem by replicating my setup with fresh repositories …
|
||||
|
||||
Please let me know if you need more information, and *so* many thanks for git-annex!
|
||||
"""]]
|
48
doc/bugs/Build_with_aws_head_fails.mdwn
Normal file
48
doc/bugs/Build_with_aws_head_fails.mdwn
Normal file
|
@ -0,0 +1,48 @@
|
|||
### Please describe the problem.
|
||||
https://github.com/aristidb/aws/issues/206 was recently resolved in https://github.com/aristidb/aws/pull/213.
|
||||
|
||||
A newer version will be tagged imminently according to https://github.com/aristidb/aws/issues/206#issuecomment-260214736.
|
||||
|
||||
With the http-conduit (<2.2.0) constraint removed from git-annex.cabal, and the aws dependency set to use aws head (currently c8806dc), the git-annex build fails.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
Remove the http-conduit (<2.2.0) constraint and attempt to build git-annex with aws head.
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
macOS 10.11, git-annex 6.20161118.
|
||||
|
||||
### Please provide any additional information below.
|
||||
Full build log: https://gist.github.com/ilovezfs/15bcd8f1086b3d825beff58140e04eec
|
||||
[[!format sh """
|
||||
[ 90 of 542] Compiling Types.Crypto ( Types/Crypto.hs, dist/dist-sandbox-6b15e8f0/build/git-annex/git-annex-tmp/Types/Crypto.o )
|
||||
[ 91 of 542] Compiling Utility.Metered ( Utility/Metered.hs, dist/dist-sandbox-6b15e8f0/build/git-annex/git-annex-tmp/Utility/Metered.o )
|
||||
[ 92 of 542] Compiling Messages.JSON ( Messages/JSON.hs, dist/dist-sandbox-6b15e8f0/build/git-annex/git-annex-tmp/Messages/JSON.o )
|
||||
[ 93 of 542] Compiling Utility.Url ( Utility/Url.hs, dist/dist-sandbox-6b15e8f0/build/git-annex/git-annex-tmp/Utility/Url.o )
|
||||
|
||||
Utility/Url.hs:354:34: error:
|
||||
• The constructor ‘StatusCodeException’ should have 2 arguments, but has been given 3
|
||||
• In the pattern: StatusCodeException s _ _
|
||||
In an equation for ‘matchStatusCodeException’:
|
||||
matchStatusCodeException want e@(StatusCodeException s _ _)
|
||||
| want s = Just e
|
||||
| otherwise = Nothing
|
||||
|
||||
Utility/Url.hs:354:34: error:
|
||||
• Couldn't match expected type ‘HttpException’
|
||||
with actual type ‘HttpExceptionContent’
|
||||
• In the pattern: StatusCodeException s _ _
|
||||
In an equation for ‘matchStatusCodeException’:
|
||||
matchStatusCodeException want e@(StatusCodeException s _ _)
|
||||
| want s = Just e
|
||||
| otherwise = Nothing
|
||||
cabal: Leaving directory '.'
|
||||
cabal: Error: some packages failed to install:
|
||||
git-annex-6.20161118 failed during the building phase. The exception was:
|
||||
ExitFailure 1
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
Yes :)
|
||||
|
|
@ -0,0 +1,53 @@
|
|||
### Please describe the problem.
|
||||
|
||||
I'm seeing some inconsistent results between runs of `git annex fsck` and `git annex whereis` that I'm not able to explain. When I run `git annex fsck`, it reports a few keys that only have 1 copy, and advises me to make more copies. If I run `git annex whereis --key <key>`, git annex confirms that it only knows about 1 copy of this key. If I then use `git log --stat -S'<key>'` to find the actual file that it refers to, and run `git annex whereis <file>`, git annex report 9 copies of this file. Checking on remotes shows that these files do exist on the remote, so why does `git annex fsck` and `git annex whereis` mis-report the number of copies when querying for the key - but not for the actual filename? Additionally, `git annex find --lackingcopies 1` doesn't return any results, but should if there are actually files with not enough copies?
|
||||
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
5.20151208-1build1 on Ubuntu Xenial, one remote running 5.20141024~bpo70+1 on Debian Wheezy
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
|
||||
[william@hactar ~/Pictures/Photo Library]$ git annex whereis SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9
|
||||
git-annex: SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9 not found
|
||||
git-annex: whereis: 1 failed
|
||||
[william@hactar ~/Pictures/Photo Library]$ git annex whereis --key SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9
|
||||
whereis SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9 (1 copy)
|
||||
7691934f-2542-4103-9122-2db4e6cfc887 -- hactar [here]
|
||||
ok
|
||||
[william@hactar ~/Pictures/Photo Library]$ git annex fsck --key SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9
|
||||
fsck SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9
|
||||
Only 1 of 3 trustworthy copies exist of SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9
|
||||
Back it up with git-annex copy.
|
||||
failed
|
||||
(recording state in git...)
|
||||
git-annex: fsck: 1 failed
|
||||
[william@hactar ~/Pictures/Photo Library]$ git log --stat -S'SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9'
|
||||
[william@hactar ~/Pictures/Photo Library]$ git annex whereis 2009/05/05/P1040890.JPG
|
||||
whereis 2009/05/05/P1040890.JPG (9 copies)
|
||||
0e825a69-1927-4f62-b731-6f3e98bba998 -- william@marvin:/media/backup/annex/photos [marvin]
|
||||
1b728ab5-1e32-45a6-bc11-2a4bfdc9d6ab -- backup1
|
||||
5c0caa42-b489-467b-a612-9590fa9d5a94 -- backup2
|
||||
7691934f-2542-4103-9122-2db4e6cfc887 -- hactar [here]
|
||||
894b2216-72e0-40e1-8765-1386e1e9e4b4 -- backup3
|
||||
96f19fa8-d385-4e8b-b000-61ee15993a70 -- backup3
|
||||
a862b121-d794-4af4-bb56-21adfe8962f2 -- S3
|
||||
b083f8ae-42fb-41f0-a2a3-4e7c9f93aadb -- [guide]
|
||||
bf021ce9-465b-4419-86e7-bddfd208fca4 -- git@newzaphod:~/repositories/annex/photos.git [zaphod]
|
||||
ok
|
||||
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
I trust Git Annex to keep hundreds of GB of data safe, and it has never failed me - despite my best efforts
|
|
@ -0,0 +1,17 @@
|
|||
[[!comment format=mdwn
|
||||
username="justin.lebar@7a36fcafc322d9a381e89f08ab6289033c6dde91"
|
||||
nickname="justin.lebar"
|
||||
avatar="http://cdn.libravatar.org/avatar/9fca4b61a1ab555f231851e7543f9a3e"
|
||||
subject="comment 2"
|
||||
date="2016-11-20T03:47:23Z"
|
||||
content="""
|
||||
Thanks for your reply, Joey. Sorry for the delay getting back to this -- I didn't realize I hadn't enabled notifications on the thread.
|
||||
|
||||
The GCS docs suggest that 400 errors should be accompanied by an explanation in the reply body.
|
||||
|
||||
> Error responses usually include a JSON document in the response body, which contains information about the error.
|
||||
|
||||
https://cloud.google.com/storage/docs/json_api/v1/status-codes
|
||||
|
||||
Do you think we're not getting an http response body here, or that it's not being printed out?
|
||||
"""]]
|
|
@ -0,0 +1,16 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://launchpad.net/~stephane-gourichon-lpad"
|
||||
nickname="stephane-gourichon-lpad"
|
||||
avatar="http://cdn.libravatar.org/avatar/02d4a0af59175f9123720b4481d55a769ba954e20f6dd9b2792217d9fa0c6089"
|
||||
subject="Known bug, fixed."
|
||||
date="2016-11-23T18:04:27Z"
|
||||
content="""
|
||||
This is a known bug introduced in 6.20161012 and fixed in 6.20161031.
|
||||
|
||||
Solution is: just update your copy of git-annex. At this time most recent is 6.20161119 .
|
||||
|
||||
For more details, see changelog at https://github.com/joeyh/git-annex/blob/master/CHANGELOG#L53
|
||||
|
||||
|
||||
|
||||
"""]]
|
|
@ -0,0 +1,12 @@
|
|||
[[!comment format=mdwn
|
||||
username="t.z.mates"
|
||||
avatar="http://cdn.libravatar.org/avatar/90f15fad216078fd08d62cc676487925"
|
||||
subject="comment 2"
|
||||
date="2016-11-19T04:42:25Z"
|
||||
content="""
|
||||
Thanks for looking into it; I just checked again, and even on the newest version (6.20161118 binary), I'm still experiencing the behavior. However, I checked on an older OpenSuse box I have, and there it works (6.20161031 from OpenSuse repo).
|
||||
|
||||
Since my two machines experiencing the problem are both running arch, it seems it's somehow related to that distro. I've checked both installing via the binary (from kitenet) and from the arch community repo, but both produce the same behavior. Further, the OpenSuse install has the same build flags as the binaries, so that doesn't seem to be it. Are there any other diagnostics I can run?
|
||||
|
||||
This particular problem isn't very troublesome (it doesn't seem to have any material impact aside from error messages); however, I also occasionally experience a more serious bug. Namely, when certain (seemingly random) files are added to the repo locked, their content disappears and the symlink is broken (this is the other problem I alluded to in the description). I suspect that problem is related to this one though, since it also only affects my arch machines. I haven't yet submitted a report for that bug yet, though, since I can't reliably replicate it.
|
||||
"""]]
|
|
@ -0,0 +1,41 @@
|
|||
### Please describe the problem.
|
||||
|
||||
When addurl'ing a big file with .gitattributes configured to add only some files directly into git (and 'git annex add' operating correctly), addurl adds large files straight into git.
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
git-annex version: 6.20161018+gitgf3c366a-1~ndall+1
|
||||
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
$> cat .gitattributes
|
||||
* annex.backend=MD5E
|
||||
* annex.largefiles=(largerthan=100kb)
|
||||
*.json annex.largefiles=nothing
|
||||
*.txt annex.largefiles=nothing
|
||||
*.tsv annex.largefiles=nothing
|
||||
*.nii.gz annex.largefiles=(largerthan=0kb)
|
||||
*.tgz annex.largefiles=(largerthan=0kb)
|
||||
*.tar.gz annex.largefiles=(largerthan=0kb)
|
||||
*.gz annex.largefiles=(largerthan=0kb)
|
||||
|
||||
$> git annex addurl http://fcp-indi.s3.amazonaws.com/data/Projects/HBNSSI/RawDataTars/sub-0031121_baseline.tar.gz\?versionId\=7FvexHgyazWF.dUo238FA7XRiK0FWQDw.
|
||||
addurl fcp_indi.s3.amazonaws.com_data_Projects_HBNSSI_RawDataTars_sub_0031121_baseline.tar.gz_versionId_7FvexHgyazWF.dUo238FA7XRiK0FWQDw. (downloading http://fcp-indi.s3.amazonaws.com/data/Projects/HBNSSI/RawDataTars/sub-0031121_baseline.tar.gz?versionId=7FvexHgyazWF.dUo238FA7XRiK0FWQDw. ...)
|
||||
/mnt/btrfs/datasets/datalad/crawl-misc/indi/ 100%[==============================================================================================>] 195.44M 21.2MB/s in 12s
|
||||
(non-large file; adding content to git repository) ok
|
||||
(recording state in git...)
|
||||
cached/staged changes:
|
||||
\u2026r.gz_versionId_7FvexHgyazWF.dUo238FA7XRiK0FWQDw. | Bin 0 -> 204937338 bytes
|
||||
|
||||
$> ls -l fcp_indi.s3.amazonaws.com_data_Projects_HBNSSI_RawDataTars_sub_0031121_baseline.tar.gz_versionId_7FvexHgyazWF.dUo238FA7XRiK0FWQDw.
|
||||
-rw------- 1 yoh datalad 204937338 Oct 25 17:30 fcp_indi.s3.amazonaws.com_data_Projects_HBNSSI_RawDataTars_sub_0031121_baseline.tar.gz_versionId_7FvexHgyazWF.dUo238FA7XRiK0FWQDw.
|
||||
cached/staged changes:
|
||||
\u2026r.gz_versionId_7FvexHgyazWF.dUo238FA7XRiK0FWQDw. | Bin 0 -> 204937338 bytes
|
||||
|
||||
"""]]
|
||||
|
||||
[[!meta author=yoh]]
|
||||
|
||||
> [[fixed|done]] --[[Joey]]
|
|
@ -0,0 +1,20 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 1"""
|
||||
date="2016-11-21T15:12:54Z"
|
||||
content="""
|
||||
It's sufficient to have "* annex.largefiles=(largerthan=100kb)"
|
||||
in .gitattributes.
|
||||
|
||||
Even "* annex.largefiles=(largerthan=0kb)" will reproduce it.
|
||||
|
||||
Ok, I see why.. It's running the largefile matcher on the destination file
|
||||
before it renames the temp file to it!
|
||||
|
||||
Seems to have been broken this way ever since addurl got largefiles
|
||||
support. Testing didn't catch it because it only affects largefiles
|
||||
expressions that need to examine the file.
|
||||
|
||||
Fixed in git. Audited other checkFileMatcher calls for this problem;
|
||||
the rest are ok.
|
||||
"""]]
|
|
@ -0,0 +1,10 @@
|
|||
[[!comment format=mdwn
|
||||
username="CandyAngel"
|
||||
avatar="http://cdn.libravatar.org/avatar/15c0aade8bec5bf004f939dd73cf9ed8"
|
||||
subject="comment 4"
|
||||
date="2016-11-25T20:27:07Z"
|
||||
content="""
|
||||
I really don't know what to say. I can't even figure out which computer I updated git-annex on to test if it was still happening.. let alone reproduce it anymore. It does work fine.
|
||||
|
||||
I'm so sorry to bother you with this, I've done something stupid! This is exactly why you ask for a transcript of bugs occurring. (Feel free to use this as an example for why you ask for them, so some good can come of it at least..).
|
||||
"""]]
|
|
@ -0,0 +1,30 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://launchpad.net/~felixonmars"
|
||||
nickname="felixonmars"
|
||||
avatar="http://cdn.libravatar.org/avatar/17284a3bb2e4ad9d3be8fab31f49865be9c1dc22143c728de731fe800a335d38"
|
||||
subject="comment 1"
|
||||
date="2016-11-28T04:17:12Z"
|
||||
content="""
|
||||
aws has merged a PR to support http-conduit 2.2, but git-annex itself doesn't build with the new component yet:
|
||||
|
||||
```
|
||||
[ 95 of 544] Compiling Utility.Url ( Utility/Url.hs, dist/build/git-annex/git-annex-tmp/Utility/Url.o )
|
||||
|
||||
Utility/Url.hs:354:34: error:
|
||||
* The constructor `StatusCodeException' should have 2 arguments, but has been given 3
|
||||
* In the pattern: StatusCodeException s _ _
|
||||
In an equation for `matchStatusCodeException':
|
||||
matchStatusCodeException want e@(StatusCodeException s _ _)
|
||||
| want s = Just e
|
||||
| otherwise = Nothing
|
||||
|
||||
Utility/Url.hs:354:34: error:
|
||||
* Couldn't match expected type `HttpException'
|
||||
with actual type `HttpExceptionContent'
|
||||
* In the pattern: StatusCodeException s _ _
|
||||
In an equation for `matchStatusCodeException':
|
||||
matchStatusCodeException want e@(StatusCodeException s _ _)
|
||||
| want s = Just e
|
||||
| otherwise = Nothing
|
||||
```
|
||||
"""]]
|
34
doc/bugs/git-annex_fromkey_barfs_on_utf-8_input.mdwn
Normal file
34
doc/bugs/git-annex_fromkey_barfs_on_utf-8_input.mdwn
Normal file
|
@ -0,0 +1,34 @@
|
|||
### Please describe the problem.
|
||||
|
||||
I'm sending a stream of keys and filenames to git-annex fromkey on stdin, and it errors out with "git-annex: <stdin>: hGetContents: invalid argument (invalid byte sequence)". On the other hand yipdw tried to reproduce this and it worked fine for him, so I must be doing something wrong.
|
||||
|
||||
I have LANG=en_US.UTF-8 set in my environment, if that matters.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
[[!format sh """
|
||||
echo "MD5-s3263532--0b4d070eff7baa8ef314ca330aecb71f é" | git-annex fromkey
|
||||
"""]]
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
[[!format sh """
|
||||
git-annex version: 6.20161118-g0a34f08
|
||||
build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify XMPP ConcurrentOutput TorrentParser MagicMime Feeds Quvi
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
|
||||
remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
|
||||
local repository version: 5
|
||||
supported repository versions: 3 5 6
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5
|
||||
operating system: linux x86_64
|
||||
"""]]
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
Note that this is indeed valid utf-8:
|
||||
|
||||
[[!format sh """
|
||||
db48x ~ projects IA.BAK-server echo "é" | hexdump -C
|
||||
00000000 c3 a9 0a |...|
|
||||
00000003
|
||||
"""]]
|
|
@ -0,0 +1,17 @@
|
|||
[[!comment format=mdwn
|
||||
username="scottgorlin@a32946b2aad278883c1690a0753241583a9855b9"
|
||||
nickname="scottgorlin"
|
||||
avatar="http://cdn.libravatar.org/avatar/2dd1fc8add62bbf4ffefac081b322563"
|
||||
subject="IgnoreUnknown Include considered harmful?"
|
||||
date="2016-11-23T20:07:45Z"
|
||||
content="""
|
||||
As noted, include appears to not work on a mac at the moment. This means git-annex silently ignores the included configs, which may be required to ssh to the remotes of interest. This is happening to me.
|
||||
|
||||
My understanding is that ssh aliases are the recommended way of juggling multiple private keys amongst multiple hosts, so it is a required part of many git workflows. In this particular case, I have set up git annex on a NAS which does not allow multiple ssh users (QNAP) and the authentication is done only via key identity, not username. Thus, host aliases are necessary.
|
||||
|
||||
If one config can't include another, I would prefer an early failure indicating a problem with the config file, or better, a solution where git-annex doesn't require a config. In this scenario, git fetch remote_name and git annex copy --to remotename do not resolve to the same alias definitions (the latter is missing because of the ignored config!).
|
||||
|
||||
I got my setup to work only by finding and manually editing <repo>/.git/annex/ssh_config, which to my knowledge is undocumented (ie when is it written? do any commands change it?); manual mucking around inside .git to me is not a good practice, and for now I have two different alias's defined (in repo and in ~/.ssh/config)
|
||||
|
||||
|
||||
"""]]
|
|
@ -0,0 +1,15 @@
|
|||
[[!comment format=mdwn
|
||||
username="palday@91f366b5178879146d2b6e1e53bfa21389ee89a8"
|
||||
nickname="palday"
|
||||
avatar="http://cdn.libravatar.org/avatar/077a63af75ddba159980fbf88690f401"
|
||||
subject="Temporary workaround until the brew formula is updated"
|
||||
date="2016-11-29T02:17:52Z"
|
||||
content="""
|
||||
The homebrew formula doesn't yet this fix, but you can get around the problem in the meantime by getting a newer SSH via homebrew:
|
||||
|
||||
```
|
||||
brew install homebrew/dupes/openssh
|
||||
```
|
||||
|
||||
You can then choose to keep that or get rid of it when the formula for git annex is later updated.
|
||||
"""]]
|
31
doc/devblog/day_428-429__git_push_to_hiddden_service.mdwn
Normal file
31
doc/devblog/day_428-429__git_push_to_hiddden_service.mdwn
Normal file
|
@ -0,0 +1,31 @@
|
|||
The `tor` branch is coming along nicely.
|
||||
|
||||
This weekend, I continued working on the P2P protocol, implementing
|
||||
it for network sockets, and extending it to support connecting up
|
||||
git-send-pack/git-receive-pack.
|
||||
|
||||
There was a bit of a detour when I split the Free monad into two separate
|
||||
ones, one for Net operations and the other for Local filesystem operations.
|
||||
|
||||
This weekend's work was sponsored by Thomas Hochstein on Patreon.
|
||||
|
||||
----
|
||||
|
||||
Today, implemented a `git-remote-tor-annex` command that git will
|
||||
use for tor-annex:: urls, and made `git annex remotedaemon`
|
||||
serve the tor hidden service.
|
||||
|
||||
Now I have git push/pull working to the hidden service, for example:
|
||||
|
||||
git pull tor-annex::eeaytkuhaupbarfi.onion:47651
|
||||
|
||||
That works very well, but does not yet check that the user is authorized
|
||||
to use the repo, beyond knowing the onion address. And currently
|
||||
it only works in git-annex repos; with some tweaks it should
|
||||
also work in plain git repos.
|
||||
|
||||
Next, I need to teach git-annex how to access tor-annex remotes.
|
||||
And after that, an interface in the webapp for setting them up and
|
||||
connecting them together.
|
||||
|
||||
Today's work was sponsored by Josh Taylor on Patreon.
|
41
doc/forum/Git-annex_link_to_different_file_names.mdwn
Normal file
41
doc/forum/Git-annex_link_to_different_file_names.mdwn
Normal file
|
@ -0,0 +1,41 @@
|
|||
This is a recreation of a stackexchange question, in case the community here is more knowledgeable.
|
||||
|
||||
Link to stackexchange question : http://unix.stackexchange.com/questions/325753/git-annex-link-to-different-file-names
|
||||
|
||||
Content :
|
||||
"Maybe this is just a crazy use case that doesn't work, but I was wondering if there's a way to build a file's history from files with different file names. I'm exploring this idea because I'd like to have a git-annex system but I can't force my coworkers to adapt.
|
||||
|
||||
Here's what I have in mind :
|
||||
|
||||
Folder 1, managed by coworkers (On a shared disk) :
|
||||
|
||||
drawing_shop_12_nov_2015.pdf
|
||||
drawing_shop_13_nov_2015.pdf
|
||||
drawing_asbuilt_14_nov_2015.pdf
|
||||
drawing_asbuilt_rev1_15_nov_2015.pdf
|
||||
|
||||
And
|
||||
|
||||
Git-annex, managed by me :
|
||||
|
||||
drawing.pdf
|
||||
|
||||
(with a shop branch and a asbuilt branch)
|
||||
|
||||
The git-annex's drawing.pdf would have an history like this :
|
||||
|
||||
[shop]
|
||||
|
|
||||
Commit A "Initial shop drawing"
|
||||
|
|
||||
Commit B "Add corrections from Wizzbasket"
|
||||
\
|
||||
|
|
||||
[asbuilt]
|
||||
Commit C "Reflect as built"
|
||||
|
|
||||
Commit D "Change dweezelbox block for simplicity"
|
||||
|
||||
But somehow the "managed by coworkers" repo would be a direct mode repo with Commit A pointing to drawing_shop_12_nov_2015.pdf, Commit B to drawing_shop_13_nov_2015.pdf etc.
|
||||
|
||||
Can this be done?"
|
27
doc/forum/Odd_Hybrid_Symlinks_To_Content.mdwn
Normal file
27
doc/forum/Odd_Hybrid_Symlinks_To_Content.mdwn
Normal file
|
@ -0,0 +1,27 @@
|
|||
I've somehow managed to get my indirect repository to symlink to literal content instead of object files.
|
||||
|
||||
By this I mean literally the symlink is pointing at the contents of the file as the filename.
|
||||
|
||||
So if I have a blah.txt file with this content:
|
||||
|
||||
* First line
|
||||
* second line
|
||||
|
||||
And I ls -al to view the symlink pointer, it shows up as this:
|
||||
|
||||
* blah.txt -> First line?second line
|
||||
|
||||
It literally has the contents of the file as the destination filename.
|
||||
|
||||
I've tried a couple things I could think of to re-symlink the files, but they don't seem to do anything as they think everything is fine:
|
||||
|
||||
* git annex indirect //returns nothing
|
||||
* git annex lock blah.txt //returns nothing
|
||||
* git annex fix blah.txt //returns nothing
|
||||
* git annex fsck //returns nothing
|
||||
|
||||
I'm actually able to find several of these files hanging around by searching for all symlinks that don't point to something in the .git directory.
|
||||
|
||||
Is there a way for me to replace the symlinks with correct symlinks to the objects in .git/annex? Can it even figure out which ones it was supposed to point to if the symlinks are messed up (are filenames -> content hashes stored anywhere else)?
|
||||
|
||||
Else I might have to go do some manual rebasing and history editing to try to undo the bad commits manually. I've synced this repo to another direct repo so I'll need to figure out how to manually fix that repo too (using proxy). From what I can tell the annex/direct/master seems to be same as master and synced/master branches? Is there an [[internals]] page for direct branches besides [[direct_mode]] so I know what should be fixed where?
|
47
doc/forum/Preserving_Directories_in_Metadata_Views.mdwn
Normal file
47
doc/forum/Preserving_Directories_in_Metadata_Views.mdwn
Normal file
|
@ -0,0 +1,47 @@
|
|||
I want to use metadata views to sort files into top-level directories based on a tag, but then preserve the directory structure underneath that. I'm having trouble with this.
|
||||
|
||||
Say I have an annex at `~/annex` with a structure like this:
|
||||
|
||||
$ tree
|
||||
.
|
||||
├── foo
|
||||
│ └── bar
|
||||
│ ├── one.txt
|
||||
│ ├── three.txt
|
||||
│ └── two.txt
|
||||
└── waldo
|
||||
└── fred
|
||||
├── a.txt
|
||||
├── b.txt
|
||||
└── c.txt
|
||||
|
||||
I tag some of the files with `blah`:
|
||||
|
||||
$ git annex metadata -t blah foo/bar/*
|
||||
|
||||
Now I want to change my view to only see those files with a certain tag, but I want to maintain their directory structure, ie I want to end up with something like this:
|
||||
|
||||
$ tree
|
||||
.
|
||||
├── blah
|
||||
│ └── foo
|
||||
│ └── bar
|
||||
│ ├── one.txt
|
||||
│ ├── three.txt
|
||||
│ └── two.txt
|
||||
|
||||
If I do `git annex view blah` I see the files `one.txt`, `two.txt` and `three.txt` but they are in the top level of `~/annex`. The `foo` and `bar` directories are not present.
|
||||
|
||||
If I do `git annex view blah "/=*"` then the files I present under the `foo` directory, but the `bar` subdirectory is not there.
|
||||
|
||||
It would also be fine if I could just hide the files that did not have the `blah` tag, so that I ended up with this:
|
||||
|
||||
$ tree
|
||||
.
|
||||
├── foo
|
||||
│ └── bar
|
||||
│ ├── one.txt
|
||||
│ ├── three.txt
|
||||
│ └── two.txt
|
||||
|
||||
Is something like this possible?
|
|
@ -0,0 +1,32 @@
|
|||
I am attempting to set up automatic two-way synchronization between my laptop and a server via ssh by running assistant on both machines. I want to have both machines be non-bare and unlocked.
|
||||
|
||||
On the rhel server:
|
||||
|
||||
$ mkdir ~/annex
|
||||
$ cd ~/annex
|
||||
$ git init
|
||||
$ git annex init u --version=6
|
||||
$ echo This is test file 1. >testfile1.txt
|
||||
$ git annex add testfile1.txt
|
||||
$ git annex sync
|
||||
$ git remote add ml2 ssh://laptop/Users/username/annex
|
||||
$ git annex adjust --unlock
|
||||
$ git annex wanted . standard
|
||||
$ git annex group . client
|
||||
|
||||
On my mac laptop:
|
||||
|
||||
$ cd ~/
|
||||
$ git clone ssh://server/home/username/annex
|
||||
$ cd annex
|
||||
$ git annex init ml2 --version=6
|
||||
$ git annex sync
|
||||
$ git annex adjust --unlock
|
||||
$ git annex wanted . standard
|
||||
$ git annex group . client
|
||||
|
||||
Everything seems to work when I manually sync. But when I run
|
||||
|
||||
$ git annex assistant
|
||||
|
||||
on both machines, I only get one-way automatic synchronization. Changes on the laptop are immediately propagated to the server. But changes on the server do not show up on the laptop until I manually sync. What am I doing wrong?
|
|
@ -0,0 +1,17 @@
|
|||
[[!comment format=mdwn
|
||||
username="neocryptek@659edac901ffbc8e541a974f8f18987eeafc63bd"
|
||||
nickname="neocryptek"
|
||||
avatar="http://cdn.libravatar.org/avatar/d9bfdefa9b503f1ac4844a686618374e"
|
||||
subject="comment 2"
|
||||
date="2016-11-21T22:26:52Z"
|
||||
content="""
|
||||
Right, though bup also requires installation on the server. I'm looking for a way to store content into a vanilla git repo (as I don't have permission to install anything custom on the server).
|
||||
|
||||
Since I want to store the content outside of git annex, it feels like a special remote. Though ideally it would have human readable files like:
|
||||
|
||||
* <https://git-annex.branchable.com/todo/dumb__44___unsafe__44___human-readable_backend/>
|
||||
|
||||
But since it's git and not just a normal (single version) filesystem, it could dedupe and save previous versions. Is there an easy way to hook git up safely to the external remote protocol:
|
||||
|
||||
* [[special_remotes/external]]
|
||||
"""]]
|
|
@ -20,8 +20,6 @@ Multiple pairs of file and key can be given in a single command line.
|
|||
Allow rekeying of even files whose content is not currently available.
|
||||
Use with caution.
|
||||
|
||||
# OPTIONS
|
||||
|
||||
# SEE ALSO
|
||||
|
||||
[[git-annex]](1)
|
||||
|
|
|
@ -18,7 +18,7 @@ several more. Handy if you don't otherwise have git installed.
|
|||
|
||||
## autobuilds
|
||||
|
||||
Thanks to Dartmouth for hosting the autobuilder.
|
||||
Thanks to Dartmouth College for hosting the autobuilder.
|
||||
|
||||
* [autobuild of git-annex.dmg](https://downloads.kitenet.net/git-annex/autobuild/x86_64-apple-yosemite/git-annex.dmg) ([build logs](https://downloads.kitenet.net/git-annex/autobuild/x86_64-apple-yosemite/))
|
||||
|
||||
|
|
|
@ -1,8 +1,12 @@
|
|||
git-annex now does Windows!
|
||||
|
||||
* First, [install Git for Windows](http://git-scm.com/downloads)
|
||||
* First, [install Git for Windows](http://git-scm.com/downloads)
|
||||
|
||||
Important: **Get the 32 bit version not the 64 bit version.**
|
||||
(Note that msysgit is no longer supported.)
|
||||
If you installed the 64 bit version of git, then parts of git-annex will
|
||||
still run, however, some features, including tools like rsync, will
|
||||
not work.
|
||||
|
||||
* Then, [install git-annex](https://downloads.kitenet.net/git-annex/windows/current/)
|
||||
|
||||
This port is now in reasonably good shape for command-line use of
|
||||
|
@ -18,7 +22,7 @@ important thing is that it should end with "All tests passed".
|
|||
## autobuilds
|
||||
|
||||
A daily build is also available, thanks to Yury V. Zaytsev and
|
||||
[NEST](http://nest-initiative.org/).
|
||||
Dartmouth College.
|
||||
|
||||
* [download](https://downloads.kitenet.net/git-annex/autobuild/windows/) ([build logs](https://qa.nest-initiative.org/view/msysGit/job/msysgit-git-annex-assistant-test/))
|
||||
|
||||
|
|
|
@ -0,0 +1,9 @@
|
|||
[[!comment format=mdwn
|
||||
username="davidriod@e75b369a4b1cced29c14354bce7493c61f00b1c7"
|
||||
nickname="davidriod"
|
||||
avatar="http://cdn.libravatar.org/avatar/d6e327bd88b88802d6f0c20c83f682a2"
|
||||
subject="Sharing rsync special remote between repository"
|
||||
date="2016-11-24T19:23:42Z"
|
||||
content="""
|
||||
I was wondering if it is possible to share a rsync special remote between repository which are not parented in any way. The use case would be that even if these repositories are not related at all they still may contains the same binary file. It would be useful to have a single rsync remote in order to reduce space usage. I think it could work as the object names are based on their checksum, but I wonder if anyone has already try that ?
|
||||
"""]]
|
|
@ -0,0 +1,82 @@
|
|||
[[!comment format=mdwn
|
||||
username="StephaneGourichon"
|
||||
avatar="http://cdn.libravatar.org/avatar/8cea01af2c7a8bf529d0a3d918ed4abf"
|
||||
subject="Walkthrough of a prudent retroactive annex."
|
||||
date="2016-11-24T11:27:59Z"
|
||||
content="""
|
||||
Been using the one-liner. Despite the warning, I'm not dead yet.
|
||||
|
||||
There's much more to do than the one-liner.
|
||||
|
||||
This post offers instructions.
|
||||
|
||||
# First simple try: slow
|
||||
|
||||
Was slow (estimated >600s for 189 commits).
|
||||
|
||||
# In tmpfs: about 6 times faster
|
||||
|
||||
I have cloned repository into /run/user/1000/rewrite-git, which is a tmpfs mount point. (Machine has plenty of RAM.)
|
||||
|
||||
There I also did `git annex init`, git-annex found its state branches.
|
||||
|
||||
On second try I also did
|
||||
|
||||
git checkout -t remotes/origin/synced/master
|
||||
|
||||
So that filter-branch would clean that, too.
|
||||
|
||||
There, `filter-branch` operation finished in 90s first try, 149s second try.
|
||||
|
||||
`.git/objects` wasn't smaller.
|
||||
|
||||
# Practicing reduction on clone
|
||||
|
||||
This produced no visible benefit:
|
||||
|
||||
time git gc --aggressive
|
||||
time git repack -a -d
|
||||
|
||||
Even cloning and retrying on clone. Oh, but I should have done `git clone file:///path` as said on git-filter-branch man page's section titled \"CHECKLIST FOR SHRINKING A REPOSITORY\"
|
||||
|
||||
This (as seen on https://rtyley.github.io/bfg-repo-cleaner/ ) was efficient:
|
||||
|
||||
git reflog expire --expire=now --all && git gc --prune=now --aggressive
|
||||
|
||||
`.git/objects` shrunk from 148M to 58M
|
||||
|
||||
All this was on a clone of the repo in tmpfs.
|
||||
|
||||
# Propagating cleaned up branches to origin
|
||||
|
||||
This confirmed that filter-branch did not change last tree:
|
||||
|
||||
git diff remotes/origin/master..master
|
||||
git diff remotes/origin/synced/master synced/master
|
||||
|
||||
This, expectedly, was refused:
|
||||
|
||||
git push origin master
|
||||
git push origin synced/master
|
||||
|
||||
On origin, I checked out the hash of current master, then on tmpfs clone
|
||||
|
||||
git push -f origin master
|
||||
git push -f origin synced/master
|
||||
|
||||
Looks good.
|
||||
|
||||
I'm not doing the aggressive shrink now, because of the \"two orders of magnitude more caution than normal filter-branch\" recommended by arand.
|
||||
|
||||
# Now what? Check if precious not broken
|
||||
|
||||
I'm planning to do the same operation on the other repos, then :
|
||||
|
||||
* if everything seems right,
|
||||
* if `git annex sync` works between all those fellows
|
||||
* etc,
|
||||
* then I would perform the reflog expire, gc prune on some then all of them, etc.
|
||||
|
||||
Joey, does this seem okay? Any comment?
|
||||
|
||||
"""]]
|
|
@ -0,0 +1,9 @@
|
|||
[[!comment format=mdwn
|
||||
username="scottgorlin@a32946b2aad278883c1690a0753241583a9855b9"
|
||||
nickname="scottgorlin"
|
||||
avatar="http://cdn.libravatar.org/avatar/2dd1fc8add62bbf4ffefac081b322563"
|
||||
subject="Coldline"
|
||||
date="2016-11-21T00:49:23Z"
|
||||
content="""
|
||||
Wanted to add that \"storageclass=COLDLINE\" appears to work seamlessly, both from my mac and arm NAS. As far as I can tell, this appears to be a no-brainer vs glacier - builtin git annex client, simpler/cheaper billing, and no 4 hour delay!
|
||||
"""]]
|
24
doc/todo/renameremote.mdwn
Normal file
24
doc/todo/renameremote.mdwn
Normal file
|
@ -0,0 +1,24 @@
|
|||
Sometimes a name has been used for a special remote, and you want to change
|
||||
the name. A common reason is that the special remote has become dead, and
|
||||
you want to reuse the name for a new special remote.
|
||||
|
||||
Initremote prevents reusing a name when the old one exists, even if the old
|
||||
one is dead. And that makes sense in general, because a dead remote can
|
||||
come back sometimes, and that would leave the repo with two special remotes
|
||||
with the same name, and so enableremote would need to be run with a uuid
|
||||
instead of a name to specify which one to enable, which is not a desirable
|
||||
state of affairs.
|
||||
|
||||
So, add `git annex renameremote oldname newname`. This could also do a `git
|
||||
remote rename`, or equivilant. (`git remote rename` gets confused by special
|
||||
remotes not having a fetch url and fails; this can be worked around by
|
||||
manually renaming the stanza in git config.)
|
||||
|
||||
Implementing that would need a way to remove the old name from remote.log.
|
||||
We can't remove lines from union merged files, but what we could do is
|
||||
add a new line like:
|
||||
|
||||
- name=oldname timestamp=<latest>
|
||||
|
||||
And in parsing remote.log, if the UUID is "-", don't include the
|
||||
remote with that name in the the resulting map.
|
Loading…
Reference in a new issue