Merge branch 'master' into tor

This commit is contained in:
Joey Hess 2016-11-29 15:45:29 -04:00
commit 398345cb26
No known key found for this signature in database
GPG key ID: C910D9222512E3C7
35 changed files with 697 additions and 20 deletions

View file

@ -2,7 +2,7 @@
-
- Copyright 2013 Joey Hess <id@joeyh.name>
-
- Licensed under the GNU AGPL version 3 or higher.
- Licensed under the GNU GPL version 3 or higher.
-}
module Assistant.Fsck where

View file

@ -2,7 +2,7 @@
-
- Copyright 2013 Joey Hess <id@joeyh.name>
-
- Licensed under the GNU AGPL version 3 or higher.
- Licensed under the GNU GPL version 3 or higher.
-}
module Assistant.Gpg where

View file

@ -2,7 +2,7 @@
-
- Copyright 2013 Joey Hess <id@joeyh.name>
-
- Licensed under the GNU AGPL version 3 or higher.
- Licensed under the GNU GPL version 3 or higher.
-}
{-# LANGUAGE CPP #-}

View file

@ -2,7 +2,7 @@
-
- Copyright 2013 Joey Hess <id@joeyh.name>
-
- Licensed under the GNU AGPL version 3 or higher.
- Licensed under the GNU GPL version 3 or higher.
-}
{-# LANGUAGE CPP #-}

View file

@ -2,7 +2,7 @@
-
- Copyright 2013 Joey Hess <id@joeyh.name>
-
- Licensed under the GNU AGPL version 3 or higher.
- Licensed under the GNU GPL version 3 or higher.
-}
{-# LANGUAGE CPP #-}

View file

@ -6,8 +6,12 @@ git-annex (6.20161119) UNRELEASED; urgency=medium
hidden service.
* remotedaemon: Fork to background by default. Added --foreground switch
to enable old behavior.
* addurl: Fix bug in checking annex.largefiles expressions using
largerthan, mimetype, and smallerthan; the first two always failed
to match, and the latter always matched.
* Relicense 5 source files that are not part of the webapp from AGPL to GPL.
-- Joey Hess <id@joeyh.name> Sun, 20 Nov 2016 14:10:15 -0400
-- Joey Hess <id@joeyh.name> Mon, 21 Nov 2016 11:27:50 -0400
git-annex (6.20161118) unstable; urgency=medium

View file

@ -340,13 +340,18 @@ cleanup :: UUID -> URLString -> FilePath -> Key -> Maybe FilePath -> Annex ()
cleanup u url file key mtmp = case mtmp of
Nothing -> go
Just tmp -> do
-- Move to final location for large file check.
liftIO $ renameFile tmp file
largematcher <- largeFilesMatcher
ifM (checkFileMatcher largematcher file)
( go
, do
liftIO $ renameFile tmp file
void $ Command.Add.addSmall file
)
large <- checkFileMatcher largematcher file
if large
then do
-- Move back to tmp because addAnnexedFile
-- needs the file in a different location
-- than the work tree file.
liftIO $ renameFile file tmp
go
else void $ Command.Add.addSmall file
where
go = do
maybeShowJSON $ JSONChunk [("key", key2file key)]

View file

@ -44,7 +44,7 @@ perform :: FilePath -> Key -> Key -> CommandPerform
perform file oldkey newkey = do
ifM (inAnnex oldkey)
( unlessM (linkKey file oldkey newkey) $
error "failed"
giveup "failed"
, unlessM (Annex.getState Annex.force) $
giveup $ file ++ " is not available (use --force to override)"
)

View file

@ -29,7 +29,7 @@ start = parse
where
parse (name:[]) = go name performGet
parse (name:expr:[]) = go name $ \uuid -> do
showStart "schedile" name
showStart "schedule" name
performSet expr uuid
parse _ = giveup "Specify a repository."

View file

@ -0,0 +1,34 @@
[[!comment format=mdwn
username="boh"
avatar="http://cdn.libravatar.org/avatar/e7fa2d1c5d95e323fe48887f7f827b1f"
subject="comment 9"
date="2016-11-27T12:23:20Z"
content="""
Seems as if the problem still exists in 6.20161118 (Debian).
I have three repositories (among others), `jolla`, `sts-3xx`, and `here`. `jolla` and `here` are in group `manual`, `sts-3xx` is `backup`; `here` and `sts-3xx` have assistants running, `jolla` not. `jolla` and `sts-3xx` have slightly older versions of git-annex installed.
Now, when I copy a file from `here` to `jolla` like this
git annex copy real_programmers.png -t jolla
the file is subsequently dropped by the assistant:
```
drop real_programmers.png (locking jolla...) [2016-11-27 13:00:02.667376556] chat: ssh [\"-S\",\".git/annex/ssh/jolla\",\"-o\",\"ControlMaster
=auto\",\"-o\",\"ControlPersist=yes\",\"-F\",\".git/annex/ssh.config\",\"-T\",\"jolla\",\"git-annex-shell 'lockcontent' '/~/Music/media/' '--debug' '
SHA256E-s84499--ff98a733cc0122858fb11433c720e2d038fec190a3d36380d0e7e8dab468f883.png' --uuid 5298e3ce-1106-4d5e-b052-0aee4b27a344\"]
(locking sts-3xx...) [2016-11-27 13:00:03.252473676] chat: ssh [..., \"git-annex-shell 'lockcontent' '/backups/exot/media/' '--debug' 'SHA256E-s84499--ff98a733cc0122858fb11433c720e2d038fec190a3d 36380d0e7e8dab468f883.png' --uuid 1fec6253-171d-4f86-885b-e233be2d65ec\"]
(lockcontent failed) [2016-11-27 13:00:03.486158016] process done ExitFailure 1
(checking sts-3xx...) [2016-11-27 13:00:03.487047149] call: ssh [..., \"git-annex-shell 'inannex' '/backups/exot/media/' '--debug' 'SHA256E-s84499--ff98a733cc0122858fb11433c720e2d038fec190a3d363 80d0e7e8dab468f883.png' --uuid 1fec6253-171d-4f86-885b-e233be2d65ec\"]
[2016-11-27 13:00:03.76435136] process done ExitSuccess
[2016-11-27 13:00:03.764705754] Dropping from here proof: Just (SafeDropProof (NumCopies 2) [RecentlyVerifiedCopy UUID \"1fec6253-171d-4 f86-885b-e233be2d65ec\",LockedCopy UUID \"5298e3ce-1106-4d5e-b052-0aee4b27a344\"] (Just (ContentRemovalLock (Key {keyName = \"ff98a733cc012 2858fb11433c720e2d038fec190a3d36380d0e7e8dab468f883.png\", keyBackendName = \"SHA256E\", keySize = Just 84499, keyMtime = Nothing, keyChun kSize = Nothing, keyChunkNum = Nothing}))))
[2016-11-27 13:00:04.24333081] process done ExitFailure 1
ok
[2016-11-27 13:00:04.251232455] dropped real_programmers.png (from here) (copies now 4) : drop wanted after Upload UUID \"5298e3ce-1106- 4d5e-b052-0aee4b27a344\" real_programmers.png Just 84499
```
However, I failed to reproduce the problem by replicating my setup with fresh repositories …
Please let me know if you need more information, and *so* many thanks for git-annex!
"""]]

View file

@ -0,0 +1,48 @@
### Please describe the problem.
https://github.com/aristidb/aws/issues/206 was recently resolved in https://github.com/aristidb/aws/pull/213.
A newer version will be tagged imminently according to https://github.com/aristidb/aws/issues/206#issuecomment-260214736.
With the http-conduit (<2.2.0) constraint removed from git-annex.cabal, and the aws dependency set to use aws head (currently c8806dc), the git-annex build fails.
### What steps will reproduce the problem?
Remove the http-conduit (<2.2.0) constraint and attempt to build git-annex with aws head.
### What version of git-annex are you using? On what operating system?
macOS 10.11, git-annex 6.20161118.
### Please provide any additional information below.
Full build log: https://gist.github.com/ilovezfs/15bcd8f1086b3d825beff58140e04eec
[[!format sh """
[ 90 of 542] Compiling Types.Crypto ( Types/Crypto.hs, dist/dist-sandbox-6b15e8f0/build/git-annex/git-annex-tmp/Types/Crypto.o )
[ 91 of 542] Compiling Utility.Metered ( Utility/Metered.hs, dist/dist-sandbox-6b15e8f0/build/git-annex/git-annex-tmp/Utility/Metered.o )
[ 92 of 542] Compiling Messages.JSON ( Messages/JSON.hs, dist/dist-sandbox-6b15e8f0/build/git-annex/git-annex-tmp/Messages/JSON.o )
[ 93 of 542] Compiling Utility.Url ( Utility/Url.hs, dist/dist-sandbox-6b15e8f0/build/git-annex/git-annex-tmp/Utility/Url.o )
Utility/Url.hs:354:34: error:
• The constructor StatusCodeException should have 2 arguments, but has been given 3
• In the pattern: StatusCodeException s _ _
In an equation for matchStatusCodeException:
matchStatusCodeException want e@(StatusCodeException s _ _)
| want s = Just e
| otherwise = Nothing
Utility/Url.hs:354:34: error:
• Couldn't match expected type HttpException
with actual type HttpExceptionContent
• In the pattern: StatusCodeException s _ _
In an equation for matchStatusCodeException:
matchStatusCodeException want e@(StatusCodeException s _ _)
| want s = Just e
| otherwise = Nothing
cabal: Leaving directory '.'
cabal: Error: some packages failed to install:
git-annex-6.20161118 failed during the building phase. The exception was:
ExitFailure 1
"""]]
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
Yes :)

View file

@ -0,0 +1,53 @@
### Please describe the problem.
I'm seeing some inconsistent results between runs of `git annex fsck` and `git annex whereis` that I'm not able to explain. When I run `git annex fsck`, it reports a few keys that only have 1 copy, and advises me to make more copies. If I run `git annex whereis --key <key>`, git annex confirms that it only knows about 1 copy of this key. If I then use `git log --stat -S'<key>'` to find the actual file that it refers to, and run `git annex whereis <file>`, git annex report 9 copies of this file. Checking on remotes shows that these files do exist on the remote, so why does `git annex fsck` and `git annex whereis` mis-report the number of copies when querying for the key - but not for the actual filename? Additionally, `git annex find --lackingcopies 1` doesn't return any results, but should if there are actually files with not enough copies?
### What steps will reproduce the problem?
### What version of git-annex are you using? On what operating system?
5.20151208-1build1 on Ubuntu Xenial, one remote running 5.20141024~bpo70+1 on Debian Wheezy
### Please provide any additional information below.
[[!format sh """
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
[william@hactar ~/Pictures/Photo Library]$ git annex whereis SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9
git-annex: SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9 not found
git-annex: whereis: 1 failed
[william@hactar ~/Pictures/Photo Library]$ git annex whereis --key SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9
whereis SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9 (1 copy)
7691934f-2542-4103-9122-2db4e6cfc887 -- hactar [here]
ok
[william@hactar ~/Pictures/Photo Library]$ git annex fsck --key SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9
fsck SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9
Only 1 of 3 trustworthy copies exist of SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9
Back it up with git-annex copy.
failed
(recording state in git...)
git-annex: fsck: 1 failed
[william@hactar ~/Pictures/Photo Library]$ git log --stat -S'SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9'
[william@hactar ~/Pictures/Photo Library]$ git annex whereis 2009/05/05/P1040890.JPG
whereis 2009/05/05/P1040890.JPG (9 copies)
0e825a69-1927-4f62-b731-6f3e98bba998 -- william@marvin:/media/backup/annex/photos [marvin]
1b728ab5-1e32-45a6-bc11-2a4bfdc9d6ab -- backup1
5c0caa42-b489-467b-a612-9590fa9d5a94 -- backup2
7691934f-2542-4103-9122-2db4e6cfc887 -- hactar [here]
894b2216-72e0-40e1-8765-1386e1e9e4b4 -- backup3
96f19fa8-d385-4e8b-b000-61ee15993a70 -- backup3
a862b121-d794-4af4-bb56-21adfe8962f2 -- S3
b083f8ae-42fb-41f0-a2a3-4e7c9f93aadb -- [guide]
bf021ce9-465b-4419-86e7-bddfd208fca4 -- git@newzaphod:~/repositories/annex/photos.git [zaphod]
ok
# End of transcript or log.
"""]]
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
I trust Git Annex to keep hundreds of GB of data safe, and it has never failed me - despite my best efforts

View file

@ -0,0 +1,17 @@
[[!comment format=mdwn
username="justin.lebar@7a36fcafc322d9a381e89f08ab6289033c6dde91"
nickname="justin.lebar"
avatar="http://cdn.libravatar.org/avatar/9fca4b61a1ab555f231851e7543f9a3e"
subject="comment 2"
date="2016-11-20T03:47:23Z"
content="""
Thanks for your reply, Joey. Sorry for the delay getting back to this -- I didn't realize I hadn't enabled notifications on the thread.
The GCS docs suggest that 400 errors should be accompanied by an explanation in the reply body.
> Error responses usually include a JSON document in the response body, which contains information about the error.
https://cloud.google.com/storage/docs/json_api/v1/status-codes
Do you think we're not getting an http response body here, or that it's not being printed out?
"""]]

View file

@ -0,0 +1,16 @@
[[!comment format=mdwn
username="https://launchpad.net/~stephane-gourichon-lpad"
nickname="stephane-gourichon-lpad"
avatar="http://cdn.libravatar.org/avatar/02d4a0af59175f9123720b4481d55a769ba954e20f6dd9b2792217d9fa0c6089"
subject="Known bug, fixed."
date="2016-11-23T18:04:27Z"
content="""
This is a known bug introduced in 6.20161012 and fixed in 6.20161031.
Solution is: just update your copy of git-annex. At this time most recent is 6.20161119 .
For more details, see changelog at https://github.com/joeyh/git-annex/blob/master/CHANGELOG#L53
"""]]

View file

@ -0,0 +1,12 @@
[[!comment format=mdwn
username="t.z.mates"
avatar="http://cdn.libravatar.org/avatar/90f15fad216078fd08d62cc676487925"
subject="comment 2"
date="2016-11-19T04:42:25Z"
content="""
Thanks for looking into it; I just checked again, and even on the newest version (6.20161118 binary), I'm still experiencing the behavior. However, I checked on an older OpenSuse box I have, and there it works (6.20161031 from OpenSuse repo).
Since my two machines experiencing the problem are both running arch, it seems it's somehow related to that distro. I've checked both installing via the binary (from kitenet) and from the arch community repo, but both produce the same behavior. Further, the OpenSuse install has the same build flags as the binaries, so that doesn't seem to be it. Are there any other diagnostics I can run?
This particular problem isn't very troublesome (it doesn't seem to have any material impact aside from error messages); however, I also occasionally experience a more serious bug. Namely, when certain (seemingly random) files are added to the repo locked, their content disappears and the symlink is broken (this is the other problem I alluded to in the description). I suspect that problem is related to this one though, since it also only affects my arch machines. I haven't yet submitted a report for that bug yet, though, since I can't reliably replicate it.
"""]]

View file

@ -0,0 +1,41 @@
### Please describe the problem.
When addurl'ing a big file with .gitattributes configured to add only some files directly into git (and 'git annex add' operating correctly), addurl adds large files straight into git.
### What version of git-annex are you using? On what operating system?
git-annex version: 6.20161018+gitgf3c366a-1~ndall+1
### Please provide any additional information below.
[[!format sh """
$> cat .gitattributes
* annex.backend=MD5E
* annex.largefiles=(largerthan=100kb)
*.json annex.largefiles=nothing
*.txt annex.largefiles=nothing
*.tsv annex.largefiles=nothing
*.nii.gz annex.largefiles=(largerthan=0kb)
*.tgz annex.largefiles=(largerthan=0kb)
*.tar.gz annex.largefiles=(largerthan=0kb)
*.gz annex.largefiles=(largerthan=0kb)
$> git annex addurl http://fcp-indi.s3.amazonaws.com/data/Projects/HBNSSI/RawDataTars/sub-0031121_baseline.tar.gz\?versionId\=7FvexHgyazWF.dUo238FA7XRiK0FWQDw.
addurl fcp_indi.s3.amazonaws.com_data_Projects_HBNSSI_RawDataTars_sub_0031121_baseline.tar.gz_versionId_7FvexHgyazWF.dUo238FA7XRiK0FWQDw. (downloading http://fcp-indi.s3.amazonaws.com/data/Projects/HBNSSI/RawDataTars/sub-0031121_baseline.tar.gz?versionId=7FvexHgyazWF.dUo238FA7XRiK0FWQDw. ...)
/mnt/btrfs/datasets/datalad/crawl-misc/indi/ 100%[==============================================================================================>] 195.44M 21.2MB/s in 12s
(non-large file; adding content to git repository) ok
(recording state in git...)
cached/staged changes:
\u2026r.gz_versionId_7FvexHgyazWF.dUo238FA7XRiK0FWQDw. | Bin 0 -> 204937338 bytes
$> ls -l fcp_indi.s3.amazonaws.com_data_Projects_HBNSSI_RawDataTars_sub_0031121_baseline.tar.gz_versionId_7FvexHgyazWF.dUo238FA7XRiK0FWQDw.
-rw------- 1 yoh datalad 204937338 Oct 25 17:30 fcp_indi.s3.amazonaws.com_data_Projects_HBNSSI_RawDataTars_sub_0031121_baseline.tar.gz_versionId_7FvexHgyazWF.dUo238FA7XRiK0FWQDw.
cached/staged changes:
\u2026r.gz_versionId_7FvexHgyazWF.dUo238FA7XRiK0FWQDw. | Bin 0 -> 204937338 bytes
"""]]
[[!meta author=yoh]]
> [[fixed|done]] --[[Joey]]

View file

@ -0,0 +1,20 @@
[[!comment format=mdwn
username="joey"
subject="""comment 1"""
date="2016-11-21T15:12:54Z"
content="""
It's sufficient to have "* annex.largefiles=(largerthan=100kb)"
in .gitattributes.
Even "* annex.largefiles=(largerthan=0kb)" will reproduce it.
Ok, I see why.. It's running the largefile matcher on the destination file
before it renames the temp file to it!
Seems to have been broken this way ever since addurl got largefiles
support. Testing didn't catch it because it only affects largefiles
expressions that need to examine the file.
Fixed in git. Audited other checkFileMatcher calls for this problem;
the rest are ok.
"""]]

View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="CandyAngel"
avatar="http://cdn.libravatar.org/avatar/15c0aade8bec5bf004f939dd73cf9ed8"
subject="comment 4"
date="2016-11-25T20:27:07Z"
content="""
I really don't know what to say. I can't even figure out which computer I updated git-annex on to test if it was still happening.. let alone reproduce it anymore. It does work fine.
I'm so sorry to bother you with this, I've done something stupid! This is exactly why you ask for a transcript of bugs occurring. (Feel free to use this as an example for why you ask for them, so some good can come of it at least..).
"""]]

View file

@ -0,0 +1,30 @@
[[!comment format=mdwn
username="https://launchpad.net/~felixonmars"
nickname="felixonmars"
avatar="http://cdn.libravatar.org/avatar/17284a3bb2e4ad9d3be8fab31f49865be9c1dc22143c728de731fe800a335d38"
subject="comment 1"
date="2016-11-28T04:17:12Z"
content="""
aws has merged a PR to support http-conduit 2.2, but git-annex itself doesn't build with the new component yet:
```
[ 95 of 544] Compiling Utility.Url ( Utility/Url.hs, dist/build/git-annex/git-annex-tmp/Utility/Url.o )
Utility/Url.hs:354:34: error:
* The constructor `StatusCodeException' should have 2 arguments, but has been given 3
* In the pattern: StatusCodeException s _ _
In an equation for `matchStatusCodeException':
matchStatusCodeException want e@(StatusCodeException s _ _)
| want s = Just e
| otherwise = Nothing
Utility/Url.hs:354:34: error:
* Couldn't match expected type `HttpException'
with actual type `HttpExceptionContent'
* In the pattern: StatusCodeException s _ _
In an equation for `matchStatusCodeException':
matchStatusCodeException want e@(StatusCodeException s _ _)
| want s = Just e
| otherwise = Nothing
```
"""]]

View file

@ -0,0 +1,34 @@
### Please describe the problem.
I'm sending a stream of keys and filenames to git-annex fromkey on stdin, and it errors out with "git-annex: <stdin>: hGetContents: invalid argument (invalid byte sequence)". On the other hand yipdw tried to reproduce this and it worked fine for him, so I must be doing something wrong.
I have LANG=en_US.UTF-8 set in my environment, if that matters.
### What steps will reproduce the problem?
[[!format sh """
echo "MD5-s3263532--0b4d070eff7baa8ef314ca330aecb71f é" | git-annex fromkey
"""]]
### What version of git-annex are you using? On what operating system?
[[!format sh """
git-annex version: 6.20161118-g0a34f08
build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify XMPP ConcurrentOutput TorrentParser MagicMime Feeds Quvi
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
local repository version: 5
supported repository versions: 3 5 6
upgrade supported from repository versions: 0 1 2 3 4 5
operating system: linux x86_64
"""]]
### Please provide any additional information below.
Note that this is indeed valid utf-8:
[[!format sh """
 db48x  ~  projects  IA.BAK-server  echo "é" | hexdump -C
00000000 c3 a9 0a |...|
00000003
"""]]

View file

@ -0,0 +1,17 @@
[[!comment format=mdwn
username="scottgorlin@a32946b2aad278883c1690a0753241583a9855b9"
nickname="scottgorlin"
avatar="http://cdn.libravatar.org/avatar/2dd1fc8add62bbf4ffefac081b322563"
subject="IgnoreUnknown Include considered harmful?"
date="2016-11-23T20:07:45Z"
content="""
As noted, include appears to not work on a mac at the moment. This means git-annex silently ignores the included configs, which may be required to ssh to the remotes of interest. This is happening to me.
My understanding is that ssh aliases are the recommended way of juggling multiple private keys amongst multiple hosts, so it is a required part of many git workflows. In this particular case, I have set up git annex on a NAS which does not allow multiple ssh users (QNAP) and the authentication is done only via key identity, not username. Thus, host aliases are necessary.
If one config can't include another, I would prefer an early failure indicating a problem with the config file, or better, a solution where git-annex doesn't require a config. In this scenario, git fetch remote_name and git annex copy --to remotename do not resolve to the same alias definitions (the latter is missing because of the ignored config!).
I got my setup to work only by finding and manually editing <repo>/.git/annex/ssh_config, which to my knowledge is undocumented (ie when is it written? do any commands change it?); manual mucking around inside .git to me is not a good practice, and for now I have two different alias's defined (in repo and in ~/.ssh/config)
"""]]

View file

@ -0,0 +1,15 @@
[[!comment format=mdwn
username="palday@91f366b5178879146d2b6e1e53bfa21389ee89a8"
nickname="palday"
avatar="http://cdn.libravatar.org/avatar/077a63af75ddba159980fbf88690f401"
subject="Temporary workaround until the brew formula is updated"
date="2016-11-29T02:17:52Z"
content="""
The homebrew formula doesn't yet this fix, but you can get around the problem in the meantime by getting a newer SSH via homebrew:
```
brew install homebrew/dupes/openssh
```
You can then choose to keep that or get rid of it when the formula for git annex is later updated.
"""]]

View file

@ -0,0 +1,31 @@
The `tor` branch is coming along nicely.
This weekend, I continued working on the P2P protocol, implementing
it for network sockets, and extending it to support connecting up
git-send-pack/git-receive-pack.
There was a bit of a detour when I split the Free monad into two separate
ones, one for Net operations and the other for Local filesystem operations.
This weekend's work was sponsored by Thomas Hochstein on Patreon.
----
Today, implemented a `git-remote-tor-annex` command that git will
use for tor-annex:: urls, and made `git annex remotedaemon`
serve the tor hidden service.
Now I have git push/pull working to the hidden service, for example:
git pull tor-annex::eeaytkuhaupbarfi.onion:47651
That works very well, but does not yet check that the user is authorized
to use the repo, beyond knowing the onion address. And currently
it only works in git-annex repos; with some tweaks it should
also work in plain git repos.
Next, I need to teach git-annex how to access tor-annex remotes.
And after that, an interface in the webapp for setting them up and
connecting them together.
Today's work was sponsored by Josh Taylor on Patreon.

View file

@ -0,0 +1,41 @@
This is a recreation of a stackexchange question, in case the community here is more knowledgeable.
Link to stackexchange question : http://unix.stackexchange.com/questions/325753/git-annex-link-to-different-file-names
Content :
"Maybe this is just a crazy use case that doesn't work, but I was wondering if there's a way to build a file's history from files with different file names. I'm exploring this idea because I'd like to have a git-annex system but I can't force my coworkers to adapt.
Here's what I have in mind :
Folder 1, managed by coworkers (On a shared disk) :
drawing_shop_12_nov_2015.pdf
drawing_shop_13_nov_2015.pdf
drawing_asbuilt_14_nov_2015.pdf
drawing_asbuilt_rev1_15_nov_2015.pdf
And
Git-annex, managed by me :
drawing.pdf
(with a shop branch and a asbuilt branch)
The git-annex's drawing.pdf would have an history like this :
[shop]
|
Commit A "Initial shop drawing"
|
Commit B "Add corrections from Wizzbasket"
\
|
[asbuilt]
Commit C "Reflect as built"
|
Commit D "Change dweezelbox block for simplicity"
But somehow the "managed by coworkers" repo would be a direct mode repo with Commit A pointing to drawing_shop_12_nov_2015.pdf, Commit B to drawing_shop_13_nov_2015.pdf etc.
Can this be done?"

View file

@ -0,0 +1,27 @@
I've somehow managed to get my indirect repository to symlink to literal content instead of object files.
By this I mean literally the symlink is pointing at the contents of the file as the filename.
So if I have a blah.txt file with this content:
* First line
* second line
And I ls -al to view the symlink pointer, it shows up as this:
* blah.txt -> First line?second line
It literally has the contents of the file as the destination filename.
I've tried a couple things I could think of to re-symlink the files, but they don't seem to do anything as they think everything is fine:
* git annex indirect //returns nothing
* git annex lock blah.txt //returns nothing
* git annex fix blah.txt //returns nothing
* git annex fsck //returns nothing
I'm actually able to find several of these files hanging around by searching for all symlinks that don't point to something in the .git directory.
Is there a way for me to replace the symlinks with correct symlinks to the objects in .git/annex? Can it even figure out which ones it was supposed to point to if the symlinks are messed up (are filenames -> content hashes stored anywhere else)?
Else I might have to go do some manual rebasing and history editing to try to undo the bad commits manually. I've synced this repo to another direct repo so I'll need to figure out how to manually fix that repo too (using proxy). From what I can tell the annex/direct/master seems to be same as master and synced/master branches? Is there an [[internals]] page for direct branches besides [[direct_mode]] so I know what should be fixed where?

View file

@ -0,0 +1,47 @@
I want to use metadata views to sort files into top-level directories based on a tag, but then preserve the directory structure underneath that. I'm having trouble with this.
Say I have an annex at `~/annex` with a structure like this:
$ tree
.
├── foo
│   └── bar
│   ├── one.txt
│   ├── three.txt
│   └── two.txt
└── waldo
└── fred
├── a.txt
├── b.txt
└── c.txt
I tag some of the files with `blah`:
$ git annex metadata -t blah foo/bar/*
Now I want to change my view to only see those files with a certain tag, but I want to maintain their directory structure, ie I want to end up with something like this:
$ tree
.
├── blah
│   └── foo
│   └── bar
│   ├── one.txt
│   ├── three.txt
│   └── two.txt
If I do `git annex view blah` I see the files `one.txt`, `two.txt` and `three.txt` but they are in the top level of `~/annex`. The `foo` and `bar` directories are not present.
If I do `git annex view blah "/=*"` then the files I present under the `foo` directory, but the `bar` subdirectory is not there.
It would also be fine if I could just hide the files that did not have the `blah` tag, so that I ended up with this:
$ tree
.
├── foo
│   └── bar
│   ├── one.txt
│   ├── three.txt
│   └── two.txt
Is something like this possible?

View file

@ -0,0 +1,32 @@
I am attempting to set up automatic two-way synchronization between my laptop and a server via ssh by running assistant on both machines. I want to have both machines be non-bare and unlocked.
On the rhel server:
$ mkdir ~/annex
$ cd ~/annex
$ git init
$ git annex init u --version=6
$ echo This is test file 1. >testfile1.txt
$ git annex add testfile1.txt
$ git annex sync
$ git remote add ml2 ssh://laptop/Users/username/annex
$ git annex adjust --unlock
$ git annex wanted . standard
$ git annex group . client
On my mac laptop:
$ cd ~/
$ git clone ssh://server/home/username/annex
$ cd annex
$ git annex init ml2 --version=6
$ git annex sync
$ git annex adjust --unlock
$ git annex wanted . standard
$ git annex group . client
Everything seems to work when I manually sync. But when I run
$ git annex assistant
on both machines, I only get one-way automatic synchronization. Changes on the laptop are immediately propagated to the server. But changes on the server do not show up on the laptop until I manually sync. What am I doing wrong?

View file

@ -0,0 +1,17 @@
[[!comment format=mdwn
username="neocryptek@659edac901ffbc8e541a974f8f18987eeafc63bd"
nickname="neocryptek"
avatar="http://cdn.libravatar.org/avatar/d9bfdefa9b503f1ac4844a686618374e"
subject="comment 2"
date="2016-11-21T22:26:52Z"
content="""
Right, though bup also requires installation on the server. I'm looking for a way to store content into a vanilla git repo (as I don't have permission to install anything custom on the server).
Since I want to store the content outside of git annex, it feels like a special remote. Though ideally it would have human readable files like:
* <https://git-annex.branchable.com/todo/dumb__44___unsafe__44___human-readable_backend/>
But since it's git and not just a normal (single version) filesystem, it could dedupe and save previous versions. Is there an easy way to hook git up safely to the external remote protocol:
* [[special_remotes/external]]
"""]]

View file

@ -20,8 +20,6 @@ Multiple pairs of file and key can be given in a single command line.
Allow rekeying of even files whose content is not currently available.
Use with caution.
# OPTIONS
# SEE ALSO
[[git-annex]](1)

View file

@ -18,7 +18,7 @@ several more. Handy if you don't otherwise have git installed.
## autobuilds
Thanks to Dartmouth for hosting the autobuilder.
Thanks to Dartmouth College for hosting the autobuilder.
* [autobuild of git-annex.dmg](https://downloads.kitenet.net/git-annex/autobuild/x86_64-apple-yosemite/git-annex.dmg) ([build logs](https://downloads.kitenet.net/git-annex/autobuild/x86_64-apple-yosemite/))

View file

@ -1,8 +1,12 @@
git-annex now does Windows!
* First, [install Git for Windows](http://git-scm.com/downloads)
* First, [install Git for Windows](http://git-scm.com/downloads)
Important: **Get the 32 bit version not the 64 bit version.**
(Note that msysgit is no longer supported.)
If you installed the 64 bit version of git, then parts of git-annex will
still run, however, some features, including tools like rsync, will
not work.
* Then, [install git-annex](https://downloads.kitenet.net/git-annex/windows/current/)
This port is now in reasonably good shape for command-line use of
@ -18,7 +22,7 @@ important thing is that it should end with "All tests passed".
## autobuilds
A daily build is also available, thanks to Yury V. Zaytsev and
[NEST](http://nest-initiative.org/).
Dartmouth College.
* [download](https://downloads.kitenet.net/git-annex/autobuild/windows/) ([build logs](https://qa.nest-initiative.org/view/msysGit/job/msysgit-git-annex-assistant-test/))

View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="davidriod@e75b369a4b1cced29c14354bce7493c61f00b1c7"
nickname="davidriod"
avatar="http://cdn.libravatar.org/avatar/d6e327bd88b88802d6f0c20c83f682a2"
subject="Sharing rsync special remote between repository"
date="2016-11-24T19:23:42Z"
content="""
I was wondering if it is possible to share a rsync special remote between repository which are not parented in any way. The use case would be that even if these repositories are not related at all they still may contains the same binary file. It would be useful to have a single rsync remote in order to reduce space usage. I think it could work as the object names are based on their checksum, but I wonder if anyone has already try that ?
"""]]

View file

@ -0,0 +1,82 @@
[[!comment format=mdwn
username="StephaneGourichon"
avatar="http://cdn.libravatar.org/avatar/8cea01af2c7a8bf529d0a3d918ed4abf"
subject="Walkthrough of a prudent retroactive annex."
date="2016-11-24T11:27:59Z"
content="""
Been using the one-liner. Despite the warning, I'm not dead yet.
There's much more to do than the one-liner.
This post offers instructions.
# First simple try: slow
Was slow (estimated >600s for 189 commits).
# In tmpfs: about 6 times faster
I have cloned repository into /run/user/1000/rewrite-git, which is a tmpfs mount point. (Machine has plenty of RAM.)
There I also did `git annex init`, git-annex found its state branches.
On second try I also did
git checkout -t remotes/origin/synced/master
So that filter-branch would clean that, too.
There, `filter-branch` operation finished in 90s first try, 149s second try.
`.git/objects` wasn't smaller.
# Practicing reduction on clone
This produced no visible benefit:
time git gc --aggressive
time git repack -a -d
Even cloning and retrying on clone. Oh, but I should have done `git clone file:///path` as said on git-filter-branch man page's section titled \"CHECKLIST FOR SHRINKING A REPOSITORY\"
This (as seen on https://rtyley.github.io/bfg-repo-cleaner/ ) was efficient:
git reflog expire --expire=now --all && git gc --prune=now --aggressive
`.git/objects` shrunk from 148M to 58M
All this was on a clone of the repo in tmpfs.
# Propagating cleaned up branches to origin
This confirmed that filter-branch did not change last tree:
git diff remotes/origin/master..master
git diff remotes/origin/synced/master synced/master
This, expectedly, was refused:
git push origin master
git push origin synced/master
On origin, I checked out the hash of current master, then on tmpfs clone
git push -f origin master
git push -f origin synced/master
Looks good.
I'm not doing the aggressive shrink now, because of the \"two orders of magnitude more caution than normal filter-branch\" recommended by arand.
# Now what? Check if precious not broken
I'm planning to do the same operation on the other repos, then :
* if everything seems right,
* if `git annex sync` works between all those fellows
* etc,
* then I would perform the reflog expire, gc prune on some then all of them, etc.
Joey, does this seem okay? Any comment?
"""]]

View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="scottgorlin@a32946b2aad278883c1690a0753241583a9855b9"
nickname="scottgorlin"
avatar="http://cdn.libravatar.org/avatar/2dd1fc8add62bbf4ffefac081b322563"
subject="Coldline"
date="2016-11-21T00:49:23Z"
content="""
Wanted to add that \"storageclass=COLDLINE\" appears to work seamlessly, both from my mac and arm NAS. As far as I can tell, this appears to be a no-brainer vs glacier - builtin git annex client, simpler/cheaper billing, and no 4 hour delay!
"""]]

View file

@ -0,0 +1,24 @@
Sometimes a name has been used for a special remote, and you want to change
the name. A common reason is that the special remote has become dead, and
you want to reuse the name for a new special remote.
Initremote prevents reusing a name when the old one exists, even if the old
one is dead. And that makes sense in general, because a dead remote can
come back sometimes, and that would leave the repo with two special remotes
with the same name, and so enableremote would need to be run with a uuid
instead of a name to specify which one to enable, which is not a desirable
state of affairs.
So, add `git annex renameremote oldname newname`. This could also do a `git
remote rename`, or equivilant. (`git remote rename` gets confused by special
remotes not having a fetch url and fails; this can be worked around by
manually renaming the stanza in git config.)
Implementing that would need a way to remove the old name from remote.log.
We can't remove lines from union merged files, but what we could do is
add a new line like:
- name=oldname timestamp=<latest>
And in parsing remote.log, if the UUID is "-", don't include the
remote with that name in the the resulting map.