From 23ff3521362bf03352e1284d48f88f2512795651 Mon Sep 17 00:00:00 2001 From: oliv5 Date: Mon, 16 Jul 2018 17:04:06 +0000 Subject: [PATCH 1/3] Added a comment --- ...comment_4_cd56ee6915acb9389876b7a4b7c0e728._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/forum/Shared_pubkeys__58___decrypting_files_in_special_remotes_without_git-annex/comment_4_cd56ee6915acb9389876b7a4b7c0e728._comment diff --git a/doc/forum/Shared_pubkeys__58___decrypting_files_in_special_remotes_without_git-annex/comment_4_cd56ee6915acb9389876b7a4b7c0e728._comment b/doc/forum/Shared_pubkeys__58___decrypting_files_in_special_remotes_without_git-annex/comment_4_cd56ee6915acb9389876b7a4b7c0e728._comment new file mode 100644 index 0000000000..a063c28025 --- /dev/null +++ b/doc/forum/Shared_pubkeys__58___decrypting_files_in_special_remotes_without_git-annex/comment_4_cd56ee6915acb9389876b7a4b7c0e728._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="oliv5" + avatar="http://cdn.libravatar.org/avatar/d7f0d33c51583bbd8578e4f1f9f8cf4b" + subject="comment 4" + date="2018-07-16T17:04:06Z" + content=""" +Yes, I had a look at the cipher, and I cannot tell much about it. It is ... random :) And yes, I tried the removal of the '\n' characters like in pubkey encryption scheme, with no success so far. + +I'll post my test script, which is a basic variation of the one given in [Decrypting_files_in_special_remotes_without_git-annex](https://git-annex.branchable.com/tips/Decrypting_files_in_special_remotes_without_git-annex/) to make multiple calls to lookup_key(). +"""]] From 1f8d04d34c53d2712e79a0ba63553b146d982718 Mon Sep 17 00:00:00 2001 From: andrew Date: Mon, 16 Jul 2018 22:58:30 +0000 Subject: [PATCH 2/3] Added a comment --- ...t_5_feda43ef61bc7676c3fd8c9308605650._comment | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 doc/forum/Shared_pubkeys__58___decrypting_files_in_special_remotes_without_git-annex/comment_5_feda43ef61bc7676c3fd8c9308605650._comment diff --git a/doc/forum/Shared_pubkeys__58___decrypting_files_in_special_remotes_without_git-annex/comment_5_feda43ef61bc7676c3fd8c9308605650._comment b/doc/forum/Shared_pubkeys__58___decrypting_files_in_special_remotes_without_git-annex/comment_5_feda43ef61bc7676c3fd8c9308605650._comment new file mode 100644 index 0000000000..8f8f936b06 --- /dev/null +++ b/doc/forum/Shared_pubkeys__58___decrypting_files_in_special_remotes_without_git-annex/comment_5_feda43ef61bc7676c3fd8c9308605650._comment @@ -0,0 +1,16 @@ +[[!comment format=mdwn + username="andrew" + avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435" + subject="comment 5" + date="2018-07-16T22:58:30Z" + content=""" +OK. Thanks for all of the patience with the back-and-forth. After trying to follow these steps and re-reading your posts I believe that what you are trying to do is unfortunately not possible with your current setup. + +Although you are using public keys to encrypt your content, `git-annex` is hashing the original key names (using a one-way hash) aka SHA1 Digest signed with your public key before storing on the special remote. The public key is just used by the digest algorithm for signing and does not enable you to recover the hashed key. The `git-annex` special remote protocol does not require special remotes to actually store `git-annex` keys, it only requires that special remotes can retrieve content *given* a key. + +In Joey's source code `Crypto.hs` in reference to this key generation process he does have a comment to this effect, \"The encryption does not need to be reversable\". I assume Joey used hashes for simplicity and so the filenames could stay short. + +In [Future proofing / disaster recovery with an encrypted special remote](http://git-annex.branchable.com/forum/Future_proofing___47___disaster_recovery_with_an_encrypted_special_remote/) Joey also mentions “That's what's \"special\" about special remotes vs regular git remotes: They only store the content of annexed files and not the git repository. Back up the git repository separately” + +So, it seems, in order to recover the original key names you will either have to keep a backup of the original repository or create a new special remote that stores these in a recoverable fashion (instead of using a digest). Perhaps some git commit hook that zips up the `.git` directory and adds it to your repository could be of use? +"""]] From 08570f7330d631e0cc0eeb7ef3723c4551aa6f7b Mon Sep 17 00:00:00 2001 From: jlebar Date: Tue, 17 Jul 2018 05:12:04 +0000 Subject: [PATCH 3/3] Added a comment --- ..._2a6ef51c96a914be5d248b526ad37394._comment | 37 +++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100644 doc/bugs/git_fsck_duplicateEntries_errors_when_using_adjusted_branch/comment_3_2a6ef51c96a914be5d248b526ad37394._comment diff --git a/doc/bugs/git_fsck_duplicateEntries_errors_when_using_adjusted_branch/comment_3_2a6ef51c96a914be5d248b526ad37394._comment b/doc/bugs/git_fsck_duplicateEntries_errors_when_using_adjusted_branch/comment_3_2a6ef51c96a914be5d248b526ad37394._comment new file mode 100644 index 0000000000..d2edf68b24 --- /dev/null +++ b/doc/bugs/git_fsck_duplicateEntries_errors_when_using_adjusted_branch/comment_3_2a6ef51c96a914be5d248b526ad37394._comment @@ -0,0 +1,37 @@ +[[!comment format=mdwn + username="jlebar" + avatar="http://cdn.libravatar.org/avatar/9fca4b61a1ab555f231851e7543f9a3e" + subject="comment 3" + date="2018-07-17T05:12:03Z" + content=""" +Thanks a lot, Joey. + +I wrote a script that replaces each directory or filename with a salted hash: + + #!/usr/bin/env python + + import sys + import hashlib + + def hash(s): + m = hashlib.sha256() + m.update(s) + m.update('') + return m.hexdigest() + + for line in sys.stdin: + print '/'.join(hash(p) for p in line.split('/')) + +Then I ran + + $ git ls-files | python hash_paths.py | bzip2 > repo_paths.bz2 # attached + +To make something you can correlate with the git fsck errors, I ran + + $ git fsck |& grep duplicateEntries | cut -f 4 -d ' ' | sed -e 's/://' | xargs -n1 git ls-tree | grep -v ' tree ' > blobs + $ paste <(cut -f 1 blobs) <(cut -f 2 blobs | python hash_paths.py) | bzip2 > fsck_errors.bz2 # attached + +So the second column in fsck_errors is the salted+hashed filename like in the comment above. You should be able to correlate the \"filenames\" in fsck_errors with the paths in repo_paths. + +I'll email you the relevant files. +"""]]