remove old closed bugs and todo items to speed up wiki updates and reduce size
Remove closed bugs and todos that were last edited or commented before 2022. Except for ones tagged projects/* since projects like datalad want to keep around records of old deleted bugs longer. Command line used: for f in $(grep -l '|done\]\]' -- ./*.mdwn); do if ! grep -q "projects/" "$f"; then d="$(echo "$f" | sed 's/.mdwn$//')"; if [ -z "$(git log --since=01-01-2022 --pretty=oneline -- "$f")" -a -z "$(git log --since=01-01-2022 --pretty=oneline -- "$d")" ]; then git rm -- "./$f" ; git rm -rf "./$d"; fi; fi; done for f in $(grep -l '\[\[done\]\]' -- ./*.mdwn); do if ! grep -q "projects/" "$f"; then d="$(echo "$f" | sed 's/.mdwn$//')"; if [ -z "$(git log --since=01-01-2022 --pretty=oneline -- "$f")" -a -z "$(git log --since=01-01-2022 --pretty=oneline -- "$d")" ]; then git rm -- "./$f" ; git rm -rf "./$d"; fi; fi; done
This commit is contained in:
Joey Hess
parent
acdd5fbab6
commit
4d90053e17
427 changed files with 0 additions and 15690 deletions
|
|
@ -1,13 +0,0 @@
|
|||
### Please describe the problem.
|
||||
|
||||
cryptonite supports the hash skein512_256, which is just a truncated version of skein512. Could you please add support for using this as a key-value backend? Please note that cryptohash doesn't support this hash, so it'll have to go into the ifdef block alongside the sha3 hashes.
|
||||
|
||||
Rationale: I've just done a benchmark of the different hash implementations in cryptonite, and the skein hashes beat all other hashes (except md5) by a large margin, so I'd like to use them. As with the other hashes, skein512 is a bit faster than skein256 (on x86-64), but I'd like to avoid the overly long file names it produces.
|
||||
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
I'm just trying git-annex for the first time, but it definitly looks great.
|
||||
|
||||
> Ok, going to reject this since the speedup is minimal, and there are
|
||||
> other faster hashes anyway, like blake2. [[done]] --[[Joey]]
|
||||
|
|
@ -1,15 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 1"""
|
||||
date="2016-10-31T16:54:30Z"
|
||||
content="""
|
||||
What kind of performance benefit are we talking about here?
|
||||
|
||||
It seems it would be a bit hard to implement this, since Backend.Hash uses
|
||||
a HashSize that is just an Int, which would have to be extended with the
|
||||
number of bits to use of a longer hash. Not hard hard but it will
|
||||
complicate the code.
|
||||
|
||||
(Also, the name "SKEIN512_256" is pretty long itself to include in a
|
||||
filename.")
|
||||
"""]]
|
||||
|
|
@ -1,13 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="aranea@650e41fad422f2a4d6f36ca1f20d41b7c0f18ab7"
|
||||
nickname="aranea"
|
||||
avatar="http://cdn.libravatar.org/avatar/8574023ce00757ca95b1708b7306602a"
|
||||
subject="comment 2"
|
||||
date="2016-10-31T21:29:42Z"
|
||||
content="""
|
||||
> What kind of performance benefit are we talking about here?
|
||||
|
||||
Not much. In my benchmark, SKEIN512 was about 10% faster thank SKEIN256, and SKEIN256 has quite reasonable speed anyway. Thus, adding SKEIN512_256 is not neccessary if it complicated the code too much.
|
||||
|
||||
Thanks anyway!
|
||||
"""]]
|
||||
|
|
@ -1,47 +0,0 @@
|
|||
### Please describe the problem.
|
||||
|
||||
Adding new files to a gcrypt remote ends up eating all inodes.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
I have a docs directory, 780 files in it, various types, various subdirectories. I add these to a git annex repo. Create a gcrypt remote on a usb drive and use assistant to copy. Eventually all sorts of stuff stops working since there are no inodes left.
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
Arch Linux 64 bit, stable. Git annex 6.20160613-8
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
|
||||
Remote bare repo looks as follows:
|
||||
|
||||
annex/objects - contains 4096 directories (000, 001, ...)
|
||||
|
||||
annex/objects/XXX - contains ~1380 directories (001, ...)
|
||||
|
||||
annex/objects/XXX/XXX - contains two or three directories, e.g. GPGHMACSHA1--3939af6b89c30015490ce9e19f9051bb9e9fe64e
|
||||
|
||||
annex/objects/XXX/XXX/GPGHMACSHA1-XXXX - contains one or two files of the same name
|
||||
|
||||
Let's multiply that out: 4096 * 1380 * 2 * 2 = 22,609,920
|
||||
|
||||
Plus . and .. for all the directories, and boom, all my 59 million inodes are gone, before I even manage to store my 780 files!
|
||||
|
||||
Disk is about 20% full.
|
||||
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
|
||||
|
||||
[[!tag moreinfo]]
|
||||
|
||||
> [[closing|done]], the bug reporter did not seem to show that this was a
|
||||
> bug and not just an unfortunate case of too many versions of files.
|
||||
> --[[Joey]]
|
||||
|
|
@ -1,15 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 1"""
|
||||
date="2016-10-05T16:56:25Z"
|
||||
content="""
|
||||
If your remote contains 4096*1380 "GPGHMACSHA1" directories
|
||||
with files in them then there are apparently 5652480 annexed objects
|
||||
stored in this remote... not 780.
|
||||
|
||||
Unless the files in those directories are not the encrypted content
|
||||
of annexed objects, but I don't know what else they could be.
|
||||
|
||||
You'll need to follow up with more information about what these files are,
|
||||
or a way to reproduce this.
|
||||
"""]]
|
||||
File diff suppressed because it is too large
Load diff
|
|
@ -1,13 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 3"""
|
||||
date="2016-10-26T18:07:51Z"
|
||||
content="""
|
||||
Well these look like the names I'd expect to see used for encrypted files.
|
||||
|
||||
If you had the assistant watching some files that were frequently changing,
|
||||
then it could lead to something like this, since many commits would be made
|
||||
of many versions of a file, and each version backed up to a new encrypted
|
||||
file in the special remote. You can take a look at `git log -S` in the git
|
||||
repository and see if there are many commits of some of your files.
|
||||
"""]]
|
||||
|
|
@ -1,63 +0,0 @@
|
|||
### Please describe the problem.
|
||||
Often, when I `git annex get` a large file over ssh, it will hang in the middle, and I'll have to C-c to cancel it, and try again. This strategy works, but I often have to keep canceling and retrying 5-10 times for a large file. This is unfortunate. I think maybe rsync is getting overwhelmed.
|
||||
|
||||
Request: Could you please add a flag --autoretry so that if `get` hangs, cancel and try again. This generally works, because of the rsync --partial flag, so it eventually gets to the end of the file. Thanks!
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
git-annex version: 6.20160511-1~ubuntu14.04.1~ppa1
|
||||
Ubuntu 14.04
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
|
||||
$ git annex get --debug
|
||||
[2016-08-26 15:59:19.476393] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--cached","-z","--"]
|
||||
get clerk/H101-2.json [2016-08-26 15:59:19.483388] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"]
|
||||
[2016-08-26 15:59:19.485433] process done ExitSuccess
|
||||
[2016-08-26 15:59:19.485676] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
|
||||
[2016-08-26 15:59:19.48755] process done ExitSuccess
|
||||
[2016-08-26 15:59:19.487727] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/git-annex..4455cba13d289aad3c25ddc097b5ca02e7fc0ffb","--pretty=%H","-n1"]
|
||||
[2016-08-26 15:59:19.489793] process done ExitSuccess
|
||||
[2016-08-26 15:59:19.4899] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/git-annex..7af97d408ed894922cd08bd2091dab5bb1cf1df9","--pretty=%H","-n1"]
|
||||
[2016-08-26 15:59:19.49133] process done ExitSuccess
|
||||
[2016-08-26 15:59:19.491939] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
|
||||
(from origin...)
|
||||
[2016-08-26 15:59:19.495688] read: rsync ["--progress","--inplace","--perms","-e","'ssh' '-S' '.git/annex/ssh/voteviewdev' '-o' 'ControlMaster=auto' '-o' 'ControlPersist=yes' '-T' 'voteviewdev' 'git-annex-shell ''sendkey'' ''/srv/annex'' ''--debug'' ''SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json'' --uuid ff59187e-f48c-47d0-b75d-320a6a4ce115 ''--'' ''remoteuuid=da1361a1-3f1a-47b8-8165-daf8c30d543d'' ''unlocked='' ''direct='' ''associatedfile=clerk/H101-2.json'' ''--'''","--","dummy:",".git/annex/tmp/SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json"]
|
||||
[2016-08-26 15:58:37.018182] transfer start
|
||||
[2016-08-26 15:58:37.023135] call: rsync ["--server","-t","--inplace","-e.Lsf",".","--sender","../../srv/annex/.git/annex/objects/2G/fP/SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json/SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json"]
|
||||
SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json
|
||||
49,101,968 98% 46.83MB/s 0:00:00 [2016-08-26 15:59:22.809273] feed: ssh ["-S",".git/annex/ssh/voteviewdev","-o","ControlMaster=auto","-o","ControlPersist=yes","-T","voteviewdev","git-annex-shell 'transferinfo' '/srv/annex' '--debug' 'SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json' --uuid ff59187e-f48c-47d0-b75d-320a6a4ce115 '--' 'remoteuuid=da1361a1-3f1a-47b8-8165-daf8c30d543d' 'associatedfile=clerk/H101-2.json' '--'"]
|
||||
49,892,641 100% 39.52MB/s 0:00:01 (xfr#1, to-chk=0/1)
|
||||
[2016-08-26 15:58:39.034666] process done ExitSuccess
|
||||
[2016-08-26 15:58:39.038811] transfer done
|
||||
[2016-08-26 15:59:23.09308] process done ExitSuccess
|
||||
(checksum...) [2016-08-26 15:59:23.093582] read: sha256sum [".git/annex/tmp/SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json"]
|
||||
[2016-08-26 15:59:23.331872] process done ExitSuccess
|
||||
ok
|
||||
get clerk/H102-1.json (from origin...)
|
||||
[2016-08-26 15:59:23.337838] read: rsync ["--progress","--inplace","--perms","-e","'ssh' '-S' '.git/annex/ssh/voteviewdev' '-o' 'ControlMaster=auto' '-o' 'ControlPersist=yes' '-T' 'voteviewdev' 'git-annex-shell ''sendkey'' ''/srv/annex'' ''--debug'' ''SHA256E-s41311329--69c3b054a3fe9676133605730d85b7fcef8696f6782d402a524e41b836253891.json'' --uuid ff59187e-f48c-47d0-b75d-320a6a4ce115 ''--'' ''remoteuuid=da1361a1-3f1a-47b8-8165-daf8c30d543d'' ''unlocked='' ''direct='' ''associatedfile=clerk/H102-1.json'' ''--'''","--","dummy:",".git/annex/tmp/SHA256E-s41311329--69c3b054a3fe9676133605730d85b7fcef8696f6782d402a524e41b836253891.json"]
|
||||
[2016-08-26 15:59:23.351903] process done ExitSuccess
|
||||
[2016-08-26 15:58:39.394898] transfer start
|
||||
[2016-08-26 15:58:39.400323] call: rsync ["--server","-t","--inplace","-e.Lsf",".","--sender","../../srv/annex/.git/annex/objects/6Z/QP/SHA256E-s41311329--69c3b054a3fe9676133605730d85b7fcef8696f6782d402a524e41b836253891.json/SHA256E-s41311329--69c3b054a3fe9676133605730d85b7fcef8696f6782d402a524e41b836253891.json"]
|
||||
SHA256E-s41311329--69c3b054a3fe9676133605730d85b7fcef8696f6782d402a524e41b836253891.json
|
||||
1,998,848 4% 1.91MB/s 0:00:20 [2016-08-26 15:59:24.608637] feed: ssh ["-S",".git/annex/ssh/voteviewdev","-o","ControlMaster=auto","-o","ControlPersist=yes","-T","voteviewdev","git-annex-shell 'transferinfo' '/srv/annex' '--debug' 'SHA256E-s41311329--69c3b054a3fe9676133605730d85b7fcef8696f6782d402a524e41b836253891.json' --uuid ff59187e-f48c-47d0-b75d-320a6a4ce115 '--' 'remoteuuid=da1361a1-3f1a-47b8-8165-daf8c30d543d' 'associatedfile=clerk/H102-1.json' '--'"]
|
||||
10,616,832 25% 1.54MB/s 0:00:19
|
||||
|
||||
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
|
||||
|
||||
[[!meta title="Detect stalled transfer and retry or abort it"]]
|
||||
|
||||
> [[done]], see annex.stalldetection to enable --[[Joey]]
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="adamboche@d77483462c747486c7a7341a9080d1341eeddfe6"
|
||||
nickname="adamboche"
|
||||
subject="Retry rsync on failure"
|
||||
date="2016-08-26T23:32:46Z"
|
||||
content="""
|
||||
This may be useful for providing an in-product solution to this problem.
|
||||
http://unix.stackexchange.com/a/165417/126433
|
||||
"""]]
|
||||
|
|
@ -1,66 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 2"""
|
||||
date="2016-10-26T18:26:35Z"
|
||||
content="""
|
||||
The most common way a network connection can stall like this is when
|
||||
moving to a different wifi network: the connection is open but
|
||||
no more data will be received. I suppose other kinds of network
|
||||
glitches could also lead to this kind of situation.
|
||||
|
||||
ssh has some things, like ServerAliveInterval and TCPKeepAlive,
|
||||
that it can use to detect such problems. You may find them useful.
|
||||
|
||||
----
|
||||
|
||||
As for the retrying once a stall is detected, some transfers use
|
||||
`forwardRetry` which will automatically retry as long as the failed try
|
||||
managed to send some data. But the get/move/copy commands currently use
|
||||
`noRetry`. I can't find any justification for not always using
|
||||
`forwardRetry`; I think that it was added for the assistant originally and
|
||||
the other stuff just never switched over.
|
||||
|
||||
Only problem I can think of is, if there actually is a ssh password
|
||||
prompt, it would prompt again on retry. But most people using git-annex
|
||||
with ssh have something in place to make ssh not prompt repeatedly for
|
||||
passwords.
|
||||
|
||||
So, I've gone ahead and enabled `forwardRetry` for everything.
|
||||
|
||||
----
|
||||
|
||||
Occurs to me that git-annex could try to notice when a transfer is not
|
||||
progressing, by reusing the existing progress metering code.
|
||||
|
||||
Since some remotes don't update the progress meter, this could
|
||||
only be used to detect stalls after the progress meter has been updated
|
||||
at least once. If the stall occurs earlier than that, it would not be able
|
||||
to be detected.
|
||||
|
||||
It seems quite hard to come up with a good timeout value to detect a
|
||||
stalled connection. Often progress meters are updated after every small
|
||||
(eg 32kb) chunk transferred. But others might poll periodically, or might
|
||||
use a larger chunk size. It's even possible that some special remotes
|
||||
are looking at a percent output by some program, and only update the meter
|
||||
when the percent transferred changes -- in which case it could be many
|
||||
minutes in between each meter update when a large file is being
|
||||
transferred.
|
||||
|
||||
If the timeout is too short, git-annex will stall in a new way, by
|
||||
constantly killing "stalled" connections before they can send enough data.
|
||||
|
||||
----
|
||||
|
||||
So it really seems better to fix the ssh connection to not stall, since
|
||||
that is not so heuristic a fix. Seems like git-annex could force
|
||||
ServerAliveInterval to be set, and perhaps lower ServerAliveCountMax from 3
|
||||
to 1. The ssh BatchMode setting sets the former to 300, so a stalled
|
||||
connection will time out after 15 minutes. But BatchMode also disables
|
||||
prompting, and git-annex should not disable that.
|
||||
|
||||
Catch is, what if the user has configured ssh with some
|
||||
other ServerAliveInterval value? We don't want git-annex to override that.
|
||||
|
||||
(git-annex does have a rudimentary .ssh/config parser, but it's not
|
||||
good enough to handle eg, "Host *.example.com ")
|
||||
"""]]
|
||||
|
|
@ -1,28 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 3"""
|
||||
date="2016-10-26T19:55:18Z"
|
||||
content="""
|
||||
On the ssh config, one way to do it is to pass -F with a config
|
||||
file that git-annex generates. It could look like:
|
||||
|
||||
Include ~/.ssh/config
|
||||
Include /etc/ssh/ssh_config
|
||||
ServerAliveInterval 60
|
||||
|
||||
Since ssh uses the first config setting it sees, if `~/.ssh/config`
|
||||
or `/etc/ssh/ssh_config` set a ServerAliveInterval that one will be used,
|
||||
and otherwise the value git-annex sets will be used.
|
||||
|
||||
But.. Ssh enables TCPKeepAlive by default. You'd think that would be enough
|
||||
to detect this kind of problem.
|
||||
|
||||
There do seem to be reasons for users to disable TCPKeepAlive;
|
||||
perhaps it causes annoying disconnects when there's a minor hiccough,
|
||||
or a firewall does not support it.
|
||||
|
||||
If the problem is that users are disabling TCPKeepAlive, then
|
||||
having git-annex enable ServerAliveInterval makes sense.
|
||||
|
||||
Ok; implemented this.
|
||||
"""]]
|
||||
|
|
@ -1,16 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 4"""
|
||||
date="2016-12-13T16:05:42Z"
|
||||
content="""
|
||||
Could the original bug reporter please show what your ~/.ssh/config
|
||||
contains? As far as I can tell, ssh's TCPKeepAlive option, which is
|
||||
supposed to be enabled by default, unless you have disabled it, should
|
||||
avoid such problems.
|
||||
|
||||
It may also help to set ServerAliveInterval.
|
||||
|
||||
Unfortunately, my attempt to make git-annex set ServerAliveInterval
|
||||
when running ssh broke too many systems with old sshed, and I have had to
|
||||
revert it.
|
||||
"""]]
|
||||
|
|
@ -1,30 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 5"""
|
||||
date="2018-03-29T17:36:48Z"
|
||||
content="""
|
||||
We have annex.retry etc configuration now to control retries of transfers.
|
||||
|
||||
Stall detection would be good to have in git-annex. Currently, when a
|
||||
transfer is stalled, the progress meter stops updating, so displays
|
||||
an ETA that's wrong.
|
||||
|
||||
Since the progress meter is already getting called periodically when
|
||||
a transfer is not stalled, a stall detector could perhaps be added
|
||||
into that, to abort stalled transfers. The tricky part being that a few
|
||||
types of transfers don't use the internal progress meters but rely on an
|
||||
external command's progress display, and the stall detection would need to
|
||||
be disabled for those.
|
||||
|
||||
Hmm, a password prompt for eg ssh can also "stall"
|
||||
a transfer for a long time. Perhaps the thing to do is wait until
|
||||
the meter gets updated once, and only then start the stall detector.
|
||||
|
||||
But what would the stall detector do when it does detect a stalled
|
||||
transfer? It should perhaps cancel the transfer action, but I don't know
|
||||
how to do that; the transfer action may have eg run a process, which would
|
||||
need to be canceled, or it may have a network connection open. Simply
|
||||
killing the transfer thread won't stop a process that it started.
|
||||
And when an external special remote is performing the transfer, there's
|
||||
nothing in that protocol to cancel a transfer.
|
||||
"""]]
|
||||
|
|
@ -1,47 +0,0 @@
|
|||
### Please describe the problem.
|
||||
|
||||
After not syncing my Android repo for a while, I tried to sync it. By some combination of starting up the assistant, killing it, running `git annex sync --content`, and killing *it*, I managed to get the Android client (operating in direct mode) to decide that I had manually deleted a whole bunch of files that had in fact just not been created yet, and to create a commit to that effect, which it promptly synced out to my other repos.
|
||||
|
||||
I then synced in my main direct mode crippled filesystem repo with the only copy of some files, and got a bunch of messages that Git Annex was deleting files I wanted to keep. I killed that sync with a ctrl+c.
|
||||
|
||||
My problem (***UPDATE**: solved) is: how do I revert the offending commit and restore my files?
|
||||
|
||||
My other question (not yet solved) is: how do I prevent this happening again? Is there a way I can pre-clear commits and not accept those that delete files without manual confirmation? Or should I just stop being mean to the Android client and hope it doesn't decide to delete things it shouldn't delete again?
|
||||
|
||||
I've tried a `git annex proxy -- git revert HASHOFBADCOMMIT`, but (as I killed Git Annex before it got through recording that it had trashed my files), I just get:
|
||||
|
||||
```
|
||||
error: Your local changes would be overwritten by revert.
|
||||
hint: Commit your changes or stash them to proceed.
|
||||
fatal: revert failed
|
||||
```
|
||||
|
||||
When syncing in a direct mode repo, does Git Annex happily delete the last copy of a file that appears to have been deleted somewhere else? Or does it save it until you manually clean up unused data, by moving it somewhere under .git?
|
||||
|
||||
**UPDATE**: A `git annex sync`, of all things, in the direct mode repository seems to have brought the files and their contents back. It created a commit that undid the deleting commit, except for the deletion of a duplicate copy of one file, which I don't need to have.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
It's not entirely clear. Some combination of interrupting and restarting the Android app can make it think that files have been deleted when they really have never been created.
|
||||
|
||||
Alternately, to get the problem I really want to fix now, delete a file in one repo, sync the delete to a direct-mode repo with the only copy. Then somehow undo the delete from the direct mode repo and restore the content of the file.
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
The PC has 6.20160903-g1c0b2b4 and Android has 6.20160714.
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
|
||||
> Direct mode was known to have some bugs that could probably have led
|
||||
> to this kind of behavior. It has been replaced, so [[done]] --[[Joey]]
|
||||
|
|
@ -1,20 +0,0 @@
|
|||
### Please describe the problem.
|
||||
I ran git-annex unused on a few annexes and they dropped files that were still being used. The annexes were fully synced and git-annex list showed the files in the annex they were dropped from.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
I'm not sure. The files lost appear to be more recently added files, so this bug must be in more recent versions. I will edit and fill this out when I know more.
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
7.20191230-gef6d1e327 on Pop! OS 20.04 (Ubuntu based)
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
I've used git-annex to maintain large file sets for years now and there is no better tool I know of.
|
||||
|
||||
[[!tag moreinfo]]
|
||||
|
||||
> Closing as requested information has not been provided. If you followup
|
||||
> we can reopen it. [[done]] --[[Joey]]
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="Lukey"
|
||||
avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
|
||||
subject="comment 1"
|
||||
date="2020-11-23T19:00:55Z"
|
||||
content="""
|
||||
There are some details missing here. Did git-annex actually drop the last copy of one of the files? Usually git annex will refuse to do this. What does `git annex fsck --fast --quiet` say?
|
||||
"""]]
|
||||
|
|
@ -1,27 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 2"""
|
||||
date="2020-11-24T18:37:58Z"
|
||||
content="""
|
||||
> The annexes were fully synced and git-annex list showed the files in the annex they were dropped from.
|
||||
|
||||
That statement does not make a lot of sense. If both repos have been
|
||||
synced with the other, then git-annex list could only display the file
|
||||
if it was still in the master branch. But then git-annex unused would
|
||||
certianly not have considered it unused. (Unless it has an insanely bad bug,
|
||||
but the test suite tests that it doesn't have that bug, so it seems likely
|
||||
it does not.)
|
||||
|
||||
But also, if both repos were synced with one-another, and you had dropped the
|
||||
file from one repo, then git-annex list in other other repo would know that and
|
||||
would not show the file as still being in it.
|
||||
|
||||
Which suggests you mean something else by "fully synced" than being fully in
|
||||
sync at that point. Or that you were mistaken about them being fully in sync.
|
||||
|
||||
And if you were mistaken about that, it's easy to set up situations where
|
||||
two repos are not fully in sync, and so in one, the file is to all appearances
|
||||
unused, but in the other one, the file is still in use by a branch.
|
||||
|
||||
So, without a way to reproduce this, I have to assume it's user error.
|
||||
"""]]
|
||||
|
|
@ -1,36 +0,0 @@
|
|||
### Please describe the problem.
|
||||
|
||||
Building the latest version from source, whatever method I use eventually fails on the language-javascript-0.7.1.0 dependency. The error (stack output) below.
|
||||
|
||||
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
1. Install ghc 8.10.7 via ghcup
|
||||
2. Check out latest source
|
||||
3. Issue any of the supported build commands, watch the failure eventually happen
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
git master (tag 8.20211123 has the same issue)
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
|
||||
language-javascript > Configuring language-javascript-0.7.1.0...
|
||||
language-javascript > build
|
||||
language-javascript > Preprocessing library for language-javascript-0.7.1.0..
|
||||
language-javascript > happy: src/Language/JavaScript/Parser/Grammar7.y: hGetContents: invalid argument (invalid byte sequence)
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
Yes, it's wonderful.
|
||||
|
||||
> Upstream bug is <https://github.com/simonmar/happy/issues/157>
|
||||
> and this cannot be fixed in git-annex, so [[done]] --[[Joey]]
|
||||
|
|
@ -1,18 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 1"""
|
||||
date="2021-12-13T16:49:37Z"
|
||||
content="""
|
||||
Here is an upstream bug report about the same problem,
|
||||
<https://github.com/erikd/language-javascript/issues/86>
|
||||
|
||||
Unfortunately the author of language-javascript has decided to not support
|
||||
building it except in a unicode locale. So you will need to set
|
||||
LANG or similar to a unicode locale to avoid the problem.
|
||||
|
||||
I don't see anything I can do about this in git-annex, this is not a direct
|
||||
dependency and it is uncreasingly unusual for users to not use a unicode
|
||||
locale, so while I would rather compilers (and everything else) did not crash
|
||||
in these situations, I expect that anyone not using a unicode locale is
|
||||
going to see lots of problems like this in the future.
|
||||
"""]]
|
||||
|
|
@ -1,11 +0,0 @@
|
|||
I did a "git annex add" of a bunch of files on a storage server with low IOPS, and saw this:
|
||||
|
||||
git-annex: /tank/Media/Pictures/.git/annex/tmp/430_32b_SHA256E-s4464838--c1785a76ee1451f602e93c99c147e214705004e541de8256d74a3be3717d15e5.jpg.log: openBinaryFile: resource busy (file is locked)
|
||||
failed
|
||||
|
||||
How is that even possible, when the server is doing nothing else?
|
||||
|
||||
[[!tag moreinfo]]
|
||||
|
||||
> Closing as I've been waiting for more information for 6 years. [[done]]
|
||||
> --[[Joey]]
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="4.152.108.211"
|
||||
subject="comment 1"
|
||||
date="2013-01-05T21:13:09Z"
|
||||
content="""
|
||||
Is this a Solaris system?
|
||||
"""]]
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://me.yahoo.com/a/2grhJvAC049fJnvALDXek.6MRZMTlg--#eec89"
|
||||
nickname="John"
|
||||
subject="comment 2"
|
||||
date="2013-01-20T03:38:43Z"
|
||||
content="""
|
||||
No, it's CentOS 6.3 x86_64.,
|
||||
"""]]
|
||||
|
|
@ -1,28 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="209.250.56.102"
|
||||
subject="moreinfo"
|
||||
date="2014-03-19T20:49:47Z"
|
||||
content="""
|
||||
What I don't understand about this is, how does `open` fail due to a file being locked? This is Linux, it doesn't have mandatory locking that I know of, and git-annex certianly doesn't use such a thing.
|
||||
|
||||
I really need a way to reproduce this and/or a strace. As it is, I've never seen this reported by anyone else and don't understand the failure mode at all.
|
||||
|
||||
The relevant part of the code seems to be here:
|
||||
|
||||
[[!format haskell \"\"\"
|
||||
setJournalFile :: JournalLocked -> FilePath -> String -> Annex ()
|
||||
setJournalFile _jl file content = do
|
||||
tmp <- fromRepo gitAnnexTmpMiscDir
|
||||
createAnnexDirectory =<< fromRepo gitAnnexJournalDir
|
||||
createAnnexDirectory tmp
|
||||
-- journal file is written atomically
|
||||
jfile <- fromRepo $ journalFile file
|
||||
let tmpfile = tmp </> takeFileName jfile
|
||||
liftIO $ do
|
||||
writeBinaryFile tmpfile content
|
||||
moveFile tmpfile jfile
|
||||
\"\"\"]]
|
||||
|
||||
While there is some ctnl locking going on, it locks a special sentinal file, not the file it's writing to.
|
||||
"""]]
|
||||
|
|
@ -1,22 +0,0 @@
|
|||
I may be misunderstanding the purpose of mincopies, but I found this surprising. I would expect annex to not drop the file, because doing so violates the mincopies setting:
|
||||
|
||||
```
|
||||
luckbox:orig patmaddox$ git annex mincopies
|
||||
3
|
||||
luckbox:orig patmaddox$ git annex whereis testmincopies
|
||||
whereis testmincopies (3 copies)
|
||||
1c09b94f-eed3-425d-9bbe-49aa5e575ed9 -- [s3]
|
||||
c2bafb10-bf48-4ae5-a1b9-d142f2bea86a -- patmaddox@luckbox.local:~/Desktop/annex_test [here]
|
||||
c8144467-348b-476d-8464-5dfe98580f0b -- patmaddox@istudo.local:~/Desktop/annex [istudo]
|
||||
ok
|
||||
luckbox:orig patmaddox$ git annex drop testmincopies
|
||||
drop testmincopies (locking istudo...) ok
|
||||
(recording state in git...)
|
||||
luckbox:orig patmaddox$ git annex whereis testmincopies
|
||||
whereis testmincopies (2 copies)
|
||||
1c09b94f-eed3-425d-9bbe-49aa5e575ed9 -- [s3]
|
||||
c8144467-348b-476d-8464-5dfe98580f0b -- patmaddox@istudo.local:~/Desktop/annex [istudo]
|
||||
ok
|
||||
```
|
||||
|
||||
> [[fixed|done]] --[[Joey]]
|
||||
|
|
@ -1,25 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 1"""
|
||||
date="2021-04-27T17:06:45Z"
|
||||
content="""
|
||||
Please, when filing a bug report, especially one that has the potential
|
||||
to be a major data loss bug, provide enough information to reproduce the bug
|
||||
report. This includes a version number, and ideally showing how to create a
|
||||
new repository and configure it, from scratch, so that the bug occurs.
|
||||
|
||||
I suspect that what is happening here is, you don't have numcopies set to a
|
||||
number >= mincopies. And there's code that only checks numcopies, with the
|
||||
assumption that if it makes sure numcopies is satisfied, mincopies must be
|
||||
satisfied since it's a smaller or equal number.
|
||||
|
||||
Of course, that's just a guess, but it was the first way I was able to
|
||||
reproduce a behavior like you show.
|
||||
|
||||
I guess that's a bug, but it's not a major data loss bug since numcopies
|
||||
defaults to 1 it will preserve at least that many. Also, it's not a very
|
||||
wise configuration because mincopies is a new feature and if an older
|
||||
version of git-annex, that does not support it, is used with this
|
||||
repository, it will not know about mincopies and will only enforce
|
||||
numcopies.
|
||||
"""]]
|
||||
|
|
@ -1,58 +0,0 @@
|
|||
### Please describe the problem.
|
||||
|
||||
Trying to execute an external remote shell script on windows fails.
|
||||
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
$git annex initremote test type=external externaltype=rclone encryption=none
|
||||
initremote test git-annex: Cannot run git-annex-remote-rclone -- Make sure it's in your PATH and is executable.
|
||||
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
windows, using git-bash
|
||||
|
||||
git-annex version: 6.20170214-g2233a50
|
||||
build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV ConcurrentOutput TorrentParser Feeds Quvi
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
|
||||
local repository version: 5
|
||||
supported repository versions: 3 5 6
|
||||
upgrade supported from repository versions: 2 3 4 5
|
||||
operating system: mingw32 i386
|
||||
|
||||
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
I've also tried fooling around with the proxy, which may help with pinpointing the issue?
|
||||
|
||||
[[!format sh """
|
||||
$ git-annex-remote-rclone
|
||||
VERSION 1
|
||||
|
||||
$ git annex proxy -- git-annex-remote-rclone
|
||||
git-annex: git-annex-remote-rclone: createProcess: does not exist (No such file or directory)
|
||||
failed
|
||||
git-annex: proxy: 1 failed
|
||||
|
||||
$ git annex proxy -- which git-annex-remote-rclone
|
||||
/d/bin/git-annex-remote-rclone
|
||||
|
||||
$ git annex proxy -- /d/bin/git-annex-remote-rclone
|
||||
git-annex: D:/bin/git-annex-remote-rclone: createProcess: invalid argument (Exec format error)
|
||||
failed
|
||||
git-annex: proxy: 1 failed
|
||||
|
||||
$ git annex proxy -- sh /d/bin/git-annex-remote-rclone
|
||||
VERSION 1
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
Oh yeah! This software is awesome. After getting used to having "dummy" shortcuts to content I don't currently have, with the simple ability to get/drop that content, I can't believe I haven't seen this anywhere before. If there is anything more impressive than this software, it's the support it has had from Joey over all this time. I'd have pulled my hair out long ago. :P
|
||||
|
||||
> [[fixed|done]] although untested --[[Joey]]
|
||||
|
|
@ -1,11 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="jason.dixon.email@aa0e536a2ec2877d6f666108dbbc6e39bbe87ac0"
|
||||
nickname="jason.dixon.email"
|
||||
avatar="http://cdn.libravatar.org/avatar/fbe9050fc83bbd536d307d87ea14d4bc"
|
||||
subject="Perhaps relevant"
|
||||
date="2017-03-08T17:31:30Z"
|
||||
content="""
|
||||
Not sure if this is helpful or relevant, but it may be?
|
||||
|
||||
https://github.com/jgm/pandoc/issues/3247#issuecomment-262036875
|
||||
"""]]
|
||||
|
|
@ -1,16 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 2"""
|
||||
date="2017-03-08T19:20:21Z"
|
||||
content="""
|
||||
git-annex on Windows already checks if external special remotes start
|
||||
with #! and tries to handle the shebang, since Windows does not do that
|
||||
natively. That was added in version 6.20160907.
|
||||
([[todo/refactor_shebang_handling_code_for_wider_use]])
|
||||
|
||||
`git-annex proxy` does not do that, and I feel it's out of scope for it to
|
||||
do so. So the proxy stuff you tried is a red herring.
|
||||
|
||||
It might be useful to use --debug to see what command git-annex tries to
|
||||
execute when running the external special remote program.
|
||||
"""]]
|
||||
|
|
@ -1,14 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 3"""
|
||||
date="2017-03-08T19:36:57Z"
|
||||
content="""
|
||||
Looking into it a bit more, the problem seems to be that findShellCommand
|
||||
expects a path to a file to examine, but when it's used for an external
|
||||
special remote, it's only given the name of the command.
|
||||
|
||||
So, I fixed it by searching for the command in the PATH.
|
||||
|
||||
I have still not tested if this works on Windows, but probably, I think.
|
||||
As long as PATH is set on Windows at least.
|
||||
"""]]
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="jason.dixon.email@aa0e536a2ec2877d6f666108dbbc6e39bbe87ac0"
|
||||
nickname="jason.dixon.email"
|
||||
avatar="http://cdn.libravatar.org/avatar/fbe9050fc83bbd536d307d87ea14d4bc"
|
||||
subject="re: comment 3"
|
||||
date="2017-03-08T21:58:14Z"
|
||||
content="""
|
||||
Awesome, thanks Joey. Yeah that sounds plausible. I'll check it out whenever the next version comes by. Thanks. Keep up the amazing work! :)
|
||||
"""]]
|
||||
|
|
@ -1,53 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="datamanager"
|
||||
avatar="http://cdn.libravatar.org/avatar/7d4ca7c5e571d4740ef072b83a746c12"
|
||||
subject="I think I am having the same issue"
|
||||
date="2021-04-28T01:19:46Z"
|
||||
content="""
|
||||
Hello, I think I am having the same issue, though some particulars seem different. It's marked as solved, but what exactly is the solution?
|
||||
|
||||
I tried running `git annex enableremote --debug google-drive`, and got this output:
|
||||
|
||||
```
|
||||
[2021-04-28 01:11:19.033938395] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"git-annex\"]
|
||||
[2021-04-28 01:11:19.880161468] process done ExitSuccess
|
||||
[2021-04-28 01:11:19.881748395] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"]
|
||||
[2021-04-28 01:11:20.742986051] process done ExitSuccess
|
||||
[2021-04-28 01:11:20.760170947] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..b8c2aac9d4d666ae5c8ee2041c643052ecf7dcbe\",\"--pretty=%H\",\"-n1\"]
|
||||
[2021-04-28 01:11:21.239519332] process done ExitSuccess
|
||||
[2021-04-28 01:11:21.240506519] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..bd130925066b8c62855ac88d4640bda4b4d14afa\",\"--pretty=%H\",\"-n1\"]
|
||||
[2021-04-28 01:11:21.621569176] process done ExitSuccess
|
||||
[2021-04-28 01:11:21.627188551] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"cat-file\",\"--batch\"]
|
||||
[2021-04-28 01:11:21.629520009] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"cat-file\",\"--batch-check=%(objectname) %(objecttype) %(objectsize)\"]
|
||||
enableremote google-drive [2021-04-28 01:11:22.153897457] chat: /data/data/com.termux/files/usr/bin/git-annex-remote-rclone []
|
||||
(encryption update) (to gpg keys: redacted) [2021-04-28 01:11:22.164967925] chat: /data/data/com.termux/files/usr/bin/git-annex-remote-rclone []
|
||||
|
||||
git-annex: Cannot run /data/data/com.termux/files/usr/bin/git-annex-remote-rclone -- Make sure it's executable and that its dependencies are installed.
|
||||
failed
|
||||
[2021-04-28 01:11:22.1806623] process done ExitSuccess
|
||||
[2021-04-28 01:11:22.184542509] process done ExitSuccess
|
||||
git-annex: enableremote: 1 failed
|
||||
```
|
||||
|
||||
This remote is configured with gpg, so I tried a test repository without encryption. I attempted to initialize it with `git annex initremote --debug google-drive type=external externaltype=rclone target=google-drive prefix=git-annex-android-test chunk=50MiB encryption=shared mac=HMACSHA512 rclone_layout=lower`, and got this output:
|
||||
|
||||
```
|
||||
[2021-04-28 01:17:33.983261638] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"git-annex\"]
|
||||
[2021-04-28 01:17:34.314992575] process done ExitSuccess
|
||||
[2021-04-28 01:17:34.315285908] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"]
|
||||
[2021-04-28 01:17:34.774339919] process done ExitSuccess
|
||||
[2021-04-28 01:17:34.810030752] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..ae1ad94cc2cf05b139560772923f5196793a1015\",\"--pretty=%H\",\"-n1\"]
|
||||
[2021-04-28 01:17:35.111290491] process done ExitSuccess
|
||||
[2021-04-28 01:17:35.125354033] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"cat-file\",\"--batch\"]
|
||||
[2021-04-28 01:17:35.129850856] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"cat-file\",\"--batch-check=%(objectname) %(objecttype) %(objectsize)\"]
|
||||
initremote google-drive [2021-04-28 01:17:35.690712627] chat: /data/data/com.termux/files/usr/bin/git-annex-remote-rclone []
|
||||
|
||||
git-annex: Cannot run /data/data/com.termux/files/usr/bin/git-annex-remote-rclone -- Make sure it's executable and that its dependencies are installed.
|
||||
failed
|
||||
[2021-04-28 01:17:35.70162346] process done ExitSuccess
|
||||
[2021-04-28 01:17:35.703906741] process done ExitSuccess
|
||||
git-annex: initremote: 1 failed
|
||||
```
|
||||
|
||||
Hopefully someone can help me, thanks!
|
||||
"""]]
|
||||
|
|
@ -1,15 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 6"""
|
||||
date="2021-07-13T16:58:59Z"
|
||||
content="""
|
||||
@datamanager this bug report is about a Windows-specific problem,
|
||||
and it was fixed in 2017. You seem to be using Android, unless
|
||||
Windows has gotten a termux app that I am not aware of?
|
||||
|
||||
It seems likely to me that if you look at
|
||||
/data/data/com.termux/files/usr/bin/git-annex-remote-rclone,
|
||||
it will start with "#!/bin/bash", and perhaps your android device
|
||||
does not have a /bin/bash to run it with. Modifying the script to
|
||||
use the path to termux's bash would probably fix that.
|
||||
"""]]
|
||||
|
|
@ -1,30 +0,0 @@
|
|||
### Please describe the problem.
|
||||
I normally use two clients with the assistant (lets call them A and B) and they are usually online at different times.
|
||||
Sometimes, after deleting a file in A, I turn on the client in B and when rescanning, instead of deleting that same file from B, it adds it back to A.
|
||||
|
||||
I can always delete the file again, but it's a bit annoying. Anyway, it's a lot better than data loss ;)
|
||||
|
||||
Might be worthy to mention that the computer running client B is quite slow and has a slow HD. I have 4 git-annex repositories and when I turn on the computer the rescan can easily take up a couple of hours.
|
||||
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
B -> is turned off
|
||||
|
||||
A -> delete file
|
||||
|
||||
B -> turn on and wait for rescan
|
||||
|
||||
B -> readds the file instead of deleting it
|
||||
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
6.20160923 (Debian Sid)
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
I've been using git-annex for years with 4 different repositories, 3 of them pretty big (> 50 GB) and I'm really happy! :)
|
||||
|
||||
> This sounds like the git-annex assistant, and probably direct mode,
|
||||
> which was pretty buggy. Its replacement avoids those problems,
|
||||
> so I'm guessing the bug got fixed. If you still see it, please comment.
|
||||
> [[done]] --[[Joey]]
|
||||
|
|
@ -1,27 +0,0 @@
|
|||
Apologies if this isn't the right section. I considered posting it at https://git-annex.branchable.com/tips/unlocked_files/ but thought this might be a better place.
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
v7
|
||||
|
||||
First of all, I want to say that I'm a *HUGE* fan of Joey and git annex. I've been using and following it for 5+ years now, and absolutely love it.
|
||||
|
||||
Unfortunately, now for the first time I saw a change that worries me. One of the core tenets of git annex is "future proofing" in a "world that's forgotten git and git annex" and the latest changes go against that. There seems to be a lot of "magic" under the covers with smudge and clean filters, no longer using standard symlinks to identify files and instead relying on smudge filters which would make life without git and git annex _very_ painful.
|
||||
|
||||
I understand that this is an improvement over direct mode, but I view direct mode as a crutch only for people/hosts who need it (and I've used it too). I wish the new "unlocked" files behaved the same way. Just because _one_ of my hosts needs a crutch doesn't mean the rest of them (and the git history) all need to have the data identified by a string that relies on git annex internals. /annex/objects/xx isn't too complicated, but this violates the core tenet that got me on board git annex on day 1.
|
||||
|
||||
One more concern I have is the fact that "git add" now adds files in this mode, which is just a big no no. Not even lfs does this with all the complicated smudge/clean work it does.
|
||||
|
||||
(On a closing note, I want to add that knowing Joey, he's probably thought through all of this and has very valid reasons it was this way.)
|
||||
|
||||
TL;DR
|
||||
Please provide support (or better still enable as default) the following behavior:
|
||||
|
||||
* "git add" just adds files to git, not annex
|
||||
* The v7 unlocked mode stores data in git annex(I mean the git-annex branch), and ideally uses standard mechanism like symlinks to track data, with smudge/clean filters used *only* when necessary. (My guess is that the reason it is this way now is because git probably doesn't like doing a type change of a file)
|
||||
|
||||
|
||||
|
||||
> I don't think this is a bug, if you don't like unlocked files you do not
|
||||
> need to use them, or if you're using someone else's repo you can `git
|
||||
> annex adjust --lock` to convert from unlocked to locked. So [[done]]
|
||||
> --[[Joey]]
|
||||
|
|
@ -1,14 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 1"""
|
||||
date="2018-12-18T16:11:58Z"
|
||||
content="""
|
||||
> Just because _one_ of my hosts needs a crutch doesn't mean the rest of them (and the git history) all need to have the data identified by a string that relies on git annex internals
|
||||
|
||||
So use `git annex adjust --unlock` on that one host and the rest won't see it.
|
||||
|
||||
> /annex/objects/xx isn't too complicated
|
||||
|
||||
It is *exactly* as complicated as the same thing in the target of a symlink.
|
||||
It can be converted to a symlink with a small shell script.
|
||||
"""]]
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="CandyAngel"
|
||||
avatar="http://cdn.libravatar.org/avatar/15c0aade8bec5bf004f939dd73cf9ed8"
|
||||
subject="comment 2"
|
||||
date="2018-12-20T09:19:15Z"
|
||||
content="""
|
||||
> It is exactly as complicated as the same thing in the target of a symlink. It can be converted to a symlink with a small shell script.
|
||||
|
||||
Err.. by definition, requiring an additional step means it is more complicated. So not \"exactly as complicated\" :P
|
||||
"""]]
|
||||
|
|
@ -1,104 +0,0 @@
|
|||
I'm not sure if this is a bug, or if I'm just misunderstanding how to
|
||||
use `import`. Happy to move this to the forum, if that's more
|
||||
appropriate.
|
||||
|
||||
--
|
||||
|
||||
Importing from a directory special-remote into a nested directory
|
||||
which contains existing files that are more than 1 level deep tries to
|
||||
overwrite existing files?
|
||||
|
||||
```shell
|
||||
# Import to level 1, with existing files in level 1 - WORKS
|
||||
#export import_location=level1 && export existing_file_location=level1
|
||||
|
||||
# Import to level 2, with existing files in level 1 - WORKS
|
||||
#export import_location=level1/level2 && export existing_file_location=level1
|
||||
|
||||
# Import to level 2, with existing files in level 2 - DOES NOT WORK
|
||||
#export import_location=level1/level2 && export existing_file_location=level1/level2
|
||||
|
||||
# Import to level 3, with existing files in level 1 - WORKS
|
||||
#export import_location=level1/level2/level3 && export existing_file_location=level1
|
||||
|
||||
# Import to level 3, with existing files in level 2 - DOES NOT WORK
|
||||
#export import_location=level1/level2/level3 && export existing_file_location=level1/level2
|
||||
|
||||
# Import to level 3, with existing files in level 3 - DOES NOT WORK
|
||||
#export import_location=level1/level2/level3 && export existing_file_location=level1/level2/level3
|
||||
|
||||
# Import to level 4, with existing files in level 1 - WORKS
|
||||
#export import_location=level1/level2/level3/level4 && export existing_file_location=level1
|
||||
|
||||
# Import to level 4, with existing files in level 2 - DOES NOT WORK
|
||||
#export import_location=level1/level2/level3/level4 && export existing_file_location=level1/level2
|
||||
|
||||
# Import to level 4, with existing files in level 3 - DOES NOT WORK
|
||||
#export import_location=level1/level2/level3/level4 && export existing_file_location=level1/level2/level3
|
||||
|
||||
# Import to level 4, with existing files in level 4 - DOES NOT WORK
|
||||
export import_location=level1/level2/level3/level4 && export existing_file_location=level1/level2/level3/level4
|
||||
|
||||
|
||||
mkdir remote-directory
|
||||
echo "import content" > remote-directory/import-file.txt
|
||||
|
||||
mkdir repo
|
||||
cd repo
|
||||
git init
|
||||
git annex init
|
||||
git annex initremote myremote type=directory directory=../remote-directory encryption=none importtree=yes exporttree=yes
|
||||
|
||||
mkdir -p ${import_location}
|
||||
echo "content" > ${existing_file_location}/existing-file.txt
|
||||
git annex add --force-large
|
||||
git commit -m "add existing file to annex"
|
||||
|
||||
git annex import master:${import_location} --from myremote
|
||||
git annex merge myremote/master
|
||||
```
|
||||
|
||||
This is the output of the final scenario:
|
||||
|
||||
```shell
|
||||
$ git annex merge myremote/master
|
||||
merge myremote/master
|
||||
error: The following untracked working tree files would be overwritten by merge:
|
||||
level1/level2/level3/level4/existing-file.txt
|
||||
Please move or remove them before you merge.
|
||||
Aborting
|
||||
failed
|
||||
git-annex: merge: 1 failed
|
||||
```
|
||||
|
||||
--
|
||||
|
||||
Running this after encountering the situation above also causes
|
||||
strange things to happen on the `myremote/master` branch:
|
||||
|
||||
```shell
|
||||
git annex import master:${import_location} --from myremote
|
||||
git annex import master:${import_location} --from myremote
|
||||
git annex import master:${import_location} --from myremote
|
||||
git annex import master:${import_location} --from myremote
|
||||
```
|
||||
|
||||
The `myremote/master` branch appears to be stuck in some sort of loop,
|
||||
trying to remove and add the files over and over?
|
||||
|
||||
--
|
||||
|
||||
```shell
|
||||
$ git annex version
|
||||
git-annex version: 8.20210310
|
||||
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV
|
||||
dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.27 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.4 http-client-0.6.4.1 persistent-sqlite-2.11.1.0 torrent-10000.1.1 uuid-1.3.14 yesod-1.6.1.0
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
|
||||
operating system: linux x86_64
|
||||
supported repository versions: 8
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5 6 7
|
||||
local repository version: 8
|
||||
```
|
||||
|
||||
> [[fixed|done]], thanks for an excellent test case. --[[Joey]]
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="Lukey"
|
||||
avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
|
||||
subject="comment 1"
|
||||
date="2021-03-20T18:28:47Z"
|
||||
content="""
|
||||
Hi, <br>
|
||||
I don't think you should change the `import_location` for a special-remote after the first import.
|
||||
"""]]
|
||||
|
|
@ -1,53 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="gerta"
|
||||
avatar="http://cdn.libravatar.org/avatar/35a862512e6a10052014537b0131e482"
|
||||
subject="comment 2"
|
||||
date="2021-03-21T11:56:26Z"
|
||||
content="""
|
||||
Sorry, I think the way I formatted the question made it confusing.
|
||||
|
||||
--
|
||||
|
||||
Starting in an empty directory, and running this:
|
||||
|
||||
```shell
|
||||
export import_location=level1/level2/level3/level4 && export existing_file_location=level1/level2/level3/level4
|
||||
|
||||
|
||||
mkdir remote-directory
|
||||
echo \"import content\" > remote-directory/import-file.txt
|
||||
|
||||
mkdir repo
|
||||
cd repo
|
||||
git init
|
||||
git annex init
|
||||
git annex initremote myremote type=directory directory=../remote-directory encryption=none importtree=yes exporttree=yes
|
||||
|
||||
mkdir -p ${import_location}
|
||||
echo \"content\" > ${existing_file_location}/existing-file.txt
|
||||
git annex add --force-large
|
||||
git commit -m \"add existing file to annex\"
|
||||
|
||||
git annex import master:${import_location} --from myremote
|
||||
git annex merge myremote/master
|
||||
```
|
||||
|
||||
results in the merge failing with this output:
|
||||
|
||||
```
|
||||
$ git annex merge myremote/master
|
||||
merge myremote/master
|
||||
error: The following untracked working tree files would be overwritten by merge:
|
||||
level1/level2/level3/level4/existing-file.txt
|
||||
Please move or remove them before you merge.
|
||||
Aborting
|
||||
failed
|
||||
git-annex: merge: 1 failed
|
||||
```
|
||||
|
||||
Should this work, or am I doing something wrong?
|
||||
|
||||
--
|
||||
|
||||
All the other combinations of `import_location` and `existing_file_location` in the original post were other things that I tried while trying to work out if I was using `import` correctly, but each scenario was run in a clean starting directory.
|
||||
"""]]
|
||||
|
|
@ -1,27 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 3"""
|
||||
date="2021-03-26T17:38:23Z"
|
||||
content="""
|
||||
It seems that git-annex import is generating a tree containing 2 subtrees
|
||||
with the same name. While ls-tree -r shows this as a tree containing
|
||||
the 2 files like we'd want to have after the import, looking at it more
|
||||
closely shows the problem:
|
||||
|
||||
# git ls-tree -r remotes/myremote/master
|
||||
120000 blob a180a12b4c192cb67498f67e476196050cbac770 level1/level2/level3/level4/import-file.txt
|
||||
120000 blob 55a266f9e9629be3d8eba3ad4a65b15d8614e1f8 level1/level2/level3/level4/existing-file.txt
|
||||
# git ls-tree remotes/myremote/master
|
||||
040000 tree d77ae5e1f666f8f13795c27ee85a55ce0d1c5136 level1
|
||||
# git ls-tree d77ae5e1f666f8f13795c27ee85a55ce0d1c5136
|
||||
040000 tree 79300b8717445f07eee006a0ba8992fb8adee3cd level2
|
||||
040000 tree 28092869684c815a23fb4d0d0753d598f352e9cc level2
|
||||
# git ls-tree -r 79300b8717445f07eee006a0ba8992fb8adee3cd
|
||||
120000 blob a180a12b4c192cb67498f67e476196050cbac770 level3/level4/import-file.txt
|
||||
# git ls-tree -r 28092869684c815a23fb4d0d0753d598f352e9cc
|
||||
120000 blob 55a266f9e9629be3d8eba3ad4a65b15d8614e1f8 level3/level4/existing-file.txt
|
||||
|
||||
I assume this is giving git merge its trouble, although it seems to be
|
||||
failing in a highly surprising way since it's getting confused about
|
||||
what files are tracked by git in the tree this is being merged into.
|
||||
"""]]
|
||||
|
|
@ -1,11 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 4"""
|
||||
date="2021-03-26T18:46:19Z"
|
||||
content="""
|
||||
Note that level1/level2 is sufficient for git-annex to generate the bad
|
||||
tree. I also saw git merge fail at that depth.
|
||||
|
||||
I've fixed this, at least the test case given and some other similar ones
|
||||
I thought of to try.
|
||||
"""]]
|
||||
|
|
@ -1,119 +0,0 @@
|
|||
### Please describe the problem.
|
||||
|
||||
I have a git annex remote on s3 configured to push things to glacier rather than normal storage. Compared to regular s3 things in glacier are not immediately available and must be "restored" before they can be downloaded (the trade off is that data which is untouched long term is quite a lot cheaper per GB). I'm using the DEEP_ARCHIVE storage class (configured using the `storageclass` key in the remote's config, I didn't fiddle with the s3 bucket lifecycle at all). I think the following applies to any Glacier stored objects, the class just changes how long a restore will take.
|
||||
|
||||
My annexed objects are > 1GB and the s3 remote is chunked at 1GB granularity.
|
||||
|
||||
When I attempt to `git annex get` such an object the error message misleading refers to the unchunked path e.g. `/SHA256E-s6749536256--f76639fa11276b4045844e6110035c15e6803acc38d77847c2e4a2be1b1850ca.iso` rather than `/SHA256E-s6749536256-S1000000000-C1--f76639fa11276b4045844e6110035c15e6803acc38d77847c2e4a2be1b1850ca.iso` etc, which sent me down a blind alleyway for a bit. If I `git annex get -d` then I can see in the log that it tries the chunked path first and the falls back to the unchunked. It would be useful if the non-verbose error message listed the first attempt and the fallback. It would be even better if it could be aware enough of Glacier to point out that some list objects need to be manually restored in order to be retrieved.
|
||||
|
||||
In my quest to manually restore I could not for the life of me figure out (even going into the plumbing layers of git etc) how to retrieve a list of the chunks needed. I can get the key from `git annex info` easily enough and then `aws s3api list-objects --bucket <...> --prefix` to look for chunks of objects with the `SHA256E-s6749536256` prefix which works ok so long as all objects in the annex are different sizes -- AWS CLI seems to only lets you filter by prefix not a glob. I could probably list everything and extract what I wanted with `jq` but I _think_ there are cost implications to listing everything (although I might be wrong about that, and it wouldn't be a lot of money for my use case in any event).
|
||||
|
||||
Fixing those two minor issues (the error reporting and the ability to get the list of chunks) would be a massive improvement to the usability of S3/glacier remotes IMHO, especially if the output of the latter were consumable by scripts.
|
||||
|
||||
I will also include my manual steps to restore in the final section, it would be amazing of git annex could learn to do all this itself though...
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
Given an S3/glacier remote with chunked objects in it just a `git annex get` for an object in it will do.
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
8.20210223-1 on Debian, from the Debian archive.
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
Issue with `git annex get` error logging:
|
||||
|
||||
[[!format sh """
|
||||
|
||||
$ git annex get OBJECT.iso
|
||||
get OBJECT.iso (from s3...)
|
||||
|
||||
HttpExceptionRequest Request {
|
||||
host = "<REDACTED>"
|
||||
port = 80
|
||||
secure = False
|
||||
requestHeaders = [("Date","Sun, 25 Apr 2021 09:42:56 GMT"),("Authorization","<REDACTED>")]
|
||||
path = "/SHA256E-s6749536256--f76639fa11276b4045844e6110035c15e6803acc38d77847c2e4a2be1b1850ca.iso"
|
||||
queryString = ""
|
||||
method = "GET"
|
||||
proxy = Nothing
|
||||
rawBody = False
|
||||
redirectCount = 10
|
||||
responseTimeout = ResponseTimeoutDefault
|
||||
requestVersion = HTTP/1.1
|
||||
}
|
||||
(StatusCodeException (Response {responseStatus = Status {statusCode = 404, statusMessage = "Not Found"}, responseVersion = HTTP/1.1, responseHeaders = [("x-amz-request-id","YVCPYD2RW4QEWMWN"),("x-amz-id-2","6dJY8ceWlLOSNIyTTchniLm5+cvJLovbMZL44YjNmViGwfChQSmWLl6VI6E5sFNDbMpwUeBhpbA="),("Content-Type","application/xml"),("Transfer-Encoding","chunked"),("Date","Sun, 25 Apr 2021 09:42:55 GMT"),("Server","AmazonS3")], responseBody = (), responseCookieJar = CJ {expose = []}, responseClose' = ResponseClose}) "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>SHA256E-s6749536256--f76639fa11276b4045844e6110035c15e6803acc38d77847c2e4a2be1b1850ca.iso</Key><RequestId>YVCPYD2RW4QEWMWN</RequestId><HostId>6dJY8ceWlLOSNIyTTchniLm5+cvJLovbMZL44YjNmViGwfChQSmWLl6VI6E5sFNDbMpwUeBhpbA=</HostId></Error>")
|
||||
|
||||
Unable to access these remotes: s3
|
||||
|
||||
No other repository is known to contain the file.
|
||||
|
||||
(Note that these git remotes have annex-ignore set: origin)
|
||||
failed
|
||||
git-annex: get: 1 failed
|
||||
|
||||
$ git annex get -d OBJECT.iso
|
||||
[...]
|
||||
(from s3...)
|
||||
[2021-04-25 10:43:44.098104491] Path: "/SHA256E-s6749536256-S1000000000-C1--f76639fa11276b4045844e6110035c15e6803acc38d77847c2e4a2be1b1850ca.iso"
|
||||
[...]
|
||||
[2021-04-25 10:43:44.200461839] Response status: Status {statusCode = 403, statusMessage = "Forbidden"}
|
||||
[...]
|
||||
[2021-04-25 10:43:44.238156623] Response status: Status {statusCode = 404, statusMessage = "Not Found"}
|
||||
[...]
|
||||
*** error message as above ***
|
||||
|
||||
"""]]
|
||||
|
||||
It's notable that the status codes differ, since the chunk is present but not currently accessible while the unchunked just isn't there.
|
||||
|
||||
Manually fetching things:
|
||||
|
||||
[[!format sh """
|
||||
|
||||
: 1. Figure out the key and the prefix on it:
|
||||
|
||||
$ git annex info OBJECT.iso
|
||||
file: OBJECT.iso
|
||||
size: 6.75 gigabytes
|
||||
key: SHA256E-s6749536256--f76639fa11276b4045844e6110035c15e6803acc38d77847c2e4a2be1b1850ca.iso
|
||||
present: false
|
||||
|
||||
: 2. Find the number of chunks using the prefix:
|
||||
|
||||
$ aws s3api list-objects --bucket <BUCKET> --prefix SHA256E-s6749536256 | jq '.Contents[].Key'
|
||||
"SHA256E-s6749536256-S1000000000-C1--f76639fa11276b4045844e6110035c15e6803acc38d77847c2e4a2be1b1850ca.iso"
|
||||
"SHA256E-s6749536256-S1000000000-C2--f76639fa11276b4045844e6110035c15e6803acc38d77847c2e4a2be1b1850ca.iso"
|
||||
"SHA256E-s6749536256-S1000000000-C3--f76639fa11276b4045844e6110035c15e6803acc38d77847c2e4a2be1b1850ca.iso"
|
||||
"SHA256E-s6749536256-S1000000000-C4--f76639fa11276b4045844e6110035c15e6803acc38d77847c2e4a2be1b1850ca.iso"
|
||||
"SHA256E-s6749536256-S1000000000-C5--f76639fa11276b4045844e6110035c15e6803acc38d77847c2e4a2be1b1850ca.iso"
|
||||
"SHA256E-s6749536256-S1000000000-C6--f76639fa11276b4045844e6110035c15e6803acc38d77847c2e4a2be1b1850ca.iso"
|
||||
"SHA256E-s6749536256-S1000000000-C7--f76639fa11276b4045844e6110035c15e6803acc38d77847c2e4a2be1b1850ca.iso"
|
||||
|
||||
: 3. Request a restore of those chunks:
|
||||
|
||||
$ for i in $(seq 1 7) ; do aws s3api restore-object --bucket <BUCKET> --key SHA256E-s6749536256-S1000000000-C${i}--f76639fa11276b4045844e6110035c15e6803acc38d77847c2e4a2be1b1850ca.iso --restore-request Days=1 ; done
|
||||
|
||||
: 4. Poll for completion, for DEEP_ARCHIVE restores happen on the order of hours.
|
||||
|
||||
$ until
|
||||
for i in $(seq 1 7) ; do aws s3api head-object --bucket <BUCKET> --key SHA256E-s6749536256-S1000000000-C${i}--f76639fa11276b4045844e6110035c15e6803acc38d77847c2e4a2be1b1850ca.iso ; done | jq -r .Restore
|
||||
git annex get OBJECT.iso
|
||||
do echo "$(date): sleeping..." ; sleep 1h; done
|
||||
|
||||
ongoing-request="true"
|
||||
ongoing-request="true"
|
||||
...eventually becoming
|
||||
ongoing-request="false", expiry-date="Mon, 26 Apr 2021 00:00:00 GMT"
|
||||
... for all objects then the "git annex get" succeeds and the loop exits
|
||||
|
||||
|
||||
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
Yes, for loads of stuff. It's awesome, thanks!
|
||||
|
||||
> [[closed|done]], see my comment --[[Joey]]
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="Lukey"
|
||||
avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
|
||||
subject="comment 1"
|
||||
date="2021-04-25T11:53:29Z"
|
||||
content="""
|
||||
I think that is expected, from [[special_remotes/glacier]]: \"To deal with this, commands like \"git-annex get\" request Glacier start the retrieval process, and will fail due to the data not yet being available. \"
|
||||
"""]]
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="ijc@c69abafeb65fa2e784811fc549e9976a5cf4b903"
|
||||
nickname="ijc"
|
||||
avatar="http://cdn.libravatar.org/avatar/83432d9f01a0bedc575703583f7aa7c6"
|
||||
subject="comment 2"
|
||||
date="2021-05-02T17:54:56Z"
|
||||
content="""
|
||||
I don't think git-annex is starting any retrieval process, I have to do it manually. I wasn't sure if it was supposed to (it doesn't seem to try) but maybe that's an aspect of this bug too.
|
||||
"""]]
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="Lukey"
|
||||
avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
|
||||
subject="comment 3"
|
||||
date="2021-05-02T18:46:34Z"
|
||||
content="""
|
||||
Sorry, I just noticed that you are using s3 and not glacier-cli (which the [[special_remotes/glacier/]] special-remote uses). So it indeed is a bug, since git-annex doesn't handle glacier with s3 as yet far as I know. Maybe you have more luck with the glacier special-remote?
|
||||
"""]]
|
||||
|
|
@ -1,13 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="ijc@c69abafeb65fa2e784811fc549e9976a5cf4b903"
|
||||
nickname="ijc"
|
||||
avatar="http://cdn.libravatar.org/avatar/83432d9f01a0bedc575703583f7aa7c6"
|
||||
subject="comment 4"
|
||||
date="2021-05-03T15:45:14Z"
|
||||
content="""
|
||||
Thanks Lukey. I'm not really sure how I missed (or how I found and then for some reason discounted) the glacier-cli backend but I think the fact I've started with s3 means I'm stuck with it unless I want to reupload a dozen TB of data and/or pay enormous retrieval/move fees (AIUI this is a property of the AWS end of things, not a git-annex issue, although s3 DEEP_ARCHIVE uses Glacier it's not the same objects/apis as going at glacier direct).
|
||||
|
||||
FWIW the docs for storageclass at https://git-annex.branchable.com/special_remotes/S3/ refer one to s3cmd(1) for the list of valid values which includes DEEP_ARCHIVE, so that might be how I came to do things this way. Perhaps a pointer to glacier-cli at that point would be appropriate?
|
||||
|
||||
I also just found https://git-annex.branchable.com/todo/wishlist__58___Restore_s3_files_moved_to_Glacier/ which is related to this but involves s3 lifecycles moving things between s3 and glacier on schedules etc which I think precludes glacier-cli. I've used that in other contexts (perhaps another reason I ended up following this path here too) but never with git-annex.
|
||||
"""]]
|
||||
|
|
@ -1,19 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 5"""
|
||||
date="2021-10-14T16:17:14Z"
|
||||
content="""
|
||||
I have added a note to the S3 documentation about `DEEP_ARCHIVE` and the
|
||||
glacier special remote.
|
||||
|
||||
I have made git-annex display the exception for the more likely chunked
|
||||
location, rather than the less likely unchunked location, when retrieving
|
||||
from both locations fails. Although it's still possible for there to be
|
||||
situations where the exception if displays is not for the location where
|
||||
the content actually is. Eg, if the chunk size of the remote has
|
||||
changed over time.
|
||||
|
||||
I think that todo is basically talking about the same desire to make the S3
|
||||
remote support these glacier-style storage classes, in one way or another,
|
||||
and so I think this bug report can be closed as otherwise a duplicate of it.
|
||||
"""]]
|
||||
|
|
@ -1,55 +0,0 @@
|
|||
### Please describe the problem.
|
||||
|
||||
I'm seeing some inconsistent results between runs of `git annex fsck` and `git annex whereis` that I'm not able to explain. When I run `git annex fsck`, it reports a few keys that only have 1 copy, and advises me to make more copies. If I run `git annex whereis --key <key>`, git annex confirms that it only knows about 1 copy of this key. If I then use `git log --stat -S'<key>'` to find the actual file that it refers to, and run `git annex whereis <file>`, git annex report 9 copies of this file. Checking on remotes shows that these files do exist on the remote, so why does `git annex fsck` and `git annex whereis` mis-report the number of copies when querying for the key - but not for the actual filename? Additionally, `git annex find --lackingcopies 1` doesn't return any results, but should if there are actually files with not enough copies?
|
||||
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
5.20151208-1build1 on Ubuntu Xenial, one remote running 5.20141024~bpo70+1 on Debian Wheezy
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
|
||||
[william@hactar ~/Pictures/Photo Library]$ git annex whereis SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9
|
||||
git-annex: SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9 not found
|
||||
git-annex: whereis: 1 failed
|
||||
[william@hactar ~/Pictures/Photo Library]$ git annex whereis --key SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9
|
||||
whereis SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9 (1 copy)
|
||||
7691934f-2542-4103-9122-2db4e6cfc887 -- hactar [here]
|
||||
ok
|
||||
[william@hactar ~/Pictures/Photo Library]$ git annex fsck --key SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9
|
||||
fsck SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9
|
||||
Only 1 of 3 trustworthy copies exist of SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9
|
||||
Back it up with git-annex copy.
|
||||
failed
|
||||
(recording state in git...)
|
||||
git-annex: fsck: 1 failed
|
||||
[william@hactar ~/Pictures/Photo Library]$ git log --stat -S'SHA256E-s1071765--dbaa7f32ee44c28d6a1f0c8095e8dfd8b4ec433b144085d5097425303a510ea9'
|
||||
[william@hactar ~/Pictures/Photo Library]$ git annex whereis 2009/05/05/P1040890.JPG
|
||||
whereis 2009/05/05/P1040890.JPG (9 copies)
|
||||
0e825a69-1927-4f62-b731-6f3e98bba998 -- william@marvin:/media/backup/annex/photos [marvin]
|
||||
1b728ab5-1e32-45a6-bc11-2a4bfdc9d6ab -- backup1
|
||||
5c0caa42-b489-467b-a612-9590fa9d5a94 -- backup2
|
||||
7691934f-2542-4103-9122-2db4e6cfc887 -- hactar [here]
|
||||
894b2216-72e0-40e1-8765-1386e1e9e4b4 -- backup3
|
||||
96f19fa8-d385-4e8b-b000-61ee15993a70 -- backup3
|
||||
a862b121-d794-4af4-bb56-21adfe8962f2 -- S3
|
||||
b083f8ae-42fb-41f0-a2a3-4e7c9f93aadb -- [guide]
|
||||
bf021ce9-465b-4419-86e7-bddfd208fca4 -- git@newzaphod:~/repositories/annex/photos.git [zaphod]
|
||||
ok
|
||||
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
I trust Git Annex to keep hundreds of GB of data safe, and it has never failed me - despite my best efforts
|
||||
|
||||
> [[closing|done]] per my comment --[[Joey]]
|
||||
|
|
@ -1,12 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 1"""
|
||||
date="2016-12-13T16:42:08Z"
|
||||
content="""
|
||||
The obvious reason for this would be if the file no longer points to that
|
||||
same key. Perhaps the file got modified and the key is the old version of
|
||||
the file.
|
||||
|
||||
That would explain everything you showed, so currently I don't see any
|
||||
bug..
|
||||
"""]]
|
||||
|
|
@ -1,68 +0,0 @@
|
|||
### Please describe the problem.
|
||||
|
||||
Metadata are not stored in a consistent format. It seems more like git-annex chooses the "smallest" charset able to hold the data, i.e. US-ASCII, unless there are latin1 characters, and only UTF-8 if there are UTF-8 characters that are not in latin1
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
% git init
|
||||
Initialized empty Git repository in /home/madduck/.tmp/cdt.GlIevu/.git/
|
||||
|
||||
% git annex init
|
||||
init ok
|
||||
(recording state in git...)
|
||||
|
||||
% date > a
|
||||
|
||||
% git annex add a
|
||||
add a ok
|
||||
(recording state in git...)
|
||||
|
||||
% git annex metadata -s one=$(echo US-ASCII | iconv -tus-ascii) a
|
||||
metadata a
|
||||
lastchanged=2016-09-25@13-18-57
|
||||
one=US-ASCII
|
||||
one-lastchanged=2016-09-25@13-18-57
|
||||
ok
|
||||
(recording state in git...)
|
||||
|
||||
% git annex metadata -s two=$(echo lätin1 | iconv -tlatin1) a
|
||||
metadata a
|
||||
lastchanged=2016-09-25@13-19-37
|
||||
one=US-ASCII
|
||||
one-lastchanged=2016-09-25@13-18-57
|
||||
two=lätin1
|
||||
two-lastchanged=2016-09-25@13-19-37
|
||||
ok
|
||||
(recording state in git...)
|
||||
|
||||
% git annex metadata -s three=$(echo unicode… | iconv -tutf8) a
|
||||
metadata a
|
||||
lastchanged=2016-09-25@13-19-41
|
||||
one=US-ASCII
|
||||
one-lastchanged=2016-09-25@13-18-57
|
||||
three=unicode…
|
||||
three-lastchanged=2016-09-25@13-19-41
|
||||
two=lätin1
|
||||
two-lastchanged=2016-09-25@13-19-37
|
||||
ok
|
||||
(recording state in git...)
|
||||
|
||||
% git annex metadata -g three a | iconv -tutf8
|
||||
unicode…
|
||||
|
||||
% git annex metadata -g two a | iconv -tutf8
|
||||
liconv: illegal input sequence at position 1
|
||||
|
||||
% git annex metadata -g one a | iconv -tutf8
|
||||
US-ASCII
|
||||
|
||||
% git annex metadata -g two a | iconv -flatin1 -tutf8
|
||||
lätin1
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
6.20160808-1
|
||||
|
||||
[[!tag moreinfo]]
|
||||
|
||||
> [[done]] --[[Joey]]
|
||||
|
|
@ -1,12 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 1"""
|
||||
date="2016-09-26T20:50:26Z"
|
||||
content="""
|
||||
The metadata you get out should always be encoded the same as the metadata
|
||||
you put in. The encoding, or encodings used are up to you.
|
||||
|
||||
Are you seeing metadata queries returning a different sequence of bytes
|
||||
than the sequence of bytes that were originally stored? If not, I don't
|
||||
think this is a bug.
|
||||
"""]]
|
||||
|
|
@ -1,13 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 2"""
|
||||
date="2021-01-29T19:32:25Z"
|
||||
content="""
|
||||
joey@darkstar:~>echo lätin1 | iconv -tlatin1 | iconv -tutf8
|
||||
liconv: illegal input sequence at position 1
|
||||
|
||||
This is identical to what you did, except I have replaced git-annex
|
||||
with a pipe.
|
||||
|
||||
So, not a bug.
|
||||
"""]]
|
||||
|
|
@ -1,53 +0,0 @@
|
|||
### Please describe the problem.
|
||||
Build error 1 during build process of git-annex on alpine 5.15 using GHC 9.0.1.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
Build git-annex on alpine 5.15 using following APKBUILD:
|
||||
https://github.com/ayakael/aports/blob/testing/git-annex/testing/git-annex/APKBUILD
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
git-annex: 20211123
|
||||
os: alpine linux 5.15
|
||||
ghc: 9.0.1
|
||||
cabal: 3.6.2.0
|
||||
|
||||
### Please provide any additional information below.
|
||||
I made an attempt of fixing by reverting the first fix attempt done on line 181 by commit 2739adc. It gave the same error, but it is most likely related. I know very little of haskell, thus could not push the debugging further.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
[411 of 670] Compiling Remote.Glacier ( Remote/Glacier.hs, /var/build/aports/testing/git-annex/src/git-annex-8.20211123/dist-newstyle/build/x86_64-linux/ghc-9.0.1/git-annex-8.20211123/build/git-annex/git-annex-tmp/Remote/Glacier.o, /var/build/aports/testing/git-annex/src/git-annex-8.20211123/dist-newstyle/build/x86_64-linux/ghc-9.0.1/git-annex-8.20211123/build/git-annex/git-annex-tmp/Remote/Glacier.dyn_o )
|
||||
|
||||
Remote/Glacier.hs:179:12: error:
|
||||
• Couldn't match type: Key
|
||||
-> MeterUpdate
|
||||
-> Maybe Utility.Hash.IncrementalVerifier
|
||||
-> (ContentSource -> Annex a0)
|
||||
-> Annex a0
|
||||
with: forall a.
|
||||
Key
|
||||
-> MeterUpdate
|
||||
-> Maybe Utility.Hash.IncrementalVerifier
|
||||
-> (ContentSource -> Annex a)
|
||||
-> Annex a
|
||||
Expected: Remote -> Retriever
|
||||
Actual: Remote
|
||||
-> Key
|
||||
-> MeterUpdate
|
||||
-> Maybe Utility.Hash.IncrementalVerifier
|
||||
-> (ContentSource -> Annex a0)
|
||||
-> Annex a0
|
||||
• In the expression: byteRetriever . retrieve'
|
||||
In an equation for ‘retrieve’: retrieve = byteRetriever . retrieve'
|
||||
|
|
||||
179 | retrieve = byteRetriever . retrieve'
|
||||
| ^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
make: *** [Makefile:58: git-annex] Error 1
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
Oh absolutely. This version compiles fine under alpine 5.14, which uses GHC 8.8.4. I've been using this software for many years without major issues, and it's a linchpin of my backup infrastructure. Thanks for the all the good work!
|
||||
|
||||
> [[fixed|done]] --[[Joey]]
|
||||
|
|
@ -1,48 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="account@dc612ad075297e574ebc3eb9a5b8ab6e753510dc"
|
||||
nickname="account"
|
||||
avatar="http://cdn.libravatar.org/avatar/6a2e457685ce943a3d85b0a6d1ffb11a"
|
||||
subject="Further fix attempts"
|
||||
date="2021-12-01T03:25:35Z"
|
||||
content="""
|
||||
I made another attempt at a fix by porting whatever changes you made to stack.yaml on 2739adc to git-annex.cabal using the following patch.
|
||||
[[!format sh \"\"\"
|
||||
From 05030463daca2582f2512cf14515393971263a40 Mon Sep 17 00:00:00 2001
|
||||
From: \"build@apk-groulx\" <build@apk-groulx.praxis>
|
||||
Date: Tue, 30 Nov 2021 06:48:09 +0000
|
||||
Subject: [PATCH 1/1] Added dependency for ghc 9.0.1 for cabal
|
||||
|
||||
---
|
||||
git-annex.cabal | 6 ++++--
|
||||
1 file changed, 4 insertions(+), 2 deletions(-)
|
||||
|
||||
diff --git a/git-annex.cabal b/git-annex.cabal
|
||||
index 9285858cd..1a3ee52ff 100644
|
||||
--- a/git-annex.cabal
|
||||
+++ b/git-annex.cabal
|
||||
@@ -330,7 +330,9 @@ Executable git-annex
|
||||
directory (>= 1.2),
|
||||
disk-free-space,
|
||||
filepath,
|
||||
- filepath-bytestring (>= 1.4.2.1.1),
|
||||
+ filepath-bytestring (>= 1.4.2.1.8),
|
||||
+ base16-bytestring (>=0.1.1.7),
|
||||
+ base64-bytestring (>=0.0.3),
|
||||
IfElse,
|
||||
monad-logger (>= 0.3.10),
|
||||
free,
|
||||
@@ -425,7 +427,7 @@ Executable git-annex
|
||||
Other-Modules: Utility.GitLFS
|
||||
|
||||
if flag(HttpClientRestricted)
|
||||
- Build-Depends: http-client-restricted (>= 0.0.2)
|
||||
+ Build-Depends: http-client-restricted (>= 0.0.4)
|
||||
CPP-Options: -DWITH_HTTP_CLIENT_RESTRICTED
|
||||
else
|
||||
Other-Modules: Utility.HttpManagerRestricted
|
||||
--
|
||||
2.34.0
|
||||
\"\"\"]]
|
||||
|
||||
The same error occurs, unfortunately.
|
||||
"""]]
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 2"""
|
||||
date="2021-12-08T19:19:38Z"
|
||||
content="""
|
||||
Seems this breakage is due to https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0287-simplify-subsumption.rst
|
||||
|
||||
Fixed it.
|
||||
"""]]
|
||||
|
|
@ -1,11 +0,0 @@
|
|||
### Please describe the problem.
|
||||
There appears to be no way of getting the current configuration of special remotes (e.g. url, keyid). Maybe there should be git-annex-showremote (or similar).
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
Configure the special remote (e.g. webdav), and then forget what settings you specified ;)
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
7.20190129-3 (Debian)
|
||||
|
||||
> [[done]] seems the reporter found a way to get what they wanted.
|
||||
> --[[Joey]]
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="derobert"
|
||||
avatar="http://cdn.libravatar.org/avatar/05b48b72766177b3b0a6ff4afdb70790"
|
||||
subject="comment 1"
|
||||
date="2019-05-16T08:12:45Z"
|
||||
content="""
|
||||
Did you try `git-annex info «remote»` (where «remote» could be the name or the UUID).
|
||||
"""]]
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="aragilar"
|
||||
avatar="http://cdn.libravatar.org/avatar/7390f7c0dd07d43ca827a5446b65c102"
|
||||
subject="comment 2"
|
||||
date="2019-05-16T08:26:28Z"
|
||||
content="""
|
||||
Weird, I tried that on the same remote on a different system, and it gave what I wanted... I'll have to work out why that didn't work the first time. Thanks @derobert!
|
||||
"""]]
|
||||
|
|
@ -1,28 +0,0 @@
|
|||
### Please describe the problem.
|
||||
When stall detection is enabled, debug output for transfers with external special remotes appears to be hidden.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
git config annex.stalldetection true
|
||||
git annex copy file --to=externalspecialremote --debug 2>&1 | grep PROGRESS
|
||||
# no lines output
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
I'm on RHEL7 and this problem is present in git master for me ever since the introduction of stall detection: 135757d64 through 4d6f74477
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
I've written ESRPs and recovered many lost files =)
|
||||
|
||||
> [[fixed|done]] --[[Joey]]
|
||||
File diff suppressed because it is too large
Load diff
|
|
@ -1,79 +0,0 @@
|
|||
Hello,
|
||||
|
||||
### Please describe the problem.
|
||||
I am unable to create a gcrypt remote with encryption=shared set. It allways complains with: "git-annex: cannot use gcrypt remote without encryption enabled", however the same setup works when using encryption=hybrid instead.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
$ git annex initremote test type=gcrypt encryption=shared gitrepo=../annex-target
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
I tried it with:
|
||||
The Version in Debian Buster (6.20180913), latest stable and daily build.
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
Transcript:
|
||||
[[!format sh """
|
||||
|
||||
xxx@xxx:~/Documents$ mkdir annex-test
|
||||
xxx@xxx:~/Documents$ cd annex-test/
|
||||
xxx@xxx:~/Documents/annex-test$ git init
|
||||
Initialized empty Git repository in /home/xxx/Documents/annex-test/.git/
|
||||
xxx@xxx:~/Documents/annex-test$ git annex init test
|
||||
init test ok
|
||||
(recording state in git...)
|
||||
xxx@xxx:~/Documents/annex-test$ cd ..
|
||||
xxx@xxx:~/Documents$ git init --bare annex-target
|
||||
Initialized empty Git repository in /home/xxx/Documents/annex-target/
|
||||
xxx@xxx:~/Documents$ cd annex-test/
|
||||
xxx@xxx:~/Documents/annex-test$ touch test
|
||||
xxx@xxx:~/Documents/annex-test$ git annex add test
|
||||
add test ok
|
||||
(recording state in git...)
|
||||
xxx@xxx:~/Documents/annex-test$ git annex sync
|
||||
commit
|
||||
[master (root-commit) 4f63e05] git-annex in test
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 120000 test
|
||||
ok
|
||||
|
||||
# This fails
|
||||
xxx@xxx:~/Documents/annex-test$ git annex initremote test type=gcrypt encryption=shared gitrepo=../annex-target
|
||||
initremote test (encryption setup) (encryption key stored in git repository)
|
||||
git-annex: cannot use gcrypt remote without encryption enabled
|
||||
failed
|
||||
git-annex: initremote: 1 failed
|
||||
xxx@xxx:~/Documents/annex-test$ git annex initremote test type=gcrypt encryption=hybrid gitrepo=../annex-target keyid=30C027C6B72964DD
|
||||
|
||||
git-annex: There is already a remote named "test"
|
||||
failed
|
||||
git-annex: initremote: 1 failed
|
||||
xxx@xxx:~/Documents/annex-test$ git remote remove test
|
||||
|
||||
# But this works
|
||||
xxx@xxx:~/Documents/annex-test$ git annex initremote test type=gcrypt encryption=hybrid gitrepo=../annex-target keyid=30C027C6B72964DD
|
||||
initremote test (encryption setup) (to gpg keys: 30C027C6B72964DD) gcrypt: Repository not found: ../annex-target
|
||||
gcrypt: Repository not found: ../annex-target
|
||||
gcrypt: Setting up new repository
|
||||
gcrypt: Remote ID is :id:aHvfIDaNhjyLzBgYnmII
|
||||
Enumerating objects: 10, done.
|
||||
Counting objects: 100% (10/10), done.
|
||||
Compressing objects: 100% (7/7), done.
|
||||
Total 10 (delta 1), reused 0 (delta 0)
|
||||
gcrypt: Encrypting to: -r 30C027C6B72964DD
|
||||
gcrypt: Requesting manifest signature
|
||||
To gcrypt::../annex-target
|
||||
* [new branch] git-annex -> git-annex
|
||||
ok
|
||||
(recording state in git...)
|
||||
xxx@xxx:~/Documents/annex-test$
|
||||
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
I think git-annex is awesome. It already helps me managing the limited diskspace on my laptop and its possibilities seem endless.
|
||||
|
||||
> This was fixed over a year ago in
|
||||
> [[!commit 8401b09e32b9378fa4c02515255a743d8f0ad4b1]] [[done]] --[[Joey]]
|
||||
|
|
@ -1,15 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="lukasstraub2@bbbb2ef261a0994edda5f5f55999dfac5998d4e5"
|
||||
nickname="lukasstraub2"
|
||||
avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
|
||||
subject="Workaround"
|
||||
date="2018-12-04T19:37:37Z"
|
||||
content="""
|
||||
As a workaround, I can include a \"gnupg\" Directory in the Repo and point the GNUPGHOME environment Variable at it. Then I simply create a single Key there and add the gcrypt Repo with encryption=hybrid. Altough I have to add the following to the gpg.conf (in the gnupg Directory inside the Repo) to prevent gpg from writing to the directory afterwards:
|
||||
pinentry-mode loopback
|
||||
no-auto-check-trustdb
|
||||
no-random-seed-file
|
||||
no-permission-warning
|
||||
quiet
|
||||
|
||||
"""]]
|
||||
|
|
@ -1,57 +0,0 @@
|
|||
### Please describe the problem.
|
||||
|
||||
Utility/libkqueue.h is missing from git-annex-8.20210127.tar.gz as hosted on hackage.
|
||||
|
||||
```
|
||||
SHA256 (/usr/ports/distfiles/hackage/git-annex-8.20210127.tar.gz) = 636f439420f28a92a8250b9f8b422567e439315f3f2d23cb0bea3b0a0e5755c3
|
||||
% tar ztf /usr/ports/distfiles/hackage/git-annex-8.20210127.tar.gz | grep libkqu
|
||||
git-annex-8.20210127/Utility/libkqueue.c
|
||||
```
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
Building on `(! os(solaris) && ! os(gnu) && ! os(linux))` causes a complaint about this file missing.
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
git-annex-8.20210127 on OpenBSD 6.8-current amd64
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
$ doas pkg_add libmagic
|
||||
$ cabal get git-annex-8.20210127
|
||||
$ cabal v2-build --extra-include-dirs=/usr/local/include --extra-lib-dirs=/usr/local/lib
|
||||
...
|
||||
checking UPGRADE_LOCATION... not available
|
||||
checking git... yes
|
||||
checking git version... 2.30.0
|
||||
checking cp -a... yes
|
||||
checking cp -p... yes
|
||||
checking cp --preserve=timestamps... no
|
||||
checking cp --reflink=auto... no
|
||||
checking xargs -0... yes
|
||||
checking rsync... yes
|
||||
checking curl... yes
|
||||
checking bup... no
|
||||
checking nice... yes
|
||||
checking ionice... no
|
||||
checking nocache... no
|
||||
checking gpg... gpg
|
||||
checking lsof... not available
|
||||
checking git-remote-gcrypt... not available
|
||||
checking ssh connection caching... yes
|
||||
fatal: not a git repository (or any parent up to mount point /)
|
||||
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
|
||||
Configuring git-annex-8.20210127...
|
||||
setup: Missing dependency on a foreign library:
|
||||
* Missing (or bad) header file: Utility/libkqueue.h
|
||||
...
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before?
|
||||
|
||||
Nope, but hopefully packaging up the file will make it a little easier for me to try the next time :)
|
||||
|
||||
|
||||
> [[fixed|done]] in git for next release --[[Joey]]
|
||||
|
|
@ -1,43 +0,0 @@
|
|||
### Please describe the problem.
|
||||
|
||||
```
|
||||
> git annex get Narnia/
|
||||
get Narnia/Course of a Generation/01 Sail Around the World.mp3 (from Seagate...)
|
||||
SHA256E-s8395599--2fea961006a279f0765c45755b35a06f0a4fc6bfbab6118182ebc693d7b47a91.mp3
|
||||
8,395,599 100% 29.65MB/s 0:00:00 (xfr#1, to-chk=0/1)
|
||||
(checksum...) ^C⏎
|
||||
```
|
||||
|
||||
```
|
||||
> mpv ~/Music/sorted/Narnia/Course\ of\ a\ Generation/
|
||||
Playing: /home/philip/Music/sorted/Narnia/Course of a Generation/
|
||||
[file] This is a directory - adding to playlist.
|
||||
|
||||
Playing: /home/philip/Music/sorted/Narnia/Course of a Generation/01 Sail Around the World.mp3
|
||||
Failed to recognize file format.
|
||||
|
||||
Playing: /home/philip/Music/sorted/Narnia/Course of a Generation/02 When the Stars Are Falling.mp3
|
||||
```
|
||||
|
||||
```
|
||||
> git annex version
|
||||
git-annex version: 6.20161012
|
||||
build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify XMPP ConcurrentOutput TorrentParser MagicMime Feeds Quvi
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
|
||||
remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
|
||||
local repository version: 6
|
||||
supported repository versions: 3 5 6
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5
|
||||
operating system: linux x86_64
|
||||
```
|
||||
|
||||
Any consecutive `git annex get` commands don’t notice that the file is not completely transferred and leave it in a broken state.
|
||||
`git annex get --failed` does not correct the problem.
|
||||
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
Yes, it (kind of) works for keeping my music library in sync.
|
||||
|
||||
> [[closing|done]] as it's not clear there's really a bug here. Please
|
||||
> followup to my comment with more information if you disagree.
|
||||
|
|
@ -1,22 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 1"""
|
||||
date="2016-11-16T18:36:34Z"
|
||||
content="""
|
||||
Thing is, git-annex get does not update the file in place. Only once the
|
||||
entire file is downloaded, and its content is verified correct is it moved
|
||||
into a place where you can access it.
|
||||
|
||||
So, it seems much more likely to me that the content of the file, as
|
||||
originally added to git-annex, was bad, and the it had just finished
|
||||
verifying the content and moving it into place when you interruped the
|
||||
command.
|
||||
|
||||
Please check with `git annex fsck` on the file and see if it determines
|
||||
it has the content git-annex expects it to have.
|
||||
|
||||
However, I notice you're using a v6 repository. Is the file an unlocked
|
||||
file? It's possible that in that specific case there could be a bug.
|
||||
I've interrupted `git annex get` on a nearly daily basis for years, but
|
||||
v6 is still experimental and not as well tested.
|
||||
"""]]
|
||||
|
|
@ -1,89 +0,0 @@
|
|||
### Please describe the problem.
|
||||
After successfully adding tags to files, I'm unable to view them. I paste the input and output to reproduce the problem below.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
$ git annex init
|
||||
init
|
||||
Detected a filesystem without fifo support.
|
||||
|
||||
Disabling ssh connection caching.
|
||||
|
||||
Detected a crippled filesystem.
|
||||
(scanning for unlocked files...)
|
||||
|
||||
Entering an adjusted branch where files are unlocked as this filesystem does not support locked files.
|
||||
|
||||
Switched to branch 'adjusted/master(unlocked)'
|
||||
ok
|
||||
(recording state in git...)
|
||||
|
||||
$ touch test-file.txt
|
||||
|
||||
$ git annex add .
|
||||
add test-file.txt
|
||||
ok
|
||||
(recording state in git...)
|
||||
|
||||
$ git commit -a -m "Adds a file."
|
||||
warning: LF will be replaced by CRLF in test-file.txt.
|
||||
The file will have its original line endings in your working directory
|
||||
[adjusted/master(unlocked) b19bcbb] Adds a file.
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 100644 test-file.txt
|
||||
|
||||
$ git annex metadata test-file.txt -t a -t b -t c
|
||||
metadata test-file.txt
|
||||
lastchanged=2021-03-17@18-31-54
|
||||
tag=a
|
||||
tag=b
|
||||
tag=c
|
||||
tag-lastchanged=2021-03-17@18-31-54
|
||||
ok
|
||||
(recording state in git...)
|
||||
|
||||
$ git annex view tag=*
|
||||
view (searching...) fatal: Unable to create 'D:\test-dir\.git\annex\viewindex.lock': File exists.
|
||||
|
||||
Another git process seems to be running in this repository, e.g.
|
||||
an editor opened by 'git commit'. Please make sure all processes
|
||||
are terminated then try again. If it still fails, a git process
|
||||
may have crashed in this repository earlier:
|
||||
remove the file manually to continue.
|
||||
|
||||
git-annex: failed to read sha from git write-tree
|
||||
CallStack (from HasCallStack):
|
||||
error, called at .\Git\Sha.hs:23:15 in main:Git.Sha
|
||||
failed
|
||||
git-annex: view: 1 failed
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
$ git annex version
|
||||
git-annex version: 8.20200815-g335aae266
|
||||
build flags: Assistant Webapp Pairing S3 WebDAV TorrentParser Feeds Testsuite
|
||||
dependency versions: aws-0.21.1 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.4 feed-1.2.0.1 ghc-8.6.5 http-client-0.5.14 persistent-sqlite-2.9.3 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0.1
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256EBLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs hook external
|
||||
operating system: mingw32 x86_64
|
||||
supported repository versions: 8
|
||||
upgrade supported from repository versions: 2 3 4 5 6 7
|
||||
local repository version: 8
|
||||
|
||||
And I'm using the Git Bash terminal on Windows 10 Pro.
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
I'm pretty hyped about git annex. I finally sat down and started backing up my data on different drives and git annex is about the perfect solution. Or at least it does everything I want it to (especially if the above tagging issue can be resolved). I've even thought about writing little cli apps over it, like an ejournal or recipe organizer.
|
||||
|
||||
|
||||
> [[fixed|done]] already some time ago --[[Joey]]
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="Lukey"
|
||||
avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
|
||||
subject="comment 1"
|
||||
date="2021-03-20T17:33:31Z"
|
||||
content="""
|
||||
Hi, <br>
|
||||
This is already fixed in version 8.20201103. However, the windows build seems to be outdated. You can try the latest [autobuild](http://datasets.datalad.org/datalad/packages/windows/) instead.
|
||||
"""]]
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 2"""
|
||||
date="2021-03-22T14:18:03Z"
|
||||
content="""
|
||||
The published windows build uses that autobuild and is up-to-date.
|
||||
The bug reporter just needs to upgrade.
|
||||
"""]]
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="Lukey"
|
||||
avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
|
||||
subject="comment 3"
|
||||
date="2021-03-22T17:19:37Z"
|
||||
content="""
|
||||
Hmm, on the [[windows install page|https://git-annex.branchable.com/install/Windows/]], the 2nd link \"Then, install git-annex: Windows 7+ download\" leads to [this](https://downloads.kitenet.net/git-annex/windows/7/current/), which says \"Last modified: 2020-09-08\".
|
||||
|
||||
"""]]
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 4"""
|
||||
date="2021-03-22T17:51:08Z"
|
||||
content="""
|
||||
Oh I see, forgot that page still linked to the windows 7 build. We lost
|
||||
that autobuilder and with windows 7 fairly EOL only the windows 10 build is
|
||||
being updated. I've changed the link now.
|
||||
"""]]
|
||||
|
|
@ -1,41 +0,0 @@
|
|||
### Please describe the problem.
|
||||
When `git-annex add`ing files with multiple dots in them, the `SHA256E`, `MD5E` (and presumably other `*E`) backends take the extension from the second-to-last dot, instead of the last dot.
|
||||
This annoyed me because I have some photographs with names like `YYYY-mm-dd HH.MM.SS.jpg`. However, it might be intentional considering a `file.tar.gz` would have `tar.gz`.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
[[!format sh """
|
||||
$ touch a a.b a.b.c a.b.c.d
|
||||
$ git-annex add .
|
||||
add a ok
|
||||
add a.b ok
|
||||
add a.b.c ok
|
||||
add a.b.c.d ok
|
||||
$ git-annex lookupkey *
|
||||
SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
|
||||
SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.b
|
||||
SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.b.c
|
||||
SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.c.d
|
||||
"""]]
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
[[!format sh """
|
||||
$ git-annex version
|
||||
git-annex version: 6.20170101+gitg93d69b1-1~ndall+1
|
||||
build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Quvi
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
|
||||
local repository version: 5
|
||||
supported repository versions: 3 5 6
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5
|
||||
operating system: linux x86_64
|
||||
"""]]
|
||||
|
||||
NeuroDebian's `git-annex-standalone` package on Xubuntu 16.04. (Also with a Debian sid chroot with their own `git-annex 6.20161210-1`)
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
I'm trying to analyze and organize my huge `Photos` folder into a neat git-annex repository (with yet another [project of mine](https://www.github.com/alpernebbi/albumin)). It's a huge mess.
|
||||
|
||||
Keep up the great work! Also thanks for fixing [my UTF-8 problem](https://git-annex.branchable.com/bugs/Linux_standalone__39__s_metadata_--batch_can__39__t_parse_UTF-8/) as well.
|
||||
|
||||
> [[Closing|done]] per my comment --[[Joey]]
|
||||
|
|
@ -1,27 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 1"""
|
||||
date="2017-02-20T15:53:29Z"
|
||||
content="""
|
||||
It is indeed to support things like `.tar.gz`, `.pdf.gz` etc.
|
||||
|
||||
There's really no way to distinguish between what's desired to be part of
|
||||
the extension and not, other than heuristics. The heuristics for this are
|
||||
fairly narrow. Other than not allowing purely numeric parts in extenssions,
|
||||
or only taking the last extension, I don't see any change that could help
|
||||
with this case.
|
||||
|
||||
Bear in mind that the only reason the extension is included at all is
|
||||
because some strange programs, especially on OSX, follow symlinks and look
|
||||
at the symlink of the link destination to try to guess what kind of file it
|
||||
is. There's no good reason for a program to do that, and if you're not
|
||||
using programs with that problem, you can just use the SHA256 backend and
|
||||
leave off the extensions.
|
||||
|
||||
Bearing in mind that the extension is part of a big long ugly key name that
|
||||
you don't generally need to do anything with, including an extra short
|
||||
possibly not-really-extension part doesn't seem like a usability problem.
|
||||
The only real problem would be if you had two files with the same content,
|
||||
but different "extensions", since this would prevent them being
|
||||
deduplicated to the same key.
|
||||
"""]]
|
||||
|
|
@ -1,73 +0,0 @@
|
|||
### Please describe the problem.
|
||||
Unlocking a large file on the Synology NAS results in "Cannot handle files this big" errors with every following other git command (git status, git diff, ... and hereby git annex status/sync etc.).
|
||||
|
||||
Unfortunately I miss the technical understanding how the pointer files are hidden from git - I've seen the smudge/clean filters though.
|
||||
|
||||
How does it work?
|
||||
After a file is unlocked, it is physically part of the working tree and also part of the git history, so a git status/diff will naturally try to index/check that file without running the annex filter, which then results in this error message, right?
|
||||
So how should these unlocked pointer files be hidden technically to not be indexed and checked by git?
|
||||
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
$ git init
|
||||
Initialized empty Git repository in /volume1/homes/admin/git-annex/test3/.git/
|
||||
$ git annex init --version 8
|
||||
init (scanning for unlocked files...)
|
||||
ok
|
||||
(recording state in git...)
|
||||
$ ls -lah
|
||||
total 12K
|
||||
drwxr-xr-x 3 admin users 4.0K May 16 17:00 .
|
||||
drwxr-xr-x 8 admin users 4.0K May 16 17:00 ..
|
||||
drwxr-xr-x 9 admin users 4.0K May 16 17:01 .git
|
||||
-rw-r--r-- 1 admin users 20G May 16 17:00 output
|
||||
$ git annex add output
|
||||
add output
|
||||
$ git annex sync
|
||||
$ git annex unlock output
|
||||
$ git annex status
|
||||
fatal: Cannot handle files this big
|
||||
git-annex: git status failed
|
||||
$ git status
|
||||
fatal: Cannot handle files this big
|
||||
$ git diff
|
||||
fatal: Cannot handle files this big
|
||||
|
||||
Why does git even feel responsible to load this file?
|
||||
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
Synology NAS
|
||||
|
||||
git-annex version: 8.20200331-g111b747be
|
||||
build flags: Assistant Webapp Pairing S3 WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite
|
||||
dependency versions: aws-0.20 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.3 feed-1.0.1.0 ghc-8.6.5 http-client-0.5.14 persistent-sqlite-2.9.3 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs hook external
|
||||
operating system: linux arm
|
||||
supported repository versions: 8
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5 6 7
|
||||
local repository version: 8
|
||||
|
||||
git version 2.26.1
|
||||
|
||||
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
Unlocking files (with thin set to true) seems to be the perfect solution for me - I just have to understand what's going on under the hood ;)
|
||||
|
||||
> [[closing|done]] because this appears to be a git problem if anything,
|
||||
> there does not seem to be anything in git-annex that could be changed
|
||||
> to avoid it --[[Joey]]
|
||||
|
|
@ -1,22 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 1"""
|
||||
date="2020-05-19T17:03:09Z"
|
||||
content="""
|
||||
git is configured to filter the file through git-annex, so git does not
|
||||
need to deal with the large content of the file.
|
||||
|
||||
But this error message seems to come from part of git that just stats the
|
||||
file, and looks at its size. So there's nothing git-annex is configuring
|
||||
wrong, and using git-lfs would likewise make git fail the same.
|
||||
|
||||
I don't think this is a bug in git-annex. It should be reported to the git
|
||||
developers. The error comes from `xsize_t`, and according to git's commit
|
||||
dc49cd769b5fa6b7e0114b051c34a849828a7603 it was intended to detect cases
|
||||
where git can't possibly malloc enough memory for a file, because `size_t`
|
||||
is not wide enough, compared with `off_t`.
|
||||
|
||||
On linux amd64, both are 8. On linux i386 and arm, both are 4. So I don't
|
||||
think any usual linux systems are affected by this, only unusual systems
|
||||
that use different sizes for them.
|
||||
"""]]
|
||||
|
|
@ -1,28 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="braun.markus89@51b521a42cc994db864df308627bd6454f9c309d"
|
||||
nickname="braun.markus89"
|
||||
avatar="http://cdn.libravatar.org/avatar/c11d06a0d9db6a9472b05ee01c342ca4"
|
||||
subject="comment 2"
|
||||
date="2020-05-20T13:54:23Z"
|
||||
content="""
|
||||
Thanks for your answer.
|
||||
|
||||
Short follow up question.
|
||||
When I do exactly the same for a 2G file, something similar happens:
|
||||
|
||||
$ git annex sync --debug
|
||||
[2020-05-20 15:48:19.441795963] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"git-annex\"]
|
||||
[2020-05-20 15:48:19.459542967] process done ExitSuccess
|
||||
[2020-05-20 15:48:19.460055539] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"]
|
||||
[2020-05-20 15:48:19.47249456] process done ExitSuccess
|
||||
[2020-05-20 15:48:19.473466546] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..9655aad25802451eb83141096fb9275aa36fe810\",\"--pretty=%H\",\"-n1\"]
|
||||
[2020-05-20 15:48:19.487917815] process done ExitSuccess
|
||||
[2020-05-20 15:48:19.489243941] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"cat-file\",\"--batch\"]
|
||||
[2020-05-20 15:48:19.490737137] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"cat-file\",\"--batch-check=%(objectname) %(objecttype) %(objectsize)\"]
|
||||
commit
|
||||
[2020-05-20 15:48:19.506415618] call: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"commit\",\"-a\",\"-m\",\"git-annex in admin@Paintower:~/git-annex/test\"]
|
||||
fatal: mmap failed: Cannot allocate memory
|
||||
|
||||
So why does the \"git commit\" allocate so much memory? It seems like it tries to handle the file content itself? Or is it a malloc failure caused by git annex smudge?
|
||||
|
||||
"""]]
|
||||
|
|
@ -1,18 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 2"""
|
||||
date="2020-05-21T16:54:48Z"
|
||||
content="""
|
||||
I don't think there's any reason why git should need to allocate memory for
|
||||
the whole file. All it's doing is reading the file and sending it over a
|
||||
pipe, and git-annex actually hangs up the pipe before reading anything.
|
||||
So git could read a few bytes, try to send them, and give up.
|
||||
|
||||
The last time I looked at it, in [[!commit a96972015dd76271b46432151e15d5d38d7151ff]]
|
||||
having git-annex close the pipe immediately did avoid git overallocating
|
||||
memory. I don't know what else git-annex could do to avoid any bad behavior
|
||||
from git.
|
||||
|
||||
If your git is older than 2.5, it would certainly explain it, since the
|
||||
above workaround needed git 2.5.
|
||||
"""]]
|
||||
|
|
@ -1,25 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="braun.markus89@51b521a42cc994db864df308627bd6454f9c309d"
|
||||
nickname="braun.markus89"
|
||||
avatar="http://cdn.libravatar.org/avatar/c11d06a0d9db6a9472b05ee01c342ca4"
|
||||
subject="comment 4"
|
||||
date="2020-05-25T11:06:05Z"
|
||||
content="""
|
||||
git version is 2.26.1, so this should be fine.
|
||||
|
||||
I guess, git-annex couldn't do it any better, still git tries to overallocate memory.
|
||||
My Synology NAS only got 1gb of memory (at least 600mb are used all the time), so I wonder why \"hanging up\" the pipe works when unlocking 1gb file but not for 2gb. But the Synology linux is a little bit weird, so I have to give up on debugging there. The 1gb memory spec of my NAS is ridiculously low (maybe too low for git anyways) and cannot be upgraded....
|
||||
|
||||
For the sake of documentation for other synology users with a low-spec NAS
|
||||
|
||||
Workaround that did work
|
||||
|
||||
* create a user with uid/gid matching the NAS user and mount via NFSv3. On the client system the memory is sufficient to run every git-annex command.
|
||||
|
||||
Workarounds that did not work out
|
||||
|
||||
* NFSv4 with idmapping (configuring the Kerberos authentication would have taken a lot of time and not even sure if it would have worked out in the end)
|
||||
* SSHFS (the sshfs server provided by Synology seems to be broken, resulting in broken symlinks on my linux client system -> which is obviously a no go for git annex ;-) )
|
||||
|
||||
|
||||
"""]]
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="braun.markus89@51b521a42cc994db864df308627bd6454f9c309d"
|
||||
nickname="braun.markus89"
|
||||
avatar="http://cdn.libravatar.org/avatar/c11d06a0d9db6a9472b05ee01c342ca4"
|
||||
subject="comment 5"
|
||||
date="2020-05-25T11:07:07Z"
|
||||
content="""
|
||||
so, thank you once again, Joey, in particular for this great tool ;-)
|
||||
"""]]
|
||||
|
|
@ -1,31 +0,0 @@
|
|||
### Please describe the problem.
|
||||
|
||||
someone kinda could consider it a 'feature' but it complicates use of the output since then progressbar should jump down and some progressbar libraries
|
||||
do not "support" that
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
initiate download, interrupt it, try to redownload it... I guess in some cases redownload doesn't start at the point where it was previously interrupted but somewhat before, or restarts altogether. But annex first reports in --json-progress the size of previously downloaded portion and then goes down. see below
|
||||
|
||||
so, my life would be easier, if annex did not report "unconfirmed initial progress" at all I guess
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
6.20170307+gitg24ade8a25-1~ndall+1
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
$> ls -l .git/annex/tmp; k=$(/bin/ls .git/annex/tmp | head -n 1); [ -z "$k" ] || git annex get --json --json-progress --key $k
|
||||
total 3796
|
||||
-rw------- 1 yoh yoh 3887104 Mar 24 17:06 MD5E-s4108657--e055fc250b37b313c0904f3687bbed1c
|
||||
{"byte-progress":3887104,"action":{"command":"get","note":"from origin...","key":"MD5E-s4108657--e055fc250b37b313c0904f3687bbed1c","file":null},"total-size":4108657,"percent-progress":"94.61%"}
|
||||
{"byte-progress":2068480,"action":{"command":"get","note":"from origin...","key":"MD5E-s4108657--e055fc250b37b313c0904f3687bbed1c","file":null},"total-size":4108657,"percent-progress":"50.34%"}
|
||||
{"command":"get","note":"checksum...","success":true,"key":"MD5E-s4108657--e055fc250b37b313c0904f3687bbed1c","file":null}
|
||||
|
||||
"""]]
|
||||
|
||||
|
||||
> [[Closing|done]] since it seems I fixed the most egrarious case where
|
||||
> this could happen, and since it does seem that, in general, it's ok for
|
||||
> git-annex to report progress that is not strictly increasing. --[[Joey]]
|
||||
|
|
@ -1,29 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 1"""
|
||||
date="2017-05-25T17:56:48Z"
|
||||
content="""
|
||||
That looks like a git remote accessed perhaps by rsync, or perhaps locally?
|
||||
|
||||
I'd be surprised if a rsync transfer did this, because AFAIK all progress
|
||||
updates come from rsync's own progress display, and that does not jump
|
||||
backward.
|
||||
|
||||
Local file copies (when not using rsync), and some other types of remotes,
|
||||
poll the size of the temp file to determine how much data has been
|
||||
received, and so if the transfer doesn't resume, they will do this. **I've
|
||||
made it avoid reporting the file size until the file size has changed once,
|
||||
which avoids the problem in this case.**
|
||||
|
||||
Another way it could happen is when a transfer fails partway and git-annex
|
||||
immediately retries and the retry fails to resume. In
|
||||
this case, the progress would go to some percent for the first transfer,
|
||||
and then could reset to a lower percent for the retry, and that
|
||||
reflects what's really happening. Eg, 50% of it transferred and now
|
||||
we've unfortunately started over at 0%.
|
||||
|
||||
I could make the reported progress always be monotonically increasing, but
|
||||
then in that retry cases it would just seem to stall, perhaps for a long
|
||||
period of time. Not sure that's better than a progress display that while
|
||||
annoying, reflects what's really going on.
|
||||
"""]]
|
||||
|
|
@ -1,21 +0,0 @@
|
|||
Without this patch, building on OpenBSD fails with a type error in `Utility/DirWatcher/Kqueue.hs`.
|
||||
|
||||
I'm running OpenBSD current. This patch is against master (commit 7d18153955).
|
||||
|
||||
```
|
||||
diff --git a/Utility/DirWatcher/Kqueue.hs b/Utility/DirWatcher/Kqueue.hs
|
||||
index 4a1c55ae9..0c84f67c1 100644
|
||||
--- a/Utility/DirWatcher/Kqueue.hs
|
||||
+++ b/Utility/DirWatcher/Kqueue.hs
|
||||
@@ -133,7 +133,7 @@ removeSubDir dirmap dir = do
|
||||
mapM_ Posix.closeFd $ M.keys toremove
|
||||
return rest
|
||||
where
|
||||
- (toremove, rest) = M.partition (dirContains dir . dirName) dirmap
|
||||
+ (toremove, rest) = M.partition (dirContains (toRawFilePath dir) . toRawFilePath . dirName) dirmap
|
||||
|
||||
findDirContents :: DirMap -> FilePath -> [FilePath]
|
||||
findDirContents dirmap dir = concatMap absolutecontents $ search
|
||||
```
|
||||
|
||||
> [[done]] --[[Joey]]
|
||||
|
|
@ -1,12 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="falsifian"
|
||||
avatar="http://cdn.libravatar.org/avatar/59c3c23c500d20d83ecb9d1f149be9ae"
|
||||
subject="tests mostly pass"
|
||||
date="2021-01-31T05:07:16Z"
|
||||
content="""
|
||||
FYI, `git annex test` has just a few failures, which can all be blamed on my setup.
|
||||
|
||||
(git-annex is not in PATH seen by commands run via ssh, so I get a bunch of failures like \"/bin/sh: git-annex-shell: not found\". In practice I work around this by setting remote.xxx.annex-shell=/home/falsifian/bin/git-annex-shell in other repos.)
|
||||
|
||||
PS if you want to credit me in a commit message, I'm `James Cook <falsifian@falsifian.org>`.
|
||||
"""]]
|
||||
|
|
@ -1,7 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 2"""
|
||||
date="2021-02-01T15:54:22Z"
|
||||
content="""
|
||||
Thank you, I've applied that.
|
||||
"""]]
|
||||
|
|
@ -1,144 +0,0 @@
|
|||
### Please describe the problem.
|
||||
|
||||
git-annex has issues when trying to deal with SSH (and possibly other kinds) of URLs which have the form:
|
||||
|
||||
```
|
||||
ssh://user@host/~
|
||||
```
|
||||
|
||||
When git-annex tries to perform tilde-expansion the path part of the URL on the remote side,
|
||||
it runs into problems because the function responsible for doing this (`expandTilde` in `Git/Construct.hs`)
|
||||
does not correctly handle the expansion of home directory paths which do not end in a slash,
|
||||
such as `~` or `/~`. It will correctly handle strings like `/~/` or `~/`, which is why SSH
|
||||
URLs of the form `ssh://user@host/~/` *will* work.
|
||||
|
||||
Examining the definition of `expandTilde` makes it clear why this is true:
|
||||
|
||||
```haskell
|
||||
expandTilde :: FilePath -> IO FilePath
|
||||
#ifdef mingw32_HOST_OS
|
||||
expandTilde = return
|
||||
#else
|
||||
expandTilde = expandt True
|
||||
where
|
||||
expandt _ [] = return ""
|
||||
expandt _ ('/':cs) = do
|
||||
v <- expandt True cs
|
||||
return ('/':v)
|
||||
expandt True ('~':'/':cs) = do
|
||||
h <- myHomeDir
|
||||
return $ h </> cs
|
||||
expandt True ('~':cs) = do
|
||||
let (name, rest) = findname "" cs
|
||||
u <- getUserEntryForName name
|
||||
return $ homeDirectory u </> rest
|
||||
expandt _ (c:cs) = do
|
||||
v <- expandt False cs
|
||||
return (c:v)
|
||||
findname n [] = (n, "")
|
||||
findname n (c:cs)
|
||||
| c == '/' = (n, cs)
|
||||
| otherwise = findname (n++[c]) cs
|
||||
```
|
||||
|
||||
The expression `expandTilde "~"` will eventually match the fourth pattern for `expandt`.
|
||||
Since `cs == ""` in this context, `name` will also evaluate to `""`.
|
||||
This means that `getUserEntryForName` will be called with the null string as an argument.
|
||||
Since there is no user on the system with the null string as a username,
|
||||
`getUserEntryForName` will throw an exception.
|
||||
This will cause git-annex to spit out an error message:
|
||||
|
||||
```
|
||||
get testfile (from origin...)
|
||||
git-annex-shell: getUserEntryForName: does not exist (no such user)
|
||||
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
|
||||
rsync error: error in rsync protocol data stream (code 12) at io.c(235) [Receiver=3.1.3]
|
||||
|
||||
rsync failed -- run git annex again to resume file transfer
|
||||
|
||||
Unable to access these remotes: origin
|
||||
|
||||
Try making some of these repositories available:
|
||||
1f5118ff-a50e-4bf1-a372-960774bce0ab -- user@A:~/ [origin]
|
||||
failed
|
||||
git-annex: get: 1 failed
|
||||
```
|
||||
|
||||
Fixing the problem is simple enough.
|
||||
All that needs to be done is to add an equation for `expandt` to handle the case where `~` appears at the end of a string.
|
||||
See the following patch:
|
||||
|
||||
```
|
||||
From 680873923197f5eec15365b3e47e3fa05b9573be Mon Sep 17 00:00:00 2001
|
||||
From: Grond <grond66@riseup.net>
|
||||
Date: Thu, 14 Jan 2021 18:16:31 -0800
|
||||
Subject: [PATCH] Fix expandTilde so that it can handle tildes at the end of
|
||||
it's input
|
||||
|
||||
---
|
||||
Git/Construct.hs | 1 +
|
||||
1 file changed, 1 insertion(+)
|
||||
|
||||
diff --git a/Git/Construct.hs b/Git/Construct.hs
|
||||
index 8b63ac480..a369bc4a6 100644
|
||||
--- a/Git/Construct.hs
|
||||
+++ b/Git/Construct.hs
|
||||
@@ -187,6 +187,7 @@ expandTilde = expandt True
|
||||
expandt True ('~':'/':cs) = do
|
||||
h <- myHomeDir
|
||||
return $ h </> cs
|
||||
+ expandt True "~" = myHomeDir
|
||||
expandt True ('~':cs) = do
|
||||
let (name, rest) = findname "" cs
|
||||
u <- getUserEntryForName name
|
||||
--
|
||||
2.20.1
|
||||
|
||||
```
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
1. Create `testfile` in a git-annex repo of your home directory on host `A`
|
||||
2. Run `git annex add testfile` in the repo on `A`
|
||||
3. Run `git commit`
|
||||
4. Clone your home directory on `A` onto host `B` using `git clone ssh://me@A/~ homedir_A`
|
||||
5. `cd` into `homedir_A`
|
||||
6. Run `git annex get testfile`
|
||||
7. Watch git-annex fail to fetch the file
|
||||
8. Run `git remote set-url origin ssh://me@A/~/` to set the remote URL to be something git-annex can deal with
|
||||
9. Run `git annex get testfile` again
|
||||
10. Watch git-annex suddenly succeed
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
I'm running Debian 10.7.
|
||||
|
||||
The output of `git annex version` is:
|
||||
|
||||
```
|
||||
git-annex version: 7.20190129
|
||||
build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite
|
||||
dependency versions: aws-0.20 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.3 feed-1.0.0.0 ghc-8.4.4 http-client-0.5.13.1 persistent-sqlite-2.8.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external
|
||||
operating system: linux x86_64
|
||||
supported repository versions: 5 7
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5 6
|
||||
local repository version: 5
|
||||
```
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
Definitely! I'm currently writing some personal file synchronization software that uses git-annex for myself, which is how I noticed this bug.
|
||||
|
||||
> Thanks for a perfect bug report and patch. [[done]] --[[Joey]]
|
||||
|
|
@ -1,40 +0,0 @@
|
|||
### Please describe the problem.
|
||||
|
||||
`cabal v2-install` produces many lines like this in the build log:
|
||||
|
||||
/bin/sh: ./Build/mdwn2man: cannot execute - Permission denied
|
||||
|
||||
and the man pages don't get copied to `~/.cabal/store/...`.
|
||||
|
||||
My guess is that Cabal is losing the file mode during some tarring/untarring.
|
||||
|
||||
Here's a patch that fixes it. (It makes the `#/usr/bin/env perl` at the start of the file redundant; not sure if you want to delete it.)
|
||||
|
||||
```
|
||||
diff --git a/Build/Mans.hs b/Build/Mans.hs
|
||||
index 9fb29d4a3..672dcd71c 100644
|
||||
--- a/Build/Mans.hs
|
||||
+++ b/Build/Mans.hs
|
||||
@@ -38,7 +38,8 @@ buildMans = do
|
||||
if (Just srcm > destm)
|
||||
then do
|
||||
r <- system $ unwords
|
||||
- [ "./Build/mdwn2man"
|
||||
+ [ "perl"
|
||||
+ , "Build/mdwn2man"
|
||||
, progName src
|
||||
, "1"
|
||||
, src
|
||||
```
|
||||
|
||||
(Related question: is the installation supposed to put the man pages anywhere other than `~/.cabal/store/ghc-XXX/git-annex-XXX`? I.e. is it up to the user to either add that directory to `MANPATH` or copy them somewhere from there?)
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
OpenBSD current
|
||||
|
||||
Cabal 3.4.0.0
|
||||
|
||||
git-annex master (commit `4e5a27841`)
|
||||
|
||||
> [[done]] --[[Joey]]
|
||||
|
|
@ -1,17 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 1"""
|
||||
date="2021-02-08T18:20:31Z"
|
||||
content="""
|
||||
Reproduced by running `cabal unpack git-annex`. While the tarball on
|
||||
hackage does have execute bits, somehow cabal is not preserving them.
|
||||
Applied your patch, thanks.
|
||||
|
||||
Cabal's ability to install anything more than a command into PATH is very
|
||||
limited, and also there's not really a good way to install man pages
|
||||
in a user's home directory in a way that will make man find them (ie,
|
||||
there's probably one way or less per OS, and all different).
|
||||
So it's best to use the Makefile (from a git clone of the repo)
|
||||
if you want to install git-annex in a more complete manner,
|
||||
ie system wide or generating a package of its files.
|
||||
"""]]
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="falsifian"
|
||||
avatar="http://cdn.libravatar.org/avatar/59c3c23c500d20d83ecb9d1f149be9ae"
|
||||
subject="comment 2"
|
||||
date="2021-02-12T03:52:37Z"
|
||||
content="""
|
||||
Thanks for the tip. (FYI: I'm working on packaging it for OpenBSD. Currently we just have a hack in place that manually runs the mdwn2man script after the cabal build. The Cabal packaging module it's built on probably isn't designed to call a Makefile but there may be some way to do it. In any case, it seems to work, so we might just leave it as is.)
|
||||
"""]]
|
||||
|
|
@ -1,52 +0,0 @@
|
|||
This is a satellite issue the one on [strange mimetype driven addition](http://git-annex.branchable.com/bugs/manages_to_incorrectly_add_to_annex_instead_of_git_based_on___34__mimetype__34___-_we_cannot_figure_it_out_why/?updated)
|
||||
|
||||
We do get report on addition of the file twice (when it switches from annex to git?)
|
||||
|
||||
[[!format sh """
|
||||
(git)smaug:/mnt/btrfs/scrap/tmp/SIMON[master]data_BIDS
|
||||
$> rm -f TEST.txt; cat BADFILE.txt >| TEST.txt; file --mime TEST.txt; git annex add TEST.txt
|
||||
TEST.txt: text/plain; charset=utf-8
|
||||
add TEST.txt ok
|
||||
(recording state in git...)
|
||||
|
||||
$> rm -f TEST.txt; sed -e 's,[{}],,g' BADFILE.txt >| TEST.txt; file --mime TEST.txt; git annex add TEST.txt
|
||||
TEST.txt: text/plain; charset=utf-8
|
||||
add TEST.txt (non-large file; adding content to git repository) ok
|
||||
add TEST.txt (non-large file; adding content to git repository) ok
|
||||
(recording state in git...)
|
||||
|
||||
"""]]
|
||||
|
||||
and here is that run with `--debug`:
|
||||
|
||||
[[!format sh """
|
||||
$> rm -f TEST.txt; sed -e 's,[{}],,g' BADFILE.txt >| TEST.txt; file --mime TEST.txt; git annex add --debug TEST.txt
|
||||
TEST.txt: text/plain; charset=utf-8
|
||||
[2019-04-26 09:57:45.451632679] read: git ["--git-dir=../.git","--work-tree=..","--literal-pathspecs","symbolic-ref","-q","HEAD"]
|
||||
[2019-04-26 09:57:45.455061047] process done ExitSuccess
|
||||
[2019-04-26 09:57:45.455134123] read: git ["--git-dir=../.git","--work-tree=..","--literal-pathspecs","show-ref","refs/heads/master"]
|
||||
[2019-04-26 09:57:45.466478693] process done ExitSuccess
|
||||
[2019-04-26 09:57:45.466612578] read: git ["--git-dir=../.git","--work-tree=..","--literal-pathspecs","ls-files","--others","--exclude-standard","-z","--","TEST.txt"]
|
||||
[2019-04-26 09:57:45.471421295] read: git ["--git-dir=../.git","--work-tree=..","--literal-pathspecs","ls-files","--modified","-z","--","TEST.txt"]
|
||||
[2019-04-26 09:57:45.479659995] chat: git ["--git-dir=../.git","--work-tree=..","--literal-pathspecs","check-attr","-z","--stdin","annex.backend","annex.numcopies","annex.largefiles","--"]
|
||||
[2019-04-26 09:57:45.479997471] read: git ["--version"]
|
||||
[2019-04-26 09:57:45.482476867] process done ExitSuccess
|
||||
add TEST.txt (non-large file; adding content to git repository) ok
|
||||
[2019-04-26 09:57:45.494729569] read: git ["--git-dir=../.git","--work-tree=..","--literal-pathspecs","diff","--name-only","--diff-filter=T","-z","--","TEST.txt"]
|
||||
[2019-04-26 09:57:45.498615035] chat: git ["--git-dir=../.git","--work-tree=..","--literal-pathspecs","cat-file","--batch"]
|
||||
[2019-04-26 09:57:45.499023359] chat: git ["--git-dir=../.git","--work-tree=..","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
|
||||
add TEST.txt (non-large file; adding content to git repository) ok
|
||||
(recording state in git...)
|
||||
[2019-04-26 09:57:45.509449016] feed: xargs ["-0","git","--git-dir=../.git","--work-tree=..","--literal-pathspecs","add","--"]
|
||||
[2019-04-26 09:57:45.54937333] process done ExitSuccess
|
||||
[2019-04-26 09:57:45.550142214] process done ExitSuccess
|
||||
[2019-04-26 09:57:45.550690643] process done ExitSuccess
|
||||
[2019-04-26 09:57:45.551282227] process done ExitSuccess
|
||||
|
||||
|
||||
"""]]
|
||||
|
||||
annex 7.20190219+git191-g2d6a364d4-1~ndall+1
|
||||
|
||||
> [[fixed|done]] this affected v5 repos but was fixed in v6 and above.
|
||||
> --[[Joey]]
|
||||
|
|
@ -1,14 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 1"""
|
||||
date="2019-05-06T16:30:41Z"
|
||||
content="""
|
||||
This is because git-annex add makes multiple passes looking for files in
|
||||
different situations. In particular, the withFilesMaybeModified and
|
||||
withFilesOldUnlocked passes both find the file.
|
||||
|
||||
I don't immediately see a way to improve it. But it is avoided in v6 mode.
|
||||
|
||||
(I'm fairly sure someone, possibly even you, has reported this same issue
|
||||
before, so it's probably still open somewhere.)
|
||||
"""]]
|
||||
|
|
@ -1,54 +0,0 @@
|
|||
In some occasions `annex.adjustedbranchrefresh` is ignored when `git annex sync` is run in a branch created with `adjust --unlock-present`.
|
||||
|
||||
If `annex.adjustedbranchrefresh` is set to 1, one would expect git-annex to automatically adjust the branch once a file has been `git annex add`-ed or the repository is `git annex sync`-ed. However this does not happen and a manual `git annex adjust --unlock-present` is required.
|
||||
|
||||
Is this a bug or am I misunderstanding how `annex.adjustedbranchrefresh` is supposed to work?
|
||||
|
||||
> It is a bug --[[Joey]]
|
||||
|
||||
The following script reproduces this bug.
|
||||
|
||||
```
|
||||
#!/bin/bash
|
||||
|
||||
set -eux
|
||||
|
||||
rm -Rvf /tmp/an-repo.git && mkdir /tmp/an-repo.git && cd /tmp/an-repo.git
|
||||
git init --bare
|
||||
n=$(date +%s) ; mkdir /tmp/ga-$n && cd /tmp/ga-$n
|
||||
git clone --no-local --no-hardlinks /tmp/an-repo.git
|
||||
cd an-repo/
|
||||
|
||||
git config user.email "email@example.com" ; git config user.name "Name Name"
|
||||
git config annex.thin true
|
||||
git config annex.adjustedbranchrefresh 1
|
||||
git config remote.origin.annex-ignore true
|
||||
|
||||
# 8.20201117 is the version in the standalone tarball of 8.20201127
|
||||
~/Applications/git-annex/8.20201117-ga314537cd/runshell bash -c '
|
||||
git annex init foobar
|
||||
|
||||
echo "aaaa" > a && echo "bbbb" > b
|
||||
git annex add a b
|
||||
git annex sync
|
||||
|
||||
git annex adjust --unlock-present
|
||||
git annex sync
|
||||
|
||||
echo "cccc" > c && echo "dddd" > d
|
||||
git annex add c d
|
||||
|
||||
echo "## before sync"
|
||||
stat -c "%n: %F" a b c d
|
||||
|
||||
git annex sync
|
||||
echo "## after sync"
|
||||
stat -c "%n: %F" a b c d # should show four regular files, but shows two files and two symlinks
|
||||
|
||||
git annex sync --content;
|
||||
echo "## after sync --content"
|
||||
stat -c "%n: %F" a b c d # ibid
|
||||
'
|
||||
```
|
||||
|
||||
> [[fixed|done]] --[[Joey]]
|
||||
|
|
@ -1,14 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 1"""
|
||||
date="2021-01-04T20:57:22Z"
|
||||
content="""
|
||||
So I think the whole problem is that, git-annex add (and import, addurl,
|
||||
etc) should add the files unlocked when in an unlockpresent branch.
|
||||
|
||||
I don't think git-annex sync needs to deal with this, problably.
|
||||
|
||||
Don't think this really has anything to do with adjustedbranchrefresh.
|
||||
That's about updates after getting/dropping files, and that's not been done
|
||||
in this case.
|
||||
"""]]
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 2"""
|
||||
date="2021-01-25T17:46:03Z"
|
||||
content="""
|
||||
Oh, git-annex add already adds it unlocked in an adjusted unlocked branch,
|
||||
so that just needs to be done for this new type of branch too.
|
||||
"""]]
|
||||
|
|
@ -1,153 +0,0 @@
|
|||
### Please describe the problem.
|
||||
git-annex doesn't seem to honor the remote.\<name\>.annex-rsync-upload-options or annex.rsync-upload-options anymore. The problem seems to be on the server side. I noticed it when upgrading the ancient git-annex on my server from the version in stretch (6.20170101, --bwlimit worked fine with that) to the version in stretch-backports (7.20190129, --bwlimit broken). On the client I'm running bullseye and git-annex version 8.20210223. I then upgraded git-annex on the server to 8.20210903, the rsync-options don't work with that either.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
#### Client:
|
||||
|
||||
root@dition:/mnt/sneakerdisk9# mkdir testannex
|
||||
root@dition:/mnt/sneakerdisk9# cd testannex
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git init
|
||||
hint: Using 'master' as the name for the initial branch. This default branch name
|
||||
hint: is subject to change. To configure the initial branch name to use in all
|
||||
hint: of your new repositories, which will suppress this warning, call:
|
||||
hint:
|
||||
hint: git config --global init.defaultBranch <name>
|
||||
hint:
|
||||
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
|
||||
hint: 'development'. The just-created branch can be renamed via this command:
|
||||
hint:
|
||||
hint: git branch -m <name>
|
||||
Initialized empty Git repository in /mnt/sneakerdisk9/testannex/.git/
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git annex init dition
|
||||
init dition (scanning for unlocked files...)
|
||||
ok
|
||||
(recording state in git...)
|
||||
root@dition:/mnt/sneakerdisk9/testannex# dd if=/dev/random of=bigtestfile bs=1MiB count=500
|
||||
500+0 records in
|
||||
500+0 records out
|
||||
524288000 bytes (524 MB, 500 MiB) copied, 6.00526 s, 87.3 MB/s
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git annex add bigtestfile
|
||||
add bigtestfile
|
||||
ok
|
||||
(recording state in git...)
|
||||
|
||||
#### Server:
|
||||
|
||||
dition@holm:/mnt/btrfs/testannex.git$ git init --bare
|
||||
Initialized empty Git repository in /mnt/btrfs/testannex.git/
|
||||
|
||||
#### Client:
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git remote add holm ssh://dition@holm:1322/mnt/btrfs/testannex.git
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git push --all holm
|
||||
Enumerating objects: 10, done.
|
||||
Counting objects: 100% (10/10), done.
|
||||
Delta compression using up to 4 threads
|
||||
Compressing objects: 100% (7/7), done.
|
||||
Writing objects: 100% (10/10), 871 bytes | 217.00 KiB/s, done.
|
||||
Total 10 (delta 0), reused 0 (delta 0), pack-reused 0
|
||||
To ssh://holm:1322/mnt/btrfs/testannex.git
|
||||
* [new branch] git-annex -> git-annex
|
||||
|
||||
#### Server:
|
||||
|
||||
dition@holm:/mnt/btrfs/testannex.git$ git annex init holm
|
||||
init holm ok
|
||||
(recording state in git...)
|
||||
|
||||
#### Client:
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git config --add annex.rsync-upload-options "--bwlimit 1024"
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git annex sync
|
||||
commit
|
||||
[master (root-commit) d0f28fe] git-annex in dition
|
||||
Committer: root <root@dition.tulpe>
|
||||
Your name and email address were configured automatically based
|
||||
on your username and hostname. Please check that they are accurate.
|
||||
You can suppress this message by setting them explicitly. Run the
|
||||
following command and follow the instructions in your editor to edit
|
||||
your configuration file:
|
||||
|
||||
git config --global --edit
|
||||
|
||||
After doing this, you may fix the identity used for this commit with:
|
||||
|
||||
git commit --amend --reset-author
|
||||
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 120000 bigtestfile
|
||||
ok
|
||||
pull holm
|
||||
remote: Counting objects: 3, done.
|
||||
remote: Compressing objects: 100% (3/3), done.
|
||||
remote: Total 3 (delta 0), reused 0 (delta 0)
|
||||
Unpacking objects: 100% (3/3), 363 bytes | 121.00 KiB/s, done.
|
||||
From ssh://holm:1322/mnt/btrfs/testannex
|
||||
5ace97f..70df03f git-annex -> holm/git-annex
|
||||
ok
|
||||
(merging holm/git-annex into git-annex...)
|
||||
push holm
|
||||
Enumerating objects: 3, done.
|
||||
Counting objects: 100% (3/3), done.
|
||||
Delta compression using up to 4 threads
|
||||
Compressing objects: 100% (2/2), done.
|
||||
Writing objects: 100% (3/3), 314 bytes | 314.00 KiB/s, done.
|
||||
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
|
||||
To ssh://holm:1322/mnt/btrfs/testannex.git
|
||||
* [new branch] git-annex -> synced/git-annex
|
||||
* [new branch] master -> synced/master
|
||||
ok
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git annex copy --to holm bigtestfile
|
||||
copy bigtestfile (to holm...)
|
||||
4% 19.27 MiB 3 MiB/s 2m31s
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
#### Server:
|
||||
Debian stretch amd64 with git-annex from the neurodebian repo
|
||||
|
||||
dition@holm:/mnt/btrfs/testannex.git$ git annex version
|
||||
git-annex version: 8.20210903-1~ndall+1
|
||||
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV
|
||||
dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1
|
||||
persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 S
|
||||
HA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B51
|
||||
2 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256
|
||||
BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WOR
|
||||
M URL X*
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg
|
||||
hook external
|
||||
operating system: linux x86_64
|
||||
supported repository versions: 8
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5 6 7
|
||||
|
||||
#### Client:
|
||||
|
||||
Debian bullseye arm64
|
||||
|
||||
root@dition:/mnt/sneakerdisk9# git annex version
|
||||
git-annex version: 8.20210223
|
||||
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV
|
||||
dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
|
||||
operating system: linux aarch64
|
||||
supported repository versions: 8
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5 6 7
|
||||
|
||||
I can also reproduce this with a debian buster client on amd64 with git-annex 8.20200330
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
As mentioned, the problem was introduced somewhere between 6.20170101 and 7.20190129.
|
||||
|
||||
annex.rsync-download-options "--bwlimit 1024" is ignored as well:
|
||||
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git config --add annex.rsync-download-options "--bwlimit 1024"
|
||||
root@dition:/mnt/sneakerdisk9/testannex# git annex get --from holm bigtestfile
|
||||
get bigtestfile (from holm...)
|
||||
10% 51.82 MiB 19 MiB/s 23s
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
I've been using git-annex for many years with great success. It's a pleasure to use, I don't know how I ever managed to get by without it!
|
||||
|
||||
> Closing as a duplicate of the bwlimit todo. [[done]] --[[Joey]]
|
||||
|
|
@ -1,18 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 1"""
|
||||
date="2021-09-21T17:24:59Z"
|
||||
content="""
|
||||
Since that old version of git-annex, it has changed to using a different
|
||||
protocol than rsync for transfers over ssh. So the rsync options no longer
|
||||
apply to that. They are still used when git-annex does use rsync, either a
|
||||
rsync special remote or a server with too old a version of git-annex to use
|
||||
the new protocol.
|
||||
|
||||
I think the main thing lost by this is bandwidth throttling.
|
||||
There is an open todo at [[todo/bwlimit]] to implement that in a way that
|
||||
will work more broadly than rsync's --bwlimit.
|
||||
|
||||
Maybe also --ipv4/--ipv6, but ssh configs can probably be used to
|
||||
accomplish the same thing as that.
|
||||
"""]]
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 2"""
|
||||
date="2021-09-22T19:30:15Z"
|
||||
content="""
|
||||
I've now implemented remote.name.annex-bwlimit, which works for most
|
||||
remotes, and can be used instead of the rsync options for git remotes
|
||||
that no longer use rsync.
|
||||
"""]]
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="weinzwang"
|
||||
avatar="http://cdn.libravatar.org/avatar/e73d7d9e358f3b974d283fb0834cc5d9"
|
||||
subject="comment 3"
|
||||
date="2021-09-23T21:40:50Z"
|
||||
content="""
|
||||
Awesome, thank you!
|
||||
"""]]
|
||||
|
|
@ -1,52 +0,0 @@
|
|||
The assistant's git-annex repair sometimes happens when git fsck does not
|
||||
actually detect any problems.
|
||||
|
||||
See [[Git_repos_corrupt_themselves]] of which this was part of the cause,
|
||||
although the data loss part of that was solved.
|
||||
|
||||
I saw this happen on my sister's laptop, in a freshly cloned repo,
|
||||
with a git-annex version that fixed that data loss.
|
||||
assistant was set up to fsck and on the very first fsck it started git
|
||||
repair. git fsck reported no problems at all. --[[Joey]]
|
||||
|
||||
> .git/annex/fsckresults/$uuid was empty, which means that
|
||||
> writeFsckResults was called with FsckFailed. So apparently
|
||||
> the fsck exited nonzero for some reason, but did not detect
|
||||
> any misssing shas.
|
||||
>
|
||||
> Reproed on my own laptop, with the family annex. This reproduces it about
|
||||
> 50% of the time: Clone over ssh; git-annex init;
|
||||
> git remote rm origin; git annex schedule here 'fsck self 30m every day at
|
||||
> any time'; git annex assistant; kill git-annex fsck process
|
||||
>
|
||||
> Confirmed that it's getting FsckFailed.
|
||||
>
|
||||
> Hypothesis: Maybe fsck is failing due to some other change
|
||||
> that is being made to the git repo by the assistant
|
||||
> at the same time it's running?
|
||||
> I noticed some files being downloaded from the web at the same
|
||||
> time the failed fsck was running.
|
||||
>
|
||||
> Fsck output to stdout is empty, stderr is:
|
||||
>
|
||||
> missing commit 4da14c19140e4c240358af4518d83661713ab044
|
||||
>
|
||||
> Intriguingly, that commit is present. It is a commit on the git-annex
|
||||
> branch. And fscking again succeeds. So, fsck found a reference to a
|
||||
> commit object that had not yet been written to disk. This feels like a
|
||||
> bug in git, because if it were interrupted there the repo would be left
|
||||
> in a bad state.
|
||||
>
|
||||
> Anyway, git-annex verifies that the commit is present, to double-check
|
||||
> it understood fsck correctly. And it is. So it is not considered a
|
||||
> problem. But, fsck still exits nonzero because it thinks there was a
|
||||
> problem. And that's the problem.
|
||||
>
|
||||
> Fixed by making git-annex assistant ignore fsck nonzero
|
||||
> exit status when it does not find any missing objects.
|
||||
> Since any actual failure that makes fsck do that can't
|
||||
> be distinguished from a false positive. I left git-annex repair
|
||||
> unchanged, because if the user knows the repo is badly broken and explictly
|
||||
> runs it, they would be surprised if it didn't repair.
|
||||
>
|
||||
> [[fixed|done]] --[[Joey]]
|
||||
|
|
@ -1,54 +0,0 @@
|
|||
### Please describe the problem.
|
||||
|
||||
When running the assistant, it seems to want to drop all files in the worktree from my NAS which is in the transfer group. This fails because of numcopies thankfully but it strikes me as very odd and potentially dangerous for it to do this when it knows that it can't drop these files.
|
||||
|
||||
It also unnecessarily spams the logs.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
```
|
||||
git annex assistant --stop
|
||||
git annex assistant
|
||||
```
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
```
|
||||
git-annex version: 8.20210428
|
||||
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV
|
||||
dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.28 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.4 http-client-0.6.4.1 persistent-sqlite-2.11.1.0 torrent-10000.1.1 uuid-1.3.14 yesod-1.6.1.1
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
|
||||
operating system: linux x86_64
|
||||
supported repository versions: 8
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5 6 7
|
||||
local repository version: 8
|
||||
```
|
||||
|
||||
https://github.com/Atemu/nixpkgs/tree/498831397e77a265c240cf8f8a7d15e738f2f05b
|
||||
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
The log is full of
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
drop SOTERIA path/to/file (unsafe)
|
||||
Could only verify the existence of 1 out of 2 necessary copies
|
||||
|
||||
Maybe add some of these git remotes (git remote add ...):
|
||||
d7d728f7-891a-4035-a758-c7ee80a8017a -- MB-<redacted>
|
||||
|
||||
(Use --force to override this check, or adjust numcopies.)
|
||||
failed
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
|
||||
> notabug per my comment; [[done]] --[[Joey]]
|
||||
|
|
@ -1,33 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 1"""
|
||||
date="2021-05-10T14:53:34Z"
|
||||
content="""
|
||||
Currently the assistant simply tries to drop when it thinks it's possible
|
||||
that the content may be droppable. It doesn't check if the drop is allowed
|
||||
before trying to drop. Because that would be redundant with the check
|
||||
that's done when actually dropping.
|
||||
|
||||
I don't see anything dangerous about this, same as running `git annex drop`
|
||||
on lots of files and having them be preserved by numcopies is not
|
||||
dangerous.
|
||||
|
||||
If it's a bug at all, it's only that it should be silent if
|
||||
it is unable to perform the drop due to numcopies. However,
|
||||
making it silent about that also seems like it would make it harder to
|
||||
figure out what is preventing things from being dropping in situations
|
||||
where you do expect drops to happen.
|
||||
|
||||
As in, ironically, your specific case! You have a transfer remote, which is
|
||||
having files pile up on it, despite them apparently having been transferred
|
||||
from it to both of the repos you want it to transfer them to. Since your
|
||||
local repo cannot access the other repo, it cannot verify it has the
|
||||
content and so leaves a copy on the transfer remote.
|
||||
|
||||
If you had a problem with that transfer remote filling up, and nothing was
|
||||
ever logged about why it was not dropping from it, it would be hard to
|
||||
understand what was going on and how to fix it (eg by trusting the other
|
||||
repo, or adding it as a remote, or lowering numcopies to 1).
|
||||
|
||||
So the logging seems like a good thing to me.
|
||||
"""]]
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="Atemu"
|
||||
avatar="http://cdn.libravatar.org/avatar/d1f0f4275931c552403f4c6707bead7a"
|
||||
subject="comment 2"
|
||||
date="2021-05-18T16:23:09Z"
|
||||
content="""
|
||||
That is true indeed; thanks for pointing it out!
|
||||
|
||||
Though I'd prefer if it printed a summary like: \"Warning: Couldn't perform drop of n files because the necessary copies' existence could not be verified. Make your remotes reachable by this repo or adjust numcopies.\"; that'd be a lot clearer.
|
||||
"""]]
|
||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue