Commit graph

30940 commits

Author SHA1 Message Date
Joey Hess
258647ce7d
improve docs
I saw a user open a forum post that looked like they had read this man
page, but were unaware of git-annex unlock, and were looking for its
functionality. They later deleted the post, probably when they found
git-annex unlock. Looking at this man page, it was a bit unclear about
the motivation for the command.

Yes, I'm warching... ;-)
2021-11-12 14:08:36 -04:00
Joey Hess
2248b35fb8
Merge branch 'master' of ssh://git-annex.branchable.com 2021-11-12 13:29:22 -04:00
Joey Hess
51b73ea1fc
migrate: New --remove-size option
While intended for converting URL keys added by addurl --fast to be
as if added by addurl --relaxed, it can also be used to remove size
from other types of keys. Although that is not likely to be useful
for checksummed keys, I suppose it could be used for WORM or other
non-checksum keys.

Specifying the --remove-size option does not prevent other migrations
from taking effect if there's a key upgrade to perform, or if the
backend has changed. So --backend=URL needs to be used to prevent
migrating an URL key to the default backend.

Note that it's not possible to use git-annex migrate to convert from a
non-URL key to an URL key, as URL keys cannot be generated, except by
addurl. So while this can get the same effect as --relaxed would have
when addurl --fast was used, when --fast was not used, it won't work, or
if --backend=URL is not used will remove the size but not prevent
checksum verification, which is not useful. Due to this complexity, I
decided not to mention it in the git-annex addurl man page.

Sponsored-by: Jochen Bartl on Patreon
2021-11-12 13:28:28 -04:00
iimog@2c22a44141070c04b943932b697818a686859677
71f87dabd2 Added a comment: It works 2021-11-12 08:14:33 +00:00
Joey Hess
f3326b8b5a
git-lfs gitlab interoperability fix
git-lfs: Fix interoperability with gitlab's implementation of the git-lfs
protocol, which requests Content-Encoding chunked.

Sponsored-by: Dartmouth College's Datalad project
2021-11-10 13:51:11 -04:00
Joey Hess
dee462f536
tag as datalad 2021-11-09 16:06:41 -04:00
Joey Hess
3bdac41090
comment 2021-11-09 16:04:54 -04:00
Joey Hess
886d60dabe
Merge branch 'master' of ssh://git-annex.branchable.com 2021-11-09 15:53:13 -04:00
Joey Hess
9121154a75
new todo 2021-11-09 15:52:17 -04:00
Ilya_Shlyakhter
61a6bdc386 Added a comment: clarification re: smudge filter and annex.supportunlocked 2021-11-09 18:28:10 +00:00
iimog@2c22a44141070c04b943932b697818a686859677
173d8d0e51 2021-11-09 15:30:42 +00:00
jkniiv
a741d3308f Added a comment 2021-11-09 08:48:33 +00:00
fireboy
95c37fb262 removed 2021-11-09 04:47:41 +00:00
fireboy
6682b04a27 2021-11-09 01:48:01 +00:00
Joey Hess
9d3ce224e3
uninit edge cases
* uninit: Avoid error message when no commits have been made to the
  repository yet.
* uninit: Avoid error message when there is no git-annex branch.

Sponsored-by: Svenne Krap on Patreon
2021-11-08 16:47:00 -04:00
Joey Hess
9cb02ca29e
comment 2021-11-08 16:28:43 -04:00
Joey Hess
020148bd06
comment 2021-11-08 16:23:05 -04:00
Joey Hess
f410f9d7ca
Merge branch 'master' of ssh://git-annex.branchable.com 2021-11-08 16:20:57 -04:00
Joey Hess
7e3180226d
devblog 2021-11-08 16:20:43 -04:00
Joey Hess
a0758bdd10
dynamically disable filter-process in restagePointerFile when it would be slower
Based on my earlier benchmark, I have a rough cost model for how
expensive it is for git-annex smudge to be run on a file, vs
how expensive it is for a gigabyte of a file's content to be read and
piped through to filter-process.

So, using that cost model, it can decide if using filter-process will
be more or less expensive than running the smudge filter on the files to
be restaged.

It turned out to be *really* annoying to temporarily disable
filter-process. I did find a way, but urk, this is horrible. Notice
that, if it's interrupted with it disabled, it will remain disabled
until the next time restagePointerFile runs. Which could be some time
later. If the user runs `git add` or `git checkout` on a lot of small
files before that, they will see slower than expected performance.

(This commit also deletes where I wrote down the benchmark results
earlier.)

Sponsored-by: Noam Kremen on Patreon
2021-11-08 16:20:34 -04:00
Lukey
f32cb96a34 Added a comment 2021-11-08 19:38:35 +00:00
Ilya_Shlyakhter
54b82e4fef Added a comment 2021-11-08 16:34:30 +00:00
xeruf
d632c781c2 Suggest description 2021-11-07 20:50:03 +00:00
xeruf
fdd3321e81 2021-11-07 19:25:09 +00:00
Joey Hess
054c803f8d
benchmarking of filter-process vs smudge/clean
No firm conclusions yet, but it's doing better than I would have
expected.

Sponsored-by: Graham Spencer on Patreon
2021-11-05 13:37:53 -04:00
Joey Hess
099e8fe061
close 2021-11-05 12:46:56 -04:00
Joey Hess
7551c7ab54
Merge branch 'master' of ssh://git-annex.branchable.com 2021-11-05 12:46:14 -04:00
Joey Hess
cdf7639954
update 2021-11-05 11:00:45 -04:00
jkniiv
d5a06d51c3 Added a comment 2021-11-05 10:25:32 +00:00
jkniiv
beabf83ffb restated: plenty of unit tests fail both under Windows and Ubuntu 18.04 2021-11-05 06:02:29 +00:00
jkniiv
2407f35eb8 plenty of unit tests fail on Windows 2021-11-05 03:34:16 +00:00
Joey Hess
afca7d66f6
Merge branch 'master' of ssh://git-annex.branchable.com 2021-11-04 15:24:46 -04:00
Joey Hess
b25a138e22
update for git-annex filter-process 2021-11-04 15:15:26 -04:00
Joey Hess
8dd91be867
mention filter-process as v9 material 2021-11-04 15:05:24 -04:00
Joey Hess
d392d8dec8
update with actual command to run 2021-11-04 15:03:51 -04:00
Joey Hess
916c5a7619
Merge branch 'master' into long-running-smudge 2021-11-04 15:03:28 -04:00
Joey Hess
68257e9076
add git-annex filter-process
filter-process: New command that can make git add/checkout faster when
there are a lot of unlocked annexed files or non-annexed files, but that
also makes git add of large annexed files slower.

Use it by running: git
config filter.annex.process 'git-annex filter-process'

Fully tested and working, but I have not benchmarked it at all.
And, incremental hashing is not done when git add uses it, so extra work is
done in that case.

Sponsored-by: Mark Reidenbach on Patreon
2021-11-04 15:02:36 -04:00
CandyAngel
06f7ad2045 removed 2021-11-04 16:06:17 +00:00
CandyAngel
4515eb2a58 Added a comment 2021-11-04 16:04:07 +00:00
Joey Hess
e0b4a66a54
Merge branch 'master' of ssh://git-annex.branchable.com 2021-11-03 16:06:49 -04:00
Joey Hess
efe0554f22
devblog 2021-11-03 16:06:32 -04:00
Joey Hess
bf1408f7bf
long-running-smudge branch started 2021-11-03 15:44:05 -04:00
bjornw
c853992e2f 2021-11-02 23:12:33 +00:00
bjornw
d8b5c8479f 2021-11-02 23:10:14 +00:00
Joey Hess
ac05422703
comment 2021-11-02 15:24:22 -04:00
Joey Hess
be690a0717
comment 2021-11-02 15:16:15 -04:00
Joey Hess
38ba8cca1b
investigation results
Also, close dup bug.
2021-11-02 15:06:20 -04:00
Joey Hess
438e5b56aa
tighter --json parsing for metadata
metadata --batch --json: Reject input whose "fields" does not consist of
arrays of strings. Such invalid input used to be silently ignored.

Used to be that parseJSON for a JSONActionItem ran parseJSON separately
for the itemAdded, and if that failed, did not propagate the error. That
allowed different items with differently named fields to be parsed.
But it was actually only used to parse "fields" for metadata, so that
flexability is not needed.

The fix is just to parse "fields" as-is. AddJSONActionItemFields is needed
only because of the wonky way Command.MetaData adds onto the started json
object.

Note that this line got a dummy type signature added,
just because the type checker needs it to be some type.
itemFields = Nothing :: Maybe Bool
Since it's Nothing, it doesn't really matter what type it is,
and the value gets turned into json and is then thrown away.

Sponsored-by: Kevin Mueller on Patreon
2021-11-01 14:42:37 -04:00
Joey Hess
70c06b4434
comment 2021-11-01 13:42:48 -04:00
Joey Hess
77b932b0f1
Merge branch 'master' of ssh://git-annex.branchable.com 2021-11-01 13:41:19 -04:00