Commit graph

4079 commits

Author SHA1 Message Date
Joey Hess
f64eff9355
test: Added --test-with-git-config option
Sponsored-by: Dartmouth College's DANDI project
2022-09-22 15:58:45 -04:00
Joey Hess
d049228fd3
comment and todo 2022-09-22 14:48:32 -04:00
Joey Hess
34e313f786
annex.diskreserve default increased from 1 mb to 100 mb
It's hard to know what's a good default for this. But 1 mb seems way too
small, because it's very easy for a git pull or some similar operation
that we don't think of as using much space to use up 1 mb of space.

Most people would want to free up some space if a filesystem only had 100
mb free. But on a small VPS, it's probably not uncommon to have only 1 gb
free. So 1 gb is too large for annex.diskreserve.

While old 1 gb USB keys are around, it's unlikely that anyone is
relying on them to shuttle annex data around; it would be worth anyone's
time to upgrade to a 32 gb or larger cheap modern USB key ($5).

Sponsored-by: Kevin Mueller on Patreon
2022-09-21 15:00:13 -04:00
Joey Hess
94216c99a7
comment and todo 2022-09-21 14:32:42 -04:00
Joey Hess
90da0a5e59
comment 2022-09-21 13:04:47 -04:00
yarikoptic
f3a99db14b Added a comment 2022-09-20 22:50:46 +00:00
nick.guenther@e418ed3c763dff37995c2ed5da4232a7c6cee0a9
0602d659af Added a comment 2022-09-20 21:30:51 +00:00
Joey Hess
3d51f866d3
comment 2022-09-20 14:02:09 -04:00
Joey Hess
f5a3a12360
wontfix 2022-09-20 13:54:34 -04:00
Joey Hess
0a4871cb65
comment 2022-09-16 12:39:43 -04:00
prancewit
f9417c9b20 Added a comment 2022-09-16 08:39:09 +00:00
Joey Hess
9164d9587c
general purpose design for this todo
Sponsored-by: Boyd Stephen Smith Jr. on Patreon
2022-09-15 14:26:18 -04:00
prancewit
bef6eb5d02 Added a comment 2022-09-13 21:36:38 +00:00
prancewit
9f5f960548 removed 2022-09-13 21:34:39 +00:00
prancewit
518105f89c Added a comment 2022-09-13 21:32:45 +00:00
prancewit
e0e16187a8 Added a comment: My current use case 2022-09-13 19:45:23 +00:00
Joey Hess
e13444fb2b
comments 2022-09-13 12:46:05 -04:00
prancewit
0b3da75e62 2022-09-12 19:06:11 +00:00
Joey Hess
c2184e2295
Merge branch 'master' of ssh://git-annex.branchable.com 2022-09-09 14:46:55 -04:00
Joey Hess
c62fe5e9a8
avoid redundant prompt for http password in git-annex get that does autoinit
autoEnableSpecialRemotes runs a subprocess, and if the uuid for a git
remote has not been probed yet, that will do a http get that will prompt
for a password. And then the parent process will subsequently prompt
for a password when getting annexed files from the remote.

So the solution is for autoEnableSpecialRemotes to run remoteList before
the subprocess, which will probe for the uuid for the git remote in the
same process that will later be used to get annexed files.

But, Remote.Git imports Annex.Init, and Remote.List imports Remote.Git,
so Annex.Init cannot import Remote.List. Had to pass remoteList into
functions in Annex.Init to get around this dependency loop.
2022-09-09 14:43:43 -04:00
Joey Hess
9621beabc4
cache credentials in memory when doing http basic auth to a git remote
When accessing a git remote over http needs a git credential prompt for a
password, cache it for the lifetime of the git-annex process, rather than
repeatedly prompting.

The git-lfs special remote already caches the credential when discovering
the endpoint. And presumably commands like git pull do as well, since they
may download multiple urls from a remote.

The TMVar CredentialCache is read, so two concurrent calls to
getBasicAuthFromCredential will both prompt for a credential.
There would already be two concurrent password prompts in such a case,
and existing uses of `prompt` probably avoid it. Anyway, it's no worse
than before.
2022-09-09 14:20:32 -04:00
yarikoptic
9811a02b67 Added a comment 2022-09-09 16:51:47 +00:00
Joey Hess
adb2f5cc00
comment 2022-09-09 12:39:28 -04:00
Joey Hess
28fc8d491f
comment 2022-09-09 12:30:46 -04:00
yarikoptic
f895adbb05 initial todo on multiple passwords entry for get 2022-09-06 21:02:01 +00:00
jgoerzen
254c204da2 Added a comment: mtree can help 2022-09-03 17:30:58 +00:00
yarikoptic
55252d1d4f Added a comment 2022-08-30 19:38:49 +00:00
Joey Hess
b7ff67d176
update 2022-08-30 15:22:06 -04:00
Joey Hess
e1dd689630
Merge branch 'master' of ssh://git-annex.branchable.com 2022-08-30 15:20:46 -04:00
Joey Hess
a93163d6f7
optimise linker in linux standalone tarballs
Trick the linker into not doing unncessary work searching for optimised
libraries that are not present, by symlinking the directories where
optimised libs would be to the main lib dir.

This reduces the ENOENT of git-annex init by about 1/2. The linker always
finds the files where it looks first time now. I have not looked at what
the wall clock speedup might be, it's probably rather small.

If a x86-64-v5 comes to be, the list will need to be extended. And there
may be other directories used on some machines that I have missed. Not done
for arm64 yet, or any uncommon architectures.

Sponsored-by: Dartmouth College's Datalad project
2022-08-30 15:20:04 -04:00
Joey Hess
345e60a623
comment 2022-08-30 13:36:53 -04:00
yarikoptic
04689aa74d Added a comment 2022-08-29 20:46:12 +00:00
Ilya_Shlyakhter
8df2e2dbd8 added suggestion for git-annex-import to import symlinks 2022-08-29 18:45:35 +00:00
Ilya_Shlyakhter
a1efed2a9c Added a comment: testremote cleanup 2022-08-29 18:09:24 +00:00
Joey Hess
d4fd966396
avoid dup check of guardSafeToUseRepo
Speeds up init slightly, and reduces the number of syscalls by the
dynamic linker.

Sponsored-by: Dartmouth College's Datalad project
2022-08-29 13:52:58 -04:00
Joey Hess
f184dca7de
comment 2022-08-29 13:38:42 -04:00
Joey Hess
f8524d6ecb
comment 2022-08-29 12:45:58 -04:00
Ilya_Shlyakhter
86e0e8fce1 Added a comment: testremote cleanup 2022-08-28 20:15:39 +00:00
yarikoptic
ab8ffb406c initial report on libpcre lookups 2022-08-27 00:53:22 +00:00
yarikoptic
3ff0d9ac66 Added a comment 2022-08-24 19:27:00 +00:00
Joey Hess
5e45befa1b
comment 2022-08-24 15:17:43 -04:00
yarikoptic
9750dd5e21 Added a comment 2022-08-24 18:40:10 +00:00
yarikoptic
8e5b22af2f Added a comment 2022-08-24 18:35:25 +00:00
yarikoptic
26c152bae1 Added a comment 2022-08-24 18:32:53 +00:00
Joey Hess
33083a952a
comment 2022-08-24 13:28:43 -04:00
Joey Hess
ef037b9028
comment 2022-08-24 13:26:42 -04:00
Joey Hess
a124a53fa2
comment 2022-08-24 13:17:56 -04:00
Joey Hess
ea445260ba
comment 2022-08-24 13:13:45 -04:00
yarikoptic
cc64741410 Added a comment 2022-08-24 15:42:10 +00:00
yarikoptic
4344943d64 added "reproducer" 2022-08-24 15:25:33 +00:00
yarikoptic
749593af7d request add --batch to be able to handle directories 2022-08-24 15:12:19 +00:00
Ilya_Shlyakhter
ba5c8cba44 added suggestion to add clean-up after a testremote failure 2022-08-24 14:52:36 +00:00
Joey Hess
28921af543
remove old closed bugs and todo items to speed up wiki updates and reduce size
Remove closed bugs and todos that were last edited or commented before 2021.

Except for ones tagged projects/* since projects like datalad want to keep
around records of old deleted bugs longer.

Command line used:

    for f in $(grep -l '|done\]\]' -- ./*.mdwn); do if ! grep -q "projects/" "$f"; then d="$(echo "$f" | sed 's/.mdwn$//')"; if [ -z "$(git log --since=01-01-2021 --pretty=oneline -- "$f")" -a -z "$(git log --since=01-01-2021 --pretty=oneline -- "$d")" ]; then git rm -- "./$f" ; git rm -rf "./$d"; fi; fi; done
    for f in $(grep -l '\[\[done\]\]' -- ./*.mdwn); do if ! grep -q "projects/" "$f"; then d="$(echo "$f" | sed 's/.mdwn$//')"; if [ -z "$(git log --since=01-01-2021 --pretty=oneline -- "$f")" -a -z "$(git log --since=01-01-2021 --pretty=oneline -- "$d")" ]; then git rm -- "./$f" ; git rm -rf "./$d"; fi; fi; done
2022-08-22 12:27:10 -04:00
Yaroslav Halchenko
0151976676
Typo fix unncessary -> unnecessary.
Detected while reading recent CHANGELOG entry but then decided to apply
to entire codebase and docs since why not?
2022-08-20 09:40:19 -04:00
Joey Hess
cd9fd6e28c
fix case of Win32 2022-08-04 12:17:27 -04:00
jkniiv
2930afded3 Added a comment: typo in Utility/LockFile/Windows.hs 2022-08-03 20:46:14 +00:00
Joey Hess
f185140d3e
Merge branch 'master' of ssh://git-annex.branchable.com 2022-08-03 11:16:50 -04:00
Joey Hess
3a513cfe73
add --dry-run: New option
This is intended for users who want to see what it would output in order to
eg, check if a file would be added to git or the annex. It is not intended
as a way for scripts to get information.

Sponsored-by: Dartmouth College's Datalad project
2022-08-03 11:16:04 -04:00
jkniiv
e5a83d29bc Added a comment 2022-08-03 15:14:23 +00:00
Joey Hess
54b3bfed4b
comment 2022-08-03 10:21:08 -04:00
Joey Hess
c6ce2150f4
Merge branch 'master' of ssh://git-annex.branchable.com 2022-08-03 09:53:56 -04:00
yarikoptic
d297e9ba78 initial report on wanting --dry-run 2022-08-02 21:06:06 +00:00
Joey Hess
472f5c142b
Use createFile_NoRetry from win32 2.13.3.1
Sponsored-by: Tobias Ammann on Patreon
2022-08-02 10:45:39 -04:00
Joey Hess
f6f46eb9eb
comment 2022-08-01 12:45:41 -04:00
Joey Hess
89095abc26
comment 2022-08-01 12:38:39 -04:00
Joey Hess
0c0724e9a8
response 2022-08-01 12:36:44 -04:00
Joey Hess
08be64b6f6
comment 2022-07-29 12:30:33 -04:00
Joey Hess
be19a68276
new matching options --want-get-by and --want-drop-by
Sponsored-by: Graham Spencer on Patreon
2022-07-28 13:26:03 -04:00
kdm9
11f7e68e16 Added a comment: Bump for --want-get/drop-by 2022-07-26 18:19:37 +00:00
Joey Hess
d4b2c4a3fe
comment and fix my incorrect earlier comment 2022-07-25 16:30:22 -04:00
yarikoptic
dfa8298f46 Added a comment 2022-07-25 19:03:42 +00:00
Joey Hess
7f77405f60
comment 2022-07-25 12:50:56 -04:00
Joey Hess
7e4ee80e9a
comment 2022-07-25 12:35:58 -04:00
yarikoptic
b40ac7af3d Added a comment 2022-07-24 16:52:33 +00:00
Joey Hess
2de61f2e5a
comment 2022-07-22 11:38:33 -04:00
jkniiv
3d105f44bb Added a comment 2022-07-21 20:08:55 +00:00
yarikoptic
152f82796a Added a comment 2022-07-21 14:03:56 +00:00
Joey Hess
27e0108097
comment 2022-07-20 13:51:26 -04:00
Joey Hess
03ce01dcf9
comment 2022-07-20 13:44:24 -04:00
Joey Hess
8b4f7af605
Merge branch 'master' of ssh://git-annex.branchable.com 2022-07-20 13:34:37 -04:00
Joey Hess
a0746d2027
fixed 2022-07-20 13:32:26 -04:00
Joey Hess
05b96a1acf
Merge branch 'append' 2022-07-20 13:24:04 -04:00
Joey Hess
4e88137a28
prevent appends except when annex.alwayscompact=false
I would like for a new repo version to enable appends, but to do so
safely would need a v11 followed by a 1 year delay followed by a v12
that does it. Since a similar v9 and v10 transition is currently
happening, and is less than 6 months along in most repos, it does not
feel wise to stack up another year-long transition behind that. What if
I need to hurry up a new repo version for some other change?

Added todo so I remember to make this change at some time when a v11
and probably v12 repo version do make sense.

Sponsored-by: Dartmouth College's DANDI project
2022-07-20 13:23:55 -04:00
Joey Hess
c933b0074f
comment 2022-07-19 18:10:45 -04:00
yarikoptic
78642ededb Added a comment 2022-07-19 21:29:02 +00:00
Joey Hess
d832a7c211
thoughts 2022-07-19 15:45:08 -04:00
daven.quinn@d0ed4e0e5e4462d9a74a5d5a8fbd1b17f85db13e
0568fc872d 2022-07-19 07:57:37 +00:00
Joey Hess
fd6e01d9b6
comment 2022-07-18 17:02:41 -04:00
Joey Hess
2cb634c373
comment 2022-07-18 16:56:31 -04:00
yarikoptic
cab61b88e0 Added a comment 2022-07-18 19:23:02 +00:00
Joey Hess
b04435ea27
Merge branch 'master' of ssh://git-annex.branchable.com 2022-07-18 14:45:23 -04:00
Joey Hess
efee53f433
comments 2022-07-18 14:45:03 -04:00
Ilya_Shlyakhter
1155bbb3db added suggestion to record ETags in URL- keys 2022-07-18 16:43:28 +00:00
yarikoptic
eee1169ad5 Added a comment 2022-07-15 19:33:50 +00:00
Joey Hess
8bc9381d8d
design work 2022-07-15 15:06:40 -04:00
Joey Hess
7c8c5ffe8e
Merge branch 'master' of ssh://git-annex.branchable.com 2022-07-15 11:10:44 -04:00
Joey Hess
f561602484
comment 2022-07-15 11:10:40 -04:00
jkniiv
c2cdf0f61f Added a comment 2022-07-14 20:32:06 +00:00
Joey Hess
94b50c61b3
comment 2022-07-14 16:09:48 -04:00
yarikoptic
f2c30bcb07 Added a comment 2022-07-14 19:42:58 +00:00
Joey Hess
2e57da226c
comments 2022-07-14 15:08:01 -04:00
Joey Hess
78da3e2783
close 2022-07-14 13:53:32 -04:00
Joey Hess
557542d621
comment 2022-07-14 13:51:59 -04:00
yarikoptic
06981c6c5a Added a comment 2022-07-14 17:00:40 +00:00
yarikoptic
3c948423a9 Added a comment 2022-07-14 16:50:15 +00:00
Joey Hess
ad467791c1
optimise journal writes to not mkdir journal directory when it already exists
Sponsored-by: Dartmouth College's DANDI project
2022-07-14 12:29:39 -04:00
Joey Hess
5e407304a2
comment with a question 2022-07-14 12:13:28 -04:00
yarikoptic
c4cca7e6c6 initial request for more efficient registerurl 2022-07-14 13:40:16 +00:00
Joey Hess
6c7550ba62
comment 2022-07-13 17:27:30 -04:00
yarikoptic
5764450a80 Added a comment 2022-07-13 20:05:34 +00:00
yarikoptic
fa126d1ac9 Added a comment 2022-07-13 19:04:40 +00:00
Joey Hess
7c7b7ac9b9
followup 2022-07-13 14:53:46 -04:00
Joey Hess
68e9b7f987
comment 2022-07-13 13:44:43 -04:00
yarikoptic
2d71b83e7f initial complain/call for a more efficient journal? 2022-07-13 16:19:55 +00:00
Joey Hess
323c78b191
comment
(and fix an old comment's formatting)
2022-07-12 16:47:49 -04:00
Joey Hess
ba13c1e2ac
depend on version of Win32 that exports c_createFile 2022-07-12 16:28:01 -04:00
jkniiv
125feb6da0 I should use the correct term 2022-07-09 11:31:11 +00:00
jkniiv
70357a9bac corr. typo with meta directive 2022-07-09 08:27:20 +00:00
jkniiv
92efe0b8e2 not to be the bringer of bad news re: Windows, yet here we are 2022-07-09 08:23:18 +00:00
yarikoptic
22453fca7c request for anonymous access to S3 2022-07-07 19:50:15 +00:00
Joey Hess
8040ecf9b8
final readonly values moves to AnnexRead
At this point I've checked all AnnexState values and these were all that
remained that could move.

Pity that Annex.repo can't move, but it gets modified sometimes..

A couple of AnnexState values are set by options and could be AnnexRead,
but happen to use Annex when being set.

Sponsored-by: Max Thoursie on Patreon
2022-06-28 16:04:58 -04:00
Joey Hess
cb9cf30c48
move several readonly values to AnnexRead
This improves performance to a small extent in several places.

Sponsored-by: Tobias Ammann on Patreon
2022-06-28 15:40:19 -04:00
Joey Hess
655235319b
comment 2022-06-28 13:04:41 -04:00
Joey Hess
53a4de4968
close 2022-06-28 12:58:00 -04:00
Joey Hess
a153480667
comments 2022-06-28 12:22:32 -04:00
Atemu
4517e9e41c Added a comment: Still useful 2022-06-25 16:36:38 +00:00
Joey Hess
e5c418b99c
convert bug to todo 2022-06-22 14:23:46 -04:00
Atemu
c6b1121088 Added a comment 2022-06-22 12:34:45 +00:00
Joey Hess
49ab467784
comment 2022-06-13 14:20:54 -04:00
Joey Hess
4f23931439
add todo based on forum post 2022-06-13 13:12:18 -04:00
Atemu
c0e648e921 2022-06-11 14:58:11 +00:00
aurelia@b44312a63326710de6cea9c43290e5debbd55607
4150433ef4 Added a comment 2022-06-06 12:38:26 +00:00
Joey Hess
1cacfd1b19
avoid setEnv in test framework when tasty is running
setEnv is not thread safe and could cause a getEnv by another thread to
segfault, or perhaps other had behavior. This is particularly a problem
when using tasty, because tasty runs the test in a thread, and a getEnv
in another thread.

The use of top-level TMVars is ugly, but ok because only 1 test actually
runs at a time per process. Because it has to chdir into the test repo.

The setEnv that remains happens before tasty is running.

Sponsored-by: Dartmouth College's Datalad project
2022-05-18 16:45:20 -04:00
Joey Hess
ebb76f0486
avoid setEnv while testing gpg
setEnv is not thread safe and could cause a getEnv by another thread to
segfault, or perhaps other had behavior.

Sponsored-by: Dartmouth College's Datalad project
2022-05-18 16:05:11 -04:00
Joey Hess
4ba7a97d8c
add comment I made earlier 2022-05-18 14:57:27 -04:00
Joey Hess
6bf09a38ed
comment and open a todo 2022-05-18 14:57:14 -04:00
Joey Hess
12119d8bfe
Merge branch 'master' of ssh://git-annex.branchable.com 2022-05-16 15:38:22 -04:00
Joey Hess
aa414d97c9
make fsck normalize object locations
The purpose of this is to fix situations where the annex object file is
stored in a directory structure other than where annex symlinks point to.

But it will also move object files from the hashdirmixed back to
hashdirlower if the repo configuration makes that the normal location.
It would have been more work to avoid that than to let it do it.

Sponsored-by: Dartmouth College's Datalad project
2022-05-16 15:38:06 -04:00
Joey Hess
5a98f2d509
avoid creating content directory when locking content
If the content directory does not exist, then it does not make sense to
lock the content file, as it also does not exist, and so it's ok for the
lock operation to fail.

This avoids potential races where the content file exists but is then
deleted/renamed, while another process sees that it exists and goes to
lock it, resulting in a dangling lock file in an otherwise empty object
directory.

Also renamed modifyContent to modifyContentDir since it is not only
necessarily used for modifying content files, but also other files in
the content directory.

Sponsored-by: Dartmouth College's Datalad project
2022-05-16 12:34:56 -04:00
yarikoptic
5079891302 Added a comment: shell helper 2022-05-13 16:51:23 +00:00
yarikoptic
e37dddbacb Added a comment 2022-05-13 15:12:29 +00:00
Joey Hess
b6c7819803
thoughts 2022-05-10 14:17:29 -04:00
Joey Hess
c5b5fd364a
thoughts 2022-05-10 14:14:41 -04:00
Joey Hess
997e96ef5e
thoughts 2022-05-10 13:25:38 -04:00
Joey Hess
3b83224e57
followup 2022-05-09 16:10:31 -04:00
Joey Hess
136ad17b94
Merge branch 'master' of ssh://git-annex.branchable.com 2022-05-09 15:54:07 -04:00
Joey Hess
e8a601aa24
incremental verification for retrieval from import remotes
Sponsored-by: Dartmouth College's Datalad project
2022-05-09 15:39:43 -04:00
yarikoptic
dfc3bf4daa Added a comment: hm... 2022-05-09 19:10:10 +00:00
Joey Hess
2f2701137d
incremental verification for retrieval from all export remotes
Only for export remotes so far, not export/import.

Sponsored-by: Dartmouth College's Datalad project
2022-05-09 13:49:33 -04:00
Joey Hess
52b768db45
datalad metadata 2022-05-09 11:32:29 -04:00