Commit graph

26054 commits

Author SHA1 Message Date
Joey Hess
157f41427f
bug 2019-06-12 15:00:28 -04:00
Joey Hess
e589a9b3fc
moving this to a bug 2019-06-12 15:00:14 -04:00
Joey Hess
7b5aad2452
Merge commit '6f8322b8f72f3399d4c28426749db5d01742001d' into starting 2019-06-12 14:50:59 -04:00
Joey Hess
6f8322b8f7
close; not a bug 2019-06-12 14:45:25 -04:00
Joey Hess
8e5ea28c26
finish CommandStart transition
The hoped for optimisation of CommandStart with -J did not materialize.
In fact, not runnign CommandStart in parallel is slower than -J3.
So, CommandStart are still run in parallel.

(The actual bad performance I've been seeing with -J in my big repo
has to do with building the remoteList.)

But, this is still progress toward making -J faster, because it gets rid
of the onlyActionOn roadblock in the way of making CommandCleanup jobs
run separate from CommandPerform jobs.

Added OnlyActionOn constructor for ActionItem which fixes the
onlyActionOn breakage in the last commit.

Made CustomOutput include an ActionItem, so even things using it can
specify OnlyActionOn.

In Command.Move and Command.Sync, there were CommandStarts that used
includeCommandAction, so output messages, which is no longer allowed.
Fixed by using startingCustomOutput, but that's still not quite right,
since it prevents message display for the includeCommandAction run
inside it too.
2019-06-12 13:24:01 -04:00
Ilya_Shlyakhter
3820829407 re: status of v7 2019-06-12 15:53:12 +00:00
Ilya_Shlyakhter
d2658d9537 an issue involving repos cloned with --single-branch 2019-06-11 23:30:26 +00:00
anthony@ad39673d230d75cbfd19d2757d754030049c7673
fd7f316482 Added a comment 2019-06-10 16:52:23 +00:00
anthony@ad39673d230d75cbfd19d2757d754030049c7673
d5a0bf3ae9 clarify it's the new android installer 2019-06-10 16:38:44 +00:00
anthony@ad39673d230d75cbfd19d2757d754030049c7673
856affe859 initial report 2019-06-10 16:37:31 +00:00
kirelagin@6d93475882c55a329fedae6be1971868a775ec7e
c0f4788cdb Added a comment: Workaround? 2019-06-08 13:03:50 +00:00
Joey Hess
8d573a653b
Merge branch 'master' of ssh://git-annex.branchable.com 2019-06-07 19:34:46 -04:00
Joey Hess
1d92846e54
bug report from MacGyver.mdwn 2019-06-07 19:34:21 -04:00
jamie@b5676b90eec0401ca8faac7c972eaf5676891601
0424549599 removed 2019-06-07 15:19:15 +00:00
Joey Hess
132ec9e005
devblog 2019-06-06 17:16:30 -04:00
Joey Hess
3893d84764
todo 2019-06-06 12:02:27 -04:00
Joey Hess
6cefe071b1
Merge branch 'master' of ssh://git-annex.branchable.com 2019-06-05 20:19:27 -04:00
Joey Hess
4088ce49ce
devblog 2019-06-05 20:18:59 -04:00
Joey Hess
3eac4e01a4
idea 2019-06-05 19:43:01 -04:00
Joey Hess
659640e224
separate queue for cleanup actions
When running multiple concurrent actions, the cleanup phase is run in a
separate queue than the main action queue. This can make some commands
faster, because less time is spent on bookkeeping in between each file
transfer.

But as far as I can see, nothing will be sped up much by this yet, because
all the existing cleanup actions are very light-weight. This is just groundwork
for deferring checksum verification to cleanup time.

This change does mean that if the user expects -J2 will mean that they see no
more than 2 jobs running at a time, they may be surprised to see 4 in some
cases (if the cleanup actions are slow enough to notice).

It might also make sense to enable background cleanup without the -J,
for at least one cleanup action. Indeed, that's the behavior that -J1
has now. At some point in the future, it make make sense to make the
behavior with no -J the same as -J1. The only reason it's not currently
is that git-annex can build w/o concurrent-output, and also any bugs
in concurrent-output (such as perhaps misbehaving on non-VT100 compatible
terminals) are avoided by default by only using it when -J is used.
2019-06-05 17:54:35 -04:00
kyle
b390256bd1 Added a comment: uuid.log is also not created 2019-06-05 15:18:30 +00:00
kyle
0d60b2f2c5 2019-06-05 13:57:02 +00:00
Joey Hess
6e51b9ae88
clarify 2019-06-04 21:49:53 -04:00
Joey Hess
082e1f1738
Don't try to import .git directories from special remotes
Because git does not support storing git repositories inside a git
repository.
2019-06-04 15:14:20 -04:00
Joey Hess
500f72ec3d
comment typo 2019-06-04 14:40:07 -04:00
Joey Hess
7dcc815c29
more thoughts 2019-06-04 14:38:55 -04:00
Joey Hess
cd20dc4158
thoughts 2019-06-04 14:13:15 -04:00
Joey Hess
418e842d93
Merge branch 'master' of ssh://git-annex.branchable.com 2019-06-02 15:38:37 -04:00
Joey Hess
f0cba96a92
update 2019-06-02 15:26:50 -04:00
git-annex.branchable@794aec3ca2af16d925f13a04f7e00a6dfe523186
7a5b534ea8 2019-06-02 10:16:58 +00:00
Joey Hess
fa2c8d8fd9
Merge branch 'master' of ssh://git-annex.branchable.com 2019-06-01 09:41:59 -04:00
Joey Hess
f05875f00b
bug 2019-06-01 09:41:39 -04:00
Ilya_Shlyakhter
196548a1b2 a place for pull requests 2019-05-31 17:05:23 +00:00
joer962500
22f1b6673e Added a comment: asus tablet stuck loading screen 2019-05-31 07:24:27 +00:00
Joey Hess
5d681a3ce6
Merge branch 'master' of ssh://git-annex.branchable.com 2019-05-30 16:05:41 -04:00
Joey Hess
6136e299a2
add back support for following http to ftp redirects
Did not test build with http-client < 0.5 and while I tried to support
it, the ifdefed parts may needs some fixes.
2019-05-30 16:04:59 -04:00
Joey Hess
67c06f5121
add back support for ftp urls
Add back support for ftp urls, which was disabled as part of the fix for
security hole CVE-2018-10857 (except for configurations which enabled curl
and bypassed public IP address restrictions). Now it will work if allowed
by annex.security.allowed-ip-addresses.
2019-05-30 14:51:34 -04:00
Joey Hess
1871295765
rename annex.security.allowed-http-addresses
Renamed annex.security.allowed-http-addresses to
annex.security.allowed-ip-addresses because it is not really specific to
the http protocol, also limiting eg, git-annex's use of ftp and via
youtube-dl, several other protocols.

The old name for the config will still work.

If both old and new name are set, the new name will win.
2019-05-30 12:43:40 -04:00
Joey Hess
135404b33e
update 2019-05-30 12:28:05 -04:00
Joey Hess
af5246cf04
thoughts 2019-05-30 12:00:45 -04:00
emanuele.olivetti@47d88ed185b03191e25329caa6fabc2efb3118b2
23daba2e99 Added a comment 2019-05-29 17:36:06 +00:00
Joey Hess
b993d26dff
plan 2019-05-29 10:29:20 -04:00
Joey Hess
b114a90fae
comment 2019-05-29 09:59:34 -04:00
Joey Hess
9831d366a0
Merge branch 'master' of ssh://git-annex.branchable.com 2019-05-29 09:24:52 -04:00
Joey Hess
1dde73dd98
comment 2019-05-29 09:24:25 -04:00
emanuele.olivetti@47d88ed185b03191e25329caa6fabc2efb3118b2
a3bef03fae Added a comment: Misconfiguration in GHC 2019-05-28 20:38:07 +00:00
Joey Hess
39ab026bab
devblog 2019-05-28 16:25:11 -04:00
Joey Hess
b6dfcf529e
indent 2019-05-28 16:21:35 -04:00
Joey Hess
a5c5ce5e48
Merge branch 'master' of ssh://git-annex.branchable.com 2019-05-28 16:17:00 -04:00
Joey Hess
08aa8e2010
break out export and import appending
The import protocol is WiP
2019-05-28 16:16:17 -04:00
Joey Hess
4818e3606c
reorg special remote docs
Made responses to git-annex requests be listed under each request.

This did lead to a little duplication since some replies are used for 2
requests, but it also makes it much clearer and easier to see how the
protocol works.

And, it makes each request self-contained, so they can be split out into
separate pages.
2019-05-28 14:01:19 -04:00
Ilya_Shlyakhter
afb1be754a Added a comment: readonly special remotes] 2019-05-28 17:25:19 +00:00
emanuele.olivetti@47d88ed185b03191e25329caa6fabc2efb3118b2
89abda9efd Added a comment: git-annex not working on arm at all? 2019-05-28 17:23:57 +00:00
root@99e0a30209b856a064fa7d3e42d26b562bdf991c
a03a9cedf1 2019-05-28 16:53:35 +00:00
Joey Hess
63fa6b4a44
fix wrong statement 2019-05-28 12:05:49 -04:00
Joey Hess
bb635fffa2
reorg page 2019-05-28 11:53:49 -04:00
Joey Hess
c1ed0293b0
improve docs about removeExportDirectory 2019-05-28 11:16:01 -04:00
Joey Hess
8960f259b8
make readonly export remotes really be readonly
When a remote is configured to be readonly, don't allow changing what's
exported to it.

This was missed in the original export remote implementation, but it makes
sense for a readonly export remote to not be allowed to change.
2019-05-28 11:04:28 -04:00
Joey Hess
4c48efeb35
comment 2019-05-28 10:41:25 -04:00
Joey Hess
ca5ee26858
Merge branch 'master' of ssh://git-annex.branchable.com 2019-05-28 10:28:30 -04:00
ndj
4cb8eec696 Added a comment 2019-05-28 03:36:53 +00:00
emanuele.olivetti@47d88ed185b03191e25329caa6fabc2efb3118b2
182943ed60 2019-05-26 11:05:38 +00:00
Ilya_Shlyakhter
bf3c032a31 strange interaction of checkpresentkey and annex-readonly=true 2019-05-24 03:05:32 +00:00
emmet.obrien@e4a3124a9cf70a33bd02e6cab55d0db0352ce9e1
82dfd2d9e7 2019-05-23 20:39:29 +00:00
yarikoptic
8fe962dba5 Added a comment: configuration variable 2019-05-23 19:31:46 +00:00
Joey Hess
2353d80906
remove done item 2019-05-23 14:24:25 -04:00
Joey Hess
e678d0d0ed
Merge branch 'master' of ssh://git-annex.branchable.com 2019-05-23 14:15:04 -04:00
Joey Hess
c877af6192
comment 2019-05-23 14:12:59 -04:00
Joey Hess
87c70c278d
comment 2019-05-23 14:00:20 -04:00
Joey Hess
1a315a8704
comment 2019-05-23 13:37:47 -04:00
Joey Hess
cd5e8be2dc
comment 2019-05-23 13:36:59 -04:00
Joey Hess
c0907909c8
comment 2019-05-23 13:33:53 -04:00
Ilya_Shlyakhter
8ba2109395 Added a comment 2019-05-23 17:07:40 +00:00
Joey Hess
f2a54e3401
Android: Improve installation process when the user's login shell is not bash.
~/.profile works for bash, but not all other login shells.

This setting PATH is a minor convenience for users, particuarly since
typing on android is so much harder. The usual linux standalone bundle
just expects the user to know how to add it to PATH. I don't want this
code to grow special cases for every possible login shell. So displaying a
message to the presumably minority who don't use bash seems like the best
choice.

Longer term, I'd hope termux gets some way to set an environment variable
for all login shells. Systems using PAM can, via ~/.pam_environment. Or
alternatively, add a git-annex package to termux, even if just an installer
package. I'd rather spend time on either of those than on making this minor
thing support more login shells.

This commit was sponsored by mo on Patreon.
2019-05-23 13:06:31 -04:00
Joey Hess
a14f6ce758
fix repo description setting bugs
* init: When the repository already has a description, don't change it.
* describe: When run with no description parameter it used to set
  the description to "", now it will error out.
2019-05-23 12:51:01 -04:00
Joey Hess
d5582865b6
comment 2019-05-23 12:25:51 -04:00
Joey Hess
1e00f861ab
comment 2019-05-23 12:12:44 -04:00
andrew
66b1f2bd15 Added a comment 2019-05-21 23:33:57 +00:00
Joey Hess
52df1b2015
devblog 2019-05-21 15:05:57 -04:00
Joey Hess
d98dc13345
update for preferred content 2019-05-21 14:56:45 -04:00
Joey Hess
e06feb7316
honor preferred content when importing
Importing from a special remote honors its preferred content too; unwanted
files are not imported. But, some preferred content expressions can't be
checked before files are imported, and trying to import with such an
expression will fail.

Tested this with scenarios including changing the preferred content
expression and making sure merging the import didn't delete files that were
no longer wanted.

There was one minor inefficiency mentioned in the todo that I punted on.
2019-05-21 14:38:06 -04:00
Joey Hess
ec11575d17
hairyness 2019-05-21 12:54:57 -04:00
Joey Hess
3b9a19171a
Merge branch 'master' into preferred 2019-05-21 11:34:45 -04:00
Joey Hess
5e1221ad53
Improve shape of commit tree when importing from unversioned special remotes
Make the import have the previous import as a parent, so eg `git log --stat`
displays a useful diff.

Also a minor optimisation, only calculate the depth of the imported history
once.
2019-05-21 11:32:54 -04:00
Joey Hess
5af9e7f3d0
break out a todo 2019-05-21 11:10:13 -04:00
Joey Hess
e51ca437c4
Merge branch 'master' of ssh://git-annex.branchable.com 2019-05-20 16:46:26 -04:00
Joey Hess
357f420ba3
devblog 2019-05-20 16:46:14 -04:00
Joey Hess
97fd9da6e7
add back non-preferred files to imported tree
Prevents merging the import from deleting the non-preferred files from
the branch it's merged into.

adjustTree previously appended the new list of items to the old, which
could result in it generating a tree with multiple files with the same
name. That is not good and confuses some parts of git. Gave it a
function to resolve such conflicts.

That allowed dealing with the problem of what happens when the import
contains some files (or subtrees) with the same name as files that were
filtered out of the export. The files from the import win.
2019-05-20 16:43:52 -04:00
Joey Hess
7d177b78e4
docs for export preferred content
This includes a note about how include= and exclude= match when exporting
a subtree. I don't know if the note is prominent enough, but the
behavior seems unsurprising enough.
2019-05-20 12:06:02 -04:00
branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e
8bec41ceed Added a comment 2019-05-20 15:23:45 +00:00
Joey Hess
12451ea010
Merge branch 'master' into preferred 2019-05-20 10:00:03 -04:00
Joey Hess
fb69a8c215
retitle 2019-05-20 09:59:12 -04:00
Joey Hess
4894bfe3ea
comment 2019-05-20 09:58:36 -04:00
Joey Hess
2246aef3b8
Merge branch 'master' of ssh://git-annex.branchable.com 2019-05-20 09:51:25 -04:00
Joey Hess
cc6eb46ac2
Merge branch 'master' of ssh://git-annex.branchable.com 2019-05-20 09:50:56 -04:00
branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e
06bc46eaa2 Added a comment: it's zsh 2019-05-19 10:05:27 +00:00
anthony@2b89f08c5c67f83d920fe464d7363db8f45cec20
b7a8264c2f Added a comment 2019-05-19 08:39:09 +00:00
branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e
02a3160ca5 Added a comment: .profile doesn't take effect 2019-05-18 19:46:09 +00:00
Joey Hess
2d7f295b90
todo 2019-05-17 21:15:36 -04:00
ply
c70f344af4 Added a comment 2019-05-17 20:55:48 +00:00
Ilya_Shlyakhter
d803c1c331 reported issue with git-annex-init silently overwriting repo description 2019-05-17 17:51:25 +00:00
Joey Hess
8958556fe3
thought 2019-05-16 20:41:17 -04:00
Ilya_Shlyakhter
33c16fd936 reported still seeing errors with git-annex-add 2019-05-16 18:36:44 +00:00
aragilar
bea10be73c Added a comment 2019-05-16 08:26:28 +00:00
derobert
1a33402476 Added a comment 2019-05-16 08:12:45 +00:00
aragilar
117c009f23 2019-05-16 06:27:23 +00:00
anthony@ad39673d230d75cbfd19d2757d754030049c7673
2d7186b43d Added a comment 2019-05-15 17:55:58 +00:00
branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e
a275f65662 Added a comment: Are we talking about a different installer? 2019-05-14 19:57:41 +00:00
Joey Hess
428435c37c
comment 2019-05-14 15:30:38 -04:00
Joey Hess
6a012c7d0c
Merge branch 'master' of ssh://git-annex.branchable.com 2019-05-14 15:25:21 -04:00
Joey Hess
24c8b1b15a
update 2019-05-14 15:25:09 -04:00
Ilya_Shlyakhter
846bf51421 Added a comment 2019-05-14 19:23:33 +00:00
Joey Hess
f4de45332c
wording 2019-05-14 15:18:43 -04:00
anthony@ad39673d230d75cbfd19d2757d754030049c7673
f986210657 Added a comment: Should be in ~/.profile 2019-05-14 19:15:08 +00:00
anthony@ad39673d230d75cbfd19d2757d754030049c7673
4e7444cbf1 2019-05-14 19:02:27 +00:00
Joey Hess
5266da2cdd
docs for importing preferred content 2019-05-14 14:43:51 -04:00
Joey Hess
156069f621
Merge branch 'master' into preferred 2019-05-14 14:29:49 -04:00
Joey Hess
f29da41367
fix man page formatting 2019-05-14 14:29:40 -04:00
Joey Hess
9411a7c93c
matching preferred content before key is known
This will let import try to match preferred content expressions before
downloading the content and generating its key.

If an expression needs a key, it preferredContentParser with
preferredContentKeylessTokens will fail to parse it.

standard and groupwanted are not in preferredContentKeylessTokens
because they may refer to an expression that refers to a key.
That needs further work to support them.
2019-05-14 14:28:23 -04:00
Joey Hess
a3e24ed533
more design work 2019-05-14 11:49:23 -04:00
Joey Hess
c5a61ee808
closing in on final design for this 2019-05-14 10:52:00 -04:00
michael.hanke@c60e12358aa3fc6060531bdead1f530ac4d582ec
53bee6dc5f Added a comment: Re: Empty lines sent by git-annex to an external special remote 2019-05-14 08:48:27 +00:00
michael.hanke@c60e12358aa3fc6060531bdead1f530ac4d582ec
c4abfcc368 Added a comment: Empty lines sent by git-annex to an external special remote 2019-05-14 06:36:12 +00:00
Ilya_Shlyakhter
0610789285 Added a comment: checksums of remote data 2019-05-13 22:03:38 +00:00
branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e
4c7f38fc2c 2019-05-12 17:02:07 +00:00
ndj
d45dbd678c 2019-05-11 18:33:25 +00:00
Joey Hess
0c7569bb6f
close 2019-05-10 13:59:39 -04:00
Joey Hess
c77d79d343
close old todo 2019-05-10 13:54:32 -04:00
Joey Hess
ae562ad4d7
update old todo item with what still needs doing
removed old comments that are no longer relevant
2019-05-10 13:52:40 -04:00
Joey Hess
daa0c6c1c6
close old todo 2019-05-10 13:35:55 -04:00
Joey Hess
d32143e7ad
close 2019-05-10 13:34:44 -04:00
Joey Hess
ccfb800fa6
Merge branch 'master' of ssh://git-annex.branchable.com 2019-05-10 13:31:49 -04:00
Joey Hess
82186ca58f
annex.jobs=cpus etc
Added the ability to run one job per CPU (core), by setting annex.jobs=cpus,
or using option --jobs=cpus or -Jcpus.

Built with future expansion in mind, including not defaulting matching on
Concurrency so more constructors can later be added, and using "cpu"
instead of "0".
2019-05-10 13:27:08 -04:00
Joey Hess
459bbd9005
update roadmap 2019-05-10 12:35:14 -04:00
CandyAngel
0076e9fa03 2019-05-10 11:29:10 +00:00
Ilya_Shlyakhter
e0c73c7f29 Added a comment 2019-05-09 21:07:39 +00:00
yarikoptic
a022f42ab2 new issue on inability to access ftp urls forwarded to from http URL 2019-05-09 20:42:34 +00:00
Ilya_Shlyakhter
63f6389e64 problem with multi-threaded copying to an S3 remote. 2019-05-09 03:00:34 +00:00
strmd
aa66648d2f Added a comment 2019-05-08 17:23:24 +00:00
quzhengrobot@6237b7730f478778d064c18a5a314535129357dd
dff0a44287 Added a comment: Android - gitlab repo 2019-05-08 11:35:04 +00:00
Joey Hess
8b44548d0e
comment 2019-05-07 13:20:03 -04:00
Joey Hess
aaeb85361c
Merge branch 'wip' 2019-05-07 13:07:45 -04:00
Joey Hess
0dead2d6ec
Merge branch 'master' of ssh://git-annex.branchable.com 2019-05-07 13:06:30 -04:00
Joey Hess
7264863a01
add news item for git-annex 7.20190507 2019-05-07 13:06:14 -04:00
Joey Hess
2d33122215
avoid ingest lockdown file escaping the withOtherTmp call
Fixes bug that caused git-annex to fail to add a file when another
git-annex process cleaned up the temp directory it was using.

Solution is just to push withOtherTmp out to a higher level, so that
the whole ingest process can be completed inside it.

But in the assistant, that was not practical to do, since withOtherTmp runs
in the Annex monad and the assistant does not. Worked around by introducing
a separate temp directory that only the assistant uses for lockdown.
Since only one assistant can run at a time, it's easy to clean up that
directory of old cruft at startup.
2019-05-07 13:04:57 -04:00
Ilya_Shlyakhter
cc7c20f428 Added a comment 2019-05-07 16:53:10 +00:00
Joey Hess
f98e97669e
comment 2019-05-07 12:10:03 -04:00
Joey Hess
eb592a1f20
Merge branch 'master' of ssh://git-annex.branchable.com 2019-05-07 12:08:30 -04:00
Joey Hess
d6198442ce
comment 2019-05-07 12:08:07 -04:00
Ilya_Shlyakhter
6d5ba46776 Added a comment 2019-05-07 16:04:58 +00:00