Update documentation (#3549)
* Updating README.md files as well as man pages This aligns with the latest changes in the official documentation. It also aligns dotnet test readme.md to official docs and introduces a new doc in the "Documentation" directory that deals with dotnet test protocols.
This commit is contained in:
parent
18c8a2724a
commit
4da4f95e21
16 changed files with 971 additions and 404 deletions
170
Documentation/manpages/sdk/dotnet-test.1
Normal file
170
Documentation/manpages/sdk/dotnet-test.1
Normal file
|
@ -0,0 +1,170 @@
|
|||
.\" Automatically generated by Pandoc 1.15.1
|
||||
.\"
|
||||
.hy
|
||||
.TH "DOTNET\-TEST" "1" "April 2016" "" ""
|
||||
.SH Dotnet Test, Adapters and Test Runners
|
||||
.PP
|
||||
This document covers the interactions between dotnet test, a potential
|
||||
adapter (like VS) and test runners (like
|
||||
xunit (https://github.com/dotnet/coreclr.xunit)).
|
||||
.PP
|
||||
It describes the communication protocol for these agents, the parameters
|
||||
that the runner needs to support and the modes on which dotnet test and
|
||||
the runner work.
|
||||
.SS Running modes
|
||||
.PP
|
||||
Dotnet test supports two running modes:
|
||||
.IP "1." 3
|
||||
Console: In console mode, dotnet test simply executes fully whatever
|
||||
command gets passed to it and outputs the results.
|
||||
Anytime you invoke dotnet test without passing \-\-port, it will run in
|
||||
console mode, which in turn will cause the runner to run in console
|
||||
mode.
|
||||
.IP "2." 3
|
||||
Design time: Anytime you pass a port to dotnet test, we will run in
|
||||
design time.
|
||||
That means that dotnet test will connect to that port using TCP and will
|
||||
then exchange a established set of messages with whatever else is
|
||||
connected to that port.
|
||||
When this happens, the runner also receives a port (a new one, mind you)
|
||||
that dotnet test will use to communicate with it.
|
||||
The reason why the runner also uses TCP to communicate with dotnet test
|
||||
is because in design mode, it is not sufficient to just output results
|
||||
to the console.
|
||||
We need to send the adapter structure messages containing the results of
|
||||
the test execution.
|
||||
.SS Communication protocol in design time.
|
||||
.IP "1." 3
|
||||
Because during design time, dotnet test connects to a port when it
|
||||
starts up, the adapter needs to be listening on that port otherwise
|
||||
dotnet test will fail.
|
||||
We did it like this so that the adapter could reserve all the ports it
|
||||
needs by binding and listening to them before dotnet test ran and tried
|
||||
to get ports for the runner.
|
||||
.IP "2." 3
|
||||
Once dotnet test starts, it sends a TestSession.Connected message to the
|
||||
adapter indicating that it is ready to receive messages.
|
||||
.IP "3." 3
|
||||
It is possible to send an optional version
|
||||
check (https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/Messages/ProtocolVersionMessage.cs)
|
||||
message with the adapter version of the protocol in it.
|
||||
Dotnet test will send back the version of the protocol that it supports.
|
||||
.PP
|
||||
All messages have the format described here:
|
||||
Message.cs (https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/Messages/Message.cs).
|
||||
The payload formats for each message is described in links to the
|
||||
classes used to de/serialize the information in the description of the
|
||||
protocol.
|
||||
.SS Test Execution
|
||||
.PP
|
||||
[IMAGE: alt
|
||||
tag (../../../../Documentation/images/DotnetTestExecuteTests.png)]
|
||||
.IP "1." 3
|
||||
After the optional version check, the adapter sends a
|
||||
TestExecution.GetTestRunnerProcessStartInfo, with the
|
||||
tests (https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/Messages/RunTestsMessage.cs)
|
||||
it wants to execute inside of it.
|
||||
Dotnet test sends back a FileName and Arguments inside a
|
||||
TestStartInfo (https://github.com/dotnet/cli/blob/rel/1.0.0/src/dotnet/commands/dotnet-test/TestStartInfo.cs)
|
||||
payload that the adapter can use to start the runner.
|
||||
In the past, we would send the list of tests to run as part of that
|
||||
argument, but we were actually going over the command line size limit
|
||||
for some test projects.
|
||||
.IP "2." 3
|
||||
As part of the arguments, we send a port that the runner should connect
|
||||
to and for executing tests, a \-\-wait\-command flag, that indicates
|
||||
that the runner should connect to the port and wait for commands,
|
||||
instead of going ahead and executing the tests.
|
||||
.IP "3." 3
|
||||
At this point, the adapter can launch the runner (and attach to it for
|
||||
debugging if it chooses to).
|
||||
.IP "4." 3
|
||||
Once the runner starts, it sends dotnet test a TestRunner.WaitCommand
|
||||
message that indicates it is ready to receive commands, at which point
|
||||
dotnet test sends a TestRunner.Execute with the list of
|
||||
tests (https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/Messages/RunTestsMessage.cs)
|
||||
to run.
|
||||
This bypasses the command line size limit described above.
|
||||
.IP "5." 3
|
||||
The runner then sends dotnet test (and it passes forward to the adapter)
|
||||
a TestExecution.TestStarted for each tests as they start with the
|
||||
test (https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/Test.cs)
|
||||
information inside of it.
|
||||
.IP "6." 3
|
||||
The runner also sends dotnet test (and it forwards to the adapter) a
|
||||
TestExecution.TestResult for each test with the individual
|
||||
result (https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/TestResult.cs)
|
||||
of the test.
|
||||
.IP "7." 3
|
||||
After all tests finish, the runner sends a TestRunner.Completed message
|
||||
to dotnet test, which dotnet test sends as TestExecution.Completed to
|
||||
the adapter.
|
||||
.IP "8." 3
|
||||
Once the adapter is done, it sends dotnet test a TestSession.Terminate
|
||||
which will cause dotnet test to shutdown.
|
||||
.SS Test discovery
|
||||
.PP
|
||||
[IMAGE: alt
|
||||
tag (../../../..//Documentation/images/DotnetTestDiscoverTests.png)]
|
||||
.IP "1." 3
|
||||
After the optional version check, the adapter sends a
|
||||
TestDiscovery.Start message.
|
||||
Because in this case, the adapter does not need to attach to the
|
||||
process, dotnet test will start the runner itself.
|
||||
Also, since there is no long list of arguments to be passed to the
|
||||
runner, no \-\-wait\-command flag is needed to be passed to the runner.
|
||||
dotnet test only passes a \-\-list argument to the runner, which means
|
||||
the runner should not run the tests, just list them.
|
||||
.IP "2." 3
|
||||
The runner then sends dotnet test (and it passes forward to the adapter)
|
||||
a TestDiscovery.TestFound for each
|
||||
test (https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/Test.cs)
|
||||
found.
|
||||
.IP "3." 3
|
||||
After all tests are discovered, the runner sends a TestRunner.Completed
|
||||
message to dotnet test, which dotnet test sends as
|
||||
TestDiscovery.Completed to the adapter.
|
||||
.IP "4." 3
|
||||
Once the adapter is done, it sends dotnet test a TestSession.Terminate
|
||||
which will cause dotnet test to shutdown.
|
||||
.SS Dotnet test parameters
|
||||
.PP
|
||||
Any parameters not accepted by dotnet test will be forwarded to the
|
||||
runner.
|
||||
This is the list of parameters supported by dotnet test from its own
|
||||
help print out:
|
||||
.PP
|
||||
Usage: dotnet test [arguments] [options]
|
||||
.PP
|
||||
Arguments: The project to test, defaults to the current directory.
|
||||
Can be a path to a project.json or a project directory.
|
||||
.PP
|
||||
Options: \-?|\-h|\-\-help Show help information \-\-parentProcessId Used
|
||||
by IDEs to specify their process ID.
|
||||
Test will exit if the parent process does.
|
||||
\-\-port Used by IDEs to specify a port number to listen for a
|
||||
connection.
|
||||
\-c|\-\-configuration Configuration under which to build \-o|\-\-output
|
||||
Directory in which to find the binaries to be run
|
||||
\-b|\-\-build\-base\-path Directory in which to find temporary outputs
|
||||
\-f|\-\-framework Looks for test binaries for a specific framework
|
||||
\-r|\-\-runtime Look for test binaries for a for the specified runtime
|
||||
\-\-no\-build Do not build project before testing
|
||||
.SS Minimum parameters that the runner needs to support
|
||||
.IP \[bu] 2
|
||||
AssemblyUnderTest: Path to the dll that contains the tests to be run.
|
||||
.IP \[bu] 2
|
||||
\-\-port: Used by dotnet test to specify a port number that the runner
|
||||
should connect to.
|
||||
.IP \[bu] 2
|
||||
\-\-list: Indicates that the tests should only be listed and not
|
||||
executed.
|
||||
.IP \[bu] 2
|
||||
\-\-designtime: Indicates that the runner is running in design time, for
|
||||
instance, inside an adapter.
|
||||
.IP \[bu] 2
|
||||
\-\-wait\-command: Indicates that the runner should wait to receive
|
||||
commands through the TCP channel instead of running tests right away.
|
||||
We use this to get around the command line size limit.
|
||||
.SH AUTHORS
|
||||
Microsoft Corporation dotnetclifeedback\@microsoft.com.
|
Loading…
Add table
Add a link
Reference in a new issue