Update dotnet manpages for Unix
This commit is contained in:
parent
2e9c0b7b06
commit
b828418021
24 changed files with 3294 additions and 537 deletions
|
@ -1,170 +1,281 @@
|
|||
.\" Automatically generated by Pandoc 1.15.1
|
||||
.\"t
|
||||
.\" Automatically generated by Pandoc 2.1.3
|
||||
.\"
|
||||
.TH "dotnet test command \- .NET Core CLI" "1" "" "" ".NET Core"
|
||||
.hy
|
||||
.TH "DOTNET\-TEST" "1" "April 2016" "" ""
|
||||
.SH Dotnet Test, Adapters and Test Runners
|
||||
.SH dotnet test
|
||||
.PP
|
||||
This document covers the interactions between dotnet test, a potential
|
||||
adapter (like VS) and test runners (like
|
||||
xunit (https://github.com/dotnet/coreclr.xunit)).
|
||||
.SH NAME
|
||||
.PP
|
||||
It describes the communication protocol for these agents, the parameters
|
||||
that the runner needs to support and the modes on which dotnet test and
|
||||
the runner work.
|
||||
.SS Running modes
|
||||
\f[C]dotnet\ test\f[] \- .NET test driver used to execute unit tests.
|
||||
.SH SYNOPSIS
|
||||
.SS .NET Core 2.x
|
||||
.IP
|
||||
.nf
|
||||
\f[C]
|
||||
dotnet\ test\ [<PROJECT>]\ [\-a|\-\-test\-adapter\-path]\ [\-c|\-\-configuration]\ [\-\-collect]\ [\-d|\-\-diag]\ [\-f|\-\-framework]\ [\-\-filter]\ [\-l|\-\-logger]\ [\-\-no\-build]\ [\-\-no\-restore]\ [\-o|\-\-output]\ [\-r|\-\-results\-directory]\ [\-s|\-\-settings]\ [\-t|\-\-list\-tests]\ [\-v|\-\-verbosity]
|
||||
dotnet\ test\ [\-h|\-\-help]
|
||||
\f[]
|
||||
.fi
|
||||
.SS .NET Core 1.x
|
||||
.IP
|
||||
.nf
|
||||
\f[C]
|
||||
dotnet\ test\ [<PROJECT>]\ [\-a|\-\-test\-adapter\-path]\ [\-c|\-\-configuration]\ [\-d|\-\-diag]\ [\-f|\-\-framework]\ [\-\-filter]\ [\-l|\-\-logger]\ [\-\-no\-build]\ [\-o|\-\-output]\ [\-s|\-\-settings]\ [\-t|\-\-list\-tests]\ \ [\-v|\-\-verbosity]
|
||||
dotnet\ test\ [\-h|\-\-help]
|
||||
\f[]
|
||||
.fi
|
||||
.PP
|
||||
Dotnet test supports two running modes:
|
||||
.IP "1." 3
|
||||
Console: In console mode, dotnet test simply executes fully whatever
|
||||
command gets passed to it and outputs the results.
|
||||
Anytime you invoke dotnet test without passing \-\-port, it will run in
|
||||
console mode, which in turn will cause the runner to run in console
|
||||
mode.
|
||||
.IP "2." 3
|
||||
Design time: Anytime you pass a port to dotnet test, we will run in
|
||||
design time.
|
||||
That means that dotnet test will connect to that port using TCP and will
|
||||
then exchange a established set of messages with whatever else is
|
||||
connected to that port.
|
||||
When this happens, the runner also receives a port (a new one, mind you)
|
||||
that dotnet test will use to communicate with it.
|
||||
The reason why the runner also uses TCP to communicate with dotnet test
|
||||
is because in design mode, it is not sufficient to just output results
|
||||
to the console.
|
||||
We need to send the adapter structure messages containing the results of
|
||||
the test execution.
|
||||
.SS Communication protocol in design time.
|
||||
.IP "1." 3
|
||||
Because during design time, dotnet test connects to a port when it
|
||||
starts up, the adapter needs to be listening on that port otherwise
|
||||
dotnet test will fail.
|
||||
We did it like this so that the adapter could reserve all the ports it
|
||||
needs by binding and listening to them before dotnet test ran and tried
|
||||
to get ports for the runner.
|
||||
.IP "2." 3
|
||||
Once dotnet test starts, it sends a TestSession.Connected message to the
|
||||
adapter indicating that it is ready to receive messages.
|
||||
.IP "3." 3
|
||||
It is possible to send an optional version
|
||||
check (https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/Messages/ProtocolVersionMessage.cs)
|
||||
message with the adapter version of the protocol in it.
|
||||
Dotnet test will send back the version of the protocol that it supports.
|
||||
* * * * *
|
||||
.SH DESCRIPTION
|
||||
.PP
|
||||
All messages have the format described here:
|
||||
Message.cs (https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/Messages/Message.cs).
|
||||
The payload formats for each message is described in links to the
|
||||
classes used to de/serialize the information in the description of the
|
||||
protocol.
|
||||
.SS Test Execution
|
||||
The \f[C]dotnet\ test\f[] command is used to execute unit tests in a given project.
|
||||
The \f[C]dotnet\ test\f[] command launches the test runner console application specified for a project.
|
||||
The test runner executes the tests defined for a unit test framework (for example, MSTest, NUnit, or xUnit) and reports the success or failure of each test.
|
||||
The test runner and the unit test library are packaged as NuGet packages and are restored as ordinary dependencies for the project.
|
||||
.PP
|
||||
[IMAGE: alt
|
||||
tag (../../../../Documentation/images/DotnetTestExecuteTests.png)]
|
||||
.IP "1." 3
|
||||
After the optional version check, the adapter sends a
|
||||
TestExecution.GetTestRunnerProcessStartInfo, with the
|
||||
tests (https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/Messages/RunTestsMessage.cs)
|
||||
it wants to execute inside of it.
|
||||
Dotnet test sends back a FileName and Arguments inside a
|
||||
TestStartInfo (https://github.com/dotnet/cli/blob/rel/1.0.0/src/dotnet/commands/dotnet-test/TestStartInfo.cs)
|
||||
payload that the adapter can use to start the runner.
|
||||
In the past, we would send the list of tests to run as part of that
|
||||
argument, but we were actually going over the command line size limit
|
||||
for some test projects.
|
||||
.IP "2." 3
|
||||
As part of the arguments, we send a port that the runner should connect
|
||||
to and for executing tests, a \-\-wait\-command flag, that indicates
|
||||
that the runner should connect to the port and wait for commands,
|
||||
instead of going ahead and executing the tests.
|
||||
.IP "3." 3
|
||||
At this point, the adapter can launch the runner (and attach to it for
|
||||
debugging if it chooses to).
|
||||
.IP "4." 3
|
||||
Once the runner starts, it sends dotnet test a TestRunner.WaitCommand
|
||||
message that indicates it is ready to receive commands, at which point
|
||||
dotnet test sends a TestRunner.Execute with the list of
|
||||
tests (https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/Messages/RunTestsMessage.cs)
|
||||
to run.
|
||||
This bypasses the command line size limit described above.
|
||||
.IP "5." 3
|
||||
The runner then sends dotnet test (and it passes forward to the adapter)
|
||||
a TestExecution.TestStarted for each tests as they start with the
|
||||
test (https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/Test.cs)
|
||||
information inside of it.
|
||||
.IP "6." 3
|
||||
The runner also sends dotnet test (and it forwards to the adapter) a
|
||||
TestExecution.TestResult for each test with the individual
|
||||
result (https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/TestResult.cs)
|
||||
of the test.
|
||||
.IP "7." 3
|
||||
After all tests finish, the runner sends a TestRunner.Completed message
|
||||
to dotnet test, which dotnet test sends as TestExecution.Completed to
|
||||
the adapter.
|
||||
.IP "8." 3
|
||||
Once the adapter is done, it sends dotnet test a TestSession.Terminate
|
||||
which will cause dotnet test to shutdown.
|
||||
.SS Test discovery
|
||||
Test projects specify the test runner using an ordinary \f[C]<PackageReference>\f[] element, as seen in the following sample project file:
|
||||
.PP
|
||||
[IMAGE: alt
|
||||
tag (../../../..//Documentation/images/DotnetTestDiscoverTests.png)]
|
||||
.IP "1." 3
|
||||
After the optional version check, the adapter sends a
|
||||
TestDiscovery.Start message.
|
||||
Because in this case, the adapter does not need to attach to the
|
||||
process, dotnet test will start the runner itself.
|
||||
Also, since there is no long list of arguments to be passed to the
|
||||
runner, no \-\-wait\-command flag is needed to be passed to the runner.
|
||||
dotnet test only passes a \-\-list argument to the runner, which means
|
||||
the runner should not run the tests, just list them.
|
||||
.IP "2." 3
|
||||
The runner then sends dotnet test (and it passes forward to the adapter)
|
||||
a TestDiscovery.TestFound for each
|
||||
test (https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/Test.cs)
|
||||
found.
|
||||
.IP "3." 3
|
||||
After all tests are discovered, the runner sends a TestRunner.Completed
|
||||
message to dotnet test, which dotnet test sends as
|
||||
TestDiscovery.Completed to the adapter.
|
||||
.IP "4." 3
|
||||
Once the adapter is done, it sends dotnet test a TestSession.Terminate
|
||||
which will cause dotnet test to shutdown.
|
||||
.SS Dotnet test parameters
|
||||
[!code\-xmlXUnit Basic Template]
|
||||
.SS Arguments
|
||||
.PP
|
||||
Any parameters not accepted by dotnet test will be forwarded to the
|
||||
runner.
|
||||
This is the list of parameters supported by dotnet test from its own
|
||||
help print out:
|
||||
\f[C]PROJECT\f[]
|
||||
.PP
|
||||
Usage: dotnet test [arguments] [options]
|
||||
Specifies a path to the test project.
|
||||
If omitted, it defaults to current directory.
|
||||
.SH OPTIONS
|
||||
.SS .NET Core 2.x
|
||||
.PP
|
||||
Arguments: The project to test, defaults to the current directory.
|
||||
Can be a path to a project.json or a project directory.
|
||||
\f[C]\-a|\-\-test\-adapter\-path\ <PATH_TO_ADAPTER>\f[]
|
||||
.PP
|
||||
Options: \-?|\-h|\-\-help Show help information \-\-parentProcessId Used
|
||||
by IDEs to specify their process ID.
|
||||
Test will exit if the parent process does.
|
||||
\-\-port Used by IDEs to specify a port number to listen for a
|
||||
connection.
|
||||
\-c|\-\-configuration Configuration under which to build \-o|\-\-output
|
||||
Directory in which to find the binaries to be run
|
||||
\-b|\-\-build\-base\-path Directory in which to find temporary outputs
|
||||
\-f|\-\-framework Looks for test binaries for a specific framework
|
||||
\-r|\-\-runtime Look for test binaries for a for the specified runtime
|
||||
\-\-no\-build Do not build project before testing
|
||||
.SS Minimum parameters that the runner needs to support
|
||||
.IP \[bu] 2
|
||||
AssemblyUnderTest: Path to the dll that contains the tests to be run.
|
||||
.IP \[bu] 2
|
||||
\-\-port: Used by dotnet test to specify a port number that the runner
|
||||
should connect to.
|
||||
.IP \[bu] 2
|
||||
\-\-list: Indicates that the tests should only be listed and not
|
||||
executed.
|
||||
.IP \[bu] 2
|
||||
\-\-designtime: Indicates that the runner is running in design time, for
|
||||
instance, inside an adapter.
|
||||
.IP \[bu] 2
|
||||
\-\-wait\-command: Indicates that the runner should wait to receive
|
||||
commands through the TCP channel instead of running tests right away.
|
||||
We use this to get around the command line size limit.
|
||||
Use the custom test adapters from the specified path in the test run.
|
||||
.PP
|
||||
\f[C]\-c|\-\-configuration\ {Debug|Release}\f[]
|
||||
.PP
|
||||
Defines the build configuration.
|
||||
The default value is \f[C]Debug\f[], but your project's configuration could override this default SDK setting.
|
||||
.PP
|
||||
\f[C]\-\-collect\ <DATA_COLLECTOR_FRIENDLY_NAME>\f[]
|
||||
.PP
|
||||
Enables data collector for the test run.
|
||||
For more information, see Monitor and analyze test run.
|
||||
.PP
|
||||
\f[C]\-d|\-\-diag\ <PATH_TO_DIAGNOSTICS_FILE>\f[]
|
||||
.PP
|
||||
Enables diagnostic mode for the test platform and write diagnostic messages to the specified file.
|
||||
.PP
|
||||
\f[C]\-f|\-\-framework\ <FRAMEWORK>\f[]
|
||||
.PP
|
||||
Looks for test binaries for a specific framework.
|
||||
.PP
|
||||
\f[C]\-\-filter\ <EXPRESSION>\f[]
|
||||
.PP
|
||||
Filters out tests in the current project using the given expression.
|
||||
For more information, see the Filter option details section.
|
||||
For additional information and examples on how to use selective unit test filtering, see Running selective unit tests.
|
||||
.PP
|
||||
\f[C]\-h|\-\-help\f[]
|
||||
.PP
|
||||
Prints out a short help for the command.
|
||||
.PP
|
||||
\f[C]\-l|\-\-logger\ <LoggerUri/FriendlyName>\f[]
|
||||
.PP
|
||||
Specifies a logger for test results.
|
||||
.PP
|
||||
\f[C]\-\-no\-build\f[]
|
||||
.PP
|
||||
Does not build the test project prior to running it.
|
||||
.PP
|
||||
\f[C]\-\-no\-restore\f[]
|
||||
.PP
|
||||
Doesn't perform an implicit restore when running the command.
|
||||
.PP
|
||||
\f[C]\-o|\-\-output\ <OUTPUT_DIRECTORY>\f[]
|
||||
.PP
|
||||
Directory in which to find the binaries to run.
|
||||
.PP
|
||||
\f[C]\-r|\-\-results\-directory\ <PATH>\f[]
|
||||
.PP
|
||||
The directory where the test results are going to be placed.
|
||||
The specified directory will be created if it doesn't exist.
|
||||
.PP
|
||||
\f[C]\-s|\-\-settings\ <SETTINGS_FILE>\f[]
|
||||
.PP
|
||||
Settings to use when running tests.
|
||||
.PP
|
||||
\f[C]\-t|\-\-list\-tests\f[]
|
||||
.PP
|
||||
List all of the discovered tests in the current project.
|
||||
.PP
|
||||
\f[C]\-v|\-\-verbosity\ <LEVEL>\f[]
|
||||
.PP
|
||||
Sets the verbosity level of the command.
|
||||
Allowed values are \f[C]q[uiet]\f[], \f[C]m[inimal]\f[], \f[C]n[ormal]\f[], \f[C]d[etailed]\f[], and \f[C]diag[nostic]\f[].
|
||||
.SS .NET Core 1.x
|
||||
.PP
|
||||
\f[C]\-a|\-\-test\-adapter\-path\ <PATH_TO_ADAPTER>\f[]
|
||||
.PP
|
||||
Use the custom test adapters from the specified path in the test run.
|
||||
.PP
|
||||
\f[C]\-c|\-\-configuration\ {Debug|Release}\f[]
|
||||
.PP
|
||||
Defines the build configuration.
|
||||
The default value is \f[C]Debug\f[], but your project's configuration could override this default SDK setting.
|
||||
.PP
|
||||
\f[C]\-d|\-\-diag\ <PATH_TO_DIAGNOSTICS_FILE>\f[]
|
||||
.PP
|
||||
Enables diagnostic mode for the test platform and write diagnostic messages to the specified file.
|
||||
.PP
|
||||
\f[C]\-f|\-\-framework\ <FRAMEWORK>\f[]
|
||||
.PP
|
||||
Looks for test binaries for a specific framework.
|
||||
.PP
|
||||
\f[C]\-\-filter\ <EXPRESSION>\f[]
|
||||
.PP
|
||||
Filters out tests in the current project using the given expression.
|
||||
For more information, see the Filter option details section.
|
||||
For additional information and examples on how to use selective unit test filtering, see Running selective unit tests.
|
||||
.PP
|
||||
\f[C]\-h|\-\-help\f[]
|
||||
.PP
|
||||
Prints out a short help for the command.
|
||||
.PP
|
||||
\f[C]\-l|\-\-logger\ <LoggerUri/FriendlyName>\f[]
|
||||
.PP
|
||||
Specifies a logger for test results.
|
||||
.PP
|
||||
\f[C]\-\-no\-build\f[]
|
||||
.PP
|
||||
Does not build the test project prior to running it.
|
||||
.PP
|
||||
\f[C]\-o|\-\-output\ <OUTPUT_DIRECTORY>\f[]
|
||||
.PP
|
||||
Directory in which to find the binaries to run.
|
||||
.PP
|
||||
\f[C]\-s|\-\-settings\ <SETTINGS_FILE>\f[]
|
||||
.PP
|
||||
Settings to use when running tests.
|
||||
.PP
|
||||
\f[C]\-t|\-\-list\-tests\f[]
|
||||
.PP
|
||||
List all of the discovered tests in the current project.
|
||||
.PP
|
||||
\f[C]\-v|\-\-verbosity\ <LEVEL>\f[]
|
||||
.PP
|
||||
Sets the verbosity level of the command.
|
||||
Allowed values are \f[C]q[uiet]\f[], \f[C]m[inimal]\f[], \f[C]n[ormal]\f[], \f[C]d[etailed]\f[], and \f[C]diag[nostic]\f[].
|
||||
.PP
|
||||
* * * * *
|
||||
.SH EXAMPLES
|
||||
.PP
|
||||
Run the tests in the project in the current directory:
|
||||
.PP
|
||||
\f[C]dotnet\ test\f[]
|
||||
.PP
|
||||
Run the tests in the \f[C]test1\f[] project:
|
||||
.PP
|
||||
\f[C]dotnet\ test\ ~/projects/test1/test1.csproj\f[]
|
||||
.SS Filter option details
|
||||
.PP
|
||||
\f[C]\-\-filter\ <EXPRESSION>\f[]
|
||||
.PP
|
||||
\f[C]<Expression>\f[] has the format \f[C]<property><operator><value>[|&<Expression>]\f[].
|
||||
.PP
|
||||
\f[C]<property>\f[] is an attribute of the \f[C]Test\ Case\f[].
|
||||
The following are the properties supported by popular unit test frameworks:
|
||||
.PP
|
||||
.TS
|
||||
tab(@);
|
||||
c l.
|
||||
T{
|
||||
Test Framework
|
||||
T}@T{
|
||||
Supported properties
|
||||
T}
|
||||
_
|
||||
.TE
|
||||
| MSTest |
|
||||
FullyQualifiedName
|
||||
Name
|
||||
ClassName
|
||||
Priority
|
||||
TestCategory
|
||||
| | Xunit |
|
||||
FullyQualifiedName
|
||||
DisplayName
|
||||
Traits
|
||||
.IP
|
||||
.nf
|
||||
\f[C]
|
||||
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |
|
||||
\f[]
|
||||
.fi
|
||||
.PP
|
||||
The \f[C]<operator>\f[] describes the relationship between the property and the value:
|
||||
.PP
|
||||
.TS
|
||||
tab(@);
|
||||
c l.
|
||||
T{
|
||||
Operator
|
||||
T}@T{
|
||||
Function
|
||||
T}
|
||||
_
|
||||
T{
|
||||
\f[C]=\f[]
|
||||
T}@T{
|
||||
Exact match
|
||||
T}
|
||||
T{
|
||||
\f[C]!=\f[]
|
||||
T}@T{
|
||||
Not exact match
|
||||
T}
|
||||
T{
|
||||
\f[C]~\f[]
|
||||
T}@T{
|
||||
Contains
|
||||
T}
|
||||
.TE
|
||||
.PP
|
||||
\f[C]<value>\f[] is a string.
|
||||
All the lookups are case insensitive.
|
||||
.PP
|
||||
An expression without an \f[C]<operator>\f[] is automatically considered as a \f[C]contains\f[] on \f[C]FullyQualifiedName\f[] property (for example, \f[C]dotnet\ test\ \-\-filter\ xyz\f[] is same as \f[C]dotnet\ test\ \-\-filter\ FullyQualifiedName~xyz\f[]).
|
||||
.PP
|
||||
Expressions can be joined with conditional operators:
|
||||
.PP
|
||||
.TS
|
||||
tab(@);
|
||||
c c.
|
||||
T{
|
||||
Operator
|
||||
T}@T{
|
||||
Function
|
||||
T}
|
||||
_
|
||||
T{
|
||||
| \ \
|
||||
T}@T{
|
||||
OR \ \ \
|
||||
T}
|
||||
T{
|
||||
\f[C]&\f[]
|
||||
T}@T{
|
||||
AND
|
||||
T}
|
||||
.TE
|
||||
.PP
|
||||
You can enclose expressions in parenthesis when using conditional operators (for example, \f[C](Name~TestMethod1)\ |\ (Name~TestMethod2)\f[]).
|
||||
.PP
|
||||
For additional information and examples on how to use selective unit test filtering, see Running selective unit tests.
|
||||
.SS See also
|
||||
.PP
|
||||
Frameworks and Targets
|
||||
.PD 0
|
||||
.P
|
||||
.PD
|
||||
\&.NET Core Runtime IDentifier (RID) catalog
|
||||
.SH AUTHORS
|
||||
Microsoft Corporation dotnetclifeedback\@microsoft.com.
|
||||
mairaw.
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue