The perf tests use `dotnet restore` to download test dependencies. The
perf tests' project.json may sometimes refer to a version of the
framework that is not yet published, which can cause the restore command
to fail. This change addresses the issue by adding a fallback nuget feed
that contains not-yet-published version of the framework.
* Add script to run and compare CLI perf tests
tests/Performance/run-perftests.py is a Python3 script that fetches all
of the dependencies needed to run the perf tests in test/Performance
locally. It can run the perf tests for a single dotnet.exe, or run two
dotnet.exe instances sequentially and compare the results between them.
Basic usage for a single test:
run-perftests.py --name "runid" ...\dotnet.exe
Usage for a comparison run:
run-perftests.py --name "runid" ...\dotnet.exe --base ...\dotnet.exe
For more detailed usage:
run-perftests.py -h
* run-perftests: fix publish xunit.perf.runner.cli
The code that builds xunit-performance would skip publishing
Microsoft.DotNet.xunit.performance.runner.cli due to a bug in the
condition to check whether it was published already or not. To fix the
issue and simplify the logic, I'm making it always publish when
building the project, instead of building and publishing separately.
* run-perftests: add support for python2
* run-perftests.py: fix framework version issue
The perf test harness was failing with "stage 0" binaries due to an issue
finding the correct installed framework version. The fix is to delete the
project.lock.json followed by a dotnet restore before each run, using the
dotnet.exe that is about to be tested. (Kudos to @brianrob for the debugging
help and suggested fix!)
