Commit graph

195 commits

Author SHA1 Message Date
Jon Fortescue
f5c955d584 Switch to 1ES R&D pools on main 2021-10-04 11:11:05 -07:00
MichaelSimons
ad7094b0db Fix source-build merge conflicts 2021-10-01 19:21:21 +00:00
MichaelSimons
2f7c1a94d6 Disable offline source-build tarball CI 2021-10-01 18:36:48 +00:00
MichaelSimons
7094f386a9 Disable offline source-build tarball CI 2021-10-01 18:21:43 +00:00
Sarah Oslund
cef94cb9e0 Merge remote-tracking branch 'upstream/release/6.0.1xx-rc2' into HEAD 2021-10-01 09:18:12 -07:00
Michael Simons
0e1b32b8ee
[ArPow] Add support to build source-build from specific installer commit (#12186)
* Add support to build source-build from specific installer commit

* Fix nuget.client repo name
2021-09-29 20:48:16 +00:00
Łukasz Patalas
3435ea263c
[release/6.0.1xx-rc2] Migrate to 1ES hosted pools (#12056) 2021-09-28 07:40:29 -05:00
Michael Simons
e68cea7f97
Apply source-build patches during tarball creation (#12133) 2021-09-28 07:39:40 -05:00
Michael Simons
3694b8b984
[ArPow] Run smoke tests for tarball CI (#11023) (#12055) 2021-09-23 19:02:27 -05:00
Matt Mitchell
87c927be27
Add source build tarball setup nuget feeds and remove unused group (#12081) 2021-09-22 22:05:24 +00:00
Michael Simons
d26f9a313a
[ArPow] Update prebuilts and enable offline CI (#12072)
* Update prebuilts and enable offline CI

* Update previous source-built and add offline source-link patch
2021-09-22 14:19:05 -05:00
Matt Mitchell
1e9ab7323f
Add source build tarball setup nuget feeds and remove unused group (#12079) 2021-09-22 12:15:31 -07:00
Michael Simons
4a23a7ed10
Update commit shas for source build (#12066) 2021-09-22 10:22:47 -05:00
Logan Bussell
3d8501fba2
[ArPow] Add text-only packages to tarball (#11967)
* add text-only packages to tarball

* add BuildTextOnlyPackages target to tarball build targets

* fix typo

* add content to text only package nuspec files

* Change reference to single nuspec file

* Update based on PR review

* prevent inner buildlog from being overwritten

* condition SourceBuildIntermediate.proj copy and build on the file not existing

* condition entire copy target on SourceBuildIntermediateProjTarget file existing

* Revert "condition entire copy target on SourceBuildIntermediateProjTarget file existing"

This reverts commit 995e106c81751b7d9597abf0c120963c2ab5374b.

* Revert "condition SourceBuildIntermediate.proj copy and build on the file not existing"

This reverts commit 3fbaa11bdb34cf9bb8f006f4c27bbd16783cac09.

* Revert "prevent inner buildlog from being overwritten"

This reverts commit f994e885fd7e0938dcc80150f3a0a8aad215f014.

Co-authored-by: dseefeld <dseefeld@microsoft.com>
2021-09-21 21:06:41 -05:00
Matt Mitchell
cce77edfe1
[main] Tweaks for RTM stable building and servicing (#11936)
* Tweaks for RTM stable building and servicing
- When building from source offline in a stable configuration, the shared frameworks versions will match the directory from which the runtime archives are located in, both stable. This is in contrast to when not building from source, in which the runtime archives will appear in non-stable directories.
- The targeting pack versions should not be assumed to always match the runtime framework versions, and the blob directories containing them are the same. Add new dependencies so that we can track where the targeting pack directory is over time.
- PGO archives coming from runtime are always built non-stable. This means the download location needs to be tweaks, and crossgen calls need to look for shared framework binaries in the non-stable locations.
- For source-build, the templating should move to a new non-stable dependency. This is used to determine the version of the source build intermediate package.

* Generate the full nuget version in the source build tarball job.
The source build tarball job needs the full nuget version so that it can publish the blob to the right location (`Sdk/<full version>/source-<stable version>.zip). This involves being able to call the GenerateFullNuGetVersion target and the targets it depends on.

* Use correct package version prop for crossgen targets
2021-09-14 14:59:33 -07:00
Matt Mitchell
1131a93542
[release/6.0.1xx] Tweaks for RTM stable building and servicing (#11938)
* Tweaks for RTM stable building and servicing
- When building from source offline in a stable configuration, the shared frameworks versions will match the directory from which the runtime archives are located in, both stable. This is in contrast to when not building from source, in which the runtime archives will appear in non-stable directories.
- The targeting pack versions should not be assumed to always match the runtime framework versions, and the blob directories containing them are the same. Add new dependencies so that we can track where the targeting pack directory is over time.
- PGO archives coming from runtime are always built non-stable. This means the download location needs to be tweaks, and crossgen calls need to look for shared framework binaries in the non-stable locations.
- For source-build, the templating should move to a new non-stable dependency. This is used to determine the version of the source build intermediate package.

* Generate the full nuget version in the source build tarball job.
The source build tarball job needs the full nuget version so that it can publish the blob to the right location (`Sdk/<full version>/source-<stable version>.zip). This involves being able to call the GenerateFullNuGetVersion target and the targets it depends on.
2021-09-13 12:01:04 -07:00
Michael Simons
1bfb50655c
[ArPow] Update prebuilts and pipeline build agents (#11779) (#11974) 2021-09-13 13:01:07 -05:00
Michael Simons
052907276a
Update source-build commit shas (#11968) 2021-09-11 00:36:20 +00:00
Dan Seefeldt
1ee668a856
Port logic from 5.0 to clean binaries from tarball (#11955) 2021-09-10 22:21:07 +00:00
Dan Seefeldt
dae9febf42
Update dependency from dotnet/source-build-reference-packages (#11966) 2021-09-10 15:37:10 -05:00
Dan Seefeldt
1afc0b6892
Update dependency from dotnet/source-build-reference-packages (#11928) 2021-09-08 16:43:44 +00:00
Dan Seefeldt
3efe376de1
[ArPow] Update dependency from dotnet/source-build-reference-packages (#11915)
* Update dependency from dotnet/source-build-reference-packages

* Change in SourceBuild dir to trigger SB CI
2021-09-08 08:29:39 -05:00
Dan Seefeldt
290df5778e
[ArPow] Reduce the size of the tarball. (#11662)
* Copy instead of clone and remove .git/objects directories

* Add depth 1 to fetch command

* Add commit count to git-info for installer repo

* Update targets to fix dependency ordering

Some repos take a dependency on PrepareInnerSourceBuildRepoRoot,
assuming that the inner repo will be setup.  The condition for copy
instead of clone caused a conflict.
Keep PrepareInnerSourceBuildRepoRoot and make it depend on
either copy or clone completing.

* Don't copy artifacts dir

* Actually don't remove artifacts directory

* Update prebuilt version
2021-09-01 08:31:15 +00:00
Michael Simons
c61bcfc01f
[ArPow] Update prebuilts and pipeline build agents (#11779) 2021-09-01 04:20:55 +00:00
Michael Simons
f9cc5150e0
[ArPow] Update pipeline to capture prebuild-report (#11688) 2021-08-26 15:54:04 -05:00
Michael Simons
fe0187f55e
[ArPow] Misc changes to reduce tarball build size (#11683) 2021-08-26 16:30:50 +00:00
Michael Simons
3be0fd4f08
Source-build updates to handle internal repo names (#11661) 2021-08-25 01:46:35 +00:00
Chris Rummel
f3ea71b28f
[ArPow] Add NuGet.Client (#11489)
* [main] Update dependencies from dotnet/sdk (#11565)

[main] Update dependencies from dotnet/sdk
- Coherency Updates:
  - Microsoft.AspNetCore.App.Ref: from 6.0.0-rc.1.21417.2 to 6.0.0-rc.1.21417.8 (parent: Microsoft.NET.Sdk)
  - Microsoft.AspNetCore.App.Ref.Internal: from 6.0.0-rc.1.21417.2 to 6.0.0-rc.1.21417.8 (parent: Microsoft.NET.Sdk)
  - Microsoft.AspNetCore.App.Runtime.win-x64: from 6.0.0-rc.1.21417.2 to 6.0.0-rc.1.21417.8 (parent: Microsoft.NET.Sdk)
  - VS.Redist.Common.AspNetCore.SharedFramework.x64.6.0: from 6.0.0-rc.1.21417.2 to 6.0.0-rc.1.21417.8 (parent: Microsoft.NET.Sdk)
  - dotnet-dev-certs: from 6.0.0-rc.1.21417.2 to 6.0.0-rc.1.21417.8 (parent: Microsoft.NET.Sdk)
  - dotnet-user-secrets: from 6.0.0-rc.1.21417.2 to 6.0.0-rc.1.21417.8 (parent: Microsoft.NET.Sdk)

* Update dependencies from https://github.com/dotnet/sdk build 20210817.21 (#11569)

[main] Update dependencies from dotnet/sdk
- Coherency Updates:
  - Microsoft.NET.Test.Sdk: from 17.0.0-preview-20210817-01 to 17.0.0-preview-20210817-02 (parent: Microsoft.NET.Sdk)

* Update dependencies from https://github.com/dotnet/arcade build 20210817.1 (#11570)

[main] Update dependencies from dotnet/arcade
- Coherency Updates:
  - Microsoft.SourceLink.GitHub: from 1.1.0-beta-21309-01 to 1.1.0-beta-21416-05 (parent: Microsoft.DotNet.Arcade.Sdk)
  - Microsoft.DotNet.XliffTasks: from 1.0.0-beta.21412.1 to 1.0.0-beta.21414.1 (parent: Microsoft.DotNet.Arcade.Sdk)

* [ArPow] Add runtime repo to tarball (#11506)

* Add runtime repo / update runtime project

* Update runtime-portable logs and properties

* Create runtime-portable git-info

* Fix unpack of IntermediatePackages

* Re-enable references to runtime

* Pass TargetRid and NonPortable to runtime build

* Only build allconfigurations in non-portable build

* Add SDK patch to disable new CA warning

* Increase built tarball timeout

* Update prebuilt version

* Remove 5.0.3xx and preview 7
Add rc1, 6.0 daily, and move main to 7.0

* Removing 7.0 for now as there aren't builds yet.

* Update dependencies from https://github.com/dotnet/arcade build 20210818.12 (#11594)

[main] Update dependencies from dotnet/arcade
- Coherency Updates:
  - Microsoft.SourceLink.GitHub: from 1.1.0-beta-21416-05 to 1.1.0-beta-21417-01 (parent: Microsoft.DotNet.Arcade.Sdk)
  - Microsoft.DotNet.XliffTasks: from 1.0.0-beta.21414.1 to 1.0.0-beta.21417.1 (parent: Microsoft.DotNet.Arcade.Sdk)

* [ArPow] Add installer repo to tarball (#11483)

* Fix the rc2 aka.ms links

* Add NuGet

* Make safe repo name safer.

* some more props filename stuff

* Patches to get the repo to build

* Use NuGet build script instead of manually calling msbuild.

* Add patch for source-build in build.sh in nuget.

* Use existing 5.0 nuget patch.

* Fixes for build scripts and .NET 6 SDK upgrade.

* Remove obsolete patch - this was caused by not using the correct source-build SDK.

* Address code review comment for source-build build script.

* Restore reference from installer to NuGet.

* Fix patch not applying.

* Don't try to pass Arcade build args to MSBuild.

* Fix script typo.

* Patch NuGet.Frameworks to produce netstandard2.0 for VSTest.

* Did this patch the wrong way around.

* Ignore  failure in no-network case

* UPdate prebuilts version.

Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: Dan Seefeldt <dseefeld@microsoft.com>
Co-authored-by: Marc Paine <marcpop@microsoft.com>
Co-authored-by: Michael Simons <msimons@microsoft.com>
Co-authored-by: Matt Galbraith <mattgal@microsoft.com>
2021-08-24 20:01:06 +00:00
Michael Simons
7c4b93e5b7
[ArPow Tarball] Tier 3 - Add roslyn-analyzers repo to arpow tarball (#11114) 2021-08-24 15:33:44 +00:00
Chris Rummel
f932fe8d77
[ArPow] Merge new repos to release/6.0.1xx (#11620)
* [ArPow] Add runtime repo to tarball (#11506)

* Add runtime repo / update runtime project

* Update runtime-portable logs and properties

* Create runtime-portable git-info

* Fix unpack of IntermediatePackages

* Re-enable references to runtime

* Pass TargetRid and NonPortable to runtime build

* Only build allconfigurations in non-portable build

* Add SDK patch to disable new CA warning

* Increase built tarball timeout

* Update prebuilt version

* [ArPow] Add installer repo to tarball (#11483)

* Ignore  failure in no-network case

Co-authored-by: Dan Seefeldt <dseefeld@microsoft.com>
Co-authored-by: Michael Simons <msimons@microsoft.com>
Co-authored-by: Matt Galbraith <mattgal@microsoft.com>
2021-08-23 22:05:58 +00:00
Michael Simons
f2576ec09d
[ArPow] Add installer repo to tarball (#11483) 2021-08-19 10:11:11 -05:00
Dan Seefeldt
6bd49343d9
[ArPow] Add runtime repo to tarball (#11506)
* Add runtime repo / update runtime project

* Update runtime-portable logs and properties

* Create runtime-portable git-info

* Fix unpack of IntermediatePackages

* Re-enable references to runtime

* Pass TargetRid and NonPortable to runtime build

* Only build allconfigurations in non-portable build

* Add SDK patch to disable new CA warning

* Increase built tarball timeout

* Update prebuilt version
2021-08-18 17:18:01 +00:00
Matt Mitchell
9994b20baa
Changes for internal runtime build support for .NET 6 (#11422)
- Cherry pick the internal runtime download changes over
- Cherry pick changes to the arcade SB template over (making the next update a noop and unblocking P7+)
- Hoist out the variable groups and parameters needed for the internal download to the top level yaml file.
- Remove the old DownloadFile task and replace this with the Arcade version. Specifically this allows us to remove
  set/use of the DOTNETCLIMSRC_READ_SAS_TOKEN environment variable and instead rely on that task's ability to decode and
  use a base64 encoded SAS token. The reason for the environment variable usage before was that the
  non-encoded SAS token was getting mangled by msbuild/bash/etc. on non-Windows OSs.
- Update the source build tarball template with support for internal runtimes

Update to source build template
2021-08-06 13:54:33 -07:00
Michael Simons
5268979bef
[ArPow] Add offline tarball build infra and CI (#11403)
* Add offline build to CI

* Add sourcelink patches to unblock offline aspnetcore build
2021-08-06 12:50:48 -05:00
Dan Seefeldt
33c0d0250d
[ArPow Tarball] Clean up src directories when building (#11421)
* Clean up src directories when building

* Updates based on PR review comments
2021-08-06 09:12:04 -05:00
Michael Simons
2897e82c96
Add prebuilt acquisition and production build infra (#11417) 2021-08-05 15:19:38 -05:00
Michael Simons
30734f0b61
Add tarball pipeline logic to publish Private.SourceBuilt.Artifacts (#11297) 2021-07-27 16:51:44 +00:00
Dan Seefeldt
702587dcbb
[ArPow] Update SBRP to get version with updated ilasm (#11233)
* Update SBRP to get version with updated ilasm

Also, remove the prebuilts for packages that weren't being built
correctly.

* Add empty prebuilt directory
2021-07-23 16:25:22 -05:00
Chris Rummel
91afacfc09
[ArPow] Temporary fix for reference packages with broken extension methods (#11092)
* Add some needed prebuilt packages to override temporarily bad ref packages.

* Remove bad ref packages.

* Also don't copy new versions of the bad SBRP packages after the SBRP build.

* Account for missing intermediate packages and different paths.

* Address code review comment.
2021-07-09 22:43:30 +00:00
Dan Seefeldt
5048e0f143
[Arpow Tarball] Add flag to build with online sources (#11091)
* Add SBRP packages to prev source-built

* Don't delete SBRP packages

* Add BuildWithOnlineSources flag

* Build tarball in CI with online sources

* Remove OfflineBuild / add BuildWithOnlineSources

* Add --online option to tarball build.sh

* Additional cleanup for tarball build

* Rename baseline file

* Update baseline

* Remove acquire-darc.proj

* Default DotNetBuildOffline to true

* Fix line endings in RemoveInternetSources

* Update src/SourceBuild/Arcade/eng/common/templates/steps/source-build-run-tarball-build.yml

Co-authored-by: Michael Simons <msimons@microsoft.com>

Co-authored-by: Michael Simons <msimons@microsoft.com>
2021-07-09 21:42:42 +00:00
Michael Simons
40235466e9
Fix source-build ci and build infra (#11088) 2021-07-08 20:09:54 -05:00
Michael Simons
addb1b1961
Refactor tarball creation to avoid retrieving all git history (#11029) 2021-07-07 09:24:46 -05:00
Michael Simons
8288ae2e6d
Add source-build tarball build ci (#10995) 2021-06-29 08:33:40 -05:00
Michael Simons
69803dae86
Create tarball file and add tarball creation CI (#10972) 2021-06-28 07:36:57 -05:00
Dan Seefeldt
832010fdf6
Initial checkin of source-build tarball build infrastructure (#10961)
* Initial checkin of source-build tarball build infra

* Add a couple more comments

* Update eng/SourceBuild.Version.Details.xml based on PR review

Co-authored-by: Chris Rummel <crummel@microsoft.com>

* Updates based on PR review comments

Co-authored-by: Chris Rummel <crummel@microsoft.com>
2021-06-24 16:36:04 -05:00