Commit graph

95 commits

Author SHA1 Message Date
Michael Simons
84217d6e27
Specify depth when updating submodules (#12588) 2021-11-11 08:22:24 -06:00
Michael Simons
4eaf8ab130
Merge pull request #12562 from MichaelSimons/ci-improvements
Enable offline build and capture artifacts in PR validation
2021-11-02 08:52:17 -05:00
Michael Simons
c020befd81
Merge pull request #12507 from omajid/arpow-tarball-remove-files-with-non-open-source-licenses
[release/6.0.1xx] [ArPow] Delete optional files under non-open source licenses
2021-10-25 13:58:22 -05:00
Omair Majid
6de9c8d897 [ArPow] Delete optional files under non-open source licenses
These files are not required as part of source-build, and are under
non-free licenses. Delete them when building the source-build tarball.

Fixes: https://github.com/dotnet/source-build/issues/2359
2021-10-25 10:23:45 -04:00
Davis Goodin
24f68585c3 Prepare source-build for prebuiltless build 2021-10-22 14:20:30 -05:00
MichaelSimons
ee7b08fdb6 Edit per code review 2021-10-12 02:23:02 +00:00
MichaelSimons
0cec6c187e Handle workload manifests as text-only packages in source-build 2021-10-08 19:39:22 +00:00
MichaelSimons
39c6c2cb89 Disable offline and smoke-tests in CI 2021-10-08 12:17:17 +00:00
MichaelSimons
fce1e1b787 Re-enable tarball PR validation 2021-10-05 15:59:33 +00:00
MichaelSimons
7094f386a9 Disable offline source-build tarball CI 2021-10-01 18:21:43 +00:00
Michael Simons
0e1b32b8ee
[ArPow] Add support to build source-build from specific installer commit (#12186)
* Add support to build source-build from specific installer commit

* Fix nuget.client repo name
2021-09-29 20:48:16 +00:00
Łukasz Patalas
3435ea263c
[release/6.0.1xx-rc2] Migrate to 1ES hosted pools (#12056) 2021-09-28 07:40:29 -05:00
Michael Simons
e68cea7f97
Apply source-build patches during tarball creation (#12133) 2021-09-28 07:39:40 -05:00
Michael Simons
3694b8b984
[ArPow] Run smoke tests for tarball CI (#11023) (#12055) 2021-09-23 19:02:27 -05:00
Michael Simons
d26f9a313a
[ArPow] Update prebuilts and enable offline CI (#12072)
* Update prebuilts and enable offline CI

* Update previous source-built and add offline source-link patch
2021-09-22 14:19:05 -05:00
Matt Mitchell
1e9ab7323f
Add source build tarball setup nuget feeds and remove unused group (#12079) 2021-09-22 12:15:31 -07:00
Michael Simons
4a23a7ed10
Update commit shas for source build (#12066) 2021-09-22 10:22:47 -05:00
Logan Bussell
3d8501fba2
[ArPow] Add text-only packages to tarball (#11967)
* add text-only packages to tarball

* add BuildTextOnlyPackages target to tarball build targets

* fix typo

* add content to text only package nuspec files

* Change reference to single nuspec file

* Update based on PR review

* prevent inner buildlog from being overwritten

* condition SourceBuildIntermediate.proj copy and build on the file not existing

* condition entire copy target on SourceBuildIntermediateProjTarget file existing

* Revert "condition entire copy target on SourceBuildIntermediateProjTarget file existing"

This reverts commit 995e106c81751b7d9597abf0c120963c2ab5374b.

* Revert "condition SourceBuildIntermediate.proj copy and build on the file not existing"

This reverts commit 3fbaa11bdb34cf9bb8f006f4c27bbd16783cac09.

* Revert "prevent inner buildlog from being overwritten"

This reverts commit f994e885fd7e0938dcc80150f3a0a8aad215f014.

Co-authored-by: dseefeld <dseefeld@microsoft.com>
2021-09-21 21:06:41 -05:00
Matt Mitchell
1131a93542
[release/6.0.1xx] Tweaks for RTM stable building and servicing (#11938)
* Tweaks for RTM stable building and servicing
- When building from source offline in a stable configuration, the shared frameworks versions will match the directory from which the runtime archives are located in, both stable. This is in contrast to when not building from source, in which the runtime archives will appear in non-stable directories.
- The targeting pack versions should not be assumed to always match the runtime framework versions, and the blob directories containing them are the same. Add new dependencies so that we can track where the targeting pack directory is over time.
- PGO archives coming from runtime are always built non-stable. This means the download location needs to be tweaks, and crossgen calls need to look for shared framework binaries in the non-stable locations.
- For source-build, the templating should move to a new non-stable dependency. This is used to determine the version of the source build intermediate package.

* Generate the full nuget version in the source build tarball job.
The source build tarball job needs the full nuget version so that it can publish the blob to the right location (`Sdk/<full version>/source-<stable version>.zip). This involves being able to call the GenerateFullNuGetVersion target and the targets it depends on.
2021-09-13 12:01:04 -07:00
Michael Simons
052907276a
Update source-build commit shas (#11968) 2021-09-11 00:36:20 +00:00
Dan Seefeldt
1ee668a856
Port logic from 5.0 to clean binaries from tarball (#11955) 2021-09-10 22:21:07 +00:00
Dan Seefeldt
dae9febf42
Update dependency from dotnet/source-build-reference-packages (#11966) 2021-09-10 15:37:10 -05:00
Dan Seefeldt
1afc0b6892
Update dependency from dotnet/source-build-reference-packages (#11928) 2021-09-08 16:43:44 +00:00
Dan Seefeldt
3efe376de1
[ArPow] Update dependency from dotnet/source-build-reference-packages (#11915)
* Update dependency from dotnet/source-build-reference-packages

* Change in SourceBuild dir to trigger SB CI
2021-09-08 08:29:39 -05:00
Dan Seefeldt
290df5778e
[ArPow] Reduce the size of the tarball. (#11662)
* Copy instead of clone and remove .git/objects directories

* Add depth 1 to fetch command

* Add commit count to git-info for installer repo

* Update targets to fix dependency ordering

Some repos take a dependency on PrepareInnerSourceBuildRepoRoot,
assuming that the inner repo will be setup.  The condition for copy
instead of clone caused a conflict.
Keep PrepareInnerSourceBuildRepoRoot and make it depend on
either copy or clone completing.

* Don't copy artifacts dir

* Actually don't remove artifacts directory

* Update prebuilt version
2021-09-01 08:31:15 +00:00
Michael Simons
c61bcfc01f
[ArPow] Update prebuilts and pipeline build agents (#11779) 2021-09-01 04:20:55 +00:00
Michael Simons
f9cc5150e0
[ArPow] Update pipeline to capture prebuild-report (#11688) 2021-08-26 15:54:04 -05:00
Michael Simons
fe0187f55e
[ArPow] Misc changes to reduce tarball build size (#11683) 2021-08-26 16:30:50 +00:00
Michael Simons
3be0fd4f08
Source-build updates to handle internal repo names (#11661) 2021-08-25 01:46:35 +00:00
Chris Rummel
f3ea71b28f
[ArPow] Add NuGet.Client (#11489)
* [main] Update dependencies from dotnet/sdk (#11565)

[main] Update dependencies from dotnet/sdk
- Coherency Updates:
  - Microsoft.AspNetCore.App.Ref: from 6.0.0-rc.1.21417.2 to 6.0.0-rc.1.21417.8 (parent: Microsoft.NET.Sdk)
  - Microsoft.AspNetCore.App.Ref.Internal: from 6.0.0-rc.1.21417.2 to 6.0.0-rc.1.21417.8 (parent: Microsoft.NET.Sdk)
  - Microsoft.AspNetCore.App.Runtime.win-x64: from 6.0.0-rc.1.21417.2 to 6.0.0-rc.1.21417.8 (parent: Microsoft.NET.Sdk)
  - VS.Redist.Common.AspNetCore.SharedFramework.x64.6.0: from 6.0.0-rc.1.21417.2 to 6.0.0-rc.1.21417.8 (parent: Microsoft.NET.Sdk)
  - dotnet-dev-certs: from 6.0.0-rc.1.21417.2 to 6.0.0-rc.1.21417.8 (parent: Microsoft.NET.Sdk)
  - dotnet-user-secrets: from 6.0.0-rc.1.21417.2 to 6.0.0-rc.1.21417.8 (parent: Microsoft.NET.Sdk)

* Update dependencies from https://github.com/dotnet/sdk build 20210817.21 (#11569)

[main] Update dependencies from dotnet/sdk
- Coherency Updates:
  - Microsoft.NET.Test.Sdk: from 17.0.0-preview-20210817-01 to 17.0.0-preview-20210817-02 (parent: Microsoft.NET.Sdk)

* Update dependencies from https://github.com/dotnet/arcade build 20210817.1 (#11570)

[main] Update dependencies from dotnet/arcade
- Coherency Updates:
  - Microsoft.SourceLink.GitHub: from 1.1.0-beta-21309-01 to 1.1.0-beta-21416-05 (parent: Microsoft.DotNet.Arcade.Sdk)
  - Microsoft.DotNet.XliffTasks: from 1.0.0-beta.21412.1 to 1.0.0-beta.21414.1 (parent: Microsoft.DotNet.Arcade.Sdk)

* [ArPow] Add runtime repo to tarball (#11506)

* Add runtime repo / update runtime project

* Update runtime-portable logs and properties

* Create runtime-portable git-info

* Fix unpack of IntermediatePackages

* Re-enable references to runtime

* Pass TargetRid and NonPortable to runtime build

* Only build allconfigurations in non-portable build

* Add SDK patch to disable new CA warning

* Increase built tarball timeout

* Update prebuilt version

* Remove 5.0.3xx and preview 7
Add rc1, 6.0 daily, and move main to 7.0

* Removing 7.0 for now as there aren't builds yet.

* Update dependencies from https://github.com/dotnet/arcade build 20210818.12 (#11594)

[main] Update dependencies from dotnet/arcade
- Coherency Updates:
  - Microsoft.SourceLink.GitHub: from 1.1.0-beta-21416-05 to 1.1.0-beta-21417-01 (parent: Microsoft.DotNet.Arcade.Sdk)
  - Microsoft.DotNet.XliffTasks: from 1.0.0-beta.21414.1 to 1.0.0-beta.21417.1 (parent: Microsoft.DotNet.Arcade.Sdk)

* [ArPow] Add installer repo to tarball (#11483)

* Fix the rc2 aka.ms links

* Add NuGet

* Make safe repo name safer.

* some more props filename stuff

* Patches to get the repo to build

* Use NuGet build script instead of manually calling msbuild.

* Add patch for source-build in build.sh in nuget.

* Use existing 5.0 nuget patch.

* Fixes for build scripts and .NET 6 SDK upgrade.

* Remove obsolete patch - this was caused by not using the correct source-build SDK.

* Address code review comment for source-build build script.

* Restore reference from installer to NuGet.

* Fix patch not applying.

* Don't try to pass Arcade build args to MSBuild.

* Fix script typo.

* Patch NuGet.Frameworks to produce netstandard2.0 for VSTest.

* Did this patch the wrong way around.

* Ignore  failure in no-network case

* UPdate prebuilts version.

Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: Dan Seefeldt <dseefeld@microsoft.com>
Co-authored-by: Marc Paine <marcpop@microsoft.com>
Co-authored-by: Michael Simons <msimons@microsoft.com>
Co-authored-by: Matt Galbraith <mattgal@microsoft.com>
2021-08-24 20:01:06 +00:00
Michael Simons
7c4b93e5b7
[ArPow Tarball] Tier 3 - Add roslyn-analyzers repo to arpow tarball (#11114) 2021-08-24 15:33:44 +00:00
Chris Rummel
f932fe8d77
[ArPow] Merge new repos to release/6.0.1xx (#11620)
* [ArPow] Add runtime repo to tarball (#11506)

* Add runtime repo / update runtime project

* Update runtime-portable logs and properties

* Create runtime-portable git-info

* Fix unpack of IntermediatePackages

* Re-enable references to runtime

* Pass TargetRid and NonPortable to runtime build

* Only build allconfigurations in non-portable build

* Add SDK patch to disable new CA warning

* Increase built tarball timeout

* Update prebuilt version

* [ArPow] Add installer repo to tarball (#11483)

* Ignore  failure in no-network case

Co-authored-by: Dan Seefeldt <dseefeld@microsoft.com>
Co-authored-by: Michael Simons <msimons@microsoft.com>
Co-authored-by: Matt Galbraith <mattgal@microsoft.com>
2021-08-23 22:05:58 +00:00
Matt Mitchell
9994b20baa
Changes for internal runtime build support for .NET 6 (#11422)
- Cherry pick the internal runtime download changes over
- Cherry pick changes to the arcade SB template over (making the next update a noop and unblocking P7+)
- Hoist out the variable groups and parameters needed for the internal download to the top level yaml file.
- Remove the old DownloadFile task and replace this with the Arcade version. Specifically this allows us to remove
  set/use of the DOTNETCLIMSRC_READ_SAS_TOKEN environment variable and instead rely on that task's ability to decode and
  use a base64 encoded SAS token. The reason for the environment variable usage before was that the
  non-encoded SAS token was getting mangled by msbuild/bash/etc. on non-Windows OSs.
- Update the source build tarball template with support for internal runtimes

Update to source build template
2021-08-06 13:54:33 -07:00
Michael Simons
5268979bef
[ArPow] Add offline tarball build infra and CI (#11403)
* Add offline build to CI

* Add sourcelink patches to unblock offline aspnetcore build
2021-08-06 12:50:48 -05:00
Dan Seefeldt
33c0d0250d
[ArPow Tarball] Clean up src directories when building (#11421)
* Clean up src directories when building

* Updates based on PR review comments
2021-08-06 09:12:04 -05:00
Michael Simons
2897e82c96
Add prebuilt acquisition and production build infra (#11417) 2021-08-05 15:19:38 -05:00
Michael Simons
30734f0b61
Add tarball pipeline logic to publish Private.SourceBuilt.Artifacts (#11297) 2021-07-27 16:51:44 +00:00
Dan Seefeldt
702587dcbb
[ArPow] Update SBRP to get version with updated ilasm (#11233)
* Update SBRP to get version with updated ilasm

Also, remove the prebuilts for packages that weren't being built
correctly.

* Add empty prebuilt directory
2021-07-23 16:25:22 -05:00
Chris Rummel
91afacfc09
[ArPow] Temporary fix for reference packages with broken extension methods (#11092)
* Add some needed prebuilt packages to override temporarily bad ref packages.

* Remove bad ref packages.

* Also don't copy new versions of the bad SBRP packages after the SBRP build.

* Account for missing intermediate packages and different paths.

* Address code review comment.
2021-07-09 22:43:30 +00:00
Dan Seefeldt
5048e0f143
[Arpow Tarball] Add flag to build with online sources (#11091)
* Add SBRP packages to prev source-built

* Don't delete SBRP packages

* Add BuildWithOnlineSources flag

* Build tarball in CI with online sources

* Remove OfflineBuild / add BuildWithOnlineSources

* Add --online option to tarball build.sh

* Additional cleanup for tarball build

* Rename baseline file

* Update baseline

* Remove acquire-darc.proj

* Default DotNetBuildOffline to true

* Fix line endings in RemoveInternetSources

* Update src/SourceBuild/Arcade/eng/common/templates/steps/source-build-run-tarball-build.yml

Co-authored-by: Michael Simons <msimons@microsoft.com>

Co-authored-by: Michael Simons <msimons@microsoft.com>
2021-07-09 21:42:42 +00:00
Michael Simons
40235466e9
Fix source-build ci and build infra (#11088) 2021-07-08 20:09:54 -05:00
Michael Simons
addb1b1961
Refactor tarball creation to avoid retrieving all git history (#11029) 2021-07-07 09:24:46 -05:00
Michael Simons
8288ae2e6d
Add source-build tarball build ci (#10995) 2021-06-29 08:33:40 -05:00
Michael Simons
69803dae86
Create tarball file and add tarball creation CI (#10972) 2021-06-28 07:36:57 -05:00
Dan Seefeldt
832010fdf6
Initial checkin of source-build tarball build infrastructure (#10961)
* Initial checkin of source-build tarball build infra

* Add a couple more comments

* Update eng/SourceBuild.Version.Details.xml based on PR review

Co-authored-by: Chris Rummel <crummel@microsoft.com>

* Updates based on PR review comments

Co-authored-by: Chris Rummel <crummel@microsoft.com>
2021-06-24 16:36:04 -05:00