8dc41755cb
* Switch repo to use native AzDO container support
This is a major refactor of the YAML used by the installer repo. The goal is to utilize the native container support that AzDO provides, rather than building containers on the fly and issuing commands using custom infrastructure. To do this, the YAML requires a bit of a refactor. The matrix strategy approach used by the repo to build a ton of different OS's does not work with containers, because a matrix strategy only changes the variables available to each build command. It cannot change the AzDO host environment. In order to resolve this, I refactored build.yml to take and use optional container names. In the process of doing this, I discovered a number of other things about the old YAML that just happened to 'accidentally' work and fixed them or did general cleanup. Including:
- This construct, used in build.yml, does not work as you might expect. If the parameter is not declared as a boolean (or not declared at all), this evaluates to "does this parameter exist", not "is it true":
```
${{ if parameters.pgoInstrument }}:
```
- I fully specified all the parameters and their types to avoid issues in the future.
- Build pool selection was moved to build.yml
- Removed some unused parameters.
- There was a **very** subtle indentation change here: 762d2966ee/.vsts-ci.yml (L275-L281). This meant that this leg ran in both PR and official builds. I have no idea whether this was the intention or not, but I kept it this way and reorganized the file.
**One change of note:** One of the upsides of the original matrix based approach is that job dependencies are simple. The jobs generated by the matrix are referred to in `dependsOn` lists only by the original job that contains the matrix. That keeps the dependsOn list small even if the number of jobs generated is large. Installer has a large set of independently addressable jobs now. Normally, we would solve this by using the arcade jobs template. The jobs template takes a set of jobs and automatically adds dependsOn for the publishing jobs. BUT, AzDO does not allow templates to be passed as parameters to other templates. This would mean that you couldn't use the build.yml template in conjunction with the jobs template and would have to list all the installer jobs as dependencies. This list would be hard to keep up to date. To avoid this, I used a new feature of publishing (`publishAssetsImmediately`) which uses the Publish To Build Asset Registry job to do the actual publishing call, and put it in a separate phase, then eliminated the post-build.yml call. This means that the publishing stage depends on all jobs in the build stage, and does not need to address them individually. Eliminating the post-build.yml stage may seem odd, but this is what actually happens when `publishAssetsImmediately` is set to true anyway.
367 lines
No EOL
12 KiB
YAML
367 lines
No EOL
12 KiB
YAML
trigger:
|
|
batch: true
|
|
branches:
|
|
include:
|
|
- main
|
|
- master
|
|
- release/*
|
|
- internal/release/3.*
|
|
- internal/release/5.*
|
|
- internal/release/6.*
|
|
- internal/release/7.*
|
|
|
|
variables:
|
|
- name: _PublishUsingPipelines
|
|
value: false
|
|
- ${{ if or(startswith(variables['Build.SourceBranch'], 'refs/heads/release/'), startswith(variables['Build.SourceBranch'], 'refs/heads/internal/release/'), eq(variables['Build.Reason'], 'Manual')) }}:
|
|
- name: PostBuildSign
|
|
value: false
|
|
- ${{ else }}:
|
|
- name: PostBuildSign
|
|
value: true
|
|
- ${{ if and(ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
|
|
- group: DotNet-DotNetCli-Storage
|
|
- group: DotNet-Installer-SDLValidation-Params
|
|
- name: _PublishUsingPipelines
|
|
value: true
|
|
|
|
- name: _InternalRuntimeDownloadArgs
|
|
value: ''
|
|
|
|
- ${{ if eq(variables['System.TeamProject'], 'internal') }}:
|
|
- group: DotNetBuilds storage account read tokens
|
|
- name: _InternalRuntimeDownloadArgs
|
|
value: /p:DotNetRuntimeSourceFeed=https://dotnetbuilds.blob.core.windows.net/internal
|
|
/p:DotNetRuntimeSourceFeedKey=$(dotnetbuilds-internal-container-read-token-base64)
|
|
/p:dotnetbuilds-internal-container-read-token-base64=$(dotnetbuilds-internal-container-read-token-base64)
|
|
|
|
stages:
|
|
- stage: Build
|
|
jobs:
|
|
# This job is for build retry configuration.
|
|
- job: Publish_Build_Configuration
|
|
pool:
|
|
${{ if eq(variables['System.TeamProject'], 'public') }}:
|
|
name: NetCore-Public
|
|
demands: ImageOverride -equals Build.Windows.Amd64.VS2022.Pre.Open
|
|
${{ if eq(variables['System.TeamProject'], 'internal') }}:
|
|
name: NetCore1ESPool-Internal
|
|
demands: ImageOverride -equals Build.Windows.Amd64.VS2022.Pre
|
|
steps:
|
|
- publish: $(Build.SourcesDirectory)\eng\buildConfiguration
|
|
artifact: buildConfiguration
|
|
displayName: Publish Build Config
|
|
|
|
## PR-only jobs
|
|
|
|
- ${{ if or(eq(variables['System.TeamProject'], 'public'), in(variables['Build.Reason'], 'PullRequest')) }}:
|
|
|
|
## Windows
|
|
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Windows_NT
|
|
jobName: Build_Debug_x64
|
|
buildConfiguration: Debug
|
|
buildArchitecture: x64
|
|
additionalBuildParameters: '/p:PublishInternalAsset=true'
|
|
runTests: true
|
|
|
|
## Linux
|
|
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Linux
|
|
jobName: Build_Ubuntu_18_04_Debug_x64
|
|
container: 'mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-20220916154732-3c53da6'
|
|
buildConfiguration: Debug
|
|
buildArchitecture: x64
|
|
linuxPortable: true
|
|
runTests: true
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Linux
|
|
jobName: Build_Fedora_36_Debug_x64
|
|
container: 'mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-36-20220912173100-a09384f'
|
|
buildConfiguration: Debug
|
|
buildArchitecture: x64
|
|
linuxPortable: true
|
|
runTests: true
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Linux
|
|
jobName: Build_CentOS_7_Debug_x64
|
|
container: 'mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-20220912172913-d16db59'
|
|
buildConfiguration: Debug
|
|
buildArchitecture: x64
|
|
linuxPortable: false
|
|
runTests: true
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Linux
|
|
jobName: Build_Debian_Stretch_Debug_x64
|
|
container: 'mcr.microsoft.com/dotnet-buildtools/prereqs:debian-stretch-20220912173009-94fc78a'
|
|
buildConfiguration: Debug
|
|
buildArchitecture: x64
|
|
additionalBuildParameters: '/p:BuildSdkDeb=true'
|
|
linuxPortable: false
|
|
runTests: true
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Linux
|
|
jobName: Build_Arm64_Debug
|
|
buildConfiguration: Debug
|
|
buildArchitecture: arm64
|
|
runtimeIdentifier: 'linux-arm64'
|
|
linuxPortable: true
|
|
# Never run tests on arm64
|
|
runTests: false
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Linux
|
|
jobName: Build_Linux_musl_Debug_x64
|
|
container: 'mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-WithNode-20220916182008-f0ea7ba'
|
|
buildConfiguration: Debug
|
|
buildArchitecture: x64
|
|
runtimeIdentifier: 'linux-musl-x64'
|
|
# Pass in HostOSName when running on alpine
|
|
additionalBuildParameters: '/p:HostOSName="linux-musl"'
|
|
linuxPortable: false
|
|
runTests: true
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Linux
|
|
jobName: Build_LinuxPortable_Release_x64
|
|
buildConfiguration: Release
|
|
buildArchitecture: x64
|
|
linuxPortable: true
|
|
runTests: true
|
|
|
|
# MacOS
|
|
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Darwin
|
|
jobName: Build_Release_x64
|
|
buildConfiguration: Release
|
|
buildArchitecture: x64
|
|
runTests: true
|
|
|
|
## Official/PGO instrumentation Builds
|
|
|
|
- ${{ if and(ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
|
|
|
|
## Windows
|
|
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Windows_NT
|
|
jobName: Build_Release_x64
|
|
buildConfiguration: Release
|
|
buildArchitecture: x64
|
|
additionalBuildParameters: '/p:PublishInternalAsset=true'
|
|
runTests: false
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Windows_NT
|
|
jobName: Build_Release_x86
|
|
buildConfiguration: Release
|
|
buildArchitecture: x86
|
|
runTests: false
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Windows_NT
|
|
jobName: Build_Release_arm64
|
|
buildConfiguration: Release
|
|
buildArchitecture: arm64
|
|
runTests: false
|
|
|
|
## Linux
|
|
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Linux
|
|
jobName: Build_Arm_Release
|
|
buildConfiguration: Release
|
|
buildArchitecture: arm
|
|
runtimeIdentifier: 'linux-arm'
|
|
linuxPortable: true
|
|
runTests: false
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Linux
|
|
jobName: Build_Arm64_Release
|
|
buildConfiguration: Release
|
|
buildArchitecture: arm64
|
|
runtimeIdentifier: 'linux-arm64'
|
|
linuxPortable: true
|
|
runTests: false
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Linux
|
|
jobName: Build_Linux_musl_Release_arm
|
|
container: 'mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-cross-20220916154619-56ef508'
|
|
buildConfiguration: Release
|
|
buildArchitecture: arm
|
|
runtimeIdentifier: 'linux-musl-arm'
|
|
additionalBuildParameters: '/p:OSName="linux-musl"'
|
|
linuxPortable: false
|
|
runTests: false
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Linux
|
|
jobName: Build_Linux_musl_Release_arm64
|
|
buildConfiguration: Release
|
|
buildArchitecture: arm64
|
|
runtimeIdentifier: 'linux-musl-arm64'
|
|
additionalBuildParameters: '/p:OSName="linux-musl"'
|
|
linuxPortable: false
|
|
runTests: false
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Linux
|
|
jobName: Build_Linux_musl_Release_x64
|
|
container: 'mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-WithNode-20220916182008-f0ea7ba'
|
|
buildConfiguration: Release
|
|
buildArchitecture: x64
|
|
runtimeIdentifier: 'linux-musl-x64'
|
|
# Pass in HostOSName when running on alpine
|
|
additionalBuildParameters: '/p:HostOSName="linux-musl"'
|
|
linuxPortable: false
|
|
runTests: false
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Linux
|
|
jobName: Build_Linux_Portable_Deb_Release_x64
|
|
container: 'mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-debpkg-20220916154732-cfdd435'
|
|
buildConfiguration: Release
|
|
buildArchitecture: x64
|
|
# Do not publish zips and tarballs. The linux-x64 binaries are
|
|
# already published by Build_LinuxPortable_Release_x64
|
|
additionalBuildParameters: '/p:PublishBinariesAndBadge=false /p:BuildSdkDeb=true'
|
|
linuxPortable: true
|
|
runTests: false
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Linux
|
|
jobName: Build_Linux_Portable_Rpm_Release_x64
|
|
container: 'mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-rpmpkg-20220912172913-d0fa36f'
|
|
buildConfiguration: Release
|
|
buildArchitecture: x64
|
|
# Do not publish zips and tarballs. The linux-x64 binaries are
|
|
# already published by Build_LinuxPortable_Release_x64
|
|
additionalBuildParameters: '/p:PublishBinariesAndBadge=false'
|
|
linuxPortable: true
|
|
runTests: false
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Linux
|
|
jobName: Build_Linux_Portable_Rpm_Release_Arm64
|
|
container: 'mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-rpmpkg-20220912172913-d0fa36f'
|
|
buildConfiguration: Release
|
|
buildArchitecture: arm64
|
|
runtimeIdentifier: 'linux-arm64'
|
|
# Do not publish zips and tarballs. The linux-x64 binaries are
|
|
# already published by Build_LinuxPortable_Release_x64
|
|
additionalBuildParameters: '/p:PublishBinariesAndBadge=false /p:CLIBUILD_SKIP_TESTS=true'
|
|
linuxPortable: true
|
|
runTests: false
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Linux
|
|
jobName: Build_LinuxPortable_Release_x64
|
|
buildConfiguration: Release
|
|
buildArchitecture: x64
|
|
linuxPortable: true
|
|
runTests: false
|
|
|
|
# MacOS
|
|
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Darwin
|
|
jobName: Build_Release_x64
|
|
buildConfiguration: Release
|
|
buildArchitecture: x64
|
|
runTests: false
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Darwin
|
|
jobName: Build_Release_arm64
|
|
runtimeIdentifier: 'osx-arm64'
|
|
buildConfiguration: Release
|
|
buildArchitecture: arm64
|
|
runTests: false
|
|
|
|
## Windows PGO Instrumentation builds
|
|
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Windows_NT
|
|
pgoInstrument: true
|
|
jobName: Build_Release_x64
|
|
buildConfiguration: Release
|
|
buildArchitecture: x64
|
|
additionalBuildParameters: '/p:PublishInternalAsset=true'
|
|
runTests: false
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Windows_NT
|
|
pgoInstrument: true
|
|
jobName: Build_Release_x86
|
|
buildConfiguration: Release
|
|
buildArchitecture: x86
|
|
runTests: false
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Windows_NT
|
|
pgoInstrument: true
|
|
jobName: Build_Release_arm64
|
|
buildConfiguration: Release
|
|
buildArchitecture: arm64
|
|
runTests: false
|
|
|
|
## Linux PGO Instrumentation builds
|
|
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Linux
|
|
pgoInstrument: true
|
|
jobName: Build_LinuxPortable_Release_x64
|
|
buildConfiguration: Release
|
|
buildArchitecture: x64
|
|
linuxPortable: true
|
|
runTests: false
|
|
|
|
- template: /eng/build.yml
|
|
parameters:
|
|
agentOs: Linux
|
|
pgoInstrument: true
|
|
jobName: Build_Release_arm64
|
|
buildConfiguration: Release
|
|
buildArchitecture: arm64
|
|
linuxPortable: true
|
|
runTests: false
|
|
|
|
- template: /eng/common/templates/jobs/source-build.yml
|
|
|
|
- template: /src/SourceBuild/Arcade/eng/common/templates/job/source-build-create-tarball.yml
|
|
|
|
- template: /src/SourceBuild/Arcade/eng/common/templates/job/source-build-run-tarball-build.yml
|
|
parameters:
|
|
dependsOn: Source_Build_Create_Tarball
|
|
condition: eq(dependencies.Source_Build_Create_Tarball.outputs['Tarball_Build_Check._includeTarballBuild'], 'true')
|
|
|
|
- ${{ if and(ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
|
|
- stage: Publish
|
|
dependsOn:
|
|
- Build
|
|
jobs:
|
|
- template: /eng/common/templates/job/publish-build-assets.yml
|
|
parameters:
|
|
publishUsingPipelines: true
|
|
publishAssetsImmediately: true
|
|
pool:
|
|
${{ if eq(variables['System.TeamProject'], 'internal') }}:
|
|
name: NetCore1ESPool-Internal
|
|
demands: ImageOverride -equals Build.Windows.Amd64.VS2022 |