mirrors/dotnet-installer
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

add a spec for the new runtime configuration file

Browse source 
(the file formerly known as "deps")
This commit is contained in:
Andrew Stanton-Nurse 2016-01-07 12:33:31 -08:00
parent 845f2c56e6
commit 03885e876f
2 changed files with 270 additions and 29 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

36
Documentation/specs/corehost.md
View file

@ -29,31 +29,9 @@ Paths in this file are **always** specified using `/`, even on Windows. They mus
This index is loaded when needed during the Resolution Process (see below).
### Dependencies File
### Runtime Configuration File
The `.deps` file is a file located alongside the native host. It is stored in a CSV format, with header, of the following format:
```
LibraryType,LibraryName,LibraryVersion,LibraryHash,AssetType,AssetName,AssetRelativePath,AssetServicable
"Package","Microsoft.CodeAnalysis.Common","1.2.0-beta-20151117-04","sha512-4O3f1iRswbWDz70AGTk0tBA1ivslJUuq37FSx4d0+ZuoBFxNO4oY7ReJgndOJYefaxeTMsCVHSVHJMYOJJphtQ==","runtime","Microsoft.CodeAnalysis","lib/portable-net45+win8/Microsoft.CodeAnalysis.dll","true"
"Package","Microsoft.CodeAnalysis.Common","1.2.0-beta-20151117-04","sha512-4O3f1iRswbWDz70AGTk0tBA1ivslJUuq37FSx4d0+ZuoBFxNO4oY7ReJgndOJYefaxeTMsCVHSVHJMYOJJphtQ==","resources","Microsoft.CodeAnalysis","lib/portable-net45+win8/fr-FR/Microsoft.CodeAnalysis.resources.dll","false"
```
Fields:
* `LibraryType` -> The type of Library that provides the Asset. Currently always `Package` (other Library types include Projects and Framework Assemblies but these aren't relevant to `corehost`)
* `LibraryName` -> The name of the Library that provides the Asset. For a Package, this is the Package ID
* `LibraryVersion` -> The version of the Library that provides the Asset. For a Package, this is the Package Version
* `LibraryHash` -> The hash of the Library that provides the Asset. For a Package, this is a hash of the `.nupkg` file. The format is `[algorithm]-[base64-encoded hash]` where `[algorithm]` must always be `sha512` (in the current version)
* `AssetType` -> The type of the Asset within the Library. `corehost` recognizes two types: `runtime` and `native`, representing Managed Assemblies and Native Libraries respectively. These names are designed to mirror the names in `project.lock.json`.
* `AssetName` -> The base name of the Asset. This is generally just the file name, without the extension (also removing "sub-extensions" like `ni` for native images of managed assemblies)
* `AssetRelativePath` -> The relative path to the Asset file within the root of the package in the packages cache. Entries for localized resources will have the culture specific folder as the last directory in the relative path (i.e. `fr-FR/Foo.resources.dll`). See "Satellite Assemblies" below for more details. As with the servicing file, paths in this file **always** use `/` as a directory separator.
* `AssetServicable` -> Indicates if the asset is servicable. `true` indicates that it is servicable, `false` indicates that it is not
A few notes about the format:
* The current code does not include the header, this is being changed.
* All fields MUST be quoted (for simplicity), even if they don't contain a ','
The runtime configuration file is used to determine settings to apply to the runtime during initialization and for building the TPA and Native Library Search Path lists. See the [spec for the runtime configuration file](runtime-configuration-file.md) for more information.
### Files in the application folder
@ -66,7 +44,7 @@ Only assemblies listed in the dependencies file can be resolved from a package c
* `DOTNET_PACKAGES` - The primary package cache. If not set, defaults to `$HOME/.nuget/packages` on Unix or `%LOCALAPPDATA%\NuGet\Packages` (TBD) on Windows. **NOTE**: Currently the host uses different folders as we are still coordinating with NuGet to get the directories right (there are compat considerations). Currently we always use `$HOME/.dnx/packages`(Unix)/`%USERPROFILE%\.dnx\packages`(Win).
* `DOTNET_PACKAGES_CACHE` - The secondary cache. This is used by shared hosters (such as Azure) to provide a cache of pre-downloaded common packages on a faster disk. If not set, it is not used.
Given the Package ID, Package Version, Package Hash and Asset Relative Path provided in the dependencies file, **and the assembly is not serviced** (see the full resolution algorithm below) resolution proceeds as follows (Unix-style paths will be used for convenience but these variables and paths all apply to Windows as well):
Given the Package ID, Package Version, Package Hash and Asset Relative Path provided in the runtime configuration file, **and the assembly is not serviced** (see the full resolution algorithm below) resolution proceeds as follows (Unix-style paths will be used for convenience but these variables and paths all apply to Windows as well):
1. If `DOTNET_PACKAGES_CACHE` is non-empty, read the file `$DOTNET_PACKAGES_CACHE/[Package ID]/[Package Version]/[Package Id].[Package Version].nupkg.sha512` if present. If the file is present and the content matches the `[Package Hash]` value from the dependencies file. Use that location as the Package Root and go to 3
2. Using `DOTNET_PACKAGES`, or it's default value, use `$DOTNET_PACKAGES/[Package ID]/[Package Version]` as the Package Root
@ -74,19 +52,19 @@ Given the Package ID, Package Version, Package Hash and Asset Relative Path prov
## Assembly Resolution
During host start-up, the host identifies if a `.deps` file is present and loads it. It also scans the files located in the Application Base and determines the assembly name for each (using the file name). It builds a set of assembly names to be loaded by the union of the assembly names listed in `.deps` file and the assemblies located in the Application Base.
During host start-up, the host identifies if a runtime configuration file is present and loads it. It also scans the files located in the Application Base and determines the assembly name for each (using the file name). It builds a set of assembly names to be loaded by the union of the assembly names listed in runtime configuration file and the assemblies located in the Application Base.
A `.deps` file is **not** required to successfully launch an application, but without it, all the dependent assemblies must be located within the same folder as the application. Also, since servicing is performed on the basis of packages, an application without a `.deps` file cannot use the servicing index to override the location of assemblies.
A runtime configuration file is **not** required to successfully launch an application, but without it, all the dependent assemblies must be located within the same folder as the application. Also, since servicing is performed on the basis of packages, an application without a runtime configuration file file cannot use the servicing index to override the location of assemblies.
The host looks for the `.deps` file in the Application Base with the file name `[AssemblyName].deps`. The path to the deps file can be overriden by specifying `--depsfile:{path to deps file}` as the first parameter to the application.
Given the set of assembly names, the host performs the following resolution. In some steps, the Package ID/Version/Relative Path data is required. This is only available if the assembly was listed in the deps file. If the assembly comes from the app-local folder, these resolution steps are skipped.
1. If there is an entry in the servicing index for the Package ID/Version/Relative Path associated with the assembly, the Servicing Root is concatenated with the New Asset Path from the index and used as the Assembly Path. This occurs **even if the assembly is also located app-local**, as long as it is also in the `.deps` file.
1. If there is an entry in the servicing index for the Package ID/Version/Relative Path associated with the assembly, the Servicing Root is concatenated with the New Asset Path from the index and used as the Assembly Path. This occurs **even if the assembly is also located app-local**, as long as it is also in the runtime configuration file.
2. If there is a file in the Application Base with the file name `[AssemblyName].dll`, `[AssemblyName].ni.dll`, `[AssemblyName].exe`, or `[AssemblyName].ni.exe` (in that order), it its full path is used as the Assembly Path
3. The Assembly Path is resolved out of the Package Caches using the algorithm above (in 'Files from package caches').
A similar process is used to produce a list of Native Library Paths. Native libraries are listed in the `.deps` file (marked with a special `native` type token) and searched in the same way as managed assemblies (Servicing, then app-local, then package caches). The main exception is that the app-local file extensions vary by platform (`.dll` on Windows, `.so` on Linux, `.dylib` on Mac OS X)
A similar process is used to produce a list of Native Library Paths. Native libraries are listed in the runtime configuration file (under the `native` asset section) and searched in the same way as managed assemblies (Servicing, then app-local, then package caches). The main exception is that the app-local file extensions vary by platform (`.dll` on Windows, `.so` on Linux, `.dylib` on Mac OS X)
Once a the list of assemblies and native libraries is produced, the host will check for duplicates. If both an `.ni.dll`/`.ni.exe` image and a `.dll`/`.exe` assembly are found for an assembly, the native image will be preferred and the IL-only assembly will be removed. The presence of two different paths for the same assembly name will be considered an error. The managed assemblies are provided in the Trusted Platform Assemblies (TPA) list for the CoreCLR during startup. The folder paths for each native library are deduplicated and provided in the Native Search Paths list for the CoreCLR during startup.

263
Documentation/specs/runtime-configuration-file.md Normal file
View file

@ -0,0 +1,263 @@
# Runtime Configuration Files
The runtime configuration files store the dependencies of an application (formerly stored in the `.deps` file). They also include runtime configuration options, such as the Garbage Collector mode. Optionally they can also include data for runtime compilation (compilation settings used to compile the original application, and reference assemblies used by the application).
**Note:** This document doesn't provide full explanations as to why individual items are needed in this file. That is covered in the [`corehost` spec](corehost.md) and via the `Microsoft.Extensions.DependencyModel` assembly.
## What produces the files and where are they?
There are two runtime configuration files for a particular application. Given a project named `MyApp`, the compilation process produces the following files (on Windows, other platforms are similar):
* `MyApp.dll` - The managed assembly for `MyApp`, including an ECMA-compliant entry point token.
* `MyApp.exe` - A copy of the `corehost.exe` executable.
* `MyApp.config.json` - An **optional** configuration file containing runtime configuration settings.
* `MyApp.deps.json` - A list of dependencies, as well as compilation context data and compilation dependencies. Not technically required, but required to use the servicing or package cache/shared package install features.
## File format
The files are both JSON files stored in UTF-8 encoding. Below are sample files. Note that not all sections are required and some will be opt-in only (see below for more details). The `.config.json` file is completely optional, and in the `.deps.json` file, only the `runtimeTarget`, `targets` and `libraries` sections are required (and within the `targets` section, only the runtime-specific target is required).
### [appname].config.json
```json
{
"runtimeConfig": {
"gcServer": true,
"gcConcurrent": false
}
}
```
### [appname].deps.json
```json
{
"runtimeTarget": "DNXCore,Version=v5.0/osx.10.10-x64",
"compilationOptions": {
"defines": [ "DEBUG" ]
},
"targets": {
"DNXCore,Version=v5.0": {
"MyApp/1.0": {
"type": "project",
"dependencies": {
"AspNet.Mvc": "1.0.0"
}
},
"System.Foo/1.0.0": {
"type": "package",
},
"System.Banana/1.0.0": {
"type": "package",
"dependencies": {
"System.Foo": "1.0.0"
},
"compile": {
"ref/dotnet5.4/System.Banana.dll": { }
}
}
},
"DNXCore,Version=v5.0/osx.10.10-x64": {
"MyApp/1.0": {
"type": "project",
"dependencies": {
"AspNet.Mvc": "1.0.0"
}
},
"System.Foo/1.0.0": {
"type": "package",
"runtime": {
"lib/dnxcore50/System.Foo.dll": { }
}
},
"System.Banana/1.0.0": {
"type": "package",
"dependencies": {
"System.Foo": "1.0.0"
},
"runtime": {
"lib/dnxcore50/System.Banana.dll": { }
},
"resources": {
"lib/dnxcore50/fr-FR/System.Banana.resources.dll": { "locale": "fr-FR" }
},
"native": {
"runtimes/osx.10.10-x64/native/libbananahelper.dylib": { }
}
}
}
},
"libraries": {
"MyApp/1.0": {
"type": "project"
},
"System.Foo/1.0": {
"type": "package",
"serviceable": true,
"sha512": "[base64 string]"
},
"System.Banana/1.0": {
"type": "package",
"sha512": "[base64 string]"
}
}
}
```
## Sections
### `runtimeConfig` Section (`.config.json`)
This section is copied verbatim from an identical section in the input `project.json` file (with the exception of the `target` parameter which is generated by the compilation process). The `runtimeConfig` section specifies parameters to be provided to the runtime during initialization. Known parameters include:
* `gcServer` - Boolean indicating if the server GC should be used (Default: _TBD_). Note: This is designed to mirror the existing [app.config](https://msdn.microsoft.com/en-us/library/ms229357.aspx) setting)
* `gcConcurrent` - Boolean indicating if background garbage collection should be used (Default: _TBD_). Note: This is designed to mirror the existing [app.config](https://msdn.microsoft.com/en-us/library/yhwwzef8.aspx) setting).
* Others _TBD_
These settings are read by `corehost` to determine how to initialize the runtime. All versions of `corehost` **must ignore** settings in this section that they do not understand (thus allowing new settings to be added in later versions).
### `compilationOptions` Section (`.deps.json`)
This section is copied by storing the merged `compilationOptions` from the input `project.json`. The `project.json` can define three sets of compilation options: Global, Per-Configuration, and Per-Framework. However, the `runtime.config.json` is specific to a configuration and framework so there is only one merged section here.
The exact settings found here are specific to the compiler that produced the original application binary. Some example settings include: `defines`, `languageVersion` (C#/VB), `allowUnsafe` (C#), etc.
As an example, here is a possible `project.json` file:
```json
{
"compilationOptions": {
"allowUnsafe": true
},
"frameworks": {
"net451": {
"compilationOptions": {
"defines": [ "DESKTOP_CLR" ]
}
},
"dnxcore50": {
"compilationOptions": {
"defines": [ "CORE_CLR" ]
}
}
},
"configurations": {
"Debug": {
"compilationOptions": {
"defines": [ "DEBUG_MODE" ]
}
}
}
}
```
When this project is built for `dnxcore50` in the `Debug` configuration, the outputted `MyApp.deps.json` file will have the following `compilationOptions` section:
```json
{
"compilationOptions": {
"allowUnsafe": true,
"defines": [ "CORE_CLR", "DEBUG_MODE" ]
}
}
```
### `runtimeTarget` property (`.deps.json`)
This property contains the name of the target from `targets` that should be used by the runtime. This is present to simplify `corehost` so that it does not have to parse or understand target names.
### `targets` Section (`.deps.json`)
This section contains subsetted data from the input `project.lock.json`.
Each property under `targets` describes a "target", which is a collection of libraries required by the application when run or compiled in a certain framework and platform context. A target **must** specify a Framework name, and **may** specify a Runtime Identifier. Targets without Runtime Identifiers represent the dependencies and assets used for compiling the application for a particular framework. Targets with Runtime Identifiers represent the dependencies and assets used for running the application under a particular framework and on the platform defined by the Runtime Identifier. In the example above, the `DNXCore,Version=v5.0` target lists the dependencies and assets used to compile the application for `dnxcore50`, and the `DNXCore,Version=v5.0/osx.10.10-x64` target lists the dependencies and assets used to run the application on `dnxcore50` on a 64-bit Mac OS X 10.10 machine.
There will always be two targets in the `runtime.config.json` file: A compilation target, and a runtime target. The compilation target will be named with the framework name used for the compilation (`DNXCore,Version=v5.0` in the example above). The runtime target will be named with the framework name and runtime identifier used to execute the application (`DNXCore,Version=v5.0/osx.10.10-x64` in the example above). However, the runtime target will also be identified by name in the `runtimeOptions` section, so that `corehost` need not parse and understand target names.
The content of each target property in the JSON is a JSON object. Each property of that JSON object represents a single dependency required by the application when compiled for/run on that target. The name of the property contains the ID and Version of the dependency in the form `[Id]/[Version]`. The content of the property is another JSON object containing metadata about the dependency.
The `type` property of a dependency object defines what kind of entity satisfied the dependency. Possible values include `project` and `package` (further comments on dependency types below).
**Open Question:** `type` is also present in the `libraries` section. We don't really need it in both. It's in both now because the lock file does that and we want the formats to be similar. Should we remove it?
The `dependencies` property of a dependency object defines the ID and Version of direct dependencies of this node. It is a JSON object where the property names are the ID of the dependency and the content of each property is the Version of the dependency.
The `runtime` property of a dependency object lists the relative paths to Managed Assemblies required to be available at runtime in order to satisfy this dependency. The paths are relative to the location of the Dependency (see below for further details on locating a dependency).
The `resources` property of a dependency object lists the relative paths and locales of Managed Satellite Assemblies which provide resources for other languages. Each item contains a `locale` property specifying the [IETF Language Tag](https://en.wikipedia.org/wiki/IETF_language_tag) for the satellite assembly (or more specifically, a value usable in the Culture field for a CLR Assembly Name).
The `native` property of a dependency object lists the relative paths to Native Libraries required to be available at runtime in order to satisfy this dependency. The paths are relative to the location of the Dependency (see below for further details on locating a dependency).
In compilation targets, the `runtime`, `resources` and `native` properties of a dependency are omitted, because they are not relevant to compilation. Similarly, in runtime targets, the `compile` property is omitted, because it is not relevant to runtime.
Only dependencies with a `type` value of `package` will have asset lists (`compile`, `runtime`, `resources`, `native`). Dependencies which are satisfied by projects will have all of their assets copied to the output directory, so they will not be listed in this file.
### `libraries` Section (`.deps.json`)
This section contains a union of all the dependencies found in the various targets, and contains common metadata for them. Specifically, it contains the `type`, as well as a boolean indicating if the library can be serviced (`servicable`, only for `package`-typed libraries) and a SHA-512 hash of the package file (`sha512`, only for `package`-typed libraries.
**Open Question**: We could probably exclude projects from this set in order to reduce duplication. The main reason this is a separate section is because that's how the lock file is formatted and we want to try an keep this format the same if possible.
## How the file is used
The file is read by two different components:
* `corehost` uses it to determine what to place on the TPA and Native Library Search Path lists, as well as what runtime settings to apply (GC type, etc.). See [the `corehost` spec](corehost.md).
* `Microsoft.Extensions.DependencyModel` uses it to allow a running managed application to query various data about it's dependencies. For example:
* To find all dependencies that depend on a particular package (used by ASP.NET MVC and other plugin-based systems to identify assemblies that should be searched for possible plugin implementations)
* To determine the reference assemblies used by the application when it was compiled in order to allow runtime compilation to use the same reference assemblies (used by ASP.NET Razor to compile views)
* To determine the compilation settings used by the application in order to allow runtime compilation to use the same settings (also used by ASP.NET Razor views).
## Opt-In Compilation Data
Some of the sections in the `.deps.json` file contain data used for runtime compilation. This data is not provided in the file by default. Instead, a project.json setting `preserveCompilationContext` must be set to true in order to ensure this data is added. Without this setting, the `compilationOptions` will not be present in the file, and the `targets` section will contain only the runtime target. For example, if the `preserveCompilationContext` setting was not present in the `project.json` that generated the above example, the `.deps.json` file would only contain the following content:
```json
{
"targets": {
"DNXCore,Version=v5.0/osx.10.10-x64": {
"MyApp/1.0": {
"type": "project",
"dependencies": {
"AspNet.Mvc": "1.0.0"
}
},
"System.Foo/1.0.0": {
"type": "package",
"runtime": {
"lib/dnxcore50/System.Foo.dll": { }
}
},
"System.Banana/1.0.0": {
"type": "package",
"dependencies": {
"System.Foo": "1.0.0"
},
"runtime": {
"lib/dnxcore50/System.Banana.dll": { }
},
"resources": {
"lib/dnxcore50/fr-FR/System.Banana.resources.dll": { "locale": "fr-FR" }
},
"native": {
"runtimes/osx.10.10-x64/native/libbananahelper.dylib": { }
}
}
}
},
"libraries": {
"MyApp/1.0": {
"type": "project"
},
"System.Foo/1.0": {
"type": "package",
"serviceable": true,
"sha512": "[base64 string]"
},
"System.Banana/1.0": {
"type": "package",
"sha512": "[base64 string]"
}
}
}
```