mirrors/electron
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge pull request #11240 from electron/upgrading_docs

Browse source 
Upgrading docs for node, crashpad, and chromium
This commit is contained in:
shelley vohr 2017-11-24 11:16:18 +01:00 committed by GitHub
commit 00999a07d8
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 338 additions and 171 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

171
docs/development/upgrading-chrome.md
View file

@ -1,171 +0,0 @@
# Upgrading Chromium Workflow
This document is meant to serve as an overview of what steps are needed
on each Chromium upgrade in Electron.
- Upgrade libcc to a new Chromium version
- Make Electron code compatible with the new libcc
- Update Electron dependencies (crashpad, NodeJS, etc.) if needed
- Make internal builds of libcc and electron
- Update Electron docs if necessary
## Upgrade `libcc` to a new Chromium
### Steps
### 1. Get the code and initialize the project:
```sh
$ git clone git@github.com:electron/libchromiumcontent.git
$ cd libchromiumcontent
$ ./script/bootstrap -v
```
### 2. Find the new beta/stable Chromium version from [OmahaProxy](https://omahaproxy.appspot.com/).
### 3. Put it into the `libchromiumcontent/VERSION` file, then run `$ ./script/update`
- It will probably fail applying patches.
### 4. Fix `*.patch` files in the `/patches` and `/patches-mas` folders.
### 5. (Optional) Run a separate script to apply patches (`script/update` uses it internally):
```sh
$ ./script/apply-patches
```
- There is also another script `/script/patch.py` that could be more useful, check `--help` to learn how it works with `$ ./script/patch.py -h`
### 6. Run the build when all patches can be applied without errors
```sh
$ ./script/build
```
- If some patches are no longer compatible with the Chromium code, fix compilation errors.
### 7. When build succeeds, create a `dist` for Electron
`$ ./script/create-dist --no_zip`
- It will create `dist/main` folder in the root of the libcc repo, you will need it to build Electron.
### 8. (Optional) Update script contents if there are errors resultant of some files being removed or renamed. (`--no_zip` prevents script from create `dist` archives, you don't need them.)
## Update Electron Code
### Steps
### 1. Get the code:
```sh
$ git clone git@github.com:electron/electron.git
$ cd electron
```
### 2. If you already have libcc built on you machine in its own repo, you need to tell Electron explicitly to use it:
```sh
$ ./script/bootstrap.py -v \
--libcc_source_path <libcc_folder>/src \
--libcc_shared_library_path <libcc_folder>/shared_library \
--libcc_static_library_path <libcc_folder>/static_library
```
- If you haven't yet built libcc but it's already supposed to be upgraded to a new Chromium, bootstrap Electron as usual
`$ ./script/bootstrap.py -v`
- Ensure that libcc submodule (`vendor/libchromiumcontent`) points to a right revision
### 3. Set CLANG_REVISION in` script/update-clang.sh` to match the version Chromium is using.
- Located in `electron/libchromiumcontent/src/tools/clang/scripts/update.py`
### 4. Checkout Chromium if you haven't already: https://chromium.googlesource.com/chromium/src.git/+/{VERSION}/tools/clang/scripts/update.py
- (Replace the `{VERSION}` placeholder in the url above to the Chromium version libcc uses.)
### 5. Build Electron.
- Try to build Debug version first: `$ ./script/build.py -c D`
- You will need it to run tests
### 6. Fix compilation and linking errors
### 7. Ensure that Release build can be built too: `$ ./script/build.py -c R`
- Often the Release build will have different linking errors that you'll need to fix.
- Some compilation and linking errors are caused by missing source/object files in the libcc `dist`
### 8. Update `./script/create-dist` in the libcc repo, recreate a `dist`, and run Electron bootstrap script once again.
### Tips for fixing compilation errors
- Fix build config errors first
- Fix fatal errors first, like missing files and errors related to compiler flags or defines
- Try to identify complex errors as soon as possible.
- Ask for help if you're not sure how to fix them
- Disable all Electron features, fix the build, then enable them one by one
- Add more build flags to disable features in build-time.
When a Debug build of Electron succeeds, run the tests:
`$ ./script/test.py`
Fix the failing tests.
Follow all the steps above to fix Electron code on all supported platforms.
## Update Crashpad
- Electron's crashpad fork: https://github.com/electron/crashpad
- Primary crashpad repo: https://chromium.googlesource.com/crashpad/crashpad/
### Steps
If there are any compilation errors related to the Crashpad, it probably means you need to update the fork to a newer revision:
### 1. Clone Electron's fork of the Crashpad, add the main repo as another remote:
```sh
$ git clone https://github.com/electron/crashpad && cd crashpad
$ git remote add upstream https://chromium.googlesource.com/crashpad/crashpad/ && git fetch upstream
```
### 2. Find a revision Chromium uses in src/third_party/crashpad/README.chromium
### 3. Rebase the master branch to that REVISION
```sh
$ git rebase REVISION
```
- If there are nontrivial conflicts during rebase, consult with the commit's author.
### 4. Add a tag with the next Electron version to a top commit in the master branch
- e.g. if current Electron version is `1.11.*`, then use tag `electron-1.12.0`, because it will be used in the next version.
### 5. Ensure Electron-specific commits in the master branch on GitHub are properly tagged
- If they aren't, force push will make them orphans and they'll be lost.
### 6. Force push to Electron's crashpad fork.
### 7. Update `/vendor/crashpad` submodule in the Electron's upgrade branch to point to the updated crashpad.
## Update NodeJS
- Upgrade `vendor/node` to the Node release that corresponds to the v8 version
being used in the new Chromium release. See the v8 versions in Node on
https://nodejs.org/en/download/releases for more details.
- You can find v8 version Chromium is using on [OmahaProxy](http://omahaproxy.appspot.com).
If it's not available check `v8/include/v8-version.h`
in the Chromium checkout.
## Verify ffmpeg Support
Electron ships with a version of `ffmpeg` that includes proprietary codecs by
default. A version without these codecs is built and distributed with each
release as well. Each Chrome upgrade should verify that switching this version is
still supported.
You can verify Electron's support for multiple `ffmpeg` builds by loading the
following page. It should work with the default `ffmpeg` library distributed
with Electron and not work with the `ffmpeg` library built without proprietary
codecs.
```html
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Proprietary Codec Check</title>
</head>
<body>
<p>Checking if Electron is using proprietary codecs by loading video from http://www.quirksmode.org/html5/videos/big_buck_bunny.mp4</p>
<p id="outcome"></p>
<video style="display:none" src="http://www.quirksmode.org/html5/videos/big_buck_bunny.mp4" autoplay></video>
<script>
const video = document.querySelector('video')
video.addEventListener('error', ({target}) => {
if (target.error.code === target.error.MEDIA_ERR_SRC_NOT_SUPPORTED) {
document.querySelector('#outcome').textContent = 'Not using proprietary codecs, video emitted source not supported error event.'
} else {
document.querySelector('#outcome').textContent = `Unexpected error: ${target.error.code}`
}
})
video.addEventListener('playing', () => {
document.querySelector('#outcome').textContent = 'Using proprietary codecs, video started playing.'
})
</script>
</body>
</html>
```
## Useful Links
- [Chrome Release Schedule](https://www.chromium.org/developers/calendar)
- [OmahaProxy](http://omahaproxy.appspot.com)
- [Chromium Issue Tracker](https://bugs.chromium.org/p/chromium)

136
docs/development/upgrading-chromium.md Normal file
View file

@ -0,0 +1,136 @@
# Upgrading Chromium
This document is meant to serve as an overview of what steps are needed
on each Chromium upgrade in Electron.
- Upgrade libcc to a new Chromium version
- Make Electron code compatible with the new libcc
- Update Electron dependencies (crashpad, NodeJS, etc.) if needed
- Make internal builds of libcc and electron
- Update Electron docs if necessary
## Upgrade `libcc` to a new Chromium
1. Get the code and initialize the project:
- ```sh
$ git clone git@github.com:electron/libchromiumcontent.git
$ cd libchromiumcontent
$ ./script/bootstrap -v```
2. Find the new beta/stable Chromium version from [OmahaProxy](https://omahaproxy.appspot.com/).
3. Put it into the `libchromiumcontent/VERSION` file, then run `$ ./script/update`
- It will probably fail applying patches.
4. Fix `*.patch` files in the `/patches` and `/patches-mas` folders.
5. (Optional) Run a separate script to apply patches (`script/update` uses it internally):
- `$ ./script/apply-patches`
- There is also another script `/script/patch.py` that could be more useful
- Check `--help` to learn how it works with `$ ./script/patch.py -h`
6. Run the build when all patches can be applied without errors
- `$ ./script/build`
- If some patches are no longer compatible with the Chromium code, fix compilation errors.
7. When build succeeds, create a `dist` for Electron
- `$ ./script/create-dist --no_zip`
- It will create `dist/main` folder in the root of the libcc repo
- You will need it to build Electron.
8. (Optional) Update script contents if there are errors resultant of some files being removed or renamed. (`--no_zip` prevents script from create `dist` archives, you don't need them.)
## Update Electron Code
1. Get the code:
- ```sh
$ git clone git@github.com:electron/electron.git
$ cd electron
```
2. If you already have libcc built on you machine in its own repo, you need to tell Electron explicitly to use it:
- ```sh
$ ./script/bootstrap.py -v \
--libcc_source_path <libcc_folder>/src \
--libcc_shared_library_path <libcc_folder>/shared_library \
--libcc_static_library_path <libcc_folder>/static_library
```
- If you haven't yet built libcc but it's already supposed to be upgraded to a new Chromium, bootstrap Electron as usual
`$ ./script/bootstrap.py -v`
- Ensure that libcc submodule (`vendor/libchromiumcontent`) points to a right revision
3. Set CLANG_REVISION in` script/update-clang.sh` to match the version Chromium is using.
- Located in `electron/libchromiumcontent/src/tools/clang/scripts/update.py`
4. Checkout Chromium if you haven't already:
- https://chromium.googlesource.com/chromium/src.git/+/{VERSION}/tools/clang/scripts/update.py
- (Replace the `{VERSION}` placeholder in the url above to the Chromium version libcc uses.)
5. Build Electron.
- Try to build Debug version first: `$ ./script/build.py -c D`
- You will need it to run tests
6. Fix compilation and linking errors
7. Ensure that Release build can be built too
- `$ ./script/build.py -c R`
- Often the Release build will have different linking errors that you'll need to fix.
- Some compilation and linking errors are caused by missing source/object files in the libcc `dist`
8. Update `./script/create-dist` in the libcc repo, recreate a `dist`, and run Electron bootstrap script once again.
### Tips for fixing compilation errors
- Fix build config errors first
- Fix fatal errors first, like missing files and errors related to compiler flags or defines
- Try to identify complex errors as soon as possible.
- Ask for help if you're not sure how to fix them
- Disable all Electron features, fix the build, then enable them one by one
- Add more build flags to disable features in build-time.
When a Debug build of Electron succeeds, run the tests:
`$ ./script/test.py`
Fix the failing tests.
Follow all the steps above to fix Electron code on all supported platforms.
## Updating Crashpad
If there are any compilation errors related to the Crashpad, it probably means you need to update the fork to a newer revision: see [Upgrading Crashpad](https://github.com/electron/electron/tree/master/docs/development/upgrading-crashpad.md) for instructions on how to do that.
## Updating NodeJS
Upgrade `vendor/node` to the Node release that corresponds to the v8 version being used in the new Chromium release. See the v8 versions in Node on
See [Upgrading Node](https://github.com/electron/electron/tree/master/docs/development/upgrading-node.md) for instructions on how to do this.
## Verify ffmpeg Support
Electron ships with a version of `ffmpeg` that includes proprietary codecs by default. A version without these codecs is built and distributed with each release as well. Each Chrome upgrade should verify that switching this version is still supported.
You can verify Electron's support for multiple `ffmpeg` builds by loading the following page. It should work with the default `ffmpeg` library distributed with Electron and not work with the `ffmpeg` library built without proprietary codecs.
```html
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Proprietary Codec Check</title>
</head>
<body>
<p>Checking if Electron is using proprietary codecs by loading video from http://www.quirksmode.org/html5/videos/big_buck_bunny.mp4</p>
<p id="outcome"></p>
<video style="display:none" src="http://www.quirksmode.org/html5/videos/big_buck_bunny.mp4" autoplay></video>
<script>
const video = document.querySelector('video')
video.addEventListener('error', ({target}) => {
if (target.error.code === target.error.MEDIA_ERR_SRC_NOT_SUPPORTED) {
document.querySelector('#outcome').textContent = 'Not using proprietary codecs, video emitted source not supported error event.'
} else {
document.querySelector('#outcome').textContent = `Unexpected error: ${target.error.code}`
}
})
video.addEventListener('playing', () => {
document.querySelector('#outcome').textContent = 'Using proprietary codecs, video started playing.'
})
</script>
</body>
</html>
```
## Useful Links
- [Chrome Release Schedule](https://www.chromium.org/developers/calendar)
- [OmahaProxy](http://omahaproxy.appspot.com)
- [Chromium Issue Tracker](https://bugs.chromium.org/p/chromium)

42
docs/development/upgrading-crashpad.md Normal file
View file

@ -0,0 +1,42 @@
# Upgrading Crashpad
1. Get the version of crashpad that we're going to use.
- `libcc/src/third_party/crashpad/README.chromium` will have a line `Revision:` with a checksum
- We need to check out the correponding branch.
- Clone Google's crashpad (https://chromium.googlesource.com/crashpad/crashpad)
- `git clone https://chromium.googlesource.com/crashpad/crashpad`
- Check out the branch with the revision checksum:
- `git checkout <revision checksum>`
- Add electron's crashpad fork as a remote
- `git remote add electron https://github.com/electron/crashpad`
- Check out a new branch for the update
- `git checkout -b electron-crashpad-vA.B.C.D`
- `A.B.C.D` is the Chromium version found in `libcc/VERSION`
and will be something like `62.0.3202.94`
2. Make a checklist of the Electron patches that need to be applied
with `git log --oneline`
- Or view http://github.com/electron/crashpad/commits/previous-branch-name
3. For each patch:
- In `electron-crashpad-vA.B.C.D`, cherry-pick the patch's checksum
- `git cherry-pick <checksum>`
- Resolve any conflicts
- Make sure it builds then add, commit, and push work to electron's crashpad fork
- `git push electron electron-crashpad-vA.B.C.D`
4. Update Electron to build the new crashpad:
- `cd vendor/crashpad`
- `git fetch`
- `git checkout electron-crashpad-v62.0.3202.94`
5. Regenerate Ninja files against both targets
- From Electron root's root, run `script/update.py`
- `script/build.py -c D --target=crashpad_client`
- `script/build.py -c D --target=crashpad_handler`
- Both should build with no errors
6. Push changes to submodule reference
- (From electron root) `git add vendor/crashpad`
- `git push origin upgrade-to-chromium-62`

160
docs/development/upgrading-node.md Normal file
View file

@ -0,0 +1,160 @@
# Upgrading Node
## Discussion
One upgrade issue is building all of Electron with a single copy
of V8 to ensure compatability. This is important because
upstream Node and [libchromiumcontent](upgrading-chrome.md)
both use their own versions of V8.
Upgrading Node is much easier than upgrading libchromiumcontent,
so fewer conflicts arise if one upgrades libchromiumcontent first,
then chooses the upstream Node release whose V8 is closest to it.
Electron has its own [Node fork](https://github.com/electron/node)
with modifications for the V8 build details mentioned above
and for exposing API needed by Electron. Once an upstream Node
release is chosen, it's placed in a branch in Electron's Node fork
and any Electron Node patches are applied there.
Another factor is that the Node project patches its version of V8.
As mentioned above, Electron builds everything with a single copy
of V8, so Node's V8 patches must be ported to that copy.
Once all of Electron's dependencies are building and using the same
copy of V8, the next step is to fix any Electron code issues caused
by the Node upgrade.
[FIXME] something about a Node debugger in Atom that we (e.g. deepak)
use and need to confirm doesn't break with the Node upgrade?
So in short, the primary steps are:
1. Update Electron's Node fork to the desired version
2. Backport Node's V8 patches to our copy of V8
3. Update Electron to use new version of Node
* Update submodules
* Update Node.js build configuration
## Updating Electron's Node [fork](https://github.com/electron/node)
1. Create a branch in https://github.com/electron/node: `electron-node-vX.X.X`
- `vX.X.X` Must use a version of node compatible with our current version of chromium
2. Re-apply our commits from the previous version of node we were using (`vY.Y.Y`) to `v.X.X.X`
- Check release tag and select the range of commits we need to re-apply
- Cherry-pick commit range:
1. Checkout both `vY.Y.Y` & `v.X.X.X`
2. `git cherry-pick FIRST_COMMIT_HASH..LAST_COMMIT_HASH`
- Resolve merge conflicts in each file encountered, then:
1. `git add <conflict-file>`
2. `git cherry-pick --continue`
3. Repeat until finished
## Updating [V8](https://github.com/electron/node/src/V8) Patches
We need to generate a patch file from each patch applied to V8.
1. Get a copy of Electron's libcc fork
- `$ git clone https://github.com/electron/libchromiumcontent`
2. Run `script/update` to get the latest libcc
- This will be time-consuming
3. Remove our copies of the old Node v8 patches
- (In libchromiumcontent repo) Read `patches/v8/README.md` to see which patchfiles
were created during the last update
- Remove those files from `patches/v8/`:
- `git rm` the patchfiles
- edit `patches/v8/README.md`
- commit these removals
4. Inspect Node [repo](https://github.com/electron/node) to see what patches upstream Node
used with their v8 after bumping its version
- `git log --oneline deps/V8`
5. Create a checklist of the patches. This is useful for tracking your work and for
having a quick reference of commit hashes to use in the `git diff-tree` step below.
6. Read `patches/v8/README.md` to see which patchfiles came from the previous version of V8 and therefore need to be removed.
- Delete each patchfile referenced in `patches/v8/README.md`
7. For each patch, do:
- (In node repo) `git diff-tree --patch HASH > ~/path_to_libchromiumcontent/patches/v8/xxx-patch_name.patch`
- `xxx` is an incremented three-digit number (to force patch order)
- `patch_name` should loosely match the node commit messages,
e.g. `030-cherry_pick_cc55747,patch` if the Node commit message was "cherry-pick cc55747"
- (remainder of steps in libchromium repo)
Manually edit the `.patch` file to match upstream V8's directory:
- If a diff section has no instances of `deps/V8`, remove it altogether.
- We don’t want those patches because we’re only patching V8.
- Replace instances of `a/deps/v8`/filename.ext` with `a/filename.ext`
- This is needed because upstream Node keeps its V8 files in a subdirectory
- Ensure that local status is clean: `git status` to make sure there are no unstaged changes.
- Confirm that the patch applies cleanly with
`script/patch.py -r src/V8 -p patches/v8/xxx-patch_name.patch.patch`
- Create a new copy of the patch:
- `cd src/v8 && git diff > ../../test.patch && cd ../..`
- This is needed because the first patch has Node commit checksums that we don't want
- Confirm that checksums are the only difference between the two patches:
- `diff -u test.patch patches/v8/xxx-patch_name.patch`
- Replace the old patch with the new:
- `mv test.patch patches/v8/xxx-patch_name.patch`
- Add the patched code to the index _without_ committing:
- `cd src/v8 && git add . && cd ../..`
- We don't want to commit the changes (they're kept in the patchfiles)
but need them locally so that they don't show up in subsequent diffs
while we iterate through more patches
- Add the patch file to the index:
- `git add a patches/v8/`
- (Optionally) commit each patch file to ensure you can back up if you mess up a step:
- `git commit patches/v8/`
8. Update `patches/v8/README.md` with references to all new patches that have been added so that the next person will know which need to be removed.
9. Update Electron's submodule references:
- ```sh
cd electron/vendor/node
electron/vendor/node$ git fetch
electron/vendor/node$ git checkout electron-node-vA.B.C
electron/vendor/node$ cd ../libchromiumcontent
electron/vendor/libchromiumcontent$ git fetch
electron/vendor/libchromiumcontent$ git checkout upgrade-to-chromium-X
electron/vendor/libchromiumcontent$ cd ../..
electron$ git add vendor
electron$ git commit -m "update submodule referefences for node and libc"
electron$ git pso upgrade-to-chromium-62
electron$ script/bootstrap.py -d
electron$ script/build.py -c -D
```
## Notes
- libcc and V8 are treated as a single unit
- Node maintains its own fork of V8
- They backport a small amount of things as needed
- Documentation in node about how [they work with V8](https://nodejs.org/api/v8.html)
- We update code such that we only use one copy of V8 across all of electron
- E.g electron, libcc, and node
- We don’t track upstream closely due to logistics:
- Upstream uses multiple repos and so merging into a single repo
would result in lost history. So we only update when we’re planning
a node version bump in electron.
- libcc is large and time-consuming to update, so we typically
choose the node version based on which of its releases has a version
of V8 that’s closest to the version in libcc that we’re using.
- We sometimes have to wait for the next periodic Node release
because it will sync more closely with the version of V8 in the new libcc
- Electron keeps all its patches in libcc because it’s simpler than
maintaining different repos for patches for each upstream project.
- Crashpad, node, libcc, etc. patches are all kept in the same place
- Building node:
- There’s a chance we need to change our build configuration
to match the build flags that node wants in `node/common.gypi`