diff --git a/docs/development/upgrading-chrome.md b/docs/development/upgrading-chrome.md deleted file mode 100644 index 07e137b97f3c..000000000000 --- a/docs/development/upgrading-chrome.md +++ /dev/null @@ -1,171 +0,0 @@ -# Upgrading Chromium Workflow - -This document is meant to serve as an overview of what steps are needed -on each Chromium upgrade in Electron. - -- Upgrade libcc to a new Chromium version -- Make Electron code compatible with the new libcc -- Update Electron dependencies (crashpad, NodeJS, etc.) if needed -- Make internal builds of libcc and electron -- Update Electron docs if necessary - - -## Upgrade `libcc` to a new Chromium - -### Steps -### 1. Get the code and initialize the project: -```sh -$ git clone git@github.com:electron/libchromiumcontent.git -$ cd libchromiumcontent -$ ./script/bootstrap -v -``` -### 2. Find the new beta/stable Chromium version from [OmahaProxy](https://omahaproxy.appspot.com/). -### 3. Put it into the `libchromiumcontent/VERSION` file, then run `$ ./script/update` - - It will probably fail applying patches. -### 4. Fix `*.patch` files in the `/patches` and `/patches-mas` folders. -### 5. (Optional) Run a separate script to apply patches (`script/update` uses it internally): - ```sh - $ ./script/apply-patches - ``` - - There is also another script `/script/patch.py` that could be more useful, check `--help` to learn how it works with `$ ./script/patch.py -h` -### 6. Run the build when all patches can be applied without errors - ```sh - $ ./script/build - ``` - - If some patches are no longer compatible with the Chromium code, fix compilation errors. -### 7. When build succeeds, create a `dist` for Electron - `$ ./script/create-dist --no_zip` - - It will create `dist/main` folder in the root of the libcc repo, you will need it to build Electron. -### 8. (Optional) Update script contents if there are errors resultant of some files being removed or renamed. (`--no_zip` prevents script from create `dist` archives, you don't need them.) - - -## Update Electron Code - -### Steps -### 1. Get the code: -```sh -$ git clone git@github.com:electron/electron.git -$ cd electron -``` -### 2. If you already have libcc built on you machine in its own repo, you need to tell Electron explicitly to use it: - ```sh - $ ./script/bootstrap.py -v \ - --libcc_source_path /src \ - --libcc_shared_library_path /shared_library \ - --libcc_static_library_path /static_library - ``` -- If you haven't yet built libcc but it's already supposed to be upgraded to a new Chromium, bootstrap Electron as usual - `$ ./script/bootstrap.py -v` - - Ensure that libcc submodule (`vendor/libchromiumcontent`) points to a right revision - -### 3. Set CLANG_REVISION in` script/update-clang.sh` to match the version Chromium is using. -- Located in `electron/libchromiumcontent/src/tools/clang/scripts/update.py` - -### 4. Checkout Chromium if you haven't already: https://chromium.googlesource.com/chromium/src.git/+/{VERSION}/tools/clang/scripts/update.py - - (Replace the `{VERSION}` placeholder in the url above to the Chromium version libcc uses.) -### 5. Build Electron. -- Try to build Debug version first: `$ ./script/build.py -c D` -- You will need it to run tests -### 6. Fix compilation and linking errors -### 7. Ensure that Release build can be built too: `$ ./script/build.py -c R` - - Often the Release build will have different linking errors that you'll need to fix. - - Some compilation and linking errors are caused by missing source/object files in the libcc `dist` -### 8. Update `./script/create-dist` in the libcc repo, recreate a `dist`, and run Electron bootstrap script once again. - -### Tips for fixing compilation errors -- Fix build config errors first -- Fix fatal errors first, like missing files and errors related to compiler flags or defines -- Try to identify complex errors as soon as possible. - - Ask for help if you're not sure how to fix them -- Disable all Electron features, fix the build, then enable them one by one -- Add more build flags to disable features in build-time. - -When a Debug build of Electron succeeds, run the tests: -`$ ./script/test.py` -Fix the failing tests. - -Follow all the steps above to fix Electron code on all supported platforms. - - -## Update Crashpad - -- Electron's crashpad fork: https://github.com/electron/crashpad -- Primary crashpad repo: https://chromium.googlesource.com/crashpad/crashpad/ - - -### Steps -If there are any compilation errors related to the Crashpad, it probably means you need to update the fork to a newer revision: - -### 1. Clone Electron's fork of the Crashpad, add the main repo as another remote: - ```sh - $ git clone https://github.com/electron/crashpad && cd crashpad - $ git remote add upstream https://chromium.googlesource.com/crashpad/crashpad/ && git fetch upstream - ``` -### 2. Find a revision Chromium uses in src/third_party/crashpad/README.chromium -### 3. Rebase the master branch to that REVISION - ```sh - $ git rebase REVISION - ``` - - If there are nontrivial conflicts during rebase, consult with the commit's author. -### 4. Add a tag with the next Electron version to a top commit in the master branch - - e.g. if current Electron version is `1.11.*`, then use tag `electron-1.12.0`, because it will be used in the next version. -### 5. Ensure Electron-specific commits in the master branch on GitHub are properly tagged - - If they aren't, force push will make them orphans and they'll be lost. -### 6. Force push to Electron's crashpad fork. -### 7. Update `/vendor/crashpad` submodule in the Electron's upgrade branch to point to the updated crashpad. - - -## Update NodeJS - -- Upgrade `vendor/node` to the Node release that corresponds to the v8 version - being used in the new Chromium release. See the v8 versions in Node on - https://nodejs.org/en/download/releases for more details. - - You can find v8 version Chromium is using on [OmahaProxy](http://omahaproxy.appspot.com). - If it's not available check `v8/include/v8-version.h` - in the Chromium checkout. - -## Verify ffmpeg Support - -Electron ships with a version of `ffmpeg` that includes proprietary codecs by -default. A version without these codecs is built and distributed with each -release as well. Each Chrome upgrade should verify that switching this version is -still supported. - -You can verify Electron's support for multiple `ffmpeg` builds by loading the -following page. It should work with the default `ffmpeg` library distributed -with Electron and not work with the `ffmpeg` library built without proprietary -codecs. - -```html - - - - - Proprietary Codec Check - - -

Checking if Electron is using proprietary codecs by loading video from http://www.quirksmode.org/html5/videos/big_buck_bunny.mp4

-

- - - - -``` - -## Useful Links - -- [Chrome Release Schedule](https://www.chromium.org/developers/calendar) -- [OmahaProxy](http://omahaproxy.appspot.com) -- [Chromium Issue Tracker](https://bugs.chromium.org/p/chromium) diff --git a/docs/development/upgrading-chromium.md b/docs/development/upgrading-chromium.md new file mode 100644 index 000000000000..b7a08c7053d5 --- /dev/null +++ b/docs/development/upgrading-chromium.md @@ -0,0 +1,136 @@ +# Upgrading Chromium + +This document is meant to serve as an overview of what steps are needed +on each Chromium upgrade in Electron. + +- Upgrade libcc to a new Chromium version +- Make Electron code compatible with the new libcc +- Update Electron dependencies (crashpad, NodeJS, etc.) if needed +- Make internal builds of libcc and electron +- Update Electron docs if necessary + + +## Upgrade `libcc` to a new Chromium + +1. Get the code and initialize the project: + - ```sh + $ git clone git@github.com:electron/libchromiumcontent.git + $ cd libchromiumcontent + $ ./script/bootstrap -v``` +2. Find the new beta/stable Chromium version from [OmahaProxy](https://omahaproxy.appspot.com/). +3. Put it into the `libchromiumcontent/VERSION` file, then run `$ ./script/update` + - It will probably fail applying patches. +4. Fix `*.patch` files in the `/patches` and `/patches-mas` folders. +5. (Optional) Run a separate script to apply patches (`script/update` uses it internally): + - `$ ./script/apply-patches` + - There is also another script `/script/patch.py` that could be more useful + - Check `--help` to learn how it works with `$ ./script/patch.py -h` +6. Run the build when all patches can be applied without errors + - `$ ./script/build` + - If some patches are no longer compatible with the Chromium code, fix compilation errors. +7. When build succeeds, create a `dist` for Electron + - `$ ./script/create-dist --no_zip` + - It will create `dist/main` folder in the root of the libcc repo + - You will need it to build Electron. +8. (Optional) Update script contents if there are errors resultant of some files being removed or renamed. (`--no_zip` prevents script from create `dist` archives, you don't need them.) + + +## Update Electron Code + +1. Get the code: + - ```sh + $ git clone git@github.com:electron/electron.git + $ cd electron + ``` +2. If you already have libcc built on you machine in its own repo, you need to tell Electron explicitly to use it: + - ```sh + $ ./script/bootstrap.py -v \ + --libcc_source_path /src \ + --libcc_shared_library_path /shared_library \ + --libcc_static_library_path /static_library + ``` +- If you haven't yet built libcc but it's already supposed to be upgraded to a new Chromium, bootstrap Electron as usual + `$ ./script/bootstrap.py -v` + - Ensure that libcc submodule (`vendor/libchromiumcontent`) points to a right revision + +3. Set CLANG_REVISION in` script/update-clang.sh` to match the version Chromium is using. + - Located in `electron/libchromiumcontent/src/tools/clang/scripts/update.py` + +4. Checkout Chromium if you haven't already: + - https://chromium.googlesource.com/chromium/src.git/+/{VERSION}/tools/clang/scripts/update.py + - (Replace the `{VERSION}` placeholder in the url above to the Chromium version libcc uses.) +5. Build Electron. + - Try to build Debug version first: `$ ./script/build.py -c D` + - You will need it to run tests +6. Fix compilation and linking errors +7. Ensure that Release build can be built too + - `$ ./script/build.py -c R` + - Often the Release build will have different linking errors that you'll need to fix. + - Some compilation and linking errors are caused by missing source/object files in the libcc `dist` +8. Update `./script/create-dist` in the libcc repo, recreate a `dist`, and run Electron bootstrap script once again. + +### Tips for fixing compilation errors +- Fix build config errors first +- Fix fatal errors first, like missing files and errors related to compiler flags or defines +- Try to identify complex errors as soon as possible. + - Ask for help if you're not sure how to fix them +- Disable all Electron features, fix the build, then enable them one by one +- Add more build flags to disable features in build-time. + +When a Debug build of Electron succeeds, run the tests: +`$ ./script/test.py` +Fix the failing tests. + +Follow all the steps above to fix Electron code on all supported platforms. + + +## Updating Crashpad + +If there are any compilation errors related to the Crashpad, it probably means you need to update the fork to a newer revision: see [Upgrading Crashpad](https://github.com/electron/electron/tree/master/docs/development/upgrading-crashpad.md) for instructions on how to do that. + + +## Updating NodeJS + +Upgrade `vendor/node` to the Node release that corresponds to the v8 version being used in the new Chromium release. See the v8 versions in Node on + +See [Upgrading Node](https://github.com/electron/electron/tree/master/docs/development/upgrading-node.md) for instructions on how to do this. + +## Verify ffmpeg Support + +Electron ships with a version of `ffmpeg` that includes proprietary codecs by default. A version without these codecs is built and distributed with each release as well. Each Chrome upgrade should verify that switching this version is still supported. + +You can verify Electron's support for multiple `ffmpeg` builds by loading the following page. It should work with the default `ffmpeg` library distributed with Electron and not work with the `ffmpeg` library built without proprietary codecs. + +```html + + + + + Proprietary Codec Check + + +

Checking if Electron is using proprietary codecs by loading video from http://www.quirksmode.org/html5/videos/big_buck_bunny.mp4

+

+ + + + +``` + +## Useful Links + +- [Chrome Release Schedule](https://www.chromium.org/developers/calendar) +- [OmahaProxy](http://omahaproxy.appspot.com) +- [Chromium Issue Tracker](https://bugs.chromium.org/p/chromium) diff --git a/docs/development/upgrading-crashpad.md b/docs/development/upgrading-crashpad.md new file mode 100644 index 000000000000..0eb31a8f5966 --- /dev/null +++ b/docs/development/upgrading-crashpad.md @@ -0,0 +1,42 @@ +# Upgrading Crashpad + +1. Get the version of crashpad that we're going to use. + - `libcc/src/third_party/crashpad/README.chromium` will have a line `Revision:` with a checksum + - We need to check out the correponding branch. + - Clone Google's crashpad (https://chromium.googlesource.com/crashpad/crashpad) + - `git clone https://chromium.googlesource.com/crashpad/crashpad` + - Check out the branch with the revision checksum: + - `git checkout ` + - Add electron's crashpad fork as a remote + - `git remote add electron https://github.com/electron/crashpad` + - Check out a new branch for the update + - `git checkout -b electron-crashpad-vA.B.C.D` + - `A.B.C.D` is the Chromium version found in `libcc/VERSION` + and will be something like `62.0.3202.94` + +2. Make a checklist of the Electron patches that need to be applied + with `git log --oneline` + - Or view http://github.com/electron/crashpad/commits/previous-branch-name + +3. For each patch: + - In `electron-crashpad-vA.B.C.D`, cherry-pick the patch's checksum + - `git cherry-pick ` + - Resolve any conflicts + - Make sure it builds then add, commit, and push work to electron's crashpad fork + - `git push electron electron-crashpad-vA.B.C.D` + +4. Update Electron to build the new crashpad: + - `cd vendor/crashpad` + - `git fetch` + - `git checkout electron-crashpad-v62.0.3202.94` +5. Regenerate Ninja files against both targets + - From Electron root's root, run `script/update.py` + - `script/build.py -c D --target=crashpad_client` + - `script/build.py -c D --target=crashpad_handler` + - Both should build with no errors +6. Push changes to submodule reference + - (From electron root) `git add vendor/crashpad` + - `git push origin upgrade-to-chromium-62` + + + diff --git a/docs/development/upgrading-node.md b/docs/development/upgrading-node.md new file mode 100644 index 000000000000..0424cedfc5dc --- /dev/null +++ b/docs/development/upgrading-node.md @@ -0,0 +1,160 @@ +# Upgrading Node + +## Discussion + +One upgrade issue is building all of Electron with a single copy +of V8 to ensure compatability. This is important because +upstream Node and [libchromiumcontent](upgrading-chrome.md) +both use their own versions of V8. + +Upgrading Node is much easier than upgrading libchromiumcontent, +so fewer conflicts arise if one upgrades libchromiumcontent first, +then chooses the upstream Node release whose V8 is closest to it. + +Electron has its own [Node fork](https://github.com/electron/node) +with modifications for the V8 build details mentioned above +and for exposing API needed by Electron. Once an upstream Node +release is chosen, it's placed in a branch in Electron's Node fork +and any Electron Node patches are applied there. + +Another factor is that the Node project patches its version of V8. +As mentioned above, Electron builds everything with a single copy +of V8, so Node's V8 patches must be ported to that copy. + +Once all of Electron's dependencies are building and using the same +copy of V8, the next step is to fix any Electron code issues caused +by the Node upgrade. + +[FIXME] something about a Node debugger in Atom that we (e.g. deepak) +use and need to confirm doesn't break with the Node upgrade? + +So in short, the primary steps are: + +1. Update Electron's Node fork to the desired version +2. Backport Node's V8 patches to our copy of V8 +3. Update Electron to use new version of Node + * Update submodules + * Update Node.js build configuration + +## Updating Electron's Node [fork](https://github.com/electron/node) + +1. Create a branch in https://github.com/electron/node: `electron-node-vX.X.X` + - `vX.X.X` Must use a version of node compatible with our current version of chromium +2. Re-apply our commits from the previous version of node we were using (`vY.Y.Y`) to `v.X.X.X` + - Check release tag and select the range of commits we need to re-apply + - Cherry-pick commit range: + 1. Checkout both `vY.Y.Y` & `v.X.X.X` + 2. `git cherry-pick FIRST_COMMIT_HASH..LAST_COMMIT_HASH` + - Resolve merge conflicts in each file encountered, then: + 1. `git add ` + 2. `git cherry-pick --continue` + 3. Repeat until finished + + +## Updating [V8](https://github.com/electron/node/src/V8) Patches + +We need to generate a patch file from each patch applied to V8. + +1. Get a copy of Electron's libcc fork + - `$ git clone https://github.com/electron/libchromiumcontent` +2. Run `script/update` to get the latest libcc + - This will be time-consuming +3. Remove our copies of the old Node v8 patches + - (In libchromiumcontent repo) Read `patches/v8/README.md` to see which patchfiles + were created during the last update + - Remove those files from `patches/v8/`: + - `git rm` the patchfiles + - edit `patches/v8/README.md` + - commit these removals +4. Inspect Node [repo](https://github.com/electron/node) to see what patches upstream Node + used with their v8 after bumping its version + - `git log --oneline deps/V8` +5. Create a checklist of the patches. This is useful for tracking your work and for + having a quick reference of commit hashes to use in the `git diff-tree` step below. +6. Read `patches/v8/README.md` to see which patchfiles came from the previous version of V8 and therefore need to be removed. + - Delete each patchfile referenced in `patches/v8/README.md` +7. For each patch, do: + - (In node repo) `git diff-tree --patch HASH > ~/path_to_libchromiumcontent/patches/v8/xxx-patch_name.patch` + - `xxx` is an incremented three-digit number (to force patch order) + - `patch_name` should loosely match the node commit messages, + e.g. `030-cherry_pick_cc55747,patch` if the Node commit message was "cherry-pick cc55747" + - (remainder of steps in libchromium repo) + Manually edit the `.patch` file to match upstream V8's directory: + - If a diff section has no instances of `deps/V8`, remove it altogether. + - We don’t want those patches because we’re only patching V8. + - Replace instances of `a/deps/v8`/filename.ext` with `a/filename.ext` + - This is needed because upstream Node keeps its V8 files in a subdirectory + - Ensure that local status is clean: `git status` to make sure there are no unstaged changes. + - Confirm that the patch applies cleanly with + `script/patch.py -r src/V8 -p patches/v8/xxx-patch_name.patch.patch` + - Create a new copy of the patch: + - `cd src/v8 && git diff > ../../test.patch && cd ../..` + - This is needed because the first patch has Node commit checksums that we don't want + - Confirm that checksums are the only difference between the two patches: + - `diff -u test.patch patches/v8/xxx-patch_name.patch` + - Replace the old patch with the new: + - `mv test.patch patches/v8/xxx-patch_name.patch` + - Add the patched code to the index _without_ committing: + - `cd src/v8 && git add . && cd ../..` + - We don't want to commit the changes (they're kept in the patchfiles) + but need them locally so that they don't show up in subsequent diffs + while we iterate through more patches + - Add the patch file to the index: + - `git add a patches/v8/` + - (Optionally) commit each patch file to ensure you can back up if you mess up a step: + - `git commit patches/v8/` +8. Update `patches/v8/README.md` with references to all new patches that have been added so that the next person will know which need to be removed. +9. Update Electron's submodule references: + - ```sh + cd electron/vendor/node + electron/vendor/node$ git fetch + electron/vendor/node$ git checkout electron-node-vA.B.C + electron/vendor/node$ cd ../libchromiumcontent + electron/vendor/libchromiumcontent$ git fetch + electron/vendor/libchromiumcontent$ git checkout upgrade-to-chromium-X + electron/vendor/libchromiumcontent$ cd ../.. + electron$ git add vendor + electron$ git commit -m "update submodule referefences for node and libc" + electron$ git pso upgrade-to-chromium-62 + electron$ script/bootstrap.py -d + electron$ script/build.py -c -D + ``` + + + +## Notes + +- libcc and V8 are treated as a single unit +- Node maintains its own fork of V8 + - They backport a small amount of things as needed + - Documentation in node about how [they work with V8](https://nodejs.org/api/v8.html) +- We update code such that we only use one copy of V8 across all of electron + - E.g electron, libcc, and node +- We don’t track upstream closely due to logistics: + - Upstream uses multiple repos and so merging into a single repo + would result in lost history. So we only update when we’re planning + a node version bump in electron. +- libcc is large and time-consuming to update, so we typically + choose the node version based on which of its releases has a version + of V8 that’s closest to the version in libcc that we’re using. + - We sometimes have to wait for the next periodic Node release + because it will sync more closely with the version of V8 in the new libcc + - Electron keeps all its patches in libcc because it’s simpler than + maintaining different repos for patches for each upstream project. + - Crashpad, node, libcc, etc. patches are all kept in the same place + - Building node: + - There’s a chance we need to change our build configuration + to match the build flags that node wants in `node/common.gypi` + + + + + + + + + + + + +