Automate release (#10827)

* Create prepare-release script

* Add script to merge release

* Cleanup/add logging

* Move release process out of upload.py

* Add cleanup release branch

* Update release doc to reflect new scripts

* Fix to allow running with notesOnly

Also fixup release name and body when beta release.

* Fix issues found during release

* Use getRelease instead of getAssets

github.repos.getAssets is limited to 30 entries which means we may not get back the file we are looking for.

* Documentation corrections

docs/development/releasing.md

@@ -2,47 +2,51 @@
 
 This document describes the process for releasing a new version of Electron.
 
-## Find out what version change is needed
+## Determine which branch to release from
-Is this a major, minor, patch, or beta version change? Read the [Version Change Rules](../tutorial/electron-versioning.md#version-change-rules) to find out.
-
-## Create a temporary branch
 
 - **If releasing beta,** create a new branch from `master`.
-- **If releasing a stable version,** create a new branch from the beta branch you're stablizing.
+- **If releasing a stable version,** create a new branch from the beta branch you're stabilizing.
 
-Name the new branch `release` or anything you like.
+## Find out what version change is needed
+Run `npm run prepare-release -- --notesOnly` to view auto generated release
+notes.  The notes generated should help you determine if this is a major, minor,
+patch, or beta version change. Read the
+[Version Change Rules](../tutorial/electron-versioning.md#version-change-rules) for more information.
 
+## Run the prepare-release script
+The prepare release script will do the following:
+1. Check if a release is already in process and if so it will halt.
+2. Create a release branch.
+3. Bump the version number in several files.  See [this bump commit] for an example.
+4. Create a draft release on GitHub with auto-generated release notes
+5. Push the release branch so that the release builds get built.
+Once you have determined which type of version change is needed, run the
+`prepare-release` script with arguments according to your need:
+- `[major|minor|patch|beta]` to increment one of the version numbers, or
+- `--stable` to indicate this is a stable version
+
+For example:
+
+### Major version change
 ```sh
-git checkout master
+npm run prepare-release -- major
-git pull
-git checkout -b release
 ```
-
+### Minor version change
-This branch is created as a precaution to prevent any merged PRs from sneaking into a release between the time the temporary release branch is created and the CI builds are complete.
-
-## Check for extant drafts
-
-The upload script [looks for an existing draft release](https://github.com/electron/electron/blob/7961a97d7ddbed657c6c867cc8426e02c236c077/script/upload.py#L173-L181). To prevent your new release
-from clobbering an existing draft, check [the releases page] and
-make sure there are no drafts.
-
-## Bump the version
-
-Run the `bump-version` script with arguments according to your need:
-- `--bump=[major|minor|patch|beta]` to increment one of the version numbers, or
-- `--stable` to indicate this is a stable version, or
-- `--version={version}` to set version number directly.
-
-**Note**: you can use both `--bump` and `--stable` simultaneously.
-
-There is also a `dry-run` flag you can use to make sure the version number generated is correct before committing.
-
 ```sh
-npm run bump-version -- --bump=patch --stable
+npm run prepare-release -- minor
-git push origin HEAD
+```
+### Patch version change
+```sh
+npm run prepare-release -- patch
+```
+### Beta version change
+```sh
+npm run prepare-release -- beta
+```
+### Promote beta to stable
+```sh
+npm run prepare-release -- --stable
 ```
-
-This will bump the version number in several files. See [this bump commit] for an example.
 
 ## Wait for builds :hourglass_flowing_sand:
 
@@ -159,65 +163,46 @@ This release is published to [npm](https://www.npmjs.com/package/electron) under
 1. Uncheck the `prerelease` checkbox if you're publishing a stable release; leave it checked for beta releases.
 1. Click 'Save draft'. **Do not click 'Publish release'!**
 1. Wait for all builds to pass before proceeding.
+1. You can run `npm run release --validateRelease` to verify that all of the
+required files have been created for the release.
 
 ## Merge temporary branch
+Once the release builds have finished, merge the `release` branch back into
+the source release branch using the `merge-release` script.
+If the branch cannot be successfully merged back this script will automatically
+rebase the `release` branch and push the changes which will trigger the release
+builds again, which means you will need to wait for the release builds to run
+again before proceeding.
 
-Merge the temporary branch back into master, without creating a merge commit:
+### Merging back into master
-
 ```sh
-git checkout master
+npm run merge-release -- master
-git merge release --no-commit
-git push origin master
 ```
 
-If this fails, rebase with master and rebuild:
+### Merging back into old release branch
-
 ```sh
-git pull
+npm run merge-release -- 1-7-x
-git checkout release
-git rebase master
-git push origin HEAD
 ```
 
-## Run local debug build
-
-Run local debug build to verify that you are actually building the version you want. Sometimes you thought you were doing a release for a new version, but you're actually not.
-
-```sh
-npm run build
-npm start
-```
-
-Verify the window is displaying the current updated version.
-
-## Set environment variables
-
-You'll need to set the following environment variables to publish a release. Ask another team member for these credentials.
-
-- `ELECTRON_S3_BUCKET`
-- `ELECTRON_S3_ACCESS_KEY`
-- `ELECTRON_S3_SECRET_KEY`
-- `ELECTRON_GITHUB_TOKEN` - A personal access token with "repo" scope.
-
-You will only need to do this once.
-
 ## Publish the release
 
-This script will download the binaries and generate the node headers and the .lib linker used on Windows by node-gyp to build native modules.
+Once the merge has finished successfully, run the `release` script
+via `npm run release` to finish the release process.  This script will do the
+following:
+1. Build the project to validate that the correct version number is being released.
+2. Download the binaries and generate the node headers and the .lib linker used
+on Windows by node-gyp to build native modules.
+3. Create and upload the SHASUMS files stored on S3 for the node files.
+4. Create and upload the SHASUMS256.txt file stored on the GitHub release.
+5. Validate that all of the required files are present on GitHub and S3 and have
+the correct checksums as specified in the SHASUMS files.
+6. Publish the release on GitHub
+7. Delete the `release` branch.
 
-```sh
+## Publish to npm
-npm run release
-```
 
-Note: Many distributions of Python still ship with old HTTPS certificates. You may see a `InsecureRequestWarning`, but it can be disregarded.
+Once the publish is successful, run `npm run publish-to-npm` to publish to
-
+release to npm.
-## Delete the temporary branch
-
-```sh
-git checkout master
-git branch -D release # delete local branch
-git push origin :release # delete remote branch
-```
 
 [the releases page]: https://github.com/electron/electron/releases
 [this bump commit]: https://github.com/electron/electron/commit/78ec1b8f89b3886b856377a1756a51617bc33f5a

package.json

@@ -9,15 +9,18 @@
     "check-for-leaks": "^1.0.2",
     "colors": "^1.1.2",
     "dotenv-safe": "^4.0.4",
+    "dugite": "^1.45.0",
     "electabul": "~0.0.4",
     "electron-docs-linter": "^2.3.3",
     "electron-typescript-definitions": "^1.2.7",
     "github": "^9.2.0",
-    "heads": "^1.3.0",
     "husky": "^0.14.3",
+    "minimist": "^1.2.0",
+    "nugget": "^2.0.1",
     "request": "^2.68.0",
     "standard": "^8.4.0",
     "standard-markdown": "^4.0.0",
+    "sumchecker": "^2.0.2",
     "temp": "^0.8.3"
   },
   "standard": {
@@ -49,12 +52,14 @@
     "lint-api-docs-js": "standard-markdown docs && standard-markdown docs-translations",
     "create-api-json": "electron-docs-linter docs --outfile=out/electron-api.json --version=$npm_package_version",
     "create-typescript-definitions": "npm run create-api-json && electron-typescript-definitions --in=out/electron-api.json --out=out/electron.d.ts",
+    "merge-release": "node ./script/merge-release.js",
     "preinstall": "node -e 'process.exit(0)'",
     "publish-to-npm": "node ./script/publish-to-npm.js",
     "prepack": "check-for-leaks",
     "prepush": "check-for-leaks",
-    "prerelease": "node ./script/prerelease",
+    "prepare-release": "node ./script/prepare-release.js",
-    "release": "./script/upload.py -p",
+    "prerelease": "python ./script/bootstrap.py -v --dev && npm run build",
+    "release": "node ./script/release.js",
     "repl": "python ./script/start.py --interactive",
     "start": "python ./script/start.py",
     "test": "python ./script/test.py"

script/bump-version.py

@@ -90,6 +90,7 @@ def main():
     update_package_json(version, suffix)
     tag_version(version, suffix)
 
+  print 'Bumped to version: {0}'.format(version + suffix)
 
 def increase_version(versions, index):
   for i in range(index + 1, 4):

script/merge-release.js

@@ -0,0 +1,116 @@
+#!/usr/bin/env node
+
+require('colors')
+const assert = require('assert')
+const branchToRelease = process.argv[2]
+const fail = '\u2717'.red
+const { GitProcess, GitError } = require('dugite')
+const pass = '\u2713'.green
+const path = require('path')
+const pkg = require('../package.json')
+
+assert(process.env.ELECTRON_GITHUB_TOKEN, 'ELECTRON_GITHUB_TOKEN not found in environment')
+if (!branchToRelease) {
+  console.log(`Usage: merge-release branch`)
+  process.exit(1)
+}
+const gitDir = path.resolve(__dirname, '..')
+
+async function callGit (args, errorMessage, successMessage) {
+  let gitResult = await GitProcess.exec(args, gitDir)
+  if (gitResult.exitCode === 0) {
+    console.log(`${pass} ${successMessage}`)
+    return true
+  } else {
+    console.log(`${fail} ${errorMessage} ${gitResult.stderr}`)
+    process.exit(1)
+  }
+}
+
+async function checkoutBranch (branchName) {
+  console.log(`Checking out ${branchName}.`)
+  let errorMessage = `Error checking out branch ${branchName}:`
+  let successMessage = `Successfully checked out branch ${branchName}.`
+  return await callGit(['checkout', branchName], errorMessage, successMessage)
+}
+
+async function commitMerge () {
+  console.log(`Committing the merge for v${pkg.version}`)
+  let errorMessage = `Error committing merge:`
+  let successMessage = `Successfully committed the merge for v${pkg.version}`
+  let gitArgs = ['commit', '-m', `v${pkg.version}`]
+  return await callGit(gitArgs, errorMessage, successMessage)
+}
+
+async function mergeReleaseIntoBranch (branchName) {
+  console.log(`Merging release branch into ${branchName}.`)
+  let mergeArgs = ['merge', 'release', '--squash']
+  let mergeDetails = await GitProcess.exec(mergeArgs, gitDir)
+  if (mergeDetails.exitCode === 0) {
+    return true
+  } else {
+    const error = GitProcess.parseError(mergeDetails.stderr)
+    if (error === GitError.MergeConflicts) {
+      console.log(`${fail} Could not merge release branch into ${branchName} ` +
+        `due to merge conflicts.`)
+      return false
+    } else {
+      console.log(`${fail} Could not merge release branch into ${branchName} ` +
+        `due to an error: ${mergeDetails.stderr}.`)
+      process.exit(1)
+    }
+  }
+}
+
+async function pushBranch (branchName) {
+  console.log(`Pushing branch ${branchName}.`)
+  let pushArgs = ['push', 'origin', branchName]
+  let errorMessage = `Could not push branch ${branchName} due to an error:`
+  let successMessage = `Successfully pushed branch ${branchName}.`
+  return await callGit(pushArgs, errorMessage, successMessage)
+}
+
+async function pull () {
+  console.log(`Performing a git pull`)
+  let errorMessage = `Could not pull due to an error:`
+  let successMessage = `Successfully performed a git pull`
+  return await callGit(['pull'], errorMessage, successMessage)
+}
+
+async function rebase (targetBranch) {
+  console.log(`Rebasing release branch from ${targetBranch}`)
+  let errorMessage = `Could not rebase due to an error:`
+  let successMessage = `Successfully rebased release branch from ` +
+    `${targetBranch}`
+  return await callGit(['rebase', targetBranch], errorMessage, successMessage)
+}
+
+async function mergeRelease () {
+  await checkoutBranch(branchToRelease)
+  let mergeSuccess = await mergeReleaseIntoBranch(branchToRelease)
+  if (mergeSuccess) {
+    console.log(`${pass} Successfully merged release branch into ` +
+      `${branchToRelease}.`)
+    await commitMerge()
+    let pushSuccess = await pushBranch(branchToRelease)
+    if (pushSuccess) {
+      console.log(`${pass} Success!!! ${branchToRelease} now has the latest release!`)
+    }
+  } else {
+    console.log(`Trying rebase of ${branchToRelease} into release branch.`)
+    await pull()
+    await checkoutBranch('release')
+    let rebaseResult = await rebase(branchToRelease)
+    if (rebaseResult) {
+      let pushResult = pushBranch('HEAD')
+      if (pushResult) {
+        console.log(`Rebase of ${branchToRelease} into release branch was ` +
+          `successful.  Let release builds run and then try this step again.`)
+      }
+      // Exit as failure so release doesn't continue
+      process.exit(1)
+    }
+  }
+}
+
+mergeRelease()

script/prepare-release.js

@@ -0,0 +1,173 @@
+#!/usr/bin/env node
+
+require('colors')
+const args = require('minimist')(process.argv.slice(2))
+const assert = require('assert')
+const { execSync } = require('child_process')
+const fail = '\u2717'.red
+const { GitProcess, GitError } = require('dugite')
+const GitHub = require('github')
+const pass = '\u2713'.green
+const path = require('path')
+const pkg = require('../package.json')
+const versionType = args._[0]
+
+// TODO (future) automatically determine version based on conventional commits
+// via conventional-recommended-bump
+
+assert(process.env.ELECTRON_GITHUB_TOKEN, 'ELECTRON_GITHUB_TOKEN not found in environment')
+if (!versionType && !args.notesOnly) {
+  console.log(`Usage: prepare-release versionType [major | minor | patch | beta]` +
+     ` (--stable) (--notesOnly)`)
+  process.exit(1)
+}
+
+const github = new GitHub()
+const gitDir = path.resolve(__dirname, '..')
+github.authenticate({type: 'token', token: process.env.ELECTRON_GITHUB_TOKEN})
+
+async function createReleaseBranch () {
+  console.log(`Creating release branch.`)
+  let checkoutDetails = await GitProcess.exec([ 'checkout', '-b', 'release' ], gitDir)
+  if (checkoutDetails.exitCode === 0) {
+    console.log(`${pass} Successfully created the release branch.`)
+  } else {
+    const error = GitProcess.parseError(checkoutDetails.stderr)
+    if (error === GitError.BranchAlreadyExists) {
+      console.log(`${fail} Release branch already exists, aborting prepare ` +
+        `release process.`)
+    } else {
+      console.log(`${fail} Error creating release branch: ` +
+        `${checkoutDetails.stderr}`)
+    }
+    process.exit(1)
+  }
+}
+
+function getNewVersion () {
+  console.log(`Bumping for new "${versionType}" version.`)
+  let bumpScript = path.join(__dirname, 'bump-version.py')
+  let scriptArgs = [bumpScript, `--bump ${versionType}`]
+  if (args.stable) {
+    scriptArgs.push('--stable')
+  }
+  try {
+    let bumpVersion = execSync(scriptArgs.join(' '), {encoding: 'UTF-8'})
+    bumpVersion = bumpVersion.substr(bumpVersion.indexOf(':') + 1).trim()
+    let newVersion = `v${bumpVersion}`
+    console.log(`${pass} Successfully bumped version to ${newVersion}`)
+    return newVersion
+  } catch (err) {
+    console.log(`${fail} Could not bump version, error was:`, err)
+  }
+}
+
+async function getCurrentBranch (gitDir) {
+  console.log(`Determining current git branch`)
+  let gitArgs = ['rev-parse', '--abbrev-ref', 'HEAD']
+  let branchDetails = await GitProcess.exec(gitArgs, gitDir)
+  if (branchDetails.exitCode === 0) {
+    let currentBranch = branchDetails.stdout.trim()
+    console.log(`${pass} Successfully determined current git branch is ` +
+      `${currentBranch}`)
+    return currentBranch
+  } else {
+    let error = GitProcess.parseError(branchDetails.stderr)
+    console.log(`${fail} Could not get details for the current branch,
+      error was ${branchDetails.stderr}`, error)
+    process.exit(1)
+  }
+}
+
+async function getReleaseNotes (currentBranch) {
+  console.log(`Generating release notes for ${currentBranch}.`)
+  let githubOpts = {
+    owner: 'electron',
+    repo: 'electron',
+    base: `v${pkg.version}`,
+    head: currentBranch
+  }
+  let releaseNotes = '(placeholder)\n'
+  console.log(`Checking for commits from ${pkg.version} to ${currentBranch}`)
+  let commitComparison = await github.repos.compareCommits(githubOpts)
+    .catch(err => {
+      console.log(`{$fail} Error checking for commits from ${pkg.version} to ` +
+        `${currentBranch}`, err)
+      process.exit(1)
+    })
+
+  commitComparison.data.commits.forEach(commitEntry => {
+    let commitMessage = commitEntry.commit.message
+    if (commitMessage.toLowerCase().indexOf('merge') > -1) {
+      releaseNotes += `${commitMessage} \n`
+    }
+  })
+  console.log(`${pass} Done generating release notes for ${currentBranch}.`)
+  return releaseNotes
+}
+
+async function createRelease (branchToTarget, isBeta) {
+  let releaseNotes = await getReleaseNotes(branchToTarget)
+  let newVersion = getNewVersion()
+  const githubOpts = {
+    owner: 'electron',
+    repo: 'electron'
+  }
+  console.log(`Checking for existing draft release.`)
+  let releases = await github.repos.getReleases(githubOpts)
+    .catch(err => {
+      console.log('$fail} Could not get releases.  Error was', err)
+    })
+  let drafts = releases.data.filter(release => release.draft)
+  if (drafts.length > 0) {
+    console.log(`${fail} Aborting because draft release for
+      ${drafts[0].release.tag_name} already exists.`)
+    process.exit(1)
+  }
+  console.log(`${pass} A draft release does not exist; creating one.`)
+  githubOpts.body = releaseNotes
+  githubOpts.draft = true
+  githubOpts.name = `electron ${newVersion}`
+  if (isBeta) {
+    githubOpts.body = `Note: This is a beta release.  Please file new issues ` +
+      `for any bugs you find in it.\n \n This release is published to npm ` +
+      `under the beta tag and can be installed via npm install electron@beta, ` +
+      `or npm i electron@${newVersion.substr(1)}.`
+    githubOpts.name = `${githubOpts.name}`
+    githubOpts.prerelease = true
+  }
+  githubOpts.tag_name = newVersion
+  githubOpts.target_commitish = branchToTarget
+  await github.repos.createRelease(githubOpts)
+    .catch(err => {
+      console.log(`${fail} Error creating new release: `, err)
+      process.exit(1)
+    })
+  console.log(`${pass} Draft release for ${newVersion} has been created.`)
+}
+
+async function pushRelease () {
+  let pushDetails = await GitProcess.exec(['push', 'origin', 'HEAD'], gitDir)
+  if (pushDetails.exitCode === 0) {
+    console.log(`${pass} Successfully pushed the release branch.  Wait for ` +
+      `release builds to finish before running "npm run release".`)
+  } else {
+    console.log(`${fail} Error pushing the release branch: ` +
+        `${pushDetails.stderr}`)
+    process.exit(1)
+  }
+}
+
+async function prepareRelease (isBeta, notesOnly) {
+  let currentBranch = await getCurrentBranch(gitDir)
+  if (notesOnly) {
+    let releaseNotes = await getReleaseNotes(currentBranch)
+    console.log(`Draft release notes are: ${releaseNotes}`)
+  } else {
+    await createReleaseBranch()
+    await createRelease(currentBranch, isBeta)
+    await pushRelease()
+  }
+}
+
+prepareRelease(!args.stable, args.notesOnly)

script/prerelease.js

@@ -1,114 +0,0 @@
-#!/usr/bin/env node
-
-require('colors')
-const assert = require('assert')
-const GitHub = require('github')
-const heads = require('heads')
-const pkg = require('../package.json')
-const pass = '\u2713'.green
-const fail = '\u2717'.red
-let failureCount = 0
-
-assert(process.env.ELECTRON_GITHUB_TOKEN, 'ELECTRON_GITHUB_TOKEN not found in environment')
-
-const github = new GitHub()
-github.authenticate({type: 'token', token: process.env.ELECTRON_GITHUB_TOKEN})
-github.repos.getReleases({owner: 'electron', repo: 'electron'})
-  .then(res => {
-    const releases = res.data
-    const drafts = releases
-      .filter(release => release.draft) // comment out for testing
-      // .filter(release => release.tag_name === 'v1.7.5') // uncomment for testing
-
-    check(drafts.length === 1, 'one draft exists', true)
-    const draft = drafts[0]
-
-    check(draft.tag_name === `v${pkg.version}`, `draft release version matches local package.json (v${pkg.version})`)
-    check(draft.body.length > 50 && !draft.body.includes('(placeholder)'), 'draft has release notes')
-
-    const requiredAssets = assetsForVersion(draft.tag_name).sort()
-    const extantAssets = draft.assets.map(asset => asset.name).sort()
-
-    requiredAssets.forEach(asset => {
-      check(extantAssets.includes(asset), asset)
-    })
-
-    const s3Urls = s3UrlsForVersion(draft.tag_name)
-    heads(s3Urls)
-      .then(results => {
-        results.forEach((result, i) => {
-          check(result === 200, s3Urls[i])
-        })
-
-        process.exit(failureCount > 0 ? 1 : 0)
-      })
-      .catch(err => {
-        console.error('Error making HEAD requests for S3 assets')
-        console.error(err)
-        process.exit(1)
-      })
-  })
-
-function check (condition, statement, exitIfFail = false) {
-  if (condition) {
-    console.log(`${pass} ${statement}`)
-  } else {
-    failureCount++
-    console.log(`${fail} ${statement}`)
-    if (exitIfFail) process.exit(1)
-  }
-}
-
-function assetsForVersion (version) {
-  const patterns = [
-    'electron-{{VERSION}}-darwin-x64-dsym.zip',
-    'electron-{{VERSION}}-darwin-x64-symbols.zip',
-    'electron-{{VERSION}}-darwin-x64.zip',
-    'electron-{{VERSION}}-linux-arm-symbols.zip',
-    'electron-{{VERSION}}-linux-arm.zip',
-    'electron-{{VERSION}}-linux-arm64-symbols.zip',
-    'electron-{{VERSION}}-linux-arm64.zip',
-    'electron-{{VERSION}}-linux-armv7l-symbols.zip',
-    'electron-{{VERSION}}-linux-armv7l.zip',
-    'electron-{{VERSION}}-linux-ia32-symbols.zip',
-    'electron-{{VERSION}}-linux-ia32.zip',
-    'electron-{{VERSION}}-linux-x64-symbols.zip',
-    'electron-{{VERSION}}-linux-x64.zip',
-    'electron-{{VERSION}}-mas-x64-dsym.zip',
-    'electron-{{VERSION}}-mas-x64-symbols.zip',
-    'electron-{{VERSION}}-mas-x64.zip',
-    'electron-{{VERSION}}-win32-ia32-pdb.zip',
-    'electron-{{VERSION}}-win32-ia32-symbols.zip',
-    'electron-{{VERSION}}-win32-ia32.zip',
-    'electron-{{VERSION}}-win32-x64-pdb.zip',
-    'electron-{{VERSION}}-win32-x64-symbols.zip',
-    'electron-{{VERSION}}-win32-x64.zip',
-    'electron-api.json',
-    'electron.d.ts',
-    'ffmpeg-{{VERSION}}-darwin-x64.zip',
-    'ffmpeg-{{VERSION}}-linux-arm.zip',
-    'ffmpeg-{{VERSION}}-linux-arm64.zip',
-    'ffmpeg-{{VERSION}}-linux-armv7l.zip',
-    'ffmpeg-{{VERSION}}-linux-ia32.zip',
-    'ffmpeg-{{VERSION}}-linux-x64.zip',
-    'ffmpeg-{{VERSION}}-mas-x64.zip',
-    'ffmpeg-{{VERSION}}-win32-ia32.zip',
-    'ffmpeg-{{VERSION}}-win32-x64.zip'
-  ]
-  return patterns.map(pattern => pattern.replace(/{{VERSION}}/g, version))
-}
-
-function s3UrlsForVersion (version) {
-  const bucket = 'https://gh-contractor-zcbenz.s3.amazonaws.com/'
-  const patterns = [
-    'atom-shell/dist/{{VERSION}}/iojs-{{VERSION}}-headers.tar.gz',
-    'atom-shell/dist/{{VERSION}}/iojs-{{VERSION}}.tar.gz',
-    'atom-shell/dist/{{VERSION}}/node-{{VERSION}}.tar.gz',
-    'atom-shell/dist/{{VERSION}}/node.lib',
-    'atom-shell/dist/{{VERSION}}/win-x64/iojs.lib',
-    'atom-shell/dist/{{VERSION}}/win-x86/iojs.lib',
-    'atom-shell/dist/{{VERSION}}/x64/node.lib',
-    'atom-shell/dist/index.json'
-  ]
-  return patterns.map(pattern => bucket + pattern.replace(/{{VERSION}}/g, version))
-}

script/publish-to-npm.js

@@ -114,7 +114,7 @@ new Promise((resolve, reject) => {
       cwd: tempDir
     })
     const checkVersion = childProcess.execSync(`${path.join(tempDir, 'node_modules', '.bin', 'electron')} -v`)
-    assert.strictEqual(checkVersion.toString().trim(), `v${rootPackageJson.version}`)
+    assert.ok((`v${rootPackageJson.version}`.indexOf(checkVersion.toString().trim()) === 0), `Version is correct`)
     resolve(tarballPath)
   })
 })

script/release.js

@@ -0,0 +1,462 @@
+#!/usr/bin/env node
+
+require('colors')
+const args = require('minimist')(process.argv.slice(2))
+const assert = require('assert')
+const fs = require('fs')
+const { execSync } = require('child_process')
+const GitHub = require('github')
+const { GitProcess } = require('dugite')
+const nugget = require('nugget')
+const pkg = require('../package.json')
+const pkgVersion = `v${pkg.version}`
+const pass = '\u2713'.green
+const path = require('path')
+const fail = '\u2717'.red
+const sumchecker = require('sumchecker')
+const temp = require('temp').track()
+const { URL } = require('url')
+let failureCount = 0
+
+assert(process.env.ELECTRON_GITHUB_TOKEN, 'ELECTRON_GITHUB_TOKEN not found in environment')
+
+const github = new GitHub({
+  followRedirects: false
+})
+github.authenticate({type: 'token', token: process.env.ELECTRON_GITHUB_TOKEN})
+const gitDir = path.resolve(__dirname, '..')
+
+async function getDraftRelease (version, skipValidation) {
+  let releaseInfo = await github.repos.getReleases({owner: 'electron', repo: 'electron'})
+  let drafts
+  let versionToCheck
+  if (version) {
+    drafts = releaseInfo.data
+      .filter(release => release.tag_name === version)
+    versionToCheck = version
+  } else {
+    drafts = releaseInfo.data
+      .filter(release => release.draft)
+    versionToCheck = pkgVersion
+  }
+
+  const draft = drafts[0]
+  if (!skipValidation) {
+    failureCount = 0
+    check(drafts.length === 1, 'one draft exists', true)
+    check(draft.tag_name === versionToCheck, `draft release version matches local package.json (${versionToCheck})`)
+    if (versionToCheck.indexOf('beta')) {
+      check(draft.prerelease, 'draft is a prerelease')
+    }
+    check(draft.body.length > 50 && !draft.body.includes('(placeholder)'), 'draft has release notes')
+    check((failureCount === 0), `Draft release looks good to go.`, true)
+  }
+  return draft
+}
+
+async function validateReleaseAssets (release) {
+  const requiredAssets = assetsForVersion(release.tag_name).sort()
+  const extantAssets = release.assets.map(asset => asset.name).sort()
+  const downloadUrls = release.assets.map(asset => asset.browser_download_url).sort()
+
+  failureCount = 0
+  requiredAssets.forEach(asset => {
+    check(extantAssets.includes(asset), asset)
+  })
+  check((failureCount === 0), `All required GitHub assets exist for release`, true)
+
+  if (release.draft) {
+    await verifyAssets(release)
+  } else {
+    await verifyShasums(downloadUrls)
+      .catch(err => {
+        console.log(`${fail} error verifyingShasums`, err)
+      })
+  }
+  const s3Urls = s3UrlsForVersion(release.tag_name)
+  await verifyShasums(s3Urls, true)
+}
+
+function check (condition, statement, exitIfFail = false) {
+  if (condition) {
+    console.log(`${pass} ${statement}`)
+  } else {
+    failureCount++
+    console.log(`${fail} ${statement}`)
+    if (exitIfFail) process.exit(1)
+  }
+}
+
+function assetsForVersion (version) {
+  const patterns = [
+    `electron-${version}-darwin-x64-dsym.zip`,
+    `electron-${version}-darwin-x64-symbols.zip`,
+    `electron-${version}-darwin-x64.zip`,
+    `electron-${version}-linux-arm-symbols.zip`,
+    `electron-${version}-linux-arm.zip`,
+    `electron-${version}-linux-arm64-symbols.zip`,
+    `electron-${version}-linux-arm64.zip`,
+    `electron-${version}-linux-armv7l-symbols.zip`,
+    `electron-${version}-linux-armv7l.zip`,
+    `electron-${version}-linux-ia32-symbols.zip`,
+    `electron-${version}-linux-ia32.zip`,
+    `electron-${version}-linux-x64-symbols.zip`,
+    `electron-${version}-linux-x64.zip`,
+    `electron-${version}-mas-x64-dsym.zip`,
+    `electron-${version}-mas-x64-symbols.zip`,
+    `electron-${version}-mas-x64.zip`,
+    `electron-${version}-win32-ia32-pdb.zip`,
+    `electron-${version}-win32-ia32-symbols.zip`,
+    `electron-${version}-win32-ia32.zip`,
+    `electron-${version}-win32-x64-pdb.zip`,
+    `electron-${version}-win32-x64-symbols.zip`,
+    `electron-${version}-win32-x64.zip`,
+    `electron-api.json`,
+    `electron.d.ts`,
+    `ffmpeg-${version}-darwin-x64.zip`,
+    `ffmpeg-${version}-linux-arm.zip`,
+    `ffmpeg-${version}-linux-arm64.zip`,
+    `ffmpeg-${version}-linux-armv7l.zip`,
+    `ffmpeg-${version}-linux-ia32.zip`,
+    `ffmpeg-${version}-linux-x64.zip`,
+    `ffmpeg-${version}-mas-x64.zip`,
+    `ffmpeg-${version}-win32-ia32.zip`,
+    `ffmpeg-${version}-win32-x64.zip`,
+    `SHASUMS256.txt`
+  ]
+  return patterns
+}
+
+function s3UrlsForVersion (version) {
+  const bucket = `https://gh-contractor-zcbenz.s3.amazonaws.com/`
+  const patterns = [
+    `${bucket}atom-shell/dist/${version}/iojs-${version}-headers.tar.gz`,
+    `${bucket}atom-shell/dist/${version}/iojs-${version}.tar.gz`,
+    `${bucket}atom-shell/dist/${version}/node-${version}.tar.gz`,
+    `${bucket}atom-shell/dist/${version}/node.lib`,
+    `${bucket}atom-shell/dist/${version}/win-x64/iojs.lib`,
+    `${bucket}atom-shell/dist/${version}/win-x86/iojs.lib`,
+    `${bucket}atom-shell/dist/${version}/x64/node.lib`,
+    `${bucket}atom-shell/dist/${version}/SHASUMS.txt`,
+    `${bucket}atom-shell/dist/${version}/SHASUMS256.txt`,
+    `${bucket}atom-shell/dist/index.json`
+  ]
+  return patterns
+}
+
+function checkVersion () {
+  console.log(`Verifying that app version matches package version ${pkgVersion}.`)
+  let startScript = path.join(__dirname, 'start.py')
+  let appVersion = runScript(startScript, ['--version']).trim()
+  check((pkgVersion.indexOf(appVersion) === 0), `App version ${appVersion} matches ` +
+    `package version ${pkgVersion}.`, true)
+}
+
+function runScript (scriptName, scriptArgs, cwd) {
+  let scriptCommand = `${scriptName} ${scriptArgs.join(' ')}`
+  let scriptOptions = {
+    encoding: 'UTF-8'
+  }
+  if (cwd) {
+    scriptOptions.cwd = cwd
+  }
+  try {
+    return execSync(scriptCommand, scriptOptions)
+  } catch (err) {
+    console.log(`${fail} Error running ${scriptName}`, err)
+    process.exit(1)
+  }
+}
+
+function uploadNodeShasums () {
+  console.log('Uploading Node SHASUMS file to S3.')
+  let scriptPath = path.join(__dirname, 'upload-node-checksums.py')
+  runScript(scriptPath, ['-v', pkgVersion])
+  console.log(`${pass} Done uploading Node SHASUMS file to S3.`)
+}
+
+function uploadIndexJson () {
+  console.log('Uploading index.json to S3.')
+  let scriptPath = path.join(__dirname, 'upload-index-json.py')
+  runScript(scriptPath, [])
+  console.log(`${pass} Done uploading index.json to S3.`)
+}
+
+async function createReleaseShasums (release) {
+  let fileName = 'SHASUMS256.txt'
+  let existingAssets = release.assets.filter(asset => asset.name === fileName)
+  if (existingAssets.length > 0) {
+    console.log(`${fileName} already exists on GitHub; deleting before creating new file.`)
+    await github.repos.deleteAsset({
+      owner: 'electron',
+      repo: 'electron',
+      id: existingAssets[0].id
+    }).catch(err => {
+      console.log(`${fail} Error deleting ${fileName} on GitHub:`, err)
+    })
+  }
+  console.log(`Creating and uploading the release ${fileName}.`)
+  let scriptPath = path.join(__dirname, 'merge-electron-checksums.py')
+  let checksums = runScript(scriptPath, ['-v', pkgVersion])
+  console.log(`${pass} Generated release SHASUMS.`)
+  let filePath = await saveShaSumFile(checksums, fileName)
+  console.log(`${pass} Created ${fileName} file.`)
+  await uploadShasumFile(filePath, fileName, release)
+  console.log(`${pass} Successfully uploaded ${fileName} to GitHub.`)
+}
+
+async function uploadShasumFile (filePath, fileName, release) {
+  let githubOpts = {
+    owner: 'electron',
+    repo: 'electron',
+    id: release.id,
+    filePath,
+    name: fileName
+  }
+  return await github.repos.uploadAsset(githubOpts)
+    .catch(err => {
+      console.log(`${fail} Error uploading ${filePath} to GitHub:`, err)
+      process.exit(1)
+    })
+}
+
+function saveShaSumFile (checksums, fileName) {
+  return new Promise((resolve, reject) => {
+    temp.open(fileName, (err, info) => {
+      if (err) {
+        console.log(`${fail} Could not create ${fileName} file`)
+        process.exit(1)
+      } else {
+        fs.writeFileSync(info.fd, checksums)
+        fs.close(info.fd, (err) => {
+          if (err) {
+            console.log(`${fail} Could close ${fileName} file`)
+            process.exit(1)
+          }
+          resolve(info.path)
+        })
+      }
+    })
+  })
+}
+
+async function publishRelease (release) {
+  let githubOpts = {
+    owner: 'electron',
+    repo: 'electron',
+    id: release.id,
+    tag_name: release.tag_name,
+    draft: false
+  }
+  return await github.repos.editRelease(githubOpts)
+    .catch(err => {
+      console.log(`${fail} Error publishing release:`, err)
+      process.exit(1)
+    })
+}
+
+async function makeRelease (releaseToValidate) {
+  if (releaseToValidate) {
+    console.log(`Validating release ${args.validateRelease}`)
+    let release = await getDraftRelease(args.validateRelease)
+    await validateReleaseAssets(release)
+  } else {
+    checkVersion()
+    let draftRelease = await getDraftRelease()
+    uploadNodeShasums()
+    uploadIndexJson()
+    await createReleaseShasums(draftRelease)
+    // Fetch latest version of release before verifying
+    draftRelease = await getDraftRelease(pkgVersion, true)
+    await validateReleaseAssets(draftRelease)
+    await publishRelease(draftRelease)
+    await cleanupReleaseBranch()
+    console.log(`${pass} SUCCESS!!! Release has been published. Please run ` +
+      `"npm run publish-to-npm" to publish release to npm.`)
+  }
+}
+
+async function makeTempDir () {
+  return new Promise((resolve, reject) => {
+    temp.mkdir('electron-publish', (err, dirPath) => {
+      if (err) {
+        reject(err)
+      } else {
+        resolve(dirPath)
+      }
+    })
+  })
+}
+
+async function verifyAssets (release) {
+  let downloadDir = await makeTempDir()
+  let githubOpts = {
+    owner: 'electron',
+    repo: 'electron',
+    headers: {
+      Accept: 'application/octet-stream'
+    }
+  }
+  console.log(`Downloading files from GitHub to verify shasums`)
+  let shaSumFile = 'SHASUMS256.txt'
+  let filesToCheck = await Promise.all(release.assets.map(async (asset) => {
+    githubOpts.id = asset.id
+    let assetDetails = await github.repos.getAsset(githubOpts)
+    await downloadFiles(assetDetails.meta.location, downloadDir, false, asset.name)
+    return asset.name
+  })).catch(err => {
+    console.log(`${fail} Error downloading files from GitHub`, err)
+    process.exit(1)
+  })
+  filesToCheck = filesToCheck.filter(fileName => fileName !== shaSumFile)
+  let checkerOpts
+  await validateChecksums({
+    algorithm: 'sha256',
+    filesToCheck,
+    fileDirectory: downloadDir,
+    shaSumFile,
+    checkerOpts,
+    fileSource: 'GitHub'
+  })
+}
+
+function downloadFiles (urls, directory, quiet, targetName) {
+  return new Promise((resolve, reject) => {
+    let nuggetOpts = {
+      dir: directory
+    }
+    if (quiet) {
+      nuggetOpts.quiet = quiet
+    }
+    if (targetName) {
+      nuggetOpts.target = targetName
+    }
+    nugget(urls, nuggetOpts, (err) => {
+      if (err) {
+        reject(err)
+      } else {
+        resolve()
+      }
+    })
+  })
+}
+
+async function verifyShasums (urls, isS3) {
+  let fileSource = isS3 ? 'S3' : 'GitHub'
+  console.log(`Downloading files from ${fileSource} to verify shasums`)
+  let downloadDir = await makeTempDir()
+  let filesToCheck = []
+  try {
+    if (!isS3) {
+      await downloadFiles(urls, downloadDir)
+      filesToCheck = urls.map(url => {
+        let currentUrl = new URL(url)
+        return path.basename(currentUrl.pathname)
+      }).filter(file => file.indexOf('SHASUMS') === -1)
+    } else {
+      const s3VersionPath = `/atom-shell/dist/${pkgVersion}/`
+      await Promise.all(urls.map(async (url) => {
+        let currentUrl = new URL(url)
+        let dirname = path.dirname(currentUrl.pathname)
+        let filename = path.basename(currentUrl.pathname)
+        let s3VersionPathIdx = dirname.indexOf(s3VersionPath)
+        if (s3VersionPathIdx === -1 || dirname === s3VersionPath) {
+          if (s3VersionPathIdx !== -1 && filename.indexof('SHASUMS') === -1) {
+            filesToCheck.push(filename)
+          }
+          await downloadFiles(url, downloadDir, true)
+        } else {
+          let subDirectory = dirname.substr(s3VersionPathIdx + s3VersionPath.length)
+          let fileDirectory = path.join(downloadDir, subDirectory)
+          try {
+            fs.statSync(fileDirectory)
+          } catch (err) {
+            fs.mkdirSync(fileDirectory)
+          }
+          filesToCheck.push(path.join(subDirectory, filename))
+          await downloadFiles(url, fileDirectory, true)
+        }
+      }))
+    }
+  } catch (err) {
+    console.log(`${fail} Error downloading files from ${fileSource}`, err)
+    process.exit(1)
+  }
+  console.log(`${pass} Successfully downloaded the files from ${fileSource}.`)
+  let checkerOpts
+  if (isS3) {
+    checkerOpts = { defaultTextEncoding: 'binary' }
+  }
+
+  await validateChecksums({
+    algorithm: 'sha256',
+    filesToCheck,
+    fileDirectory: downloadDir,
+    shaSumFile: 'SHASUMS256.txt',
+    checkerOpts,
+    fileSource
+  })
+
+  if (isS3) {
+    await validateChecksums({
+      algorithm: 'sha1',
+      filesToCheck,
+      fileDirectory: downloadDir,
+      shaSumFile: 'SHASUMS.txt',
+      checkerOpts,
+      fileSource
+    })
+  }
+}
+
+async function validateChecksums (validationArgs) {
+  console.log(`Validating checksums for files from ${validationArgs.fileSource} ` +
+    `against ${validationArgs.shaSumFile}.`)
+  let shaSumFilePath = path.join(validationArgs.fileDirectory, validationArgs.shaSumFile)
+  let checker = new sumchecker.ChecksumValidator(validationArgs.algorithm,
+    shaSumFilePath, validationArgs.checkerOpts)
+  await checker.validate(validationArgs.fileDirectory, validationArgs.filesToCheck)
+    .catch(err => {
+      if (err instanceof sumchecker.ChecksumMismatchError) {
+        console.error(`${fail} The checksum of ${err.filename} from ` +
+          `${validationArgs.fileSource} did not match the shasum in ` +
+          `${validationArgs.shaSumFile}`)
+      } else if (err instanceof sumchecker.ChecksumParseError) {
+        console.error(`${fail} The checksum file ${validationArgs.shaSumFile} ` +
+          `from ${validationArgs.fileSource} could not be parsed.`, err)
+      } else if (err instanceof sumchecker.NoChecksumFoundError) {
+        console.error(`${fail} The file ${err.filename} from ` +
+          `${validationArgs.fileSource} was not in the shasum file ` +
+          `${validationArgs.shaSumFile}.`)
+      } else {
+        console.error(`${fail} Error matching files from ` +
+          `${validationArgs.fileSource} shasums in ${validationArgs.shaSumFile}.`, err)
+      }
+      process.exit(1)
+    })
+  console.log(`${pass} All files from ${validationArgs.fileSource} match ` +
+    `shasums defined in ${validationArgs.shaSumFile}.`)
+}
+
+async function cleanupReleaseBranch () {
+  console.log(`Cleaning up release branch.`)
+  let errorMessage = `Could not delete local release branch.`
+  let successMessage = `Successfully deleted local release branch.`
+  await callGit(['branch', '-D', 'release'], errorMessage, successMessage)
+  errorMessage = `Could not delete remote release branch.`
+  successMessage = `Successfully deleted remote release branch.`
+  return await callGit(['push', 'origin', ':release'], errorMessage, successMessage)
+}
+
+async function callGit (args, errorMessage, successMessage) {
+  let gitResult = await GitProcess.exec(args, gitDir)
+  if (gitResult.exitCode === 0) {
+    console.log(`${pass} ${successMessage}`)
+    return true
+  } else {
+    console.log(`${fail} ${errorMessage} ${gitResult.stderr}`)
+    process.exit(1)
+  }
+}
+
+makeRelease(args.validateRelease)

script/upload-to-github.js

@@ -28,8 +28,8 @@ function uploadToGitHub () {
     if (retry < 4) {
       console.log(`Error uploading ${fileName} to GitHub, will retry.  Error was:`, err)
       retry++
-      github.repos.getAssets(githubOpts).then(assets => {
+      github.repos.getRelease(githubOpts).then(release => {
-        let existingAssets = assets.data.filter(asset => asset.name === fileName)
+        let existingAssets = release.data.assets.filter(asset => asset.name === fileName)
         if (existingAssets.length > 0) {
           console.log(`${fileName} already exists; will delete before retrying upload.`)
           github.repos.deleteAsset({

script/upload.py

@@ -36,17 +36,16 @@ PDB_NAME = get_zip_name(PROJECT_NAME, ELECTRON_VERSION, 'pdb')
 def main():
   args = parse_args()
 
-  if not args.publish_release:
+  if not dist_newer_than_head():
-    if not dist_newer_than_head():
+    run_python_script('create-dist.py')
-      run_python_script('create-dist.py')
 
-    build_version = get_electron_build_version()
+  build_version = get_electron_build_version()
-    if not ELECTRON_VERSION.startswith(build_version):
+  if not ELECTRON_VERSION.startswith(build_version):
-      error = 'Tag name ({0}) should match build version ({1})\n'.format(
+    error = 'Tag name ({0}) should match build version ({1})\n'.format(
-          ELECTRON_VERSION, build_version)
+        ELECTRON_VERSION, build_version)
-      sys.stderr.write(error)
+    sys.stderr.write(error)
-      sys.stderr.flush()
+    sys.stderr.flush()
-      return 1
+    return 1
 
   github = GitHub(auth_token())
   releases = github.repos(ELECTRON_REPO).releases.get()
@@ -64,24 +63,6 @@ def main():
     release = create_or_get_release_draft(github, releases, args.version,
                                           tag_exists)
 
-  if args.publish_release:
-    # Upload the Node SHASUMS*.txt.
-    run_python_script('upload-node-checksums.py', '-v', ELECTRON_VERSION)
-
-    # Upload the index.json.
-    run_python_script('upload-index-json.py')
-
-    # Create and upload the Electron SHASUMS*.txt
-    release_electron_checksums(release)
-
-    # Press the publish button.
-    publish_release(github, release['id'])
-
-    # TODO: run publish-to-npm script here
-
-    # Do not upload other files when passed "-p".
-    return
-
   # Upload Electron with GitHub Releases API.
   upload_electron(github, release, os.path.join(DIST_DIR, DIST_NAME))
   upload_electron(github, release, os.path.join(DIST_DIR, SYMBOLS_NAME))
@@ -206,16 +187,6 @@ def create_release_draft(github, tag):
   return r
 
 
-def release_electron_checksums(release):
-  checksums = run_python_script('merge-electron-checksums.py',
-                                '-v', ELECTRON_VERSION)
-  filename = 'SHASUMS256.txt'
-  filepath = os.path.join(SOURCE_ROOT, filename)
-  with open(filepath, 'w') as sha_file:
-      sha_file.write(checksums.decode('utf-8'))
-  upload_io_to_github(release, filename, filepath)
-
-
 def upload_electron(github, release, file_path):
   # Delete the original file before uploading in CI.
   filename = os.path.basename(file_path)
@@ -263,11 +234,6 @@ def upload_sha256_checksum(version, file_path):
         'atom-shell/tmp/{0}'.format(version), [checksum_path])
 
 
-def publish_release(github, release_id):
-  data = dict(draft=False)
-  github.repos(ELECTRON_REPO).releases(release_id).patch(data=data)
-
-
 def auth_token():
   token = get_env_var('GITHUB_TOKEN')
   message = ('Error: Please set the $ELECTRON_GITHUB_TOKEN '