The CI script to ensure no LavaMoat policy changes are required has
been failing despite there being no changes. It turns out that the
command used to check for changes (`git diff-index`) was failing
despite the lack of changes because the file was written again by
`yarn lavamoat:auto` but git hadn't gotten around to updating its index
since the write occurred, so it was considering it as changed until it
verified it wasn't [1].
The command has been replaced by `git diff --exit-code --quiet`, which
should do exactly the same thing except that it forces git to update
its internal cache to verify whether changes are present.
[1]: https://stackoverflow.com/questions/34807971/why-does-git-diff-index-head-result-change-for-touched-files-after-git-diff-or-g
A CI job has been added to ensure the `allow-scripts` config and the
LavaMoat auto-generated policy is up-to-date. This will only run on
release branches and the `master` branch, because it's too difficult a
requirement to meet for each PR for contributors on macOS, due to
differences in the dependency graph caused by optional dependencies.
The `allow-scripts` and LavaMoat policy have both been updated using
`yarn allow-scripts auto` and `yarn lavamoat:auto`.
The `.sh` file extension is now used for all Bash scripts. This ensures
the files are recognized as Bash scripts by the ShellCheck CI job, and
by editors/IDEs for improved syntax highlighting.
* use @lavamoat/allow-scripts for package postinstall allow list
* dnode: set "weak" to false
Co-authored-by: kumavis <kumavis@users.noreply.github.com>
Co-authored-by: Erik Marks <25517051+rekmarks@users.noreply.github.com>
Dependencies are now cached between builds, using a checksum of the
`yarn.lock` file as the cache key. The `node_modules` directory and the
`.har` file from the install are cached and restored, so that we ensure
the record of the install is always preserved alongside the
dependencies.
The consolidation of the `collect-har-artifact` script was to make it
easier to cache the `.har` file along with the dependencies.
The version bump script referenced the old file path for the manifest.
It was stored as a single file called `manifest.json`, but it was split
into `_base.json` and platform-specific manifests in #8140.
The manifest version bump script has been updated to reference
`_base.json`, which is the piece that has the version property.
* Cleanup beforeunload handler after transaction is resolved
The notification window was updated to reject transactions upon close
in #6340. A handler that rejects the transaction was added to
`window.onbeforeunload`, and it was cleared in `actions.js` if it was
confirmed or rejected.
However, the `onbeforeunload` handler remained uncleared if the
transaction was resolved in another window. This results in the
transaction being rejected when the notification window closes, even
long after the transaction is submitted and confirmed. This has been
the cause of many problems with the Firefox e2e tests.
Instead the `onbeforeunload` handler is cleared in the
`componentWillUnmount` lifecycle function, alongside where it's set in
the first place. This ensures that it's correctly unset regardless
of how the transaction was resolved, and it better matches user
expectations.
* Fix indentation and remove redundant export
The `run-all.sh` Bash script now uses consistent indentation, and is
consistent about only re-exporting the Ganache arguments when they
change.
* Ensure transactions are completed before checking balance
Various intermittent e2e test failures appear to be caused by React
re-rendering the transaction list during the test, as the transaction
goes from pending to confirmed. To avoid this race condition, the
transaction is now explicitly looked for in the confirmed transaction
list in each of the tests using this pattern.
* Enable all e2e tests on Firefox
The remaining tests that were disabled on Firefox now work correctly.
Only a few timing adjustments were needed.
* Update Firefox used in CI
Firefox v70 is now used on CI instead of v68. This necessitated
rewriting the function where the extension ID was obtained because the
Firefox extensions page was redesigned.
* ci - install deps - limit install scripts to those needed for build
* Update .circleci/scripts/deps-install.sh
Co-Authored-By: Mark Stacey <markjstacey@gmail.com>
* ci - install deps - expand install scripts needed for tests
* ci - install deps - expand install scripts needed for integration tests
* ci - install deps - fix node-sass script ref
* github - set codeowners for scripts/deps-install
* development - add utility to show deps with install scripts
* lint fix
* deps - move read-installed to devDeps
* ci - install deps with "--har" flag to capture network activity
* ci - add yarn install HAR logs to build-artifacts
* ci - yarn har - fix typo
* Update .circleci/scripts/collect-har-artifact.sh
Co-Authored-By: Mark Stacey <markjstacey@gmail.com>
* Publish GitHub release from master branch
This ensures that changes made on `develop` since branching for the
release are not included. It also ensures that the final release
sourcemaps line-up correctly (they were always build on master)`.
* Consolidate publish jobs
The jobs `job-publish-release` and `create_github_release` both handle
different parts of publishing a release. They have been consolidated
into a single `job-publish-release` job.
* Update release script to expect a merge commit
The release script was originally written to be run on `develop`, so it
expected the current commit to be a result of `Squash & Merge`. Now
that it's run on `master`, it will generally be run against a merge
commit.
The version is now extracted from the commit message using a regular
expression that should work on all version of Bash v3+, and should be
tolerant of both merge commits and `Squash & Merge` commits.
* Target `master` with release PR
`master` is now targeted by the release PR instead of `develop`, as
the release has to be created from the master branch.
The update to `develop` is handled after the release by a PR from
`master` to `develop`, which is created automatically after the
release.
* Publish GitHub release from master branch
This ensures that changes made on `develop` since branching for the
release are not included. It also ensures that the final release
sourcemaps line-up correctly (they were always build on master)`.
* Consolidate publish jobs
The jobs `job-publish-release` and `create_github_release` both handle
different parts of publishing a release. They have been consolidated
into a single `job-publish-release` job.
* Update release script to expect a merge commit
The release script was originally written to be run on `develop`, so it
expected the current commit to be a result of `Squash & Merge`. Now
that it's run on `master`, it will generally be run against a merge
commit.
The version is now extracted from the commit message using a regular
expression that should work on all version of Bash v3+, and should be
tolerant of both merge commits and `Squash & Merge` commits.
* Target `master` with release PR
`master` is now targeted by the release PR instead of `develop`, as
the release has to be created from the master branch.
The update to `develop` is handled after the release by a PR from
`master` to `develop`, which is created automatically after the
release.
As a solution to the constant lockfile churn issues we've had with
`npm`, the project now uses `yarn` to manage dependencies.
The `package-lock.json` file has been replaced with `yarn.lock`, which
was created using `yarn import`. It should approximate the contents of
`package-lock.json` fairly well, though there may be some changes due to
deduplication. The codeowners file has been updated to reference this
new lockfile.
All documentation and npm scripts have been updated to reference `yarn`
rather than `npm`. Note that running scripts using `npm run` still works
fine, but it seemed better to switch those to `yarn` as well to avoid
confusion.
The `npm-audit` Bash script has been replaced with `yarn-audit`. The
output of `yarn audit` is a bit different than `npm audit` in that it
returns a bitmask to describe which severity issues were found. This
made it simpler to check the results directly from the Bash script, so
the associated `npm-audit-check.js` script was no longer required. The
output should be exactly the same, and the information is still sourced
from the same place (the npm registry).
The new `yarn-audit` script does have an external dependency: `jq`.
However, `jq` is already assumed to be present by another CI script, and
is present on all CI images we use. `jq` was not added to `package.json`
as a dependency because there is no official package on the npm
registry, just wrapper scripts. We don't need it anywhere exept on CI
anyway.
The section in `CONTRIBUTING` about how to develop inside the
`node_modules` folder was removed, as the advice was a bit dated, and
wasn't specific to this project anyway.
The npm audit script was auditing all dependencies, then filtering the
results to just the advisories concerning production dependencies. This
was done by checking the boolean `dev` and `optional` properties of each
`findings` entry in each advisory.
The `dev` and `optional` properties are now missing, which is resulting
in dev advisories being mistakenly identified as affecting production.
This check has been removed, and instead the `--production` flag is used
when calling `npm audit`. This accomplishes the same goal without
relying as much upon the audit output format.
The `--production` flag was added in `npm` `v6.10.0`, so `npm` has been
updated to the current latest stable (`v6.10.2`) for the `test-deps`
job. It was also updated on the `prep-deps-npm` job to ensure
consistency in behaviour. The other jobs only use `npm run` which hasn't
changed substantially in some time, so compatibility isn't really a
concern for those.
`audit.json` has also been added to `.gitignore`. It was accidentally
checked in once while working on this branch.
* ci: Rename full_test to test_and_release
* ci: Add scripts to automate GH releases
* Add .bak files to .gitignore
* ci: Add reviewer to the auto version PR
Two important notes:
1. The time it takes to download is negligble compared to e2e test runs
2. Since we cannot use environment variables in CircleCI cache keys we
can't cache the download correctly and have it update when we switch
firefox versions—this isn't the end of the world because of point 1
Mark Stacey
Etienne Dusseault
kumavis
Whymarrh Whitby
Bruno Barbieri
kumavis