This is a follow-up to #15318, which fixed a problem with environment
variables. Every function in this module that passes options related to
environment variables has been updated with a doc comment. This should
make it clearer which options are mandatory and which are optional,
hopefully preventing a similar mistake from happening in the future.
The environment variables `IN_TEST` and `METAMASK_DEBUG` were not
being set to `false` correctly. Instead those variables were being
skipped, and were resolved to `undefined` at runtime. This is confusing
because the other environment variables do not work that way - they can
be set to false.
The build script has been updated to ensure those two environment
variables are always set to `true` or `false` - never `undefined`.
Additionally, the `METAMASK_VERSION` environment variable was being
omitted from the `app-init.js` bundle. For the sake of consistency,
that has also been restored.
Some of the functions in `development/build/scripts.js` have been
renamed to better describe their function, and to be more consistent
with other similar functions.
Two unused options have been removed from the `createNormalBundle`
function in the build script: 'extraEntries` and `modulesToExpose`.'
Both of these options were used in the old "main" bundles, before we
began using the "factored" bundles. They have been unused since #11080.
This reverts commit f09ab88891, reversing
changes made to effc761e0e.
This is being temporarily reverted to make it easier to release an
urgent fix for v10.15.1.
#14583 broke the development build scripts (e.g. `yarn start`) by adding a positional argument to a package script (`build:dev`) that is used and passed positional arguments in the build script itself. This PR removes the positional argument from the `build:dev` script and `yarn start` now works again. In addition, the `--apply-lavamoat` flag is properly forwarded to child processes, which was not the case in the original implementation.
To test, `yarn start` should work and LavaMoat should _not_ be applied, in distinction to `yarn build:dev dev --apply-lavamoat=true`. Whether LavaMoat is applied can be determined by checking whether `Object.isFrozen(Object.prototype)` is `true` (with LavaMoat) or `false` (without LavaMoat).
Adds a new flag, `--apply-lavamoat`, to the main build script. The flag controls whether LavaMoat is actually applied to the output of the build process. The flag defaults to `true`, but we explicitly set it to `false` in the `start` package script. Meanwhile, the `start:lavamoat` script is modified such that it applies LavaMoat to the build output in development mode, but it no longer runs the build process itself under LavaMoat as there aren't very compelling reasons to do so.
This change is motivated by the fact that development builds do not have their own dedicated LavaMoat policies, which causes development builds to fail since #14537. The downside of this change is that LavaMoat-related failures will not be detected when running `yarn start`. @kumavis has plans for fixing this problem in a future major version of the `@lavamoat` suite.
The phishing warning page URL environment variable has been renamed
from `PHISHING_PAGE_URL` to `PHISHING_WARNING_PAGE_URL`. We call this
page the "phishing warning page" everywhere else, and this name seemed
better suited (it's not a phishing page itself).
The variable has been listed and documented in `.metamaskrc.dist` as
well.
The e2e tests have been updated for `@metamask/phishing-warning@1.1.0`.
The iframe case was updated with a new design, which required test
changes. The third test that was meant to ensure the phishing page
can't redirect to an extension page has been updated to navigate
directly to the phishing warning page and setting the URL manually via
query parameters, as that was the only way to test that redirect.
An externally hosted phishing warning page is now used rather than the
built-in phishing warning page.The phishing page warning URL is set via
configuration file or environment variable. The default URL is either
the expected production URL or `http://localhost:9999/` for e2e testing
environments.
The new external phishing page includes a design change when it is
loaded within an iframe. In that case it now shows a condensed message,
and prompts the user to open the full warning page in a new tab to see
more details or bypass the warning. This is to prevent a clickjacking
attack from safelisting a site without user consent.
The new external phishing page also includes a simple caching service
worker to ensure it continues to work offline (or if our hosting goes
offline), as long as the user has successfully loaded the page at least
once. We also load the page temporarily during the extension startup
process to trigger the service worker installation.
The old phishing page and all related lines have been removed. The
property `web_accessible_resources` has also been removed from the
manifest. The only entry apart from the phishing page was `inpage.js`,
and we don't need that to be web accessible anymore because we inject
the script inline into each page rather than loading the file directly.
New e2e tests have been added to cover more phishing warning page
functionality, including the "safelist" action and the "iframe" case.
#14583 broke the development build scripts (e.g. `yarn start`) by adding a positional argument to a package script (`build:dev`) that is used and passed positional arguments in the build script itself. This PR removes the positional argument from the `build:dev` script and `yarn start` now works again. In addition, the `--apply-lavamoat` flag is properly forwarded to child processes, which was not the case in the original implementation.
To test, `yarn start` should work and LavaMoat should _not_ be applied, in distinction to `yarn build:dev dev --apply-lavamoat=true`. Whether LavaMoat is applied can be determined by checking whether `Object.isFrozen(Object.prototype)` is `true` (with LavaMoat) or `false` (without LavaMoat).
Adds a new flag, `--apply-lavamoat`, to the main build script. The flag controls whether LavaMoat is actually applied to the output of the build process. The flag defaults to `true`, but we explicitly set it to `false` in the `start` package script. Meanwhile, the `start:lavamoat` script is modified such that it applies LavaMoat to the build output in development mode, but it no longer runs the build process itself under LavaMoat as there aren't very compelling reasons to do so.
This change is motivated by the fact that development builds do not have their own dedicated LavaMoat policies, which causes development builds to fail since #14537. The downside of this change is that LavaMoat-related failures will not be detected when running `yarn start`. @kumavis has plans for fixing this problem in a future major version of the `@lavamoat` suite.
* lavamoat - apply lavamoat protections to popup and notification
* build - enable lavamoat for home
* lavamoat - add missing ui overrides for react family
* deps/patches - patch zxcvbn for ses compat
Certain build steps accidentally omitted the `version` variable. It has
now been restored to all steps, ensuring that all environment variables
are correctly injected into all bundles.
A check has been added to the Sentry setup module to ensure the release
is not omitted in the future.
Certain build steps accidentally omitted the `version` variable. It has
now been restored to all steps, ensuring that all environment variables
are correctly injected into all bundles.
A check has been added to the Sentry setup module to ensure the release
is not omitted in the future.
This commit modifies the build system so that TypeScript files can be
transpiled into ES5 just like JavaScript files.
Note that this commit does NOT change the build system to run TypeScript
files through the TypeScript compiler. In other words, no files will be
type-checked at the build stage, as we expect type-checking to be
handled elsewhere (live, via your editor integration with `tsserver`,
and before a PR is merged, via `yarn lint`). Rather, we merely instruct
Babel to strip TypeScript-specific syntax from any files that have it,
as if those files had been written using JavaScript syntax alone.
Why take this approach? Because it prevents the build process from being
negatively impacted with respect to performance (as TypeScript takes a
significant amount of time to run).
It's worth noting the downside of this approach: because we aren't
running files through TypeScript, but relying on Babel's [TypeScript
transform][1] to identify TypeScript syntax, this transform has to keep
up with any syntax changes that TypeScript adds in the future. In fact
there are a few syntactical forms that Babel already does not recognize.
These forms are rare or are deprecated by TypeScript, so I don't
consider them to be a blocker, but it's worth noting just in case it
comes up later. Also, any settings we place in `tsconfig.json` will be
completely ignored by Babel. Again, this isn't a blocker because there
are some analogs for the most important settings reflected in the
options we can pass to the transform. These and other caveats are
detailed in the [documentation for the transform][2].
[1]: https://babeljs.io/docs/en/babel-plugin-transform-typescript
[2]: https://babeljs.io/docs/en/babel-plugin-transform-typescript#caveats
The version of a build is now derived from both the `version` field in
`package.json` and the requested build type and version. The build type
and version are added onto the manifest version as a suffix, according
to the SemVer prerelease format.
We already have support in the extension for versions of this format,
but to apply a Flask or Beta version required manual updates to
`package.json`. Now it can be done just with build arguments.
A `get-version` module was created to make it easier to generate the
version in the various places we do that during the build. It was
created in the `development/lib` directory because it will be used by
other non-build development scripts in a future PR.
The `BuildType` constant was extracted to its own module as well, and
moved to the `development/lib` directory. This was to make it clear
that it's used by various different development scripts, not just the
build.
The version of a build is now derived from both the `version` field in
`package.json` and the requested build type and version. The build type
and version are added onto the manifest version as a suffix, according
to the SemVer prerelease format.
We already have support in the extension for versions of this format,
but to apply a Flask or Beta version required manual updates to
`package.json`. Now it can be done just with build arguments.
A `get-version` module was created to make it easier to generate the
version in the various places we do that during the build. It was
created in the `development/lib` directory because it will be used by
other non-build development scripts in a future PR.
The `BuildType` constant was extracted to its own module as well, and
moved to the `development/lib` directory. This was to make it clear
that it's used by various different development scripts, not just the
build.
If an error occurs while running Browserify, the stream that Browserify
creates will emit an `error` event. However, this event is not being
handled, so Node will catch it instead. But the error message it
produces is very nebulous, as it merely spits out the stream object and
completely ignores the actual error that occurred. So this commit
listens for the `error` event and outputs the error.
One note here is that when we are outputting the error, we must get
around a bug that exists in Endo where if you pass an Error object to
`console.{log,error,info,debug}` then you will just see `{}` on-screen.
We get around this by printing `err.stack`.
ESLint rules have been added to enforce our JSDoc conventions. These
rules were introduced by updating `@metamask/eslint-config` to v9.
Some of the rules have been disabled because the effort to fix all lint
errors was too high. It might be easiest to enable these rules one
directory at a time, or one rule at a time.
Most of the changes in this PR were a result of running
`yarn lint:fix`. There were a handful of manual changes that seemed
obvious and simple to make. Anything beyond that and the rule was left
disabled.
The ESLint config for the extension explicitly includes support for
Prettier. However, this is already being provided by our global ESLint
config (`@metamask/eslint-config`). Therefore there is no need to
include it here. In fact, this is causing weird issues where the `curly`
option is getting overridden somehow. After this change, these syntaxes
are invalid:
``` javascript
if (foo) return;
```
``` javascript
if (foo) return 'bar';
```
`remote-redux-devtools` is now explicitly excluded and disabled in non-
dev builds, and in the `testDev` build. This was causing console errors
in the `testDev` build during e2e tests, which would cause certain
tests to fail.
This was already only supposed to be enabled for development builds,
but this library used the `NODE_ENV` environment variable to make that
determination. This gives us more control over when it's disabled.
The React dev tools can result in console errors if dev tools is not
open during the test. Some of our e2e tests fail if there are any
console errors, so these errors break those tests.
`react-devtools` has been completely disabled for `testDev` builds to
make debugging e2e tests easier. The React dev tools can still be used
from development builds.
A propType error was showing up during e2e tests with a `testDev`
build. It was caused by `process.env.IN_TEST` being treated as a
boolean, when in fact it is either the string `'true'` or a boolean.
`IN_TEST` has been updated to always be a boolean. `loose-envify` has
no trouble injecting boolean values, so there's no reason to treat this
as a string.
The environment variables used for test builds was wrong for certain
bundles because the `testing` flag wasn't passed through to the
function that determines which environment variables to inject.
Effectively this means that test builds on `master` were going to the
production `metamask` Sentry project rather than the `test-metamask`
project. This has been the case since #11080.
The `testing` flag is now included for all bundles, and test builds now
use the `test-metamask` Sentry project in all cases.
The LavaMoat policy generation script would sporadically fail because
it ran the build concurrently three times, and the build includes
steps that delete the `dist` directory and write to it. So if one build
process tried to write to the directory after another deleted it, it
would fail.
This was solved by adding a new `--policy-only` flag to the build
script, and a new `scripts:prod` task. The `scripts:prod` task only
runs the script tasks for prod, rather than the entire build process.
The `--policy-only` flag stops the script tasks once the policy has
been written, and stops any other files from being written to disk.
This prevents the three concurrent build processes from getting in each
others way, and it dramatically speeds up the process.
The LavaMoat policy generation script would sporadically fail because
it ran the build concurrently three times, and the build includes
steps that delete the `dist` directory and write to it. So if one build
process tried to write to the directory after another deleted it, it
would fail.
This was solved by adding a new `--policy-only` flag to the build
script, and a new `scripts:prod` task. The `scripts:prod` task only
runs the script tasks for prod, rather than the entire build process.
The `--policy-only` flag stops the script tasks once the policy has
been written, and stops any other files from being written to disk.
This prevents the three concurrent build processes from getting in each
others way, and it dramatically speeds up the process.
The environment variables used for test builds was wrong for certain
bundles because the `testing` flag wasn't passed through to the
function that determines which environment variables to inject.
Effectively this means that test builds on `master` were going to the
production `metamask` Sentry project rather than the `test-metamask`
project. This has been the case since #11080.
The `testing` flag is now included for all bundles, and test builds now
use the `test-metamask` Sentry project in all cases.
This PR adds one LavaMoat background script policy or each build type. It also renames the build system policy directory from `node` to `build-system` to make its purpose more clear. Each build type has the original `policy-override.json` for `main` builds. The `.prettierignore` file has been updated to match the locations of the new auto-generated policy files.
We need to maintain separate policies for each build type because each type will produce different bundles with different internal and external modules.
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions.
The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details.
### Code Fencing and ESLint
When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor.
### File Exclusion
For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`.
As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice.
Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
Mark Stacey
Brad Decker
Jyoti Puri
ryanml
Dan J Miller
kumavis
Erik Marks
Elliot Winkler
David Walsh
Niranjana Binoy