Environment variables are now considered as higher-precedence than
configuration by our build system. This means that if the same value is
set in `.metamaskrc` and in an environment variable, the environment
variable is what will be used. Previously the reverse was true, the
configuration would take precedence.
It is conventional for CLI tools to consider environment variables as
higher precedence than configuration. This makes our build system less
surprising for most people.
The portfolio URL added in #15407 was meant to be configurable via
environment variable or the `.metamaskrc` file, but that configuration
was broken. Instead the default value was always used.
That configuration has been fixed. The portfolio URL can be set either
by environment variable or configuration file.
* Version v10.18.4
* Fix default currency symbol for `wallet_addEthereumChain` + improve warnings for data that doesn't match our validation expectations (#15201)
* set more appropriate default for ticker symbol when wallet_addEthereumChain is called
* throw error to dapp when site suggests network with same chainId but different ticker symbol from already added network, instead of showing error and disabled notification to user
* Fix Provider Tracking Metrics (#15082)
* fix filetype audit (#15334)
* Remove decentralized 4byte function signature registry since it contains incorrect signatures and we can't algorithmically check for best option when 4byte.directory is down (#15300)
* remove decentralized 4byte function signature registry since it is griefed and we can't algorithmically check for best option when 4byte is down
* add migration
* remove nock of on chain registry call in getMethodDataAsync test
* remove audit exclusion (#15346)
* Updates `eth-lattice-keyring` to v0.10.0 (#15261)
This is mainly associated with an update in GridPlus SDK and enables
better strategies for fetching calldata decoder data.
`eth-lattice-keyring` changes:
GridPlus/eth-lattice-keyring@v0.7.3...v0.10.0
`gridplus-sdk` changes (which includes a codebase rewrite):
GridPlus/gridplus-sdk@v1.2.3...v2.2.2
* Fix 'block link explorer on custom networks' (#13870)
* Created a logic for the 'Add a block explorer URL'
Removed unused message
Message logic rollback
Modified history push operation
WIP: Pushing before rebasing
Applied requested changes
Removed unintenionally added code
* Lint fix
* Metrics fixed
* Stop injecting provider on docs.google.com (#15459)
* Fix setting of gasPrice when on non-eip 1559 networks (#15628)
* Fix setting of gasPrice when on non-eip 1559 networks
* Fix unit tests
* Fix logic
* Update ui/ducks/send/send.test.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* [GridPlus] Bumps `eth-lattice-keyring` to v0.11.0 (#15490)
* [GridPlus] Bumps `gridplus-sdk` to v2.2.4 (#15561)
* remove exclusions for mismatched object jsdoc type casing (#15351)
* Improve `tokenId` parsing and clean up `useAssetDetails` hook (#15304)
* Fix state creation in setupSentryGetStateGlobal (#15635)
* filter breadcrumbs for improved clarity while debugging sentry errors (#15639)
* Update v10.18.4 changelog (#15645)
* Auto generated changelog
* Update 10.18.4 changelog
* Run lavamoat:auto
* Call metrics event for wallet type selection at the right time (#15591)
* Fix Sentry in LavaMoat contexts (#15672)
Our Sentry setup relies upon application state, but it wasn't able to
access it in LavaMoat builds because it's running in a separate
Compartment.
A patch has been introduced to the LavaMoat runtime to allow the root
Compartment to mutate the `rootGlobals` object, which is accessible
from outside the compartment as well. This lets us expose application
state to our Sentry integration.
* Fix Sentry deduplication of events that were never sent (#15677)
The Sentry `Dedupe` integration has been filtering out our events, even
when they were never sent due to our `beforeSend` handler. It was
wrongly identifying them as duplicates because it has no knowledge of
`beforeSend` or whether they were actually sent or not.
To resolve this, the filtering we were doing in `beforeSend` has been
moved to a Sentry integration. This integration is installed ahead of
the `Dedupe` integration, so `Dedupe` should never find out about any
events that we filter out, and thus will never consider them as sent
when they were not.
* Replace `lavamoat-runtime.js` patch (#15682)
A patch made in #15672 was found to be unnecessary. Instead of setting
a `rootGlobals` object upon construction of the root compartment, we
are now creating a `sentryHooks` object in the initial top-level
compartment. I hadn't realized at the time that the root compartment
would inherit all properties of the initial compartment `globalThis`.
This accomplishes the same goals as #15672 except without needing a
patch.
* Update v10.18.4 changelog
* Fix lint issues
* Update yarn.lock
* Update `depcheck` to latest version (#15690)
`depcheck` has been updated to the latest version. This version pins
`@babel/parser` to v7.16.4 because of unresolved bugs in v7.16.5 that
result in `depcheck` failing to parse TypeScript files correctly.
We had a Yarn resolution in place to ensure `@babel/parser@7.16.4` was
being used already. That resolution is no longer needed so it has been
removed. This should resove the issue the dev team has been seeing
lately where `yarn` and `yarn-deduplicate` disagree about the state the
lockfile should be in.
* Update yarn.lock
* Update LavaMoat policy
* deduplicate
* Update LavaMoat build policy
Co-authored-by: MetaMask Bot <metamaskbot@users.noreply.github.com>
Co-authored-by: Alex Donesky <adonesky@gmail.com>
Co-authored-by: Brad Decker <bhdecker84@gmail.com>
Co-authored-by: Alex Miller <asmiller1989@gmail.com>
Co-authored-by: Filip Sekulic <filip.sekulic@consensys.net>
Co-authored-by: Erik Marks <25517051+rekmarks@users.noreply.github.com>
Co-authored-by: Dan J Miller <danjm.com@gmail.com>
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
Co-authored-by: seaona <54408225+seaona@users.noreply.github.com>
Co-authored-by: seaona <mariona@gmx.es>
Co-authored-by: PeterYinusa <peter.yinusa@consensys.net>
Recently in #15468 the name of the scripts task used by the LavaMoat
policy generation script was renamed from `scripts:prod` to
`scripts:dist`, but we neglected to change this name in the LavaMoat
policy generation script itself.
The script task has now been updated so that the script works again,
and the LavaMoat policy generation script has been re-run.
Validation has been added to the build script when the "prod" target is
selected. We now ensure that all expected environment variables are
set, and that no extra environment variables are present (which might
indicate that the wrong configuration file is being used).
The `prod` target uses a new `.metamaskprodrc` configuration file. Each
required variable can be specified either via environment variable or
via this config file. CI will continue set these via environment
variable, but for local manual builds we can use the config file to
simplify the build process and ensure consistency.
A new "dist" target has been added to preserve the ability to build a
"production-like" build without this validation.
The config validation is invoked early in the script, in the CLI
argument parsing step, so that it would fail more quickly. Otherwise
we'd have to wait a few minutes longer for the validation to run.
This required some refactoring, moving functions to the utility module
and moving the config to a dedicated module.
Additionally, support has been added for all environment variables to
be set via the config file. Previously the values `PUBNUB_PUB_KEY`,
`PUBNUB_SUB_KEY`, `SENTRY_DSN`, and `SWAPS_USE_DEV_APIS` could only be
set via environment variable. Now, all of these variables can be set
either way.
Closes#15003
This PR converts `generate-lavamoat-policies.sh` to `.js` using Yargs. This makes it easier to only generate policy files for a specific build type (using the `-t` flag), which is often useful during Flask development. In addition, the `lavamoat:background:auto` scripts are renamed, and the main readme is updated with some useful tips.
Note that `lavamoat:background:auto:dev` is removed and `lavamoat:background:auto` should be used during local development.
* User actions benchmark and artifacts
* Lint and fix identation
* Fix lint
* Updated path
* lint
* Add user actions benchmark to pre release job
* Remove title
* Out path updated
* See if url is finally fixed
* Adding some console logs
* lint
* fix lint
* fix lint
* Updated persisting and store artifacts path
* Added MetaMask bot correct link and remove console logs
* Remove console log
* Sort Imports
* Fix lint
* Update loadAccount function and prop name for clarity to loadNewAccount
* Run yarn setup
* Fix yarn
* Update Create Account element for Create account
* Remove unnecessary step on send
Co-authored-by: Jyoti Puri <jyotipuri@gmail.com>
As we convert parts of the codebase to TypeScript, we will want a way to
track progress. This commit adds a dashboard which displays all of the
files that we wish to convert to TypeScript and which files we've
already converted.
The list of all possible files to convert is predetermined by walking
the dependency graph of each entrypoint the build system uses to compile
the extension (the files that the entrypoint imports, the files that the
imports import, etc). The list should not need to be regenerated, but
you can do it by running:
yarn ts-migration:enumerate
The dashboard is implemented as a separate React app. The CircleCI
configuration has been updated so that when a new commit is pushed, the
React app is built and stored in the CircleCI artifacts. When a PR is
merged, the built files will be pushed to a separate repo whose sole
purpose is to serve the dashboard via GitHub Pages (this is the same
way that the Storybook works). All of the app code and script to build
the app are self-contained under
`development/ts-migration-dashboard`. To build this app yourself, you
can run:
yarn ts-migration:dashboard:build
or if you want to build automatically as you change files, run:
yarn ts-migration:dashboard:watch
Then open the following file in your browser (there is no server
component):
development/ts-migration-dashboard/build/index.html
Finally, although you shouldn't have to do this, to manually deploy the
dashboard once built, you can run:
git remote add ts-migration-dashboard git@github.com:MetaMask/metamask-extension-ts-migration-dashboard.git
yarn ts-migration:dashboard:deploy
There is a SES bug that results in errors being printed to the console
as `{}`[1]. The known workaround is to print the error stack rather
than printing the error directly. This affects our build script when it
is run with LavaMoat.
We used this workaround in one place in the build script already, but
not in the handler for task errors. We now use it in both places.
The workaround has been moved to a function that we can use throughout
the build script.
[1]: https://github.com/endojs/endo/issues/944
We use the `rc` package to read the `.metamaskrc` configuration file,
which is in "ini" format. This package has been replaced by the `ini`
package.
The `rc` package was not actively maintained, and it has had recent
security vulnerabilities. But most importantly, the config object
returned by `rc` includes a bunch of extra information that made build
script validation [1] difficult to implement. Specifically, it made it
challenging to ensure no extra environment variables were present.
The `ini` package on the other hand is simple, well maintained, and
is simpler to use. This package doesn't add any extra properties to the
object it returns, making validation easy.
[1]: https://github.com/MetaMask/metamask-extension/issues/15003
The "scripts" portion of the build script has been refactored to pass
the "build target" throughout the file. The "build target" is the
target environment for the build, reflected by the command used to
start the build (e.g. "dev", "prod", "test", or "testDev").
Beforehand we derived the variables `devMode` and `testing` from this
build target, and passed these throughout the script. However, there is
a future change [1] that requires adding a new build target that acts
like "prod" in some ways but not others. It was easier to refactor to
pass through `buildTarget` directly than it was to add a _third_
boolean flag to indirectly represent the target.
The existence of the "testDev" target made it convenient to still have
the `testing` and `devMode` flag, so helper functions were added to
derive those values from the build target. I anticipate that these will
only be needed temporarily though. We will probably be able to get rid
of the `testDev` target and the related complexities when we start
adding more flags (like `--watch`[2] and `--minify`[3]) to the build
script directly.
[1]: https://github.com/MetaMask/metamask-extension/issues/15003
[2]: https://github.com/MetaMask/metamask-extension/issues/12767
[3]: https://github.com/MetaMask/metamask-extension/issues/12768
* Fix "app-init" injection
The way we were injecting variables into the `app-init.js` bundle was
accidentally overwriting the bundle output with the raw `app-init.js`
source file. This is a problem because the bundling process handles a
lot of things we care about like source maps, polyfills and other
necessary Babel transformations, environment variable injection, and
minification.
Instead of using string replacement to inject variables, we are now
using environment variables. The old string replacement strategy has
been removed, and the `app-init.js` module is now generated using the
same process as our other bundles.
A new option, "extraEnvironmentVariables", was added to allow us to
inject environment variables specifically for this bundle.
* Add check to ensure APPLY_LAVAMOAT is set
This is a follow-up to #15318, which fixed a problem with environment
variables. Every function in this module that passes options related to
environment variables has been updated with a doc comment. This should
make it clearer which options are mandatory and which are optional,
hopefully preventing a similar mistake from happening in the future.
The environment variables `IN_TEST` and `METAMASK_DEBUG` were not
being set to `false` correctly. Instead those variables were being
skipped, and were resolved to `undefined` at runtime. This is confusing
because the other environment variables do not work that way - they can
be set to false.
The build script has been updated to ensure those two environment
variables are always set to `true` or `false` - never `undefined`.
Additionally, the `METAMASK_VERSION` environment variable was being
omitted from the `app-init.js` bundle. For the sake of consistency,
that has also been restored.
Some of the functions in `development/build/scripts.js` have been
renamed to better describe their function, and to be more consistent
with other similar functions.
Two unused options have been removed from the `createNormalBundle`
function in the build script: 'extraEntries` and `modulesToExpose`.'
Both of these options were used in the old "main" bundles, before we
began using the "factored" bundles. They have been unused since #11080.
Currently the build .zip has its time set to the Unix epoch, which
apparently causes problems on certain operating systems when in a
timezone that is behind GMT.
The build timestamp has been changed to MetaMask's birthday. Time
zone adjustments will no longer result in invalid dates.
This reverts commit f09ab88891, reversing
changes made to effc761e0e.
This is being temporarily reverted to make it easier to release an
urgent fix for v10.15.1.
The build script now uses `yargs` rather than `minimist`. The CLI is
now better documented, and we have additional validation for each
option.
A patch for `yargs` was required because it would blow up on the line
`Error.captureStackTrace`. For some reason when running under LavaMoat,
that property did not exist.
Closes#12766
#14583 broke the development build scripts (e.g. `yarn start`) by adding a positional argument to a package script (`build:dev`) that is used and passed positional arguments in the build script itself. This PR removes the positional argument from the `build:dev` script and `yarn start` now works again. In addition, the `--apply-lavamoat` flag is properly forwarded to child processes, which was not the case in the original implementation.
To test, `yarn start` should work and LavaMoat should _not_ be applied, in distinction to `yarn build:dev dev --apply-lavamoat=true`. Whether LavaMoat is applied can be determined by checking whether `Object.isFrozen(Object.prototype)` is `true` (with LavaMoat) or `false` (without LavaMoat).
Adds a new flag, `--apply-lavamoat`, to the main build script. The flag controls whether LavaMoat is actually applied to the output of the build process. The flag defaults to `true`, but we explicitly set it to `false` in the `start` package script. Meanwhile, the `start:lavamoat` script is modified such that it applies LavaMoat to the build output in development mode, but it no longer runs the build process itself under LavaMoat as there aren't very compelling reasons to do so.
This change is motivated by the fact that development builds do not have their own dedicated LavaMoat policies, which causes development builds to fail since #14537. The downside of this change is that LavaMoat-related failures will not be detected when running `yarn start`. @kumavis has plans for fixing this problem in a future major version of the `@lavamoat` suite.
Filip Sekulic
Mark Stacey
ryanml
Matthew Epps
legobeat
MetaMask Bot
Erik Marks
Niranjana Binoy
seaona
Elliot Winkler
Brad Decker
Sam Gbafa
Jyoti Puri
Dan J Miller
PeterYinusa
kumavis