* Add benchmark to CI
The page load benchmark for Chrome is now run during CI, and the
results are collected and summarized in the `metamaskbot` comment.
Closes#6881
* Double default number of samples
The number of default samples was changed from 10 to 20. The results
from 10 samples would show statistically significant changes in page
load times between builds, so weren't a sufficiently useful metric.
The only web API that our usage of DetectRTC relied upon was
'enumerateDevices', which is supported and stable among all of our
supported browsers.
Note that the error handling here is a little... non-standard, and the
logic around how Firefox and Brave are handled should be justified, but
I've left the logic as-is for now to keep this PR small.
The WebRTC spec is fairly stable these days, particularly among the
browsers we support. We don't need this shim for anything. I'm guessing
it may have been added primarily with IE in mind.
The filename is seeded by a simple use of Math.random() pulling from an alphanumeric character bank, as opposed to a more cryptographically random solution. This provides a simple layer of difficulty for bad actors to seek out the recovery phrase file.
A margin of error metric has been added, which is calculated from a 95%
confidence interval. This confidence interval is calculated using
Student's t-distribution, which is generally preferred for smaller
sample sizes (< ~30) of populations following a normal distribution.
The script `benchmark.js` will collect page load metrics from the
extension, and print them to a file or the console. A method for
collecting metrics was added to the web driver to help with this.
This script will calculate the min, max, average, and standard
deviation for four metrics: 'firstPaint', 'domContentLoaded', 'load',
and 'domInteractive'. The variation between samples is sometimes high,
with the results varying between samples if only 3 were taken. However,
all tests I've done locally with 5 samples have produced results within
one standard deviation of each other. The default number of samples has
been set to 10, which should be more than enough to produce consistent
results.
The benchmark can be run with the npm script `benchmark:chrome` or
`benchmark:firefox`, e.g. `yarn benchmark:chrome`.
The 'can retype the seed phrase' test would fail sometimes when one of
the words in the seed phrase was a subset of another word (e.g. 'issue'
and 'tissue'). This is because the selector used to find the word
looked for the first element that contained the text, rather than an
exact match.
To simplify the selector and make it more reliable, test ids were added
to each seed phrase word. The selector now uses CSS instead of XPath,
and it only finds exact matches.
A test id was also added to the div containing the shuffled seed words
to select from, so that the chosen seed words wouldn't be selected
in place of the real target when the same word appears twice.
The `confirm-seed-phrase` component extends PureComponent, so it
doesn't need a `shouldComponentUpdate` function. The state is
effectively immutable, as all state is either a primitive type or is
updated with new instances rather than mutation.
Removing this function will silence a warning message printed to the
console during e2e tests (React doesn't want you to set this function
on `PureComponent`s).
Removing this function also exposed an unused piece of state, which has
also been removed.
There have been intermittent test failures at the beginning of various
e2e test runs. Most tests start with waiting for the 'Welcome' button
to be visible and enabled, which means waiting for the loading screen
to go away.
It looks like the reason the test intermittently fails is that
sometimes the loading screen doesn't appear until a few moments _after_
the page loads (or that it vanishes and comes back).
It was rather difficult to track down each possible cause for the
loading screens, so in the meantime a pause has been added at the start
of each run. This should hopefully suffice to ensure the momentary gap
in loading has been passed by the time the first test starts up.
The `withFixtures` helper will instantiate ganache, a web driver, and
a fixture server initialized with the given set of fixtures. It is
meant to facilitating writing small, isolated e2e tests.
The first example test has been added: simple-send. It ensures that the
user can send 1 ETH to another account.
These new e2e tests will run during the normal e2e test run.
Closes#6548
This reverts commit 4b4c00e94f. The
original change was a possible optimization of CI, though it ended up
not having a huge impact. It was thought that it may have broken source
maps, because the test build overwrote the `dist` directory that the
source maps were written to. However this turned out not to be the
case, as the changes to `dist` are never persisted to the workspace.
This is being re-introduced because the test build is needed for an
additional job (the page load benchmark), and sharing the same build
for all three jobs would surely be faster than building it separately
three times.
The driver now has a page navigation function that can navigate to any
of the three primary pages used in the extension. Additional pages and
support of paths can be added later as needed.
As of #7663, an in-memory store was used in place of local storage
during e2e tests, to facilitate the use of state fixtures. However,
this made it difficult to export state during a test run. The
instructions for exporting state to create fixtures assumed that local
storage was being used.
A new global function has been added to the background context to allow
exporting state. This method is available during testing and
development, and it works with either local storage or the in-memory
store. The fixture instructions have been updated to reference this new
function.
The signature request e2e tests were previously using ropsten. This
expectation was even hard-coded into the test contract dapp.
Instead the contract-dapp has been updated to use the current `chainId`
when calling `signTypedData` (falling back to the `networkId` if
`chainId` is not set). The fixture used by `signature-request` has been
updated to use ganache.
The switch case has been moved to a separate function so that the
initialization steps following the web driver instantiation could more
easily be deduplicated.
The e2e tests were failing intermittently after removing an account
because the account was shown as not deleted after the removal. I
suspect this was because the account _had_ been removed, but that
change to the background state hadn't yet propagated to the UI.
The background state is now synced before the loading overlay for
removing the account is removed, ensuring that the removed account
cannot be seen in the UI after removal.
* Remove unused functions from `mapDispatchToProps`
The actions import was also updated to import only the two actions
used, rather than all actions.
* Remove unused container component
Well, technically it was the props injected by this container that were
unused. The container served no purpose, so the component it surrounded
is now used directly instead.
* Remove both unused `getCurrentViewContext` selectors
* Remove unused SHOW_CONFIG_PAGE action
* Remove checks for `currentView` with name `config`
Now that the SHOW_CONFIG_PAGE action has been removed, it's no longer
possible for `currentView.name` to be set to that value.
* Remove unused `wallet-view` container props
* Delete unused SHOW_SEND_PAGE and SHOW_ADD_TOKEN_PAGE actions
* Remove unused `account-menu.container` props
* Remove unused SHOW_INFO_PAGE action
* Remove unused SET_NEW_ACCOUNT_FORM action
* Wait until element is clickable before clicking in e2e tests
A new `findClickableElement` has been added to the webdriver to allow
finding an element and blocking until it's both visible and enabled.
This is now used by the pre-existing `clickElement` method as well.
All cases where something is clicked in the e2e tests have been
updated to use one of these methods, to ensure we don't run into
intermittent failures when loading screens take longer than usual.
As of #7753, the `onBlur` prop is no longer used for the `token-input`
and `currency-input` components, and the associated wrapper components
and the shared underlying component. It as been removed from all of
them.
These rows on the Advanced Settings page were being looked up in the
e2e tests by the order they appeared in. Instead they're now referenced
by data id, so that we can add new settings and re-arrange them without
breaking the e2e tests.
The transaction navigation in the e2e tests has been made simpler with
the addition of data attributes to help with finding the navigation
buttons. Each button is now labelled according to its purpose.
Mark Stacey
Whymarrh Whitby
ryanml
Erik Marks
Dan J Miller
Frankie