The e2e test driver used to perform the initial navigation
automatically within the `buildWebDriver` function, so that that step
wouldn't need to be repeated at the beginning of each test. However
this prevented you from doing any setup in the test before the first
navigation.
The navigation has now been moved into each individual test. It should
be functionally equivalent, except now it's possible to control exactly
when the first navigation occurs.
A 1 second delay was also removed, as it didn't seem to be necessary
when testing this. It was initially added as an attempted fix to an
intermittent failure. It did not fix that failure.
* Fix require-unicode-regexp issues
See [`require-unicode-regexp`](https://eslint.org/docs/rules/require-unicode-regexp) for more information.
This change enables `require-unicode-regexp` and fixes the issues raised by the rule.
* Remove case-insensitive flag from regexps
The fullscreen UI now shows roughly the same design as the popup UI.
A few additional changes depicted in the new fullscreen designs will
be implemented in subsequent PRs (e.g. the inline buttons on assets)
This was done now to make asset pages easier to implement. Implementing
asset pages solely for the popup UI would have been complicated by the
fact that we use viewport size to switch between the two layouts, so we
would have had to re-route upon resizing the window.
* Add benchmark to CI
The page load benchmark for Chrome is now run during CI, and the
results are collected and summarized in the `metamaskbot` comment.
Closes#6881
* Double default number of samples
The number of default samples was changed from 10 to 20. The results
from 10 samples would show statistically significant changes in page
load times between builds, so weren't a sufficiently useful metric.
A margin of error metric has been added, which is calculated from a 95%
confidence interval. This confidence interval is calculated using
Student's t-distribution, which is generally preferred for smaller
sample sizes (< ~30) of populations following a normal distribution.
The script `benchmark.js` will collect page load metrics from the
extension, and print them to a file or the console. A method for
collecting metrics was added to the web driver to help with this.
This script will calculate the min, max, average, and standard
deviation for four metrics: 'firstPaint', 'domContentLoaded', 'load',
and 'domInteractive'. The variation between samples is sometimes high,
with the results varying between samples if only 3 were taken. However,
all tests I've done locally with 5 samples have produced results within
one standard deviation of each other. The default number of samples has
been set to 10, which should be more than enough to produce consistent
results.
The benchmark can be run with the npm script `benchmark:chrome` or
`benchmark:firefox`, e.g. `yarn benchmark:chrome`.
Erik Marks
Mark Stacey
Brad Decker
Whymarrh Whitby