A Metamask fork with Infura removed and default networks editable
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
ciphermask/development/build/scripts.js

855 lines
25 KiB

Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
const { callbackify } = require('util');
const path = require('path');
const { writeFileSync, readFileSync } = require('fs');
const EventEmitter = require('events');
const gulp = require('gulp');
const watch = require('gulp-watch');
const Vinyl = require('vinyl');
const source = require('vinyl-source-stream');
const buffer = require('vinyl-buffer');
const log = require('fancy-log');
const browserify = require('browserify');
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
const watchify = require('watchify');
const babelify = require('babelify');
const brfs = require('brfs');
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
const envify = require('loose-envify/custom');
const sourcemaps = require('gulp-sourcemaps');
const applySourceMap = require('vinyl-sourcemaps-apply');
const pify = require('pify');
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
const through = require('through2');
const endOfStream = pify(require('end-of-stream'));
const labeledStreamSplicer = require('labeled-stream-splicer').obj;
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
const wrapInStream = require('pumpify').obj;
const Sqrl = require('squirrelly');
const lavapack = require('@lavamoat/lavapack');
const lavamoatBrowserify = require('lavamoat-browserify');
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
const terser = require('terser');
const bifyModuleGroups = require('bify-module-groups');
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
const metamaskrc = require('rc')('metamask', {
INFURA_PROJECT_ID: process.env.INFURA_PROJECT_ID,
INFURA_BETA_PROJECT_ID: process.env.INFURA_BETA_PROJECT_ID,
INFURA_FLASK_PROJECT_ID: process.env.INFURA_FLASK_PROJECT_ID,
INFURA_PROD_PROJECT_ID: process.env.INFURA_PROD_PROJECT_ID,
ONBOARDING_V2: process.env.ONBOARDING_V2,
COLLECTIBLES_V1: process.env.COLLECTIBLES_V1,
SEGMENT_HOST: process.env.SEGMENT_HOST,
SEGMENT_WRITE_KEY: process.env.SEGMENT_WRITE_KEY,
SEGMENT_BETA_WRITE_KEY: process.env.SEGMENT_BETA_WRITE_KEY,
SEGMENT_FLASK_WRITE_KEY: process.env.SEGMENT_FLASK_WRITE_KEY,
SEGMENT_PROD_WRITE_KEY: process.env.SEGMENT_PROD_WRITE_KEY,
SENTRY_DSN_DEV:
process.env.SENTRY_DSN_DEV ||
'https://f59f3dd640d2429d9d0e2445a87ea8e1@sentry.io/273496',
});
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
const { streamFlatMap } = require('../stream-flat-map.js');
const { version } = require('../../package.json');
const {
createTask,
composeParallel,
composeSeries,
runInChildProcess,
} = require('./task');
const {
createRemoveFencedCodeTransform,
} = require('./transforms/remove-fenced-code');
const { BuildType } = require('./utils');
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
/**
* The build environment. This describes the environment this build was produced in.
*/
const ENVIRONMENT = {
DEVELOPMENT: 'development',
PRODUCTION: 'production',
OTHER: 'other',
PULL_REQUEST: 'pull-request',
RELEASE_CANDIDATE: 'release-candidate',
STAGING: 'staging',
TESTING: 'testing',
};
/**
* Get a value from the configuration, and confirm that it is set.
*
* @param {string} key - The configuration key to retrieve.
* @returns {string} The config entry requested.
* @throws {Error} Throws if the requested key is missing.
*/
function getConfigValue(key) {
const value = metamaskrc[key];
if (!value) {
throw new Error(`Missing config entry for '${key}'`);
}
return value;
}
/**
* Get the appropriate Infura project ID.
*
* @param {object} options - The Infura project ID options.
* @param {BuildType} options.buildType - The current build type.
* @param {ENVIRONMENT[keyof ENVIRONMENT]} options.environment - The build environment.
* @param {boolean} options.testing - Whether the current build is a test build or not.
* @returns {string} The Infura project ID.
*/
function getInfuraProjectId({ buildType, environment, testing }) {
if (testing) {
return '00000000000000000000000000000000';
} else if (environment !== ENVIRONMENT.PRODUCTION) {
// Skip validation because this is unset on PRs from forks.
return metamaskrc.INFURA_PROJECT_ID;
} else if (buildType === BuildType.main) {
return getConfigValue('INFURA_PROD_PROJECT_ID');
} else if (buildType === BuildType.beta) {
return getConfigValue('INFURA_BETA_PROJECT_ID');
} else if (buildType === BuildType.flask) {
return getConfigValue('INFURA_FLASK_PROJECT_ID');
}
throw new Error(`Invalid build type: '${buildType}'`);
}
/**
* Get the appropriate Segment write key.
*
* @param {object} options - The Segment write key options.
* @param {BuildType} options.buildType - The current build type.
* @param {keyof ENVIRONMENT} options.environment - The current build environment.
* @returns {string} The Segment write key.
*/
function getSegmentWriteKey({ buildType, environment }) {
if (environment !== ENVIRONMENT.PRODUCTION) {
// Skip validation because this is unset on PRs from forks, and isn't necessary for development builds.
return metamaskrc.SEGMENT_WRITE_KEY;
} else if (buildType === BuildType.main) {
return getConfigValue('SEGMENT_PROD_WRITE_KEY');
} else if (buildType === BuildType.beta) {
return getConfigValue('SEGMENT_BETA_WRITE_KEY');
} else if (buildType === BuildType.flask) {
return getConfigValue('SEGMENT_FLASK_WRITE_KEY');
}
throw new Error(`Invalid build type: '${buildType}'`);
}
const noopWriteStream = through.obj((_file, _fileEncoding, callback) =>
callback(),
);
module.exports = createScriptTasks;
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
function createScriptTasks({
browserPlatforms,
buildType,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
ignoredFiles,
isLavaMoat,
livereload,
shouldLintFenceFiles,
policyOnly,
}) {
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
// internal tasks
const core = {
// dev tasks (live reload)
dev: createTasksForBuildJsExtension({
taskPrefix: 'scripts:core:dev',
devMode: true,
}),
testDev: createTasksForBuildJsExtension({
taskPrefix: 'scripts:core:test-live',
devMode: true,
testing: true,
}),
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
// built for CI tests
test: createTasksForBuildJsExtension({
taskPrefix: 'scripts:core:test',
testing: true,
}),
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
// production
prod: createTasksForBuildJsExtension({ taskPrefix: 'scripts:core:prod' }),
};
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
// high level tasks
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
const { dev, test, testDev, prod } = core;
return { dev, test, testDev, prod };
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
function createTasksForBuildJsExtension({ taskPrefix, devMode, testing }) {
const standardEntryPoints = ['background', 'ui', 'content-script'];
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
const standardSubtask = createTask(
`${taskPrefix}:standardEntryPoints`,
createFactoredBuild({
browserPlatforms,
buildType,
devMode,
entryFiles: standardEntryPoints.map((label) => {
if (label === 'content-script') {
return './app/vendor/trezor/content-script.js';
}
return `./app/scripts/${label}.js`;
}),
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
ignoredFiles,
policyOnly,
shouldLintFenceFiles,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
testing,
}),
);
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
// inpage must be built before contentscript
// because inpage bundle result is included inside contentscript
const contentscriptSubtask = createTask(
`${taskPrefix}:contentscript`,
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
createTaskForBundleContentscript({ devMode, testing }),
);
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
// this can run whenever
const disableConsoleSubtask = createTask(
`${taskPrefix}:disable-console`,
createTaskForBundleDisableConsole({ devMode, testing }),
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
);
// this can run whenever
const installSentrySubtask = createTask(
`${taskPrefix}:sentry`,
createTaskForBundleSentry({ devMode, testing }),
);
const phishingDetectSubtask = createTask(
`${taskPrefix}:phishing-detect`,
createTaskForBundlePhishingDetect({ devMode, testing }),
);
// task for initiating browser livereload
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
const initiateLiveReload = async () => {
if (devMode) {
// trigger live reload when the bundles are updated
// this is not ideal, but overcomes the limitations:
// - run from the main process (not child process tasks)
// - after the first build has completed (thus the timeout)
// - build tasks never "complete" when run with livereload + child process
setTimeout(() => {
watch('./dist/*/*.js', (event) => {
livereload.changed(event.path);
});
}, 75e3);
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
}
};
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
// make each bundle run in a separate process
const allSubtasks = [
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
standardSubtask,
contentscriptSubtask,
disableConsoleSubtask,
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
installSentrySubtask,
phishingDetectSubtask,
].map((subtask) =>
runInChildProcess(subtask, {
buildType,
isLavaMoat,
policyOnly,
shouldLintFenceFiles,
}),
);
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
// make a parent task that runs each task in a child thread
return composeParallel(initiateLiveReload, ...allSubtasks);
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
}
function createTaskForBundleDisableConsole({ devMode, testing }) {
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
const label = 'disable-console';
return createNormalBundle({
browserPlatforms,
buildType,
destFilepath: `${label}.js`,
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
devMode,
entryFilepath: `./app/scripts/${label}.js`,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
ignoredFiles,
label,
testing,
policyOnly,
shouldLintFenceFiles,
});
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
}
function createTaskForBundleSentry({ devMode, testing }) {
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
const label = 'sentry-install';
return createNormalBundle({
browserPlatforms,
buildType,
destFilepath: `${label}.js`,
devMode,
entryFilepath: `./app/scripts/${label}.js`,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
ignoredFiles,
label,
testing,
policyOnly,
shouldLintFenceFiles,
});
}
function createTaskForBundlePhishingDetect({ devMode, testing }) {
const label = 'phishing-detect';
return createNormalBundle({
buildType,
browserPlatforms,
destFilepath: `${label}.js`,
devMode,
entryFilepath: `./app/scripts/${label}.js`,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
ignoredFiles,
label,
testing,
policyOnly,
shouldLintFenceFiles,
});
}
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
// the "contentscript" bundle contains the "inpage" bundle
function createTaskForBundleContentscript({ devMode, testing }) {
const inpage = 'inpage';
const contentscript = 'contentscript';
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
return composeSeries(
createNormalBundle({
buildType,
browserPlatforms,
destFilepath: `${inpage}.js`,
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
devMode,
entryFilepath: `./app/scripts/${inpage}.js`,
label: inpage,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
ignoredFiles,
policyOnly,
shouldLintFenceFiles,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
testing,
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
}),
createNormalBundle({
buildType,
browserPlatforms,
destFilepath: `${contentscript}.js`,
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
devMode,
entryFilepath: `./app/scripts/${contentscript}.js`,
label: contentscript,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
ignoredFiles,
policyOnly,
shouldLintFenceFiles,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
testing,
}),
);
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
}
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
function createFactoredBuild({
browserPlatforms,
buildType,
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
devMode,
entryFiles,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
ignoredFiles,
policyOnly,
shouldLintFenceFiles,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
testing,
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
}) {
return async function () {
// create bundler setup and apply defaults
const buildConfiguration = createBuildConfiguration();
buildConfiguration.label = 'primary';
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
const { bundlerOpts, events } = buildConfiguration;
// devMode options
const reloadOnChange = Boolean(devMode);
const minify = Boolean(devMode) === false;
const envVars = getEnvironmentVariables({ buildType, devMode, testing });
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
setupBundlerDefaults(buildConfiguration, {
buildType,
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
devMode,
envVars,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
ignoredFiles,
policyOnly,
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
minify,
reloadOnChange,
shouldLintFenceFiles,
testing,
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
});
// set bundle entries
bundlerOpts.entries = [...entryFiles];
// setup lavamoat
// lavamoat will add lavapack but it will be removed by bify-module-groups
// we will re-add it later by installing a lavapack runtime
const lavamoatOpts = {
policy: path.resolve(
__dirname,
`../../lavamoat/browserify/${buildType}/policy.json`,
),
policyName: buildType,
policyOverride: path.resolve(
__dirname,
`../../lavamoat/browserify/${buildType}/policy-override.json`,
),
writeAutoPolicy: process.env.WRITE_AUTO_POLICY,
};
Object.assign(bundlerOpts, lavamoatBrowserify.args);
bundlerOpts.plugin.push([lavamoatBrowserify, lavamoatOpts]);
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
// setup bundle factoring with bify-module-groups plugin
// note: this will remove lavapack, but its ok bc we manually readd it later
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
Object.assign(bundlerOpts, bifyModuleGroups.plugin.args);
bundlerOpts.plugin = [...bundlerOpts.plugin, [bifyModuleGroups.plugin]];
// instrument pipeline
let sizeGroupMap;
events.on('configurePipeline', ({ pipeline }) => {
// to be populated by the group-by-size transform
sizeGroupMap = new Map();
pipeline.get('groups').unshift(
// factor modules
bifyModuleGroups.groupByFactor({
entryFileToLabel(filepath) {
return path.parse(filepath).name;
},
}),
// cap files at 2 mb
bifyModuleGroups.groupBySize({
sizeLimit: 2e6,
groupingMap: sizeGroupMap,
}),
);
// converts each module group into a single vinyl file containing its bundle
const moduleGroupPackerStream = streamFlatMap((moduleGroup) => {
const filename = `${moduleGroup.label}.js`;
const childStream = wrapInStream(
moduleGroup.stream,
// we manually readd lavapack here bc bify-module-groups removes it
lavapack({ raw: true, hasExports: true, includePrelude: false }),
source(filename),
);
return childStream;
});
pipeline.get('vinyl').unshift(moduleGroupPackerStream, buffer());
// add lavamoat policy loader file to packer output
moduleGroupPackerStream.push(
new Vinyl({
path: 'policy-load.js',
contents: lavapack.makePolicyLoaderStream(lavamoatOpts),
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
}),
);
// setup bundle destination
browserPlatforms.forEach((platform) => {
const dest = `./dist/${platform}/`;
const destination = policyOnly ? noopWriteStream : gulp.dest(dest);
pipeline.get('dest').push(destination);
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
});
});
// wait for bundle completion for postprocessing
events.on('bundleDone', () => {
// Skip HTML generation if nothing is to be written to disk
if (policyOnly) {
return;
}
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
const commonSet = sizeGroupMap.get('common');
// create entry points for each file
for (const [groupLabel, groupSet] of sizeGroupMap.entries()) {
// skip "common" group, they are added to all other groups
if (groupSet === commonSet) {
continue;
}
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
switch (groupLabel) {
case 'ui': {
renderHtmlFile({
htmlName: 'popup',
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
groupSet,
commonSet,
browserPlatforms,
useLavamoat: false,
});
renderHtmlFile({
htmlName: 'notification',
groupSet,
commonSet,
browserPlatforms,
useLavamoat: false,
});
renderHtmlFile({
htmlName: 'home',
groupSet,
commonSet,
browserPlatforms,
useLavamoat: false,
});
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
break;
}
case 'background': {
renderHtmlFile({
htmlName: 'background',
groupSet,
commonSet,
browserPlatforms,
useLavamoat: true,
});
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
break;
}
case 'content-script': {
renderHtmlFile({
htmlName: 'trezor-usb-permissions',
groupSet,
commonSet,
browserPlatforms,
useLavamoat: false,
});
break;
}
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
default: {
throw new Error(
`build/scripts - unknown groupLabel "${groupLabel}"`,
);
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
}
}
}
});
await bundleIt(buildConfiguration);
};
}
function createNormalBundle({
browserPlatforms,
buildType,
destFilepath,
devMode,
entryFilepath,
extraEntries = [],
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
ignoredFiles,
label,
policyOnly,
modulesToExpose,
shouldLintFenceFiles,
testing,
}) {
return async function () {
// create bundler setup and apply defaults
const buildConfiguration = createBuildConfiguration();
buildConfiguration.label = label;
const { bundlerOpts, events } = buildConfiguration;
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
// devMode options
const reloadOnChange = Boolean(devMode);
const minify = Boolean(devMode) === false;
const envVars = getEnvironmentVariables({ buildType, devMode, testing });
setupBundlerDefaults(buildConfiguration, {
buildType,
devMode,
envVars,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
ignoredFiles,
policyOnly,
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
minify,
reloadOnChange,
shouldLintFenceFiles,
testing,
});
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
// set bundle entries
bundlerOpts.entries = [...extraEntries];
if (entryFilepath) {
bundlerOpts.entries.push(entryFilepath);
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
if (modulesToExpose) {
bundlerOpts.require = bundlerOpts.require.concat(modulesToExpose);
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
// instrument pipeline
events.on('configurePipeline', ({ pipeline }) => {
// convert bundle stream to gulp vinyl stream
// and ensure file contents are buffered
pipeline.get('vinyl').push(source(destFilepath));
pipeline.get('vinyl').push(buffer());
// setup bundle destination
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
browserPlatforms.forEach((platform) => {
const dest = `./dist/${platform}/`;
const destination = policyOnly ? noopWriteStream : gulp.dest(dest);
pipeline.get('dest').push(destination);
});
});
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
await bundleIt(buildConfiguration);
};
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
function createBuildConfiguration() {
const label = '(unnamed bundle)';
const events = new EventEmitter();
const bundlerOpts = {
entries: [],
transform: [],
plugin: [],
require: [],
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
// non-standard bify options
manualExternal: [],
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
manualIgnore: [],
};
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
return { bundlerOpts, events, label };
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
function setupBundlerDefaults(
buildConfiguration,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
{
buildType,
devMode,
envVars,
ignoredFiles,
policyOnly,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
minify,
reloadOnChange,
shouldLintFenceFiles,
testing,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
},
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
) {
const { bundlerOpts } = buildConfiguration;
Object.assign(bundlerOpts, {
// Source transforms
transform: [
// Remove code that should be excluded from builds of the current type
createRemoveFencedCodeTransform(buildType, shouldLintFenceFiles),
// Transpile top-level code
babelify,
// Inline `fs.readFileSync` files
brfs,
],
// Use entryFilepath for moduleIds, easier to determine origin file
fullPaths: devMode,
// For sourcemaps
debug: true,
});
// Ensure react-devtools are not included in non-dev builds
if (!devMode || testing) {
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
bundlerOpts.manualIgnore.push('react-devtools');
bundlerOpts.manualIgnore.push('remote-redux-devtools');
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
}
// Inject environment variables via node-style `process.env`
if (envVars) {
bundlerOpts.transform.push([envify(envVars), { global: true }]);
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
// Ensure that any files that should be ignored are excluded from the build
if (ignoredFiles) {
bundlerOpts.manualExclude = ignoredFiles;
}
// Setup reload on change
if (reloadOnChange) {
setupReloadOnChange(buildConfiguration);
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
if (!policyOnly) {
if (minify) {
setupMinification(buildConfiguration);
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
// Setup source maps
setupSourcemaps(buildConfiguration, { devMode });
}
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
function setupReloadOnChange({ bundlerOpts, events }) {
// Add plugin to options
Object.assign(bundlerOpts, {
plugin: [...bundlerOpts.plugin, watchify],
// Required by watchify
cache: {},
packageCache: {},
});
// Instrument pipeline
events.on('configurePipeline', ({ bundleStream }) => {
// Handle build error to avoid breaking build process
// (eg on syntax error)
bundleStream.on('error', (err) => {
gracefulError(err);
});
});
}
function setupMinification(buildConfiguration) {
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
const minifyOpts = {
mangle: {
reserved: ['MetamaskInpageProvider'],
},
};
const { events } = buildConfiguration;
events.on('configurePipeline', ({ pipeline }) => {
pipeline.get('minify').push(
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
// this is the "gulp-terser-js" wrapper around the latest version of terser
through.obj(
callbackify(async (file, _enc) => {
const input = {
[file.sourceMap.file]: file.contents.toString(),
};
const opts = {
sourceMap: {
filename: file.sourceMap.file,
content: file.sourceMap,
},
...minifyOpts,
};
const res = await terser.minify(input, opts);
file.contents = Buffer.from(res.code);
applySourceMap(file, res.map);
return file;
}),
),
);
});
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
function setupSourcemaps(buildConfiguration, { devMode }) {
const { events } = buildConfiguration;
events.on('configurePipeline', ({ pipeline }) => {
pipeline.get('sourcemaps:init').push(sourcemaps.init({ loadMaps: true }));
pipeline
.get('sourcemaps:write')
// Use inline source maps for development due to Chrome DevTools bug
// https://bugs.chromium.org/p/chromium/issues/detail?id=931675
.push(
devMode
? sourcemaps.write()
: sourcemaps.write('../sourcemaps', { addComment: false }),
);
});
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
async function bundleIt(buildConfiguration) {
const { label, bundlerOpts, events } = buildConfiguration;
const bundler = browserify(bundlerOpts);
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
// manually apply non-standard options
bundler.external(bundlerOpts.manualExternal);
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
bundler.ignore(bundlerOpts.manualIgnore);
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
if (Array.isArray(bundlerOpts.manualExclude)) {
bundler.exclude(bundlerOpts.manualExclude);
}
// output build logs to terminal
bundler.on('log', log);
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
3 years ago
// forward update event (used by watchify)
bundler.on('update', () => performBundle());
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
console.log(`Bundle start: "${label}"`);
await performBundle();
console.log(`Bundle end: "${label}"`);
async function performBundle() {
// this pipeline is created for every bundle
// the labels are all the steps you can hook into
const pipeline = labeledStreamSplicer([
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
'groups',
[],
'vinyl',
[],
'sourcemaps:init',
[],
'minify',
[],
'sourcemaps:write',
[],
'dest',
[],
]);
const bundleStream = bundler.bundle();
// trigger build pipeline instrumentations
events.emit('configurePipeline', { pipeline, bundleStream });
// start bundle, send into pipeline
bundleStream.pipe(pipeline);
// nothing will consume pipeline, so let it flow
pipeline.resume();
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
await endOfStream(pipeline);
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
// call the completion event to handle any post-processing
events.emit('bundleDone');
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
5 years ago
}
}
function getEnvironmentVariables({ buildType, devMode, testing }) {
const environment = getEnvironment({ devMode, testing });
if (environment === ENVIRONMENT.PRODUCTION && !process.env.SENTRY_DSN) {
throw new Error('Missing SENTRY_DSN environment variable');
}
return {
METAMASK_DEBUG: devMode,
METAMASK_ENVIRONMENT: environment,
METAMASK_VERSION: version,
METAMASK_BUILD_TYPE: buildType,
NODE_ENV: devMode ? ENVIRONMENT.DEVELOPMENT : ENVIRONMENT.PRODUCTION,
IN_TEST: testing,
PUBNUB_SUB_KEY: process.env.PUBNUB_SUB_KEY || '',
PUBNUB_PUB_KEY: process.env.PUBNUB_PUB_KEY || '',
CONF: devMode ? metamaskrc : {},
SENTRY_DSN: process.env.SENTRY_DSN,
SENTRY_DSN_DEV: metamaskrc.SENTRY_DSN_DEV,
INFURA_PROJECT_ID: getInfuraProjectId({ buildType, environment, testing }),
SEGMENT_HOST: metamaskrc.SEGMENT_HOST,
SEGMENT_WRITE_KEY: getSegmentWriteKey({ buildType, environment }),
SWAPS_USE_DEV_APIS: process.env.SWAPS_USE_DEV_APIS === '1',
ONBOARDING_V2: metamaskrc.ONBOARDING_V2 === '1',
COLLECTIBLES_V1: metamaskrc.COLLECTIBLES_V1 === '1',
};
}
function getEnvironment({ devMode, testing }) {
// get environment slug
if (devMode) {
return ENVIRONMENT.DEVELOPMENT;
} else if (testing) {
return ENVIRONMENT.TESTING;
} else if (process.env.CIRCLE_BRANCH === 'master') {
return ENVIRONMENT.PRODUCTION;
} else if (
/^Version-v(\d+)[.](\d+)[.](\d+)/u.test(process.env.CIRCLE_BRANCH)
) {
return ENVIRONMENT.RELEASE_CANDIDATE;
} else if (process.env.CIRCLE_BRANCH === 'develop') {
return ENVIRONMENT.STAGING;
} else if (process.env.CIRCLE_PULL_REQUEST) {
return ENVIRONMENT.PULL_REQUEST;
}
return ENVIRONMENT.OTHER;
}
function renderHtmlFile({
htmlName,
groupSet,
commonSet,
browserPlatforms,
useLavamoat,
}) {
if (useLavamoat === undefined) {
throw new Error(
'build/scripts/renderHtmlFile - must specify "useLavamoat" option',
);
}
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
const htmlFilePath = `./app/${htmlName}.html`;
const htmlTemplate = readFileSync(htmlFilePath, 'utf8');
const jsBundles = [...commonSet.values(), ...groupSet.values()].map(
(label) => `./${label}.js`,
);
const htmlOutput = Sqrl.render(htmlTemplate, { jsBundles, useLavamoat });
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
3 years ago
browserPlatforms.forEach((platform) => {
const dest = `./dist/${platform}/${htmlName}.html`;
// we dont have a way of creating async events atm
writeFileSync(dest, htmlOutput);
});
}
function beep() {
process.stdout.write('\x07');
}
function gracefulError(err) {
console.warn(err);
beep();
}