Bundlesize stats over time (#15209)

* Adding tasks for MV3 test build

* more changes

* fix

* fix

* fix

* MV3 CI fixes

* fixes

* fix

* Initial work to capture the logs, write to file, and parse it.

Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>

* chrome debug logs

* Pull logs from webdriver and save to json file.

Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>

* comment out stats stuffs

Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>

* Lint Fixes

Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>

* Remove console.log

Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>

* fixes

* Adding build artifact for initialisation time

* change

* fix lint

* fix

* fix

* fix

* fix

* fixes

* Capturing load time stats

* fix

* fix

* fix

* fix

* fix build

* fix

* fix

* fix

* fix

* fix

* Initial work to capture the logs, write to file, and parse it.

Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>

* chrome debug logs

* Pull logs from webdriver and save to json file.

Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>

* comment out stats stuffs

Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>

* Lint Fixes

Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>

* Remove console.log

Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>

* fix

* fix

* fix

* fix

* Initial work to capture the logs, write to file, and parse it.

Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>

chrome debug logs

Pull logs from webdriver and save to json file.

Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>

Remove console.log

Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>

split output of lavamoat to 2 files. background and ui

Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>

* fix

* fix

* fix

* Enable logging to chrome_debug only if in MV3

Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>

* fix

* fix

* fix

* fixes

* fix

* fix

* fix

* fix

* fixes

* test

* test

* fix

* fix

* MV3 bundle size stats

* fix

* Committing bundle size status to extension_bundlesize_stats repo

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* Initial commit

* Initial commit

* Initial commit

* Initial commit

* fixes

* fix

* fix

* fix

* fix

* fix

* fix

Co-authored-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>
Co-authored-by: PeterYinusa <peter.yinusa@consensys.net>

.circleci/config.yml

@@ -627,6 +627,9 @@ jobs:
       - run:
           name: Run page load benchmark
           command: yarn mv3:stats:chrome --out test-artifacts/chrome/mv3
+      - run:
+          name: Record bundle size at commit
+          command: ./.circleci/scripts/bundle-stats-commit.sh
       - store_artifacts:
           path: test-artifacts
           destination: test-artifacts

.circleci/scripts/bundle-stats-commit.sh

@@ -0,0 +1,70 @@
+#!/usr/bin/env bash
+
+set -e
+set -u
+set -o pipefail
+
+if [[ "${CI:-}" != 'true' ]]
+then
+    printf '%s\n' 'CI environment variable must be set to true'
+    exit 1
+fi
+
+if [[ "${CIRCLECI:-}" != 'true' ]]
+then
+    printf '%s\n' 'CIRCLECI environment variable must be set to true'
+    exit 1
+fi
+
+if [[ -z "${GITHUB_TOKEN:-}" ]]
+then
+    printf '%s\n' 'GITHUB_TOKEN environment variable must be set'
+    exit 1
+elif [[ -z "${GITHUB_TOKEN_USER:-}" ]]
+then
+    printf '%s\n' 'GITHUB_TOKEN_USER environment variable must be set'
+    exit 1
+fi
+
+printf '%s\n' 'Commit the manifest version and changelog if the manifest has changed'
+
+if [[ "${CIRCLE_BRANCH}" != "develop" ]]
+then
+    printf 'This is not develop branch'
+    exit 0
+fi
+
+mkdir temp
+
+git config --global user.email "metamaskbot@users.noreply.github.com"
+
+git config --global user.name "MetaMask Bot"
+
+git clone git@github.com:MetaMask/extension_bundlesize_stats.git temp
+
+if [[ -f "temp/stats/bundle_size_stats-${CIRCLE_SHA1}.json" ]]
+then
+    printf 'Bundle size of the commit is already recorded'
+    cd ..
+    rm -rf temp
+    exit 0
+fi
+
+cp -R test-artifacts/chrome/mv3/bundle_size_stats.json temp/stats
+
+echo " bundle_size_stats-${CIRCLE_SHA1}.json" >> temp/stats/fileList.txt
+
+mv temp/stats/bundle_size_stats.json "temp/stats/bundle_size_stats-${CIRCLE_SHA1}.json"
+
+cd temp
+
+git add .
+
+git commit --message "Bundle size at commit: ${CIRCLE_SHA1}"
+
+repo_slug="$CIRCLE_PROJECT_USERNAME/extension_bundlesize_stats"
+git push "https://$GITHUB_TOKEN_USER:$GITHUB_TOKEN@github.com/$repo_slug" main
+
+cd ..
+
+rm -rf temp

test/e2e/mv3-perf-stats/bundle-size.js

@@ -54,7 +54,7 @@ async function main() {
   const backgroundFileList = [];
   const uiFileList = [];
 
-  const files = fs.readdir(distFolder);
+  const files = await fs.readdir(distFolder);
   for (let i = 0; i < files.length; i++) {
     const file = files[i];
     if (CommonFileRegex.test(file)) {