The home for Hyperlane core contracts, sdk packages, and other infrastructure
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
hyperlane-monorepo/.github/workflows/test.yml

344 lines
9.8 KiB

name: test
on:
# Triggers the workflow on pushes to main branch, ignoring md files
push:
branches: [main]
# Triggers on pull requests, ignoring md files
pull_request:
branches:
- '*'
merge_group:
workflow_dispatch:
concurrency:
group: e2e-${{ github.ref }}
cancel-in-progress: ${{ github.ref_name != 'main' }}
env:
LOG_LEVEL: DEBUG
LOG_FORMAT: PRETTY
CARGO_TERM_COLOR: always
RUST_BACKTRACE: full
jobs:
yarn-install:
runs-on: ubuntu-latest
steps:
- uses: actions/setup-node@v4
with:
node-version: 18
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
submodules: recursive
- name: yarn-cache
uses: buildjet/cache@v4
with:
path: |
**/node_modules
.yarn
key: ${{ runner.os }}-yarn-4.5.1-cache-${{ hashFiles('./yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-4.5.1-cache-
- name: yarn-install
run: |
yarn install
CHANGES=$(git status -s --ignore-submodules)
if [[ ! -z $CHANGES ]]; then
echo "Changes found: $CHANGES"
git diff
exit 1
fi
lint-prettier:
runs-on: ubuntu-latest
needs: [yarn-install]
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
# check out full history
fetch-depth: 0
- name: yarn-cache
uses: buildjet/cache@v4
with:
path: |
**/node_modules
.yarn
key: ${{ runner.os }}-yarn-4.5.1-cache-${{ hashFiles('./yarn.lock') }}
fail-on-cache-miss: true
- name: lint
run: yarn lint
- name: prettier
run: |
yarn prettier
CHANGES=$(git status -s)
if [[ ! -z $CHANGES ]]; then
echo "Changes found: $CHANGES"
exit 1
fi
yarn-test:
runs-on: ubuntu-latest
feat: remove extraneous caching (#4469) feat: remove yarn build cache - saves 600MB of cache space PER COMMIT - doesn't impact overall per-commit CI time - better CI time in the general case due to fewer cache evictions on the e2e matrix job - [with caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797745470) vs [no caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797213927?pr=4469) drive-by: disable docker caching for agent build - currently cache has no impact on build time of the agent image - rerunning builds for a commit or pushing empty commits should in theory lead to quicker builds with layer caching, but the main problem is that the `RUN` step of building the agents is not cached by the docker action - only RUNs with bind mounts get a checksum computed [(reference)](https://docs.docker.com/build/cache/invalidation/#general-rules) - disabling docker build caching could help save a bunch of cache space - best way to 4x speed here is moving this job to a buildjet 8vcpu runner (read [here](https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/4445) for more info) drive-by: TLC - update to latest docker build action - provides a nice build summary and can generate artifacts that help break down where build time is spent - update check-env before build, given the current deprecation annotations - https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ - coverage step doesn't need to wait for `yarn-test` anymore
2 months ago
needs: [yarn-install]
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
submodules: recursive
fetch-depth: 0
- name: foundry-install
uses: foundry-rs/foundry-toolchain@v1
feat: remove extraneous caching (#4469) feat: remove yarn build cache - saves 600MB of cache space PER COMMIT - doesn't impact overall per-commit CI time - better CI time in the general case due to fewer cache evictions on the e2e matrix job - [with caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797745470) vs [no caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797213927?pr=4469) drive-by: disable docker caching for agent build - currently cache has no impact on build time of the agent image - rerunning builds for a commit or pushing empty commits should in theory lead to quicker builds with layer caching, but the main problem is that the `RUN` step of building the agents is not cached by the docker action - only RUNs with bind mounts get a checksum computed [(reference)](https://docs.docker.com/build/cache/invalidation/#general-rules) - disabling docker build caching could help save a bunch of cache space - best way to 4x speed here is moving this job to a buildjet 8vcpu runner (read [here](https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/4445) for more info) drive-by: TLC - update to latest docker build action - provides a nice build summary and can generate artifacts that help break down where build time is spent - update check-env before build, given the current deprecation annotations - https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ - coverage step doesn't need to wait for `yarn-test` anymore
2 months ago
- name: yarn-build
uses: ./.github/actions/yarn-build-with-cache
with:
feat: remove extraneous caching (#4469) feat: remove yarn build cache - saves 600MB of cache space PER COMMIT - doesn't impact overall per-commit CI time - better CI time in the general case due to fewer cache evictions on the e2e matrix job - [with caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797745470) vs [no caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797213927?pr=4469) drive-by: disable docker caching for agent build - currently cache has no impact on build time of the agent image - rerunning builds for a commit or pushing empty commits should in theory lead to quicker builds with layer caching, but the main problem is that the `RUN` step of building the agents is not cached by the docker action - only RUNs with bind mounts get a checksum computed [(reference)](https://docs.docker.com/build/cache/invalidation/#general-rules) - disabling docker build caching could help save a bunch of cache space - best way to 4x speed here is moving this job to a buildjet 8vcpu runner (read [here](https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/4445) for more info) drive-by: TLC - update to latest docker build action - provides a nice build summary and can generate artifacts that help break down where build time is spent - update check-env before build, given the current deprecation annotations - https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ - coverage step doesn't need to wait for `yarn-test` anymore
2 months ago
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Checkout registry
uses: ./.github/actions/checkout-registry
- name: Unit Tests
run: yarn test:ci
cli-e2e-matrix:
runs-on: ubuntu-latest
needs: [yarn-install]
strategy:
fail-fast: false
matrix:
test:
- core
- relay
- warp-read
- warp-apply
- warp-deploy
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
submodules: recursive
fetch-depth: 0
- name: foundry-install
uses: foundry-rs/foundry-toolchain@v1
- name: yarn-build
uses: ./.github/actions/yarn-build-with-cache
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Checkout registry
uses: ./.github/actions/checkout-registry
- name: CLI e2e tests (${{ matrix.test }})
run: yarn --cwd typescript/cli test:e2e
env:
CLI_E2E_TEST: ${{ matrix.test }}
cli-e2e:
runs-on: ubuntu-latest
needs: cli-e2e-matrix
if: always()
steps:
- name: Check cli-e2e matrix status
if: ${{ needs.cli-e2e-matrix.result != 'success' }}
run: |
echo "CLI E2E tests failed"
exit 1
Agent transaction overrides, ensure agent configs are updated in CI (#3563) ### Description - Adds some basic transaction overrides for Ethereum chains to be used by the agents. This follows the existing config schema in chain metadata that we have been using on the TS side for a while. The overrides are as follows: - `transactionOverrides.gasPrice` - if specified, non EIP-1559 txs are used, and they use this gasPrice - `transactionOverrides.gasLimit` - if specified, this takes precedence over any gas estimation that's done and is the gas limit that's used for txs. (This is pretty niche and I don't imagine we'll ever wanna use this internally, but happens to cover the use case described [here](https://discord.com/channels/935678348330434570/984123861144600587/1224617234580766741), which is nice to cover) - `transactionOverrides.maxFeePerGas` - if specified and if the chain supports EIP 1559 txs, is used for the `maxFeePerGas` - `transactionOverrides.maxPriorityFeePerGas` - if specified and if the chain supports EIP 1559 txs, is used for the `maxPriorityFeePerGas` - Updates some of the transaction overrides in mainnet3 / testnet4 to be more reasonable now that we'll be using them to submit transactions pretty frequently - Note that atm the transaction overrides are most easily applied to the agents via the config JSONs, and not via our K8s configuration. The upshot of this is if we wanna change a transaction override right now, the easiest way to do it will be to run `update-agent-config.ts` and then build a new image. Created https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3565 to track making this experience better - Part of this change means that it's more important to keep the `transactionOverrides` specified in the SDK / infra up to date with the agent config. So I ended up doing https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3325 along the way, which also required doing https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3311. - To fix the issue with arbitrum blocks, we now require the index.from to be specified in the chain metadata if the chain's technical stack is arbitrum nitro - A new script, `update-agent-config.ts` is added, and it's ran for mainnet3 and testnet4 in CI to make sure there's no diff (similar idea to what we do with prettier) - Along the way I renamed mainnet3_config.json and testnet4_config.json to mainnet_config.json and testnet_config.json. This has been a source of confusion with ppl in the past - in v2 we named them without any numeric suffixes, so we now just do that again ### Drive-by changes - There's a drive-by to poll more frequently for pending transactions (2 seconds). The default is 7 seconds in Ethers, and for many chains that have really quick block times, this may be a way for us to speed up us learning that a transaction is included - Don't enforce gas on the inevm routes, looks like there's some weirdness there that was reported by Nam ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3562 - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3325 - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3311 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->
8 months ago
agent-configs:
runs-on: ubuntu-latest
feat: remove extraneous caching (#4469) feat: remove yarn build cache - saves 600MB of cache space PER COMMIT - doesn't impact overall per-commit CI time - better CI time in the general case due to fewer cache evictions on the e2e matrix job - [with caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797745470) vs [no caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797213927?pr=4469) drive-by: disable docker caching for agent build - currently cache has no impact on build time of the agent image - rerunning builds for a commit or pushing empty commits should in theory lead to quicker builds with layer caching, but the main problem is that the `RUN` step of building the agents is not cached by the docker action - only RUNs with bind mounts get a checksum computed [(reference)](https://docs.docker.com/build/cache/invalidation/#general-rules) - disabling docker build caching could help save a bunch of cache space - best way to 4x speed here is moving this job to a buildjet 8vcpu runner (read [here](https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/4445) for more info) drive-by: TLC - update to latest docker build action - provides a nice build summary and can generate artifacts that help break down where build time is spent - update check-env before build, given the current deprecation annotations - https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ - coverage step doesn't need to wait for `yarn-test` anymore
2 months ago
needs: [yarn-install]
Agent transaction overrides, ensure agent configs are updated in CI (#3563) ### Description - Adds some basic transaction overrides for Ethereum chains to be used by the agents. This follows the existing config schema in chain metadata that we have been using on the TS side for a while. The overrides are as follows: - `transactionOverrides.gasPrice` - if specified, non EIP-1559 txs are used, and they use this gasPrice - `transactionOverrides.gasLimit` - if specified, this takes precedence over any gas estimation that's done and is the gas limit that's used for txs. (This is pretty niche and I don't imagine we'll ever wanna use this internally, but happens to cover the use case described [here](https://discord.com/channels/935678348330434570/984123861144600587/1224617234580766741), which is nice to cover) - `transactionOverrides.maxFeePerGas` - if specified and if the chain supports EIP 1559 txs, is used for the `maxFeePerGas` - `transactionOverrides.maxPriorityFeePerGas` - if specified and if the chain supports EIP 1559 txs, is used for the `maxPriorityFeePerGas` - Updates some of the transaction overrides in mainnet3 / testnet4 to be more reasonable now that we'll be using them to submit transactions pretty frequently - Note that atm the transaction overrides are most easily applied to the agents via the config JSONs, and not via our K8s configuration. The upshot of this is if we wanna change a transaction override right now, the easiest way to do it will be to run `update-agent-config.ts` and then build a new image. Created https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3565 to track making this experience better - Part of this change means that it's more important to keep the `transactionOverrides` specified in the SDK / infra up to date with the agent config. So I ended up doing https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3325 along the way, which also required doing https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3311. - To fix the issue with arbitrum blocks, we now require the index.from to be specified in the chain metadata if the chain's technical stack is arbitrum nitro - A new script, `update-agent-config.ts` is added, and it's ran for mainnet3 and testnet4 in CI to make sure there's no diff (similar idea to what we do with prettier) - Along the way I renamed mainnet3_config.json and testnet4_config.json to mainnet_config.json and testnet_config.json. This has been a source of confusion with ppl in the past - in v2 we named them without any numeric suffixes, so we now just do that again ### Drive-by changes - There's a drive-by to poll more frequently for pending transactions (2 seconds). The default is 7 seconds in Ethers, and for many chains that have really quick block times, this may be a way for us to speed up us learning that a transaction is included - Don't enforce gas on the inevm routes, looks like there's some weirdness there that was reported by Nam ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3562 - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3325 - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3311 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->
8 months ago
strategy:
fail-fast: false
matrix:
environment: [mainnet3, testnet4]
steps:
- uses: actions/checkout@v4
Agent transaction overrides, ensure agent configs are updated in CI (#3563) ### Description - Adds some basic transaction overrides for Ethereum chains to be used by the agents. This follows the existing config schema in chain metadata that we have been using on the TS side for a while. The overrides are as follows: - `transactionOverrides.gasPrice` - if specified, non EIP-1559 txs are used, and they use this gasPrice - `transactionOverrides.gasLimit` - if specified, this takes precedence over any gas estimation that's done and is the gas limit that's used for txs. (This is pretty niche and I don't imagine we'll ever wanna use this internally, but happens to cover the use case described [here](https://discord.com/channels/935678348330434570/984123861144600587/1224617234580766741), which is nice to cover) - `transactionOverrides.maxFeePerGas` - if specified and if the chain supports EIP 1559 txs, is used for the `maxFeePerGas` - `transactionOverrides.maxPriorityFeePerGas` - if specified and if the chain supports EIP 1559 txs, is used for the `maxPriorityFeePerGas` - Updates some of the transaction overrides in mainnet3 / testnet4 to be more reasonable now that we'll be using them to submit transactions pretty frequently - Note that atm the transaction overrides are most easily applied to the agents via the config JSONs, and not via our K8s configuration. The upshot of this is if we wanna change a transaction override right now, the easiest way to do it will be to run `update-agent-config.ts` and then build a new image. Created https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3565 to track making this experience better - Part of this change means that it's more important to keep the `transactionOverrides` specified in the SDK / infra up to date with the agent config. So I ended up doing https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3325 along the way, which also required doing https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3311. - To fix the issue with arbitrum blocks, we now require the index.from to be specified in the chain metadata if the chain's technical stack is arbitrum nitro - A new script, `update-agent-config.ts` is added, and it's ran for mainnet3 and testnet4 in CI to make sure there's no diff (similar idea to what we do with prettier) - Along the way I renamed mainnet3_config.json and testnet4_config.json to mainnet_config.json and testnet_config.json. This has been a source of confusion with ppl in the past - in v2 we named them without any numeric suffixes, so we now just do that again ### Drive-by changes - There's a drive-by to poll more frequently for pending transactions (2 seconds). The default is 7 seconds in Ethers, and for many chains that have really quick block times, this may be a way for us to speed up us learning that a transaction is included - Don't enforce gas on the inevm routes, looks like there's some weirdness there that was reported by Nam ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3562 - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3325 - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3311 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->
8 months ago
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
fetch-depth: 0
feat: remove extraneous caching (#4469) feat: remove yarn build cache - saves 600MB of cache space PER COMMIT - doesn't impact overall per-commit CI time - better CI time in the general case due to fewer cache evictions on the e2e matrix job - [with caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797745470) vs [no caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797213927?pr=4469) drive-by: disable docker caching for agent build - currently cache has no impact on build time of the agent image - rerunning builds for a commit or pushing empty commits should in theory lead to quicker builds with layer caching, but the main problem is that the `RUN` step of building the agents is not cached by the docker action - only RUNs with bind mounts get a checksum computed [(reference)](https://docs.docker.com/build/cache/invalidation/#general-rules) - disabling docker build caching could help save a bunch of cache space - best way to 4x speed here is moving this job to a buildjet 8vcpu runner (read [here](https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/4445) for more info) drive-by: TLC - update to latest docker build action - provides a nice build summary and can generate artifacts that help break down where build time is spent - update check-env before build, given the current deprecation annotations - https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ - coverage step doesn't need to wait for `yarn-test` anymore
2 months ago
- name: yarn-build
uses: ./.github/actions/yarn-build-with-cache
Agent transaction overrides, ensure agent configs are updated in CI (#3563) ### Description - Adds some basic transaction overrides for Ethereum chains to be used by the agents. This follows the existing config schema in chain metadata that we have been using on the TS side for a while. The overrides are as follows: - `transactionOverrides.gasPrice` - if specified, non EIP-1559 txs are used, and they use this gasPrice - `transactionOverrides.gasLimit` - if specified, this takes precedence over any gas estimation that's done and is the gas limit that's used for txs. (This is pretty niche and I don't imagine we'll ever wanna use this internally, but happens to cover the use case described [here](https://discord.com/channels/935678348330434570/984123861144600587/1224617234580766741), which is nice to cover) - `transactionOverrides.maxFeePerGas` - if specified and if the chain supports EIP 1559 txs, is used for the `maxFeePerGas` - `transactionOverrides.maxPriorityFeePerGas` - if specified and if the chain supports EIP 1559 txs, is used for the `maxPriorityFeePerGas` - Updates some of the transaction overrides in mainnet3 / testnet4 to be more reasonable now that we'll be using them to submit transactions pretty frequently - Note that atm the transaction overrides are most easily applied to the agents via the config JSONs, and not via our K8s configuration. The upshot of this is if we wanna change a transaction override right now, the easiest way to do it will be to run `update-agent-config.ts` and then build a new image. Created https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3565 to track making this experience better - Part of this change means that it's more important to keep the `transactionOverrides` specified in the SDK / infra up to date with the agent config. So I ended up doing https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3325 along the way, which also required doing https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3311. - To fix the issue with arbitrum blocks, we now require the index.from to be specified in the chain metadata if the chain's technical stack is arbitrum nitro - A new script, `update-agent-config.ts` is added, and it's ran for mainnet3 and testnet4 in CI to make sure there's no diff (similar idea to what we do with prettier) - Along the way I renamed mainnet3_config.json and testnet4_config.json to mainnet_config.json and testnet_config.json. This has been a source of confusion with ppl in the past - in v2 we named them without any numeric suffixes, so we now just do that again ### Drive-by changes - There's a drive-by to poll more frequently for pending transactions (2 seconds). The default is 7 seconds in Ethers, and for many chains that have really quick block times, this may be a way for us to speed up us learning that a transaction is included - Don't enforce gas on the inevm routes, looks like there's some weirdness there that was reported by Nam ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3562 - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3325 - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3311 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->
8 months ago
with:
feat: remove extraneous caching (#4469) feat: remove yarn build cache - saves 600MB of cache space PER COMMIT - doesn't impact overall per-commit CI time - better CI time in the general case due to fewer cache evictions on the e2e matrix job - [with caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797745470) vs [no caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797213927?pr=4469) drive-by: disable docker caching for agent build - currently cache has no impact on build time of the agent image - rerunning builds for a commit or pushing empty commits should in theory lead to quicker builds with layer caching, but the main problem is that the `RUN` step of building the agents is not cached by the docker action - only RUNs with bind mounts get a checksum computed [(reference)](https://docs.docker.com/build/cache/invalidation/#general-rules) - disabling docker build caching could help save a bunch of cache space - best way to 4x speed here is moving this job to a buildjet 8vcpu runner (read [here](https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/4445) for more info) drive-by: TLC - update to latest docker build action - provides a nice build summary and can generate artifacts that help break down where build time is spent - update check-env before build, given the current deprecation annotations - https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ - coverage step doesn't need to wait for `yarn-test` anymore
2 months ago
ref: ${{ github.event.pull_request.head.sha || github.sha }}
Agent transaction overrides, ensure agent configs are updated in CI (#3563) ### Description - Adds some basic transaction overrides for Ethereum chains to be used by the agents. This follows the existing config schema in chain metadata that we have been using on the TS side for a while. The overrides are as follows: - `transactionOverrides.gasPrice` - if specified, non EIP-1559 txs are used, and they use this gasPrice - `transactionOverrides.gasLimit` - if specified, this takes precedence over any gas estimation that's done and is the gas limit that's used for txs. (This is pretty niche and I don't imagine we'll ever wanna use this internally, but happens to cover the use case described [here](https://discord.com/channels/935678348330434570/984123861144600587/1224617234580766741), which is nice to cover) - `transactionOverrides.maxFeePerGas` - if specified and if the chain supports EIP 1559 txs, is used for the `maxFeePerGas` - `transactionOverrides.maxPriorityFeePerGas` - if specified and if the chain supports EIP 1559 txs, is used for the `maxPriorityFeePerGas` - Updates some of the transaction overrides in mainnet3 / testnet4 to be more reasonable now that we'll be using them to submit transactions pretty frequently - Note that atm the transaction overrides are most easily applied to the agents via the config JSONs, and not via our K8s configuration. The upshot of this is if we wanna change a transaction override right now, the easiest way to do it will be to run `update-agent-config.ts` and then build a new image. Created https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3565 to track making this experience better - Part of this change means that it's more important to keep the `transactionOverrides` specified in the SDK / infra up to date with the agent config. So I ended up doing https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3325 along the way, which also required doing https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3311. - To fix the issue with arbitrum blocks, we now require the index.from to be specified in the chain metadata if the chain's technical stack is arbitrum nitro - A new script, `update-agent-config.ts` is added, and it's ran for mainnet3 and testnet4 in CI to make sure there's no diff (similar idea to what we do with prettier) - Along the way I renamed mainnet3_config.json and testnet4_config.json to mainnet_config.json and testnet_config.json. This has been a source of confusion with ppl in the past - in v2 we named them without any numeric suffixes, so we now just do that again ### Drive-by changes - There's a drive-by to poll more frequently for pending transactions (2 seconds). The default is 7 seconds in Ethers, and for many chains that have really quick block times, this may be a way for us to speed up us learning that a transaction is included - Don't enforce gas on the inevm routes, looks like there's some weirdness there that was reported by Nam ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3562 - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3325 - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3311 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->
8 months ago
- name: Checkout registry
uses: ./.github/actions/checkout-registry
Agent transaction overrides, ensure agent configs are updated in CI (#3563) ### Description - Adds some basic transaction overrides for Ethereum chains to be used by the agents. This follows the existing config schema in chain metadata that we have been using on the TS side for a while. The overrides are as follows: - `transactionOverrides.gasPrice` - if specified, non EIP-1559 txs are used, and they use this gasPrice - `transactionOverrides.gasLimit` - if specified, this takes precedence over any gas estimation that's done and is the gas limit that's used for txs. (This is pretty niche and I don't imagine we'll ever wanna use this internally, but happens to cover the use case described [here](https://discord.com/channels/935678348330434570/984123861144600587/1224617234580766741), which is nice to cover) - `transactionOverrides.maxFeePerGas` - if specified and if the chain supports EIP 1559 txs, is used for the `maxFeePerGas` - `transactionOverrides.maxPriorityFeePerGas` - if specified and if the chain supports EIP 1559 txs, is used for the `maxPriorityFeePerGas` - Updates some of the transaction overrides in mainnet3 / testnet4 to be more reasonable now that we'll be using them to submit transactions pretty frequently - Note that atm the transaction overrides are most easily applied to the agents via the config JSONs, and not via our K8s configuration. The upshot of this is if we wanna change a transaction override right now, the easiest way to do it will be to run `update-agent-config.ts` and then build a new image. Created https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3565 to track making this experience better - Part of this change means that it's more important to keep the `transactionOverrides` specified in the SDK / infra up to date with the agent config. So I ended up doing https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3325 along the way, which also required doing https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3311. - To fix the issue with arbitrum blocks, we now require the index.from to be specified in the chain metadata if the chain's technical stack is arbitrum nitro - A new script, `update-agent-config.ts` is added, and it's ran for mainnet3 and testnet4 in CI to make sure there's no diff (similar idea to what we do with prettier) - Along the way I renamed mainnet3_config.json and testnet4_config.json to mainnet_config.json and testnet_config.json. This has been a source of confusion with ppl in the past - in v2 we named them without any numeric suffixes, so we now just do that again ### Drive-by changes - There's a drive-by to poll more frequently for pending transactions (2 seconds). The default is 7 seconds in Ethers, and for many chains that have really quick block times, this may be a way for us to speed up us learning that a transaction is included - Don't enforce gas on the inevm routes, looks like there's some weirdness there that was reported by Nam ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3562 - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3325 - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3311 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->
8 months ago
- name: Generate ${{ matrix.environment }} agent config
run: |
cd typescript/infra
yarn tsx ./scripts/agents/update-agent-config.ts -e ${{ matrix.environment }}
CHANGES=$(git status -s)
if [[ ! -z $CHANGES ]]; then
echo "Changes found in agent config: $CHANGES"
exit 1
fi
e2e-matrix:
runs-on: buildjet-8vcpu-ubuntu-2204
if: github.event_name == 'push' || (github.event_name == 'pull_request' && github.base_ref == 'main' || github.base_ref == 'cli-2.0') || github.event_name == 'merge_group'
feat: remove extraneous caching (#4469) feat: remove yarn build cache - saves 600MB of cache space PER COMMIT - doesn't impact overall per-commit CI time - better CI time in the general case due to fewer cache evictions on the e2e matrix job - [with caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797745470) vs [no caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797213927?pr=4469) drive-by: disable docker caching for agent build - currently cache has no impact on build time of the agent image - rerunning builds for a commit or pushing empty commits should in theory lead to quicker builds with layer caching, but the main problem is that the `RUN` step of building the agents is not cached by the docker action - only RUNs with bind mounts get a checksum computed [(reference)](https://docs.docker.com/build/cache/invalidation/#general-rules) - disabling docker build caching could help save a bunch of cache space - best way to 4x speed here is moving this job to a buildjet 8vcpu runner (read [here](https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/4445) for more info) drive-by: TLC - update to latest docker build action - provides a nice build summary and can generate artifacts that help break down where build time is spent - update check-env before build, given the current deprecation annotations - https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ - coverage step doesn't need to wait for `yarn-test` anymore
2 months ago
needs: [yarn-install]
strategy:
fail-fast: false
matrix:
e2e-type: [cosmwasm, non-cosmwasm]
steps:
- uses: actions/setup-node@v4
with:
node-version: 18
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
submodules: recursive
fetch-depth: 0
- name: foundry-install
uses: foundry-rs/foundry-toolchain@v1
- name: setup rust
uses: dtolnay/rust-toolchain@stable
- name: rust cache
uses: Swatinem/rust-cache@v2
with:
prefix-key: 'v2-rust-e2e'
shared-key: ${{ matrix.e2e-type }}
cache-provider: 'buildjet'
save-if: ${{ !startsWith(github.ref, 'refs/heads/gh-readonly-queue') }}
workspaces: |
./rust/main
${{ matrix.e2e-type == 'non-cosmwasm' && './rust/sealevel' || '' }}
- name: Free disk space
run: |
# Based on https://github.com/actions/runner-images/issues/2840#issuecomment-790492173
sudo rm -rf /usr/share/dotnet
sudo rm -rf /opt/ghc
sudo rm -rf "/usr/local/share/boost"
sudo rm -rf "$AGENT_TOOLSDIRECTORY"
- name: Install mold linker
uses: rui314/setup-mold@v1
with:
mold-version: 2.0.0
make-default: true
feat: remove extraneous caching (#4469) feat: remove yarn build cache - saves 600MB of cache space PER COMMIT - doesn't impact overall per-commit CI time - better CI time in the general case due to fewer cache evictions on the e2e matrix job - [with caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797745470) vs [no caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797213927?pr=4469) drive-by: disable docker caching for agent build - currently cache has no impact on build time of the agent image - rerunning builds for a commit or pushing empty commits should in theory lead to quicker builds with layer caching, but the main problem is that the `RUN` step of building the agents is not cached by the docker action - only RUNs with bind mounts get a checksum computed [(reference)](https://docs.docker.com/build/cache/invalidation/#general-rules) - disabling docker build caching could help save a bunch of cache space - best way to 4x speed here is moving this job to a buildjet 8vcpu runner (read [here](https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/4445) for more info) drive-by: TLC - update to latest docker build action - provides a nice build summary and can generate artifacts that help break down where build time is spent - update check-env before build, given the current deprecation annotations - https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ - coverage step doesn't need to wait for `yarn-test` anymore
2 months ago
- name: yarn-build
uses: ./.github/actions/yarn-build-with-cache
with:
feat: remove extraneous caching (#4469) feat: remove yarn build cache - saves 600MB of cache space PER COMMIT - doesn't impact overall per-commit CI time - better CI time in the general case due to fewer cache evictions on the e2e matrix job - [with caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797745470) vs [no caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797213927?pr=4469) drive-by: disable docker caching for agent build - currently cache has no impact on build time of the agent image - rerunning builds for a commit or pushing empty commits should in theory lead to quicker builds with layer caching, but the main problem is that the `RUN` step of building the agents is not cached by the docker action - only RUNs with bind mounts get a checksum computed [(reference)](https://docs.docker.com/build/cache/invalidation/#general-rules) - disabling docker build caching could help save a bunch of cache space - best way to 4x speed here is moving this job to a buildjet 8vcpu runner (read [here](https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/4445) for more info) drive-by: TLC - update to latest docker build action - provides a nice build summary and can generate artifacts that help break down where build time is spent - update check-env before build, given the current deprecation annotations - https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ - coverage step doesn't need to wait for `yarn-test` anymore
2 months ago
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Install system dependencies
run: |
sudo apt-get update -qq
sudo apt-get install -qq -y libudev-dev pkg-config protobuf-compiler
- name: Checkout registry
uses: ./.github/actions/checkout-registry
- name: agent tests (CosmWasm)
run: cargo test --release --package run-locally --bin run-locally --features cosmos test-utils -- cosmos::test --nocapture
if: matrix.e2e-type == 'cosmwasm'
working-directory: ./rust/main
env:
RUST_BACKTRACE: 'full'
- name: Check for Rust file changes
id: check-rust-changes
run: |
if [[ -n "$(git diff ${{ github.event.pull_request.head.sha || github.sha }} ${{ github.event.pull_request.base.sha }} -- ./rust)" ]]; then
echo "rust_changes=true" >> $GITHUB_OUTPUT
echo "$(git diff ${{ github.event.pull_request.head.sha || github.sha }} ${{ github.event.pull_request.base.sha }} -- ./rust)"
else
echo "rust_changes=false" >> $GITHUB_OUTPUT
fi
- name: agent tests (EVM and Sealevel)
run: cargo run --release --bin run-locally --features test-utils
if: matrix.e2e-type == 'non-cosmwasm'
working-directory: ./rust/main
env:
E2E_CI_MODE: 'true'
E2E_CI_TIMEOUT_SEC: '600'
E2E_KATHY_MESSAGES: '20'
RUST_BACKTRACE: 'full'
SEALEVEL_ENABLED: ${{ steps.check-rust-changes.outputs.rust_changes }}
env-test:
runs-on: ubuntu-latest
env:
MAINNET3_ARBITRUM_RPC_URLS: ${{ secrets.MAINNET3_ARBITRUM_RPC_URLS }}
MAINNET3_OPTIMISM_RPC_URLS: ${{ secrets.MAINNET3_OPTIMISM_RPC_URLS }}
timeout-minutes: 10
feat: remove extraneous caching (#4469) feat: remove yarn build cache - saves 600MB of cache space PER COMMIT - doesn't impact overall per-commit CI time - better CI time in the general case due to fewer cache evictions on the e2e matrix job - [with caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797745470) vs [no caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797213927?pr=4469) drive-by: disable docker caching for agent build - currently cache has no impact on build time of the agent image - rerunning builds for a commit or pushing empty commits should in theory lead to quicker builds with layer caching, but the main problem is that the `RUN` step of building the agents is not cached by the docker action - only RUNs with bind mounts get a checksum computed [(reference)](https://docs.docker.com/build/cache/invalidation/#general-rules) - disabling docker build caching could help save a bunch of cache space - best way to 4x speed here is moving this job to a buildjet 8vcpu runner (read [here](https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/4445) for more info) drive-by: TLC - update to latest docker build action - provides a nice build summary and can generate artifacts that help break down where build time is spent - update check-env before build, given the current deprecation annotations - https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ - coverage step doesn't need to wait for `yarn-test` anymore
2 months ago
needs: [yarn-install]
strategy:
fail-fast: false
matrix:
environment: [mainnet3]
chain: [ethereum, arbitrum, optimism, inevm, viction]
module: [core, igp]
include:
- environment: testnet4
chain: sepolia
module: core
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: foundry-install
uses: foundry-rs/foundry-toolchain@v1
feat: remove extraneous caching (#4469) feat: remove yarn build cache - saves 600MB of cache space PER COMMIT - doesn't impact overall per-commit CI time - better CI time in the general case due to fewer cache evictions on the e2e matrix job - [with caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797745470) vs [no caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797213927?pr=4469) drive-by: disable docker caching for agent build - currently cache has no impact on build time of the agent image - rerunning builds for a commit or pushing empty commits should in theory lead to quicker builds with layer caching, but the main problem is that the `RUN` step of building the agents is not cached by the docker action - only RUNs with bind mounts get a checksum computed [(reference)](https://docs.docker.com/build/cache/invalidation/#general-rules) - disabling docker build caching could help save a bunch of cache space - best way to 4x speed here is moving this job to a buildjet 8vcpu runner (read [here](https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/4445) for more info) drive-by: TLC - update to latest docker build action - provides a nice build summary and can generate artifacts that help break down where build time is spent - update check-env before build, given the current deprecation annotations - https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ - coverage step doesn't need to wait for `yarn-test` anymore
2 months ago
- name: yarn-build
uses: ./.github/actions/yarn-build-with-cache
with:
feat: remove extraneous caching (#4469) feat: remove yarn build cache - saves 600MB of cache space PER COMMIT - doesn't impact overall per-commit CI time - better CI time in the general case due to fewer cache evictions on the e2e matrix job - [with caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797745470) vs [no caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797213927?pr=4469) drive-by: disable docker caching for agent build - currently cache has no impact on build time of the agent image - rerunning builds for a commit or pushing empty commits should in theory lead to quicker builds with layer caching, but the main problem is that the `RUN` step of building the agents is not cached by the docker action - only RUNs with bind mounts get a checksum computed [(reference)](https://docs.docker.com/build/cache/invalidation/#general-rules) - disabling docker build caching could help save a bunch of cache space - best way to 4x speed here is moving this job to a buildjet 8vcpu runner (read [here](https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/4445) for more info) drive-by: TLC - update to latest docker build action - provides a nice build summary and can generate artifacts that help break down where build time is spent - update check-env before build, given the current deprecation annotations - https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ - coverage step doesn't need to wait for `yarn-test` anymore
2 months ago
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Checkout registry
uses: ./.github/actions/checkout-registry
- name: Fork test ${{ matrix.environment }} ${{ matrix.module }} ${{ matrix.chain }} deployment
run: cd typescript/infra && ./fork.sh ${{ matrix.environment }} ${{ matrix.module }} ${{ matrix.chain }}
coverage:
runs-on: ubuntu-latest
feat: remove extraneous caching (#4469) feat: remove yarn build cache - saves 600MB of cache space PER COMMIT - doesn't impact overall per-commit CI time - better CI time in the general case due to fewer cache evictions on the e2e matrix job - [with caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797745470) vs [no caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797213927?pr=4469) drive-by: disable docker caching for agent build - currently cache has no impact on build time of the agent image - rerunning builds for a commit or pushing empty commits should in theory lead to quicker builds with layer caching, but the main problem is that the `RUN` step of building the agents is not cached by the docker action - only RUNs with bind mounts get a checksum computed [(reference)](https://docs.docker.com/build/cache/invalidation/#general-rules) - disabling docker build caching could help save a bunch of cache space - best way to 4x speed here is moving this job to a buildjet 8vcpu runner (read [here](https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/4445) for more info) drive-by: TLC - update to latest docker build action - provides a nice build summary and can generate artifacts that help break down where build time is spent - update check-env before build, given the current deprecation annotations - https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ - coverage step doesn't need to wait for `yarn-test` anymore
2 months ago
needs: [yarn-install]
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
fetch-depth: 0
feat: remove extraneous caching (#4469) feat: remove yarn build cache - saves 600MB of cache space PER COMMIT - doesn't impact overall per-commit CI time - better CI time in the general case due to fewer cache evictions on the e2e matrix job - [with caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797745470) vs [no caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797213927?pr=4469) drive-by: disable docker caching for agent build - currently cache has no impact on build time of the agent image - rerunning builds for a commit or pushing empty commits should in theory lead to quicker builds with layer caching, but the main problem is that the `RUN` step of building the agents is not cached by the docker action - only RUNs with bind mounts get a checksum computed [(reference)](https://docs.docker.com/build/cache/invalidation/#general-rules) - disabling docker build caching could help save a bunch of cache space - best way to 4x speed here is moving this job to a buildjet 8vcpu runner (read [here](https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/4445) for more info) drive-by: TLC - update to latest docker build action - provides a nice build summary and can generate artifacts that help break down where build time is spent - update check-env before build, given the current deprecation annotations - https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ - coverage step doesn't need to wait for `yarn-test` anymore
2 months ago
- name: yarn-build
uses: ./.github/actions/yarn-build-with-cache
with:
feat: remove extraneous caching (#4469) feat: remove yarn build cache - saves 600MB of cache space PER COMMIT - doesn't impact overall per-commit CI time - better CI time in the general case due to fewer cache evictions on the e2e matrix job - [with caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797745470) vs [no caching](https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/10797213927?pr=4469) drive-by: disable docker caching for agent build - currently cache has no impact on build time of the agent image - rerunning builds for a commit or pushing empty commits should in theory lead to quicker builds with layer caching, but the main problem is that the `RUN` step of building the agents is not cached by the docker action - only RUNs with bind mounts get a checksum computed [(reference)](https://docs.docker.com/build/cache/invalidation/#general-rules) - disabling docker build caching could help save a bunch of cache space - best way to 4x speed here is moving this job to a buildjet 8vcpu runner (read [here](https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/4445) for more info) drive-by: TLC - update to latest docker build action - provides a nice build summary and can generate artifacts that help break down where build time is spent - update check-env before build, given the current deprecation annotations - https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ - coverage step doesn't need to wait for `yarn-test` anymore
2 months ago
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: foundry-install
uses: foundry-rs/foundry-toolchain@v1
- name: Run tests with coverage
run: yarn coverage
env:
NODE_OPTIONS: --max_old_space_size=4096
- name: Upload coverage reports to Codecov with GitHub Action
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}