hyperlane-monorepo

Commit Graph

Author	SHA1	Message	Date
Yorke Rhodes	497db63f5b	Prevent AWS KMS throttling in validator (#2391 ) ### Description Observed in validator ``` Request ID: Some("c0cc123e-da52-47f4-a2d3-aeddd034a2f5") Body: {"__type":"ThrottlingException","message":"You have exceeded the rate at which you may call KMS. Reduce the frequency of your calls."} ``` - Add sleep and checkpoint sync cache for rate limit ### Drive-by changes - Skip tree in checkpoint submitter span ### Backward compatibility Yes ### Testing Local testing	1 year ago
Yorke Rhodes	1fff74ebd8	Reduce hyper client pool idle timeout to 20s (#2384 ) ### Description - configures hyper http client to close connections after 20s - makes singleton signer retry (5) times before bubbling error up ### Backward compatibility Yes ### Testing Local binary testing against cloud resources ![Screen Shot 2023-06-13 at 5 45 47 PM](https://github.com/hyperlane-xyz/hyperlane-monorepo/assets/3020995/de47a03c-b40f-49fd-a32d-c8da6ecffa6e)	1 year ago
Asa Oines	b75cfc4c6b	Start scraper message_dispatch sync from high watermark (#2377 ) ### Description This PR addresses an issue in which BSC RPC providers have a very hard time providing very old transactions. This caused BSC scraping to fail, since the scraper had never actually indexed very early BSC messages (e.g. nonce 0). By starting at the latest scraped nonce - 1, the scraper will bypass this issue. ### Drive-by changes None	1 year ago
Asa Oines	06931bc75a	Fix BSC indexing start block (#2375 )	1 year ago
Yorke Rhodes	a93daec7a2	Implement single signer task in validator (#2342 ) ### Description Implements singleton signer which allows multiple tasks to use a shared signer that is signing on a single task and migrates validator to use a singleton KMS signer ### Drive-by changes Modifies base agent to allow self to be mutable in run ### Related issues - Fixes error from sharing KMS signer connection across validator tasks https://github.com/rusoto/rusoto/issues/1766 ### Backward compatibility Yes ### Testing E2E Tests --------- Co-authored-by: Mattie Conover <git@mconover.dev>	1 year ago
Yorke Rhodes	eb17a9af15	Block validator submitter on nonzero mailbox count (#2341 ) ### Description Ensures all checkpoint submitters wait for nonzero mailbox count ### Drive-by Changes Notably always spawns backfill submitter after seeing nonzero mailbox count ### Related issues Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2340 ### Backward compatibility Yes ### Testing Updated E2E Tests	1 year ago
Mattie Conover	efad4e1326	Omniscient relayer infra (#2299 ) ### Description - Adds support for omniscient relayers - Updates config formats to allow better overideability by agent role - Enables deploying selected agents rather than having to deploy them all at once - Merges scraper scripts and configuration ### Drive-by changes Lots of cleanup was made in the infrastructure scripts and source as it was already being touched by the above. While this PR touches a lot of files, many of the changes are just because we have so many scripts and a small change to something they all use instantly can lead to many files changed. ### Related issues - Fixes hyperlane-xyz/issues#438 - Fixes hyperlane-xyz/issues#447 - Fixes #1664 ### Backward compatibility _Are these changes backward compatible?_ No _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	1 year ago
Mattie Conover	a66652f9f2	Reduce log noise (#2336 ) ### Description Seeing a ton of errors for process estimate costs even though they are fully being handled elsewhere and are also not really an error in the first place. This makes it harder as an operator to spot real errors so I think we should eliminate this error log. (We also already log this error elsewhere so it does not add any new information). ![image](https://github.com/hyperlane-xyz/hyperlane-monorepo/assets/4187749/f149a870-52f3-444c-aa62-84b6aa822f44) This also targets the `ret` logs produced by tracing since many of them are unhelpful and most of the cases where we actually need a log we already have one. ### Drive-by changes None ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	1 year ago
Mattie Conover	c661f7dccb	Handle additional error case in retrying provider (#2330 ) ### Description Adds a fix for this error message I noticed in the logs: ``` (code: 3, message: execution reverted: INSUFFICIENT_BALANCE, ...) ``` ### Drive-by changes None ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	1 year ago
Yorke Rhodes	a85b88fe9e	Add instructions for running agents locally against cloud resources to `rust/README` (#2322 ) ### Description Update rust README with instructions for running agents locally against cloud resources. --------- Co-authored-by: Mattie Conover <git@mconover.dev>	1 year ago
Yorke Rhodes	2deb9b8973	Fix AWS KMS Signer connection issues (#2319 ) ### Description - Update ethers-rs fork - Create new KMS client instead of using OnceCell ### Backward compatibility Yes No ### Testing Running validator locally with replica of validator DB	2 years ago
Asa Oines	d57ae5f628	Do not spawn validator backwards pass if count is zero (#2315 ) ### Description Fixes a bug in which the validator would panic if the mailbox was empty. ### Drive-by changes None	2 years ago
Yorke Rhodes	e4b124cdde	Fix bug in postgres message coercion (#2310 ) ### Description Goerli had a message dispatched (nonce=1964) with a destination domain that failed to be converted from i32 to u32 when being retrieved from the database. This fixes these bugs for domain ids and nonce and improves logs for this type of failure. ### Backward compatibility Scraper needs to be redeployed ### Testing Ran scraper against read replica of explorer DB	2 years ago
Asa Oines	2ace873f9e	Gracefully handle inconsistent RPC providers in ForwardMessageSyncCursor (#2302 ) ### Description RPC providers can be inconsistent. When a message has just been inserted into the mailbox, it's possible that one Mailbox.count() query returns N and a subsequent query returns N-1. ### Drive-by changes None	2 years ago
Yorke Rhodes	bb0d688311	Add origin filter to relayer msg processor (#2301 ) ### Description Ensures the relayer never attempts to process a message destined for it's own origin domain. ### Related issues - Fixes https://github.com/hyperlane-xyz/issues/issues/464 ### Backward compatibility Yes ### Testing E2E tests	2 years ago
Asa Oines	90191d43d0	Fix bug in MessageSyncCursor rewind logic (#2297 ) ### Description This PR fixes a bug in the MessageSyncCursor rewind logic that causes the cursor to get stuck. The cursor would look for the message with nonce `n`, find it, and move its `next_block` to the block `b_n` at which `n` was dispatched. It would then index starting at `b_n`, find `n` again, interpret that as a "discontinuity", and rewind back to `b_n`, getting itself stuck permanently. We should be moving to `b_n` and not e.g. `b_n + 1` because we have no guarantee that `n+1` was not also present in `b_n` but just dropped by a flaky `eth_getLogs` implementation. The fix is to only consider messages with nonce > `n` when looking for discontinuities when our target nonce is `n + 1`. ### Drive-by changes - Modifies the E2E tests to use a chunk size of 1, allowing us to reproduce this failure mode without the filtering fix. - Debug log on rewind - Comments to clarify fast forward logic	2 years ago
Mattie Conover	347f9beac8	Revert database prefix (#2294 ) ### Description Small change to remove the domain id from the prefix in the rocks database. ### Drive-by changes No ### Related issues - https://discord.com/channels/935678348330434570/1111394973439504444 ### Backward compatibility _Are these changes backward compatible?_ Yes - returns to last release _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests	2 years ago
Yorke Rhodes	8d9ee49f67	Fix env tests for new multisig variants (#2283 ) Fixes env tests failing in CI https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/5081007361/jobs/9128835519. Adds logging to ism factory deployer and ism deployments.	2 years ago
Yorke Rhodes	50f04db1fa	Make merkle proofs optional on multisig ISM (#2173 ) ### Description Validators currently sign `(root, index)` checkpoints and during verification, a `message` is passed as calldata, an `id()` is derived, and a `proof` of `id()` at `index` in `root` is verified This provides “all or nothing” censorship resistance guarantees because a validator can only sign roots to allow any contained messages to be processed. We have considered alternatives where validators sign `message` directly and we lose censorship resistance in exchange for eliminating merkle proof verification gas costs. However, if validators sign `(root, index, message)` tuples, we can skip merkle proof verification on the destination chain while still maintaining censorship resistance by providing two valid metadata formats: 1. existing validator signatures and merkle proof verification of inclusion 2. including merkle proof verification for pathway where validators are censoring `message` It’s worth noting the validator is required to index event data to produce this new signature format. However, this does not require historical indexing and new validators being spun up can simply begin indexing from tip. See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2187 for validator changes See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2248 for relayer and e2e test changes ### Drive-by changes Merkle index also optional ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2192 ### Backward compatibility - new ISM deployment is necessary (we could upgrade implementation in theory) - Validator and relayer upgrades ### Testing Unit (fuzz) Tests, E2E tests	2 years ago
Asa Oines	8aa7b62bea	Minor logging improvements to agents (#2271 ) ### Description Removes some unnecessary clutter, adds domain to some spans ### Drive-by changes None	2 years ago
josedev-union	8a12608220	feat(helm): Support configurable SecretStore type (#2257 ) ### Description This PR makes the SecretStore type as configurable for all external secrets instead of using fixed `ClusterSecretStore` type. ### Drive-by changes _Are there any minor or drive-by changes also included?_ ### Related issues - Fixes #[issue number here] ### Backward compatibility _Are these changes backward compatible?_ No `externalSecrets.clusterSecretStore` should be replaced with `externalSecrets.StoreName` in the values. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Signed-off-by: josedev-union <josedev-union@users.noreply.github.com> Co-authored-by: josedev-union <josedev-union@users.noreply.github.com>	2 years ago
Asa Oines	ec63ed19c5	Fix CI flakiness (#2269 ) ### Description - Don't panic if E2E test fails to fetch a metric - Run coverage commands sequentially ### Drive-by changes - Cleans up some lint errors in forge tests	2 years ago
Asa Oines	73fb967ec2	Properly store gas payment expenditure data in agent rocks db (#2268 ) ### Description This PR fixes a bug that resulted in gas payment expenditures not being properly stored in the rocks DB, making them unretrievable. ### Drive-by changes - Introduces a macro to HyperlaneRocksDB to prevent similar bugs from happening in the future	2 years ago
Asa Oines	63562c7211	Refactor agent event indexing (#2246 ) ### Description This PR refactors event indexing in the agents, allowing similar logic to be shared across multiple event types (i.e. messages, deliveries, gas payments) and database types (i.e. the Relayer rocks DB and Scraper SQL DB). Furthermore, it adds new syncing modes by way of `MessageSyncCursors` that take advantage of the monotonically increasing dispatched message nonce to sync more intelligently. ### Drive-by changes - Fixes a bug in the existing cursor that caused the same block range to be indexed three times - Modifies kathy to get rid of the idea of "rounds", just sends messages with a sleep in between - Minor modifications to the e2e test for performance - Expand macros in settings - Add scraper to e2e test ### Opportunities for improvement - We can further reduce RPC usage (or improve latency) by sharing the view of the latest finalized block number between cursors - We can speed up the effective time for (a relayer to start deliving messages \| the scraper to scraper recent events) by creating forward/backward cursors for gas payments and deliveries where the backwards cursor terminates at index_settings.from - We can remove the need for index_settings.from by terminating backwards cursors based on the block number that the first message was dispatched at ### Related issues - Fixes #[issue number here] ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ E2E tests --------- Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Sergei Patrikeev	3711b64de3	feat: support WebIdentityProvider for AWS credentials [#2168 ]. (#2188 ) ### Description Support of web identity AWS credentials: https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2168 ### Drive-by changes `WebIdentityProvider` resides in `rusoto_sts`, I had to add a dependency on `rusoto_sts = "*"`. ### Backward compatibility _Are these changes backward compatible?_ Yes. The newly added `AwsChainCredentialsProvider` preserves 100% compatibility with clients using `EnvironmentProvider`. We tested the following scenarios: 1) Run agent with `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` / `AWS_SESSION_TOKEN` credentials — the current default behavior — works as before. 2) Without envs from p.1 but with `AWS_WEB_IDENTITY_TOKEN_FILE` / `AWS_ROLE_ARN` / `AWS_ROLE_SESSION_NAME` injected by AWS EKS pod identity [webhook](https://github.com/aws/amazon-eks-pod-identity-webhook) — works well. We ran the agents for 24+ hours to make sure the JWT gets refreshed automatically. 3) Without any envs — the app expectedly fails with "environment variable not found" error. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing Manual: we deployed a Hyperlane agent Docker image built from our [fork](https://github.com/tvl-labs/hyperlane-monorepo/pkgs/container/hyperlane-monorepo).	2 years ago
Trevor Porter	9743d74272	Deploy key funder (#2206 ) ### Description Deploy key funder to include #2190 ### Drive-by changes no ### Related issues n/a ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Deployed	2 years ago
Mattie Conover	5e38cdf31a	Omniscient Relayer (#2195 ) ### Description Adds support for running a relayer which delivers messages from multiple origin chains instead of just one origin chain. ### Drive-by changes _Are there any minor or drive-by changes also included?_ - The E2E pipeline got a major re-work. It both runs faster and also tests the new changes. - SerialSubmitter has been abstracted to support additional operation types such as oracle updates - SerialSubmitter has been updated to actually confirm a transaction was included and is safe from re-orgs - SerialSubmitter has been updated to have higher throughput by pipelining operations ### Related issues - Blocked by #2148 - Fixes #1899 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Asa Oines	1e9d0be9b3	Fix DomainRoutingIsm tests (#2255 ) ### Description Fixes a bug in the DomainRoutingIsm tests that curiously only started showing in CI a few days ago. ### Drive-by changes None	2 years ago
Asa Oines	f6ad1e9a95	Validator automatically self-announces (#2177 ) ### Description This PR adds the ability for validators to self-announce themselves, so long as the validator key has a large enough balance to pay for the tx. This required setting the retry provider to skip retrying reverting transactions, since with a quorum provider and a very speedy blockchain (like hardhat) the transaction is broadcast multiple times and the later transactions conflict with the first one. ### Drive-by changes None ### Related issues - Fixes #2120 ### Backward compatibility _Are these changes backward compatible?_ Yes --------- Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Mattie Conover	24f9642ea5	Agent docs and config improvements (#2148 ) ### Description Bunch of minor tweaks and updates based on what we learned from EthTokyo. - Updated Rust readme and the root readme to align with the hyperlane docs - Defaults to a 5 sec validator interval now - Makes the metrics port default to `9090` and have a soft failure if it can't start - Now will automatically create the checkpointsyncer directory if it is missing - Hyperlane database now officially uses the domain name and id as a prefix internally to support multiple relayers using the same database - ### Drive-by changes Refactored and cleaned up the database code. Removed a lot of duplicate/unnecessary/redundant functions. ### Related issues - Fixes #2113 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes - This changes the database prefixes internally to use domain name and domain id; also prepares us for #1899 ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Trevor Porter	1e43a3e1de	Lower L2 funding multiplier from 10 to 5 (#2190 ) ### Description We increased the desired balances for relayers on L2s significantly to 0.5. The target key funder balance for L2s was previously 10x the relayer target balance - we probably shouldn't be bridging over 5 ETH. This lowers the multiplier to 5, so that we shoot for a 2.5 ETH key funder balance on L2s ### Drive-by changes none ### Related issues n/a ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	2 years ago
Asa Oines	3adb6de5a5	Relayer warns if validator not announced (#2160 ) ### Description Explicitly call out if an ISM is configured with a validator that has not been announced. ### Related issues - Fixes #2120	2 years ago
J M Rossy	66ae4e4a8d	Bump verison to 1.3.4 (#71 )	2 years ago
Nam Chu Hoai	85a1bf0fba	Deploy LL + CircleBridgeAdapter to mainnet (#2166 ) ### Description Deploy LL + CircleBridgeAdapter to mainnet with the relevant changes and artifacts	2 years ago
Mattie Conover	95d8fdd330	Script to fix submodules (#2018 ) ### Description I keep finding myself running a few commands in sequence because of the submodules so I figured it was about time to turn it into a script to make things easier for us. ### Drive-by changes None ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual	2 years ago
J M Rossy	355b5ed44e	Bump versions to 1.3.4 (#2167 )	2 years ago
J M Rossy	4bf5958bc1	Bump verison to 1.3.4 (#54 )	2 years ago
Nam Chu Hoai	16c2c7eb75	Deploy gnosis IQS (#2044 ) ### Description PR deploys the iqs router on gnosis (deployed and verified manually)	2 years ago
Yorke Rhodes	1375a994d2	Decrease kathy frequency and make relayer target balance buffers larger (#2165 ) ### Description Config changes for kathy and key funder ### Related issues - Fixes https://github.com/hyperlane-xyz/issues/issues/440 ### Backward compatibility Yes ### Infrastructure - Deployed key funder on mainnet - Deployed kathy on mainnet and testnet ### Testing None	2 years ago
Nam Chu Hoai	6bac228efb	Verify proxy even if implementation is verified (#2164 ) ### Description `ContractVerifier` would skip the proxy verification if the implementation was already verified. This changes it so that we will still attempt proxy verification.	2 years ago
Nam Chu Hoai	482c8e77f5	Minor fixes to get LL deploys running smoothly (#2125 ) ### Description This PR makes some minor modifications to the deployment structure for the Liquidity Layer. The most important thing is the embedding of the `proxyAdmin` addresses in the LL addresses file which is not pretty, but I did not want to put it in the SDK right now either, but could be convinced. In the process of running this, the LL router IGPs have been updated which is non-material, but otherwise the deploy script runs as a no-op now.	2 years ago
Mattie Conover	35bcd97805	Rust release artifacts (#2150 ) ### Description This adds a new pipeline to build binaries for multiple platforms that we can package with releases. Example run: https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/4852298633/jobs/8647117945 ### Drive-by changes No ### Related issues - Fixes #2147 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual	2 years ago
Mattie Conover	0cc4600931	Update Agents (#2159 ) ### Description Update rust toolchain and packages. ### Drive-by changes No ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests	2 years ago
J M Rossy	84d7e72910	Various SDK tweaks and fixes (#2129 ) ### Description - Rename RpcPagination fields in ChainMetadata for clarity - Add optional maxBlockAge field to RpcPagination - Use RetryProvider and StaticJsonRpcProvider by default in MultiProvider - Improves RetryProvider and fixes #2115 - Enable users to customize MultiProvider provider creation behavior - Replace `ExplorerFamily` with template literal type - Improve ChainMetadata validation - Add back hyperlaneContractAddresses ### Backward compatibility No, small breaking changes: - Rename RpcPagination to RpcPaginationOptions - Rename providerBuilder to defaultProviderBuilder - Remove RetryProvider - Remove unused events functions, including getEvents, queryAnnotatedEvents, Annotated, and TSContract	2 years ago
Asa Oines	6b058e7014	Add checkConfig to HyperlaneRouterDeployer (#2156 ) ### Description Adds a pre-deploy check to HyperlaneDeployer, with an implementation for HyperlaneRouterDeployer that does a best-effort check that the ISM and IGP are properly configured for each chain. ### Drive-by changes - Move TestIsm to TestMultisigIsm for compatibility with pre-deploy checks ### Related issues - Fixes #2038 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual	2 years ago
Asa Oines	d574b9185a	Add useful error message when token config cannot be built (#70 )	2 years ago
Asa Oines	5f72e7fd92	Use node 16 when compiling contract types (#2146 ) ### Description To address https://github.com/NomicFoundation/hardhat/issues/3877 Also reverts #2140	2 years ago
Asa Oines	3b121e872d	Publish 1.3.3 (#2144 ) Publishes 1.3.3, moving the submodules back to their respective mains.	2 years ago
Asa Oines	5047ac5130	Publish 1.3.3 (#69 )	2 years ago
Asa Oines	d7c80d7908	Publish 1.3.3 (#53 )	2 years ago

... 2 3 4 5 6 ...

1724 Commits (03c92e1e346319c3a179e2355311c104b6f7e0c4) All Branches Search

1724 Commits (03c92e1e346319c3a179e2355311c104b6f7e0c4)

All Branches