hyperlane-monorepo

Commit Graph

Author	SHA1	Message	Date
Yorke Rhodes	eb17a9af15	Block validator submitter on nonzero mailbox count (#2341 ) ### Description Ensures all checkpoint submitters wait for nonzero mailbox count ### Drive-by Changes Notably always spawns backfill submitter after seeing nonzero mailbox count ### Related issues Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2340 ### Backward compatibility Yes ### Testing Updated E2E Tests	1 year ago
Mattie Conover	efad4e1326	Omniscient relayer infra (#2299 ) ### Description - Adds support for omniscient relayers - Updates config formats to allow better overideability by agent role - Enables deploying selected agents rather than having to deploy them all at once - Merges scraper scripts and configuration ### Drive-by changes Lots of cleanup was made in the infrastructure scripts and source as it was already being touched by the above. While this PR touches a lot of files, many of the changes are just because we have so many scripts and a small change to something they all use instantly can lead to many files changed. ### Related issues - Fixes hyperlane-xyz/issues#438 - Fixes hyperlane-xyz/issues#447 - Fixes #1664 ### Backward compatibility _Are these changes backward compatible?_ No _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	1 year ago
Mattie Conover	a66652f9f2	Reduce log noise (#2336 ) ### Description Seeing a ton of errors for process estimate costs even though they are fully being handled elsewhere and are also not really an error in the first place. This makes it harder as an operator to spot real errors so I think we should eliminate this error log. (We also already log this error elsewhere so it does not add any new information). ![image](https://github.com/hyperlane-xyz/hyperlane-monorepo/assets/4187749/f149a870-52f3-444c-aa62-84b6aa822f44) This also targets the `ret` logs produced by tracing since many of them are unhelpful and most of the cases where we actually need a log we already have one. ### Drive-by changes None ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	1 year ago
Mattie Conover	c661f7dccb	Handle additional error case in retrying provider (#2330 ) ### Description Adds a fix for this error message I noticed in the logs: ``` (code: 3, message: execution reverted: INSUFFICIENT_BALANCE, ...) ``` ### Drive-by changes None ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	1 year ago
Yorke Rhodes	a85b88fe9e	Add instructions for running agents locally against cloud resources to `rust/README` (#2322 ) ### Description Update rust README with instructions for running agents locally against cloud resources. --------- Co-authored-by: Mattie Conover <git@mconover.dev>	1 year ago
Yorke Rhodes	2deb9b8973	Fix AWS KMS Signer connection issues (#2319 ) ### Description - Update ethers-rs fork - Create new KMS client instead of using OnceCell ### Backward compatibility Yes No ### Testing Running validator locally with replica of validator DB	2 years ago
Asa Oines	d57ae5f628	Do not spawn validator backwards pass if count is zero (#2315 ) ### Description Fixes a bug in which the validator would panic if the mailbox was empty. ### Drive-by changes None	2 years ago
Yorke Rhodes	e4b124cdde	Fix bug in postgres message coercion (#2310 ) ### Description Goerli had a message dispatched (nonce=1964) with a destination domain that failed to be converted from i32 to u32 when being retrieved from the database. This fixes these bugs for domain ids and nonce and improves logs for this type of failure. ### Backward compatibility Scraper needs to be redeployed ### Testing Ran scraper against read replica of explorer DB	2 years ago
Asa Oines	2ace873f9e	Gracefully handle inconsistent RPC providers in ForwardMessageSyncCursor (#2302 ) ### Description RPC providers can be inconsistent. When a message has just been inserted into the mailbox, it's possible that one Mailbox.count() query returns N and a subsequent query returns N-1. ### Drive-by changes None	2 years ago
Yorke Rhodes	bb0d688311	Add origin filter to relayer msg processor (#2301 ) ### Description Ensures the relayer never attempts to process a message destined for it's own origin domain. ### Related issues - Fixes https://github.com/hyperlane-xyz/issues/issues/464 ### Backward compatibility Yes ### Testing E2E tests	2 years ago
Asa Oines	90191d43d0	Fix bug in MessageSyncCursor rewind logic (#2297 ) ### Description This PR fixes a bug in the MessageSyncCursor rewind logic that causes the cursor to get stuck. The cursor would look for the message with nonce `n`, find it, and move its `next_block` to the block `b_n` at which `n` was dispatched. It would then index starting at `b_n`, find `n` again, interpret that as a "discontinuity", and rewind back to `b_n`, getting itself stuck permanently. We should be moving to `b_n` and not e.g. `b_n + 1` because we have no guarantee that `n+1` was not also present in `b_n` but just dropped by a flaky `eth_getLogs` implementation. The fix is to only consider messages with nonce > `n` when looking for discontinuities when our target nonce is `n + 1`. ### Drive-by changes - Modifies the E2E tests to use a chunk size of 1, allowing us to reproduce this failure mode without the filtering fix. - Debug log on rewind - Comments to clarify fast forward logic	2 years ago
Mattie Conover	347f9beac8	Revert database prefix (#2294 ) ### Description Small change to remove the domain id from the prefix in the rocks database. ### Drive-by changes No ### Related issues - https://discord.com/channels/935678348330434570/1111394973439504444 ### Backward compatibility _Are these changes backward compatible?_ Yes - returns to last release _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests	2 years ago
Yorke Rhodes	50f04db1fa	Make merkle proofs optional on multisig ISM (#2173 ) ### Description Validators currently sign `(root, index)` checkpoints and during verification, a `message` is passed as calldata, an `id()` is derived, and a `proof` of `id()` at `index` in `root` is verified This provides “all or nothing” censorship resistance guarantees because a validator can only sign roots to allow any contained messages to be processed. We have considered alternatives where validators sign `message` directly and we lose censorship resistance in exchange for eliminating merkle proof verification gas costs. However, if validators sign `(root, index, message)` tuples, we can skip merkle proof verification on the destination chain while still maintaining censorship resistance by providing two valid metadata formats: 1. existing validator signatures and merkle proof verification of inclusion 2. including merkle proof verification for pathway where validators are censoring `message` It’s worth noting the validator is required to index event data to produce this new signature format. However, this does not require historical indexing and new validators being spun up can simply begin indexing from tip. See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2187 for validator changes See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2248 for relayer and e2e test changes ### Drive-by changes Merkle index also optional ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2192 ### Backward compatibility - new ISM deployment is necessary (we could upgrade implementation in theory) - Validator and relayer upgrades ### Testing Unit (fuzz) Tests, E2E tests	2 years ago
Asa Oines	8aa7b62bea	Minor logging improvements to agents (#2271 ) ### Description Removes some unnecessary clutter, adds domain to some spans ### Drive-by changes None	2 years ago
josedev-union	8a12608220	feat(helm): Support configurable SecretStore type (#2257 ) ### Description This PR makes the SecretStore type as configurable for all external secrets instead of using fixed `ClusterSecretStore` type. ### Drive-by changes _Are there any minor or drive-by changes also included?_ ### Related issues - Fixes #[issue number here] ### Backward compatibility _Are these changes backward compatible?_ No `externalSecrets.clusterSecretStore` should be replaced with `externalSecrets.StoreName` in the values. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Signed-off-by: josedev-union <josedev-union@users.noreply.github.com> Co-authored-by: josedev-union <josedev-union@users.noreply.github.com>	2 years ago
Asa Oines	ec63ed19c5	Fix CI flakiness (#2269 ) ### Description - Don't panic if E2E test fails to fetch a metric - Run coverage commands sequentially ### Drive-by changes - Cleans up some lint errors in forge tests	2 years ago
Asa Oines	73fb967ec2	Properly store gas payment expenditure data in agent rocks db (#2268 ) ### Description This PR fixes a bug that resulted in gas payment expenditures not being properly stored in the rocks DB, making them unretrievable. ### Drive-by changes - Introduces a macro to HyperlaneRocksDB to prevent similar bugs from happening in the future	2 years ago
Asa Oines	63562c7211	Refactor agent event indexing (#2246 ) ### Description This PR refactors event indexing in the agents, allowing similar logic to be shared across multiple event types (i.e. messages, deliveries, gas payments) and database types (i.e. the Relayer rocks DB and Scraper SQL DB). Furthermore, it adds new syncing modes by way of `MessageSyncCursors` that take advantage of the monotonically increasing dispatched message nonce to sync more intelligently. ### Drive-by changes - Fixes a bug in the existing cursor that caused the same block range to be indexed three times - Modifies kathy to get rid of the idea of "rounds", just sends messages with a sleep in between - Minor modifications to the e2e test for performance - Expand macros in settings - Add scraper to e2e test ### Opportunities for improvement - We can further reduce RPC usage (or improve latency) by sharing the view of the latest finalized block number between cursors - We can speed up the effective time for (a relayer to start deliving messages \| the scraper to scraper recent events) by creating forward/backward cursors for gas payments and deliveries where the backwards cursor terminates at index_settings.from - We can remove the need for index_settings.from by terminating backwards cursors based on the block number that the first message was dispatched at ### Related issues - Fixes #[issue number here] ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ E2E tests --------- Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Sergei Patrikeev	3711b64de3	feat: support WebIdentityProvider for AWS credentials [#2168 ]. (#2188 ) ### Description Support of web identity AWS credentials: https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2168 ### Drive-by changes `WebIdentityProvider` resides in `rusoto_sts`, I had to add a dependency on `rusoto_sts = "*"`. ### Backward compatibility _Are these changes backward compatible?_ Yes. The newly added `AwsChainCredentialsProvider` preserves 100% compatibility with clients using `EnvironmentProvider`. We tested the following scenarios: 1) Run agent with `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` / `AWS_SESSION_TOKEN` credentials — the current default behavior — works as before. 2) Without envs from p.1 but with `AWS_WEB_IDENTITY_TOKEN_FILE` / `AWS_ROLE_ARN` / `AWS_ROLE_SESSION_NAME` injected by AWS EKS pod identity [webhook](https://github.com/aws/amazon-eks-pod-identity-webhook) — works well. We ran the agents for 24+ hours to make sure the JWT gets refreshed automatically. 3) Without any envs — the app expectedly fails with "environment variable not found" error. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing Manual: we deployed a Hyperlane agent Docker image built from our [fork](https://github.com/tvl-labs/hyperlane-monorepo/pkgs/container/hyperlane-monorepo).	2 years ago
Mattie Conover	5e38cdf31a	Omniscient Relayer (#2195 ) ### Description Adds support for running a relayer which delivers messages from multiple origin chains instead of just one origin chain. ### Drive-by changes _Are there any minor or drive-by changes also included?_ - The E2E pipeline got a major re-work. It both runs faster and also tests the new changes. - SerialSubmitter has been abstracted to support additional operation types such as oracle updates - SerialSubmitter has been updated to actually confirm a transaction was included and is safe from re-orgs - SerialSubmitter has been updated to have higher throughput by pipelining operations ### Related issues - Blocked by #2148 - Fixes #1899 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Asa Oines	f6ad1e9a95	Validator automatically self-announces (#2177 ) ### Description This PR adds the ability for validators to self-announce themselves, so long as the validator key has a large enough balance to pay for the tx. This required setting the retry provider to skip retrying reverting transactions, since with a quorum provider and a very speedy blockchain (like hardhat) the transaction is broadcast multiple times and the later transactions conflict with the first one. ### Drive-by changes None ### Related issues - Fixes #2120 ### Backward compatibility _Are these changes backward compatible?_ Yes --------- Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Mattie Conover	24f9642ea5	Agent docs and config improvements (#2148 ) ### Description Bunch of minor tweaks and updates based on what we learned from EthTokyo. - Updated Rust readme and the root readme to align with the hyperlane docs - Defaults to a 5 sec validator interval now - Makes the metrics port default to `9090` and have a soft failure if it can't start - Now will automatically create the checkpointsyncer directory if it is missing - Hyperlane database now officially uses the domain name and id as a prefix internally to support multiple relayers using the same database - ### Drive-by changes Refactored and cleaned up the database code. Removed a lot of duplicate/unnecessary/redundant functions. ### Related issues - Fixes #2113 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes - This changes the database prefixes internally to use domain name and domain id; also prepares us for #1899 ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Asa Oines	3adb6de5a5	Relayer warns if validator not announced (#2160 ) ### Description Explicitly call out if an ISM is configured with a validator that has not been announced. ### Related issues - Fixes #2120	2 years ago
Mattie Conover	35bcd97805	Rust release artifacts (#2150 ) ### Description This adds a new pipeline to build binaries for multiple platforms that we can package with releases. Example run: https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/4852298633/jobs/8647117945 ### Drive-by changes No ### Related issues - Fixes #2147 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual	2 years ago
Mattie Conover	0cc4600931	Update Agents (#2159 ) ### Description Update rust toolchain and packages. ### Drive-by changes No ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests	2 years ago
Mattie Conover	ed17ddc32f	Agent Argument Parsing (#2132 ) ### Description All configs can now be set by arguments as well as env and config files. The arguments will take the highest priority and overwrite the others. We should also update the docs to include this new support when it gets released, but since it is an entirely new feature it also can be a stealth release which we document after the fact. ### Drive-by changes ### Related issues - Fixes #2034 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Mattie Conover	3e5b20c88e	Better validation of signers during config parsing (#2139 ) ### Description This improves the validation during config parsing to ensure that all the expected signers are configured which means the errors will have the correct config path. ### Drive-by changes Also updated the checks for missing chain configurations to be more clear and use the chain config path rather than the config that indicated the config was required and removes a now redundant check. ### Related issues - Fixes #2101 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual	2 years ago
Mattie Conover	669a610c16	Better checks on config files (#2134 ) ### Description This adds some additional validation to the config files that are provided by users to make sure that they are valid before trying to load them. ### Drive-by changes None ### Related issues - Fixes #2104 ### Backward compatibility _Are these changes backward compatible?_ Yes - probably, might be an edge case it drops support for like if someone was using a config file without a `.json` extension _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual	2 years ago
Mattie Conover	df09062706	Fix domain config unwrap bug (#2133 ) ### Description Fixes a bug in chain config parsing where it was not checking if values were actually present and lead to an annoying unwrap panic rather than the expected clean message. ### Drive-by changes Also fixed the issue for addresses. ### Related issues - Fixes #2105 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	2 years ago
Mattie Conover	b56b022f88	Hyperlane-Core WASM support (#2110 ) ### Description This shuffles some code around and changes dependencies to make `hyperlane-core` WASM compatible and adds a check to the pipeline to ensure it stays that way. To do this it does refactor hyperlane db to the `base` crate, which makes more sense IMO anyway as that is an implementation detail of the agents and not part of the core protocol. ### Drive-by changes ### Related issues - Fixes #1977 ### Backward compatibility _Are these changes backward compatible?_ Yes-ish - It does change where things are but it does not change support. Should only impact people who directly use the libraries. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Mattie Conover	910e5c84e9	Error message to indicate if there are no validators (#2099 ) ### Description Adds a more descriptive error message for failing to reach validator quorum when there are no validators defined. ### Drive-by changes No ### Related issues - Fixes #2031 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	2 years ago
Nam Chu Hoai	40cc4a631a	Update moonbeam reorg period (#1982 ) ### Description Updates the moonbeam reorg period. - Already updated the docs page - Deployed this to mainnet2 as well - Communicated the recommendation to validators ### Drive-by changes - none ### Related issues - Fixes https://github.com/hyperlane-xyz/issues/issues/417 ### Backward compatibility Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	2 years ago
Asa Oines	2900737c8f	Add HyperlaneIsmFactoryDeployer (#2068 ) ### Description This PR separates ISM deployment from HyperlaneCoreDeployer. Furthermore, it adds support for Routing and Aggregation ISM deployment via a recursive IsmConfig type. The MultisigIsm address is removed from the artifacts. Instead, the default ISM will be fetched at run time and compared against a provided IsmConfig. If there are any differences, a new defaultIsm will be deployed. ### Drive-by changes None ### Related issues None ### Backward compatibility _Are these changes backward compatible?_ No _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes, the default LegacyMultisigIsms will no longer be supported by the checker. The governance script will instead deploy a routing ISM and set that as the default ISM. ### Testing _What kind of testing have these changes undergone?_ Unit Tests	2 years ago
Asa Oines	1d0e067166	Add relayer support for MultisigIsm and RoutingIsm (#2074 ) ### Description This PR adds support in the relayer for MultisigIsm (non-legacy) and RoutingIsm metadata building. AggregationIsm support is not yet implemented. ### Drive-by changes None ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ end-to-end	2 years ago
Mattie Conover	1b7d41ad8e	Fix docs link (#2097 ) ### Description Minor fix to docs URL linked to by configuration errors. ### Drive-by changes None ### Related issues - https://discord.com/channels/935678348330434570/1096400530722537513 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	2 years ago
Mattie Conover	79669902de	More handling for eth_feeHistory (#2095 ) ### Description Adds additional error contexts for `eth_feeHistory` which should not be retried. Was seeing the error pop up for `celo` and `alfajores`. https://cloudlogging.app.goo.gl/SR7gV3PK5QzbnpFt8 ### Drive-by changes None ### Related issues - https://discord.com/channels/935678348330434570/1096400530722537513 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	2 years ago
Mattie Conover	d36cf6141a	Add link to reference (#2092 ) ### Description Forgot this in the other PR, just adds a link to the reference when there is a parsing error. ### Drive-by changes None ### Related issues - From #2070 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	2 years ago
Mattie Conover	94e6b11f94	Fix payment enforcement config parsing bug (#2093 ) ### Description Fixes a bug where we were first parsing the matching list as a string and then trying to convert it which was impacting the expected config type. ### Drive-by changes None ### Related issues - https://discord.com/channels/935678348330434570/1096400530722537513 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests	2 years ago
Mattie Conover	91837d8a20	Remove txsubmission type (#2094 ) ### Description Removes a vestigial organ from the gelato days. We no longer need a config to denote txsubmission type. ### Drive-by changes No ### Related issues - https://discord.com/channels/935678348330434570/1096400530722537513 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests	2 years ago
Mattie Conover	1722f4ee7e	Better Agent Configuration Parsing (#2070 ) ### Description This is a significant change to how we parse user configs. The main goal of this PR is to give a clear path and env key to any errors that are encountered when parsing configuration. In order to accomplish this, I have crated a new trait `FromRawConf` which defines a set of steps to validate and convert individual fields. Also for every major config struct, there is now a `Raw` variant of it which is used by the `config` library to reduce errors that occur during deserialization itself. `Raw` types should always use the most basic form of a value, such as `String`, `StrOrInt`, `bool`, and ideally optional in all cases. Missing values should be handled during the raw conversion and not by `config`. I also needed to make changes to a number of types stored in the parsed config types to force validation forward to the parsing step instead of doing it when we read the config value. This also required me to create a filter to prevent trying to validate certain configs that we would not ever need to load. These changes can also be built on later to support something other than `config` if we choose to, or add support for merging configs from multiple sources since everything is optional. ### Drive-by changes - Default to `http` for connection type - Default to no gas payment enforcement if not specified instead of failing - `ChainSetup` -> `ChainConf` - `GasPaymentEnforcementConfig` -> `GasPaymentEnforcementConf` - Made ethereum connection configuration more forgiving - Moved hyperlane base settings from `mod.rs` to `base.rs` - Moved config chain tests in hyperlane core to `tests` dir to fix a cyclical import problem - Extension traits to help with config in hyperlane core - Moved `StrOrInt` to new hyperlane core `config` module - Support for parsing `U256` from `StrOrInt` - Removed `HexString` type which is now redundant - Updated base settings to use hyperlane domain - Use `heyKey` as signer type if `type` is not specified and `key` is - Moved hyperlane ethereum chain config to a new module ### Related issues - Fixes #2033 - Fixes #2012 ### Backward compatibility _Are these changes backward compatible?_ Yes - This should take in configs in the same shape but be more forgiving in a few places _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Test	2 years ago
Asa Oines	1edbecf9e3	Lag count in Validator agent (#2073 ) ### Description This PR adds a lag to `mailbox.count()` in the validator agent. Without this lag, validators can shut down after the first message because the latestCheckpoint call is lagged and will fail due to the lagged count still being zero. ### Drive-by changes None ### Related issues - Fixes #1764 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None --------- Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Mattie Conover	e91c3100c2	Better Indexing (#2036 ) ### Description * Logs the distance from the tip * Calculates and logs the time to fully sync * Changes indexing timeout when catching up from 1s to 100ms ### Drive-by changes * Cleans up some log messages at the `info` level. * Refactors the indexing logic to be a bit cleaner/simpler * Updated all indexers to use the eta calculations ### Related issues - Fixes #1872 - Fixes #1851 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual	2 years ago
Mattie Conover	b3a1ea5d3a	RPC Error Handling (#2021 ) ### Description We have known for a while that our rules around when to retry certain calls has been a bit rough around the edges. This PR attempts to better handle a number of cases by retrying errors that are server errors and not retrying errors that we expect to have re-occur consistently on retry. I don't expect this to be perfect, but I hope it will get us about 90%+ of the way there. We can followup in the future to capture additional cases. My log investigation: [gcp link](https://console.cloud.google.com/logs/query;cursorTimestamp=2023-02-24T15:24:38.674835362Z;query=severity%3DWARNING%0AjsonPayload.target%3D~%22hyperlane_ethereum::%2528retrying%7Cfallback%2529%22%0A%2528jsonPayload.span.method%3D~%22.eth_.%22%20OR%20jsonPayload.fields.method%3D~%22.eth_.%22%2529%0A-jsonPayload.fields.text%3D%22%7B%5C%22message%5C%22:%5C%22no%20Route%20matched%20with%20those%20values%5C%22%7D%22%0A-jsonPayload.fields.error%3D~%22.connection%20closed%20before%20message%20completed.%22%0A-jsonPayload.fields.error%3D~%22.tcp%20connect%20error.%22%0A-jsonPayload.fields.text%3D%22%3Chtml%3E%5Cr%5Cn%3Chead%3E%3Ctitle%3E502%20Bad%20Gateway%3C%2Ftitle%3E%3C%2Fhead%3E%5Cr%5Cn%3Cbody%3E%5Cr%5Cn%3Ccenter%3E%3Ch1%3E502%20Bad%20Gateway%3C%2Fh1%3E%3C%2Fcenter%3E%5Cr%5Cn%3C%2Fbody%3E%5Cr%5Cn%3C%2Fhtml%3E%5Cr%5Cn%22%0A-jsonPayload.fields.error%3D%22%2528code:%20429,%20message:%20Your%20app%20has%20exceeded%20its%20compute%20units%20per%20second%20capacity.%20If%20you%20have%20retries%20enabled,%20you%20can%20safely%20ignore%20this%20message.%20If%20not,%20check%20out%20https:%2F%2Fdocs.alchemy.com%2Freference%2Fthroughput,%20data:%20None%2529%22%0A-jsonPayload.fields.text%3D%22default%20backend%20-%20404%22%0A-jsonPayload.fields.text%3D%22Bad%20Gateway%22%0A-jsonPayload.fields.error%3D~%22.Connection%20reset%20by%20peer%20%2528os%20error%20104%2529%22%0A-jsonPayload.fields.text%3D%22API%20call%20rejected%20because%20chain%20is%20not%20done%20bootstrapping%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32601,%20message:%20the%20method%20eth_feeHistory%20does%20not%20exist%2Fis%20not%20available,%20data:%20None%2529%22%0A-jsonPayload.target%3D%22hyperlane_ethereum::fallback%22%0A-jsonPayload.fields.error%3D~%22operation%20timed%20out%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32000,%20message:%20404%20Not%20Found:%20%7B%5C%22jsonrpc%5C%22:%5C%222.0%5C%22,%5C%22error%5C%22:%7B%5C%22code%5C%22:404,%5C%22message%5C%22:%5C%22arb1-sequencer%20rate%20limit%20hit.%20%20Try%20again%201%20minute%5C%22%7D%7D,%20data:%20None%2529%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32000,%20message:%20429%20Too%20Many%20Requests:%20%7B%5C%22jsonrpc%5C%22:%5C%222.0%5C%22,%5C%22error%5C%22:%7B%5C%22code%5C%22:429,%5C%22message%5C%22:%5C%22Public%20RPC%20Rate%20Limit%20Hit,%20limit%20will%20reset%20in%2060%20seconds%5C%22%7D%7D,%20data:%20None%2529%22%0A-jsonPayload.fields.error%3D~%22unable%20to%20get%20local%20issuer%20certificate%22%0A-jsonPayload.fields.error%3D%22EOF%20while%20parsing%20a%20value%20at%20line%201%20column%200%22%0A-jsonPayload.fields.text%3D%22internal%20service%20failure%5Cn%22%0A-jsonPayload.fields.text%3D%22%3Chtml%3E%5Cr%5Cn%3Chead%3E%3Ctitle%3E503%20Service%20Temporarily%20Unavailable%3C%2Ftitle%3E%3C%2Fhead%3E%5Cr%5Cn%3Cbody%3E%5Cr%5Cn%3Ccenter%3E%3Ch1%3E503%20Service%20Temporarily%20Unavailable%3C%2Fh1%3E%3C%2Fcenter%3E%5Cr%5Cn%3Chr%3E%3Ccenter%3Enginx%3C%2Fcenter%3E%5Cr%5Cn%3C%2Fbody%3E%5Cr%5Cn%3C%2Fhtml%3E%5Cr%5Cn%22%0A-jsonPayload.fields.text%3D%22%3Chtml%3E%5Cr%5Cn%3Chead%3E%3Ctitle%3E502%20Bad%20Gateway%3C%2Ftitle%3E%3C%2Fhead%3E%5Cr%5Cn%3Cbody%3E%5Cr%5Cn%3Ccenter%3E%3Ch1%3E502%20Bad%20Gateway%3C%2Fh1%3E%3C%2Fcenter%3E%5Cr%5Cn%3Chr%3E%3Ccenter%3Enginx%3C%2Fcenter%3E%5Cr%5Cn%3C%2Fbody%3E%5Cr%5Cn%3C%2Fhtml%3E%5Cr%5Cn%22%0A-jsonPayload.fields.error%3D~%22error%20trying%20to%20connect:%20error:14094410:SSL%20routines:ssl3_read_bytes:sslv3%20alert%20handshake%20failure:..%2Fssl%2Frecord%2Frec_layer_s3.c:1543:SSL%20alert%20number%2040%22%0A-jsonPayload.fields.text%3D~%22The%20gateway%20cannot%20get%20a%20response,%20please%20try%20again%20or%20contact%20the%20administrator%22%0A-jsonPayload.fields.error%3D~%22Connection%20reset%20by%20peer%20%5C%2528os%20error%20104%5C%2529%22%0A-jsonPayload.fields.text%3D%22%3Chtml%3E%5Cr%5Cn%3Chead%3E%3Ctitle%3E502%20Bad%20Gateway%3C%2Ftitle%3E%3C%2Fhead%3E%5Cr%5Cn%3Cbody%3E%5Cr%5Cn%3Ccenter%3E%3Ch1%3E502%20Bad%20Gateway%3C%2Fh1%3E%3C%2Fcenter%3E%5Cr%5Cn%3Chr%3E%3Ccenter%3Ecloudflare%3C%2Fcenter%3E%5Cr%5Cn%3C%2Fbody%3E%5Cr%5Cn%3C%2Fhtml%3E%5Cr%5Cn%22%0A-jsonPayload.fields.text%3D%22%7B%5C%22jsonrpc%5C%22:%5C%222.0%5C%22,%5C%22error%5C%22:%7B%5C%22code%5C%22:0,%5C%22message%5C%22:%5C%22we%20can't%20execute%20this%20request%5C%22%7D,%5C%22id%5C%22:null%7D%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32000,%20message:%20transaction%20underpriced,%20data:%20None%2529%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32000,%20message:%20replacement%20transaction%20underpriced,%20data:%20None%2529%22%0A-jsonPayload.fields.text%3D%22%7B%5C%22jsonrpc%5C%22:%5C%222.0%5C%22,%5C%22result%5C%22:%7B%5C%22code%5C%22:429,%5C%22message%5C%22:%5C%22Total%20number%20of%20requests%20exceeded.%20Want%20higher%20rate%20limit%3F%20Contact%20us%20at%20sales@gateway.fm%5C%22%7D,%5C%22id%5C%22:%5C%22%5C%22%7D%22%0A-jsonPayload.fields.text%3D%22%3Chtml%3E%5Cr%5Cn%3Chead%3E%3Ctitle%3E502%20Bad%20Gateway%3C%2Ftitle%3E%3C%2Fhead%3E%5Cr%5Cn%3Cbody%3E%5Cr%5Cn%3Ccenter%3E%3Ch1%3E502%20Bad%20Gateway%3C%2Fh1%3E%3C%2Fcenter%3E%5Cr%5Cn%3Chr%3E%3Ccenter%3Enginx%2F1.20.2%3C%2Fcenter%3E%5Cr%5Cn%3C%2Fbody%3E%5Cr%5Cn%3C%2Fhtml%3E%5Cr%5Cn%22%0A-jsonPayload.fields.text%3D~%22504%20ERROR%22%0A-jsonPayload.fields.text%3D%22Gateway%20Timeout%22%0A-jsonPayload.fields.text%3D%22404%20page%20not%20found%5Cn%22%0A-jsonPayload.fields.error%3D~%22error%20trying%20to%20connect:%20unexpected%20EOF%22%0A-jsonPayload.fields.error%3D~%22execution%20reverted:%20No%20router%20enrolled%20for%20domain.%20Did%20you%20specify%20the%20right%20domain%20ID%3F%22%0A-jsonPayload.fields.error%3D~%22VM%20Exception%20while%20processing%20transaction:%20revert%20No%20router%20enrolled%20for%20domain.%20Did%20you%20specify%20the%20right%20domain%20ID%3F%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32000,%20message:%20invalid%20transaction:%20nonce%20too%20low,%20data:%20None%2529%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32000,%20message:%20already%20known,%20data:%20None%2529%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32010,%20message:%20AlreadyKnown,%20data:%20None%2529%22%0A-jsonPayload.fields.error%3D~%22message:%20nonce%20too%20low%22%0A-jsonPayload.fields.error%3D~%22message:%20execution%20reverted:%20delivered%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32603,%20message:%20already%20known,%20data:%20None%2529%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32000,%20message:%20ALREADY_EXISTS:%20already%20known,%20data:%20None%2529%22%0A-jsonPayload.fields.text%3D%22%7B%5Cn%20%20%5C%22message%5C%22:%5C%22An%20invalid%20response%20was%20received%20from%20the%20upstream%20server%5C%22%5Cn%7D%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32601,%20message:%20Method%20not%20found,%20data:%20Some%2528Object%20%7B%5C%22method%5C%22:%20String%2528%5C%22'eth_getLogs'%20is%20not%20available%20on%20our%20public%20API.%20Head%20over%20to%20https:%2F%2Fdocs.blastapi.io%2Fblast-documentation%2Ftutorials-and-guides%2Fusing-blast-to-get-a-blockchain-endpoint%20for%20more%20information%5C%22%2529%7D%2529%2529%22%0A-jsonPayload.fields.error%3D~%22insufficient%20funds%22%0A-jsonPayload.fields.text%3D%22%7B%5C%22jsonrpc%5C%22:%5C%222.0%5C%22,%5C%22result%5C%22:%7B%5C%22code%5C%22:429,%5C%22message%5C%22:%5C%22Total%20number%20of%20requests%20exceeded%5C%22%7D,%5C%22id%5C%22:%5C%22%5C%22%7D%22%0A-jsonPayload.fields.text%3D~%22data%20not%20instance%20of%20model%22;summaryFields=jsonPayload%252Ffields%252Ftext,jsonPayload%252Fspan%252Fmethod,jsonPayload%252Ffields%252Ferror:false:32:beginning;timeRange=2023-02-16T06:56:50.422Z%2F2023-03-23T20:56:50.422Z?project=abacus-labs-dev) Since that will eventually rot I am also listing my query here: ``` severity=WARNING jsonPayload.target=~"hyperlane_ethereum::(retrying\|fallback)" (jsonPayload.span.method=~".eth_." OR jsonPayload.fields.method=~".eth_.") -jsonPayload.fields.text="{\"message\":\"no Route matched with those values\"}" -jsonPayload.fields.error=~".connection closed before message completed." -jsonPayload.fields.error=~".tcp connect error." -jsonPayload.fields.text="<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n</body>\r\n</html>\r\n" -jsonPayload.fields.error="(code: 429, message: Your app has exceeded its compute units per second capacity. If you have retries enabled, you can safely ignore this message. If not, check out https://docs.alchemy.com/reference/throughput, data: None)" -jsonPayload.fields.text="default backend - 404" -jsonPayload.fields.text="Bad Gateway" -jsonPayload.fields.error=~".Connection reset by peer (os error 104)" -jsonPayload.fields.text="API call rejected because chain is not done bootstrapping" -jsonPayload.fields.error="(code: -32601, message: the method eth_feeHistory does not exist/is not available, data: None)" -jsonPayload.target="hyperlane_ethereum::fallback" -jsonPayload.fields.error=~"operation timed out" -jsonPayload.fields.error="(code: -32000, message: 404 Not Found: {\"jsonrpc\":\"2.0\",\"error\":{\"code\":404,\"message\":\"arb1-sequencer rate limit hit. Try again 1 minute\"}}, data: None)" -jsonPayload.fields.error="(code: -32000, message: 429 Too Many Requests: {\"jsonrpc\":\"2.0\",\"error\":{\"code\":429,\"message\":\"Public RPC Rate Limit Hit, limit will reset in 60 seconds\"}}, data: None)" -jsonPayload.fields.error=~"unable to get local issuer certificate" -jsonPayload.fields.error="EOF while parsing a value at line 1 column 0" -jsonPayload.fields.text="internal service failure\n" -jsonPayload.fields.text="<html>\r\n<head><title>503 Service Temporarily Unavailable</title></head>\r\n<body>\r\n<center><h1>503 Service Temporarily Unavailable</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n" -jsonPayload.fields.text="<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n" -jsonPayload.fields.error=~"error trying to connect: error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:../ssl/record/rec_layer_s3.c:1543:SSL alert number 40" -jsonPayload.fields.text=~"The gateway cannot get a response, please try again or contact the administrator" -jsonPayload.fields.error=~"Connection reset by peer \(os error 104\)" -jsonPayload.fields.text="<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n<hr><center>cloudflare</center>\r\n</body>\r\n</html>\r\n" -jsonPayload.fields.text="{\"jsonrpc\":\"2.0\",\"error\":{\"code\":0,\"message\":\"we can't execute this request\"},\"id\":null}" -jsonPayload.fields.error="(code: -32000, message: transaction underpriced, data: None)" -jsonPayload.fields.error="(code: -32000, message: replacement transaction underpriced, data: None)" -jsonPayload.fields.text="{\"jsonrpc\":\"2.0\",\"result\":{\"code\":429,\"message\":\"Total number of requests exceeded. Want higher rate limit? Contact us at sales@gateway.fm\"},\"id\":\"\"}" -jsonPayload.fields.text="<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.20.2</center>\r\n</body>\r\n</html>\r\n" -jsonPayload.fields.text=~"504 ERROR" -jsonPayload.fields.text="Gateway Timeout" -jsonPayload.fields.text="404 page not found\n" -jsonPayload.fields.error=~"error trying to connect: unexpected EOF" -jsonPayload.fields.error=~"execution reverted: No router enrolled for domain. Did you specify the right domain ID?" -jsonPayload.fields.error=~"VM Exception while processing transaction: revert No router enrolled for domain. Did you specify the right domain ID?" -jsonPayload.fields.error="(code: -32000, message: invalid transaction: nonce too low, data: None)" -jsonPayload.fields.error="(code: -32000, message: already known, data: None)" -jsonPayload.fields.error="(code: -32010, message: AlreadyKnown, data: None)" -jsonPayload.fields.error=~"message: nonce too low" -jsonPayload.fields.error=~"message: execution reverted: delivered" -jsonPayload.fields.error="(code: -32603, message: already known, data: None)" -jsonPayload.fields.error="(code: -32000, message: ALREADY_EXISTS: already known, data: None)" -jsonPayload.fields.text="{\n \"message\":\"An invalid response was received from the upstream server\"\n}" -jsonPayload.fields.error="(code: -32601, message: Method not found, data: Some(Object {\"method\": String(\"'eth_getLogs' is not available on our public API. Head over to https://docs.blastapi.io/blast-documentation/tutorials-and-guides/using-blast-to-get-a-blockchain-endpoint for more information\")}))" -jsonPayload.fields.error=~"insufficient funds" -jsonPayload.fields.text="{\"jsonrpc\":\"2.0\",\"result\":{\"code\":429,\"message\":\"Total number of requests exceeded\"},\"id\":\"\"}" -jsonPayload.fields.text=~"data not instance of model" ``` Which managed to categorize errors over a large time range and I used to verify that my logic is sound. ### Drive-by changes - Moves code logic to avoid duplication ### Related issues - Fixes #1944 - Fixes hyperlane-xyz/issues#412 - Fixes hyperlane-xyz/issues#414 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual	2 years ago
Mattie Conover	7d6e5d5281	Rust Fuel Updates (#2029 ) ### Description This brings fuel up to the latest version. The main change required was to to abigen process. After some digging I figured out they just moved it to a new crate. ### Drive-by changes - Updated an error message I ran into when trying to run the validator to be more descriptive - Found an unlinked file which I was able to delete - Add `*.ignore` files to gitignore. This allows me to block out some configs I was using for testing. ### Related issues - Fixes hyperlane-xyz/issues#410 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual - Ran the validator against a local node	2 years ago
Asa Oines	f25feecfa7	Scaffolding for multiple ISM types in the relayer (#2028 ) ### Description This PR adds scaffolding in the relayer to support building metadata for multiple ISM types. To add a new ISM type: - Add a new implementation of MetadataBuilder for that type - Add the type to the SupportedIsmTypes enum - Add the type to the match statement in BaseMetadataBuilder ### Drive-by changes None ### Related issues - #1757 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ E2E tests	2 years ago
Asa Oines	63558af85d	Use interfaces in agents (#2022 ) ### Description Move agents to use contract interfaces artifacts rather than the artifacts themselves. This ensures we don't anything that we can't guarantee is in every implementation of that interface. Particularly relevant for IGPs and ISMs. ### Drive-by changes - Moves `solidity/interfaces` to `solidity/contracts/interfaces` so that their artifacts can be used in `update_abis.sh` ### Related issues - #1757 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	2 years ago
Yorke Rhodes	6f7495320b	Regenerate test fixtures and unskip hardhat tests (#2035 )	2 years ago
Mattie Conover	c78377587e	Build only the agent bins in docker (#2006 ) ### Description Minor change to slightly speed up agent image build times. We only need to build the agent binaries in the docker pipeline. ### Drive-by changes None ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests	2 years ago
Mattie Conover	45214c3d30	Added additional error context to missing values (#2007 ) ### Description Adds context for - `originchainname` - `destinationchainnames` - `chainstoscrape` ### Drive-by changes _Are there any minor or drive-by changes also included?_ ### Related issues - Fixes https://discord.com/channels/935678348330434570/935678739663192184/1088534089646227516 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None --------- Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com>	2 years ago
Asa Oines	733826e36c	Fix E2E test (#2011 )	2 years ago

... 3 4 5 6 7 ...

817 Commits (683a96257e446c4901022d589b5a618ef9c9f9d8)