162 Commits (c6cea785ae778f90d584ad5b6969ca335067b642)

Author	SHA1	Message	Date
Daniel Savu	c6cea785ae	V3 agents feedback (#2780) ... ### Description Part of the remediations from https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2746 --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	1 year ago
Daniel Savu	af5bd88aa8	V3 agents rebase (#2746) ... ### Description It's your favourite PR coming right back... V3 agents! Closes https://github.com/hyperlane-xyz/issues/issues/561 Builds on top of https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2742 Depends on https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2681 for e2e testing This PR includes: - [x] Merkle tree hook indexer - [x] Merkle tree builder task - [x] Update submitter to trigger retries if no proof is available yet Slightly more detailed overview of the work here: https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2720#issuecomment-1724038643 <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: -f <kunalarora1729@gmail.com> Co-authored-by: Trevor Porter <trkporter@ucdavis.edu> Co-authored-by: Kunal Arora <55632507+aroralanuk@users.noreply.github.com> Co-authored-by: Mattie Conover <git@mconover.dev> Co-authored-by: Guillaume Bouvignies <guillaumebouvignies@gmail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz> Co-authored-by: Guillaume Bouvignies <guillaume.bouvignies@kurtosistech.com>	1 year ago
Kunal Arora	f0e4f2b89b	Prevent invalid destination griefing for the relayer (#2703) ... ### Description - Adds destinationDomain to the `GasPayment` event - recording destination domain as `destination` while reading events for the relayer ### Drive-by changes - none ### Related issues - fixes https://github.com/hyperlane-xyz/issues/issues/475 ### Backward compatibility No, change in event emitted and relayer indexing ### Testing - Unit --------- Signed-off-by: -f <kunalarora1729@gmail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz>	1 year ago
Daniel Savu	903af462d3	chore(sealevel igp indexing): reduce log verbosity (#2724) ... Removes some unnecessary logs from the sealevel igp indexing fn. Also instruments the eta calculator in response to https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2723 ### Description <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	1 year ago
Trevor Porter	2d8fced77a	Support using Squads multisig with sealevel tooling, support ownership transfers in tooling (#2700) ... ### Description Sadly, there's no programmatic way to submit transactions to Squads via an API or anything. As a workaround, if `-k` is passed a Pubkey and not a path to a Keypair as the payer, transactions will instead be logged in a base58 serialized format that can then be copied into Squads. Instructions for exactly how to do this are found in our Notion So this includes: * Allow for the payer to just be a Pubkey without a Keypair * Some commands & functions to allow for ownership transfer ### Drive-by changes * There was a whole directory `rust/chains/hyperlane-sealevel/src/solana` that somehow got into main, I deleted this * Cleaned up some logs to be formatted more nicely ### Related issues part of https://github.com/hyperlane-xyz/issues/issues/545 ### Backward compatibility backward compatible ### Testing Transferred ownership of devnet --------- Co-authored-by: Daniel Savu <23065004+daniel-savu@users.noreply.github.com>	1 year ago
Daniel Savu	71e8988ccd	Sealevel igp indexing (#2585) ... ### Description Depends on https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2583 Indexes IGP payments related to the relayer's data pda address. Unless this address is specified in the config (`sealevel.relayer_account `), no filtering is applied and all IGP payments are stored in the local database. <!-- What's included in this PR? --> ### Drive-by changes - Sets `HYP_BASE_GASPAYMENTENFORCEMENT` in `run-locally` for the relayer, to test that it correctly indexes the IGP payment before submitting the message - A new config section (`sealevel`) is added to the relayer - The `MessageIndexer` trait is replaced with `SequenceIndexer<HyperlaneMessage>`, renaming `fetch_count_at_tip` to `sequence_at_tip`. `SequenceIndexer` is now common to both the message and igp indexers. - The `parse_addr` macro is modified so it can be reused when parsing the sealevel relayer address config too - `rust/utils/sealevel-test.bash` is included because I was using it to test locally, but I can remove it if the sealevel e2e test already does all the steps there @mattiecnvr - Performs a `try_into` conversion that can be removed once https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2610 is done ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2501 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? --> e2e tests but the pipeline is failing, likely fixed by https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2602 --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	1 year ago
Mattie Conover	389029dafb	Agent Config Base Parsing (#2665) ... ### Description This is a big one for #2215... 1. Splits the parsing of configs from the config definitions to support both the old, new, and future parsers simultaneously. (We might want to rename it from parser to config format or something; not set on this name). 2. Implements the new config format without switching to it or enabling it. This means this PR does introduce dead code that is going to be used in the next PR. ### Drive-by changes - Deleted some unused (or single use) macros defined in hyperlane base - Updated the definition of the config parsing trait to simplify implementation - Added some more ergonomic methods of handling errors in config parsing - Removed any raw `unwrap()` I noticed since these cause difficult to debug errors in prod. - Added some additional validation to catch code mistakes in config path definitions ### Related issues - Progress on #2215 - Comes after #2658 ### Backward compatibility Yes ### Testing Manual (E2E only for now to make sure I did not break the old config format)	1 year ago
Daniel Savu	51b81b84fa	fix(sealevel): data encoding in `set_return_data` (#2635) ... ### Description Suffixes `set_return_data` bytes with a non-zero byte to prevent a truncation bug ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2589 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	1 year ago
Daniel Savu	326eced987	Revert "refactor(IndexMode): remove `IndexRange`… (#2562)" (#2655) ... This reverts commit 127294decc, to include a cursor bugfix in main and remove an IGP indexing bug added by that PR. ### Description <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	1 year ago
Mattie Conover	8cbfd3cbc0	Support 64-byte txn ids (attempt 2) (#2612) ... ### Description Adds support to TxOutcome for 512-bit ids and to log meta to support Solana. Now that we have primitive types in our crate we were able to create a better solution than before which just stores the EVM 256bit hashes inside of the 512bit one with leading zeros rather than needing to over engineer an enum container for it. ### Drive-by changes - Removed some dead code - Improved primitive type support for 512bit types - Improved serialization support for 512bit types ### Related issues - Fixes #2252 - Replaces #2256 ### Backward compatibility _Are these changes backward compatible?_ Yes-ish, changes one of the internal database key strings. At standup we discussed the possibility of migrating the data and concluded that was a can of worms we did not want to open. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit tests	1 year ago
Mattie Conover	a5e1815746	Rust updates (#2619) ... ### Description Let this one slip a while. Updates our version of rust and as many packages as did not conflict with solana. I think we are going to need to figure something better out going forward with solana since if we have a couple more envs like this, it is going to make updates very hard. ### Drive-by changes None ### Related issues General Maintenance ### Backward compatibility Yes ### Testing Manual & Unit Tests	1 year ago
Mattie Conover	9ed3a24902	Solana e2e tests (#2578) ... ### Description Adds support for testing solana with the e2e tests. This involves 1) Downloading the pre-built solana cli tools 2) Setting up the solana configuration 3) Downloading the pre-built solana programs we need (see https://github.com/hyperlane-xyz/solana-program-library/releases/) 4) The solana programs in the repo 5) Start the solana validator 6) Deploy the solana programs 7) Deploy a warp route 8) Initialize the multisig ism and validator announce 9) Initialize a transfer 10) Wait for the message to be delivered In addition these were "woven" in with the existing E2E test logic, so for instance we only run one new validator and the existing relayer was extended to support these additional chains. This will make it much easier later on to support testes between environments. ### Drive-by changes - Fix a bug in the relayer when trying to index the mailbox - Add support in the hyperlane sealevel CLI for custom solana config paths - Fix a linker error on linux causes by not specifying `no-entrypoint` - Cleaned up unnecessary `no-entrypoint` features in libraries - Minor refactor to how we run commands to avoid arg passing - Created an easy way to define tasks that run asyncly `as_task` - Split up main logic flow into a couple files for easier reading - Use mold linker to speed up builds (well, the linking part) - Removed support for `log_all` to simplify code pathways - Added context when a child process ends unexpectedly for easier debugging - Fixed a bug in the validator where it would infinitely retry to send an announcement on failure without waiting - Cleaned up solana configs - Fixed processes hanging on exit (very annoying when testing locally since you have to manually go kill them) - Added stderr logging to the hyperlane sealevel CLI subprocess calls ### Related issues Fixes #2415 ### Backward compatibility Yes ### Testing Manual	1 year ago
Daniel Savu	127294decc	refactor(IndexMode): remove `IndexRange`, use default index mode per env (#2562) ... ### Description Removes the `index_mode` config field, default to whichever mode is currently supported by our existing envs. Doing this allows for safely removing runtime checks to see if the correct index mode was configured. Additional work is required to support multiple indexing modes per env. Check this discord thread for the full discussion: https://discord.com/channels/935678348330434570/961711527092682783/1133347733411987568 ### Drive-by changes - `IndexRange` is removed entirely and replaced by `RangeInclusive<u32>` - Moves `is_arbitrum_nitro` into a bigger existing `impl HyperlaneDomain` block for clarity ### Related issues This is part 1 of https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2511 ### Backward compatibility Yes ### Testing Relies on existing e2e tests	1 year ago
Trevor Porter	8c5983933b	Sealevel (#2404) ... ### Description Opening for that sweet sweet agent image ### Drive-by changes _Are there any minor or drive-by changes also included?_ ### Related issues - Fixes #[issue number here] ### Backward compatibility _Are these changes backward compatible?_ Yes No _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None Yes ### Testing _What kind of testing have these changes undergone?_ None Manual Unit Tests --------- Co-authored-by: Steven Sloboda <steven@eclipse.builders> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz> Co-authored-by: J M Rossy <jm.rossy@gmail.com> Co-authored-by: Mattie Conover <git@mconover.dev> Co-authored-by: Asa Oines <asaoines@gmail.com> Co-authored-by: Sergei Patrikeev <serejke.best@gmail.com> Co-authored-by: Daniel Savu <23065004+daniel-savu@users.noreply.github.com> Co-authored-by: Kunal Arora <55632507+aroralanuk@users.noreply.github.com> Co-authored-by: yorhodes <yorkerhodesiv@gmail.com> Co-authored-by: Yorke Rhodes <yorke@useabacus.network> Co-authored-by: Yorke Rhodes <email@yorke.dev> Co-authored-by: Asa Oines <asaoines@Asas-MacBook-Pro.local> Co-authored-by: Kunal Arora <kuarora@ucsd.edu> Co-authored-by: Anett <44020788+anettrolikova@users.noreply.github.com> Co-authored-by: Yaqub Mahmoud <yaqub320@gmail.com> Co-authored-by: Alex <alex@alexbh.dev>	1 year ago
Alex	da25b06254	CCIP Read ISM (#2398) ... ### Description CCIP Read ISM implementation, - [x] Contracts - [x] Relayer - [x] Example HTTP backend ### Drive-by changes _Are there any minor or drive-by changes also included?_ ### Related issues - Fixes #[2211] ### Backward compatibility _Are these changes backward compatible?_ Yes, relayers will skip messages they can't handle. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes, relayers won't be able to submit messages that have a CCIP Read ISM. ### Testing _What kind of testing have these changes undergone?_ - [x] Manual - [x] Unit Tests ### General approach - [x] Get a basic [CCIP Read implementation working](https://github.com/AlexBHarley/permissionless-chainlink-feeds/tree/main/contracts) - [x] Build a [mock service that serves CCIP Read compatible data](https://github.com/AlexBHarley/permissionless-chainlink-feeds/tree/main/apps/api) - [x] Update the Relayer to support CCIP Read ISMs	1 year ago
Daniel Savu	6ee5868353	fix(validator): announce directly if no gas estimate (#2454) ... ### Description It has been reported that `contract_call.tx.max_cost()` can fail / not exist on some networks. When that's the case, default to assuming no additional validator gas is required and just send the tx. If the tx fails, report the failure and suggest that gas may be insufficient in the logged error. ### Drive-by changes None ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2426 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None --------- Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com>	1 year ago
Daniel Savu	7889385501	Relayer aggregation ism (#2411) ... ### Description Adds AggregationISM support to the relayer, by adding the scaffolding, sub-ism metadata fetching, and custom metadata encoding. ### Drive-by changes - adds `dry_run_verify(...)` to the `InterchainSecurityModule` trait and implements it for `EthereumInterchainSecurityModule` - changes the default ISM in the test environment from `routingIsm(multisigIsm)` to `routingIsm(aggregationIsm([multisigIsm, messageIdIsm], threshold = 1))` ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/1757 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing The AggregationISM metadata encoding is tested using unit tests whose input was taken from the tests in `AggregationIsm.t.sol`, by printing the individual ISM metadatas and the final metadata. The overall AggregationISM is tested with the `run-locally` script.	1 year ago
Daniel Savu	61892d1d5a	feat(fallback provider): re-prioritize providers on error (#2369) ... ### Description Closes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2189 Reprioritize providers in the `FallbackProvider` when they stop receiving new blocks (although a `max_block_time` has elapsed). The `JsonRpcClient::request(...)` implementation now takes a snapshot of the provider priority list and iterates on that. This is to minimize lock contention - no lock is held across an `await` so there is no deadlock risk. If one of the errors above occurs, the priority list is overwritten. Note this may cause a prior reprioritization to be lost, if the repriotization occured after the current tokio task's snapshot was taken. I'm not entirely sure whether a specific `FallbackProvider` is referenced by more than one task at a time though. ### Drive-by changes A `PrometheusJsonRpcClientConfigExt` trait is added, so the provider can be mocked. ### Testing Unit tests.	1 year ago
Yorke Rhodes	1fff74ebd8	Reduce hyper client pool idle timeout to 20s (#2384) ... ### Description - configures hyper http client to close connections after 20s - makes singleton signer retry (5) times before bubbling error up ### Backward compatibility Yes ### Testing Local binary testing against cloud resources 	1 year ago
Yorke Rhodes	a93daec7a2	Implement single signer task in validator (#2342) ... ### Description Implements singleton signer which allows multiple tasks to use a shared signer that is signing on a single task and migrates validator to use a singleton KMS signer ### Drive-by changes Modifies base agent to allow self to be mutable in run ### Related issues - Fixes error from sharing KMS signer connection across validator tasks https://github.com/rusoto/rusoto/issues/1766 ### Backward compatibility Yes ### Testing E2E Tests --------- Co-authored-by: Mattie Conover <git@mconover.dev>	1 year ago
Mattie Conover	a66652f9f2	Reduce log noise (#2336) ... ### Description Seeing a ton of errors for process estimate costs even though they are fully being handled elsewhere and are also not really an error in the first place. This makes it harder as an operator to spot real errors so I think we should eliminate this error log. (We also already log this error elsewhere so it does not add any new information).  This also targets the `ret` logs produced by tracing since many of them are unhelpful and most of the cases where we actually need a log we already have one. ### Drive-by changes None ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	1 year ago
Mattie Conover	c661f7dccb	Handle additional error case in retrying provider (#2330) ... ### Description Adds a fix for this error message I noticed in the logs: ``` (code: 3, message: execution reverted: INSUFFICIENT_BALANCE, ...) ``` ### Drive-by changes None ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	1 year ago
Yorke Rhodes	2deb9b8973	Fix AWS KMS Signer connection issues (#2319) ... ### Description - Update ethers-rs fork - Create new KMS client instead of using OnceCell ### Backward compatibility Yes No ### Testing Running validator locally with replica of validator DB	2 years ago
Yorke Rhodes	50f04db1fa	Make merkle proofs optional on multisig ISM (#2173) ... ### Description Validators currently sign `(root, index)` checkpoints and during verification, a `message` is passed as calldata, an `id()` is derived, and a `proof` of `id()` at `index` in `root` is verified This provides “all or nothing” censorship resistance guarantees because a validator can only sign roots to allow any contained messages to be processed. We have considered alternatives where validators sign `message` directly and we lose censorship resistance in exchange for eliminating merkle proof verification gas costs. However, if validators sign `(root, index, message)` tuples, we can skip merkle proof verification on the destination chain while still maintaining censorship resistance by providing two valid metadata formats: 1. existing validator signatures and merkle proof verification of inclusion 2. including merkle proof verification for pathway where validators are censoring `message` It’s worth noting the validator is required to index event data to produce this new signature format. However, this does not require historical indexing and new validators being spun up can simply begin indexing from tip. See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2187 for validator changes See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2248 for relayer and e2e test changes ### Drive-by changes Merkle index also optional ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2192 ### Backward compatibility - new ISM deployment is necessary (we could upgrade implementation in theory) - Validator and relayer upgrades ### Testing Unit (fuzz) Tests, E2E tests	2 years ago
Asa Oines	63562c7211	Refactor agent event indexing (#2246) ... ### Description This PR refactors event indexing in the agents, allowing similar logic to be shared across multiple event types (i.e. messages, deliveries, gas payments) and database types (i.e. the Relayer rocks DB and Scraper SQL DB). Furthermore, it adds new syncing modes by way of `MessageSyncCursors` that take advantage of the monotonically increasing dispatched message nonce to sync more intelligently. ### Drive-by changes - Fixes a bug in the existing cursor that caused the same block range to be indexed three times - Modifies kathy to get rid of the idea of "rounds", just sends messages with a sleep in between - Minor modifications to the e2e test for performance - Expand macros in settings - Add scraper to e2e test ### Opportunities for improvement - We can further reduce RPC usage (or improve latency) by sharing the view of the latest finalized block number between cursors - We can speed up the effective time for (a relayer to start deliving messages | the scraper to scraper recent events) by creating forward/backward cursors for gas payments and deliveries where the backwards cursor terminates at index_settings.from - We can remove the need for index_settings.from by terminating backwards cursors based on the block number that the first message was dispatched at ### Related issues - Fixes #[issue number here] ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ E2E tests --------- Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Asa Oines	f6ad1e9a95	Validator automatically self-announces (#2177) ... ### Description This PR adds the ability for validators to self-announce themselves, so long as the validator key has a large enough balance to pay for the tx. This required setting the retry provider to skip retrying reverting transactions, since with a quorum provider and a very speedy blockchain (like hardhat) the transaction is broadcast multiple times and the later transactions conflict with the first one. ### Drive-by changes None ### Related issues - Fixes #2120 ### Backward compatibility _Are these changes backward compatible?_ Yes --------- Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Mattie Conover	ed17ddc32f	Agent Argument Parsing (#2132) ... ### Description All configs can now be set by arguments as well as env and config files. The arguments will take the highest priority and overwrite the others. We should also update the docs to include this new support when it gets released, but since it is an entirely new feature it also can be a stealth release which we document after the fact. ### Drive-by changes ### Related issues - Fixes #2034 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Mattie Conover	b56b022f88	Hyperlane-Core WASM support (#2110) ... ### Description This shuffles some code around and changes dependencies to make `hyperlane-core` WASM compatible and adds a check to the pipeline to ensure it stays that way. To do this it does refactor hyperlane db to the `base` crate, which makes more sense IMO anyway as that is an implementation detail of the agents and not part of the core protocol. ### Drive-by changes ### Related issues - Fixes #1977 ### Backward compatibility _Are these changes backward compatible?_ Yes-ish - It does change where things are but it does not change support. Should only impact people who directly use the libraries. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Asa Oines	1d0e067166	Add relayer support for MultisigIsm and RoutingIsm (#2074) ... ### Description This PR adds support in the relayer for MultisigIsm (non-legacy) and RoutingIsm metadata building. AggregationIsm support is not yet implemented. ### Drive-by changes None ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ end-to-end	2 years ago
Mattie Conover	79669902de	More handling for eth_feeHistory (#2095) ... ### Description Adds additional error contexts for `eth_feeHistory` which should not be retried. Was seeing the error pop up for `celo` and `alfajores`. https://cloudlogging.app.goo.gl/SR7gV3PK5QzbnpFt8 ### Drive-by changes None ### Related issues - https://discord.com/channels/935678348330434570/1096400530722537513 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	2 years ago
Mattie Conover	1722f4ee7e	Better Agent Configuration Parsing (#2070) ... ### Description This is a significant change to how we parse user configs. The main goal of this PR is to give a clear path and env key to any errors that are encountered when parsing configuration. In order to accomplish this, I have crated a new trait `FromRawConf` which defines a set of steps to validate and convert individual fields. Also for every major config struct, there is now a `Raw*` variant of it which is used by the `config` library to reduce errors that occur during deserialization itself. `Raw*` types should always use the most basic form of a value, such as `String`, `StrOrInt`, `bool`, and ideally optional in all cases. Missing values should be handled during the raw conversion and not by `config`. I also needed to make changes to a number of types stored in the parsed config types to force validation forward to the parsing step instead of doing it when we read the config value. This also required me to create a filter to prevent trying to validate certain configs that we would not ever need to load. These changes can also be built on later to support something other than `config` if we choose to, or add support for merging configs from multiple sources since everything is optional. ### Drive-by changes - Default to `http` for connection type - Default to no gas payment enforcement if not specified instead of failing - `ChainSetup` -> `ChainConf` - `GasPaymentEnforcementConfig` -> `GasPaymentEnforcementConf` - Made ethereum connection configuration more forgiving - Moved hyperlane base settings from `mod.rs` to `base.rs` - Moved config chain tests in hyperlane core to `tests` dir to fix a cyclical import problem - Extension traits to help with config in hyperlane core - Moved `StrOrInt` to new hyperlane core `config` module - Support for parsing `U256` from `StrOrInt` - Removed `HexString` type which is now redundant - Updated base settings to use hyperlane domain - Use `heyKey` as signer type if `type` is not specified and `key` is - Moved hyperlane ethereum chain config to a new module ### Related issues - Fixes #2033 - Fixes #2012 ### Backward compatibility _Are these changes backward compatible?_ Yes - This should take in configs in the same shape but be more forgiving in a few places _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Test	2 years ago
Asa Oines	1edbecf9e3	Lag count in Validator agent (#2073) ... ### Description This PR adds a lag to `mailbox.count()` in the validator agent. Without this lag, validators can shut down after the first message because the latestCheckpoint call is lagged and will fail due to the lagged count still being zero. ### Drive-by changes None ### Related issues - Fixes #1764 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None --------- Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Mattie Conover	e91c3100c2	Better Indexing (#2036) ... ### Description * Logs the distance from the tip * Calculates and logs the time to fully sync * Changes indexing timeout when catching up from 1s to 100ms ### Drive-by changes * Cleans up some log messages at the `info` level. * Refactors the indexing logic to be a bit cleaner/simpler * Updated all indexers to use the eta calculations ### Related issues - Fixes #1872 - Fixes #1851 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual	2 years ago
Mattie Conover	b3a1ea5d3a	RPC Error Handling (#2021) ... ### Description We have known for a while that our rules around when to retry certain calls has been a bit rough around the edges. This PR attempts to better handle a number of cases by retrying errors that are server errors and not retrying errors that we expect to have re-occur consistently on retry. I don't expect this to be perfect, but I hope it will get us about 90%+ of the way there. We can followup in the future to capture additional cases. My log investigation: [gcp link](https://console.cloud.google.com/logs/query;cursorTimestamp=2023-02-24T15:24:38.674835362Z;query=severity%3DWARNING%0AjsonPayload.target%3D~%22hyperlane_ethereum::%2528retrying%7Cfallback%2529%22%0A%2528jsonPayload.span.method%3D~%22.*eth_.*%22%20OR%20jsonPayload.fields.method%3D~%22.*eth_.*%22%2529%0A-jsonPayload.fields.text%3D%22%7B%5C%22message%5C%22:%5C%22no%20Route%20matched%20with%20those%20values%5C%22%7D%22%0A-jsonPayload.fields.error%3D~%22.*connection%20closed%20before%20message%20completed.*%22%0A-jsonPayload.fields.error%3D~%22.*tcp%20connect%20error.*%22%0A-jsonPayload.fields.text%3D%22%3Chtml%3E%5Cr%5Cn%3Chead%3E%3Ctitle%3E502%20Bad%20Gateway%3C%2Ftitle%3E%3C%2Fhead%3E%5Cr%5Cn%3Cbody%3E%5Cr%5Cn%3Ccenter%3E%3Ch1%3E502%20Bad%20Gateway%3C%2Fh1%3E%3C%2Fcenter%3E%5Cr%5Cn%3C%2Fbody%3E%5Cr%5Cn%3C%2Fhtml%3E%5Cr%5Cn%22%0A-jsonPayload.fields.error%3D%22%2528code:%20429,%20message:%20Your%20app%20has%20exceeded%20its%20compute%20units%20per%20second%20capacity.%20If%20you%20have%20retries%20enabled,%20you%20can%20safely%20ignore%20this%20message.%20If%20not,%20check%20out%20https:%2F%2Fdocs.alchemy.com%2Freference%2Fthroughput,%20data:%20None%2529%22%0A-jsonPayload.fields.text%3D%22default%20backend%20-%20404%22%0A-jsonPayload.fields.text%3D%22Bad%20Gateway%22%0A-jsonPayload.fields.error%3D~%22.*Connection%20reset%20by%20peer%20%2528os%20error%20104%2529%22%0A-jsonPayload.fields.text%3D%22API%20call%20rejected%20because%20chain%20is%20not%20done%20bootstrapping%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32601,%20message:%20the%20method%20eth_feeHistory%20does%20not%20exist%2Fis%20not%20available,%20data:%20None%2529%22%0A-jsonPayload.target%3D%22hyperlane_ethereum::fallback%22%0A-jsonPayload.fields.error%3D~%22operation%20timed%20out%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32000,%20message:%20404%20Not%20Found:%20%7B%5C%22jsonrpc%5C%22:%5C%222.0%5C%22,%5C%22error%5C%22:%7B%5C%22code%5C%22:404,%5C%22message%5C%22:%5C%22arb1-sequencer%20rate%20limit%20hit.%20%20Try%20again%201%20minute%5C%22%7D%7D,%20data:%20None%2529%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32000,%20message:%20429%20Too%20Many%20Requests:%20%7B%5C%22jsonrpc%5C%22:%5C%222.0%5C%22,%5C%22error%5C%22:%7B%5C%22code%5C%22:429,%5C%22message%5C%22:%5C%22Public%20RPC%20Rate%20Limit%20Hit,%20limit%20will%20reset%20in%2060%20seconds%5C%22%7D%7D,%20data:%20None%2529%22%0A-jsonPayload.fields.error%3D~%22unable%20to%20get%20local%20issuer%20certificate%22%0A-jsonPayload.fields.error%3D%22EOF%20while%20parsing%20a%20value%20at%20line%201%20column%200%22%0A-jsonPayload.fields.text%3D%22internal%20service%20failure%5Cn%22%0A-jsonPayload.fields.text%3D%22%3Chtml%3E%5Cr%5Cn%3Chead%3E%3Ctitle%3E503%20Service%20Temporarily%20Unavailable%3C%2Ftitle%3E%3C%2Fhead%3E%5Cr%5Cn%3Cbody%3E%5Cr%5Cn%3Ccenter%3E%3Ch1%3E503%20Service%20Temporarily%20Unavailable%3C%2Fh1%3E%3C%2Fcenter%3E%5Cr%5Cn%3Chr%3E%3Ccenter%3Enginx%3C%2Fcenter%3E%5Cr%5Cn%3C%2Fbody%3E%5Cr%5Cn%3C%2Fhtml%3E%5Cr%5Cn%22%0A-jsonPayload.fields.text%3D%22%3Chtml%3E%5Cr%5Cn%3Chead%3E%3Ctitle%3E502%20Bad%20Gateway%3C%2Ftitle%3E%3C%2Fhead%3E%5Cr%5Cn%3Cbody%3E%5Cr%5Cn%3Ccenter%3E%3Ch1%3E502%20Bad%20Gateway%3C%2Fh1%3E%3C%2Fcenter%3E%5Cr%5Cn%3Chr%3E%3Ccenter%3Enginx%3C%2Fcenter%3E%5Cr%5Cn%3C%2Fbody%3E%5Cr%5Cn%3C%2Fhtml%3E%5Cr%5Cn%22%0A-jsonPayload.fields.error%3D~%22error%20trying%20to%20connect:%20error:14094410:SSL%20routines:ssl3_read_bytes:sslv3%20alert%20handshake%20failure:..%2Fssl%2Frecord%2Frec_layer_s3.c:1543:SSL%20alert%20number%2040%22%0A-jsonPayload.fields.text%3D~%22The%20gateway%20cannot%20get%20a%20response,%20please%20try%20again%20or%20contact%20the%20administrator%22%0A-jsonPayload.fields.error%3D~%22Connection%20reset%20by%20peer%20%5C%2528os%20error%20104%5C%2529%22%0A-jsonPayload.fields.text%3D%22%3Chtml%3E%5Cr%5Cn%3Chead%3E%3Ctitle%3E502%20Bad%20Gateway%3C%2Ftitle%3E%3C%2Fhead%3E%5Cr%5Cn%3Cbody%3E%5Cr%5Cn%3Ccenter%3E%3Ch1%3E502%20Bad%20Gateway%3C%2Fh1%3E%3C%2Fcenter%3E%5Cr%5Cn%3Chr%3E%3Ccenter%3Ecloudflare%3C%2Fcenter%3E%5Cr%5Cn%3C%2Fbody%3E%5Cr%5Cn%3C%2Fhtml%3E%5Cr%5Cn%22%0A-jsonPayload.fields.text%3D%22%7B%5C%22jsonrpc%5C%22:%5C%222.0%5C%22,%5C%22error%5C%22:%7B%5C%22code%5C%22:0,%5C%22message%5C%22:%5C%22we%20can't%20execute%20this%20request%5C%22%7D,%5C%22id%5C%22:null%7D%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32000,%20message:%20transaction%20underpriced,%20data:%20None%2529%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32000,%20message:%20replacement%20transaction%20underpriced,%20data:%20None%2529%22%0A-jsonPayload.fields.text%3D%22%7B%5C%22jsonrpc%5C%22:%5C%222.0%5C%22,%5C%22result%5C%22:%7B%5C%22code%5C%22:429,%5C%22message%5C%22:%5C%22Total%20number%20of%20requests%20exceeded.%20Want%20higher%20rate%20limit%3F%20Contact%20us%20at%20sales@gateway.fm%5C%22%7D,%5C%22id%5C%22:%5C%22%5C%22%7D%22%0A-jsonPayload.fields.text%3D%22%3Chtml%3E%5Cr%5Cn%3Chead%3E%3Ctitle%3E502%20Bad%20Gateway%3C%2Ftitle%3E%3C%2Fhead%3E%5Cr%5Cn%3Cbody%3E%5Cr%5Cn%3Ccenter%3E%3Ch1%3E502%20Bad%20Gateway%3C%2Fh1%3E%3C%2Fcenter%3E%5Cr%5Cn%3Chr%3E%3Ccenter%3Enginx%2F1.20.2%3C%2Fcenter%3E%5Cr%5Cn%3C%2Fbody%3E%5Cr%5Cn%3C%2Fhtml%3E%5Cr%5Cn%22%0A-jsonPayload.fields.text%3D~%22504%20ERROR%22%0A-jsonPayload.fields.text%3D%22Gateway%20Timeout%22%0A-jsonPayload.fields.text%3D%22404%20page%20not%20found%5Cn%22%0A-jsonPayload.fields.error%3D~%22error%20trying%20to%20connect:%20unexpected%20EOF%22%0A-jsonPayload.fields.error%3D~%22execution%20reverted:%20No%20router%20enrolled%20for%20domain.%20Did%20you%20specify%20the%20right%20domain%20ID%3F%22%0A-jsonPayload.fields.error%3D~%22VM%20Exception%20while%20processing%20transaction:%20revert%20No%20router%20enrolled%20for%20domain.%20Did%20you%20specify%20the%20right%20domain%20ID%3F%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32000,%20message:%20invalid%20transaction:%20nonce%20too%20low,%20data:%20None%2529%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32000,%20message:%20already%20known,%20data:%20None%2529%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32010,%20message:%20AlreadyKnown,%20data:%20None%2529%22%0A-jsonPayload.fields.error%3D~%22message:%20nonce%20too%20low%22%0A-jsonPayload.fields.error%3D~%22message:%20execution%20reverted:%20delivered%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32603,%20message:%20already%20known,%20data:%20None%2529%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32000,%20message:%20ALREADY_EXISTS:%20already%20known,%20data:%20None%2529%22%0A-jsonPayload.fields.text%3D%22%7B%5Cn%20%20%5C%22message%5C%22:%5C%22An%20invalid%20response%20was%20received%20from%20the%20upstream%20server%5C%22%5Cn%7D%22%0A-jsonPayload.fields.error%3D%22%2528code:%20-32601,%20message:%20Method%20not%20found,%20data:%20Some%2528Object%20%7B%5C%22method%5C%22:%20String%2528%5C%22'eth_getLogs'%20is%20not%20available%20on%20our%20public%20API.%20Head%20over%20to%20https:%2F%2Fdocs.blastapi.io%2Fblast-documentation%2Ftutorials-and-guides%2Fusing-blast-to-get-a-blockchain-endpoint%20for%20more%20information%5C%22%2529%7D%2529%2529%22%0A-jsonPayload.fields.error%3D~%22insufficient%20funds%22%0A-jsonPayload.fields.text%3D%22%7B%5C%22jsonrpc%5C%22:%5C%222.0%5C%22,%5C%22result%5C%22:%7B%5C%22code%5C%22:429,%5C%22message%5C%22:%5C%22Total%20number%20of%20requests%20exceeded%5C%22%7D,%5C%22id%5C%22:%5C%22%5C%22%7D%22%0A-jsonPayload.fields.text%3D~%22data%20not%20instance%20of%20model%22;summaryFields=jsonPayload%252Ffields%252Ftext,jsonPayload%252Fspan%252Fmethod,jsonPayload%252Ffields%252Ferror:false:32:beginning;timeRange=2023-02-16T06:56:50.422Z%2F2023-03-23T20:56:50.422Z?project=abacus-labs-dev) Since that will eventually rot I am also listing my query here: ``` severity=WARNING jsonPayload.target=~"hyperlane_ethereum::(retrying|fallback)" (jsonPayload.span.method=~".*eth_.*" OR jsonPayload.fields.method=~".*eth_.*") -jsonPayload.fields.text="{\"message\":\"no Route matched with those values\"}" -jsonPayload.fields.error=~".*connection closed before message completed.*" -jsonPayload.fields.error=~".*tcp connect error.*" -jsonPayload.fields.text="<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n</body>\r\n</html>\r\n" -jsonPayload.fields.error="(code: 429, message: Your app has exceeded its compute units per second capacity. If you have retries enabled, you can safely ignore this message. If not, check out https://docs.alchemy.com/reference/throughput, data: None)" -jsonPayload.fields.text="default backend - 404" -jsonPayload.fields.text="Bad Gateway" -jsonPayload.fields.error=~".*Connection reset by peer (os error 104)" -jsonPayload.fields.text="API call rejected because chain is not done bootstrapping" -jsonPayload.fields.error="(code: -32601, message: the method eth_feeHistory does not exist/is not available, data: None)" -jsonPayload.target="hyperlane_ethereum::fallback" -jsonPayload.fields.error=~"operation timed out" -jsonPayload.fields.error="(code: -32000, message: 404 Not Found: {\"jsonrpc\":\"2.0\",\"error\":{\"code\":404,\"message\":\"arb1-sequencer rate limit hit. Try again 1 minute\"}}, data: None)" -jsonPayload.fields.error="(code: -32000, message: 429 Too Many Requests: {\"jsonrpc\":\"2.0\",\"error\":{\"code\":429,\"message\":\"Public RPC Rate Limit Hit, limit will reset in 60 seconds\"}}, data: None)" -jsonPayload.fields.error=~"unable to get local issuer certificate" -jsonPayload.fields.error="EOF while parsing a value at line 1 column 0" -jsonPayload.fields.text="internal service failure\n" -jsonPayload.fields.text="<html>\r\n<head><title>503 Service Temporarily Unavailable</title></head>\r\n<body>\r\n<center><h1>503 Service Temporarily Unavailable</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n" -jsonPayload.fields.text="<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n" -jsonPayload.fields.error=~"error trying to connect: error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:../ssl/record/rec_layer_s3.c:1543:SSL alert number 40" -jsonPayload.fields.text=~"The gateway cannot get a response, please try again or contact the administrator" -jsonPayload.fields.error=~"Connection reset by peer \(os error 104\)" -jsonPayload.fields.text="<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n<hr><center>cloudflare</center>\r\n</body>\r\n</html>\r\n" -jsonPayload.fields.text="{\"jsonrpc\":\"2.0\",\"error\":{\"code\":0,\"message\":\"we can't execute this request\"},\"id\":null}" -jsonPayload.fields.error="(code: -32000, message: transaction underpriced, data: None)" -jsonPayload.fields.error="(code: -32000, message: replacement transaction underpriced, data: None)" -jsonPayload.fields.text="{\"jsonrpc\":\"2.0\",\"result\":{\"code\":429,\"message\":\"Total number of requests exceeded. Want higher rate limit? Contact us at sales@gateway.fm\"},\"id\":\"\"}" -jsonPayload.fields.text="<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.20.2</center>\r\n</body>\r\n</html>\r\n" -jsonPayload.fields.text=~"504 ERROR" -jsonPayload.fields.text="Gateway Timeout" -jsonPayload.fields.text="404 page not found\n" -jsonPayload.fields.error=~"error trying to connect: unexpected EOF" -jsonPayload.fields.error=~"execution reverted: No router enrolled for domain. Did you specify the right domain ID?" -jsonPayload.fields.error=~"VM Exception while processing transaction: revert No router enrolled for domain. Did you specify the right domain ID?" -jsonPayload.fields.error="(code: -32000, message: invalid transaction: nonce too low, data: None)" -jsonPayload.fields.error="(code: -32000, message: already known, data: None)" -jsonPayload.fields.error="(code: -32010, message: AlreadyKnown, data: None)" -jsonPayload.fields.error=~"message: nonce too low" -jsonPayload.fields.error=~"message: execution reverted: delivered" -jsonPayload.fields.error="(code: -32603, message: already known, data: None)" -jsonPayload.fields.error="(code: -32000, message: ALREADY_EXISTS: already known, data: None)" -jsonPayload.fields.text="{\n \"message\":\"An invalid response was received from the upstream server\"\n}" -jsonPayload.fields.error="(code: -32601, message: Method not found, data: Some(Object {\"method\": String(\"'eth_getLogs' is not available on our public API. Head over to https://docs.blastapi.io/blast-documentation/tutorials-and-guides/using-blast-to-get-a-blockchain-endpoint for more information\")}))" -jsonPayload.fields.error=~"insufficient funds" -jsonPayload.fields.text="{\"jsonrpc\":\"2.0\",\"result\":{\"code\":429,\"message\":\"Total number of requests exceeded\"},\"id\":\"\"}" -jsonPayload.fields.text=~"data not instance of model" ``` Which managed to categorize errors over a large time range and I used to verify that my logic is sound. ### Drive-by changes - Moves code logic to avoid duplication ### Related issues - Fixes #1944 - Fixes hyperlane-xyz/issues#412 - Fixes hyperlane-xyz/issues#414 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual	2 years ago
Mattie Conover	7d6e5d5281	Rust Fuel Updates (#2029) ... ### Description This brings fuel up to the latest version. The main change required was to to abigen process. After some digging I figured out they just moved it to a new crate. ### Drive-by changes - Updated an error message I ran into when trying to run the validator to be more descriptive - Found an unlinked file which I was able to delete - Add `*.ignore` files to gitignore. This allows me to block out some configs I was using for testing. ### Related issues - Fixes hyperlane-xyz/issues#410 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual - Ran the validator against a local node	2 years ago
Asa Oines	f25feecfa7	Scaffolding for multiple ISM types in the relayer (#2028) ... ### Description This PR adds scaffolding in the relayer to support building metadata for multiple ISM types. To add a new ISM type: - Add a new implementation of MetadataBuilder for that type - Add the type to the SupportedIsmTypes enum - Add the type to the match statement in BaseMetadataBuilder ### Drive-by changes None ### Related issues - #1757 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ E2E tests	2 years ago
Asa Oines	63558af85d	Use interfaces in agents (#2022) ... ### Description Move agents to use contract interfaces artifacts rather than the artifacts themselves. This ensures we don't anything that we can't guarantee is in every implementation of that interface. Particularly relevant for IGPs and ISMs. ### Drive-by changes - Moves `solidity/interfaces` to `solidity/contracts/interfaces` so that their artifacts can be used in `update_abis.sh` ### Related issues - #1757 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	2 years ago
Yorke Rhodes	6f7495320b	Regenerate test fixtures and unskip hardhat tests (#2035)	2 years ago
Mattie Conover	1c2eb1bc07	Scraper Refresh (#1932) ... ### Description Since we need to re-scrape everything anyway, wanted to go ahead and make some improvements and bug fixes now. * Adds code to scrape gas payments from the IGP during the sync loop * Updates gas payments table to work with new gas payment format * Fixed support with scraper for new configuration format (adds configurable list of chains to scrape) * Adds a new view for messages to display all relevant information * Adds a new view for gas payments to show total amounts paid for a given message * Fixed a bug introduced by seaorm (0.10) where `last_inserted_id` switched from being the first insert id to being the last * Pulled in a fix from @serejke in #1925 * Updated seaorm to 0.11.1 * Adds support for sepolia * Improved logging output at various levels ### Drive-by changes * Updated fallback provider to still retry even if it only has one url * Adds support for postgres `numeric` types to store Wei values (supported as of seaorm 0.11) * Switches to `bytea` representations of binary values * Switched from the `chrono` library to `time` (`chrono` is outdated and uses an unpatched version of `time`) * Removed redundant timestamp from `message` table * Cleans up some unused code ### Related issues - Fixes #1925 - Fixes #1156 ### Backward compatibility _Are these changes backward compatible?_ No - this has a new database format and for this reason defines entirely new migrations. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes - I changed it to `scraper3` so that we can run it along side the old scraper. It now also lines up with the name in the database. There is a new env var to specify what chains are scraped `HYP_BASE_CHAINSTOSCRAPE`. ### Testing _What kind of testing have these changes undergone?_ Manual - Been running the scraper locally and also deployed a current version of it to testnet3 for a subset of chains.	2 years ago
Mattie Conover	4657fcc737	Better rust package management (#1975) ... ### Description At the suggestion of @tkporter this moves common dependencies and configuration to use the new(ish) cargo workspace configuration feature. This was added in July 2022 and allows us to ensure all versions and configurations are the same across different crates. It does not decrease the typing, but it does reduce the updates required when we go to update a package and keeps duplicates next to each other. This PR does not change any package versions or add/remove any packages. It only moves the configuration. ### Drive-by changes Also sorted packages by name. ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual	2 years ago
Nam Chu Hoai	9abce515a4	Reduce IGP indexer verbosity (#1965) ... ### Description Similar to the change we had to the mailbox to avoid polluting the logs	2 years ago
Nam Chu Hoai	4cda702a30	Require >30 gwei for Polygon (#1952) ... ### Description Fixes #1933. It's not pretty but I would like to sleep tonight	2 years ago
Mattie Conover	52404d25a6	Better context to track down errors during init (#1911) ... ### Description - Additional context to error to track down what chain's configuration is invalid. ### Drive-by changes None ### Related issues - Fixes https://discord.com/channels/935678348330434570/961711527092682783/1082783165803937853 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual	2 years ago
Trevor Porter	81ad2295fe	Move to estimateRetryableTicket to get L2 gas for arbitrum chains (#1949) ... ### Description * Moves to `estimateRetryableTicket` to get Arbitrum L2 gas ### Drive-by changes none ### Related issues - Fixes #1728 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit tests - haven't deployed yet tho	2 years ago
Trevor Porter	7fb935e801	Arbitrum gas amount handling in gas enforcement policies (#1912) ... ### Description See #1728 for detailed explanation Uses Arbitrum's NodeInterface to get the L1 cost of a tx so we can isolate the L2 gas and use that for gas enforcement policies. Very open to alternative ideas in the way I wrote it as well ### Drive-by changes No ### Related issues - Fixes #1728 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests - haven't deployed yet, but plan to deploy to rc testnet soon	2 years ago
Mattie Conover	0049cca1ba	Support On Chain Fee Quoting (#1658) ... ### Description * New gas enforcement policy that uses the on-chain-fee-quoting values. * Change gas enforcement policies to return the gas limit that should be used to submit the transaction. * Change gelato submitter to use the gas limit provided by the gas enforcement policy. * Now indexes the gas amounts for the payments A couple of notes: * We might want to test this with the e2e test. * I took a look into the retying provider and it looks like it already does not retry on failed transaction submissions so I don't think anything needs to be done there after all. ### Drive-by changes ### Related issues - Fixes #1535 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes - changes the internal database format for message payments ### Testing _What kind of testing have these changes undergone?_ Unit Tests --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	2 years ago
Asa Oines	33a77c35e1	Submit txs with EIP 1559 in relayer (#1869)	2 years ago
Mattie Conover	03d32f4009	Eliminate trivial constructors (#1847)	2 years ago
Mattie Conover	29f0c78f5e	Fix message logs (#1825)	2 years ago
Mattie Conover	8105913dea	Improve agent logging (#1797)	2 years ago