hyperlane-monorepo

Commit Graph

Author	SHA1	Message	Date
Trevor Porter	ece2be507a	Cosmos / CosmWasm agents (#2865 ) ### Description <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: Mattie Conover <git@mconover.dev> Co-authored-by: hashableric <hashableric@gmail.com> Co-authored-by: byeongsu-hong <hong@byeongsu.dev> Co-authored-by: Daniel Savu <23065004+daniel-savu@users.noreply.github.com> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz>	1 year ago
Nam Chu Hoai	e4eed2ae3d	fix: e2e working with aggregation 2/2 of `message_id` and `merkle_root` (#2861 ) ### Description - Removed the double mapping logic from the the `merkle_tree_builder.get_proof()` for fixing the following issue: > `get_proof` was expecting a nonce which it then uses to retrieve the `leaf_index` i.e. nonce -> message_id -> leaf_index and then `prove_against_previous` but in our case, we already got the leaf_index in merkle_tree_multisig.rs so what we end up doing is trying to look up the leaf_index twice and either not finding the message_id or proving wrong leaf_index which both causes the "cannot fetch metadata" for the merkle tree builder ### Drive-by changes - additional logging for aggregation to indicate which ism (moduleType, address) we are missing - trace -> debug logging for `sign_and_submit_checkpoint` ### Related issues - closes https://github.com/hyperlane-xyz/issues/issues/695 ### Backward compatibility Yes ### Testing e2e --------- Co-authored-by: -f <kunalarora1729@gmail.com> Co-authored-by: Kunal Arora <55632507+aroralanuk@users.noreply.github.com>	1 year ago
Yorke Rhodes	136e7c8d32	Fix validator checkpoint syncer latest index (#2858 ) Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2856	1 year ago
Nam Chu Hoai	f05c50c92e	Less noisy logs (#2863 ) ### Description Hides some cosmos logs from debug level ### Drive-by changes Only shows submission logs if there is queue in the checkpoint submitter	1 year ago
Yorke Rhodes	3501557d38	Remove legacy multisig from agents (#2839 ) Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2837	1 year ago
Trevor Porter	88346bfb64	Rm finality blocks and refactor checkpoint_submitter to be resilient to edge cases (#2836 ) ### Description * https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/532 wasn't actually fully closed out - finality_blocks was used still for indexing. Sadly testnet4 infra wasn't configuring finality blocks anymore, so we weren't indexing only finalized blocks. Moved fully to reorg_period * Refactored `checkpoint_submitter` in light of races revealed by the above ^ problem. It used to assume that message indexing and the `latest_checkpoint()` call would align with one another, and wasn't resilient to the tree already being past the correctness checkpoint. A couple situations were possible before that aren't now: a. Indexing is ahead of the latest_checkpoint() call, which will result in tree ingesting the new indexed messages and the tree being ahead of the correctness checkpoint :( b. It's possible for the tree() call that constructs the tree initially to be made against a block that's after the next latest_checkpoint() call, which would result in the tree being ahead of the correctness checkpoint from the very beginning :( ### Drive-by changes removed a function that wasn't being used anymore ### Related issues https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/532 ### Backward compatibility Removes finality blocks entirely ### Testing Builds, e2e	1 year ago
Yorke Rhodes	1d18549755	Merge main to v3 (#2812 )	1 year ago
Kunal Arora	9168cca6d2	validator deploy tweaks (#2784 ) ### Description - fixes to deploy testnet4 for agents ### Drive-by changes - truncating labels to length 63 - adding `.relayer` to `agentConfig.contextChainNames` ### Related issues - for https://github.com/hyperlane-xyz/issues/issues/574 ### Backward compatibility Yes ### Testing Manual --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com>	1 year ago
Yorke Rhodes	401b32dddd	Put f64 in hyperlane-core behind float flag (#2788 )	1 year ago
Daniel Savu	c6cea785ae	V3 agents feedback (#2780 ) ### Description Part of the remediations from https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2746 --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	1 year ago
Daniel Savu	af5bd88aa8	V3 agents rebase (#2746 ) ### Description It's your favourite PR coming right back... V3 agents! Closes https://github.com/hyperlane-xyz/issues/issues/561 Builds on top of https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2742 Depends on https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2681 for e2e testing This PR includes: - [x] Merkle tree hook indexer - [x] Merkle tree builder task - [x] Update submitter to trigger retries if no proof is available yet Slightly more detailed overview of the work here: https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2720#issuecomment-1724038643 <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: -f <kunalarora1729@gmail.com> Co-authored-by: Trevor Porter <trkporter@ucdavis.edu> Co-authored-by: Kunal Arora <55632507+aroralanuk@users.noreply.github.com> Co-authored-by: Mattie Conover <git@mconover.dev> Co-authored-by: Guillaume Bouvignies <guillaumebouvignies@gmail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz> Co-authored-by: Guillaume Bouvignies <guillaume.bouvignies@kurtosistech.com>	1 year ago
Mattie Conover	741c9bb553	Update Rust (#2765 ) ### Description Regular rust version and package updates. ### Drive-by changes None ### Related issues ### Backward compatibility Yes ### Testing Unit Tests	1 year ago
Daniel Savu	d65c4e7e0a	Eta calculator fixes (#2735 ) ## Bug 1 Closes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2723 The relayer panic is caused by an overflow, bc of dividing by ~`6.540888459481895e-211`. On my local, the effective rate of indexing starts at `0.61`. ``` {"timestamp":"2023-09-15T09:57:10.746276Z","level":"INFO","fields":{"message":"~~~ blocks_processed: 2508, tip_progression: 2042, elapsed: 757.10340475, old_rate: Some(0.6155037701275111), effective_rate: 0.6155037701275111"},"target":"hyperlane_base::contract_sync::eta_calculator","span":{"domain":"solanadevnet","label":"gas_payments","name":"ContractSync"},"spans":[{"domain":"solanadevnet","label":"gas_payments","name":"ContractSync"}]} ``` But then both the `blocks_processed` and the `tip_progression` are consistently zero, which makes the `new_rate` be zero (`eea423ad04/rust/hyperlane-base/src/contract_sync/eta_calculator.rs (L41)`), and over time it takes over the entire moving average window to make it almost zero, leading to an overflow. 15 mins after that initial log, the effective rate already became `0.00038`. The reason for blocks_processed and tip_progression consistently being zero after the first log is that `eta_calculator.calculate(from, tip)` is always called with the same from and tip although it expects to get the latest values. ### The fix the tip wasn't being set after the sequence_and_tip query here: `eea423ad04/rust/hyperlane-base/src/contract_sync/cursor.rs (L565)` And then the to and from are calculated based on it: `eea423ad04/rust/hyperlane-base/src/contract_sync/cursor.rs (L550)` So even though the sync_state internal variables were kept up-to-date, the min(...) would cause the issue. The first PR commit fixes this. ## Bug 2 There was another bug in the eta calculator, caused by it only using `next_block` to approximate synced state, which doesn't apply to sequence indexing. The way the eta calculator is called had to be changed to use abstract measures of sync progression (which could be blocks or sequences). The second PR commit fixes this, afaict.	1 year ago
Kunal Arora	f0e4f2b89b	Prevent invalid destination griefing for the relayer (#2703 ) ### Description - Adds destinationDomain to the `GasPayment` event - recording destination domain as `destination` while reading events for the relayer ### Drive-by changes - none ### Related issues - fixes https://github.com/hyperlane-xyz/issues/issues/475 ### Backward compatibility No, change in event emitted and relayer indexing ### Testing - Unit --------- Signed-off-by: -f <kunalarora1729@gmail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz>	1 year ago
Nam Chu Hoai	7ab7cf5447	Indicate no quorum checkpoint (#2737 ) ### Description Currently, when the validator is not signing, there is just a unhelpful "Could not fetch metadata" log line. This indicates that no quorum could be found at the debug level as well as adding the checkpoint syncer information ### Backward compatibility Yes ### Testing Manual	1 year ago
Daniel Savu	882fc8b49a	use `warn` logging level for eta errors (#2732 ) ### Description the `debug!` logs either are not occurring very often, or are not displayed at all in the gcp logs <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	1 year ago
Daniel Savu	35fdc74f7c	bump eta calculator log prio (#2726 ) ### Description <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	1 year ago
Daniel Savu	903af462d3	chore(sealevel igp indexing): reduce log verbosity (#2724 ) Removes some unnecessary logs from the sealevel igp indexing fn. Also instruments the eta calculator in response to https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2723 ### Description <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	1 year ago
Daniel Savu	71e8988ccd	Sealevel igp indexing (#2585 ) ### Description Depends on https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2583 Indexes IGP payments related to the relayer's data pda address. Unless this address is specified in the config (`sealevel.relayer_account `), no filtering is applied and all IGP payments are stored in the local database. <!-- What's included in this PR? --> ### Drive-by changes - Sets `HYP_BASE_GASPAYMENTENFORCEMENT` in `run-locally` for the relayer, to test that it correctly indexes the IGP payment before submitting the message - A new config section (`sealevel`) is added to the relayer - The `MessageIndexer` trait is replaced with `SequenceIndexer<HyperlaneMessage>`, renaming `fetch_count_at_tip` to `sequence_at_tip`. `SequenceIndexer` is now common to both the message and igp indexers. - The `parse_addr` macro is modified so it can be reused when parsing the sealevel relayer address config too - `rust/utils/sealevel-test.bash` is included because I was using it to test locally, but I can remove it if the sealevel e2e test already does all the steps there @mattiecnvr - Performs a `try_into` conversion that can be removed once https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2610 is done ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2501 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? --> e2e tests but the pipeline is failing, likely fixed by https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2602 --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	1 year ago
Mattie Conover	17a6e79ea5	Rust agent config parsing (#2687 ) ### Description This does two things: 1) It parses the agent-specific configs for the relayer, validator, and scraper. 2) It removes the rigid structures that had been in place for parsing that were called out earlier and instead parses the serde json values directly. This PR does not cut over to the new versions just yet. That can happen once we start integrating it into the testing and deployment pipelines. ### Drive-by changes - Derive more was added and used where it made sense - The old config macro was removed ### Related issues - Progress on #2215 ### Backward compatibility Yes ### Testing None	1 year ago
Guillaume Bouvignies	478826b3a0	feat: Add possibility to configure a subfolder inside an S3 bucket (#2647 ) ### Description This change adds the possibility to configure a subfolder inside an S3 bucket, to avoid having to spin up an entire bucket for hyperlane. ### Drive-by changes Nothing ### Related issues None, chatted about it on Slack with @nambrot ### Backward compatibility Yes I made the field optional in the `RawCheckpointSyncerConf` so that current config are still valid and will continue to read/write at the root of the bucket. That being said, I'm new to this code so I hope I got it right ### Testing None --------- Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com>	1 year ago
Mattie Conover	389029dafb	Agent Config Base Parsing (#2665 ) ### Description This is a big one for #2215... 1. Splits the parsing of configs from the config definitions to support both the old, new, and future parsers simultaneously. (We might want to rename it from parser to config format or something; not set on this name). 2. Implements the new config format without switching to it or enabling it. This means this PR does introduce dead code that is going to be used in the next PR. ### Drive-by changes - Deleted some unused (or single use) macros defined in hyperlane base - Updated the definition of the config parsing trait to simplify implementation - Added some more ergonomic methods of handling errors in config parsing - Removed any raw `unwrap()` I noticed since these cause difficult to debug errors in prod. - Added some additional validation to catch code mistakes in config path definitions ### Related issues - Progress on #2215 - Comes after #2658 ### Backward compatibility Yes ### Testing Manual (E2E only for now to make sure I did not break the old config format)	1 year ago
Mattie Conover	7c989e7eaa	Agent Config Parsing Casing Fixes (#2666 ) ### Description This is a fix for an issue in the arguments and environment where we are unable to correctly support attribute names due to the casing. The main issue cannot be fully fixed practically by this PR because it would cause some configuration cases to break. For now it adds manual exceptions for the command line argument case and leaves envs as they were. There are then two new parsers that can be used with the new config format. ### Drive-by changes Also adds this to argument parser since it was trivial to copy/paste it and it will allow us to harden parsing when we switch over. ### Related issues - Fixes #2662 - Fixes #2663 - Progress on #2215 ### Backward compatibility Yes ### Testing Unit Tests	1 year ago
Daniel Savu	326eced987	Revert "refactor(IndexMode): remove `IndexRange`… (#2562 )" (#2655 ) This reverts commit `127294decc`, to include a cursor bugfix in main and remove an IGP indexing bug added by that PR. ### Description <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	1 year ago
Mattie Conover	644db5eee1	Remove unused tracing logic (#2639 ) ### Description When I was working on #2215 I noticed that we are not using either zipkin or jaeger at all and possibly never have. Ripping these out to reduce dead code and also simplify our package dependencies. ### Drive-by changes None ### Related issues Discovered through #2215 ### Backward compatibility Yes - we were not using these features and did not document their existence ### Testing Manual - Verified that when running deploy-agents, the generated configs do not have jaeger or zipkin enabled on `main` and then tested to make sure they still work in this branch	1 year ago
Mattie Conover	041cf92d57	Fix insertion errors for txn and block (#2629 ) ### Description This resolves the insertion conflict that is being caused by #2246 that introduced multiple concurrent tasks to scrape. This does not address the underlying concurrency issue I discovered, however. Please review the issue comments for more details. ### Drive-by changes - Cleans up logs a bit (This was useful during testing and I think is useful globally) - Removed an unused file ### Related issues Fixes hyperlane-xyz/issues#493 ### Backward compatibility Yes ### Testing Manual	1 year ago
Mattie Conover	8cbfd3cbc0	Support 64-byte txn ids (attempt 2) (#2612 ) ### Description Adds support to TxOutcome for 512-bit ids and to log meta to support Solana. Now that we have primitive types in our crate we were able to create a better solution than before which just stores the EVM 256bit hashes inside of the 512bit one with leading zeros rather than needing to over engineer an enum container for it. ### Drive-by changes - Removed some dead code - Improved primitive type support for 512bit types - Improved serialization support for 512bit types ### Related issues - Fixes #2252 - Replaces #2256 ### Backward compatibility _Are these changes backward compatible?_ Yes-ish, changes one of the internal database key strings. At standup we discussed the possibility of migrating the data and concluded that was a can of worms we did not want to open. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit tests	1 year ago
Mattie Conover	a5e1815746	Rust updates (#2619 ) ### Description Let this one slip a while. Updates our version of rust and as many packages as did not conflict with solana. I think we are going to need to figure something better out going forward with solana since if we have a couple more envs like this, it is going to make updates very hard. ### Drive-by changes None ### Related issues General Maintenance ### Backward compatibility Yes ### Testing Manual & Unit Tests	1 year ago
Mattie Conover	9ed3a24902	Solana e2e tests (#2578 ) ### Description Adds support for testing solana with the e2e tests. This involves 1) Downloading the pre-built solana cli tools 2) Setting up the solana configuration 3) Downloading the pre-built solana programs we need (see https://github.com/hyperlane-xyz/solana-program-library/releases/) 4) The solana programs in the repo 5) Start the solana validator 6) Deploy the solana programs 7) Deploy a warp route 8) Initialize the multisig ism and validator announce 9) Initialize a transfer 10) Wait for the message to be delivered In addition these were "woven" in with the existing E2E test logic, so for instance we only run one new validator and the existing relayer was extended to support these additional chains. This will make it much easier later on to support testes between environments. ### Drive-by changes - Fix a bug in the relayer when trying to index the mailbox - Add support in the hyperlane sealevel CLI for custom solana config paths - Fix a linker error on linux causes by not specifying `no-entrypoint` - Cleaned up unnecessary `no-entrypoint` features in libraries - Minor refactor to how we run commands to avoid arg passing - Created an easy way to define tasks that run asyncly `as_task` - Split up main logic flow into a couple files for easier reading - Use mold linker to speed up builds (well, the linking part) - Removed support for `log_all` to simplify code pathways - Added context when a child process ends unexpectedly for easier debugging - Fixed a bug in the validator where it would infinitely retry to send an announcement on failure without waiting - Cleaned up solana configs - Fixed processes hanging on exit (very annoying when testing locally since you have to manually go kill them) - Added stderr logging to the hyperlane sealevel CLI subprocess calls ### Related issues Fixes #2415 ### Backward compatibility Yes ### Testing Manual	1 year ago
Daniel Savu	127294decc	refactor(IndexMode): remove `IndexRange`, use default index mode per env (#2562 ) ### Description Removes the `index_mode` config field, default to whichever mode is currently supported by our existing envs. Doing this allows for safely removing runtime checks to see if the correct index mode was configured. Additional work is required to support multiple indexing modes per env. Check this discord thread for the full discussion: https://discord.com/channels/935678348330434570/961711527092682783/1133347733411987568 ### Drive-by changes - `IndexRange` is removed entirely and replaced by `RangeInclusive<u32>` - Moves `is_arbitrum_nitro` into a bigger existing `impl HyperlaneDomain` block for clarity ### Related issues This is part 1 of https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2511 ### Backward compatibility Yes ### Testing Relies on existing e2e tests	1 year ago
Trevor Porter	8c5983933b	Sealevel (#2404 ) ### Description Opening for that sweet sweet agent image ### Drive-by changes _Are there any minor or drive-by changes also included?_ ### Related issues - Fixes #[issue number here] ### Backward compatibility _Are these changes backward compatible?_ Yes No _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None Yes ### Testing _What kind of testing have these changes undergone?_ None Manual Unit Tests --------- Co-authored-by: Steven Sloboda <steven@eclipse.builders> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz> Co-authored-by: J M Rossy <jm.rossy@gmail.com> Co-authored-by: Mattie Conover <git@mconover.dev> Co-authored-by: Asa Oines <asaoines@gmail.com> Co-authored-by: Sergei Patrikeev <serejke.best@gmail.com> Co-authored-by: Daniel Savu <23065004+daniel-savu@users.noreply.github.com> Co-authored-by: Kunal Arora <55632507+aroralanuk@users.noreply.github.com> Co-authored-by: yorhodes <yorkerhodesiv@gmail.com> Co-authored-by: Yorke Rhodes <yorke@useabacus.network> Co-authored-by: Yorke Rhodes <email@yorke.dev> Co-authored-by: Asa Oines <asaoines@Asas-MacBook-Pro.local> Co-authored-by: Kunal Arora <kuarora@ucsd.edu> Co-authored-by: Anett <44020788+anettrolikova@users.noreply.github.com> Co-authored-by: Yaqub Mahmoud <yaqub320@gmail.com> Co-authored-by: Alex <alex@alexbh.dev>	1 year ago
Alex	da25b06254	CCIP Read ISM (#2398 ) ### Description CCIP Read ISM implementation, - [x] Contracts - [x] Relayer - [x] Example HTTP backend ### Drive-by changes _Are there any minor or drive-by changes also included?_ ### Related issues - Fixes #[2211] ### Backward compatibility _Are these changes backward compatible?_ Yes, relayers will skip messages they can't handle. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes, relayers won't be able to submit messages that have a CCIP Read ISM. ### Testing _What kind of testing have these changes undergone?_ - [x] Manual - [x] Unit Tests ### General approach - [x] Get a basic [CCIP Read implementation working](https://github.com/AlexBHarley/permissionless-chainlink-feeds/tree/main/contracts) - [x] Build a [mock service that serves CCIP Read compatible data](https://github.com/AlexBHarley/permissionless-chainlink-feeds/tree/main/apps/api) - [x] Update the Relayer to support CCIP Read ISMs	1 year ago
Daniel Savu	7889385501	Relayer aggregation ism (#2411 ) ### Description Adds AggregationISM support to the relayer, by adding the scaffolding, sub-ism metadata fetching, and custom metadata encoding. ### Drive-by changes - adds `dry_run_verify(...)` to the `InterchainSecurityModule` trait and implements it for `EthereumInterchainSecurityModule` - changes the default ISM in the test environment from `routingIsm(multisigIsm)` to `routingIsm(aggregationIsm([multisigIsm, messageIdIsm], threshold = 1))` ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/1757 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing The AggregationISM metadata encoding is tested using unit tests whose input was taken from the tests in `AggregationIsm.t.sol`, by printing the individual ISM metadatas and the final metadata. The overall AggregationISM is tested with the `run-locally` script.	1 year ago
Mattie Conover	420383eb54	Treat chain names as lowercase (#2249 ) ### Description Had to do a little digging but I think this should fix not only the issue that was encountered but fully add support for whatever case they want, it will just treat it as lowercase but still display it correctly for their config paths. The problem was that `HyperlaneDomain` was already lowercasing the name but we didn't lowercase the key that we used in `from_config_filtered` for the base `RawSettings` to look it up during parsing so we couldn't find the values. ### Drive-by changes None ### Related issues - Fixes #2103 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	1 year ago
Daniel Savu	5d05e58470	fix(contract_sync): saturating math in eta calculator (#2453 ) ### Description It was observed that this function can be called with a `last_block` that is greater than the `currenct_block`, which results in an underflow. Use saturating math to prevent this. ### Drive-by changes ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	1 year ago
Trevor Porter	8e8e47236b	Fix misleading config error to do with connection configs (#2444 ) ### Description If you don't specify a connection config, atm the error message says you're missing the `domain`. I ran into this previously and found it hard to debug; a validator operator in discord also ran into this ### Drive-by changes n/a ### Related issues n/a ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	1 year ago
Daniel Savu	fdc86f687c	Reconstruct relayer `OpQueue` from HyperlaneDB (#2382 ) ### Description Stores the `num_retries` field of `PendingMessage` at its own prefix in HyperlaneDB, so that `PendingMessage` can be reconstructed with the same exponential backoff in the `next_attempt_after` field and avoid making unnecesary RPC calls on restart. To use persisted data, `PendingMessage` is constructed with `from_persisted_retries` instead of the default constructor. If reading from the db fails, the new constructor just defaults to the regular constructor, so it's worth considering replacing the regular constructor with this one entirely. ### Drive-by changes A new db prefix is added (`PENDING_MESSAGE_RETRY_COUNT_FOR_MESSAGE_ID `), as well as related getter / setter methods. Using the new prefix helps avoid a migration. I do see a `processed_by_nonce` prefix, which could be migrated to store data related to `PendingMessage` in general so let me know what you think. Could be a future improvement to merge its prefix with the prefix added in this PR. ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/1902 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ A processor is run on some dummy database entries. The `SerialSubmitter`'s end of the channel is used to ensure that the incoming `PendingMessage`s have the correct backoff in `_next_attempt_after`.	1 year ago
Asa Oines	aaddba776b	Fix latest indexed block gauge (#2387 ) ### Description This PR fixes a bug that affected the latest indexed block gauge for message_dispatch. This allows the gauge to be updated to the "correct" value whether or not a getLogs query actually takes place. ### Drive-by changes None	1 year ago
Yorke Rhodes	1fff74ebd8	Reduce hyper client pool idle timeout to 20s (#2384 ) ### Description - configures hyper http client to close connections after 20s - makes singleton signer retry (5) times before bubbling error up ### Backward compatibility Yes ### Testing Local binary testing against cloud resources ![Screen Shot 2023-06-13 at 5 45 47 PM](https://github.com/hyperlane-xyz/hyperlane-monorepo/assets/3020995/de47a03c-b40f-49fd-a32d-c8da6ecffa6e)	1 year ago
Asa Oines	b75cfc4c6b	Start scraper message_dispatch sync from high watermark (#2377 ) ### Description This PR addresses an issue in which BSC RPC providers have a very hard time providing very old transactions. This caused BSC scraping to fail, since the scraper had never actually indexed very early BSC messages (e.g. nonce 0). By starting at the latest scraped nonce - 1, the scraper will bypass this issue. ### Drive-by changes None	1 year ago
Yorke Rhodes	a93daec7a2	Implement single signer task in validator (#2342 ) ### Description Implements singleton signer which allows multiple tasks to use a shared signer that is signing on a single task and migrates validator to use a singleton KMS signer ### Drive-by changes Modifies base agent to allow self to be mutable in run ### Related issues - Fixes error from sharing KMS signer connection across validator tasks https://github.com/rusoto/rusoto/issues/1766 ### Backward compatibility Yes ### Testing E2E Tests --------- Co-authored-by: Mattie Conover <git@mconover.dev>	1 year ago
Yorke Rhodes	2deb9b8973	Fix AWS KMS Signer connection issues (#2319 ) ### Description - Update ethers-rs fork - Create new KMS client instead of using OnceCell ### Backward compatibility Yes No ### Testing Running validator locally with replica of validator DB	2 years ago
Yorke Rhodes	e4b124cdde	Fix bug in postgres message coercion (#2310 ) ### Description Goerli had a message dispatched (nonce=1964) with a destination domain that failed to be converted from i32 to u32 when being retrieved from the database. This fixes these bugs for domain ids and nonce and improves logs for this type of failure. ### Backward compatibility Scraper needs to be redeployed ### Testing Ran scraper against read replica of explorer DB	2 years ago
Asa Oines	2ace873f9e	Gracefully handle inconsistent RPC providers in ForwardMessageSyncCursor (#2302 ) ### Description RPC providers can be inconsistent. When a message has just been inserted into the mailbox, it's possible that one Mailbox.count() query returns N and a subsequent query returns N-1. ### Drive-by changes None	2 years ago
Asa Oines	90191d43d0	Fix bug in MessageSyncCursor rewind logic (#2297 ) ### Description This PR fixes a bug in the MessageSyncCursor rewind logic that causes the cursor to get stuck. The cursor would look for the message with nonce `n`, find it, and move its `next_block` to the block `b_n` at which `n` was dispatched. It would then index starting at `b_n`, find `n` again, interpret that as a "discontinuity", and rewind back to `b_n`, getting itself stuck permanently. We should be moving to `b_n` and not e.g. `b_n + 1` because we have no guarantee that `n+1` was not also present in `b_n` but just dropped by a flaky `eth_getLogs` implementation. The fix is to only consider messages with nonce > `n` when looking for discontinuities when our target nonce is `n + 1`. ### Drive-by changes - Modifies the E2E tests to use a chunk size of 1, allowing us to reproduce this failure mode without the filtering fix. - Debug log on rewind - Comments to clarify fast forward logic	2 years ago
Mattie Conover	347f9beac8	Revert database prefix (#2294 ) ### Description Small change to remove the domain id from the prefix in the rocks database. ### Drive-by changes No ### Related issues - https://discord.com/channels/935678348330434570/1111394973439504444 ### Backward compatibility _Are these changes backward compatible?_ Yes - returns to last release _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests	2 years ago
Yorke Rhodes	50f04db1fa	Make merkle proofs optional on multisig ISM (#2173 ) ### Description Validators currently sign `(root, index)` checkpoints and during verification, a `message` is passed as calldata, an `id()` is derived, and a `proof` of `id()` at `index` in `root` is verified This provides “all or nothing” censorship resistance guarantees because a validator can only sign roots to allow any contained messages to be processed. We have considered alternatives where validators sign `message` directly and we lose censorship resistance in exchange for eliminating merkle proof verification gas costs. However, if validators sign `(root, index, message)` tuples, we can skip merkle proof verification on the destination chain while still maintaining censorship resistance by providing two valid metadata formats: 1. existing validator signatures and merkle proof verification of inclusion 2. including merkle proof verification for pathway where validators are censoring `message` It’s worth noting the validator is required to index event data to produce this new signature format. However, this does not require historical indexing and new validators being spun up can simply begin indexing from tip. See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2187 for validator changes See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2248 for relayer and e2e test changes ### Drive-by changes Merkle index also optional ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2192 ### Backward compatibility - new ISM deployment is necessary (we could upgrade implementation in theory) - Validator and relayer upgrades ### Testing Unit (fuzz) Tests, E2E tests	2 years ago
Asa Oines	8aa7b62bea	Minor logging improvements to agents (#2271 ) ### Description Removes some unnecessary clutter, adds domain to some spans ### Drive-by changes None	2 years ago
Asa Oines	73fb967ec2	Properly store gas payment expenditure data in agent rocks db (#2268 ) ### Description This PR fixes a bug that resulted in gas payment expenditures not being properly stored in the rocks DB, making them unretrievable. ### Drive-by changes - Introduces a macro to HyperlaneRocksDB to prevent similar bugs from happening in the future	2 years ago
Asa Oines	63562c7211	Refactor agent event indexing (#2246 ) ### Description This PR refactors event indexing in the agents, allowing similar logic to be shared across multiple event types (i.e. messages, deliveries, gas payments) and database types (i.e. the Relayer rocks DB and Scraper SQL DB). Furthermore, it adds new syncing modes by way of `MessageSyncCursors` that take advantage of the monotonically increasing dispatched message nonce to sync more intelligently. ### Drive-by changes - Fixes a bug in the existing cursor that caused the same block range to be indexed three times - Modifies kathy to get rid of the idea of "rounds", just sends messages with a sleep in between - Minor modifications to the e2e test for performance - Expand macros in settings - Add scraper to e2e test ### Opportunities for improvement - We can further reduce RPC usage (or improve latency) by sharing the view of the latest finalized block number between cursors - We can speed up the effective time for (a relayer to start deliving messages \| the scraper to scraper recent events) by creating forward/backward cursors for gas payments and deliveries where the backwards cursor terminates at index_settings.from - We can remove the need for index_settings.from by terminating backwards cursors based on the block number that the first message was dispatched at ### Related issues - Fixes #[issue number here] ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ E2E tests --------- Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago

1 2 3

111 Commits (2da6ccebe8716f40df00eceec4e5473277af429b)