hyperlane-monorepo

Commit Graph

Author	SHA1	Message	Date
Trevor Porter	3f88aa6f6e	Observability of validators for relayers (#3057 ) ### Description Goal of this was to have insight into validators of important sets being "up" Introduces a new metric used by relayers: `hyperlane_observed_validator_latest_index`, e.g.: ``` hyperlane_observed_validator_latest_index{agent="relayer",app_context="default_ism",destination="test1",hyperlane_baselib_version="0.1.0",origin="test2",validator="0x9965507d1a55bcc2695c58ba16fb37d819b0a4dc"} 664 hyperlane_observed_validator_latest_index{agent="relayer",app_context="default_ism",destination="test1",hyperlane_baselib_version="0.1.0",origin="test3",validator="0x976ea74026e726554db657fa54763abd0c3a0aa9"} 641 hyperlane_observed_validator_latest_index{agent="relayer",app_context="default_ism",destination="test2",hyperlane_baselib_version="0.1.0",origin="test1",validator="0x15d34aaf54267db7d7c367839aaf71a00a2c6a65"} 670 hyperlane_observed_validator_latest_index{agent="relayer",app_context="default_ism",destination="test2",hyperlane_baselib_version="0.1.0",origin="test3",validator="0x976ea74026e726554db657fa54763abd0c3a0aa9"} 665 hyperlane_observed_validator_latest_index{agent="relayer",app_context="default_ism",destination="test3",hyperlane_baselib_version="0.1.0",origin="test1",validator="0x15d34aaf54267db7d7c367839aaf71a00a2c6a65"} 652 hyperlane_observed_validator_latest_index{agent="relayer",app_context="default_ism",destination="test3",hyperlane_baselib_version="0.1.0",origin="test2",validator="0x9965507d1a55bcc2695c58ba16fb37d819b0a4dc"} 664 hyperlane_observed_validator_latest_index{agent="relayer",app_context="testapp",destination="test1",hyperlane_baselib_version="0.1.0",origin="test2",validator="0x9965507d1a55bcc2695c58ba16fb37d819b0a4dc"} 658 hyperlane_observed_validator_latest_index{agent="relayer",app_context="testapp",destination="test1",hyperlane_baselib_version="0.1.0",origin="test3",validator="0x976ea74026e726554db657fa54763abd0c3a0aa9"} 641 ``` Tapping into metadata building for multisig ISMs, the relayer will update the metric with the latest indices for the validators in a set. In order to prevent the cardinality being ridiculously high, only certain validator sets are tracked. This is done by introducing an `app_context` label (I'm very open to other names here, for some reason whenever idk how to name some kind of identifier I end up calling it a context 😆) The app context can either be: - if a new setting, --metricAppContexts, is specified, a message will be classified based off the first matching list it matches. E.g. `--metricAppContexts '[{"name": "testapp", "matchingList": [{"recipient_address": "0xd84379ceae14aa33c123af12424a37803f885889", "destination_domain": 13371 }] }]'`. This is nice for e.g. warp route deployments, where the ISM is maybe not a default ISM, and can be changed - if a message doesn't get classified this way, it can also be classified with the "default_ism" app context, which is just for any message that happens to use the default ISM as its "root" ISM This way we have insight in to the default ISM and any application-specific ISMs. Some things to note: - it's possible for a message to actually have more than one validator set, e.g. if it's using an aggregation ISM. In this case, we'll have metrics on the union of all validator sets for that app context - Some effort is required to make sure that metrics don't stick around for a validator that has actually been removed from the set. To handle this, we cache the validator set for an app context and clear out the entire set each time we set the metrics ### Drive-by changes - Zod's nonempty function for strings is deprecated, moves to `.min(1)` instead ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/1762 ### Backward compatibility yes ### Testing Ran locally - I think i'll probably add something in e2e tests, but opening now	11 months ago
Daniel Savu	f73ee0b273	last round of PR remediations (#2791 ) Remediations for https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2746, in addition to https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2780	11 months ago
Daniel Savu	d18f7ae8e1	Cosmos gas price config (#3042 ) ### Description Adds a cosmos-specific config item for setting the minimum gas price, in the format returned by the `cosmos.base.node.v1beta1.Service/Config` cosmos-sdk grpc endpoint. ### Related issues - Fixes https://github.com/hyperlane-xyz/issues/issues/810 ### Backward compatibility No. Will break existing cosmos configs. ### Testing None yet, will test in e2e	11 months ago
Nam Chu Hoai	a888cf7ce9	Actually default to none policy (#2921 ) (#3058 ) ### Description Cherry-picks #2921 which got merged into the wrong branch	12 months ago
Daniel Savu	36de5bcc3c	chore: reduce agents log verbosity, reword (#3026 ) There are way too many `No message found in DB for leaf index` logs to have them at debug level. Also the wording makes it seem like an error when it's not.	12 months ago
Daniel Savu	77aa58c581	Relayer balance metrics (#2976 ) Done: - scaffolding for fetching custom agent metrics - abstractions for building a metrics fetcher for a given VM - querying cosmos balances; e2e tested. - querying evm balances; e2e tested. - Note that as a result, evm addresses are now no longer zero-padded when printed in the logs. This may break existing log queries - fixed a nasty bug on ubuntu where wasmd (osmosisd dependency, part of the grpc query flow) would panic when a block is specified via `x-cosmos-block-height`. The fix was to bump the version of osmosisd from `19.0.0` to `20.5.0`. Note that when running e2e on Mac OS, the osmosis version in use is still 19.0.0. That's because we need a fork that publishes a darwin target binary (currently pointing [here](https://github.com/hashableric/osmosis/releases/download/v19.0.0-mnts/osmosisd-19.0.0-mnts-darwin-arm64.tar.gz)) For follow up PR: - sealevel balance querying I'm open to all renaming suggestions, I just tried to speed through and didn't ponder names too much Relates to https://github.com/hyperlane-xyz/issues/issues/701 Closes https://github.com/hyperlane-xyz/issues/issues/702 (because the balance becomes available in the metrics endpoint for polling)	12 months ago
Trevor Porter	ece2be507a	Cosmos / CosmWasm agents (#2865 ) ### Description <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: Mattie Conover <git@mconover.dev> Co-authored-by: hashableric <hashableric@gmail.com> Co-authored-by: byeongsu-hong <hong@byeongsu.dev> Co-authored-by: Daniel Savu <23065004+daniel-savu@users.noreply.github.com> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz>	1 year ago
Nam Chu Hoai	87796e2ac0	Set a default gas policy (#2830 ) ### Description When no gas enforcement policy is set, by default, we should apply None --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	1 year ago
Nam Chu Hoai	e4eed2ae3d	fix: e2e working with aggregation 2/2 of `message_id` and `merkle_root` (#2861 ) ### Description - Removed the double mapping logic from the the `merkle_tree_builder.get_proof()` for fixing the following issue: > `get_proof` was expecting a nonce which it then uses to retrieve the `leaf_index` i.e. nonce -> message_id -> leaf_index and then `prove_against_previous` but in our case, we already got the leaf_index in merkle_tree_multisig.rs so what we end up doing is trying to look up the leaf_index twice and either not finding the message_id or proving wrong leaf_index which both causes the "cannot fetch metadata" for the merkle tree builder ### Drive-by changes - additional logging for aggregation to indicate which ism (moduleType, address) we are missing - trace -> debug logging for `sign_and_submit_checkpoint` ### Related issues - closes https://github.com/hyperlane-xyz/issues/issues/695 ### Backward compatibility Yes ### Testing e2e --------- Co-authored-by: -f <kunalarora1729@gmail.com> Co-authored-by: Kunal Arora <55632507+aroralanuk@users.noreply.github.com>	1 year ago
Yorke Rhodes	3501557d38	Remove legacy multisig from agents (#2839 ) Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2837	1 year ago
Trevor Porter	88346bfb64	Rm finality blocks and refactor checkpoint_submitter to be resilient to edge cases (#2836 ) ### Description * https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/532 wasn't actually fully closed out - finality_blocks was used still for indexing. Sadly testnet4 infra wasn't configuring finality blocks anymore, so we weren't indexing only finalized blocks. Moved fully to reorg_period * Refactored `checkpoint_submitter` in light of races revealed by the above ^ problem. It used to assume that message indexing and the `latest_checkpoint()` call would align with one another, and wasn't resilient to the tree already being past the correctness checkpoint. A couple situations were possible before that aren't now: a. Indexing is ahead of the latest_checkpoint() call, which will result in tree ingesting the new indexed messages and the tree being ahead of the correctness checkpoint :( b. It's possible for the tree() call that constructs the tree initially to be made against a block that's after the next latest_checkpoint() call, which would result in the tree being ahead of the correctness checkpoint from the very beginning :( ### Drive-by changes removed a function that wasn't being used anymore ### Related issues https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/532 ### Backward compatibility Removes finality blocks entirely ### Testing Builds, e2e	1 year ago
Nam Chu Hoai	746026aa40	Fix MerkleRootMultisigIsm Metadata builder (#2831 ) ### Description Somehow missed that in https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2784/files#diff-c74afa5055013fc40d4626107d9d086be0f21e87907784532f71e8bfe2e1b02bR38	1 year ago
Yorke Rhodes	1d18549755	Merge main to v3 (#2812 )	1 year ago
Kunal Arora	9168cca6d2	validator deploy tweaks (#2784 ) ### Description - fixes to deploy testnet4 for agents ### Drive-by changes - truncating labels to length 63 - adding `.relayer` to `agentConfig.contextChainNames` ### Related issues - for https://github.com/hyperlane-xyz/issues/issues/574 ### Backward compatibility Yes ### Testing Manual --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com>	1 year ago
Daniel Savu	c6cea785ae	V3 agents feedback (#2780 ) ### Description Part of the remediations from https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2746 --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	1 year ago
Daniel Savu	af5bd88aa8	V3 agents rebase (#2746 ) ### Description It's your favourite PR coming right back... V3 agents! Closes https://github.com/hyperlane-xyz/issues/issues/561 Builds on top of https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2742 Depends on https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2681 for e2e testing This PR includes: - [x] Merkle tree hook indexer - [x] Merkle tree builder task - [x] Update submitter to trigger retries if no proof is available yet Slightly more detailed overview of the work here: https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2720#issuecomment-1724038643 <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: -f <kunalarora1729@gmail.com> Co-authored-by: Trevor Porter <trkporter@ucdavis.edu> Co-authored-by: Kunal Arora <55632507+aroralanuk@users.noreply.github.com> Co-authored-by: Mattie Conover <git@mconover.dev> Co-authored-by: Guillaume Bouvignies <guillaumebouvignies@gmail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz> Co-authored-by: Guillaume Bouvignies <guillaume.bouvignies@kurtosistech.com>	1 year ago
Trevor Porter	3f3306c35e	Helloworld program and tooling (#2744 ) ### Description * HelloWorld program added to `rust/sealevel/programs/helloworld` * Some changes to the Sealevel tooling * Refactored the old warp-route-specific deploy management into a more generic framework for idempotently deploying and managing router apps in Sealevel * Added helloworld deploy tooling * Added ISM & owner checking to this router deployment tooling * Added `--require-tx-approval` to prevent txs from being called without first prompting * Added a bunch of new commands for creating txs with certain instructions that were needed along the way -- e.g. setting an ISM, deploying a specific multisig ISM (previously only one would be deployed as a part of `core deploy` * Added a command for configuring multisig ISMs, `multisig-ism-message-id configure`, that takes in a JSON file of multisig ISM configs and applies them onchain * A bit of cleanup / refactor - e.g. removed some old commands like `mailbox receive` * Added foreignDeployments into `RouterApp` * Because RouterApp takes in `contractsMap: HyperlaneContractsMap<Factories>,`, which require attached contracts, a new `readonly foreignDeployments: ChainMap<Address> = {},` is added to the constructor * These foreignDeployments are considered in the return value of `remoteChains(chainName: string)`, but not in `chains()` -- this means that `chains()` now concretely means "chains that can be deployed to / interacted with and that there is an entry in `contractsMap` for, and `remoteChains(chainName: string)` returns any and all remote chains, regardless of whether they can be deployed to / interacted with * Added complete ISM support to the HyperlaneRouterChecker * when checking the ISM, if there's not a match and the ISM is a config, then the ISM will be deployed * Also added RouterViolation, before it'd just throw if there was a violation * Updated the Helloworld, IGP, and core tooling to work when AEE deployments are also configured * Moved to Routing ISM -> Aggregation ISM -> Merkle / Message ID multisig setup Some things to note: * atm there are a few places that have a TODO saying to remove something after some multisig txs are executed, I plan to revisit these after we get some sigs * I've deployed the mainnet sealevel version of helloworld, but haven't been able to enroll it in the EVM chains yet. Waiting for some multisig activity here ### Drive-by changes ### Related issues #2502 ### Backward compatibility I believe it should all be backward compatible ### Testing Deployed, ran checkers, etc	1 year ago
Nam Chu Hoai	a82a8148cb	Cap Msg backoff (#2763 ) ### Description Caps the msg backoff instead of going to insanely large durations ### Backward compatibility Yes ### Testing None	1 year ago
Mattie Conover	741c9bb553	Update Rust (#2765 ) ### Description Regular rust version and package updates. ### Drive-by changes None ### Related issues ### Backward compatibility Yes ### Testing Unit Tests	1 year ago
Kunal Arora	f0e4f2b89b	Prevent invalid destination griefing for the relayer (#2703 ) ### Description - Adds destinationDomain to the `GasPayment` event - recording destination domain as `destination` while reading events for the relayer ### Drive-by changes - none ### Related issues - fixes https://github.com/hyperlane-xyz/issues/issues/475 ### Backward compatibility No, change in event emitted and relayer indexing ### Testing - Unit --------- Signed-off-by: -f <kunalarora1729@gmail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz>	1 year ago
Mattie Conover	892cc5df9c	Support base58 in matching list (#2717 ) ### Description Fixes a current deployment issue due to some changes in the infra deployment. Also caught an error in the schema for the matching list type and updated it.	1 year ago
Daniel Savu	71e8988ccd	Sealevel igp indexing (#2585 ) ### Description Depends on https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2583 Indexes IGP payments related to the relayer's data pda address. Unless this address is specified in the config (`sealevel.relayer_account `), no filtering is applied and all IGP payments are stored in the local database. <!-- What's included in this PR? --> ### Drive-by changes - Sets `HYP_BASE_GASPAYMENTENFORCEMENT` in `run-locally` for the relayer, to test that it correctly indexes the IGP payment before submitting the message - A new config section (`sealevel`) is added to the relayer - The `MessageIndexer` trait is replaced with `SequenceIndexer<HyperlaneMessage>`, renaming `fetch_count_at_tip` to `sequence_at_tip`. `SequenceIndexer` is now common to both the message and igp indexers. - The `parse_addr` macro is modified so it can be reused when parsing the sealevel relayer address config too - `rust/utils/sealevel-test.bash` is included because I was using it to test locally, but I can remove it if the sealevel e2e test already does all the steps there @mattiecnvr - Performs a `try_into` conversion that can be removed once https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2610 is done ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2501 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? --> e2e tests but the pipeline is failing, likely fixed by https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2602 --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	1 year ago
Mattie Conover	17a6e79ea5	Rust agent config parsing (#2687 ) ### Description This does two things: 1) It parses the agent-specific configs for the relayer, validator, and scraper. 2) It removes the rigid structures that had been in place for parsing that were called out earlier and instead parses the serde json values directly. This PR does not cut over to the new versions just yet. That can happen once we start integrating it into the testing and deployment pipelines. ### Drive-by changes - Derive more was added and used where it made sense - The old config macro was removed ### Related issues - Progress on #2215 ### Backward compatibility Yes ### Testing None	1 year ago
Mattie Conover	389029dafb	Agent Config Base Parsing (#2665 ) ### Description This is a big one for #2215... 1. Splits the parsing of configs from the config definitions to support both the old, new, and future parsers simultaneously. (We might want to rename it from parser to config format or something; not set on this name). 2. Implements the new config format without switching to it or enabling it. This means this PR does introduce dead code that is going to be used in the next PR. ### Drive-by changes - Deleted some unused (or single use) macros defined in hyperlane base - Updated the definition of the config parsing trait to simplify implementation - Added some more ergonomic methods of handling errors in config parsing - Removed any raw `unwrap()` I noticed since these cause difficult to debug errors in prod. - Added some additional validation to catch code mistakes in config path definitions ### Related issues - Progress on #2215 - Comes after #2658 ### Backward compatibility Yes ### Testing Manual (E2E only for now to make sure I did not break the old config format)	1 year ago
Daniel Savu	326eced987	Revert "refactor(IndexMode): remove `IndexRange`… (#2562 )" (#2655 ) This reverts commit `127294decc`, to include a cursor bugfix in main and remove an IGP indexing bug added by that PR. ### Description <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	1 year ago
Mattie Conover	644db5eee1	Remove unused tracing logic (#2639 ) ### Description When I was working on #2215 I noticed that we are not using either zipkin or jaeger at all and possibly never have. Ripping these out to reduce dead code and also simplify our package dependencies. ### Drive-by changes None ### Related issues Discovered through #2215 ### Backward compatibility Yes - we were not using these features and did not document their existence ### Testing Manual - Verified that when running deploy-agents, the generated configs do not have jaeger or zipkin enabled on `main` and then tested to make sure they still work in this branch	1 year ago
Mattie Conover	8cbfd3cbc0	Support 64-byte txn ids (attempt 2) (#2612 ) ### Description Adds support to TxOutcome for 512-bit ids and to log meta to support Solana. Now that we have primitive types in our crate we were able to create a better solution than before which just stores the EVM 256bit hashes inside of the 512bit one with leading zeros rather than needing to over engineer an enum container for it. ### Drive-by changes - Removed some dead code - Improved primitive type support for 512bit types - Improved serialization support for 512bit types ### Related issues - Fixes #2252 - Replaces #2256 ### Backward compatibility _Are these changes backward compatible?_ Yes-ish, changes one of the internal database key strings. At standup we discussed the possibility of migrating the data and concluded that was a can of worms we did not want to open. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit tests	1 year ago
Mattie Conover	a5e1815746	Rust updates (#2619 ) ### Description Let this one slip a while. Updates our version of rust and as many packages as did not conflict with solana. I think we are going to need to figure something better out going forward with solana since if we have a couple more envs like this, it is going to make updates very hard. ### Drive-by changes None ### Related issues General Maintenance ### Backward compatibility Yes ### Testing Manual & Unit Tests	1 year ago
Daniel Savu	127294decc	refactor(IndexMode): remove `IndexRange`, use default index mode per env (#2562 ) ### Description Removes the `index_mode` config field, default to whichever mode is currently supported by our existing envs. Doing this allows for safely removing runtime checks to see if the correct index mode was configured. Additional work is required to support multiple indexing modes per env. Check this discord thread for the full discussion: https://discord.com/channels/935678348330434570/961711527092682783/1133347733411987568 ### Drive-by changes - `IndexRange` is removed entirely and replaced by `RangeInclusive<u32>` - Moves `is_arbitrum_nitro` into a bigger existing `impl HyperlaneDomain` block for clarity ### Related issues This is part 1 of https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2511 ### Backward compatibility Yes ### Testing Relies on existing e2e tests	1 year ago
Trevor Porter	8c5983933b	Sealevel (#2404 ) ### Description Opening for that sweet sweet agent image ### Drive-by changes _Are there any minor or drive-by changes also included?_ ### Related issues - Fixes #[issue number here] ### Backward compatibility _Are these changes backward compatible?_ Yes No _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None Yes ### Testing _What kind of testing have these changes undergone?_ None Manual Unit Tests --------- Co-authored-by: Steven Sloboda <steven@eclipse.builders> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz> Co-authored-by: J M Rossy <jm.rossy@gmail.com> Co-authored-by: Mattie Conover <git@mconover.dev> Co-authored-by: Asa Oines <asaoines@gmail.com> Co-authored-by: Sergei Patrikeev <serejke.best@gmail.com> Co-authored-by: Daniel Savu <23065004+daniel-savu@users.noreply.github.com> Co-authored-by: Kunal Arora <55632507+aroralanuk@users.noreply.github.com> Co-authored-by: yorhodes <yorkerhodesiv@gmail.com> Co-authored-by: Yorke Rhodes <yorke@useabacus.network> Co-authored-by: Yorke Rhodes <email@yorke.dev> Co-authored-by: Asa Oines <asaoines@Asas-MacBook-Pro.local> Co-authored-by: Kunal Arora <kuarora@ucsd.edu> Co-authored-by: Anett <44020788+anettrolikova@users.noreply.github.com> Co-authored-by: Yaqub Mahmoud <yaqub320@gmail.com> Co-authored-by: Alex <alex@alexbh.dev>	1 year ago
Alex	da25b06254	CCIP Read ISM (#2398 ) ### Description CCIP Read ISM implementation, - [x] Contracts - [x] Relayer - [x] Example HTTP backend ### Drive-by changes _Are there any minor or drive-by changes also included?_ ### Related issues - Fixes #[2211] ### Backward compatibility _Are these changes backward compatible?_ Yes, relayers will skip messages they can't handle. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes, relayers won't be able to submit messages that have a CCIP Read ISM. ### Testing _What kind of testing have these changes undergone?_ - [x] Manual - [x] Unit Tests ### General approach - [x] Get a basic [CCIP Read implementation working](https://github.com/AlexBHarley/permissionless-chainlink-feeds/tree/main/contracts) - [x] Build a [mock service that serves CCIP Read compatible data](https://github.com/AlexBHarley/permissionless-chainlink-feeds/tree/main/apps/api) - [x] Update the Relayer to support CCIP Read ISMs	1 year ago
Kunal Arora	d69a3f639a	No metadata relayer support for hooks (#2495 ) ### Description - Adding `NoMetadataRelayer` moduleType which returns an empty `vec![]` on `MetadataBuilder::build()` - Adding `HyperlaneHookDeployer` which lets you deploy the hook, ISM and a testRecipient which specifies to use the `AbtractNativeISM` ### Drive-by changes - none ### Related issues - Accompanies #2316 - Fixes https://github.com/hyperlane-xyz/issues/issues/379 for Optimism - Related to adding native hooks to Mailbox V3 https://github.com/hyperlane-xyz/issues/issues/486 ### Backward compatibility Backward compatible? Yes Tooling changes needed? Need to update node to ^18 ### Testing - Manual on testnet3	1 year ago
Daniel Savu	7889385501	Relayer aggregation ism (#2411 ) ### Description Adds AggregationISM support to the relayer, by adding the scaffolding, sub-ism metadata fetching, and custom metadata encoding. ### Drive-by changes - adds `dry_run_verify(...)` to the `InterchainSecurityModule` trait and implements it for `EthereumInterchainSecurityModule` - changes the default ISM in the test environment from `routingIsm(multisigIsm)` to `routingIsm(aggregationIsm([multisigIsm, messageIdIsm], threshold = 1))` ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/1757 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing The AggregationISM metadata encoding is tested using unit tests whose input was taken from the tests in `AggregationIsm.t.sol`, by printing the individual ISM metadatas and the final metadata. The overall AggregationISM is tested with the `run-locally` script.	1 year ago
Mattie Conover	16a29edb00	Dedup chain parsing (#2464 ) ### Description Missed this by accident from #2249 ### Drive-by changes None ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests	1 year ago
Mattie Conover	420383eb54	Treat chain names as lowercase (#2249 ) ### Description Had to do a little digging but I think this should fix not only the issue that was encountered but fully add support for whatever case they want, it will just treat it as lowercase but still display it correctly for their config paths. The problem was that `HyperlaneDomain` was already lowercasing the name but we didn't lowercase the key that we used in `from_config_filtered` for the base `RawSettings` to look it up during parsing so we couldn't find the values. ### Drive-by changes None ### Related issues - Fixes #2103 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	1 year ago
Daniel Savu	fdc86f687c	Reconstruct relayer `OpQueue` from HyperlaneDB (#2382 ) ### Description Stores the `num_retries` field of `PendingMessage` at its own prefix in HyperlaneDB, so that `PendingMessage` can be reconstructed with the same exponential backoff in the `next_attempt_after` field and avoid making unnecesary RPC calls on restart. To use persisted data, `PendingMessage` is constructed with `from_persisted_retries` instead of the default constructor. If reading from the db fails, the new constructor just defaults to the regular constructor, so it's worth considering replacing the regular constructor with this one entirely. ### Drive-by changes A new db prefix is added (`PENDING_MESSAGE_RETRY_COUNT_FOR_MESSAGE_ID `), as well as related getter / setter methods. Using the new prefix helps avoid a migration. I do see a `processed_by_nonce` prefix, which could be migrated to store data related to `PendingMessage` in general so let me know what you think. Could be a future improvement to merge its prefix with the prefix added in this PR. ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/1902 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ A processor is run on some dummy database entries. The `SerialSubmitter`'s end of the channel is used to ensure that the incoming `PendingMessage`s have the correct backoff in `_next_attempt_after`.	1 year ago
Daniel Savu	1921c45885	chore(relayer): improve ISM address error message (#2345 ) Closes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2090 Recent improvements from https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2195 already catch the error in that user report. This PR just adds a hint as to why fetching may have failed; in that report, the user's `fallback` function was returning an empty value which caused casting to an `address` to fail and revert, so the error occurs because the returned value is malformed. Thanks to @tkporter for walking me through the contracts to better understand the scope of https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2090.	1 year ago
Yorke Rhodes	a93daec7a2	Implement single signer task in validator (#2342 ) ### Description Implements singleton signer which allows multiple tasks to use a shared signer that is signing on a single task and migrates validator to use a singleton KMS signer ### Drive-by changes Modifies base agent to allow self to be mutable in run ### Related issues - Fixes error from sharing KMS signer connection across validator tasks https://github.com/rusoto/rusoto/issues/1766 ### Backward compatibility Yes ### Testing E2E Tests --------- Co-authored-by: Mattie Conover <git@mconover.dev>	1 year ago
Yorke Rhodes	bb0d688311	Add origin filter to relayer msg processor (#2301 ) ### Description Ensures the relayer never attempts to process a message destined for it's own origin domain. ### Related issues - Fixes https://github.com/hyperlane-xyz/issues/issues/464 ### Backward compatibility Yes ### Testing E2E tests	2 years ago
Yorke Rhodes	50f04db1fa	Make merkle proofs optional on multisig ISM (#2173 ) ### Description Validators currently sign `(root, index)` checkpoints and during verification, a `message` is passed as calldata, an `id()` is derived, and a `proof` of `id()` at `index` in `root` is verified This provides “all or nothing” censorship resistance guarantees because a validator can only sign roots to allow any contained messages to be processed. We have considered alternatives where validators sign `message` directly and we lose censorship resistance in exchange for eliminating merkle proof verification gas costs. However, if validators sign `(root, index, message)` tuples, we can skip merkle proof verification on the destination chain while still maintaining censorship resistance by providing two valid metadata formats: 1. existing validator signatures and merkle proof verification of inclusion 2. including merkle proof verification for pathway where validators are censoring `message` It’s worth noting the validator is required to index event data to produce this new signature format. However, this does not require historical indexing and new validators being spun up can simply begin indexing from tip. See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2187 for validator changes See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2248 for relayer and e2e test changes ### Drive-by changes Merkle index also optional ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2192 ### Backward compatibility - new ISM deployment is necessary (we could upgrade implementation in theory) - Validator and relayer upgrades ### Testing Unit (fuzz) Tests, E2E tests	2 years ago
Asa Oines	8aa7b62bea	Minor logging improvements to agents (#2271 ) ### Description Removes some unnecessary clutter, adds domain to some spans ### Drive-by changes None	2 years ago
Asa Oines	73fb967ec2	Properly store gas payment expenditure data in agent rocks db (#2268 ) ### Description This PR fixes a bug that resulted in gas payment expenditures not being properly stored in the rocks DB, making them unretrievable. ### Drive-by changes - Introduces a macro to HyperlaneRocksDB to prevent similar bugs from happening in the future	2 years ago
Asa Oines	63562c7211	Refactor agent event indexing (#2246 ) ### Description This PR refactors event indexing in the agents, allowing similar logic to be shared across multiple event types (i.e. messages, deliveries, gas payments) and database types (i.e. the Relayer rocks DB and Scraper SQL DB). Furthermore, it adds new syncing modes by way of `MessageSyncCursors` that take advantage of the monotonically increasing dispatched message nonce to sync more intelligently. ### Drive-by changes - Fixes a bug in the existing cursor that caused the same block range to be indexed three times - Modifies kathy to get rid of the idea of "rounds", just sends messages with a sleep in between - Minor modifications to the e2e test for performance - Expand macros in settings - Add scraper to e2e test ### Opportunities for improvement - We can further reduce RPC usage (or improve latency) by sharing the view of the latest finalized block number between cursors - We can speed up the effective time for (a relayer to start deliving messages \| the scraper to scraper recent events) by creating forward/backward cursors for gas payments and deliveries where the backwards cursor terminates at index_settings.from - We can remove the need for index_settings.from by terminating backwards cursors based on the block number that the first message was dispatched at ### Related issues - Fixes #[issue number here] ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ E2E tests --------- Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Mattie Conover	5e38cdf31a	Omniscient Relayer (#2195 ) ### Description Adds support for running a relayer which delivers messages from multiple origin chains instead of just one origin chain. ### Drive-by changes _Are there any minor or drive-by changes also included?_ - The E2E pipeline got a major re-work. It both runs faster and also tests the new changes. - SerialSubmitter has been abstracted to support additional operation types such as oracle updates - SerialSubmitter has been updated to actually confirm a transaction was included and is safe from re-orgs - SerialSubmitter has been updated to have higher throughput by pipelining operations ### Related issues - Blocked by #2148 - Fixes #1899 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Mattie Conover	24f9642ea5	Agent docs and config improvements (#2148 ) ### Description Bunch of minor tweaks and updates based on what we learned from EthTokyo. - Updated Rust readme and the root readme to align with the hyperlane docs - Defaults to a 5 sec validator interval now - Makes the metrics port default to `9090` and have a soft failure if it can't start - Now will automatically create the checkpointsyncer directory if it is missing - Hyperlane database now officially uses the domain name and id as a prefix internally to support multiple relayers using the same database - ### Drive-by changes Refactored and cleaned up the database code. Removed a lot of duplicate/unnecessary/redundant functions. ### Related issues - Fixes #2113 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes - This changes the database prefixes internally to use domain name and domain id; also prepares us for #1899 ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Asa Oines	3adb6de5a5	Relayer warns if validator not announced (#2160 ) ### Description Explicitly call out if an ISM is configured with a validator that has not been announced. ### Related issues - Fixes #2120	2 years ago
Mattie Conover	3e5b20c88e	Better validation of signers during config parsing (#2139 ) ### Description This improves the validation during config parsing to ensure that all the expected signers are configured which means the errors will have the correct config path. ### Drive-by changes Also updated the checks for missing chain configurations to be more clear and use the chain config path rather than the config that indicated the config was required and removes a now redundant check. ### Related issues - Fixes #2101 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual	2 years ago
Mattie Conover	b56b022f88	Hyperlane-Core WASM support (#2110 ) ### Description This shuffles some code around and changes dependencies to make `hyperlane-core` WASM compatible and adds a check to the pipeline to ensure it stays that way. To do this it does refactor hyperlane db to the `base` crate, which makes more sense IMO anyway as that is an implementation detail of the agents and not part of the core protocol. ### Drive-by changes ### Related issues - Fixes #1977 ### Backward compatibility _Are these changes backward compatible?_ Yes-ish - It does change where things are but it does not change support. Should only impact people who directly use the libraries. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Mattie Conover	910e5c84e9	Error message to indicate if there are no validators (#2099 ) ### Description Adds a more descriptive error message for failing to reach validator quorum when there are no validators defined. ### Drive-by changes No ### Related issues - Fixes #2031 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	2 years ago
Asa Oines	1d0e067166	Add relayer support for MultisigIsm and RoutingIsm (#2074 ) ### Description This PR adds support in the relayer for MultisigIsm (non-legacy) and RoutingIsm metadata building. AggregationIsm support is not yet implemented. ### Drive-by changes None ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ end-to-end	2 years ago

1 2 3 4

177 Commits (0cad4f7407f2d25a93e17c001bca9a2e16b52c93)