hyperlane-monorepo

Commit Graph

Author	SHA1	Message	Date
Trevor Porter	26a26f4073	Agent transaction overrides, ensure agent configs are updated in CI (#3563 ) ### Description - Adds some basic transaction overrides for Ethereum chains to be used by the agents. This follows the existing config schema in chain metadata that we have been using on the TS side for a while. The overrides are as follows: - `transactionOverrides.gasPrice` - if specified, non EIP-1559 txs are used, and they use this gasPrice - `transactionOverrides.gasLimit` - if specified, this takes precedence over any gas estimation that's done and is the gas limit that's used for txs. (This is pretty niche and I don't imagine we'll ever wanna use this internally, but happens to cover the use case described [here](https://discord.com/channels/935678348330434570/984123861144600587/1224617234580766741), which is nice to cover) - `transactionOverrides.maxFeePerGas` - if specified and if the chain supports EIP 1559 txs, is used for the `maxFeePerGas` - `transactionOverrides.maxPriorityFeePerGas` - if specified and if the chain supports EIP 1559 txs, is used for the `maxPriorityFeePerGas` - Updates some of the transaction overrides in mainnet3 / testnet4 to be more reasonable now that we'll be using them to submit transactions pretty frequently - Note that atm the transaction overrides are most easily applied to the agents via the config JSONs, and not via our K8s configuration. The upshot of this is if we wanna change a transaction override right now, the easiest way to do it will be to run `update-agent-config.ts` and then build a new image. Created https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3565 to track making this experience better - Part of this change means that it's more important to keep the `transactionOverrides` specified in the SDK / infra up to date with the agent config. So I ended up doing https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3325 along the way, which also required doing https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3311. - To fix the issue with arbitrum blocks, we now require the index.from to be specified in the chain metadata if the chain's technical stack is arbitrum nitro - A new script, `update-agent-config.ts` is added, and it's ran for mainnet3 and testnet4 in CI to make sure there's no diff (similar idea to what we do with prettier) - Along the way I renamed mainnet3_config.json and testnet4_config.json to mainnet_config.json and testnet_config.json. This has been a source of confusion with ppl in the past - in v2 we named them without any numeric suffixes, so we now just do that again ### Drive-by changes - There's a drive-by to poll more frequently for pending transactions (2 seconds). The default is 7 seconds in Ethers, and for many chains that have really quick block times, this may be a way for us to speed up us learning that a transaction is included - Don't enforce gas on the inevm routes, looks like there's some weirdness there that was reported by Nam ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3562 - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3325 - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3311 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	8 months ago
Noah Bayindirli 🥂	38358ececd	feat: deprecate Mumbai testnet (#3521 ) ### Description - Removes all reference to Mumbai testnet, since it is soon being [deprecated](https://polygon.technology/blog/polygon-pos-is-cooking-the-napoli-upgrade-means-better-ux-the-mumbai-testnet-takes-a-bow) by the Polygon team - Monorepo docker tag: https://console.cloud.google.com/gcr/images/abacus-labs-dev/global/hyperlane-monorepo@sha256:d4b44359cae9c4e3bdeb6f965c59d056f7fce72c42069adc0dea038ac600785a/details?authuser=5&project=abacus-labs-dev&tab=info - Agent docker tag: https://console.cloud.google.com/gcr/images/abacus-labs-dev/global/hyperlane-agent@sha256:a578d96f7f06721d0b6cd8095d6a1fa5d22f01b59eba3320ee3b1fe34ca3c5df/details?authuser=5&project=abacus-labs-dev&tab=info ### Drive-by changes - Adds missing Testnet domains, Fuji & BSC Testnet - Fix for failing `test-rs` CI check with ``` error[E0004]: non-exhaustive patterns: `chain::KnownHyperlaneDomain::Fuji` and `chain::KnownHyperlaneDomain::BinanceSmartChainTestnet` not covered --> hyperlane-core/src/utils.rs:226:15 \| 226 \| match $v { \| ^^ patterns `chain::KnownHyperlaneDomain::Fuji` and `chain::KnownHyperlaneDomain::BinanceSmartChainTestnet` not covered ``` ### Related issues - Dependency on #3523 - Fixes #1181 ### Backward compatibility - [x] Backwards compatible - [x] Requires docker image tags to be added post-build - [x] Requires agent artifact cleanup in cloud account: https://www.notion.so/hyperlanexyz/Runbook-83c755f2652943289cf98cd1309487a8?pvs=4#77730653a12f409dacfbd427cab545d3 ### Testing * Build success * `yarn hyperlane chains list -e testnet` does not return `mumbai` * fork.sh script success locally	8 months ago
Daniel Savu	ff6b06079e	Half the relayer timeout on v3 (#3508 ) ### Description V3 counterpart of https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/3431 Co-authored-by: nambrot <nambrot@googlemail.com>	8 months ago
Trevor Porter	176e710cae	Fix BSC gas price issues by treating chains with a base fee of 0 as non-1559 (#3462 ) ### Description - Fixes #3396 - See v2 backport here #3463 So turns out this is because: - BSC nodes tend to enforce a minimum gas price of 3 gwei when you submit a tx - it's possible for validators to include privileged txs at a lower gas price, so in practice (probably some mev stuff ?) there are a bunch of txs in blocks with transactions lower than 3 gwei - BSC's 1559 situation is they have a base fee of zero, so it's basically like they just use legacy txs - when we try to figure out what to pay in prio fees, we are suggested a 1 gwei priority fee: ``` $ cast rpc eth_feeHistory 10 latest "[5.0]" --rpc-url https://rpc.ankr.com/bsc {"oldestBlock":"0x23706f7","reward":[["0x3b9aca00"]],"baseFeePerGas":["0x0","0x0"],"gasUsedRatio":[0.1149265]} $ cast td 0x3b9aca00 1000000000 ``` So options are: 1. tx overrides for bsc, set it to 3 gwei 2. use a different eip 1559 estimator and set the percentile used in eth_feeHistory to something higher. Atm the percentile is 5%, if you change this to 50% we seem to get 3 gwei. This would have consequences for other chains tho 3. another heuristic, to fall back to legacy txs (where we can trust the eth_gasPrice to give us an accurate price) whenever the base fee is 0 I'm a tx override hater and we don't have this in the agents so far so I'm gonna go with 3. It's the least intrusive change	8 months ago
Paul Balaji	76bd730103	feat: remove goerli testnets (#3389 )	9 months ago
Daniel Savu	e737d998ff	Cosmos block-by-block indexing (#3245 ) ### Description Implements cosmos block-by-block indexing, parallelizing where possible: the `block` and `block_results` calls, and the individual block queries within a range. Also reuses the existing `handle_txs` logic, to improve readability. ### Drive-by changes Adds `tokio::task::JoinError` to the `hyperlane-core` error enum ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3234 - Added some hacky retrying logic to `get_logs_in_block`, since we were seeing lots of `503` http errors. In https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3264 we should make sure to remove this and propagate the error instead, to retry via the cursor / indexer logic. I edited the description of 3264 to reflect this ### Backward compatibility Yes ### Testing e2e --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	9 months ago
Trevor Porter	9a83eb4d57	Sequence-aware indexing refactor (#3262 ) ### Description Part 1 of an indexing refactor. This mostly focuses on the sequence-aware cursors. Changes: - Some renames, e.g. `SequenceIndexer` -> `SequenceAwareIndexer`, `sequence_and_tip` -> `latest_sequence_count_and_tip` - Moved cursors out of `contract_sync/cursor.rs` into `contract_sync/cursors/*` - The goal was to make the the Forward sequence-aware cursor, the Backward sequence-aware, and the ForwardBackward sequence aware cursors more resilient, easier to understand, and better tested. - General strategy for doing this was to: - Only move onto a new range to index if the logs passed into the cursor's `update` function justify this - If updated with any weird data, like gaps in logs, rewind the cursor and retry previously queried ranges I'd recommend starting the review in the following order: 1. `rust/hyperlane-base/src/contract_sync/cursors/sequence_aware/forward.rs` - where `ForwardSequenceAwareSyncCursor` & tests live 2. `rust/hyperlane-base/src/contract_sync/cursors/sequence_aware/backward.rs` - where `BackwardSequenceAwareSyncCursor` & tests live 3. `rust/hyperlane-base/src/contract_sync/cursors/sequence_aware/mod.rs` - where `ForwardBackwardSequenceAwareSyncCursor` lives ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	9 months ago
Daniel Savu	f11878ad08	CCIP Read metadata builder fix (#3246 ) ### Description `H160`'s `ToString` implementation adds ellipsis points instead of outputting the full hex string, for some reason (e.g. `0xc66a…7b6f`). This will result in an error when `Mailbox.process` is called on the CCIP Read ISM. ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	10 months ago
Daniel Savu	3cbb06a07a	feat: fetch chain-specific metrics in standalone task (#3214 ) ### Description Spawning a tokio task to fetch chain-specific metrics in provider middleware was wasteful because we ended up with 15 tasks doing the same job. Now there is a single task that is spawned inside the relayer, which I added in the struct that used to only fetch relayer balance. Some questions: - I'm realising in the current state I'm probably removing these metrics from validators. Lmk if yes and I'll add back - the chain-specific metrics were moved out of `MiddlewareMetrics` because they're no longer part of middleware, but I wasn't sure where to place them. Maybe `CoreMetrics` is a better place than the current `custom_metrics.rs` file? `tokio-metrics` turned out not to be useful because we'd need to instrument every call site of `tokio::spawn` with it. We should still do it but as a separate task imo. ### Drive-by changes - This PR also makes it very easy to add the metrics tasks for new chains, by extending the `HyperlaneProvider` trait with a `get_chain_metrics` call which seems reasonably general to me to have implemented by all providers. (Bc we currently only track these for evm chains) - the description, naming and logic of the `gas_price` metric was also changed to support new chains ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3047 ### Backward compatibility Yes ### Testing Still need to manually test	10 months ago
Daniel Savu	f2a0e92c7f	Cosmos grpc fallbackprovider (#3139 ) ### Description Implements grpc fallback provider logic for cosmos - initially tried implementing the fallback provider deprioritization logic at middleware level like in the EVM. The difference between ethers and cosmrs is that in the latter, middleware can only live at the transport layer (`tower` crate level). - based on this github [issue](https://github.com/hyperium/tonic/issues/733), that actually doesn't look possible, because the http::Request type isn't `Clone` so it can't be submitted to multiple providers - ended up implementing the fallback provider at the application layer, by keeping an array of grpc channels - There is now a `call` method in `hyperlane_core::FallbackProvider` which I'm actually really happy with. This method handles the fallbackprovider-specific logic by taking in an async closure, running it on each provider, and iterating providers if the closure call fails. In `grpc.rs` you can see how this is slightly verbose but I think it's quite manageable. The only part that bugs me is having to duplicate `Pin::from(Box::from(future))`, but that's need afaict because the regular closure returns an anonymous type - adds `grpcUrls` and `customGrpcUrls` config items - tests the cosmos fallback provider e2e ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues - Fixes: https://github.com/hyperlane-xyz/issues/issues/998 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	10 months ago
Daniel Savu	c9ef41c109	Fallback Provider Refactor (#3127 ) ### Description - Generalizes the fallback provider. Now any provider type can be used in it, as long as it implements `Into<Box<dyn BlockNumberGetter>> + Clone` (aka it can be wrapped into a struct that reads the current block #, and it can be cloned). - moves the fallback provider implementation to hyperlane-core - had serious issues with getting versions to successfully resolve - next time I run into issues, strongly inclined to create a new crate with dependencies for all chains, but which are only depended on by the agents codebase (rather than the contract as well). Chatted with @tkporter about this. - now fallbackprovider needs to be wrapped in a newtype struct for chain-specific trait implementations - If `Deref` is implemented for the newtype struct, all the function calls will still work (although using Deref for newtypes is [debatable](https://users.rust-lang.org/t/understanding-the-perils-of-deref/47958/18)) - the `request` function required by the `JsonRpcClient` still has some reusable logic but I left it there until it's clearer what can be reused by the cosmos fallback provider - the fallback provider unit tests still live in hyperlane-ethereum because of the logic in `require` ### Drive-by changes - needed to use `async-rwlock` instead of `tokio` for the `RwLock` structure used by the fallback provider for interior mutability. This is because the solana program requirements had rust pinned too far into the past for `tokio` to resolve to a version. ### Related issues - Fixes https://github.com/hyperlane-xyz/issues/issues/892 ### Backward compatibility Yes ### Testing Unit tests that were already there for the fallback provider	10 months ago
Daniel Savu	cdbaf9ef02	fix: dedup eth logs (#3108 ) ### Description Deduplicates ethereum logs by collecting them into a HashSet before returning a Vec. I couldn't find a way to deduplicate using an ethers config, but since `fetch_logs` is only called once in hyperlane-base, I'm doing the deduplication there "globally". I also noted in doc comments that duplicates may be returned. ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3070 ### Backward compatibility Yes ### Testing Will rely on e2e passing but haven't added unit tests for this scenario	11 months ago
Daniel Savu	d18f7ae8e1	Cosmos gas price config (#3042 ) ### Description Adds a cosmos-specific config item for setting the minimum gas price, in the format returned by the `cosmos.base.node.v1beta1.Service/Config` cosmos-sdk grpc endpoint. ### Related issues - Fixes https://github.com/hyperlane-xyz/issues/issues/810 ### Backward compatibility No. Will break existing cosmos configs. ### Testing None yet, will test in e2e	11 months ago
Daniel Savu	8e44bc1b81	feat: sealevel merkle indexer + e2e (#3053 ) Depends on https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/3048 Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3046 Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2754	11 months ago
Daniel Savu	77aa58c581	Relayer balance metrics (#2976 ) Done: - scaffolding for fetching custom agent metrics - abstractions for building a metrics fetcher for a given VM - querying cosmos balances; e2e tested. - querying evm balances; e2e tested. - Note that as a result, evm addresses are now no longer zero-padded when printed in the logs. This may break existing log queries - fixed a nasty bug on ubuntu where wasmd (osmosisd dependency, part of the grpc query flow) would panic when a block is specified via `x-cosmos-block-height`. The fix was to bump the version of osmosisd from `19.0.0` to `20.5.0`. Note that when running e2e on Mac OS, the osmosis version in use is still 19.0.0. That's because we need a fork that publishes a darwin target binary (currently pointing [here](https://github.com/hashableric/osmosis/releases/download/v19.0.0-mnts/osmosisd-19.0.0-mnts-darwin-arm64.tar.gz)) For follow up PR: - sealevel balance querying I'm open to all renaming suggestions, I just tried to speed through and didn't ponder names too much Relates to https://github.com/hyperlane-xyz/issues/issues/701 Closes https://github.com/hyperlane-xyz/issues/issues/702 (because the balance becomes available in the metrics endpoint for polling)	12 months ago
Trevor Porter	1cebc39182	Use Polygon gas oracle (#2965 ) ### Description After investigating #2959, I found the following this is a known problem with Polygon, explanation here https://github.com/ethers-io/ethers.js/issues/2828#issuecomment-1283014250 Fun fact Asa looked into this once https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/771 Here's a discussion it in Foundry, which I found hoping that ethers-rs folks had ran into this before https://github.com/foundry-rs/foundry/issues/1703 Foundry fixed this by using the Polygon gas station oracle which seems to be recommended path https://github.com/foundry-rs/foundry/pull/3368/files#diff-c89a4bbf7a90da118dcf00c5fe70eba78f8e5d95662bb5f039a353113e95042bR205 There's actually a polygon ethers middleware for this https://docs.rs/ethers/latest/ethers/middleware/gas_oracle/polygon/struct.Polygon.html So I (originally) borrowed this code from Foundry https://github.com/foundry-rs/foundry/blob/master/crates/common/src/provider.rs#L254-L290 Changed to use Middlewares This also means we can remove our existing janky Polygon logic ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	1 year ago
Trevor Porter	ece2be507a	Cosmos / CosmWasm agents (#2865 ) ### Description <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: Mattie Conover <git@mconover.dev> Co-authored-by: hashableric <hashableric@gmail.com> Co-authored-by: byeongsu-hong <hong@byeongsu.dev> Co-authored-by: Daniel Savu <23065004+daniel-savu@users.noreply.github.com> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz>	1 year ago
Nam Chu Hoai	e4eed2ae3d	fix: e2e working with aggregation 2/2 of `message_id` and `merkle_root` (#2861 ) ### Description - Removed the double mapping logic from the the `merkle_tree_builder.get_proof()` for fixing the following issue: > `get_proof` was expecting a nonce which it then uses to retrieve the `leaf_index` i.e. nonce -> message_id -> leaf_index and then `prove_against_previous` but in our case, we already got the leaf_index in merkle_tree_multisig.rs so what we end up doing is trying to look up the leaf_index twice and either not finding the message_id or proving wrong leaf_index which both causes the "cannot fetch metadata" for the merkle tree builder ### Drive-by changes - additional logging for aggregation to indicate which ism (moduleType, address) we are missing - trace -> debug logging for `sign_and_submit_checkpoint` ### Related issues - closes https://github.com/hyperlane-xyz/issues/issues/695 ### Backward compatibility Yes ### Testing e2e --------- Co-authored-by: -f <kunalarora1729@gmail.com> Co-authored-by: Kunal Arora <55632507+aroralanuk@users.noreply.github.com>	1 year ago
Yorke Rhodes	3501557d38	Remove legacy multisig from agents (#2839 ) Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2837	1 year ago
Trevor Porter	88346bfb64	Rm finality blocks and refactor checkpoint_submitter to be resilient to edge cases (#2836 ) ### Description * https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/532 wasn't actually fully closed out - finality_blocks was used still for indexing. Sadly testnet4 infra wasn't configuring finality blocks anymore, so we weren't indexing only finalized blocks. Moved fully to reorg_period * Refactored `checkpoint_submitter` in light of races revealed by the above ^ problem. It used to assume that message indexing and the `latest_checkpoint()` call would align with one another, and wasn't resilient to the tree already being past the correctness checkpoint. A couple situations were possible before that aren't now: a. Indexing is ahead of the latest_checkpoint() call, which will result in tree ingesting the new indexed messages and the tree being ahead of the correctness checkpoint :( b. It's possible for the tree() call that constructs the tree initially to be made against a block that's after the next latest_checkpoint() call, which would result in the tree being ahead of the correctness checkpoint from the very beginning :( ### Drive-by changes removed a function that wasn't being used anymore ### Related issues https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/532 ### Backward compatibility Removes finality blocks entirely ### Testing Builds, e2e	1 year ago
Yorke Rhodes	1d18549755	Merge main to v3 (#2812 )	1 year ago
Kunal Arora	9168cca6d2	validator deploy tweaks (#2784 ) ### Description - fixes to deploy testnet4 for agents ### Drive-by changes - truncating labels to length 63 - adding `.relayer` to `agentConfig.contextChainNames` ### Related issues - for https://github.com/hyperlane-xyz/issues/issues/574 ### Backward compatibility Yes ### Testing Manual --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com>	1 year ago
Daniel Savu	c6cea785ae	V3 agents feedback (#2780 ) ### Description Part of the remediations from https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2746 --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	1 year ago
Daniel Savu	af5bd88aa8	V3 agents rebase (#2746 ) ### Description It's your favourite PR coming right back... V3 agents! Closes https://github.com/hyperlane-xyz/issues/issues/561 Builds on top of https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2742 Depends on https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2681 for e2e testing This PR includes: - [x] Merkle tree hook indexer - [x] Merkle tree builder task - [x] Update submitter to trigger retries if no proof is available yet Slightly more detailed overview of the work here: https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2720#issuecomment-1724038643 <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: -f <kunalarora1729@gmail.com> Co-authored-by: Trevor Porter <trkporter@ucdavis.edu> Co-authored-by: Kunal Arora <55632507+aroralanuk@users.noreply.github.com> Co-authored-by: Mattie Conover <git@mconover.dev> Co-authored-by: Guillaume Bouvignies <guillaumebouvignies@gmail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz> Co-authored-by: Guillaume Bouvignies <guillaume.bouvignies@kurtosistech.com>	1 year ago
Mattie Conover	741c9bb553	Update Rust (#2765 ) ### Description Regular rust version and package updates. ### Drive-by changes None ### Related issues ### Backward compatibility Yes ### Testing Unit Tests	1 year ago
Kunal Arora	f0e4f2b89b	Prevent invalid destination griefing for the relayer (#2703 ) ### Description - Adds destinationDomain to the `GasPayment` event - recording destination domain as `destination` while reading events for the relayer ### Drive-by changes - none ### Related issues - fixes https://github.com/hyperlane-xyz/issues/issues/475 ### Backward compatibility No, change in event emitted and relayer indexing ### Testing - Unit --------- Signed-off-by: -f <kunalarora1729@gmail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz>	1 year ago
Daniel Savu	71e8988ccd	Sealevel igp indexing (#2585 ) ### Description Depends on https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2583 Indexes IGP payments related to the relayer's data pda address. Unless this address is specified in the config (`sealevel.relayer_account `), no filtering is applied and all IGP payments are stored in the local database. <!-- What's included in this PR? --> ### Drive-by changes - Sets `HYP_BASE_GASPAYMENTENFORCEMENT` in `run-locally` for the relayer, to test that it correctly indexes the IGP payment before submitting the message - A new config section (`sealevel`) is added to the relayer - The `MessageIndexer` trait is replaced with `SequenceIndexer<HyperlaneMessage>`, renaming `fetch_count_at_tip` to `sequence_at_tip`. `SequenceIndexer` is now common to both the message and igp indexers. - The `parse_addr` macro is modified so it can be reused when parsing the sealevel relayer address config too - `rust/utils/sealevel-test.bash` is included because I was using it to test locally, but I can remove it if the sealevel e2e test already does all the steps there @mattiecnvr - Performs a `try_into` conversion that can be removed once https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2610 is done ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2501 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? --> e2e tests but the pipeline is failing, likely fixed by https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2602 --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	1 year ago
Mattie Conover	389029dafb	Agent Config Base Parsing (#2665 ) ### Description This is a big one for #2215... 1. Splits the parsing of configs from the config definitions to support both the old, new, and future parsers simultaneously. (We might want to rename it from parser to config format or something; not set on this name). 2. Implements the new config format without switching to it or enabling it. This means this PR does introduce dead code that is going to be used in the next PR. ### Drive-by changes - Deleted some unused (or single use) macros defined in hyperlane base - Updated the definition of the config parsing trait to simplify implementation - Added some more ergonomic methods of handling errors in config parsing - Removed any raw `unwrap()` I noticed since these cause difficult to debug errors in prod. - Added some additional validation to catch code mistakes in config path definitions ### Related issues - Progress on #2215 - Comes after #2658 ### Backward compatibility Yes ### Testing Manual (E2E only for now to make sure I did not break the old config format)	1 year ago
Daniel Savu	326eced987	Revert "refactor(IndexMode): remove `IndexRange`… (#2562 )" (#2655 ) This reverts commit `127294decc`, to include a cursor bugfix in main and remove an IGP indexing bug added by that PR. ### Description <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	1 year ago
Mattie Conover	a5e1815746	Rust updates (#2619 ) ### Description Let this one slip a while. Updates our version of rust and as many packages as did not conflict with solana. I think we are going to need to figure something better out going forward with solana since if we have a couple more envs like this, it is going to make updates very hard. ### Drive-by changes None ### Related issues General Maintenance ### Backward compatibility Yes ### Testing Manual & Unit Tests	1 year ago
Daniel Savu	127294decc	refactor(IndexMode): remove `IndexRange`, use default index mode per env (#2562 ) ### Description Removes the `index_mode` config field, default to whichever mode is currently supported by our existing envs. Doing this allows for safely removing runtime checks to see if the correct index mode was configured. Additional work is required to support multiple indexing modes per env. Check this discord thread for the full discussion: https://discord.com/channels/935678348330434570/961711527092682783/1133347733411987568 ### Drive-by changes - `IndexRange` is removed entirely and replaced by `RangeInclusive<u32>` - Moves `is_arbitrum_nitro` into a bigger existing `impl HyperlaneDomain` block for clarity ### Related issues This is part 1 of https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2511 ### Backward compatibility Yes ### Testing Relies on existing e2e tests	1 year ago
Trevor Porter	8c5983933b	Sealevel (#2404 ) ### Description Opening for that sweet sweet agent image ### Drive-by changes _Are there any minor or drive-by changes also included?_ ### Related issues - Fixes #[issue number here] ### Backward compatibility _Are these changes backward compatible?_ Yes No _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None Yes ### Testing _What kind of testing have these changes undergone?_ None Manual Unit Tests --------- Co-authored-by: Steven Sloboda <steven@eclipse.builders> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz> Co-authored-by: J M Rossy <jm.rossy@gmail.com> Co-authored-by: Mattie Conover <git@mconover.dev> Co-authored-by: Asa Oines <asaoines@gmail.com> Co-authored-by: Sergei Patrikeev <serejke.best@gmail.com> Co-authored-by: Daniel Savu <23065004+daniel-savu@users.noreply.github.com> Co-authored-by: Kunal Arora <55632507+aroralanuk@users.noreply.github.com> Co-authored-by: yorhodes <yorkerhodesiv@gmail.com> Co-authored-by: Yorke Rhodes <yorke@useabacus.network> Co-authored-by: Yorke Rhodes <email@yorke.dev> Co-authored-by: Asa Oines <asaoines@Asas-MacBook-Pro.local> Co-authored-by: Kunal Arora <kuarora@ucsd.edu> Co-authored-by: Anett <44020788+anettrolikova@users.noreply.github.com> Co-authored-by: Yaqub Mahmoud <yaqub320@gmail.com> Co-authored-by: Alex <alex@alexbh.dev>	1 year ago
Alex	da25b06254	CCIP Read ISM (#2398 ) ### Description CCIP Read ISM implementation, - [x] Contracts - [x] Relayer - [x] Example HTTP backend ### Drive-by changes _Are there any minor or drive-by changes also included?_ ### Related issues - Fixes #[2211] ### Backward compatibility _Are these changes backward compatible?_ Yes, relayers will skip messages they can't handle. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes, relayers won't be able to submit messages that have a CCIP Read ISM. ### Testing _What kind of testing have these changes undergone?_ - [x] Manual - [x] Unit Tests ### General approach - [x] Get a basic [CCIP Read implementation working](https://github.com/AlexBHarley/permissionless-chainlink-feeds/tree/main/contracts) - [x] Build a [mock service that serves CCIP Read compatible data](https://github.com/AlexBHarley/permissionless-chainlink-feeds/tree/main/apps/api) - [x] Update the Relayer to support CCIP Read ISMs	1 year ago
Daniel Savu	6ee5868353	fix(validator): announce directly if no gas estimate (#2454 ) ### Description It has been reported that `contract_call.tx.max_cost()` can fail / not exist on some networks. When that's the case, default to assuming no additional validator gas is required and just send the tx. If the tx fails, report the failure and suggest that gas may be insufficient in the logged error. ### Drive-by changes None ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2426 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None --------- Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com>	1 year ago
Daniel Savu	7889385501	Relayer aggregation ism (#2411 ) ### Description Adds AggregationISM support to the relayer, by adding the scaffolding, sub-ism metadata fetching, and custom metadata encoding. ### Drive-by changes - adds `dry_run_verify(...)` to the `InterchainSecurityModule` trait and implements it for `EthereumInterchainSecurityModule` - changes the default ISM in the test environment from `routingIsm(multisigIsm)` to `routingIsm(aggregationIsm([multisigIsm, messageIdIsm], threshold = 1))` ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/1757 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing The AggregationISM metadata encoding is tested using unit tests whose input was taken from the tests in `AggregationIsm.t.sol`, by printing the individual ISM metadatas and the final metadata. The overall AggregationISM is tested with the `run-locally` script.	1 year ago
Daniel Savu	61892d1d5a	feat(fallback provider): re-prioritize providers on error (#2369 ) ### Description Closes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2189 Reprioritize providers in the `FallbackProvider` when they stop receiving new blocks (although a `max_block_time` has elapsed). The `JsonRpcClient::request(...)` implementation now takes a snapshot of the provider priority list and iterates on that. This is to minimize lock contention - no lock is held across an `await` so there is no deadlock risk. If one of the errors above occurs, the priority list is overwritten. Note this may cause a prior reprioritization to be lost, if the repriotization occured after the current tokio task's snapshot was taken. I'm not entirely sure whether a specific `FallbackProvider` is referenced by more than one task at a time though. ### Drive-by changes A `PrometheusJsonRpcClientConfigExt` trait is added, so the provider can be mocked. ### Testing Unit tests.	1 year ago
Yorke Rhodes	1fff74ebd8	Reduce hyper client pool idle timeout to 20s (#2384 ) ### Description - configures hyper http client to close connections after 20s - makes singleton signer retry (5) times before bubbling error up ### Backward compatibility Yes ### Testing Local binary testing against cloud resources ![Screen Shot 2023-06-13 at 5 45 47 PM](https://github.com/hyperlane-xyz/hyperlane-monorepo/assets/3020995/de47a03c-b40f-49fd-a32d-c8da6ecffa6e)	1 year ago
Yorke Rhodes	a93daec7a2	Implement single signer task in validator (#2342 ) ### Description Implements singleton signer which allows multiple tasks to use a shared signer that is signing on a single task and migrates validator to use a singleton KMS signer ### Drive-by changes Modifies base agent to allow self to be mutable in run ### Related issues - Fixes error from sharing KMS signer connection across validator tasks https://github.com/rusoto/rusoto/issues/1766 ### Backward compatibility Yes ### Testing E2E Tests --------- Co-authored-by: Mattie Conover <git@mconover.dev>	1 year ago
Mattie Conover	a66652f9f2	Reduce log noise (#2336 ) ### Description Seeing a ton of errors for process estimate costs even though they are fully being handled elsewhere and are also not really an error in the first place. This makes it harder as an operator to spot real errors so I think we should eliminate this error log. (We also already log this error elsewhere so it does not add any new information). ![image](https://github.com/hyperlane-xyz/hyperlane-monorepo/assets/4187749/f149a870-52f3-444c-aa62-84b6aa822f44) This also targets the `ret` logs produced by tracing since many of them are unhelpful and most of the cases where we actually need a log we already have one. ### Drive-by changes None ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	1 year ago
Mattie Conover	c661f7dccb	Handle additional error case in retrying provider (#2330 ) ### Description Adds a fix for this error message I noticed in the logs: ``` (code: 3, message: execution reverted: INSUFFICIENT_BALANCE, ...) ``` ### Drive-by changes None ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	1 year ago
Yorke Rhodes	2deb9b8973	Fix AWS KMS Signer connection issues (#2319 ) ### Description - Update ethers-rs fork - Create new KMS client instead of using OnceCell ### Backward compatibility Yes No ### Testing Running validator locally with replica of validator DB	2 years ago
Yorke Rhodes	50f04db1fa	Make merkle proofs optional on multisig ISM (#2173 ) ### Description Validators currently sign `(root, index)` checkpoints and during verification, a `message` is passed as calldata, an `id()` is derived, and a `proof` of `id()` at `index` in `root` is verified This provides “all or nothing” censorship resistance guarantees because a validator can only sign roots to allow any contained messages to be processed. We have considered alternatives where validators sign `message` directly and we lose censorship resistance in exchange for eliminating merkle proof verification gas costs. However, if validators sign `(root, index, message)` tuples, we can skip merkle proof verification on the destination chain while still maintaining censorship resistance by providing two valid metadata formats: 1. existing validator signatures and merkle proof verification of inclusion 2. including merkle proof verification for pathway where validators are censoring `message` It’s worth noting the validator is required to index event data to produce this new signature format. However, this does not require historical indexing and new validators being spun up can simply begin indexing from tip. See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2187 for validator changes See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2248 for relayer and e2e test changes ### Drive-by changes Merkle index also optional ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2192 ### Backward compatibility - new ISM deployment is necessary (we could upgrade implementation in theory) - Validator and relayer upgrades ### Testing Unit (fuzz) Tests, E2E tests	2 years ago
Asa Oines	63562c7211	Refactor agent event indexing (#2246 ) ### Description This PR refactors event indexing in the agents, allowing similar logic to be shared across multiple event types (i.e. messages, deliveries, gas payments) and database types (i.e. the Relayer rocks DB and Scraper SQL DB). Furthermore, it adds new syncing modes by way of `MessageSyncCursors` that take advantage of the monotonically increasing dispatched message nonce to sync more intelligently. ### Drive-by changes - Fixes a bug in the existing cursor that caused the same block range to be indexed three times - Modifies kathy to get rid of the idea of "rounds", just sends messages with a sleep in between - Minor modifications to the e2e test for performance - Expand macros in settings - Add scraper to e2e test ### Opportunities for improvement - We can further reduce RPC usage (or improve latency) by sharing the view of the latest finalized block number between cursors - We can speed up the effective time for (a relayer to start deliving messages \| the scraper to scraper recent events) by creating forward/backward cursors for gas payments and deliveries where the backwards cursor terminates at index_settings.from - We can remove the need for index_settings.from by terminating backwards cursors based on the block number that the first message was dispatched at ### Related issues - Fixes #[issue number here] ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ E2E tests --------- Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Asa Oines	f6ad1e9a95	Validator automatically self-announces (#2177 ) ### Description This PR adds the ability for validators to self-announce themselves, so long as the validator key has a large enough balance to pay for the tx. This required setting the retry provider to skip retrying reverting transactions, since with a quorum provider and a very speedy blockchain (like hardhat) the transaction is broadcast multiple times and the later transactions conflict with the first one. ### Drive-by changes None ### Related issues - Fixes #2120 ### Backward compatibility _Are these changes backward compatible?_ Yes --------- Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Mattie Conover	ed17ddc32f	Agent Argument Parsing (#2132 ) ### Description All configs can now be set by arguments as well as env and config files. The arguments will take the highest priority and overwrite the others. We should also update the docs to include this new support when it gets released, but since it is an entirely new feature it also can be a stealth release which we document after the fact. ### Drive-by changes ### Related issues - Fixes #2034 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Mattie Conover	b56b022f88	Hyperlane-Core WASM support (#2110 ) ### Description This shuffles some code around and changes dependencies to make `hyperlane-core` WASM compatible and adds a check to the pipeline to ensure it stays that way. To do this it does refactor hyperlane db to the `base` crate, which makes more sense IMO anyway as that is an implementation detail of the agents and not part of the core protocol. ### Drive-by changes ### Related issues - Fixes #1977 ### Backward compatibility _Are these changes backward compatible?_ Yes-ish - It does change where things are but it does not change support. Should only impact people who directly use the libraries. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Asa Oines	1d0e067166	Add relayer support for MultisigIsm and RoutingIsm (#2074 ) ### Description This PR adds support in the relayer for MultisigIsm (non-legacy) and RoutingIsm metadata building. AggregationIsm support is not yet implemented. ### Drive-by changes None ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ end-to-end	2 years ago
Mattie Conover	79669902de	More handling for eth_feeHistory (#2095 ) ### Description Adds additional error contexts for `eth_feeHistory` which should not be retried. Was seeing the error pop up for `celo` and `alfajores`. https://cloudlogging.app.goo.gl/SR7gV3PK5QzbnpFt8 ### Drive-by changes None ### Related issues - https://discord.com/channels/935678348330434570/1096400530722537513 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	2 years ago
Mattie Conover	1722f4ee7e	Better Agent Configuration Parsing (#2070 ) ### Description This is a significant change to how we parse user configs. The main goal of this PR is to give a clear path and env key to any errors that are encountered when parsing configuration. In order to accomplish this, I have crated a new trait `FromRawConf` which defines a set of steps to validate and convert individual fields. Also for every major config struct, there is now a `Raw` variant of it which is used by the `config` library to reduce errors that occur during deserialization itself. `Raw` types should always use the most basic form of a value, such as `String`, `StrOrInt`, `bool`, and ideally optional in all cases. Missing values should be handled during the raw conversion and not by `config`. I also needed to make changes to a number of types stored in the parsed config types to force validation forward to the parsing step instead of doing it when we read the config value. This also required me to create a filter to prevent trying to validate certain configs that we would not ever need to load. These changes can also be built on later to support something other than `config` if we choose to, or add support for merging configs from multiple sources since everything is optional. ### Drive-by changes - Default to `http` for connection type - Default to no gas payment enforcement if not specified instead of failing - `ChainSetup` -> `ChainConf` - `GasPaymentEnforcementConfig` -> `GasPaymentEnforcementConf` - Made ethereum connection configuration more forgiving - Moved hyperlane base settings from `mod.rs` to `base.rs` - Moved config chain tests in hyperlane core to `tests` dir to fix a cyclical import problem - Extension traits to help with config in hyperlane core - Moved `StrOrInt` to new hyperlane core `config` module - Support for parsing `U256` from `StrOrInt` - Removed `HexString` type which is now redundant - Updated base settings to use hyperlane domain - Use `heyKey` as signer type if `type` is not specified and `key` is - Moved hyperlane ethereum chain config to a new module ### Related issues - Fixes #2033 - Fixes #2012 ### Backward compatibility _Are these changes backward compatible?_ Yes - This should take in configs in the same shape but be more forgiving in a few places _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Test	2 years ago
Asa Oines	1edbecf9e3	Lag count in Validator agent (#2073 ) ### Description This PR adds a lag to `mailbox.count()` in the validator agent. Without this lag, validators can shut down after the first message because the latestCheckpoint call is lagged and will fail due to the lagged count still being zero. ### Drive-by changes None ### Related issues - Fixes #1764 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None --------- Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago

1 2

94 Commits (2e439423e743694226e877fe3c63d664684083d2)