hyperlane-monorepo

Commit Graph

Author	SHA1	Message	Date
Trevor Porter	700240eb8a	Ancient8 new ism, misc infra fixes (#3606 ) ### Description - Deploys ancient8 with a new ISM where we are not quorum - Pulls out warp config to live in a single file / function so that deploy.ts and check-deploy.ts are in sync. Otherwise fork.sh is broken for warp routes - Shuffles around the aw multisig serialization stuff - now they're segmented by environment and context and are written to `config/{env}/aw-validators/{context}.json`. This was in response to some confusing merging logic, see https://discord.com/channels/935678348330434570/1228377321082327141/1229464197297868910 ### Drive-by changes Needed to specify pino to work in pretty mode for warp monitoring :/ this is necessary because pino doesn't work otherwise. After #3582 we should change this ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	8 months ago
Daniel Savu	f4626eb4fe	Allow protocol-specific indexing (#3544 ) ### Description - Generalizes contract syncer hashmaps to store trait objects instead of concrete structs, since both `SequencedDataContractSync` and `WatermarkContractSync` implement `ContractSyncer` - The nice thing this enables is being able to instantiate a different type of syncer for the same contract, for different chain protocols (e.g. for IGP, we can use a sequenced syncer on sealevel and cosmwasm, and a watermark syncer on EVM). - The problem is that `ContractSyncer` is implemented on different trait bounds on `SequencedDataContractSync` and `WatermarkContractSync`. Most notably, the `SequencedDataContractSync` impl bounds the indexed type by the `Sequenced` trait. This means that if a type is to be indexed by more than one syncer strategy, it needs to implement the union of bounds of those strategies - Blocker: In the IGP, this means that `InterchainGasPayment` needs to implement `Sequenced` and we have no obvious way of getting that just from the type. Even if the sequence is derived from the type, it has to be a sensible value, since indexing reliability depends on it. - Similar to the point above, we now have `contract_syncs`, a single function that instantiates contract syncs for all chains. Since `contract_syncs` must be able to build a `ContractSyncer` trait obj for all syncer types, it bounds both `T` (the indexed type) and `D` (the log store type) with a union of bounds, so it may be too strict for some types in the future. - The scraper now uses forward-backward indexing instead of just forward indexing, because that simplified the refactor - there's now a convenience trait called `TryFromWithMetrics` that allows adding this bound: `SequenceIndexer<T>: TryFromWithMetrics<ChainConf>,`. This isn't really extendable if we use non-sequence indexers, but it's fine for now since this is used everywhere - replaces the macro-based contract syncer builder with generic fns Still todo: - add new `IndexingDecorator<T>` struct that can wrap `InterchainGasPayment`, so we can use sequence-aware indexing for the IGP - remove sequence logic from the watermark cursor ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	8 months ago
Trevor Porter	26a26f4073	Agent transaction overrides, ensure agent configs are updated in CI (#3563 ) ### Description - Adds some basic transaction overrides for Ethereum chains to be used by the agents. This follows the existing config schema in chain metadata that we have been using on the TS side for a while. The overrides are as follows: - `transactionOverrides.gasPrice` - if specified, non EIP-1559 txs are used, and they use this gasPrice - `transactionOverrides.gasLimit` - if specified, this takes precedence over any gas estimation that's done and is the gas limit that's used for txs. (This is pretty niche and I don't imagine we'll ever wanna use this internally, but happens to cover the use case described [here](https://discord.com/channels/935678348330434570/984123861144600587/1224617234580766741), which is nice to cover) - `transactionOverrides.maxFeePerGas` - if specified and if the chain supports EIP 1559 txs, is used for the `maxFeePerGas` - `transactionOverrides.maxPriorityFeePerGas` - if specified and if the chain supports EIP 1559 txs, is used for the `maxPriorityFeePerGas` - Updates some of the transaction overrides in mainnet3 / testnet4 to be more reasonable now that we'll be using them to submit transactions pretty frequently - Note that atm the transaction overrides are most easily applied to the agents via the config JSONs, and not via our K8s configuration. The upshot of this is if we wanna change a transaction override right now, the easiest way to do it will be to run `update-agent-config.ts` and then build a new image. Created https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3565 to track making this experience better - Part of this change means that it's more important to keep the `transactionOverrides` specified in the SDK / infra up to date with the agent config. So I ended up doing https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3325 along the way, which also required doing https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3311. - To fix the issue with arbitrum blocks, we now require the index.from to be specified in the chain metadata if the chain's technical stack is arbitrum nitro - A new script, `update-agent-config.ts` is added, and it's ran for mainnet3 and testnet4 in CI to make sure there's no diff (similar idea to what we do with prettier) - Along the way I renamed mainnet3_config.json and testnet4_config.json to mainnet_config.json and testnet_config.json. This has been a source of confusion with ppl in the past - in v2 we named them without any numeric suffixes, so we now just do that again ### Drive-by changes - There's a drive-by to poll more frequently for pending transactions (2 seconds). The default is 7 seconds in Ethers, and for many chains that have really quick block times, this may be a way for us to speed up us learning that a transaction is included - Don't enforce gas on the inevm routes, looks like there's some weirdness there that was reported by Nam ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3562 - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3325 - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3311 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	8 months ago
Noah Bayindirli 🥂	38358ececd	feat: deprecate Mumbai testnet (#3521 ) ### Description - Removes all reference to Mumbai testnet, since it is soon being [deprecated](https://polygon.technology/blog/polygon-pos-is-cooking-the-napoli-upgrade-means-better-ux-the-mumbai-testnet-takes-a-bow) by the Polygon team - Monorepo docker tag: https://console.cloud.google.com/gcr/images/abacus-labs-dev/global/hyperlane-monorepo@sha256:d4b44359cae9c4e3bdeb6f965c59d056f7fce72c42069adc0dea038ac600785a/details?authuser=5&project=abacus-labs-dev&tab=info - Agent docker tag: https://console.cloud.google.com/gcr/images/abacus-labs-dev/global/hyperlane-agent@sha256:a578d96f7f06721d0b6cd8095d6a1fa5d22f01b59eba3320ee3b1fe34ca3c5df/details?authuser=5&project=abacus-labs-dev&tab=info ### Drive-by changes - Adds missing Testnet domains, Fuji & BSC Testnet - Fix for failing `test-rs` CI check with ``` error[E0004]: non-exhaustive patterns: `chain::KnownHyperlaneDomain::Fuji` and `chain::KnownHyperlaneDomain::BinanceSmartChainTestnet` not covered --> hyperlane-core/src/utils.rs:226:15 \| 226 \| match $v { \| ^^ patterns `chain::KnownHyperlaneDomain::Fuji` and `chain::KnownHyperlaneDomain::BinanceSmartChainTestnet` not covered ``` ### Related issues - Dependency on #3523 - Fixes #1181 ### Backward compatibility - [x] Backwards compatible - [x] Requires docker image tags to be added post-build - [x] Requires agent artifact cleanup in cloud account: https://www.notion.so/hyperlanexyz/Runbook-83c755f2652943289cf98cd1309487a8?pvs=4#77730653a12f409dacfbd427cab545d3 ### Testing * Build success * `yarn hyperlane chains list -e testnet` does not return `mumbai` * fork.sh script success locally	8 months ago
Daniel Savu	00fcaf74db	fix: warn on tx reverts; log cosmos tx results (#3497 ) ### Description <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	8 months ago
Daniel Savu	6e39cf01af	Relayer retry endpoints (#3476 ) ### Description Adds a `/message_retry` relayer endpoint, which supports retrying OpQueue operations either by ID or by destination domain. Such a request will be sent to an `mpmc` channel's receiving end in the OpQueue, which is emptied when `OpQueue::pop` is called. Example calls: ``` GET http://127.0.0.1:60843/message_retry?destination_domain=42 GET http://127.0.0.1:60843/message_retry?message_id=0x46910b1329ee53c86a023b322e9ca1c17e5f9f0bee789c77b0abced0a173d714 ``` If the endpoint is called specifying both filters, two requests will be sent across the channel, one for each condition. Efficiency note: The entire queue is iterated over if there's at least one retry request in the channel. The good thing is that rebuilding the heap is `O(n)` so not too bad. ### Drive-by changes - Removed the usage of `enum_dispatch`, because it would have made unit tests messy. The drawback is that we have to work around object safety restrictions in `PendingOperation` - for instance it can't be cloned (but can be stored behind an `Arc` if we ever need this). - Added new methods to `PendingOperation`, either to match by ID, or to implement heap element prioritization without depending on the concrete `PendingMessage` type (as was done before). Additions: `id()`, `origin_domain()`, `priority()`. Don't think this conflicts with any operations we may add in the future (e.g. gas oracle updates). - Although `tokio`'s broadcast channel is multi-producer-multi-consumer by default, the consumer end isn't `Clone` - you instead need to have a producer to call `.subscribe()` on to get a new consumer, so I added an `MpmcChannel` struct to encapsulate keeping a producer around to get new consumers. - OpQueue is moved into its own file and has unit tests added for retries (covering broadcast to multi-queue, and both retry types). - Did a bit of agent `server `cleanup, including `EigenNodeApi` ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3098 ### Backward compatibility Yes ### Testing Unit tests for the server route (channel transmitter logic) and for the OpQueue retries (channel receiver logic).	8 months ago
Daniel Savu	8ea0bde7c9	Use app context classifier in relayer submitter queues (#3385 ) ### Description Includes the `app_context` classification in `PendingMessage`, and adds trait methods on `PendingOperation` to require always having such a label on `OpQueue` operations. This is done by reusing the matching list logic from the validator checkpoint labels (https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/3057). The nice thing is that this enables later support for retrying a group of `OpQueue` operations just by specifying the `app_context` label, without adding any new logic, since these labels are essentially matching list results. One downside to using `app_context` for retries is that the endpoint caller is constrained to only the matching lists defined by the relayer operator - however imo only the relayer operator that should be able to trigger retries. ### Drive-by changes The `OpQueue` type alias is converted to an actual struct, that stores the queue label (for metrics purposes), and also the `IntGaugeVec` metric: the generic group of metrics associated with that queue (basically only `submitter_queue_length` currently). ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3240 ### Backward compatibility Yes ### Testing Manual, by spinning up a relayer for injective and inevm. Sample metrics, from `--metricAppContexts '[{"name": "injectivelabel", "matchingList": [{"destination_domain": 6909546 }] }, {"name": "inevmlabel", "matchingList": [{"destination_domain": 2525 }] }]'` ``` hyperlane_submitter_queue_length{agent="relayer",app_context="inevmlabel",hyperlane_baselib_version="0.1.0",queue_name="confirm_queue",remote="inevm"} 11 hyperlane_submitter_queue_length{agent="relayer",app_context="inevmlabel",hyperlane_baselib_version="0.1.0",queue_name="prepare_queue",remote="inevm"} 0 hyperlane_submitter_queue_length{agent="relayer",app_context="injectivelabel",hyperlane_baselib_version="0.1.0",queue_name="confirm_queue",remote="injective"} 63 hyperlane_submitter_queue_length{agent="relayer",app_context="injectivelabel",hyperlane_baselib_version="0.1.0",queue_name="prepare_queue",remote="injective"} 13281 ```	9 months ago
Paul Balaji	76bd730103	feat: remove goerli testnets (#3389 )	9 months ago
Daniel Savu	ee1f11b4e4	fix(agents): add inevm to known domains and make igp indexing log war… (#3387 ) ### Description adds inevm to known domains and makes igp indexing log warn level ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	9 months ago
Daniel Savu	dd8ac43c97	Validator task retries (#3361 ) ### Description - Makes validator tasks infallible by adding retries - This fixes a bug where certain tasks would return an error an shut down, affecting liveness. This was desired in the relayer since we don't want individual chain failures to affect the liveness of other chains. Now validator tasks either terminate or panic, and panicking will be propagated by `try_join_all`, causing the agent to shut down. - A thing to keep in mind in general is that agents will only terminate if a task panics. If it returns an `Err` but doesn't panic, the task won't be respawned. We should consider the implications of this in the scraper too. If this isn't desired, should consider using `select_all!` ### Drive-by changes - retry logic is moved from `rust/chains/hyperlane-cosmos/src/providers/rpc.rs` into `rust/hyperlane-core/src/rpc_clients/retry.rs`, so we're even closer to turning it into a retrying provider - changes several fn signatures within the validator to now return `ChainCommunicationError`s instead of `eyre::Report`s, for compatibility with the retry logic. Also makes `DbError` convertible to `ChainCommunicationError`, for the same reason. This achieves some progress on https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2878 - allows the backfill checkpoint submitter task to terminate, since `try_join_all` is tolerant of this (as described above) ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3349 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	9 months ago
Daniel Savu	e737d998ff	Cosmos block-by-block indexing (#3245 ) ### Description Implements cosmos block-by-block indexing, parallelizing where possible: the `block` and `block_results` calls, and the individual block queries within a range. Also reuses the existing `handle_txs` logic, to improve readability. ### Drive-by changes Adds `tokio::task::JoinError` to the `hyperlane-core` error enum ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3234 - Added some hacky retrying logic to `get_logs_in_block`, since we were seeing lots of `503` http errors. In https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3264 we should make sure to remove this and propagate the error instead, to retry via the cursor / indexer logic. I edited the description of 3264 to reflect this ### Backward compatibility Yes ### Testing e2e --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	9 months ago
Daniel Savu	2283f1b1d1	wip: deploy agents to plumetestnet (#3306 ) ### Description <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu> Co-authored-by: -f <kunalarora1729@gmail.com>	9 months ago
Trevor Porter	9a83eb4d57	Sequence-aware indexing refactor (#3262 ) ### Description Part 1 of an indexing refactor. This mostly focuses on the sequence-aware cursors. Changes: - Some renames, e.g. `SequenceIndexer` -> `SequenceAwareIndexer`, `sequence_and_tip` -> `latest_sequence_count_and_tip` - Moved cursors out of `contract_sync/cursor.rs` into `contract_sync/cursors/*` - The goal was to make the the Forward sequence-aware cursor, the Backward sequence-aware, and the ForwardBackward sequence aware cursors more resilient, easier to understand, and better tested. - General strategy for doing this was to: - Only move onto a new range to index if the logs passed into the cursor's `update` function justify this - If updated with any weird data, like gaps in logs, rewind the cursor and retry previously queried ranges I'd recommend starting the review in the following order: 1. `rust/hyperlane-base/src/contract_sync/cursors/sequence_aware/forward.rs` - where `ForwardSequenceAwareSyncCursor` & tests live 2. `rust/hyperlane-base/src/contract_sync/cursors/sequence_aware/backward.rs` - where `BackwardSequenceAwareSyncCursor` & tests live 3. `rust/hyperlane-base/src/contract_sync/cursors/sequence_aware/mod.rs` - where `ForwardBackwardSequenceAwareSyncCursor` lives ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	9 months ago
Daniel Savu	c037206928	remove leftover basegoerli mentions (#3290 ) ### Description The scraper is still trying to connect to basegoerli although this network is now down ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	9 months ago
Kunal Arora	13fb461888	feat:add eigen node endpoints to agent API (#3095 ) ### Description - compatibility with eigenlayer's node spec here: https://eigen.nethermind.io/docs/spec/api/ - changing from hyper to axum as a preferred choice of server handling - adding server trait to validator, relayer, and server (to customize behavior for each in the future) ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues - fixes https://github.com/hyperlane-xyz/issues/issues/853 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	9 months ago
Trevor Porter	6d4266ae93	Allow unknown domains in the scraper (#3276 ) ### Description Spotted a regression ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	10 months ago
Daniel Savu	155c85eda2	Infallible relayer tasks (#3257 ) ### Description The intended behavior of agents is described by @tkporter's comment here: https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/3257#discussion_r1489233847 ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3209 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	10 months ago
Lee	dcd78e916d	fix: use sender as bytes variables (#3252 ) ### Description Fixes the CCIP read json body to use the `sender_as_bytes` variable. This fixes the issue of truncated sender ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> Co-authored-by: Le Yu <leyu@Les-MacBook-Pro.fios-router.home> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com>	10 months ago
Daniel Savu	f11878ad08	CCIP Read metadata builder fix (#3246 ) ### Description `H160`'s `ToString` implementation adds ellipsis points instead of outputting the full hex string, for some reason (e.g. `0xc66a…7b6f`). This will result in an error when `Mailbox.process` is called on the CCIP Read ISM. ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	10 months ago
Daniel Savu	3cbb06a07a	feat: fetch chain-specific metrics in standalone task (#3214 ) ### Description Spawning a tokio task to fetch chain-specific metrics in provider middleware was wasteful because we ended up with 15 tasks doing the same job. Now there is a single task that is spawned inside the relayer, which I added in the struct that used to only fetch relayer balance. Some questions: - I'm realising in the current state I'm probably removing these metrics from validators. Lmk if yes and I'll add back - the chain-specific metrics were moved out of `MiddlewareMetrics` because they're no longer part of middleware, but I wasn't sure where to place them. Maybe `CoreMetrics` is a better place than the current `custom_metrics.rs` file? `tokio-metrics` turned out not to be useful because we'd need to instrument every call site of `tokio::spawn` with it. We should still do it but as a separate task imo. ### Drive-by changes - This PR also makes it very easy to add the metrics tasks for new chains, by extending the `HyperlaneProvider` trait with a `get_chain_metrics` call which seems reasonably general to me to have implemented by all providers. (Bc we currently only track these for evm chains) - the description, naming and logic of the `gas_price` metric was also changed to support new chains ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3047 ### Backward compatibility Yes ### Testing Still need to manually test	10 months ago
Trevor Porter	473b43f001	Injective & inevm agents (#3155 ) ### Description - Deploys injective & inevm validators, adds them to the hyperlane context relayer, and starts scraping inevm ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz> Co-authored-by: nambrot <nambrot@googlemail.com> Co-authored-by: Daniel Savu <23065004+daniel-savu@users.noreply.github.com>	10 months ago
Ivan Temchenko	65e2ea50f9	Google Cloud Storage CheckpointSyncer implementation (#3156 ) ### Description GSC builder and implementer classes; Documentation comments; Examples; Anonymous test on public bucket access; ### Drive-by changes `cargo doc` build fixes: * replaced square braces of non-existing items with `'` ; * raw URL surrounded by `<>`; ### Related issues Fixes #2242 Part 1 ### Backward compatibility Yes ### Testing Unit Tests	10 months ago
Daniel Savu	f2a0e92c7f	Cosmos grpc fallbackprovider (#3139 ) ### Description Implements grpc fallback provider logic for cosmos - initially tried implementing the fallback provider deprioritization logic at middleware level like in the EVM. The difference between ethers and cosmrs is that in the latter, middleware can only live at the transport layer (`tower` crate level). - based on this github [issue](https://github.com/hyperium/tonic/issues/733), that actually doesn't look possible, because the http::Request type isn't `Clone` so it can't be submitted to multiple providers - ended up implementing the fallback provider at the application layer, by keeping an array of grpc channels - There is now a `call` method in `hyperlane_core::FallbackProvider` which I'm actually really happy with. This method handles the fallbackprovider-specific logic by taking in an async closure, running it on each provider, and iterating providers if the closure call fails. In `grpc.rs` you can see how this is slightly verbose but I think it's quite manageable. The only part that bugs me is having to duplicate `Pin::from(Box::from(future))`, but that's need afaict because the regular closure returns an anonymous type - adds `grpcUrls` and `customGrpcUrls` config items - tests the cosmos fallback provider e2e ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues - Fixes: https://github.com/hyperlane-xyz/issues/issues/998 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	10 months ago
Huulu	3ea5dd7b66	chore(rust): proofreading (#3152 ) Came across few typos while browsing ./rust. One of them is an error message displayed to the user, rest are comments. Hope I could be of any help. Have a nice week-end Co-authored-by: Daniel Savu <23065004+daniel-savu@users.noreply.github.com>	10 months ago
Yorke Rhodes	8fe24830ee	Deploy viction core and warp routes (#3067 ) ### Description Deploys core and warp to viction ### Related issues - Fixes https://github.com/hyperlane-xyz/issues/issues/850 - Fixes https://github.com/hyperlane-xyz/issues/issues/859 ### Backward compatibility Yes ### Testing Fork tests --------- Co-authored-by: J M Rossy <jm.rossy@gmail.com> Co-authored-by: Trevor Porter <trkporter@ucdavis.edu> Co-authored-by: Daniel Savu <23065004+daniel-savu@users.noreply.github.com> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com>	10 months ago
Trevor Porter	6845f9f4b8	Fix parsing sequences of H256s in a matchinglist via agent settings (#3180 ) ### Description Upon trying to use a blacklist with an array of H256s as the `recipientAddress`, was getting a panic: ``` running 1 test parse_value: Array [Object {"destinationdomain": Number(11155111), "origindomain": Number(1399811151), "recipientaddress": Array [String("0x6AD4DEBA8A147d000C09de6465267a9047d1c217"), String("0x6AD4DEBA8A147d000C09de6465267a9047d1c218")], "senderaddress": Array [String("0x6AD4DEBA8A147d000C09de6465267a9047d1c217"), String("0x6AD4DEBA8A147d000C09de6465267a9047d1c218")]}] "Expected matching list" thread 'settings::matching_list::test::supports_sequence_h256s' panicked at 'called `Result::unwrap()` on an `Err` value: ConfigParsingError([(ConfigPath([]), Expected matching list Caused by: invalid type: string "0x6AD4DEBA8A147d000C09de6465267a9047d1c217", expected a borrowed string Location: /Users/trevor/abacus-monorepo/rust/hyperlane-base/src/settings/parser/json_value_parser.rs:259:14)])', agents/relayer/src/settings/matching_list.rs:473:60 stack backtrace: 0: rust_begin_unwind at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/panicking.rs:593:5 1: core::panicking::panic_fmt at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/panicking.rs:67:14 2: core::result::unwrap_failed at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/result.rs:1651:5 3: core::result::Result<T,E>::unwrap at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/result.rs:1076:23 4: relayer::settings::matching_list::test::supports_sequence_h256s at ./src/settings/matching_list.rs:473:9 5: relayer::settings::matching_list::test::supports_sequence_h256s::{{closure}} at ./src/settings/matching_list.rs:459:34 6: core::ops::function::FnOnce::call_once at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/ops/function.rs:250:5 7: core::ops::function::FnOnce::call_once at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/ops/function.rs:250:5 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. test settings::matching_list::test::supports_sequence_h256s ... FAILED ``` Honestly not too sure about the implications of this being a String vs &str ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	10 months ago
Trevor Porter	3f88aa6f6e	Observability of validators for relayers (#3057 ) ### Description Goal of this was to have insight into validators of important sets being "up" Introduces a new metric used by relayers: `hyperlane_observed_validator_latest_index`, e.g.: ``` hyperlane_observed_validator_latest_index{agent="relayer",app_context="default_ism",destination="test1",hyperlane_baselib_version="0.1.0",origin="test2",validator="0x9965507d1a55bcc2695c58ba16fb37d819b0a4dc"} 664 hyperlane_observed_validator_latest_index{agent="relayer",app_context="default_ism",destination="test1",hyperlane_baselib_version="0.1.0",origin="test3",validator="0x976ea74026e726554db657fa54763abd0c3a0aa9"} 641 hyperlane_observed_validator_latest_index{agent="relayer",app_context="default_ism",destination="test2",hyperlane_baselib_version="0.1.0",origin="test1",validator="0x15d34aaf54267db7d7c367839aaf71a00a2c6a65"} 670 hyperlane_observed_validator_latest_index{agent="relayer",app_context="default_ism",destination="test2",hyperlane_baselib_version="0.1.0",origin="test3",validator="0x976ea74026e726554db657fa54763abd0c3a0aa9"} 665 hyperlane_observed_validator_latest_index{agent="relayer",app_context="default_ism",destination="test3",hyperlane_baselib_version="0.1.0",origin="test1",validator="0x15d34aaf54267db7d7c367839aaf71a00a2c6a65"} 652 hyperlane_observed_validator_latest_index{agent="relayer",app_context="default_ism",destination="test3",hyperlane_baselib_version="0.1.0",origin="test2",validator="0x9965507d1a55bcc2695c58ba16fb37d819b0a4dc"} 664 hyperlane_observed_validator_latest_index{agent="relayer",app_context="testapp",destination="test1",hyperlane_baselib_version="0.1.0",origin="test2",validator="0x9965507d1a55bcc2695c58ba16fb37d819b0a4dc"} 658 hyperlane_observed_validator_latest_index{agent="relayer",app_context="testapp",destination="test1",hyperlane_baselib_version="0.1.0",origin="test3",validator="0x976ea74026e726554db657fa54763abd0c3a0aa9"} 641 ``` Tapping into metadata building for multisig ISMs, the relayer will update the metric with the latest indices for the validators in a set. In order to prevent the cardinality being ridiculously high, only certain validator sets are tracked. This is done by introducing an `app_context` label (I'm very open to other names here, for some reason whenever idk how to name some kind of identifier I end up calling it a context 😆) The app context can either be: - if a new setting, --metricAppContexts, is specified, a message will be classified based off the first matching list it matches. E.g. `--metricAppContexts '[{"name": "testapp", "matchingList": [{"recipient_address": "0xd84379ceae14aa33c123af12424a37803f885889", "destination_domain": 13371 }] }]'`. This is nice for e.g. warp route deployments, where the ISM is maybe not a default ISM, and can be changed - if a message doesn't get classified this way, it can also be classified with the "default_ism" app context, which is just for any message that happens to use the default ISM as its "root" ISM This way we have insight in to the default ISM and any application-specific ISMs. Some things to note: - it's possible for a message to actually have more than one validator set, e.g. if it's using an aggregation ISM. In this case, we'll have metrics on the union of all validator sets for that app context - Some effort is required to make sure that metrics don't stick around for a validator that has actually been removed from the set. To handle this, we cache the validator set for an app context and clear out the entire set each time we set the metrics ### Drive-by changes - Zod's nonempty function for strings is deprecated, moves to `.min(1)` instead ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/1762 ### Backward compatibility yes ### Testing Ran locally - I think i'll probably add something in e2e tests, but opening now	11 months ago
Daniel Savu	f73ee0b273	last round of PR remediations (#2791 ) Remediations for https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2746, in addition to https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2780	12 months ago
Daniel Savu	d18f7ae8e1	Cosmos gas price config (#3042 ) ### Description Adds a cosmos-specific config item for setting the minimum gas price, in the format returned by the `cosmos.base.node.v1beta1.Service/Config` cosmos-sdk grpc endpoint. ### Related issues - Fixes https://github.com/hyperlane-xyz/issues/issues/810 ### Backward compatibility No. Will break existing cosmos configs. ### Testing None yet, will test in e2e	12 months ago
Daniel Savu	8e44bc1b81	feat: sealevel merkle indexer + e2e (#3053 ) Depends on https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/3048 Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3046 Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2754	12 months ago
Nam Chu Hoai	a888cf7ce9	Actually default to none policy (#2921 ) (#3058 ) ### Description Cherry-picks #2921 which got merged into the wrong branch	12 months ago
Daniel Savu	36de5bcc3c	chore: reduce agents log verbosity, reword (#3026 ) There are way too many `No message found in DB for leaf index` logs to have them at debug level. Also the wording makes it seem like an error when it's not.	1 year ago
Daniel Savu	77aa58c581	Relayer balance metrics (#2976 ) Done: - scaffolding for fetching custom agent metrics - abstractions for building a metrics fetcher for a given VM - querying cosmos balances; e2e tested. - querying evm balances; e2e tested. - Note that as a result, evm addresses are now no longer zero-padded when printed in the logs. This may break existing log queries - fixed a nasty bug on ubuntu where wasmd (osmosisd dependency, part of the grpc query flow) would panic when a block is specified via `x-cosmos-block-height`. The fix was to bump the version of osmosisd from `19.0.0` to `20.5.0`. Note that when running e2e on Mac OS, the osmosis version in use is still 19.0.0. That's because we need a fork that publishes a darwin target binary (currently pointing [here](https://github.com/hashableric/osmosis/releases/download/v19.0.0-mnts/osmosisd-19.0.0-mnts-darwin-arm64.tar.gz)) For follow up PR: - sealevel balance querying I'm open to all renaming suggestions, I just tried to speed through and didn't ponder names too much Relates to https://github.com/hyperlane-xyz/issues/issues/701 Closes https://github.com/hyperlane-xyz/issues/issues/702 (because the balance becomes available in the metrics endpoint for polling)	1 year ago
Trevor Porter	ece2be507a	Cosmos / CosmWasm agents (#2865 ) ### Description <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: Mattie Conover <git@mconover.dev> Co-authored-by: hashableric <hashableric@gmail.com> Co-authored-by: byeongsu-hong <hong@byeongsu.dev> Co-authored-by: Daniel Savu <23065004+daniel-savu@users.noreply.github.com> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz>	1 year ago
Nam Chu Hoai	87796e2ac0	Set a default gas policy (#2830 ) ### Description When no gas enforcement policy is set, by default, we should apply None --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	1 year ago
Nam Chu Hoai	e4eed2ae3d	fix: e2e working with aggregation 2/2 of `message_id` and `merkle_root` (#2861 ) ### Description - Removed the double mapping logic from the the `merkle_tree_builder.get_proof()` for fixing the following issue: > `get_proof` was expecting a nonce which it then uses to retrieve the `leaf_index` i.e. nonce -> message_id -> leaf_index and then `prove_against_previous` but in our case, we already got the leaf_index in merkle_tree_multisig.rs so what we end up doing is trying to look up the leaf_index twice and either not finding the message_id or proving wrong leaf_index which both causes the "cannot fetch metadata" for the merkle tree builder ### Drive-by changes - additional logging for aggregation to indicate which ism (moduleType, address) we are missing - trace -> debug logging for `sign_and_submit_checkpoint` ### Related issues - closes https://github.com/hyperlane-xyz/issues/issues/695 ### Backward compatibility Yes ### Testing e2e --------- Co-authored-by: -f <kunalarora1729@gmail.com> Co-authored-by: Kunal Arora <55632507+aroralanuk@users.noreply.github.com>	1 year ago
Yorke Rhodes	136e7c8d32	Fix validator checkpoint syncer latest index (#2858 ) Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2856	1 year ago
Kunal Arora	c44fd29914	Deploy contracts to mainnet3 (#2826 ) ### Description - Deploys mainnet3 (and removes mainnet2) ### Drive-by changes - Changes protocol fees to 0 - Changes owners to deployer key (temporarily) ### Related issues - closes https://github.com/hyperlane-xyz/issues/issues/655 ### Testing Some v2 related tests broken --------- Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com>	1 year ago
Nam Chu Hoai	f05c50c92e	Less noisy logs (#2863 ) ### Description Hides some cosmos logs from debug level ### Drive-by changes Only shows submission logs if there is queue in the checkpoint submitter	1 year ago
Yorke Rhodes	3501557d38	Remove legacy multisig from agents (#2839 ) Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2837	1 year ago
Trevor Porter	88346bfb64	Rm finality blocks and refactor checkpoint_submitter to be resilient to edge cases (#2836 ) ### Description * https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/532 wasn't actually fully closed out - finality_blocks was used still for indexing. Sadly testnet4 infra wasn't configuring finality blocks anymore, so we weren't indexing only finalized blocks. Moved fully to reorg_period * Refactored `checkpoint_submitter` in light of races revealed by the above ^ problem. It used to assume that message indexing and the `latest_checkpoint()` call would align with one another, and wasn't resilient to the tree already being past the correctness checkpoint. A couple situations were possible before that aren't now: a. Indexing is ahead of the latest_checkpoint() call, which will result in tree ingesting the new indexed messages and the tree being ahead of the correctness checkpoint :( b. It's possible for the tree() call that constructs the tree initially to be made against a block that's after the next latest_checkpoint() call, which would result in the tree being ahead of the correctness checkpoint from the very beginning :( ### Drive-by changes removed a function that wasn't being used anymore ### Related issues https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/532 ### Backward compatibility Removes finality blocks entirely ### Testing Builds, e2e	1 year ago
Nam Chu Hoai	746026aa40	Fix MerkleRootMultisigIsm Metadata builder (#2831 ) ### Description Somehow missed that in https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2784/files#diff-c74afa5055013fc40d4626107d9d086be0f21e87907784532f71e8bfe2e1b02bR38	1 year ago
Yorke Rhodes	1d18549755	Merge main to v3 (#2812 )	1 year ago
Kunal Arora	9168cca6d2	validator deploy tweaks (#2784 ) ### Description - fixes to deploy testnet4 for agents ### Drive-by changes - truncating labels to length 63 - adding `.relayer` to `agentConfig.contextChainNames` ### Related issues - for https://github.com/hyperlane-xyz/issues/issues/574 ### Backward compatibility Yes ### Testing Manual --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com>	1 year ago
Daniel Savu	c6cea785ae	V3 agents feedback (#2780 ) ### Description Part of the remediations from https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2746 --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	1 year ago
Kunal Arora	35c176fe90	Testnet4 deployment (#2781 ) fixes https://github.com/hyperlane-xyz/issues/issues/574	1 year ago
Daniel Savu	af5bd88aa8	V3 agents rebase (#2746 ) ### Description It's your favourite PR coming right back... V3 agents! Closes https://github.com/hyperlane-xyz/issues/issues/561 Builds on top of https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2742 Depends on https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2681 for e2e testing This PR includes: - [x] Merkle tree hook indexer - [x] Merkle tree builder task - [x] Update submitter to trigger retries if no proof is available yet Slightly more detailed overview of the work here: https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2720#issuecomment-1724038643 <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: -f <kunalarora1729@gmail.com> Co-authored-by: Trevor Porter <trkporter@ucdavis.edu> Co-authored-by: Kunal Arora <55632507+aroralanuk@users.noreply.github.com> Co-authored-by: Mattie Conover <git@mconover.dev> Co-authored-by: Guillaume Bouvignies <guillaumebouvignies@gmail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz> Co-authored-by: Guillaume Bouvignies <guillaume.bouvignies@kurtosistech.com>	1 year ago
Trevor Porter	3f3306c35e	Helloworld program and tooling (#2744 ) ### Description * HelloWorld program added to `rust/sealevel/programs/helloworld` * Some changes to the Sealevel tooling * Refactored the old warp-route-specific deploy management into a more generic framework for idempotently deploying and managing router apps in Sealevel * Added helloworld deploy tooling * Added ISM & owner checking to this router deployment tooling * Added `--require-tx-approval` to prevent txs from being called without first prompting * Added a bunch of new commands for creating txs with certain instructions that were needed along the way -- e.g. setting an ISM, deploying a specific multisig ISM (previously only one would be deployed as a part of `core deploy` * Added a command for configuring multisig ISMs, `multisig-ism-message-id configure`, that takes in a JSON file of multisig ISM configs and applies them onchain * A bit of cleanup / refactor - e.g. removed some old commands like `mailbox receive` * Added foreignDeployments into `RouterApp` * Because RouterApp takes in `contractsMap: HyperlaneContractsMap<Factories>,`, which require attached contracts, a new `readonly foreignDeployments: ChainMap<Address> = {},` is added to the constructor * These foreignDeployments are considered in the return value of `remoteChains(chainName: string)`, but not in `chains()` -- this means that `chains()` now concretely means "chains that can be deployed to / interacted with and that there is an entry in `contractsMap` for, and `remoteChains(chainName: string)` returns any and all remote chains, regardless of whether they can be deployed to / interacted with * Added complete ISM support to the HyperlaneRouterChecker * when checking the ISM, if there's not a match and the ISM is a config, then the ISM will be deployed * Also added RouterViolation, before it'd just throw if there was a violation * Updated the Helloworld, IGP, and core tooling to work when AEE deployments are also configured * Moved to Routing ISM -> Aggregation ISM -> Merkle / Message ID multisig setup Some things to note: * atm there are a few places that have a TODO saying to remove something after some multisig txs are executed, I plan to revisit these after we get some sigs * I've deployed the mainnet sealevel version of helloworld, but haven't been able to enroll it in the EVM chains yet. Waiting for some multisig activity here ### Drive-by changes ### Related issues #2502 ### Backward compatibility I believe it should all be backward compatible ### Testing Deployed, ran checkers, etc	1 year ago
Nam Chu Hoai	a82a8148cb	Cap Msg backoff (#2763 ) ### Description Caps the msg backoff instead of going to insanely large durations ### Backward compatibility Yes ### Testing None	1 year ago
Mattie Conover	741c9bb553	Update Rust (#2765 ) ### Description Regular rust version and package updates. ### Drive-by changes None ### Related issues ### Backward compatibility Yes ### Testing Unit Tests	1 year ago

1 2 3 4 5 ...

297 Commits (2e439423e743694226e877fe3c63d664684083d2)