hyperlane-monorepo

The home for Hyperlane core contracts, sdk packages, and other infrastructure

blockchain hyperlane interchain monorepo

History

Daniel Savu dd8ac43c97 Validator task retries (#3361 ) ### Description - Makes validator tasks infallible by adding retries - This fixes a bug where certain tasks would return an error an shut down, affecting liveness. This was desired in the relayer since we don't want individual chain failures to affect the liveness of other chains. Now validator tasks either terminate or panic, and panicking will be propagated by `try_join_all`, causing the agent to shut down. - A thing to keep in mind in general is that agents will only terminate if a task panics. If it returns an `Err` but doesn't panic, the task won't be respawned. We should consider the implications of this in the scraper too. If this isn't desired, should consider using `select_all!` ### Drive-by changes - retry logic is moved from `rust/chains/hyperlane-cosmos/src/providers/rpc.rs` into `rust/hyperlane-core/src/rpc_clients/retry.rs`, so we're even closer to turning it into a retrying provider - changes several fn signatures within the validator to now return `ChainCommunicationError`s instead of `eyre::Report`s, for compatibility with the retry logic. Also makes `DbError` convertible to `ChainCommunicationError`, for the same reason. This achieves some progress on https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2878 - allows the backfill checkpoint submitter task to terminate, since `try_join_all` is tolerant of this (as described above) ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3349 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	9 months ago
..
src	Validator task retries (#3361 )	9 months ago
Cargo.toml	Cosmos block-by-block indexing (#3245 )	9 months ago

### Description

- Makes validator tasks infallible by adding retries 
- This fixes a bug where certain tasks would return an error an shut
down, affecting liveness. This was desired in the relayer since we don't
want individual chain failures to affect the liveness of other chains.
Now validator tasks either terminate or panic, and panicking will be
propagated by `try_join_all`, causing the agent to shut down.
- A thing to keep in mind in general is that agents will only terminate
if a task panics. If it returns an `Err` but doesn't panic, the task
won't be respawned. We should consider the implications of this in the
scraper too. If this isn't desired, should consider using `select_all!`

### Drive-by changes

- retry logic is moved from
`rust/chains/hyperlane-cosmos/src/providers/rpc.rs` into
`rust/hyperlane-core/src/rpc_clients/retry.rs`, so we're even closer to
turning it into a retrying provider
- changes several fn signatures within the validator to now return
`ChainCommunicationError`s instead of `eyre::Report`s, for compatibility
with the retry logic. Also makes `DbError` convertible to
`ChainCommunicationError`, for the same reason. This achieves some
progress on
https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2878
- allows the backfill checkpoint submitter task to terminate, since
`try_join_all` is tolerant of this (as described above)

### Related issues

- Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3349


### Backward compatibility

<!--
Are these changes backward compatible? Are there any infrastructure
implications, e.g. changes that would prohibit deploying older commits
using this infra tooling?

Yes/No
-->

### Testing

<!--
What kind of testing have these changes undergone?

None/Manual/Unit Tests
-->

9 months ago

src

Validator task retries (#3361 )

9 months ago

Cargo.toml

Cosmos block-by-block indexing (#3245 )

9 months ago