hyperlane-monorepo

The home for Hyperlane core contracts, sdk packages, and other infrastructure

blockchain hyperlane interchain monorepo

History

Daniel Savu b643fba90a feat: use multiple `mpsc` channels instead of `broadcast` for async interface (#4190 ) ### Description Follow up to https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/4180 Lowers txid broadcast channel capacity from 1M to 30k and refactors it to use a vec of `mpsc` under the hood. ### Context The heap profiler showed that each new chain takes up 100MB of memory because of the txid broadcast channel. In addition to this, the tokio broadcast channel is a ring buffer where new items simply overwrite old items, making it hard to safely lower the capacity. As such, to be able to safely lower the capacity of this channel, usage of `tokio`'s `broadcast` channel is replaced with a `Vec` of `mpsc`s, which offer an async `send` interface for backpressure. So instead of overwriting old items, senders will block on sends, which makes sense in our case as we absolutely don't want to miss items. The submitter is slower than the indexer based on empirical observation, so blocking on channel size doesn't bottleneck throughput. I noticed the max number of channel items we can reach is 29k, by running the RC relayer after an 8 day break, during which around 160k messages were submitted. Lowering the capacity to 30k should cause us to only encounter backpressure in very rare cases, and the footprint of this channel would lower from 100MB to 3MB. ### Drive-by changes The delivery indexer in the scraper is no longer passed a txid receiver, since message deliveries are not expected to occur in the same tx as message dispatches. ### Related issues - Fixes https://github.com/hyperlane-xyz/issues/issues/1293 ### Testing E2E, by adding invariants to make sure txid indexing works --------- Co-authored-by: Trevor Porter <tkporter4@gmail.com>	4 months ago
..
src	feat: use multiple `mpsc` channels instead of `broadcast` for async interface (#4190 )	4 months ago
Cargo.toml	feat: batching igp (#3870 )	6 months ago

feat: use multiple `mpsc` channels instead of `broadcast` for async interface (#4190 )

### Description

Follow up to
https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/4180

Lowers txid broadcast channel capacity from 1M to 30k and refactors it
to use a vec of `mpsc` under the hood.

### Context
The heap profiler showed that each new chain takes up 100MB of memory
because of the txid broadcast channel. In addition to this, the tokio
broadcast channel is a ring buffer where new items simply overwrite old
items, making it hard to safely lower the capacity.

As such, to be able to safely lower the capacity of this channel, usage
of `tokio`'s `broadcast` channel is replaced with a `Vec` of `mpsc`s,
which offer an async `send` interface for backpressure. So instead of
overwriting old items, senders will block on sends, which makes sense in
our case as we absolutely don't want to miss items. The submitter is
slower than the indexer based on empirical observation, so blocking on
channel size doesn't bottleneck throughput.

I noticed the max number of channel items we can reach is 29k, by
running the RC relayer after an 8 day break, during which around 160k
messages were submitted. Lowering the capacity to 30k should cause us to
only encounter backpressure in very rare cases, and the footprint of
this channel would lower from 100MB to 3MB.

### Drive-by changes

The delivery indexer in the scraper is no longer passed a txid receiver,
since message deliveries are not expected to occur in the same tx as
message dispatches.

### Related issues

- Fixes https://github.com/hyperlane-xyz/issues/issues/1293

### Testing

E2E, by adding invariants to make sure txid indexing works

---------

Co-authored-by: Trevor Porter <tkporter4@gmail.com>

4 months ago

src

feat: use multiple `mpsc` channels instead of `broadcast` for async interface (#4190 )

4 months ago

Cargo.toml

feat: batching igp (#3870 )

6 months ago