hyperlane-monorepo

Commit Graph

Author	SHA1	Message	Date
Daniel Savu	61892d1d5a	feat(fallback provider): re-prioritize providers on error (#2369 ) ### Description Closes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2189 Reprioritize providers in the `FallbackProvider` when they stop receiving new blocks (although a `max_block_time` has elapsed). The `JsonRpcClient::request(...)` implementation now takes a snapshot of the provider priority list and iterates on that. This is to minimize lock contention - no lock is held across an `await` so there is no deadlock risk. If one of the errors above occurs, the priority list is overwritten. Note this may cause a prior reprioritization to be lost, if the repriotization occured after the current tokio task's snapshot was taken. I'm not entirely sure whether a specific `FallbackProvider` is referenced by more than one task at a time though. ### Drive-by changes A `PrometheusJsonRpcClientConfigExt` trait is added, so the provider can be mocked. ### Testing Unit tests.	1 year ago
Daniel Savu	fdc86f687c	Reconstruct relayer `OpQueue` from HyperlaneDB (#2382 ) ### Description Stores the `num_retries` field of `PendingMessage` at its own prefix in HyperlaneDB, so that `PendingMessage` can be reconstructed with the same exponential backoff in the `next_attempt_after` field and avoid making unnecesary RPC calls on restart. To use persisted data, `PendingMessage` is constructed with `from_persisted_retries` instead of the default constructor. If reading from the db fails, the new constructor just defaults to the regular constructor, so it's worth considering replacing the regular constructor with this one entirely. ### Drive-by changes A new db prefix is added (`PENDING_MESSAGE_RETRY_COUNT_FOR_MESSAGE_ID `), as well as related getter / setter methods. Using the new prefix helps avoid a migration. I do see a `processed_by_nonce` prefix, which could be migrated to store data related to `PendingMessage` in general so let me know what you think. Could be a future improvement to merge its prefix with the prefix added in this PR. ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/1902 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ A processor is run on some dummy database entries. The `SerialSubmitter`'s end of the channel is used to ensure that the incoming `PendingMessage`s have the correct backoff in `_next_attempt_after`.	1 year ago
Mattie Conover	81ec08ffdc	Fix agent release pipeline (#2428 ) ### Description Fixes the agent artifact release pipeline's failing linux build. Created the branch `mattie/release-artifacts` to actually publish a version based on released version. ### Drive-by changes None ### Related issues - Fixes #2431 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual	1 year ago
Yorke Rhodes	c12a896d50	Absorb token and helloworld submodules (#2435 ) It is very painful to update any dependencies across repos and publish new releases due to submodule juggling. Fresh clones are also broken. Removes token and helloworld submodules and merges in as regular version tracked packages to same path. Followed https://nuclearsquid.com/writings/subtree-merging-and-you/ for preserving history.	1 year ago
Yorke Rhodes	f235e4ce0f	Add solhint to helloworld and token	1 year ago
Yorke Rhodes	a147bdc620	Fix token CI	1 year ago
Yorke Rhodes	638b3f5474	Unify lint configs	1 year ago
Yorke Rhodes	d0b2a6a249	Fix sdk lint	1 year ago
Yorke Rhodes	ebaf652192	Fix builds and tests	1 year ago
Yorke Rhodes	d9cd552c62	Merge CI configs	1 year ago
Yorke Rhodes	16146f0f03	Absorb helloworld	1 year ago
Yorke Rhodes	11d9d0bf0d	Absorb token	1 year ago
Mattie Conover	0ed4e84406	Deployed scraper (#2432 ) ### Description Updated the scraper to fix a metric issue. ### Drive-by changes None ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual	1 year ago
Kunal Arora	445da4fb0d	Adding Optimism native bridge support (#2316 ) ### Description - Adds support for the native Optimism bridge (L1->L2) to send messageID to be verified by the custom ISM on the rollup. - Only the smart contract component. - Outside Mailbox and needs to be called separately. - Uses Openzeppelin's CrossChainEnabled pattern for ensuring the right cross chain sender address. ### Drive-by changes - None ### Related issues - Fixes https://github.com/hyperlane-xyz/issues/issues/379 - Opens https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2409 - Related to https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2381 ### Backward compatibility - Needs modifying the relayer to read OptimismISM's storage to wait for `receivedEmitters` mapping to get updated before retrying `process` on the mailbox. ### Testing - Unit Tests with Foundry forks and Optimism's L1-L2 messenger contracts. --------- Co-authored-by: Asa Oines <asaoines@gmail.com>	1 year ago
Nam Chu Hoai	73651d1e46	Scale down gnosis validators to 1 (#2407 ) Scale down gnosis validators to 1	1 year ago
Asa Oines	aaddba776b	Fix latest indexed block gauge (#2387 ) ### Description This PR fixes a bug that affected the latest indexed block gauge for message_dispatch. This allows the gauge to be updated to the "correct" value whether or not a getLogs query actually takes place. ### Drive-by changes None	1 year ago
Asa Oines	e317c6c85d	Sealevel chains should always use hex keys for relayers (#2402 ) ### Description This PR adds support for protocol-type awareness in infra. A new field `type` is added to chainMetadata. This field is inspected during key creation and relayer helm chart creation. Relayer keys for sealevel chains are set explicitly to always be hex keys. ### Drive-by changes None ### Related issues - https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2350 ### Backward compatibility _Are these changes backward compatible?_ Yes	1 year ago
Trevor Porter	4bf065548c	Bump to 1.3.5 (#2401 ) ### Description Releases 1.3.5, mostly to include #2392 ### Drive-by changes n/a ### Related issues - Part of #2366 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	1 year ago
Trevor Porter	6992a4cc1f	Bump to 1.3.5 (#72 ) * Bump @hyperlane-xyz/* dependencies to 1.3.5 * Bump to 1.3.5	1 year ago
Trevor Porter	a20d5f9b74	1.3.5 (#55 )	1 year ago
Daniel Savu	1921c45885	chore(relayer): improve ISM address error message (#2345 ) Closes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2090 Recent improvements from https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2195 already catch the error in that user report. This PR just adds a hint as to why fetching may have failed; in that report, the user's `fallback` function was returning an empty value which caused casting to an `address` to fail and revert, so the error occurs because the returned value is malformed. Thanks to @tkporter for walking me through the contracts to better understand the scope of https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2090.	1 year ago
Trevor Porter	186fcdab36	Existing deployment support in Router tooling, small ISM factory fixes (#2392 ) ### Description To accommodate warp routes that exist on EVM chains and other execution environments, we need the ability to enroll remote routers on EVM-based routers that weren't deployed by the router deployer. This adds a new optional `existingDeployment` property to the `RouterConfig`, which means that deployment will not be attempted to this chain, but it will be enrolled as a remote router on any contracts that are deployed to. The general flow looks something like: * Deploy the warp route contracts on all EVM chains, enrolling each other as routers * Deploy the warp route contracts on the alt execution environment chains, also configuring the existing EVM contract deployments as `existingDeployment`s so that they are enrolled as routers on the alt execution environment contracts * Now equipped with all the router addresses, any new non-EVM router addresses are configured as `existingDeployment`s, which now enrolls these routers on the EVM chains Note there's still a lot left to be desired even with this change -- e.g. much of the tooling expects the MultiProvider to include metadata (& even providers) for each and every chain. Because the MultiProvider is of course EVM only, things get a bit weird. For now, I've been able to deploy warp routes with configs relating to a sealevel chain successfully by just running a local Anvil instance and setting the RPC URL of the sealevel config to that anvil instance. We'll want to readdress this at some point, but for now I'd prefer to have a flow that just works and we can be more intentional about bigger deploy / tooling changes to support alt execution environments in the future This also includes some small changes to the ISM factory - because we werent actually waiting for the `enrollValidators` tx to succeed before trying `setThreshold`, the gas estimation of `setThreshold` would fail because the threshold of 1 exceeded the current on-chain validator set size of 0. It also sometimes resulted in nonce contention I was able to get everything in hyperlane-deploy working locally by having dependencies be local filepaths The order of operations for things iiuc looks like: 1. Merge this 2. Ship new SDK version 3. Update hyperlane-token to use this new SDK version 4. Ship new hyperlane-token version 5. Update hyperlane-deploy to use the new SDK version and the new hyperlane-token ### Drive-by changes none ### Related issues - partially https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2366 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ unit tests, ran deploy tooling	1 year ago
Yorke Rhodes	feb6d2b4e9	Release new validators internally (#2390 ) ### Description Updates hyperlane context mainnet2 and testnet3 validator image tags (to whats already deployed)	1 year ago
Asa Oines	c445869793	Deploy scraper to testnet3 (#2351 ) ### Description With tag `2deb9b8-20230602-205342`	1 year ago
Asa Oines	663115d716	Deploy mainnet scraper (#2376 ) ### Description Updates the deployment to pick up #2375	1 year ago
Nam Chu Hoai	60ce3a5f4b	Deploy testnet3 key funder with quorum (#2307 ) Deploy testnet3 key funder with quorum	1 year ago
Yorke Rhodes	497db63f5b	Prevent AWS KMS throttling in validator (#2391 ) ### Description Observed in validator ``` Request ID: Some("c0cc123e-da52-47f4-a2d3-aeddd034a2f5") Body: {"__type":"ThrottlingException","message":"You have exceeded the rate at which you may call KMS. Reduce the frequency of your calls."} ``` - Add sleep and checkpoint sync cache for rate limit ### Drive-by changes - Skip tree in checkpoint submitter span ### Backward compatibility Yes ### Testing Local testing	1 year ago
Yorke Rhodes	1fff74ebd8	Reduce hyper client pool idle timeout to 20s (#2384 ) ### Description - configures hyper http client to close connections after 20s - makes singleton signer retry (5) times before bubbling error up ### Backward compatibility Yes ### Testing Local binary testing against cloud resources ![Screen Shot 2023-06-13 at 5 45 47 PM](https://github.com/hyperlane-xyz/hyperlane-monorepo/assets/3020995/de47a03c-b40f-49fd-a32d-c8da6ecffa6e)	1 year ago
Asa Oines	b75cfc4c6b	Start scraper message_dispatch sync from high watermark (#2377 ) ### Description This PR addresses an issue in which BSC RPC providers have a very hard time providing very old transactions. This caused BSC scraping to fail, since the scraper had never actually indexed very early BSC messages (e.g. nonce 0). By starting at the latest scraped nonce - 1, the scraper will bypass this issue. ### Drive-by changes None	1 year ago
Asa Oines	06931bc75a	Fix BSC indexing start block (#2375 )	1 year ago
Yorke Rhodes	a93daec7a2	Implement single signer task in validator (#2342 ) ### Description Implements singleton signer which allows multiple tasks to use a shared signer that is signing on a single task and migrates validator to use a singleton KMS signer ### Drive-by changes Modifies base agent to allow self to be mutable in run ### Related issues - Fixes error from sharing KMS signer connection across validator tasks https://github.com/rusoto/rusoto/issues/1766 ### Backward compatibility Yes ### Testing E2E Tests --------- Co-authored-by: Mattie Conover <git@mconover.dev>	1 year ago
Yorke Rhodes	eb17a9af15	Block validator submitter on nonzero mailbox count (#2341 ) ### Description Ensures all checkpoint submitters wait for nonzero mailbox count ### Drive-by Changes Notably always spawns backfill submitter after seeing nonzero mailbox count ### Related issues Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2340 ### Backward compatibility Yes ### Testing Updated E2E Tests	1 year ago
Mattie Conover	efad4e1326	Omniscient relayer infra (#2299 ) ### Description - Adds support for omniscient relayers - Updates config formats to allow better overideability by agent role - Enables deploying selected agents rather than having to deploy them all at once - Merges scraper scripts and configuration ### Drive-by changes Lots of cleanup was made in the infrastructure scripts and source as it was already being touched by the above. While this PR touches a lot of files, many of the changes are just because we have so many scripts and a small change to something they all use instantly can lead to many files changed. ### Related issues - Fixes hyperlane-xyz/issues#438 - Fixes hyperlane-xyz/issues#447 - Fixes #1664 ### Backward compatibility _Are these changes backward compatible?_ No _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Mattie Conover	a66652f9f2	Reduce log noise (#2336 ) ### Description Seeing a ton of errors for process estimate costs even though they are fully being handled elsewhere and are also not really an error in the first place. This makes it harder as an operator to spot real errors so I think we should eliminate this error log. (We also already log this error elsewhere so it does not add any new information). ![image](https://github.com/hyperlane-xyz/hyperlane-monorepo/assets/4187749/f149a870-52f3-444c-aa62-84b6aa822f44) This also targets the `ret` logs produced by tracing since many of them are unhelpful and most of the cases where we actually need a log we already have one. ### Drive-by changes None ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	2 years ago
Mattie Conover	c661f7dccb	Handle additional error case in retrying provider (#2330 ) ### Description Adds a fix for this error message I noticed in the logs: ``` (code: 3, message: execution reverted: INSUFFICIENT_BALANCE, ...) ``` ### Drive-by changes None ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ None	2 years ago
Yorke Rhodes	a85b88fe9e	Add instructions for running agents locally against cloud resources to `rust/README` (#2322 ) ### Description Update rust README with instructions for running agents locally against cloud resources. --------- Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Yorke Rhodes	2deb9b8973	Fix AWS KMS Signer connection issues (#2319 ) ### Description - Update ethers-rs fork - Create new KMS client instead of using OnceCell ### Backward compatibility Yes No ### Testing Running validator locally with replica of validator DB	2 years ago
Asa Oines	d57ae5f628	Do not spawn validator backwards pass if count is zero (#2315 ) ### Description Fixes a bug in which the validator would panic if the mailbox was empty. ### Drive-by changes None	2 years ago
Yorke Rhodes	e4b124cdde	Fix bug in postgres message coercion (#2310 ) ### Description Goerli had a message dispatched (nonce=1964) with a destination domain that failed to be converted from i32 to u32 when being retrieved from the database. This fixes these bugs for domain ids and nonce and improves logs for this type of failure. ### Backward compatibility Scraper needs to be redeployed ### Testing Ran scraper against read replica of explorer DB	2 years ago
Asa Oines	2ace873f9e	Gracefully handle inconsistent RPC providers in ForwardMessageSyncCursor (#2302 ) ### Description RPC providers can be inconsistent. When a message has just been inserted into the mailbox, it's possible that one Mailbox.count() query returns N and a subsequent query returns N-1. ### Drive-by changes None	2 years ago
Yorke Rhodes	bb0d688311	Add origin filter to relayer msg processor (#2301 ) ### Description Ensures the relayer never attempts to process a message destined for it's own origin domain. ### Related issues - Fixes https://github.com/hyperlane-xyz/issues/issues/464 ### Backward compatibility Yes ### Testing E2E tests	2 years ago
Asa Oines	90191d43d0	Fix bug in MessageSyncCursor rewind logic (#2297 ) ### Description This PR fixes a bug in the MessageSyncCursor rewind logic that causes the cursor to get stuck. The cursor would look for the message with nonce `n`, find it, and move its `next_block` to the block `b_n` at which `n` was dispatched. It would then index starting at `b_n`, find `n` again, interpret that as a "discontinuity", and rewind back to `b_n`, getting itself stuck permanently. We should be moving to `b_n` and not e.g. `b_n + 1` because we have no guarantee that `n+1` was not also present in `b_n` but just dropped by a flaky `eth_getLogs` implementation. The fix is to only consider messages with nonce > `n` when looking for discontinuities when our target nonce is `n + 1`. ### Drive-by changes - Modifies the E2E tests to use a chunk size of 1, allowing us to reproduce this failure mode without the filtering fix. - Debug log on rewind - Comments to clarify fast forward logic	2 years ago
Mattie Conover	347f9beac8	Revert database prefix (#2294 ) ### Description Small change to remove the domain id from the prefix in the rocks database. ### Drive-by changes No ### Related issues - https://discord.com/channels/935678348330434570/1111394973439504444 ### Backward compatibility _Are these changes backward compatible?_ Yes - returns to last release _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests	2 years ago
Yorke Rhodes	8d9ee49f67	Fix env tests for new multisig variants (#2283 ) Fixes env tests failing in CI https://github.com/hyperlane-xyz/hyperlane-monorepo/actions/runs/5081007361/jobs/9128835519. Adds logging to ism factory deployer and ism deployments.	2 years ago
Yorke Rhodes	50f04db1fa	Make merkle proofs optional on multisig ISM (#2173 ) ### Description Validators currently sign `(root, index)` checkpoints and during verification, a `message` is passed as calldata, an `id()` is derived, and a `proof` of `id()` at `index` in `root` is verified This provides “all or nothing” censorship resistance guarantees because a validator can only sign roots to allow any contained messages to be processed. We have considered alternatives where validators sign `message` directly and we lose censorship resistance in exchange for eliminating merkle proof verification gas costs. However, if validators sign `(root, index, message)` tuples, we can skip merkle proof verification on the destination chain while still maintaining censorship resistance by providing two valid metadata formats: 1. existing validator signatures and merkle proof verification of inclusion 2. including merkle proof verification for pathway where validators are censoring `message` It’s worth noting the validator is required to index event data to produce this new signature format. However, this does not require historical indexing and new validators being spun up can simply begin indexing from tip. See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2187 for validator changes See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2248 for relayer and e2e test changes ### Drive-by changes Merkle index also optional ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2192 ### Backward compatibility - new ISM deployment is necessary (we could upgrade implementation in theory) - Validator and relayer upgrades ### Testing Unit (fuzz) Tests, E2E tests	2 years ago
Asa Oines	8aa7b62bea	Minor logging improvements to agents (#2271 ) ### Description Removes some unnecessary clutter, adds domain to some spans ### Drive-by changes None	2 years ago
josedev-union	8a12608220	feat(helm): Support configurable SecretStore type (#2257 ) ### Description This PR makes the SecretStore type as configurable for all external secrets instead of using fixed `ClusterSecretStore` type. ### Drive-by changes _Are there any minor or drive-by changes also included?_ ### Related issues - Fixes #[issue number here] ### Backward compatibility _Are these changes backward compatible?_ No `externalSecrets.clusterSecretStore` should be replaced with `externalSecrets.StoreName` in the values. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Signed-off-by: josedev-union <josedev-union@users.noreply.github.com> Co-authored-by: josedev-union <josedev-union@users.noreply.github.com>	2 years ago
Asa Oines	ec63ed19c5	Fix CI flakiness (#2269 ) ### Description - Don't panic if E2E test fails to fetch a metric - Run coverage commands sequentially ### Drive-by changes - Cleans up some lint errors in forge tests	2 years ago
Asa Oines	73fb967ec2	Properly store gas payment expenditure data in agent rocks db (#2268 ) ### Description This PR fixes a bug that resulted in gas payment expenditures not being properly stored in the rocks DB, making them unretrievable. ### Drive-by changes - Introduces a macro to HyperlaneRocksDB to prevent similar bugs from happening in the future	2 years ago
Asa Oines	63562c7211	Refactor agent event indexing (#2246 ) ### Description This PR refactors event indexing in the agents, allowing similar logic to be shared across multiple event types (i.e. messages, deliveries, gas payments) and database types (i.e. the Relayer rocks DB and Scraper SQL DB). Furthermore, it adds new syncing modes by way of `MessageSyncCursors` that take advantage of the monotonically increasing dispatched message nonce to sync more intelligently. ### Drive-by changes - Fixes a bug in the existing cursor that caused the same block range to be indexed three times - Modifies kathy to get rid of the idea of "rounds", just sends messages with a sleep in between - Minor modifications to the e2e test for performance - Expand macros in settings - Add scraper to e2e test ### Opportunities for improvement - We can further reduce RPC usage (or improve latency) by sharing the view of the latest finalized block number between cursors - We can speed up the effective time for (a relayer to start deliving messages \| the scraper to scraper recent events) by creating forward/backward cursors for gas payments and deliveries where the backwards cursor terminates at index_settings.from - We can remove the need for index_settings.from by terminating backwards cursors based on the block number that the first message was dispatched at ### Related issues - Fixes #[issue number here] ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ E2E tests --------- Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago

1 2 3 4 5 ...

1600 Commits (61892d1d5a541009395e975b11323eda4ed23d1a) All Branches Search

1600 Commits (61892d1d5a541009395e975b11323eda4ed23d1a)

All Branches