32 Commits (707db4a2793004053fc6be963672db3f16f28e6b)

Author	SHA1	Message	Date
Trevor Porter	18d1d0d54e	Separate agent chain configurations, re-add some chains to infra, fix eclipsetestnet & solanatestnet configs (#3327) ... ### Description - solanatestnet and eclipsetestnet were accidentally moved in the agent JSON to not be in the `chains` object. This moves them back to the right place, and adds a test in `infra` to make sure that the agent JSON chains exactly match the environment chain names. There's opportunity for future improvement for exact consistency between TS artifacts / chain metadata and what exists in the agent JSONs, this is tracked here https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3325 - Adds back eclipsetestnet, solanatestnet, and mumbai to the list of testnet4 chains. These seem to have been accidentally removed - I guess we never deployed to mainnet3 since we started writing to aw-multisig.json. So this writes mainnet chains for the first time - Made some modest fixes to the address persistence logic: - Previously, it didn't handle AWS & GCP keys for the same role well. For non-EVM chains, we have a GCP version of a key for submitting transactions. These exist for relayers, kathy, and validators. This resulted in contention for the relayer key - sometimes the GCP key would get written, sometimes the AWS key. For validators, we'd write both the validator checkpoint signer (AWS) and the chain signer (GCP). Now we just flat out prefer the AWS key. We already only wrote the address persistence with EVM keys in mind, so this isn't a regression - Moves to very explicitly specifying which chains will be ran on the `hyperlane` context agents - Instead of sharing a list of chains for both agents and all contract management, now it's separate. This avoids us making quick fixes and removing e.g. non-EVM chains to make ethereum-specific contract tooling happy, but accidentally removing agent support for these chains - Each chain must be specified for each role in an `AgentChainConfig` indicating whether that chain is enabled. This being consistent with the environment's chain names is enforced at runtime & in a test that's ran in CI. Explicitly indicating whether a chain is supported or not and keeping this consistent will hopefully result in fewer unintentional changes to sets of configured chains - This also allows us during partner integrations to merge a PR with the contract deploy but without necessarily running agents yet. We can just set the chain as disabled to not deploy to it ### Drive-by changes - Removes the `disabled` property for chains in our infra - this is no longer a thing and isn't needed anymore ### Related issues Fixes #3326 Fixes #2572 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	9 months ago
Daniel Savu	b03c90191a	chore: increase relayer memory in helm chart (#3297) ... ### Description Updating the helm chart so we no longer exceed the requested memory. Checked the scraper and validators too and those have enough resources. ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	9 months ago
Trevor Porter	473b43f001	Injective & inevm agents (#3155) ... ### Description - Deploys injective & inevm validators, adds them to the hyperlane context relayer, and starts scraping inevm ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz> Co-authored-by: nambrot <nambrot@googlemail.com> Co-authored-by: Daniel Savu <23065004+daniel-savu@users.noreply.github.com>	10 months ago
Daniel Savu	f2a0e92c7f	Cosmos grpc fallbackprovider (#3139) ... ### Description Implements grpc fallback provider logic for cosmos - initially tried implementing the fallback provider deprioritization logic at middleware level like in the EVM. The difference between ethers and cosmrs is that in the latter, middleware can only live at the transport layer (`tower` crate level). - based on this github [issue](https://github.com/hyperium/tonic/issues/733), that actually doesn't look possible, because the http::Request type isn't `Clone` so it can't be submitted to multiple providers - ended up implementing the fallback provider at the application layer, by keeping an array of grpc channels - There is now a `call` method in `hyperlane_core::FallbackProvider` which I'm actually really happy with. This method handles the fallbackprovider-specific logic by taking in an async closure, running it on each provider, and iterating providers if the closure call fails. In `grpc.rs` you can see how this is slightly verbose but I think it's quite manageable. The only part that bugs me is having to duplicate `Pin::from(Box::from(future))`, but that's need afaict because the regular closure returns an anonymous type - adds `grpcUrls` and `customGrpcUrls` config items - tests the cosmos fallback provider e2e ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues - Fixes: https://github.com/hyperlane-xyz/issues/issues/998 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	10 months ago
Nam Chu Hoai	68d4f2f149	Infra for neutron & neutrontestnet (#2869) ... ### Description <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	1 year ago
Trevor Porter	b71050a0d4	Fix long validator name for polygonzkevmtestnet (#2832) ... ### Description * polygonzkevmtestnet's validator name was too long because of the `controller-revision-hash` label, see https://github.com/kubernetes/kubernetes/issues/64023 * Seeks to preserve the `-validator` suffix in the name, but truncates the earlier bit - in practice this results in validator pods like `polygonzkevmtestnet-validator-hyperlane-ag-validator-0`. This felt like the most backward compatible change -- of course we could consider trying to get a name like `polygonzkevmtestnet-hyperlane-agent-validator-0` but this would break backward compatibility more dramatically because we'd need to change all the other validator pods. Imo we should just really avoid long chain names or consider a more breaking change to our infra ### Drive-by changes n/a ### Related issues n/a ### Backward compatibility Mostly ### Testing Deployed	1 year ago
Kunal Arora	9168cca6d2	validator deploy tweaks (#2784) ... ### Description - fixes to deploy testnet4 for agents ### Drive-by changes - truncating labels to length 63 - adding `.relayer` to `agentConfig.contextChainNames` ### Related issues - for https://github.com/hyperlane-xyz/issues/issues/574 ### Backward compatibility Yes ### Testing Manual --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com>	1 year ago
Yorke Rhodes	a9e8b17fce	Stash deploy progress (#2783) ... ### Description - Deployer changes and artifacts from testnet4 deployment ### Drive-by changes - a single external-secrets instance lives on the cluster, and we are running testnet3 and testnet4 on the same cluster - json parsing errors were silently swallowed, moved to `mustFromJson` to prevent this ### Drive-by changes ### Backward compatibility Yes ### Testing Manual --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	1 year ago
Daniel Savu	af5bd88aa8	V3 agents rebase (#2746) ... ### Description It's your favourite PR coming right back... V3 agents! Closes https://github.com/hyperlane-xyz/issues/issues/561 Builds on top of https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2742 Depends on https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2681 for e2e testing This PR includes: - [x] Merkle tree hook indexer - [x] Merkle tree builder task - [x] Update submitter to trigger retries if no proof is available yet Slightly more detailed overview of the work here: https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2720#issuecomment-1724038643 <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: -f <kunalarora1729@gmail.com> Co-authored-by: Trevor Porter <trkporter@ucdavis.edu> Co-authored-by: Kunal Arora <55632507+aroralanuk@users.noreply.github.com> Co-authored-by: Mattie Conover <git@mconover.dev> Co-authored-by: Guillaume Bouvignies <guillaumebouvignies@gmail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz> Co-authored-by: Guillaume Bouvignies <guillaume.bouvignies@kurtosistech.com>	1 year ago
josedev-union	76467daf3a	feat(helm): Support chain signer key as external secret (#2543) ... ### Description This PR sets chain signer keys in the external secret. related to https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2458 ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> Co-authored-by: josedev-union <josedev-union@users.noreply.github.com> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com>	1 year ago
josedev-union	0d3dd6076d	helm: Support different signer key per a chain in relayer (#2555) ... ### Description This PR supports a different signer key per a chain in relayer. ### Drive-by changes Add env var `HYP_BASE_RELAYCHAINS` ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: josedev-union <josedev-union@users.noreply.github.com>	1 year ago
Mattie Conover	644db5eee1	Remove unused tracing logic (#2639) ... ### Description When I was working on #2215 I noticed that we are not using either zipkin or jaeger at all and possibly never have. Ripping these out to reduce dead code and also simplify our package dependencies. ### Drive-by changes None ### Related issues Discovered through #2215 ### Backward compatibility Yes - we were not using these features and did not document their existence ### Testing Manual - Verified that when running deploy-agents, the generated configs do not have jaeger or zipkin enabled on `main` and then tested to make sure they still work in this branch	1 year ago
Trevor Porter	c7ad0f4433	Fix deploying validators when there's an empty validator set (#2565) ... ### Description Turns out when there aren't any validators in the set, which is the case for solanadevnet & zbctestnet on the RC context (https://github.com/hyperlane-xyz/hyperlane-monorepo/blob/main/typescript/infra/config/environments/testnet3/validators.ts#L195), the helm value `hyperlane.validator.configs` wasn't being overridden with CLI value args. So the `configs` in values.yaml, which has a length of 1 with some keys and empty values to illustrate (part of) the expected format of the config, was being used. So then in the Helm template we'd end up iterate through the configs of len 1 :( The fix is to just set the default `configs` to `[]` I also considered adding a check to just skip the chain if the validator set is 0, because there's usually no use in deploying them anyways. But we may be deploying to a chain that used to have a non-zero validator set that we now want to be 0, so we may as well just deploy it and deal with the fact that there will be some helm releases relating to a validator set of 0 🤷 ### Drive-by changes n/a ### Related issues n/a ### Backward compatibility backward compatible ### Testing deployed	1 year ago
Mattie Conover	efad4e1326	Omniscient relayer infra (#2299) ... ### Description - Adds support for omniscient relayers - Updates config formats to allow better overideability by agent role - Enables deploying selected agents rather than having to deploy them all at once - Merges scraper scripts and configuration ### Drive-by changes Lots of cleanup was made in the infrastructure scripts and source as it was already being touched by the above. While this PR touches a lot of files, many of the changes are just because we have so many scripts and a small change to something they all use instantly can lead to many files changed. ### Related issues - Fixes hyperlane-xyz/issues#438 - Fixes hyperlane-xyz/issues#447 - Fixes #1664 ### Backward compatibility _Are these changes backward compatible?_ No _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	1 year ago
josedev-union	8a12608220	feat(helm): Support configurable SecretStore type (#2257) ... ### Description This PR makes the SecretStore type as configurable for all external secrets instead of using fixed `ClusterSecretStore` type. ### Drive-by changes _Are there any minor or drive-by changes also included?_ ### Related issues - Fixes #[issue number here] ### Backward compatibility _Are these changes backward compatible?_ No `externalSecrets.clusterSecretStore` should be replaced with `externalSecrets.StoreName` in the values. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Signed-off-by: josedev-union <josedev-union@users.noreply.github.com> Co-authored-by: josedev-union <josedev-union@users.noreply.github.com>	2 years ago
Mattie Conover	1c2eb1bc07	Scraper Refresh (#1932) ... ### Description Since we need to re-scrape everything anyway, wanted to go ahead and make some improvements and bug fixes now. * Adds code to scrape gas payments from the IGP during the sync loop * Updates gas payments table to work with new gas payment format * Fixed support with scraper for new configuration format (adds configurable list of chains to scrape) * Adds a new view for messages to display all relevant information * Adds a new view for gas payments to show total amounts paid for a given message * Fixed a bug introduced by seaorm (0.10) where `last_inserted_id` switched from being the first insert id to being the last * Pulled in a fix from @serejke in #1925 * Updated seaorm to 0.11.1 * Adds support for sepolia * Improved logging output at various levels ### Drive-by changes * Updated fallback provider to still retry even if it only has one url * Adds support for postgres `numeric` types to store Wei values (supported as of seaorm 0.11) * Switches to `bytea` representations of binary values * Switched from the `chrono` library to `time` (`chrono` is outdated and uses an unpatched version of `time`) * Removed redundant timestamp from `message` table * Cleans up some unused code ### Related issues - Fixes #1925 - Fixes #1156 ### Backward compatibility _Are these changes backward compatible?_ No - this has a new database format and for this reason defines entirely new migrations. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes - I changed it to `scraper3` so that we can run it along side the old scraper. It now also lines up with the name in the database. There is a new env var to specify what chains are scraped `HYP_BASE_CHAINSTOSCRAPE`. ### Testing _What kind of testing have these changes undergone?_ Manual - Been running the scraper locally and also deployed a current version of it to testnet3 for a subset of chains.	2 years ago
Mattie Conover	1c004f4b10	Remove gelato support (#1946) ... ### Description We have not been using Gelato anymore so thought it would be nice to remove this code and avoid continuing to maintain it. I know I have had to update it a few times to maintain parity. As an added perk it should slightly speed up build times since it removes an entire crate. ### Drive-by changes None ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes - As we do not use this code _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests	2 years ago
Nam Chu Hoai	97c7f52801	Remove GasPaymentEnforcementPolicy::MeetsEstimatedCost (#1910) ... ### Description As discussed offline, to establish the social contract of the relayer accepting the quotes of its associated IGP on the origin chain, this PR removes the GasPaymentEnforcementPolicy that was assessing gas payments at tx submission time via off-chain oracles.	2 years ago
Mattie Conover	0049cca1ba	Support On Chain Fee Quoting (#1658) ... ### Description * New gas enforcement policy that uses the on-chain-fee-quoting values. * Change gas enforcement policies to return the gas limit that should be used to submit the transaction. * Change gelato submitter to use the gas limit provided by the gas enforcement policy. * Now indexes the gas amounts for the payments A couple of notes: * We might want to test this with the e2e test. * I took a look into the retying provider and it looks like it already does not retry on failed transaction submissions so I don't think anything needs to be done there after all. ### Drive-by changes ### Related issues - Fixes #1535 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes - changes the internal database format for message payments ### Testing _What kind of testing have these changes undergone?_ Unit Tests --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	2 years ago
Mattie Conover	599722a7ca	Load all configs by default (#1887) ... ### Description This removes `RUN_ENV` and `BASE_CONFIG` from the Rust config loading process and also from the helm charts and TS configs as appropriate. ### Drive-by changes - Flattens config dir structure - Removes the test config from the git repo. - Fixed a bug in the deploy script that caused the e2e pipeline to break when the config file was removed - Removes `environment` from the config files - Updates TS code to use `runEnv` for deployment - Updates TS code to type environment with `DeployEnvironment` instead of `string` ### Related issues - Relates to #1850 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual - Updated e2e test to verify these don't cause a config loading issue.	2 years ago
Trevor Porter	63db3ceeb8	Remove testnet3 relayer blacklist (#1890) ... Remove testnet3 blacklist	2 years ago
Mattie Conover	836179c9ad	Better default stack traces (#1886) ... * Update to use color-eyre by default for stack traces * Use backtrace setting to define config formatting for debugging	2 years ago
Mattie Conover	7a121a8d80	Make relayer require destination chains (#1884) ... * Make destination chains required for relayers * Update helm chart to include destination chains * Added notice for missing destination chain names	2 years ago
Mattie Conover	8105913dea	Improve agent logging (#1797)	2 years ago
Trevor Porter	1f88d0dfa0	Add whitelist to gas enforcement, whitelist interchain queries (#1732) ... * Add gas payment whitelist to relayer * Add interchain query routers to the gas enforcement whitelist * update testnet3 rc image * don't really wanna mess with rc * nit comment * pr comments	2 years ago
Mattie Conover	6a958f6956	Fallback testing (#1731) ... * Use httpFallback for rc * Update sha for rc * Add commit dates * Deploy RC * Deploy to testnet3 * Use fallback for testnet3	2 years ago
Nam Chu Hoai	7501a865fd	Deploy Jan 25 (#1665)	2 years ago
Mattie Conover	65b10f1376	Signer changes (#1557) ... * Signer and signable ethereum signer changes move signers into chain config (chains.rs) Undo removal of traits in hyperlane-core/traits/mod.rs fix chains.rs validator submit remove signers from hyperlane core Announcements and checkpoint Remove test_output trait_builder.rs hyperlane base settings mod base settings signers base settings mod trait builder multisig Undo checkpoint syncer change for now local storage s3 storage validator metadata builder validator submit relayer run locally configs config and deployment fmt * WIP * some refactoring of signing * fmt * Null signer in config * Fix after merge	2 years ago
Mattie Conover	1ff35f63e2	Domain config (#1553) ... * Domain config * implementation -> protocol	2 years ago
Asa Oines	ece89baee7	Relayer dynamically fetches signed checkpoints and merkle proofs (#1486)	2 years ago
Mattie Conover	e8ed5f88ce	Deploy scraper for testnet3 (#1374) ... * How did those get there * abacus -> hyperlane * remove deprecated domains * Deleted dead code * Update sea-orm * Simplify settings post v2 changes * Split up settings * Update contexts * Fix scraper provider address regression during merge * Updated scraper deployment configs	2 years ago
Asa Oines	73aa067ae2	Add testnet3 environment (#1314)	2 years ago