12 Commits (d1bf212bf7dfa8e146950c3760fddad5fc102720)

Author	SHA1	Message	Date
Trevor Porter	18d1d0d54e	Separate agent chain configurations, re-add some chains to infra, fix eclipsetestnet & solanatestnet configs (#3327) ... ### Description - solanatestnet and eclipsetestnet were accidentally moved in the agent JSON to not be in the `chains` object. This moves them back to the right place, and adds a test in `infra` to make sure that the agent JSON chains exactly match the environment chain names. There's opportunity for future improvement for exact consistency between TS artifacts / chain metadata and what exists in the agent JSONs, this is tracked here https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/3325 - Adds back eclipsetestnet, solanatestnet, and mumbai to the list of testnet4 chains. These seem to have been accidentally removed - I guess we never deployed to mainnet3 since we started writing to aw-multisig.json. So this writes mainnet chains for the first time - Made some modest fixes to the address persistence logic: - Previously, it didn't handle AWS & GCP keys for the same role well. For non-EVM chains, we have a GCP version of a key for submitting transactions. These exist for relayers, kathy, and validators. This resulted in contention for the relayer key - sometimes the GCP key would get written, sometimes the AWS key. For validators, we'd write both the validator checkpoint signer (AWS) and the chain signer (GCP). Now we just flat out prefer the AWS key. We already only wrote the address persistence with EVM keys in mind, so this isn't a regression - Moves to very explicitly specifying which chains will be ran on the `hyperlane` context agents - Instead of sharing a list of chains for both agents and all contract management, now it's separate. This avoids us making quick fixes and removing e.g. non-EVM chains to make ethereum-specific contract tooling happy, but accidentally removing agent support for these chains - Each chain must be specified for each role in an `AgentChainConfig` indicating whether that chain is enabled. This being consistent with the environment's chain names is enforced at runtime & in a test that's ran in CI. Explicitly indicating whether a chain is supported or not and keeping this consistent will hopefully result in fewer unintentional changes to sets of configured chains - This also allows us during partner integrations to merge a PR with the contract deploy but without necessarily running agents yet. We can just set the chain as disabled to not deploy to it ### Drive-by changes - Removes the `disabled` property for chains in our infra - this is no longer a thing and isn't needed anymore ### Related issues Fixes #3326 Fixes #2572 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	9 months ago
Trevor Porter	473b43f001	Injective & inevm agents (#3155) ... ### Description - Deploys injective & inevm validators, adds them to the hyperlane context relayer, and starts scraping inevm ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz> Co-authored-by: nambrot <nambrot@googlemail.com> Co-authored-by: Daniel Savu <23065004+daniel-savu@users.noreply.github.com>	10 months ago
Daniel Savu	f2a0e92c7f	Cosmos grpc fallbackprovider (#3139) ... ### Description Implements grpc fallback provider logic for cosmos - initially tried implementing the fallback provider deprioritization logic at middleware level like in the EVM. The difference between ethers and cosmrs is that in the latter, middleware can only live at the transport layer (`tower` crate level). - based on this github [issue](https://github.com/hyperium/tonic/issues/733), that actually doesn't look possible, because the http::Request type isn't `Clone` so it can't be submitted to multiple providers - ended up implementing the fallback provider at the application layer, by keeping an array of grpc channels - There is now a `call` method in `hyperlane_core::FallbackProvider` which I'm actually really happy with. This method handles the fallbackprovider-specific logic by taking in an async closure, running it on each provider, and iterating providers if the closure call fails. In `grpc.rs` you can see how this is slightly verbose but I think it's quite manageable. The only part that bugs me is having to duplicate `Pin::from(Box::from(future))`, but that's need afaict because the regular closure returns an anonymous type - adds `grpcUrls` and `customGrpcUrls` config items - tests the cosmos fallback provider e2e ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues - Fixes: https://github.com/hyperlane-xyz/issues/issues/998 ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests -->	10 months ago
Nam Chu Hoai	68d4f2f149	Infra for neutron & neutrontestnet (#2869) ... ### Description <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	1 year ago
Kunal Arora	9168cca6d2	validator deploy tweaks (#2784) ... ### Description - fixes to deploy testnet4 for agents ### Drive-by changes - truncating labels to length 63 - adding `.relayer` to `agentConfig.contextChainNames` ### Related issues - for https://github.com/hyperlane-xyz/issues/issues/574 ### Backward compatibility Yes ### Testing Manual --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com>	1 year ago
Yorke Rhodes	a9e8b17fce	Stash deploy progress (#2783) ... ### Description - Deployer changes and artifacts from testnet4 deployment ### Drive-by changes - a single external-secrets instance lives on the cluster, and we are running testnet3 and testnet4 on the same cluster - json parsing errors were silently swallowed, moved to `mustFromJson` to prevent this ### Drive-by changes ### Backward compatibility Yes ### Testing Manual --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	1 year ago
Daniel Savu	af5bd88aa8	V3 agents rebase (#2746) ... ### Description It's your favourite PR coming right back... V3 agents! Closes https://github.com/hyperlane-xyz/issues/issues/561 Builds on top of https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2742 Depends on https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2681 for e2e testing This PR includes: - [x] Merkle tree hook indexer - [x] Merkle tree builder task - [x] Update submitter to trigger retries if no proof is available yet Slightly more detailed overview of the work here: https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2720#issuecomment-1724038643 <!-- What's included in this PR? --> ### Drive-by changes <!-- Are there any minor or drive-by changes also included? --> ### Related issues <!-- - Fixes #[issue number here] --> ### Backward compatibility <!-- Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling? Yes/No --> ### Testing <!-- What kind of testing have these changes undergone? None/Manual/Unit Tests --> --------- Co-authored-by: -f <kunalarora1729@gmail.com> Co-authored-by: Trevor Porter <trkporter@ucdavis.edu> Co-authored-by: Kunal Arora <55632507+aroralanuk@users.noreply.github.com> Co-authored-by: Mattie Conover <git@mconover.dev> Co-authored-by: Guillaume Bouvignies <guillaumebouvignies@gmail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz> Co-authored-by: Guillaume Bouvignies <guillaume.bouvignies@kurtosistech.com>	1 year ago
Mattie Conover	efad4e1326	Omniscient relayer infra (#2299) ... ### Description - Adds support for omniscient relayers - Updates config formats to allow better overideability by agent role - Enables deploying selected agents rather than having to deploy them all at once - Merges scraper scripts and configuration ### Drive-by changes Lots of cleanup was made in the infrastructure scripts and source as it was already being touched by the above. While this PR touches a lot of files, many of the changes are just because we have so many scripts and a small change to something they all use instantly can lead to many files changed. ### Related issues - Fixes hyperlane-xyz/issues#438 - Fixes hyperlane-xyz/issues#447 - Fixes #1664 ### Backward compatibility _Are these changes backward compatible?_ No _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
josedev-union	8a12608220	feat(helm): Support configurable SecretStore type (#2257) ... ### Description This PR makes the SecretStore type as configurable for all external secrets instead of using fixed `ClusterSecretStore` type. ### Drive-by changes _Are there any minor or drive-by changes also included?_ ### Related issues - Fixes #[issue number here] ### Backward compatibility _Are these changes backward compatible?_ No `externalSecrets.clusterSecretStore` should be replaced with `externalSecrets.StoreName` in the values. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Signed-off-by: josedev-union <josedev-union@users.noreply.github.com> Co-authored-by: josedev-union <josedev-union@users.noreply.github.com>	2 years ago
Mattie Conover	1c004f4b10	Remove gelato support (#1946) ... ### Description We have not been using Gelato anymore so thought it would be nice to remove this code and avoid continuing to maintain it. I know I have had to update it a few times to maintain parity. As an added perk it should slightly speed up build times since it removes an entire crate. ### Drive-by changes None ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes - As we do not use this code _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests	2 years ago
Mattie Conover	6a958f6956	Fallback testing (#1731) ... * Use httpFallback for rc * Update sha for rc * Add commit dates * Deploy RC * Deploy to testnet3 * Use fallback for testnet3	2 years ago
Asa Oines	73aa067ae2	Add testnet3 environment (#1314)	2 years ago
Asa Oines	f15632f26c	Finalize rename from abacus->hyperlane (#1312) ... * Rename abacus->hyperlane in agents * Rename in typescript * Rename abacus->hyperlane in dockerfile	2 years ago
Trevor Porter	8f9f7add72	Scraper infrastructure (#1145) ... * gitignore * wip * more wip * Mostly works * Update scraper config values * Prevent deploying to wrong context * throw error instead of exit * remove dry-run from scraper deploy * must be non-zero volume size * Add image tag for scraper deploy * Fix env config for scraper * minor cleanup * Minor cleaning * No volume Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Trevor Porter	986b531c64	ABC_ -> HYP_ (#1116)	2 years ago
Trevor Porter	ea5b584fe8	Support new Gelato relay API (#1041) ... * All compiles * Rename files fwd request -> sponsored call * More scaffolding to get the API key in there * Cleaning up * Rename task_status_call to task_status * Finish rename * rm useForDisabledOriginChains * Introduce TransactionSubmitterType * submitter type -> submission type * Pass around gelato_config instead of the sponsor api key * Final cleanup * Nit * Use default tx submission type * Some renames * Nits after some testing	2 years ago
Trevor Porter	a067a6235f	Minor fixes, deploy (#1051) ... Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com>	2 years ago
Trevor Porter	893d8efc46	QuorumProvider testing in RC (#1018) ... * Update Cargo.lock * wip * Cleanup * Deploy to all chains * Fix build, still only write http connectionType in config * Use QuorumProvider for the outbox too * Use newer img	2 years ago
Trevor Porter	75b116cd56	Add "contexts" to infrastructure tooling, this time with no circular dependency (#770) ... * Add contexts * Make validator configs and relayer configs optional * Introduce contexts * Get rid of coreConfig.agent, replace with coreConfig.agents * fix lint * Deployed, just needs some work with RPC urls * Add missing label in rust, disable undesired inboxes * Correctly disable chains, lol * Nit, rename chainNames to contextChainNames * nits * Fix getValidatorKeys * AWS flowcarbon testnet2; don't use default context * Better error when context is undefined * Add relayer funder cronjob context * Update flowcarbon testnet2 img * Fix helm parsing * whitelist * Make Contexts enum higher up * Make CheckpointSyncerType const * Move assertContext to avoid import cycle	2 years ago
Trevor Porter	84400b15c7	Revert "Add "contexts" to infrastructure tooling (#683)" (#741) ... This reverts commit e4a2811176. Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Trevor Porter	e4a2811176	Add "contexts" to infrastructure tooling (#683) ... * Add contexts * Make validator configs and relayer configs optional * Introduce contexts * Get rid of coreConfig.agent, replace with coreConfig.agents * fix lint * Deployed, just needs some work with RPC urls * Add missing label in rust, disable undesired inboxes * Correctly disable chains, lol * Nit, rename chainNames to contextChainNames * nits * Fix getValidatorKeys * AWS flowcarbon testnet2; don't use default context * Better error when context is undefined * Add relayer funder cronjob context * Update flowcarbon testnet2 img * Fix helm parsing * whitelist * Make Contexts enum higher up * Make CheckpointSyncerType const	2 years ago
webbhorn	3ec86b0e26	Configs/env vars: s/OPT_/ABC_. (#565) ... * Transition to Abacus-themed env var prefix. Previously we had lots of environment variables prefixed with OPT as a holdover from the Optics fork. This PR tries to migrate them to Abacus-specific prefix of ABC, which is something we'd like to do prior to handing operational responsibility for running agents to any third parties, since there will be some inherent resistance in persuading a group like that to rename all configuration variables. * Update image to use ABC_ instead of OPT_, rm --debug --dry-run flags from agent helm command Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	3 years ago
Trevor Porter	54a41d5a4b	More testnet2 tweaks (#571) ... * Reduce kathy frequency, add fuji * enable all kathys * Change Mumbai finality blocks 500 -> 32 * Always re-pull secrets, update mumbai to use 32 block finality blocks	3 years ago
Trevor Porter	6593e47f1a	Support agents with AWS keys (#361) ... * Add some tracing to S3 reading and writing * Add AgentAwsUser, refactor into aws dir * Some signer changes * Add AWS credentials to relayer external secret * Redeploy dev * upgrade agents * Log when indexed messages * Looks like relayer is all good * Validator workin * Needs cleanup but nearly there * Get AWS credentials when needed * cleaning * mv some common fns * Fix output env vars * Final clean (I think) * Better fix for annoying race cond * prettier * rm some tracing * Add message_count to messages indexed log	3 years ago
Trevor Porter	c7fa1272b4	Dev environment & updated infrastructure (#352) ... * First starting at adding some correct configs and messing with helm charts * Created local test docker container here * Adding back infrastructure for test * Add checkpointer to Dockerfile, a bunch of helm chart changes * use docker buildkit and correct image name in build.sh * Checkpointer deploy works * Create dev keys, add support for multiple validators when creating keys * Deploy dev * Add dev config * Deploy is working * Stepping toward a better config... * prettier * Use ChainAgentConfig * Use stricter typing, thank you Yorke * Add some trailing newlines where needed * S3 validators work * Almost entirely working * fix test env, bunch of cleanup * prettier * Clean * Add aws signers for validator statefulset * fix package-locks * Better values.yaml, rely on validators configmap * prettier * Make kathy key creation not conditional * Different local storage paths for test checkpoint syncers * Rename replicaChain * Make relayer types all mandatory * Fix output-agent-env-vars.ts * clean up * Try build cache * setup buildkit * Try GHA caching	3 years ago
Trevor Porter	fec07d21fe	Remove docs & all optics-related rust (#322) ... * optics -> abacus in infra * rm docs * rename contract-metrics prom metrics optics-* -> abacus-* * Rm watcher, updater, a bunch of optics things in abacus-base * rm agents/processor * Removing a whole lot of updater / home / replica stuff from abacus-core * Deleted a lot throughout -- some still commented out and a lot of cleanup needed * Slowly cleaning up, kathy is where the remaining errors are * Delete some commented out code * makes kathy an abacus agent * compiling woo * cargo fmt * compiling with all features * removing some update references * Ignore handles_missing_rpc_checkpoints * cargo fmt * Add signedCheckpoint.json * rm some references to updates, delete unnecessary abis * rm all references to home or replica * compiling, remove som e unused code * fix test warning * nit * New README - needs a better pass, but no optics references for now * mv rust/helm/optics-agent rust/helm/abacus-agent * Note on us not indexing updates * Get checkpointer working from local env var output	3 years ago
Trevor Porter	46bd6793c9	Move agents to external-secrets, no longer requiring a .env file (#145) ... * Use external-secrets for agents, working on dev, need to handle aws credentials and possibly others * Get AWS support working * Move toward not requiring AWS keys in a local .env * Use us-west-2 as AWS regions in config rather than requiring env vars * Pretty annoying refactor to asynchronously get some secrets from GCP that were previously env vars * Everything seems to be working * Don't pass in any secrets as helm values * Self nit * Fix * Prettier * Rm some unused dotenv	3 years ago