8 Commits (3a09f54b209e41d659777c141e94027950615f20)

Author	SHA1	Message	Date
Mattie Conover	efad4e1326	Omniscient relayer infra (#2299) ... ### Description - Adds support for omniscient relayers - Updates config formats to allow better overideability by agent role - Enables deploying selected agents rather than having to deploy them all at once - Merges scraper scripts and configuration ### Drive-by changes Lots of cleanup was made in the infrastructure scripts and source as it was already being touched by the above. While this PR touches a lot of files, many of the changes are just because we have so many scripts and a small change to something they all use instantly can lead to many files changed. ### Related issues - Fixes hyperlane-xyz/issues#438 - Fixes hyperlane-xyz/issues#447 - Fixes #1664 ### Backward compatibility _Are these changes backward compatible?_ No _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	1 year ago
josedev-union	8a12608220	feat(helm): Support configurable SecretStore type (#2257) ... ### Description This PR makes the SecretStore type as configurable for all external secrets instead of using fixed `ClusterSecretStore` type. ### Drive-by changes _Are there any minor or drive-by changes also included?_ ### Related issues - Fixes #[issue number here] ### Backward compatibility _Are these changes backward compatible?_ No `externalSecrets.clusterSecretStore` should be replaced with `externalSecrets.StoreName` in the values. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Signed-off-by: josedev-union <josedev-union@users.noreply.github.com> Co-authored-by: josedev-union <josedev-union@users.noreply.github.com>	2 years ago
Nam Chu Hoai	97c7f52801	Remove GasPaymentEnforcementPolicy::MeetsEstimatedCost (#1910) ... ### Description As discussed offline, to establish the social contract of the relayer accepting the quotes of its associated IGP on the origin chain, this PR removes the GasPaymentEnforcementPolicy that was assessing gas payments at tx submission time via off-chain oracles.	2 years ago
Mattie Conover	0049cca1ba	Support On Chain Fee Quoting (#1658) ... ### Description * New gas enforcement policy that uses the on-chain-fee-quoting values. * Change gas enforcement policies to return the gas limit that should be used to submit the transaction. * Change gelato submitter to use the gas limit provided by the gas enforcement policy. * Now indexes the gas amounts for the payments A couple of notes: * We might want to test this with the e2e test. * I took a look into the retying provider and it looks like it already does not retry on failed transaction submissions so I don't think anything needs to be done there after all. ### Drive-by changes ### Related issues - Fixes #1535 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes - changes the internal database format for message payments ### Testing _What kind of testing have these changes undergone?_ Unit Tests --------- Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	2 years ago
Trevor Porter	1f88d0dfa0	Add whitelist to gas enforcement, whitelist interchain queries (#1732) ... * Add gas payment whitelist to relayer * Add interchain query routers to the gas enforcement whitelist * update testnet3 rc image * don't really wanna mess with rc * nit comment * pr comments	2 years ago
Nam Chu Hoai	7501a865fd	Deploy Jan 25 (#1665)	2 years ago
Mattie Conover	65b10f1376	Signer changes (#1557) ... * Signer and signable ethereum signer changes move signers into chain config (chains.rs) Undo removal of traits in hyperlane-core/traits/mod.rs fix chains.rs validator submit remove signers from hyperlane core Announcements and checkpoint Remove test_output trait_builder.rs hyperlane base settings mod base settings signers base settings mod trait builder multisig Undo checkpoint syncer change for now local storage s3 storage validator metadata builder validator submit relayer run locally configs config and deployment fmt * WIP * some refactoring of signing * fmt * Null signer in config * Fix after merge	2 years ago
Asa Oines	73aa067ae2	Add testnet3 environment (#1314)	2 years ago
Asa Oines	f15632f26c	Finalize rename from abacus->hyperlane (#1312) ... * Rename abacus->hyperlane in agents * Rename in typescript * Rename abacus->hyperlane in dockerfile	2 years ago
Trevor Porter	8f9f7add72	Scraper infrastructure (#1145) ... * gitignore * wip * more wip * Mostly works * Update scraper config values * Prevent deploying to wrong context * throw error instead of exit * remove dry-run from scraper deploy * must be non-zero volume size * Add image tag for scraper deploy * Fix env config for scraper * minor cleanup * Minor cleaning * No volume Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Trevor Porter	986b531c64	ABC_ -> HYP_ (#1116)	2 years ago
Trevor Porter	7040e5c8a4	Gas enforcement policy requiring payment to meet estimated costs (#1083) ... * All compiles * Rename files fwd request -> sponsored call * More scaffolding to get the API key in there * Cleaning up * Rename task_status_call to task_status * Finish rename * rm useForDisabledOriginChains * Introduce TransactionSubmitterType * submitter type -> submission type * Pass around gelato_config instead of the sponsor api key * Final cleanup * Nit * Use default tx submission type * Wip * wip * Getting there * Getting there * Refactor of serial submitter * Some renames * cargo fmt * Some testing * Nits after some testing * A bit of a refactor of policies * Adding some chain enums * Single source of truth for abacus domain IDs * Add local test chains * Get coingecko api key via external-secrets * Move some things around, rm gelato oracle api interaction * Gonna move to abacus-core * Move chain stuff to abacus-core * nit * comment nit * Deploy tooling * MeetsEstimatedCost policy test * PR comments, still need to merge domain enums * Single AbacusDomain enum	2 years ago
Trevor Porter	75b116cd56	Add "contexts" to infrastructure tooling, this time with no circular dependency (#770) ... * Add contexts * Make validator configs and relayer configs optional * Introduce contexts * Get rid of coreConfig.agent, replace with coreConfig.agents * fix lint * Deployed, just needs some work with RPC urls * Add missing label in rust, disable undesired inboxes * Correctly disable chains, lol * Nit, rename chainNames to contextChainNames * nits * Fix getValidatorKeys * AWS flowcarbon testnet2; don't use default context * Better error when context is undefined * Add relayer funder cronjob context * Update flowcarbon testnet2 img * Fix helm parsing * whitelist * Make Contexts enum higher up * Make CheckpointSyncerType const * Move assertContext to avoid import cycle	2 years ago
Trevor Porter	84400b15c7	Revert "Add "contexts" to infrastructure tooling (#683)" (#741) ... This reverts commit e4a2811176. Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Trevor Porter	e4a2811176	Add "contexts" to infrastructure tooling (#683) ... * Add contexts * Make validator configs and relayer configs optional * Introduce contexts * Get rid of coreConfig.agent, replace with coreConfig.agents * fix lint * Deployed, just needs some work with RPC urls * Add missing label in rust, disable undesired inboxes * Correctly disable chains, lol * Nit, rename chainNames to contextChainNames * nits * Fix getValidatorKeys * AWS flowcarbon testnet2; don't use default context * Better error when context is undefined * Add relayer funder cronjob context * Update flowcarbon testnet2 img * Fix helm parsing * whitelist * Make Contexts enum higher up * Make CheckpointSyncerType const	2 years ago
webbhorn	3ec86b0e26	Configs/env vars: s/OPT_/ABC_. (#565) ... * Transition to Abacus-themed env var prefix. Previously we had lots of environment variables prefixed with OPT as a holdover from the Optics fork. This PR tries to migrate them to Abacus-specific prefix of ABC, which is something we'd like to do prior to handing operational responsibility for running agents to any third parties, since there will be some inherent resistance in persuading a group like that to rename all configuration variables. * Update image to use ABC_ instead of OPT_, rm --debug --dry-run flags from agent helm command Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	2 years ago
Trevor Porter	54a41d5a4b	More testnet2 tweaks (#571) ... * Reduce kathy frequency, add fuji * enable all kathys * Change Mumbai finality blocks 500 -> 32 * Always re-pull secrets, update mumbai to use 32 block finality blocks	2 years ago
Nam Chu Hoai	cc6c2db5d8	Testnet deploy (#369) ... * Got keys for testnet * Testnet deploy * Add rust configs * prettier * Nits, update governance config * new agent image * Rename secret reference * Have kathy also checkpoint * Only checkpoint when count > 1 * Move relayer to unique key for (source, remote) pair * prettier * Clean up get-aws-key-addresses * get GCP keys half working * let's clean this up * First steps at refactor * Migrate delete-keys over * support key rotation * Create and get key all working * update relayer external secret * Prettier * Update ethers-rs * Debugging testnet deployment (#370) * Dont fail on inbox checkpointing tx error * fix docker build * Debug * More instrumentation * Lint * Use the correct signers * Lint * prettier * suffix -> index, go back to 1 key per agent * rm relayer user * Create user and key identifiers * Move some fns to key-utils, fix gcp secret names * Correct kathy & checkpointer key names * Back to Promse.all in deploy-agents, clean * Don't require GCP keys to be created with agentconfig * fix * pure json response for get-key-addresses Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	3 years ago
Trevor Porter	6593e47f1a	Support agents with AWS keys (#361) ... * Add some tracing to S3 reading and writing * Add AgentAwsUser, refactor into aws dir * Some signer changes * Add AWS credentials to relayer external secret * Redeploy dev * upgrade agents * Log when indexed messages * Looks like relayer is all good * Validator workin * Needs cleanup but nearly there * Get AWS credentials when needed * cleaning * mv some common fns * Fix output env vars * Final clean (I think) * Better fix for annoying race cond * prettier * rm some tracing * Add message_count to messages indexed log	3 years ago
Trevor Porter	c7fa1272b4	Dev environment & updated infrastructure (#352) ... * First starting at adding some correct configs and messing with helm charts * Created local test docker container here * Adding back infrastructure for test * Add checkpointer to Dockerfile, a bunch of helm chart changes * use docker buildkit and correct image name in build.sh * Checkpointer deploy works * Create dev keys, add support for multiple validators when creating keys * Deploy dev * Add dev config * Deploy is working * Stepping toward a better config... * prettier * Use ChainAgentConfig * Use stricter typing, thank you Yorke * Add some trailing newlines where needed * S3 validators work * Almost entirely working * fix test env, bunch of cleanup * prettier * Clean * Add aws signers for validator statefulset * fix package-locks * Better values.yaml, rely on validators configmap * prettier * Make kathy key creation not conditional * Different local storage paths for test checkpoint syncers * Rename replicaChain * Make relayer types all mandatory * Fix output-agent-env-vars.ts * clean up * Try build cache * setup buildkit * Try GHA caching	3 years ago
Trevor Porter	fec07d21fe	Remove docs & all optics-related rust (#322) ... * optics -> abacus in infra * rm docs * rename contract-metrics prom metrics optics-* -> abacus-* * Rm watcher, updater, a bunch of optics things in abacus-base * rm agents/processor * Removing a whole lot of updater / home / replica stuff from abacus-core * Deleted a lot throughout -- some still commented out and a lot of cleanup needed * Slowly cleaning up, kathy is where the remaining errors are * Delete some commented out code * makes kathy an abacus agent * compiling woo * cargo fmt * compiling with all features * removing some update references * Ignore handles_missing_rpc_checkpoints * cargo fmt * Add signedCheckpoint.json * rm some references to updates, delete unnecessary abis * rm all references to home or replica * compiling, remove som e unused code * fix test warning * nit * New README - needs a better pass, but no optics references for now * mv rust/helm/optics-agent rust/helm/abacus-agent * Note on us not indexing updates * Get checkpointer working from local env var output	3 years ago
Trevor Porter	46bd6793c9	Move agents to external-secrets, no longer requiring a .env file (#145) ... * Use external-secrets for agents, working on dev, need to handle aws credentials and possibly others * Get AWS support working * Move toward not requiring AWS keys in a local .env * Use us-west-2 as AWS regions in config rather than requiring env vars * Pretty annoying refactor to asynchronously get some secrets from GCP that were previously env vars * Everything seems to be working * Don't pass in any secrets as helm values * Self nit * Fix * Prettier * Rm some unused dotenv	3 years ago