388 Commits (47f4f10f14f86b423f9d2242c21bb66f008e4293)

Author	SHA1	Message	Date
Trevor Porter	75b116cd56	Add "contexts" to infrastructure tooling, this time with no circular dependency (#770) ... * Add contexts * Make validator configs and relayer configs optional * Introduce contexts * Get rid of coreConfig.agent, replace with coreConfig.agents * fix lint * Deployed, just needs some work with RPC urls * Add missing label in rust, disable undesired inboxes * Correctly disable chains, lol * Nit, rename chainNames to contextChainNames * nits * Fix getValidatorKeys * AWS flowcarbon testnet2; don't use default context * Better error when context is undefined * Add relayer funder cronjob context * Update flowcarbon testnet2 img * Fix helm parsing * whitelist * Make Contexts enum higher up * Make CheckpointSyncerType const * Move assertContext to avoid import cycle	2 years ago
webbhorn	401b6e2de9	run-locally: Run 'yarn prettier' after hardhat. (#750) ... Currently the process of starting/running our local hardhat setup results in an annoying irrelevant diff created against a test.json config file. The diff is due to lack of a file-terminating newline character. A better version of this change would probably fix the typescript script that touches / generates that file, but I'm completely unfamiliar with that setup (and TS in general) so opting for this hack for now.	2 years ago
webbhorn	274efcd34d	Restore reverted PR to .gitignore run-locally-induced diffs. (#746) ... This reverts commit 023eacd44b. Also introduce two minor changes: * End rust/.gitignore with a newline. * End test.json with newline, otherwise run-locally removes it, introducing an annoying diff.	2 years ago
Trevor Porter	84400b15c7	Revert "Add "contexts" to infrastructure tooling (#683)" (#741) ... This reverts commit e4a2811176. Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
webbhorn	c819982db4	Reduce log severity for already-sent messages. (#735) ... It is expected for the 'processor' module to churn through the entire history of sent messages on relayer init, and it is expected/normal behavior to discover that the vast majority of those have already been sent. Therefore, it is cluttering / inappropriate to log those events at info! severity, and instead preferable to record them at debug severity. Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Trevor Porter	e4a2811176	Add "contexts" to infrastructure tooling (#683) ... * Add contexts * Make validator configs and relayer configs optional * Introduce contexts * Get rid of coreConfig.agent, replace with coreConfig.agents * fix lint * Deployed, just needs some work with RPC urls * Add missing label in rust, disable undesired inboxes * Correctly disable chains, lol * Nit, rename chainNames to contextChainNames * nits * Fix getValidatorKeys * AWS flowcarbon testnet2; don't use default context * Better error when context is undefined * Add relayer funder cronjob context * Update flowcarbon testnet2 img * Fix helm parsing * whitelist * Make Contexts enum higher up * Make CheckpointSyncerType const	2 years ago
Mattie Conover	f1357469a1	reduce "indexed block heights" log level (#719)	2 years ago
Trevor Porter	bf0b5b9a47	Re-add abacus/context label (#716)	2 years ago
webbhorn	87400211a5	Check delivery status on inbox before attempting delivery (#712) ... Restores functionality accidentally lost in #637 wherein we proactively check inbox message_status() call before attempting delivery of a message. * Check whether message has already been delivered before attempting delivery * s/deliver/process	2 years ago
Mattie Conover	8d0509a4d2	Operational cleanup (#707) ... * Reduce log level for common message * retry more of the errors and downgrade log level for non-stop cases * cleanup * reduce error logs	2 years ago
Mattie Conover	ef1ec1c24e	Update metric labels (#697)	2 years ago
webbhorn	8b79a14b51	Set message_delivered phase correctly in serial_submitter (#695) ... Update two metrics upon successful delivery of message: * counter reflecting number of messages successfully delivered during process lifetime * gauge of highest leaf index of any message delivered to this inbox so far during process lifetime Also updates run-locally integration test to verify these metric values upon test termination.	2 years ago
J M Rossy	80ffce0e3a	Fix path to SDK consts in infra core deploy script (#690) ... * Update working dirs for yarn ops * Fix path to SDK consts dir * Run test deploy * Update db path from cargo run-locally * Run prettier * Add outbox addr to kathy log * Fix rust fmt lint Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Mattie Conover	023eacd44b	Revert "stop checking in test configs since they are generated automatically (#685)" (#687) ... This reverts commit 5ec2d82740.	2 years ago
Mattie Conover	5ec2d82740	stop checking in test configs since they are generated automatically (#685)	2 years ago
Mattie Conover	0644e4b22d	E2E testing workflow (#679) ... * Work on new e2e script * new e2e mimics old * Support CI mode * Kathy CI mode * remove old script * e2e workflow * Update readme * fix workflow * split build from run * change build target * Use lighter-weight http client * fix typo * better handling for stopping children * Fix url * Support multiple filters * cache node modules * cache yarn build * ci cleanup * Cleanup and updates * requested changes	2 years ago
webbhorn	dd63f70da1	Gelato Plumbing (#637) ... This PR prepares for the introduction of Gelato ForwardRequest API usage. However, it only refactors current code and introduces a prerequisite interface. In particular, the existing MessageProcessor was previously responsible both for message discovery via AbacusDB, as well as managing the submission of that message to the destination chain via some RPC provider. Since we want to have the option to deliver those messages over Gelato, a natural place to split things up might be between the AbacusDB message discovery phase, and the message delivery phase. This CL formalizes that notion by moving the submission logic into a SerialSubmitter submodule. The MessageProcessor and SerialSubmitter only communicate via a channel now. A subsequent PR will introduce GelatoSubmitter, which will route messages from the MessageProcessor to the Gelato API according to the new configuration options introduced in this PR.	2 years ago
Trevor Porter	954d4bc866	Remote checkpointer infra (#673) ... * Remote checkpointer infra * rm more values * fix	2 years ago
Nam Chu Hoai	c80426383c	Only pass non-zero reorg periods as lag to latest_checkpoint (#676)	2 years ago
Trevor Porter	38cdae0f22	Some more correct finality numbers (#671) ... * More correct finality numbers * change all configs	2 years ago
Trevor Porter	ba8cdd04e0	Add destination domain to InterchainGasPaymaster.payGasFor (#668) ... * Add _destinationDomain param to payGasFor * update abis * Better comment Co-authored-by: Yorke Rhodes <yorke@useabacus.network>	2 years ago
Yorke Rhodes	ed99b22316	Add `InterchainGasPaymaster` back to APIs (#663) ... * Revert typescript changes from #558 * Add 0x0 addresses for IGP to environment artifacts * Update test addresses with IGP * Require explicit Router external initialize * Improve connection client SDK tools * Make IGP upgradeable and proxy in core deploy	2 years ago
Mattie Conover	3064991639	Better Retrying provider (#654) ... * WIP * Update retrying provider to only retry on connection issues * use better base retry time logic * cleanup * don't retry serialization errors * simplify logs * better error messages * Update loop logic * fix lint	2 years ago
Mattie Conover	02f32b61d0	Use current index for validator latest processed gauge init (#646)	2 years ago
Mattie Conover	de20710466	Rust updates (#644) ... * Update config and trim down macro code * Update rust * Package updates * Updates and cleanup * make clippy happy	2 years ago
Mattie Conover	fbf7130313	Provider request reduction (#638) ... * add contract function from call data to metric and no longer use histogram * fix warning * Get function names for metrics * Fix relayer metrics fetch interval * Fix validator metrics updates * initialize the latest processed checkpoint * Fix clippy errors	2 years ago
Mattie Conover	a7a14f3156	Fix configs (#639) ... * fix configs * print outbox addr * print outbox addr * Have deterministic contract addresses for deployments * Fix prettier * Write to rust config folder directly * Revert "print outbox addr" This reverts commit f4a6385079. * Revert "print outbox addr" This reverts commit ad9e5fb587. Co-authored-by: nambrot <nambrot@googlemail.com>	2 years ago
Nam Chu Hoai	8c9c38918e	Do not create inbox contracts if validator agent (#635)	2 years ago
Mattie Conover	1f7beb3b9c	Better Abigen (#633) ... * Abigen tools * Generate abis with buildscript * Generate abis with buildscript for middleware * Simplify code * update lock * Fix pipeline order, must run check first new to generate files	2 years ago
Trevor Porter	5e6ab10e63	Mainnet agent deploy (#614) ... * Stash * Trim fat * Fix lint * Add rust configs * Add reorg periods for mainnet * testnet2 reorg periods * Update deploy.ts Co-authored-by: yorhodes <yorke@useabacus.network>	2 years ago
Mattie Conover	aa92873088	Oneline Backtraces (#627) ... * add backtrace lib * Pull over code for printing backtraces * Oneline backtraces * code shuffle * fmt * remove stale comment * cleanup * refactoring	2 years ago
webbhorn	efe80ee8d7	Add a comment explaining an agent config "rune". (#619) ... I couldn't find any comment explaining how the correspondence between environment variable names and JSON config file hierarchy works. I think you just have to figure this out by asking someone or internalizing the decl_settings! implementation. This is really surprising behavior to me, and people might benefit from a comment in the settings mod.rs file explaining what's going on at a high level.	2 years ago
webbhorn	018c73206b	Gelato: ForwardRequest and TaskStatus calls, and EIP-712 signing for ForwardRequests (#511) ... Goal of this PR is to add support for posting ForwardRequest messages to Gelato and obtaining the status of those requests post-submission-to-Gelato. The key abstractions introduced in this PR are: ForwardRequestCall (in fwd_req_call.rs): Logic for making a single, individual RPC to the Gelato endpoint responsible for accepting the initial forward request submission, which it then enqueues and handles asynchronously. fwd_req_sig.rs, which contains an impl of the ethers::types::transaction::eip712::Eip712 trait for ForwardRequestOpArgs. This allows an ethers::signers::Signer to sign our message before we submit it to the relevant Gelato endpoint. TaskStatusCall in task_status_call.rs: Analogous to ForwardRequestCall in that it encapsulates the logic and state required to make an individual call to a Gelato endpoint to get the status of a previously-submitted request. Chain in chains.rs: Represents chains that are relevant for Abacus and/or supported by Gelato. Handles conversions between the chain_id integer representation expected by Gelato and other types. Also provides the per-chain addresses we need when signing gelato requests for supported chains.	2 years ago
Mattie Conover	f566bb40c8	Validator Metrics (#609) ... * minor refactoring * Add metrics to validator	2 years ago
Mattie Conover	4ec3e2124a	Even more metrics (#595) ... * Allow Raw messages to be parsed without cloning * Add hardcoded mapping between domain and chain * Remove extra clone * Report dst chain for `dispatch` phase of `last_known_message_leaf_index` * Update state call for outbox * Add outbox state metric	2 years ago
webbhorn	3ec86b0e26	Configs/env vars: s/OPT_/ABC_. (#565) ... * Transition to Abacus-themed env var prefix. Previously we had lots of environment variables prefixed with OPT as a holdover from the Optics fork. This PR tries to migrate them to Abacus-specific prefix of ABC, which is something we'd like to do prior to handing operational responsibility for running agents to any third parties, since there will be some inherent resistance in persuading a group like that to rename all configuration variables. * Update image to use ABC_ instead of OPT_, rm --debug --dry-run flags from agent helm command Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	2 years ago
webbhorn	630e20bbc2	validator: Ensure non-empty outbox on init. (#586) ... validator: Ensure non-empty outbox on init. This addresses the issue described in #575 wherein validators crash after new contract deployment due to failed calls on outbox.latest_checkpoint(). The calls fail due to u256 underflow on count()-1, since count() is zero on init, and latest_checkpoint() wants to return an **index**, which is count() - 1. This PR adds a retry loop just prior to entering the existing main validator-submit-loop. I chose to poll outbox.count() rather than try to decode the error code or revert reason on failed calls to latest_checkpoint() so that we avoid tightly coupling this behavior to particular error codes or messages, and in fact may want to preserve exisitng behavior when calls to outbox.latest_checkpoint() fail for other reasons. Tested with a modified `rust/run-locally.sh` which asks Kathy to sleep for 5 minutes prior to sending messages. Without this change, the validator crashes. With this change, the validator does not crash, and after 5 minutes, starts signing checkpoints as designed.	2 years ago
Mattie Conover	4d47540bb8	Metrics improvements (#574) ... * add config for chain name * Add wallets to config for prometheus middleware * add igp contract name * better insert * Use our own chain names * clean conf * fix naming of chain * be more generic to say native currency rather than token * Use `Native` as token symbol	2 years ago
Mattie Conover	4d9603c868	Relayer whitelist (#570) ... * Update ABIs, compiles. Removed Inbox indexer and cached checkpoints from the Inbox * Rm checkpoint indexing test * Update InboxValidatorManager, just need to connect the pieces now * Add channels * So close * Fix InboxValidatorManager deploy * Make message processor slightly more readable * Create loop-control crate * Try to bring some sanity to message processor * Fix bug where prover_sync wasn't in line with latest_signed_checkpoint * Rm immediate message processing, clean up * rm abacus-cli * more cleanup * rename / rm some settings * TS renames / rms * Lower some processing failure logs to info * Checkpoint fetcher doesn't need the CommittedMessages * Hardcode kathy dispatching * Move to watch channel * I'm sorry clippy * nits * rm some nonce related stuff * more refactoring * cleanup * Add whitelist settings * Whitelist filtering * fix docker build * use relayed terminology * Change whitelist tuple order * naming change * move whitelist check to before fetching proof * Update span event * make messages singular * Update error * minor cleanup * Update configs in scripts * Whitelist testing and parsing fixes Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	2 years ago
Trevor Porter	54a41d5a4b	More testnet2 tweaks (#571) ... * Reduce kathy frequency, add fuji * enable all kathys * Change Mumbai finality blocks 500 -> 32 * Always re-pull secrets, update mumbai to use 32 block finality blocks	2 years ago
Trevor Porter	1793fffbeb	Testnet2 agent deployment (#568) ... * Deploy testnet2 contracts * Add testnet2 configs to rust * don't panic for pre-london chains * Don't checkpoint, retry fewer times, and configure kathy properly * clean up * cleaninngggg Co-authored-by: Asa Oines <asaoines@Asas-MacBook-Pro.local>	2 years ago
Asa Oines	0a81c910a3	Move InterchainGasPaymaster from AbacusConnectionManager to AbacusConnectionClient (#558)	2 years ago
Trevor Porter	d120a76c1e	No more cached checkpoints in agents (#550) ... * Update ABIs, compiles. Removed Inbox indexer and cached checkpoints from the Inbox * Rm checkpoint indexing test * Update InboxValidatorManager, just need to connect the pieces now * Add channels * So close * Fix InboxValidatorManager deploy * Fix bug where prover_sync wasn't in line with latest_signed_checkpoint * Rm immediate message processing, clean up * rm abacus-cli * more cleanup * rename / rm some settings * TS renames / rms * Lower some processing failure logs to info * Checkpoint fetcher doesn't need the CommittedMessages * Hardcode kathy dispatching * Move to watch channel * I'm sorry clippy * nits * rm some nonce related stuff * PR comments	3 years ago
Trevor Porter	29970fe326	Index GasPayment events in Relayer (#504) ... * Add InterchainGasPaymaster traits and ethereum structs * Add abacuse-base interchain_gas * Actually setting up indexer * Setting up DB usage * wip * Add interchain gas paymaster to AbacusAgentCore * Sync gas payments in the relayer * It runs wooooooo * Use U256 for gas payments * nits, rm InterchainGasPaymasterEvents * Add chain_name, fix metrics * More name -> chain_name rename * Correctly rename contract_name => chain_name, move InterchainGasPayment from traits to types * split up contract_sync a bit * cargo fmt * nit * clean * Add metrics * to_string => encode_hex * make clippy happy * move contract_sync/common.rs stuff into mod.rs * string interpolation for logs * some nits * mv types/interchain_gas into types/mod * Idempotent GasPayment indexing * Use new contract ABI & update rust code * Rename RandomBadRecipient -> TestSendReceiver * use nit * only index finalized blocks * Run with finalityBlocks * Add InterchainGasPaymaster to update_abis * Some PR comments	3 years ago
Trevor Porter	5050eee9a4	Use new contract ABI & update rust code (#519)	3 years ago
Mattie Conover	71ce3e285b	Fix duplicate metric labels (#518)	3 years ago
Mattie Conover	d687d3e230	More Metrics (#495) ... * metrics refactoring * metrics refactoring * Make brain hurt less by using a damn function * patch through metrics to abacus_ethereum inbox and outbox * WIP * Minor cleanup * work on time update * added label arrays * refactoring * Metrics for sending txns * Metrics middleware * cleanup * Cleanup after merge * Add metrics config and setup prom metrics with core metrics * Minor fixes to docs * Better trait building * rename * Weave metrics through code * Use core metrics instead * Removed some unused code * fmt * change feature layout * fix docker build * Start scraping metrics on an interval * fmt * fix duplicate label error * change features * fix u256 conversion fn * refactor * Support default values in config * Minor fixes * Use f64 conversion from primitive-types * remove unused code * fix const name * improve docs * better var name * remove unused code * remove unused code	3 years ago
J M Rossy	7caaa4e5c6	[Chore] Fix VSCode formatting and prettier precommit (#501) ... - Fix VSCode prettier format on save - Add lint-staged to fix prettier precommit hook - Show root folder in vscode monorepo view - Format vscode settings files	3 years ago
webbhorn	6b3822f664	Misc. agent cleanup and dead code deletion (#470) ... * Fix path to agent::Agent in comment. Otherwise `cargo doc` emits a rustdoc::broken_intra_doc_links warning. * Use automatic link in comment for rustdoc. Otherwise 'cargo doc' emits a warning. * Delete checkpointer agent. It is not used in current deployment. * Delete kms-cli. Not used in current deployment (no references). * Include "--all" in precommit cargo fmt -- --check invocation. * Delete unused and hidden src/bin programs. These appear to have been introduced over a year ago and appear to be dead code now. Unhide them from vscode workspace too -- since they were hidden, automated refactoring tools wouldn't find them until compiler failed. * Update github CODEOWNERS to current Abacus team. Drop optional authors field from Cargo.tomls. * Drop processordb from .gitignore. Processor crate has been deleted. * Delete balance-exporter crate. It is not used / no references. * Hoist abacus-cli into its own top-level crate. It is the only tool now, no reason to hide it inside of a tools directory. * Delete old relaese.sh, which only relates to Celo Optics. * Cleanup top-level rust README.md. - Architecture paragraphs outdated, delete. - Point to run-locally.sh instead of documentation. * Explain a bit more about crate deps in README. * Dockerfile fix after tools directory restructuring. We had previously hoisted abacus-cli out since it was the only remaining tool. * Fix path to agent::Agent in comment. Otherwise `cargo doc` emits a rustdoc::broken_intra_doc_links warning. * Use automatic link in coment for rustdoc. Otherwise 'cargo doc' emits a warning. * Delete checkpointer agent. It is not used in current deployment. * Delete kms-cli. Not used in current deployment (no references). * Include "--all" in precommit cargo fmt -- --check invocation. * Delete unused and hidden src/bin programs. These appear to have been introduced over a year ago and appear to be dead code now. Unhide them from vscode workspace too -- since they were hidden, automated refactoring tools wouldn't find them until compiler failed. * Update github CODEOWNERS to current Abacus team. Drop optional authors field from Cargo.tomls. * Drop processordb from .gitignore. Processor crate has been deleted. * Delete balance-exporter crate. It is not used / no references. * Hoist abacus-cli into its own top-level crate. It is the only tool now, no reason to hide it inside of a tools directory. * Delete old relaese.sh, which only relates to Celo Optics. * Cleanup top-level rust README.md. - Architecture paragraphs outdated, delete. - Point to run-locally.sh instead of documentation. * Explain a bit more about crate deps in README. * Dockerfile fix after tools directory restructuring. We had previously hoisted abacus-cli out since it was the only remaining tool.	3 years ago
webbhorn	d40b2d776c	Reliability/correctness fixes for run-locally.sh. (#480) ... * Use ephemeral sig and db dirs for agents. If agents discover an old instance of their db files, or old validator signatures (likely from a prior invocation of run-locally.sh), they will operate based on the old state, which has diverged from the hardhat state, and thus be incorret. * Log agent errors to script stdout. * Log kathy stdout in backgrounded subshell. * Readonly env vars.	3 years ago