575 Commits (f5a74006ecf21fe18bdb67426d7a39f804b96a59)

Author	SHA1	Message	Date
webbhorn	dd63f70da1	Gelato Plumbing (#637) ... This PR prepares for the introduction of Gelato ForwardRequest API usage. However, it only refactors current code and introduces a prerequisite interface. In particular, the existing MessageProcessor was previously responsible both for message discovery via AbacusDB, as well as managing the submission of that message to the destination chain via some RPC provider. Since we want to have the option to deliver those messages over Gelato, a natural place to split things up might be between the AbacusDB message discovery phase, and the message delivery phase. This CL formalizes that notion by moving the submission logic into a SerialSubmitter submodule. The MessageProcessor and SerialSubmitter only communicate via a channel now. A subsequent PR will introduce GelatoSubmitter, which will route messages from the MessageProcessor to the Gelato API according to the new configuration options introduced in this PR.	2 years ago
Trevor Porter	954d4bc866	Remote checkpointer infra (#673) ... * Remote checkpointer infra * rm more values * fix	2 years ago
Nam Chu Hoai	c80426383c	Only pass non-zero reorg periods as lag to latest_checkpoint (#676)	2 years ago
Trevor Porter	38cdae0f22	Some more correct finality numbers (#671) ... * More correct finality numbers * change all configs	2 years ago
Trevor Porter	ba8cdd04e0	Add destination domain to InterchainGasPaymaster.payGasFor (#668) ... * Add _destinationDomain param to payGasFor * update abis * Better comment Co-authored-by: Yorke Rhodes <yorke@useabacus.network>	2 years ago
Yorke Rhodes	ed99b22316	Add `InterchainGasPaymaster` back to APIs (#663) ... * Revert typescript changes from #558 * Add 0x0 addresses for IGP to environment artifacts * Update test addresses with IGP * Require explicit Router external initialize * Improve connection client SDK tools * Make IGP upgradeable and proxy in core deploy	2 years ago
Mattie Conover	3064991639	Better Retrying provider (#654) ... * WIP * Update retrying provider to only retry on connection issues * use better base retry time logic * cleanup * don't retry serialization errors * simplify logs * better error messages * Update loop logic * fix lint	2 years ago
Mattie Conover	02f32b61d0	Use current index for validator latest processed gauge init (#646)	2 years ago
Mattie Conover	de20710466	Rust updates (#644) ... * Update config and trim down macro code * Update rust * Package updates * Updates and cleanup * make clippy happy	2 years ago
Mattie Conover	fbf7130313	Provider request reduction (#638) ... * add contract function from call data to metric and no longer use histogram * fix warning * Get function names for metrics * Fix relayer metrics fetch interval * Fix validator metrics updates * initialize the latest processed checkpoint * Fix clippy errors	2 years ago
Mattie Conover	a7a14f3156	Fix configs (#639) ... * fix configs * print outbox addr * print outbox addr * Have deterministic contract addresses for deployments * Fix prettier * Write to rust config folder directly * Revert "print outbox addr" This reverts commit f4a6385079. * Revert "print outbox addr" This reverts commit ad9e5fb587. Co-authored-by: nambrot <nambrot@googlemail.com>	2 years ago
Nam Chu Hoai	8c9c38918e	Do not create inbox contracts if validator agent (#635)	2 years ago
Mattie Conover	1f7beb3b9c	Better Abigen (#633) ... * Abigen tools * Generate abis with buildscript * Generate abis with buildscript for middleware * Simplify code * update lock * Fix pipeline order, must run check first new to generate files	2 years ago
Trevor Porter	5e6ab10e63	Mainnet agent deploy (#614) ... * Stash * Trim fat * Fix lint * Add rust configs * Add reorg periods for mainnet * testnet2 reorg periods * Update deploy.ts Co-authored-by: yorhodes <yorke@useabacus.network>	2 years ago
Mattie Conover	aa92873088	Oneline Backtraces (#627) ... * add backtrace lib * Pull over code for printing backtraces * Oneline backtraces * code shuffle * fmt * remove stale comment * cleanup * refactoring	2 years ago
webbhorn	efe80ee8d7	Add a comment explaining an agent config "rune". (#619) ... I couldn't find any comment explaining how the correspondence between environment variable names and JSON config file hierarchy works. I think you just have to figure this out by asking someone or internalizing the decl_settings! implementation. This is really surprising behavior to me, and people might benefit from a comment in the settings mod.rs file explaining what's going on at a high level.	2 years ago
webbhorn	018c73206b	Gelato: ForwardRequest and TaskStatus calls, and EIP-712 signing for ForwardRequests (#511) ... Goal of this PR is to add support for posting ForwardRequest messages to Gelato and obtaining the status of those requests post-submission-to-Gelato. The key abstractions introduced in this PR are: ForwardRequestCall (in fwd_req_call.rs): Logic for making a single, individual RPC to the Gelato endpoint responsible for accepting the initial forward request submission, which it then enqueues and handles asynchronously. fwd_req_sig.rs, which contains an impl of the ethers::types::transaction::eip712::Eip712 trait for ForwardRequestOpArgs. This allows an ethers::signers::Signer to sign our message before we submit it to the relevant Gelato endpoint. TaskStatusCall in task_status_call.rs: Analogous to ForwardRequestCall in that it encapsulates the logic and state required to make an individual call to a Gelato endpoint to get the status of a previously-submitted request. Chain in chains.rs: Represents chains that are relevant for Abacus and/or supported by Gelato. Handles conversions between the chain_id integer representation expected by Gelato and other types. Also provides the per-chain addresses we need when signing gelato requests for supported chains.	2 years ago
Mattie Conover	f566bb40c8	Validator Metrics (#609) ... * minor refactoring * Add metrics to validator	2 years ago
Mattie Conover	4ec3e2124a	Even more metrics (#595) ... * Allow Raw messages to be parsed without cloning * Add hardcoded mapping between domain and chain * Remove extra clone * Report dst chain for `dispatch` phase of `last_known_message_leaf_index` * Update state call for outbox * Add outbox state metric	2 years ago
webbhorn	3ec86b0e26	Configs/env vars: s/OPT_/ABC_. (#565) ... * Transition to Abacus-themed env var prefix. Previously we had lots of environment variables prefixed with OPT as a holdover from the Optics fork. This PR tries to migrate them to Abacus-specific prefix of ABC, which is something we'd like to do prior to handing operational responsibility for running agents to any third parties, since there will be some inherent resistance in persuading a group like that to rename all configuration variables. * Update image to use ABC_ instead of OPT_, rm --debug --dry-run flags from agent helm command Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	2 years ago
webbhorn	630e20bbc2	validator: Ensure non-empty outbox on init. (#586) ... validator: Ensure non-empty outbox on init. This addresses the issue described in #575 wherein validators crash after new contract deployment due to failed calls on outbox.latest_checkpoint(). The calls fail due to u256 underflow on count()-1, since count() is zero on init, and latest_checkpoint() wants to return an **index**, which is count() - 1. This PR adds a retry loop just prior to entering the existing main validator-submit-loop. I chose to poll outbox.count() rather than try to decode the error code or revert reason on failed calls to latest_checkpoint() so that we avoid tightly coupling this behavior to particular error codes or messages, and in fact may want to preserve exisitng behavior when calls to outbox.latest_checkpoint() fail for other reasons. Tested with a modified `rust/run-locally.sh` which asks Kathy to sleep for 5 minutes prior to sending messages. Without this change, the validator crashes. With this change, the validator does not crash, and after 5 minutes, starts signing checkpoints as designed.	3 years ago
Mattie Conover	4d47540bb8	Metrics improvements (#574) ... * add config for chain name * Add wallets to config for prometheus middleware * add igp contract name * better insert * Use our own chain names * clean conf * fix naming of chain * be more generic to say native currency rather than token * Use `Native` as token symbol	3 years ago
Mattie Conover	4d9603c868	Relayer whitelist (#570) ... * Update ABIs, compiles. Removed Inbox indexer and cached checkpoints from the Inbox * Rm checkpoint indexing test * Update InboxValidatorManager, just need to connect the pieces now * Add channels * So close * Fix InboxValidatorManager deploy * Make message processor slightly more readable * Create loop-control crate * Try to bring some sanity to message processor * Fix bug where prover_sync wasn't in line with latest_signed_checkpoint * Rm immediate message processing, clean up * rm abacus-cli * more cleanup * rename / rm some settings * TS renames / rms * Lower some processing failure logs to info * Checkpoint fetcher doesn't need the CommittedMessages * Hardcode kathy dispatching * Move to watch channel * I'm sorry clippy * nits * rm some nonce related stuff * more refactoring * cleanup * Add whitelist settings * Whitelist filtering * fix docker build * use relayed terminology * Change whitelist tuple order * naming change * move whitelist check to before fetching proof * Update span event * make messages singular * Update error * minor cleanup * Update configs in scripts * Whitelist testing and parsing fixes Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	3 years ago
Trevor Porter	54a41d5a4b	More testnet2 tweaks (#571) ... * Reduce kathy frequency, add fuji * enable all kathys * Change Mumbai finality blocks 500 -> 32 * Always re-pull secrets, update mumbai to use 32 block finality blocks	3 years ago
Trevor Porter	1793fffbeb	Testnet2 agent deployment (#568) ... * Deploy testnet2 contracts * Add testnet2 configs to rust * don't panic for pre-london chains * Don't checkpoint, retry fewer times, and configure kathy properly * clean up * cleaninngggg Co-authored-by: Asa Oines <asaoines@Asas-MacBook-Pro.local>	3 years ago
Asa Oines	0a81c910a3	Move InterchainGasPaymaster from AbacusConnectionManager to AbacusConnectionClient (#558)	3 years ago
Trevor Porter	d120a76c1e	No more cached checkpoints in agents (#550) ... * Update ABIs, compiles. Removed Inbox indexer and cached checkpoints from the Inbox * Rm checkpoint indexing test * Update InboxValidatorManager, just need to connect the pieces now * Add channels * So close * Fix InboxValidatorManager deploy * Fix bug where prover_sync wasn't in line with latest_signed_checkpoint * Rm immediate message processing, clean up * rm abacus-cli * more cleanup * rename / rm some settings * TS renames / rms * Lower some processing failure logs to info * Checkpoint fetcher doesn't need the CommittedMessages * Hardcode kathy dispatching * Move to watch channel * I'm sorry clippy * nits * rm some nonce related stuff * PR comments	3 years ago
Trevor Porter	29970fe326	Index GasPayment events in Relayer (#504) ... * Add InterchainGasPaymaster traits and ethereum structs * Add abacuse-base interchain_gas * Actually setting up indexer * Setting up DB usage * wip * Add interchain gas paymaster to AbacusAgentCore * Sync gas payments in the relayer * It runs wooooooo * Use U256 for gas payments * nits, rm InterchainGasPaymasterEvents * Add chain_name, fix metrics * More name -> chain_name rename * Correctly rename contract_name => chain_name, move InterchainGasPayment from traits to types * split up contract_sync a bit * cargo fmt * nit * clean * Add metrics * to_string => encode_hex * make clippy happy * move contract_sync/common.rs stuff into mod.rs * string interpolation for logs * some nits * mv types/interchain_gas into types/mod * Idempotent GasPayment indexing * Use new contract ABI & update rust code * Rename RandomBadRecipient -> TestSendReceiver * use nit * only index finalized blocks * Run with finalityBlocks * Add InterchainGasPaymaster to update_abis * Some PR comments	3 years ago
Trevor Porter	5050eee9a4	Use new contract ABI & update rust code (#519)	3 years ago
Mattie Conover	71ce3e285b	Fix duplicate metric labels (#518)	3 years ago
Mattie Conover	d687d3e230	More Metrics (#495) ... * metrics refactoring * metrics refactoring * Make brain hurt less by using a damn function * patch through metrics to abacus_ethereum inbox and outbox * WIP * Minor cleanup * work on time update * added label arrays * refactoring * Metrics for sending txns * Metrics middleware * cleanup * Cleanup after merge * Add metrics config and setup prom metrics with core metrics * Minor fixes to docs * Better trait building * rename * Weave metrics through code * Use core metrics instead * Removed some unused code * fmt * change feature layout * fix docker build * Start scraping metrics on an interval * fmt * fix duplicate label error * change features * fix u256 conversion fn * refactor * Support default values in config * Minor fixes * Use f64 conversion from primitive-types * remove unused code * fix const name * improve docs * better var name * remove unused code * remove unused code	3 years ago
J M Rossy	7caaa4e5c6	[Chore] Fix VSCode formatting and prettier precommit (#501) ... - Fix VSCode prettier format on save - Add lint-staged to fix prettier precommit hook - Show root folder in vscode monorepo view - Format vscode settings files	3 years ago
webbhorn	6b3822f664	Misc. agent cleanup and dead code deletion (#470) ... * Fix path to agent::Agent in comment. Otherwise `cargo doc` emits a rustdoc::broken_intra_doc_links warning. * Use automatic link in comment for rustdoc. Otherwise 'cargo doc' emits a warning. * Delete checkpointer agent. It is not used in current deployment. * Delete kms-cli. Not used in current deployment (no references). * Include "--all" in precommit cargo fmt -- --check invocation. * Delete unused and hidden src/bin programs. These appear to have been introduced over a year ago and appear to be dead code now. Unhide them from vscode workspace too -- since they were hidden, automated refactoring tools wouldn't find them until compiler failed. * Update github CODEOWNERS to current Abacus team. Drop optional authors field from Cargo.tomls. * Drop processordb from .gitignore. Processor crate has been deleted. * Delete balance-exporter crate. It is not used / no references. * Hoist abacus-cli into its own top-level crate. It is the only tool now, no reason to hide it inside of a tools directory. * Delete old relaese.sh, which only relates to Celo Optics. * Cleanup top-level rust README.md. - Architecture paragraphs outdated, delete. - Point to run-locally.sh instead of documentation. * Explain a bit more about crate deps in README. * Dockerfile fix after tools directory restructuring. We had previously hoisted abacus-cli out since it was the only remaining tool. * Fix path to agent::Agent in comment. Otherwise `cargo doc` emits a rustdoc::broken_intra_doc_links warning. * Use automatic link in coment for rustdoc. Otherwise 'cargo doc' emits a warning. * Delete checkpointer agent. It is not used in current deployment. * Delete kms-cli. Not used in current deployment (no references). * Include "--all" in precommit cargo fmt -- --check invocation. * Delete unused and hidden src/bin programs. These appear to have been introduced over a year ago and appear to be dead code now. Unhide them from vscode workspace too -- since they were hidden, automated refactoring tools wouldn't find them until compiler failed. * Update github CODEOWNERS to current Abacus team. Drop optional authors field from Cargo.tomls. * Drop processordb from .gitignore. Processor crate has been deleted. * Delete balance-exporter crate. It is not used / no references. * Hoist abacus-cli into its own top-level crate. It is the only tool now, no reason to hide it inside of a tools directory. * Delete old relaese.sh, which only relates to Celo Optics. * Cleanup top-level rust README.md. - Architecture paragraphs outdated, delete. - Point to run-locally.sh instead of documentation. * Explain a bit more about crate deps in README. * Dockerfile fix after tools directory restructuring. We had previously hoisted abacus-cli out since it was the only remaining tool.	3 years ago
webbhorn	d40b2d776c	Reliability/correctness fixes for run-locally.sh. (#480) ... * Use ephemeral sig and db dirs for agents. If agents discover an old instance of their db files, or old validator signatures (likely from a prior invocation of run-locally.sh), they will operate based on the old state, which has diverged from the hardhat state, and thus be incorret. * Log agent errors to script stdout. * Log kathy stdout in backgrounded subshell. * Readonly env vars.	3 years ago
webbhorn	61f270aec5	Fix publisher for rust-analyezr extension. (#496) ... Without this change, vscode emits a warning that maktlad.rust-analyzer is not in the extensions marketplace. Since May 16, rust-lang is the new publisher for the rust-analyzer extension: https://twitter.com/rust_analyzer/status/1526181089391280128?cxt=HHwWgICwgazniq4qAAAA	3 years ago
Nam Chu Hoai	9e081a43f6	Fix agent config generation (#482)	3 years ago
Mattie Conover	36cb6296ce	Replace `boxed_trait` macro (#467) ... * remove macro * Cleanup * refactoring * Cleanup * Minor refactor * minor cleanup * more refactoring * dedup a bit * Attempt to fix pipeline	3 years ago
webbhorn	ba29b50a8f	Replace running-locally.md docs with a script. (#466) ... This bash script builds the relevant components, including relayer and validator agents, launches them in the required order once dependent services are ready, and writes stdout/stderr logs to unified datetime-based directory.	3 years ago
Mattie Conover	2a96c607f5	Update `report_tx` to be a function instead of a macro (#456) ... * Make brain hurt less by using a damn function * rename mod to tx	3 years ago
Mattie Conover	f78e79ab9b	Better Agent Error Logging (#412) ... * Make errors print on one line * Replace color-eyre with eyre in core and base * Replace color-eyre with eyre other libs * Log should be imported without patch version * Update tracing and opentelemetry * Only need openssl dynamic libs * Fix no-default-feature build * Custom eyre handler * Removed unused log dep * Cleanup paste dep * More package cleanup * Update tokio * Minor refactor	3 years ago
Mattie Conover	37af9c15a0	Running locally fixes and docs (#410) ... * Update readme with instructions on how to run locally * Better error message * Fix kathy task * Update rust configs * Update envs * Move to rust dir * Update rust readme	3 years ago
Nam Chu Hoai	d5bdd405c1	Remove auroratestnet from testnet environment (#404)	3 years ago
Mattie Conover	3c312d732c	improve dockerfile (#401)	3 years ago
Mattie Conover	9cc9d3ab52	Rust dependency cleanup + Lastest stable (#381) ... * ignore jetbrains IDE configs * Update to stable rust 1.60 * Remove unused deps from cargo projects * allow the previously unused fields to still be unused after new warnings * update dockerfile	3 years ago
Mattie Conover	50172104fa	Ported http retrying logic from nomad (#390) ... * Ported retrying logic from nomad * remove unused import	3 years ago
Yorke Rhodes	1d572ba887	Add vscode workspaces and settings (#387) ... * Add recommended root extensions * Add typescript and solidity workspace settings * Add rust settings and extensions * Update some root settings * Comment unused rust vars as revealed by analyzer * Remove root extensions in favor of workspace * Update README with VSCode hint	3 years ago
Nam Chu Hoai	cc6c2db5d8	Testnet deploy (#369) ... * Got keys for testnet * Testnet deploy * Add rust configs * prettier * Nits, update governance config * new agent image * Rename secret reference * Have kathy also checkpoint * Only checkpoint when count > 1 * Move relayer to unique key for (source, remote) pair * prettier * Clean up get-aws-key-addresses * get GCP keys half working * let's clean this up * First steps at refactor * Migrate delete-keys over * support key rotation * Create and get key all working * update relayer external secret * Prettier * Update ethers-rs * Debugging testnet deployment (#370) * Dont fail on inbox checkpointing tx error * fix docker build * Debug * More instrumentation * Lint * Use the correct signers * Lint * prettier * suffix -> index, go back to 1 key per agent * rm relayer user * Create user and key identifiers * Move some fns to key-utils, fix gcp secret names * Correct kathy & checkpointer key names * Back to Promse.all in deploy-agents, clean * Don't require GCP keys to be created with agentconfig * fix * pure json response for get-key-addresses Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	3 years ago
Nam Chu Hoai	cee9a2d8ef	Add more tracing information (#362) ... * Add more tracing information * PR review	3 years ago
Trevor Porter	6593e47f1a	Support agents with AWS keys (#361) ... * Add some tracing to S3 reading and writing * Add AgentAwsUser, refactor into aws dir * Some signer changes * Add AWS credentials to relayer external secret * Redeploy dev * upgrade agents * Log when indexed messages * Looks like relayer is all good * Validator workin * Needs cleanup but nearly there * Get AWS credentials when needed * cleaning * mv some common fns * Fix output env vars * Final clean (I think) * Better fix for annoying race cond * prettier * rm some tracing * Add message_count to messages indexed log	3 years ago
Nam Chu Hoai	2d7234455d	Support multiple inboxes per domain (#349) ... * Support multiple inboxes per domain * Rename in comment * PR review * PR review	3 years ago