30 Commits (946e220edfd297aa8da4f6682d7b7ae19a11fa86)

Author	SHA1	Message	Date
Mattie Conover	0644e4b22d	E2E testing workflow (#679) ... * Work on new e2e script * new e2e mimics old * Support CI mode * Kathy CI mode * remove old script * e2e workflow * Update readme * fix workflow * split build from run * change build target * Use lighter-weight http client * fix typo * better handling for stopping children * Fix url * Support multiple filters * cache node modules * cache yarn build * ci cleanup * Cleanup and updates * requested changes	2 years ago
webbhorn	dd63f70da1	Gelato Plumbing (#637) ... This PR prepares for the introduction of Gelato ForwardRequest API usage. However, it only refactors current code and introduces a prerequisite interface. In particular, the existing MessageProcessor was previously responsible both for message discovery via AbacusDB, as well as managing the submission of that message to the destination chain via some RPC provider. Since we want to have the option to deliver those messages over Gelato, a natural place to split things up might be between the AbacusDB message discovery phase, and the message delivery phase. This CL formalizes that notion by moving the submission logic into a SerialSubmitter submodule. The MessageProcessor and SerialSubmitter only communicate via a channel now. A subsequent PR will introduce GelatoSubmitter, which will route messages from the MessageProcessor to the Gelato API according to the new configuration options introduced in this PR.	2 years ago
Mattie Conover	1f7beb3b9c	Better Abigen (#633) ... * Abigen tools * Generate abis with buildscript * Generate abis with buildscript for middleware * Simplify code * update lock * Fix pipeline order, must run check first new to generate files	2 years ago
Mattie Conover	aa92873088	Oneline Backtraces (#627) ... * add backtrace lib * Pull over code for printing backtraces * Oneline backtraces * code shuffle * fmt * remove stale comment * cleanup * refactoring	2 years ago
webbhorn	018c73206b	Gelato: ForwardRequest and TaskStatus calls, and EIP-712 signing for ForwardRequests (#511) ... Goal of this PR is to add support for posting ForwardRequest messages to Gelato and obtaining the status of those requests post-submission-to-Gelato. The key abstractions introduced in this PR are: ForwardRequestCall (in fwd_req_call.rs): Logic for making a single, individual RPC to the Gelato endpoint responsible for accepting the initial forward request submission, which it then enqueues and handles asynchronously. fwd_req_sig.rs, which contains an impl of the ethers::types::transaction::eip712::Eip712 trait for ForwardRequestOpArgs. This allows an ethers::signers::Signer to sign our message before we submit it to the relevant Gelato endpoint. TaskStatusCall in task_status_call.rs: Analogous to ForwardRequestCall in that it encapsulates the logic and state required to make an individual call to a Gelato endpoint to get the status of a previously-submitted request. Chain in chains.rs: Represents chains that are relevant for Abacus and/or supported by Gelato. Handles conversions between the chain_id integer representation expected by Gelato and other types. Also provides the per-chain addresses we need when signing gelato requests for supported chains.	2 years ago
Mattie Conover	4d9603c868	Relayer whitelist (#570) ... * Update ABIs, compiles. Removed Inbox indexer and cached checkpoints from the Inbox * Rm checkpoint indexing test * Update InboxValidatorManager, just need to connect the pieces now * Add channels * So close * Fix InboxValidatorManager deploy * Make message processor slightly more readable * Create loop-control crate * Try to bring some sanity to message processor * Fix bug where prover_sync wasn't in line with latest_signed_checkpoint * Rm immediate message processing, clean up * rm abacus-cli * more cleanup * rename / rm some settings * TS renames / rms * Lower some processing failure logs to info * Checkpoint fetcher doesn't need the CommittedMessages * Hardcode kathy dispatching * Move to watch channel * I'm sorry clippy * nits * rm some nonce related stuff * more refactoring * cleanup * Add whitelist settings * Whitelist filtering * fix docker build * use relayed terminology * Change whitelist tuple order * naming change * move whitelist check to before fetching proof * Update span event * make messages singular * Update error * minor cleanup * Update configs in scripts * Whitelist testing and parsing fixes Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	2 years ago
Trevor Porter	d120a76c1e	No more cached checkpoints in agents (#550) ... * Update ABIs, compiles. Removed Inbox indexer and cached checkpoints from the Inbox * Rm checkpoint indexing test * Update InboxValidatorManager, just need to connect the pieces now * Add channels * So close * Fix InboxValidatorManager deploy * Fix bug where prover_sync wasn't in line with latest_signed_checkpoint * Rm immediate message processing, clean up * rm abacus-cli * more cleanup * rename / rm some settings * TS renames / rms * Lower some processing failure logs to info * Checkpoint fetcher doesn't need the CommittedMessages * Hardcode kathy dispatching * Move to watch channel * I'm sorry clippy * nits * rm some nonce related stuff * PR comments	3 years ago
Mattie Conover	d687d3e230	More Metrics (#495) ... * metrics refactoring * metrics refactoring * Make brain hurt less by using a damn function * patch through metrics to abacus_ethereum inbox and outbox * WIP * Minor cleanup * work on time update * added label arrays * refactoring * Metrics for sending txns * Metrics middleware * cleanup * Cleanup after merge * Add metrics config and setup prom metrics with core metrics * Minor fixes to docs * Better trait building * rename * Weave metrics through code * Use core metrics instead * Removed some unused code * fmt * change feature layout * fix docker build * Start scraping metrics on an interval * fmt * fix duplicate label error * change features * fix u256 conversion fn * refactor * Support default values in config * Minor fixes * Use f64 conversion from primitive-types * remove unused code * fix const name * improve docs * better var name * remove unused code * remove unused code	3 years ago
webbhorn	6b3822f664	Misc. agent cleanup and dead code deletion (#470) ... * Fix path to agent::Agent in comment. Otherwise `cargo doc` emits a rustdoc::broken_intra_doc_links warning. * Use automatic link in comment for rustdoc. Otherwise 'cargo doc' emits a warning. * Delete checkpointer agent. It is not used in current deployment. * Delete kms-cli. Not used in current deployment (no references). * Include "--all" in precommit cargo fmt -- --check invocation. * Delete unused and hidden src/bin programs. These appear to have been introduced over a year ago and appear to be dead code now. Unhide them from vscode workspace too -- since they were hidden, automated refactoring tools wouldn't find them until compiler failed. * Update github CODEOWNERS to current Abacus team. Drop optional authors field from Cargo.tomls. * Drop processordb from .gitignore. Processor crate has been deleted. * Delete balance-exporter crate. It is not used / no references. * Hoist abacus-cli into its own top-level crate. It is the only tool now, no reason to hide it inside of a tools directory. * Delete old relaese.sh, which only relates to Celo Optics. * Cleanup top-level rust README.md. - Architecture paragraphs outdated, delete. - Point to run-locally.sh instead of documentation. * Explain a bit more about crate deps in README. * Dockerfile fix after tools directory restructuring. We had previously hoisted abacus-cli out since it was the only remaining tool. * Fix path to agent::Agent in comment. Otherwise `cargo doc` emits a rustdoc::broken_intra_doc_links warning. * Use automatic link in coment for rustdoc. Otherwise 'cargo doc' emits a warning. * Delete checkpointer agent. It is not used in current deployment. * Delete kms-cli. Not used in current deployment (no references). * Include "--all" in precommit cargo fmt -- --check invocation. * Delete unused and hidden src/bin programs. These appear to have been introduced over a year ago and appear to be dead code now. Unhide them from vscode workspace too -- since they were hidden, automated refactoring tools wouldn't find them until compiler failed. * Update github CODEOWNERS to current Abacus team. Drop optional authors field from Cargo.tomls. * Drop processordb from .gitignore. Processor crate has been deleted. * Delete balance-exporter crate. It is not used / no references. * Hoist abacus-cli into its own top-level crate. It is the only tool now, no reason to hide it inside of a tools directory. * Delete old relaese.sh, which only relates to Celo Optics. * Cleanup top-level rust README.md. - Architecture paragraphs outdated, delete. - Point to run-locally.sh instead of documentation. * Explain a bit more about crate deps in README. * Dockerfile fix after tools directory restructuring. We had previously hoisted abacus-cli out since it was the only remaining tool.	3 years ago
Trevor Porter	fec07d21fe	Remove docs & all optics-related rust (#322) ... * optics -> abacus in infra * rm docs * rename contract-metrics prom metrics optics-* -> abacus-* * Rm watcher, updater, a bunch of optics things in abacus-base * rm agents/processor * Removing a whole lot of updater / home / replica stuff from abacus-core * Deleted a lot throughout -- some still commented out and a lot of cleanup needed * Slowly cleaning up, kathy is where the remaining errors are * Delete some commented out code * makes kathy an abacus agent * compiling woo * cargo fmt * compiling with all features * removing some update references * Ignore handles_missing_rpc_checkpoints * cargo fmt * Add signedCheckpoint.json * rm some references to updates, delete unnecessary abis * rm all references to home or replica * compiling, remove som e unused code * fix test warning * nit * New README - needs a better pass, but no optics references for now * mv rust/helm/optics-agent rust/helm/abacus-agent * Note on us not indexing updates * Get checkpointer working from local env var output	3 years ago
Nam Chu Hoai	4a28875af8	Add Validator scaffold (#213)	3 years ago
Nam Chu Hoai	2cb643486f	Add checkpointer scaffold (#183) ... * Add checkpointer scaffold * Add Abacus ABIs to abacus-ethereum (#186) * Add Abacus ABIs * Build on all PRs * PR comments * lint * Lint * Add new Abacus Outbox/Inbox wrappers (#189) * Add new Abacus Outbox/Inbox wrappers * PR review * PR comments	3 years ago
Nam Chu Hoai	d4c8720700	Rename rust packages (#128) ... * Rename rust packages * rustfmt * More rename * Rename update_abis * Rename metrics.rs	3 years ago
Luke Tchang	40d0ceb253	refactor/feature: split up indexing/contract/db logic + make watcher functional again (#925) ... * refactor: prefixes all loads and stores with home_name or entity in AgentDB * fmt: cargo fmt * refactor: use AgentDB everywhere and scap HomeDB * refactor: rename AgentDB OpticsDB * fix: put watcher back into cargo.toml and fix OpticsDB there * feature: adds ReplicaIndexer and moves index() to Common * refactor: implement CachingHome and CachingReplica + chain agnostic indexer, agent still broken due to not using CachingHome * prog: agent with CachingHome still broken; plan is to provide metrics obj and index-time * fix: contract sync stores messages too * refactor: successfully builds with CachingHome on agent that can spawn sync task * feature: adds unused CachingReplica * refactor: remove contract_name from indexer and give field to contract_sync * refactor: replaces all Replicas usage with CachingReplica * refactor: only CachingHome/Replica reach out to db and implement Event traits * fix: watcher tests work again * fix: removes db references from ethereum home and replica * rebase: rebase big indexer refactor on TypedDB/OpticsDB changes from updater split PR * fix: update crates and silence abigen! warnings for lint * feature: make watcher functional again * rename: spawn_sync --> sync * refactor: fetch_messages returns empty vec by default and home implements method * fix: update watcher-partial config and rename index tasks sync tasks * refactor: HomeIndexer now separate trait than CommonIndexer * doc: document Indexer trait and ContractSync * refactor: Homes/Replicas serve as Arc wrappers * fix: watcher stores update but not dependent on latest root * doc: document watcher fraud behavior * fix: indexers return _sorted_ updates and messages	3 years ago
Conner Swann	7b2c4a21df	refactor: split updater tasks for safety (#909) ... * refactor: split updater tasks, refactor opticsdb * feature: asrefs for db types * refactor: refactor typeddb to hold entity and pair with all loads/stores * ci: cargo +nightly * ci: use nightly toolchain * fix: use 1.56 stable not nightly * fix: updater can never overwrite an existing update with a conflicting one * fix: also check suggested update for state change guard Co-authored-by: Luke Tchang <ltchang@stanford.edu>	3 years ago
ember arlynx	a240facc6d	Balance exporter tool skeleton (#793) ... * First pass at balance exporter tool * this was broken with windows unicode paths * promote optics_base to a facade * optics-ethereum has many fewer lines of code * remove a layer of indirection * update all the other code * fmt * post-mege * flatten core::types::\* * impl Chain for ethers * move balance-exporter into tools * post-merge fixes * post-merge resolution * put Default back	3 years ago
Luke Tchang	eb685bdbf8	refactor: watcher home indexing rewrite (#809) ... * refactor: add HomeDB struct (raw db not replaced yet) * refactor: EthereumHome and HomeIndexer use HomeDB instead of DB * refactor: UpdateHandler uses HomeDB * refactor: processor/prover-sync use HomeDB * refactor: error cleanup * refactor: implements run_all without metrics * refactor: watcher uses oneshot to receive double update * fix: update comments * fix: panic on run and run_many * tracing: add more info when double update found * rebase: fix store_latest_update rename * test: add default impls to make watcher unit tests compile * fix: run_loop takes HomeDB, watcher struct holds DB * run_watch_tasks borrows &self instead of many params * rebase: fix unused imports	3 years ago
Luke Tchang	ab8913df12	feature: add db state dump command to `optics-cli` (#874) ... * refactor: pull out prove into subcommand * refactor: structopt not clap * fix: leaf or leaf_index args * prog: draft for dbstate command without printing * prog: prints unordered list of roots, block nums, and leaves * fix: decomposes run function and tidies up printing * nit: rename committed_root to update_root at print stage * fix: dockerfiles specifies optics-cli instead of prove-cli * feature: allow saving to json file * fix: use retrieve_metadata instead after history rearragement * doc: fix info message and kv comment	3 years ago
James Prestwich	16dacb5d08	Prestwich/prove cli (#823) ... * feature: simple prove cli * bug: set chainid * feature: skip re-processing and re-proving * refactor: improve prove cli legibility * refactor: more concise leaf idx selection * doc: readme for prove-cli * fix docker build * fix: prover-cli fixes typo: exampl and hyphens in readme bug: multiple -l * refactor: prover_cli uses HomeDB keyed by RPC * feature: prove-cli allows AWS usage * bug: use home name as cli arg * bug: properly retrieve and use chain_id in prove-cli * feature: prove CLI contains lookups for replica and rpc Co-authored-by: Conner Swann <me@connerswann.me>	3 years ago
James Prestwich	39b7fc5964	Prover sync refactoring (#781) ... * refactor: display impl for ProverSync * refactor: make prover internal only * refactor: remove non-functional gas escalator * refactor: disable watcher, task management refactor * fixed Docker build Co-authored-by: Conner Swann <me@connerswann.me>	3 years ago
ember arlynx	5f041c9db1	drive-by cleanups (#592) ... * reorganize rust code into dirs * rust: ignore vendor dir * how did kathy go missing? * drive-by cleanups: abstraction in update_abis, hyperlink, shell scripts are mostly shellcheck clean * rust readme: note agents dir * cargo fmt --all * remove stray comment * cargo update * why am i fighting my computer rn * patch over sloppy rebase * shore up test vector file handling * clarify how to add a new chain * add rust-toolchain.toml for 1.54 * DRY abis_dir * chore: add clippy installation to CI * github actions will use rust-toolchain.yaml now * move rust-toolchain so the action will find it rustup seems to scan upwards for this file, should pick it up fine when using from within * use obsolete rust-toolchain file location, because the action is naive * chore: specify toolchain in rust.yml * chore: specify toolchain in rust.yml in all workflows * chore: return clippy to rust.yml Co-authored-by: ember arlynx <ember@lunar.software> Co-authored-by: James Prestwich <james@prestwi.ch>	3 years ago
James Prestwich	13644d5957	KMS CLI (#547) ... * feature: sweep bin * refactor: hardcode gas req for simple sweep tx * refactor: update to latest version of ethers * feature: kms cli * bug: short designator conflict + docs * bug: fix value parsing * chore: temp fix for simple sends * refactor: delete deprecated kms bins, roll features into kms-cli	3 years ago
Luke Tchang	01b6aa43ea	test: adds optics-test crate and unit test for watcher::check_double_update (#138) ... * refactor: extracts double update check to helper * prog: mockall derive debug on Common and Home broken * feature: MockHomeContract in optics-base * test: adds test for watcher UpdateHandler check_double_update * feature: adds replica mock contract * refactor: moves mocks to new optics-test crate * fix: satisfy clippy gods * fix: removes dev dependencies from optics-base * fix: fixes optics-test docstring and unecessary assert_eq Co-authored-by: James Prestwich <prestwich@clabs.co>	4 years ago
Anna Carroll	69f64b04f8	feature: add untested Processor daemon (#97) ... * feature: add untested Processor daemon * feature: add an untested processor daemon * docs: clean up and clarify the watcher * bug: fix watcher loop conflating leaf index and sequence * feature: improved handling for main watcher loop - info logging on loop resets due to acceptable faults - returns an error if the local prover is out of sync with the home chain Co-authored-by: James Prestwich <prestwich@clabs.co>	4 years ago
Erin Hales	5d3668495a	enhancement: move abi files to shared folder (#94) ... * moved abis to shared folder * basic (surely flawed) optics ethereum package * bug: fix circular imports, rearrange files and paths, lint * fix: unused import * fix: serde Deserialize * fix test issues, delete settings * enhancement: separated out home and replica abis * fix: pub use statements * cargo update and lint * fix: doc code snippet, trim dependencies	4 years ago
Luke Tchang	c57b9ded00	feature: adds untested Watcher daemon (#73) ... * enhance: adds Update event emission to Replica and signed_update_by_old root to Replica abi * feature: adds core run logic for watcher observing single replica * feature: adds watch_home logic and run_many override * refactor: moves signed_update_by_old_root to common * refactor: uses Common enum for Home/Replica * refactor: removes enum and uses ?Sized to remove Sized constraint * fix: removes unnecessary import * fix: runs cargo fmt and replaces references to origin_slip44 with domain * fix: fixes deadlock setup for history read/write * fix: fixes poll_signed_update to use signed_update_by_new_root * fix: fixes off-by-one error for topics for signed_update abi methods * docs: fixes comment formatting on signed_update methods	4 years ago
Luke Tchang	05e551f26a	fix: readds settings and can_confirm abi function to relayer (#70)	4 years ago
James Prestwich	f4078c3961	Add Kathy (#48) ... * fix: use kebab-case in configs * feature: add kathy, a chatty friend * feat: add default impl for chat generators	4 years ago
James Prestwich	4a57874d8c	feature: adds an untested updater daemon (#19) ... * feature: adds an untested updater daemon * feature: check that the updater is using the correct keys before polling for updates * refactor: simplify OpticsAgent::run_report_error * chore: improve documentation and remove unnecessary map_errs * bug: correct env var prefix in updater settings	4 years ago
James Prestwich	fd838e0b0c	feature: implement replica trait on replica contract abigen	4 years ago