116 Commits (027a51d68044f4348c1c6ded1c241f173735b979)

Author	SHA1	Message	Date
Nam Chu Hoai	23c3e4cd22	Add lookback for updates/messages in contract_sync (#144) ... * Apply contract_sync mod patch * Add IndexDataTypes * Fmt * Do not store updates/messages when encountering an invalid continuation If we store them we may end up invariably overwriting the latest_leaf_index database entry which keeps track of "contigousness". I.e. this would cause holes of size >1 to cause the agents being unable to recover * Index both messages and updates on the processor * remove comment * Followups for the contract-sync patch (#146) * Readd abacus_last_known_message_leaf_index to message indexing in the processor * Remove tipbuffer functionality * Additional contract_sync tests (#147) * Fix test case for when batch includes the already known updates/messages * Modify tests to include multiple ranges of missing updates/events	3 years ago
Nam Chu Hoai	d4c8720700	Rename rust packages (#128) ... * Rename rust packages * rustfmt * More rename * Rename update_abis * Rename metrics.rs	3 years ago
Luke Tchang	40d0ceb253	refactor/feature: split up indexing/contract/db logic + make watcher functional again (#925) ... * refactor: prefixes all loads and stores with home_name or entity in AgentDB * fmt: cargo fmt * refactor: use AgentDB everywhere and scap HomeDB * refactor: rename AgentDB OpticsDB * fix: put watcher back into cargo.toml and fix OpticsDB there * feature: adds ReplicaIndexer and moves index() to Common * refactor: implement CachingHome and CachingReplica + chain agnostic indexer, agent still broken due to not using CachingHome * prog: agent with CachingHome still broken; plan is to provide metrics obj and index-time * fix: contract sync stores messages too * refactor: successfully builds with CachingHome on agent that can spawn sync task * feature: adds unused CachingReplica * refactor: remove contract_name from indexer and give field to contract_sync * refactor: replaces all Replicas usage with CachingReplica * refactor: only CachingHome/Replica reach out to db and implement Event traits * fix: watcher tests work again * fix: removes db references from ethereum home and replica * rebase: rebase big indexer refactor on TypedDB/OpticsDB changes from updater split PR * fix: update crates and silence abigen! warnings for lint * feature: make watcher functional again * rename: spawn_sync --> sync * refactor: fetch_messages returns empty vec by default and home implements method * fix: update watcher-partial config and rename index tasks sync tasks * refactor: HomeIndexer now separate trait than CommonIndexer * doc: document Indexer trait and ContractSync * refactor: Homes/Replicas serve as Arc wrappers * fix: watcher stores update but not dependent on latest root * doc: document watcher fraud behavior * fix: indexers return _sorted_ updates and messages	3 years ago
Luke Tchang	f007a299c8	feature: add metrics/tracking to remaining agents (#922) ... * feature: add num relayed messages metrics/tracking to relayer * refactor: remove processor-specific nonce storage method from optics db * fix: use countervec for relayer instead of guage * feature: updater uses intcounter vec for submitted update count * fix: remove db from relayer since it now uses countervec with no rocksdb * fix: updates relayed not messages oops * fix: update submitter takes the correct countervec	3 years ago
Conner Swann	7b2c4a21df	refactor: split updater tasks for safety (#909) ... * refactor: split updater tasks, refactor opticsdb * feature: asrefs for db types * refactor: refactor typeddb to hold entity and pair with all loads/stores * ci: cargo +nightly * ci: use nightly toolchain * fix: use 1.56 stable not nightly * fix: updater can never overwrite an existing update with a conflicting one * fix: also check suggested update for state change guard Co-authored-by: Luke Tchang <ltchang@stanford.edu>	3 years ago
ember arlynx	a240facc6d	Balance exporter tool skeleton (#793) ... * First pass at balance exporter tool * this was broken with windows unicode paths * promote optics_base to a facade * optics-ethereum has many fewer lines of code * remove a layer of indirection * update all the other code * fmt * post-mege * flatten core::types::\* * impl Chain for ethers * move balance-exporter into tools * post-merge fixes * post-merge resolution * put Default back	3 years ago
Luke Tchang	eb685bdbf8	refactor: watcher home indexing rewrite (#809) ... * refactor: add HomeDB struct (raw db not replaced yet) * refactor: EthereumHome and HomeIndexer use HomeDB instead of DB * refactor: UpdateHandler uses HomeDB * refactor: processor/prover-sync use HomeDB * refactor: error cleanup * refactor: implements run_all without metrics * refactor: watcher uses oneshot to receive double update * fix: update comments * fix: panic on run and run_many * tracing: add more info when double update found * rebase: fix store_latest_update rename * test: add default impls to make watcher unit tests compile * fix: run_loop takes HomeDB, watcher struct holds DB * run_watch_tasks borrows &self instead of many params * rebase: fix unused imports	3 years ago
James Prestwich	985bd7850c	feature: configuration to push proofs to s3 buckets (#821) ... * feature: configuration to push proofs to s3 buckets * refactor: key s3 proofs by their index, not their leaf hash * chore: improve processor bootup logging * fix: add name and bucket to debug for Pusher * bug: check that content-length is gt 0 * bug: expect 3 labels in next_message_nonce metric * bug: properly handle non-existent key * fix: log messages for proof pushing * feature: upload message to s3 with proof	3 years ago
Luke Tchang	ab8913df12	feature: add db state dump command to `optics-cli` (#874) ... * refactor: pull out prove into subcommand * refactor: structopt not clap * fix: leaf or leaf_index args * prog: draft for dbstate command without printing * prog: prints unordered list of roots, block nums, and leaves * fix: decomposes run function and tidies up printing * nit: rename committed_root to update_root at print stage * fix: dockerfiles specifies optics-cli instead of prove-cli * feature: allow saving to json file * fix: use retrieve_metadata instead after history rearragement * doc: fix info message and kv comment	3 years ago
James Prestwich	16dacb5d08	Prestwich/prove cli (#823) ... * feature: simple prove cli * bug: set chainid * feature: skip re-processing and re-proving * refactor: improve prove cli legibility * refactor: more concise leaf idx selection * doc: readme for prove-cli * fix docker build * fix: prover-cli fixes typo: exampl and hyphens in readme bug: multiple -l * refactor: prover_cli uses HomeDB keyed by RPC * feature: prove-cli allows AWS usage * bug: use home name as cli arg * bug: properly retrieve and use chain_id in prove-cli * feature: prove CLI contains lookups for replica and rpc Co-authored-by: Conner Swann <me@connerswann.me>	3 years ago
James Prestwich	0aba12bb9c	Prestiwch/polygon metrics fix (#783) ... * refactor: move block height tracking to the HomeIndexer, temporarily disable replica block height tracking * chore: enhance updater main comment * refactor: beta reduction	3 years ago
James Prestwich	39b7fc5964	Prover sync refactoring (#781) ... * refactor: display impl for ProverSync * refactor: make prover internal only * refactor: remove non-functional gas escalator * refactor: disable watcher, task management refactor * fixed Docker build Co-authored-by: Conner Swann <me@connerswann.me>	3 years ago
James Prestwich	11d2ec0c5d	prestwich/home indexing (#768) ... * refactor: move db into optics-core, pass db to home instantiation * feature: prototype chain indexer * chore: add todo comment * feature: make chunk_size configurable * refactor: improve DB through out the codebase * bug: use to in indexer spawn * feature: tracing in home indexer * refactor: indexer settings are strings * feature: add setting to disable replica #767 * feature: store latest leaf_index in db * refactor: make fetch_leaf dependent only on DB * feature: add index config to kovan testnet * chore: increase polygon chunk size to 9999	3 years ago
ember arlynx	2767fd921c	feature: additional agent metrics (#746) ... * use a label for replica dimensioning, not a separate metric name * use labels in updater metric for attestation count * don't use non-Vec metrics * track the block height metric * fix typos	3 years ago
Conner Swann	2ffd4ee915	cargo.lock (#724)	3 years ago
ember arlynx	d5745362c6	rust agents: expose prometheus metrics (#670) ... * expose prometheus metrics from agents currently not actually updated in the course of execution * cargo update * log when starting metrics server, or not * metricsport -> metrics * metrics ought to be string * set content-type	3 years ago
Anna Carroll	5ba0bed73a	Final Contract Cleaning + Renaming (#698) ... * Rename Pending to Proven; make case for enums consistent * Replica docs * GovernanceRouter docs * Fix test error message * XAppConnectionManager docs * Remove is_owner from rust * Remove isOwner test * Rename enqueue => dispatch * Update tests to remove enqueue * Rename enqueue in rust * Fix: disapatch * Rename sequences => nonces * Rename sequences => nonces * fix lint * Rename current => committedRoot * Rename leaf => messageHash * Rename endBlock => recoveryActiveAt * Fix test gen	3 years ago
James Prestwich	db75e6714b	feature: BridgeRouter forbids transfer to 0 addr (#696)	3 years ago
James Prestwich	b2cb90d6cf	chore: add explicit ws dep to work with new ethers (#669)	3 years ago
ember arlynx	5f041c9db1	drive-by cleanups (#592) ... * reorganize rust code into dirs * rust: ignore vendor dir * how did kathy go missing? * drive-by cleanups: abstraction in update_abis, hyperlink, shell scripts are mostly shellcheck clean * rust readme: note agents dir * cargo fmt --all * remove stray comment * cargo update * why am i fighting my computer rn * patch over sloppy rebase * shore up test vector file handling * clarify how to add a new chain * add rust-toolchain.toml for 1.54 * DRY abis_dir * chore: add clippy installation to CI * github actions will use rust-toolchain.yaml now * move rust-toolchain so the action will find it rustup seems to scan upwards for this file, should pick it up fine when using from within * use obsolete rust-toolchain file location, because the action is naive * chore: specify toolchain in rust.yml * chore: specify toolchain in rust.yml in all workflows * chore: return clippy to rust.yml Co-authored-by: ember arlynx <ember@lunar.software> Co-authored-by: James Prestwich <james@prestwi.ch>	3 years ago
James Prestwich	4f56d8a51b	feature: log hex signature in update methods	3 years ago
James Prestwich	c576771bac	chore: bump ethers to use legacy feature in optics-core	3 years ago
James Prestwich	13644d5957	KMS CLI (#547) ... * feature: sweep bin * refactor: hardcode gas req for simple sweep tx * refactor: update to latest version of ethers * feature: kms cli * bug: short designator conflict + docs * bug: fix value parsing * chore: temp fix for simple sends * refactor: delete deprecated kms bins, roll features into kms-cli	3 years ago
James Prestwich	7b501442b1	refactor: add aws signer (#516) ... * refactor: add aws signer * lint: clippy and fmt * lint: fix unsequenced reads * feature: add a small bin to print key address	3 years ago
James Prestwich	a38f4f5153	chore: bump ethers to main in all cargo.toml files	3 years ago
James Prestwich	aff52ce2a9	bug: tracing and pending txn fixes ... - use branch of ethers-rs with pendingtx poll delay - use tracing::Instrumented in most agent trait methods - add in_current_span to most spawns - add an id generator to jaeger tracing	3 years ago
James Prestwich	677f68c1e1	feature: zipkin	3 years ago
James Prestwich	5722a698f2	Refactor tracing subscriber instatiation (#466) ... * WIP(broken): start breaking out tracing config * WIP: custom abstraction over log formats * WIP(feat): clean jaeger instantiation in Subscriber layers * refactor: improve jaeger layer instantiation and application. Add filter layer * lint: self by value on copy type * feature: implement Jaeger configuration * bug: specify reqwest collector client and remove doc fragment * refactor: use std convert traits where possible * refactor: introduce DroppedError	3 years ago
James Prestwich	1593d3b0c9	feature: time out txns that have been pending confirmation for 10m (#457)	3 years ago
James Prestwich	a37a6ed776	Prestwich/rust config (#419) ... * improve configuration handling of env vars - improve decl_settings macro to name the declared structs - change domains and timers to String - rename chat_gen to chat in kathy settings - panic on failed uint parsing in settings deser * bug: update typescript deploy to output string domains * chore: update new deploy conf to match new format * chore: set kathy destination to 3000 for now * refactor: ensure kathy dispatches messages to each comnfigured replica	3 years ago
James Prestwich	41b19b70dd	refactor: add prove_and_process to replicas enum	4 years ago
James Prestwich	76ab2034d8	refactor: fix the annoying process bug. misc rust updates	4 years ago
Luke Tchang	ae28e02316	test/bug: verify Rust/Solidity OpticsMessage and leaf equivalence (#363) ... * feature: outputs hardcoded merkle proof * fix: wrong sender address * debug: print sender receiver for StampedMessage * prog: leaf check failing * fix: fixes message pack ordering bug and outputs correct message and leaf json * fix: add message and leaf output to precommit fn * fix: use hex string instead of hardcoded str * feature: adds output proof test function	4 years ago
Luke Tchang	c4b34fb2df	test: adds mock ConnectionManager interface + watcher failure test cases (#314)	4 years ago
Luke Tchang	befde049a6	fix: update signer to support EIP-155 (#329) ... * fix: provider with SignerMiddleware udpates chain_id of signer to match provider * fix: add ethers wallets feature flags to optics-core cargo.toml	4 years ago
James Prestwich	d3788f0457	refactor: move Ethereum settings to optics-ethereum (#266)	4 years ago
Luke Tchang	0fa4283a70	refactor: restructures agent config locations (#254) ... * feature: restructures configs and from_settings * fix: settings takes agent name from macro * fix: replaces toml with json * fix: removes redundant agent config info * fix: derives prefix from agent name * fix: readds kathy config and deletes optics base config * fix: fixes kebab case rename for base settings and removes extra db_path fields	4 years ago
James Prestwich	2f943b5c20	fix: respect the clippy	4 years ago
Luke Tchang	9bc2feff4b	fix: update deploy scripts after upgradability changes (#208) ... * Move ownDomain to initialize; logically paired with instance, not implementation * fix: updates deploy scripts to deploy upgradable contract suite * fix: fixes commas in replica.rs collision avoidance * fix: fixes object used in console logs * fix: style fixes on replica deploy script and addition of lastProcessed field * fix: removes duplicated line after rebase on main * fix: rename executables Co-authored-by: anna-caroll <anna.s.carroll@gmail.com>	4 years ago
Luke Tchang	d12c50845f	feature: persists watcher history to disk (#192) ... * feature: adds disk persistence to updater and checks for conflicting updates * feature: gets db_path from settings block * feature: adds open_db util and replaces in updater * test: adds test_util func to run db tests in closure * feature: adds serde * feature: proversync saves new leaves to disk * feature: adds loading prover from disk * rebase: removes duplication after rebase on updater-history branch * feature: proversync saves new leaves to disk * feature: adds loading prover from disk * feature: replaces hashmap with db * rebase: removes duplicated import * test: uses run_test util for watcher tests * fix: replaces bincode with encode/decode * fix: fix after rebase on prover persistence PR * fix: removes unecessary serialize/deserialize	4 years ago
Luke Tchang	73b989285e	feature/refactor: saves/loads processor prover-tree to/from disk + fixes abstraction layers between prover and merkle (#189) ... * refactor: joins two chain api calls and spawns task for wait and check * feature: adds disk persistence to updater and checks for conflicting updates * feature: gets db_path from settings block * feature: adds open_db util and replaces in updater * feature: adds encode/decode to optics-core lib structs * test: adds test_util func to run db tests in closure * feature: adds serde * feature: proversync saves new leaves to disk * feature: adds loading prover from disk * fix: replaces ne_bytes to be_bytes * rebase: removes duplication after rebase on updater-history branch * fix: removes bincode * refactor: fixes abstraction layers between prover and merkle variants * refactor: uses prefix_iterator in prover * fix: has rocksdb imports draw from github * refactor: adds UsingPersistence trait and adds to updater, prover, proversync * refactor: update UsingPersistence to use Encode + Decode * feature: iterator in UsingPersistence Co-authored-by: James Prestwich <james@prestwi.ch>	4 years ago
Luke Tchang	b9694211f4	feature: adds updater history and persists to disk (#180) ... * refactor: joins two chain api calls and spawns task for wait and check * refactor: poll_and_handle_update returns join handle * feature: adds disk persistence to updater and checks for conflicting updates * feature: gets db_path from settings block * feature: adds open_db util and replaces in updater * feature: adds encode/decode to optics-core lib structs * test: adds test_util func to run db tests in closure * fix: uses Encode::to_vec	4 years ago
Luke Tchang	e7e25db05f	tests: relayer tests (#153) ... * refactor: separates prover and prover_sync; all agents compiling except watcher * refactor: adds EthereumReplica variant and fixes agents * refactor: adds MockReplica variant to Replicas * fmt: runs cargo fmt * test: adds updater poll_and_handle_update test * test: adds polls_and_relays_update test to relayer * test: adds poll_confirm test to relayer * fix: removes duplicated additions after rebase * fix: removes TxOutcome::default after rebase	4 years ago
Luke Tchang	1cdc8c8614	test: updater tests (#152) ... * prog: adds Homes type to optics-base and has AgentCore take in Arc<Homes> * refactor: separates prover and prover_sync; all agents compiling except watcher * refactor: adds EthereumReplica variant and fixes agents * refactor: adds MockReplica variant to Replicas * fmt: runs cargo fmt * test: adds updater poll_and_handle_update test * fix: removes duplicates after bad rebase * test: adds no update path test to updater test * fix: uses cfg_attr to only warn missing docs in test	4 years ago
Luke Tchang	9cae2f469a	refactor: contract trait refactor + prover/prover_sync separation (#151) ... * prog: adds Homes type to optics-base and has AgentCore take in Arc<Homes> * refactor: separates prover and prover_sync; all agents compiling except watcher * refactor: adds EthereumReplica variant and fixes agents * refactor: adds MockReplica variant to Replicas * fmt: runs cargo fmt * feature: adds From implementations for Homes and Replicas * feature: adds From<EthereumHome> for Homes * feature: adds From<EthereumReplica> for Replicas * feature: adds Other variant to Homes and Replicas * lint: use Box in all homes and replicas to satisfy clippy * bug: fix Ethereum handling in ChainSetup::try_into_ functions Co-authored-by: James Prestwich <james@prestwi.ch>	4 years ago
James Prestwich	1f333b6c02	refactor: improve processor task handling (#147) ... * refactor: improve processor task handling * refactor: change OpticsAgent::run to return a join handle * refactor: change default behavior for Agent to cancel all tasks instead of continue	4 years ago
Luke Tchang	01b6aa43ea	test: adds optics-test crate and unit test for watcher::check_double_update (#138) ... * refactor: extracts double update check to helper * prog: mockall derive debug on Common and Home broken * feature: MockHomeContract in optics-base * test: adds test for watcher UpdateHandler check_double_update * feature: adds replica mock contract * refactor: moves mocks to new optics-test crate * fix: satisfy clippy gods * fix: removes dev dependencies from optics-base * fix: fixes optics-test docstring and unecessary assert_eq Co-authored-by: James Prestwich <prestwich@clabs.co>	4 years ago
Luke Tchang	c3c1c184d9	feature: adds basic chat generators to Kathy (#101) ... * feature: adds static, random, and ordered list ChatGenerator types to kathy * refactor: adds enum interior state and no longer returns vec * feature: ordered list variant uses counter and gen_chat returns Option<Message> * refactor: replaces usize with AtomicUsize * rebase: fixes errors after agent refactor * feature: adds config blocks for generators and instantiation from settings * fix: removes rwlock * feature: makes destination and recipient configurable for static messages * fix: runs cargo update and fixes length.clone() --> *length	4 years ago
Anna Carroll	69f64b04f8	feature: add untested Processor daemon (#97) ... * feature: add untested Processor daemon * feature: add an untested processor daemon * docs: clean up and clarify the watcher * bug: fix watcher loop conflating leaf index and sequence * feature: improved handling for main watcher loop - info logging on loop resets due to acceptable faults - returns an error if the local prover is out of sync with the home chain Co-authored-by: James Prestwich <prestwich@clabs.co>	4 years ago
Erin Hales	5d3668495a	enhancement: move abi files to shared folder (#94) ... * moved abis to shared folder * basic (surely flawed) optics ethereum package * bug: fix circular imports, rearrange files and paths, lint * fix: unused import * fix: serde Deserialize * fix test issues, delete settings * enhancement: separated out home and replica abis * fix: pub use statements * cargo update and lint * fix: doc code snippet, trim dependencies	4 years ago