43 Commits (ebfbdaa708c223c750ad9d17d9b8154903fc5546)

Author	SHA1	Message	Date
Nam Chu Hoai	d3fbd1f9f9	Add relayer scaffold (#225) ... * Add relayer scaffold * Lint * Add Outbox indexing to relayer and determine messages destined for inbox (#227) * Add Outbox indexing to relayer and determine messages destined for inbox * Lint * PR review * Support message processing in relayer (#232) * Support message processing in relayer * Lint * lint * PR review * Add clarifying comment * Be more specific between polling_interval and submission_latency	3 years ago
Nam Chu Hoai	23c3e4cd22	Add lookback for updates/messages in contract_sync (#144) ... * Apply contract_sync mod patch * Add IndexDataTypes * Fmt * Do not store updates/messages when encountering an invalid continuation If we store them we may end up invariably overwriting the latest_leaf_index database entry which keeps track of "contigousness". I.e. this would cause holes of size >1 to cause the agents being unable to recover * Index both messages and updates on the processor * remove comment * Followups for the contract-sync patch (#146) * Readd abacus_last_known_message_leaf_index to message indexing in the processor * Remove tipbuffer functionality * Additional contract_sync tests (#147) * Fix test case for when batch includes the already known updates/messages * Modify tests to include multiple ranges of missing updates/events	3 years ago
Nam Chu Hoai	e3c1b3bdcc	Support tip buffers for contract_sync (#138) ... * Support tip buffers for contract_sync * Pass IndexSettings * Add tipBuffer to deploy tooling * Run prettier * Deploy on dev	3 years ago
Nam Chu Hoai	d4c8720700	Rename rust packages (#128) ... * Rename rust packages * rustfmt * More rename * Rename update_abis * Rename metrics.rs	3 years ago
Nam Chu Hoai	91a411e2c9	Add gauge to track last_known_message_leaf_index (#117) ... * Add gauge to track last_known_message_leaf_index * Fix lint * Lint * Lint	3 years ago
Nam Chu Hoai	45d9ce9e9b	Repeat processor reverted txs (#48) ... * Repeat processor reverted txs * Adjust error message	3 years ago
nambrot	632cc53ee1	rustfmt	3 years ago
Nam Chu Hoai	8877a98aa8	Support selective index-only for processors (#17) ... * Support selective index-only for processors * Support no indexonly config	3 years ago
Nam Chu Hoai	b6caa0da55	Fix S3 proof pushing (#13) ... * Fix S3 proof pushing * Support AWS in getAgentVars * Add staging output-agent-env-vars	3 years ago
Luke Tchang	40d0ceb253	refactor/feature: split up indexing/contract/db logic + make watcher functional again (#925) ... * refactor: prefixes all loads and stores with home_name or entity in AgentDB * fmt: cargo fmt * refactor: use AgentDB everywhere and scap HomeDB * refactor: rename AgentDB OpticsDB * fix: put watcher back into cargo.toml and fix OpticsDB there * feature: adds ReplicaIndexer and moves index() to Common * refactor: implement CachingHome and CachingReplica + chain agnostic indexer, agent still broken due to not using CachingHome * prog: agent with CachingHome still broken; plan is to provide metrics obj and index-time * fix: contract sync stores messages too * refactor: successfully builds with CachingHome on agent that can spawn sync task * feature: adds unused CachingReplica * refactor: remove contract_name from indexer and give field to contract_sync * refactor: replaces all Replicas usage with CachingReplica * refactor: only CachingHome/Replica reach out to db and implement Event traits * fix: watcher tests work again * fix: removes db references from ethereum home and replica * rebase: rebase big indexer refactor on TypedDB/OpticsDB changes from updater split PR * fix: update crates and silence abigen! warnings for lint * feature: make watcher functional again * rename: spawn_sync --> sync * refactor: fetch_messages returns empty vec by default and home implements method * fix: update watcher-partial config and rename index tasks sync tasks * refactor: HomeIndexer now separate trait than CommonIndexer * doc: document Indexer trait and ContractSync * refactor: Homes/Replicas serve as Arc wrappers * fix: watcher stores update but not dependent on latest root * doc: document watcher fraud behavior * fix: indexers return _sorted_ updates and messages	3 years ago
Luke Tchang	f007a299c8	feature: add metrics/tracking to remaining agents (#922) ... * feature: add num relayed messages metrics/tracking to relayer * refactor: remove processor-specific nonce storage method from optics db * fix: use countervec for relayer instead of guage * feature: updater uses intcounter vec for submitted update count * fix: remove db from relayer since it now uses countervec with no rocksdb * fix: updates relayed not messages oops * fix: update submitter takes the correct countervec	3 years ago
Conner Swann	7b2c4a21df	refactor: split updater tasks for safety (#909) ... * refactor: split updater tasks, refactor opticsdb * feature: asrefs for db types * refactor: refactor typeddb to hold entity and pair with all loads/stores * ci: cargo +nightly * ci: use nightly toolchain * fix: use 1.56 stable not nightly * fix: updater can never overwrite an existing update with a conflicting one * fix: also check suggested update for state change guard Co-authored-by: Luke Tchang <ltchang@stanford.edu>	3 years ago
Luke Tchang	ea32adf3af	refactor: scrap HomeDB for generalized OpticsDB (#902) ... * refactor: comment out homedb and have typeddb use arg prefixes * refactor: prefixes all loads and stores with home_name or entity in AgentDB * fmt: cargo fmt * fix: keyed encodable/decodable * refactor: use AgentDB everywhere and scap HomeDB * refactor: rename AgentDB OpticsDB * fix: put watcher back into cargo.toml and fix OpticsDB there	3 years ago
ember arlynx	a240facc6d	Balance exporter tool skeleton (#793) ... * First pass at balance exporter tool * this was broken with windows unicode paths * promote optics_base to a facade * optics-ethereum has many fewer lines of code * remove a layer of indirection * update all the other code * fmt * post-mege * flatten core::types::\* * impl Chain for ethers * move balance-exporter into tools * post-merge fixes * post-merge resolution * put Default back	3 years ago
Luke Tchang	0e5b04bd1f	fix: increment next_message_nonce for ReplicaProcessor and temporarily fix stuck processor (#879) ... * metrics: increment next_nonce for ReplicaProcessor and add info spans * obsv: info spans in home indexer * fix linter * obsv: add origin chain to message retrieval info span * actually copy clis into release image * FIX: proversync always stores proofs during spawn loop (TEMP FIX) * fix: debug -> info span * tracing: add info span for processing new message Co-authored-by: Conner Swann <me@connerswann.me>	3 years ago
James Prestwich	985bd7850c	feature: configuration to push proofs to s3 buckets (#821) ... * feature: configuration to push proofs to s3 buckets * refactor: key s3 proofs by their index, not their leaf hash * chore: improve processor bootup logging * fix: add name and bucket to debug for Pusher * bug: check that content-length is gt 0 * bug: expect 3 labels in next_message_nonce metric * bug: properly handle non-existent key * fix: log messages for proof pushing * feature: upload message to s3 with proof	3 years ago
Luke Tchang	07bd730d4e	fix: fix processor index/nonce conflation and add per-replica nonce storage (#850) ... * docs/fix: fixes nonce/index conflation in processor and adds kv store docs * fix: rename next_index next_nonce and add new IntGaugeVec for each Replica processor * fix: Replica processors share single int guage vec for next_nonces * fix: change db keys to be prefixed with value indicator + standardize use of leaf instead of leaf_hash * docs: fix out of date docs * fix: rebase on main and fix cli to use leaf instead of leaf_hash * fix: replica processor label matches int gauge vec labels	3 years ago
Conner Swann	6275eaa3f5	fix: fix overloaded index config var (#843)	3 years ago
James Prestwich	eed0b96c51	feature: optionally skip replica tasks (#839) ... * feature: optionally skip replica tasks * refactor: use index_only in processor	3 years ago
Luke Tchang	890c5495bf	fix: fix potential out-of-order updates when `HomeIndexer` syncing (#824) ... * fix: order update events and add race condition TODO * docs: remove race condition comment * doc: clarify edge case where we try to prove leaf with no update yet * fix: proversync check stops at prover.count() * rebase: fix merge conflicts after rebasing on HomeDB merged PR * fix: proversync check stops at prover.count()	3 years ago
Luke Tchang	8e54b713bd	refactor: deduplicate db keys by adding `HomeDB` (#803)	3 years ago
ember arlynx	e23625fbef	feature: record span durations as a metric (#759) ... * record span durations as a metric * kathy is just like the others * show rest of comment its contents were hiding in my brain * -Default	3 years ago
James Prestwich	d413b9f22a	bug: ProverSync from_disk now properly recovers tree state, proofs, etc. (#791) ... * bug: ProverSync from_disk now properly recovers tree state, proofs, etc. * bug: handle proof-out-of-range edge case	3 years ago
James Prestwich	0aba12bb9c	Prestiwch/polygon metrics fix (#783) ... * refactor: move block height tracking to the HomeIndexer, temporarily disable replica block height tracking * chore: enhance updater main comment * refactor: beta reduction	3 years ago
James Prestwich	39b7fc5964	Prover sync refactoring (#781) ... * refactor: display impl for ProverSync * refactor: make prover internal only * refactor: remove non-functional gas escalator * refactor: disable watcher, task management refactor * fixed Docker build Co-authored-by: Conner Swann <me@connerswann.me>	3 years ago
James Prestwich	ce79bb1c5a	refactor: improve db and test roundtrip store/load (#774) ... * refactor: improve db and test roundtrip store/load * refactor: improve tracing messages in processor * refactor: home event getters rely only on db now * feature: more tx info in report_tx tracing	3 years ago
James Prestwich	11d2ec0c5d	prestwich/home indexing (#768) ... * refactor: move db into optics-core, pass db to home instantiation * feature: prototype chain indexer * chore: add todo comment * feature: make chunk_size configurable * refactor: improve DB through out the codebase * bug: use to in indexer spawn * feature: tracing in home indexer * refactor: indexer settings are strings * feature: add setting to disable replica #767 * feature: store latest leaf_index in db * refactor: make fetch_leaf dependent only on DB * feature: add index config to kovan testnet * chore: increase polygon chunk size to 9999	3 years ago
ember arlynx	2767fd921c	feature: additional agent metrics (#746) ... * use a label for replica dimensioning, not a separate metric name * use labels in updater metric for attestation count * don't use non-Vec metrics * track the block height metric * fix typos	3 years ago
James Prestwich	4a86e6301b	chore: remove unused param from prover_sync spawn	3 years ago
James Prestwich	49f0316ee6	bug: updater runs only 1 instance	3 years ago
James Prestwich	060620a99f	bug: fix filters after dispatch event signature changed	3 years ago
James Prestwich	c9a1cd520b	bug: metrics no longer panics when attempting to register multiple replicas (#735)	3 years ago
ember arlynx	d5745362c6	rust agents: expose prometheus metrics (#670) ... * expose prometheus metrics from agents currently not actually updated in the course of execution * cargo update * log when starting metrics server, or not * metricsport -> metrics * metrics ought to be string * set content-type	3 years ago
Anna Carroll	6f5f51d97a	Remove queue from Replica (#703) ... * Remove queue from Replica * Update rust * Lint * Fix test order	3 years ago
James Prestwich	3d72c2db63	refactor: polling_interval -> interval, updater_pause -> pause (#701) ... * refactor: polling_interval -> interval, updater_pause -> pause * refactor: message_interval -> interval * helm updates for interval and pause Co-authored-by: Conner Swann <me@connerswann.me>	3 years ago
Anna Carroll	5ba0bed73a	Final Contract Cleaning + Renaming (#698) ... * Rename Pending to Proven; make case for enums consistent * Replica docs * GovernanceRouter docs * Fix test error message * XAppConnectionManager docs * Remove is_owner from rust * Remove isOwner test * Rename enqueue => dispatch * Update tests to remove enqueue * Rename enqueue in rust * Fix: disapatch * Rename sequences => nonces * Rename sequences => nonces * fix lint * Rename current => committedRoot * Rename leaf => messageHash * Rename endBlock => recoveryActiveAt * Fix test gen	3 years ago
James Prestwich	ff5fa65479	feature: slight rate limit in prover_sync fetch_leaf (#687)	3 years ago
James Prestwich	e5328d3105	bug: remove ancient debug rate limiter in prover_sync (#686)	3 years ago
James Prestwich	0f0cf57edb	feature: add span and debugs to fetch_leaf (#683)	3 years ago
James Prestwich	7492491313	refactor: slightly more verbose processor bootup trace	3 years ago
ember arlynx	22e9c1adc2	refactor processor to have nicer behavioural traces (#593) ... * rust agents: use OTLP for tracing the processor has also been tweaked impl Encode/Decode for u32 * final cleanups * stray comment b gon	3 years ago
Anna Carroll	567631e510	Remove nextToProcess from Replica (#629) ... * Remove nextToProcess from Replica * Get tests passing * package-lock * refactor: remove next_to_process from rust Co-authored-by: James Prestwich <james@prestwi.ch>	3 years ago
ember arlynx	5f041c9db1	drive-by cleanups (#592) ... * reorganize rust code into dirs * rust: ignore vendor dir * how did kathy go missing? * drive-by cleanups: abstraction in update_abis, hyperlink, shell scripts are mostly shellcheck clean * rust readme: note agents dir * cargo fmt --all * remove stray comment * cargo update * why am i fighting my computer rn * patch over sloppy rebase * shore up test vector file handling * clarify how to add a new chain * add rust-toolchain.toml for 1.54 * DRY abis_dir * chore: add clippy installation to CI * github actions will use rust-toolchain.yaml now * move rust-toolchain so the action will find it rustup seems to scan upwards for this file, should pick it up fine when using from within * use obsolete rust-toolchain file location, because the action is naive * chore: specify toolchain in rust.yml * chore: specify toolchain in rust.yml in all workflows * chore: return clippy to rust.yml Co-authored-by: ember arlynx <ember@lunar.software> Co-authored-by: James Prestwich <james@prestwi.ch>	3 years ago