hyperlane-monorepo

Commit Graph

Author	SHA1	Message	Date
Nam Chu Hoai	23c3e4cd22	Add lookback for updates/messages in contract_sync (#144 ) * Apply contract_sync mod patch * Add IndexDataTypes * Fmt * Do not store updates/messages when encountering an invalid continuation If we store them we may end up invariably overwriting the latest_leaf_index database entry which keeps track of "contigousness". I.e. this would cause holes of size >1 to cause the agents being unable to recover * Index both messages and updates on the processor * remove comment * Followups for the contract-sync patch (#146) * Readd abacus_last_known_message_leaf_index to message indexing in the processor * Remove tipbuffer functionality * Additional contract_sync tests (#147) * Fix test case for when batch includes the already known updates/messages * Modify tests to include multiple ranges of missing updates/events	3 years ago
Nam Chu Hoai	e3c1b3bdcc	Support tip buffers for contract_sync (#138 ) * Support tip buffers for contract_sync * Pass IndexSettings * Add tipBuffer to deploy tooling * Run prettier * Deploy on dev	3 years ago
Nam Chu Hoai	d4c8720700	Rename rust packages (#128 ) * Rename rust packages * rustfmt * More rename * Rename update_abis * Rename metrics.rs	3 years ago
Nam Chu Hoai	91a411e2c9	Add gauge to track last_known_message_leaf_index (#117 ) * Add gauge to track last_known_message_leaf_index * Fix lint * Lint * Lint	3 years ago
Luke Tchang	40d0ceb253	refactor/feature: split up indexing/contract/db logic + make watcher functional again (#925 ) * refactor: prefixes all loads and stores with home_name or entity in AgentDB * fmt: cargo fmt * refactor: use AgentDB everywhere and scap HomeDB * refactor: rename AgentDB OpticsDB * fix: put watcher back into cargo.toml and fix OpticsDB there * feature: adds ReplicaIndexer and moves index() to Common * refactor: implement CachingHome and CachingReplica + chain agnostic indexer, agent still broken due to not using CachingHome * prog: agent with CachingHome still broken; plan is to provide metrics obj and index-time * fix: contract sync stores messages too * refactor: successfully builds with CachingHome on agent that can spawn sync task * feature: adds unused CachingReplica * refactor: remove contract_name from indexer and give field to contract_sync * refactor: replaces all Replicas usage with CachingReplica * refactor: only CachingHome/Replica reach out to db and implement Event traits * fix: watcher tests work again * fix: removes db references from ethereum home and replica * rebase: rebase big indexer refactor on TypedDB/OpticsDB changes from updater split PR * fix: update crates and silence abigen! warnings for lint * feature: make watcher functional again * rename: spawn_sync --> sync * refactor: fetch_messages returns empty vec by default and home implements method * fix: update watcher-partial config and rename index tasks sync tasks * refactor: HomeIndexer now separate trait than CommonIndexer * doc: document Indexer trait and ContractSync * refactor: Homes/Replicas serve as Arc wrappers * fix: watcher stores update but not dependent on latest root * doc: document watcher fraud behavior * fix: indexers return _sorted_ updates and messages	3 years ago
Conner Swann	7b2c4a21df	refactor: split updater tasks for safety (#909 ) * refactor: split updater tasks, refactor opticsdb * feature: asrefs for db types * refactor: refactor typeddb to hold entity and pair with all loads/stores * ci: cargo +nightly * ci: use nightly toolchain * fix: use 1.56 stable not nightly * fix: updater can never overwrite an existing update with a conflicting one * fix: also check suggested update for state change guard Co-authored-by: Luke Tchang <ltchang@stanford.edu>	3 years ago
Luke Tchang	ea32adf3af	refactor: scrap HomeDB for generalized OpticsDB (#902 ) * refactor: comment out homedb and have typeddb use arg prefixes * refactor: prefixes all loads and stores with home_name or entity in AgentDB * fmt: cargo fmt * fix: keyed encodable/decodable * refactor: use AgentDB everywhere and scap HomeDB * refactor: rename AgentDB OpticsDB * fix: put watcher back into cargo.toml and fix OpticsDB there	3 years ago
ember arlynx	a240facc6d	Balance exporter tool skeleton (#793 ) * First pass at balance exporter tool * this was broken with windows unicode paths * promote optics_base to a facade * optics-ethereum has many fewer lines of code * remove a layer of indirection * update all the other code * fmt * post-mege * flatten core::types::\* * impl Chain for ethers * move balance-exporter into tools * post-merge fixes * post-merge resolution * put Default back	3 years ago
Luke Tchang	eb685bdbf8	refactor: watcher home indexing rewrite (#809 ) * refactor: add HomeDB struct (raw db not replaced yet) * refactor: EthereumHome and HomeIndexer use HomeDB instead of DB * refactor: UpdateHandler uses HomeDB * refactor: processor/prover-sync use HomeDB * refactor: error cleanup * refactor: implements run_all without metrics * refactor: watcher uses oneshot to receive double update * fix: update comments * fix: panic on run and run_many * tracing: add more info when double update found * rebase: fix store_latest_update rename * test: add default impls to make watcher unit tests compile * fix: run_loop takes HomeDB, watcher struct holds DB * run_watch_tasks borrows &self instead of many params * rebase: fix unused imports	3 years ago
Luke Tchang	890c5495bf	fix: fix potential out-of-order updates when `HomeIndexer` syncing (#824 ) * fix: order update events and add race condition TODO * docs: remove race condition comment * doc: clarify edge case where we try to prove leaf with no update yet * fix: proversync check stops at prover.count() * rebase: fix merge conflicts after rebasing on HomeDB merged PR * fix: proversync check stops at prover.count()	3 years ago
ember arlynx	e23625fbef	feature: record span durations as a metric (#759 ) * record span durations as a metric * kathy is just like the others * show rest of comment its contents were hiding in my brain * -Default	3 years ago
James Prestwich	0aba12bb9c	Prestiwch/polygon metrics fix (#783 ) * refactor: move block height tracking to the HomeIndexer, temporarily disable replica block height tracking * chore: enhance updater main comment * refactor: beta reduction	3 years ago
James Prestwich	11d2ec0c5d	prestwich/home indexing (#768 ) * refactor: move db into optics-core, pass db to home instantiation * feature: prototype chain indexer * chore: add todo comment * feature: make chunk_size configurable * refactor: improve DB through out the codebase * bug: use to in indexer spawn * feature: tracing in home indexer * refactor: indexer settings are strings * feature: add setting to disable replica #767 * feature: store latest leaf_index in db * refactor: make fetch_leaf dependent only on DB * feature: add index config to kovan testnet * chore: increase polygon chunk size to 9999	3 years ago
ember arlynx	d5745362c6	rust agents: expose prometheus metrics (#670 ) * expose prometheus metrics from agents currently not actually updated in the course of execution * cargo update * log when starting metrics server, or not * metricsport -> metrics * metrics ought to be string * set content-type	3 years ago
James Prestwich	3d72c2db63	refactor: polling_interval -> interval, updater_pause -> pause (#701 ) * refactor: polling_interval -> interval, updater_pause -> pause * refactor: message_interval -> interval * helm updates for interval and pause Co-authored-by: Conner Swann <me@connerswann.me>	3 years ago
Anna Carroll	5ba0bed73a	Final Contract Cleaning + Renaming (#698 ) * Rename Pending to Proven; make case for enums consistent * Replica docs * GovernanceRouter docs * Fix test error message * XAppConnectionManager docs * Remove is_owner from rust * Remove isOwner test * Rename enqueue => dispatch * Update tests to remove enqueue * Rename enqueue in rust * Fix: disapatch * Rename sequences => nonces * Rename sequences => nonces * fix lint * Rename current => committedRoot * Rename leaf => messageHash * Rename endBlock => recoveryActiveAt * Fix test gen	3 years ago
ember arlynx	5f041c9db1	drive-by cleanups (#592 ) * reorganize rust code into dirs * rust: ignore vendor dir * how did kathy go missing? * drive-by cleanups: abstraction in update_abis, hyperlink, shell scripts are mostly shellcheck clean * rust readme: note agents dir * cargo fmt --all * remove stray comment * cargo update * why am i fighting my computer rn * patch over sloppy rebase * shore up test vector file handling * clarify how to add a new chain * add rust-toolchain.toml for 1.54 * DRY abis_dir * chore: add clippy installation to CI * github actions will use rust-toolchain.yaml now * move rust-toolchain so the action will find it rustup seems to scan upwards for this file, should pick it up fine when using from within * use obsolete rust-toolchain file location, because the action is naive * chore: specify toolchain in rust.yml * chore: specify toolchain in rust.yml in all workflows * chore: return clippy to rust.yml Co-authored-by: ember arlynx <ember@lunar.software> Co-authored-by: James Prestwich <james@prestwi.ch>	3 years ago
James Prestwich	7b501442b1	refactor: add aws signer (#516 ) * refactor: add aws signer * lint: clippy and fmt * lint: fix unsequenced reads * feature: add a small bin to print key address	3 years ago
James Prestwich	aff52ce2a9	bug: tracing and pending txn fixes - use branch of ethers-rs with pendingtx poll delay - use tracing::Instrumented in most agent trait methods - add in_current_span to most spawns - add an id generator to jaeger tracing	3 years ago
James Prestwich	a37a6ed776	Prestwich/rust config (#419 ) * improve configuration handling of env vars - improve decl_settings macro to name the declared structs - change domains and timers to String - rename chat_gen to chat in kathy settings - panic on failed uint parsing in settings deser * bug: update typescript deploy to output string domains * chore: update new deploy conf to match new format * chore: set kathy destination to 3000 for now * refactor: ensure kathy dispatches messages to each comnfigured replica	3 years ago
James Prestwich	6487fcb3eb	Processing Refactor (#409 ) * refactor: have proversync cache proofs upon tree update * refactor: start introducing acceptableRoot into replica * refactor: set confirmAt to 1 for initial root * refactor: remove previous from replica trait * refactor: processor no longer bails on missing proof * tests: fix the tests * refactor: remove last use of tokio::time::interval * refactor: remove redundant condition in replica	3 years ago
James Prestwich	5489c5f22c	refactor: break signers out of replicas in rust configs (#381 ) * refactor(wip): break signers out of replicas in rust configs * chore: update configs files with new format	4 years ago
Luke Tchang	c4b34fb2df	test: adds mock ConnectionManager interface + watcher failure test cases (#314 )	4 years ago
James Prestwich	7bfe50bd5f	Prestwich/watcher unenroll (#324 ) * feature: adds xapp interface enum to optics-base * feature: adds xapp manager to watcher and has watcher unenroll replicas * refactor: adds Vec<ConnectionManagers> and separates double update handling from unenrollment * fix: watcher-partial config takes vec of ConnectionManager configs * fix: updates comment and adds tracing instrumentation * fix: bad ConnectionManager setup reports error instead of unwrap * fix: concurrently awaits double_update and unenroll tasks * fix: actually awaits two vec of futures concurrently * fix: signed failure notification uses home domain (fixing conflation of replica/home domain) * fix: remove origin_domain after rebase * fix: rename double_update_res double_update * refactor: always use full-length identifier in failure notification Co-authored-by: Luke Tchang <ltchang@stanford.edu>	4 years ago
Erin Hales	676c495668	refactor: rename domains rust (#306 ) * refactor: nuke origin_domain * refactor: rename domain_hash * refactor: rename update local_domain to match solidity * refactor: update local_domain to domain * refactor: rename replica destination_domain * fix: remames origin in home/replica and converts back to domain_hash * fix: rust json testcase output field renames * fix: renames contract domain hash functions homeDomainHash * rename: update.domain --> update.homeDomain * fix: variable names and casing * fix: minor things, delete extra vector file * fix: homeDomainHash test loop * enhance: move common functionality to _signatureHash * fix: rename internal homeDomainHash func * fix: lint * fix: sucky merge * fix: bad rebase * fix: domain field accidentally got deleted * fix: overwrite instead of append for updating precommit script (#311) * fix: rename remaining 'origin' references * fix: use origin for Rust and Solidity formatted messages Co-authored-by: Luke Tchang <ltchang@stanford.edu>	4 years ago
Luke Tchang	94fd095043	refactor: renames UsingOptics contract XappConnectionManager (#258 ) * refactor: renames UsingOptics contract XappConnectionManager * fix: Xapp --> XApp	4 years ago
Luke Tchang	72d6ae20fb	feature: replaces Signer generic with enum (#246 ) * fix: only takes domain and updater * fix: unenrollReplica reverts instead of silent failure and events emitted * feature: adds failure notification structs * fix: update after rebasing on main with watcher disk changes * fix: removes stale db_paths from updater and watcher * test: adds test to verify watcher SignedFailureNotifications in Solidity * refactor: adds and uses Addresses enum to support 20 and 32 byte addresses * refactor: make an OpticsIdentifier type * feature: adds signer enum variants * feature: fixes signer enum and adds to updater and watcher * fix: adds SignersError enum to accomodate LedgerError * fix: removes Addresses enum after rebase * fix: deletes duplicate impls after rebase * fix: uses ? instead of map_err Co-authored-by: James Prestwich <prestwich@clabs.co>	4 years ago
Luke Tchang	dc52d01882	feature: adds watcher support for UsingOptics unenrollment (#229 ) * fix: only takes domain and updater * fix: unenrollReplica reverts instead of silent failure and events emitted * feature: adds failure notification structs * feature: adds signer to watcher and produces signed failure notifications on fraud * fix: update after rebasing on main with watcher disk changes * fix: removes stale db_paths from updater and watcher * prog: adds test function in optics-core to produce signed updates for common test suite (broken) * fix: renames previousRoot oldRoot and fixes originDomain to match Common tests * test: adds test to verify watcher SignedFailureNotifications in Solidity * refactor: adds and uses Addresses enum to support 20 and 32 byte addresses * refactor: make an OpticsIdentifier type * feature: add function to cast identifier to ethereum * fix: fixes replica deployment in UsingOptics tests Co-authored-by: James Prestwich <prestwich@clabs.co>	4 years ago
James Prestwich	1174800624	refactor: move DB to agentcore (#228 ) - add db_path to settings for all agents - refactor updater poll_and_handle_update to use a mutex guard - add db() method to optics agent trait	4 years ago
Luke Tchang	d12c50845f	feature: persists watcher history to disk (#192 ) * feature: adds disk persistence to updater and checks for conflicting updates * feature: gets db_path from settings block * feature: adds open_db util and replaces in updater * test: adds test_util func to run db tests in closure * feature: adds serde * feature: proversync saves new leaves to disk * feature: adds loading prover from disk * rebase: removes duplication after rebase on updater-history branch * feature: proversync saves new leaves to disk * feature: adds loading prover from disk * feature: replaces hashmap with db * rebase: removes duplicated import * test: uses run_test util for watcher tests * fix: replaces bincode with encode/decode * fix: fix after rebase on prover persistence PR * fix: removes unecessary serialize/deserialize	4 years ago
Luke Tchang	5a651d7eb0	test: adds watcher tests (#154 ) * refactor: separates prover and prover_sync; all agents compiling except watcher * refactor: adds EthereumReplica variant and fixes agents * refactor: adds MockReplica variant to Replicas * fmt: runs cargo fmt * test: adds updater poll_and_handle_update test * test: adds polls_and_relays_update test to relayer * fix: removes duplicated additions after rebase * test: adds updater poll_and_handle_update test * fix: removes duplicated additions after rebase * refactor: breaks apart watcher component loops * test: adds contract watcher poll and send update test * test: adds checkpoint method to Homes and Replicas for testing * test: adds history sync update history test	4 years ago
Luke Tchang	9cae2f469a	refactor: contract trait refactor + prover/prover_sync separation (#151 ) * prog: adds Homes type to optics-base and has AgentCore take in Arc<Homes> * refactor: separates prover and prover_sync; all agents compiling except watcher * refactor: adds EthereumReplica variant and fixes agents * refactor: adds MockReplica variant to Replicas * fmt: runs cargo fmt * feature: adds From implementations for Homes and Replicas * feature: adds From<EthereumHome> for Homes * feature: adds From<EthereumReplica> for Replicas * feature: adds Other variant to Homes and Replicas * lint: use Box in all homes and replicas to satisfy clippy * bug: fix Ethereum handling in ChainSetup::try_into_ functions Co-authored-by: James Prestwich <james@prestwi.ch>	4 years ago
James Prestwich	1f333b6c02	refactor: improve processor task handling (#147 ) * refactor: improve processor task handling * refactor: change OpticsAgent::run to return a join handle * refactor: change default behavior for Agent to cancel all tasks instead of continue	4 years ago
Luke Tchang	01b6aa43ea	test: adds optics-test crate and unit test for watcher::check_double_update (#138 ) * refactor: extracts double update check to helper * prog: mockall derive debug on Common and Home broken * feature: MockHomeContract in optics-base * test: adds test for watcher UpdateHandler check_double_update * feature: adds replica mock contract * refactor: moves mocks to new optics-test crate * fix: satisfy clippy gods * fix: removes dev dependencies from optics-base * fix: fixes optics-test docstring and unecessary assert_eq Co-authored-by: James Prestwich <prestwich@clabs.co>	4 years ago
James Prestwich	ae81ba9f60	docs: add message layout to bridge readme (#127 ) * docs: add message layout to bridge readme * docs: fix a copy+paste in the Bridge Readme	4 years ago
James Prestwich	01144db102	lint: satisfy clippy a bit more	4 years ago
James Prestwich	b7280e677d	refactor: improve the OpticsAgent interface by introducing AgentCore (#104 ) * refactor: improve the OpticsAgent interface by introducing AgentCore * refactor: normalize configs and add name key to ChainConfs * bug: fix list of tables in updater toml config * refactor: add an output type to the agent trait for wathcer double updates * docs: add a few comments to the watcher loop * refactor: make watcher handling of missing updates explicit * feature: watcher syncs history backwards * refactor: split watcher responsibilities into smaller message-passing structs * feature: implement a shutdown function for the Watcher * refactor: DRY up task cancellation * chore: add tracing to more watcher methods * refactor: dedup and move agent macros to a shared location * chore: fix an ugly import * refactor: use macros to DRY up agent and settings declarations * feature: resolve potential race condition by explicit from hash in watcher syncs	4 years ago
Luke Tchang	0d5fbbe23b	feature/WIP: watcher checks for fraudulent updates (#88 ) * feature: separates watch into watch_replica and watch_home and checks for fraudulent updates for replica * docs: adds documentation for watcher behavior * todo: adds todo! macro * docs: updates watcher description to include fraudulent updates * fix: has watcher check for fraudulent updates by submitting to home * refactor: has check_double_update return error and handles in calling scope * refactor: improve watcher reporting of double updates Co-authored-by: James Prestwich <prestwich@clabs.co>	4 years ago
Luke Tchang	c57b9ded00	feature: adds untested Watcher daemon (#73 ) * enhance: adds Update event emission to Replica and signed_update_by_old root to Replica abi * feature: adds core run logic for watcher observing single replica * feature: adds watch_home logic and run_many override * refactor: moves signed_update_by_old_root to common * refactor: uses Common enum for Home/Replica * refactor: removes enum and uses ?Sized to remove Sized constraint * fix: removes unnecessary import * fix: runs cargo fmt and replaces references to origin_slip44 with domain * fix: fixes deadlock setup for history read/write * fix: fixes poll_signed_update to use signed_update_by_new_root * fix: fixes off-by-one error for topics for signed_update abi methods * docs: fixes comment formatting on signed_update methods	4 years ago

17 Commits (b23d24c0a6eaf7cdb297619801e73f065fac9196)