10 Commits (33a77c35e14c61dbd09f628baf87db4f3512320b)

Author	SHA1	Message	Date
Mattie Conover	03d32f4009	Eliminate trivial constructors (#1847)	2 years ago
Mattie Conover	1b45f8cad1	Rust Updates (#1845) ... * Update rust lang version * Fix new clippy lint * Update packages * Update rocksdb * Updates	2 years ago
Mattie Conover	8105913dea	Improve agent logging (#1797)	2 years ago
Mattie Conover	4b7e7ca9bb	Improve config errors (#1776) ... * Remove path to error as it was not working * Better domain parsing errors * Locked min version of config lib * Improve error messages for config loading	2 years ago
Mattie Conover	e5e794eda4	Fuel Validators (#1474) ... * Fuel ABI gen * WIP * Progress * Use HyperlaneDomain * Progress on fuel init * Support unknown domains * Work on handling wallets and signers for fuel * Move signer output test * Work on signers * Fixing up compilation * Update to use HyperlaneDomain * Fixes * WIP * Update Mailbox ABI * Move signer to be in chain conf and change rpc style to implementation * Better loader debugging * WIP * Merge fixes and refactor * Fix "rpcStyle" after merge * Simplify chain mappings * Fix tests * Remove excess runtime deps from validator * Use agent core in scraper and remove DB assumptions * More core cleanup * Remove duplication for checkpoint syncers * Better error messages for debugging * Debugging * Cleanup * remove signers * fmt * Fix after merge * Fixes after merge * Fixes after merge * remove random inclusion * Remove unused files * Requested changes from PR * Simplify deps and update sha3 * Fix after merge	2 years ago
Mattie Conover	f6aa1f6039	Fallback provider (#1571) ... * Created fallback provider * support fallback provider * WIP testing * Fix testing * Use http provider so we can better inspect errors * Undo change * Fix comment	2 years ago
Mattie Conover	fcba422718	Better errors in core (#1404) ... * Better errors * Remove unused error types * fmt * Better error when object is not found from a provider	2 years ago
Mattie Conover	f814286998	Update packages (#1385) ... * Update ethers * Update packages * Fix build error	2 years ago
Mattie Conover	e8ed5f88ce	Deploy scraper for testnet3 (#1374) ... * How did those get there * abacus -> hyperlane * remove deprecated domains * Deleted dead code * Update sea-orm * Simplify settings post v2 changes * Split up settings * Update contexts * Fix scraper provider address regression during merge * Updated scraper deployment configs	2 years ago
Asa Oines	f15632f26c	Finalize rename from abacus->hyperlane (#1312) ... * Rename abacus->hyperlane in agents * Rename in typescript * Rename abacus->hyperlane in dockerfile	2 years ago
Mattie Conover	0ba82d1918	Proper RPC Useage fix (#1282) ... * Dedupe indexer range calculations * Better handling of tip updates * Don't use to block for interchain gas indexing * Rename to SyncBlockRangeCursor * use elapsed * Fix init case for debug mode * Work on mocking for outbox indexer test * Fix tests * Use from instead of start for interchain gas indexing db cursor * warn on failure to get block range from cursor * Fixed typo * Document test sleep in outbox	2 years ago
Mattie Conover	b4d06f9639	Message Explorer Inbox Scraping (#1159) ... * Better hex processing * Cleanup clippy errors * Fix parsing of 160 bit values * Support multiple contracts with cursors * InboxIndexer for getting message processed events * Work on inbox indexing and delivered message linking * Use message hashes to link deliveries and also add hash indexes for common lookups * split up settings struct * Support scraper config for main fn * Multiple config parsing for scraper * Better docs and more reliant config loading * Refactoring building of contract items * Cut down on duplication * Support inbox indexer building * fmt * Scraper settings loading re-work * Fix errors, warnings, and cleanup * Fix clippy warnings * Fix hash index not supporting uniqueness * Fix after entity update * Remove special case * Better error messages for missing config values * Env prefix filtering * Improve error context for url without base error * Fix inbox domain bug * Actually I was wrong about domains, misinterpreted config * Fix loading settings method * Fix upsert error * Debugged and fixed incorrect txn linking with messages and deliveries * Cleanup * Update config for scraper * Update config for scraper * Delivered message linking; again. * Reduce call frequency of message linking query * Fixed typo * Fix init mistake * changed to agent_prefix * Domain settings * Use a loop * Add more docs * Remove semicolen * Increase linking frequency * Add moonbeam domain * Fmt * Renamed settings struct * Add docs * Add test for FROM_HEX_CHARS array	2 years ago
Trevor Porter	8f656d1f34	Update ethers-rs to include QuorumProvider fix for methods like eth_estimateGas (#1201)	2 years ago
Mattie Conover	fe904f22cc	Runnable Scraper (#1140) ... * Fix metric ordering (#1043) * Remove redunant enums * WIP * block cursor and date time * add chrono * add mods * Work on scraper logic * Init plumbing and config parsing * Stub out inbox loading * Update metric type * Refactoring out weird opt struct * Minor refactoring * Sync logic for scraper outbox reading messages * Fix param names * Stubs and docs to lay out next steps * Add way to get a contract address * read latest message index and work on storing messages * Ported LogMeta and refactored CheckpointMeta out + moved RawCommittedMessage * Remove redundant CheckpointWithMeta struct * Update outbox indexer to include metadata when fetching messages * WIP storing messages * Work on processing blocks and txns * Work on processing blocks and txns * weave data through for message insertion * cleanup * Scraper config * Support multiple index setting configs * Fix panics loading config * Fix db generation * Fix loading wrong outbox from config * Removed deprecated testnets from scraper configs * Fix outbox loading using wrong outbox config * Refactoring out unused index config from ethereum outbox indexer * Fix `todo` panic * Fixed handling of block/txn fetching/storing * allow disabling scraper outboxes * Fix elusive unwrap error * Cleanup and fix `ensure_blocks_and_txns` * Logging that apparently does nothing rn * Refactoring to support initializing tracing before general agent initialization. * Simplify agent main fns * Include deprecated domains * regen entities * Update index names to be more consistent * Remove destination fkey for messages as it does not have to be a valid id * Better logging * remove constraint on block height for the moment * Fix reverse id issue * Fix initial cursor insertion * fix test * appease clippy for now * Get scraper bin in Dockerfile too * append only cursors * append only cursor fix * Fix handling of empty insertion * Reduce query load * Scraper config for mainnet * Remove metrics fn from agent definition * Remove unused checkpoint fn * fix idx name * Order by height instead of id for cursors * Don't fetch message leaf index for every tick * Better var name * Add comment * Removed dead code stub * Better docs * Updated param name * Comment for pr * Added comment * Added comment * Removed unused code * Add doc on domains * Add comment. Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	2 years ago
Mattie Conover	c37aa61b91	Update ethers (#1058) ... * Update run-locally to test quorum provider better * Update ethers-rs	2 years ago
Mattie Conover	1a7e74ea7c	Update ethers (#1037)	2 years ago
Trevor Porter	000b67cf16	Upgrade ethers-rs to fix Optimsim gas estimation (#1023)	2 years ago
Mattie Conover	d316ae68d5	update ethers-rs (#1004)	2 years ago
Mattie Conover	333b18ecfe	Quorum provider (#934) ... * Add quorum provider support * Update Rust * Pull Http logic out of retrying provider * first pass at retrying provider for quorum provider * change weight parsing * Better logging and handling for quorum retry logic * Better logging * Appease new clippy rules * Update packages in vein attempt to make things work * quorum provider testing * switch to our version of ethers-rs * Update config * Support quorum provider * Use quorum provider for testing * Update packages * Update	2 years ago
webbhorn	ae03a7343b	Fix broken test `contract_sync::outbox::test::handles_missing_rpc_messages`. (#811) ... Before this PR, the test will panic due to outbox syncer running in a concurrent tokio task after the test completes and invoking mocked functions without expectations set. The panic is also swallowed by a separate bug and is only visible if tests are run under `cargo test -- --show-output`. After this PR, we no longer see a backtrace / panic when running with `cargo test -- --show-output`. We also correctly mark the test as failed in `cargo test` if we introduce a new panic in the test or DB harness code. Some additional changes to try and improve things adjacent to this test: * use tmpfs instead of disk + panic-unwind-detection for the RocksDB contents to avoid the panic detection complexity * avoid the current correctness dependency on sleep(from_secs(5)) in the test. This is inherently racy and it is not much harder or more complex to just select() with a timeout	2 years ago
Mattie Conover	de20710466	Rust updates (#644) ... * Update config and trim down macro code * Update rust * Package updates * Updates and cleanup * make clippy happy	2 years ago
Mattie Conover	aa92873088	Oneline Backtraces (#627) ... * add backtrace lib * Pull over code for printing backtraces * Oneline backtraces * code shuffle * fmt * remove stale comment * cleanup * refactoring	2 years ago
Mattie Conover	d687d3e230	More Metrics (#495) ... * metrics refactoring * metrics refactoring * Make brain hurt less by using a damn function * patch through metrics to abacus_ethereum inbox and outbox * WIP * Minor cleanup * work on time update * added label arrays * refactoring * Metrics for sending txns * Metrics middleware * cleanup * Cleanup after merge * Add metrics config and setup prom metrics with core metrics * Minor fixes to docs * Better trait building * rename * Weave metrics through code * Use core metrics instead * Removed some unused code * fmt * change feature layout * fix docker build * Start scraping metrics on an interval * fmt * fix duplicate label error * change features * fix u256 conversion fn * refactor * Support default values in config * Minor fixes * Use f64 conversion from primitive-types * remove unused code * fix const name * improve docs * better var name * remove unused code * remove unused code	3 years ago
webbhorn	6b3822f664	Misc. agent cleanup and dead code deletion (#470) ... * Fix path to agent::Agent in comment. Otherwise `cargo doc` emits a rustdoc::broken_intra_doc_links warning. * Use automatic link in comment for rustdoc. Otherwise 'cargo doc' emits a warning. * Delete checkpointer agent. It is not used in current deployment. * Delete kms-cli. Not used in current deployment (no references). * Include "--all" in precommit cargo fmt -- --check invocation. * Delete unused and hidden src/bin programs. These appear to have been introduced over a year ago and appear to be dead code now. Unhide them from vscode workspace too -- since they were hidden, automated refactoring tools wouldn't find them until compiler failed. * Update github CODEOWNERS to current Abacus team. Drop optional authors field from Cargo.tomls. * Drop processordb from .gitignore. Processor crate has been deleted. * Delete balance-exporter crate. It is not used / no references. * Hoist abacus-cli into its own top-level crate. It is the only tool now, no reason to hide it inside of a tools directory. * Delete old relaese.sh, which only relates to Celo Optics. * Cleanup top-level rust README.md. - Architecture paragraphs outdated, delete. - Point to run-locally.sh instead of documentation. * Explain a bit more about crate deps in README. * Dockerfile fix after tools directory restructuring. We had previously hoisted abacus-cli out since it was the only remaining tool. * Fix path to agent::Agent in comment. Otherwise `cargo doc` emits a rustdoc::broken_intra_doc_links warning. * Use automatic link in coment for rustdoc. Otherwise 'cargo doc' emits a warning. * Delete checkpointer agent. It is not used in current deployment. * Delete kms-cli. Not used in current deployment (no references). * Include "--all" in precommit cargo fmt -- --check invocation. * Delete unused and hidden src/bin programs. These appear to have been introduced over a year ago and appear to be dead code now. Unhide them from vscode workspace too -- since they were hidden, automated refactoring tools wouldn't find them until compiler failed. * Update github CODEOWNERS to current Abacus team. Drop optional authors field from Cargo.tomls. * Drop processordb from .gitignore. Processor crate has been deleted. * Delete balance-exporter crate. It is not used / no references. * Hoist abacus-cli into its own top-level crate. It is the only tool now, no reason to hide it inside of a tools directory. * Delete old relaese.sh, which only relates to Celo Optics. * Cleanup top-level rust README.md. - Architecture paragraphs outdated, delete. - Point to run-locally.sh instead of documentation. * Explain a bit more about crate deps in README. * Dockerfile fix after tools directory restructuring. We had previously hoisted abacus-cli out since it was the only remaining tool.	3 years ago
Mattie Conover	f78e79ab9b	Better Agent Error Logging (#412) ... * Make errors print on one line * Replace color-eyre with eyre in core and base * Replace color-eyre with eyre other libs * Log should be imported without patch version * Update tracing and opentelemetry * Only need openssl dynamic libs * Fix no-default-feature build * Custom eyre handler * Removed unused log dep * Cleanup paste dep * More package cleanup * Update tokio * Minor refactor	3 years ago
Nam Chu Hoai	9ab0c16dd6	Add CheckpointSyncer configuration in Validator and support S3 (#266) ... * Add CheckpointSyncer configuration in Validator and support S3 * Lint * PR review	3 years ago
Nam Chu Hoai	4a28875af8	Add Validator scaffold (#213)	3 years ago
Nam Chu Hoai	23c3e4cd22	Add lookback for updates/messages in contract_sync (#144) ... * Apply contract_sync mod patch * Add IndexDataTypes * Fmt * Do not store updates/messages when encountering an invalid continuation If we store them we may end up invariably overwriting the latest_leaf_index database entry which keeps track of "contigousness". I.e. this would cause holes of size >1 to cause the agents being unable to recover * Index both messages and updates on the processor * remove comment * Followups for the contract-sync patch (#146) * Readd abacus_last_known_message_leaf_index to message indexing in the processor * Remove tipbuffer functionality * Additional contract_sync tests (#147) * Fix test case for when batch includes the already known updates/messages * Modify tests to include multiple ranges of missing updates/events	3 years ago
Nam Chu Hoai	d4c8720700	Rename rust packages (#128) ... * Rename rust packages * rustfmt * More rename * Rename update_abis * Rename metrics.rs	3 years ago
Conner Swann	7b2c4a21df	refactor: split updater tasks for safety (#909) ... * refactor: split updater tasks, refactor opticsdb * feature: asrefs for db types * refactor: refactor typeddb to hold entity and pair with all loads/stores * ci: cargo +nightly * ci: use nightly toolchain * fix: use 1.56 stable not nightly * fix: updater can never overwrite an existing update with a conflicting one * fix: also check suggested update for state change guard Co-authored-by: Luke Tchang <ltchang@stanford.edu>	3 years ago
ember arlynx	d5745362c6	rust agents: expose prometheus metrics (#670) ... * expose prometheus metrics from agents currently not actually updated in the course of execution * cargo update * log when starting metrics server, or not * metricsport -> metrics * metrics ought to be string * set content-type	3 years ago
ember arlynx	5f041c9db1	drive-by cleanups (#592) ... * reorganize rust code into dirs * rust: ignore vendor dir * how did kathy go missing? * drive-by cleanups: abstraction in update_abis, hyperlink, shell scripts are mostly shellcheck clean * rust readme: note agents dir * cargo fmt --all * remove stray comment * cargo update * why am i fighting my computer rn * patch over sloppy rebase * shore up test vector file handling * clarify how to add a new chain * add rust-toolchain.toml for 1.54 * DRY abis_dir * chore: add clippy installation to CI * github actions will use rust-toolchain.yaml now * move rust-toolchain so the action will find it rustup seems to scan upwards for this file, should pick it up fine when using from within * use obsolete rust-toolchain file location, because the action is naive * chore: specify toolchain in rust.yml * chore: specify toolchain in rust.yml in all workflows * chore: return clippy to rust.yml Co-authored-by: ember arlynx <ember@lunar.software> Co-authored-by: James Prestwich <james@prestwi.ch>	3 years ago
James Prestwich	13644d5957	KMS CLI (#547) ... * feature: sweep bin * refactor: hardcode gas req for simple sweep tx * refactor: update to latest version of ethers * feature: kms cli * bug: short designator conflict + docs * bug: fix value parsing * chore: temp fix for simple sends * refactor: delete deprecated kms bins, roll features into kms-cli	3 years ago
James Prestwich	7b501442b1	refactor: add aws signer (#516) ... * refactor: add aws signer * lint: clippy and fmt * lint: fix unsequenced reads * feature: add a small bin to print key address	3 years ago
James Prestwich	a38f4f5153	chore: bump ethers to main in all cargo.toml files	3 years ago
James Prestwich	aff52ce2a9	bug: tracing and pending txn fixes ... - use branch of ethers-rs with pendingtx poll delay - use tracing::Instrumented in most agent trait methods - add in_current_span to most spawns - add an id generator to jaeger tracing	3 years ago
James Prestwich	677f68c1e1	feature: zipkin	3 years ago
James Prestwich	5722a698f2	Refactor tracing subscriber instatiation (#466) ... * WIP(broken): start breaking out tracing config * WIP: custom abstraction over log formats * WIP(feat): clean jaeger instantiation in Subscriber layers * refactor: improve jaeger layer instantiation and application. Add filter layer * lint: self by value on copy type * feature: implement Jaeger configuration * bug: specify reqwest collector client and remove doc fragment * refactor: use std convert traits where possible * refactor: introduce DroppedError	3 years ago
James Prestwich	a37a6ed776	Prestwich/rust config (#419) ... * improve configuration handling of env vars - improve decl_settings macro to name the declared structs - change domains and timers to String - rename chat_gen to chat in kathy settings - panic on failed uint parsing in settings deser * bug: update typescript deploy to output string domains * chore: update new deploy conf to match new format * chore: set kathy destination to 3000 for now * refactor: ensure kathy dispatches messages to each comnfigured replica	3 years ago
Luke Tchang	73b989285e	feature/refactor: saves/loads processor prover-tree to/from disk + fixes abstraction layers between prover and merkle (#189) ... * refactor: joins two chain api calls and spawns task for wait and check * feature: adds disk persistence to updater and checks for conflicting updates * feature: gets db_path from settings block * feature: adds open_db util and replaces in updater * feature: adds encode/decode to optics-core lib structs * test: adds test_util func to run db tests in closure * feature: adds serde * feature: proversync saves new leaves to disk * feature: adds loading prover from disk * fix: replaces ne_bytes to be_bytes * rebase: removes duplication after rebase on updater-history branch * fix: removes bincode * refactor: fixes abstraction layers between prover and merkle variants * refactor: uses prefix_iterator in prover * fix: has rocksdb imports draw from github * refactor: adds UsingPersistence trait and adds to updater, prover, proversync * refactor: update UsingPersistence to use Encode + Decode * feature: iterator in UsingPersistence Co-authored-by: James Prestwich <james@prestwi.ch>	4 years ago
Luke Tchang	b9694211f4	feature: adds updater history and persists to disk (#180) ... * refactor: joins two chain api calls and spawns task for wait and check * refactor: poll_and_handle_update returns join handle * feature: adds disk persistence to updater and checks for conflicting updates * feature: gets db_path from settings block * feature: adds open_db util and replaces in updater * feature: adds encode/decode to optics-core lib structs * test: adds test_util func to run db tests in closure * fix: uses Encode::to_vec	4 years ago
Luke Tchang	9cae2f469a	refactor: contract trait refactor + prover/prover_sync separation (#151) ... * prog: adds Homes type to optics-base and has AgentCore take in Arc<Homes> * refactor: separates prover and prover_sync; all agents compiling except watcher * refactor: adds EthereumReplica variant and fixes agents * refactor: adds MockReplica variant to Replicas * fmt: runs cargo fmt * feature: adds From implementations for Homes and Replicas * feature: adds From<EthereumHome> for Homes * feature: adds From<EthereumReplica> for Replicas * feature: adds Other variant to Homes and Replicas * lint: use Box in all homes and replicas to satisfy clippy * bug: fix Ethereum handling in ChainSetup::try_into_ functions Co-authored-by: James Prestwich <james@prestwi.ch>	4 years ago
Erin Hales	5d3668495a	enhancement: move abi files to shared folder (#94) ... * moved abis to shared folder * basic (surely flawed) optics ethereum package * bug: fix circular imports, rearrange files and paths, lint * fix: unused import * fix: serde Deserialize * fix test issues, delete settings * enhancement: separated out home and replica abis * fix: pub use statements * cargo update and lint * fix: doc code snippet, trim dependencies	4 years ago
Luke Tchang	319ebd4369	feature: adds untested Relayer daemon (fixed history) (#55) ... * refactor: fixed camel casing and simplified canConfirm function in Replica.sol * fix: relayer now runs from settings * refactor: split polling updates and polling confirms into joined helper functions # Conflicts: # rust/relayer/src/relayer.rs * feature: adds tracing subscriber to example and relayer::main.rs * feature: adds tracing errors and instrument(err) to relayer * fix: removes duplicates after rebase # Conflicts: # rust/optics-base/Cargo.toml * fix: added relayer dependencies to Cargo.lock (cherry-pick)	4 years ago
James Prestwich	9951a90c29	refactor: allow ethereum node to be a signer	4 years ago
James Prestwich	4a57874d8c	feature: adds an untested updater daemon (#19) ... * feature: adds an untested updater daemon * feature: check that the updater is using the correct keys before polling for updates * refactor: simplify OpticsAgent::run_report_error * chore: improve documentation and remove unnecessary map_errs * bug: correct env var prefix in updater settings	4 years ago
James Prestwich	880273a123	feature: adds tracing and a basic config for the tracing setup (#31) ... * feature: adds tracing and a basic config for the tracing setup * refactor: change some settings tags to camelCase for consistency * fix: correct logger level match	4 years ago
James Prestwich	ca7e773db8	refactor: simplify dependencies	4 years ago
James Prestwich	fc63fed64c	refactor: adds a bit more logging and error reporting to OpticsAgent	4 years ago
James Prestwich	5ea330cfbb	feature: OpticsAgent trait and refactoring contract traits	4 years ago