hyperlane-monorepo

Commit Graph

Author	SHA1	Message	Date
Yorke Rhodes	50f04db1fa	Make merkle proofs optional on multisig ISM (#2173 ) ### Description Validators currently sign `(root, index)` checkpoints and during verification, a `message` is passed as calldata, an `id()` is derived, and a `proof` of `id()` at `index` in `root` is verified This provides “all or nothing” censorship resistance guarantees because a validator can only sign roots to allow any contained messages to be processed. We have considered alternatives where validators sign `message` directly and we lose censorship resistance in exchange for eliminating merkle proof verification gas costs. However, if validators sign `(root, index, message)` tuples, we can skip merkle proof verification on the destination chain while still maintaining censorship resistance by providing two valid metadata formats: 1. existing validator signatures and merkle proof verification of inclusion 2. including merkle proof verification for pathway where validators are censoring `message` It’s worth noting the validator is required to index event data to produce this new signature format. However, this does not require historical indexing and new validators being spun up can simply begin indexing from tip. See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2187 for validator changes See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2248 for relayer and e2e test changes ### Drive-by changes Merkle index also optional ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2192 ### Backward compatibility - new ISM deployment is necessary (we could upgrade implementation in theory) - Validator and relayer upgrades ### Testing Unit (fuzz) Tests, E2E tests	2 years ago
Mattie Conover	b56b022f88	Hyperlane-Core WASM support (#2110 ) ### Description This shuffles some code around and changes dependencies to make `hyperlane-core` WASM compatible and adds a check to the pipeline to ensure it stays that way. To do this it does refactor hyperlane db to the `base` crate, which makes more sense IMO anyway as that is an implementation detail of the agents and not part of the core protocol. ### Drive-by changes ### Related issues - Fixes #1977 ### Backward compatibility _Are these changes backward compatible?_ Yes-ish - It does change where things are but it does not change support. Should only impact people who directly use the libraries. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Mattie Conover	1722f4ee7e	Better Agent Configuration Parsing (#2070 ) ### Description This is a significant change to how we parse user configs. The main goal of this PR is to give a clear path and env key to any errors that are encountered when parsing configuration. In order to accomplish this, I have crated a new trait `FromRawConf` which defines a set of steps to validate and convert individual fields. Also for every major config struct, there is now a `Raw` variant of it which is used by the `config` library to reduce errors that occur during deserialization itself. `Raw` types should always use the most basic form of a value, such as `String`, `StrOrInt`, `bool`, and ideally optional in all cases. Missing values should be handled during the raw conversion and not by `config`. I also needed to make changes to a number of types stored in the parsed config types to force validation forward to the parsing step instead of doing it when we read the config value. This also required me to create a filter to prevent trying to validate certain configs that we would not ever need to load. These changes can also be built on later to support something other than `config` if we choose to, or add support for merging configs from multiple sources since everything is optional. ### Drive-by changes - Default to `http` for connection type - Default to no gas payment enforcement if not specified instead of failing - `ChainSetup` -> `ChainConf` - `GasPaymentEnforcementConfig` -> `GasPaymentEnforcementConf` - Made ethereum connection configuration more forgiving - Moved hyperlane base settings from `mod.rs` to `base.rs` - Moved config chain tests in hyperlane core to `tests` dir to fix a cyclical import problem - Extension traits to help with config in hyperlane core - Moved `StrOrInt` to new hyperlane core `config` module - Support for parsing `U256` from `StrOrInt` - Removed `HexString` type which is now redundant - Updated base settings to use hyperlane domain - Use `heyKey` as signer type if `type` is not specified and `key` is - Moved hyperlane ethereum chain config to a new module ### Related issues - Fixes #2033 - Fixes #2012 ### Backward compatibility _Are these changes backward compatible?_ Yes - This should take in configs in the same shape but be more forgiving in a few places _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Test	2 years ago
Asa Oines	f25feecfa7	Scaffolding for multiple ISM types in the relayer (#2028 ) ### Description This PR adds scaffolding in the relayer to support building metadata for multiple ISM types. To add a new ISM type: - Add a new implementation of MetadataBuilder for that type - Add the type to the SupportedIsmTypes enum - Add the type to the match statement in BaseMetadataBuilder ### Drive-by changes None ### Related issues - #1757 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ E2E tests	2 years ago
Mattie Conover	4657fcc737	Better rust package management (#1975 ) ### Description At the suggestion of @tkporter this moves common dependencies and configuration to use the new(ish) cargo workspace configuration feature. This was added in July 2022 and allows us to ensure all versions and configurations are the same across different crates. It does not decrease the typing, but it does reduce the updates required when we go to update a package and keeps duplicates next to each other. This PR does not change any package versions or add/remove any packages. It only moves the configuration. ### Drive-by changes Also sorted packages by name. ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual	2 years ago
Mattie Conover	03d32f4009	Eliminate trivial constructors (#1847 )	2 years ago
Mattie Conover	1b45f8cad1	Rust Updates (#1845 ) * Update rust lang version * Fix new clippy lint * Update packages * Update rocksdb * Updates	2 years ago
Mattie Conover	8105913dea	Improve agent logging (#1797 )	2 years ago
Mattie Conover	4b7e7ca9bb	Improve config errors (#1776 ) * Remove path to error as it was not working * Better domain parsing errors * Locked min version of config lib * Improve error messages for config loading	2 years ago
Mattie Conover	e5e794eda4	Fuel Validators (#1474 ) * Fuel ABI gen * WIP * Progress * Use HyperlaneDomain * Progress on fuel init * Support unknown domains * Work on handling wallets and signers for fuel * Move signer output test * Work on signers * Fixing up compilation * Update to use HyperlaneDomain * Fixes * WIP * Update Mailbox ABI * Move signer to be in chain conf and change rpc style to implementation * Better loader debugging * WIP * Merge fixes and refactor * Fix "rpcStyle" after merge * Simplify chain mappings * Fix tests * Remove excess runtime deps from validator * Use agent core in scraper and remove DB assumptions * More core cleanup * Remove duplication for checkpoint syncers * Better error messages for debugging * Debugging * Cleanup * remove signers * fmt * Fix after merge * Fixes after merge * Fixes after merge * remove random inclusion * Remove unused files * Requested changes from PR * Simplify deps and update sha3 * Fix after merge	2 years ago
Mattie Conover	f6aa1f6039	Fallback provider (#1571 ) * Created fallback provider * support fallback provider * WIP testing * Fix testing * Use http provider so we can better inspect errors * Undo change * Fix comment	2 years ago
Mattie Conover	fcba422718	Better errors in core (#1404 ) * Better errors * Remove unused error types * fmt * Better error when object is not found from a provider	2 years ago
Mattie Conover	39a24fd2ea	Use primitive types instead of ethers (#1387 ) * ethereum Connection -> ConnectionConf * Use hyperlane_core for primitives * Fixes after merge * fmt	2 years ago
Mattie Conover	f814286998	Update packages (#1385 ) * Update ethers * Update packages * Fix build error	2 years ago
Mattie Conover	e8ed5f88ce	Deploy scraper for testnet3 (#1374 ) * How did those get there * abacus -> hyperlane * remove deprecated domains * Deleted dead code * Update sea-orm * Simplify settings post v2 changes * Split up settings * Update contexts * Fix scraper provider address regression during merge * Updated scraper deployment configs	2 years ago
Asa Oines	f15632f26c	Finalize rename from abacus->hyperlane (#1312 ) * Rename abacus->hyperlane in agents * Rename in typescript * Rename abacus->hyperlane in dockerfile	2 years ago
Trevor Porter	8f656d1f34	Update ethers-rs to include QuorumProvider fix for methods like eth_estimateGas (#1201 )	2 years ago
Trevor Porter	7040e5c8a4	Gas enforcement policy requiring payment to meet estimated costs (#1083 ) * All compiles * Rename files fwd request -> sponsored call * More scaffolding to get the API key in there * Cleaning up * Rename task_status_call to task_status * Finish rename * rm useForDisabledOriginChains * Introduce TransactionSubmitterType * submitter type -> submission type * Pass around gelato_config instead of the sponsor api key * Final cleanup * Nit * Use default tx submission type * Wip * wip * Getting there * Getting there * Refactor of serial submitter * Some renames * cargo fmt * Some testing * Nits after some testing * A bit of a refactor of policies * Adding some chain enums * Single source of truth for abacus domain IDs * Add local test chains * Get coingecko api key via external-secrets * Move some things around, rm gelato oracle api interaction * Gonna move to abacus-core * Move chain stuff to abacus-core * nit * comment nit * Deploy tooling * MeetsEstimatedCost policy test * PR comments, still need to merge domain enums * Single AbacusDomain enum	2 years ago
Mattie Conover	9cc91d9c51	Refactoring out redundant enums (#1060 ) * Fix metric ordering (#1043) * Remove redunant enums * Remove checkpoint fn that was not actually in use	2 years ago
Mattie Conover	c37aa61b91	Update ethers (#1058 ) * Update run-locally to test quorum provider better * Update ethers-rs	2 years ago
Mattie Conover	1a7e74ea7c	Update ethers (#1037 )	2 years ago
Trevor Porter	000b67cf16	Upgrade ethers-rs to fix Optimsim gas estimation (#1023 )	2 years ago
Mattie Conover	d316ae68d5	update ethers-rs (#1004 )	2 years ago
Mattie Conover	333b18ecfe	Quorum provider (#934 ) * Add quorum provider support * Update Rust * Pull Http logic out of retrying provider * first pass at retrying provider for quorum provider * change weight parsing * Better logging and handling for quorum retry logic * Better logging * Appease new clippy rules * Update packages in vein attempt to make things work * quorum provider testing * switch to our version of ethers-rs * Update config * Support quorum provider * Use quorum provider for testing * Update packages * Update	2 years ago
webbhorn	a8d5d0bd56	Bring `domain_and_chain!` macro up to date, and add test coverage to keep it there (#812 ) Fixes #786 Adds arbitrum, bsc, and optimism and their domains to the `domain_and_chain!` macro, which will enable portions of our monitoring to properly identify traffic relating to this chains instead of filling with "unknown" today. Also introduces a unit test that searches for any json config files it can find, tries to parse them all, takes the union of all chain names and domain mappings it can see, and ensures certain internal consistency properties hold and that the `domain_and_chain!` macro is aware of them. It's possible that this could be eventually adapated for json configs living in `typescript/infra` instead of `rust` if we go that direction. My hope was just to avoid a situation where we forget to keep things synchronized again.	2 years ago
Mattie Conover	de20710466	Rust updates (#644 ) * Update config and trim down macro code * Update rust * Package updates * Updates and cleanup * make clippy happy	2 years ago
webbhorn	6b3822f664	Misc. agent cleanup and dead code deletion (#470 ) * Fix path to agent::Agent in comment. Otherwise `cargo doc` emits a rustdoc::broken_intra_doc_links warning. * Use automatic link in comment for rustdoc. Otherwise 'cargo doc' emits a warning. * Delete checkpointer agent. It is not used in current deployment. * Delete kms-cli. Not used in current deployment (no references). * Include "--all" in precommit cargo fmt -- --check invocation. * Delete unused and hidden src/bin programs. These appear to have been introduced over a year ago and appear to be dead code now. Unhide them from vscode workspace too -- since they were hidden, automated refactoring tools wouldn't find them until compiler failed. * Update github CODEOWNERS to current Abacus team. Drop optional authors field from Cargo.tomls. * Drop processordb from .gitignore. Processor crate has been deleted. * Delete balance-exporter crate. It is not used / no references. * Hoist abacus-cli into its own top-level crate. It is the only tool now, no reason to hide it inside of a tools directory. * Delete old relaese.sh, which only relates to Celo Optics. * Cleanup top-level rust README.md. - Architecture paragraphs outdated, delete. - Point to run-locally.sh instead of documentation. * Explain a bit more about crate deps in README. * Dockerfile fix after tools directory restructuring. We had previously hoisted abacus-cli out since it was the only remaining tool. * Fix path to agent::Agent in comment. Otherwise `cargo doc` emits a rustdoc::broken_intra_doc_links warning. * Use automatic link in coment for rustdoc. Otherwise 'cargo doc' emits a warning. * Delete checkpointer agent. It is not used in current deployment. * Delete kms-cli. Not used in current deployment (no references). * Include "--all" in precommit cargo fmt -- --check invocation. * Delete unused and hidden src/bin programs. These appear to have been introduced over a year ago and appear to be dead code now. Unhide them from vscode workspace too -- since they were hidden, automated refactoring tools wouldn't find them until compiler failed. * Update github CODEOWNERS to current Abacus team. Drop optional authors field from Cargo.tomls. * Drop processordb from .gitignore. Processor crate has been deleted. * Delete balance-exporter crate. It is not used / no references. * Hoist abacus-cli into its own top-level crate. It is the only tool now, no reason to hide it inside of a tools directory. * Delete old relaese.sh, which only relates to Celo Optics. * Cleanup top-level rust README.md. - Architecture paragraphs outdated, delete. - Point to run-locally.sh instead of documentation. * Explain a bit more about crate deps in README. * Dockerfile fix after tools directory restructuring. We had previously hoisted abacus-cli out since it was the only remaining tool.	3 years ago
Mattie Conover	f78e79ab9b	Better Agent Error Logging (#412 ) * Make errors print on one line * Replace color-eyre with eyre in core and base * Replace color-eyre with eyre other libs * Log should be imported without patch version * Update tracing and opentelemetry * Only need openssl dynamic libs * Fix no-default-feature build * Custom eyre handler * Removed unused log dep * Cleanup paste dep * More package cleanup * Update tokio * Minor refactor	3 years ago
Mattie Conover	9cc9d3ab52	Rust dependency cleanup + Lastest stable (#381 ) * ignore jetbrains IDE configs * Update to stable rust 1.60 * Remove unused deps from cargo projects * allow the previously unused fields to still be unused after new warnings * update dockerfile	3 years ago
Nam Chu Hoai	d4c8720700	Rename rust packages (#128 ) * Rename rust packages * rustfmt * More rename * Rename update_abis * Rename metrics.rs	3 years ago
Conner Swann	7b2c4a21df	refactor: split updater tasks for safety (#909 ) * refactor: split updater tasks, refactor opticsdb * feature: asrefs for db types * refactor: refactor typeddb to hold entity and pair with all loads/stores * ci: cargo +nightly * ci: use nightly toolchain * fix: use 1.56 stable not nightly * fix: updater can never overwrite an existing update with a conflicting one * fix: also check suggested update for state change guard Co-authored-by: Luke Tchang <ltchang@stanford.edu>	3 years ago
ember arlynx	a240facc6d	Balance exporter tool skeleton (#793 ) * First pass at balance exporter tool * this was broken with windows unicode paths * promote optics_base to a facade * optics-ethereum has many fewer lines of code * remove a layer of indirection * update all the other code * fmt * post-mege * flatten core::types::\* * impl Chain for ethers * move balance-exporter into tools * post-merge fixes * post-merge resolution * put Default back	3 years ago
James Prestwich	0aba12bb9c	Prestiwch/polygon metrics fix (#783 ) * refactor: move block height tracking to the HomeIndexer, temporarily disable replica block height tracking * chore: enhance updater main comment * refactor: beta reduction	3 years ago
James Prestwich	11d2ec0c5d	prestwich/home indexing (#768 ) * refactor: move db into optics-core, pass db to home instantiation * feature: prototype chain indexer * chore: add todo comment * feature: make chunk_size configurable * refactor: improve DB through out the codebase * bug: use to in indexer spawn * feature: tracing in home indexer * refactor: indexer settings are strings * feature: add setting to disable replica #767 * feature: store latest leaf_index in db * refactor: make fetch_leaf dependent only on DB * feature: add index config to kovan testnet * chore: increase polygon chunk size to 9999	3 years ago
James Prestwich	b2cb90d6cf	chore: add explicit ws dep to work with new ethers (#669 )	3 years ago
James Prestwich	c576771bac	chore: bump ethers to use legacy feature in optics-core	3 years ago
James Prestwich	13644d5957	KMS CLI (#547 ) * feature: sweep bin * refactor: hardcode gas req for simple sweep tx * refactor: update to latest version of ethers * feature: kms cli * bug: short designator conflict + docs * bug: fix value parsing * chore: temp fix for simple sends * refactor: delete deprecated kms bins, roll features into kms-cli	3 years ago
James Prestwich	7b501442b1	refactor: add aws signer (#516 ) * refactor: add aws signer * lint: clippy and fmt * lint: fix unsequenced reads * feature: add a small bin to print key address	3 years ago
James Prestwich	a38f4f5153	chore: bump ethers to main in all cargo.toml files	3 years ago
James Prestwich	aff52ce2a9	bug: tracing and pending txn fixes - use branch of ethers-rs with pendingtx poll delay - use tracing::Instrumented in most agent trait methods - add in_current_span to most spawns - add an id generator to jaeger tracing	3 years ago
Luke Tchang	ae28e02316	test/bug: verify Rust/Solidity OpticsMessage and leaf equivalence (#363 ) * feature: outputs hardcoded merkle proof * fix: wrong sender address * debug: print sender receiver for StampedMessage * prog: leaf check failing * fix: fixes message pack ordering bug and outputs correct message and leaf json * fix: add message and leaf output to precommit fn * fix: use hex string instead of hardcoded str * feature: adds output proof test function	4 years ago
Luke Tchang	befde049a6	fix: update signer to support EIP-155 (#329 ) * fix: provider with SignerMiddleware udpates chain_id of signer to match provider * fix: add ethers wallets feature flags to optics-core cargo.toml	4 years ago
Luke Tchang	59efd7c10b	feature: rust output functions conditionally run in precommit (#309 ) * feature: adds rust output bins to precommit * fix: compiles abis first and formats json * fix: json formatting * fix: use git diff-index for json * fix: git add abis if updated	4 years ago
Luke Tchang	2dd857bbb8	feature: adds untested ProverSync struct to optics-core (#78 ) * feature: basic ProverSync setup * prog: basic setup for linking enqueuings to roots * refactor: modifies Dispatch event to contain treeSize field and condenses destination and sequence into single field * feature: adds leaf_by_tree_size methods to Home abi and logic for updating local tree in prover_sync * format: runs cargo fmt * prog: comments out unused code from previous design * fix: replaces slip44 references with domain * refactor: moves interval to parameter instead of ProverSync struct field * fix: deletes commented out code from NewLeaf design * refactor: modifies ProverSync error handling to bubble up ProverSyncError type * fix: removes slip44 reference * refactor: adds incremental merkle tree to ProverSync and batch updates prover on successful incremental update * fmt: runs cargo fmt * fix: fixes raw_message_by_sequence filtering after destination and sequence event fields were combined to single field in Home * fix: simplifies error handling, adds retry counter for failed api calls, and clones incremental merkle * fix: renames leaves_by_tree_size to leaves_by_tree_index * fix/docs: fixes Home Dispatch event to use leafIndex instead of treeSize and fixes docs to reflect change * fix: changes num_retries to u32 * refactor: cleans up error handling * docs: adds comment about why leafIndex is count() - 1 * fix: deletes old NewLeaf event from Home * fmt: runs cargo fmt * fix: removes num_retries field * refactor: simplifies destination_and_sequence util * fix: fixes typo in warning message * refactor: derives transparent errors * docs: improves comments for prover-sync/incremental merkle behavior * fix: restarts poll_updates loop if prover root was updated while we were building leaf vector from incremental merkle * feature: adds ProverSync::new function * fix: ProverSync stops polling updates on invalid local_root * fmt: runs cargo fmt	4 years ago
Luke Tchang	1f22e023eb	test: adds tests for Common contract (fixed history) (#57 ) * test: adds output function to print hashes used in solidity/test/domainHashTestCases.json * test: adds domainHashTestCases.json file * test: adds signature and domainHash calculation tests to common tests * fix: runs prettier on contracts	4 years ago
Anna Carroll	c4ae975310	test: Merkle contract and rust implementation (#52 )	4 years ago
James Prestwich	880273a123	feature: adds tracing and a basic config for the tracing setup (#31 ) * feature: adds tracing and a basic config for the tracing setup * refactor: change some settings tags to camelCase for consistency * fix: correct logger level match	4 years ago
James Prestwich	ca7e773db8	refactor: simplify dependencies	4 years ago
James Prestwich	8fb2db9269	refactor: traits for on-chain Home and Replica	4 years ago
James Prestwich	823a3497aa	refactor: docstrings + fix sig verification + toy example	4 years ago

16 Commits (04bdce0877b0c1c0ed8ec5ef1780e73edddfe95e)