134 Commits (d57ae5f628bcf3bc0ebcac2c832ad2821a4a5cbb)

Author	SHA1	Message	Date
Yorke Rhodes	50f04db1fa	Make merkle proofs optional on multisig ISM (#2173) ... ### Description Validators currently sign `(root, index)` checkpoints and during verification, a `message` is passed as calldata, an `id()` is derived, and a `proof` of `id()` at `index` in `root` is verified This provides “all or nothing” censorship resistance guarantees because a validator can only sign roots to allow any contained messages to be processed. We have considered alternatives where validators sign `message` directly and we lose censorship resistance in exchange for eliminating merkle proof verification gas costs. However, if validators sign `(root, index, message)` tuples, we can skip merkle proof verification on the destination chain while still maintaining censorship resistance by providing two valid metadata formats: 1. existing validator signatures and merkle proof verification of inclusion 2. including merkle proof verification for pathway where validators are censoring `message` It’s worth noting the validator is required to index event data to produce this new signature format. However, this does not require historical indexing and new validators being spun up can simply begin indexing from tip. See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2187 for validator changes See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2248 for relayer and e2e test changes ### Drive-by changes Merkle index also optional ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2192 ### Backward compatibility - new ISM deployment is necessary (we could upgrade implementation in theory) - Validator and relayer upgrades ### Testing Unit (fuzz) Tests, E2E tests	2 years ago
Asa Oines	ec63ed19c5	Fix CI flakiness (#2269) ... ### Description - Don't panic if E2E test fails to fetch a metric - Run coverage commands sequentially ### Drive-by changes - Cleans up some lint errors in forge tests	2 years ago
Sergei Patrikeev	3711b64de3	feat: support WebIdentityProvider for AWS credentials [#2168]. (#2188) ... ### Description Support of web identity AWS credentials: https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2168 ### Drive-by changes `WebIdentityProvider` resides in `rusoto_sts`, I had to add a dependency on `rusoto_sts = "*"`. ### Backward compatibility _Are these changes backward compatible?_ Yes. The newly added `AwsChainCredentialsProvider` preserves 100% compatibility with clients using `EnvironmentProvider`. We tested the following scenarios: 1) Run agent with `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` / `AWS_SESSION_TOKEN` credentials — the current default behavior — works as before. 2) Without envs from p.1 but with `AWS_WEB_IDENTITY_TOKEN_FILE` / `AWS_ROLE_ARN` / `AWS_ROLE_SESSION_NAME` injected by AWS EKS pod identity [webhook](https://github.com/aws/amazon-eks-pod-identity-webhook) — works well. We ran the agents for 24+ hours to make sure the JWT gets refreshed automatically. 3) Without any envs — the app expectedly fails with "environment variable not found" error. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing Manual: we deployed a Hyperlane agent Docker image built from our [fork](https://github.com/tvl-labs/hyperlane-monorepo/pkgs/container/hyperlane-monorepo).	2 years ago
Mattie Conover	5e38cdf31a	Omniscient Relayer (#2195) ... ### Description Adds support for running a relayer which delivers messages from multiple origin chains instead of just one origin chain. ### Drive-by changes _Are there any minor or drive-by changes also included?_ - The E2E pipeline got a major re-work. It both runs faster and also tests the new changes. - SerialSubmitter has been abstracted to support additional operation types such as oracle updates - SerialSubmitter has been updated to actually confirm a transaction was included and is safe from re-orgs - SerialSubmitter has been updated to have higher throughput by pipelining operations ### Related issues - Blocked by #2148 - Fixes #1899 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Mattie Conover	0cc4600931	Update Agents (#2159) ... ### Description Update rust toolchain and packages. ### Drive-by changes No ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests	2 years ago
Mattie Conover	ed17ddc32f	Agent Argument Parsing (#2132) ... ### Description All configs can now be set by arguments as well as env and config files. The arguments will take the highest priority and overwrite the others. We should also update the docs to include this new support when it gets released, but since it is an entirely new feature it also can be a stealth release which we document after the fact. ### Drive-by changes ### Related issues - Fixes #2034 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Mattie Conover	b56b022f88	Hyperlane-Core WASM support (#2110) ... ### Description This shuffles some code around and changes dependencies to make `hyperlane-core` WASM compatible and adds a check to the pipeline to ensure it stays that way. To do this it does refactor hyperlane db to the `base` crate, which makes more sense IMO anyway as that is an implementation detail of the agents and not part of the core protocol. ### Drive-by changes ### Related issues - Fixes #1977 ### Backward compatibility _Are these changes backward compatible?_ Yes-ish - It does change where things are but it does not change support. Should only impact people who directly use the libraries. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Mattie Conover	1722f4ee7e	Better Agent Configuration Parsing (#2070) ... ### Description This is a significant change to how we parse user configs. The main goal of this PR is to give a clear path and env key to any errors that are encountered when parsing configuration. In order to accomplish this, I have crated a new trait `FromRawConf` which defines a set of steps to validate and convert individual fields. Also for every major config struct, there is now a `Raw*` variant of it which is used by the `config` library to reduce errors that occur during deserialization itself. `Raw*` types should always use the most basic form of a value, such as `String`, `StrOrInt`, `bool`, and ideally optional in all cases. Missing values should be handled during the raw conversion and not by `config`. I also needed to make changes to a number of types stored in the parsed config types to force validation forward to the parsing step instead of doing it when we read the config value. This also required me to create a filter to prevent trying to validate certain configs that we would not ever need to load. These changes can also be built on later to support something other than `config` if we choose to, or add support for merging configs from multiple sources since everything is optional. ### Drive-by changes - Default to `http` for connection type - Default to no gas payment enforcement if not specified instead of failing - `ChainSetup` -> `ChainConf` - `GasPaymentEnforcementConfig` -> `GasPaymentEnforcementConf` - Made ethereum connection configuration more forgiving - Moved hyperlane base settings from `mod.rs` to `base.rs` - Moved config chain tests in hyperlane core to `tests` dir to fix a cyclical import problem - Extension traits to help with config in hyperlane core - Moved `StrOrInt` to new hyperlane core `config` module - Support for parsing `U256` from `StrOrInt` - Removed `HexString` type which is now redundant - Updated base settings to use hyperlane domain - Use `heyKey` as signer type if `type` is not specified and `key` is - Moved hyperlane ethereum chain config to a new module ### Related issues - Fixes #2033 - Fixes #2012 ### Backward compatibility _Are these changes backward compatible?_ Yes - This should take in configs in the same shape but be more forgiving in a few places _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Test	2 years ago
Mattie Conover	7d6e5d5281	Rust Fuel Updates (#2029) ... ### Description This brings fuel up to the latest version. The main change required was to to abigen process. After some digging I figured out they just moved it to a new crate. ### Drive-by changes - Updated an error message I ran into when trying to run the validator to be more descriptive - Found an unlinked file which I was able to delete - Add `*.ignore` files to gitignore. This allows me to block out some configs I was using for testing. ### Related issues - Fixes hyperlane-xyz/issues#410 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual - Ran the validator against a local node	2 years ago
Asa Oines	f25feecfa7	Scaffolding for multiple ISM types in the relayer (#2028) ... ### Description This PR adds scaffolding in the relayer to support building metadata for multiple ISM types. To add a new ISM type: - Add a new implementation of MetadataBuilder for that type - Add the type to the SupportedIsmTypes enum - Add the type to the match statement in BaseMetadataBuilder ### Drive-by changes None ### Related issues - #1757 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ E2E tests	2 years ago
Mattie Conover	82f4b5a5f4	Rust updates (#1979) ... ### Description Time for the monthly rust language update and rust package updates. ### Drive-by changes - Clippy fixes ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests	2 years ago
Mattie Conover	1c2eb1bc07	Scraper Refresh (#1932) ... ### Description Since we need to re-scrape everything anyway, wanted to go ahead and make some improvements and bug fixes now. * Adds code to scrape gas payments from the IGP during the sync loop * Updates gas payments table to work with new gas payment format * Fixed support with scraper for new configuration format (adds configurable list of chains to scrape) * Adds a new view for messages to display all relevant information * Adds a new view for gas payments to show total amounts paid for a given message * Fixed a bug introduced by seaorm (0.10) where `last_inserted_id` switched from being the first insert id to being the last * Pulled in a fix from @serejke in #1925 * Updated seaorm to 0.11.1 * Adds support for sepolia * Improved logging output at various levels ### Drive-by changes * Updated fallback provider to still retry even if it only has one url * Adds support for postgres `numeric` types to store Wei values (supported as of seaorm 0.11) * Switches to `bytea` representations of binary values * Switched from the `chrono` library to `time` (`chrono` is outdated and uses an unpatched version of `time`) * Removed redundant timestamp from `message` table * Cleans up some unused code ### Related issues - Fixes #1925 - Fixes #1156 ### Backward compatibility _Are these changes backward compatible?_ No - this has a new database format and for this reason defines entirely new migrations. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes - I changed it to `scraper3` so that we can run it along side the old scraper. It now also lines up with the name in the database. There is a new env var to specify what chains are scraped `HYP_BASE_CHAINSTOSCRAPE`. ### Testing _What kind of testing have these changes undergone?_ Manual - Been running the scraper locally and also deployed a current version of it to testnet3 for a subset of chains.	2 years ago
Mattie Conover	1c004f4b10	Remove gelato support (#1946) ... ### Description We have not been using Gelato anymore so thought it would be nice to remove this code and avoid continuing to maintain it. I know I have had to update it a few times to maintain parity. As an added perk it should slightly speed up build times since it removes an entire crate. ### Drive-by changes None ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes - As we do not use this code _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests	2 years ago
Nam Chu Hoai	97c7f52801	Remove GasPaymentEnforcementPolicy::MeetsEstimatedCost (#1910) ... ### Description As discussed offline, to establish the social contract of the relayer accepting the quotes of its associated IGP on the origin chain, this PR removes the GasPaymentEnforcementPolicy that was assessing gas payments at tx submission time via off-chain oracles.	2 years ago
Mattie Conover	03d32f4009	Eliminate trivial constructors (#1847)	2 years ago
Mattie Conover	1b45f8cad1	Rust Updates (#1845) ... * Update rust lang version * Fix new clippy lint * Update packages * Update rocksdb * Updates	2 years ago
Mattie Conover	8105913dea	Improve agent logging (#1797)	2 years ago
Mattie Conover	4b7e7ca9bb	Improve config errors (#1776) ... * Remove path to error as it was not working * Better domain parsing errors * Locked min version of config lib * Improve error messages for config loading	2 years ago
Mattie Conover	f0c4afb463	Rust Updates (#1594) ... * Update rust version * Update packages	2 years ago
Mattie Conover	e5e794eda4	Fuel Validators (#1474) ... * Fuel ABI gen * WIP * Progress * Use HyperlaneDomain * Progress on fuel init * Support unknown domains * Work on handling wallets and signers for fuel * Move signer output test * Work on signers * Fixing up compilation * Update to use HyperlaneDomain * Fixes * WIP * Update Mailbox ABI * Move signer to be in chain conf and change rpc style to implementation * Better loader debugging * WIP * Merge fixes and refactor * Fix "rpcStyle" after merge * Simplify chain mappings * Fix tests * Remove excess runtime deps from validator * Use agent core in scraper and remove DB assumptions * More core cleanup * Remove duplication for checkpoint syncers * Better error messages for debugging * Debugging * Cleanup * remove signers * fmt * Fix after merge * Fixes after merge * Fixes after merge * remove random inclusion * Remove unused files * Requested changes from PR * Simplify deps and update sha3 * Fix after merge	2 years ago
Mattie Conover	f6aa1f6039	Fallback provider (#1571) ... * Created fallback provider * support fallback provider * WIP testing * Fix testing * Use http provider so we can better inspect errors * Undo change * Fix comment	2 years ago
Mattie Conover	7353d605df	Fuel scaffolding (#1401) ... * Fuel scaffolding * Fix primitive type import * Remove unused dep	2 years ago
Mattie Conover	fcba422718	Better errors in core (#1404) ... * Better errors * Remove unused error types * fmt * Better error when object is not found from a provider	2 years ago
Mattie Conover	39a24fd2ea	Use primitive types instead of ethers (#1387) ... * ethereum Connection -> ConnectionConf * Use hyperlane_core for primitives * Fixes after merge * fmt	2 years ago
Mattie Conover	f814286998	Update packages (#1385) ... * Update ethers * Update packages * Fix build error	2 years ago
Mattie Conover	e8ed5f88ce	Deploy scraper for testnet3 (#1374) ... * How did those get there * abacus -> hyperlane * remove deprecated domains * Deleted dead code * Update sea-orm * Simplify settings post v2 changes * Split up settings * Update contexts * Fix scraper provider address regression during merge * Updated scraper deployment configs	2 years ago
Asa Oines	f15632f26c	Finalize rename from abacus->hyperlane (#1312) ... * Rename abacus->hyperlane in agents * Rename in typescript * Rename abacus->hyperlane in dockerfile	2 years ago
Mattie Conover	0ba82d1918	Proper RPC Useage fix (#1282) ... * Dedupe indexer range calculations * Better handling of tip updates * Don't use to block for interchain gas indexing * Rename to SyncBlockRangeCursor * use elapsed * Fix init case for debug mode * Work on mocking for outbox indexer test * Fix tests * Use from instead of start for interchain gas indexing db cursor * warn on failure to get block range from cursor * Fixed typo * Document test sleep in outbox	2 years ago
Mattie Conover	0b09754922	Message explorer cleanup (#1251) ... * Better hex processing * Cleanup clippy errors * Fix parsing of 160 bit values * Support multiple contracts with cursors * InboxIndexer for getting message processed events * Work on inbox indexing and delivered message linking * Use message hashes to link deliveries and also add hash indexes for common lookups * split up settings struct * Support scraper config for main fn * Multiple config parsing for scraper * Better docs and more reliant config loading * Refactoring building of contract items * Cut down on duplication * Support inbox indexer building * fmt * Scraper settings loading re-work * Fix errors, warnings, and cleanup * Fix clippy warnings * Fix hash index not supporting uniqueness * Fix after entity update * Remove special case * Better error messages for missing config values * Env prefix filtering * Improve error context for url without base error * Fix inbox domain bug * Actually I was wrong about domains, misinterpreted config * Fix loading settings method * Fix upsert error * Debugged and fixed incorrect txn linking with messages and deliveries * Cleanup * Update config for scraper * Update config for scraper * Delivered message linking; again. * Reduce call frequency of message linking query * Fixed typo * Fix init mistake * changed to agent_prefix * Domain settings * Use a loop * Add more docs * Remove semicolen * Increase linking frequency * Add moonbeam domain * Fmt * Renamed settings struct * Add docs * Add test for FROM_HEX_CHARS array * WIP * WIP * Work on provider building * WIP attempt to create different make provider with connection * Create AbacusProviders * Update tables * Update tables and revert seaorm version * Pull data from providers * Better span tracing * Fixes after merge * Apparently this file was excluded * fix addr len error * Better contexts for error decyphering during config parsing * Add optimismgoerli * Add other rollups from #1176 * correct gas used field * Fix build * Cleanup * Update transactions to have additional EIP-1559 data and also default times for easier manual ops * Fix warnings * Some splits * Flatten dir structure a bit * WIP refactoring * #1229 add index on message timestamp * Refactor out db txn fns * Refactor out db block fns * Remove the message linker * Split of db logic into sub mods * Update rust version and sea-orm and satisfy new clippy requirments * new rust version feature + cleanup * Split up syncing logic * Remove use of cell since it was making things more complicated * Fix tracing * Doc cleanup * Record data even if no messages were returned * Reduce tuple use * Keep sea-orm 0.9.3 for now * Update comment. * Remove old commentˆ * Store instead of record * Fix message leaf index metric * Move dead code allowance to fns * Remove redundant type def * Docs * fmt * Move delivery impl * More docs * Fmt * Fixes after testing * Reduce logging verbosity from span * Use IN instead of OR for finding lists of hashes * Remove unused code	2 years ago
Mattie Conover	05e10b656d	Message Explorer Block and Txn Data from Providers (#1218) ... * Better hex processing * Cleanup clippy errors * Fix parsing of 160 bit values * Support multiple contracts with cursors * InboxIndexer for getting message processed events * Work on inbox indexing and delivered message linking * Use message hashes to link deliveries and also add hash indexes for common lookups * split up settings struct * Support scraper config for main fn * Multiple config parsing for scraper * Better docs and more reliant config loading * Refactoring building of contract items * Cut down on duplication * Support inbox indexer building * fmt * Scraper settings loading re-work * Fix errors, warnings, and cleanup * Fix clippy warnings * Fix hash index not supporting uniqueness * Fix after entity update * Remove special case * Better error messages for missing config values * Env prefix filtering * Improve error context for url without base error * Fix inbox domain bug * Actually I was wrong about domains, misinterpreted config * Fix loading settings method * Fix upsert error * Debugged and fixed incorrect txn linking with messages and deliveries * Cleanup * Update config for scraper * Update config for scraper * Delivered message linking; again. * Reduce call frequency of message linking query * Fixed typo * Fix init mistake * changed to agent_prefix * Domain settings * Use a loop * Add more docs * Remove semicolen * Increase linking frequency * Add moonbeam domain * Fmt * Renamed settings struct * Add docs * Add test for FROM_HEX_CHARS array * WIP * WIP * Work on provider building * WIP attempt to create different make provider with connection * Create AbacusProviders * Update tables * Update tables and revert seaorm version * Pull data from providers * Better span tracing * Fixes after merge * Apparently this file was excluded * fix addr len error * Better contexts for error decyphering during config parsing * Add optimismgoerli * Add other rollups from #1176 * correct gas used field * Fix build * Cleanup * Update transactions to have additional EIP-1559 data and also default times for easier manual ops * Fix warnings * Remove status from abacus core * ETH instead of AGOR * Remove block gas info * Make address optional for contract locator * Removed debug assert * Revert "Make address optional for contract locator" This reverts commit 5f8dd8f360. * chain agnostic zero address for provider construction * Fix var setting * Don't scale floats * Remove debug asserts	2 years ago
Mattie Conover	ad1340e094	Gelato error information (#1199) ... * Display gelato response when failing to parse response * Fix clippy err	2 years ago
Mattie Conover	b4d06f9639	Message Explorer Inbox Scraping (#1159) ... * Better hex processing * Cleanup clippy errors * Fix parsing of 160 bit values * Support multiple contracts with cursors * InboxIndexer for getting message processed events * Work on inbox indexing and delivered message linking * Use message hashes to link deliveries and also add hash indexes for common lookups * split up settings struct * Support scraper config for main fn * Multiple config parsing for scraper * Better docs and more reliant config loading * Refactoring building of contract items * Cut down on duplication * Support inbox indexer building * fmt * Scraper settings loading re-work * Fix errors, warnings, and cleanup * Fix clippy warnings * Fix hash index not supporting uniqueness * Fix after entity update * Remove special case * Better error messages for missing config values * Env prefix filtering * Improve error context for url without base error * Fix inbox domain bug * Actually I was wrong about domains, misinterpreted config * Fix loading settings method * Fix upsert error * Debugged and fixed incorrect txn linking with messages and deliveries * Cleanup * Update config for scraper * Update config for scraper * Delivered message linking; again. * Reduce call frequency of message linking query * Fixed typo * Fix init mistake * changed to agent_prefix * Domain settings * Use a loop * Add more docs * Remove semicolen * Increase linking frequency * Add moonbeam domain * Fmt * Renamed settings struct * Add docs * Add test for FROM_HEX_CHARS array	2 years ago
Trevor Porter	8f656d1f34	Update ethers-rs to include QuorumProvider fix for methods like eth_estimateGas (#1201)	2 years ago
Mattie Conover	fe904f22cc	Runnable Scraper (#1140) ... * Fix metric ordering (#1043) * Remove redunant enums * WIP * block cursor and date time * add chrono * add mods * Work on scraper logic * Init plumbing and config parsing * Stub out inbox loading * Update metric type * Refactoring out weird opt struct * Minor refactoring * Sync logic for scraper outbox reading messages * Fix param names * Stubs and docs to lay out next steps * Add way to get a contract address * read latest message index and work on storing messages * Ported LogMeta and refactored CheckpointMeta out + moved RawCommittedMessage * Remove redundant CheckpointWithMeta struct * Update outbox indexer to include metadata when fetching messages * WIP storing messages * Work on processing blocks and txns * Work on processing blocks and txns * weave data through for message insertion * cleanup * Scraper config * Support multiple index setting configs * Fix panics loading config * Fix db generation * Fix loading wrong outbox from config * Removed deprecated testnets from scraper configs * Fix outbox loading using wrong outbox config * Refactoring out unused index config from ethereum outbox indexer * Fix `todo` panic * Fixed handling of block/txn fetching/storing * allow disabling scraper outboxes * Fix elusive unwrap error * Cleanup and fix `ensure_blocks_and_txns` * Logging that apparently does nothing rn * Refactoring to support initializing tracing before general agent initialization. * Simplify agent main fns * Include deprecated domains * regen entities * Update index names to be more consistent * Remove destination fkey for messages as it does not have to be a valid id * Better logging * remove constraint on block height for the moment * Fix reverse id issue * Fix initial cursor insertion * fix test * appease clippy for now * Get scraper bin in Dockerfile too * append only cursors * append only cursor fix * Fix handling of empty insertion * Reduce query load * Scraper config for mainnet * Remove metrics fn from agent definition * Remove unused checkpoint fn * fix idx name * Order by height instead of id for cursors * Don't fetch message leaf index for every tick * Better var name * Add comment * Removed dead code stub * Better docs * Updated param name * Comment for pr * Added comment * Added comment * Removed unused code * Add doc on domains * Add comment. Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	2 years ago
Trevor Porter	7040e5c8a4	Gas enforcement policy requiring payment to meet estimated costs (#1083) ... * All compiles * Rename files fwd request -> sponsored call * More scaffolding to get the API key in there * Cleaning up * Rename task_status_call to task_status * Finish rename * rm useForDisabledOriginChains * Introduce TransactionSubmitterType * submitter type -> submission type * Pass around gelato_config instead of the sponsor api key * Final cleanup * Nit * Use default tx submission type * Wip * wip * Getting there * Getting there * Refactor of serial submitter * Some renames * cargo fmt * Some testing * Nits after some testing * A bit of a refactor of policies * Adding some chain enums * Single source of truth for abacus domain IDs * Add local test chains * Get coingecko api key via external-secrets * Move some things around, rm gelato oracle api interaction * Gonna move to abacus-core * Move chain stuff to abacus-core * nit * comment nit * Deploy tooling * MeetsEstimatedCost policy test * PR comments, still need to merge domain enums * Single AbacusDomain enum	2 years ago
Trevor Porter	ea5b584fe8	Support new Gelato relay API (#1041) ... * All compiles * Rename files fwd request -> sponsored call * More scaffolding to get the API key in there * Cleaning up * Rename task_status_call to task_status * Finish rename * rm useForDisabledOriginChains * Introduce TransactionSubmitterType * submitter type -> submission type * Pass around gelato_config instead of the sponsor api key * Final cleanup * Nit * Use default tx submission type * Some renames * Nits after some testing	2 years ago
Mattie Conover	9cc91d9c51	Refactoring out redundant enums (#1060) ... * Fix metric ordering (#1043) * Remove redunant enums * Remove checkpoint fn that was not actually in use	2 years ago
Mattie Conover	c37aa61b91	Update ethers (#1058) ... * Update run-locally to test quorum provider better * Update ethers-rs	2 years ago
Mattie Conover	f19110f4de	Quorum metrics (#1039) ... * Refactor ethers-prometheus in preparation * Rename metrics struct * Add JsonRpcMetrics wrapper * Plumbing new metrics in * Rename label for clarity	2 years ago
Mattie Conover	44361545f8	Message explorer entities (#998) ... * Make domain id the primary key * Generation code to build entities * generated entity code * update docs * Switch to hex strings for hashes and addresses * fix log config * Switch to varchar(64) * Fix stdio -> stderr	2 years ago
Mattie Conover	1a7e74ea7c	Update ethers (#1037)	2 years ago
Trevor Porter	000b67cf16	Upgrade ethers-rs to fix Optimsim gas estimation (#1023)	2 years ago
Trevor Porter	3e60d9aa29	Update Cargo.lock (#1007)	2 years ago
Mattie Conover	5b06d0280c	Scraper DB generation and schema (#991) ... * Cleanup cargo config * BaseAgent trait * Pull `run_all` fn out of BaseAgent into static fn * Add `run` to `BaseAgent` * Scraper crate init * Create tables * WIP * data init * Init script * Fix * Attempting to run migration * fix ordering * fix domain creation * fix block table creation * fix checkpoint table * fix table cursor * fix table transaction + cleanup * enum type * progress * gas payments * message * delivered message * message state * cleanup * cleanup warnings * undo * Unique by domain and height * update index names * Change to OriginTxId * Add timestamp to messages * comment	2 years ago
webbhorn	02f9589fdc	Drop rust-Kathy (#713)	2 years ago
Mattie Conover	333b18ecfe	Quorum provider (#934) ... * Add quorum provider support * Update Rust * Pull Http logic out of retrying provider * first pass at retrying provider for quorum provider * change weight parsing * Better logging and handling for quorum retry logic * Better logging * Appease new clippy rules * Update packages in vein attempt to make things work * quorum provider testing * switch to our version of ethers-rs * Update config * Support quorum provider * Use quorum provider for testing * Update packages * Update	2 years ago
Nam Chu Hoai	805942e034	Add a 60s timeout to http requests (#973)	2 years ago
Trevor Porter	0691b7b326	Gelato relayer scaffolding (#950) ... * Revive fwd_req_op. * Transform iter map into a for loop in relayer.rs. * cargo fmt * Plumb signer into GelatoSubmitter. * Provide inbox signer to GelatoSubmitter init. This might fail, in which case the relayer can't really do anything for that inbox if it is configured to use Gelato. In that case, bail!(..). Bailing requires that the boot/init boilerplate in the relayer handle errors, so plumb/wrap those all in Result<> return types and update accordingly. * Plumb deps up to ForwardRequestArgs construction. * Plumb actual target chain, and rename chain --> target_chain. * Use 0xEE for native token always. * Plumb outbox chain id * Plumb address of IVM on target chain * Just use zero bytes instead of todo! to satisfy clippy etc. * Plumb IVM BaseContract into GelatoSubmitter so that we can use it to encode calldata for IVM.process(). * cargo fmt again * How to marshal bytes. * Add interface for accessing inbox contract eth address * Plumb inbox address into the gelato submitter finally. * cargo fmt * Actually return the address instead of todo!()... * Actually build the byte array of calldata! * Better error msg to fit < 100c * defer introducing fwd_req_op until next PR * Handful of pre-review cleanups: - Only give GelatoSubmitter the inbox domain instead of the whole InboxContracts - Don't wrap http: reqwest::Client in an Arc. It's unnecessary and handled internally for you, and better to clone() so that cxn pool reused. - Shorter names from imports where noticed - stub op.run() impl - shorter submitter field names where appropriate - dont pass GelatoConf to GelatoSubmitter - get correct visibility in abigen'd build code (and allow no doc comments to satisfy clippy) * rename data to call_data * cargo fmt * Defer field name change for chain_id and target in FwdReqArgs * Cleanup of relayer.rs including more clear data validation and imoprt scoping. * cargo fmt * Improve (?) address handling for inbox and ivm. * for InboxValidatorManagerVariants::contract_address(), actually delegate to the Mock variant implementation's contract_address() call instead of over-eagerly returning None. * Stub impl of _contract_address in mockall mock! Inbox contract impl. * for Other variants, match arm does unimplemented!(), so we avoid having to wrap in an Option, making cleaner at call sites * Minimize namespace qualifications where possible for Address-like types * attempt to make the IVM contract_address() for abacus-ethereum impl parallel to inbox w/r/t types * Move abacus domain conversion out of gelato::Chain and into private helper in gelato_submitter * More logical ordering of GelatoSubmitter struct fields * cargo fmt * Drop extra comment, clarify log * Plumb sponsor address from configs into GelatoSubmitter. Interpreted as a source chain address. * cargo fmt --all * stop trying to wrap http client in an arc, it's counterproductive * Write Ok once instead of every line * formatting * Do not separate with empty line the inner attributes found in lib.rs post-abigen-buildscript-exec Otherwise cargo fmt finds a diff and presubmits halt. * spelling typo * Change GelatoSubmitter construction from new to helper in relayer.rs. Also use simpler construction for tokenizing the proof for merkle trees when generating the forward request args struct. Was needlessly convoluted before using default trait unnecessarily etc. Use writeln instead of write in abigen util since clippy complains otherwise. * Get ivm_base_contract from ABI string directly rather than plumbing. * pass GelatoConf by as_ref() which refs inner type rather than &Option<GelatoConf> * Drop unrelated comment in gelato/chains.rs, noted elsewhere * Revive fwd_req_op. * Transform iter map into a for loop in relayer.rs. * cargo fmt * Plumb signer into GelatoSubmitter. * Provide inbox signer to GelatoSubmitter init. This might fail, in which case the relayer can't really do anything for that inbox if it is configured to use Gelato. In that case, bail!(..). Bailing requires that the boot/init boilerplate in the relayer handle errors, so plumb/wrap those all in Result<> return types and update accordingly. * Plumb deps up to ForwardRequestArgs construction. * Plumb actual target chain, and rename chain --> target_chain. * Use 0xEE for native token always. * Plumb outbox chain id * Plumb address of IVM on target chain * Just use zero bytes instead of todo! to satisfy clippy etc. * Plumb IVM BaseContract into GelatoSubmitter so that we can use it to encode calldata for IVM.process(). * cargo fmt again * How to marshal bytes. * Add interface for accessing inbox contract eth address * Plumb inbox address into the gelato submitter finally. * cargo fmt * Actually return the address instead of todo!()... * Actually build the byte array of calldata! * Better error msg to fit < 100c * defer introducing fwd_req_op until next PR * Handful of pre-review cleanups: - Only give GelatoSubmitter the inbox domain instead of the whole InboxContracts - Don't wrap http: reqwest::Client in an Arc. It's unnecessary and handled internally for you, and better to clone() so that cxn pool reused. - Shorter names from imports where noticed - stub op.run() impl - shorter submitter field names where appropriate - dont pass GelatoConf to GelatoSubmitter - get correct visibility in abigen'd build code (and allow no doc comments to satisfy clippy) * rename data to call_data * cargo fmt * Defer field name change for chain_id and target in FwdReqArgs * Cleanup of relayer.rs including more clear data validation and imoprt scoping. * cargo fmt * Improve (?) address handling for inbox and ivm. * for InboxValidatorManagerVariants::contract_address(), actually delegate to the Mock variant implementation's contract_address() call instead of over-eagerly returning None. * Stub impl of _contract_address in mockall mock! Inbox contract impl. * for Other variants, match arm does unimplemented!(), so we avoid having to wrap in an Option, making cleaner at call sites * Minimize namespace qualifications where possible for Address-like types * attempt to make the IVM contract_address() for abacus-ethereum impl parallel to inbox w/r/t types * Move abacus domain conversion out of gelato::Chain and into private helper in gelato_submitter * More logical ordering of GelatoSubmitter struct fields * cargo fmt * Drop extra comment, clarify log * Plumb sponsor address from configs into GelatoSubmitter. Interpreted as a source chain address. * cargo fmt --all * stop trying to wrap http client in an arc, it's counterproductive * Write Ok once instead of every line * formatting * Do not separate with empty line the inner attributes found in lib.rs post-abigen-buildscript-exec Otherwise cargo fmt finds a diff and presubmits halt. * spelling typo * Change GelatoSubmitter construction from new to helper in relayer.rs. Also use simpler construction for tokenizing the proof for merkle trees when generating the forward request args struct. Was needlessly convoluted before using default trait unnecessarily etc. Use writeln instead of write in abigen util since clippy complains otherwise. * Get ivm_base_contract from ABI string directly rather than plumbing. * pass GelatoConf by as_ref() which refs inner type rather than &Option<GelatoConf> * Drop unrelated comment in gelato/chains.rs, noted elsewhere * Wrap Inbox::contract_address() result in an Option. * Don't over-zealously mark pub(crate) struct's members pub(crate). pub is good enough. * More natural iter map construction in Token byte packing for call_data on process. * More natural iter map construction in Token byte packing for call_data on process. * Restore fwd req op struct and fmt * rm unnecessary Results * add with_capacity * Make contract_address return Address, not Option<Address> * rm unimplemeneted * Rm address field of validator manager * Mid refactor of creating GelatoSubmitter * add inbox_gelato_chain * cleaning * Some shuffling around * some moving around * Finishing touches * Remove unnecessary type definition & some unused code Co-authored-by: webbhorn <webb.horn@gmail.com>	2 years ago
webbhorn	a8d5d0bd56	Bring `domain_and_chain!` macro up to date, and add test coverage to keep it there (#812) ... Fixes #786 Adds arbitrum, bsc, and optimism and their domains to the `domain_and_chain!` macro, which will enable portions of our monitoring to properly identify traffic relating to this chains instead of filling with "unknown" today. Also introduces a unit test that searches for any json config files it can find, tries to parse them all, takes the union of all chain names and domain mappings it can see, and ensures certain internal consistency properties hold and that the `domain_and_chain!` macro is aware of them. It's possible that this could be eventually adapated for json configs living in `typescript/infra` instead of `rust` if we go that direction. My hope was just to avoid a situation where we forget to keep things synchronized again.	2 years ago
webbhorn	ae03a7343b	Fix broken test `contract_sync::outbox::test::handles_missing_rpc_messages`. (#811) ... Before this PR, the test will panic due to outbox syncer running in a concurrent tokio task after the test completes and invoking mocked functions without expectations set. The panic is also swallowed by a separate bug and is only visible if tests are run under `cargo test -- --show-output`. After this PR, we no longer see a backtrace / panic when running with `cargo test -- --show-output`. We also correctly mark the test as failed in `cargo test` if we introduce a new panic in the test or DB harness code. Some additional changes to try and improve things adjacent to this test: * use tmpfs instead of disk + panic-unwind-detection for the RocksDB contents to avoid the panic detection complexity * avoid the current correctness dependency on sleep(from_secs(5)) in the test. This is inherently racy and it is not much harder or more complex to just select() with a timeout	2 years ago