142 Commits (2fc4903a9098adaf1be837ad7b63e125713e3531)

Author	SHA1	Message	Date
Mattie Conover	9ed3a24902	Solana e2e tests (#2578) ... ### Description Adds support for testing solana with the e2e tests. This involves 1) Downloading the pre-built solana cli tools 2) Setting up the solana configuration 3) Downloading the pre-built solana programs we need (see https://github.com/hyperlane-xyz/solana-program-library/releases/) 4) The solana programs in the repo 5) Start the solana validator 6) Deploy the solana programs 7) Deploy a warp route 8) Initialize the multisig ism and validator announce 9) Initialize a transfer 10) Wait for the message to be delivered In addition these were "woven" in with the existing E2E test logic, so for instance we only run one new validator and the existing relayer was extended to support these additional chains. This will make it much easier later on to support testes between environments. ### Drive-by changes - Fix a bug in the relayer when trying to index the mailbox - Add support in the hyperlane sealevel CLI for custom solana config paths - Fix a linker error on linux causes by not specifying `no-entrypoint` - Cleaned up unnecessary `no-entrypoint` features in libraries - Minor refactor to how we run commands to avoid arg passing - Created an easy way to define tasks that run asyncly `as_task` - Split up main logic flow into a couple files for easier reading - Use mold linker to speed up builds (well, the linking part) - Removed support for `log_all` to simplify code pathways - Added context when a child process ends unexpectedly for easier debugging - Fixed a bug in the validator where it would infinitely retry to send an announcement on failure without waiting - Cleaned up solana configs - Fixed processes hanging on exit (very annoying when testing locally since you have to manually go kill them) - Added stderr logging to the hyperlane sealevel CLI subprocess calls ### Related issues Fixes #2415 ### Backward compatibility Yes ### Testing Manual	1 year ago
Trevor Porter	21acccac3d	Sealevel IGP program (#2574) ... ### Description * Sealevel IGP program * Includes tests * Minor changes to some other Sealevel program tests to use some common test utils ### Drive-by changes n/a ### Related issues #2217 ### Backward compatibility Yes ### Testing Unit / functional	1 year ago
Trevor Porter	8c5983933b	Sealevel (#2404) ... ### Description Opening for that sweet sweet agent image ### Drive-by changes _Are there any minor or drive-by changes also included?_ ### Related issues - Fixes #[issue number here] ### Backward compatibility _Are these changes backward compatible?_ Yes No _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None Yes ### Testing _What kind of testing have these changes undergone?_ None Manual Unit Tests --------- Co-authored-by: Steven Sloboda <steven@eclipse.builders> Co-authored-by: Nam Chu Hoai <nambrot@googlemail.com> Co-authored-by: Yorke Rhodes <yorke@hyperlane.xyz> Co-authored-by: J M Rossy <jm.rossy@gmail.com> Co-authored-by: Mattie Conover <git@mconover.dev> Co-authored-by: Asa Oines <asaoines@gmail.com> Co-authored-by: Sergei Patrikeev <serejke.best@gmail.com> Co-authored-by: Daniel Savu <23065004+daniel-savu@users.noreply.github.com> Co-authored-by: Kunal Arora <55632507+aroralanuk@users.noreply.github.com> Co-authored-by: yorhodes <yorkerhodesiv@gmail.com> Co-authored-by: Yorke Rhodes <yorke@useabacus.network> Co-authored-by: Yorke Rhodes <email@yorke.dev> Co-authored-by: Asa Oines <asaoines@Asas-MacBook-Pro.local> Co-authored-by: Kunal Arora <kuarora@ucsd.edu> Co-authored-by: Anett <44020788+anettrolikova@users.noreply.github.com> Co-authored-by: Yaqub Mahmoud <yaqub320@gmail.com> Co-authored-by: Alex <alex@alexbh.dev>	1 year ago
Mattie Conover	c1c86c7953	E2E Sealevel Prep (#2551) ... ### Description This PR hardens the e2e test pipeline to failure cases we have seen frequently and also makes the code more generic to better support future environments such as sealevel. The original ticket describes making one or more traits to help with this but after much experimentation and consideration, I feel that doing so would make the code unnecessarily complex and less flexible to future changes. My solution instead after talking with @tkporter regarding the sealvel e2e test pathway is to: 1) get utility code out of the way to make it more clear what the execution path is 2) simplify the process of running build tasks in the background so that as we add more envs we can have multiple things running concurrently without difficulty 3) standardize our program execution pattern to reduce the understanding overhead of the main execution path The main thing I kept running into every time I tried to create traits is that we ultimately need a lot of env and context which means either arg passing or even more structures to hold them or we statically code it into that module. The last option was the cleanest but at that point there isn't much difference between creating a trait and just defining a function to call. And even then there is still some live data we define as we go in main that is bulky to pass around but it might become worth it depending. Also the amount of support code for traits kept trying to ballon because we would need something that constructs the "components", and then runs each of the stages, but it also would need to still know what order to run things in, possibly even within each of those steps. ### Drive-by changes - Now watches agents to detect early termination - Standardizes log prefixes to distinguish between e2e logs and sub process logs - Now logs what commands are run - Standardized the program args concept to reduce param passing and better support reusable configurations - Has all build commands run asynchronously by default to prevent `spawn` commands in main - Fixed an issue where some rust components were being built twice, once as debug and again as release ### Related issues - Fixes hyperlane-xyz/issues#501 - Should help debug recent flakiness if not resolve it ### Backward compatibility Yes ### Testing Manual	1 year ago
Alex	da25b06254	CCIP Read ISM (#2398) ... ### Description CCIP Read ISM implementation, - [x] Contracts - [x] Relayer - [x] Example HTTP backend ### Drive-by changes _Are there any minor or drive-by changes also included?_ ### Related issues - Fixes #[2211] ### Backward compatibility _Are these changes backward compatible?_ Yes, relayers will skip messages they can't handle. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes, relayers won't be able to submit messages that have a CCIP Read ISM. ### Testing _What kind of testing have these changes undergone?_ - [x] Manual - [x] Unit Tests ### General approach - [x] Get a basic [CCIP Read implementation working](https://github.com/AlexBHarley/permissionless-chainlink-feeds/tree/main/contracts) - [x] Build a [mock service that serves CCIP Read compatible data](https://github.com/AlexBHarley/permissionless-chainlink-feeds/tree/main/apps/api) - [x] Update the Relayer to support CCIP Read ISMs	1 year ago
Daniel Savu	7889385501	Relayer aggregation ism (#2411) ... ### Description Adds AggregationISM support to the relayer, by adding the scaffolding, sub-ism metadata fetching, and custom metadata encoding. ### Drive-by changes - adds `dry_run_verify(...)` to the `InterchainSecurityModule` trait and implements it for `EthereumInterchainSecurityModule` - changes the default ISM in the test environment from `routingIsm(multisigIsm)` to `routingIsm(aggregationIsm([multisigIsm, messageIdIsm], threshold = 1))` ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/1757 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing The AggregationISM metadata encoding is tested using unit tests whose input was taken from the tests in `AggregationIsm.t.sol`, by printing the individual ISM metadatas and the final metadata. The overall AggregationISM is tested with the `run-locally` script.	1 year ago
Daniel Savu	61892d1d5a	feat(fallback provider): re-prioritize providers on error (#2369) ... ### Description Closes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2189 Reprioritize providers in the `FallbackProvider` when they stop receiving new blocks (although a `max_block_time` has elapsed). The `JsonRpcClient::request(...)` implementation now takes a snapshot of the provider priority list and iterates on that. This is to minimize lock contention - no lock is held across an `await` so there is no deadlock risk. If one of the errors above occurs, the priority list is overwritten. Note this may cause a prior reprioritization to be lost, if the repriotization occured after the current tokio task's snapshot was taken. I'm not entirely sure whether a specific `FallbackProvider` is referenced by more than one task at a time though. ### Drive-by changes A `PrometheusJsonRpcClientConfigExt` trait is added, so the provider can be mocked. ### Testing Unit tests.	1 year ago
Yorke Rhodes	2deb9b8973	Fix AWS KMS Signer connection issues (#2319) ... ### Description - Update ethers-rs fork - Create new KMS client instead of using OnceCell ### Backward compatibility Yes No ### Testing Running validator locally with replica of validator DB	2 years ago
Yorke Rhodes	50f04db1fa	Make merkle proofs optional on multisig ISM (#2173) ... ### Description Validators currently sign `(root, index)` checkpoints and during verification, a `message` is passed as calldata, an `id()` is derived, and a `proof` of `id()` at `index` in `root` is verified This provides “all or nothing” censorship resistance guarantees because a validator can only sign roots to allow any contained messages to be processed. We have considered alternatives where validators sign `message` directly and we lose censorship resistance in exchange for eliminating merkle proof verification gas costs. However, if validators sign `(root, index, message)` tuples, we can skip merkle proof verification on the destination chain while still maintaining censorship resistance by providing two valid metadata formats: 1. existing validator signatures and merkle proof verification of inclusion 2. including merkle proof verification for pathway where validators are censoring `message` It’s worth noting the validator is required to index event data to produce this new signature format. However, this does not require historical indexing and new validators being spun up can simply begin indexing from tip. See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2187 for validator changes See https://github.com/hyperlane-xyz/hyperlane-monorepo/pull/2248 for relayer and e2e test changes ### Drive-by changes Merkle index also optional ### Related issues - Fixes https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2192 ### Backward compatibility - new ISM deployment is necessary (we could upgrade implementation in theory) - Validator and relayer upgrades ### Testing Unit (fuzz) Tests, E2E tests	2 years ago
Asa Oines	ec63ed19c5	Fix CI flakiness (#2269) ... ### Description - Don't panic if E2E test fails to fetch a metric - Run coverage commands sequentially ### Drive-by changes - Cleans up some lint errors in forge tests	2 years ago
Sergei Patrikeev	3711b64de3	feat: support WebIdentityProvider for AWS credentials [#2168]. (#2188) ... ### Description Support of web identity AWS credentials: https://github.com/hyperlane-xyz/hyperlane-monorepo/issues/2168 ### Drive-by changes `WebIdentityProvider` resides in `rusoto_sts`, I had to add a dependency on `rusoto_sts = "*"`. ### Backward compatibility _Are these changes backward compatible?_ Yes. The newly added `AwsChainCredentialsProvider` preserves 100% compatibility with clients using `EnvironmentProvider`. We tested the following scenarios: 1) Run agent with `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` / `AWS_SESSION_TOKEN` credentials — the current default behavior — works as before. 2) Without envs from p.1 but with `AWS_WEB_IDENTITY_TOKEN_FILE` / `AWS_ROLE_ARN` / `AWS_ROLE_SESSION_NAME` injected by AWS EKS pod identity [webhook](https://github.com/aws/amazon-eks-pod-identity-webhook) — works well. We ran the agents for 24+ hours to make sure the JWT gets refreshed automatically. 3) Without any envs — the app expectedly fails with "environment variable not found" error. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing Manual: we deployed a Hyperlane agent Docker image built from our [fork](https://github.com/tvl-labs/hyperlane-monorepo/pkgs/container/hyperlane-monorepo).	2 years ago
Mattie Conover	5e38cdf31a	Omniscient Relayer (#2195) ... ### Description Adds support for running a relayer which delivers messages from multiple origin chains instead of just one origin chain. ### Drive-by changes _Are there any minor or drive-by changes also included?_ - The E2E pipeline got a major re-work. It both runs faster and also tests the new changes. - SerialSubmitter has been abstracted to support additional operation types such as oracle updates - SerialSubmitter has been updated to actually confirm a transaction was included and is safe from re-orgs - SerialSubmitter has been updated to have higher throughput by pipelining operations ### Related issues - Blocked by #2148 - Fixes #1899 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Mattie Conover	0cc4600931	Update Agents (#2159) ... ### Description Update rust toolchain and packages. ### Drive-by changes No ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests	2 years ago
Mattie Conover	ed17ddc32f	Agent Argument Parsing (#2132) ... ### Description All configs can now be set by arguments as well as env and config files. The arguments will take the highest priority and overwrite the others. We should also update the docs to include this new support when it gets released, but since it is an entirely new feature it also can be a stealth release which we document after the fact. ### Drive-by changes ### Related issues - Fixes #2034 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Mattie Conover	b56b022f88	Hyperlane-Core WASM support (#2110) ... ### Description This shuffles some code around and changes dependencies to make `hyperlane-core` WASM compatible and adds a check to the pipeline to ensure it stays that way. To do this it does refactor hyperlane db to the `base` crate, which makes more sense IMO anyway as that is an implementation detail of the agents and not part of the core protocol. ### Drive-by changes ### Related issues - Fixes #1977 ### Backward compatibility _Are these changes backward compatible?_ Yes-ish - It does change where things are but it does not change support. Should only impact people who directly use the libraries. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Tests	2 years ago
Mattie Conover	1722f4ee7e	Better Agent Configuration Parsing (#2070) ... ### Description This is a significant change to how we parse user configs. The main goal of this PR is to give a clear path and env key to any errors that are encountered when parsing configuration. In order to accomplish this, I have crated a new trait `FromRawConf` which defines a set of steps to validate and convert individual fields. Also for every major config struct, there is now a `Raw*` variant of it which is used by the `config` library to reduce errors that occur during deserialization itself. `Raw*` types should always use the most basic form of a value, such as `String`, `StrOrInt`, `bool`, and ideally optional in all cases. Missing values should be handled during the raw conversion and not by `config`. I also needed to make changes to a number of types stored in the parsed config types to force validation forward to the parsing step instead of doing it when we read the config value. This also required me to create a filter to prevent trying to validate certain configs that we would not ever need to load. These changes can also be built on later to support something other than `config` if we choose to, or add support for merging configs from multiple sources since everything is optional. ### Drive-by changes - Default to `http` for connection type - Default to no gas payment enforcement if not specified instead of failing - `ChainSetup` -> `ChainConf` - `GasPaymentEnforcementConfig` -> `GasPaymentEnforcementConf` - Made ethereum connection configuration more forgiving - Moved hyperlane base settings from `mod.rs` to `base.rs` - Moved config chain tests in hyperlane core to `tests` dir to fix a cyclical import problem - Extension traits to help with config in hyperlane core - Moved `StrOrInt` to new hyperlane core `config` module - Support for parsing `U256` from `StrOrInt` - Removed `HexString` type which is now redundant - Updated base settings to use hyperlane domain - Use `heyKey` as signer type if `type` is not specified and `key` is - Moved hyperlane ethereum chain config to a new module ### Related issues - Fixes #2033 - Fixes #2012 ### Backward compatibility _Are these changes backward compatible?_ Yes - This should take in configs in the same shape but be more forgiving in a few places _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual Unit Test	2 years ago
Mattie Conover	7d6e5d5281	Rust Fuel Updates (#2029) ... ### Description This brings fuel up to the latest version. The main change required was to to abigen process. After some digging I figured out they just moved it to a new crate. ### Drive-by changes - Updated an error message I ran into when trying to run the validator to be more descriptive - Found an unlinked file which I was able to delete - Add `*.ignore` files to gitignore. This allows me to block out some configs I was using for testing. ### Related issues - Fixes hyperlane-xyz/issues#410 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Manual - Ran the validator against a local node	2 years ago
Asa Oines	f25feecfa7	Scaffolding for multiple ISM types in the relayer (#2028) ... ### Description This PR adds scaffolding in the relayer to support building metadata for multiple ISM types. To add a new ISM type: - Add a new implementation of MetadataBuilder for that type - Add the type to the SupportedIsmTypes enum - Add the type to the match statement in BaseMetadataBuilder ### Drive-by changes None ### Related issues - #1757 ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ E2E tests	2 years ago
Mattie Conover	82f4b5a5f4	Rust updates (#1979) ... ### Description Time for the monthly rust language update and rust package updates. ### Drive-by changes - Clippy fixes ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests	2 years ago
Mattie Conover	1c2eb1bc07	Scraper Refresh (#1932) ... ### Description Since we need to re-scrape everything anyway, wanted to go ahead and make some improvements and bug fixes now. * Adds code to scrape gas payments from the IGP during the sync loop * Updates gas payments table to work with new gas payment format * Fixed support with scraper for new configuration format (adds configurable list of chains to scrape) * Adds a new view for messages to display all relevant information * Adds a new view for gas payments to show total amounts paid for a given message * Fixed a bug introduced by seaorm (0.10) where `last_inserted_id` switched from being the first insert id to being the last * Pulled in a fix from @serejke in #1925 * Updated seaorm to 0.11.1 * Adds support for sepolia * Improved logging output at various levels ### Drive-by changes * Updated fallback provider to still retry even if it only has one url * Adds support for postgres `numeric` types to store Wei values (supported as of seaorm 0.11) * Switches to `bytea` representations of binary values * Switched from the `chrono` library to `time` (`chrono` is outdated and uses an unpatched version of `time`) * Removed redundant timestamp from `message` table * Cleans up some unused code ### Related issues - Fixes #1925 - Fixes #1156 ### Backward compatibility _Are these changes backward compatible?_ No - this has a new database format and for this reason defines entirely new migrations. _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ Yes - I changed it to `scraper3` so that we can run it along side the old scraper. It now also lines up with the name in the database. There is a new env var to specify what chains are scraped `HYP_BASE_CHAINSTOSCRAPE`. ### Testing _What kind of testing have these changes undergone?_ Manual - Been running the scraper locally and also deployed a current version of it to testnet3 for a subset of chains.	2 years ago
Mattie Conover	1c004f4b10	Remove gelato support (#1946) ... ### Description We have not been using Gelato anymore so thought it would be nice to remove this code and avoid continuing to maintain it. I know I have had to update it a few times to maintain parity. As an added perk it should slightly speed up build times since it removes an entire crate. ### Drive-by changes None ### Related issues ### Backward compatibility _Are these changes backward compatible?_ Yes - As we do not use this code _Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?_ None ### Testing _What kind of testing have these changes undergone?_ Unit Tests	2 years ago
Nam Chu Hoai	97c7f52801	Remove GasPaymentEnforcementPolicy::MeetsEstimatedCost (#1910) ... ### Description As discussed offline, to establish the social contract of the relayer accepting the quotes of its associated IGP on the origin chain, this PR removes the GasPaymentEnforcementPolicy that was assessing gas payments at tx submission time via off-chain oracles.	2 years ago
Mattie Conover	03d32f4009	Eliminate trivial constructors (#1847)	2 years ago
Mattie Conover	1b45f8cad1	Rust Updates (#1845) ... * Update rust lang version * Fix new clippy lint * Update packages * Update rocksdb * Updates	2 years ago
Mattie Conover	8105913dea	Improve agent logging (#1797)	2 years ago
Mattie Conover	4b7e7ca9bb	Improve config errors (#1776) ... * Remove path to error as it was not working * Better domain parsing errors * Locked min version of config lib * Improve error messages for config loading	2 years ago
Mattie Conover	f0c4afb463	Rust Updates (#1594) ... * Update rust version * Update packages	2 years ago
Mattie Conover	e5e794eda4	Fuel Validators (#1474) ... * Fuel ABI gen * WIP * Progress * Use HyperlaneDomain * Progress on fuel init * Support unknown domains * Work on handling wallets and signers for fuel * Move signer output test * Work on signers * Fixing up compilation * Update to use HyperlaneDomain * Fixes * WIP * Update Mailbox ABI * Move signer to be in chain conf and change rpc style to implementation * Better loader debugging * WIP * Merge fixes and refactor * Fix "rpcStyle" after merge * Simplify chain mappings * Fix tests * Remove excess runtime deps from validator * Use agent core in scraper and remove DB assumptions * More core cleanup * Remove duplication for checkpoint syncers * Better error messages for debugging * Debugging * Cleanup * remove signers * fmt * Fix after merge * Fixes after merge * Fixes after merge * remove random inclusion * Remove unused files * Requested changes from PR * Simplify deps and update sha3 * Fix after merge	2 years ago
Mattie Conover	f6aa1f6039	Fallback provider (#1571) ... * Created fallback provider * support fallback provider * WIP testing * Fix testing * Use http provider so we can better inspect errors * Undo change * Fix comment	2 years ago
Mattie Conover	7353d605df	Fuel scaffolding (#1401) ... * Fuel scaffolding * Fix primitive type import * Remove unused dep	2 years ago
Mattie Conover	fcba422718	Better errors in core (#1404) ... * Better errors * Remove unused error types * fmt * Better error when object is not found from a provider	2 years ago
Mattie Conover	39a24fd2ea	Use primitive types instead of ethers (#1387) ... * ethereum Connection -> ConnectionConf * Use hyperlane_core for primitives * Fixes after merge * fmt	2 years ago
Mattie Conover	f814286998	Update packages (#1385) ... * Update ethers * Update packages * Fix build error	2 years ago
Mattie Conover	e8ed5f88ce	Deploy scraper for testnet3 (#1374) ... * How did those get there * abacus -> hyperlane * remove deprecated domains * Deleted dead code * Update sea-orm * Simplify settings post v2 changes * Split up settings * Update contexts * Fix scraper provider address regression during merge * Updated scraper deployment configs	2 years ago
Asa Oines	f15632f26c	Finalize rename from abacus->hyperlane (#1312) ... * Rename abacus->hyperlane in agents * Rename in typescript * Rename abacus->hyperlane in dockerfile	2 years ago
Mattie Conover	0ba82d1918	Proper RPC Useage fix (#1282) ... * Dedupe indexer range calculations * Better handling of tip updates * Don't use to block for interchain gas indexing * Rename to SyncBlockRangeCursor * use elapsed * Fix init case for debug mode * Work on mocking for outbox indexer test * Fix tests * Use from instead of start for interchain gas indexing db cursor * warn on failure to get block range from cursor * Fixed typo * Document test sleep in outbox	2 years ago
Mattie Conover	0b09754922	Message explorer cleanup (#1251) ... * Better hex processing * Cleanup clippy errors * Fix parsing of 160 bit values * Support multiple contracts with cursors * InboxIndexer for getting message processed events * Work on inbox indexing and delivered message linking * Use message hashes to link deliveries and also add hash indexes for common lookups * split up settings struct * Support scraper config for main fn * Multiple config parsing for scraper * Better docs and more reliant config loading * Refactoring building of contract items * Cut down on duplication * Support inbox indexer building * fmt * Scraper settings loading re-work * Fix errors, warnings, and cleanup * Fix clippy warnings * Fix hash index not supporting uniqueness * Fix after entity update * Remove special case * Better error messages for missing config values * Env prefix filtering * Improve error context for url without base error * Fix inbox domain bug * Actually I was wrong about domains, misinterpreted config * Fix loading settings method * Fix upsert error * Debugged and fixed incorrect txn linking with messages and deliveries * Cleanup * Update config for scraper * Update config for scraper * Delivered message linking; again. * Reduce call frequency of message linking query * Fixed typo * Fix init mistake * changed to agent_prefix * Domain settings * Use a loop * Add more docs * Remove semicolen * Increase linking frequency * Add moonbeam domain * Fmt * Renamed settings struct * Add docs * Add test for FROM_HEX_CHARS array * WIP * WIP * Work on provider building * WIP attempt to create different make provider with connection * Create AbacusProviders * Update tables * Update tables and revert seaorm version * Pull data from providers * Better span tracing * Fixes after merge * Apparently this file was excluded * fix addr len error * Better contexts for error decyphering during config parsing * Add optimismgoerli * Add other rollups from #1176 * correct gas used field * Fix build * Cleanup * Update transactions to have additional EIP-1559 data and also default times for easier manual ops * Fix warnings * Some splits * Flatten dir structure a bit * WIP refactoring * #1229 add index on message timestamp * Refactor out db txn fns * Refactor out db block fns * Remove the message linker * Split of db logic into sub mods * Update rust version and sea-orm and satisfy new clippy requirments * new rust version feature + cleanup * Split up syncing logic * Remove use of cell since it was making things more complicated * Fix tracing * Doc cleanup * Record data even if no messages were returned * Reduce tuple use * Keep sea-orm 0.9.3 for now * Update comment. * Remove old commentˆ * Store instead of record * Fix message leaf index metric * Move dead code allowance to fns * Remove redundant type def * Docs * fmt * Move delivery impl * More docs * Fmt * Fixes after testing * Reduce logging verbosity from span * Use IN instead of OR for finding lists of hashes * Remove unused code	2 years ago
Mattie Conover	05e10b656d	Message Explorer Block and Txn Data from Providers (#1218) ... * Better hex processing * Cleanup clippy errors * Fix parsing of 160 bit values * Support multiple contracts with cursors * InboxIndexer for getting message processed events * Work on inbox indexing and delivered message linking * Use message hashes to link deliveries and also add hash indexes for common lookups * split up settings struct * Support scraper config for main fn * Multiple config parsing for scraper * Better docs and more reliant config loading * Refactoring building of contract items * Cut down on duplication * Support inbox indexer building * fmt * Scraper settings loading re-work * Fix errors, warnings, and cleanup * Fix clippy warnings * Fix hash index not supporting uniqueness * Fix after entity update * Remove special case * Better error messages for missing config values * Env prefix filtering * Improve error context for url without base error * Fix inbox domain bug * Actually I was wrong about domains, misinterpreted config * Fix loading settings method * Fix upsert error * Debugged and fixed incorrect txn linking with messages and deliveries * Cleanup * Update config for scraper * Update config for scraper * Delivered message linking; again. * Reduce call frequency of message linking query * Fixed typo * Fix init mistake * changed to agent_prefix * Domain settings * Use a loop * Add more docs * Remove semicolen * Increase linking frequency * Add moonbeam domain * Fmt * Renamed settings struct * Add docs * Add test for FROM_HEX_CHARS array * WIP * WIP * Work on provider building * WIP attempt to create different make provider with connection * Create AbacusProviders * Update tables * Update tables and revert seaorm version * Pull data from providers * Better span tracing * Fixes after merge * Apparently this file was excluded * fix addr len error * Better contexts for error decyphering during config parsing * Add optimismgoerli * Add other rollups from #1176 * correct gas used field * Fix build * Cleanup * Update transactions to have additional EIP-1559 data and also default times for easier manual ops * Fix warnings * Remove status from abacus core * ETH instead of AGOR * Remove block gas info * Make address optional for contract locator * Removed debug assert * Revert "Make address optional for contract locator" This reverts commit 5f8dd8f360. * chain agnostic zero address for provider construction * Fix var setting * Don't scale floats * Remove debug asserts	2 years ago
Mattie Conover	ad1340e094	Gelato error information (#1199) ... * Display gelato response when failing to parse response * Fix clippy err	2 years ago
Mattie Conover	b4d06f9639	Message Explorer Inbox Scraping (#1159) ... * Better hex processing * Cleanup clippy errors * Fix parsing of 160 bit values * Support multiple contracts with cursors * InboxIndexer for getting message processed events * Work on inbox indexing and delivered message linking * Use message hashes to link deliveries and also add hash indexes for common lookups * split up settings struct * Support scraper config for main fn * Multiple config parsing for scraper * Better docs and more reliant config loading * Refactoring building of contract items * Cut down on duplication * Support inbox indexer building * fmt * Scraper settings loading re-work * Fix errors, warnings, and cleanup * Fix clippy warnings * Fix hash index not supporting uniqueness * Fix after entity update * Remove special case * Better error messages for missing config values * Env prefix filtering * Improve error context for url without base error * Fix inbox domain bug * Actually I was wrong about domains, misinterpreted config * Fix loading settings method * Fix upsert error * Debugged and fixed incorrect txn linking with messages and deliveries * Cleanup * Update config for scraper * Update config for scraper * Delivered message linking; again. * Reduce call frequency of message linking query * Fixed typo * Fix init mistake * changed to agent_prefix * Domain settings * Use a loop * Add more docs * Remove semicolen * Increase linking frequency * Add moonbeam domain * Fmt * Renamed settings struct * Add docs * Add test for FROM_HEX_CHARS array	2 years ago
Trevor Porter	8f656d1f34	Update ethers-rs to include QuorumProvider fix for methods like eth_estimateGas (#1201)	2 years ago
Mattie Conover	fe904f22cc	Runnable Scraper (#1140) ... * Fix metric ordering (#1043) * Remove redunant enums * WIP * block cursor and date time * add chrono * add mods * Work on scraper logic * Init plumbing and config parsing * Stub out inbox loading * Update metric type * Refactoring out weird opt struct * Minor refactoring * Sync logic for scraper outbox reading messages * Fix param names * Stubs and docs to lay out next steps * Add way to get a contract address * read latest message index and work on storing messages * Ported LogMeta and refactored CheckpointMeta out + moved RawCommittedMessage * Remove redundant CheckpointWithMeta struct * Update outbox indexer to include metadata when fetching messages * WIP storing messages * Work on processing blocks and txns * Work on processing blocks and txns * weave data through for message insertion * cleanup * Scraper config * Support multiple index setting configs * Fix panics loading config * Fix db generation * Fix loading wrong outbox from config * Removed deprecated testnets from scraper configs * Fix outbox loading using wrong outbox config * Refactoring out unused index config from ethereum outbox indexer * Fix `todo` panic * Fixed handling of block/txn fetching/storing * allow disabling scraper outboxes * Fix elusive unwrap error * Cleanup and fix `ensure_blocks_and_txns` * Logging that apparently does nothing rn * Refactoring to support initializing tracing before general agent initialization. * Simplify agent main fns * Include deprecated domains * regen entities * Update index names to be more consistent * Remove destination fkey for messages as it does not have to be a valid id * Better logging * remove constraint on block height for the moment * Fix reverse id issue * Fix initial cursor insertion * fix test * appease clippy for now * Get scraper bin in Dockerfile too * append only cursors * append only cursor fix * Fix handling of empty insertion * Reduce query load * Scraper config for mainnet * Remove metrics fn from agent definition * Remove unused checkpoint fn * fix idx name * Order by height instead of id for cursors * Don't fetch message leaf index for every tick * Better var name * Add comment * Removed dead code stub * Better docs * Updated param name * Comment for pr * Added comment * Added comment * Removed unused code * Add doc on domains * Add comment. Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	2 years ago
Trevor Porter	7040e5c8a4	Gas enforcement policy requiring payment to meet estimated costs (#1083) ... * All compiles * Rename files fwd request -> sponsored call * More scaffolding to get the API key in there * Cleaning up * Rename task_status_call to task_status * Finish rename * rm useForDisabledOriginChains * Introduce TransactionSubmitterType * submitter type -> submission type * Pass around gelato_config instead of the sponsor api key * Final cleanup * Nit * Use default tx submission type * Wip * wip * Getting there * Getting there * Refactor of serial submitter * Some renames * cargo fmt * Some testing * Nits after some testing * A bit of a refactor of policies * Adding some chain enums * Single source of truth for abacus domain IDs * Add local test chains * Get coingecko api key via external-secrets * Move some things around, rm gelato oracle api interaction * Gonna move to abacus-core * Move chain stuff to abacus-core * nit * comment nit * Deploy tooling * MeetsEstimatedCost policy test * PR comments, still need to merge domain enums * Single AbacusDomain enum	2 years ago
Trevor Porter	ea5b584fe8	Support new Gelato relay API (#1041) ... * All compiles * Rename files fwd request -> sponsored call * More scaffolding to get the API key in there * Cleaning up * Rename task_status_call to task_status * Finish rename * rm useForDisabledOriginChains * Introduce TransactionSubmitterType * submitter type -> submission type * Pass around gelato_config instead of the sponsor api key * Final cleanup * Nit * Use default tx submission type * Some renames * Nits after some testing	2 years ago
Mattie Conover	9cc91d9c51	Refactoring out redundant enums (#1060) ... * Fix metric ordering (#1043) * Remove redunant enums * Remove checkpoint fn that was not actually in use	2 years ago
Mattie Conover	c37aa61b91	Update ethers (#1058) ... * Update run-locally to test quorum provider better * Update ethers-rs	2 years ago
Mattie Conover	f19110f4de	Quorum metrics (#1039) ... * Refactor ethers-prometheus in preparation * Rename metrics struct * Add JsonRpcMetrics wrapper * Plumbing new metrics in * Rename label for clarity	2 years ago
Mattie Conover	44361545f8	Message explorer entities (#998) ... * Make domain id the primary key * Generation code to build entities * generated entity code * update docs * Switch to hex strings for hashes and addresses * fix log config * Switch to varchar(64) * Fix stdio -> stderr	2 years ago
Mattie Conover	1a7e74ea7c	Update ethers (#1037)	2 years ago
Trevor Porter	000b67cf16	Upgrade ethers-rs to fix Optimsim gas estimation (#1023)	2 years ago