24 Commits (6ac9e9f1d4539f6f027536c5fffa26e6c5e9e355)

Author	SHA1	Message	Date
Mattie Conover	0b09754922	Message explorer cleanup (#1251) ... * Better hex processing * Cleanup clippy errors * Fix parsing of 160 bit values * Support multiple contracts with cursors * InboxIndexer for getting message processed events * Work on inbox indexing and delivered message linking * Use message hashes to link deliveries and also add hash indexes for common lookups * split up settings struct * Support scraper config for main fn * Multiple config parsing for scraper * Better docs and more reliant config loading * Refactoring building of contract items * Cut down on duplication * Support inbox indexer building * fmt * Scraper settings loading re-work * Fix errors, warnings, and cleanup * Fix clippy warnings * Fix hash index not supporting uniqueness * Fix after entity update * Remove special case * Better error messages for missing config values * Env prefix filtering * Improve error context for url without base error * Fix inbox domain bug * Actually I was wrong about domains, misinterpreted config * Fix loading settings method * Fix upsert error * Debugged and fixed incorrect txn linking with messages and deliveries * Cleanup * Update config for scraper * Update config for scraper * Delivered message linking; again. * Reduce call frequency of message linking query * Fixed typo * Fix init mistake * changed to agent_prefix * Domain settings * Use a loop * Add more docs * Remove semicolen * Increase linking frequency * Add moonbeam domain * Fmt * Renamed settings struct * Add docs * Add test for FROM_HEX_CHARS array * WIP * WIP * Work on provider building * WIP attempt to create different make provider with connection * Create AbacusProviders * Update tables * Update tables and revert seaorm version * Pull data from providers * Better span tracing * Fixes after merge * Apparently this file was excluded * fix addr len error * Better contexts for error decyphering during config parsing * Add optimismgoerli * Add other rollups from #1176 * correct gas used field * Fix build * Cleanup * Update transactions to have additional EIP-1559 data and also default times for easier manual ops * Fix warnings * Some splits * Flatten dir structure a bit * WIP refactoring * #1229 add index on message timestamp * Refactor out db txn fns * Refactor out db block fns * Remove the message linker * Split of db logic into sub mods * Update rust version and sea-orm and satisfy new clippy requirments * new rust version feature + cleanup * Split up syncing logic * Remove use of cell since it was making things more complicated * Fix tracing * Doc cleanup * Record data even if no messages were returned * Reduce tuple use * Keep sea-orm 0.9.3 for now * Update comment. * Remove old commentˆ * Store instead of record * Fix message leaf index metric * Move dead code allowance to fns * Remove redundant type def * Docs * fmt * Move delivery impl * More docs * Fmt * Fixes after testing * Reduce logging verbosity from span * Use IN instead of OR for finding lists of hashes * Remove unused code	2 years ago
Mattie Conover	b4d06f9639	Message Explorer Inbox Scraping (#1159) ... * Better hex processing * Cleanup clippy errors * Fix parsing of 160 bit values * Support multiple contracts with cursors * InboxIndexer for getting message processed events * Work on inbox indexing and delivered message linking * Use message hashes to link deliveries and also add hash indexes for common lookups * split up settings struct * Support scraper config for main fn * Multiple config parsing for scraper * Better docs and more reliant config loading * Refactoring building of contract items * Cut down on duplication * Support inbox indexer building * fmt * Scraper settings loading re-work * Fix errors, warnings, and cleanup * Fix clippy warnings * Fix hash index not supporting uniqueness * Fix after entity update * Remove special case * Better error messages for missing config values * Env prefix filtering * Improve error context for url without base error * Fix inbox domain bug * Actually I was wrong about domains, misinterpreted config * Fix loading settings method * Fix upsert error * Debugged and fixed incorrect txn linking with messages and deliveries * Cleanup * Update config for scraper * Update config for scraper * Delivered message linking; again. * Reduce call frequency of message linking query * Fixed typo * Fix init mistake * changed to agent_prefix * Domain settings * Use a loop * Add more docs * Remove semicolen * Increase linking frequency * Add moonbeam domain * Fmt * Renamed settings struct * Add docs * Add test for FROM_HEX_CHARS array	2 years ago
Trevor Porter	8f656d1f34	Update ethers-rs to include QuorumProvider fix for methods like eth_estimateGas (#1201)	2 years ago
Trevor Porter	986b531c64	ABC_ -> HYP_ (#1116)	2 years ago
Mattie Conover	c37aa61b91	Update ethers (#1058) ... * Update run-locally to test quorum provider better * Update ethers-rs	2 years ago
Trevor Porter	f544a5df03	Add gas payment enforcement policy to require a minimum payment (#1031) ... * Add gas payment enforcement policy * Add infra, prettier * Better logs, add IGPs into configs * Fix e2e * Fix clippy	2 years ago
Mattie Conover	44361545f8	Message explorer entities (#998) ... * Make domain id the primary key * Generation code to build entities * generated entity code * update docs * Switch to hex strings for hashes and addresses * fix log config * Switch to varchar(64) * Fix stdio -> stderr	2 years ago
Mattie Conover	1a7e74ea7c	Update ethers (#1037)	2 years ago
Trevor Porter	000b67cf16	Upgrade ethers-rs to fix Optimsim gas estimation (#1023)	2 years ago
Trevor Porter	09ef472844	Fix e2e (#1021)	2 years ago
Trevor Porter	893d8efc46	QuorumProvider testing in RC (#1018) ... * Update Cargo.lock * wip * Cleanup * Deploy to all chains * Fix build, still only write http connectionType in config * Use QuorumProvider for the outbox too * Use newer img	2 years ago
Mattie Conover	d316ae68d5	update ethers-rs (#1004)	2 years ago
Mattie Conover	333b18ecfe	Quorum provider (#934) ... * Add quorum provider support * Update Rust * Pull Http logic out of retrying provider * first pass at retrying provider for quorum provider * change weight parsing * Better logging and handling for quorum retry logic * Better logging * Appease new clippy rules * Update packages in vein attempt to make things work * quorum provider testing * switch to our version of ethers-rs * Update config * Support quorum provider * Use quorum provider for testing * Update packages * Update	2 years ago
Trevor Porter	0691b7b326	Gelato relayer scaffolding (#950) ... * Revive fwd_req_op. * Transform iter map into a for loop in relayer.rs. * cargo fmt * Plumb signer into GelatoSubmitter. * Provide inbox signer to GelatoSubmitter init. This might fail, in which case the relayer can't really do anything for that inbox if it is configured to use Gelato. In that case, bail!(..). Bailing requires that the boot/init boilerplate in the relayer handle errors, so plumb/wrap those all in Result<> return types and update accordingly. * Plumb deps up to ForwardRequestArgs construction. * Plumb actual target chain, and rename chain --> target_chain. * Use 0xEE for native token always. * Plumb outbox chain id * Plumb address of IVM on target chain * Just use zero bytes instead of todo! to satisfy clippy etc. * Plumb IVM BaseContract into GelatoSubmitter so that we can use it to encode calldata for IVM.process(). * cargo fmt again * How to marshal bytes. * Add interface for accessing inbox contract eth address * Plumb inbox address into the gelato submitter finally. * cargo fmt * Actually return the address instead of todo!()... * Actually build the byte array of calldata! * Better error msg to fit < 100c * defer introducing fwd_req_op until next PR * Handful of pre-review cleanups: - Only give GelatoSubmitter the inbox domain instead of the whole InboxContracts - Don't wrap http: reqwest::Client in an Arc. It's unnecessary and handled internally for you, and better to clone() so that cxn pool reused. - Shorter names from imports where noticed - stub op.run() impl - shorter submitter field names where appropriate - dont pass GelatoConf to GelatoSubmitter - get correct visibility in abigen'd build code (and allow no doc comments to satisfy clippy) * rename data to call_data * cargo fmt * Defer field name change for chain_id and target in FwdReqArgs * Cleanup of relayer.rs including more clear data validation and imoprt scoping. * cargo fmt * Improve (?) address handling for inbox and ivm. * for InboxValidatorManagerVariants::contract_address(), actually delegate to the Mock variant implementation's contract_address() call instead of over-eagerly returning None. * Stub impl of _contract_address in mockall mock! Inbox contract impl. * for Other variants, match arm does unimplemented!(), so we avoid having to wrap in an Option, making cleaner at call sites * Minimize namespace qualifications where possible for Address-like types * attempt to make the IVM contract_address() for abacus-ethereum impl parallel to inbox w/r/t types * Move abacus domain conversion out of gelato::Chain and into private helper in gelato_submitter * More logical ordering of GelatoSubmitter struct fields * cargo fmt * Drop extra comment, clarify log * Plumb sponsor address from configs into GelatoSubmitter. Interpreted as a source chain address. * cargo fmt --all * stop trying to wrap http client in an arc, it's counterproductive * Write Ok once instead of every line * formatting * Do not separate with empty line the inner attributes found in lib.rs post-abigen-buildscript-exec Otherwise cargo fmt finds a diff and presubmits halt. * spelling typo * Change GelatoSubmitter construction from new to helper in relayer.rs. Also use simpler construction for tokenizing the proof for merkle trees when generating the forward request args struct. Was needlessly convoluted before using default trait unnecessarily etc. Use writeln instead of write in abigen util since clippy complains otherwise. * Get ivm_base_contract from ABI string directly rather than plumbing. * pass GelatoConf by as_ref() which refs inner type rather than &Option<GelatoConf> * Drop unrelated comment in gelato/chains.rs, noted elsewhere * Revive fwd_req_op. * Transform iter map into a for loop in relayer.rs. * cargo fmt * Plumb signer into GelatoSubmitter. * Provide inbox signer to GelatoSubmitter init. This might fail, in which case the relayer can't really do anything for that inbox if it is configured to use Gelato. In that case, bail!(..). Bailing requires that the boot/init boilerplate in the relayer handle errors, so plumb/wrap those all in Result<> return types and update accordingly. * Plumb deps up to ForwardRequestArgs construction. * Plumb actual target chain, and rename chain --> target_chain. * Use 0xEE for native token always. * Plumb outbox chain id * Plumb address of IVM on target chain * Just use zero bytes instead of todo! to satisfy clippy etc. * Plumb IVM BaseContract into GelatoSubmitter so that we can use it to encode calldata for IVM.process(). * cargo fmt again * How to marshal bytes. * Add interface for accessing inbox contract eth address * Plumb inbox address into the gelato submitter finally. * cargo fmt * Actually return the address instead of todo!()... * Actually build the byte array of calldata! * Better error msg to fit < 100c * defer introducing fwd_req_op until next PR * Handful of pre-review cleanups: - Only give GelatoSubmitter the inbox domain instead of the whole InboxContracts - Don't wrap http: reqwest::Client in an Arc. It's unnecessary and handled internally for you, and better to clone() so that cxn pool reused. - Shorter names from imports where noticed - stub op.run() impl - shorter submitter field names where appropriate - dont pass GelatoConf to GelatoSubmitter - get correct visibility in abigen'd build code (and allow no doc comments to satisfy clippy) * rename data to call_data * cargo fmt * Defer field name change for chain_id and target in FwdReqArgs * Cleanup of relayer.rs including more clear data validation and imoprt scoping. * cargo fmt * Improve (?) address handling for inbox and ivm. * for InboxValidatorManagerVariants::contract_address(), actually delegate to the Mock variant implementation's contract_address() call instead of over-eagerly returning None. * Stub impl of _contract_address in mockall mock! Inbox contract impl. * for Other variants, match arm does unimplemented!(), so we avoid having to wrap in an Option, making cleaner at call sites * Minimize namespace qualifications where possible for Address-like types * attempt to make the IVM contract_address() for abacus-ethereum impl parallel to inbox w/r/t types * Move abacus domain conversion out of gelato::Chain and into private helper in gelato_submitter * More logical ordering of GelatoSubmitter struct fields * cargo fmt * Drop extra comment, clarify log * Plumb sponsor address from configs into GelatoSubmitter. Interpreted as a source chain address. * cargo fmt --all * stop trying to wrap http client in an arc, it's counterproductive * Write Ok once instead of every line * formatting * Do not separate with empty line the inner attributes found in lib.rs post-abigen-buildscript-exec Otherwise cargo fmt finds a diff and presubmits halt. * spelling typo * Change GelatoSubmitter construction from new to helper in relayer.rs. Also use simpler construction for tokenizing the proof for merkle trees when generating the forward request args struct. Was needlessly convoluted before using default trait unnecessarily etc. Use writeln instead of write in abigen util since clippy complains otherwise. * Get ivm_base_contract from ABI string directly rather than plumbing. * pass GelatoConf by as_ref() which refs inner type rather than &Option<GelatoConf> * Drop unrelated comment in gelato/chains.rs, noted elsewhere * Wrap Inbox::contract_address() result in an Option. * Don't over-zealously mark pub(crate) struct's members pub(crate). pub is good enough. * More natural iter map construction in Token byte packing for call_data on process. * More natural iter map construction in Token byte packing for call_data on process. * Restore fwd req op struct and fmt * rm unnecessary Results * add with_capacity * Make contract_address return Address, not Option<Address> * rm unimplemeneted * Rm address field of validator manager * Mid refactor of creating GelatoSubmitter * add inbox_gelato_chain * cleaning * Some shuffling around * some moving around * Finishing touches * Remove unnecessary type definition & some unused code Co-authored-by: webbhorn <webb.horn@gmail.com>	2 years ago
Trevor Porter	f24aa96a0c	Filter by outbox address (#820)	2 years ago
Mattie Conover	aa73b93f38	make timeout longer for contract deploy (#802)	2 years ago
webbhorn	401b6e2de9	run-locally: Run 'yarn prettier' after hardhat. (#750) ... Currently the process of starting/running our local hardhat setup results in an annoying irrelevant diff created against a test.json config file. The diff is due to lack of a file-terminating newline character. A better version of this change would probably fix the typescript script that touches / generates that file, but I'm completely unfamiliar with that setup (and TS in general) so opting for this hack for now.	2 years ago
webbhorn	8b79a14b51	Set message_delivered phase correctly in serial_submitter (#695) ... Update two metrics upon successful delivery of message: * counter reflecting number of messages successfully delivered during process lifetime * gauge of highest leaf index of any message delivered to this inbox so far during process lifetime Also updates run-locally integration test to verify these metric values upon test termination.	2 years ago
J M Rossy	80ffce0e3a	Fix path to SDK consts in infra core deploy script (#690) ... * Update working dirs for yarn ops * Fix path to SDK consts dir * Run test deploy * Update db path from cargo run-locally * Run prettier * Add outbox addr to kathy log * Fix rust fmt lint Co-authored-by: Mattie Conover <git@mconover.dev>	2 years ago
Mattie Conover	0644e4b22d	E2E testing workflow (#679) ... * Work on new e2e script * new e2e mimics old * Support CI mode * Kathy CI mode * remove old script * e2e workflow * Update readme * fix workflow * split build from run * change build target * Use lighter-weight http client * fix typo * better handling for stopping children * Fix url * Support multiple filters * cache node modules * cache yarn build * ci cleanup * Cleanup and updates * requested changes	2 years ago
webbhorn	dd63f70da1	Gelato Plumbing (#637) ... This PR prepares for the introduction of Gelato ForwardRequest API usage. However, it only refactors current code and introduces a prerequisite interface. In particular, the existing MessageProcessor was previously responsible both for message discovery via AbacusDB, as well as managing the submission of that message to the destination chain via some RPC provider. Since we want to have the option to deliver those messages over Gelato, a natural place to split things up might be between the AbacusDB message discovery phase, and the message delivery phase. This CL formalizes that notion by moving the submission logic into a SerialSubmitter submodule. The MessageProcessor and SerialSubmitter only communicate via a channel now. A subsequent PR will introduce GelatoSubmitter, which will route messages from the MessageProcessor to the Gelato API according to the new configuration options introduced in this PR.	2 years ago
Mattie Conover	1f7beb3b9c	Better Abigen (#633) ... * Abigen tools * Generate abis with buildscript * Generate abis with buildscript for middleware * Simplify code * update lock * Fix pipeline order, must run check first new to generate files	2 years ago
Mattie Conover	aa92873088	Oneline Backtraces (#627) ... * add backtrace lib * Pull over code for printing backtraces * Oneline backtraces * code shuffle * fmt * remove stale comment * cleanup * refactoring	2 years ago
Mattie Conover	4d9603c868	Relayer whitelist (#570) ... * Update ABIs, compiles. Removed Inbox indexer and cached checkpoints from the Inbox * Rm checkpoint indexing test * Update InboxValidatorManager, just need to connect the pieces now * Add channels * So close * Fix InboxValidatorManager deploy * Make message processor slightly more readable * Create loop-control crate * Try to bring some sanity to message processor * Fix bug where prover_sync wasn't in line with latest_signed_checkpoint * Rm immediate message processing, clean up * rm abacus-cli * more cleanup * rename / rm some settings * TS renames / rms * Lower some processing failure logs to info * Checkpoint fetcher doesn't need the CommittedMessages * Hardcode kathy dispatching * Move to watch channel * I'm sorry clippy * nits * rm some nonce related stuff * more refactoring * cleanup * Add whitelist settings * Whitelist filtering * fix docker build * use relayed terminology * Change whitelist tuple order * naming change * move whitelist check to before fetching proof * Update span event * make messages singular * Update error * minor cleanup * Update configs in scripts * Whitelist testing and parsing fixes Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	2 years ago