505 Commits (656880871aa415a55772884a55b49af1562642de)

Author	SHA1	Message	Date
webbhorn	630e20bbc2	validator: Ensure non-empty outbox on init. (#586) ... validator: Ensure non-empty outbox on init. This addresses the issue described in #575 wherein validators crash after new contract deployment due to failed calls on outbox.latest_checkpoint(). The calls fail due to u256 underflow on count()-1, since count() is zero on init, and latest_checkpoint() wants to return an **index**, which is count() - 1. This PR adds a retry loop just prior to entering the existing main validator-submit-loop. I chose to poll outbox.count() rather than try to decode the error code or revert reason on failed calls to latest_checkpoint() so that we avoid tightly coupling this behavior to particular error codes or messages, and in fact may want to preserve exisitng behavior when calls to outbox.latest_checkpoint() fail for other reasons. Tested with a modified `rust/run-locally.sh` which asks Kathy to sleep for 5 minutes prior to sending messages. Without this change, the validator crashes. With this change, the validator does not crash, and after 5 minutes, starts signing checkpoints as designed.	3 years ago
Mattie Conover	4d47540bb8	Metrics improvements (#574) ... * add config for chain name * Add wallets to config for prometheus middleware * add igp contract name * better insert * Use our own chain names * clean conf * fix naming of chain * be more generic to say native currency rather than token * Use `Native` as token symbol	3 years ago
Mattie Conover	4d9603c868	Relayer whitelist (#570) ... * Update ABIs, compiles. Removed Inbox indexer and cached checkpoints from the Inbox * Rm checkpoint indexing test * Update InboxValidatorManager, just need to connect the pieces now * Add channels * So close * Fix InboxValidatorManager deploy * Make message processor slightly more readable * Create loop-control crate * Try to bring some sanity to message processor * Fix bug where prover_sync wasn't in line with latest_signed_checkpoint * Rm immediate message processing, clean up * rm abacus-cli * more cleanup * rename / rm some settings * TS renames / rms * Lower some processing failure logs to info * Checkpoint fetcher doesn't need the CommittedMessages * Hardcode kathy dispatching * Move to watch channel * I'm sorry clippy * nits * rm some nonce related stuff * more refactoring * cleanup * Add whitelist settings * Whitelist filtering * fix docker build * use relayed terminology * Change whitelist tuple order * naming change * move whitelist check to before fetching proof * Update span event * make messages singular * Update error * minor cleanup * Update configs in scripts * Whitelist testing and parsing fixes Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	3 years ago
Trevor Porter	54a41d5a4b	More testnet2 tweaks (#571) ... * Reduce kathy frequency, add fuji * enable all kathys * Change Mumbai finality blocks 500 -> 32 * Always re-pull secrets, update mumbai to use 32 block finality blocks	3 years ago
Trevor Porter	1793fffbeb	Testnet2 agent deployment (#568) ... * Deploy testnet2 contracts * Add testnet2 configs to rust * don't panic for pre-london chains * Don't checkpoint, retry fewer times, and configure kathy properly * clean up * cleaninngggg Co-authored-by: Asa Oines <asaoines@Asas-MacBook-Pro.local>	3 years ago
Asa Oines	0a81c910a3	Move InterchainGasPaymaster from AbacusConnectionManager to AbacusConnectionClient (#558)	3 years ago
Trevor Porter	d120a76c1e	No more cached checkpoints in agents (#550) ... * Update ABIs, compiles. Removed Inbox indexer and cached checkpoints from the Inbox * Rm checkpoint indexing test * Update InboxValidatorManager, just need to connect the pieces now * Add channels * So close * Fix InboxValidatorManager deploy * Fix bug where prover_sync wasn't in line with latest_signed_checkpoint * Rm immediate message processing, clean up * rm abacus-cli * more cleanup * rename / rm some settings * TS renames / rms * Lower some processing failure logs to info * Checkpoint fetcher doesn't need the CommittedMessages * Hardcode kathy dispatching * Move to watch channel * I'm sorry clippy * nits * rm some nonce related stuff * PR comments	3 years ago
Trevor Porter	29970fe326	Index GasPayment events in Relayer (#504) ... * Add InterchainGasPaymaster traits and ethereum structs * Add abacuse-base interchain_gas * Actually setting up indexer * Setting up DB usage * wip * Add interchain gas paymaster to AbacusAgentCore * Sync gas payments in the relayer * It runs wooooooo * Use U256 for gas payments * nits, rm InterchainGasPaymasterEvents * Add chain_name, fix metrics * More name -> chain_name rename * Correctly rename contract_name => chain_name, move InterchainGasPayment from traits to types * split up contract_sync a bit * cargo fmt * nit * clean * Add metrics * to_string => encode_hex * make clippy happy * move contract_sync/common.rs stuff into mod.rs * string interpolation for logs * some nits * mv types/interchain_gas into types/mod * Idempotent GasPayment indexing * Use new contract ABI & update rust code * Rename RandomBadRecipient -> TestSendReceiver * use nit * only index finalized blocks * Run with finalityBlocks * Add InterchainGasPaymaster to update_abis * Some PR comments	3 years ago
Trevor Porter	5050eee9a4	Use new contract ABI & update rust code (#519)	3 years ago
Mattie Conover	71ce3e285b	Fix duplicate metric labels (#518)	3 years ago
Mattie Conover	d687d3e230	More Metrics (#495) ... * metrics refactoring * metrics refactoring * Make brain hurt less by using a damn function * patch through metrics to abacus_ethereum inbox and outbox * WIP * Minor cleanup * work on time update * added label arrays * refactoring * Metrics for sending txns * Metrics middleware * cleanup * Cleanup after merge * Add metrics config and setup prom metrics with core metrics * Minor fixes to docs * Better trait building * rename * Weave metrics through code * Use core metrics instead * Removed some unused code * fmt * change feature layout * fix docker build * Start scraping metrics on an interval * fmt * fix duplicate label error * change features * fix u256 conversion fn * refactor * Support default values in config * Minor fixes * Use f64 conversion from primitive-types * remove unused code * fix const name * improve docs * better var name * remove unused code * remove unused code	3 years ago
J M Rossy	7caaa4e5c6	[Chore] Fix VSCode formatting and prettier precommit (#501) ... - Fix VSCode prettier format on save - Add lint-staged to fix prettier precommit hook - Show root folder in vscode monorepo view - Format vscode settings files	3 years ago
webbhorn	6b3822f664	Misc. agent cleanup and dead code deletion (#470) ... * Fix path to agent::Agent in comment. Otherwise `cargo doc` emits a rustdoc::broken_intra_doc_links warning. * Use automatic link in comment for rustdoc. Otherwise 'cargo doc' emits a warning. * Delete checkpointer agent. It is not used in current deployment. * Delete kms-cli. Not used in current deployment (no references). * Include "--all" in precommit cargo fmt -- --check invocation. * Delete unused and hidden src/bin programs. These appear to have been introduced over a year ago and appear to be dead code now. Unhide them from vscode workspace too -- since they were hidden, automated refactoring tools wouldn't find them until compiler failed. * Update github CODEOWNERS to current Abacus team. Drop optional authors field from Cargo.tomls. * Drop processordb from .gitignore. Processor crate has been deleted. * Delete balance-exporter crate. It is not used / no references. * Hoist abacus-cli into its own top-level crate. It is the only tool now, no reason to hide it inside of a tools directory. * Delete old relaese.sh, which only relates to Celo Optics. * Cleanup top-level rust README.md. - Architecture paragraphs outdated, delete. - Point to run-locally.sh instead of documentation. * Explain a bit more about crate deps in README. * Dockerfile fix after tools directory restructuring. We had previously hoisted abacus-cli out since it was the only remaining tool. * Fix path to agent::Agent in comment. Otherwise `cargo doc` emits a rustdoc::broken_intra_doc_links warning. * Use automatic link in coment for rustdoc. Otherwise 'cargo doc' emits a warning. * Delete checkpointer agent. It is not used in current deployment. * Delete kms-cli. Not used in current deployment (no references). * Include "--all" in precommit cargo fmt -- --check invocation. * Delete unused and hidden src/bin programs. These appear to have been introduced over a year ago and appear to be dead code now. Unhide them from vscode workspace too -- since they were hidden, automated refactoring tools wouldn't find them until compiler failed. * Update github CODEOWNERS to current Abacus team. Drop optional authors field from Cargo.tomls. * Drop processordb from .gitignore. Processor crate has been deleted. * Delete balance-exporter crate. It is not used / no references. * Hoist abacus-cli into its own top-level crate. It is the only tool now, no reason to hide it inside of a tools directory. * Delete old relaese.sh, which only relates to Celo Optics. * Cleanup top-level rust README.md. - Architecture paragraphs outdated, delete. - Point to run-locally.sh instead of documentation. * Explain a bit more about crate deps in README. * Dockerfile fix after tools directory restructuring. We had previously hoisted abacus-cli out since it was the only remaining tool.	3 years ago
webbhorn	d40b2d776c	Reliability/correctness fixes for run-locally.sh. (#480) ... * Use ephemeral sig and db dirs for agents. If agents discover an old instance of their db files, or old validator signatures (likely from a prior invocation of run-locally.sh), they will operate based on the old state, which has diverged from the hardhat state, and thus be incorret. * Log agent errors to script stdout. * Log kathy stdout in backgrounded subshell. * Readonly env vars.	3 years ago
webbhorn	61f270aec5	Fix publisher for rust-analyezr extension. (#496) ... Without this change, vscode emits a warning that maktlad.rust-analyzer is not in the extensions marketplace. Since May 16, rust-lang is the new publisher for the rust-analyzer extension: https://twitter.com/rust_analyzer/status/1526181089391280128?cxt=HHwWgICwgazniq4qAAAA	3 years ago
Nam Chu Hoai	9e081a43f6	Fix agent config generation (#482)	3 years ago
Mattie Conover	36cb6296ce	Replace `boxed_trait` macro (#467) ... * remove macro * Cleanup * refactoring * Cleanup * Minor refactor * minor cleanup * more refactoring * dedup a bit * Attempt to fix pipeline	3 years ago
webbhorn	ba29b50a8f	Replace running-locally.md docs with a script. (#466) ... This bash script builds the relevant components, including relayer and validator agents, launches them in the required order once dependent services are ready, and writes stdout/stderr logs to unified datetime-based directory.	3 years ago
Mattie Conover	2a96c607f5	Update `report_tx` to be a function instead of a macro (#456) ... * Make brain hurt less by using a damn function * rename mod to tx	3 years ago
Mattie Conover	f78e79ab9b	Better Agent Error Logging (#412) ... * Make errors print on one line * Replace color-eyre with eyre in core and base * Replace color-eyre with eyre other libs * Log should be imported without patch version * Update tracing and opentelemetry * Only need openssl dynamic libs * Fix no-default-feature build * Custom eyre handler * Removed unused log dep * Cleanup paste dep * More package cleanup * Update tokio * Minor refactor	3 years ago
Mattie Conover	37af9c15a0	Running locally fixes and docs (#410) ... * Update readme with instructions on how to run locally * Better error message * Fix kathy task * Update rust configs * Update envs * Move to rust dir * Update rust readme	3 years ago
Nam Chu Hoai	d5bdd405c1	Remove auroratestnet from testnet environment (#404)	3 years ago
Mattie Conover	3c312d732c	improve dockerfile (#401)	3 years ago
Mattie Conover	9cc9d3ab52	Rust dependency cleanup + Lastest stable (#381) ... * ignore jetbrains IDE configs * Update to stable rust 1.60 * Remove unused deps from cargo projects * allow the previously unused fields to still be unused after new warnings * update dockerfile	3 years ago
Mattie Conover	50172104fa	Ported http retrying logic from nomad (#390) ... * Ported retrying logic from nomad * remove unused import	3 years ago
Yorke Rhodes	1d572ba887	Add vscode workspaces and settings (#387) ... * Add recommended root extensions * Add typescript and solidity workspace settings * Add rust settings and extensions * Update some root settings * Comment unused rust vars as revealed by analyzer * Remove root extensions in favor of workspace * Update README with VSCode hint	3 years ago
Nam Chu Hoai	cc6c2db5d8	Testnet deploy (#369) ... * Got keys for testnet * Testnet deploy * Add rust configs * prettier * Nits, update governance config * new agent image * Rename secret reference * Have kathy also checkpoint * Only checkpoint when count > 1 * Move relayer to unique key for (source, remote) pair * prettier * Clean up get-aws-key-addresses * get GCP keys half working * let's clean this up * First steps at refactor * Migrate delete-keys over * support key rotation * Create and get key all working * update relayer external secret * Prettier * Update ethers-rs * Debugging testnet deployment (#370) * Dont fail on inbox checkpointing tx error * fix docker build * Debug * More instrumentation * Lint * Use the correct signers * Lint * prettier * suffix -> index, go back to 1 key per agent * rm relayer user * Create user and key identifiers * Move some fns to key-utils, fix gcp secret names * Correct kathy & checkpointer key names * Back to Promse.all in deploy-agents, clean * Don't require GCP keys to be created with agentconfig * fix * pure json response for get-key-addresses Co-authored-by: Trevor Porter <trkporter@ucdavis.edu>	3 years ago
Nam Chu Hoai	cee9a2d8ef	Add more tracing information (#362) ... * Add more tracing information * PR review	3 years ago
Trevor Porter	6593e47f1a	Support agents with AWS keys (#361) ... * Add some tracing to S3 reading and writing * Add AgentAwsUser, refactor into aws dir * Some signer changes * Add AWS credentials to relayer external secret * Redeploy dev * upgrade agents * Log when indexed messages * Looks like relayer is all good * Validator workin * Needs cleanup but nearly there * Get AWS credentials when needed * cleaning * mv some common fns * Fix output env vars * Final clean (I think) * Better fix for annoying race cond * prettier * rm some tracing * Add message_count to messages indexed log	3 years ago
Nam Chu Hoai	2d7234455d	Support multiple inboxes per domain (#349) ... * Support multiple inboxes per domain * Rename in comment * PR review * PR review	3 years ago
Nam Chu Hoai	5e254f0512	Add gauges to track agents (#360)	3 years ago
Trevor Porter	c7fa1272b4	Dev environment & updated infrastructure (#352) ... * First starting at adding some correct configs and messing with helm charts * Created local test docker container here * Adding back infrastructure for test * Add checkpointer to Dockerfile, a bunch of helm chart changes * use docker buildkit and correct image name in build.sh * Checkpointer deploy works * Create dev keys, add support for multiple validators when creating keys * Deploy dev * Add dev config * Deploy is working * Stepping toward a better config... * prettier * Use ChainAgentConfig * Use stricter typing, thank you Yorke * Add some trailing newlines where needed * S3 validators work * Almost entirely working * fix test env, bunch of cleanup * prettier * Clean * Add aws signers for validator statefulset * fix package-locks * Better values.yaml, rely on validators configmap * prettier * Make kathy key creation not conditional * Different local storage paths for test checkpoint syncers * Rename replicaChain * Make relayer types all mandatory * Fix output-agent-env-vars.ts * clean up * Try build cache * setup buildkit * Try GHA caching	3 years ago
Nam Chu Hoai	e5ea4d42a9	0-indexed Checkpoints (#344) ... * Checkpoints are zero indexed * Checkpoints are not one-indexed anymore * Agent changes for 0-indexed checkpoints * Disallow checkpoint index 0 * Can just check indicies again * Proper isCheckpoint * Dispatch dummy message * fix before * PR review Co-authored-by: Asa Oines <asaoines@Asas-MacBook-Pro.local>	3 years ago
Trevor Porter	47ce61ab62	S3 checkpointer: write numeric index at index_key (#353)	3 years ago
Nam Chu Hoai	dc6abedc1d	Remove Outbox nonce and checkpointedRoot from dispatch event (#341) ... * Remove Outbox nonce and checkpointedRoot from dispatch event * Fix db_state * Fix npm build * Fix test * Fix test * Explicitly pass leaf_index to message hash calculation * remove retrying * Update message.json * Remove nonce from AbacusMessage in sdk * Fix test * PR review	3 years ago
Nam Chu Hoai	0216d28dcf	Merge Inbox prove and process (#339) ... * Merge Inbox prove and process * PR reviews * Package-lock	3 years ago
Trevor Porter	b5f121b837	Multisig validator rust agents (#342) ... * add MultisigCheckpointSyncer * cargo fmt * Nits * Some comments, clean up * rm inbox's submit_checkpoint, get settings compiling * Move some checkpoint structs to core, add inbox validator manager base & core * Update existing Inbox/Outbox ABIs, add InboxValidatorManager ABI * Add ethereum inbox validator manager * Add inbox validator manager to AbacusAgentCore * Put everything together * Updated some configs, nearly able to run the relayer * Update config to have InboxValidatorManager alongside Inbox * Able to run it * Remove now unnecessary AbacusAgentCore::new * Make MultisigSignedCheckpoint::try_from not take in a mut * Use InboxContracts in relayer * cleaning up * nit * clippy * Check all checkpoints are the same in the try_from for MultisigSignedCheckpoint	3 years ago
Trevor Porter	fec07d21fe	Remove docs & all optics-related rust (#322) ... * optics -> abacus in infra * rm docs * rename contract-metrics prom metrics optics-* -> abacus-* * Rm watcher, updater, a bunch of optics things in abacus-base * rm agents/processor * Removing a whole lot of updater / home / replica stuff from abacus-core * Deleted a lot throughout -- some still commented out and a lot of cleanup needed * Slowly cleaning up, kathy is where the remaining errors are * Delete some commented out code * makes kathy an abacus agent * compiling woo * cargo fmt * compiling with all features * removing some update references * Ignore handles_missing_rpc_checkpoints * cargo fmt * Add signedCheckpoint.json * rm some references to updates, delete unnecessary abis * rm all references to home or replica * compiling, remove som e unused code * fix test warning * nit * New README - needs a better pass, but no optics references for now * mv rust/helm/optics-agent rust/helm/abacus-agent * Note on us not indexing updates * Get checkpointer working from local env var output	3 years ago
Trevor Porter	6772725332	Rename home domain hash (#321) ... * Rename domain hash str * outbox_domain_hash -> domain_hash, fix vector * Fix sol tests	3 years ago
Nam Chu Hoai	babf5985a3	Fix minor things with agent development (#310) ... * Fix minor things with agent development * Lint	3 years ago
Nam Chu Hoai	cc8a8b40ea	Message Processor Error Handling (#294) ... * Refactor message processor * Fetch Proofs directly instead of the DB (#295) * Fetch Proofs directly instead of the DB * Support retrying of message processes with expoenential backoff (#308) * Support retrying of message processes with expoenential backoff * Minor message processing todos (#309) * Minor message processing todos * Configure relayer for immediate message processing * Catch prover inconsistency * Remove TODO and rely on configuration for now * Only retry when time to retry as come * PR review * PR review * Lint * lint	3 years ago
Nam Chu Hoai	ae3c26189f	Start from last signed checkpoint (#292) ... Co-authored-by: Asa Oines <asaoines@gmail.com>	3 years ago
Nam Chu Hoai	f6168cbe47	Add MessageProcessor (#287) ... * Add MessageProcessor * Wait for indexed messages * PR review * Fix subtle message processing bugs (#290) * Fix suble message processing bugs * Lint * Simplify	3 years ago
Nam Chu Hoai	a9b509e973	Ignore messages during message processing in TipProver (#281)	3 years ago
Nam Chu Hoai	fae4152221	Set a functioning recipient for kathy (#280)	3 years ago
Nam Chu Hoai	535fc17378	Finalize Abacus key roles (#273)	3 years ago
Nam Chu Hoai	a34858b1be	Read checkpoint syncer from relayer settings (#274)	3 years ago
Nam Chu Hoai	0cc15e67a2	Make deploy optics-environment independent (#275)	3 years ago
Trevor Porter	328941fc2c	Checkpointer policy v1 (#217) ... * Add script for running local dev environment * Prettier * Move Checkpointer to (Abacus)Agent * Lint * Fix * Commit dev files to run checkpointer * Use @abacus-network/utils in solidity/abacus-core/scripts/abacus.ts * Compiles, some warnings to think of * rename checkpoint() -> create_checkpoint() * cargo fmt * Move to index based decision * More info * Move from (H256, u32) to Checkpoint * cargo fmt * Add some scripts for testing * Actually fix merge conflicts * Add latency * Nits * rm testing utils * rm 1000_config.json * nit * fix package-lock.json * Better names * Sleep instead of using last_checkpoint_time * cargo fmt * Rm comment * Add checkpointer to output-agent-env-vars Co-authored-by: Asa Oines <a@celo.org> Co-authored-by: nambrot <nambrot@googlemail.com>	3 years ago
Nam Chu Hoai	9ab0c16dd6	Add CheckpointSyncer configuration in Validator and support S3 (#266) ... * Add CheckpointSyncer configuration in Validator and support S3 * Lint * PR review	3 years ago