hyperlane-monorepo/rust

Optics Rust implementations

Setup

• install rustup
  • link here
• setup pre-commit hooks: cp ../pre-commit.sh ../.git/hooks/pre-commit
  • Note: To bypass pre-commit hooks, pass --no-verify after commit message

Note: You should be running >= version 1.52.1 of the rustc compiler, you can see that version with this command and should see similar output:

$ rustup --version
rustup 1.24.2 (755e2b07e 2021-05-12)
info: This is the version for the rustup toolchain manager, not the rustc compiler.
info: The currently active `rustc` version is `rustc 1.52.1 (9bc8c42bb 2021-05-09)`

Useful cargo commands

• cargo doc --open
  • generate documentation and open it in a web browser
• cargo build
  • compile the project
• cargo run --example example
  • run the default executable for the current project
• cargo test
  • run the tests

Useful cargo extensions

• tree
  • show the dependency tree. Allows searching for specific packages
  • install: cargo install cargo-tree
  • invoke: cargo tree
• clippy
  • search the codebase for a large number of lints and bad patterns
  • install: rustup component add clippy
  • invoke: cargo clippy
• expand
  • expand macros and procedural macros. Show the code generated by the preprocessor
  • useful for debugging #[macros] and macros!()
  • install: cargo install cargo-expand
  • invoke cargo expand path::to::module

Architecture

The on-chain portions of optics are written in Solidity. The rust portions are exclusively off-chain. Later, there may be on-chain rust for Near/Solana/ Polkadot.

Optics will be managed by a number of small off-chain programs ("agents"). Each of these will have a specific role. We want these roles to be simple, and easily described. Each of these agents will connect to a home chain and any number of replicas. They need to be configured with chain connection details and have access to a reliable node for each chain.

Some agent sketches:

• updater
  • Needs only a connection to the home chain
  • Signs upate attestations and submits them to the home chain
• watcher
  • Observe the home chain
  • Observe as many replicas as possible
  • Cache updates
  • Check for fraud
  • Submit fraud to the home chain
  • if configured, issue emergency stop transactions
• relayer
  • Relays signed updates from the home to the replica
  • Ensures updates are confirmed in a timely manner on the replica
• processor
  • retrieve leaves from home chain
  • observe >=1 replica
  • generate proofs for the messages
  • submit messages and proofs to the replica for processing
  • config option: gas params

For Ethereum and Celo connections we use ethers-rs. Please see the docs here.

We use the tokio async runtime environment. Please see the docs here.

Repo layout

• optics-core
  • contains implementations of core primitives
  • this includes
    • traits (interfaces) for the on-chain contracts
    • model implementations of the contracts in rust
    • merkle tree implementations (for provers)
• optics-base
  • contains shared utilities for building off-chain agents
  • this includes
    • trait implementations for different chains
    • shared configuration file formats
    • basic setup for an off-chain agent
• chains/optics-ethereum
  • interfaces to the ethereum contracts
• agents
  • each of the off-chain agents implemented thus far

High-level guide to building an agent

• cargo new $AGENT_NAME
• add the new directory name to the workspace Cargo.toml
• add dependencies to the new directory's Cargo.toml
  • copy most of the dependencies from optics-base
• create a new module in src/$AGENT_NAME.rs
  • add a new struct
  • implement optics_base::agent::OpticsAgent for your struct
  • your run function is the business logic of your agent
• create a new settings module src/settings.rs
  • reuse the Settings objects from optics_base::settings
  • make sure to read the docs :)
  • add your own new settings
• in $AGENT_NAME/src/main.rs
  • add mod _____ declarations for your agent and settings modules
  • create main and setup functions
  • follow the pattern in optics-base/src/main.rs
• make a config folder and a toml file
  • Make sure to include your own settings from above

Provisoning KMS Keys

There exists a script in this repository (provision_kms_keys.py ) that facilitates KMS key provisioning for agent roles.

The script will produce a single set of keys per "environment." Where an environment is a logical set of smart contrace deployments. By default there are two environments configured, staging and production where staging is testnet deployments of the contracts and production corresponds to mainnet deployments.

The current strategy, in order to reduce complexity, is to use the same keys for transaction signing on both Celo and Ethereum networks. Should you desire, the key names to be provisioned can be modified such that the script creates unique keys per-network. Ex:

# Agent Keys
required_keys = [
    "watcher-signer-alfajores",
    "watcher-attestation-alfajores",
    "watcher-signer-kovan",
    "watcher-attestation-kovan",
    "updater-signer-alfajores",
    "updater-attestation-alfajores",
    "updater-signer-kovan",
    "updater-attestation-kovan",
    "processor-signer-alfajores",
    "processor-signer-kovan",
    "relayer-signer-alfajores",
    "relayer-signer-kovan"
]

Run the Key Provisoner Script

AWS_ACCESS_KEY_ID=accesskey AWS_SECRET_ACCESS_KEY=secretkey python3 provision_kms_keys.py  

If the required keys are not present, the script will generate them. If they keys are present, their information will be fetched and displayed non-destructively.

Upon successful operation, the script will output a table of the required keys, their ARNs, ETH addresses (for funding the accounts), and their regions.

Provision IAM Policies and Users

This is an opinionated setup that works for most general agent operations use-cases. The same permissions boundaries can be achieved through different means, like using only Key Policies

Background Reading/Documentation:

• KMS Policy Conditions: https://docs.aws.amazon.com/kms/latest/developerguide/policy-conditions.htm
• KMS Policy Examples: https://docs.aws.amazon.com/kms/latest/developerguide/customer-managed-policies.html
• CMK Alias Authorization: https://docs.aws.amazon.com/kms/latest/developerguide/alias-authorization.html

The following sequence describes how to set up IAM policies staging and production deployments.

• Create two users
  • optics-signer-staging
  • optics-signer-production
  • kms-admin
  • Save IAM credential CSV
• Create staging signer policies
  • staging-processor-signer
  • staging-relayer-signer
  • staging-updater-signer
  • staging-watcher-signer
  • With the following policy, modified appropriately:

      {
          "Version": "2012-10-17",
          "Statement": [
              {
                  "Sid": "OpticsStagingPolicy",
                  "Effect": "Allow",
                  "Action": [
                      "kms:GetPublicKey",
                      "kms:Sign"
                  ],
                  "Resource": "arn:aws:kms:*:11111111111:key/*",
                  "Condition": {
                      "ForAnyValue:StringLike": {
                          "kms:ResourceAliases": "alias/staging-processor*"
                      }
                  }
              }
          ]
      }
  

  • production-processor-signer
  • production-relayer-signer
  • production-updater-signer
  • production-watcher-signer
  • With the following policy, modified appropriately:

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "OpticsProductionPolicy",
                "Effect": "Allow",
                "Action": [
                    "kms:GetPublicKey",
                    "kms:Sign"
                ],
                "Resource": "arn:aws:kms:*:11111111111:key/*",
                "Condition": {
                    "ForAnyValue:StringLike": {
                        "kms:ResourceAliases": "alias/production-processor*"
                    }
                }
            }
        ]
    }
  

• Create kms-admin policy

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "KMSAdminPolicy",
                "Effect": "Allow",
                "Action": [
                    "kms:DescribeCustomKeyStores",
                    "kms:ListKeys",
                    "kms:DeleteCustomKeyStore",
                    "kms:GenerateRandom",
                    "kms:UpdateCustomKeyStore",
                    "kms:ListAliases",
                    "kms:DisconnectCustomKeyStore",
                    "kms:CreateKey",
                    "kms:ConnectCustomKeyStore",
                    "kms:CreateCustomKeyStore"
                ],
                "Resource": "*"
            },
            {
                "Sid": "VisualEditor1",
                "Effect": "Allow",
                "Action": "kms:*",
                "Resource": [
                    "arn:aws:kms:*:756467427867:alias/*",
                    "arn:aws:kms:*:756467427867:key/*"
                ]
            }
        ]
    }
  

  • Create IAM groups
    • staging-signer
    • production-signer
    • kms-admin
  • Add previously created users to the corresponding groups
    • optics-signer-staging -> staging-signer
    • opticics-signer-production -> production-signer
    • kms-admin -> kms-admin