CipherLabs
/
hyperlane-monorepo
mirror of https://github.com/hyperlane-xyz/hyperlane-monorepo
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[Feature] added AWS signers as option to Helm (#567)

Browse Source 
* added AWS signers as option

* updated README

* updated dockerfile
buddies-main-deployment
Conner Swann 3 years ago committed by GitHub
parent d48ef53f22
commit 93b3f22201
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
7 changed files with 286 additions and 23 deletions
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Show Stats Download Patch File Download Diff File
  1. 3
      rust/Dockerfile
  2. 146
      rust/README.md
  3. 22
      rust/helm/optics-agent/templates/kathy-statefulset.yaml
  4. 22
      rust/helm/optics-agent/templates/processor-statefulset.yaml
  5. 22
      rust/helm/optics-agent/templates/relayer-statefulset.yaml
  6. 37
      rust/helm/optics-agent/templates/updater-statefulset.yaml
  7. 57
      rust/helm/optics-agent/values.yaml

3
rust/Dockerfile
Unescape View File

@ -1,6 +1,6 @@
# syntax=docker/dockerfile:experimental
FROM rust:1.52 as builder
FROM rust:1.54 as builder
WORKDIR /usr/src
# 1a: Prepare for static linking
@ -14,6 +14,7 @@ COPY optics-core ./optics-core
COPY optics-ethereum ./optics-ethereum
COPY optics-base ./optics-base
COPY optics-test ./optics-test
COPY kms-cli ./kms-cli
COPY processor ./processor
COPY relayer ./relayer
COPY watcher ./watcher

146
rust/README.md
Unescape View File

@ -121,3 +121,149 @@ We use the tokio async runtime environment. Please see the docs
- follow the pattern in `optics-base/src/main.rs`
- make a `config` folder and a toml file
- Make sure to include your own settings from above
# Provisoning KMS Keys
There exists a script in this repository (`provision_kms_keys.py `) that facilitates KMS key provisioning for agent roles.
The script will produce a single set of keys per "environment." Where an __environment__ is a logical set of smart contrace deployments. By default there are two environments configured, `staging` and `production` where `staging` is testnet deployments of the contracts and `production` corresponds to mainnet deployments.
The current strategy, in order to reduce complexity, is to use the same keys for transaction signing on both Celo and Ethereum networks. Should you desire, the key names to be provisioned can be modified such that the script creates unique keys per-network. Ex:
```
# Agent Keys
required_keys = [
"watcher-signer-alfajores",
"watcher-attestation-alfajores",
"watcher-signer-kovan",
"watcher-attestation-kovan",
"updater-signer-alfajores",
"updater-attestation-alfajores",
"updater-signer-kovan",
"updater-attestation-kovan",
"processor-signer-alfajores",
"processor-signer-kovan",
"relayer-signer-alfajores",
"relayer-signer-kovan"
]
```
## Run the Key Provisoner Script
```
AWS_ACCESS_KEY_ID=accesskey AWS_SECRET_ACCESS_KEY=secretkey python3 provision_kms_keys.py
```
If the required keys are not present, the script will generate them. If they keys _are_ present, their information will be fetched and displayed non-destructively.
Upon successful operation, the script will output a table of the required keys, their ARNs, ETH addresses (for funding the accounts), and their regions.
## Provision IAM Policies and Users
This is an opinionated setup that works for most general agent operations use-cases. The same permissions boundaries can be achieved through different means, like using only [Key Policies](https://docs.aws.amazon.com/kms/latest/developerguide/key-policies.html)
Background Reading/Documentation:
- KMS Policy Conditions: https://docs.aws.amazon.com/kms/latest/developerguide/policy-conditions.htm
- KMS Policy Examples: https://docs.aws.amazon.com/kms/latest/developerguide/customer-managed-policies.html
- CMK Alias Authorization: https://docs.aws.amazon.com/kms/latest/developerguide/alias-authorization.html
The following sequence describes how to set up IAM policies staging and production deployments.
- Create two users
- optics-signer-staging
- optics-signer-production
- kms-admin
- Save IAM credential CSV
- Create staging signer policies
- staging-processor-signer
- staging-relayer-signer
- staging-updater-signer
- staging-watcher-signer
- With the following policy, modified appropriately:
```
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "OpticsStagingPolicy",
"Effect": "Allow",
"Action": [
"kms:GetPublicKey",
"kms:Sign"
],
"Resource": "arn:aws:kms:*:11111111111:key/*",
"Condition": {
"ForAnyValue:StringLike": {
"kms:ResourceAliases": "alias/staging-processor*"
}
}
}
]
}
```
- production-processor-signer
- production-relayer-signer
- production-updater-signer
- production-watcher-signer
- With the following policy, modified appropriately:
```
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "OpticsProductionPolicy",
"Effect": "Allow",
"Action": [
"kms:GetPublicKey",
"kms:Sign"
],
"Resource": "arn:aws:kms:*:11111111111:key/*",
"Condition": {
"ForAnyValue:StringLike": {
"kms:ResourceAliases": "alias/production-processor*"
}
}
}
]
}
```
- Create kms-admin policy
```
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "KMSAdminPolicy",
"Effect": "Allow",
"Action": [
"kms:DescribeCustomKeyStores",
"kms:ListKeys",
"kms:DeleteCustomKeyStore",
"kms:GenerateRandom",
"kms:UpdateCustomKeyStore",
"kms:ListAliases",
"kms:DisconnectCustomKeyStore",
"kms:CreateKey",
"kms:ConnectCustomKeyStore",
"kms:CreateCustomKeyStore"
],
"Resource": "*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": "kms:*",
"Resource": [
"arn:aws:kms:*:756467427867:alias/*",
"arn:aws:kms:*:756467427867:key/*"
]
}
]
}
```
- Create IAM groups
- staging-signer
- production-signer
- kms-admin
- Add previously created users to the corresponding groups
- optics-signer-staging -> staging-signer
- opticics-signer-production -> production-signer
- kms-admin -> kms-admin

22
rust/helm/optics-agent/templates/kathy-statefulset.yaml
Unescape View File

@ -50,11 +50,20 @@ spec:
imagePullPolicy: {{ .Values.image.pullPolicy }}
command: ["./kathy"]
env:
# Set AWS Keys if present in config
{{- if .Values.optics.awsAccessKeyId }}
- name: AWS_ACCESS_KEY_ID
value: {{ .Values.optics.awsAccessKeyId }}
{{- end }}
{{- if .Values.optics.awsSecretAccessKey }}
- name: AWS_SECRET_ACCESS_KEY
value: {{ .Values.optics.awsSecretAccessKey }}
{{- end }}
# Set the DB location to be the volume
- name: OPT_BASE_DBPATH
value: /usr/share/optics
# Set the log formatting
- name: OPT_BASE_TRACING_STYLE
- name: OPT_BASE_TRACING_FMT
value: "json"
# Sets the config folder to use
- name: RUN_ENV
@ -89,8 +98,17 @@ spec:
{{- end }}
# Transaction Signer Keys
{{- range .Values.optics.kathy.transactionSigners }}
{{- if .hexKey }}
- name: OPT_BASE_SIGNERS_{{ .name | upper }}_KEY
value: {{ .key }}
value: {{ .hexKey }}
{{- else }}
- name: OPT_BASE_SIGNERS_{{ .name | upper }}_TYPE
value: "aws"
- name: OPT_BASE_SIGNERS_{{ .name | upper }}_AWS_ID
value: {{ .aws.keyId }}
- name: OPT_BASE_SIGNERS_{{ .name | upper }}_AWS_REGION
value: {{ .aws.region }}
{{- end }}
{{- end }}
# Jaeger Config
{{- if .Values.optics.tracing.uri }}

22
rust/helm/optics-agent/templates/processor-statefulset.yaml
Unescape View File

@ -50,11 +50,20 @@ spec:
imagePullPolicy: {{ .Values.image.pullPolicy }}
command: ["./processor"]
env:
# Set AWS Keys if present in config
{{- if .Values.optics.awsAccessKeyId }}
- name: AWS_ACCESS_KEY_ID
value: {{ .Values.optics.awsAccessKeyId }}
{{- end }}
{{- if .Values.optics.awsSecretAccessKey }}
- name: AWS_SECRET_ACCESS_KEY
value: {{ .Values.optics.awsSecretAccessKey}}
{{- end }}
# Set the DB location to be the volume
- name: OPT_BASE_DBPATH
value: /usr/share/optics
# Set the log formatting
- name: OPT_BASE_TRACING_STYLE
- name: OPT_BASE_TRACING_FMT
value: "json"
# Sets the config folder to use
- name: RUN_ENV
@ -89,8 +98,17 @@ spec:
{{- end }}
# Transaction Signer Keys
{{- range .Values.optics.processor.transactionSigners }}
{{- if .hexKey }}
- name: OPT_BASE_SIGNERS_{{ .name | upper }}_KEY
value: {{ .key }}
value: {{ .hexKey }}
{{- else }}
- name: OPT_BASE_SIGNERS_{{ .name | upper }}_TYPE
value: "aws"
- name: OPT_BASE_SIGNERS_{{ .name | upper }}_AWS_ID
value: {{ .aws.keyId }}
- name: OPT_BASE_SIGNERS_{{ .name | upper }}_AWS_REGION
value: {{ .aws.region }}
{{- end }}
{{- end }}
# Jaeger Config
{{- if .Values.optics.tracing.uri }}

22
rust/helm/optics-agent/templates/relayer-statefulset.yaml
Unescape View File

@ -50,11 +50,20 @@ spec:
imagePullPolicy: {{ .Values.image.pullPolicy }}
command: ["./relayer"]
env:
# Set AWS Keys if present in config
{{- if .Values.optics.awsAccessKey }}
- name: AWS_ACCESS_KEY_ID
value: {{ .Values.optics.awsAccessKeyId }}
{{- end }}
{{- if .Values.optics.awsSecretKey }}
- name: AWS_SECRET_ACCESS_KEY
value: {{ .Values.optics.awsSecretAccessKey }}
{{- end }}
# Set the DB location to be the volume
- name: OPT_BASE_DBPATH
value: /usr/share/optics
# Set the log formatting
- name: OPT_BASE_TRACING_STYLE
- name: OPT_BASE_TRACING_FMT
value: "json"
# Sets the config folder to use
- name: RUN_ENV
@ -89,8 +98,17 @@ spec:
{{- end }}
# Transaction Signer Keys
{{- range .Values.optics.relayer.transactionSigners }}
{{- if .hexKey }}
- name: OPT_BASE_SIGNERS_{{ .name | upper }}_KEY
value: {{ .key }}
value: {{ .hexKey }}
{{- else }}
- name: OPT_BASE_SIGNERS_{{ .name | upper }}_TYPE
value: "aws"
- name: OPT_BASE_SIGNERS_{{ .name | upper }}_AWS_ID
value: {{ .aws.keyId }}
- name: OPT_BASE_SIGNERS_{{ .name | upper }}_AWS_REGION
value: {{ .aws.region }}
{{- end}}
{{- end }}
# Jaeger Config
{{- if .Values.optics.tracing.uri }}

37
rust/helm/optics-agent/templates/updater-statefulset.yaml
Unescape View File

@ -50,11 +50,20 @@ spec:
imagePullPolicy: {{ .Values.image.pullPolicy }}
command: ["./updater"]
env:
# Set AWS Keys if present in config
{{- if .Values.optics.awsAccessKeyId }}
- name: AWS_ACCESS_KEY_ID
value: {{ .Values.optics.awsAccessKeyId }}
{{- end }}
{{- if .Values.optics.awsSecretAccessKey }}
- name: AWS_SECRET_ACCESS_KEY
value: {{ .Values.optics.awsSecretAccessKey }}
{{- end }}
# Set the DB location to be the volume
- name: OPT_BASE_DBPATH
value: /usr/share/optics
# Set the log formatting
- name: OPT_BASE_TRACING_STYLE
- name: OPT_BASE_TRACING_FMT
value: "json"
# Sets the config folder to use
- name: RUN_ENV
@ -89,12 +98,30 @@ spec:
{{- end }}
# Transaction Signer Keys
{{- range .Values.optics.updater.transactionSigners }}
- name: OPT_UPDATER_SIGNERS_{{ .name | upper }}_KEY
value: {{ .key }}
{{- if .hexKey }}
- name: OPT_BASE_SIGNERS_{{ .name | upper }}_KEY
value: {{ .hexKey }}
{{- else }}
- name: OPT_BASE_SIGNERS_{{ .name | upper }}_TYPE
value: "aws"
- name: OPT_BASE_SIGNERS_{{ .name | upper }}_ID
value: {{ .aws.keyId }}
- name: OPT_BASE_SIGNERS_{{ .name | upper }}_REGION
value: {{ .aws.region }}
{{- end }}
{{- end }}
# Attestation Signer Key
- name: OPT_UPDATER_UPDATER_KEY
value: {{ .Values.optics.updater.attestationSigner }}
{{- if .Values.optics.updater.attestationSigner.hexKey }}
- name: OPT_BASE_UPDATER_KEY
value: {{ .Values.optics.updater.attestationSigner.hexKey }}
{{- else }}
- name: OPT_BASE_UPDATER_TYPE
value: "aws"
- name: OPT_BASE_UPDATER_ID
value: {{ .Values.optics.updater.attestationSigner.aws.keyId }}
- name: OPT_BASE_UPDATER_REGION
value: {{ .Values.optics.updater.attestationSigner.aws.region }}
{{- end }}
{{- if .Values.optics.tracing.uri }}
- name: OPT_BASE_TRACING_JAEGER_NAME
value: {{ include "optics-agent.fullname" . }}-updater

57
rust/helm/optics-agent/values.yaml
Unescape View File

@ -23,6 +23,8 @@ optics:
connectionType: # "http"
# -- Connection string pointing to an RPC endpoint for the home chain
connectionUrl: # "https://goerli.infura.io/v3/<KEY>"
awsAccessKeyId: ""
awsSecretAccessKey: ""
# -- Replica chain overrides, a sequence
replicaChains:
@ -40,41 +42,74 @@ optics:
updater:
# -- Enables or disables the updater
enabled: false
# -- Trnsaction Signing keys for home and replica(s)
# -- Transaction Signing keys for home and replica(s)
# Note: Only select one kind of key, either hexKey OR AWS. hexKey will take precedence over AWS if set.
transactionSigners:
- name: "goerli"
key: ""
hexKey: ""
aws:
keyId: ""
region: ""
- name: "alfajores"
key: ""
# -- Specialized key used by updater and watcher used to sign attestations, separate from updater.keys
attestationSigner: ""
hexKey: ""
aws:
keyId: ""
region: ""
# -- Specialized key used by updater and watcher used to sign attestations, separate from updater.transactionSigners
# Note: Only select one kind of key, either hexKey OR AWS. hexKey will take precedence over AWS if set.
attestationSigner:
hexKey: ""
aws:
keyId: ""
region: ""
# -- How long to wait between checking for updates
pollingInterval: # 5
updatePause: # 15
relayer:
# -- Enables or disables the relayer
enabled: false
# Note: Only select one kind of key, either hexKey OR AWS. hexKey will take precedence over AWS if set.
transactionSigners:
- name: "goerli"
key: ""
hexKey: ""
aws:
keyId: ""
region: ""
- name: "alfajores"
key: ""
hexKey: ""
aws:
keyId: ""
region: ""
pollingInterval: # 10
processor:
enabled: false
# Note: Only select one kind of key, either hexKey OR AWS. hexKey will take precedence over AWS if set.
transactionSigners:
- name: "goerli"
key: ""
hexKey: ""
aws:
keyId: ""
region: ""
- name: "alfajores"
key: ""
hexKey: ""
aws:
keyId: ""
region: ""
pollingInterval: # 10
kathy:
enabled: false
# Note: Only select one kind of key, either hexKey OR AWS. hexKey will take precedence over AWS if set.
transactionSigners:
- name: "goerli"
key: ""
hexKey: ""
aws:
keyId: ""
region: ""
- name: "alfajores"
key: ""
hexKey: ""
aws:
keyId: ""
region: ""
messageInterval: # 100
# -- Configuration for Kathy's message generation code
chatGenConfig:

Write Preview
Loading…
Cancel
Save