initial merge documentation (#989)

* [WIP] initial merge documentation

Signed-off-by: Alexandra Tran <alexandra.tran@consensys.net>

* more updates

Signed-off-by: Alexandra Tran <alexandra.tran@consensys.net>

* fix md

Signed-off-by: Alexandra Tran <alexandra.tran@consensys.net>

* update accept.txt

Signed-off-by: Alexandra Tran <alexandra.tran@consensys.net>

* more updates

Signed-off-by: Alexandra Tran <alexandra.tran@consensys.net>

* fix link format

Signed-off-by: Alexandra Tran <alexandra.tran@consensys.net>

* minor edits

Signed-off-by: Alexandra Tran <alexandra.tran@consensys.net>

* minor edits

Signed-off-by: Alexandra Tran <alexandra.tran@consensys.net>

* reviewer suggestions

Signed-off-by: Alexandra Tran <alexandra.tran@consensys.net>

* add tutorial

Signed-off-by: Alexandra Tran <alexandra.tran@consensys.net>

* fix md

Signed-off-by: Alexandra Tran <alexandra.tran@consensys.net>

* add beacon node note

Signed-off-by: Alexandra Tran <alexandra.tran@consensys.net>

* integrate reviewer feedback

Signed-off-by: Alexandra Tran <alexandra.tran@consensys.net>

* add diagram

Signed-off-by: Alexandra Tran <alexandra.tran@consensys.net>

* apply reviewer feedback

Signed-off-by: Alexandra Tran <alexandra.tran@consensys.net>

* update nav

Signed-off-by: Alexandra Tran <alexandra.tran@consensys.net>

* fix md

Signed-off-by: Alexandra Tran <alexandra.tran@consensys.net>
pull/1000/head
Alexandra Tran 3 years ago committed by GitHub
parent 0bfe32f4a6
commit 3898aed1d4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
  1. 2
      CI/vale/vale_styles/Vocab/Besu/accept.txt
  2. 7
      docs/Concepts/Consensus-Protocols/Overview-Consensus.md
  3. 40
      docs/Concepts/Consensus-Protocols/Proof-of-Stake.md
  4. 57
      docs/Concepts/Merge.md
  5. 2
      docs/HowTo/Get-Started/Installation-Options/Run-Docker-Image.md
  6. 13
      docs/HowTo/Get-Started/Starting-node.md
  7. 14
      docs/HowTo/Upgrade/Upgrade-Node.md
  8. 80
      docs/Reference/CLI/CLI-Syntax.md
  9. 94
      docs/Tutorials/Merge-Testnet.md
  10. BIN
      docs/images/Execution-Consensus-Clients.png
  11. 9
      docs/index.md
  12. 5
      mkdocs.yml

@ -116,10 +116,12 @@ P[oO]A
Poste
Postgres
P[oO]W
[pP]re
[pP]recompile(d)?
[pP]reidentif(y|ied)
Prometheus
Protobuf
Prysm
Prysmatic
Pushgateway
Pyrmont

@ -4,7 +4,7 @@ description: Besu consensus protocols
# Consensus protocols
Besu implements the following consensus protocols:
Besu supports the following consensus protocols:
* [QBFT](../../HowTo/Configure/Consensus-Protocols/QBFT.md) (proof of authority) - The recommended
enterprise-grade consensus protocol for private networks.
@ -12,7 +12,10 @@ Besu implements the following consensus protocols:
* [Clique](../../HowTo/Configure/Consensus-Protocols/Clique.md) (proof of authority) - Not recommended for
production use.
You can [migrate a network using Clique to another consensus protocol](../../HowTo/Configure/Consensus-Protocols/Clique.md#migrate-from-clique-to-another-consensus-protocol).
* Ethash (proof of work) - Used on Ethereum Mainnet and can also be used in
* [Proof of stake](Proof-of-Stake.md) - Used on Ethereum Mainnet post-[Merge](../../Concepts/Merge.md) and can also be
used on the [Merge testnet](../../Tutorials/Merge-Testnet.md).
* [Ethash](https://ethereum.org/en/developers/docs/consensus-mechanisms/pow/) (proof of work) - Used on Ethereum Mainnet
pre-[Merge](../../Concepts/Merge.md) and can also be used in
[small development networks](../../Tutorials/Private-Network/Create-Private-Network.md).
See a [comparison of the proof of authority consensus protocols](Comparing-PoA.md).

@ -0,0 +1,40 @@
---
description: Ethereum proof of stake
---
# Proof of stake
[The Merge](../Merge.md) transitions Ethereum Mainnet to
[proof of stake (PoS)](https://ethereum.org/en/developers/docs/consensus-mechanisms/pos/) consensus.
In Ethereum's PoS, you must run a [full node](../Merge.md#execution-and-consensus-clients) and
[stake 32 ETH](https://ethereum.org/en/staking/) to become a validator.
!!! important
Withdrawing staked ETH isn't yet supported and will be included in a separate upgrade following The Merge.
The PoS mechanism randomly chooses validators to propose or validate blocks on the
[Beacon Chain](https://ethereum.org/en/upgrades/beacon-chain/) (consensus blocks) in defined time frames (slots).
Proposers are responsible for proposing new consensus blocks, and non-proposing validators are responsible for
validating (attesting to) proposed blocks.
Validators are rewarded for proposing and attesting to consensus blocks eventually included in the Beacon Chain, and
their stake is slashed if they fail to validate or if they attest to malicious blocks, incentivizing good behavior.
Each consensus block contains an execution payload, which contains a list of transactions and other data required to
execute and validate the payload.
When a node validates a consensus block, its [consensus client](../Merge.md#execution-and-consensus-clients) processes
the block and sends the execution payload to the [execution client](../Merge.md#execution-and-consensus-clients), which:
1. Assembles a block on the execution layer (execution block).
1. Verifies pre-conditions.
1. Executes transactions.
1. Verifies post-conditions.
1. Sends the validity result back to the consensus client.
If the block is valid, the execution client includes it in the execution chain and stores the new state in execution
state storage.
If at least 128 validators attest to a consensus block within a slot, it's included in the Beacon Chain.

@ -0,0 +1,57 @@
---
description: What is the Merge?
---
# The Merge
The Ethereum upgrade known as [The Merge](https://ethereum.org/en/upgrades/merge/) will merge the [Beacon Chain] into
Ethereum Mainnet, turning Mainnet into a combination of an
[execution layer and consensus layer](#execution-and-consensus-clients).
The Merge will transition Mainnet from proof of work to [proof of stake consensus](Consensus-Protocols/Proof-of-Stake.md).
## Execution and consensus clients
After The Merge, a full Ethereum Mainnet node will be a combination of an execution client (previously called an
[Ethereum 1.0](https://blog.ethereum.org/2022/01/24/the-great-eth2-renaming/) client) and a consensus client (previously
called an [Ethereum 2.0](https://blog.ethereum.org/2022/01/24/the-great-eth2-renaming/) client).
Execution clients, such as Besu, manage the state and execute transactions on the execution layer.
Consensus clients, such as [Teku], maintain [consensus](Consensus-Protocols/Proof-of-Stake.md) on the [Beacon Chain]
(consensus layer).
Execution and consensus clients communicate with each other using the
[Engine API](https://github.com/ethereum/execution-apis/blob/main/src/engine/specification.md).
Execution clients serve [JSON-RPC API](../Reference/API-Methods.md) requests and consensus clients serve
[REST API](https://docs.teku.consensys.net/en/stable/Reference/Rest_API/Rest/) requests.
Execution clients communicate among each other in a peer-to-peer network, and consensus clients do the same.
![Ethereum Merge node](../images/Execution-Consensus-Clients.png)
### Run a node
After The Merge, a Mainnet node operator must run both an execution client and a consensus client at the same time.
You must run a node to become a Mainnet [validator](Consensus-Protocols/Proof-of-Stake.md), but you can also run a
non-validator node.
!!! note
Consensus clients that don't perform validator duties are referred to as "beacon nodes" or "Beacon Chain clients."
You can
[test Besu as an execution client on the Merge testnet](../Tutorials/Merge-Testnet.md).
You must test it with a consensus client, for example, [Teku].
You can run Besu as an execution client on Ethereum Mainnet using the same
[installation options](../HowTo/Get-Started/Installation-Options).
Consensus clients compatible with Besu include:
- [Teku] (recommended)
- [Lighthouse](https://lighthouse.sigmaprime.io/)
- [Lodestar](https://lodestar.chainsafe.io/)
- [Prysm](https://github.com/prysmaticlabs/prysm)
- [Nimbus](https://nimbus.team/)
<!-- links -->
[Beacon Chain]: https://ethereum.org/en/upgrades/beacon-chain/
[Teku]: https://docs.teku.consensys.net/en/stable/

@ -103,7 +103,7 @@ To run a node that mines blocks at a rate suitable for testing purposes with Web
docker run -p 8546:8546 --mount type=bind,source=/<myvolume/besu/testnode>,target=/var/lib/besu hyperledger/besu:latest --miner-enabled --miner-coinbase fe3b557e8fb62b89f4916b721be55ceb828dbd73 --rpc-ws-enabled --network=dev --data-path=/var/lib/besu
```
### Run a node on Rinkeby Testnet
### Run a node on Rinkeby testnet
To run a node on Rinkeby:

@ -27,8 +27,9 @@ To delete the local block data, delete the `database` directory in the
## Genesis configuration
Besu specifies the genesis configuration, and sets the network ID and bootnodes when connecting to
[Mainnet](#run-a-node-on-ethereum-mainnet), [Goerli](#run-a-node-on-goerli-testnet),
[Rinkeby](#run-a-node-on-rinkeby-testnet), and [Ropsten](#run-a-node-on-ropsten-testnet).
[Ropsten](#run-a-node-on-ropsten-testnet), [Rinkeby](#run-a-node-on-rinkeby-testnet),
[Goerli](#run-a-node-on-goerli-testnet), [Kiln](#run-a-node-on-kiln-testnet), and
[Mainnet](#run-a-node-on-ethereum-mainnet).
When you specify [`--network=dev`](../../Reference/CLI/CLI-Syntax.md#network), Besu uses the
development mode genesis configuration with a fixed low difficulty. A node started with
@ -147,6 +148,14 @@ besu --network=goerli --data-path=<path>/<goerlidata-path>
Where `<path>` and `<goerlidata-path>` are the path and directory to save the Goerli chain data to.
## Run a node on Kiln testnet
You can [test Besu as an execution client](../../Tutorials/Merge-Testnet.md#start-besu) on the
[Kiln Merge testnet](https://blog.ethereum.org/2022/03/14/kiln-merge-testnet/).
You must also run a [consensus client](../../Concepts/Merge.md#execution-and-consensus-clients) with Besu on the Merge
testnet.
For example, [Teku](https://docs.teku.consensys.net/en/stable/).
## Run a node on Ethereum Mainnet
To run a node on the Ethereum Mainnet:

@ -4,7 +4,7 @@ description: Upgrade Besu
# Upgrading your Besu node
We recommend:
When upgrading your Besu node, we recommend:
* Using an orchestration method (for example, Ansible or Chef) to keep all nodes in sync with your
desired configuration.
@ -16,14 +16,14 @@ You can use the [Ansible role on Galaxy](https://galaxy.ansible.com/pegasyseng/h
directly or customize it to suit your needs.
Upgrade the Besu version on nodes by running the play with the new version. For more information,
For more information, see the "Read Me" button on the
[Ansible Galaxy Besu page](https://galaxy.ansible.com/pegasyseng/hyperledger_besu).
For more information, select **Read Me** on the
[Ansible Galaxy Besu page](https://galaxy.ansible.com/consensys/hyperledger_besu).
The play-book:
The playbook:
1. Stops Besu
1. Downloads the updated version
1. Applies any new configuration
1. Stops Besu.
1. Downloads the updated version.
1. Applies any new configuration.
1. Starts Besu.
## Finding peers on restarting

@ -455,150 +455,153 @@ The default is `true`.
You can override the default DNS server if it's unreliable or doesn't serve TCP DNS requests, using the
[experimental option](#xhelp) `--Xp2p-dns-discovery-server=<HOST>`.
### `engine-rpc-http-port`
### `engine-host-allowlist`
=== "Syntax"
```bash
--engine-rpc-http-port=<PORT>
--engine-host-allowlist=<hostname>[,<hostname>...]... or "*"
```
=== "Example"
```bash
--engine-rpc-http-port=8550
--engine-host-allowlist=localhost,127.0.0.1
```
=== "Environment variable"
```bash
BESU_ENGINE_RPC_HTTP_PORT=8550
BESU_ENGINE_HOST_ALLOWLIST=localhost,127.0.0.1
```
=== "Configuration file"
```bash
engine-rpc-http-port=8550
engine-host-allowlist=localhost,127.0.0.1
```
The listening port for the Engine API calls (`ENGINE`, `ETH`) for JSON-RPC over HTTP.
A comma-separated list of hostnames to allow for Engine API access (applies to both HTTP and WebSocket).
### `engine-rpc-ws-port`
!!!tip
To allow all hostnames, use `"*"`. We don't recommend allowing all hostnames in production
environments.
### `engine-jwt-enabled`
=== "Syntax"
```bash
--engine-rpc-ws-port=<PORT>
--engine-jwt-enabled
```
=== "Example"
```bash
--engine-rpc-ws-port=8551
--engine-jwt-enabled=true
```
=== "Environment variable"
```bash
BESU_ENGINE_RPC_WS_PORT=8551
BESU_ENGINE_JWT_ENABLED=true
```
=== "Configuration file"
```bash
engine-rpc-ws-port=8551
engine-jwt-enabled=true
```
Enables or disables authentication for Engine APIs. The default is `false`.
The listening port for the Engine API calls (`ENGINE`, `ETH`) for JSON-RPC over WebSockets.
### `engine-host-allowlist`
### `engine-jwt-secret`
=== "Syntax"
```bash
--engine-host-allowlist=<hostname>[,<hostname>...]... or "*"
--engine-jwt-secret=<FILE>
```
=== "Example"
```bash
--engine-host-allowlist=localhost,127.0.0.1
--engine-jwt-secret=jwt.hex
```
=== "Environment variable"
```bash
BESU_ENGINE_HOST_ALLOWLIST=localhost,127.0.0.1
BESU_ENGINE_JWT_SECRET="publicKey.pem"
```
=== "Configuration file"
```bash
engine-host-allowlist=localhost,127.0.0.1
engine-jwt-secret="jwt.hex"
```
A comma-separated list of hostnames to allow for Engine API access (applies to both HTTP and WebSockets).
Shared secret used to authenticate [consensus clients](../../Concepts/Merge.md) when using the Engine JSON-RPC API (both
HTTP and WebSocket).
Contents of file must be at least 32 hex-encoded bytes and not begin with `0x`.
May be a relative or absolute path.
See an [example of how to generate this](../../Tutorials/Merge-Testnet.md#prerequisites).
!!!tip
To allow all hostnames, use `"*"`. We don't recommend allowing all hostnames in production
environments.
### `engine-jwt-enabled`
### `engine-rpc-http-port`
=== "Syntax"
```bash
--engine-jwt-enabled
--engine-rpc-http-port=<PORT>
```
=== "Example"
```bash
--engine-jwt-enabled=true
--engine-rpc-http-port=8550
```
=== "Environment variable"
```bash
BESU_ENGINE_JWT_ENABLED=true
BESU_ENGINE_RPC_HTTP_PORT=8550
```
=== "Configuration file"
```bash
engine-jwt-enabled=true
engine-rpc-http-port=8550
```
Enables or disables authentication for Engine APIs. The default is `false`.
### `engine-jwt-secret`
The listening port for the Engine API calls (`ENGINE`, `ETH`) for JSON-RPC over HTTP.
### `engine-rpc-ws-port`
=== "Syntax"
```bash
--engine-jwt-secret=<FILE>
--engine-rpc-ws-port=<PORT>
```
=== "Example"
```bash
--engine-jwt-secret=jwt.hex
--engine-rpc-ws-port=8551
```
=== "Environment variable"
```bash
BESU_ENGINE_JWT_SECRET="publicKey.pem"
BESU_ENGINE_RPC_WS_PORT=8551
```
=== "Configuration file"
```bash
engine-jwt-secret="jwt.hex"
engine-rpc-ws-port=8551
```
The shared secret used for JSON-RPC (both HTTP and WebSocket) authentication of Engine API clients.
Contents of file must be at least 32 hex-encoded bytes and not begin with `0x`. May be a relative or absolute path.
The listening port for the Engine API calls (`ENGINE`, `ETH`) for JSON-RPC over WebSocket.
### `ethstats`
@ -1645,7 +1648,8 @@ Possible values are:
| Network | Chain | Type | Default Sync Mode | Description |
|:----------|:------|:------------|:-------------------|:---------------------------------------------------------------|
| `mainnet` | ETH | Production | [FAST](#sync-mode) | The main network |
| `ropsten` | ETH | Test | [FAST](#sync-mode) | A PoW network similar to the current main Ethereum network |
| `kiln` | ETH | Test | [FAST](#sync-mode) | A PoS network similar to the main Ethereum network post-[Merge](../../Concepts/Merge.md) |
| `ropsten` | ETH | Test | [FAST](#sync-mode) | A PoW network similar to the main Ethereum network pre-[Merge](../../Concepts/Merge.md) |
| `rinkeby` | ETH | Test | [FAST](#sync-mode) | A PoA network using Clique |
| `goerli` | ETH | Test | [FAST](#sync-mode) | A PoA network using Clique |
| `dev` | ETH | Development | [FULL](#sync-mode) | A PoW network with a low difficulty to enable local CPU mining |

@ -0,0 +1,94 @@
---
Description: How to run Besu and Teku on the Merge testnet
---
# Run clients on the Merge testnet
You can test Besu as an [execution client](../Concepts/Merge.md) and [Teku](https://docs.teku.consensys.net/en/stable/)
as a [consensus client](../Concepts/Merge.md) on the [Kiln Merge testnet](https://blog.ethereum.org/2022/03/14/kiln-merge-testnet/).
## Prerequisites
- Install [Besu](../HowTo/Get-Started/Installation-Options/Options.md) and
[Teku](https://docs.teku.consensys.net/en/stable/HowTo/Get-Started/Installation-Options/Install-Binaries/).
- Generate the JWT secret:
```bash
openssl rand -hex 32 | tr -d "\n" > jwtsecret.hex
```
You must pass this file to both Besu and Teku.
This is a shared secret the clients use to authenticate each other when using the Engine API.
- Create a test Ethereum address to use as the
[default fee recipient](https://docs.teku.consensys.net/en/latest/Reference/CLI/CLI-Syntax/#validators-proposer-default-fee-recipient)
when starting Teku.
## Start Besu
Run the following command:
```bash
besu \
--network=kiln \
--rpc-http-enabled=true \
--rpc-http-host="0.0.0.0" \
--rpc-http-cors-origins="*" \
--rpc-ws-enabled=true \
--rpc-ws-host="0.0.0.0" \
--host-allowlist="*" \
--engine-host-allowlist="*" \
--engine-jwt-enabled=true \
--engine-rpc-http-port=8550 \
--engine-rpc-ws-port=8551 \
--Xmerge-support=true \
--engine-jwt-secret=<path_to_jwtsecret>
```
See the [`--engine-*`](../Reference/CLI/CLI-Syntax.md#engine-host-allowlist) CLI options for more information on running
Besu as an execution client.
## Start Teku
Run the following command:
```bash
teku \
--data-path "datadir-teku" \
--network kiln \
--p2p-discovery-bootnodes "enr:-Iq4QMCTfIMXnow27baRUb35Q8iiFHSIDBJh6hQM5Axohhf4b6Kr_cOCu0htQ5WvVqKvFgY28893DHAg8gnBAXsAVqmGAX53x8JggmlkgnY0gmlwhLKAlv6Jc2VjcDI1NmsxoQK6S-Cii_KmfFdUJL2TANL3ksaKUnNXvTCv1tLwXs0QgIN1ZHCCIyk" \
--ee-endpoint http://localhost:8550 \
--Xee-version kilnv2 \
--ee-jwt-secret-file <path_to_jwtsecret> \
--log-destination console \
--validators-proposer-default-fee-recipient=<eth_address_of_default_fee_recipient>
```
This runs Teku as a beacon node (without validator duties).
See the Teku [`--validators-*`](https://docs.teku.consensys.net/en/latest/Reference/CLI/CLI-Syntax/#validator-keys) CLI
options for information on running Teku as a validator.
After starting Besu and Teku, your node should start syncing and connecting to peers.
!!! example
=== "Besu logs"
```bash
2022-03-21 20:42:09.295-07:00 | EthScheduler-Timer-0 | INFO | FullSyncTargetManager | No sync target, waiting for peers. Current peers: 0
2022-03-21 20:42:14.298-07:00 | EthScheduler-Timer-0 | INFO | FullSyncTargetManager | No sync target, waiting for peers. Current peers: 0
2022-03-21 20:42:14.848-07:00 | nioEventLoopGroup-3-8 | INFO | FullSyncTargetManager | No sync target, waiting for peers. Current peers: 4
2022-03-21 20:42:18.452-07:00 | nioEventLoopGroup-3-8 | INFO | SyncTargetManager | Found common ancestor with peer Peer 0xab3a286b181721c794... at block 55127
2022-03-21 20:42:18.454-07:00 | nioEventLoopGroup-3-8 | INFO | PipelineChainDownloader | PipelineChain download complete
```
=== "Teku logs"
```bash
2022-03-21 20:43:24.355 INFO - Syncing *** Target slot: 76092, Head slot: 2680, Remaining slots: 73412, Connected peers: 8
2022-03-21 20:43:36.363 INFO - Syncing *** Target slot: 76093, Head slot: 2879, Remaining slots: 73214, Connected peers: 10
2022-03-21 20:43:48.327 INFO - Syncing *** Target slot: 76094, Head slot: 3080, Remaining slots: 73014, Connected peers: 8
2022-03-21 20:44:00.339 INFO - Syncing *** Target slot: 76095, Head slot: 3317, Remaining slots: 72778, Connected peers: 6
2022-03-21 20:44:12.353 INFO - Syncing *** Target slot: 76096, Head slot: 3519, Remaining slots: 72577, Connected peers: 9
```

Binary file not shown.

After

Width:  |  Height:  |  Size: 106 KiB

@ -9,10 +9,11 @@ description: Besu is an open-source Ethereum client developed under the Apache 2
## What is Hyperledger Besu?
Hyperledger Besu is an open-source Ethereum client developed under the Apache 2.0 license and
written in Java. It runs on the Ethereum public network, private networks, and test networks such
as Rinkeby, Ropsten, and Görli. Besu implements proof of work (Ethash) and proof of authority (IBFT
2.0, Clique, and QBFT) consensus mechanisms.
Hyperledger Besu is an open-source Ethereum client developed under the Apache 2.0 license and written in Java.
It runs on Ethereum Mainnet, private networks, and test networks such as Rinkeby, Ropsten, Goerli, and the Merge testnet.
Besu serves as an [execution client](Concepts/Merge.md) on Ethereum Mainnet and the Merge testnet.
Besu implements proof of authority (QBFT, IBFT 2.0, and Clique) and proof of work (Ethash) consensus mechanisms.
You can use Besu to develop enterprise applications requiring secure, high-performance transaction
processing in a private network.

@ -185,14 +185,17 @@ nav:
- Create a cluster: Tutorials/Kubernetes/Create-Cluster.md
- Deploy charts: Tutorials/Kubernetes/Deploy-Charts.md
- Maintenance: Tutorials/Kubernetes/Maintenance.md
- Production: Tutorials/Kubernetes/Production.md
- Production: Tutorials/Kubernetes/Production.md
- Run clients on the Merge testnet: Tutorials/Merge-Testnet.md
- Concepts:
- Architecture: Concepts/ArchitectureOverview.md
- Consensus protocols:
- Overview: Concepts/Consensus-Protocols/Overview-Consensus.md
- Comparing PoA consensus protocols: Concepts/Consensus-Protocols/Comparing-PoA.md
- Proof of stake: Concepts/Consensus-Protocols/Proof-of-Stake.md
- Data storage formats: Concepts/Data-Storage-Formats.md
- Events and logs: Concepts/Events-and-Logs.md
- The Merge: Concepts/Merge.md
- Mining: Concepts/Mining.md
- Monitoring: Concepts/Monitoring.md
- Network ID and chain ID: Concepts/NetworkID-And-ChainID.md

Loading…
Cancel
Save