woop/specifications/p2p/peerdiscovery.md

# Peer Discovery

## Introduction
For any decentralized blockchain network, peer discovery is the essential step to create a real p2p mesh network.
New node uses peer discovery to join the blockchain network.
In Harmony network, the new node will also need to find the right shard to join.
The proposed peer discovery process works as illustrated below.

## P2P Overlay Networks

Each Harmony node joins two P2P overlay networks:

* The **global** overlay, used for *inter*-shard communication; and
* The **shard** overlay, used for *intra*-shard communication.

For each overlay network, the node:

* Initiates up to 16 outbound peer connections; and
* Accepts up to 128 inbound peer connections.

## Bootnodes
New nodes contact any bootnode to get a list of 32 random nodes at the global level.
A list of the IP addresses of the bootnodes are hard-coded in the node software.
Alternatively, the new node may use a Harmony owned DNSseed (ex. bootnode.harmony.one) to find a list of long running bootnodes if the hard-coded bootnodes are not accepting connections due to network congestion.

## Register
To prevent Sybil and DoS attacks, new nodes need to solve a PoW puzzle in order to join the p2p network.
We support different type of nodes joining the network.
For a validator node, the difficulty of the puzzle can be adjusted to within 1 minute on a typical server machine (4 cores, 2.7 GHz).
For a light client such like a mobile device or IoT device that won't join the consensus, the puzzle should be adjusted to within 5 seconds on a typical mobile processor, ex, 2 ARM cores of 1GHz.

Bootnodes use the Kademlia based DHT to return a list of 32 peer nodes to the new node.
Each bootnode may keep up to 3,000 IP addresses of the peer nodes in the network.
The Kademlia based DHT calculates the XOR distances of the public keys of the nodes to keep nodes into different buckets.

After joining the network, for a full node that will participate in the consensus, it needs to broadcast the network via the peers returned by bootnodes for Proof-of-Stake verification.
The first message is to submit a transaction of deposit of a certain amount of Harmony tokens as the stake to join the network as validators.
Beacon chain will be responsible for validating the stakes and the transaction.
Details of the PoS staking and verification process should refer to our Beacon Chain design spec.

For a light node that will not join in the consensus, it has no requirement of the PoS verification.

Beacon chain will use randomness to generate a permutation of all nodes that are waiting to join consensus.
The randomness and node permutation is used determine which shard the new node should be joining.
Details of the sharding algorithm please refer to the White Paper.

Every new node’s public key and deposit will be recorded in the beacon chain blocks.

## Shard Discovery
New node compute the shard info based on randomness generated by beacon chain.
It then broadcasts the shard info together with its public key to the network using the shard id as a topic.
The peers within the same shard as the new node will respond to the topic based on the XOR distance in the DHT.
We are investigating the rendezvous feature of libp2p to implement the shard discovery feature.

## LibP2P
The new node are connected to the two P2P overlay networks.
Harmony utilizes libp2p as the underlying networking layer for peer discovery and p2p network transportation.
It is still a crucial task to understand the protocol messages and how the libp2p handles the messages.
We may need to fork the libp2p to fit our requirement during the development.

## Two Stages of Peer Discovery
Harmony uses two stages of peer discovery mechanism to form the overlay of p2p networks.
The first stage is to connect to beacon chain to stake and get the shard information.
The second stage is to create the real p2p network within the shard afterwards.
New nodes will always keep the connection to some beacon chain nodes for further communication.
The current implementation works like the following.
* beacon chain nodes bootstrap by contacting bootnodes using discovery rendezvous string "0". Then the beacon chain is formed.
* new nodes contact bootnodes using rendezvous string "0" to connect to beacon chain nodes.
* new nodes use pubsub to stake in beacon chain, and get shard information from beacon chain after the randomness and resharding algorithm.
* new nodes use the new shardID as the rendezvous string to connect to bootnodes again to form new p2p network at the shard level.