The core protocol of WoopChain
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
woop/node/node.go

701 lines
23 KiB

package node
import (
"container/ring"
"crypto/ecdsa"
6 years ago
"fmt"
"os"
"strings"
"sync"
"github.com/ethereum/go-ethereum/common"
"github.com/harmony-one/harmony/api/client"
msg_pb "github.com/harmony-one/harmony/api/proto/message"
proto_node "github.com/harmony-one/harmony/api/proto/node"
"github.com/harmony-one/harmony/api/service"
"github.com/harmony-one/harmony/api/service/syncing"
"github.com/harmony-one/harmony/api/service/syncing/downloader"
"github.com/harmony-one/harmony/consensus"
"github.com/harmony-one/harmony/core"
"github.com/harmony-one/harmony/core/rawdb"
"github.com/harmony-one/harmony/core/types"
"github.com/harmony-one/harmony/internal/chain"
nodeconfig "github.com/harmony-one/harmony/internal/configs/node"
"github.com/harmony-one/harmony/internal/params"
"github.com/harmony-one/harmony/internal/shardchain"
"github.com/harmony-one/harmony/internal/utils"
"github.com/harmony-one/harmony/msgq"
"github.com/harmony-one/harmony/node/worker"
"github.com/harmony-one/harmony/p2p"
p2p_host "github.com/harmony-one/harmony/p2p/host"
"github.com/harmony-one/harmony/shard"
"github.com/harmony-one/harmony/shard/committee"
[double-sign] Provide proof of double sign in slash record sent to beaconchain (#2253) * [double-sign] Commit changes in consensus needed for double-sign * [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain * [slash] Add quick iteration tool for testing double-signing * [slash] Add webhook example * [slash] Add http server for hook to trigger double sign behavior * [double-sign] Use bin/trigger-double-sign to cause a double-sign * [double-sign] Full feedback loop working * [slash] Thread through the slash records in the block proposal step * [slash] Compute the slashing rate * [double-sign] Generalize yaml malicious for many keys * [double-sign][slash] Modify data structures, verify via webhook handler * [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures * [slash] Apply to state slashing for double signing * [slash][double-sign] Checkpoint for working code that slashes on beaconchain * [slash] Keep track of the total slash and total reporters reward * [slash] Dump account state before and after the slash * [slash] Satisfy Travis * [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates * [slash] Capture the unique new delegations since snapshot as well * [slash] Filter undelegation by epoch of double sign * [slash] Add TODO of correctness needed in slash needs on off-chain data * [rpc] Fix closure issue on shardID * [slash] Add delegator to double-sign testing script * [slash] Expand crt-validator.sh with commenting printfs and make delegation * [slash] Finish track payment of leftover slash debt after undelegation runs out * [slash] Now be explicit about error wrt delegatorSlashApply * [slash] Capture specific sanity check on slash paidoff * [slash] Track slash from undelegation piecemeal * [slash][delegation] Named slice types, .String() * [slash] Do no RLP encode twice, once is enough * [slash] Remove special case of validators own delegation * [slash] Refactor approach to slash state application * [slash] Begin expanding out Verify * [slash] Slash on snapshot delegations, not current * [slash] Fix Epoch Cmp * [slash] Third iteration on slash logic * [slash] Use full slash amount * [slash] More log, whitespace * [slash] Remove Println, add log * [slash] Remove debug Println * [slash] Add record in unit test * [slash] Build Validator snapshot, current. Fill out slash record * [slash] Need to get RLP dump of a header to use in test * [slash] Factor out double sign test constants * [slash] Factor out common for validator, stub out slash application, finish out deserialization setup * [slash] Factor out data structure creation because of var lexical scoping * [slash] Seem to have pipeline of unit test e2e executing * [slash] Add expected snitch, slash amounts * [slash] Checkpoint * [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing * [config] add double-sign testnet config (#1) Signed-off-by: Leo Chen <leo@harmony.one> * [slash] Commit for as is code & data of current dump.json * [slash] Order of state operation not correct in test, hence bad results, thank you dlv * [slash] Add snapshot state dump * [slash] Pay off slash of validator own delegation correctly * [slash] Pay off slash debt with special case for min-self * [slash] Pass first scenario conclusively * [slash] 2% slash passes unit test for own delegation and external * [slash] Parameterize unit test to easily test .02 vs .80 slash * [slash] Handle own delegation correctly at 80% slash * [slash] Have 80% slash working with external delegator * [slash] Remove debug code from slash * [slash] Adjust Apply signature, test again for 2% slash * [slash] Factor out scenario in testing so can test 2% and 80% at same time * [slash] Correct balance deduction on plan delegation * [slash] Mock out ChainReader for TestVerify * [slash] Small surface area interface, now feedback loop for verify * [slash] Remove development json * [slash] trigger-double-sign consumes yaml * [slash] Remove dead code * [slash][test] Factor ValidatorWrapper into scenario * [slash][test] Add example from local-testing dump - caution might be off * [slash] Factor out mutation of slashDebt * [slash][test] Factor out tests so can easily load test-case from bytes * [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected * [slash] Satisfy Travis * [slash] Begin cleanup of PR * [slash] Apply slash from header to Finalize via state processor * [slash] Productionize code, Println => logs; adjust slash picked in newblock * [slash] Need pointer for rlp.Decode * [slash] ValidatorInformation use full wrapper * Fix median stake * [staking] Adjust MarshalJSON for Validator, Wrapper * Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279) * Refactor offchain data; Add epoch to ValidatorSnapshot * Make block onchain/offchain data commit atomically * [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment * [effective] VC eligible.go * [consensus] Redundant field in printf * [docker] import-ks for a dev account * [slash] Create BLS key for dockerfile and crt-validator.sh * [slash][docker] Easy deployment of double-sign testing * [docker] Have slash work as single docker command * [rpc] Fix median-stake RPC * [slash] Update webhook with default docker BLS key * [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go * [slash] Remove helper binary, commented out code, change to local config * [params] Factor out test genesis value * Add shard checking to Tx-Pool & correct blacklist (#2301) * [core] Fix blacklist & add shardID check * [staking + node + cmd] Fix blacklist & add shardID check * [slash] Adjust to PR comments part 1 * [docker] Use different throw away funded account * [docker] Create easier testing for delegation with private keys * [docker] Update yaml * [slash] Remove special case for slashing validator own delegation wrt min-self-delegate * [docker] Install nano as well * [slash] Early error if banned * [quorum] Expose earning account in decider marshal json * Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)" This reverts commit 9ffbf682c075b49188923c65a0bbf39ac188be00. * [slash] Add non-sanity check way to update validator * [reward] Increase percision on percentage in schedule * [slash] Adjust logs * [committee] Check eligibility of validator before doing sanity check * [slash] Update docker * [slash] Move create validator script to test * [slash] More log * [param] Make things faster * [slash][off-chain] Clear out slashes from pending in writeblockwithstate * [cross-link] Log is not error, just info * [blockchain] Not necessary to guard DeletePendingSlashingCandidates * [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that * [slash][test] Use faucet as sender, assume user imported * [slash] Test setup * [slash] reserve error for real error in logs * [slash][availability] Apply availability correct, bump signing count each block * [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed * [slash] Pay as much as can * [slash] use right nowAmt * [slash] Take away from rewards as well * [slash] iterate faster * [slash] Remove dev based timing * [slash] Add more log, sanity check incoming slash records, only count external for slash rate * [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters * [availability] More log * [slash] Simply pre slash erra slashing * [slash] Remove development code * [slash] Use height from recvMsg, todo on epoch * [staking] Not necessary to touch LastEpochInCommittee in staking_verifier * [slash] Undo ds in endpoint pattern config * [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment * [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count Co-authored-by: Leo Chen <leo@harmony.one> Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com> Co-authored-by: Rongjian Lan <rongjian@harmony.one> Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
5 years ago
"github.com/harmony-one/harmony/staking/slash"
staking "github.com/harmony-one/harmony/staking/types"
"github.com/harmony-one/harmony/webhooks"
"github.com/pkg/errors"
)
// State is a state of a node.
type State byte
// All constants except the NodeLeader below are for validators only.
const (
NodeInit State = iota // Node just started, before contacting BeaconChain
NodeWaitToJoin // Node contacted BeaconChain, wait to join Shard
NodeNotInSync // Node out of sync, might be just joined Shard or offline for a period of time
NodeOffline // Node is offline
NodeReadyForConsensus // Node is ready for doing consensus
NodeDoingConsensus // Node is already doing consensus
NodeLeader // Node is the leader of some shard.
)
const (
// NumTryBroadCast is the number of times trying to broadcast
NumTryBroadCast = 3
// ClientRxQueueSize is the number of client messages to queue before tail-dropping.
ClientRxQueueSize = 16384
// ShardRxQueueSize is the number of shard messages to queue before tail-dropping.
ShardRxQueueSize = 16384
// GlobalRxQueueSize is the number of global messages to queue before tail-dropping.
GlobalRxQueueSize = 16384
// ClientRxWorkers is the number of concurrent client message handlers.
ClientRxWorkers = 8
// ShardRxWorkers is the number of concurrent shard message handlers.
ShardRxWorkers = 32
// GlobalRxWorkers is the number of concurrent global message handlers.
GlobalRxWorkers = 32
)
func (state State) String() string {
switch state {
case NodeInit:
return "NodeInit"
case NodeWaitToJoin:
return "NodeWaitToJoin"
case NodeNotInSync:
return "NodeNotInSync"
case NodeOffline:
return "NodeOffline"
case NodeReadyForConsensus:
return "NodeReadyForConsensus"
case NodeDoingConsensus:
return "NodeDoingConsensus"
case NodeLeader:
return "NodeLeader"
}
return "Unknown"
}
const (
maxBroadcastNodes = 10 // broadcast at most maxBroadcastNodes peers that need in sync
broadcastTimeout int64 = 60 * 1000000000 // 1 mins
//SyncIDLength is the length of bytes for syncID
SyncIDLength = 20
)
// use to push new block to outofsync node
type syncConfig struct {
timestamp int64
client *downloader.Client
}
6 years ago
// Node represents a protocol-participating node in the network
type Node struct {
Consensus *consensus.Consensus // Consensus object containing all Consensus related data (e.g. committee members, signatures, commits)
BlockChannel chan *types.Block // The channel to send newly proposed blocks
ConfirmedBlockChannel chan *types.Block // The channel to send confirmed blocks
BeaconBlockChannel chan *types.Block // The channel to send beacon blocks for non-beaconchain nodes
6 years ago
pendingCXReceipts map[string]*types.CXReceiptsProof // All the receipts received but not yet processed for Consensus
pendingCXMutex sync.Mutex
// Shard databases
shardChains shardchain.Collection
6 years ago
Client *client.Client // The presence of a client object means this node will also act as a client
SelfPeer p2p.Peer // TODO(minhdoan): it could be duplicated with Self below whose is Alok work.
BCPeers []p2p.Peer // list of Beacon Chain Peers. This is needed by all nodes.
6 years ago
// TODO: Neighbors should store only neighbor nodes in the same shard
Neighbors sync.Map // All the neighbor nodes, key is the sha256 of Peer IP/Port, value is the p2p.Peer
numPeers int // Number of Peers
State State // State of the Node
stateMutex sync.Mutex // mutex for change node state
6 years ago
// BeaconNeighbors store only neighbor nodes in the beacon chain shard
BeaconNeighbors sync.Map // All the neighbor nodes, key is the sha256 of Peer IP/Port, value is the p2p.Peer
5 years ago
TxPool *core.TxPool
CxPool *core.CxPool // pool for missing cross shard receipts resend
Worker, BeaconWorker *worker.Worker
downloaderServer *downloader.Server
// Syncing component.
syncID [SyncIDLength]byte // a unique ID for the node during the state syncing process with peers
stateSync, beaconSync *syncing.StateSync
peerRegistrationRecord map[string]*syncConfig // record registration time (unixtime) of peers begin in syncing
SyncingPeerProvider SyncingPeerProvider
// The p2p host used to send/receive p2p messages
host p2p.Host
// Incoming messages to process.
clientRxQueue, shardRxQueue, globalRxQueue *msgq.Queue
// Service manager.
serviceManager *service.Manager
ContractDeployerKey *ecdsa.PrivateKey
ContractDeployerCurrentNonce uint64 // The nonce of the deployer contract at current block
ContractAddresses []common.Address
// Shard group Message Receiver
shardGroupReceiver p2p.GroupReceiver
// Global group Message Receiver, communicate with beacon chain, or cross-shard TX
globalGroupReceiver p2p.GroupReceiver
// Client Message Receiver to handle light client messages
// Beacon leader needs to use this receiver to talk to new node
clientReceiver p2p.GroupReceiver
// Duplicated Ping Message Received
duplicatedPing sync.Map
// Channel to notify consensus service to really start consensus
startConsensus chan struct{}
// node configuration, including group ID, shard ID, etc
NodeConfig *nodeconfig.ConfigType
// Chain configuration.
chainConfig params.ChainConfig
// map of service type to its message channel.
serviceMessageChan map[service.Type]chan *msg_pb.Message
isFirstTime bool // the node was started with a fresh database
// Last 1024 staking transaction error, only in memory
errorSink struct {
sync.Mutex
failedStakingTxns *ring.Ring
failedTxns *ring.Ring
}
}
// Blockchain returns the blockchain for the node's current shard.
func (node *Node) Blockchain() *core.BlockChain {
shardID := node.NodeConfig.ShardID
bc, err := node.shardChains.ShardChain(shardID)
if err != nil {
utils.Logger().Error().
Uint32("shardID", shardID).
Err(err).
Msg("cannot get shard chain")
}
return bc
}
// Beaconchain returns the beaconchain from node.
func (node *Node) Beaconchain() *core.BlockChain {
bc, err := node.shardChains.ShardChain(0)
if err != nil {
utils.Logger().Error().Err(err).Msg("cannot get beaconchain")
}
return bc
}
// TODO: make this batch more transactions
func (node *Node) tryBroadcast(tx *types.Transaction) {
msg := proto_node.ConstructTransactionListMessageAccount(types.Transactions{tx})
shardGroupID := nodeconfig.NewGroupIDByShardID(nodeconfig.ShardID(tx.ShardID()))
utils.Logger().Info().Str("shardGroupID", string(shardGroupID)).Msg("tryBroadcast")
for attempt := 0; attempt < NumTryBroadCast; attempt++ {
if err := node.host.SendMessageToGroups([]nodeconfig.GroupID{shardGroupID}, p2p_host.ConstructP2pMessage(byte(0), msg)); err != nil && attempt < NumTryBroadCast {
utils.Logger().Error().Int("attempt", attempt).Msg("Error when trying to broadcast tx")
} else {
break
}
}
}
func (node *Node) tryBroadcastStaking(stakingTx *staking.StakingTransaction) {
msg := proto_node.ConstructStakingTransactionListMessageAccount(staking.StakingTransactions{stakingTx})
[slash][consensus] Notice double sign & broadcast, factor out tech debt of consensus (#2152) * [slash] Remove dead interface, associated piping * [slash] Expand out structs * [consensus] Write to a chan when find a case of double-signing, remove dead code * [slash] Broadcast the noticing of a double signing * [rawdb] CRUD for slashing candidates * [slashing][node][proto] Broadcast the slash record after receive from consensus, handle received proto message, persist in off-chain db while pending * [slash][node][propose-block] Add verified slashes proposed into the header in block proposal * [slash][shard] Factor out external validator as method on shard state, add double-signature field * [slash][engine] Apply slash, name boolean expression for sorts, use stable sort * [slash] Abstract Ballot results so keep track of both pre and post double sign event * [slash] Fix type errors on test code * [slash] Read from correct rawdb * [slash] Add epoch based guards in CRUD of slashing * [slash] Write to correct cache for slashing candidates * [shard] Use explicit named type of BLS Signature, use convention * [slash] Fix mistake done in refactor, improper header used. Factor out fromSlice to set * [slash][node] Restore newblock to master, try again minimial change * [cx-receipts] Break up one-liner, use SliceStable, not Slice * [network] Finish refactor that makes network message headers once * [network] Simplify creation further of headers write * [slash] Adjust data structure of slash after offline discussion with RJ, Chao * [slash] Still did need signature of the double signature * [consensus] Prepare message does not have block header * [consensus] Soft reset three files to 968517d~1 * [consensus] Begin factor consensus network intended message out with prepare first * [consensus] Factor out Prepared message * [consensus] Factor out announce message creation * [consensus] Committed Message, branch on verify sender key for clearer log * [consensus] Committed Message Factor out * [consensus] Do jenkins MVP of signatures adjustment * [main][slash] Provide YAML config as webhook config for double sign event * [consensus] Adjust signatures, whitespace, lessen GC pressure * [consensus] Remove dead code * [consensus] Factor out commit overloaded message, give commit payload override in construct * [consensus] Fix travis tests * [consensus] Provide block bytes in SubmitVote(quorum.Commit) * [consensus] Factor out noisy sanity checks in BFT, move existing commit check earlier as was before * [quorum] Adjust signatures in quorum * [staking] Adjust after merge from master * [consensus] Finish refactor of consensus * [node] Fix import * [consensus] Fix travis * [consensus] Use origin/master copy of block, fix mistake of pointer to empty byte * [consensus] Less verbose bools * [consensus] Remove unused trailing mutation hook in message construct * [consensus] Address some TODOs on err, comment out double sign
5 years ago
shardGroupID := nodeconfig.NewGroupIDByShardID(
nodeconfig.ShardID(shard.BeaconChainShardID),
) // broadcast to beacon chain
utils.Logger().Info().Str("shardGroupID", string(shardGroupID)).Msg("tryBroadcastStaking")
for attempt := 0; attempt < NumTryBroadCast; attempt++ {
if err := node.host.SendMessageToGroups([]nodeconfig.GroupID{shardGroupID}, p2p_host.ConstructP2pMessage(byte(0), msg)); err != nil && attempt < NumTryBroadCast {
utils.Logger().Error().Int("attempt", attempt).Msg("Error when trying to broadcast staking tx")
} else {
break
}
}
}
// Add new transactions to the pending transaction list.
func (node *Node) addPendingTransactions(newTxs types.Transactions) []error {
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
poolTxs := types.PoolTransactions{}
for _, tx := range newTxs {
poolTxs = append(poolTxs, tx)
}
errs := node.TxPool.AddRemotes(poolTxs)
pendingCount, queueCount := node.TxPool.Stats()
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
utils.Logger().Info().
Int("length of newTxs", len(newTxs)).
Int("totalPending", pendingCount).
Int("totalQueued", queueCount).
Msg("Got more transactions")
return errs
}
// Add new staking transactions to the pending staking transaction list.
func (node *Node) addPendingStakingTransactions(newStakingTxs staking.StakingTransactions) []error {
if node.NodeConfig.ShardID == shard.BeaconChainShardID &&
node.Blockchain().Config().IsPreStaking(node.Blockchain().CurrentHeader().Epoch()) {
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
poolTxs := types.PoolTransactions{}
for _, tx := range newStakingTxs {
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
poolTxs = append(poolTxs, tx)
}
errs := node.TxPool.AddRemotes(poolTxs)
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
pendingCount, queueCount := node.TxPool.Stats()
utils.Logger().Info().
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
Int("length of newStakingTxs", len(poolTxs)).
Int("totalPending", pendingCount).
Int("totalQueued", queueCount).
Msg("Got more staking transactions")
return errs
}
return make([]error, len(newStakingTxs))
}
// AddPendingStakingTransaction staking transactions
func (node *Node) AddPendingStakingTransaction(
newStakingTx *staking.StakingTransaction,
) error {
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
if node.NodeConfig.ShardID == shard.BeaconChainShardID {
errs := node.addPendingStakingTransactions(staking.StakingTransactions{newStakingTx})
for i := range errs {
if errs[i] != nil {
return errs[i]
}
}
utils.Logger().Info().Str("Hash", newStakingTx.Hash().Hex()).Msg("Broadcasting Staking Tx")
node.tryBroadcastStaking(newStakingTx)
}
return nil
}
// AddPendingTransaction adds one new transaction to the pending transaction list.
// This is only called from SDK.
func (node *Node) AddPendingTransaction(newTx *types.Transaction) error {
if newTx.ShardID() == node.NodeConfig.ShardID {
errs := node.addPendingTransactions(types.Transactions{newTx})
for i := range errs {
if errs[i] != nil {
return errs[i]
}
}
utils.Logger().Info().Str("Hash", newTx.Hash().Hex()).Msg("Broadcasting Tx")
node.tryBroadcast(newTx)
}
return nil
}
// AddPendingReceipts adds one receipt message to pending list.
func (node *Node) AddPendingReceipts(receipts *types.CXReceiptsProof) {
node.pendingCXMutex.Lock()
defer node.pendingCXMutex.Unlock()
if receipts.ContainsEmptyField() {
utils.Logger().Info().
Int("totalPendingReceipts", len(node.pendingCXReceipts)).
Msg("CXReceiptsProof contains empty field")
return
}
blockNum := receipts.Header.Number().Uint64()
shardID := receipts.Header.ShardID()
// Sanity checks
if err := node.Blockchain().Validator().ValidateCXReceiptsProof(receipts); err != nil {
if !strings.Contains(err.Error(), rawdb.MsgNoShardStateFromDB) {
utils.Logger().Error().Err(err).Msg("[AddPendingReceipts] Invalid CXReceiptsProof")
return
}
}
// cross-shard receipt should not be coming from our shard
if s := node.Consensus.ShardID; s == shardID {
utils.Logger().Info().
Uint32("my-shard", s).
Uint32("receipt-shard", shardID).
Msg("ShardID of incoming receipt was same as mine")
return
}
if e := receipts.Header.Epoch(); blockNum == 0 ||
!node.Blockchain().Config().AcceptsCrossTx(e) {
utils.Logger().Info().
Uint64("incoming-epoch", e.Uint64()).
Msg("Incoming receipt had meaningless epoch")
return
}
key := utils.GetPendingCXKey(shardID, blockNum)
// DDoS protection
const maxCrossTxnSize = 4096
if s := len(node.pendingCXReceipts); s >= maxCrossTxnSize {
utils.Logger().Info().
Int("pending-cx-receipts-size", s).
Int("pending-cx-receipts-limit", maxCrossTxnSize).
Msg("Current pending cx-receipts reached size limit")
return
}
if _, ok := node.pendingCXReceipts[key]; ok {
utils.Logger().Info().
Int("totalPendingReceipts", len(node.pendingCXReceipts)).
Msg("Already Got Same Receipt message")
return
}
node.pendingCXReceipts[key] = receipts
utils.Logger().Info().
Int("totalPendingReceipts", len(node.pendingCXReceipts)).
Msg("Got ONE more receipt message")
}
func (node *Node) startRxPipeline(
receiver p2p.GroupReceiver, queue *msgq.Queue, numWorkers int,
) {
// consumers
for i := 0; i < numWorkers; i++ {
go queue.HandleMessages(node)
}
// provider
go node.receiveGroupMessage(receiver, queue)
}
// StartServer starts a server and process the requests by a handler.
func (node *Node) StartServer() {
// client messages are for just spectators, like plain observers
node.startRxPipeline(node.clientReceiver, node.clientRxQueue, ClientRxWorkers)
// start the goroutine to receive in my subcommittee messages
node.startRxPipeline(node.shardGroupReceiver, node.shardRxQueue, ShardRxWorkers)
// start the goroutine to receive supercommittee level messages
// FIXME (leo): we use beacon client topic as the global topic for now
node.startRxPipeline(node.globalGroupReceiver, node.globalRxQueue, GlobalRxWorkers)
select {}
}
// GetSyncID returns the syncID of this node
func (node *Node) GetSyncID() [SyncIDLength]byte {
return node.syncID
}
// New creates a new node.
[staking][validation][protocol] (#2396) * [staking][validation][protocol] Limit max bls keys * [staking-era] Fold banned and active into single field * [slash][effective] Remove LRU cache for slash, change .Active to enumeration * [slash] Remove leftover wrong usage of Logger * [slash][offchain] Only Decode if len > 0 * [offchain] cosmetic * [slash] Remove some logs in proposal * [webhook] Move webhook with call for when cannot commit block * [shard] Finally make finding subcommittee by shardID an explicit error * [node] Whitespace, prefer literal * [webhook] Report bad block to webhook * [slash] Expand verify, remove bad log usage, explicit error handle * [slash] Check on key size * [slash] Explicit upper bound of pending slashes * [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain * [multibls] Make max count allowed be 1/3 of external slots * [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee * [verify] Begin factor out of common verification approach * [project] Further remove RawJSON log, use proper epoch for snapshot * [slash] Implement verification * [slash] Implement BLS key verification of ballots * [rpc] Keep validator information as meaningful as possible * [staking] Never can stop being banned * [slash] Comments and default Unknown case of eligibility * [slash] Be explicit on what input values allowed when want to change EPOSStatus * [consensus] Remove unneeded TODO * [verify] Add proper error message * [rpc] Give back to caller their wrong chain id * [chain] Add extra map dump of delegation sizing for downstream analysis * [engine] Less code, more methods * [offchain] More leniency in handling slash bytes and delete from pending * [validator] Remove errors on bad input for edit
5 years ago
func New(
host p2p.Host,
consensusObj *consensus.Consensus,
chainDBFactory shardchain.DBFactory,
blacklist map[common.Address]struct{},
isArchival bool,
) *Node {
node := Node{}
const sinkSize = 4096
node.errorSink = struct {
sync.Mutex
failedStakingTxns *ring.Ring
failedTxns *ring.Ring
}{sync.Mutex{}, ring.New(sinkSize), ring.New(sinkSize)}
// Get the node config that's created in the harmony.go program.
if consensusObj != nil {
node.NodeConfig = nodeconfig.GetShardConfig(consensusObj.ShardID)
} else {
node.NodeConfig = nodeconfig.GetDefaultConfig()
}
copy(node.syncID[:], GenerateRandomString(SyncIDLength))
if host != nil {
node.host = host
node.SelfPeer = host.GetSelfPeer()
}
networkType := node.NodeConfig.GetNetworkType()
chainConfig := networkType.ChainConfig()
node.chainConfig = chainConfig
collection := shardchain.NewCollection(
chainDBFactory, &genesisInitializer{&node}, chain.Engine, &chainConfig,
)
if isArchival {
collection.DisableCache()
}
node.shardChains = collection
if host != nil && consensusObj != nil {
// Consensus and associated channel to communicate blocks
node.Consensus = consensusObj
6 years ago
// Load the chains.
blockchain := node.Blockchain() // this also sets node.isFirstTime if the DB is fresh
beaconChain := node.Beaconchain()
if b1, b2 := beaconChain == nil, blockchain == nil; b1 || b2 {
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
shardID := node.NodeConfig.ShardID
// HACK get the real error reason
_, err := node.shardChains.ShardChain(shardID)
fmt.Fprintf(
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
os.Stderr,
"reason:%s beaconchain-is-nil:%t shardchain-is-nil:%t",
err.Error(), b1, b2,
)
os.Exit(-1)
}
node.BlockChannel = make(chan *types.Block)
node.ConfirmedBlockChannel = make(chan *types.Block)
node.BeaconBlockChannel = make(chan *types.Block)
txPoolConfig := core.DefaultTxPoolConfig
txPoolConfig.Blacklist = blacklist
node.TxPool = core.NewTxPool(txPoolConfig, node.Blockchain().Config(), blockchain,
func(payload []types.RPCTransactionError) {
if len(payload) > 0 {
node.errorSink.Lock()
for i := range payload {
node.errorSink.failedTxns.Value = payload[i]
node.errorSink.failedTxns = node.errorSink.failedTxns.Next()
}
node.errorSink.Unlock()
}
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
},
func(payload []staking.RPCTransactionError) {
node.errorSink.Lock()
for i := range payload {
node.errorSink.failedStakingTxns.Value = payload[i]
node.errorSink.failedStakingTxns = node.errorSink.failedStakingTxns.Next()
}
node.errorSink.Unlock()
},
)
node.CxPool = core.NewCxPool(core.CxPoolSize)
node.Worker = worker.New(node.Blockchain().Config(), blockchain, chain.Engine)
if node.Blockchain().ShardID() != shard.BeaconChainShardID {
node.BeaconWorker = worker.New(
node.Beaconchain().Config(), beaconChain, chain.Engine,
)
}
[staking][validation][protocol] (#2396) * [staking][validation][protocol] Limit max bls keys * [staking-era] Fold banned and active into single field * [slash][effective] Remove LRU cache for slash, change .Active to enumeration * [slash] Remove leftover wrong usage of Logger * [slash][offchain] Only Decode if len > 0 * [offchain] cosmetic * [slash] Remove some logs in proposal * [webhook] Move webhook with call for when cannot commit block * [shard] Finally make finding subcommittee by shardID an explicit error * [node] Whitespace, prefer literal * [webhook] Report bad block to webhook * [slash] Expand verify, remove bad log usage, explicit error handle * [slash] Check on key size * [slash] Explicit upper bound of pending slashes * [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain * [multibls] Make max count allowed be 1/3 of external slots * [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee * [verify] Begin factor out of common verification approach * [project] Further remove RawJSON log, use proper epoch for snapshot * [slash] Implement verification * [slash] Implement BLS key verification of ballots * [rpc] Keep validator information as meaningful as possible * [staking] Never can stop being banned * [slash] Comments and default Unknown case of eligibility * [slash] Be explicit on what input values allowed when want to change EPOSStatus * [consensus] Remove unneeded TODO * [verify] Add proper error message * [rpc] Give back to caller their wrong chain id * [chain] Add extra map dump of delegation sizing for downstream analysis * [engine] Less code, more methods * [offchain] More leniency in handling slash bytes and delete from pending * [validator] Remove errors on bad input for edit
5 years ago
node.pendingCXReceipts = map[string]*types.CXReceiptsProof{}
node.Consensus.VerifiedNewBlock = make(chan *types.Block)
chain.Engine.SetBeaconchain(beaconChain)
// the sequence number is the next block number to be added in consensus protocol, which is
// always one more than current chain header block
node.Consensus.SetBlockNum(blockchain.CurrentBlock().NumberU64() + 1)
// Add Faucet contract to all shards, so that on testnet, we can demo wallet in explorer
if networkType != nodeconfig.Mainnet {
if node.isFirstTime {
// Setup one time smart contracts
node.AddFaucetContractToPendingTransactions()
}
}
}
utils.Logger().Info().
Interface("genesis block header", node.Blockchain().GetHeaderByNumber(0)).
Msg("Genesis block hash")
node.clientRxQueue = msgq.New(ClientRxQueueSize)
node.shardRxQueue = msgq.New(ShardRxQueueSize)
node.globalRxQueue = msgq.New(GlobalRxQueueSize)
// Setup initial state of syncing.
[staking][validation][protocol] (#2396) * [staking][validation][protocol] Limit max bls keys * [staking-era] Fold banned and active into single field * [slash][effective] Remove LRU cache for slash, change .Active to enumeration * [slash] Remove leftover wrong usage of Logger * [slash][offchain] Only Decode if len > 0 * [offchain] cosmetic * [slash] Remove some logs in proposal * [webhook] Move webhook with call for when cannot commit block * [shard] Finally make finding subcommittee by shardID an explicit error * [node] Whitespace, prefer literal * [webhook] Report bad block to webhook * [slash] Expand verify, remove bad log usage, explicit error handle * [slash] Check on key size * [slash] Explicit upper bound of pending slashes * [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain * [multibls] Make max count allowed be 1/3 of external slots * [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee * [verify] Begin factor out of common verification approach * [project] Further remove RawJSON log, use proper epoch for snapshot * [slash] Implement verification * [slash] Implement BLS key verification of ballots * [rpc] Keep validator information as meaningful as possible * [staking] Never can stop being banned * [slash] Comments and default Unknown case of eligibility * [slash] Be explicit on what input values allowed when want to change EPOSStatus * [consensus] Remove unneeded TODO * [verify] Add proper error message * [rpc] Give back to caller their wrong chain id * [chain] Add extra map dump of delegation sizing for downstream analysis * [engine] Less code, more methods * [offchain] More leniency in handling slash bytes and delete from pending * [validator] Remove errors on bad input for edit
5 years ago
node.peerRegistrationRecord = map[string]*syncConfig{}
node.startConsensus = make(chan struct{})
go node.bootstrapConsensus()
[slash][consensus] Notice double sign & broadcast, factor out tech debt of consensus (#2152) * [slash] Remove dead interface, associated piping * [slash] Expand out structs * [consensus] Write to a chan when find a case of double-signing, remove dead code * [slash] Broadcast the noticing of a double signing * [rawdb] CRUD for slashing candidates * [slashing][node][proto] Broadcast the slash record after receive from consensus, handle received proto message, persist in off-chain db while pending * [slash][node][propose-block] Add verified slashes proposed into the header in block proposal * [slash][shard] Factor out external validator as method on shard state, add double-signature field * [slash][engine] Apply slash, name boolean expression for sorts, use stable sort * [slash] Abstract Ballot results so keep track of both pre and post double sign event * [slash] Fix type errors on test code * [slash] Read from correct rawdb * [slash] Add epoch based guards in CRUD of slashing * [slash] Write to correct cache for slashing candidates * [shard] Use explicit named type of BLS Signature, use convention * [slash] Fix mistake done in refactor, improper header used. Factor out fromSlice to set * [slash][node] Restore newblock to master, try again minimial change * [cx-receipts] Break up one-liner, use SliceStable, not Slice * [network] Finish refactor that makes network message headers once * [network] Simplify creation further of headers write * [slash] Adjust data structure of slash after offline discussion with RJ, Chao * [slash] Still did need signature of the double signature * [consensus] Prepare message does not have block header * [consensus] Soft reset three files to 968517d~1 * [consensus] Begin factor consensus network intended message out with prepare first * [consensus] Factor out Prepared message * [consensus] Factor out announce message creation * [consensus] Committed Message, branch on verify sender key for clearer log * [consensus] Committed Message Factor out * [consensus] Do jenkins MVP of signatures adjustment * [main][slash] Provide YAML config as webhook config for double sign event * [consensus] Adjust signatures, whitespace, lessen GC pressure * [consensus] Remove dead code * [consensus] Factor out commit overloaded message, give commit payload override in construct * [consensus] Fix travis tests * [consensus] Provide block bytes in SubmitVote(quorum.Commit) * [consensus] Factor out noisy sanity checks in BFT, move existing commit check earlier as was before * [quorum] Adjust signatures in quorum * [staking] Adjust after merge from master * [consensus] Finish refactor of consensus * [node] Fix import * [consensus] Fix travis * [consensus] Use origin/master copy of block, fix mistake of pointer to empty byte * [consensus] Less verbose bools * [consensus] Remove unused trailing mutation hook in message construct * [consensus] Address some TODOs on err, comment out double sign
5 years ago
// Broadcast double-signers reported by consensus
if node.Consensus != nil {
go func() {
for doubleSign := range node.Consensus.SlashChan {
utils.Logger().Info().
RawJSON("double-sign-candidate", []byte(doubleSign.String())).
Msg("double sign notified by consensus leader")
// no point to broadcast the slash if we aren't even in the right epoch yet
if !node.Blockchain().Config().IsStaking(
node.Blockchain().CurrentHeader().Epoch(),
) {
return
}
if hooks := node.NodeConfig.WebHooks.Hooks; hooks != nil {
if s := hooks.Slashing; s != nil {
url := s.OnNoticeDoubleSign
go func() { webhooks.DoPost(url, &doubleSign) }()
[double-sign] Provide proof of double sign in slash record sent to beaconchain (#2253) * [double-sign] Commit changes in consensus needed for double-sign * [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain * [slash] Add quick iteration tool for testing double-signing * [slash] Add webhook example * [slash] Add http server for hook to trigger double sign behavior * [double-sign] Use bin/trigger-double-sign to cause a double-sign * [double-sign] Full feedback loop working * [slash] Thread through the slash records in the block proposal step * [slash] Compute the slashing rate * [double-sign] Generalize yaml malicious for many keys * [double-sign][slash] Modify data structures, verify via webhook handler * [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures * [slash] Apply to state slashing for double signing * [slash][double-sign] Checkpoint for working code that slashes on beaconchain * [slash] Keep track of the total slash and total reporters reward * [slash] Dump account state before and after the slash * [slash] Satisfy Travis * [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates * [slash] Capture the unique new delegations since snapshot as well * [slash] Filter undelegation by epoch of double sign * [slash] Add TODO of correctness needed in slash needs on off-chain data * [rpc] Fix closure issue on shardID * [slash] Add delegator to double-sign testing script * [slash] Expand crt-validator.sh with commenting printfs and make delegation * [slash] Finish track payment of leftover slash debt after undelegation runs out * [slash] Now be explicit about error wrt delegatorSlashApply * [slash] Capture specific sanity check on slash paidoff * [slash] Track slash from undelegation piecemeal * [slash][delegation] Named slice types, .String() * [slash] Do no RLP encode twice, once is enough * [slash] Remove special case of validators own delegation * [slash] Refactor approach to slash state application * [slash] Begin expanding out Verify * [slash] Slash on snapshot delegations, not current * [slash] Fix Epoch Cmp * [slash] Third iteration on slash logic * [slash] Use full slash amount * [slash] More log, whitespace * [slash] Remove Println, add log * [slash] Remove debug Println * [slash] Add record in unit test * [slash] Build Validator snapshot, current. Fill out slash record * [slash] Need to get RLP dump of a header to use in test * [slash] Factor out double sign test constants * [slash] Factor out common for validator, stub out slash application, finish out deserialization setup * [slash] Factor out data structure creation because of var lexical scoping * [slash] Seem to have pipeline of unit test e2e executing * [slash] Add expected snitch, slash amounts * [slash] Checkpoint * [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing * [config] add double-sign testnet config (#1) Signed-off-by: Leo Chen <leo@harmony.one> * [slash] Commit for as is code & data of current dump.json * [slash] Order of state operation not correct in test, hence bad results, thank you dlv * [slash] Add snapshot state dump * [slash] Pay off slash of validator own delegation correctly * [slash] Pay off slash debt with special case for min-self * [slash] Pass first scenario conclusively * [slash] 2% slash passes unit test for own delegation and external * [slash] Parameterize unit test to easily test .02 vs .80 slash * [slash] Handle own delegation correctly at 80% slash * [slash] Have 80% slash working with external delegator * [slash] Remove debug code from slash * [slash] Adjust Apply signature, test again for 2% slash * [slash] Factor out scenario in testing so can test 2% and 80% at same time * [slash] Correct balance deduction on plan delegation * [slash] Mock out ChainReader for TestVerify * [slash] Small surface area interface, now feedback loop for verify * [slash] Remove development json * [slash] trigger-double-sign consumes yaml * [slash] Remove dead code * [slash][test] Factor ValidatorWrapper into scenario * [slash][test] Add example from local-testing dump - caution might be off * [slash] Factor out mutation of slashDebt * [slash][test] Factor out tests so can easily load test-case from bytes * [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected * [slash] Satisfy Travis * [slash] Begin cleanup of PR * [slash] Apply slash from header to Finalize via state processor * [slash] Productionize code, Println => logs; adjust slash picked in newblock * [slash] Need pointer for rlp.Decode * [slash] ValidatorInformation use full wrapper * Fix median stake * [staking] Adjust MarshalJSON for Validator, Wrapper * Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279) * Refactor offchain data; Add epoch to ValidatorSnapshot * Make block onchain/offchain data commit atomically * [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment * [effective] VC eligible.go * [consensus] Redundant field in printf * [docker] import-ks for a dev account * [slash] Create BLS key for dockerfile and crt-validator.sh * [slash][docker] Easy deployment of double-sign testing * [docker] Have slash work as single docker command * [rpc] Fix median-stake RPC * [slash] Update webhook with default docker BLS key * [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go * [slash] Remove helper binary, commented out code, change to local config * [params] Factor out test genesis value * Add shard checking to Tx-Pool & correct blacklist (#2301) * [core] Fix blacklist & add shardID check * [staking + node + cmd] Fix blacklist & add shardID check * [slash] Adjust to PR comments part 1 * [docker] Use different throw away funded account * [docker] Create easier testing for delegation with private keys * [docker] Update yaml * [slash] Remove special case for slashing validator own delegation wrt min-self-delegate * [docker] Install nano as well * [slash] Early error if banned * [quorum] Expose earning account in decider marshal json * Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)" This reverts commit 9ffbf682c075b49188923c65a0bbf39ac188be00. * [slash] Add non-sanity check way to update validator * [reward] Increase percision on percentage in schedule * [slash] Adjust logs * [committee] Check eligibility of validator before doing sanity check * [slash] Update docker * [slash] Move create validator script to test * [slash] More log * [param] Make things faster * [slash][off-chain] Clear out slashes from pending in writeblockwithstate * [cross-link] Log is not error, just info * [blockchain] Not necessary to guard DeletePendingSlashingCandidates * [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that * [slash][test] Use faucet as sender, assume user imported * [slash] Test setup * [slash] reserve error for real error in logs * [slash][availability] Apply availability correct, bump signing count each block * [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed * [slash] Pay as much as can * [slash] use right nowAmt * [slash] Take away from rewards as well * [slash] iterate faster * [slash] Remove dev based timing * [slash] Add more log, sanity check incoming slash records, only count external for slash rate * [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters * [availability] More log * [slash] Simply pre slash erra slashing * [slash] Remove development code * [slash] Use height from recvMsg, todo on epoch * [staking] Not necessary to touch LastEpochInCommittee in staking_verifier * [slash] Undo ds in endpoint pattern config * [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment * [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count Co-authored-by: Leo Chen <leo@harmony.one> Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com> Co-authored-by: Rongjian Lan <rongjian@harmony.one> Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
5 years ago
}
}
if node.NodeConfig.ShardID != shard.BeaconChainShardID {
go node.BroadcastSlash(&doubleSign)
} else {
records := slash.Records{doubleSign}
if err := node.Blockchain().AddPendingSlashingCandidates(
records,
); err != nil {
utils.Logger().Err(err).Msg("could not add new slash to ending slashes")
[double-sign] Provide proof of double sign in slash record sent to beaconchain (#2253) * [double-sign] Commit changes in consensus needed for double-sign * [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain * [slash] Add quick iteration tool for testing double-signing * [slash] Add webhook example * [slash] Add http server for hook to trigger double sign behavior * [double-sign] Use bin/trigger-double-sign to cause a double-sign * [double-sign] Full feedback loop working * [slash] Thread through the slash records in the block proposal step * [slash] Compute the slashing rate * [double-sign] Generalize yaml malicious for many keys * [double-sign][slash] Modify data structures, verify via webhook handler * [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures * [slash] Apply to state slashing for double signing * [slash][double-sign] Checkpoint for working code that slashes on beaconchain * [slash] Keep track of the total slash and total reporters reward * [slash] Dump account state before and after the slash * [slash] Satisfy Travis * [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates * [slash] Capture the unique new delegations since snapshot as well * [slash] Filter undelegation by epoch of double sign * [slash] Add TODO of correctness needed in slash needs on off-chain data * [rpc] Fix closure issue on shardID * [slash] Add delegator to double-sign testing script * [slash] Expand crt-validator.sh with commenting printfs and make delegation * [slash] Finish track payment of leftover slash debt after undelegation runs out * [slash] Now be explicit about error wrt delegatorSlashApply * [slash] Capture specific sanity check on slash paidoff * [slash] Track slash from undelegation piecemeal * [slash][delegation] Named slice types, .String() * [slash] Do no RLP encode twice, once is enough * [slash] Remove special case of validators own delegation * [slash] Refactor approach to slash state application * [slash] Begin expanding out Verify * [slash] Slash on snapshot delegations, not current * [slash] Fix Epoch Cmp * [slash] Third iteration on slash logic * [slash] Use full slash amount * [slash] More log, whitespace * [slash] Remove Println, add log * [slash] Remove debug Println * [slash] Add record in unit test * [slash] Build Validator snapshot, current. Fill out slash record * [slash] Need to get RLP dump of a header to use in test * [slash] Factor out double sign test constants * [slash] Factor out common for validator, stub out slash application, finish out deserialization setup * [slash] Factor out data structure creation because of var lexical scoping * [slash] Seem to have pipeline of unit test e2e executing * [slash] Add expected snitch, slash amounts * [slash] Checkpoint * [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing * [config] add double-sign testnet config (#1) Signed-off-by: Leo Chen <leo@harmony.one> * [slash] Commit for as is code & data of current dump.json * [slash] Order of state operation not correct in test, hence bad results, thank you dlv * [slash] Add snapshot state dump * [slash] Pay off slash of validator own delegation correctly * [slash] Pay off slash debt with special case for min-self * [slash] Pass first scenario conclusively * [slash] 2% slash passes unit test for own delegation and external * [slash] Parameterize unit test to easily test .02 vs .80 slash * [slash] Handle own delegation correctly at 80% slash * [slash] Have 80% slash working with external delegator * [slash] Remove debug code from slash * [slash] Adjust Apply signature, test again for 2% slash * [slash] Factor out scenario in testing so can test 2% and 80% at same time * [slash] Correct balance deduction on plan delegation * [slash] Mock out ChainReader for TestVerify * [slash] Small surface area interface, now feedback loop for verify * [slash] Remove development json * [slash] trigger-double-sign consumes yaml * [slash] Remove dead code * [slash][test] Factor ValidatorWrapper into scenario * [slash][test] Add example from local-testing dump - caution might be off * [slash] Factor out mutation of slashDebt * [slash][test] Factor out tests so can easily load test-case from bytes * [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected * [slash] Satisfy Travis * [slash] Begin cleanup of PR * [slash] Apply slash from header to Finalize via state processor * [slash] Productionize code, Println => logs; adjust slash picked in newblock * [slash] Need pointer for rlp.Decode * [slash] ValidatorInformation use full wrapper * Fix median stake * [staking] Adjust MarshalJSON for Validator, Wrapper * Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279) * Refactor offchain data; Add epoch to ValidatorSnapshot * Make block onchain/offchain data commit atomically * [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment * [effective] VC eligible.go * [consensus] Redundant field in printf * [docker] import-ks for a dev account * [slash] Create BLS key for dockerfile and crt-validator.sh * [slash][docker] Easy deployment of double-sign testing * [docker] Have slash work as single docker command * [rpc] Fix median-stake RPC * [slash] Update webhook with default docker BLS key * [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go * [slash] Remove helper binary, commented out code, change to local config * [params] Factor out test genesis value * Add shard checking to Tx-Pool & correct blacklist (#2301) * [core] Fix blacklist & add shardID check * [staking + node + cmd] Fix blacklist & add shardID check * [slash] Adjust to PR comments part 1 * [docker] Use different throw away funded account * [docker] Create easier testing for delegation with private keys * [docker] Update yaml * [slash] Remove special case for slashing validator own delegation wrt min-self-delegate * [docker] Install nano as well * [slash] Early error if banned * [quorum] Expose earning account in decider marshal json * Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)" This reverts commit 9ffbf682c075b49188923c65a0bbf39ac188be00. * [slash] Add non-sanity check way to update validator * [reward] Increase percision on percentage in schedule * [slash] Adjust logs * [committee] Check eligibility of validator before doing sanity check * [slash] Update docker * [slash] Move create validator script to test * [slash] More log * [param] Make things faster * [slash][off-chain] Clear out slashes from pending in writeblockwithstate * [cross-link] Log is not error, just info * [blockchain] Not necessary to guard DeletePendingSlashingCandidates * [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that * [slash][test] Use faucet as sender, assume user imported * [slash] Test setup * [slash] reserve error for real error in logs * [slash][availability] Apply availability correct, bump signing count each block * [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed * [slash] Pay as much as can * [slash] use right nowAmt * [slash] Take away from rewards as well * [slash] iterate faster * [slash] Remove dev based timing * [slash] Add more log, sanity check incoming slash records, only count external for slash rate * [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters * [availability] More log * [slash] Simply pre slash erra slashing * [slash] Remove development code * [slash] Use height from recvMsg, todo on epoch * [staking] Not necessary to touch LastEpochInCommittee in staking_verifier * [slash] Undo ds in endpoint pattern config * [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment * [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count Co-authored-by: Leo Chen <leo@harmony.one> Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com> Co-authored-by: Rongjian Lan <rongjian@harmony.one> Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
5 years ago
}
[slash][consensus] Notice double sign & broadcast, factor out tech debt of consensus (#2152) * [slash] Remove dead interface, associated piping * [slash] Expand out structs * [consensus] Write to a chan when find a case of double-signing, remove dead code * [slash] Broadcast the noticing of a double signing * [rawdb] CRUD for slashing candidates * [slashing][node][proto] Broadcast the slash record after receive from consensus, handle received proto message, persist in off-chain db while pending * [slash][node][propose-block] Add verified slashes proposed into the header in block proposal * [slash][shard] Factor out external validator as method on shard state, add double-signature field * [slash][engine] Apply slash, name boolean expression for sorts, use stable sort * [slash] Abstract Ballot results so keep track of both pre and post double sign event * [slash] Fix type errors on test code * [slash] Read from correct rawdb * [slash] Add epoch based guards in CRUD of slashing * [slash] Write to correct cache for slashing candidates * [shard] Use explicit named type of BLS Signature, use convention * [slash] Fix mistake done in refactor, improper header used. Factor out fromSlice to set * [slash][node] Restore newblock to master, try again minimial change * [cx-receipts] Break up one-liner, use SliceStable, not Slice * [network] Finish refactor that makes network message headers once * [network] Simplify creation further of headers write * [slash] Adjust data structure of slash after offline discussion with RJ, Chao * [slash] Still did need signature of the double signature * [consensus] Prepare message does not have block header * [consensus] Soft reset three files to 968517d~1 * [consensus] Begin factor consensus network intended message out with prepare first * [consensus] Factor out Prepared message * [consensus] Factor out announce message creation * [consensus] Committed Message, branch on verify sender key for clearer log * [consensus] Committed Message Factor out * [consensus] Do jenkins MVP of signatures adjustment * [main][slash] Provide YAML config as webhook config for double sign event * [consensus] Adjust signatures, whitespace, lessen GC pressure * [consensus] Remove dead code * [consensus] Factor out commit overloaded message, give commit payload override in construct * [consensus] Fix travis tests * [consensus] Provide block bytes in SubmitVote(quorum.Commit) * [consensus] Factor out noisy sanity checks in BFT, move existing commit check earlier as was before * [quorum] Adjust signatures in quorum * [staking] Adjust after merge from master * [consensus] Finish refactor of consensus * [node] Fix import * [consensus] Fix travis * [consensus] Use origin/master copy of block, fix mistake of pointer to empty byte * [consensus] Less verbose bools * [consensus] Remove unused trailing mutation hook in message construct * [consensus] Address some TODOs on err, comment out double sign
5 years ago
}
}
}()
}
[double-sign] Provide proof of double sign in slash record sent to beaconchain (#2253) * [double-sign] Commit changes in consensus needed for double-sign * [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain * [slash] Add quick iteration tool for testing double-signing * [slash] Add webhook example * [slash] Add http server for hook to trigger double sign behavior * [double-sign] Use bin/trigger-double-sign to cause a double-sign * [double-sign] Full feedback loop working * [slash] Thread through the slash records in the block proposal step * [slash] Compute the slashing rate * [double-sign] Generalize yaml malicious for many keys * [double-sign][slash] Modify data structures, verify via webhook handler * [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures * [slash] Apply to state slashing for double signing * [slash][double-sign] Checkpoint for working code that slashes on beaconchain * [slash] Keep track of the total slash and total reporters reward * [slash] Dump account state before and after the slash * [slash] Satisfy Travis * [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates * [slash] Capture the unique new delegations since snapshot as well * [slash] Filter undelegation by epoch of double sign * [slash] Add TODO of correctness needed in slash needs on off-chain data * [rpc] Fix closure issue on shardID * [slash] Add delegator to double-sign testing script * [slash] Expand crt-validator.sh with commenting printfs and make delegation * [slash] Finish track payment of leftover slash debt after undelegation runs out * [slash] Now be explicit about error wrt delegatorSlashApply * [slash] Capture specific sanity check on slash paidoff * [slash] Track slash from undelegation piecemeal * [slash][delegation] Named slice types, .String() * [slash] Do no RLP encode twice, once is enough * [slash] Remove special case of validators own delegation * [slash] Refactor approach to slash state application * [slash] Begin expanding out Verify * [slash] Slash on snapshot delegations, not current * [slash] Fix Epoch Cmp * [slash] Third iteration on slash logic * [slash] Use full slash amount * [slash] More log, whitespace * [slash] Remove Println, add log * [slash] Remove debug Println * [slash] Add record in unit test * [slash] Build Validator snapshot, current. Fill out slash record * [slash] Need to get RLP dump of a header to use in test * [slash] Factor out double sign test constants * [slash] Factor out common for validator, stub out slash application, finish out deserialization setup * [slash] Factor out data structure creation because of var lexical scoping * [slash] Seem to have pipeline of unit test e2e executing * [slash] Add expected snitch, slash amounts * [slash] Checkpoint * [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing * [config] add double-sign testnet config (#1) Signed-off-by: Leo Chen <leo@harmony.one> * [slash] Commit for as is code & data of current dump.json * [slash] Order of state operation not correct in test, hence bad results, thank you dlv * [slash] Add snapshot state dump * [slash] Pay off slash of validator own delegation correctly * [slash] Pay off slash debt with special case for min-self * [slash] Pass first scenario conclusively * [slash] 2% slash passes unit test for own delegation and external * [slash] Parameterize unit test to easily test .02 vs .80 slash * [slash] Handle own delegation correctly at 80% slash * [slash] Have 80% slash working with external delegator * [slash] Remove debug code from slash * [slash] Adjust Apply signature, test again for 2% slash * [slash] Factor out scenario in testing so can test 2% and 80% at same time * [slash] Correct balance deduction on plan delegation * [slash] Mock out ChainReader for TestVerify * [slash] Small surface area interface, now feedback loop for verify * [slash] Remove development json * [slash] trigger-double-sign consumes yaml * [slash] Remove dead code * [slash][test] Factor ValidatorWrapper into scenario * [slash][test] Add example from local-testing dump - caution might be off * [slash] Factor out mutation of slashDebt * [slash][test] Factor out tests so can easily load test-case from bytes * [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected * [slash] Satisfy Travis * [slash] Begin cleanup of PR * [slash] Apply slash from header to Finalize via state processor * [slash] Productionize code, Println => logs; adjust slash picked in newblock * [slash] Need pointer for rlp.Decode * [slash] ValidatorInformation use full wrapper * Fix median stake * [staking] Adjust MarshalJSON for Validator, Wrapper * Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279) * Refactor offchain data; Add epoch to ValidatorSnapshot * Make block onchain/offchain data commit atomically * [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment * [effective] VC eligible.go * [consensus] Redundant field in printf * [docker] import-ks for a dev account * [slash] Create BLS key for dockerfile and crt-validator.sh * [slash][docker] Easy deployment of double-sign testing * [docker] Have slash work as single docker command * [rpc] Fix median-stake RPC * [slash] Update webhook with default docker BLS key * [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go * [slash] Remove helper binary, commented out code, change to local config * [params] Factor out test genesis value * Add shard checking to Tx-Pool & correct blacklist (#2301) * [core] Fix blacklist & add shardID check * [staking + node + cmd] Fix blacklist & add shardID check * [slash] Adjust to PR comments part 1 * [docker] Use different throw away funded account * [docker] Create easier testing for delegation with private keys * [docker] Update yaml * [slash] Remove special case for slashing validator own delegation wrt min-self-delegate * [docker] Install nano as well * [slash] Early error if banned * [quorum] Expose earning account in decider marshal json * Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)" This reverts commit 9ffbf682c075b49188923c65a0bbf39ac188be00. * [slash] Add non-sanity check way to update validator * [reward] Increase percision on percentage in schedule * [slash] Adjust logs * [committee] Check eligibility of validator before doing sanity check * [slash] Update docker * [slash] Move create validator script to test * [slash] More log * [param] Make things faster * [slash][off-chain] Clear out slashes from pending in writeblockwithstate * [cross-link] Log is not error, just info * [blockchain] Not necessary to guard DeletePendingSlashingCandidates * [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that * [slash][test] Use faucet as sender, assume user imported * [slash] Test setup * [slash] reserve error for real error in logs * [slash][availability] Apply availability correct, bump signing count each block * [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed * [slash] Pay as much as can * [slash] use right nowAmt * [slash] Take away from rewards as well * [slash] iterate faster * [slash] Remove dev based timing * [slash] Add more log, sanity check incoming slash records, only count external for slash rate * [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters * [availability] More log * [slash] Simply pre slash erra slashing * [slash] Remove development code * [slash] Use height from recvMsg, todo on epoch * [staking] Not necessary to touch LastEpochInCommittee in staking_verifier * [slash] Undo ds in endpoint pattern config * [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment * [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count Co-authored-by: Leo Chen <leo@harmony.one> Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com> Co-authored-by: Rongjian Lan <rongjian@harmony.one> Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
5 years ago
return &node
7 years ago
}
// InitConsensusWithValidators initialize shard state
// from latest epoch and update committee pub
// keys for consensus
func (node *Node) InitConsensusWithValidators() (err error) {
if node.Consensus == nil {
[staking][validation][protocol] (#2396) * [staking][validation][protocol] Limit max bls keys * [staking-era] Fold banned and active into single field * [slash][effective] Remove LRU cache for slash, change .Active to enumeration * [slash] Remove leftover wrong usage of Logger * [slash][offchain] Only Decode if len > 0 * [offchain] cosmetic * [slash] Remove some logs in proposal * [webhook] Move webhook with call for when cannot commit block * [shard] Finally make finding subcommittee by shardID an explicit error * [node] Whitespace, prefer literal * [webhook] Report bad block to webhook * [slash] Expand verify, remove bad log usage, explicit error handle * [slash] Check on key size * [slash] Explicit upper bound of pending slashes * [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain * [multibls] Make max count allowed be 1/3 of external slots * [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee * [verify] Begin factor out of common verification approach * [project] Further remove RawJSON log, use proper epoch for snapshot * [slash] Implement verification * [slash] Implement BLS key verification of ballots * [rpc] Keep validator information as meaningful as possible * [staking] Never can stop being banned * [slash] Comments and default Unknown case of eligibility * [slash] Be explicit on what input values allowed when want to change EPOSStatus * [consensus] Remove unneeded TODO * [verify] Add proper error message * [rpc] Give back to caller their wrong chain id * [chain] Add extra map dump of delegation sizing for downstream analysis * [engine] Less code, more methods * [offchain] More leniency in handling slash bytes and delete from pending * [validator] Remove errors on bad input for edit
5 years ago
utils.Logger().Error().
Msg("[InitConsensusWithValidators] consenus is nil; Cannot figure out shardID")
return errors.New(
[staking][validation][protocol] (#2396) * [staking][validation][protocol] Limit max bls keys * [staking-era] Fold banned and active into single field * [slash][effective] Remove LRU cache for slash, change .Active to enumeration * [slash] Remove leftover wrong usage of Logger * [slash][offchain] Only Decode if len > 0 * [offchain] cosmetic * [slash] Remove some logs in proposal * [webhook] Move webhook with call for when cannot commit block * [shard] Finally make finding subcommittee by shardID an explicit error * [node] Whitespace, prefer literal * [webhook] Report bad block to webhook * [slash] Expand verify, remove bad log usage, explicit error handle * [slash] Check on key size * [slash] Explicit upper bound of pending slashes * [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain * [multibls] Make max count allowed be 1/3 of external slots * [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee * [verify] Begin factor out of common verification approach * [project] Further remove RawJSON log, use proper epoch for snapshot * [slash] Implement verification * [slash] Implement BLS key verification of ballots * [rpc] Keep validator information as meaningful as possible * [staking] Never can stop being banned * [slash] Comments and default Unknown case of eligibility * [slash] Be explicit on what input values allowed when want to change EPOSStatus * [consensus] Remove unneeded TODO * [verify] Add proper error message * [rpc] Give back to caller their wrong chain id * [chain] Add extra map dump of delegation sizing for downstream analysis * [engine] Less code, more methods * [offchain] More leniency in handling slash bytes and delete from pending * [validator] Remove errors on bad input for edit
5 years ago
"[InitConsensusWithValidators] consenus is nil; Cannot figure out shardID",
)
}
shardID := node.Consensus.ShardID
blockNum := node.Blockchain().CurrentBlock().NumberU64()
node.Consensus.SetMode(consensus.Listening)
epoch := shard.Schedule.CalcEpochNumber(blockNum)
utils.Logger().Info().
Uint64("blockNum", blockNum).
Uint32("shardID", shardID).
Uint64("epoch", epoch.Uint64()).
Msg("[InitConsensusWithValidators] Try To Get PublicKeys")
shardState, err := committee.WithStakingEnabled.Compute(
epoch, node.Consensus.ChainReader,
)
if err != nil {
utils.Logger().Err(err).
Uint64("blockNum", blockNum).
Uint32("shardID", shardID).
Uint64("epoch", epoch.Uint64()).
Msg("[InitConsensusWithValidators] Failed getting shard state")
return err
}
[staking][validation][protocol] (#2396) * [staking][validation][protocol] Limit max bls keys * [staking-era] Fold banned and active into single field * [slash][effective] Remove LRU cache for slash, change .Active to enumeration * [slash] Remove leftover wrong usage of Logger * [slash][offchain] Only Decode if len > 0 * [offchain] cosmetic * [slash] Remove some logs in proposal * [webhook] Move webhook with call for when cannot commit block * [shard] Finally make finding subcommittee by shardID an explicit error * [node] Whitespace, prefer literal * [webhook] Report bad block to webhook * [slash] Expand verify, remove bad log usage, explicit error handle * [slash] Check on key size * [slash] Explicit upper bound of pending slashes * [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain * [multibls] Make max count allowed be 1/3 of external slots * [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee * [verify] Begin factor out of common verification approach * [project] Further remove RawJSON log, use proper epoch for snapshot * [slash] Implement verification * [slash] Implement BLS key verification of ballots * [rpc] Keep validator information as meaningful as possible * [staking] Never can stop being banned * [slash] Comments and default Unknown case of eligibility * [slash] Be explicit on what input values allowed when want to change EPOSStatus * [consensus] Remove unneeded TODO * [verify] Add proper error message * [rpc] Give back to caller their wrong chain id * [chain] Add extra map dump of delegation sizing for downstream analysis * [engine] Less code, more methods * [offchain] More leniency in handling slash bytes and delete from pending * [validator] Remove errors on bad input for edit
5 years ago
subComm, err := shardState.FindCommitteeByID(shardID)
if err != nil {
return err
}
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
pubKeys, err := subComm.BLSPublicKeys()
if err != nil {
utils.Logger().Error().
Uint32("shardID", shardID).
Uint64("blockNum", blockNum).
Msg("[InitConsensusWithValidators] PublicKeys is Empty, Cannot update public keys")
return errors.Wrapf(
err,
"[InitConsensusWithValidators] PublicKeys is Empty, Cannot update public keys",
)
}
for _, key := range pubKeys {
if node.Consensus.PubKey.Contains(key) {
utils.Logger().Info().
Uint64("blockNum", blockNum).
Int("numPubKeys", len(pubKeys)).
Msg("[InitConsensusWithValidators] Successfully updated public keys")
node.Consensus.UpdatePublicKeys(pubKeys)
node.Consensus.SetMode(consensus.Normal)
return nil
}
}
return nil
}
// AddPeers adds neighbors nodes
func (node *Node) AddPeers(peers []*p2p.Peer) int {
count := 0
for _, p := range peers {
key := fmt.Sprintf("%s:%s:%s", p.IP, p.Port, p.PeerID)
_, ok := node.Neighbors.LoadOrStore(key, *p)
if !ok {
// !ok means new peer is stored
count++
node.host.AddPeer(p)
node.numPeers++
continue
}
}
return count
}
// AddBeaconPeer adds beacon chain neighbors nodes
// Return false means new neighbor peer was added
// Return true means redundant neighbor peer wasn't added
func (node *Node) AddBeaconPeer(p *p2p.Peer) bool {
key := fmt.Sprintf("%s:%s:%s", p.IP, p.Port, p.PeerID)
_, ok := node.BeaconNeighbors.LoadOrStore(key, *p)
return ok
}
// isBeacon = true if the node is beacon node
func (node *Node) initNodeConfiguration() (service.NodeConfig, chan p2p.Peer) {
chanPeer := make(chan p2p.Peer)
nodeConfig := service.NodeConfig{
IsClient: node.NodeConfig.IsClient(),
Beacon: nodeconfig.NewGroupIDByShardID(shard.BeaconChainShardID),
ShardGroupID: node.NodeConfig.GetShardGroupID(),
Actions: make(map[nodeconfig.GroupID]nodeconfig.ActionType),
}
if nodeConfig.IsClient {
[staking][validation][protocol] (#2396) * [staking][validation][protocol] Limit max bls keys * [staking-era] Fold banned and active into single field * [slash][effective] Remove LRU cache for slash, change .Active to enumeration * [slash] Remove leftover wrong usage of Logger * [slash][offchain] Only Decode if len > 0 * [offchain] cosmetic * [slash] Remove some logs in proposal * [webhook] Move webhook with call for when cannot commit block * [shard] Finally make finding subcommittee by shardID an explicit error * [node] Whitespace, prefer literal * [webhook] Report bad block to webhook * [slash] Expand verify, remove bad log usage, explicit error handle * [slash] Check on key size * [slash] Explicit upper bound of pending slashes * [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain * [multibls] Make max count allowed be 1/3 of external slots * [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee * [verify] Begin factor out of common verification approach * [project] Further remove RawJSON log, use proper epoch for snapshot * [slash] Implement verification * [slash] Implement BLS key verification of ballots * [rpc] Keep validator information as meaningful as possible * [staking] Never can stop being banned * [slash] Comments and default Unknown case of eligibility * [slash] Be explicit on what input values allowed when want to change EPOSStatus * [consensus] Remove unneeded TODO * [verify] Add proper error message * [rpc] Give back to caller their wrong chain id * [chain] Add extra map dump of delegation sizing for downstream analysis * [engine] Less code, more methods * [offchain] More leniency in handling slash bytes and delete from pending * [validator] Remove errors on bad input for edit
5 years ago
nodeConfig.Actions[nodeconfig.NewClientGroupIDByShardID(shard.BeaconChainShardID)] =
nodeconfig.ActionStart
} else {
nodeConfig.Actions[node.NodeConfig.GetShardGroupID()] = nodeconfig.ActionStart
}
var err error
node.shardGroupReceiver, err = node.host.GroupReceiver(node.NodeConfig.GetShardGroupID())
if err != nil {
utils.Logger().Error().Err(err).Msg("Failed to create shard receiver")
}
[staking][validation][protocol] (#2396) * [staking][validation][protocol] Limit max bls keys * [staking-era] Fold banned and active into single field * [slash][effective] Remove LRU cache for slash, change .Active to enumeration * [slash] Remove leftover wrong usage of Logger * [slash][offchain] Only Decode if len > 0 * [offchain] cosmetic * [slash] Remove some logs in proposal * [webhook] Move webhook with call for when cannot commit block * [shard] Finally make finding subcommittee by shardID an explicit error * [node] Whitespace, prefer literal * [webhook] Report bad block to webhook * [slash] Expand verify, remove bad log usage, explicit error handle * [slash] Check on key size * [slash] Explicit upper bound of pending slashes * [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain * [multibls] Make max count allowed be 1/3 of external slots * [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee * [verify] Begin factor out of common verification approach * [project] Further remove RawJSON log, use proper epoch for snapshot * [slash] Implement verification * [slash] Implement BLS key verification of ballots * [rpc] Keep validator information as meaningful as possible * [staking] Never can stop being banned * [slash] Comments and default Unknown case of eligibility * [slash] Be explicit on what input values allowed when want to change EPOSStatus * [consensus] Remove unneeded TODO * [verify] Add proper error message * [rpc] Give back to caller their wrong chain id * [chain] Add extra map dump of delegation sizing for downstream analysis * [engine] Less code, more methods * [offchain] More leniency in handling slash bytes and delete from pending * [validator] Remove errors on bad input for edit
5 years ago
node.globalGroupReceiver, err = node.host.GroupReceiver(
nodeconfig.NewClientGroupIDByShardID(shard.BeaconChainShardID),
)
if err != nil {
utils.Logger().Error().Err(err).Msg("Failed to create global receiver")
}
node.clientReceiver, err = node.host.GroupReceiver(node.NodeConfig.GetClientGroupID())
if err != nil {
utils.Logger().Error().Err(err).Msg("Failed to create client receiver")
}
return nodeConfig, chanPeer
}
# This is a combination of 16 commits. # This is the 1st commit message: Fix travis # This is the commit message #2: [cleanup] remove is_genesis flag Signed-off-by: Leo Chen <leo@harmony.one> # This is the commit message #3: [nodetype] add nodetype to replace is_explorer Signed-off-by: Leo Chen <leo@harmony.one> # This is the commit message #4: fix beacon sync channel blocking issue # This is the commit message #5: use lastMileMux to protect lastMileBlock queue to avoid potential blocking; use sleep instead of ticker # This is the commit message #6: Fix the beacon committee check # This is the commit message #7: [release] add release action to go_executable_build.sh Signed-off-by: Leo Chen <leo@harmony.one> # This is the commit message #8: [node.sh] add -d option -d download only Signed-off-by: Leo Chen <leo@harmony.one> # This is the commit message #9: [node.sh] add -T node_type option -T node_type support node type (validator/explorer) Signed-off-by: Leo Chen <leo@harmony.one> # This is the commit message #10: [node.sh] backward compatible with older harmony binary Signed-off-by: Leo Chen <leo@harmony.one> # This is the commit message #11: [node.sh] support -i shard_id option -i shard_id specify shard_id, this is applicable only to explorer node Signed-off-by: Leo Chen <leo@harmony.one> # This is the commit message #12: Revisited api # This is the commit message #13: Fix rpc integration # This is the commit message #14: address some minor issues in comments and code # This is the commit message #15: addressed comments on others' buckets # This is the commit message #16: Add Global Access to OS Temp Directory Variable and Move DHT Files Into Temp Directory
5 years ago
// ServiceManager ...
func (node *Node) ServiceManager() *service.Manager {
return node.serviceManager
}
// ShutDown gracefully shut down the node server and dump the in-memory blockchain state into DB.
func (node *Node) ShutDown() {
node.Blockchain().Stop()
node.Beaconchain().Stop()
msg := "Successfully shut down!\n"
utils.Logger().Print(msg)
fmt.Print(msg)
os.Exit(0)
}