The core protocol of WoopChain
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
woop/staking/availability/measure.go

239 lines
5.8 KiB

[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
package availability
import (
"math/big"
"github.com/ethereum/go-ethereum/common"
"github.com/harmony-one/harmony/block"
"github.com/harmony-one/harmony/core/state"
bls2 "github.com/harmony-one/harmony/crypto/bls"
[double-sign] Provide proof of double sign in slash record sent to beaconchain (#2253) * [double-sign] Commit changes in consensus needed for double-sign * [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain * [slash] Add quick iteration tool for testing double-signing * [slash] Add webhook example * [slash] Add http server for hook to trigger double sign behavior * [double-sign] Use bin/trigger-double-sign to cause a double-sign * [double-sign] Full feedback loop working * [slash] Thread through the slash records in the block proposal step * [slash] Compute the slashing rate * [double-sign] Generalize yaml malicious for many keys * [double-sign][slash] Modify data structures, verify via webhook handler * [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures * [slash] Apply to state slashing for double signing * [slash][double-sign] Checkpoint for working code that slashes on beaconchain * [slash] Keep track of the total slash and total reporters reward * [slash] Dump account state before and after the slash * [slash] Satisfy Travis * [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates * [slash] Capture the unique new delegations since snapshot as well * [slash] Filter undelegation by epoch of double sign * [slash] Add TODO of correctness needed in slash needs on off-chain data * [rpc] Fix closure issue on shardID * [slash] Add delegator to double-sign testing script * [slash] Expand crt-validator.sh with commenting printfs and make delegation * [slash] Finish track payment of leftover slash debt after undelegation runs out * [slash] Now be explicit about error wrt delegatorSlashApply * [slash] Capture specific sanity check on slash paidoff * [slash] Track slash from undelegation piecemeal * [slash][delegation] Named slice types, .String() * [slash] Do no RLP encode twice, once is enough * [slash] Remove special case of validators own delegation * [slash] Refactor approach to slash state application * [slash] Begin expanding out Verify * [slash] Slash on snapshot delegations, not current * [slash] Fix Epoch Cmp * [slash] Third iteration on slash logic * [slash] Use full slash amount * [slash] More log, whitespace * [slash] Remove Println, add log * [slash] Remove debug Println * [slash] Add record in unit test * [slash] Build Validator snapshot, current. Fill out slash record * [slash] Need to get RLP dump of a header to use in test * [slash] Factor out double sign test constants * [slash] Factor out common for validator, stub out slash application, finish out deserialization setup * [slash] Factor out data structure creation because of var lexical scoping * [slash] Seem to have pipeline of unit test e2e executing * [slash] Add expected snitch, slash amounts * [slash] Checkpoint * [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing * [config] add double-sign testnet config (#1) Signed-off-by: Leo Chen <leo@harmony.one> * [slash] Commit for as is code & data of current dump.json * [slash] Order of state operation not correct in test, hence bad results, thank you dlv * [slash] Add snapshot state dump * [slash] Pay off slash of validator own delegation correctly * [slash] Pay off slash debt with special case for min-self * [slash] Pass first scenario conclusively * [slash] 2% slash passes unit test for own delegation and external * [slash] Parameterize unit test to easily test .02 vs .80 slash * [slash] Handle own delegation correctly at 80% slash * [slash] Have 80% slash working with external delegator * [slash] Remove debug code from slash * [slash] Adjust Apply signature, test again for 2% slash * [slash] Factor out scenario in testing so can test 2% and 80% at same time * [slash] Correct balance deduction on plan delegation * [slash] Mock out ChainReader for TestVerify * [slash] Small surface area interface, now feedback loop for verify * [slash] Remove development json * [slash] trigger-double-sign consumes yaml * [slash] Remove dead code * [slash][test] Factor ValidatorWrapper into scenario * [slash][test] Add example from local-testing dump - caution might be off * [slash] Factor out mutation of slashDebt * [slash][test] Factor out tests so can easily load test-case from bytes * [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected * [slash] Satisfy Travis * [slash] Begin cleanup of PR * [slash] Apply slash from header to Finalize via state processor * [slash] Productionize code, Println => logs; adjust slash picked in newblock * [slash] Need pointer for rlp.Decode * [slash] ValidatorInformation use full wrapper * Fix median stake * [staking] Adjust MarshalJSON for Validator, Wrapper * Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279) * Refactor offchain data; Add epoch to ValidatorSnapshot * Make block onchain/offchain data commit atomically * [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment * [effective] VC eligible.go * [consensus] Redundant field in printf * [docker] import-ks for a dev account * [slash] Create BLS key for dockerfile and crt-validator.sh * [slash][docker] Easy deployment of double-sign testing * [docker] Have slash work as single docker command * [rpc] Fix median-stake RPC * [slash] Update webhook with default docker BLS key * [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go * [slash] Remove helper binary, commented out code, change to local config * [params] Factor out test genesis value * Add shard checking to Tx-Pool & correct blacklist (#2301) * [core] Fix blacklist & add shardID check * [staking + node + cmd] Fix blacklist & add shardID check * [slash] Adjust to PR comments part 1 * [docker] Use different throw away funded account * [docker] Create easier testing for delegation with private keys * [docker] Update yaml * [slash] Remove special case for slashing validator own delegation wrt min-self-delegate * [docker] Install nano as well * [slash] Early error if banned * [quorum] Expose earning account in decider marshal json * Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)" This reverts commit 9ffbf682c075b49188923c65a0bbf39ac188be00. * [slash] Add non-sanity check way to update validator * [reward] Increase percision on percentage in schedule * [slash] Adjust logs * [committee] Check eligibility of validator before doing sanity check * [slash] Update docker * [slash] Move create validator script to test * [slash] More log * [param] Make things faster * [slash][off-chain] Clear out slashes from pending in writeblockwithstate * [cross-link] Log is not error, just info * [blockchain] Not necessary to guard DeletePendingSlashingCandidates * [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that * [slash][test] Use faucet as sender, assume user imported * [slash] Test setup * [slash] reserve error for real error in logs * [slash][availability] Apply availability correct, bump signing count each block * [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed * [slash] Pay as much as can * [slash] use right nowAmt * [slash] Take away from rewards as well * [slash] iterate faster * [slash] Remove dev based timing * [slash] Add more log, sanity check incoming slash records, only count external for slash rate * [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters * [availability] More log * [slash] Simply pre slash erra slashing * [slash] Remove development code * [slash] Use height from recvMsg, todo on epoch * [staking] Not necessary to touch LastEpochInCommittee in staking_verifier * [slash] Undo ds in endpoint pattern config * [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment * [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count Co-authored-by: Leo Chen <leo@harmony.one> Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com> Co-authored-by: Rongjian Lan <rongjian@harmony.one> Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
5 years ago
"github.com/harmony-one/harmony/internal/utils"
"github.com/harmony-one/harmony/numeric"
[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
"github.com/harmony-one/harmony/shard"
[staking][validation][protocol] (#2396) * [staking][validation][protocol] Limit max bls keys * [staking-era] Fold banned and active into single field * [slash][effective] Remove LRU cache for slash, change .Active to enumeration * [slash] Remove leftover wrong usage of Logger * [slash][offchain] Only Decode if len > 0 * [offchain] cosmetic * [slash] Remove some logs in proposal * [webhook] Move webhook with call for when cannot commit block * [shard] Finally make finding subcommittee by shardID an explicit error * [node] Whitespace, prefer literal * [webhook] Report bad block to webhook * [slash] Expand verify, remove bad log usage, explicit error handle * [slash] Check on key size * [slash] Explicit upper bound of pending slashes * [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain * [multibls] Make max count allowed be 1/3 of external slots * [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee * [verify] Begin factor out of common verification approach * [project] Further remove RawJSON log, use proper epoch for snapshot * [slash] Implement verification * [slash] Implement BLS key verification of ballots * [rpc] Keep validator information as meaningful as possible * [staking] Never can stop being banned * [slash] Comments and default Unknown case of eligibility * [slash] Be explicit on what input values allowed when want to change EPOSStatus * [consensus] Remove unneeded TODO * [verify] Add proper error message * [rpc] Give back to caller their wrong chain id * [chain] Add extra map dump of delegation sizing for downstream analysis * [engine] Less code, more methods * [offchain] More leniency in handling slash bytes and delete from pending * [validator] Remove errors on bad input for edit
5 years ago
"github.com/harmony-one/harmony/staking/effective"
staking "github.com/harmony-one/harmony/staking/types"
[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
"github.com/pkg/errors"
)
var (
measure = numeric.NewDec(2).Quo(numeric.NewDec(3))
// ErrDivByZero ..
ErrDivByZero = errors.New("toSign of availability cannot be 0, mistake in protocol")
[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
)
// BlockSigners ..
func BlockSigners(
bitmap []byte, parentCommittee *shard.Committee,
) (shard.SlotList, shard.SlotList, error) {
[staking][validation][protocol] (#2396) * [staking][validation][protocol] Limit max bls keys * [staking-era] Fold banned and active into single field * [slash][effective] Remove LRU cache for slash, change .Active to enumeration * [slash] Remove leftover wrong usage of Logger * [slash][offchain] Only Decode if len > 0 * [offchain] cosmetic * [slash] Remove some logs in proposal * [webhook] Move webhook with call for when cannot commit block * [shard] Finally make finding subcommittee by shardID an explicit error * [node] Whitespace, prefer literal * [webhook] Report bad block to webhook * [slash] Expand verify, remove bad log usage, explicit error handle * [slash] Check on key size * [slash] Explicit upper bound of pending slashes * [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain * [multibls] Make max count allowed be 1/3 of external slots * [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee * [verify] Begin factor out of common verification approach * [project] Further remove RawJSON log, use proper epoch for snapshot * [slash] Implement verification * [slash] Implement BLS key verification of ballots * [rpc] Keep validator information as meaningful as possible * [staking] Never can stop being banned * [slash] Comments and default Unknown case of eligibility * [slash] Be explicit on what input values allowed when want to change EPOSStatus * [consensus] Remove unneeded TODO * [verify] Add proper error message * [rpc] Give back to caller their wrong chain id * [chain] Add extra map dump of delegation sizing for downstream analysis * [engine] Less code, more methods * [offchain] More leniency in handling slash bytes and delete from pending * [validator] Remove errors on bad input for edit
5 years ago
committerKeys, err := parentCommittee.BLSPublicKeys()
if err != nil {
return nil, nil, err
[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
}
mask, err := bls2.NewMask(committerKeys, nil)
if err != nil {
return nil, nil, err
[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
}
if err := mask.SetMask(bitmap); err != nil {
return nil, nil, err
[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
}
payable, missing := shard.SlotList{}, shard.SlotList{}
for idx, member := range parentCommittee.Slots {
switch signed, err := mask.IndexEnabled(idx); true {
case err != nil:
return nil, nil, err
[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
case signed:
payable = append(payable, member)
default:
missing = append(missing, member)
}
}
return payable, missing, nil
}
// BallotResult returns
// (parentCommittee.Slots, payable, missing, err)
[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
func BallotResult(
parentHeader, header *block.Header, parentShardState *shard.State, shardID uint32,
[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
) (shard.SlotList, shard.SlotList, shard.SlotList, error) {
[staking][validation][protocol] (#2396) * [staking][validation][protocol] Limit max bls keys * [staking-era] Fold banned and active into single field * [slash][effective] Remove LRU cache for slash, change .Active to enumeration * [slash] Remove leftover wrong usage of Logger * [slash][offchain] Only Decode if len > 0 * [offchain] cosmetic * [slash] Remove some logs in proposal * [webhook] Move webhook with call for when cannot commit block * [shard] Finally make finding subcommittee by shardID an explicit error * [node] Whitespace, prefer literal * [webhook] Report bad block to webhook * [slash] Expand verify, remove bad log usage, explicit error handle * [slash] Check on key size * [slash] Explicit upper bound of pending slashes * [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain * [multibls] Make max count allowed be 1/3 of external slots * [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee * [verify] Begin factor out of common verification approach * [project] Further remove RawJSON log, use proper epoch for snapshot * [slash] Implement verification * [slash] Implement BLS key verification of ballots * [rpc] Keep validator information as meaningful as possible * [staking] Never can stop being banned * [slash] Comments and default Unknown case of eligibility * [slash] Be explicit on what input values allowed when want to change EPOSStatus * [consensus] Remove unneeded TODO * [verify] Add proper error message * [rpc] Give back to caller their wrong chain id * [chain] Add extra map dump of delegation sizing for downstream analysis * [engine] Less code, more methods * [offchain] More leniency in handling slash bytes and delete from pending * [validator] Remove errors on bad input for edit
5 years ago
parentCommittee, err := parentShardState.FindCommitteeByID(shardID)
if err != nil {
return nil, nil, nil, errors.Errorf(
"cannot find shard in the shard state %d %d",
parentHeader.Number(),
parentHeader.ShardID(),
[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
)
}
payable, missing, err := BlockSigners(
header.LastCommitBitmap(), parentCommittee,
)
[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
return parentCommittee.Slots, payable, missing, err
}
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
type signerKind struct {
didSign bool
committee shard.SlotList
}
func bumpCount(
bc Reader,
[double-sign] Provide proof of double sign in slash record sent to beaconchain (#2253) * [double-sign] Commit changes in consensus needed for double-sign * [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain * [slash] Add quick iteration tool for testing double-signing * [slash] Add webhook example * [slash] Add http server for hook to trigger double sign behavior * [double-sign] Use bin/trigger-double-sign to cause a double-sign * [double-sign] Full feedback loop working * [slash] Thread through the slash records in the block proposal step * [slash] Compute the slashing rate * [double-sign] Generalize yaml malicious for many keys * [double-sign][slash] Modify data structures, verify via webhook handler * [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures * [slash] Apply to state slashing for double signing * [slash][double-sign] Checkpoint for working code that slashes on beaconchain * [slash] Keep track of the total slash and total reporters reward * [slash] Dump account state before and after the slash * [slash] Satisfy Travis * [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates * [slash] Capture the unique new delegations since snapshot as well * [slash] Filter undelegation by epoch of double sign * [slash] Add TODO of correctness needed in slash needs on off-chain data * [rpc] Fix closure issue on shardID * [slash] Add delegator to double-sign testing script * [slash] Expand crt-validator.sh with commenting printfs and make delegation * [slash] Finish track payment of leftover slash debt after undelegation runs out * [slash] Now be explicit about error wrt delegatorSlashApply * [slash] Capture specific sanity check on slash paidoff * [slash] Track slash from undelegation piecemeal * [slash][delegation] Named slice types, .String() * [slash] Do no RLP encode twice, once is enough * [slash] Remove special case of validators own delegation * [slash] Refactor approach to slash state application * [slash] Begin expanding out Verify * [slash] Slash on snapshot delegations, not current * [slash] Fix Epoch Cmp * [slash] Third iteration on slash logic * [slash] Use full slash amount * [slash] More log, whitespace * [slash] Remove Println, add log * [slash] Remove debug Println * [slash] Add record in unit test * [slash] Build Validator snapshot, current. Fill out slash record * [slash] Need to get RLP dump of a header to use in test * [slash] Factor out double sign test constants * [slash] Factor out common for validator, stub out slash application, finish out deserialization setup * [slash] Factor out data structure creation because of var lexical scoping * [slash] Seem to have pipeline of unit test e2e executing * [slash] Add expected snitch, slash amounts * [slash] Checkpoint * [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing * [config] add double-sign testnet config (#1) Signed-off-by: Leo Chen <leo@harmony.one> * [slash] Commit for as is code & data of current dump.json * [slash] Order of state operation not correct in test, hence bad results, thank you dlv * [slash] Add snapshot state dump * [slash] Pay off slash of validator own delegation correctly * [slash] Pay off slash debt with special case for min-self * [slash] Pass first scenario conclusively * [slash] 2% slash passes unit test for own delegation and external * [slash] Parameterize unit test to easily test .02 vs .80 slash * [slash] Handle own delegation correctly at 80% slash * [slash] Have 80% slash working with external delegator * [slash] Remove debug code from slash * [slash] Adjust Apply signature, test again for 2% slash * [slash] Factor out scenario in testing so can test 2% and 80% at same time * [slash] Correct balance deduction on plan delegation * [slash] Mock out ChainReader for TestVerify * [slash] Small surface area interface, now feedback loop for verify * [slash] Remove development json * [slash] trigger-double-sign consumes yaml * [slash] Remove dead code * [slash][test] Factor ValidatorWrapper into scenario * [slash][test] Add example from local-testing dump - caution might be off * [slash] Factor out mutation of slashDebt * [slash][test] Factor out tests so can easily load test-case from bytes * [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected * [slash] Satisfy Travis * [slash] Begin cleanup of PR * [slash] Apply slash from header to Finalize via state processor * [slash] Productionize code, Println => logs; adjust slash picked in newblock * [slash] Need pointer for rlp.Decode * [slash] ValidatorInformation use full wrapper * Fix median stake * [staking] Adjust MarshalJSON for Validator, Wrapper * Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279) * Refactor offchain data; Add epoch to ValidatorSnapshot * Make block onchain/offchain data commit atomically * [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment * [effective] VC eligible.go * [consensus] Redundant field in printf * [docker] import-ks for a dev account * [slash] Create BLS key for dockerfile and crt-validator.sh * [slash][docker] Easy deployment of double-sign testing * [docker] Have slash work as single docker command * [rpc] Fix median-stake RPC * [slash] Update webhook with default docker BLS key * [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go * [slash] Remove helper binary, commented out code, change to local config * [params] Factor out test genesis value * Add shard checking to Tx-Pool & correct blacklist (#2301) * [core] Fix blacklist & add shardID check * [staking + node + cmd] Fix blacklist & add shardID check * [slash] Adjust to PR comments part 1 * [docker] Use different throw away funded account * [docker] Create easier testing for delegation with private keys * [docker] Update yaml * [slash] Remove special case for slashing validator own delegation wrt min-self-delegate * [docker] Install nano as well * [slash] Early error if banned * [quorum] Expose earning account in decider marshal json * Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)" This reverts commit 9ffbf682c075b49188923c65a0bbf39ac188be00. * [slash] Add non-sanity check way to update validator * [reward] Increase percision on percentage in schedule * [slash] Adjust logs * [committee] Check eligibility of validator before doing sanity check * [slash] Update docker * [slash] Move create validator script to test * [slash] More log * [param] Make things faster * [slash][off-chain] Clear out slashes from pending in writeblockwithstate * [cross-link] Log is not error, just info * [blockchain] Not necessary to guard DeletePendingSlashingCandidates * [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that * [slash][test] Use faucet as sender, assume user imported * [slash] Test setup * [slash] reserve error for real error in logs * [slash][availability] Apply availability correct, bump signing count each block * [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed * [slash] Pay as much as can * [slash] use right nowAmt * [slash] Take away from rewards as well * [slash] iterate faster * [slash] Remove dev based timing * [slash] Add more log, sanity check incoming slash records, only count external for slash rate * [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters * [availability] More log * [slash] Simply pre slash erra slashing * [slash] Remove development code * [slash] Use height from recvMsg, todo on epoch * [staking] Not necessary to touch LastEpochInCommittee in staking_verifier * [slash] Undo ds in endpoint pattern config * [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment * [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count Co-authored-by: Leo Chen <leo@harmony.one> Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com> Co-authored-by: Rongjian Lan <rongjian@harmony.one> Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
5 years ago
state *state.DB,
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
signers []signerKind,
[double-sign] Provide proof of double sign in slash record sent to beaconchain (#2253) * [double-sign] Commit changes in consensus needed for double-sign * [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain * [slash] Add quick iteration tool for testing double-signing * [slash] Add webhook example * [slash] Add http server for hook to trigger double sign behavior * [double-sign] Use bin/trigger-double-sign to cause a double-sign * [double-sign] Full feedback loop working * [slash] Thread through the slash records in the block proposal step * [slash] Compute the slashing rate * [double-sign] Generalize yaml malicious for many keys * [double-sign][slash] Modify data structures, verify via webhook handler * [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures * [slash] Apply to state slashing for double signing * [slash][double-sign] Checkpoint for working code that slashes on beaconchain * [slash] Keep track of the total slash and total reporters reward * [slash] Dump account state before and after the slash * [slash] Satisfy Travis * [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates * [slash] Capture the unique new delegations since snapshot as well * [slash] Filter undelegation by epoch of double sign * [slash] Add TODO of correctness needed in slash needs on off-chain data * [rpc] Fix closure issue on shardID * [slash] Add delegator to double-sign testing script * [slash] Expand crt-validator.sh with commenting printfs and make delegation * [slash] Finish track payment of leftover slash debt after undelegation runs out * [slash] Now be explicit about error wrt delegatorSlashApply * [slash] Capture specific sanity check on slash paidoff * [slash] Track slash from undelegation piecemeal * [slash][delegation] Named slice types, .String() * [slash] Do no RLP encode twice, once is enough * [slash] Remove special case of validators own delegation * [slash] Refactor approach to slash state application * [slash] Begin expanding out Verify * [slash] Slash on snapshot delegations, not current * [slash] Fix Epoch Cmp * [slash] Third iteration on slash logic * [slash] Use full slash amount * [slash] More log, whitespace * [slash] Remove Println, add log * [slash] Remove debug Println * [slash] Add record in unit test * [slash] Build Validator snapshot, current. Fill out slash record * [slash] Need to get RLP dump of a header to use in test * [slash] Factor out double sign test constants * [slash] Factor out common for validator, stub out slash application, finish out deserialization setup * [slash] Factor out data structure creation because of var lexical scoping * [slash] Seem to have pipeline of unit test e2e executing * [slash] Add expected snitch, slash amounts * [slash] Checkpoint * [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing * [config] add double-sign testnet config (#1) Signed-off-by: Leo Chen <leo@harmony.one> * [slash] Commit for as is code & data of current dump.json * [slash] Order of state operation not correct in test, hence bad results, thank you dlv * [slash] Add snapshot state dump * [slash] Pay off slash of validator own delegation correctly * [slash] Pay off slash debt with special case for min-self * [slash] Pass first scenario conclusively * [slash] 2% slash passes unit test for own delegation and external * [slash] Parameterize unit test to easily test .02 vs .80 slash * [slash] Handle own delegation correctly at 80% slash * [slash] Have 80% slash working with external delegator * [slash] Remove debug code from slash * [slash] Adjust Apply signature, test again for 2% slash * [slash] Factor out scenario in testing so can test 2% and 80% at same time * [slash] Correct balance deduction on plan delegation * [slash] Mock out ChainReader for TestVerify * [slash] Small surface area interface, now feedback loop for verify * [slash] Remove development json * [slash] trigger-double-sign consumes yaml * [slash] Remove dead code * [slash][test] Factor ValidatorWrapper into scenario * [slash][test] Add example from local-testing dump - caution might be off * [slash] Factor out mutation of slashDebt * [slash][test] Factor out tests so can easily load test-case from bytes * [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected * [slash] Satisfy Travis * [slash] Begin cleanup of PR * [slash] Apply slash from header to Finalize via state processor * [slash] Productionize code, Println => logs; adjust slash picked in newblock * [slash] Need pointer for rlp.Decode * [slash] ValidatorInformation use full wrapper * Fix median stake * [staking] Adjust MarshalJSON for Validator, Wrapper * Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279) * Refactor offchain data; Add epoch to ValidatorSnapshot * Make block onchain/offchain data commit atomically * [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment * [effective] VC eligible.go * [consensus] Redundant field in printf * [docker] import-ks for a dev account * [slash] Create BLS key for dockerfile and crt-validator.sh * [slash][docker] Easy deployment of double-sign testing * [docker] Have slash work as single docker command * [rpc] Fix median-stake RPC * [slash] Update webhook with default docker BLS key * [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go * [slash] Remove helper binary, commented out code, change to local config * [params] Factor out test genesis value * Add shard checking to Tx-Pool & correct blacklist (#2301) * [core] Fix blacklist & add shardID check * [staking + node + cmd] Fix blacklist & add shardID check * [slash] Adjust to PR comments part 1 * [docker] Use different throw away funded account * [docker] Create easier testing for delegation with private keys * [docker] Update yaml * [slash] Remove special case for slashing validator own delegation wrt min-self-delegate * [docker] Install nano as well * [slash] Early error if banned * [quorum] Expose earning account in decider marshal json * Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)" This reverts commit 9ffbf682c075b49188923c65a0bbf39ac188be00. * [slash] Add non-sanity check way to update validator * [reward] Increase percision on percentage in schedule * [slash] Adjust logs * [committee] Check eligibility of validator before doing sanity check * [slash] Update docker * [slash] Move create validator script to test * [slash] More log * [param] Make things faster * [slash][off-chain] Clear out slashes from pending in writeblockwithstate * [cross-link] Log is not error, just info * [blockchain] Not necessary to guard DeletePendingSlashingCandidates * [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that * [slash][test] Use faucet as sender, assume user imported * [slash] Test setup * [slash] reserve error for real error in logs * [slash][availability] Apply availability correct, bump signing count each block * [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed * [slash] Pay as much as can * [slash] use right nowAmt * [slash] Take away from rewards as well * [slash] iterate faster * [slash] Remove dev based timing * [slash] Add more log, sanity check incoming slash records, only count external for slash rate * [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters * [availability] More log * [slash] Simply pre slash erra slashing * [slash] Remove development code * [slash] Use height from recvMsg, todo on epoch * [staking] Not necessary to touch LastEpochInCommittee in staking_verifier * [slash] Undo ds in endpoint pattern config * [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment * [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count Co-authored-by: Leo Chen <leo@harmony.one> Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com> Co-authored-by: Rongjian Lan <rongjian@harmony.one> Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
5 years ago
stakedAddrSet map[common.Address]struct{},
[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
) error {
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
for _, subset := range signers {
for i := range subset.committee {
addr := subset.committee[i].EcdsaAddress
// NOTE if the signer address is not part of the staked addrs,
// then it must be a harmony operated node running,
// hence keep on going
if _, isAddrForStaked := stakedAddrSet[addr]; !isAddrForStaked {
continue
}
wrapper, err := state.ValidatorWrapper(addr)
if err != nil {
return err
}
wrapper.Counters.NumBlocksToSign.Add(
wrapper.Counters.NumBlocksToSign, common.Big1,
)
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
if subset.didSign {
wrapper.Counters.NumBlocksSigned.Add(
wrapper.Counters.NumBlocksSigned, common.Big1,
)
}
if err := state.UpdateValidatorWrapper(
addr, wrapper,
); err != nil {
return err
}
[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
}
}
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
return nil
}
// IncrementValidatorSigningCounts ..
func IncrementValidatorSigningCounts(
bc Reader,
staked *shard.StakedSlots,
state *state.DB,
signers, missing shard.SlotList,
) error {
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
return bumpCount(
bc, state, []signerKind{{false, missing}, {true, signers}},
staked.LookupSet,
)
}
// Reader ..
type Reader interface {
ReadValidatorSnapshot(
addr common.Address,
) (*staking.ValidatorWrapper, error)
[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
}
// ComputeCurrentSigning returns (signed, toSign, quotient, error)
func ComputeCurrentSigning(
snapshot, wrapper *staking.ValidatorWrapper,
) *staking.Computed {
statsNow, snapSigned, snapToSign :=
wrapper.Counters,
snapshot.Counters.NumBlocksSigned,
snapshot.Counters.NumBlocksToSign
[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
signed, toSign :=
new(big.Int).Sub(statsNow.NumBlocksSigned, snapSigned),
new(big.Int).Sub(statsNow.NumBlocksToSign, snapToSign)
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
computed := staking.NewComputed(
signed, toSign, 0, numeric.ZeroDec(), true,
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
)
if toSign.Cmp(common.Big0) == 0 {
utils.Logger().Info().
Msg("toSign is 0, perhaps did not receive crosslink proving signing")
return computed
}
if signed.Sign() == -1 {
// Shouldn't happen
utils.Logger().Error().Msg("negative number of signed blocks")
}
[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
if toSign.Sign() == -1 {
// Shouldn't happen
utils.Logger().Error().Msg("negative number of blocks to sign")
}
[double-sign] Provide proof of double sign in slash record sent to beaconchain (#2253) * [double-sign] Commit changes in consensus needed for double-sign * [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain * [slash] Add quick iteration tool for testing double-signing * [slash] Add webhook example * [slash] Add http server for hook to trigger double sign behavior * [double-sign] Use bin/trigger-double-sign to cause a double-sign * [double-sign] Full feedback loop working * [slash] Thread through the slash records in the block proposal step * [slash] Compute the slashing rate * [double-sign] Generalize yaml malicious for many keys * [double-sign][slash] Modify data structures, verify via webhook handler * [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures * [slash] Apply to state slashing for double signing * [slash][double-sign] Checkpoint for working code that slashes on beaconchain * [slash] Keep track of the total slash and total reporters reward * [slash] Dump account state before and after the slash * [slash] Satisfy Travis * [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates * [slash] Capture the unique new delegations since snapshot as well * [slash] Filter undelegation by epoch of double sign * [slash] Add TODO of correctness needed in slash needs on off-chain data * [rpc] Fix closure issue on shardID * [slash] Add delegator to double-sign testing script * [slash] Expand crt-validator.sh with commenting printfs and make delegation * [slash] Finish track payment of leftover slash debt after undelegation runs out * [slash] Now be explicit about error wrt delegatorSlashApply * [slash] Capture specific sanity check on slash paidoff * [slash] Track slash from undelegation piecemeal * [slash][delegation] Named slice types, .String() * [slash] Do no RLP encode twice, once is enough * [slash] Remove special case of validators own delegation * [slash] Refactor approach to slash state application * [slash] Begin expanding out Verify * [slash] Slash on snapshot delegations, not current * [slash] Fix Epoch Cmp * [slash] Third iteration on slash logic * [slash] Use full slash amount * [slash] More log, whitespace * [slash] Remove Println, add log * [slash] Remove debug Println * [slash] Add record in unit test * [slash] Build Validator snapshot, current. Fill out slash record * [slash] Need to get RLP dump of a header to use in test * [slash] Factor out double sign test constants * [slash] Factor out common for validator, stub out slash application, finish out deserialization setup * [slash] Factor out data structure creation because of var lexical scoping * [slash] Seem to have pipeline of unit test e2e executing * [slash] Add expected snitch, slash amounts * [slash] Checkpoint * [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing * [config] add double-sign testnet config (#1) Signed-off-by: Leo Chen <leo@harmony.one> * [slash] Commit for as is code & data of current dump.json * [slash] Order of state operation not correct in test, hence bad results, thank you dlv * [slash] Add snapshot state dump * [slash] Pay off slash of validator own delegation correctly * [slash] Pay off slash debt with special case for min-self * [slash] Pass first scenario conclusively * [slash] 2% slash passes unit test for own delegation and external * [slash] Parameterize unit test to easily test .02 vs .80 slash * [slash] Handle own delegation correctly at 80% slash * [slash] Have 80% slash working with external delegator * [slash] Remove debug code from slash * [slash] Adjust Apply signature, test again for 2% slash * [slash] Factor out scenario in testing so can test 2% and 80% at same time * [slash] Correct balance deduction on plan delegation * [slash] Mock out ChainReader for TestVerify * [slash] Small surface area interface, now feedback loop for verify * [slash] Remove development json * [slash] trigger-double-sign consumes yaml * [slash] Remove dead code * [slash][test] Factor ValidatorWrapper into scenario * [slash][test] Add example from local-testing dump - caution might be off * [slash] Factor out mutation of slashDebt * [slash][test] Factor out tests so can easily load test-case from bytes * [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected * [slash] Satisfy Travis * [slash] Begin cleanup of PR * [slash] Apply slash from header to Finalize via state processor * [slash] Productionize code, Println => logs; adjust slash picked in newblock * [slash] Need pointer for rlp.Decode * [slash] ValidatorInformation use full wrapper * Fix median stake * [staking] Adjust MarshalJSON for Validator, Wrapper * Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279) * Refactor offchain data; Add epoch to ValidatorSnapshot * Make block onchain/offchain data commit atomically * [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment * [effective] VC eligible.go * [consensus] Redundant field in printf * [docker] import-ks for a dev account * [slash] Create BLS key for dockerfile and crt-validator.sh * [slash][docker] Easy deployment of double-sign testing * [docker] Have slash work as single docker command * [rpc] Fix median-stake RPC * [slash] Update webhook with default docker BLS key * [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go * [slash] Remove helper binary, commented out code, change to local config * [params] Factor out test genesis value * Add shard checking to Tx-Pool & correct blacklist (#2301) * [core] Fix blacklist & add shardID check * [staking + node + cmd] Fix blacklist & add shardID check * [slash] Adjust to PR comments part 1 * [docker] Use different throw away funded account * [docker] Create easier testing for delegation with private keys * [docker] Update yaml * [slash] Remove special case for slashing validator own delegation wrt min-self-delegate * [docker] Install nano as well * [slash] Early error if banned * [quorum] Expose earning account in decider marshal json * Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)" This reverts commit 9ffbf682c075b49188923c65a0bbf39ac188be00. * [slash] Add non-sanity check way to update validator * [reward] Increase percision on percentage in schedule * [slash] Adjust logs * [committee] Check eligibility of validator before doing sanity check * [slash] Update docker * [slash] Move create validator script to test * [slash] More log * [param] Make things faster * [slash][off-chain] Clear out slashes from pending in writeblockwithstate * [cross-link] Log is not error, just info * [blockchain] Not necessary to guard DeletePendingSlashingCandidates * [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that * [slash][test] Use faucet as sender, assume user imported * [slash] Test setup * [slash] reserve error for real error in logs * [slash][availability] Apply availability correct, bump signing count each block * [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed * [slash] Pay as much as can * [slash] use right nowAmt * [slash] Take away from rewards as well * [slash] iterate faster * [slash] Remove dev based timing * [slash] Add more log, sanity check incoming slash records, only count external for slash rate * [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters * [availability] More log * [slash] Simply pre slash erra slashing * [slash] Remove development code * [slash] Use height from recvMsg, todo on epoch * [staking] Not necessary to touch LastEpochInCommittee in staking_verifier * [slash] Undo ds in endpoint pattern config * [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment * [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count Co-authored-by: Leo Chen <leo@harmony.one> Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com> Co-authored-by: Rongjian Lan <rongjian@harmony.one> Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
5 years ago
s1, s2 := numeric.NewDecFromBigInt(signed), numeric.NewDecFromBigInt(toSign)
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
computed.Percentage = s1.Quo(s2)
computed.IsBelowThreshold = IsBelowSigningThreshold(computed.Percentage)
return computed
}
// IsBelowSigningThreshold ..
func IsBelowSigningThreshold(quotient numeric.Dec) bool {
return quotient.LTE(measure)
}
// ComputeAndMutateEPOSStatus sets the validator to
// inactive and thereby keeping it out of
// consideration in the pool of validators for
// whenever committee selection happens in future, the
// signing threshold is 66%
func ComputeAndMutateEPOSStatus(
bc Reader,
state *state.DB,
addr common.Address,
) error {
utils.Logger().Info().Msg("begin compute for availability")
wrapper, err := state.ValidatorWrapper(addr)
if err != nil {
return err
}
if wrapper.Status == effective.Banned {
utils.Logger().Debug().Msg("Can't update EPoS status on a banned validator")
return nil
}
snapshot, err := bc.ReadValidatorSnapshot(wrapper.Address)
if err != nil {
return err
}
computed := ComputeCurrentSigning(snapshot, wrapper)
utils.Logger().
Info().Msg("check if signing percent is meeting required threshold")
const missedTooManyBlocks = true
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
switch computed.IsBelowThreshold {
case missedTooManyBlocks:
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
wrapper.Status = effective.Inactive
utils.Logger().Info().
Str("threshold", measure.String()).
Msg("validator failed availability threshold, set to inactive")
default:
// Default is no-op so validator who wants
// to leave the committee can actually leave.
}
if err := state.UpdateValidatorWrapper(
addr, wrapper,
); err != nil {
return err
[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
}
[double-sign] Provide proof of double sign in slash record sent to beaconchain (#2253) * [double-sign] Commit changes in consensus needed for double-sign * [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain * [slash] Add quick iteration tool for testing double-signing * [slash] Add webhook example * [slash] Add http server for hook to trigger double sign behavior * [double-sign] Use bin/trigger-double-sign to cause a double-sign * [double-sign] Full feedback loop working * [slash] Thread through the slash records in the block proposal step * [slash] Compute the slashing rate * [double-sign] Generalize yaml malicious for many keys * [double-sign][slash] Modify data structures, verify via webhook handler * [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures * [slash] Apply to state slashing for double signing * [slash][double-sign] Checkpoint for working code that slashes on beaconchain * [slash] Keep track of the total slash and total reporters reward * [slash] Dump account state before and after the slash * [slash] Satisfy Travis * [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates * [slash] Capture the unique new delegations since snapshot as well * [slash] Filter undelegation by epoch of double sign * [slash] Add TODO of correctness needed in slash needs on off-chain data * [rpc] Fix closure issue on shardID * [slash] Add delegator to double-sign testing script * [slash] Expand crt-validator.sh with commenting printfs and make delegation * [slash] Finish track payment of leftover slash debt after undelegation runs out * [slash] Now be explicit about error wrt delegatorSlashApply * [slash] Capture specific sanity check on slash paidoff * [slash] Track slash from undelegation piecemeal * [slash][delegation] Named slice types, .String() * [slash] Do no RLP encode twice, once is enough * [slash] Remove special case of validators own delegation * [slash] Refactor approach to slash state application * [slash] Begin expanding out Verify * [slash] Slash on snapshot delegations, not current * [slash] Fix Epoch Cmp * [slash] Third iteration on slash logic * [slash] Use full slash amount * [slash] More log, whitespace * [slash] Remove Println, add log * [slash] Remove debug Println * [slash] Add record in unit test * [slash] Build Validator snapshot, current. Fill out slash record * [slash] Need to get RLP dump of a header to use in test * [slash] Factor out double sign test constants * [slash] Factor out common for validator, stub out slash application, finish out deserialization setup * [slash] Factor out data structure creation because of var lexical scoping * [slash] Seem to have pipeline of unit test e2e executing * [slash] Add expected snitch, slash amounts * [slash] Checkpoint * [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing * [config] add double-sign testnet config (#1) Signed-off-by: Leo Chen <leo@harmony.one> * [slash] Commit for as is code & data of current dump.json * [slash] Order of state operation not correct in test, hence bad results, thank you dlv * [slash] Add snapshot state dump * [slash] Pay off slash of validator own delegation correctly * [slash] Pay off slash debt with special case for min-self * [slash] Pass first scenario conclusively * [slash] 2% slash passes unit test for own delegation and external * [slash] Parameterize unit test to easily test .02 vs .80 slash * [slash] Handle own delegation correctly at 80% slash * [slash] Have 80% slash working with external delegator * [slash] Remove debug code from slash * [slash] Adjust Apply signature, test again for 2% slash * [slash] Factor out scenario in testing so can test 2% and 80% at same time * [slash] Correct balance deduction on plan delegation * [slash] Mock out ChainReader for TestVerify * [slash] Small surface area interface, now feedback loop for verify * [slash] Remove development json * [slash] trigger-double-sign consumes yaml * [slash] Remove dead code * [slash][test] Factor ValidatorWrapper into scenario * [slash][test] Add example from local-testing dump - caution might be off * [slash] Factor out mutation of slashDebt * [slash][test] Factor out tests so can easily load test-case from bytes * [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected * [slash] Satisfy Travis * [slash] Begin cleanup of PR * [slash] Apply slash from header to Finalize via state processor * [slash] Productionize code, Println => logs; adjust slash picked in newblock * [slash] Need pointer for rlp.Decode * [slash] ValidatorInformation use full wrapper * Fix median stake * [staking] Adjust MarshalJSON for Validator, Wrapper * Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279) * Refactor offchain data; Add epoch to ValidatorSnapshot * Make block onchain/offchain data commit atomically * [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment * [effective] VC eligible.go * [consensus] Redundant field in printf * [docker] import-ks for a dev account * [slash] Create BLS key for dockerfile and crt-validator.sh * [slash][docker] Easy deployment of double-sign testing * [docker] Have slash work as single docker command * [rpc] Fix median-stake RPC * [slash] Update webhook with default docker BLS key * [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go * [slash] Remove helper binary, commented out code, change to local config * [params] Factor out test genesis value * Add shard checking to Tx-Pool & correct blacklist (#2301) * [core] Fix blacklist & add shardID check * [staking + node + cmd] Fix blacklist & add shardID check * [slash] Adjust to PR comments part 1 * [docker] Use different throw away funded account * [docker] Create easier testing for delegation with private keys * [docker] Update yaml * [slash] Remove special case for slashing validator own delegation wrt min-self-delegate * [docker] Install nano as well * [slash] Early error if banned * [quorum] Expose earning account in decider marshal json * Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)" This reverts commit 9ffbf682c075b49188923c65a0bbf39ac188be00. * [slash] Add non-sanity check way to update validator * [reward] Increase percision on percentage in schedule * [slash] Adjust logs * [committee] Check eligibility of validator before doing sanity check * [slash] Update docker * [slash] Move create validator script to test * [slash] More log * [param] Make things faster * [slash][off-chain] Clear out slashes from pending in writeblockwithstate * [cross-link] Log is not error, just info * [blockchain] Not necessary to guard DeletePendingSlashingCandidates * [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that * [slash][test] Use faucet as sender, assume user imported * [slash] Test setup * [slash] reserve error for real error in logs * [slash][availability] Apply availability correct, bump signing count each block * [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed * [slash] Pay as much as can * [slash] use right nowAmt * [slash] Take away from rewards as well * [slash] iterate faster * [slash] Remove dev based timing * [slash] Add more log, sanity check incoming slash records, only count external for slash rate * [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters * [availability] More log * [slash] Simply pre slash erra slashing * [slash] Remove development code * [slash] Use height from recvMsg, todo on epoch * [staking] Not necessary to touch LastEpochInCommittee in staking_verifier * [slash] Undo ds in endpoint pattern config * [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment * [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count Co-authored-by: Leo Chen <leo@harmony.one> Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com> Co-authored-by: Rongjian Lan <rongjian@harmony.one> Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
5 years ago
[availability] Implement inactive toggle for validators that miss threshold of signing required; (66%) of epoch (#2077) * [availability] Add function setting Validator as Inactive=true if meets threshold * [availability] Set Validators that did not meet signing threshold to inactive * [availability] Wrap Setting invalid validator only if new epoch forthcoming * [availability] Return right error value * [staking] Add Active field to EditValidator staking txn * [availability] Add validator snapshot type, thread throughout codebase * [availability] Adjust check availability on a per epoch basis * [availability] Address PR comments, simplify collection of validators * [availability] Fold ValidatorSnapshot into ValidatorWrapper * [blockchain] Move update of validator list to after availability removal of validator * [availability] Move availability signing counts to Wrapper, out of Stats * [availability] Record epoch on each validator update as well * [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written * [availability] Mutate state for validators signing in finalize * [availability] Set unavailable validators in finalize * [consensus] Remove error level for non-error log * [node] No point to broadcast crosslink if we are not in cross link time yet * [availability] Remove moved blocksigners function * [core] Give more context in failure * [availability] Provide set as filter for which validators to track on signing increase and set inactivity * [blockchain] Write snapshot of validator as is * Fix format in staking transaction (#2127) * [availability] Move increment of validator signing counter to before shard state proposal * [availability] Kick out inactive validators right before new shard state proposal * [availability] Keep logic of getting shard members as was * [state-transition] Attach Epoch number to create validator txn Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
5 years ago
return nil
}