refactoring to multibls package to avoid cyclic dependency and better structure
fixing import issue
fixing more imports
fixing a bug that causes nil pointer for explorer node
fixing a bug in the super majority vote
changing config to have same epoch for cross link and staking
fixing the compilation error
addressing PR comments
leaving todo for assuming order between pub/pri keys pairs
fixing goimports
removing unwanted warning message
* [availability] More log, remove error of div by zero
* [availability] Apply increment of count in accumulate since need crosslink
* [availability] More log
* [staking] VC easy flow of create validator and delegation, restart watchdog
* [availability] Use method instead of inline, fix Cmp mistake in measure
* [availability] Add explicit log for measure
* [availability] Begin unit test impl
* [availability] Factor out subset of methods that availability needs, expand test
* [availability] Do not increment count when LastEpochInCommittee is 0
* [availability] Remove dev scripts
* [availability] Adjust logs
* [availability] Remove test BLS keys
* [availability] Remove check on LastEpoch in bumpCount, remove spurious log
* [availability] Further simplify, current shard state is the elected validator
* [consensus] Remove log
* [project] Remove noisy, hard to know where happened log; whitespace on style
* [double-sign] Commit changes in consensus needed for double-sign
* [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain
* [slash] Add quick iteration tool for testing double-signing
* [slash] Add webhook example
* [slash] Add http server for hook to trigger double sign behavior
* [double-sign] Use bin/trigger-double-sign to cause a double-sign
* [double-sign] Full feedback loop working
* [slash] Thread through the slash records in the block proposal step
* [slash] Compute the slashing rate
* [double-sign] Generalize yaml malicious for many keys
* [double-sign][slash] Modify data structures, verify via webhook handler
* [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures
* [slash] Apply to state slashing for double signing
* [slash][double-sign] Checkpoint for working code that slashes on beaconchain
* [slash] Keep track of the total slash and total reporters reward
* [slash] Dump account state before and after the slash
* [slash] Satisfy Travis
* [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates
* [slash] Capture the unique new delegations since snapshot as well
* [slash] Filter undelegation by epoch of double sign
* [slash] Add TODO of correctness needed in slash needs on off-chain data
* [rpc] Fix closure issue on shardID
* [slash] Add delegator to double-sign testing script
* [slash] Expand crt-validator.sh with commenting printfs and make delegation
* [slash] Finish track payment of leftover slash debt after undelegation runs out
* [slash] Now be explicit about error wrt delegatorSlashApply
* [slash] Capture specific sanity check on slash paidoff
* [slash] Track slash from undelegation piecemeal
* [slash][delegation] Named slice types, .String()
* [slash] Do no RLP encode twice, once is enough
* [slash] Remove special case of validators own delegation
* [slash] Refactor approach to slash state application
* [slash] Begin expanding out Verify
* [slash] Slash on snapshot delegations, not current
* [slash] Fix Epoch Cmp
* [slash] Third iteration on slash logic
* [slash] Use full slash amount
* [slash] More log, whitespace
* [slash] Remove Println, add log
* [slash] Remove debug Println
* [slash] Add record in unit test
* [slash] Build Validator snapshot, current. Fill out slash record
* [slash] Need to get RLP dump of a header to use in test
* [slash] Factor out double sign test constants
* [slash] Factor out common for validator, stub out slash application, finish out deserialization setup
* [slash] Factor out data structure creation because of var lexical scoping
* [slash] Seem to have pipeline of unit test e2e executing
* [slash] Add expected snitch, slash amounts
* [slash] Checkpoint
* [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing
* [config] add double-sign testnet config (#1)
Signed-off-by: Leo Chen <leo@harmony.one>
* [slash] Commit for as is code & data of current dump.json
* [slash] Order of state operation not correct in test, hence bad results, thank you dlv
* [slash] Add snapshot state dump
* [slash] Pay off slash of validator own delegation correctly
* [slash] Pay off slash debt with special case for min-self
* [slash] Pass first scenario conclusively
* [slash] 2% slash passes unit test for own delegation and external
* [slash] Parameterize unit test to easily test .02 vs .80 slash
* [slash] Handle own delegation correctly at 80% slash
* [slash] Have 80% slash working with external delegator
* [slash] Remove debug code from slash
* [slash] Adjust Apply signature, test again for 2% slash
* [slash] Factor out scenario in testing so can test 2% and 80% at same time
* [slash] Correct balance deduction on plan delegation
* [slash] Mock out ChainReader for TestVerify
* [slash] Small surface area interface, now feedback loop for verify
* [slash] Remove development json
* [slash] trigger-double-sign consumes yaml
* [slash] Remove dead code
* [slash][test] Factor ValidatorWrapper into scenario
* [slash][test] Add example from local-testing dump - caution might be off
* [slash] Factor out mutation of slashDebt
* [slash][test] Factor out tests so can easily load test-case from bytes
* [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected
* [slash] Satisfy Travis
* [slash] Begin cleanup of PR
* [slash] Apply slash from header to Finalize via state processor
* [slash] Productionize code, Println => logs; adjust slash picked in newblock
* [slash] Need pointer for rlp.Decode
* [slash] ValidatorInformation use full wrapper
* Fix median stake
* [staking] Adjust MarshalJSON for Validator, Wrapper
* Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)
* Refactor offchain data; Add epoch to ValidatorSnapshot
* Make block onchain/offchain data commit atomically
* [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment
* [effective] VC eligible.go
* [consensus] Redundant field in printf
* [docker] import-ks for a dev account
* [slash] Create BLS key for dockerfile and crt-validator.sh
* [slash][docker] Easy deployment of double-sign testing
* [docker] Have slash work as single docker command
* [rpc] Fix median-stake RPC
* [slash] Update webhook with default docker BLS key
* [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go
* [slash] Remove helper binary, commented out code, change to local config
* [params] Factor out test genesis value
* Add shard checking to Tx-Pool & correct blacklist (#2301)
* [core] Fix blacklist & add shardID check
* [staking + node + cmd] Fix blacklist & add shardID check
* [slash] Adjust to PR comments part 1
* [docker] Use different throw away funded account
* [docker] Create easier testing for delegation with private keys
* [docker] Update yaml
* [slash] Remove special case for slashing validator own delegation wrt min-self-delegate
* [docker] Install nano as well
* [slash] Early error if banned
* [quorum] Expose earning account in decider marshal json
* Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)"
This reverts commit 9ffbf682c0.
* [slash] Add non-sanity check way to update validator
* [reward] Increase percision on percentage in schedule
* [slash] Adjust logs
* [committee] Check eligibility of validator before doing sanity check
* [slash] Update docker
* [slash] Move create validator script to test
* [slash] More log
* [param] Make things faster
* [slash][off-chain] Clear out slashes from pending in writeblockwithstate
* [cross-link] Log is not error, just info
* [blockchain] Not necessary to guard DeletePendingSlashingCandidates
* [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that
* [slash][test] Use faucet as sender, assume user imported
* [slash] Test setup
* [slash] reserve error for real error in logs
* [slash][availability] Apply availability correct, bump signing count each block
* [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed
* [slash] Pay as much as can
* [slash] use right nowAmt
* [slash] Take away from rewards as well
* [slash] iterate faster
* [slash] Remove dev based timing
* [slash] Add more log, sanity check incoming slash records, only count external for slash rate
* [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters
* [availability] More log
* [slash] Simply pre slash erra slashing
* [slash] Remove development code
* [slash] Use height from recvMsg, todo on epoch
* [staking] Not necessary to touch LastEpochInCommittee in staking_verifier
* [slash] Undo ds in endpoint pattern config
* [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment
* [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count
Co-authored-by: Leo Chen <leo@harmony.one>
Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
Co-authored-by: Rongjian Lan <rongjian@harmony.one>
Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
* [consensus] Remove unnecessary usage of bandwidth in network messages, small adjustments after consensus code review
* [consensus] Factor out behavior of when quorum prepare reached
* [consensus] Push code from another scratch branch
* [consensus] Still do need to accept higher than blocknum message
* [slash] Remove dead interface, associated piping
* [slash] Expand out structs
* [consensus] Write to a chan when find a case of double-signing, remove dead code
* [slash] Broadcast the noticing of a double signing
* [rawdb] CRUD for slashing candidates
* [slashing][node][proto] Broadcast the slash record after receive from consensus, handle received proto message, persist in off-chain db while pending
* [slash][node][propose-block] Add verified slashes proposed into the header in block proposal
* [slash][shard] Factor out external validator as method on shard state, add double-signature field
* [slash][engine] Apply slash, name boolean expression for sorts, use stable sort
* [slash] Abstract Ballot results so keep track of both pre and post double sign event
* [slash] Fix type errors on test code
* [slash] Read from correct rawdb
* [slash] Add epoch based guards in CRUD of slashing
* [slash] Write to correct cache for slashing candidates
* [shard] Use explicit named type of BLS Signature, use convention
* [slash] Fix mistake done in refactor, improper header used. Factor out fromSlice to set
* [slash][node] Restore newblock to master, try again minimial change
* [cx-receipts] Break up one-liner, use SliceStable, not Slice
* [network] Finish refactor that makes network message headers once
* [network] Simplify creation further of headers write
* [slash] Adjust data structure of slash after offline discussion with RJ, Chao
* [slash] Still did need signature of the double signature
* [consensus] Prepare message does not have block header
* [consensus] Soft reset three files to 968517d~1
* [consensus] Begin factor consensus network intended message out with prepare first
* [consensus] Factor out Prepared message
* [consensus] Factor out announce message creation
* [consensus] Committed Message, branch on verify sender key for clearer log
* [consensus] Committed Message Factor out
* [consensus] Do jenkins MVP of signatures adjustment
* [main][slash] Provide YAML config as webhook config for double sign event
* [consensus] Adjust signatures, whitespace, lessen GC pressure
* [consensus] Remove dead code
* [consensus] Factor out commit overloaded message, give commit payload override in construct
* [consensus] Fix travis tests
* [consensus] Provide block bytes in SubmitVote(quorum.Commit)
* [consensus] Factor out noisy sanity checks in BFT, move existing commit check earlier as was before
* [quorum] Adjust signatures in quorum
* [staking] Adjust after merge from master
* [consensus] Finish refactor of consensus
* [node] Fix import
* [consensus] Fix travis
* [consensus] Use origin/master copy of block, fix mistake of pointer to empty byte
* [consensus] Less verbose bools
* [consensus] Remove unused trailing mutation hook in message construct
* [consensus] Address some TODOs on err, comment out double sign
* [slashing] Add Slashing fields in header, only available from v3 onward
* [slashing] Add Banned field to validator struct
* [consensus] Remove dead code
* [node] Use named value for shardID of beaconchain
* [consensus] No need to memory thrash Buffer
* [slashing] Add slash record type, stub out BroadcastSlash
* [cmd] Bump 2019 -> 2020
* [votepower] Add ballot results structs
* [quorum][slash] Refactor quorum to accomdate extra data points needed for slashing
* [consensus] Begin refactor of consensus leader messages
* [consensus] Accomdate consensus for changed method signatures in quorum
* [project] Whitespace and helper function
* [block] More comments and logs for fields that do not make sense in earlier v version
* [slashing] More comments, field renames, tag Edgar TODO
* [quorum] Undo name change of Prepare to Announce, was mislead by other existing code
* [availability] Add function setting Validator as Inactive=true if meets threshold
* [availability] Set Validators that did not meet signing threshold to inactive
* [availability] Wrap Setting invalid validator only if new epoch forthcoming
* [availability] Return right error value
* [staking] Add Active field to EditValidator staking txn
* [availability] Add validator snapshot type, thread throughout codebase
* [availability] Adjust check availability on a per epoch basis
* [availability] Address PR comments, simplify collection of validators
* [availability] Fold ValidatorSnapshot into ValidatorWrapper
* [blockchain] Move update of validator list to after availability removal of validator
* [availability] Move availability signing counts to Wrapper, out of Stats
* [availability] Record epoch on each validator update as well
* [availability] Remove update validator stats in writeblockwithstate, update validator signing in proposal of new block to get correct state written
* [availability] Mutate state for validators signing in finalize
* [availability] Set unavailable validators in finalize
* [consensus] Remove error level for non-error log
* [node] No point to broadcast crosslink if we are not in cross link time yet
* [availability] Remove moved blocksigners function
* [core] Give more context in failure
* [availability] Provide set as filter for which validators to track on signing increase and set inactivity
* [blockchain] Write snapshot of validator as is
* Fix format in staking transaction (#2127)
* [availability] Move increment of validator signing counter to before shard state proposal
* [availability] Kick out inactive validators right before new shard state proposal
* [availability] Keep logic of getting shard members as was
* [state-transition] Attach Epoch number to create validator txn
Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
* Add debug info for consensus main loop
* Fix potential OOB issue on p2p message parsing
* Add msg size check to prevent OOB in msg handler
* Add constant for p2p msg prefix
* Generate Decider for base case of 50/50 nodes
* Add Quorum Achieved tests for all Staker nodes & all nodes signed
* Add QuorumAchieved tests for all Harmony nodes signed
* Add RewardThreshold tests for all above cases
* [votepower] Allow negative diff, explicit check sum to one
* [votepower] Handle error value from compute in test
* [votepower] Change log as suggested in PR comment
* [votepower] Check sum of voting power is 1 only when have at least one staked validator
* [reward] Add reward Schedule table from spreadsheet
* [reward][schedule] Make denom a function of time block rather than fixed 12.6 billion
* [reward] Fix latent mistaken because of .UnixNano usage, header uses seconds
* [unit-testing] Add test for when there are 33 Harmony nodes
* Add test for Policy
* Add test for QuorumThreshold
* [quroum] Give diff to last staked voter, make sure sum to one
* [quorum] Fix indent for travis
* [reward] Add Accumulator of rewards in DB
* [reward] Update Accumulator after checking all block rewards to state
* [reward] Address PR comments #1
* [reward] Make fresh base 18 block reward always
* [reward] Remove dead method on fakeChainReader
* [committee] Adjust logging of prev,next committee on each usage of updateconsensusinformation
* [reward] Set block accumulator only once to 0 when flip to staking epoch
* [reward] Separate out condition to write block reward to 0 once past staking epoch (fix mistake)
* [reward] lint
* [reward] Initial write of reward accumulator needs lower-level first usage
* [reward] Give block reward stake adjustment in right denomination
* [quorum] Add interface method to check if I am in committee, implement in both concrete types
* [quorum] Only handle message when I am in committee
* [quorum][consensus] Let non-member still get onCommitted messages
* [unit-testing] Add unit test for effective stake calculation
* [unit-testing] Intermediate commit
* [unit-testing] Another intermediate commit
* [unit-testing] Add unit test for Roster compute
* Remove unnecessary code
* [staking][reward] Give out block reward for cross links
* [staking][reward] Wrap reward logic based on epoch
* [votepower] Factor out votepower b/c need to compute elsewhere as well
* [reward][votepower] Award beaconchain committee members their percent due
* [reward] Only reward non-harmony slots
* [reward] Factor out only signers by commit bitmap
* [reward][votepower] Fix leftover merge based naming changes, fix nil pointer map
* [reward] Handle cross link payment succicently
* [reward] Remove unnecessary helper
* [reward] Handle legacy case of one-node-one-vote block reward (pre-staking)
* [reward] Incorrect log of reward
* [reward] Adjust back to big.Int for legacy rewarding
* [reward] Add +1 for i nonce
* [reward] Abandon formula for nonce, sizes not reliable, easier to do explicit bucketing
* [reward] Remove Prints, fix mistake on using wrong header
* [quorum] Adjust log print out for IsQuorumAchieved
* [reward] Redundant check in code path that only runs in >= staking-epoch
* [reward] Fix mistake on staked block undoing effect of voting power-sharing
Edgar Aroutiounian
Ganesha Upadhyaya
flicker-harmony
Rongjian Lan
chaosma
janet-harmony