The core protocol of WoopChain
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
woop/core/blockchain.go

347 lines
17 KiB

package core
import (
3 years ago
"github.com/harmony-one/harmony/internal/tikv/redis_helper"
"math/big"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/ethdb"
"github.com/ethereum/go-ethereum/event"
"github.com/harmony-one/harmony/block"
"github.com/harmony-one/harmony/consensus/engine"
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
"github.com/harmony-one/harmony/consensus/reward"
"github.com/harmony-one/harmony/core/rawdb"
"github.com/harmony-one/harmony/core/state"
"github.com/harmony-one/harmony/core/types"
"github.com/harmony-one/harmony/core/vm"
"github.com/harmony-one/harmony/internal/params"
"github.com/harmony-one/harmony/shard"
[slash][consensus] Notice double sign & broadcast, factor out tech debt of consensus (#2152) * [slash] Remove dead interface, associated piping * [slash] Expand out structs * [consensus] Write to a chan when find a case of double-signing, remove dead code * [slash] Broadcast the noticing of a double signing * [rawdb] CRUD for slashing candidates * [slashing][node][proto] Broadcast the slash record after receive from consensus, handle received proto message, persist in off-chain db while pending * [slash][node][propose-block] Add verified slashes proposed into the header in block proposal * [slash][shard] Factor out external validator as method on shard state, add double-signature field * [slash][engine] Apply slash, name boolean expression for sorts, use stable sort * [slash] Abstract Ballot results so keep track of both pre and post double sign event * [slash] Fix type errors on test code * [slash] Read from correct rawdb * [slash] Add epoch based guards in CRUD of slashing * [slash] Write to correct cache for slashing candidates * [shard] Use explicit named type of BLS Signature, use convention * [slash] Fix mistake done in refactor, improper header used. Factor out fromSlice to set * [slash][node] Restore newblock to master, try again minimial change * [cx-receipts] Break up one-liner, use SliceStable, not Slice * [network] Finish refactor that makes network message headers once * [network] Simplify creation further of headers write * [slash] Adjust data structure of slash after offline discussion with RJ, Chao * [slash] Still did need signature of the double signature * [consensus] Prepare message does not have block header * [consensus] Soft reset three files to 968517d~1 * [consensus] Begin factor consensus network intended message out with prepare first * [consensus] Factor out Prepared message * [consensus] Factor out announce message creation * [consensus] Committed Message, branch on verify sender key for clearer log * [consensus] Committed Message Factor out * [consensus] Do jenkins MVP of signatures adjustment * [main][slash] Provide YAML config as webhook config for double sign event * [consensus] Adjust signatures, whitespace, lessen GC pressure * [consensus] Remove dead code * [consensus] Factor out commit overloaded message, give commit payload override in construct * [consensus] Fix travis tests * [consensus] Provide block bytes in SubmitVote(quorum.Commit) * [consensus] Factor out noisy sanity checks in BFT, move existing commit check earlier as was before * [quorum] Adjust signatures in quorum * [staking] Adjust after merge from master * [consensus] Finish refactor of consensus * [node] Fix import * [consensus] Fix travis * [consensus] Use origin/master copy of block, fix mistake of pointer to empty byte * [consensus] Less verbose bools * [consensus] Remove unused trailing mutation hook in message construct * [consensus] Address some TODOs on err, comment out double sign
5 years ago
"github.com/harmony-one/harmony/staking/slash"
types2 "github.com/harmony-one/harmony/staking/types"
)
Epoch syncing - syncing only epoch blocks. (#4070) * Epoch syncing - syncing only epoch blocks. * Fix go test, added command to run. * Block signature validation. * Support getting block by height. * Fix to compile. * Http method. * Generate fix. * Install protoc-gen-go-grpc. * Fix sync, additional timeouts, fix epoch. * Chain behaviour options, optional state validation on blockchain initialization. * Added limit to avoid abuse, small fixes. * Fix duplicate code. * Fix isInSyn. * Fix method call. * Removed additional block sending. * Fix in sync. * Code clean up. * Fix beacon chain func. * Return back Beaconchain. * Fix sending only epoch blocks. * Fix failed startup with validator. * Increase recv message size. * Remove peer. * Add logs. * Add logs. * Fix msgf call. * Epoch syncing - syncing only epoch blocks. * Block signature validation. * Fix sync, additional timeouts, fix epoch. * Chain behaviour options, optional state validation on blockchain initialization. * Support getting block by height. * Http method. * Fix duplicate code. * Fix isInSyn. * Added limit to avoid abuse, small fixes. * Fix method call. * Removed additional block sending. * Fix in sync. * Small fix. * Code clean up. * Fix beacon chain func. * Return back Beaconchain. * Fix sending only epoch blocks. * Fix failed startup with validator. * Increase recv message size. * Remove peer. * Add logs. * Fix msgf call. * Sending crosslinks from beacon to non-beacon chains. * Receiving crosslinks. * Broadcast. * Fix comment. * Crosslink signal. * Crosslink hardbeat signal receiving. * Sending crosslinks from beacon to non-beacon chains. * Sending crosslinks from beacon to non-beacon chains. * Broadcast. * Fix comment. * Crosslink signal. * Crosslink signal. * Add comment to run tests. * Fix comments. * Fix comments. * Fix review comments. * Updated with last crosslink heart beat signal. * Merged crosslink signal. * Clean up code. * Add logs for cl debug. * Stop iterate on receiving result. * Add logs. * Fix in sync check. * Fix reconnect problems. * Fix tests and reduce dependency. * Updated with main. * Fix review comments. * Changed to compare epochs. * Fix crosslinks usage. * Fix initialization. * Epoch chain implementation. * Legacy sync by default. * Revert "Legacy sync by default." This reverts commit 5fa6ce002a16699cc3ca75bdd40b068dbe6cefb6. * Epoch sync for downloader. * Build docker image while tests. * Fix paths. * Updated travis. * Review improvements. Co-authored-by: Konstantin <k.potapov@softpro.com>
2 years ago
// Options contains configuration values to change blockchain behaviour.
type Options struct {
// Subset of blockchain suitable for storing last epoch blocks i.e. blocks with shard state.
EpochChain bool
}
// BlockChain represents the canonical chain given a database with a genesis
// block. The Blockchain manages chain imports, reverts, chain reorganisations.
//
Epoch syncing - syncing only epoch blocks. (#4070) * Epoch syncing - syncing only epoch blocks. * Fix go test, added command to run. * Block signature validation. * Support getting block by height. * Fix to compile. * Http method. * Generate fix. * Install protoc-gen-go-grpc. * Fix sync, additional timeouts, fix epoch. * Chain behaviour options, optional state validation on blockchain initialization. * Added limit to avoid abuse, small fixes. * Fix duplicate code. * Fix isInSyn. * Fix method call. * Removed additional block sending. * Fix in sync. * Code clean up. * Fix beacon chain func. * Return back Beaconchain. * Fix sending only epoch blocks. * Fix failed startup with validator. * Increase recv message size. * Remove peer. * Add logs. * Add logs. * Fix msgf call. * Epoch syncing - syncing only epoch blocks. * Block signature validation. * Fix sync, additional timeouts, fix epoch. * Chain behaviour options, optional state validation on blockchain initialization. * Support getting block by height. * Http method. * Fix duplicate code. * Fix isInSyn. * Added limit to avoid abuse, small fixes. * Fix method call. * Removed additional block sending. * Fix in sync. * Small fix. * Code clean up. * Fix beacon chain func. * Return back Beaconchain. * Fix sending only epoch blocks. * Fix failed startup with validator. * Increase recv message size. * Remove peer. * Add logs. * Fix msgf call. * Sending crosslinks from beacon to non-beacon chains. * Receiving crosslinks. * Broadcast. * Fix comment. * Crosslink signal. * Crosslink hardbeat signal receiving. * Sending crosslinks from beacon to non-beacon chains. * Sending crosslinks from beacon to non-beacon chains. * Broadcast. * Fix comment. * Crosslink signal. * Crosslink signal. * Add comment to run tests. * Fix comments. * Fix comments. * Fix review comments. * Updated with last crosslink heart beat signal. * Merged crosslink signal. * Clean up code. * Add logs for cl debug. * Stop iterate on receiving result. * Add logs. * Fix in sync check. * Fix reconnect problems. * Fix tests and reduce dependency. * Updated with main. * Fix review comments. * Changed to compare epochs. * Fix crosslinks usage. * Fix initialization. * Epoch chain implementation. * Legacy sync by default. * Revert "Legacy sync by default." This reverts commit 5fa6ce002a16699cc3ca75bdd40b068dbe6cefb6. * Epoch sync for downloader. * Build docker image while tests. * Fix paths. * Updated travis. * Review improvements. Co-authored-by: Konstantin <k.potapov@softpro.com>
2 years ago
// Importing blocks in to the blockchain happens according to the set of rules
Merge master to t3 0523 (#3085) * fix signers share computation to use only non-harmony nodes (#3008) * Add LastEpochInCommittee at prestaking epoch too (#3014) * write validator stats at the end of prestaking epoch (#3021) * write validator stats at the end of prestaking epoch * also need to write block reward accumulator * [build] enable static build by default (#2885) scripts/go_executable_build.sh -s (static build, default) scripts/go_executable_build.sh -S (non-static build) Signed-off-by: Leo Chen <leo@harmony.one> * Code formatting fix based on `staticcheck` (#3023) * fix some static check issues * more fixes according to staticcheck * add nil check on getHeader * Add more log info * [rpc] Revert change to v2 GetBalance RPC (#3037) * Cherrypick crosslink related fixes from t3 to master. (#3045) * Optimize crosslink verification logic and add more cache (#3032) * make crosslink broadcast smarter and more efficient (#3036) * adjust catch up speed to be a sane number * Fix crosslink broadcast condition (#3041) * Tx pool error report fix & local build fix (#3046) * [tx_list] Add transaction after you can fetch tx cost * [tx_pool] Add error sink reports for removed tx * Correct log levels for known transactions * Remove redundant hash to `enqueue` and `add` signatures * Update tests for signature changes * [build] Fix local build for MacOs * [tx_pool] Correct typo & logging level for known tx * [config] Create Testnet config * [node.sh] update testnet network config * [config] correct testnet config * [main] Enable beacon sync for explorer nodes (#3059) * [apr] should not use snapshot from last epoch, use current epoch snapshot (#3051) * use snapshot total delegation instead of current total delegation * should not use snapshot from last epoch, use current epoch snapshot (#3049) Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com> Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com> * cherrypick log level change * [rpc] Add active status back to Validator information [validator] Fix typo for ValidatorRPCEnhanced [Makefile] Change make exe to build non-static by default * [network] Added caching to node server (#3048) * [network] Added caching to node server * [network] renamed two singleflight.Group * [network] added two error handling * [network] added forget logic for single flight * [network] changed single flight to lru cache solution * [network] adjusted the cache size * [network] changed cache size to 10000 * Transaction api improvements (#3057) * [node] Add option to broadcast invalid tx (on by default) * update main args to take in broadcast invalid tx * update node.sh for new broadcast invalid tx option * [rpc] Do not submit transaction if tx has been finalized * Support legacy behavior and not return errors on tx submission * [main] make broadcasting invalid tx default false * [node] Fix formatting * Add timeouts for the explorer HTTP service * [test] State trans unit test (#3043) * completing create-validator tests generalizing create validator tests, started with edit validator tests one round completed before major refactor * [test] finished coding test TestCheckDuplicateFields * [test] finished debug TestCheckDuplicateFields * [test] Added TestDescription_EnsureLength * [test] added test in validator_test.go TestUpdateDescription * [test] Added test TestComputed_String * [test] refactored TestValidatorSanityCheck and added some message in error message returned from sanity check * [test] removed test index for debugging * [test] Added test TestValidatorWrapper_SanityCheck. Added value check for TestMarshalUnmarshalValidator. Fixed rlp usage defect in UnmarshalValidator * [test] added test TestVerifyBLSKeys * [test] added test TestContainsHarmonyBLSKeys * [test] added test TestCreateValidatorFromNewMsg * [test] fixed some tests errors * [test] added test TestUpdateValidatorFromEditMsg, last editted the test file, finished testing validator_test.go * [staking] added numeric.Dec.Copy method * [staking] added CommissionRates.Copy * [staking] add nil case handle to copy method and related test case * [test] added nil case for commissionRates.Copy test * [staking] finished CreateValidator.Copy and related test case * [staking] added EditValidator.Copy method and related test case * [test] added zero test cases for Copy method * [staking] implemented Delegate.Copy and related unit test case * [staking] added Undelegate.Copy and CollectReward.Copy method. Implemented corresponding test cases * [test] added two more simple unit tests * [test] solving the merge conflict with deep copy fix * [test] added schedule for sharding * [test] refactored double_sign_test.go * [test] comment out all test cases for now. * [test] added record Copy method for testing * [test] added the first test case in TestVerify * [test] finished test TestVerify * [test] Added test TestVerify, TestApplySlashRate, TestSetDifference * [test] half through TestDelegatorSlashApply * [test] fix the problem cause by unsafe usage of big.Int * [staking] added the copy methods to validatorWrapper * [test] added test cases for copy method for staking/validatorWrapper * [test] added test case TestApply * [test] added test case TestRate * [test] fix the golint errors * [test] removed commented out unit test code * [test] remote the empty line in imports * [test] moved copy methods to stake_testing module * [test] removed usage of interface stateDB * [test] removed empty lines in imports * [test] fixed golint package name issues * [test] removed a todo when writing code * [test] moved record.Copy to staking/slash/test * [test] add some changes * [test] added prototypes in staketest module * [test] fix the golint issue in staketest * [test] make prototype more beautiful * [test] refactored test TestCheckDuplicateFields * [test] add createValidator test data * [test] added positive test case for VerifyAndCreateValidatorFromMsg * [test] added create validator test * [test] added the positive test case for TestVerifyAndEditValidatorFromMsg * [test] added the tests case TestVerifyAndEditValidatorFromMsg * [test] fix one of the error is not triggered as expected * [test] more changes in test * [test] fix the positive test data for TestVerifyAndDelegateFromMsg * [test] Fixed two comment golint errors * [test] added delegate tests * [test] added a make function. WIP * [test] added undelegate positive test * [test] added negative test cases for undelegate * [test] added positive test. not passed. Need to add assert method in test suites * [test] added equal function to check Validator Equality * [test] added equals for staketest * [test] replaced deep equal with new equal methods: * [test] removed unused codes * [test] Finishing touch * [test] fix comment golint issue * [test] removed dead code in staking_verifier_test.go Co-authored-by: Ganesha Upadhyaya <ganeshrvce@gmail.com> * changed the sync port to dns port in flag * [network] Limit client sync connection (#3071) * [network] limit client connected peers * Changed signature and added tests * Set secure permissions on generated blskey pass file in node.sh * Fix spellin gtypo in node.sh * fix latest param handling in api v1 * added comment for the isGreaterThanLatestBlockNum check * fix a nil pointer crash due to previous revert Signed-off-by: Leo Chen <leo@harmony.one> * fix nil pointer crash Signed-off-by: Leo Chen <leo@harmony.one> * fix nil pointer of block Signed-off-by: Leo Chen <leo@harmony.one> * [rpc] add viewID and epoch to RPCMarshalBlock (#3009) * [rpc] add viewID and epoch to RPCMarshalBlock * add apt-get update * Do not rollback current block on failed block sync (#3101) Co-authored-by: Ganesha Upadhyaya <ganeshrvce@gmail.com> Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com> Co-authored-by: Jacky Wang <jackyw.se@gmail.com> Co-authored-by: Janet Liang <56005637+janet-harmony@users.noreply.github.com> Co-authored-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu> Co-authored-by: Janet Liang <janet@harmony.one> Co-authored-by: Sebastian Johnsson <sebastian.johnsson@gmail.com> Co-authored-by: hsiung <hsiung@pinterest.com> Co-authored-by: Dennis Won <jhwon0820@gmail.com> Co-authored-by: Yishuang Chen <34232522+ivorytowerdds@users.noreply.github.com>
5 years ago
// defined by the two stage validator. Processing of blocks is done using the
// Processor which processes the included transaction. The validation of the state
Merge master to t3 0523 (#3085) * fix signers share computation to use only non-harmony nodes (#3008) * Add LastEpochInCommittee at prestaking epoch too (#3014) * write validator stats at the end of prestaking epoch (#3021) * write validator stats at the end of prestaking epoch * also need to write block reward accumulator * [build] enable static build by default (#2885) scripts/go_executable_build.sh -s (static build, default) scripts/go_executable_build.sh -S (non-static build) Signed-off-by: Leo Chen <leo@harmony.one> * Code formatting fix based on `staticcheck` (#3023) * fix some static check issues * more fixes according to staticcheck * add nil check on getHeader * Add more log info * [rpc] Revert change to v2 GetBalance RPC (#3037) * Cherrypick crosslink related fixes from t3 to master. (#3045) * Optimize crosslink verification logic and add more cache (#3032) * make crosslink broadcast smarter and more efficient (#3036) * adjust catch up speed to be a sane number * Fix crosslink broadcast condition (#3041) * Tx pool error report fix & local build fix (#3046) * [tx_list] Add transaction after you can fetch tx cost * [tx_pool] Add error sink reports for removed tx * Correct log levels for known transactions * Remove redundant hash to `enqueue` and `add` signatures * Update tests for signature changes * [build] Fix local build for MacOs * [tx_pool] Correct typo & logging level for known tx * [config] Create Testnet config * [node.sh] update testnet network config * [config] correct testnet config * [main] Enable beacon sync for explorer nodes (#3059) * [apr] should not use snapshot from last epoch, use current epoch snapshot (#3051) * use snapshot total delegation instead of current total delegation * should not use snapshot from last epoch, use current epoch snapshot (#3049) Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com> Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com> * cherrypick log level change * [rpc] Add active status back to Validator information [validator] Fix typo for ValidatorRPCEnhanced [Makefile] Change make exe to build non-static by default * [network] Added caching to node server (#3048) * [network] Added caching to node server * [network] renamed two singleflight.Group * [network] added two error handling * [network] added forget logic for single flight * [network] changed single flight to lru cache solution * [network] adjusted the cache size * [network] changed cache size to 10000 * Transaction api improvements (#3057) * [node] Add option to broadcast invalid tx (on by default) * update main args to take in broadcast invalid tx * update node.sh for new broadcast invalid tx option * [rpc] Do not submit transaction if tx has been finalized * Support legacy behavior and not return errors on tx submission * [main] make broadcasting invalid tx default false * [node] Fix formatting * Add timeouts for the explorer HTTP service * [test] State trans unit test (#3043) * completing create-validator tests generalizing create validator tests, started with edit validator tests one round completed before major refactor * [test] finished coding test TestCheckDuplicateFields * [test] finished debug TestCheckDuplicateFields * [test] Added TestDescription_EnsureLength * [test] added test in validator_test.go TestUpdateDescription * [test] Added test TestComputed_String * [test] refactored TestValidatorSanityCheck and added some message in error message returned from sanity check * [test] removed test index for debugging * [test] Added test TestValidatorWrapper_SanityCheck. Added value check for TestMarshalUnmarshalValidator. Fixed rlp usage defect in UnmarshalValidator * [test] added test TestVerifyBLSKeys * [test] added test TestContainsHarmonyBLSKeys * [test] added test TestCreateValidatorFromNewMsg * [test] fixed some tests errors * [test] added test TestUpdateValidatorFromEditMsg, last editted the test file, finished testing validator_test.go * [staking] added numeric.Dec.Copy method * [staking] added CommissionRates.Copy * [staking] add nil case handle to copy method and related test case * [test] added nil case for commissionRates.Copy test * [staking] finished CreateValidator.Copy and related test case * [staking] added EditValidator.Copy method and related test case * [test] added zero test cases for Copy method * [staking] implemented Delegate.Copy and related unit test case * [staking] added Undelegate.Copy and CollectReward.Copy method. Implemented corresponding test cases * [test] added two more simple unit tests * [test] solving the merge conflict with deep copy fix * [test] added schedule for sharding * [test] refactored double_sign_test.go * [test] comment out all test cases for now. * [test] added record Copy method for testing * [test] added the first test case in TestVerify * [test] finished test TestVerify * [test] Added test TestVerify, TestApplySlashRate, TestSetDifference * [test] half through TestDelegatorSlashApply * [test] fix the problem cause by unsafe usage of big.Int * [staking] added the copy methods to validatorWrapper * [test] added test cases for copy method for staking/validatorWrapper * [test] added test case TestApply * [test] added test case TestRate * [test] fix the golint errors * [test] removed commented out unit test code * [test] remote the empty line in imports * [test] moved copy methods to stake_testing module * [test] removed usage of interface stateDB * [test] removed empty lines in imports * [test] fixed golint package name issues * [test] removed a todo when writing code * [test] moved record.Copy to staking/slash/test * [test] add some changes * [test] added prototypes in staketest module * [test] fix the golint issue in staketest * [test] make prototype more beautiful * [test] refactored test TestCheckDuplicateFields * [test] add createValidator test data * [test] added positive test case for VerifyAndCreateValidatorFromMsg * [test] added create validator test * [test] added the positive test case for TestVerifyAndEditValidatorFromMsg * [test] added the tests case TestVerifyAndEditValidatorFromMsg * [test] fix one of the error is not triggered as expected * [test] more changes in test * [test] fix the positive test data for TestVerifyAndDelegateFromMsg * [test] Fixed two comment golint errors * [test] added delegate tests * [test] added a make function. WIP * [test] added undelegate positive test * [test] added negative test cases for undelegate * [test] added positive test. not passed. Need to add assert method in test suites * [test] added equal function to check Validator Equality * [test] added equals for staketest * [test] replaced deep equal with new equal methods: * [test] removed unused codes * [test] Finishing touch * [test] fix comment golint issue * [test] removed dead code in staking_verifier_test.go Co-authored-by: Ganesha Upadhyaya <ganeshrvce@gmail.com> * changed the sync port to dns port in flag * [network] Limit client sync connection (#3071) * [network] limit client connected peers * Changed signature and added tests * Set secure permissions on generated blskey pass file in node.sh * Fix spellin gtypo in node.sh * fix latest param handling in api v1 * added comment for the isGreaterThanLatestBlockNum check * fix a nil pointer crash due to previous revert Signed-off-by: Leo Chen <leo@harmony.one> * fix nil pointer crash Signed-off-by: Leo Chen <leo@harmony.one> * fix nil pointer of block Signed-off-by: Leo Chen <leo@harmony.one> * [rpc] add viewID and epoch to RPCMarshalBlock (#3009) * [rpc] add viewID and epoch to RPCMarshalBlock * add apt-get update * Do not rollback current block on failed block sync (#3101) Co-authored-by: Ganesha Upadhyaya <ganeshrvce@gmail.com> Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com> Co-authored-by: Jacky Wang <jackyw.se@gmail.com> Co-authored-by: Janet Liang <56005637+janet-harmony@users.noreply.github.com> Co-authored-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu> Co-authored-by: Janet Liang <janet@harmony.one> Co-authored-by: Sebastian Johnsson <sebastian.johnsson@gmail.com> Co-authored-by: hsiung <hsiung@pinterest.com> Co-authored-by: Dennis Won <jhwon0820@gmail.com> Co-authored-by: Yishuang Chen <34232522+ivorytowerdds@users.noreply.github.com>
5 years ago
// is done in the second part of the validator. Failing results in aborting of
// the import.
//
// The BlockChain also helps in returning blocks from **any** chain included
// in the database as well as blocks that represents the canonical chain. It's
// important to note that GetBlock can return any block and does not need to be
// included in the canonical one where as GetBlockByNumber always represents the
// canonical chain.
type BlockChain interface {
// ValidateNewBlock validates new block.
ValidateNewBlock(block *types.Block) error
// SetHead rewinds the local chain to a new head. In the case of headers, everything
// above the new head will be deleted and the new one set. In the case of blocks
// though, the head may be further rewound if block bodies are missing (non-archive
// nodes after a fast sync).
SetHead(head uint64) error
// ShardID returns the shard Id of the blockchain.
ShardID() uint32
// CurrentBlock retrieves the current head block of the canonical chain. The
// block is retrieved from the blockchain's internal cache.
CurrentBlock() *types.Block
// Validator returns the current validator.
Validator() Validator
// Processor returns the current processor.
Processor() Processor
// State returns a new mutable state based on the current HEAD block.
State() (*state.DB, error)
// StateAt returns a new mutable state based on a particular point in time.
StateAt(root common.Hash) (*state.DB, error)
// HasBlock checks if a block is fully present in the database or not.
HasBlock(hash common.Hash, number uint64) bool
// HasState checks if state trie is fully present in the database or not.
HasState(hash common.Hash) bool
// HasBlockAndState checks if a block and associated state trie is fully present
// in the database or not, caching it if present.
HasBlockAndState(hash common.Hash, number uint64) bool
// GetBlock retrieves a block from the database by hash and number,
// caching it if found.
GetBlock(hash common.Hash, number uint64) *types.Block
// GetBlockByHash retrieves a block from the database by hash, caching it if found.
GetBlockByHash(hash common.Hash) *types.Block
// GetBlockByNumber retrieves a block from the database by number, caching it
// (associated with its hash) if found.
GetBlockByNumber(number uint64) *types.Block
// GetReceiptsByHash retrieves the receipts for all transactions in a given block.
GetReceiptsByHash(hash common.Hash) types.Receipts
// Stop stops the blockchain service. If any imports are currently in progress
// it will abort them using the procInterrupt.
Stop()
// Rollback is designed to remove a chain of links from the database that aren't
// certain enough to be valid.
Rollback(chain []common.Hash) error
// WriteBlockWithoutState writes only the block and its metadata to the database,
// but does not write any state. This is used to construct competing side forks
// up to the point where they exceed the canonical total difficulty.
WriteBlockWithoutState(block *types.Block, td *big.Int) (err error)
// WriteBlockWithState writes the block and all associated state to the database.
WriteBlockWithState(
block *types.Block, receipts []*types.Receipt,
cxReceipts []*types.CXReceipt,
stakeMsgs []types2.StakeMsg,
paid reward.Reader,
state *state.DB,
) (status WriteStatus, err error)
// GetMaxGarbageCollectedBlockNumber ..
GetMaxGarbageCollectedBlockNumber() int64
// InsertChain attempts to insert the given batch of blocks in to the canonical
// chain or, otherwise, create a fork. If an error is returned it will return
// the index number of the failing block as well an error describing what went
// wrong.
//
// After insertion is done, all accumulated events will be fired.
InsertChain(chain types.Blocks, verifyHeaders bool) (int, error)
// BadBlocks returns a list of the last 'bad blocks' that
// the client has seen on the network.
BadBlocks() []BadBlock
// CurrentHeader retrieves the current head header of the canonical chain. The
// header is retrieved from the HeaderChain's internal cache.
CurrentHeader() *block.Header
// GetHeader retrieves a block header from the database by hash and number,
// caching it if found.
GetHeader(hash common.Hash, number uint64) *block.Header
// GetHeaderByHash retrieves a block header from the database by hash, caching it if
// found.
GetHeaderByHash(hash common.Hash) *block.Header
// GetCanonicalHash returns the canonical hash for a given block number.
GetCanonicalHash(number uint64) common.Hash
// GetHeaderByNumber retrieves a block header from the database by number,
// caching it (associated with its hash) if found.
GetHeaderByNumber(number uint64) *block.Header
// Config retrieves the blockchain's chain configuration.
Config() *params.ChainConfig
// Engine retrieves the blockchain's consensus engine.
Engine() engine.Engine
// SubscribeRemovedLogsEvent registers a subscription of RemovedLogsEvent.
SubscribeRemovedLogsEvent(ch chan<- RemovedLogsEvent) event.Subscription
// SubscribeTraceEvent registers a subscription of ChainEvent.
SubscribeTraceEvent(ch chan<- TraceEvent) event.Subscription
// SubscribeChainEvent registers a subscription of ChainEvent.
SubscribeChainEvent(ch chan<- ChainEvent) event.Subscription
// SubscribeChainHeadEvent registers a subscription of ChainHeadEvent.
SubscribeChainHeadEvent(ch chan<- ChainHeadEvent) event.Subscription
// SubscribeChainSideEvent registers a subscription of ChainSideEvent.
SubscribeChainSideEvent(ch chan<- ChainSideEvent) event.Subscription
// SubscribeLogsEvent registers a subscription of []*types.Log.
SubscribeLogsEvent(ch chan<- []*types.Log) event.Subscription
// ReadShardState retrieves sharding state given the epoch number.
ReadShardState(epoch *big.Int) (*shard.State, error)
// WriteShardStateBytes saves the given sharding state under the given epoch number.
WriteShardStateBytes(db rawdb.DatabaseWriter,
epoch *big.Int, shardState []byte,
) (*shard.State, error)
Epoch syncing - syncing only epoch blocks. (#4070) * Epoch syncing - syncing only epoch blocks. * Fix go test, added command to run. * Block signature validation. * Support getting block by height. * Fix to compile. * Http method. * Generate fix. * Install protoc-gen-go-grpc. * Fix sync, additional timeouts, fix epoch. * Chain behaviour options, optional state validation on blockchain initialization. * Added limit to avoid abuse, small fixes. * Fix duplicate code. * Fix isInSyn. * Fix method call. * Removed additional block sending. * Fix in sync. * Code clean up. * Fix beacon chain func. * Return back Beaconchain. * Fix sending only epoch blocks. * Fix failed startup with validator. * Increase recv message size. * Remove peer. * Add logs. * Add logs. * Fix msgf call. * Epoch syncing - syncing only epoch blocks. * Block signature validation. * Fix sync, additional timeouts, fix epoch. * Chain behaviour options, optional state validation on blockchain initialization. * Support getting block by height. * Http method. * Fix duplicate code. * Fix isInSyn. * Added limit to avoid abuse, small fixes. * Fix method call. * Removed additional block sending. * Fix in sync. * Small fix. * Code clean up. * Fix beacon chain func. * Return back Beaconchain. * Fix sending only epoch blocks. * Fix failed startup with validator. * Increase recv message size. * Remove peer. * Add logs. * Fix msgf call. * Sending crosslinks from beacon to non-beacon chains. * Receiving crosslinks. * Broadcast. * Fix comment. * Crosslink signal. * Crosslink hardbeat signal receiving. * Sending crosslinks from beacon to non-beacon chains. * Sending crosslinks from beacon to non-beacon chains. * Broadcast. * Fix comment. * Crosslink signal. * Crosslink signal. * Add comment to run tests. * Fix comments. * Fix comments. * Fix review comments. * Updated with last crosslink heart beat signal. * Merged crosslink signal. * Clean up code. * Add logs for cl debug. * Stop iterate on receiving result. * Add logs. * Fix in sync check. * Fix reconnect problems. * Fix tests and reduce dependency. * Updated with main. * Fix review comments. * Changed to compare epochs. * Fix crosslinks usage. * Fix initialization. * Epoch chain implementation. * Legacy sync by default. * Revert "Legacy sync by default." This reverts commit 5fa6ce002a16699cc3ca75bdd40b068dbe6cefb6. * Epoch sync for downloader. * Build docker image while tests. * Fix paths. * Updated travis. * Review improvements. Co-authored-by: Konstantin <k.potapov@softpro.com>
2 years ago
// WriteHeadBlock writes head block.
WriteHeadBlock(block *types.Block) error
// ReadCommitSig retrieves the commit signature on a block.
ReadCommitSig(blockNum uint64) ([]byte, error)
// WriteCommitSig saves the commits signatures signed on a block.
WriteCommitSig(blockNum uint64, lastCommits []byte) error
// GetVdfByNumber retrieves the rand seed given the block number, return 0 if not exist.
GetVdfByNumber(number uint64) []byte
// GetVrfByNumber retrieves the randomness preimage given the block number, return 0 if not exist
GetVrfByNumber(number uint64) []byte
// ChainDb returns the database.
ChainDb() ethdb.Database
// GetEpochBlockNumber returns the first block number of the given epoch.
GetEpochBlockNumber(epoch *big.Int) (*big.Int, error)
// StoreEpochBlockNumber stores the given epoch-first block number.
StoreEpochBlockNumber(
epoch *big.Int, blockNum *big.Int,
) error
// ReadEpochVrfBlockNums retrieves block numbers with valid VRF for the specified epoch.
ReadEpochVrfBlockNums(epoch *big.Int) ([]uint64, error)
// WriteEpochVrfBlockNums saves block numbers with valid VRF for the specified epoch.
WriteEpochVrfBlockNums(epoch *big.Int, vrfNumbers []uint64) error
// ReadEpochVdfBlockNum retrieves block number with valid VDF for the specified epoch.
ReadEpochVdfBlockNum(epoch *big.Int) (*big.Int, error)
// WriteEpochVdfBlockNum saves block number with valid VDF for the specified epoch.
WriteEpochVdfBlockNum(epoch *big.Int, blockNum *big.Int) error
// WriteCrossLinks saves the hashes of crosslinks by shardID and blockNum combination key.
WriteCrossLinks(batch rawdb.DatabaseWriter, cls []types.CrossLink) error
// DeleteCrossLinks removes the hashes of crosslinks by shardID and blockNum combination key.
DeleteCrossLinks(cls []types.CrossLink) error
// ReadCrossLink retrieves crosslink given shardID and blockNum.
ReadCrossLink(shardID uint32, blockNum uint64) (*types.CrossLink, error)
// LastContinuousCrossLink saves the last crosslink of a shard
// This function will update the latest crosslink in the sense that
// any previous block's crosslink is received up to this point
// there is no missing hole between genesis to this crosslink of given shardID.
LastContinuousCrossLink(batch rawdb.DatabaseWriter, shardID uint32) error
// ReadShardLastCrossLink retrieves the last crosslink of a shard.
ReadShardLastCrossLink(shardID uint32) (*types.CrossLink, error)
// DeleteFromPendingSlashingCandidates ..
DeleteFromPendingSlashingCandidates(
processed slash.Records,
) error
// ReadPendingSlashingCandidates retrieves pending slashing candidates.
ReadPendingSlashingCandidates() slash.Records
// ReadPendingCrossLinks retrieves pending crosslinks.
ReadPendingCrossLinks() ([]types.CrossLink, error)
// CachePendingCrossLinks caches the pending crosslinks in memory.
CachePendingCrossLinks(crossLinks []types.CrossLink) error
// SavePendingCrossLinks saves the pending crosslinks in db.
SavePendingCrossLinks() error
// AddPendingSlashingCandidates appends pending slashing candidates.
AddPendingSlashingCandidates(
candidates slash.Records,
) error
// AddPendingCrossLinks appends pending crosslinks.
AddPendingCrossLinks(pendingCLs []types.CrossLink) (int, error)
// DeleteFromPendingCrossLinks delete pending crosslinks that already committed (i.e. passed in the params).
DeleteFromPendingCrossLinks(crossLinks []types.CrossLink) (int, error)
// IsSameLeaderAsPreviousBlock retrieves a block from the database by number, caching it.
IsSameLeaderAsPreviousBlock(block *types.Block) bool
// GetVMConfig returns the blockchain VM config.
GetVMConfig() *vm.Config
// ReadCXReceipts retrieves the cross shard transaction receipts of a given shard.
ReadCXReceipts(shardID uint32, blockNum uint64, blockHash common.Hash) (types.CXReceipts, error)
// CXMerkleProof calculates the cross shard transaction merkle proof of a given destination shard.
CXMerkleProof(toShardID uint32, block *types.Block) (*types.CXMerkleProof, error)
// WriteCXReceiptsProofSpent mark the CXReceiptsProof list with given unspent status
WriteCXReceiptsProofSpent(db rawdb.DatabaseWriter, cxps []*types.CXReceiptsProof) error
// IsSpent checks whether a CXReceiptsProof is spent.
IsSpent(cxp *types.CXReceiptsProof) bool
// ReadTxLookupEntry returns where the given transaction resides in the chain,
// as a (block hash, block number, index in transaction list) triple.
// returns 0, 0 if not found.
ReadTxLookupEntry(txID common.Hash) (common.Hash, uint64, uint64)
// ReadValidatorInformationAtRoot reads staking
// information of given validatorWrapper at a specific state root.
ReadValidatorInformationAtRoot(
addr common.Address, root common.Hash,
) (*types2.ValidatorWrapper, error)
// ReadValidatorInformationAtState reads staking
// information of given validatorWrapper at a specific state root.
ReadValidatorInformationAtState(
addr common.Address, state *state.DB,
) (*types2.ValidatorWrapper, error)
// ReadValidatorInformation reads staking information of given validator address.
ReadValidatorInformation(
addr common.Address,
) (*types2.ValidatorWrapper, error)
// ReadValidatorSnapshotAtEpoch reads the snapshot
// staking validator information of given validator address.
ReadValidatorSnapshotAtEpoch(
epoch *big.Int,
addr common.Address,
) (*types2.ValidatorSnapshot, error)
// ReadValidatorSnapshot reads the snapshot staking information of given validator address.
ReadValidatorSnapshot(
addr common.Address,
) (*types2.ValidatorSnapshot, error)
// WriteValidatorSnapshot writes the snapshot of provided validator.
WriteValidatorSnapshot(
batch rawdb.DatabaseWriter, snapshot *types2.ValidatorSnapshot,
) error
// ReadValidatorStats reads the stats of a validator.
ReadValidatorStats(
addr common.Address,
) (*types2.ValidatorStats, error)
// UpdateValidatorVotingPower writes the voting power for the committees.
UpdateValidatorVotingPower(
batch rawdb.DatabaseWriter,
block *types.Block,
newEpochSuperCommittee, currentEpochSuperCommittee *shard.State,
state *state.DB,
) (map[common.Address]*types2.ValidatorStats, error)
// ComputeAndUpdateAPR ...
ComputeAndUpdateAPR(
block *types.Block, now *big.Int,
wrapper *types2.ValidatorWrapper, stats *types2.ValidatorStats,
) error
// UpdateValidatorSnapshots updates the content snapshot of all validators
// Note: this should only be called within the blockchain insert process.
UpdateValidatorSnapshots(
batch rawdb.DatabaseWriter, epoch *big.Int, state *state.DB, newValidators []common.Address,
) error
// ReadValidatorList reads the addresses of current all validators.
ReadValidatorList() ([]common.Address, error)
// WriteValidatorList writes the list of validator addresses to database
// Note: this should only be called within the blockchain insert process.
WriteValidatorList(
db rawdb.DatabaseWriter, addrs []common.Address,
) error
// ReadDelegationsByDelegator reads the addresses of validators delegated by a delegator.
ReadDelegationsByDelegator(
delegator common.Address,
) (m types2.DelegationIndexes, err error)
// ReadDelegationsByDelegatorAt reads the addresses of validators delegated by a delegator at a given block.
ReadDelegationsByDelegatorAt(
delegator common.Address, blockNum *big.Int,
) (m types2.DelegationIndexes, err error)
// UpdateStakingMetaData updates the metadata of validators and delegations,
// including the full validator list and delegation indexes.
// Note: this should only be called within the blockchain insert process.
UpdateStakingMetaData(
batch rawdb.DatabaseWriter, block *types.Block,
stakeMsgs []types2.StakeMsg,
state *state.DB, epoch, newEpoch *big.Int,
) (newValidators []common.Address, err error)
// ReadBlockRewardAccumulator must only be called on beaconchain
// Note that block rewards are only for staking era.
ReadBlockRewardAccumulator(number uint64) (*big.Int, error)
// WriteBlockRewardAccumulator directly writes the BlockRewardAccumulator value
// Note: this should only be called once during staking launch.
WriteBlockRewardAccumulator(
batch rawdb.DatabaseWriter, reward *big.Int, number uint64,
) error
// UpdateBlockRewardAccumulator ..
// Note: this should only be called within the blockchain insert process.
UpdateBlockRewardAccumulator(
batch rawdb.DatabaseWriter, diff *big.Int, number uint64,
) error
// ValidatorCandidates returns the up to date validator candidates for next epoch.
ValidatorCandidates() []common.Address
// DelegatorsInformation returns up to date information of delegators of a given validator address.
DelegatorsInformation(addr common.Address) []*types2.Delegation
// GetECDSAFromCoinbase retrieve corresponding ecdsa address from Coinbase Address.
GetECDSAFromCoinbase(header *block.Header) (common.Address, error)
// SuperCommitteeForNextEpoch ...
// isVerify=true means validators use it to verify
// isVerify=false means leader is to propose.
SuperCommitteeForNextEpoch(
beacon engine.ChainReader,
header *block.Header,
isVerify bool,
) (*shard.State, error)
// EnablePruneBeaconChainFeature enabled prune BeaconChain feature.
EnablePruneBeaconChainFeature()
// IsEnablePruneBeaconChainFeature returns is enable prune BeaconChain feature.
IsEnablePruneBeaconChainFeature() bool
// CommitOffChainData write off chain data of a block onto db writer.
CommitOffChainData(
batch rawdb.DatabaseWriter,
block *types.Block,
receipts []*types.Receipt,
cxReceipts []*types.CXReceipt,
stakeMsgs []types2.StakeMsg,
payout reward.Reader,
state *state.DB,
) (status WriteStatus, err error)
3 years ago
// ========== Only For Tikv Start ==========
// return true if is tikv writer master
IsTikvWriterMaster() bool
// RedisPreempt used for tikv mode, get the redis preempt instance
RedisPreempt() *redis_helper.RedisPreempt
// SyncFromTiKVWriter used for tikv mode, all reader or follower writer used to sync block from master writer
SyncFromTiKVWriter(newBlkNum uint64, logs []*types.Log) error
// ========== Only For Tikv End ==========
}