[viewchange] encapsulate view ID in the State struct
do NOT ues consensus.current directory to set/get viewID
use the following functions
consensus.SetCurViewID, consensus.SetViewChangingID
consensus.GetCurViewID, consensus.GetViewChangingID
Signed-off-by: Leo Chen <leo@harmony.one>
This is a big PR merged many small commits together.
We add the message validation function in libp2p layer.
In the validation function, we check the following conditions
1) the p2p message is a valid consensus message
2) the p2p message sender has a valid public key
3) the sender's public key is in the current committee
4) log the number of invalid/valid messages
After the validation, the valid messages will be forward to the network,
while the invalid messages will be filtered out.
The messages intended for the validator will be handled in the consensus layer.
Signed-off-by: Leo Chen <leo@harmony.one>
* fix signers share computation to use only non-harmony nodes (#3008)
* Add LastEpochInCommittee at prestaking epoch too (#3014)
* write validator stats at the end of prestaking epoch (#3021)
* write validator stats at the end of prestaking epoch
* also need to write block reward accumulator
* [build] enable static build by default (#2885)
scripts/go_executable_build.sh -s (static build, default)
scripts/go_executable_build.sh -S (non-static build)
Signed-off-by: Leo Chen <leo@harmony.one>
* Code formatting fix based on `staticcheck` (#3023)
* fix some static check issues
* more fixes according to staticcheck
* add nil check on getHeader
* Add more log info
* [rpc] Revert change to v2 GetBalance RPC (#3037)
* Cherrypick crosslink related fixes from t3 to master. (#3045)
* Optimize crosslink verification logic and add more cache (#3032)
* make crosslink broadcast smarter and more efficient (#3036)
* adjust catch up speed to be a sane number
* Fix crosslink broadcast condition (#3041)
* Tx pool error report fix & local build fix (#3046)
* [tx_list] Add transaction after you can fetch tx cost
* [tx_pool] Add error sink reports for removed tx
* Correct log levels for known transactions
* Remove redundant hash to `enqueue` and `add` signatures
* Update tests for signature changes
* [build] Fix local build for MacOs
* [tx_pool] Correct typo & logging level for known tx
* [config] Create Testnet config
* [node.sh] update testnet network config
* [config] correct testnet config
* [main] Enable beacon sync for explorer nodes (#3059)
* [apr] should not use snapshot from last epoch, use current epoch snapshot (#3051)
* use snapshot total delegation instead of current total delegation
* should not use snapshot from last epoch, use current epoch snapshot (#3049)
Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com>
Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com>
* cherrypick log level change
* [rpc] Add active status back to Validator information
[validator] Fix typo for ValidatorRPCEnhanced
[Makefile] Change make exe to build non-static by default
* [network] Added caching to node server (#3048)
* [network] Added caching to node server
* [network] renamed two singleflight.Group
* [network] added two error handling
* [network] added forget logic for single flight
* [network] changed single flight to lru cache solution
* [network] adjusted the cache size
* [network] changed cache size to 10000
* Transaction api improvements (#3057)
* [node] Add option to broadcast invalid tx (on by default)
* update main args to take in broadcast invalid tx
* update node.sh for new broadcast invalid tx option
* [rpc] Do not submit transaction if tx has been finalized
* Support legacy behavior and not return errors on tx submission
* [main] make broadcasting invalid tx default false
* [node] Fix formatting
* Add timeouts for the explorer HTTP service
* [test] State trans unit test (#3043)
* completing create-validator tests
generalizing create validator tests, started with edit validator tests
one round completed before major refactor
* [test] finished coding test TestCheckDuplicateFields
* [test] finished debug TestCheckDuplicateFields
* [test] Added TestDescription_EnsureLength
* [test] added test in validator_test.go TestUpdateDescription
* [test] Added test TestComputed_String
* [test] refactored TestValidatorSanityCheck and added some message in error message returned from sanity check
* [test] removed test index for debugging
* [test] Added test TestValidatorWrapper_SanityCheck. Added value check for TestMarshalUnmarshalValidator. Fixed rlp usage defect in UnmarshalValidator
* [test] added test TestVerifyBLSKeys
* [test] added test TestContainsHarmonyBLSKeys
* [test] added test TestCreateValidatorFromNewMsg
* [test] fixed some tests errors
* [test] added test TestUpdateValidatorFromEditMsg, last editted the test file, finished testing validator_test.go
* [staking] added numeric.Dec.Copy method
* [staking] added CommissionRates.Copy
* [staking] add nil case handle to copy method and related test case
* [test] added nil case for commissionRates.Copy test
* [staking] finished CreateValidator.Copy and related test case
* [staking] added EditValidator.Copy method and related test case
* [test] added zero test cases for Copy method
* [staking] implemented Delegate.Copy and related unit test case
* [staking] added Undelegate.Copy and CollectReward.Copy method. Implemented corresponding test cases
* [test] added two more simple unit tests
* [test] solving the merge conflict with deep copy fix
* [test] added schedule for sharding
* [test] refactored double_sign_test.go
* [test] comment out all test cases for now.
* [test] added record Copy method for testing
* [test] added the first test case in TestVerify
* [test] finished test TestVerify
* [test] Added test TestVerify, TestApplySlashRate, TestSetDifference
* [test] half through TestDelegatorSlashApply
* [test] fix the problem cause by unsafe usage of big.Int
* [staking] added the copy methods to validatorWrapper
* [test] added test cases for copy method for staking/validatorWrapper
* [test] added test case TestApply
* [test] added test case TestRate
* [test] fix the golint errors
* [test] removed commented out unit test code
* [test] remote the empty line in imports
* [test] moved copy methods to stake_testing module
* [test] removed usage of interface stateDB
* [test] removed empty lines in imports
* [test] fixed golint package name issues
* [test] removed a todo when writing code
* [test] moved record.Copy to staking/slash/test
* [test] add some changes
* [test] added prototypes in staketest module
* [test] fix the golint issue in staketest
* [test] make prototype more beautiful
* [test] refactored test TestCheckDuplicateFields
* [test] add createValidator test data
* [test] added positive test case for VerifyAndCreateValidatorFromMsg
* [test] added create validator test
* [test] added the positive test case for TestVerifyAndEditValidatorFromMsg
* [test] added the tests case TestVerifyAndEditValidatorFromMsg
* [test] fix one of the error is not triggered as expected
* [test] more changes in test
* [test] fix the positive test data for TestVerifyAndDelegateFromMsg
* [test] Fixed two comment golint errors
* [test] added delegate tests
* [test] added a make function. WIP
* [test] added undelegate positive test
* [test] added negative test cases for undelegate
* [test] added positive test. not passed. Need to add assert method in test suites
* [test] added equal function to check Validator Equality
* [test] added equals for staketest
* [test] replaced deep equal with new equal methods:
* [test] removed unused codes
* [test] Finishing touch
* [test] fix comment golint issue
* [test] removed dead code in staking_verifier_test.go
Co-authored-by: Ganesha Upadhyaya <ganeshrvce@gmail.com>
* changed the sync port to dns port in flag
* [network] Limit client sync connection (#3071)
* [network] limit client connected peers
* Changed signature and added tests
* Set secure permissions on generated blskey pass file in node.sh
* Fix spellin gtypo in node.sh
* fix latest param handling in api v1
* added comment for the isGreaterThanLatestBlockNum check
* fix a nil pointer crash due to previous revert
Signed-off-by: Leo Chen <leo@harmony.one>
* fix nil pointer crash
Signed-off-by: Leo Chen <leo@harmony.one>
* fix nil pointer of block
Signed-off-by: Leo Chen <leo@harmony.one>
* [rpc] add viewID and epoch to RPCMarshalBlock (#3009)
* [rpc] add viewID and epoch to RPCMarshalBlock
* add apt-get update
* Do not rollback current block on failed block sync (#3101)
Co-authored-by: Ganesha Upadhyaya <ganeshrvce@gmail.com>
Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com>
Co-authored-by: Jacky Wang <jackyw.se@gmail.com>
Co-authored-by: Janet Liang <56005637+janet-harmony@users.noreply.github.com>
Co-authored-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
Co-authored-by: Janet Liang <janet@harmony.one>
Co-authored-by: Sebastian Johnsson <sebastian.johnsson@gmail.com>
Co-authored-by: hsiung <hsiung@pinterest.com>
Co-authored-by: Dennis Won <jhwon0820@gmail.com>
Co-authored-by: Yishuang Chen <34232522+ivorytowerdds@users.noreply.github.com>
* [project] Dead test
* [node] If fields always same, then just use the constant
* [project] Remove overcomplicated ctxerror
* [project] Remove more dead tests, adjust error replacing ctxerror
* [project] More dead tests, fix travis complaint on Errorf
* [node.sh] Remove is-genesis
* [rpc][validator] Extend hmy blockchain validator information
* [availability] Optimize bump count
* [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior
* [slash] Reimplement SetDifference
* [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards
* [header] Custom Marshal header for downstream, remove dev code
* [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS
* [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test
* [reward] Use excellent singleflight package for caching lookup of subcommittees
* [apr][reward] Begin APR package itself, iterate on iterface signatures
* [reward] Handle possible error from singleflight
* [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information
* [apr] Stub out computation of APR
* [effective][committee] Upgrade SlotPurchase with named fields, provide marshal
* [effective] Update Tests
* [blockchain] TODO Remove the validators no longer in committee
* [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now
* [rpc] Median-stake more semantic meaningful
* [validator] Iterate on semantic meaning of JSON representation
* [offchain] Make validator stats return explicit error
* [availability] Small typo
* [rpc] Quick visual hack until fix delete out kicked out validators
* [offchain] Delete validator from offchain that lost their slot
* [apr] Forgot to update interface signature
* [apr] Mul instead of Div
* [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain
* [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters
* [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick
* [shard] Granular errors
* [votepower][validator] Unify votepower data structure with off-chain usage
* [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats
* [votepower] Use RJs naming convention group,overall
* [votepower] Remove Println, do keep enforcing order
* [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations
* [apr] Adjust json field name
* [votepower] Only aggregate on external validator
* [votepower] Mistake on aggregation, custom presentation network-wide
* [rpc][validator][availability] Remove parameter, take into account empty snapshot
* [apr] Use snapshots from two, one epochs ago. Still have question on header
* [apr] Use GetHeaderByNumber for the header needed for time stamp
* [chain] Evict > 3 epoch old voting power
* [blockchain] Leave Delete Validator snapshot as TODO
* [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning
* [project] Address PR comments
* [committee][rpc] Move +1 to computation of epos round rather than hack mutation
* [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int
* [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC
* [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits
* [node] Add hack way to get real error reason of why cannot load shardchain
* [consensus] Expand log on current issue on nil block
* [apr] Do the actual call to compute for validator's APR
* [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC
* [apr] Incorrect error handle order
* [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check
* [shard] Add log if stakedSlots is 0
* [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet
* [committee] Remove + 1 on seat count
* [apr] Use int64() directly
* [apr] Log when odd empty nil header
* [apr] Do not crash on empty header, figure out later
* [staking][validation][protocol] Limit max bls keys
* [staking-era] Fold banned and active into single field
* [slash][effective] Remove LRU cache for slash, change .Active to enumeration
* [slash] Remove leftover wrong usage of Logger
* [slash][offchain] Only Decode if len > 0
* [offchain] cosmetic
* [slash] Remove some logs in proposal
* [webhook] Move webhook with call for when cannot commit block
* [shard] Finally make finding subcommittee by shardID an explicit error
* [node] Whitespace, prefer literal
* [webhook] Report bad block to webhook
* [slash] Expand verify, remove bad log usage, explicit error handle
* [slash] Check on key size
* [slash] Explicit upper bound of pending slashes
* [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain
* [multibls] Make max count allowed be 1/3 of external slots
* [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee
* [verify] Begin factor out of common verification approach
* [project] Further remove RawJSON log, use proper epoch for snapshot
* [slash] Implement verification
* [slash] Implement BLS key verification of ballots
* [rpc] Keep validator information as meaningful as possible
* [staking] Never can stop being banned
* [slash] Comments and default Unknown case of eligibility
* [slash] Be explicit on what input values allowed when want to change EPOSStatus
* [consensus] Remove unneeded TODO
* [verify] Add proper error message
* [rpc] Give back to caller their wrong chain id
* [chain] Add extra map dump of delegation sizing for downstream analysis
* [engine] Less code, more methods
* [offchain] More leniency in handling slash bytes and delete from pending
* [validator] Remove errors on bad input for edit
* [assignment] Make error case be explicit on public keys, log out important committee size info
* [committee] Do hmy nodes first, exit early if 0 externals to do
* [genesis] Never ignore errors
* [slash][availability] Remove log, remove leftover yaml
* [committee] Maintain same API as before
* [rpc] Enchance RPC of validator information for current epoch signing percent
* [availability] Remove excessive log
* [availability][slash] Unify yaml hook config
* [availability][webhooks] Call webhook when reach missing signing threshold
refactoring to multibls package to avoid cyclic dependency and better structure
fixing import issue
fixing more imports
fixing a bug that causes nil pointer for explorer node
fixing a bug in the super majority vote
changing config to have same epoch for cross link and staking
fixing the compilation error
addressing PR comments
leaving todo for assuming order between pub/pri keys pairs
fixing goimports
removing unwanted warning message
Rongjian Lan
Leo Chen
Jacky Wang
Ganesha Upadhyaya
Edgar Aroutiounian
flicker-harmony