This is a big PR merged many small commits together.
We add the message validation function in libp2p layer.
In the validation function, we check the following conditions
1) the p2p message is a valid consensus message
2) the p2p message sender has a valid public key
3) the sender's public key is in the current committee
4) log the number of invalid/valid messages
After the validation, the valid messages will be forward to the network,
while the invalid messages will be filtered out.
The messages intended for the validator will be handled in the consensus layer.
Signed-off-by: Leo Chen <leo@harmony.one>
* Optimize crosslink verification logic and add more cache (#3032)
* make crosslink broadcast smarter and more efficient (#3036)
* adjust catch up speed to be a sane number
* Fix crosslink broadcast condition (#3041)
* [node] Unused package level big.Int
* [p2p] Throttle receives to a max of 100
* [project] Remove msgq
* [node][p2p] Only allow 100 max message handling at time
* [node][p2p] Throttle max message handler with Weighted Semaphore - init w/ uniform weight
* [node] Use go routine for passing handled block
* [p2p] Leftover commented out
* [node] Add log for when cannot pay semaphore cost
* [node] Bump max message handlers
* [internal] Remove file_embed
* [project] Remove unused file, unused function
* [consensus] Unused values
* [config] Unused field & function
* [internal] Unused address & hash
* [project] More unused tests, values, types
* [project] Move conn logger just to bootnode
* [rpc] Remove dead functions
* [internal] More unused functions
* [project] Remove dead keystore dir package
* [p2p] Need lock to grab all Topics
* [node] Force side effect of topic join, give each topic own weighted semaphore
* [p2p] Remove unused Method
* [p2p] More dead function
* [node] Unused BCPeers field
* [node] Need to exit early if have ping
* [p2p] Remove unnecessary & confusing abstraction
* [p2p] Expose topics
* [project] Dead test
* [node] If fields always same, then just use the constant
* [project] Remove overcomplicated ctxerror
* [project] Remove more dead tests, adjust error replacing ctxerror
* [project] More dead tests, fix travis complaint on Errorf
* [node.sh] Remove is-genesis
* [project] Remove hmyclient
* [drand] Remove dead drand
* [drand] Remove dead code in proto, more dead drand things
* [project] Remove dead demo tool
* [project] Remove dead Lottery solidity
* [project] Remove dead hmy client main
* [node] Remove unused variable in test code
* [project] Remove keygen
* [project] Remove not useful test
* [rpc][validator] Extend hmy blockchain validator information
* [availability] Optimize bump count
* [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior
* [slash] Reimplement SetDifference
* [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards
* [header] Custom Marshal header for downstream, remove dev code
* [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS
* [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test
* [reward] Use excellent singleflight package for caching lookup of subcommittees
* [apr][reward] Begin APR package itself, iterate on iterface signatures
* [reward] Handle possible error from singleflight
* [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information
* [apr] Stub out computation of APR
* [effective][committee] Upgrade SlotPurchase with named fields, provide marshal
* [effective] Update Tests
* [blockchain] TODO Remove the validators no longer in committee
* [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now
* [rpc] Median-stake more semantic meaningful
* [validator] Iterate on semantic meaning of JSON representation
* [offchain] Make validator stats return explicit error
* [availability] Small typo
* [rpc] Quick visual hack until fix delete out kicked out validators
* [offchain] Delete validator from offchain that lost their slot
* [apr] Forgot to update interface signature
* [apr] Mul instead of Div
* [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain
* [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters
* [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick
* [shard] Granular errors
* [votepower][validator] Unify votepower data structure with off-chain usage
* [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats
* [votepower] Use RJs naming convention group,overall
* [votepower] Remove Println, do keep enforcing order
* [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations
* [apr] Adjust json field name
* [votepower] Only aggregate on external validator
* [votepower] Mistake on aggregation, custom presentation network-wide
* [rpc][validator][availability] Remove parameter, take into account empty snapshot
* [apr] Use snapshots from two, one epochs ago. Still have question on header
* [apr] Use GetHeaderByNumber for the header needed for time stamp
* [chain] Evict > 3 epoch old voting power
* [blockchain] Leave Delete Validator snapshot as TODO
* [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning
* [project] Address PR comments
* [committee][rpc] Move +1 to computation of epos round rather than hack mutation
* [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int
* [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC
* [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits
* [node] Add hack way to get real error reason of why cannot load shardchain
* [consensus] Expand log on current issue on nil block
* [apr] Do the actual call to compute for validator's APR
* [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC
* [apr] Incorrect error handle order
* [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check
* [shard] Add log if stakedSlots is 0
* [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet
* [committee] Remove + 1 on seat count
* [apr] Use int64() directly
* [apr] Log when odd empty nil header
* [apr] Do not crash on empty header, figure out later
* More logs and checks before processing staking txns
* fix import
* Refactor block proposal
* Various fixes and todos for auditing; add log for collect rewards
* Fix lint
* fix comment
* [staking][validation][protocol] Limit max bls keys
* [staking-era] Fold banned and active into single field
* [slash][effective] Remove LRU cache for slash, change .Active to enumeration
* [slash] Remove leftover wrong usage of Logger
* [slash][offchain] Only Decode if len > 0
* [offchain] cosmetic
* [slash] Remove some logs in proposal
* [webhook] Move webhook with call for when cannot commit block
* [shard] Finally make finding subcommittee by shardID an explicit error
* [node] Whitespace, prefer literal
* [webhook] Report bad block to webhook
* [slash] Expand verify, remove bad log usage, explicit error handle
* [slash] Check on key size
* [slash] Explicit upper bound of pending slashes
* [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain
* [multibls] Make max count allowed be 1/3 of external slots
* [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee
* [verify] Begin factor out of common verification approach
* [project] Further remove RawJSON log, use proper epoch for snapshot
* [slash] Implement verification
* [slash] Implement BLS key verification of ballots
* [rpc] Keep validator information as meaningful as possible
* [staking] Never can stop being banned
* [slash] Comments and default Unknown case of eligibility
* [slash] Be explicit on what input values allowed when want to change EPOSStatus
* [consensus] Remove unneeded TODO
* [verify] Add proper error message
* [rpc] Give back to caller their wrong chain id
* [chain] Add extra map dump of delegation sizing for downstream analysis
* [engine] Less code, more methods
* [offchain] More leniency in handling slash bytes and delete from pending
* [validator] Remove errors on bad input for edit
* [webhook] Loop over all self address
* [rpc] Remove accidential recursion
* [rpc] Never ignore errors, fix RPC with current snap info, no div by zero
* [rpc] Expose BadBlock for downstream
* [availability] Thread through missing signers for offchain usage in WriteBlockWithState
* [project] Merge with master, accept master conflicts
* [offchain] Remove inactive signers from offchain data, should fix median-stake issues
* [rpc] Test idea that returning error cause jenkins failure
* [staking] Put editvalidator back into validator list cache
* [rpc] Add network voting-power to validator information
* [votepower] Extend roster with raw percentage
* [rpc] If have validator but not snapshot, give back validator
* [rpc] Expose effective-stake as well
* [project] Remove missedSignThreshold map, use instance for stakedslots count
* [staking] Remove edit-validator idea from previous filter idea
* [assignment] Make error case be explicit on public keys, log out important committee size info
* [committee] Do hmy nodes first, exit early if 0 externals to do
* [genesis] Never ignore errors
* [slash][availability] Remove log, remove leftover yaml
* [committee] Maintain same API as before
* [rpc] Enchance RPC of validator information for current epoch signing percent
* [availability] Remove excessive log
* [availability][slash] Unify yaml hook config
* [availability][webhooks] Call webhook when reach missing signing threshold
* Define constants for tx-data and check size limit in tx-pool
* Check for raw data size limitation at RPC layer
* Check for raw data size limitation at Network layer
refactoring to multibls package to avoid cyclic dependency and better structure
fixing import issue
fixing more imports
fixing a bug that causes nil pointer for explorer node
fixing a bug in the super majority vote
changing config to have same epoch for cross link and staking
fixing the compilation error
addressing PR comments
leaving todo for assuming order between pub/pri keys pairs
fixing goimports
removing unwanted warning message
* [availability] More log, remove error of div by zero
* [availability] Apply increment of count in accumulate since need crosslink
* [availability] More log
* [staking] VC easy flow of create validator and delegation, restart watchdog
* [availability] Use method instead of inline, fix Cmp mistake in measure
* [availability] Add explicit log for measure
* [availability] Begin unit test impl
* [availability] Factor out subset of methods that availability needs, expand test
* [availability] Do not increment count when LastEpochInCommittee is 0
* [availability] Remove dev scripts
* [availability] Adjust logs
* [availability] Remove test BLS keys
* [availability] Remove check on LastEpoch in bumpCount, remove spurious log
* [availability] Further simplify, current shard state is the elected validator
* [consensus] Remove log
* [project] Remove noisy, hard to know where happened log; whitespace on style
Leo Chen
Rongjian Lan
Edgar Aroutiounian
Jian Zhen
Ganesha Upadhyaya
Janet Liang
flicker-harmony