11 Commits (89653714829d26ae98cf4eaddd98972131943fba)

Author	SHA1	Message	Date
Rongjian Lan	aa643ca856	Add out of order message processing logic (#3468)	4 years ago
Rongjian Lan	1851d7eea5	add HasSingleSender func	4 years ago
Rongjian Lan	8885eb45e0	update fbft id	4 years ago
Jacky Wang	7801a1d678	[consensus][sync] Better coordination between state sync and consensus module. (#3352) ... * [rawdb] add error handling to all rawdb write. Add fdlimit module. Fix the node stuck * [core] switch back the batch write condition in InsertReceiptChain * [rawdb] add error handling to all rawdb write. Add fdlimit module. Fix the node stuck * [consensus] refactored and optimized tryCatchup logic * [sync] added consensus last mile block in sync. * [consensus] remove time wait for consensus inform sync. Make block low chan a buffered chan * [consensus] fix rebase errors, and optimize one line code * [consensus][sync] fix golint error and added prune logic in sync * [consensus] move header verify after adding FBFT log in onPrepared * [consensus] more change on block verification logic * [consensus] fix the verified panic issue * [consensus][sync] add block verification in consensus last mile, change it to iterator * [consensus] fix two nil pointer references when running local node (Still cannot find the root cause for it) * remove coverage.txt and add to gitignore * [consensus] add leader key check. Move quorum check logic after tryCatchup and can spin state sync * [consensus] remove the leader sender check for now. Will add later * [consensus] refactor fbftlog to get rid of unsafe mapset module. Replace with map * [consensus] move the isQuorumAchived logic back. We surely need to check it before add to FBFTlog * [consensus] remove the redundant block nil check * [test] fix the consensus test * [consensus] rebase main and fix stuff. Removed isSendByLeader * [consensus] added logic to spin up sync when received message is greater than consensus block number * [consensus] more changes in consensus. Remove some spin sync logic. * fix error in main * [consensus] change the hash algorithm of the FBFTLog to get rid of rlp error * [consensus] use seperate mutex in FBFT message * [consensus] change fbft log id to a shorter form. Added unit test case	4 years ago
Edgar Aroutiounian	520332931b	[proto] Undo protobuf change, annotate with deprecated (#2773) ... * [consensus] staticcheck fixes * [proto] Remove unused file * [proto] Undo changes to proto, annotate fields with deprecated for future reference * [api] Make entire dir as was a7b7961~1 * [api] Remove dead tests * [proto] Add deprecate tags * [api] Autogen update since add deprecation tags, it just generated comments * [api] Remove manager test again * [api] Remove dead functions * [api] Whitespace * [api] Whitespace * [api] Whitespace * [p2p] Remove noisy log	5 years ago
Edgar Aroutiounian	a815e11409	[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) ... * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later	5 years ago
Ganesha Upadhyaya	ae362c24c6	multi-bls key support (#2249) ... refactoring to multibls package to avoid cyclic dependency and better structure fixing import issue fixing more imports fixing a bug that causes nil pointer for explorer node fixing a bug in the super majority vote changing config to have same epoch for cross link and staking fixing the compilation error addressing PR comments leaving todo for assuming order between pub/pri keys pairs fixing goimports removing unwanted warning message	5 years ago
flicker-harmony	aadeae045c	Refactor explorer service and add addresses fetch from db	5 years ago
Edgar Aroutiounian	c4d744e376	[double-sign] Provide proof of double sign in slash record sent to beaconchain (#2253) ... * [double-sign] Commit changes in consensus needed for double-sign * [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain * [slash] Add quick iteration tool for testing double-signing * [slash] Add webhook example * [slash] Add http server for hook to trigger double sign behavior * [double-sign] Use bin/trigger-double-sign to cause a double-sign * [double-sign] Full feedback loop working * [slash] Thread through the slash records in the block proposal step * [slash] Compute the slashing rate * [double-sign] Generalize yaml malicious for many keys * [double-sign][slash] Modify data structures, verify via webhook handler * [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures * [slash] Apply to state slashing for double signing * [slash][double-sign] Checkpoint for working code that slashes on beaconchain * [slash] Keep track of the total slash and total reporters reward * [slash] Dump account state before and after the slash * [slash] Satisfy Travis * [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates * [slash] Capture the unique new delegations since snapshot as well * [slash] Filter undelegation by epoch of double sign * [slash] Add TODO of correctness needed in slash needs on off-chain data * [rpc] Fix closure issue on shardID * [slash] Add delegator to double-sign testing script * [slash] Expand crt-validator.sh with commenting printfs and make delegation * [slash] Finish track payment of leftover slash debt after undelegation runs out * [slash] Now be explicit about error wrt delegatorSlashApply * [slash] Capture specific sanity check on slash paidoff * [slash] Track slash from undelegation piecemeal * [slash][delegation] Named slice types, .String() * [slash] Do no RLP encode twice, once is enough * [slash] Remove special case of validators own delegation * [slash] Refactor approach to slash state application * [slash] Begin expanding out Verify * [slash] Slash on snapshot delegations, not current * [slash] Fix Epoch Cmp * [slash] Third iteration on slash logic * [slash] Use full slash amount * [slash] More log, whitespace * [slash] Remove Println, add log * [slash] Remove debug Println * [slash] Add record in unit test * [slash] Build Validator snapshot, current. Fill out slash record * [slash] Need to get RLP dump of a header to use in test * [slash] Factor out double sign test constants * [slash] Factor out common for validator, stub out slash application, finish out deserialization setup * [slash] Factor out data structure creation because of var lexical scoping * [slash] Seem to have pipeline of unit test e2e executing * [slash] Add expected snitch, slash amounts * [slash] Checkpoint * [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing * [config] add double-sign testnet config (#1) Signed-off-by: Leo Chen <leo@harmony.one> * [slash] Commit for as is code & data of current dump.json * [slash] Order of state operation not correct in test, hence bad results, thank you dlv * [slash] Add snapshot state dump * [slash] Pay off slash of validator own delegation correctly * [slash] Pay off slash debt with special case for min-self * [slash] Pass first scenario conclusively * [slash] 2% slash passes unit test for own delegation and external * [slash] Parameterize unit test to easily test .02 vs .80 slash * [slash] Handle own delegation correctly at 80% slash * [slash] Have 80% slash working with external delegator * [slash] Remove debug code from slash * [slash] Adjust Apply signature, test again for 2% slash * [slash] Factor out scenario in testing so can test 2% and 80% at same time * [slash] Correct balance deduction on plan delegation * [slash] Mock out ChainReader for TestVerify * [slash] Small surface area interface, now feedback loop for verify * [slash] Remove development json * [slash] trigger-double-sign consumes yaml * [slash] Remove dead code * [slash][test] Factor ValidatorWrapper into scenario * [slash][test] Add example from local-testing dump - caution might be off * [slash] Factor out mutation of slashDebt * [slash][test] Factor out tests so can easily load test-case from bytes * [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected * [slash] Satisfy Travis * [slash] Begin cleanup of PR * [slash] Apply slash from header to Finalize via state processor * [slash] Productionize code, Println => logs; adjust slash picked in newblock * [slash] Need pointer for rlp.Decode * [slash] ValidatorInformation use full wrapper * Fix median stake * [staking] Adjust MarshalJSON for Validator, Wrapper * Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279) * Refactor offchain data; Add epoch to ValidatorSnapshot * Make block onchain/offchain data commit atomically * [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment * [effective] VC eligible.go * [consensus] Redundant field in printf * [docker] import-ks for a dev account * [slash] Create BLS key for dockerfile and crt-validator.sh * [slash][docker] Easy deployment of double-sign testing * [docker] Have slash work as single docker command * [rpc] Fix median-stake RPC * [slash] Update webhook with default docker BLS key * [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go * [slash] Remove helper binary, commented out code, change to local config * [params] Factor out test genesis value * Add shard checking to Tx-Pool & correct blacklist (#2301) * [core] Fix blacklist & add shardID check * [staking + node + cmd] Fix blacklist & add shardID check * [slash] Adjust to PR comments part 1 * [docker] Use different throw away funded account * [docker] Create easier testing for delegation with private keys * [docker] Update yaml * [slash] Remove special case for slashing validator own delegation wrt min-self-delegate * [docker] Install nano as well * [slash] Early error if banned * [quorum] Expose earning account in decider marshal json * Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)" This reverts commit 9ffbf682c0. * [slash] Add non-sanity check way to update validator * [reward] Increase percision on percentage in schedule * [slash] Adjust logs * [committee] Check eligibility of validator before doing sanity check * [slash] Update docker * [slash] Move create validator script to test * [slash] More log * [param] Make things faster * [slash][off-chain] Clear out slashes from pending in writeblockwithstate * [cross-link] Log is not error, just info * [blockchain] Not necessary to guard DeletePendingSlashingCandidates * [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that * [slash][test] Use faucet as sender, assume user imported * [slash] Test setup * [slash] reserve error for real error in logs * [slash][availability] Apply availability correct, bump signing count each block * [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed * [slash] Pay as much as can * [slash] use right nowAmt * [slash] Take away from rewards as well * [slash] iterate faster * [slash] Remove dev based timing * [slash] Add more log, sanity check incoming slash records, only count external for slash rate * [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters * [availability] More log * [slash] Simply pre slash erra slashing * [slash] Remove development code * [slash] Use height from recvMsg, todo on epoch * [staking] Not necessary to touch LastEpochInCommittee in staking_verifier * [slash] Undo ds in endpoint pattern config * [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment * [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count Co-authored-by: Leo Chen <leo@harmony.one> Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com> Co-authored-by: Rongjian Lan <rongjian@harmony.one> Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>	5 years ago
Edgar Aroutiounian	801b4b83e2	[slash][consensus] Notice double sign & broadcast, factor out tech debt of consensus (#2152) ... * [slash] Remove dead interface, associated piping * [slash] Expand out structs * [consensus] Write to a chan when find a case of double-signing, remove dead code * [slash] Broadcast the noticing of a double signing * [rawdb] CRUD for slashing candidates * [slashing][node][proto] Broadcast the slash record after receive from consensus, handle received proto message, persist in off-chain db while pending * [slash][node][propose-block] Add verified slashes proposed into the header in block proposal * [slash][shard] Factor out external validator as method on shard state, add double-signature field * [slash][engine] Apply slash, name boolean expression for sorts, use stable sort * [slash] Abstract Ballot results so keep track of both pre and post double sign event * [slash] Fix type errors on test code * [slash] Read from correct rawdb * [slash] Add epoch based guards in CRUD of slashing * [slash] Write to correct cache for slashing candidates * [shard] Use explicit named type of BLS Signature, use convention * [slash] Fix mistake done in refactor, improper header used. Factor out fromSlice to set * [slash][node] Restore newblock to master, try again minimial change * [cx-receipts] Break up one-liner, use SliceStable, not Slice * [network] Finish refactor that makes network message headers once * [network] Simplify creation further of headers write * [slash] Adjust data structure of slash after offline discussion with RJ, Chao * [slash] Still did need signature of the double signature * [consensus] Prepare message does not have block header * [consensus] Soft reset three files to 968517d~1 * [consensus] Begin factor consensus network intended message out with prepare first * [consensus] Factor out Prepared message * [consensus] Factor out announce message creation * [consensus] Committed Message, branch on verify sender key for clearer log * [consensus] Committed Message Factor out * [consensus] Do jenkins MVP of signatures adjustment * [main][slash] Provide YAML config as webhook config for double sign event * [consensus] Adjust signatures, whitespace, lessen GC pressure * [consensus] Remove dead code * [consensus] Factor out commit overloaded message, give commit payload override in construct * [consensus] Fix travis tests * [consensus] Provide block bytes in SubmitVote(quorum.Commit) * [consensus] Factor out noisy sanity checks in BFT, move existing commit check earlier as was before * [quorum] Adjust signatures in quorum * [staking] Adjust after merge from master * [consensus] Finish refactor of consensus * [node] Fix import * [consensus] Fix travis * [consensus] Use origin/master copy of block, fix mistake of pointer to empty byte * [consensus] Less verbose bools * [consensus] Remove unused trailing mutation hook in message construct * [consensus] Address some TODOs on err, comment out double sign	5 years ago
Edgar Aroutiounian	e6a4fbea4f	[committee] Factor out committee membership, provide entry for alternative committee membership (#1818) ... * [committee] Move core.ShardingSchedule to shard.Schedule * [consensus] Remove redundant PublicKeys field of Consensus as Decider maintains that * [committee] Use committee package to pick PublicKeys * [committee] Use committee inplace of CalculateShardState * [committee] Remove core/resharding.go, complete usage of committee as implementation replacement * [committee] Address PR comments	5 years ago
Edgar Aroutiounian	9f00923ac3	Factor out quorum logic from consensus, undo prior consolidation of values under core (#1717) ... * [consensus] Factor out enums to core/values, begin factor out of consensus mechanisms * [consensus] Make Mechanism explicit * [consensus] Add ViewChange to QuorumPhase * Update core/values/consensus.go Co-Authored-By: Eugene Kim <ek@harmony.one> * Update core/values/consensus.go Co-Authored-By: Eugene Kim <ek@harmony.one> * [mainnet-release] Address code comments * [staking][consensus][project] Remove txgen, factor out consensus * [consensus] Factor out PublicKeys * [txgen] Bring back txgen * [project] Undo prior consolidation of error values under core * [consensus] Update tests using quorum decider * [consensus] Fix overlooked resets during refactor * [consensus] Fix wrong check of quorum phase * [consensus] Address leftover TODO for prepare count * [consensus] Simplfy reset switch * [consensus] Fix mistake of wrong ReadSignature in ViewChange, need sender, not node PubKey	5 years ago
Eugene Kim	5bd07e8e4a	Configure commit delay via setter and not ctor arg ... Also make -delay_commit accept duration strings such as 500ms.	6 years ago
Eugene Kim	b084e3c38d	Retire isHarmonyAccount logic ... This has been superseded by commitDelay.	6 years ago
chao	817592c976	add delay in commit message sending for genesis account	6 years ago
Chao Ma	26efc233d7	m1 type message view change path works	6 years ago
chao	7cd24dae6c	modify the naming of variables to make code cleaner ... address pr comments	6 years ago
chaosma	b53d8fb5bf	pbft normal path (#851) ... * PBFT view change framework added * timeout class and pbft_log class added for view change * consensus protocol worked under PBFT normal mode	6 years ago