The core protocol of WoopChain
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
woop/core/staking_verifier.go

569 lines
19 KiB

Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
package core
import (
"bytes"
Resolve harmony-one/bounties#77: Staking precompiles (#3906) * Resolve harmony-one/bounties#77: Staking precompiles Create write capable precompiles that can perform staking transactions Add hard fork logic (EpochTBD) for these precompiles Tests for new code with at least 80% unit test coverage Staking library + tests in MaxMustermann2/harmony-staking-precompiles * Fix small typo in comment * Run goimports on files to fix Travis * Do not activate staking precompile on shard 0 * Cascade readOnly to WriteCapableContract * No overlap in readOnly + writeCapable precompiles * Use function selector instead of directive From Solidity, use abi.encodeWithSelector and match it against the exact ABI of the functions. This allows us to remove the need for a directive (32) being encoded, and thus saves 28 bytes of data. * Do not allow contracts to become validators As discussed with Jacky on #3906 * Merge harmony-one/harmony/main properly this time * Run goimports * Update gas calculation for staking precompile Please see comment in core/vm/contracts_write.go RequiredGas * Do not allow contract to become validator (2/2) * Cache StakeMsgs from precompiled transactions Add the StakeMsgs to ProcessorResult and cascade them in insertChain * Remove ContractCode fields from validators Since smart contracts can no longer beecome validators, this field is superfluous. Remove it from the Wrapper structure, and do not assign it a value when creating a validator. Build and goimports checked * Update comments in response to feedback (1) Comments to start with function names (2) Comments for public variables (3) Comment to match function name RunPrecompiledContract (4) Clarify that CreateValidatorFunc + EditValidatorFunc are still used * Fix Travis build by reverting rosetta change * Add revert capability to 3 staking tx types - Delegate - Undelegate - CollectRewards * Fix build: Update evm_test for ValidatorWrapper * Merge main into harmony-staking-precompiles * Add gas for precompile calls and allow EOA usage - Each time the precompile is called, charge the base gas fee plus data cost (if data can be parsed successfully). A gas fee is added to prevent benevolent contract deployers from subsidizing the staking transactions for EOAs through repeated assembly `delegatecall`. - Allow EOAs to use the staking precompile directly. Some changes to the Solidity library are associated with this change. - Remove bytes from parsing address, since the ABI unpacks it into an address format correctly. - Add or update tests. Test coverage report to be attached to the PR shortly. * Run goimports * Check read only and write capable for overlap * Handle precompile stakeMsgs for block proposer The staking precompile generates staking messages which are cascaded to the block via the EVM in `state_processor.go`. This change cascades them in `worker.go` to allow block proposers and block verifiers to keep the same state. * Run goimports for cf2dfac4081444e36a120c9432f4e.. * Update staking precompile epoch to 2 for localnet Bring it in line with staking epoch. Change effects all configurations except mainnet and testnet. `goimports` included. * Add read only precompile to fetch the epoch num * Move epoch precompile to 250 * precompiles: left pad the returned epoch number * chainConfig: check epochs for precompiles panic if staking precompile epoch < pre staking epoch * Add staking migration precompile - Lives at address 251 - Migrates delegations + pending undelegations from address A to B - Useful if address A is hacked - Charges gas of 21k + cost of bytes for two addresses - Does not remove existing delegations, just sets them to zero. Replicates current undelegate setup - Unit tests and `goimports` included. Integration test following shortly in MaxMustermann2/harmony-staking-precompiles * Migration precompile: merge into staking Merge the two precompiles into one, add gas calculation for migration precompile. Move epoch precompile to 251 as a result. When migrating, add undelegations to `To`'s existing undelegations, if any match the epoch. * Add migration gas test, remove panic, add check In response to review comments, add tests for migration gas wherein there are 0/1/2 delegations to migrate. Add the index out of bound check to migration gas calculator and remove panics. Lastly, re-sort migrated undelegations if no existing undelegation in the same epoch was found on `To`. * Move undelegations sorting to end of loop
3 years ago
"fmt"
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
"math/big"
Resolve harmony-one/bounties#77: Staking precompiles (#3906) * Resolve harmony-one/bounties#77: Staking precompiles Create write capable precompiles that can perform staking transactions Add hard fork logic (EpochTBD) for these precompiles Tests for new code with at least 80% unit test coverage Staking library + tests in MaxMustermann2/harmony-staking-precompiles * Fix small typo in comment * Run goimports on files to fix Travis * Do not activate staking precompile on shard 0 * Cascade readOnly to WriteCapableContract * No overlap in readOnly + writeCapable precompiles * Use function selector instead of directive From Solidity, use abi.encodeWithSelector and match it against the exact ABI of the functions. This allows us to remove the need for a directive (32) being encoded, and thus saves 28 bytes of data. * Do not allow contracts to become validators As discussed with Jacky on #3906 * Merge harmony-one/harmony/main properly this time * Run goimports * Update gas calculation for staking precompile Please see comment in core/vm/contracts_write.go RequiredGas * Do not allow contract to become validator (2/2) * Cache StakeMsgs from precompiled transactions Add the StakeMsgs to ProcessorResult and cascade them in insertChain * Remove ContractCode fields from validators Since smart contracts can no longer beecome validators, this field is superfluous. Remove it from the Wrapper structure, and do not assign it a value when creating a validator. Build and goimports checked * Update comments in response to feedback (1) Comments to start with function names (2) Comments for public variables (3) Comment to match function name RunPrecompiledContract (4) Clarify that CreateValidatorFunc + EditValidatorFunc are still used * Fix Travis build by reverting rosetta change * Add revert capability to 3 staking tx types - Delegate - Undelegate - CollectRewards * Fix build: Update evm_test for ValidatorWrapper * Merge main into harmony-staking-precompiles * Add gas for precompile calls and allow EOA usage - Each time the precompile is called, charge the base gas fee plus data cost (if data can be parsed successfully). A gas fee is added to prevent benevolent contract deployers from subsidizing the staking transactions for EOAs through repeated assembly `delegatecall`. - Allow EOAs to use the staking precompile directly. Some changes to the Solidity library are associated with this change. - Remove bytes from parsing address, since the ABI unpacks it into an address format correctly. - Add or update tests. Test coverage report to be attached to the PR shortly. * Run goimports * Check read only and write capable for overlap * Handle precompile stakeMsgs for block proposer The staking precompile generates staking messages which are cascaded to the block via the EVM in `state_processor.go`. This change cascades them in `worker.go` to allow block proposers and block verifiers to keep the same state. * Run goimports for cf2dfac4081444e36a120c9432f4e.. * Update staking precompile epoch to 2 for localnet Bring it in line with staking epoch. Change effects all configurations except mainnet and testnet. `goimports` included. * Add read only precompile to fetch the epoch num * Move epoch precompile to 250 * precompiles: left pad the returned epoch number * chainConfig: check epochs for precompiles panic if staking precompile epoch < pre staking epoch * Add staking migration precompile - Lives at address 251 - Migrates delegations + pending undelegations from address A to B - Useful if address A is hacked - Charges gas of 21k + cost of bytes for two addresses - Does not remove existing delegations, just sets them to zero. Replicates current undelegate setup - Unit tests and `goimports` included. Integration test following shortly in MaxMustermann2/harmony-staking-precompiles * Migration precompile: merge into staking Merge the two precompiles into one, add gas calculation for migration precompile. Move epoch precompile to 251 as a result. When migrating, add undelegations to `To`'s existing undelegations, if any match the epoch. * Add migration gas test, remove panic, add check In response to review comments, add tests for migration gas wherein there are 0/1/2 delegations to migrate. Add the index out of bound check to migration gas calculator and remove panics. Lastly, re-sort migrated undelegations if no existing undelegation in the same epoch was found on `To`. * Move undelegations sorting to end of loop
3 years ago
"sort"
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
8 months ago
"github.com/woop-chain/woop/staking/availability"
8 months ago
"github.com/woop-chain/woop/internal/params"
8 months ago
"github.com/woop-chain/woop/crypto/bls"
[staking][validation][protocol] (#2396) * [staking][validation][protocol] Limit max bls keys * [staking-era] Fold banned and active into single field * [slash][effective] Remove LRU cache for slash, change .Active to enumeration * [slash] Remove leftover wrong usage of Logger * [slash][offchain] Only Decode if len > 0 * [offchain] cosmetic * [slash] Remove some logs in proposal * [webhook] Move webhook with call for when cannot commit block * [shard] Finally make finding subcommittee by shardID an explicit error * [node] Whitespace, prefer literal * [webhook] Report bad block to webhook * [slash] Expand verify, remove bad log usage, explicit error handle * [slash] Check on key size * [slash] Explicit upper bound of pending slashes * [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain * [multibls] Make max count allowed be 1/3 of external slots * [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee * [verify] Begin factor out of common verification approach * [project] Further remove RawJSON log, use proper epoch for snapshot * [slash] Implement verification * [slash] Implement BLS key verification of ballots * [rpc] Keep validator information as meaningful as possible * [staking] Never can stop being banned * [slash] Comments and default Unknown case of eligibility * [slash] Be explicit on what input values allowed when want to change EPOSStatus * [consensus] Remove unneeded TODO * [verify] Add proper error message * [rpc] Give back to caller their wrong chain id * [chain] Add extra map dump of delegation sizing for downstream analysis * [engine] Less code, more methods * [offchain] More leniency in handling slash bytes and delete from pending * [validator] Remove errors on bad input for edit
5 years ago
"github.com/ethereum/go-ethereum/common"
8 months ago
"github.com/woop-chain/woop/common/denominations"
"github.com/woop-chain/woop/core/vm"
common2 "github.com/woop-chain/woop/internal/common"
"github.com/woop-chain/woop/internal/utils"
"github.com/woop-chain/woop/staking/effective"
staking "github.com/woop-chain/woop/staking/types"
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
"github.com/pkg/errors"
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
)
var (
errStateDBIsMissing = errors.New("no stateDB was provided")
errChainContextMissing = errors.New("no chain context was provided")
errEpochMissing = errors.New("no epoch was provided")
errBlockNumMissing = errors.New("no block number was provided")
)
func checkDuplicateFields(
addrs []common.Address, state vm.StateDB,
validator common.Address, identity string, blsKeys []bls.SerializedPublicKey,
) error {
checkIdentity := identity != ""
checkBlsKeys := len(blsKeys) != 0
blsKeyMap := map[bls.SerializedPublicKey]struct{}{}
for _, key := range blsKeys {
blsKeyMap[key] = struct{}{}
}
for _, addr := range addrs {
if !bytes.Equal(validator.Bytes(), addr.Bytes()) {
Resolve harmony-one/bounties#90: Add revert mechanism for UpdateValidatorWrapper (#3939) * Add revert mechanism for UpdateValidatorWrapper Closes harmony-one/bounties#90 (1) Use LRU for ValidatorWrapper objects in stateDB to plug a potential memory leak (2) Merge ValidatorWrapper and ValidatorWrapperCopy to let callers ask for either a copy, or a pointer to the cached object. Additionally, give callers the option to not deep copy delegations (which is a heavy process). Copies need to be explicitly committed (and thus can be reverted), while the pointers are committed when Finalise is called. (3) Add a UpdateValidatorWrapperWithRevert function, which is used by staking txs `Delegate`, `Undelegate`, and `CollectRewards`. Other 2 types of staking txs and `db.Finalize` continue to use UpdateValidateWrapper without revert, again, to save memoery (4) Add unit tests which check a) Revert goes through b) Wrapper is as expected after revert c) State is as expected after revert * Change back to dictionary for stateValidators Since the memory / CPU usage saved is not significantly different when using an LRU + map structure, go back to the original dictionary structure to keep code easy to read and have limited modifications. * Add tests for validator wrapper reverts As requested by @rlan35, add tests beyond just adding and reverting a delegation. The tests are successive in the sense that we do multiple modifications to the wrapper, save a snapshot before each modification and revert to each of them to confirm everything works well. This change improves test coverage of statedb.go to 66.7% from 64.8% and that of core/state to 71.9% from 70.8%, and covers all the code that has been modified by this PR in statedb.go. For clarity, the modifications to the wrapper include (1) creation of wrapper in state, (2) adding a delegation to the wrapper, (3) increasing the blocks signed, and (4) a change in the validator Name and the BlockReward. Two additional tests have been added to cover the `panic` and the `GetCode` cases.
3 years ago
wrapper, err := state.ValidatorWrapper(addr, true, false)
if err != nil {
return err
}
if checkIdentity && wrapper.Identity == identity {
return errors.Wrapf(errDupIdentity, "duplicate identity %s", identity)
}
if checkBlsKeys {
for _, existingKey := range wrapper.SlotPubKeys {
if _, ok := blsKeyMap[existingKey]; ok {
return errors.Wrapf(errDupBlsKey, "duplicate bls key %x", existingKey)
}
}
}
}
}
return nil
}
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
// TODO: add unit tests to check staking msg verification
// VerifyAndCreateValidatorFromMsg verifies the create validator message using
// the stateDB, epoch, & blocknumber and returns the validatorWrapper created
// in the process.
//
// Note that this function never updates the stateDB, it only reads from stateDB.
func VerifyAndCreateValidatorFromMsg(
stateDB vm.StateDB, chainContext ChainContext, epoch *big.Int, blockNum *big.Int, msg *staking.CreateValidator,
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
) (*staking.ValidatorWrapper, error) {
if stateDB == nil {
return nil, errStateDBIsMissing
}
if chainContext == nil {
return nil, errChainContextMissing
}
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
if epoch == nil {
return nil, errEpochMissing
}
if blockNum == nil {
return nil, errBlockNumMissing
}
if msg.Amount.Sign() == -1 {
return nil, errNegativeAmount
}
if stateDB.IsValidator(msg.ValidatorAddress) {
[staking][validation][protocol] (#2396) * [staking][validation][protocol] Limit max bls keys * [staking-era] Fold banned and active into single field * [slash][effective] Remove LRU cache for slash, change .Active to enumeration * [slash] Remove leftover wrong usage of Logger * [slash][offchain] Only Decode if len > 0 * [offchain] cosmetic * [slash] Remove some logs in proposal * [webhook] Move webhook with call for when cannot commit block * [shard] Finally make finding subcommittee by shardID an explicit error * [node] Whitespace, prefer literal * [webhook] Report bad block to webhook * [slash] Expand verify, remove bad log usage, explicit error handle * [slash] Check on key size * [slash] Explicit upper bound of pending slashes * [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain * [multibls] Make max count allowed be 1/3 of external slots * [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee * [verify] Begin factor out of common verification approach * [project] Further remove RawJSON log, use proper epoch for snapshot * [slash] Implement verification * [slash] Implement BLS key verification of ballots * [rpc] Keep validator information as meaningful as possible * [staking] Never can stop being banned * [slash] Comments and default Unknown case of eligibility * [slash] Be explicit on what input values allowed when want to change EPOSStatus * [consensus] Remove unneeded TODO * [verify] Add proper error message * [rpc] Give back to caller their wrong chain id * [chain] Add extra map dump of delegation sizing for downstream analysis * [engine] Less code, more methods * [offchain] More leniency in handling slash bytes and delete from pending * [validator] Remove errors on bad input for edit
5 years ago
return nil, errors.Wrapf(
errValidatorExist, common2.MustAddressToBech32(msg.ValidatorAddress),
)
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
}
addrs, err := chainContext.ReadValidatorList()
if err != nil {
return nil, err
}
if err := checkDuplicateFields(
addrs, stateDB,
msg.ValidatorAddress,
msg.Identity,
msg.SlotPubKeys); err != nil {
return nil, err
}
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
if !CanTransfer(stateDB, msg.ValidatorAddress, msg.Amount) {
return nil, errInsufficientBalanceForStake
}
v, err := staking.CreateValidatorFromNewMsg(msg, blockNum, epoch)
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
if err != nil {
return nil, err
}
wrapper := &staking.ValidatorWrapper{}
wrapper.Validator = *v
wrapper.Delegations = []staking.Delegation{
staking.NewDelegation(v.Address, msg.Amount),
}
Merge master to t3 0523 (#3085) * fix signers share computation to use only non-harmony nodes (#3008) * Add LastEpochInCommittee at prestaking epoch too (#3014) * write validator stats at the end of prestaking epoch (#3021) * write validator stats at the end of prestaking epoch * also need to write block reward accumulator * [build] enable static build by default (#2885) scripts/go_executable_build.sh -s (static build, default) scripts/go_executable_build.sh -S (non-static build) Signed-off-by: Leo Chen <leo@harmony.one> * Code formatting fix based on `staticcheck` (#3023) * fix some static check issues * more fixes according to staticcheck * add nil check on getHeader * Add more log info * [rpc] Revert change to v2 GetBalance RPC (#3037) * Cherrypick crosslink related fixes from t3 to master. (#3045) * Optimize crosslink verification logic and add more cache (#3032) * make crosslink broadcast smarter and more efficient (#3036) * adjust catch up speed to be a sane number * Fix crosslink broadcast condition (#3041) * Tx pool error report fix & local build fix (#3046) * [tx_list] Add transaction after you can fetch tx cost * [tx_pool] Add error sink reports for removed tx * Correct log levels for known transactions * Remove redundant hash to `enqueue` and `add` signatures * Update tests for signature changes * [build] Fix local build for MacOs * [tx_pool] Correct typo & logging level for known tx * [config] Create Testnet config * [node.sh] update testnet network config * [config] correct testnet config * [main] Enable beacon sync for explorer nodes (#3059) * [apr] should not use snapshot from last epoch, use current epoch snapshot (#3051) * use snapshot total delegation instead of current total delegation * should not use snapshot from last epoch, use current epoch snapshot (#3049) Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com> Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com> * cherrypick log level change * [rpc] Add active status back to Validator information [validator] Fix typo for ValidatorRPCEnhanced [Makefile] Change make exe to build non-static by default * [network] Added caching to node server (#3048) * [network] Added caching to node server * [network] renamed two singleflight.Group * [network] added two error handling * [network] added forget logic for single flight * [network] changed single flight to lru cache solution * [network] adjusted the cache size * [network] changed cache size to 10000 * Transaction api improvements (#3057) * [node] Add option to broadcast invalid tx (on by default) * update main args to take in broadcast invalid tx * update node.sh for new broadcast invalid tx option * [rpc] Do not submit transaction if tx has been finalized * Support legacy behavior and not return errors on tx submission * [main] make broadcasting invalid tx default false * [node] Fix formatting * Add timeouts for the explorer HTTP service * [test] State trans unit test (#3043) * completing create-validator tests generalizing create validator tests, started with edit validator tests one round completed before major refactor * [test] finished coding test TestCheckDuplicateFields * [test] finished debug TestCheckDuplicateFields * [test] Added TestDescription_EnsureLength * [test] added test in validator_test.go TestUpdateDescription * [test] Added test TestComputed_String * [test] refactored TestValidatorSanityCheck and added some message in error message returned from sanity check * [test] removed test index for debugging * [test] Added test TestValidatorWrapper_SanityCheck. Added value check for TestMarshalUnmarshalValidator. Fixed rlp usage defect in UnmarshalValidator * [test] added test TestVerifyBLSKeys * [test] added test TestContainsHarmonyBLSKeys * [test] added test TestCreateValidatorFromNewMsg * [test] fixed some tests errors * [test] added test TestUpdateValidatorFromEditMsg, last editted the test file, finished testing validator_test.go * [staking] added numeric.Dec.Copy method * [staking] added CommissionRates.Copy * [staking] add nil case handle to copy method and related test case * [test] added nil case for commissionRates.Copy test * [staking] finished CreateValidator.Copy and related test case * [staking] added EditValidator.Copy method and related test case * [test] added zero test cases for Copy method * [staking] implemented Delegate.Copy and related unit test case * [staking] added Undelegate.Copy and CollectReward.Copy method. Implemented corresponding test cases * [test] added two more simple unit tests * [test] solving the merge conflict with deep copy fix * [test] added schedule for sharding * [test] refactored double_sign_test.go * [test] comment out all test cases for now. * [test] added record Copy method for testing * [test] added the first test case in TestVerify * [test] finished test TestVerify * [test] Added test TestVerify, TestApplySlashRate, TestSetDifference * [test] half through TestDelegatorSlashApply * [test] fix the problem cause by unsafe usage of big.Int * [staking] added the copy methods to validatorWrapper * [test] added test cases for copy method for staking/validatorWrapper * [test] added test case TestApply * [test] added test case TestRate * [test] fix the golint errors * [test] removed commented out unit test code * [test] remote the empty line in imports * [test] moved copy methods to stake_testing module * [test] removed usage of interface stateDB * [test] removed empty lines in imports * [test] fixed golint package name issues * [test] removed a todo when writing code * [test] moved record.Copy to staking/slash/test * [test] add some changes * [test] added prototypes in staketest module * [test] fix the golint issue in staketest * [test] make prototype more beautiful * [test] refactored test TestCheckDuplicateFields * [test] add createValidator test data * [test] added positive test case for VerifyAndCreateValidatorFromMsg * [test] added create validator test * [test] added the positive test case for TestVerifyAndEditValidatorFromMsg * [test] added the tests case TestVerifyAndEditValidatorFromMsg * [test] fix one of the error is not triggered as expected * [test] more changes in test * [test] fix the positive test data for TestVerifyAndDelegateFromMsg * [test] Fixed two comment golint errors * [test] added delegate tests * [test] added a make function. WIP * [test] added undelegate positive test * [test] added negative test cases for undelegate * [test] added positive test. not passed. Need to add assert method in test suites * [test] added equal function to check Validator Equality * [test] added equals for staketest * [test] replaced deep equal with new equal methods: * [test] removed unused codes * [test] Finishing touch * [test] fix comment golint issue * [test] removed dead code in staking_verifier_test.go Co-authored-by: Ganesha Upadhyaya <ganeshrvce@gmail.com> * changed the sync port to dns port in flag * [network] Limit client sync connection (#3071) * [network] limit client connected peers * Changed signature and added tests * Set secure permissions on generated blskey pass file in node.sh * Fix spellin gtypo in node.sh * fix latest param handling in api v1 * added comment for the isGreaterThanLatestBlockNum check * fix a nil pointer crash due to previous revert Signed-off-by: Leo Chen <leo@harmony.one> * fix nil pointer crash Signed-off-by: Leo Chen <leo@harmony.one> * fix nil pointer of block Signed-off-by: Leo Chen <leo@harmony.one> * [rpc] add viewID and epoch to RPCMarshalBlock (#3009) * [rpc] add viewID and epoch to RPCMarshalBlock * add apt-get update * Do not rollback current block on failed block sync (#3101) Co-authored-by: Ganesha Upadhyaya <ganeshrvce@gmail.com> Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com> Co-authored-by: Jacky Wang <jackyw.se@gmail.com> Co-authored-by: Janet Liang <56005637+janet-harmony@users.noreply.github.com> Co-authored-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu> Co-authored-by: Janet Liang <janet@harmony.one> Co-authored-by: Sebastian Johnsson <sebastian.johnsson@gmail.com> Co-authored-by: hsiung <hsiung@pinterest.com> Co-authored-by: Dennis Won <jhwon0820@gmail.com> Co-authored-by: Yishuang Chen <34232522+ivorytowerdds@users.noreply.github.com>
5 years ago
wrapper.Counters.NumBlocksSigned = big.NewInt(0)
wrapper.Counters.NumBlocksToSign = big.NewInt(0)
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
wrapper.BlockReward = big.NewInt(0)
if err := wrapper.SanityCheck(); err != nil {
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
return nil, err
}
return wrapper, nil
}
// VerifyAndEditValidatorFromMsg verifies the edit validator message using
// the stateDB, chainContext and returns the edited validatorWrapper.
//
// Note that this function never updates the stateDB, it only reads from stateDB.
func VerifyAndEditValidatorFromMsg(
[staking][validation][protocol] (#2396) * [staking][validation][protocol] Limit max bls keys * [staking-era] Fold banned and active into single field * [slash][effective] Remove LRU cache for slash, change .Active to enumeration * [slash] Remove leftover wrong usage of Logger * [slash][offchain] Only Decode if len > 0 * [offchain] cosmetic * [slash] Remove some logs in proposal * [webhook] Move webhook with call for when cannot commit block * [shard] Finally make finding subcommittee by shardID an explicit error * [node] Whitespace, prefer literal * [webhook] Report bad block to webhook * [slash] Expand verify, remove bad log usage, explicit error handle * [slash] Check on key size * [slash] Explicit upper bound of pending slashes * [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain * [multibls] Make max count allowed be 1/3 of external slots * [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee * [verify] Begin factor out of common verification approach * [project] Further remove RawJSON log, use proper epoch for snapshot * [slash] Implement verification * [slash] Implement BLS key verification of ballots * [rpc] Keep validator information as meaningful as possible * [staking] Never can stop being banned * [slash] Comments and default Unknown case of eligibility * [slash] Be explicit on what input values allowed when want to change EPOSStatus * [consensus] Remove unneeded TODO * [verify] Add proper error message * [rpc] Give back to caller their wrong chain id * [chain] Add extra map dump of delegation sizing for downstream analysis * [engine] Less code, more methods * [offchain] More leniency in handling slash bytes and delete from pending * [validator] Remove errors on bad input for edit
5 years ago
stateDB vm.StateDB, chainContext ChainContext,
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
epoch, blockNum *big.Int, msg *staking.EditValidator,
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
) (*staking.ValidatorWrapper, error) {
if stateDB == nil {
return nil, errStateDBIsMissing
}
if chainContext == nil {
return nil, errChainContextMissing
}
if blockNum == nil {
return nil, errBlockNumMissing
}
if !stateDB.IsValidator(msg.ValidatorAddress) {
return nil, errValidatorNotExist
}
newBlsKeys := []bls.SerializedPublicKey{}
if msg.SlotKeyToAdd != nil {
newBlsKeys = append(newBlsKeys, *msg.SlotKeyToAdd)
}
addrs, err := chainContext.ReadValidatorList()
if err != nil {
return nil, err
}
if err := checkDuplicateFields(
addrs, stateDB,
msg.ValidatorAddress,
msg.Identity,
newBlsKeys); err != nil {
return nil, err
}
Resolve harmony-one/bounties#90: Add revert mechanism for UpdateValidatorWrapper (#3939) * Add revert mechanism for UpdateValidatorWrapper Closes harmony-one/bounties#90 (1) Use LRU for ValidatorWrapper objects in stateDB to plug a potential memory leak (2) Merge ValidatorWrapper and ValidatorWrapperCopy to let callers ask for either a copy, or a pointer to the cached object. Additionally, give callers the option to not deep copy delegations (which is a heavy process). Copies need to be explicitly committed (and thus can be reverted), while the pointers are committed when Finalise is called. (3) Add a UpdateValidatorWrapperWithRevert function, which is used by staking txs `Delegate`, `Undelegate`, and `CollectRewards`. Other 2 types of staking txs and `db.Finalize` continue to use UpdateValidateWrapper without revert, again, to save memoery (4) Add unit tests which check a) Revert goes through b) Wrapper is as expected after revert c) State is as expected after revert * Change back to dictionary for stateValidators Since the memory / CPU usage saved is not significantly different when using an LRU + map structure, go back to the original dictionary structure to keep code easy to read and have limited modifications. * Add tests for validator wrapper reverts As requested by @rlan35, add tests beyond just adding and reverting a delegation. The tests are successive in the sense that we do multiple modifications to the wrapper, save a snapshot before each modification and revert to each of them to confirm everything works well. This change improves test coverage of statedb.go to 66.7% from 64.8% and that of core/state to 71.9% from 70.8%, and covers all the code that has been modified by this PR in statedb.go. For clarity, the modifications to the wrapper include (1) creation of wrapper in state, (2) adding a delegation to the wrapper, (3) increasing the blocks signed, and (4) a change in the validator Name and the BlockReward. Two additional tests have been added to cover the `panic` and the `GetCode` cases.
3 years ago
// request a copy, but delegations are not being changed so do not deep copy them
wrapper, err := stateDB.ValidatorWrapper(msg.ValidatorAddress, false, false)
[double-sign] Provide proof of double sign in slash record sent to beaconchain (#2253) * [double-sign] Commit changes in consensus needed for double-sign * [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain * [slash] Add quick iteration tool for testing double-signing * [slash] Add webhook example * [slash] Add http server for hook to trigger double sign behavior * [double-sign] Use bin/trigger-double-sign to cause a double-sign * [double-sign] Full feedback loop working * [slash] Thread through the slash records in the block proposal step * [slash] Compute the slashing rate * [double-sign] Generalize yaml malicious for many keys * [double-sign][slash] Modify data structures, verify via webhook handler * [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures * [slash] Apply to state slashing for double signing * [slash][double-sign] Checkpoint for working code that slashes on beaconchain * [slash] Keep track of the total slash and total reporters reward * [slash] Dump account state before and after the slash * [slash] Satisfy Travis * [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates * [slash] Capture the unique new delegations since snapshot as well * [slash] Filter undelegation by epoch of double sign * [slash] Add TODO of correctness needed in slash needs on off-chain data * [rpc] Fix closure issue on shardID * [slash] Add delegator to double-sign testing script * [slash] Expand crt-validator.sh with commenting printfs and make delegation * [slash] Finish track payment of leftover slash debt after undelegation runs out * [slash] Now be explicit about error wrt delegatorSlashApply * [slash] Capture specific sanity check on slash paidoff * [slash] Track slash from undelegation piecemeal * [slash][delegation] Named slice types, .String() * [slash] Do no RLP encode twice, once is enough * [slash] Remove special case of validators own delegation * [slash] Refactor approach to slash state application * [slash] Begin expanding out Verify * [slash] Slash on snapshot delegations, not current * [slash] Fix Epoch Cmp * [slash] Third iteration on slash logic * [slash] Use full slash amount * [slash] More log, whitespace * [slash] Remove Println, add log * [slash] Remove debug Println * [slash] Add record in unit test * [slash] Build Validator snapshot, current. Fill out slash record * [slash] Need to get RLP dump of a header to use in test * [slash] Factor out double sign test constants * [slash] Factor out common for validator, stub out slash application, finish out deserialization setup * [slash] Factor out data structure creation because of var lexical scoping * [slash] Seem to have pipeline of unit test e2e executing * [slash] Add expected snitch, slash amounts * [slash] Checkpoint * [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing * [config] add double-sign testnet config (#1) Signed-off-by: Leo Chen <leo@harmony.one> * [slash] Commit for as is code & data of current dump.json * [slash] Order of state operation not correct in test, hence bad results, thank you dlv * [slash] Add snapshot state dump * [slash] Pay off slash of validator own delegation correctly * [slash] Pay off slash debt with special case for min-self * [slash] Pass first scenario conclusively * [slash] 2% slash passes unit test for own delegation and external * [slash] Parameterize unit test to easily test .02 vs .80 slash * [slash] Handle own delegation correctly at 80% slash * [slash] Have 80% slash working with external delegator * [slash] Remove debug code from slash * [slash] Adjust Apply signature, test again for 2% slash * [slash] Factor out scenario in testing so can test 2% and 80% at same time * [slash] Correct balance deduction on plan delegation * [slash] Mock out ChainReader for TestVerify * [slash] Small surface area interface, now feedback loop for verify * [slash] Remove development json * [slash] trigger-double-sign consumes yaml * [slash] Remove dead code * [slash][test] Factor ValidatorWrapper into scenario * [slash][test] Add example from local-testing dump - caution might be off * [slash] Factor out mutation of slashDebt * [slash][test] Factor out tests so can easily load test-case from bytes * [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected * [slash] Satisfy Travis * [slash] Begin cleanup of PR * [slash] Apply slash from header to Finalize via state processor * [slash] Productionize code, Println => logs; adjust slash picked in newblock * [slash] Need pointer for rlp.Decode * [slash] ValidatorInformation use full wrapper * Fix median stake * [staking] Adjust MarshalJSON for Validator, Wrapper * Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279) * Refactor offchain data; Add epoch to ValidatorSnapshot * Make block onchain/offchain data commit atomically * [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment * [effective] VC eligible.go * [consensus] Redundant field in printf * [docker] import-ks for a dev account * [slash] Create BLS key for dockerfile and crt-validator.sh * [slash][docker] Easy deployment of double-sign testing * [docker] Have slash work as single docker command * [rpc] Fix median-stake RPC * [slash] Update webhook with default docker BLS key * [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go * [slash] Remove helper binary, commented out code, change to local config * [params] Factor out test genesis value * Add shard checking to Tx-Pool & correct blacklist (#2301) * [core] Fix blacklist & add shardID check * [staking + node + cmd] Fix blacklist & add shardID check * [slash] Adjust to PR comments part 1 * [docker] Use different throw away funded account * [docker] Create easier testing for delegation with private keys * [docker] Update yaml * [slash] Remove special case for slashing validator own delegation wrt min-self-delegate * [docker] Install nano as well * [slash] Early error if banned * [quorum] Expose earning account in decider marshal json * Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)" This reverts commit 9ffbf682c075b49188923c65a0bbf39ac188be00. * [slash] Add non-sanity check way to update validator * [reward] Increase percision on percentage in schedule * [slash] Adjust logs * [committee] Check eligibility of validator before doing sanity check * [slash] Update docker * [slash] Move create validator script to test * [slash] More log * [param] Make things faster * [slash][off-chain] Clear out slashes from pending in writeblockwithstate * [cross-link] Log is not error, just info * [blockchain] Not necessary to guard DeletePendingSlashingCandidates * [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that * [slash][test] Use faucet as sender, assume user imported * [slash] Test setup * [slash] reserve error for real error in logs * [slash][availability] Apply availability correct, bump signing count each block * [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed * [slash] Pay as much as can * [slash] use right nowAmt * [slash] Take away from rewards as well * [slash] iterate faster * [slash] Remove dev based timing * [slash] Add more log, sanity check incoming slash records, only count external for slash rate * [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters * [availability] More log * [slash] Simply pre slash erra slashing * [slash] Remove development code * [slash] Use height from recvMsg, todo on epoch * [staking] Not necessary to touch LastEpochInCommittee in staking_verifier * [slash] Undo ds in endpoint pattern config * [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment * [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count Co-authored-by: Leo Chen <leo@harmony.one> Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com> Co-authored-by: Rongjian Lan <rongjian@harmony.one> Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
5 years ago
if err != nil {
return nil, err
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
}
if err := staking.UpdateValidatorFromEditMsg(&wrapper.Validator, msg, epoch); err != nil {
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
return nil, err
}
newRate := wrapper.Validator.Rate
if newRate.GT(wrapper.Validator.MaxRate) {
return nil, errCommissionRateChangeTooHigh
}
minRate := availability.MinCommissionRate(
chainContext.Config().IsMinCommissionRate(epoch),
chainContext.Config().IsHIP30(epoch),
)
if newRate.LT(minRate) {
firstEpoch := stateDB.GetValidatorFirstElectionEpoch(msg.ValidatorAddress)
promoPeriod := chainContext.Config().MinCommissionPromoPeriod.Int64()
if firstEpoch.Uint64() != 0 && big.NewInt(0).Sub(epoch, firstEpoch).Int64() >= promoPeriod {
return nil,
errors.Errorf(
"%s %d%%",
errCommissionRateChangeTooLowT,
minRate.MulInt64(100).Int64(),
)
}
}
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
snapshotValidator, err := chainContext.ReadValidatorSnapshot(wrapper.Address)
if err != nil {
Merge master to t3 0523 (#3085) * fix signers share computation to use only non-harmony nodes (#3008) * Add LastEpochInCommittee at prestaking epoch too (#3014) * write validator stats at the end of prestaking epoch (#3021) * write validator stats at the end of prestaking epoch * also need to write block reward accumulator * [build] enable static build by default (#2885) scripts/go_executable_build.sh -s (static build, default) scripts/go_executable_build.sh -S (non-static build) Signed-off-by: Leo Chen <leo@harmony.one> * Code formatting fix based on `staticcheck` (#3023) * fix some static check issues * more fixes according to staticcheck * add nil check on getHeader * Add more log info * [rpc] Revert change to v2 GetBalance RPC (#3037) * Cherrypick crosslink related fixes from t3 to master. (#3045) * Optimize crosslink verification logic and add more cache (#3032) * make crosslink broadcast smarter and more efficient (#3036) * adjust catch up speed to be a sane number * Fix crosslink broadcast condition (#3041) * Tx pool error report fix & local build fix (#3046) * [tx_list] Add transaction after you can fetch tx cost * [tx_pool] Add error sink reports for removed tx * Correct log levels for known transactions * Remove redundant hash to `enqueue` and `add` signatures * Update tests for signature changes * [build] Fix local build for MacOs * [tx_pool] Correct typo & logging level for known tx * [config] Create Testnet config * [node.sh] update testnet network config * [config] correct testnet config * [main] Enable beacon sync for explorer nodes (#3059) * [apr] should not use snapshot from last epoch, use current epoch snapshot (#3051) * use snapshot total delegation instead of current total delegation * should not use snapshot from last epoch, use current epoch snapshot (#3049) Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com> Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com> * cherrypick log level change * [rpc] Add active status back to Validator information [validator] Fix typo for ValidatorRPCEnhanced [Makefile] Change make exe to build non-static by default * [network] Added caching to node server (#3048) * [network] Added caching to node server * [network] renamed two singleflight.Group * [network] added two error handling * [network] added forget logic for single flight * [network] changed single flight to lru cache solution * [network] adjusted the cache size * [network] changed cache size to 10000 * Transaction api improvements (#3057) * [node] Add option to broadcast invalid tx (on by default) * update main args to take in broadcast invalid tx * update node.sh for new broadcast invalid tx option * [rpc] Do not submit transaction if tx has been finalized * Support legacy behavior and not return errors on tx submission * [main] make broadcasting invalid tx default false * [node] Fix formatting * Add timeouts for the explorer HTTP service * [test] State trans unit test (#3043) * completing create-validator tests generalizing create validator tests, started with edit validator tests one round completed before major refactor * [test] finished coding test TestCheckDuplicateFields * [test] finished debug TestCheckDuplicateFields * [test] Added TestDescription_EnsureLength * [test] added test in validator_test.go TestUpdateDescription * [test] Added test TestComputed_String * [test] refactored TestValidatorSanityCheck and added some message in error message returned from sanity check * [test] removed test index for debugging * [test] Added test TestValidatorWrapper_SanityCheck. Added value check for TestMarshalUnmarshalValidator. Fixed rlp usage defect in UnmarshalValidator * [test] added test TestVerifyBLSKeys * [test] added test TestContainsHarmonyBLSKeys * [test] added test TestCreateValidatorFromNewMsg * [test] fixed some tests errors * [test] added test TestUpdateValidatorFromEditMsg, last editted the test file, finished testing validator_test.go * [staking] added numeric.Dec.Copy method * [staking] added CommissionRates.Copy * [staking] add nil case handle to copy method and related test case * [test] added nil case for commissionRates.Copy test * [staking] finished CreateValidator.Copy and related test case * [staking] added EditValidator.Copy method and related test case * [test] added zero test cases for Copy method * [staking] implemented Delegate.Copy and related unit test case * [staking] added Undelegate.Copy and CollectReward.Copy method. Implemented corresponding test cases * [test] added two more simple unit tests * [test] solving the merge conflict with deep copy fix * [test] added schedule for sharding * [test] refactored double_sign_test.go * [test] comment out all test cases for now. * [test] added record Copy method for testing * [test] added the first test case in TestVerify * [test] finished test TestVerify * [test] Added test TestVerify, TestApplySlashRate, TestSetDifference * [test] half through TestDelegatorSlashApply * [test] fix the problem cause by unsafe usage of big.Int * [staking] added the copy methods to validatorWrapper * [test] added test cases for copy method for staking/validatorWrapper * [test] added test case TestApply * [test] added test case TestRate * [test] fix the golint errors * [test] removed commented out unit test code * [test] remote the empty line in imports * [test] moved copy methods to stake_testing module * [test] removed usage of interface stateDB * [test] removed empty lines in imports * [test] fixed golint package name issues * [test] removed a todo when writing code * [test] moved record.Copy to staking/slash/test * [test] add some changes * [test] added prototypes in staketest module * [test] fix the golint issue in staketest * [test] make prototype more beautiful * [test] refactored test TestCheckDuplicateFields * [test] add createValidator test data * [test] added positive test case for VerifyAndCreateValidatorFromMsg * [test] added create validator test * [test] added the positive test case for TestVerifyAndEditValidatorFromMsg * [test] added the tests case TestVerifyAndEditValidatorFromMsg * [test] fix one of the error is not triggered as expected * [test] more changes in test * [test] fix the positive test data for TestVerifyAndDelegateFromMsg * [test] Fixed two comment golint errors * [test] added delegate tests * [test] added a make function. WIP * [test] added undelegate positive test * [test] added negative test cases for undelegate * [test] added positive test. not passed. Need to add assert method in test suites * [test] added equal function to check Validator Equality * [test] added equals for staketest * [test] replaced deep equal with new equal methods: * [test] removed unused codes * [test] Finishing touch * [test] fix comment golint issue * [test] removed dead code in staking_verifier_test.go Co-authored-by: Ganesha Upadhyaya <ganeshrvce@gmail.com> * changed the sync port to dns port in flag * [network] Limit client sync connection (#3071) * [network] limit client connected peers * Changed signature and added tests * Set secure permissions on generated blskey pass file in node.sh * Fix spellin gtypo in node.sh * fix latest param handling in api v1 * added comment for the isGreaterThanLatestBlockNum check * fix a nil pointer crash due to previous revert Signed-off-by: Leo Chen <leo@harmony.one> * fix nil pointer crash Signed-off-by: Leo Chen <leo@harmony.one> * fix nil pointer of block Signed-off-by: Leo Chen <leo@harmony.one> * [rpc] add viewID and epoch to RPCMarshalBlock (#3009) * [rpc] add viewID and epoch to RPCMarshalBlock * add apt-get update * Do not rollback current block on failed block sync (#3101) Co-authored-by: Ganesha Upadhyaya <ganeshrvce@gmail.com> Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com> Co-authored-by: Jacky Wang <jackyw.se@gmail.com> Co-authored-by: Janet Liang <56005637+janet-harmony@users.noreply.github.com> Co-authored-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu> Co-authored-by: Janet Liang <janet@harmony.one> Co-authored-by: Sebastian Johnsson <sebastian.johnsson@gmail.com> Co-authored-by: hsiung <hsiung@pinterest.com> Co-authored-by: Dennis Won <jhwon0820@gmail.com> Co-authored-by: Yishuang Chen <34232522+ivorytowerdds@users.noreply.github.com>
5 years ago
return nil, errors.WithMessage(err, "validator snapshot not found.")
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
}
rateAtBeginningOfEpoch := snapshotValidator.Validator.Rate
[rpc][availability][apr] Richer validator information, implement APR, unify EPoS computation, remove fall 2019 tech debt (#2484) * [rpc][validator] Extend hmy blockchain validator information * [availability] Optimize bump count * [staking][validator][rpc] Remove validator stats rpc, fold into validator information, make existing pattern default behavior * [slash] Reimplement SetDifference * [reward][engine][network] Remove bad API from fall, begin setup for Per validator awards * [header] Custom Marshal header for downstream, remove dev code * [effective][committee] Factor out EPoS round of computation thereby unification in codebase of EPoS * [unit-test] Fix semantically wrong validator unit tests, punt on maxBLS key wrt tx-pool test * [reward] Use excellent singleflight package for caching lookup of subcommittees * [apr][reward] Begin APR package itself, iterate on iterface signatures * [reward] Handle possible error from singleflight * [rpc][validator][reward] Adjust RPC committees, singleflight on votingPower, foldStats into Validator Information * [apr] Stub out computation of APR * [effective][committee] Upgrade SlotPurchase with named fields, provide marshal * [effective] Update Tests * [blockchain] TODO Remove the validators no longer in committee * [validator][effective] More expressive string representation of eligibilty, ValidatorRPC explicit say if in committee now * [rpc] Median-stake more semantic meaningful * [validator] Iterate on semantic meaning of JSON representation * [offchain] Make validator stats return explicit error * [availability] Small typo * [rpc] Quick visual hack until fix delete out kicked out validators * [offchain] Delete validator from offchain that lost their slot * [apr] Forgot to update interface signature * [apr] Mul instead of Div * [protocol][validator] Fold block reward accum per vaidator into validator-wrapper, off-chain => on-chain * [votepower] Refactor votepower Roster, simplify aggregation of network wide rosters * [votepower][shard] Adjust roster, optimize usage of BLSPublicKey as key, use MarshalText trick * [shard] Granular errors * [votepower][validator] Unify votepower data structure with off-chain usage * [votepower][consensus][validator] Further simplify and unify votepower with off-chain, validator stats * [votepower] Use RJs naming convention group,overall * [votepower] Remove Println, do keep enforcing order * [effective][reward] Expand semantics of eligibility as it was overloaded and confusing, evict old voting power computations * [apr] Adjust json field name * [votepower] Only aggregate on external validator * [votepower] Mistake on aggregation, custom presentation network-wide * [rpc][validator][availability] Remove parameter, take into account empty snapshot * [apr] Use snapshots from two, one epochs ago. Still have question on header * [apr] Use GetHeaderByNumber for the header needed for time stamp * [chain] Evict > 3 epoch old voting power * [blockchain] Leave Delete Validator snapshot as TODO * [validator][rpc][effective] Undo changes to Protocol field, use virtual construct at RPC layer for meaning * [project] Address PR comments * [committee][rpc] Move +1 to computation of epos round rather than hack mutation * [reward] Remove entire unnecessary loop, hook on AddReward. Remove unnecessary new big int * [votepower][rpc][validator] Stick with numeric.Dec for token involved with computation, expose accumulate block-reward in RPC * [effective][committee] Track the candidates for the EPoS auction, RPC median-stake benefits * [node] Add hack way to get real error reason of why cannot load shardchain * [consensus] Expand log on current issue on nil block * [apr] Do the actual call to compute for validator's APR * [committee] Wrap SlotOrder with validator address, manifests in median-stake RPC * [apr] Incorrect error handle order * [quorum] Remove incorrect compare on bls Key, (typo), remove redundant error check * [shard] Add log if stakedSlots is 0 * [apr] More sanity check on div by zero, more lenient on error when dont have historical data yet * [committee] Remove + 1 on seat count * [apr] Use int64() directly * [apr] Log when odd empty nil header * [apr] Do not crash on empty header, figure out later
5 years ago
if rateAtBeginningOfEpoch.IsNil() ||
(!newRate.IsNil() && !rateAtBeginningOfEpoch.Equal(newRate)) {
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
wrapper.Validator.UpdateHeight = blockNum
}
[staking][validation][protocol] (#2396) * [staking][validation][protocol] Limit max bls keys * [staking-era] Fold banned and active into single field * [slash][effective] Remove LRU cache for slash, change .Active to enumeration * [slash] Remove leftover wrong usage of Logger * [slash][offchain] Only Decode if len > 0 * [offchain] cosmetic * [slash] Remove some logs in proposal * [webhook] Move webhook with call for when cannot commit block * [shard] Finally make finding subcommittee by shardID an explicit error * [node] Whitespace, prefer literal * [webhook] Report bad block to webhook * [slash] Expand verify, remove bad log usage, explicit error handle * [slash] Check on key size * [slash] Explicit upper bound of pending slashes * [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain * [multibls] Make max count allowed be 1/3 of external slots * [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee * [verify] Begin factor out of common verification approach * [project] Further remove RawJSON log, use proper epoch for snapshot * [slash] Implement verification * [slash] Implement BLS key verification of ballots * [rpc] Keep validator information as meaningful as possible * [staking] Never can stop being banned * [slash] Comments and default Unknown case of eligibility * [slash] Be explicit on what input values allowed when want to change EPOSStatus * [consensus] Remove unneeded TODO * [verify] Add proper error message * [rpc] Give back to caller their wrong chain id * [chain] Add extra map dump of delegation sizing for downstream analysis * [engine] Less code, more methods * [offchain] More leniency in handling slash bytes and delete from pending * [validator] Remove errors on bad input for edit
5 years ago
if newRate.Sub(rateAtBeginningOfEpoch).Abs().GT(
wrapper.Validator.MaxChangeRate,
) {
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
return nil, errCommissionRateChangeTooFast
}
if err := wrapper.SanityCheck(); err != nil {
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
return nil, err
}
return wrapper, nil
}
const oneThousand = 1000
const oneHundred = 100
var (
oneAsBigInt = big.NewInt(denominations.One)
minimumDelegation = new(big.Int).Mul(oneAsBigInt, big.NewInt(oneThousand))
minimumDelegationV2 = new(big.Int).Mul(oneAsBigInt, big.NewInt(oneHundred))
errDelegationTooSmall = errors.New("minimum delegation amount for a delegator has to be greater than or equal to 1000 ONE")
errDelegationTooSmallV2 = errors.New("minimum delegation amount for a delegator has to be greater than or equal to 100 ONE")
)
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
// VerifyAndDelegateFromMsg verifies the delegate message using the stateDB
// and returns the balance to be deducted by the delegator as well as the
// validatorWrapper with the delegation applied to it.
//
// Note that this function never updates the stateDB, it only reads from stateDB.
func VerifyAndDelegateFromMsg(
stateDB vm.StateDB, epoch *big.Int, msg *staking.Delegate, delegations []staking.DelegationIndex, chainConfig *params.ChainConfig,
) ([]*staking.ValidatorWrapper, *big.Int, map[common.Address]*big.Int, error) {
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
if stateDB == nil {
return nil, nil, nil, errStateDBIsMissing
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
}
if !stateDB.IsValidator(msg.ValidatorAddress) {
return nil, nil, nil, errValidatorNotExist
}
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
if msg.Amount.Sign() == -1 {
return nil, nil, nil, errNegativeAmount
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
}
if msg.Amount.Cmp(minimumDelegation) < 0 {
if chainConfig.IsMinDelegation100(epoch) {
if msg.Amount.Cmp(minimumDelegationV2) < 0 {
return nil, nil, nil, errDelegationTooSmallV2
}
} else {
return nil, nil, nil, errDelegationTooSmall
}
}
updatedValidatorWrappers := []*staking.ValidatorWrapper{}
delegateBalance := big.NewInt(0).Set(msg.Amount)
fromLockedTokens := map[common.Address]*big.Int{}
var delegateeWrapper *staking.ValidatorWrapper
if chainConfig.IsRedelegation(epoch) {
// Check if we can use tokens in undelegation to delegate (redelegate)
for i := range delegations {
delegationIndex := &delegations[i]
Resolve harmony-one/bounties#90: Add revert mechanism for UpdateValidatorWrapper (#3939) * Add revert mechanism for UpdateValidatorWrapper Closes harmony-one/bounties#90 (1) Use LRU for ValidatorWrapper objects in stateDB to plug a potential memory leak (2) Merge ValidatorWrapper and ValidatorWrapperCopy to let callers ask for either a copy, or a pointer to the cached object. Additionally, give callers the option to not deep copy delegations (which is a heavy process). Copies need to be explicitly committed (and thus can be reverted), while the pointers are committed when Finalise is called. (3) Add a UpdateValidatorWrapperWithRevert function, which is used by staking txs `Delegate`, `Undelegate`, and `CollectRewards`. Other 2 types of staking txs and `db.Finalize` continue to use UpdateValidateWrapper without revert, again, to save memoery (4) Add unit tests which check a) Revert goes through b) Wrapper is as expected after revert c) State is as expected after revert * Change back to dictionary for stateValidators Since the memory / CPU usage saved is not significantly different when using an LRU + map structure, go back to the original dictionary structure to keep code easy to read and have limited modifications. * Add tests for validator wrapper reverts As requested by @rlan35, add tests beyond just adding and reverting a delegation. The tests are successive in the sense that we do multiple modifications to the wrapper, save a snapshot before each modification and revert to each of them to confirm everything works well. This change improves test coverage of statedb.go to 66.7% from 64.8% and that of core/state to 71.9% from 70.8%, and covers all the code that has been modified by this PR in statedb.go. For clarity, the modifications to the wrapper include (1) creation of wrapper in state, (2) adding a delegation to the wrapper, (3) increasing the blocks signed, and (4) a change in the validator Name and the BlockReward. Two additional tests have been added to cover the `panic` and the `GetCode` cases.
3 years ago
// request a copy, and since delegations will be changed, copy them too
wrapper, err := stateDB.ValidatorWrapper(delegationIndex.ValidatorAddress, false, true)
if err != nil {
return nil, nil, nil, err
}
if uint64(len(wrapper.Delegations)) <= delegationIndex.Index {
utils.Logger().Warn().
Str("validator", delegationIndex.ValidatorAddress.String()).
Uint64("delegation index", delegationIndex.Index).
Int("delegations length", len(wrapper.Delegations)).
Msg("Delegation index out of bound")
return nil, nil, nil, errors.New("Delegation index out of bound")
}
delegation := &wrapper.Delegations[delegationIndex.Index]
startBalance := big.NewInt(0).Set(delegateBalance)
// Start from the oldest undelegated tokens
curIndex := 0
for ; curIndex < len(delegation.Undelegations); curIndex++ {
if delegation.Undelegations[curIndex].Epoch.Cmp(epoch) >= 0 {
break
}
if delegation.Undelegations[curIndex].Amount.Cmp(delegateBalance) <= 0 {
delegateBalance.Sub(delegateBalance, delegation.Undelegations[curIndex].Amount)
} else {
delegation.Undelegations[curIndex].Amount.Sub(
delegation.Undelegations[curIndex].Amount, delegateBalance,
)
delegateBalance = big.NewInt(0)
break
}
}
if startBalance.Cmp(delegateBalance) > 0 {
// Used undelegated token for redelegation
delegation.Undelegations = delegation.Undelegations[curIndex:]
if err := wrapper.SanityCheck(); err != nil {
return nil, nil, nil, err
}
if bytes.Equal(delegationIndex.ValidatorAddress[:], msg.ValidatorAddress[:]) {
delegateeWrapper = wrapper
}
updatedValidatorWrappers = append(updatedValidatorWrappers, wrapper)
fromLockedTokens[delegationIndex.ValidatorAddress] = big.NewInt(0).Sub(startBalance, delegateBalance)
}
}
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
}
if delegateeWrapper == nil {
var err error
Resolve harmony-one/bounties#90: Add revert mechanism for UpdateValidatorWrapper (#3939) * Add revert mechanism for UpdateValidatorWrapper Closes harmony-one/bounties#90 (1) Use LRU for ValidatorWrapper objects in stateDB to plug a potential memory leak (2) Merge ValidatorWrapper and ValidatorWrapperCopy to let callers ask for either a copy, or a pointer to the cached object. Additionally, give callers the option to not deep copy delegations (which is a heavy process). Copies need to be explicitly committed (and thus can be reverted), while the pointers are committed when Finalise is called. (3) Add a UpdateValidatorWrapperWithRevert function, which is used by staking txs `Delegate`, `Undelegate`, and `CollectRewards`. Other 2 types of staking txs and `db.Finalize` continue to use UpdateValidateWrapper without revert, again, to save memoery (4) Add unit tests which check a) Revert goes through b) Wrapper is as expected after revert c) State is as expected after revert * Change back to dictionary for stateValidators Since the memory / CPU usage saved is not significantly different when using an LRU + map structure, go back to the original dictionary structure to keep code easy to read and have limited modifications. * Add tests for validator wrapper reverts As requested by @rlan35, add tests beyond just adding and reverting a delegation. The tests are successive in the sense that we do multiple modifications to the wrapper, save a snapshot before each modification and revert to each of them to confirm everything works well. This change improves test coverage of statedb.go to 66.7% from 64.8% and that of core/state to 71.9% from 70.8%, and covers all the code that has been modified by this PR in statedb.go. For clarity, the modifications to the wrapper include (1) creation of wrapper in state, (2) adding a delegation to the wrapper, (3) increasing the blocks signed, and (4) a change in the validator Name and the BlockReward. Two additional tests have been added to cover the `panic` and the `GetCode` cases.
3 years ago
// request a copy, and since delegations will be changed, copy them too
delegateeWrapper, err = stateDB.ValidatorWrapper(msg.ValidatorAddress, false, true)
if err != nil {
return nil, nil, nil, err
}
updatedValidatorWrappers = append(updatedValidatorWrappers, delegateeWrapper)
}
// Add to existing delegation if any
found := false
for i := range delegateeWrapper.Delegations {
delegation := &delegateeWrapper.Delegations[i]
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
if bytes.Equal(delegation.DelegatorAddress.Bytes(), msg.DelegatorAddress.Bytes()) {
delegation.Amount.Add(delegation.Amount, msg.Amount)
if err := delegateeWrapper.SanityCheck(); err != nil {
return nil, nil, nil, err
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
}
found = true
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
}
}
if !found {
// Add new delegation
delegateeWrapper.Delegations = append(
delegateeWrapper.Delegations, staking.NewDelegation(
msg.DelegatorAddress, msg.Amount,
),
)
if err := delegateeWrapper.SanityCheck(); err != nil {
return nil, nil, nil, err
}
}
if delegateBalance.Cmp(big.NewInt(0)) == 0 {
// delegation fully from undelegated tokens, no need to deduct from balance.
return updatedValidatorWrappers, big.NewInt(0), fromLockedTokens, nil
}
// Still need to deduct tokens from balance for delegation
// Check if there is enough liquid token to delegate
if !CanTransfer(stateDB, msg.DelegatorAddress, delegateBalance) {
return nil, nil, nil, errors.Wrapf(
errInsufficientBalanceForStake, "totalRedelegatable: %v, balance: %v; trying to stake %v",
big.NewInt(0).Sub(msg.Amount, delegateBalance), stateDB.GetBalance(msg.DelegatorAddress), msg.Amount)
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
}
return updatedValidatorWrappers, delegateBalance, fromLockedTokens, nil
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
}
// VerifyAndUndelegateFromMsg verifies the undelegate validator message
// using the stateDB & chainContext and returns the edited validatorWrapper
// with the undelegation applied to it.
//
// Note that this function never updates the stateDB, it only reads from stateDB.
func VerifyAndUndelegateFromMsg(
stateDB vm.StateDB, epoch *big.Int, msg *staking.Undelegate,
) (*staking.ValidatorWrapper, error) {
if stateDB == nil {
return nil, errStateDBIsMissing
}
if epoch == nil {
return nil, errEpochMissing
}
[double-sign] Provide proof of double sign in slash record sent to beaconchain (#2253) * [double-sign] Commit changes in consensus needed for double-sign * [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain * [slash] Add quick iteration tool for testing double-signing * [slash] Add webhook example * [slash] Add http server for hook to trigger double sign behavior * [double-sign] Use bin/trigger-double-sign to cause a double-sign * [double-sign] Full feedback loop working * [slash] Thread through the slash records in the block proposal step * [slash] Compute the slashing rate * [double-sign] Generalize yaml malicious for many keys * [double-sign][slash] Modify data structures, verify via webhook handler * [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures * [slash] Apply to state slashing for double signing * [slash][double-sign] Checkpoint for working code that slashes on beaconchain * [slash] Keep track of the total slash and total reporters reward * [slash] Dump account state before and after the slash * [slash] Satisfy Travis * [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates * [slash] Capture the unique new delegations since snapshot as well * [slash] Filter undelegation by epoch of double sign * [slash] Add TODO of correctness needed in slash needs on off-chain data * [rpc] Fix closure issue on shardID * [slash] Add delegator to double-sign testing script * [slash] Expand crt-validator.sh with commenting printfs and make delegation * [slash] Finish track payment of leftover slash debt after undelegation runs out * [slash] Now be explicit about error wrt delegatorSlashApply * [slash] Capture specific sanity check on slash paidoff * [slash] Track slash from undelegation piecemeal * [slash][delegation] Named slice types, .String() * [slash] Do no RLP encode twice, once is enough * [slash] Remove special case of validators own delegation * [slash] Refactor approach to slash state application * [slash] Begin expanding out Verify * [slash] Slash on snapshot delegations, not current * [slash] Fix Epoch Cmp * [slash] Third iteration on slash logic * [slash] Use full slash amount * [slash] More log, whitespace * [slash] Remove Println, add log * [slash] Remove debug Println * [slash] Add record in unit test * [slash] Build Validator snapshot, current. Fill out slash record * [slash] Need to get RLP dump of a header to use in test * [slash] Factor out double sign test constants * [slash] Factor out common for validator, stub out slash application, finish out deserialization setup * [slash] Factor out data structure creation because of var lexical scoping * [slash] Seem to have pipeline of unit test e2e executing * [slash] Add expected snitch, slash amounts * [slash] Checkpoint * [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing * [config] add double-sign testnet config (#1) Signed-off-by: Leo Chen <leo@harmony.one> * [slash] Commit for as is code & data of current dump.json * [slash] Order of state operation not correct in test, hence bad results, thank you dlv * [slash] Add snapshot state dump * [slash] Pay off slash of validator own delegation correctly * [slash] Pay off slash debt with special case for min-self * [slash] Pass first scenario conclusively * [slash] 2% slash passes unit test for own delegation and external * [slash] Parameterize unit test to easily test .02 vs .80 slash * [slash] Handle own delegation correctly at 80% slash * [slash] Have 80% slash working with external delegator * [slash] Remove debug code from slash * [slash] Adjust Apply signature, test again for 2% slash * [slash] Factor out scenario in testing so can test 2% and 80% at same time * [slash] Correct balance deduction on plan delegation * [slash] Mock out ChainReader for TestVerify * [slash] Small surface area interface, now feedback loop for verify * [slash] Remove development json * [slash] trigger-double-sign consumes yaml * [slash] Remove dead code * [slash][test] Factor ValidatorWrapper into scenario * [slash][test] Add example from local-testing dump - caution might be off * [slash] Factor out mutation of slashDebt * [slash][test] Factor out tests so can easily load test-case from bytes * [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected * [slash] Satisfy Travis * [slash] Begin cleanup of PR * [slash] Apply slash from header to Finalize via state processor * [slash] Productionize code, Println => logs; adjust slash picked in newblock * [slash] Need pointer for rlp.Decode * [slash] ValidatorInformation use full wrapper * Fix median stake * [staking] Adjust MarshalJSON for Validator, Wrapper * Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279) * Refactor offchain data; Add epoch to ValidatorSnapshot * Make block onchain/offchain data commit atomically * [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment * [effective] VC eligible.go * [consensus] Redundant field in printf * [docker] import-ks for a dev account * [slash] Create BLS key for dockerfile and crt-validator.sh * [slash][docker] Easy deployment of double-sign testing * [docker] Have slash work as single docker command * [rpc] Fix median-stake RPC * [slash] Update webhook with default docker BLS key * [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go * [slash] Remove helper binary, commented out code, change to local config * [params] Factor out test genesis value * Add shard checking to Tx-Pool & correct blacklist (#2301) * [core] Fix blacklist & add shardID check * [staking + node + cmd] Fix blacklist & add shardID check * [slash] Adjust to PR comments part 1 * [docker] Use different throw away funded account * [docker] Create easier testing for delegation with private keys * [docker] Update yaml * [slash] Remove special case for slashing validator own delegation wrt min-self-delegate * [docker] Install nano as well * [slash] Early error if banned * [quorum] Expose earning account in decider marshal json * Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)" This reverts commit 9ffbf682c075b49188923c65a0bbf39ac188be00. * [slash] Add non-sanity check way to update validator * [reward] Increase percision on percentage in schedule * [slash] Adjust logs * [committee] Check eligibility of validator before doing sanity check * [slash] Update docker * [slash] Move create validator script to test * [slash] More log * [param] Make things faster * [slash][off-chain] Clear out slashes from pending in writeblockwithstate * [cross-link] Log is not error, just info * [blockchain] Not necessary to guard DeletePendingSlashingCandidates * [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that * [slash][test] Use faucet as sender, assume user imported * [slash] Test setup * [slash] reserve error for real error in logs * [slash][availability] Apply availability correct, bump signing count each block * [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed * [slash] Pay as much as can * [slash] use right nowAmt * [slash] Take away from rewards as well * [slash] iterate faster * [slash] Remove dev based timing * [slash] Add more log, sanity check incoming slash records, only count external for slash rate * [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters * [availability] More log * [slash] Simply pre slash erra slashing * [slash] Remove development code * [slash] Use height from recvMsg, todo on epoch * [staking] Not necessary to touch LastEpochInCommittee in staking_verifier * [slash] Undo ds in endpoint pattern config * [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment * [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count Co-authored-by: Leo Chen <leo@harmony.one> Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com> Co-authored-by: Rongjian Lan <rongjian@harmony.one> Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
5 years ago
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
if msg.Amount.Sign() == -1 {
return nil, errNegativeAmount
}
[double-sign] Provide proof of double sign in slash record sent to beaconchain (#2253) * [double-sign] Commit changes in consensus needed for double-sign * [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain * [slash] Add quick iteration tool for testing double-signing * [slash] Add webhook example * [slash] Add http server for hook to trigger double sign behavior * [double-sign] Use bin/trigger-double-sign to cause a double-sign * [double-sign] Full feedback loop working * [slash] Thread through the slash records in the block proposal step * [slash] Compute the slashing rate * [double-sign] Generalize yaml malicious for many keys * [double-sign][slash] Modify data structures, verify via webhook handler * [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures * [slash] Apply to state slashing for double signing * [slash][double-sign] Checkpoint for working code that slashes on beaconchain * [slash] Keep track of the total slash and total reporters reward * [slash] Dump account state before and after the slash * [slash] Satisfy Travis * [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates * [slash] Capture the unique new delegations since snapshot as well * [slash] Filter undelegation by epoch of double sign * [slash] Add TODO of correctness needed in slash needs on off-chain data * [rpc] Fix closure issue on shardID * [slash] Add delegator to double-sign testing script * [slash] Expand crt-validator.sh with commenting printfs and make delegation * [slash] Finish track payment of leftover slash debt after undelegation runs out * [slash] Now be explicit about error wrt delegatorSlashApply * [slash] Capture specific sanity check on slash paidoff * [slash] Track slash from undelegation piecemeal * [slash][delegation] Named slice types, .String() * [slash] Do no RLP encode twice, once is enough * [slash] Remove special case of validators own delegation * [slash] Refactor approach to slash state application * [slash] Begin expanding out Verify * [slash] Slash on snapshot delegations, not current * [slash] Fix Epoch Cmp * [slash] Third iteration on slash logic * [slash] Use full slash amount * [slash] More log, whitespace * [slash] Remove Println, add log * [slash] Remove debug Println * [slash] Add record in unit test * [slash] Build Validator snapshot, current. Fill out slash record * [slash] Need to get RLP dump of a header to use in test * [slash] Factor out double sign test constants * [slash] Factor out common for validator, stub out slash application, finish out deserialization setup * [slash] Factor out data structure creation because of var lexical scoping * [slash] Seem to have pipeline of unit test e2e executing * [slash] Add expected snitch, slash amounts * [slash] Checkpoint * [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing * [config] add double-sign testnet config (#1) Signed-off-by: Leo Chen <leo@harmony.one> * [slash] Commit for as is code & data of current dump.json * [slash] Order of state operation not correct in test, hence bad results, thank you dlv * [slash] Add snapshot state dump * [slash] Pay off slash of validator own delegation correctly * [slash] Pay off slash debt with special case for min-self * [slash] Pass first scenario conclusively * [slash] 2% slash passes unit test for own delegation and external * [slash] Parameterize unit test to easily test .02 vs .80 slash * [slash] Handle own delegation correctly at 80% slash * [slash] Have 80% slash working with external delegator * [slash] Remove debug code from slash * [slash] Adjust Apply signature, test again for 2% slash * [slash] Factor out scenario in testing so can test 2% and 80% at same time * [slash] Correct balance deduction on plan delegation * [slash] Mock out ChainReader for TestVerify * [slash] Small surface area interface, now feedback loop for verify * [slash] Remove development json * [slash] trigger-double-sign consumes yaml * [slash] Remove dead code * [slash][test] Factor ValidatorWrapper into scenario * [slash][test] Add example from local-testing dump - caution might be off * [slash] Factor out mutation of slashDebt * [slash][test] Factor out tests so can easily load test-case from bytes * [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected * [slash] Satisfy Travis * [slash] Begin cleanup of PR * [slash] Apply slash from header to Finalize via state processor * [slash] Productionize code, Println => logs; adjust slash picked in newblock * [slash] Need pointer for rlp.Decode * [slash] ValidatorInformation use full wrapper * Fix median stake * [staking] Adjust MarshalJSON for Validator, Wrapper * Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279) * Refactor offchain data; Add epoch to ValidatorSnapshot * Make block onchain/offchain data commit atomically * [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment * [effective] VC eligible.go * [consensus] Redundant field in printf * [docker] import-ks for a dev account * [slash] Create BLS key for dockerfile and crt-validator.sh * [slash][docker] Easy deployment of double-sign testing * [docker] Have slash work as single docker command * [rpc] Fix median-stake RPC * [slash] Update webhook with default docker BLS key * [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go * [slash] Remove helper binary, commented out code, change to local config * [params] Factor out test genesis value * Add shard checking to Tx-Pool & correct blacklist (#2301) * [core] Fix blacklist & add shardID check * [staking + node + cmd] Fix blacklist & add shardID check * [slash] Adjust to PR comments part 1 * [docker] Use different throw away funded account * [docker] Create easier testing for delegation with private keys * [docker] Update yaml * [slash] Remove special case for slashing validator own delegation wrt min-self-delegate * [docker] Install nano as well * [slash] Early error if banned * [quorum] Expose earning account in decider marshal json * Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)" This reverts commit 9ffbf682c075b49188923c65a0bbf39ac188be00. * [slash] Add non-sanity check way to update validator * [reward] Increase percision on percentage in schedule * [slash] Adjust logs * [committee] Check eligibility of validator before doing sanity check * [slash] Update docker * [slash] Move create validator script to test * [slash] More log * [param] Make things faster * [slash][off-chain] Clear out slashes from pending in writeblockwithstate * [cross-link] Log is not error, just info * [blockchain] Not necessary to guard DeletePendingSlashingCandidates * [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that * [slash][test] Use faucet as sender, assume user imported * [slash] Test setup * [slash] reserve error for real error in logs * [slash][availability] Apply availability correct, bump signing count each block * [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed * [slash] Pay as much as can * [slash] use right nowAmt * [slash] Take away from rewards as well * [slash] iterate faster * [slash] Remove dev based timing * [slash] Add more log, sanity check incoming slash records, only count external for slash rate * [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters * [availability] More log * [slash] Simply pre slash erra slashing * [slash] Remove development code * [slash] Use height from recvMsg, todo on epoch * [staking] Not necessary to touch LastEpochInCommittee in staking_verifier * [slash] Undo ds in endpoint pattern config * [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment * [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count Co-authored-by: Leo Chen <leo@harmony.one> Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com> Co-authored-by: Rongjian Lan <rongjian@harmony.one> Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
5 years ago
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
if !stateDB.IsValidator(msg.ValidatorAddress) {
return nil, errValidatorNotExist
}
[double-sign] Provide proof of double sign in slash record sent to beaconchain (#2253) * [double-sign] Commit changes in consensus needed for double-sign * [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain * [slash] Add quick iteration tool for testing double-signing * [slash] Add webhook example * [slash] Add http server for hook to trigger double sign behavior * [double-sign] Use bin/trigger-double-sign to cause a double-sign * [double-sign] Full feedback loop working * [slash] Thread through the slash records in the block proposal step * [slash] Compute the slashing rate * [double-sign] Generalize yaml malicious for many keys * [double-sign][slash] Modify data structures, verify via webhook handler * [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures * [slash] Apply to state slashing for double signing * [slash][double-sign] Checkpoint for working code that slashes on beaconchain * [slash] Keep track of the total slash and total reporters reward * [slash] Dump account state before and after the slash * [slash] Satisfy Travis * [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates * [slash] Capture the unique new delegations since snapshot as well * [slash] Filter undelegation by epoch of double sign * [slash] Add TODO of correctness needed in slash needs on off-chain data * [rpc] Fix closure issue on shardID * [slash] Add delegator to double-sign testing script * [slash] Expand crt-validator.sh with commenting printfs and make delegation * [slash] Finish track payment of leftover slash debt after undelegation runs out * [slash] Now be explicit about error wrt delegatorSlashApply * [slash] Capture specific sanity check on slash paidoff * [slash] Track slash from undelegation piecemeal * [slash][delegation] Named slice types, .String() * [slash] Do no RLP encode twice, once is enough * [slash] Remove special case of validators own delegation * [slash] Refactor approach to slash state application * [slash] Begin expanding out Verify * [slash] Slash on snapshot delegations, not current * [slash] Fix Epoch Cmp * [slash] Third iteration on slash logic * [slash] Use full slash amount * [slash] More log, whitespace * [slash] Remove Println, add log * [slash] Remove debug Println * [slash] Add record in unit test * [slash] Build Validator snapshot, current. Fill out slash record * [slash] Need to get RLP dump of a header to use in test * [slash] Factor out double sign test constants * [slash] Factor out common for validator, stub out slash application, finish out deserialization setup * [slash] Factor out data structure creation because of var lexical scoping * [slash] Seem to have pipeline of unit test e2e executing * [slash] Add expected snitch, slash amounts * [slash] Checkpoint * [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing * [config] add double-sign testnet config (#1) Signed-off-by: Leo Chen <leo@harmony.one> * [slash] Commit for as is code & data of current dump.json * [slash] Order of state operation not correct in test, hence bad results, thank you dlv * [slash] Add snapshot state dump * [slash] Pay off slash of validator own delegation correctly * [slash] Pay off slash debt with special case for min-self * [slash] Pass first scenario conclusively * [slash] 2% slash passes unit test for own delegation and external * [slash] Parameterize unit test to easily test .02 vs .80 slash * [slash] Handle own delegation correctly at 80% slash * [slash] Have 80% slash working with external delegator * [slash] Remove debug code from slash * [slash] Adjust Apply signature, test again for 2% slash * [slash] Factor out scenario in testing so can test 2% and 80% at same time * [slash] Correct balance deduction on plan delegation * [slash] Mock out ChainReader for TestVerify * [slash] Small surface area interface, now feedback loop for verify * [slash] Remove development json * [slash] trigger-double-sign consumes yaml * [slash] Remove dead code * [slash][test] Factor ValidatorWrapper into scenario * [slash][test] Add example from local-testing dump - caution might be off * [slash] Factor out mutation of slashDebt * [slash][test] Factor out tests so can easily load test-case from bytes * [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected * [slash] Satisfy Travis * [slash] Begin cleanup of PR * [slash] Apply slash from header to Finalize via state processor * [slash] Productionize code, Println => logs; adjust slash picked in newblock * [slash] Need pointer for rlp.Decode * [slash] ValidatorInformation use full wrapper * Fix median stake * [staking] Adjust MarshalJSON for Validator, Wrapper * Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279) * Refactor offchain data; Add epoch to ValidatorSnapshot * Make block onchain/offchain data commit atomically * [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment * [effective] VC eligible.go * [consensus] Redundant field in printf * [docker] import-ks for a dev account * [slash] Create BLS key for dockerfile and crt-validator.sh * [slash][docker] Easy deployment of double-sign testing * [docker] Have slash work as single docker command * [rpc] Fix median-stake RPC * [slash] Update webhook with default docker BLS key * [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go * [slash] Remove helper binary, commented out code, change to local config * [params] Factor out test genesis value * Add shard checking to Tx-Pool & correct blacklist (#2301) * [core] Fix blacklist & add shardID check * [staking + node + cmd] Fix blacklist & add shardID check * [slash] Adjust to PR comments part 1 * [docker] Use different throw away funded account * [docker] Create easier testing for delegation with private keys * [docker] Update yaml * [slash] Remove special case for slashing validator own delegation wrt min-self-delegate * [docker] Install nano as well * [slash] Early error if banned * [quorum] Expose earning account in decider marshal json * Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)" This reverts commit 9ffbf682c075b49188923c65a0bbf39ac188be00. * [slash] Add non-sanity check way to update validator * [reward] Increase percision on percentage in schedule * [slash] Adjust logs * [committee] Check eligibility of validator before doing sanity check * [slash] Update docker * [slash] Move create validator script to test * [slash] More log * [param] Make things faster * [slash][off-chain] Clear out slashes from pending in writeblockwithstate * [cross-link] Log is not error, just info * [blockchain] Not necessary to guard DeletePendingSlashingCandidates * [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that * [slash][test] Use faucet as sender, assume user imported * [slash] Test setup * [slash] reserve error for real error in logs * [slash][availability] Apply availability correct, bump signing count each block * [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed * [slash] Pay as much as can * [slash] use right nowAmt * [slash] Take away from rewards as well * [slash] iterate faster * [slash] Remove dev based timing * [slash] Add more log, sanity check incoming slash records, only count external for slash rate * [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters * [availability] More log * [slash] Simply pre slash erra slashing * [slash] Remove development code * [slash] Use height from recvMsg, todo on epoch * [staking] Not necessary to touch LastEpochInCommittee in staking_verifier * [slash] Undo ds in endpoint pattern config * [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment * [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count Co-authored-by: Leo Chen <leo@harmony.one> Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com> Co-authored-by: Rongjian Lan <rongjian@harmony.one> Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
5 years ago
Resolve harmony-one/bounties#90: Add revert mechanism for UpdateValidatorWrapper (#3939) * Add revert mechanism for UpdateValidatorWrapper Closes harmony-one/bounties#90 (1) Use LRU for ValidatorWrapper objects in stateDB to plug a potential memory leak (2) Merge ValidatorWrapper and ValidatorWrapperCopy to let callers ask for either a copy, or a pointer to the cached object. Additionally, give callers the option to not deep copy delegations (which is a heavy process). Copies need to be explicitly committed (and thus can be reverted), while the pointers are committed when Finalise is called. (3) Add a UpdateValidatorWrapperWithRevert function, which is used by staking txs `Delegate`, `Undelegate`, and `CollectRewards`. Other 2 types of staking txs and `db.Finalize` continue to use UpdateValidateWrapper without revert, again, to save memoery (4) Add unit tests which check a) Revert goes through b) Wrapper is as expected after revert c) State is as expected after revert * Change back to dictionary for stateValidators Since the memory / CPU usage saved is not significantly different when using an LRU + map structure, go back to the original dictionary structure to keep code easy to read and have limited modifications. * Add tests for validator wrapper reverts As requested by @rlan35, add tests beyond just adding and reverting a delegation. The tests are successive in the sense that we do multiple modifications to the wrapper, save a snapshot before each modification and revert to each of them to confirm everything works well. This change improves test coverage of statedb.go to 66.7% from 64.8% and that of core/state to 71.9% from 70.8%, and covers all the code that has been modified by this PR in statedb.go. For clarity, the modifications to the wrapper include (1) creation of wrapper in state, (2) adding a delegation to the wrapper, (3) increasing the blocks signed, and (4) a change in the validator Name and the BlockReward. Two additional tests have been added to cover the `panic` and the `GetCode` cases.
3 years ago
wrapper, err := stateDB.ValidatorWrapper(msg.ValidatorAddress, false, true)
[double-sign] Provide proof of double sign in slash record sent to beaconchain (#2253) * [double-sign] Commit changes in consensus needed for double-sign * [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain * [slash] Add quick iteration tool for testing double-signing * [slash] Add webhook example * [slash] Add http server for hook to trigger double sign behavior * [double-sign] Use bin/trigger-double-sign to cause a double-sign * [double-sign] Full feedback loop working * [slash] Thread through the slash records in the block proposal step * [slash] Compute the slashing rate * [double-sign] Generalize yaml malicious for many keys * [double-sign][slash] Modify data structures, verify via webhook handler * [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures * [slash] Apply to state slashing for double signing * [slash][double-sign] Checkpoint for working code that slashes on beaconchain * [slash] Keep track of the total slash and total reporters reward * [slash] Dump account state before and after the slash * [slash] Satisfy Travis * [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates * [slash] Capture the unique new delegations since snapshot as well * [slash] Filter undelegation by epoch of double sign * [slash] Add TODO of correctness needed in slash needs on off-chain data * [rpc] Fix closure issue on shardID * [slash] Add delegator to double-sign testing script * [slash] Expand crt-validator.sh with commenting printfs and make delegation * [slash] Finish track payment of leftover slash debt after undelegation runs out * [slash] Now be explicit about error wrt delegatorSlashApply * [slash] Capture specific sanity check on slash paidoff * [slash] Track slash from undelegation piecemeal * [slash][delegation] Named slice types, .String() * [slash] Do no RLP encode twice, once is enough * [slash] Remove special case of validators own delegation * [slash] Refactor approach to slash state application * [slash] Begin expanding out Verify * [slash] Slash on snapshot delegations, not current * [slash] Fix Epoch Cmp * [slash] Third iteration on slash logic * [slash] Use full slash amount * [slash] More log, whitespace * [slash] Remove Println, add log * [slash] Remove debug Println * [slash] Add record in unit test * [slash] Build Validator snapshot, current. Fill out slash record * [slash] Need to get RLP dump of a header to use in test * [slash] Factor out double sign test constants * [slash] Factor out common for validator, stub out slash application, finish out deserialization setup * [slash] Factor out data structure creation because of var lexical scoping * [slash] Seem to have pipeline of unit test e2e executing * [slash] Add expected snitch, slash amounts * [slash] Checkpoint * [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing * [config] add double-sign testnet config (#1) Signed-off-by: Leo Chen <leo@harmony.one> * [slash] Commit for as is code & data of current dump.json * [slash] Order of state operation not correct in test, hence bad results, thank you dlv * [slash] Add snapshot state dump * [slash] Pay off slash of validator own delegation correctly * [slash] Pay off slash debt with special case for min-self * [slash] Pass first scenario conclusively * [slash] 2% slash passes unit test for own delegation and external * [slash] Parameterize unit test to easily test .02 vs .80 slash * [slash] Handle own delegation correctly at 80% slash * [slash] Have 80% slash working with external delegator * [slash] Remove debug code from slash * [slash] Adjust Apply signature, test again for 2% slash * [slash] Factor out scenario in testing so can test 2% and 80% at same time * [slash] Correct balance deduction on plan delegation * [slash] Mock out ChainReader for TestVerify * [slash] Small surface area interface, now feedback loop for verify * [slash] Remove development json * [slash] trigger-double-sign consumes yaml * [slash] Remove dead code * [slash][test] Factor ValidatorWrapper into scenario * [slash][test] Add example from local-testing dump - caution might be off * [slash] Factor out mutation of slashDebt * [slash][test] Factor out tests so can easily load test-case from bytes * [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected * [slash] Satisfy Travis * [slash] Begin cleanup of PR * [slash] Apply slash from header to Finalize via state processor * [slash] Productionize code, Println => logs; adjust slash picked in newblock * [slash] Need pointer for rlp.Decode * [slash] ValidatorInformation use full wrapper * Fix median stake * [staking] Adjust MarshalJSON for Validator, Wrapper * Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279) * Refactor offchain data; Add epoch to ValidatorSnapshot * Make block onchain/offchain data commit atomically * [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment * [effective] VC eligible.go * [consensus] Redundant field in printf * [docker] import-ks for a dev account * [slash] Create BLS key for dockerfile and crt-validator.sh * [slash][docker] Easy deployment of double-sign testing * [docker] Have slash work as single docker command * [rpc] Fix median-stake RPC * [slash] Update webhook with default docker BLS key * [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go * [slash] Remove helper binary, commented out code, change to local config * [params] Factor out test genesis value * Add shard checking to Tx-Pool & correct blacklist (#2301) * [core] Fix blacklist & add shardID check * [staking + node + cmd] Fix blacklist & add shardID check * [slash] Adjust to PR comments part 1 * [docker] Use different throw away funded account * [docker] Create easier testing for delegation with private keys * [docker] Update yaml * [slash] Remove special case for slashing validator own delegation wrt min-self-delegate * [docker] Install nano as well * [slash] Early error if banned * [quorum] Expose earning account in decider marshal json * Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)" This reverts commit 9ffbf682c075b49188923c65a0bbf39ac188be00. * [slash] Add non-sanity check way to update validator * [reward] Increase percision on percentage in schedule * [slash] Adjust logs * [committee] Check eligibility of validator before doing sanity check * [slash] Update docker * [slash] Move create validator script to test * [slash] More log * [param] Make things faster * [slash][off-chain] Clear out slashes from pending in writeblockwithstate * [cross-link] Log is not error, just info * [blockchain] Not necessary to guard DeletePendingSlashingCandidates * [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that * [slash][test] Use faucet as sender, assume user imported * [slash] Test setup * [slash] reserve error for real error in logs * [slash][availability] Apply availability correct, bump signing count each block * [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed * [slash] Pay as much as can * [slash] use right nowAmt * [slash] Take away from rewards as well * [slash] iterate faster * [slash] Remove dev based timing * [slash] Add more log, sanity check incoming slash records, only count external for slash rate * [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters * [availability] More log * [slash] Simply pre slash erra slashing * [slash] Remove development code * [slash] Use height from recvMsg, todo on epoch * [staking] Not necessary to touch LastEpochInCommittee in staking_verifier * [slash] Undo ds in endpoint pattern config * [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment * [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count Co-authored-by: Leo Chen <leo@harmony.one> Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com> Co-authored-by: Rongjian Lan <rongjian@harmony.one> Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
5 years ago
if err != nil {
return nil, err
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
}
[double-sign] Provide proof of double sign in slash record sent to beaconchain (#2253) * [double-sign] Commit changes in consensus needed for double-sign * [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain * [slash] Add quick iteration tool for testing double-signing * [slash] Add webhook example * [slash] Add http server for hook to trigger double sign behavior * [double-sign] Use bin/trigger-double-sign to cause a double-sign * [double-sign] Full feedback loop working * [slash] Thread through the slash records in the block proposal step * [slash] Compute the slashing rate * [double-sign] Generalize yaml malicious for many keys * [double-sign][slash] Modify data structures, verify via webhook handler * [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures * [slash] Apply to state slashing for double signing * [slash][double-sign] Checkpoint for working code that slashes on beaconchain * [slash] Keep track of the total slash and total reporters reward * [slash] Dump account state before and after the slash * [slash] Satisfy Travis * [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates * [slash] Capture the unique new delegations since snapshot as well * [slash] Filter undelegation by epoch of double sign * [slash] Add TODO of correctness needed in slash needs on off-chain data * [rpc] Fix closure issue on shardID * [slash] Add delegator to double-sign testing script * [slash] Expand crt-validator.sh with commenting printfs and make delegation * [slash] Finish track payment of leftover slash debt after undelegation runs out * [slash] Now be explicit about error wrt delegatorSlashApply * [slash] Capture specific sanity check on slash paidoff * [slash] Track slash from undelegation piecemeal * [slash][delegation] Named slice types, .String() * [slash] Do no RLP encode twice, once is enough * [slash] Remove special case of validators own delegation * [slash] Refactor approach to slash state application * [slash] Begin expanding out Verify * [slash] Slash on snapshot delegations, not current * [slash] Fix Epoch Cmp * [slash] Third iteration on slash logic * [slash] Use full slash amount * [slash] More log, whitespace * [slash] Remove Println, add log * [slash] Remove debug Println * [slash] Add record in unit test * [slash] Build Validator snapshot, current. Fill out slash record * [slash] Need to get RLP dump of a header to use in test * [slash] Factor out double sign test constants * [slash] Factor out common for validator, stub out slash application, finish out deserialization setup * [slash] Factor out data structure creation because of var lexical scoping * [slash] Seem to have pipeline of unit test e2e executing * [slash] Add expected snitch, slash amounts * [slash] Checkpoint * [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing * [config] add double-sign testnet config (#1) Signed-off-by: Leo Chen <leo@harmony.one> * [slash] Commit for as is code & data of current dump.json * [slash] Order of state operation not correct in test, hence bad results, thank you dlv * [slash] Add snapshot state dump * [slash] Pay off slash of validator own delegation correctly * [slash] Pay off slash debt with special case for min-self * [slash] Pass first scenario conclusively * [slash] 2% slash passes unit test for own delegation and external * [slash] Parameterize unit test to easily test .02 vs .80 slash * [slash] Handle own delegation correctly at 80% slash * [slash] Have 80% slash working with external delegator * [slash] Remove debug code from slash * [slash] Adjust Apply signature, test again for 2% slash * [slash] Factor out scenario in testing so can test 2% and 80% at same time * [slash] Correct balance deduction on plan delegation * [slash] Mock out ChainReader for TestVerify * [slash] Small surface area interface, now feedback loop for verify * [slash] Remove development json * [slash] trigger-double-sign consumes yaml * [slash] Remove dead code * [slash][test] Factor ValidatorWrapper into scenario * [slash][test] Add example from local-testing dump - caution might be off * [slash] Factor out mutation of slashDebt * [slash][test] Factor out tests so can easily load test-case from bytes * [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected * [slash] Satisfy Travis * [slash] Begin cleanup of PR * [slash] Apply slash from header to Finalize via state processor * [slash] Productionize code, Println => logs; adjust slash picked in newblock * [slash] Need pointer for rlp.Decode * [slash] ValidatorInformation use full wrapper * Fix median stake * [staking] Adjust MarshalJSON for Validator, Wrapper * Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279) * Refactor offchain data; Add epoch to ValidatorSnapshot * Make block onchain/offchain data commit atomically * [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment * [effective] VC eligible.go * [consensus] Redundant field in printf * [docker] import-ks for a dev account * [slash] Create BLS key for dockerfile and crt-validator.sh * [slash][docker] Easy deployment of double-sign testing * [docker] Have slash work as single docker command * [rpc] Fix median-stake RPC * [slash] Update webhook with default docker BLS key * [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go * [slash] Remove helper binary, commented out code, change to local config * [params] Factor out test genesis value * Add shard checking to Tx-Pool & correct blacklist (#2301) * [core] Fix blacklist & add shardID check * [staking + node + cmd] Fix blacklist & add shardID check * [slash] Adjust to PR comments part 1 * [docker] Use different throw away funded account * [docker] Create easier testing for delegation with private keys * [docker] Update yaml * [slash] Remove special case for slashing validator own delegation wrt min-self-delegate * [docker] Install nano as well * [slash] Early error if banned * [quorum] Expose earning account in decider marshal json * Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)" This reverts commit 9ffbf682c075b49188923c65a0bbf39ac188be00. * [slash] Add non-sanity check way to update validator * [reward] Increase percision on percentage in schedule * [slash] Adjust logs * [committee] Check eligibility of validator before doing sanity check * [slash] Update docker * [slash] Move create validator script to test * [slash] More log * [param] Make things faster * [slash][off-chain] Clear out slashes from pending in writeblockwithstate * [cross-link] Log is not error, just info * [blockchain] Not necessary to guard DeletePendingSlashingCandidates * [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that * [slash][test] Use faucet as sender, assume user imported * [slash] Test setup * [slash] reserve error for real error in logs * [slash][availability] Apply availability correct, bump signing count each block * [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed * [slash] Pay as much as can * [slash] use right nowAmt * [slash] Take away from rewards as well * [slash] iterate faster * [slash] Remove dev based timing * [slash] Add more log, sanity check incoming slash records, only count external for slash rate * [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters * [availability] More log * [slash] Simply pre slash erra slashing * [slash] Remove development code * [slash] Use height from recvMsg, todo on epoch * [staking] Not necessary to touch LastEpochInCommittee in staking_verifier * [slash] Undo ds in endpoint pattern config * [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment * [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count Co-authored-by: Leo Chen <leo@harmony.one> Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com> Co-authored-by: Rongjian Lan <rongjian@harmony.one> Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
5 years ago
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
for i := range wrapper.Delegations {
delegation := &wrapper.Delegations[i]
if bytes.Equal(delegation.DelegatorAddress.Bytes(), msg.DelegatorAddress.Bytes()) {
if err := delegation.Undelegate(epoch, msg.Amount); err != nil {
return nil, err
}
if err := wrapper.SanityCheck(); err != nil {
// allow self delegation to go below min self delegation
// but set the status to inactive
if errors.Cause(err) == staking.ErrInvalidSelfDelegation {
wrapper.Status = effective.Inactive
} else {
return nil, err
}
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
}
return wrapper, nil
}
}
return nil, errNoDelegationToUndelegate
}
Resolve harmony-one/bounties#77: Staking precompiles (#3906) * Resolve harmony-one/bounties#77: Staking precompiles Create write capable precompiles that can perform staking transactions Add hard fork logic (EpochTBD) for these precompiles Tests for new code with at least 80% unit test coverage Staking library + tests in MaxMustermann2/harmony-staking-precompiles * Fix small typo in comment * Run goimports on files to fix Travis * Do not activate staking precompile on shard 0 * Cascade readOnly to WriteCapableContract * No overlap in readOnly + writeCapable precompiles * Use function selector instead of directive From Solidity, use abi.encodeWithSelector and match it against the exact ABI of the functions. This allows us to remove the need for a directive (32) being encoded, and thus saves 28 bytes of data. * Do not allow contracts to become validators As discussed with Jacky on #3906 * Merge harmony-one/harmony/main properly this time * Run goimports * Update gas calculation for staking precompile Please see comment in core/vm/contracts_write.go RequiredGas * Do not allow contract to become validator (2/2) * Cache StakeMsgs from precompiled transactions Add the StakeMsgs to ProcessorResult and cascade them in insertChain * Remove ContractCode fields from validators Since smart contracts can no longer beecome validators, this field is superfluous. Remove it from the Wrapper structure, and do not assign it a value when creating a validator. Build and goimports checked * Update comments in response to feedback (1) Comments to start with function names (2) Comments for public variables (3) Comment to match function name RunPrecompiledContract (4) Clarify that CreateValidatorFunc + EditValidatorFunc are still used * Fix Travis build by reverting rosetta change * Add revert capability to 3 staking tx types - Delegate - Undelegate - CollectRewards * Fix build: Update evm_test for ValidatorWrapper * Merge main into harmony-staking-precompiles * Add gas for precompile calls and allow EOA usage - Each time the precompile is called, charge the base gas fee plus data cost (if data can be parsed successfully). A gas fee is added to prevent benevolent contract deployers from subsidizing the staking transactions for EOAs through repeated assembly `delegatecall`. - Allow EOAs to use the staking precompile directly. Some changes to the Solidity library are associated with this change. - Remove bytes from parsing address, since the ABI unpacks it into an address format correctly. - Add or update tests. Test coverage report to be attached to the PR shortly. * Run goimports * Check read only and write capable for overlap * Handle precompile stakeMsgs for block proposer The staking precompile generates staking messages which are cascaded to the block via the EVM in `state_processor.go`. This change cascades them in `worker.go` to allow block proposers and block verifiers to keep the same state. * Run goimports for cf2dfac4081444e36a120c9432f4e.. * Update staking precompile epoch to 2 for localnet Bring it in line with staking epoch. Change effects all configurations except mainnet and testnet. `goimports` included. * Add read only precompile to fetch the epoch num * Move epoch precompile to 250 * precompiles: left pad the returned epoch number * chainConfig: check epochs for precompiles panic if staking precompile epoch < pre staking epoch * Add staking migration precompile - Lives at address 251 - Migrates delegations + pending undelegations from address A to B - Useful if address A is hacked - Charges gas of 21k + cost of bytes for two addresses - Does not remove existing delegations, just sets them to zero. Replicates current undelegate setup - Unit tests and `goimports` included. Integration test following shortly in MaxMustermann2/harmony-staking-precompiles * Migration precompile: merge into staking Merge the two precompiles into one, add gas calculation for migration precompile. Move epoch precompile to 251 as a result. When migrating, add undelegations to `To`'s existing undelegations, if any match the epoch. * Add migration gas test, remove panic, add check In response to review comments, add tests for migration gas wherein there are 0/1/2 delegations to migrate. Add the index out of bound check to migration gas calculator and remove panics. Lastly, re-sort migrated undelegations if no existing undelegation in the same epoch was found on `To`. * Move undelegations sorting to end of loop
3 years ago
// VerifyAndMigrateFromMsg verifies and transfers all delegations of
// msg.From to msg.To. Returns all modified validator wrappers and delegate msgs
// for metadata
// Note that this function never updates the stateDB, it only reads from stateDB.
func VerifyAndMigrateFromMsg(
stateDB vm.StateDB,
msg *staking.MigrationMsg,
fromDelegations []staking.DelegationIndex,
) ([]*staking.ValidatorWrapper,
[]interface{},
error) {
if bytes.Equal(msg.From.Bytes(), msg.To.Bytes()) {
return nil, nil, errors.New("From and To are the same address")
}
if len(fromDelegations) == 0 {
return nil, nil, errors.New("No delegations to migrate")
}
modifiedWrappers := make([]*staking.ValidatorWrapper, 0)
stakeMsgs := make([]interface{}, 0)
// iterate over all delegationIndexes by `From`
for i := range fromDelegations {
delegationIndex := &fromDelegations[i]
// find the wrapper for each delegationIndex
// request a copy, and since delegations will be changed, copy them too
wrapper, err := stateDB.ValidatorWrapper(delegationIndex.ValidatorAddress, false, true)
if err != nil {
return nil, nil, err
}
if uint64(len(wrapper.Delegations)) <= delegationIndex.Index {
utils.Logger().Warn().
Str("validator", delegationIndex.ValidatorAddress.String()).
Uint64("delegation index", delegationIndex.Index).
Int("delegations length", len(wrapper.Delegations)).
Msg("Delegation index out of bound")
return nil, nil, errors.New("Delegation index out of bound")
}
// and then find matching delegation to remove from wrapper
foundDelegation := &wrapper.Delegations[delegationIndex.Index] // note: pointer
if !bytes.Equal(foundDelegation.DelegatorAddress.Bytes(), msg.From.Bytes()) {
return nil, nil, errors.New(fmt.Sprintf("Expected %s but got %s",
msg.From.Hex(),
foundDelegation.DelegatorAddress.Hex()))
}
// Skip delegations with zero amount and empty undelegation
if foundDelegation.Amount.Cmp(common.Big0) == 0 && len(foundDelegation.Undelegations) == 0 {
continue
}
delegationAmountToMigrate := big.NewInt(0).Add(foundDelegation.Amount, big.NewInt(0))
undelegationsToMigrate := foundDelegation.Undelegations
// when undelegating we don't remove, just set the amount to zero
// to be coherent, do the same thing here (effective on wrapper since pointer)
foundDelegation.Amount = big.NewInt(0)
foundDelegation.Undelegations = make([]staking.Undelegation, 0)
// find `To` and give it to them
totalAmount := big.NewInt(0)
found := false
for i := range wrapper.Delegations {
delegation := &wrapper.Delegations[i]
if bytes.Equal(delegation.DelegatorAddress.Bytes(), msg.To.Bytes()) {
found = true
// add to existing delegation
totalAmount = delegation.Amount.Add(delegation.Amount, delegationAmountToMigrate)
// and the undelegations
for _, undelegationToMigrate := range undelegationsToMigrate {
exist := false
for _, entry := range delegation.Undelegations {
if entry.Epoch.Cmp(undelegationToMigrate.Epoch) == 0 {
exist = true
entry.Amount.Add(entry.Amount, undelegationToMigrate.Amount)
break
}
}
if !exist {
delegation.Undelegations = append(delegation.Undelegations,
undelegationToMigrate)
}
}
// Always sort the undelegate by epoch in increasing order
sort.SliceStable(
delegation.Undelegations,
func(i, j int) bool {
return delegation.Undelegations[i].Epoch.Cmp(delegation.Undelegations[j].Epoch) < 0
},
)
break
}
}
if !found { // add the delegation
wrapper.Delegations = append(
wrapper.Delegations, staking.NewDelegation(
msg.To, delegationAmountToMigrate,
),
)
totalAmount = delegationAmountToMigrate
}
if err := wrapper.SanityCheck(); err != nil {
// allow self delegation to go below min self delegation
// but set the status to inactive
if errors.Cause(err) == staking.ErrInvalidSelfDelegation {
wrapper.Status = effective.Inactive
} else {
return nil, nil, err
}
}
modifiedWrappers = append(modifiedWrappers, wrapper)
delegate := &staking.Delegate{
ValidatorAddress: wrapper.Address,
DelegatorAddress: msg.To,
Amount: totalAmount,
}
stakeMsgs = append(stakeMsgs, delegate)
}
return modifiedWrappers, stakeMsgs, nil
}
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
// VerifyAndCollectRewardsFromDelegation verifies and collects rewards
// from the given delegation slice using the stateDB. It returns all of the
// edited validatorWrappers and the sum total of the rewards.
//
// Note that this function never updates the stateDB, it only reads from stateDB.
func VerifyAndCollectRewardsFromDelegation(
stateDB vm.StateDB, delegations []staking.DelegationIndex,
) ([]*staking.ValidatorWrapper, *big.Int, error) {
if stateDB == nil {
return nil, nil, errStateDBIsMissing
}
updatedValidatorWrappers := []*staking.ValidatorWrapper{}
totalRewards := big.NewInt(0)
for i := range delegations {
delegation := &delegations[i]
Resolve harmony-one/bounties#90: Add revert mechanism for UpdateValidatorWrapper (#3939) * Add revert mechanism for UpdateValidatorWrapper Closes harmony-one/bounties#90 (1) Use LRU for ValidatorWrapper objects in stateDB to plug a potential memory leak (2) Merge ValidatorWrapper and ValidatorWrapperCopy to let callers ask for either a copy, or a pointer to the cached object. Additionally, give callers the option to not deep copy delegations (which is a heavy process). Copies need to be explicitly committed (and thus can be reverted), while the pointers are committed when Finalise is called. (3) Add a UpdateValidatorWrapperWithRevert function, which is used by staking txs `Delegate`, `Undelegate`, and `CollectRewards`. Other 2 types of staking txs and `db.Finalize` continue to use UpdateValidateWrapper without revert, again, to save memoery (4) Add unit tests which check a) Revert goes through b) Wrapper is as expected after revert c) State is as expected after revert * Change back to dictionary for stateValidators Since the memory / CPU usage saved is not significantly different when using an LRU + map structure, go back to the original dictionary structure to keep code easy to read and have limited modifications. * Add tests for validator wrapper reverts As requested by @rlan35, add tests beyond just adding and reverting a delegation. The tests are successive in the sense that we do multiple modifications to the wrapper, save a snapshot before each modification and revert to each of them to confirm everything works well. This change improves test coverage of statedb.go to 66.7% from 64.8% and that of core/state to 71.9% from 70.8%, and covers all the code that has been modified by this PR in statedb.go. For clarity, the modifications to the wrapper include (1) creation of wrapper in state, (2) adding a delegation to the wrapper, (3) increasing the blocks signed, and (4) a change in the validator Name and the BlockReward. Two additional tests have been added to cover the `panic` and the `GetCode` cases.
3 years ago
// request a copy, and since delegations will be changed (.Reward.Set), copy them too
wrapper, err := stateDB.ValidatorWrapper(delegation.ValidatorAddress, false, true)
[double-sign] Provide proof of double sign in slash record sent to beaconchain (#2253) * [double-sign] Commit changes in consensus needed for double-sign * [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain * [slash] Add quick iteration tool for testing double-signing * [slash] Add webhook example * [slash] Add http server for hook to trigger double sign behavior * [double-sign] Use bin/trigger-double-sign to cause a double-sign * [double-sign] Full feedback loop working * [slash] Thread through the slash records in the block proposal step * [slash] Compute the slashing rate * [double-sign] Generalize yaml malicious for many keys * [double-sign][slash] Modify data structures, verify via webhook handler * [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures * [slash] Apply to state slashing for double signing * [slash][double-sign] Checkpoint for working code that slashes on beaconchain * [slash] Keep track of the total slash and total reporters reward * [slash] Dump account state before and after the slash * [slash] Satisfy Travis * [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates * [slash] Capture the unique new delegations since snapshot as well * [slash] Filter undelegation by epoch of double sign * [slash] Add TODO of correctness needed in slash needs on off-chain data * [rpc] Fix closure issue on shardID * [slash] Add delegator to double-sign testing script * [slash] Expand crt-validator.sh with commenting printfs and make delegation * [slash] Finish track payment of leftover slash debt after undelegation runs out * [slash] Now be explicit about error wrt delegatorSlashApply * [slash] Capture specific sanity check on slash paidoff * [slash] Track slash from undelegation piecemeal * [slash][delegation] Named slice types, .String() * [slash] Do no RLP encode twice, once is enough * [slash] Remove special case of validators own delegation * [slash] Refactor approach to slash state application * [slash] Begin expanding out Verify * [slash] Slash on snapshot delegations, not current * [slash] Fix Epoch Cmp * [slash] Third iteration on slash logic * [slash] Use full slash amount * [slash] More log, whitespace * [slash] Remove Println, add log * [slash] Remove debug Println * [slash] Add record in unit test * [slash] Build Validator snapshot, current. Fill out slash record * [slash] Need to get RLP dump of a header to use in test * [slash] Factor out double sign test constants * [slash] Factor out common for validator, stub out slash application, finish out deserialization setup * [slash] Factor out data structure creation because of var lexical scoping * [slash] Seem to have pipeline of unit test e2e executing * [slash] Add expected snitch, slash amounts * [slash] Checkpoint * [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing * [config] add double-sign testnet config (#1) Signed-off-by: Leo Chen <leo@harmony.one> * [slash] Commit for as is code & data of current dump.json * [slash] Order of state operation not correct in test, hence bad results, thank you dlv * [slash] Add snapshot state dump * [slash] Pay off slash of validator own delegation correctly * [slash] Pay off slash debt with special case for min-self * [slash] Pass first scenario conclusively * [slash] 2% slash passes unit test for own delegation and external * [slash] Parameterize unit test to easily test .02 vs .80 slash * [slash] Handle own delegation correctly at 80% slash * [slash] Have 80% slash working with external delegator * [slash] Remove debug code from slash * [slash] Adjust Apply signature, test again for 2% slash * [slash] Factor out scenario in testing so can test 2% and 80% at same time * [slash] Correct balance deduction on plan delegation * [slash] Mock out ChainReader for TestVerify * [slash] Small surface area interface, now feedback loop for verify * [slash] Remove development json * [slash] trigger-double-sign consumes yaml * [slash] Remove dead code * [slash][test] Factor ValidatorWrapper into scenario * [slash][test] Add example from local-testing dump - caution might be off * [slash] Factor out mutation of slashDebt * [slash][test] Factor out tests so can easily load test-case from bytes * [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected * [slash] Satisfy Travis * [slash] Begin cleanup of PR * [slash] Apply slash from header to Finalize via state processor * [slash] Productionize code, Println => logs; adjust slash picked in newblock * [slash] Need pointer for rlp.Decode * [slash] ValidatorInformation use full wrapper * Fix median stake * [staking] Adjust MarshalJSON for Validator, Wrapper * Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279) * Refactor offchain data; Add epoch to ValidatorSnapshot * Make block onchain/offchain data commit atomically * [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment * [effective] VC eligible.go * [consensus] Redundant field in printf * [docker] import-ks for a dev account * [slash] Create BLS key for dockerfile and crt-validator.sh * [slash][docker] Easy deployment of double-sign testing * [docker] Have slash work as single docker command * [rpc] Fix median-stake RPC * [slash] Update webhook with default docker BLS key * [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go * [slash] Remove helper binary, commented out code, change to local config * [params] Factor out test genesis value * Add shard checking to Tx-Pool & correct blacklist (#2301) * [core] Fix blacklist & add shardID check * [staking + node + cmd] Fix blacklist & add shardID check * [slash] Adjust to PR comments part 1 * [docker] Use different throw away funded account * [docker] Create easier testing for delegation with private keys * [docker] Update yaml * [slash] Remove special case for slashing validator own delegation wrt min-self-delegate * [docker] Install nano as well * [slash] Early error if banned * [quorum] Expose earning account in decider marshal json * Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)" This reverts commit 9ffbf682c075b49188923c65a0bbf39ac188be00. * [slash] Add non-sanity check way to update validator * [reward] Increase percision on percentage in schedule * [slash] Adjust logs * [committee] Check eligibility of validator before doing sanity check * [slash] Update docker * [slash] Move create validator script to test * [slash] More log * [param] Make things faster * [slash][off-chain] Clear out slashes from pending in writeblockwithstate * [cross-link] Log is not error, just info * [blockchain] Not necessary to guard DeletePendingSlashingCandidates * [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that * [slash][test] Use faucet as sender, assume user imported * [slash] Test setup * [slash] reserve error for real error in logs * [slash][availability] Apply availability correct, bump signing count each block * [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed * [slash] Pay as much as can * [slash] use right nowAmt * [slash] Take away from rewards as well * [slash] iterate faster * [slash] Remove dev based timing * [slash] Add more log, sanity check incoming slash records, only count external for slash rate * [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters * [availability] More log * [slash] Simply pre slash erra slashing * [slash] Remove development code * [slash] Use height from recvMsg, todo on epoch * [staking] Not necessary to touch LastEpochInCommittee in staking_verifier * [slash] Undo ds in endpoint pattern config * [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment * [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count Co-authored-by: Leo Chen <leo@harmony.one> Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com> Co-authored-by: Rongjian Lan <rongjian@harmony.one> Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
5 years ago
if err != nil {
return nil, nil, err
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
}
if uint64(len(wrapper.Delegations)) > delegation.Index {
delegation := &wrapper.Delegations[delegation.Index]
[staking][validation][protocol] (#2396) * [staking][validation][protocol] Limit max bls keys * [staking-era] Fold banned and active into single field * [slash][effective] Remove LRU cache for slash, change .Active to enumeration * [slash] Remove leftover wrong usage of Logger * [slash][offchain] Only Decode if len > 0 * [offchain] cosmetic * [slash] Remove some logs in proposal * [webhook] Move webhook with call for when cannot commit block * [shard] Finally make finding subcommittee by shardID an explicit error * [node] Whitespace, prefer literal * [webhook] Report bad block to webhook * [slash] Expand verify, remove bad log usage, explicit error handle * [slash] Check on key size * [slash] Explicit upper bound of pending slashes * [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain * [multibls] Make max count allowed be 1/3 of external slots * [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee * [verify] Begin factor out of common verification approach * [project] Further remove RawJSON log, use proper epoch for snapshot * [slash] Implement verification * [slash] Implement BLS key verification of ballots * [rpc] Keep validator information as meaningful as possible * [staking] Never can stop being banned * [slash] Comments and default Unknown case of eligibility * [slash] Be explicit on what input values allowed when want to change EPOSStatus * [consensus] Remove unneeded TODO * [verify] Add proper error message * [rpc] Give back to caller their wrong chain id * [chain] Add extra map dump of delegation sizing for downstream analysis * [engine] Less code, more methods * [offchain] More leniency in handling slash bytes and delete from pending * [validator] Remove errors on bad input for edit
5 years ago
if delegation.Reward.Cmp(common.Big0) > 0 {
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
totalRewards.Add(totalRewards, delegation.Reward)
delegation.Reward.SetUint64(0)
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
}
} else {
utils.Logger().Warn().
Str("validator", delegation.ValidatorAddress.String()).
Uint64("delegation index", delegation.Index).
Int("delegations length", len(wrapper.Delegations)).
Msg("Delegation index out of bound")
return nil, nil, errors.New("Delegation index out of bound")
Abstract transactions in tx pool and add staking transaction to pool with error report (#2236) * [core] Add tx-pool txn interface & update supporting components * Add txn interface (`PoolTransaction`) for tx-pool * Update tx_journal to handle pool's txn interface * Update tx_list to handle pool's txn interface * [staking] Satisfy `PoolTransaction` interface & move error sink types * Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size` for `StakingTransaction` to satisfy `PoolTransaction` interface * Refactor `Price` to `GasPrice` for `StakingTransaction` to satisfy `PoolTransaction` interface * Move error sink related components to transaction.go * Expose `VerifyBLSKey` and `VerifyBLSKeys` * [core] Generalize tx pool & refactor error sink logic * Refactor txn logic to use `PoolTransaction` and `PoolTransactions` * Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx error sinks * Remove old & unpayable txs error reports (to error sink) since errs are already reported when adding the txs * Fix known transaction error report when adding txn batches * Add error sink reporting when failed to enqueue txs * [node] Fix error sink & update tx pool interaction * Integrate staking transaction in tx-pool * Remove staking transaction error sink * [hmy api] Integrate staking transactions from tx pool * Remove looking at tx pool for `GetTransactionByHash` * Add `PendingStakingTransactions` and update `PendingTransactions` to only return plainTx * [tests] Update all tests for tx pool txn interface & staking err sink * Update transactions to `PoolTransaction` interface * Remove `CommitTransactions` staking txn error sink * Add basic staking txn tests to tx pool tests * [node] Make all node broadcast staking tx and plain tx * [core + staking] Separate staking msg check and put in tx pool * Move `Validator` specific sanity check into its own method and call said method in `ValidatorWrapper` sanity check * Create staking msg verifiers and preprocessors in `staking_verifier.go` * Remove staking msg verification on all staking msg applications in `state_transition.go` and call new staking msg verifiers & preprocessors * Add staking msg verification to tx pool * Remove `ToShardID` from `PoolTransaction` interface and remove trivial implementation of `ToShardID` in `StakingTransaction`
5 years ago
}
updatedValidatorWrappers = append(updatedValidatorWrappers, wrapper)
}
if totalRewards.Int64() == 0 {
return nil, nil, errNoRewardsToCollect
}
return updatedValidatorWrappers, totalRewards, nil
}