* Resolve harmony-one/bounties#77: Staking precompiles
Create write capable precompiles that can perform staking transactions
Add hard fork logic (EpochTBD) for these precompiles
Tests for new code with at least 80% unit test coverage
Staking library + tests in MaxMustermann2/harmony-staking-precompiles
* Fix small typo in comment
* Run goimports on files to fix Travis
* Do not activate staking precompile on shard 0
* Cascade readOnly to WriteCapableContract
* No overlap in readOnly + writeCapable precompiles
* Use function selector instead of directive
From Solidity, use abi.encodeWithSelector and match it against the
exact ABI of the functions. This allows us to remove the need for
a directive (32) being encoded, and thus saves 28 bytes of data.
* Do not allow contracts to become validators
As discussed with Jacky on #3906
* Merge harmony-one/harmony/main properly this time
* Run goimports
* Update gas calculation for staking precompile
Please see comment in core/vm/contracts_write.go RequiredGas
* Do not allow contract to become validator (2/2)
* Cache StakeMsgs from precompiled transactions
Add the StakeMsgs to ProcessorResult and cascade them in insertChain
* Remove ContractCode fields from validators
Since smart contracts can no longer beecome validators,
this field is superfluous. Remove it from the Wrapper
structure, and do not assign it a value when creating
a validator. Build and goimports checked
* Update comments in response to feedback
(1) Comments to start with function names
(2) Comments for public variables
(3) Comment to match function name RunPrecompiledContract
(4) Clarify that CreateValidatorFunc + EditValidatorFunc are still used
* Fix Travis build by reverting rosetta change
* Add revert capability to 3 staking tx types
- Delegate
- Undelegate
- CollectRewards
* Fix build: Update evm_test for ValidatorWrapper
* Merge main into harmony-staking-precompiles
* Add gas for precompile calls and allow EOA usage
- Each time the precompile is called, charge the base gas fee plus data
cost (if data can be parsed successfully). A gas fee is added to
prevent benevolent contract deployers from subsidizing the staking
transactions for EOAs through repeated assembly `delegatecall`.
- Allow EOAs to use the staking precompile directly. Some changes to
the Solidity library are associated with this change.
- Remove bytes from parsing address, since the ABI unpacks it into an
address format correctly.
- Add or update tests. Test coverage report to be attached to the PR
shortly.
* Run goimports
* Check read only and write capable for overlap
* Handle precompile stakeMsgs for block proposer
The staking precompile generates staking messages which are cascaded to
the block via the EVM in `state_processor.go`. This change cascades them
in `worker.go` to allow block proposers and block verifiers to keep the
same state.
* Run goimports for cf2dfac4081444e36a120c9432f4e..
* Update staking precompile epoch to 2 for localnet
Bring it in line with staking epoch. Change effects all configurations
except mainnet and testnet. `goimports` included.
* Add read only precompile to fetch the epoch num
* Move epoch precompile to 250
* precompiles: left pad the returned epoch number
* chainConfig: check epochs for precompiles
panic if staking precompile epoch < pre staking epoch
* Add staking migration precompile
- Lives at address 251
- Migrates delegations + pending undelegations from address A to B
- Useful if address A is hacked
- Charges gas of 21k + cost of bytes for two addresses
- Does not remove existing delegations, just sets them to zero.
Replicates current undelegate setup
- Unit tests and `goimports` included. Integration test following
shortly in MaxMustermann2/harmony-staking-precompiles
* Migration precompile: merge into staking
Merge the two precompiles into one, add gas calculation for migration
precompile. Move epoch precompile to 251 as a result. When migrating,
add undelegations to `To`'s existing undelegations, if any match the
epoch.
* Add migration gas test, remove panic, add check
In response to review comments, add tests for migration gas wherein
there are 0/1/2 delegations to migrate. Add the index out of bound check
to migration gas calculator and remove panics. Lastly, re-sort
migrated undelegations if no existing undelegation in the same epoch was
found on `To`.
* Move undelegations sorting to end of loop
* Add ethereum transaction support
* fix fmt
* fix lint
* add new block body fork logic
* add back deleted code
* lower case block version const
* add eth txn into new version of block
* refactor back
* refactor more
* fix test code
* Fix build
* Fix build
* revert eth txn in block and block proposal
* fix build
* fix build
* fix build
* Add ethereum transaction support
* fix fmt
* fix lint
* add new block body fork logic
* add back deleted code
* lower case block version const
* add eth txn into new version of block
* refactor back
* refactor more
* fix test code
* Fix build
* Fix build
* [core] Add FindLogsWithTopic & unit test
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [hmy] Add GetDetailedBlockSignerInfo
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [hmy] Add IsCommitteeSelectionBlock
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [test] Add test transaction creation helpers
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Refactor account.go & add tests
* Move TestNewAccountIdentifier & TestGetAddress to account_test.go
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Move Operation & Tx formatting to own files
* Move Respective unit tests to own files
* Expose GetOperations & GetStakingOperations
* Expose FormatTransaction, FormatCrossShardReceiverTransaction,
FormatGenesisTransaction, FormatPreStakingRewardTransaction & FormatUndelegationPayoutTransaction
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Move TransactionMetadata to transaction_construction.go
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Update construction to use new helpers & formatters
* Make docs consistent for mempool.go
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Move all special tx & blk handling to own file
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Remove all moved fns, methods & tests from block.go
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* Fix lint & imports
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Rename all tx related files for clarity
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Rename DefaultSenderAddress to FormatDefaultSenderAddress
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Rename Currency to NativeCurrency
* This is in anticipation of HRC20 token support with rosetta
* Rename various native operation functions accordingly
* Add documentation to explain what a native token is
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix pre-staking block reward calculation
* Move getPreStakingRewardTransactionIdentifiers to block_special.go
* Add epoch to block metadata
* Update unit tests
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* Add IsLastBlockInEpoch method to Block & Header
* Refactor all uses of length check `ShardState`
* [hmy] Refactor IsCommitteeSelectionBlock to use chain.IsCommitteeSelectionBlock
* Address PR comments
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Update var names in preStakingRewardBlockTransaction
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* Rollup evm to geth v1.9.9 Muir Glacier
* fix go gen
* update intrinsic gas for istanbul
* Update statedb.Commit
* revert go gen result
* revert protobuf version
* update protobuf
* update go gen files
* set testnet epoch for evm upgrade
* processing staking and normal transactions together
* bug fix and removing unwanted method
* made sense to rename and move the TransactionsByPriceAndNonce to pool type
* accounting for prestaking blocks, need to keep the previous order for processing txs
* Add cache to state validator access; refactor snapshot read/write
* make validator update happen at commit
* fix lint
* make sure slashing applies on validator wrapper in state
* Revert live update of validtor wrapper in staking txn
* add init population of validator cache logic
* Fix validator cache commit logic
* Separate validatorWrapper func
* fix build
* Fix lint
* [project] Dead test
* [node] If fields always same, then just use the constant
* [project] Remove overcomplicated ctxerror
* [project] Remove more dead tests, adjust error replacing ctxerror
* [project] More dead tests, fix travis complaint on Errorf
* [node.sh] Remove is-genesis
* More logs and checks before processing staking txns
* fix import
* Refactor block proposal
* Various fixes and todos for auditing; add log for collect rewards
* Fix lint
* fix comment
* [staking][validation][protocol] Limit max bls keys
* [staking-era] Fold banned and active into single field
* [slash][effective] Remove LRU cache for slash, change .Active to enumeration
* [slash] Remove leftover wrong usage of Logger
* [slash][offchain] Only Decode if len > 0
* [offchain] cosmetic
* [slash] Remove some logs in proposal
* [webhook] Move webhook with call for when cannot commit block
* [shard] Finally make finding subcommittee by shardID an explicit error
* [node] Whitespace, prefer literal
* [webhook] Report bad block to webhook
* [slash] Expand verify, remove bad log usage, explicit error handle
* [slash] Check on key size
* [slash] Explicit upper bound of pending slashes
* [slash] Use right epoch snapshot, fail to verify if epoch wrong on beaconchain
* [multibls] Make max count allowed be 1/3 of external slots
* [quorum] Remove bad API of ShardIDProvider, factor out committee key as method of committee
* [verify] Begin factor out of common verification approach
* [project] Further remove RawJSON log, use proper epoch for snapshot
* [slash] Implement verification
* [slash] Implement BLS key verification of ballots
* [rpc] Keep validator information as meaningful as possible
* [staking] Never can stop being banned
* [slash] Comments and default Unknown case of eligibility
* [slash] Be explicit on what input values allowed when want to change EPOSStatus
* [consensus] Remove unneeded TODO
* [verify] Add proper error message
* [rpc] Give back to caller their wrong chain id
* [chain] Add extra map dump of delegation sizing for downstream analysis
* [engine] Less code, more methods
* [offchain] More leniency in handling slash bytes and delete from pending
* [validator] Remove errors on bad input for edit
* [webhook] Loop over all self address
* [rpc] Remove accidential recursion
* [rpc] Never ignore errors, fix RPC with current snap info, no div by zero
* [rpc] Expose BadBlock for downstream
* [availability] Thread through missing signers for offchain usage in WriteBlockWithState
* [project] Merge with master, accept master conflicts
* [offchain] Remove inactive signers from offchain data, should fix median-stake issues
* [rpc] Test idea that returning error cause jenkins failure
* [staking] Put editvalidator back into validator list cache
* [rpc] Add network voting-power to validator information
* [votepower] Extend roster with raw percentage
* [rpc] If have validator but not snapshot, give back validator
* [rpc] Expose effective-stake as well
* [project] Remove missedSignThreshold map, use instance for stakedslots count
* [staking] Remove edit-validator idea from previous filter idea
* [double-sign] Commit changes in consensus needed for double-sign
* [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain
* [slash] Add quick iteration tool for testing double-signing
* [slash] Add webhook example
* [slash] Add http server for hook to trigger double sign behavior
* [double-sign] Use bin/trigger-double-sign to cause a double-sign
* [double-sign] Full feedback loop working
* [slash] Thread through the slash records in the block proposal step
* [slash] Compute the slashing rate
* [double-sign] Generalize yaml malicious for many keys
* [double-sign][slash] Modify data structures, verify via webhook handler
* [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures
* [slash] Apply to state slashing for double signing
* [slash][double-sign] Checkpoint for working code that slashes on beaconchain
* [slash] Keep track of the total slash and total reporters reward
* [slash] Dump account state before and after the slash
* [slash] Satisfy Travis
* [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates
* [slash] Capture the unique new delegations since snapshot as well
* [slash] Filter undelegation by epoch of double sign
* [slash] Add TODO of correctness needed in slash needs on off-chain data
* [rpc] Fix closure issue on shardID
* [slash] Add delegator to double-sign testing script
* [slash] Expand crt-validator.sh with commenting printfs and make delegation
* [slash] Finish track payment of leftover slash debt after undelegation runs out
* [slash] Now be explicit about error wrt delegatorSlashApply
* [slash] Capture specific sanity check on slash paidoff
* [slash] Track slash from undelegation piecemeal
* [slash][delegation] Named slice types, .String()
* [slash] Do no RLP encode twice, once is enough
* [slash] Remove special case of validators own delegation
* [slash] Refactor approach to slash state application
* [slash] Begin expanding out Verify
* [slash] Slash on snapshot delegations, not current
* [slash] Fix Epoch Cmp
* [slash] Third iteration on slash logic
* [slash] Use full slash amount
* [slash] More log, whitespace
* [slash] Remove Println, add log
* [slash] Remove debug Println
* [slash] Add record in unit test
* [slash] Build Validator snapshot, current. Fill out slash record
* [slash] Need to get RLP dump of a header to use in test
* [slash] Factor out double sign test constants
* [slash] Factor out common for validator, stub out slash application, finish out deserialization setup
* [slash] Factor out data structure creation because of var lexical scoping
* [slash] Seem to have pipeline of unit test e2e executing
* [slash] Add expected snitch, slash amounts
* [slash] Checkpoint
* [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing
* [config] add double-sign testnet config (#1)
Signed-off-by: Leo Chen <leo@harmony.one>
* [slash] Commit for as is code & data of current dump.json
* [slash] Order of state operation not correct in test, hence bad results, thank you dlv
* [slash] Add snapshot state dump
* [slash] Pay off slash of validator own delegation correctly
* [slash] Pay off slash debt with special case for min-self
* [slash] Pass first scenario conclusively
* [slash] 2% slash passes unit test for own delegation and external
* [slash] Parameterize unit test to easily test .02 vs .80 slash
* [slash] Handle own delegation correctly at 80% slash
* [slash] Have 80% slash working with external delegator
* [slash] Remove debug code from slash
* [slash] Adjust Apply signature, test again for 2% slash
* [slash] Factor out scenario in testing so can test 2% and 80% at same time
* [slash] Correct balance deduction on plan delegation
* [slash] Mock out ChainReader for TestVerify
* [slash] Small surface area interface, now feedback loop for verify
* [slash] Remove development json
* [slash] trigger-double-sign consumes yaml
* [slash] Remove dead code
* [slash][test] Factor ValidatorWrapper into scenario
* [slash][test] Add example from local-testing dump - caution might be off
* [slash] Factor out mutation of slashDebt
* [slash][test] Factor out tests so can easily load test-case from bytes
* [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected
* [slash] Satisfy Travis
* [slash] Begin cleanup of PR
* [slash] Apply slash from header to Finalize via state processor
* [slash] Productionize code, Println => logs; adjust slash picked in newblock
* [slash] Need pointer for rlp.Decode
* [slash] ValidatorInformation use full wrapper
* Fix median stake
* [staking] Adjust MarshalJSON for Validator, Wrapper
* Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)
* Refactor offchain data; Add epoch to ValidatorSnapshot
* Make block onchain/offchain data commit atomically
* [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment
* [effective] VC eligible.go
* [consensus] Redundant field in printf
* [docker] import-ks for a dev account
* [slash] Create BLS key for dockerfile and crt-validator.sh
* [slash][docker] Easy deployment of double-sign testing
* [docker] Have slash work as single docker command
* [rpc] Fix median-stake RPC
* [slash] Update webhook with default docker BLS key
* [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go
* [slash] Remove helper binary, commented out code, change to local config
* [params] Factor out test genesis value
* Add shard checking to Tx-Pool & correct blacklist (#2301)
* [core] Fix blacklist & add shardID check
* [staking + node + cmd] Fix blacklist & add shardID check
* [slash] Adjust to PR comments part 1
* [docker] Use different throw away funded account
* [docker] Create easier testing for delegation with private keys
* [docker] Update yaml
* [slash] Remove special case for slashing validator own delegation wrt min-self-delegate
* [docker] Install nano as well
* [slash] Early error if banned
* [quorum] Expose earning account in decider marshal json
* Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)"
This reverts commit 9ffbf682c0.
* [slash] Add non-sanity check way to update validator
* [reward] Increase percision on percentage in schedule
* [slash] Adjust logs
* [committee] Check eligibility of validator before doing sanity check
* [slash] Update docker
* [slash] Move create validator script to test
* [slash] More log
* [param] Make things faster
* [slash][off-chain] Clear out slashes from pending in writeblockwithstate
* [cross-link] Log is not error, just info
* [blockchain] Not necessary to guard DeletePendingSlashingCandidates
* [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that
* [slash][test] Use faucet as sender, assume user imported
* [slash] Test setup
* [slash] reserve error for real error in logs
* [slash][availability] Apply availability correct, bump signing count each block
* [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed
* [slash] Pay as much as can
* [slash] use right nowAmt
* [slash] Take away from rewards as well
* [slash] iterate faster
* [slash] Remove dev based timing
* [slash] Add more log, sanity check incoming slash records, only count external for slash rate
* [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters
* [availability] More log
* [slash] Simply pre slash erra slashing
* [slash] Remove development code
* [slash] Use height from recvMsg, todo on epoch
* [staking] Not necessary to touch LastEpochInCommittee in staking_verifier
* [slash] Undo ds in endpoint pattern config
* [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment
* [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count
Co-authored-by: Leo Chen <leo@harmony.one>
Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
Co-authored-by: Rongjian Lan <rongjian@harmony.one>
Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
* [core] Add tx-pool txn interface & update supporting components
* Add txn interface (`PoolTransaction`) for tx-pool
* Update tx_journal to handle pool's txn interface
* Update tx_list to handle pool's txn interface
* [staking] Satisfy `PoolTransaction` interface & move error sink types
* Implement `Protected`, `ToShardID`, `To`, `Data`, `Value` and `Size`
for `StakingTransaction` to satisfy `PoolTransaction` interface
* Refactor `Price` to `GasPrice` for `StakingTransaction` to
satisfy `PoolTransaction` interface
* Move error sink related components to transaction.go
* Expose `VerifyBLSKey` and `VerifyBLSKeys`
* [core] Generalize tx pool & refactor error sink logic
* Refactor txn logic to use `PoolTransaction` and `PoolTransactions`
* Add `txPoolErrorReporter` to handle reporting to plainTx and stakingTx
error sinks
* Remove old & unpayable txs error reports (to error sink) since errs
are already reported when adding the txs
* Fix known transaction error report when adding txn batches
* Add error sink reporting when failed to enqueue txs
* [node] Fix error sink & update tx pool interaction
* Integrate staking transaction in tx-pool
* Remove staking transaction error sink
* [hmy api] Integrate staking transactions from tx pool
* Remove looking at tx pool for `GetTransactionByHash`
* Add `PendingStakingTransactions` and update `PendingTransactions` to
only return plainTx
* [tests] Update all tests for tx pool txn interface & staking err sink
* Update transactions to `PoolTransaction` interface
* Remove `CommitTransactions` staking txn error sink
* Add basic staking txn tests to tx pool tests
* [node] Make all node broadcast staking tx and plain tx
* [core + staking] Separate staking msg check and put in tx pool
* Move `Validator` specific sanity check into its own method and
call said method in `ValidatorWrapper` sanity check
* Create staking msg verifiers and preprocessors in `staking_verifier.go`
* Remove staking msg verification on all staking msg applications
in `state_transition.go` and call new staking msg verifiers & preprocessors
* Add staking msg verification to tx pool
* Remove `ToShardID` from `PoolTransaction` interface and remove trivial
implementation of `ToShardID` in `StakingTransaction`
* [slash] Remove dead interface, associated piping
* [slash] Expand out structs
* [consensus] Write to a chan when find a case of double-signing, remove dead code
* [slash] Broadcast the noticing of a double signing
* [rawdb] CRUD for slashing candidates
* [slashing][node][proto] Broadcast the slash record after receive from consensus, handle received proto message, persist in off-chain db while pending
* [slash][node][propose-block] Add verified slashes proposed into the header in block proposal
* [slash][shard] Factor out external validator as method on shard state, add double-signature field
* [slash][engine] Apply slash, name boolean expression for sorts, use stable sort
* [slash] Abstract Ballot results so keep track of both pre and post double sign event
* [slash] Fix type errors on test code
* [slash] Read from correct rawdb
* [slash] Add epoch based guards in CRUD of slashing
* [slash] Write to correct cache for slashing candidates
* [shard] Use explicit named type of BLS Signature, use convention
* [slash] Fix mistake done in refactor, improper header used. Factor out fromSlice to set
* [slash][node] Restore newblock to master, try again minimial change
* [cx-receipts] Break up one-liner, use SliceStable, not Slice
* [network] Finish refactor that makes network message headers once
* [network] Simplify creation further of headers write
* [slash] Adjust data structure of slash after offline discussion with RJ, Chao
* [slash] Still did need signature of the double signature
* [consensus] Prepare message does not have block header
* [consensus] Soft reset three files to 968517d~1
* [consensus] Begin factor consensus network intended message out with prepare first
* [consensus] Factor out Prepared message
* [consensus] Factor out announce message creation
* [consensus] Committed Message, branch on verify sender key for clearer log
* [consensus] Committed Message Factor out
* [consensus] Do jenkins MVP of signatures adjustment
* [main][slash] Provide YAML config as webhook config for double sign event
* [consensus] Adjust signatures, whitespace, lessen GC pressure
* [consensus] Remove dead code
* [consensus] Factor out commit overloaded message, give commit payload override in construct
* [consensus] Fix travis tests
* [consensus] Provide block bytes in SubmitVote(quorum.Commit)
* [consensus] Factor out noisy sanity checks in BFT, move existing commit check earlier as was before
* [quorum] Adjust signatures in quorum
* [staking] Adjust after merge from master
* [consensus] Finish refactor of consensus
* [node] Fix import
* [consensus] Fix travis
* [consensus] Use origin/master copy of block, fix mistake of pointer to empty byte
* [consensus] Less verbose bools
* [consensus] Remove unused trailing mutation hook in message construct
* [consensus] Address some TODOs on err, comment out double sign
* [project] Rename specs to specifications b/c gcc thinks specs is a special file
* [rpc][node] Refactor failed stking txn buffer as container/ring, track failed plain txn as well
* [rpc] Expose RPC for staking, plain txn error sink
* [rpc][staking] Quick way to add an staking transaction error sink for rpc consumption
* [staking] Satisfy Travis
* [rpc][staking] Expose staking transaction errors using slice, protect mutex
* [staking][rpc] Use string name of Directive
* [staking][rpc] Change func signature in tests
* [staking][rpc] Protect read with lock
Max
Rongjian Lan
Jacky Wang
Daniel Van Der Maden
Ganesha Upadhyaya
Edgar Aroutiounian
flicker-harmony
Daniel Van Der Maden
chao