* Resolve harmony-one/bounties#77: Staking precompiles
Create write capable precompiles that can perform staking transactions
Add hard fork logic (EpochTBD) for these precompiles
Tests for new code with at least 80% unit test coverage
Staking library + tests in MaxMustermann2/harmony-staking-precompiles
* Fix small typo in comment
* Run goimports on files to fix Travis
* Do not activate staking precompile on shard 0
* Cascade readOnly to WriteCapableContract
* No overlap in readOnly + writeCapable precompiles
* Use function selector instead of directive
From Solidity, use abi.encodeWithSelector and match it against the
exact ABI of the functions. This allows us to remove the need for
a directive (32) being encoded, and thus saves 28 bytes of data.
* Do not allow contracts to become validators
As discussed with Jacky on #3906
* Merge harmony-one/harmony/main properly this time
* Run goimports
* Update gas calculation for staking precompile
Please see comment in core/vm/contracts_write.go RequiredGas
* Do not allow contract to become validator (2/2)
* Cache StakeMsgs from precompiled transactions
Add the StakeMsgs to ProcessorResult and cascade them in insertChain
* Remove ContractCode fields from validators
Since smart contracts can no longer beecome validators,
this field is superfluous. Remove it from the Wrapper
structure, and do not assign it a value when creating
a validator. Build and goimports checked
* Update comments in response to feedback
(1) Comments to start with function names
(2) Comments for public variables
(3) Comment to match function name RunPrecompiledContract
(4) Clarify that CreateValidatorFunc + EditValidatorFunc are still used
* Fix Travis build by reverting rosetta change
* Add revert capability to 3 staking tx types
- Delegate
- Undelegate
- CollectRewards
* Fix build: Update evm_test for ValidatorWrapper
* Merge main into harmony-staking-precompiles
* Add gas for precompile calls and allow EOA usage
- Each time the precompile is called, charge the base gas fee plus data
cost (if data can be parsed successfully). A gas fee is added to
prevent benevolent contract deployers from subsidizing the staking
transactions for EOAs through repeated assembly `delegatecall`.
- Allow EOAs to use the staking precompile directly. Some changes to
the Solidity library are associated with this change.
- Remove bytes from parsing address, since the ABI unpacks it into an
address format correctly.
- Add or update tests. Test coverage report to be attached to the PR
shortly.
* Run goimports
* Check read only and write capable for overlap
* Handle precompile stakeMsgs for block proposer
The staking precompile generates staking messages which are cascaded to
the block via the EVM in `state_processor.go`. This change cascades them
in `worker.go` to allow block proposers and block verifiers to keep the
same state.
* Run goimports for cf2dfac4081444e36a120c9432f4e..
* Update staking precompile epoch to 2 for localnet
Bring it in line with staking epoch. Change effects all configurations
except mainnet and testnet. `goimports` included.
* Add read only precompile to fetch the epoch num
* Move epoch precompile to 250
* precompiles: left pad the returned epoch number
* chainConfig: check epochs for precompiles
panic if staking precompile epoch < pre staking epoch
* Add staking migration precompile
- Lives at address 251
- Migrates delegations + pending undelegations from address A to B
- Useful if address A is hacked
- Charges gas of 21k + cost of bytes for two addresses
- Does not remove existing delegations, just sets them to zero.
Replicates current undelegate setup
- Unit tests and `goimports` included. Integration test following
shortly in MaxMustermann2/harmony-staking-precompiles
* Migration precompile: merge into staking
Merge the two precompiles into one, add gas calculation for migration
precompile. Move epoch precompile to 251 as a result. When migrating,
add undelegations to `To`'s existing undelegations, if any match the
epoch.
* Add migration gas test, remove panic, add check
In response to review comments, add tests for migration gas wherein
there are 0/1/2 delegations to migrate. Add the index out of bound check
to migration gas calculator and remove panics. Lastly, re-sort
migrated undelegations if no existing undelegation in the same epoch was
found on `To`.
* Move undelegations sorting to end of loop
* Add revert mechanism for UpdateValidatorWrapper
Closesharmony-one/bounties#90
(1) Use LRU for ValidatorWrapper objects in stateDB to plug a potential
memory leak
(2) Merge ValidatorWrapper and ValidatorWrapperCopy to let callers ask
for either a copy, or a pointer to the cached object. Additionally,
give callers the option to not deep copy delegations (which is a
heavy process). Copies need to be explicitly committed (and thus
can be reverted), while the pointers are committed when Finalise
is called.
(3) Add a UpdateValidatorWrapperWithRevert function, which is used by
staking txs `Delegate`, `Undelegate`, and `CollectRewards`. Other
2 types of staking txs and `db.Finalize` continue to use
UpdateValidateWrapper without revert, again, to save memoery
(4) Add unit tests which check
a) Revert goes through
b) Wrapper is as expected after revert
c) State is as expected after revert
* Change back to dictionary for stateValidators
Since the memory / CPU usage saved is not significantly different when
using an LRU + map structure, go back to the original dictionary
structure to keep code easy to read and have limited modifications.
* Add tests for validator wrapper reverts
As requested by @rlan35, add tests beyond just adding and reverting a
delegation. The tests are successive in the sense that we do multiple
modifications to the wrapper, save a snapshot before each modification
and revert to each of them to confirm everything works well. This change
improves test coverage of statedb.go to 66.7% from 64.8% and that of
core/state to 71.9% from 70.8%, and covers all the code that has been
modified by this PR in statedb.go.
For clarity, the modifications to the wrapper include (1) creation of
wrapper in state, (2) adding a delegation to the wrapper, (3)
increasing the blocks signed, and (4) a change in the validator Name and
the BlockReward. Two additional tests have been added to cover the
`panic` and the `GetCode` cases.
* Add ethereum transaction support
* fix fmt
* fix lint
* add new block body fork logic
* add back deleted code
* lower case block version const
* add eth txn into new version of block
* refactor back
* refactor more
* fix test code
* Fix build
* Fix build
* revert eth txn in block and block proposal
* fix build
* fix build
* fix build
* Add ethereum transaction support
* fix fmt
* fix lint
* add new block body fork logic
* add back deleted code
* lower case block version const
* add eth txn into new version of block
* refactor back
* refactor more
* fix test code
* Fix build
* Fix build
* Rollup evm to geth v1.9.9 Muir Glacier
* fix go gen
* update intrinsic gas for istanbul
* Update statedb.Commit
* revert go gen result
* revert protobuf version
* update protobuf
* update go gen files
* set testnet epoch for evm upgrade
* [rpc] Fix sanity check to accept "latest"
* [rpc] Return as much data possible when using GetAllValidatorInformationByBlockNumber
* [rpc] Remove beacon check for getValidators, returns valid information on all shards
* [rpc] Add GetValidatorKeys to get BLS Keys of committee for a particular epoch
* [rpc] Add getBlockSignerKeys to return public bls keys of signers for a block
* [rpc] Fix lint
* [rpc] Address PR comments
* [rpc] Add common function to API backend instead of separate implementations in V1 & V2
* [rpc] Remove extra parenthesis
* [rpc] Fix lint
[validator] Fix typo
* [rpc] Revert change to sanity check in V2
* [rpc] Fix lint
* fix signers share computation to use only non-harmony nodes (#3008)
* Add LastEpochInCommittee at prestaking epoch too (#3014)
* write validator stats at the end of prestaking epoch (#3021)
* write validator stats at the end of prestaking epoch
* also need to write block reward accumulator
* [build] enable static build by default (#2885)
scripts/go_executable_build.sh -s (static build, default)
scripts/go_executable_build.sh -S (non-static build)
Signed-off-by: Leo Chen <leo@harmony.one>
* Code formatting fix based on `staticcheck` (#3023)
* fix some static check issues
* more fixes according to staticcheck
* add nil check on getHeader
* Add more log info
* [rpc] Revert change to v2 GetBalance RPC (#3037)
* Cherrypick crosslink related fixes from t3 to master. (#3045)
* Optimize crosslink verification logic and add more cache (#3032)
* make crosslink broadcast smarter and more efficient (#3036)
* adjust catch up speed to be a sane number
* Fix crosslink broadcast condition (#3041)
* Tx pool error report fix & local build fix (#3046)
* [tx_list] Add transaction after you can fetch tx cost
* [tx_pool] Add error sink reports for removed tx
* Correct log levels for known transactions
* Remove redundant hash to `enqueue` and `add` signatures
* Update tests for signature changes
* [build] Fix local build for MacOs
* [tx_pool] Correct typo & logging level for known tx
* [config] Create Testnet config
* [node.sh] update testnet network config
* [config] correct testnet config
* [main] Enable beacon sync for explorer nodes (#3059)
* [apr] should not use snapshot from last epoch, use current epoch snapshot (#3051)
* use snapshot total delegation instead of current total delegation
* should not use snapshot from last epoch, use current epoch snapshot (#3049)
Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com>
Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com>
* cherrypick log level change
* [rpc] Add active status back to Validator information
[validator] Fix typo for ValidatorRPCEnhanced
[Makefile] Change make exe to build non-static by default
* [network] Added caching to node server (#3048)
* [network] Added caching to node server
* [network] renamed two singleflight.Group
* [network] added two error handling
* [network] added forget logic for single flight
* [network] changed single flight to lru cache solution
* [network] adjusted the cache size
* [network] changed cache size to 10000
* Transaction api improvements (#3057)
* [node] Add option to broadcast invalid tx (on by default)
* update main args to take in broadcast invalid tx
* update node.sh for new broadcast invalid tx option
* [rpc] Do not submit transaction if tx has been finalized
* Support legacy behavior and not return errors on tx submission
* [main] make broadcasting invalid tx default false
* [node] Fix formatting
* Add timeouts for the explorer HTTP service
* [test] State trans unit test (#3043)
* completing create-validator tests
generalizing create validator tests, started with edit validator tests
one round completed before major refactor
* [test] finished coding test TestCheckDuplicateFields
* [test] finished debug TestCheckDuplicateFields
* [test] Added TestDescription_EnsureLength
* [test] added test in validator_test.go TestUpdateDescription
* [test] Added test TestComputed_String
* [test] refactored TestValidatorSanityCheck and added some message in error message returned from sanity check
* [test] removed test index for debugging
* [test] Added test TestValidatorWrapper_SanityCheck. Added value check for TestMarshalUnmarshalValidator. Fixed rlp usage defect in UnmarshalValidator
* [test] added test TestVerifyBLSKeys
* [test] added test TestContainsHarmonyBLSKeys
* [test] added test TestCreateValidatorFromNewMsg
* [test] fixed some tests errors
* [test] added test TestUpdateValidatorFromEditMsg, last editted the test file, finished testing validator_test.go
* [staking] added numeric.Dec.Copy method
* [staking] added CommissionRates.Copy
* [staking] add nil case handle to copy method and related test case
* [test] added nil case for commissionRates.Copy test
* [staking] finished CreateValidator.Copy and related test case
* [staking] added EditValidator.Copy method and related test case
* [test] added zero test cases for Copy method
* [staking] implemented Delegate.Copy and related unit test case
* [staking] added Undelegate.Copy and CollectReward.Copy method. Implemented corresponding test cases
* [test] added two more simple unit tests
* [test] solving the merge conflict with deep copy fix
* [test] added schedule for sharding
* [test] refactored double_sign_test.go
* [test] comment out all test cases for now.
* [test] added record Copy method for testing
* [test] added the first test case in TestVerify
* [test] finished test TestVerify
* [test] Added test TestVerify, TestApplySlashRate, TestSetDifference
* [test] half through TestDelegatorSlashApply
* [test] fix the problem cause by unsafe usage of big.Int
* [staking] added the copy methods to validatorWrapper
* [test] added test cases for copy method for staking/validatorWrapper
* [test] added test case TestApply
* [test] added test case TestRate
* [test] fix the golint errors
* [test] removed commented out unit test code
* [test] remote the empty line in imports
* [test] moved copy methods to stake_testing module
* [test] removed usage of interface stateDB
* [test] removed empty lines in imports
* [test] fixed golint package name issues
* [test] removed a todo when writing code
* [test] moved record.Copy to staking/slash/test
* [test] add some changes
* [test] added prototypes in staketest module
* [test] fix the golint issue in staketest
* [test] make prototype more beautiful
* [test] refactored test TestCheckDuplicateFields
* [test] add createValidator test data
* [test] added positive test case for VerifyAndCreateValidatorFromMsg
* [test] added create validator test
* [test] added the positive test case for TestVerifyAndEditValidatorFromMsg
* [test] added the tests case TestVerifyAndEditValidatorFromMsg
* [test] fix one of the error is not triggered as expected
* [test] more changes in test
* [test] fix the positive test data for TestVerifyAndDelegateFromMsg
* [test] Fixed two comment golint errors
* [test] added delegate tests
* [test] added a make function. WIP
* [test] added undelegate positive test
* [test] added negative test cases for undelegate
* [test] added positive test. not passed. Need to add assert method in test suites
* [test] added equal function to check Validator Equality
* [test] added equals for staketest
* [test] replaced deep equal with new equal methods:
* [test] removed unused codes
* [test] Finishing touch
* [test] fix comment golint issue
* [test] removed dead code in staking_verifier_test.go
Co-authored-by: Ganesha Upadhyaya <ganeshrvce@gmail.com>
* changed the sync port to dns port in flag
* [network] Limit client sync connection (#3071)
* [network] limit client connected peers
* Changed signature and added tests
* Set secure permissions on generated blskey pass file in node.sh
* Fix spellin gtypo in node.sh
* fix latest param handling in api v1
* added comment for the isGreaterThanLatestBlockNum check
* fix a nil pointer crash due to previous revert
Signed-off-by: Leo Chen <leo@harmony.one>
* fix nil pointer crash
Signed-off-by: Leo Chen <leo@harmony.one>
* fix nil pointer of block
Signed-off-by: Leo Chen <leo@harmony.one>
* [rpc] add viewID and epoch to RPCMarshalBlock (#3009)
* [rpc] add viewID and epoch to RPCMarshalBlock
* add apt-get update
* Do not rollback current block on failed block sync (#3101)
Co-authored-by: Ganesha Upadhyaya <ganeshrvce@gmail.com>
Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com>
Co-authored-by: Jacky Wang <jackyw.se@gmail.com>
Co-authored-by: Janet Liang <56005637+janet-harmony@users.noreply.github.com>
Co-authored-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
Co-authored-by: Janet Liang <janet@harmony.one>
Co-authored-by: Sebastian Johnsson <sebastian.johnsson@gmail.com>
Co-authored-by: hsiung <hsiung@pinterest.com>
Co-authored-by: Dennis Won <jhwon0820@gmail.com>
Co-authored-by: Yishuang Chen <34232522+ivorytowerdds@users.noreply.github.com>
* completing create-validator tests
generalizing create validator tests, started with edit validator tests
one round completed before major refactor
* [test] finished coding test TestCheckDuplicateFields
* [test] finished debug TestCheckDuplicateFields
* [test] Added TestDescription_EnsureLength
* [test] added test in validator_test.go TestUpdateDescription
* [test] Added test TestComputed_String
* [test] refactored TestValidatorSanityCheck and added some message in error message returned from sanity check
* [test] removed test index for debugging
* [test] Added test TestValidatorWrapper_SanityCheck. Added value check for TestMarshalUnmarshalValidator. Fixed rlp usage defect in UnmarshalValidator
* [test] added test TestVerifyBLSKeys
* [test] added test TestContainsHarmonyBLSKeys
* [test] added test TestCreateValidatorFromNewMsg
* [test] fixed some tests errors
* [test] added test TestUpdateValidatorFromEditMsg, last editted the test file, finished testing validator_test.go
* [staking] added numeric.Dec.Copy method
* [staking] added CommissionRates.Copy
* [staking] add nil case handle to copy method and related test case
* [test] added nil case for commissionRates.Copy test
* [staking] finished CreateValidator.Copy and related test case
* [staking] added EditValidator.Copy method and related test case
* [test] added zero test cases for Copy method
* [staking] implemented Delegate.Copy and related unit test case
* [staking] added Undelegate.Copy and CollectReward.Copy method. Implemented corresponding test cases
* [test] added two more simple unit tests
* [test] solving the merge conflict with deep copy fix
* [test] added schedule for sharding
* [test] refactored double_sign_test.go
* [test] comment out all test cases for now.
* [test] added record Copy method for testing
* [test] added the first test case in TestVerify
* [test] finished test TestVerify
* [test] Added test TestVerify, TestApplySlashRate, TestSetDifference
* [test] half through TestDelegatorSlashApply
* [test] fix the problem cause by unsafe usage of big.Int
* [staking] added the copy methods to validatorWrapper
* [test] added test cases for copy method for staking/validatorWrapper
* [test] added test case TestApply
* [test] added test case TestRate
* [test] fix the golint errors
* [test] removed commented out unit test code
* [test] remote the empty line in imports
* [test] moved copy methods to stake_testing module
* [test] removed usage of interface stateDB
* [test] removed empty lines in imports
* [test] fixed golint package name issues
* [test] removed a todo when writing code
* [test] moved record.Copy to staking/slash/test
* [test] add some changes
* [test] added prototypes in staketest module
* [test] fix the golint issue in staketest
* [test] make prototype more beautiful
* [test] refactored test TestCheckDuplicateFields
* [test] add createValidator test data
* [test] added positive test case for VerifyAndCreateValidatorFromMsg
* [test] added create validator test
* [test] added the positive test case for TestVerifyAndEditValidatorFromMsg
* [test] added the tests case TestVerifyAndEditValidatorFromMsg
* [test] fix one of the error is not triggered as expected
* [test] more changes in test
* [test] fix the positive test data for TestVerifyAndDelegateFromMsg
* [test] Fixed two comment golint errors
* [test] added delegate tests
* [test] added a make function. WIP
* [test] added undelegate positive test
* [test] added negative test cases for undelegate
* [test] added positive test. not passed. Need to add assert method in test suites
* [test] added equal function to check Validator Equality
* [test] added equals for staketest
* [test] replaced deep equal with new equal methods:
* [test] removed unused codes
* [test] Finishing touch
* [test] fix comment golint issue
* [test] removed dead code in staking_verifier_test.go
Co-authored-by: Ganesha Upadhyaya <ganeshrvce@gmail.com>
* validator stats regression due to apr bug fixed
* better coding style
* correctly computing apr accounting for epoch holes
* APRHistory for only 30 last epochs
* [test] added schedule for sharding
* [test] refactored double_sign_test.go
* [test] comment out all test cases for now.
* [test] added record Copy method for testing
* [test] added the first test case in TestVerify
* [test] finished test TestVerify
* [test] Added test TestVerify, TestApplySlashRate, TestSetDifference
* [test] half through TestDelegatorSlashApply
* [test] fix the problem cause by unsafe usage of big.Int
* [staking] added the copy methods to validatorWrapper
* [test] added test cases for copy method for staking/validatorWrapper
* [test] added test case TestApply
* [test] added test case TestRate
* [test] fix the golint errors
* [test] removed commented out unit test code
* [test] remote the empty line in imports
* [test] moved copy methods to stake_testing module
* [test] removed usage of interface stateDB
* [test] removed empty lines in imports
* [test] fixed golint package name issues
* [test] removed a todo when writing code
* [test] moved record.Copy to staking/slash/test
* [types] Add TransactionErrorSink to report failed txs
[node] Create ErrorSink handler for unique error msgs
Implemented with a LRU cache.
[node] Rename ErrorSink to TransactionErrorSink
* Rename RPCTransactionError to TransactionError
[node] Make tx error sink not return err on Add & Remove
[node] Make tx errorSink Contains check take string as tx hash param
[errorsink] Move tx error sink into errorsink pkg
[errorsink] Rename Errors and Count methods
[errorsink] Rename NewTransactionErrorSink to NewTransactionSink
[types] Move error sink to core/types
* Rename NewTransactionSink to NewTransactionErrorSink
* [types] Fix log msg for unfound errors
* [types] Rename TransactionError to TransactionErrorReport
* [core] Remove RPCTransactionError & refactor tx_pool to use TxErrorSink
* [staking] Remove RPCTransactionError
* [node] Refactor tx_pool init to use new TxErrorSink
* [main] Construct transaction error sink before initing the node
* [node] Refactor error sink reporting at RPC layer
* [rpc] Refactor returned type of ErrorSink RPCs to 1 type
* [core] Remove tx from TxErrorSink on Add to tx_pool
* [types] Make NewTransactionErrorSink not return err
* [node] Make node.New create error sink
* [cmd] Revert to origin main.go
* [core] Add TxErrorSink unit test & fix bad ErrExcessiveBLSKeys in tests
* [testnet config] Change testnet config to allow for 5*4 external keys
* [cmd] Revert main.go to original node instantiation
* [rpc] Add GetPoolStats rpc to fetch pending and queued tx counts
* completing create-validator tests
generalizing create validator tests, started with edit validator tests
one round completed before major refactor
* [test] finished coding test TestCheckDuplicateFields
* [test] finished debug TestCheckDuplicateFields
* [test] Added TestDescription_EnsureLength
* [test] added test in validator_test.go TestUpdateDescription
* [test] Added test TestComputed_String
* [test] refactored TestValidatorSanityCheck and added some message in error message returned from sanity check
* [test] removed test index for debugging
* [test] Added test TestValidatorWrapper_SanityCheck. Added value check for TestMarshalUnmarshalValidator. Fixed rlp usage defect in UnmarshalValidator
* [test] added test TestVerifyBLSKeys
* [test] added test TestContainsHarmonyBLSKeys
* [test] added test TestCreateValidatorFromNewMsg
* [test] fixed some tests errors
* [test] added test TestUpdateValidatorFromEditMsg, last editted the test file, finished testing validator_test.go
* [staking] added numeric.Dec.Copy method
* [staking] added CommissionRates.Copy
* [staking] add nil case handle to copy method and related test case
* [test] added nil case for commissionRates.Copy test
* [staking] finished CreateValidator.Copy and related test case
* [staking] added EditValidator.Copy method and related test case
* [test] added zero test cases for Copy method
* [staking] implemented Delegate.Copy and related unit test case
* [staking] added Undelegate.Copy and CollectReward.Copy method. Implemented corresponding test cases
* [test] added two more simple unit tests
* [test] solving the merge conflict with deep copy fix
* [test] restore the changes made to staking_verifier_test.go
* [test] resolve test conflicts in core
Co-authored-by: Ganesha Upadhyaya <ganeshrvce@gmail.com>
* [staking] added numeric.Dec.Copy method
* [staking] added CommissionRates.Copy
* [staking] add nil case handle to copy method and related test case
* [test] added nil case for commissionRates.Copy test
* [staking] finished CreateValidator.Copy and related test case
* [staking] added EditValidator.Copy method and related test case
* [test] added zero test cases for Copy method
* [staking] implemented Delegate.Copy and related unit test case
* [staking] added Undelegate.Copy and CollectReward.Copy method. Implemented corresponding test cases
* [test] added two more simple unit tests
* apr average over last hundred epochs
* minor
* upcounting once activation to include the impact on apr when not in committeee
* validator stats maintain APR histories as fixed length circular slice
* use map instead of slice for flexibility
* use APRHistoryLength instead of 100
* APR for currentEpochSuperCommittee.Epoch and not newEpochSuperCommittee.Epoch
* Add cache to state validator access; refactor snapshot read/write
* make validator update happen at commit
* fix lint
* make sure slashing applies on validator wrapper in state
* Revert live update of validtor wrapper in staking txn
* add init population of validator cache logic
* Fix validator cache commit logic
* Separate validatorWrapper func
* fix build
* Fix lint
* [project] Dead test
* [node] If fields always same, then just use the constant
* [project] Remove overcomplicated ctxerror
* [project] Remove more dead tests, adjust error replacing ctxerror
* [project] More dead tests, fix travis complaint on Errorf
* [node.sh] Remove is-genesis
* [internal] Remove dead RPC/wallet
* [rpc] Reduce API surface area for dead RPC
* [node] Remove dead flags, unused setters
* [project] Unused logger, too many loggers
* [core] staticcheck fixes, remove unused db keys
* [project] More dead code through, now in param and genesis
* [internal] Useless test
peekpi
Konstantin
harjas
Max
Rongjian Lan
xiaohuo
Jacky Wang
Ganesha Upadhyaya
Janet Liang
Leo Chen
Janet Liang
Daniel Van Der Maden
Edgar Aroutiounian