* Add revert mechanism for UpdateValidatorWrapper
Closesharmony-one/bounties#90
(1) Use LRU for ValidatorWrapper objects in stateDB to plug a potential
memory leak
(2) Merge ValidatorWrapper and ValidatorWrapperCopy to let callers ask
for either a copy, or a pointer to the cached object. Additionally,
give callers the option to not deep copy delegations (which is a
heavy process). Copies need to be explicitly committed (and thus
can be reverted), while the pointers are committed when Finalise
is called.
(3) Add a UpdateValidatorWrapperWithRevert function, which is used by
staking txs `Delegate`, `Undelegate`, and `CollectRewards`. Other
2 types of staking txs and `db.Finalize` continue to use
UpdateValidateWrapper without revert, again, to save memoery
(4) Add unit tests which check
a) Revert goes through
b) Wrapper is as expected after revert
c) State is as expected after revert
* Change back to dictionary for stateValidators
Since the memory / CPU usage saved is not significantly different when
using an LRU + map structure, go back to the original dictionary
structure to keep code easy to read and have limited modifications.
* Add tests for validator wrapper reverts
As requested by @rlan35, add tests beyond just adding and reverting a
delegation. The tests are successive in the sense that we do multiple
modifications to the wrapper, save a snapshot before each modification
and revert to each of them to confirm everything works well. This change
improves test coverage of statedb.go to 66.7% from 64.8% and that of
core/state to 71.9% from 70.8%, and covers all the code that has been
modified by this PR in statedb.go.
For clarity, the modifications to the wrapper include (1) creation of
wrapper in state, (2) adding a delegation to the wrapper, (3)
increasing the blocks signed, and (4) a change in the validator Name and
the BlockReward. Two additional tests have been added to cover the
`panic` and the `GetCode` cases.
1. Added some functionality to HostV2 for stream support.
2. Added new module discovery under p2p host to replace the networkInfo service.
3. Make dht datastore badger only active on bootstrap nodes. For normal nodes, badger is disabled by default. It can also be enabled by flags.
Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com>
* Add ethereum transaction support
* fix fmt
* fix lint
* add new block body fork logic
* add back deleted code
* lower case block version const
* add eth txn into new version of block
* refactor back
* refactor more
* fix test code
* Fix build
* Fix build
* revert eth txn in block and block proposal
* fix build
* fix build
* fix build
* Add ethereum transaction support
* fix fmt
* fix lint
* add new block body fork logic
* add back deleted code
* lower case block version const
* add eth txn into new version of block
* refactor back
* refactor more
* fix test code
* Fix build
* Fix build
* [core] Add FindLogsWithTopic & unit test
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [hmy] Add GetDetailedBlockSignerInfo
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [hmy] Add IsCommitteeSelectionBlock
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [test] Add test transaction creation helpers
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Refactor account.go & add tests
* Move TestNewAccountIdentifier & TestGetAddress to account_test.go
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Move Operation & Tx formatting to own files
* Move Respective unit tests to own files
* Expose GetOperations & GetStakingOperations
* Expose FormatTransaction, FormatCrossShardReceiverTransaction,
FormatGenesisTransaction, FormatPreStakingRewardTransaction & FormatUndelegationPayoutTransaction
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Move TransactionMetadata to transaction_construction.go
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Update construction to use new helpers & formatters
* Make docs consistent for mempool.go
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Move all special tx & blk handling to own file
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Remove all moved fns, methods & tests from block.go
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* Fix lint & imports
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Rename all tx related files for clarity
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Rename DefaultSenderAddress to FormatDefaultSenderAddress
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Rename Currency to NativeCurrency
* This is in anticipation of HRC20 token support with rosetta
* Rename various native operation functions accordingly
* Add documentation to explain what a native token is
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix pre-staking block reward calculation
* Move getPreStakingRewardTransactionIdentifiers to block_special.go
* Add epoch to block metadata
* Update unit tests
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* Add IsLastBlockInEpoch method to Block & Header
* Refactor all uses of length check `ShardState`
* [hmy] Refactor IsCommitteeSelectionBlock to use chain.IsCommitteeSelectionBlock
* Address PR comments
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Update var names in preStakingRewardBlockTransaction
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [cmd] Add IsOffline flag
* Add IsOffline to node config
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [node] Disable syncing and p2p msgs if offline
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Clarify syncing status msg
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [internal/config] Add DefaultLocalListenIP & DefaultPublicListenIP
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [cmd] Add IP to p2p config
* Use default local listening ip for p2p hors if node is in offline
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [p2p] Use peer ip when creating listenAddr
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [cmd] Fix unit tests & bump config version
* Fix TestAddPeer & TestConnectionToInvalidPeer p2p test
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [cmd] Fix lint
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [cmd] Address PR comments
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* Rollup evm to geth v1.9.9 Muir Glacier
* fix go gen
* update intrinsic gas for istanbul
* Update statedb.Commit
* revert go gen result
* revert protobuf version
* update protobuf
* update go gen files
* set testnet epoch for evm upgrade
* [rosetta] Add server stop
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Make network naming consistent
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Correct common package name & add error enum
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Remove needless forward of network info to services
* Implement /network/list
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Refactor errors & add operation statuses and types
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Implement NetworkOptions & update NetworkAPIService
* Rename *_service.go files to remove the suffix
* Update StartServers to use new operation types
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Impl NetworkStatus - Finish init impl of /network endpoint
* Fix import structure for rosetta.go
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [test] Make explorer run as archival for localnet
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add unit tests
* Force errors to remain the same with unit tests
* Force operations to remain the same with unit tests
* Ensure network checking works for all cases with unit tests
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add InvalidNetworkError and correct error codes
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add UnmarshalFromInterface for SubNetworkMetadata
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add network checking
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Nit fixes & add unit test for Peer Info
* Make names consistent
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add BlockNotFoundError & TransactionNotFoundError
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Implement skeleton for block transactions
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add ReceiptNotFoundError
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add receipt to formatTransaction sig for contract fails
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add currency, ExpendGasOperation, & ContractCreationOperation
* Add Error creator
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Init impl of plain transaction formatting
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Update network.go for new error constructor
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Implement stx formatter & refactor BlockTransaction
* Updated todo comments & function formatting
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Impl Block & make currency non-ptr for easy copy with custom metadata
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix collect rewards amount on transaction fetch
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix block look-up edge case & add recovery middleware
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add bocks unit tests
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix checkPeerID unit test in network_test.go
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix staking tx amount for tx ops & update inline docs
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix lint
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Refactor getStakingOperations
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix undelegate value
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Nit - fix formatting for network.go
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [node] Move genesis allocation to core & remove unused ContractDeployerKey
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix precision error & add cx receipt hash on blk fetch
* Add unit tests for supporting helper functions
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix fmt
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix cx receipt hashes for blocks
* Print stack trace on panic recovery
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [node] Nit - fix comment for StopRosetta
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [node] Expose GetMaxPeerHeight
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add SyncStatus enum
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Nit - remove redundant 'service' name in services namespace
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add server with example block & network services
* Update go.mod for rosetta SDK
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [internal/configs] Add RosettaServer to node config
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [cmd] Add rosetta port flag
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [node] Add rosetta server start & rename api.go
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Change server start to use nodeconfig of rosetta
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [cmd] Cleanup debugging prints
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix stdout print & document placeholders
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix lint & make StartSevers more consistent
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [cmd] Disable rosetta by default
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [test] Make explorer deploy rosetta server
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Use direct http server for start
* Make go.mod changes minimal
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix fmt
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix fmt of go.mod
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Use port 9700 instead of 10000
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [cmd] Bump config version
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [cmd] Add v1.0.0 config backwards compatibility test
* Included update message if old config is loaded
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [test] Do not broadcast invalid tx on localnet
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [cmd] Correct for invalid port when loading old config
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [cmd] Make rosetta variable names consistent
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
This is a big PR merged many small commits together.
We add the message validation function in libp2p layer.
In the validation function, we check the following conditions
1) the p2p message is a valid consensus message
2) the p2p message sender has a valid public key
3) the sender's public key is in the current committee
4) log the number of invalid/valid messages
After the validation, the valid messages will be forward to the network,
while the invalid messages will be filtered out.
The messages intended for the validator will be handled in the consensus layer.
Signed-off-by: Leo Chen <leo@harmony.one>
* [tx_list] Add transaction after you can fetch tx cost
* [tx_pool] Add error sink reports for removed tx
* Correct log levels for known transactions
* Remove redundant hash to `enqueue` and `add` signatures
* Update tests for signature changes
* [build] Fix local build for MacOs
* [tx_pool] Correct typo & logging level for known tx
* [tx_pool] Fix for go imports
* [tx_list] Add transaction after you can fetch tx cost
* [tx_pool] Add error sink reports for removed tx
* Correct log levels for known transactions
* Remove redundant hash to `enqueue` and `add` signatures
* Update tests for signature changes
* [build] Fix local build for MacOs
* [tx_pool] Correct typo & logging level for known tx
* [types] Add TransactionErrorSink to report failed txs
[node] Create ErrorSink handler for unique error msgs
Implemented with a LRU cache.
[node] Rename ErrorSink to TransactionErrorSink
* Rename RPCTransactionError to TransactionError
[node] Make tx error sink not return err on Add & Remove
[node] Make tx errorSink Contains check take string as tx hash param
[errorsink] Move tx error sink into errorsink pkg
[errorsink] Rename Errors and Count methods
[errorsink] Rename NewTransactionErrorSink to NewTransactionSink
[types] Move error sink to core/types
* Rename NewTransactionSink to NewTransactionErrorSink
* [types] Fix log msg for unfound errors
* [types] Rename TransactionError to TransactionErrorReport
* [core] Remove RPCTransactionError & refactor tx_pool to use TxErrorSink
* [staking] Remove RPCTransactionError
* [node] Refactor tx_pool init to use new TxErrorSink
* [main] Construct transaction error sink before initing the node
* [node] Refactor error sink reporting at RPC layer
* [rpc] Refactor returned type of ErrorSink RPCs to 1 type
* [core] Remove tx from TxErrorSink on Add to tx_pool
* [types] Make NewTransactionErrorSink not return err
* [node] Make node.New create error sink
* [cmd] Revert to origin main.go
* [core] Add TxErrorSink unit test & fix bad ErrExcessiveBLSKeys in tests
* [testnet config] Change testnet config to allow for 5*4 external keys
* [cmd] Revert main.go to original node instantiation
* [rpc] Add GetPoolStats rpc to fetch pending and queued tx counts
* processing staking and normal transactions together
* bug fix and removing unwanted method
* made sense to rename and move the TransactionsByPriceAndNonce to pool type
* accounting for prestaking blocks, need to keep the previous order for processing txs
* Add cache to state validator access; refactor snapshot read/write
* make validator update happen at commit
* fix lint
* make sure slashing applies on validator wrapper in state
* Revert live update of validtor wrapper in staking txn
* add init population of validator cache logic
* Fix validator cache commit logic
* Separate validatorWrapper func
* fix build
* Fix lint
Konstantin
Max
yahtoo
Rongjian Lan
peekpi
xiaohuo
Jacky Wang
Daniel Van Der Maden
Leo Chen
Sebastian Johnsson
Ganesha Upadhyaya