* [rawdb] add error handling to all rawdb write. Add fdlimit module. Fix the node stuck
* [core] switch back the batch write condition in InsertReceiptChain
* [rawdb] add error handling to all rawdb write. Add fdlimit module. Fix the node stuck
* [consensus] refactored and optimized tryCatchup logic
* [sync] added consensus last mile block in sync.
* [consensus] remove time wait for consensus inform sync. Make block low chan a buffered chan
* [consensus] fix rebase errors, and optimize one line code
* [consensus][sync] fix golint error and added prune logic in sync
* [consensus] move header verify after adding FBFT log in onPrepared
* [consensus] more change on block verification logic
* [consensus] fix the verified panic issue
* [consensus][sync] add block verification in consensus last mile, change it to iterator
* [consensus] fix two nil pointer references when running local node (Still cannot find the root cause for it)
* remove coverage.txt and add to gitignore
* [consensus] add leader key check. Move quorum check logic after tryCatchup and can spin state sync
* [consensus] remove the leader sender check for now. Will add later
* [consensus] refactor fbftlog to get rid of unsafe mapset module. Replace with map
* [consensus] move the isQuorumAchived logic back. We surely need to check it before add to FBFTlog
* [consensus] remove the redundant block nil check
* [test] fix the consensus test
* [consensus] rebase main and fix stuff. Removed isSendByLeader
* [consensus] added logic to spin up sync when received message is greater than consensus block number
* [consensus] more changes in consensus. Remove some spin sync logic.
* fix error in main
* [consensus] change the hash algorithm of the FBFTLog to get rid of rlp error
* [consensus] use seperate mutex in FBFT message
* [consensus] change fbft log id to a shorter form. Added unit test case
* [rawdb] add error handling to all rawdb write. Add fdlimit module. Fix the node stuck
* [core] switch back the batch write condition in InsertReceiptChain
* [core] add comments on isUnrecoverableErr
* Rollup evm to geth v1.9.9 Muir Glacier
* fix go gen
* update intrinsic gas for istanbul
* Update statedb.Commit
* revert go gen result
* revert protobuf version
* update protobuf
* update go gen files
* set testnet epoch for evm upgrade
* [consensus] enable the consensus module can spin up doSync
* [consensus] remove the blockNumLowChan buffer in onCommitted. Move tryCatchup logic before informing blockNumLow
* [sync] Remove an unnecessary time ticker in sync to reduce one sync loop time from 3.5s to 1s (based on testnet)
* [consensus] also spin up state sync at onPrepared
* [consensus] removed onCommitted block number check since redundent
* [rosetta] Add server stop
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Make network naming consistent
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Correct common package name & add error enum
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Remove needless forward of network info to services
* Implement /network/list
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Refactor errors & add operation statuses and types
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Implement NetworkOptions & update NetworkAPIService
* Rename *_service.go files to remove the suffix
* Update StartServers to use new operation types
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Impl NetworkStatus - Finish init impl of /network endpoint
* Fix import structure for rosetta.go
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [test] Make explorer run as archival for localnet
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add unit tests
* Force errors to remain the same with unit tests
* Force operations to remain the same with unit tests
* Ensure network checking works for all cases with unit tests
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add InvalidNetworkError and correct error codes
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add UnmarshalFromInterface for SubNetworkMetadata
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add network checking
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Nit fixes & add unit test for Peer Info
* Make names consistent
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add BlockNotFoundError & TransactionNotFoundError
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Implement skeleton for block transactions
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add ReceiptNotFoundError
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add receipt to formatTransaction sig for contract fails
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add currency, ExpendGasOperation, & ContractCreationOperation
* Add Error creator
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Init impl of plain transaction formatting
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Update network.go for new error constructor
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Implement stx formatter & refactor BlockTransaction
* Updated todo comments & function formatting
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Impl Block & make currency non-ptr for easy copy with custom metadata
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix collect rewards amount on transaction fetch
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix block look-up edge case & add recovery middleware
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add bocks unit tests
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix checkPeerID unit test in network_test.go
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix staking tx amount for tx ops & update inline docs
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix lint
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Refactor getStakingOperations
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix undelegate value
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Nit - fix formatting for network.go
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [node] Move genesis allocation to core & remove unused ContractDeployerKey
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix precision error & add cx receipt hash on blk fetch
* Add unit tests for supporting helper functions
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix fmt
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix cx receipt hashes for blocks
* Print stack trace on panic recovery
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [node] Nit - fix comment for StopRosetta
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [node] Expose GetMaxPeerHeight
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add SyncStatus enum
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Nit - remove redundant 'service' name in services namespace
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* Fix minor typos
* Properly sort imports
* [explorer] Add /node-sync to return if the node is in sync
* [explorer] Use different status code for node out of sync & JSON error
* [main] Do not initialize syncing as a separate go routine to ensure stateSync pointer is initialized before explorer service creation (adds 0.002 seconds to node start up time)
[sync] Fix nil pointer on syncConfig, return true since node has not yet checked if it is in sync
* [explorer] Reverse sync for display
This is a big PR merged many small commits together.
We add the message validation function in libp2p layer.
In the validation function, we check the following conditions
1) the p2p message is a valid consensus message
2) the p2p message sender has a valid public key
3) the sender's public key is in the current committee
4) log the number of invalid/valid messages
After the validation, the valid messages will be forward to the network,
while the invalid messages will be filtered out.
The messages intended for the validator will be handled in the consensus layer.
Signed-off-by: Leo Chen <leo@harmony.one>
* fix signers share computation to use only non-harmony nodes (#3008)
* Add LastEpochInCommittee at prestaking epoch too (#3014)
* write validator stats at the end of prestaking epoch (#3021)
* write validator stats at the end of prestaking epoch
* also need to write block reward accumulator
* [build] enable static build by default (#2885)
scripts/go_executable_build.sh -s (static build, default)
scripts/go_executable_build.sh -S (non-static build)
Signed-off-by: Leo Chen <leo@harmony.one>
* Code formatting fix based on `staticcheck` (#3023)
* fix some static check issues
* more fixes according to staticcheck
* add nil check on getHeader
* Add more log info
* [rpc] Revert change to v2 GetBalance RPC (#3037)
* Cherrypick crosslink related fixes from t3 to master. (#3045)
* Optimize crosslink verification logic and add more cache (#3032)
* make crosslink broadcast smarter and more efficient (#3036)
* adjust catch up speed to be a sane number
* Fix crosslink broadcast condition (#3041)
* Tx pool error report fix & local build fix (#3046)
* [tx_list] Add transaction after you can fetch tx cost
* [tx_pool] Add error sink reports for removed tx
* Correct log levels for known transactions
* Remove redundant hash to `enqueue` and `add` signatures
* Update tests for signature changes
* [build] Fix local build for MacOs
* [tx_pool] Correct typo & logging level for known tx
* [config] Create Testnet config
* [node.sh] update testnet network config
* [config] correct testnet config
* [main] Enable beacon sync for explorer nodes (#3059)
* [apr] should not use snapshot from last epoch, use current epoch snapshot (#3051)
* use snapshot total delegation instead of current total delegation
* should not use snapshot from last epoch, use current epoch snapshot (#3049)
Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com>
Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com>
* cherrypick log level change
* [rpc] Add active status back to Validator information
[validator] Fix typo for ValidatorRPCEnhanced
[Makefile] Change make exe to build non-static by default
* [network] Added caching to node server (#3048)
* [network] Added caching to node server
* [network] renamed two singleflight.Group
* [network] added two error handling
* [network] added forget logic for single flight
* [network] changed single flight to lru cache solution
* [network] adjusted the cache size
* [network] changed cache size to 10000
* Transaction api improvements (#3057)
* [node] Add option to broadcast invalid tx (on by default)
* update main args to take in broadcast invalid tx
* update node.sh for new broadcast invalid tx option
* [rpc] Do not submit transaction if tx has been finalized
* Support legacy behavior and not return errors on tx submission
* [main] make broadcasting invalid tx default false
* [node] Fix formatting
* Add timeouts for the explorer HTTP service
* [test] State trans unit test (#3043)
* completing create-validator tests
generalizing create validator tests, started with edit validator tests
one round completed before major refactor
* [test] finished coding test TestCheckDuplicateFields
* [test] finished debug TestCheckDuplicateFields
* [test] Added TestDescription_EnsureLength
* [test] added test in validator_test.go TestUpdateDescription
* [test] Added test TestComputed_String
* [test] refactored TestValidatorSanityCheck and added some message in error message returned from sanity check
* [test] removed test index for debugging
* [test] Added test TestValidatorWrapper_SanityCheck. Added value check for TestMarshalUnmarshalValidator. Fixed rlp usage defect in UnmarshalValidator
* [test] added test TestVerifyBLSKeys
* [test] added test TestContainsHarmonyBLSKeys
* [test] added test TestCreateValidatorFromNewMsg
* [test] fixed some tests errors
* [test] added test TestUpdateValidatorFromEditMsg, last editted the test file, finished testing validator_test.go
* [staking] added numeric.Dec.Copy method
* [staking] added CommissionRates.Copy
* [staking] add nil case handle to copy method and related test case
* [test] added nil case for commissionRates.Copy test
* [staking] finished CreateValidator.Copy and related test case
* [staking] added EditValidator.Copy method and related test case
* [test] added zero test cases for Copy method
* [staking] implemented Delegate.Copy and related unit test case
* [staking] added Undelegate.Copy and CollectReward.Copy method. Implemented corresponding test cases
* [test] added two more simple unit tests
* [test] solving the merge conflict with deep copy fix
* [test] added schedule for sharding
* [test] refactored double_sign_test.go
* [test] comment out all test cases for now.
* [test] added record Copy method for testing
* [test] added the first test case in TestVerify
* [test] finished test TestVerify
* [test] Added test TestVerify, TestApplySlashRate, TestSetDifference
* [test] half through TestDelegatorSlashApply
* [test] fix the problem cause by unsafe usage of big.Int
* [staking] added the copy methods to validatorWrapper
* [test] added test cases for copy method for staking/validatorWrapper
* [test] added test case TestApply
* [test] added test case TestRate
* [test] fix the golint errors
* [test] removed commented out unit test code
* [test] remote the empty line in imports
* [test] moved copy methods to stake_testing module
* [test] removed usage of interface stateDB
* [test] removed empty lines in imports
* [test] fixed golint package name issues
* [test] removed a todo when writing code
* [test] moved record.Copy to staking/slash/test
* [test] add some changes
* [test] added prototypes in staketest module
* [test] fix the golint issue in staketest
* [test] make prototype more beautiful
* [test] refactored test TestCheckDuplicateFields
* [test] add createValidator test data
* [test] added positive test case for VerifyAndCreateValidatorFromMsg
* [test] added create validator test
* [test] added the positive test case for TestVerifyAndEditValidatorFromMsg
* [test] added the tests case TestVerifyAndEditValidatorFromMsg
* [test] fix one of the error is not triggered as expected
* [test] more changes in test
* [test] fix the positive test data for TestVerifyAndDelegateFromMsg
* [test] Fixed two comment golint errors
* [test] added delegate tests
* [test] added a make function. WIP
* [test] added undelegate positive test
* [test] added negative test cases for undelegate
* [test] added positive test. not passed. Need to add assert method in test suites
* [test] added equal function to check Validator Equality
* [test] added equals for staketest
* [test] replaced deep equal with new equal methods:
* [test] removed unused codes
* [test] Finishing touch
* [test] fix comment golint issue
* [test] removed dead code in staking_verifier_test.go
Co-authored-by: Ganesha Upadhyaya <ganeshrvce@gmail.com>
* changed the sync port to dns port in flag
* [network] Limit client sync connection (#3071)
* [network] limit client connected peers
* Changed signature and added tests
* Set secure permissions on generated blskey pass file in node.sh
* Fix spellin gtypo in node.sh
* fix latest param handling in api v1
* added comment for the isGreaterThanLatestBlockNum check
* fix a nil pointer crash due to previous revert
Signed-off-by: Leo Chen <leo@harmony.one>
* fix nil pointer crash
Signed-off-by: Leo Chen <leo@harmony.one>
* fix nil pointer of block
Signed-off-by: Leo Chen <leo@harmony.one>
* [rpc] add viewID and epoch to RPCMarshalBlock (#3009)
* [rpc] add viewID and epoch to RPCMarshalBlock
* add apt-get update
* Do not rollback current block on failed block sync (#3101)
Co-authored-by: Ganesha Upadhyaya <ganeshrvce@gmail.com>
Co-authored-by: Rongjian Lan <rongjian.lan@gmail.com>
Co-authored-by: Jacky Wang <jackyw.se@gmail.com>
Co-authored-by: Janet Liang <56005637+janet-harmony@users.noreply.github.com>
Co-authored-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
Co-authored-by: Janet Liang <janet@harmony.one>
Co-authored-by: Sebastian Johnsson <sebastian.johnsson@gmail.com>
Co-authored-by: hsiung <hsiung@pinterest.com>
Co-authored-by: Dennis Won <jhwon0820@gmail.com>
Co-authored-by: Yishuang Chen <34232522+ivorytowerdds@users.noreply.github.com>
* explorer node. Skip dump for redundant blocks with lower block number than the checkpoint block number
* add a key for each block to mark whether they are dumped or not.
* make staking txn field values to be decoded on explorer node ldb storage, and set to address correctly for staking txns
* revert the txn and staking txn unification for explorer rpc
* [project] Dead test
* [node] If fields always same, then just use the constant
* [project] Remove overcomplicated ctxerror
* [project] Remove more dead tests, adjust error replacing ctxerror
* [project] More dead tests, fix travis complaint on Errorf
* [node.sh] Remove is-genesis
* [internal] Remove dead RPC/wallet
* [rpc] Reduce API surface area for dead RPC
* [node] Remove dead flags, unused setters
* [project] Unused logger, too many loggers
* [core] staticcheck fixes, remove unused db keys
* [project] More dead code through, now in param and genesis
* [internal] Useless test
* Clean up logs
* remove log
* Refactor block period logic; wait for 100% sig before timeout
* logic fixes
* optimize now init
* always wait for 2s for grace period
* staking txn. look up by hash fix on api backend rawdb storage
* node explorer staking txn 'history' RPC layer support
* fix unit test
* add error log when explorer node db instance cannot be fetched
* revert unwanted merge changes during rebase
* use already encoded tx message fields for get staking txn rpc
* update explorer node storage service for staking txns
* use hex string for staking transaction data field
* revert transaction pool apiv1 changes
Jacky Wang
Rongjian Lan
Leo Chen
Daniel Van Der Maden
Janet Liang
Sebastian Johnsson
Jong Hyuck Won
Edgar Aroutiounian
Dennis Won
Minh Doan