* fix allowed txs to be able to handle multiple txs for same from address (#4624)
* [HOTFIX] fix leader crosslink issue to not include old cross link in the propo… (#4629)
* fix leader crosslink issue to not include old cross link in the proposing block
* set higher epoch threshold for pending crosslinks to be added to proposing block
* delete old pending cross links
* delete when proposing
* delete when proposing
* delete when proposing
* delete when proposing
* minor logic change for the log
* minor logic change for the log
* minor logic change for the log
* minor logic change for the log
---------
Co-authored-by: Diego Nava <diego.nava77@hotmail.com>
* Fix for possible panic. (#4627)
* Fix, removed duplicated check.
---------
Co-authored-by: Gheis Mohammadi <Gheis.Mohammadi@gmail.com>
Co-authored-by: Diego Nava <diego.nava77@hotmail.com>
Co-authored-by: Konstantin <355847+Frozen@users.noreply.github.com>
* fix leader crosslink issue to not include old cross link in the proposing block
* set higher epoch threshold for pending crosslinks to be added to proposing block
* delete old pending cross links
* delete when proposing
* delete when proposing
* delete when proposing
* delete when proposing
* minor logic change for the log
* minor logic change for the log
* minor logic change for the log
* minor logic change for the log
---------
Co-authored-by: Diego Nava <diego.nava77@hotmail.com>
* fix allowed txs to be able to handle multiple txs for same from address
* improve tx data checking for allowed txs
Co-authored-by: Diego Nava <8563843+diego1q2w@users.noreply.github.com>
---------
Co-authored-by: Diego Nava <8563843+diego1q2w@users.noreply.github.com>
* fix null snapshot issue in chain helper's state sync new functions
* add WriteHeaderNumber to writeHeadBlock
* improve CurrentBlockNumberto handle unsaved new pivot
* fix null response data for full state sync
* add log for zero task in state worker loop
* adjust full state sync request parameters, rename stage_state
* add full state stage to the list of stages in fast sync
* add RangeMode and ChainExecutionMode to handle execution of the stream sync stage
* fix block exists issue on stage_states in stream sync
* fix double insertion in stage states
* add count for state downloader to return number of tasks
* fix travis build issue by goimports
* switch to Full Sync on pivot block, fix checking nil length in HandleRequestError
* test: add delegation type tests
These were skipped intentionally in the previous pull request
* rpc: undo all changes to 2023.3.0
* rpc: calculate SenderAddress before `ConvertToEth`
`tx.ConvertToEth` silently drops the `ShardID` and `ToShardID` fields.
This results in the hash of the transaction changing (either via removal
of these fields in the hash calculation or by automatic filling of the
values by the node's shard). The different hash calculation results in
incorrect sender address calculation.
There have been a couple of attempts to fix this issue, which created
troubles elsewhere. This pull request reverts to the behaviour seen in
2023.3.0 with a simple edit that calculates the `SenderAddress` before
dropping the fields in the call to `ConvertToEth`.
* Fix: max rate issue (#4580)
* fix: max-rate bellow the era min-rate
* fix comments
* add localnet epoch config
* update config
* update config
* update config
* update config
* add log
* remove hip30 from localnet
* disable localnet config
* engine: actually fix the 7% fee implementation
* rpc: fix transaction receipt format for eth
use the same receipt as `hmyv2_`. using a boolean variable, decide if
the addresses need to be converted to bech32. do not return a contract
address unless a contract was actually deployed in the transaction by
using a pointer address type.
* rpc: add comment indicating function is unused
with the switch to `v2.NewReceipt` for even `eth_` queries, the
`eth.NewReceipt` function is no longer used
* build: fix delegation tests
* update comment
blocks was referring to `blocks of code` and not blocks in a chain.
removed the confusing word
* rpc: remove ConvertToEth in GetBlockReceipts
* internal: max rate hard fork schedule
* internal: testnet max rate schedule
---------
Co-authored-by: Diego Nava <8563843+diego1q2w@users.noreply.github.com>
* Cleanup and fix update pub keys.
* Skip the next leader if it doesn't sign blocks.
* Comment & constant.
* Updated with dev.
* Updated with latest dev.
* Cleanup
* add new functions to p2p stream client for sharing the full states
* remove extra comments, add bytes checking
* add client tests for new p2p stream client functions
* rename new client functions
* complete tests for new functions of p2p stream client
* Fixed debug run for mac. (#4484)
* Fix debug run for mac.
* Next validator in view change. (#4492)
* NthNextValidatorHmy
* Fixed func usage.
* Additional checks.
* HIP-30 Boilerplate (#4495)
* HIP-30: sharding configuration boilerplate
* update test
* update comment
---------
Co-authored-by: Nita Neou (Soph) <soph@harmony.one>
* HIP-30: minimum validator commission of 7% (#4496)
* HIP-30: sharding configuration boilerplate
* update comments
* goimports
* HIP-30: minimum validator commission of 7%
Based on #4495, which must be merged before this PR. This PR should be
rebased with dev after #4495 is merged to retain atomicity of changes by
pull request.
* goimports
* update test
* update test
---------
Co-authored-by: Casey Gardiner <117784577+ONECasey@users.noreply.github.com>
* HIP-30: Emission split (#4497)
* HIP-30: sharding configuration boilerplate
* update comments
* goimports
* HIP-30: minimum validator commission of 7%
Based on #4495, which must be merged before this PR. This PR should be
rebased with dev after #4495 is merged to retain atomicity of changes by
pull request.
* goimports
* HIP-30: Emission split implementation
Note that the allocated split of the emission goes directly to the
recipient (and not via the Reward). This is because rewards are indexed
by validator and not by delegator, and the recipient may/may not have
any delegations which we can reward. Even if one was guaranteed to
exist, it would mess up the math of the validator.
* set up mainnet recipient of emission split
* HIP-30: Emission split addresses for non mainnet
* update test
* Update mainnet.go
---------
Co-authored-by: Casey Gardiner <117784577+ONECasey@users.noreply.github.com>
* HIP-30: Set up pre-image generation, recording, export and import (#4494)
* flags: set up preimage flags
* hip30: set up preimage import, export, api
* save pre-images by default
* add pre images api
* goimports
* commit rpc preimages file
* preimages: re-generate them using CLI
* add metrics and numbers for pre-images
* automate generation after import
* move from rpc to core
* add back core/preimages.go file
* export prometheus metric when no error importing preimage
* add preimages flags to rootflags
---------
Co-authored-by: Nita Neou (Soph) <soph@harmony.one>
* HIP-30: Shard reduction (#4498)
* HIP-30: sharding configuration boilerplate
* update comments
* goimports
* HIP-30: minimum validator commission of 7%
Based on #4495, which must be merged before this PR. This PR should be
rebased with dev after #4495 is merged to retain atomicity of changes by
pull request.
* goimports
* HIP-30: Emission split implementation
Note that the allocated split of the emission goes directly to the
recipient (and not via the Reward). This is because rewards are indexed
by validator and not by delegator, and the recipient may/may not have
any delegations which we can reward. Even if one was guaranteed to
exist, it would mess up the math of the validator.
* set up mainnet recipient of emission split
* HIP-30: Emission split addresses for non mainnet
* HIP-30: deactivate shard 2 and 3 validators
* update test
* shard reduction: update block reward
---------
Co-authored-by: Casey Gardiner <117784577+ONECasey@users.noreply.github.com>
* Fix for index. (#4504)
* Small improvements. (#4477)
* HIP-30: Balance migration (#4499)
* flags: set up preimage flags
* hip30: set up preimage import, export, api
* save pre-images by default
* add pre images api
* goimports
* commit rpc preimages file
* preimages: re-generate them using CLI
* add metrics and numbers for pre-images
* automate generation after import
* move from rpc to core
* add back core/preimages.go file
* HIP-30: sharding configuration boilerplate
* update comments
* HIP-30: minimum validator commission of 7%
Based on #4495, which must be merged before this PR. This PR should be
rebased with dev after #4495 is merged to retain atomicity of changes by
pull request.
* goimports
* HIP-30: Emission split implementation
Note that the allocated split of the emission goes directly to the
recipient (and not via the Reward). This is because rewards are indexed
by validator and not by delegator, and the recipient may/may not have
any delegations which we can reward. Even if one was guaranteed to
exist, it would mess up the math of the validator.
* set up mainnet recipient of emission split
* HIP-30: Emission split addresses for non mainnet
* HIP-30: deactivate shard 2 and 3 validators
* goimports
* update test
* migrate balance uring epoch T - 1
highly untested code. also missing is the ability to generate a
pre-migration report for future verification.
* update test
* export prometheus metric when no error importing preimage
* add comment
* test account migration in localnet
* add preimages flags to rootflags
* enable preimages on the whitelist
* add the generate method
* fix cropping log
* cropping startpoint when bigger than endpoint
* add support for the rpcblocknumer type
* enable import api
* use earlies block
* fix error catching
* make end optional for the comand line
* fix cropping logic
* improve error when apply message fails
* add balance on the error
* fix importing
* remove unused imports
---------
Co-authored-by: Nita Neou (Soph) <soph@harmony.one>
Co-authored-by: Soph <35721420+sophoah@users.noreply.github.com>
Co-authored-by: Diego Nava <diego.nava77@hotmail.com>
Co-authored-by: Diego Nava <8563843+diego1q2w@users.noreply.github.com>
* Hip30 balance migration with fix. (#4502)
* flags: set up preimage flags
* hip30: set up preimage import, export, api
* preimages: re-generate them using CLI
* move from rpc to core
* migrate balance uring epoch T - 1
* test account migration in localnet
* enable preimages on the whitelist
* add the generate method
* fix cropping log
* cropping startpoint when bigger than endpoint
* add support for the rpcblocknumer type
* enable import api
* Fixed stuck.
* Additional logs.
* Rebased on harmony-one:hip30/testing.
* Removed code duplicate.
* Fixed stuck.
* IsOneEpochBeforeHIP30 for only 1 epoch.
---------
Co-authored-by: MaxMustermann2 <82761650+MaxMustermann2@users.noreply.github.com>
Co-authored-by: Nita Neou (Soph) <soph@harmony.one>
Co-authored-by: Diego Nava <diego.nava77@hotmail.com>
Co-authored-by: Diego Nava <8563843+diego1q2w@users.noreply.github.com>
* remove double import
* rename variable
* remove unused fmt
* Fixed imports. (#4507)
* Block gas 30m. (#4501)
* BlockGas30M renamed to BlockGas30MEpoch.
* Removed redundant code.
* Hip30 : localnet account migration fix (#4508)
* flags: set up preimage flags
* hip30: set up preimage import, export, api
* save pre-images by default
* add pre images api
* goimports
* commit rpc preimages file
* preimages: re-generate them using CLI
* add metrics and numbers for pre-images
* automate generation after import
* add back core/preimages.go file
* HIP-30: sharding configuration boilerplate
* update comments
* goimports
* HIP-30: minimum validator commission of 7%
Based on #4495, which must be merged before this PR. This PR should be
rebased with dev after #4495 is merged to retain atomicity of changes by
pull request.
* goimports
* HIP-30: Emission split implementation
Note that the allocated split of the emission goes directly to the
recipient (and not via the Reward). This is because rewards are indexed
by validator and not by delegator, and the recipient may/may not have
any delegations which we can reward. Even if one was guaranteed to
exist, it would mess up the math of the validator.
* set up mainnet recipient of emission split
* HIP-30: Emission split addresses for non mainnet
* HIP-30: deactivate shard 2 and 3 validators
* goimports
* update test
* goimports
* migrate balance uring epoch T - 1
highly untested code. also missing is the ability to generate a
pre-migration report for future verification.
* update test
* export prometheus metric when no error importing preimage
* test account migration in localnet
* add preimages flags to rootflags
* enable preimages on the whitelist
* add the generate method
* fix cropping log
* cropping startpoint when bigger than endpoint
* add support for the rpcblocknumer type
* enable import api
* use earlies block
* debug logs
* fix error catching
* fix error catching
* make end optional for the comand line
* fix cropping logic
* improve error when apply message fails
* add balance on the error
* fix importing
* remove unused imports
* create preimage for genesis block
* fix consensus with No Migration Possible
* use correct header for migration
* process all tx in all block for non shard 0
---------
Co-authored-by: MaxMustermann2 <82761650+MaxMustermann2@users.noreply.github.com>
Co-authored-by: Diego Nava <diego.nava77@hotmail.com>
Co-authored-by: Diego Nava <8563843+diego1q2w@users.noreply.github.com>
* create the sate as the insertchain does
* roll back changes
* use the updated state in case there is one
* use the updated state in case there is one
* add testing fmt
* fix getReceipts rpc issue (#4511)
* pass the correct config
* Fixes.
* reduce the block number count
* add verify preimages rpc method
* write preimages on process
* commit preimages
* commit preimages
* verify root hashes after commit
* send metrics on node start
* send the verified preimages
* correct the starting block
* register the verified address
* flush the db every export and verify
* add shard label
* aggregate the recovery multisig reward (#4514)
* 1) Removed unused worker (#4512)
2) Proper error checking
3) Tests for gas 30m
* Improvements of streamsync to deploy on mainnet (#4493)
* add faultRecoveryThreshold to reset stream failures
* increase MaxStreamFailures to let stream be longer in the list
* set Concurrency to 2 for devnet to be same as MinStreams, otherwise it will rewrite MinStreams
* stream sync loop checks for ErrNotEnoughStreamsand waits for enough streams in case there are not enough connected streams in list
* fix fault recovery issue
* improve checkPrerequisites to be able to continue with minimum streams
* refactor fixValues function, put priority on MinStreams rather than Concurrency
* drop remote peer if sending empty blocks array
* goimports to fix build issue
* fix getReceipts array assignments
* fix getReceipts and add tests for it
* fix duplicate function def
* reset devnet and set 30M epoch for all network except mainnet/testnet (#4517)
* enable on devnet hip28 and hip30 together at hip30
* enable 30M epoch for all except mainnet/testnet, update devnet for restart
* remove unused var
* update partner/devnet feature activation
* Remove old devnet/partner instance config
* reduce the epoch time for devnet to 30 min (#4522)
* add GetNodeData tests for stream client, increase nodes and receipts cap (#4519)
* add tests for GetNodeData in stream protocol
* increase client cap for nodes and receipts requests
* use new(big.Int) so we don't modify the epoch value (#4523)
* add hip30 testing for devnet/partner network (#4525)
* enable hip30 epoch for testnet (#4526)
* enable hip30 epoch for testnet
* fix date comment
* set blockgas30M epoch
* enable hip30 and gas30m epoch for mainnet (#4528)
* fix preimage import bugs (#4529)
* fix preimage import bugs
* Fixed lru cache size. (#4535)
* fix decryptRaw issue for nil/empty data (#4532)
* update deprecated ioutil, improve local accounts (#4527)
* make peer connected/disconnected debug log level (#4537)
* Revert improvements. (#4520)
* Updated go lib p2p deps. (#4538)
* Flush data. (#4536)
* Rotation fix and update. (#4516)
---------
Co-authored-by: Konstantin <355847+Frozen@users.noreply.github.com>
Co-authored-by: Max <82761650+MaxMustermann2@users.noreply.github.com>
Co-authored-by: Nita Neou (Soph) <soph@harmony.one>
Co-authored-by: Soph <35721420+sophoah@users.noreply.github.com>
Co-authored-by: Diego Nava <diego.nava77@hotmail.com>
Co-authored-by: Diego Nava <8563843+diego1q2w@users.noreply.github.com>
Co-authored-by: Gheis Mohammadi <Gheis.Mohammadi@gmail.com>
Co-authored-by: “GheisMohammadi” <36589218+GheisMohammadi@users.noreply.github.com>
Co-authored-by: Adam Androulidakis <37982984+adsorptionenthalpy@users.noreply.github.com>
* enable on devnet hip28 and hip30 together at hip30
* enable 30M epoch for all except mainnet/testnet, update devnet for restart
* remove unused var
* update partner/devnet feature activation
* Remove old devnet/partner instance config
* add faultRecoveryThreshold to reset stream failures
* increase MaxStreamFailures to let stream be longer in the list
* set Concurrency to 2 for devnet to be same as MinStreams, otherwise it will rewrite MinStreams
* stream sync loop checks for ErrNotEnoughStreamsand waits for enough streams in case there are not enough connected streams in list
* fix fault recovery issue
* improve checkPrerequisites to be able to continue with minimum streams
* refactor fixValues function, put priority on MinStreams rather than Concurrency
* drop remote peer if sending empty blocks array
* goimports to fix build issue
* fix getReceipts array assignments
* fix getReceipts and add tests for it
* flags: set up preimage flags
* hip30: set up preimage import, export, api
* save pre-images by default
* add pre images api
* goimports
* commit rpc preimages file
* preimages: re-generate them using CLI
* add metrics and numbers for pre-images
* automate generation after import
* move from rpc to core
* goimports
* add back core/preimages.go file
* HIP-30: sharding configuration boilerplate
* update comments
* goimports
* HIP-30: minimum validator commission of 7%
Based on #4495, which must be merged before this PR. This PR should be
rebased with dev after #4495 is merged to retain atomicity of changes by
pull request.
* goimports
* HIP-30: Emission split implementation
Note that the allocated split of the emission goes directly to the
recipient (and not via the Reward). This is because rewards are indexed
by validator and not by delegator, and the recipient may/may not have
any delegations which we can reward. Even if one was guaranteed to
exist, it would mess up the math of the validator.
* set up mainnet recipient of emission split
* HIP-30: Emission split addresses for non mainnet
* HIP-30: deactivate shard 2 and 3 validators
* goimports
* update test
* goimports
* migrate balance uring epoch T - 1
highly untested code. also missing is the ability to generate a
pre-migration report for future verification.
* update test
* export prometheus metric when no error importing preimage
* add comment
* test account migration in localnet
* add preimages flags to rootflags
* enable preimages on the whitelist
* add the generate method
* fix cropping log
* fix cropping log
* cropping startpoint when bigger than endpoint
* add support for the rpcblocknumer type
* enable import api
* use earlies block
* debug logs
* debug logs
* debug logs
* debug logs
* fix error catching
* fix error catching
* make end optional for the comand line
* fix cropping logic
* improve error when apply message fails
* add balance on the error
* fix importing
* remove unused imports
* create preimage for genesis block
* fix consensus with No Migration Possible
* use correct header for migration
* process all tx in all block for non shard 0
---------
Co-authored-by: MaxMustermann2 <82761650+MaxMustermann2@users.noreply.github.com>
Co-authored-by: Diego Nava <diego.nava77@hotmail.com>
Co-authored-by: Diego Nava <8563843+diego1q2w@users.noreply.github.com>
* flags: set up preimage flags
* hip30: set up preimage import, export, api
* preimages: re-generate them using CLI
* move from rpc to core
* migrate balance uring epoch T - 1
highly untested code. also missing is the ability to generate a
pre-migration report for future verification.
* test account migration in localnet
* enable preimages on the whitelist
* add the generate method
* fix cropping log
* fix cropping log
* cropping startpoint when bigger than endpoint
* add support for the rpcblocknumer type
* enable import api
* Fixed stuck.
* Additional logs.
* Cleanup.
* Rebased on harmony-one:hip30/testing.
* Removed code duplicate.
* Fixed stuck.
* IsOneEpochBeforeHIP30 for only 1 epoch.
---------
Co-authored-by: MaxMustermann2 <82761650+MaxMustermann2@users.noreply.github.com>
Co-authored-by: Nita Neou (Soph) <soph@harmony.one>
Co-authored-by: Diego Nava <diego.nava77@hotmail.com>
Co-authored-by: Diego Nava <8563843+diego1q2w@users.noreply.github.com>
* flags: set up preimage flags
* hip30: set up preimage import, export, api
* save pre-images by default
* add pre images api
* goimports
* commit rpc preimages file
* preimages: re-generate them using CLI
* add metrics and numbers for pre-images
* automate generation after import
* move from rpc to core
* goimports
* add back core/preimages.go file
* HIP-30: sharding configuration boilerplate
* update comments
* goimports
* HIP-30: minimum validator commission of 7%
Based on #4495, which must be merged before this PR. This PR should be
rebased with dev after #4495 is merged to retain atomicity of changes by
pull request.
* goimports
* HIP-30: Emission split implementation
Note that the allocated split of the emission goes directly to the
recipient (and not via the Reward). This is because rewards are indexed
by validator and not by delegator, and the recipient may/may not have
any delegations which we can reward. Even if one was guaranteed to
exist, it would mess up the math of the validator.
* set up mainnet recipient of emission split
* HIP-30: Emission split addresses for non mainnet
* HIP-30: deactivate shard 2 and 3 validators
* goimports
* update test
* goimports
* migrate balance uring epoch T - 1
highly untested code. also missing is the ability to generate a
pre-migration report for future verification.
* update test
* export prometheus metric when no error importing preimage
* add comment
* test account migration in localnet
* add preimages flags to rootflags
* enable preimages on the whitelist
* add the generate method
* fix cropping log
* fix cropping log
* cropping startpoint when bigger than endpoint
* add support for the rpcblocknumer type
* enable import api
* use earlies block
* debug logs
* debug logs
* debug logs
* debug logs
* fix error catching
* fix error catching
* make end optional for the comand line
* fix cropping logic
* improve error when apply message fails
* add balance on the error
* fix importing
* remove unused imports
---------
Co-authored-by: Nita Neou (Soph) <soph@harmony.one>
Co-authored-by: Soph <35721420+sophoah@users.noreply.github.com>
Co-authored-by: Diego Nava <diego.nava77@hotmail.com>
Co-authored-by: Diego Nava <8563843+diego1q2w@users.noreply.github.com>
* HIP-30: sharding configuration boilerplate
* update comments
* goimports
* HIP-30: minimum validator commission of 7%
Based on #4495, which must be merged before this PR. This PR should be
rebased with dev after #4495 is merged to retain atomicity of changes by
pull request.
* goimports
* HIP-30: Emission split implementation
Note that the allocated split of the emission goes directly to the
recipient (and not via the Reward). This is because rewards are indexed
by validator and not by delegator, and the recipient may/may not have
any delegations which we can reward. Even if one was guaranteed to
exist, it would mess up the math of the validator.
* set up mainnet recipient of emission split
* HIP-30: Emission split addresses for non mainnet
* HIP-30: deactivate shard 2 and 3 validators
* update test
* update test
* shard reduction: update block reward
---------
Co-authored-by: Casey Gardiner <117784577+ONECasey@users.noreply.github.com>
* flags: set up preimage flags
* hip30: set up preimage import, export, api
* save pre-images by default
* add pre images api
* goimports
* commit rpc preimages file
* preimages: re-generate them using CLI
* add metrics and numbers for pre-images
* automate generation after import
* move from rpc to core
* goimports
* add back core/preimages.go file
* goimports
* goimports
* export prometheus metric when no error importing preimage
* add preimages flags to rootflags
---------
Co-authored-by: Nita Neou (Soph) <soph@harmony.one>
* HIP-30: sharding configuration boilerplate
* update comments
* goimports
* HIP-30: minimum validator commission of 7%
Based on #4495, which must be merged before this PR. This PR should be
rebased with dev after #4495 is merged to retain atomicity of changes by
pull request.
* goimports
* HIP-30: Emission split implementation
Note that the allocated split of the emission goes directly to the
recipient (and not via the Reward). This is because rewards are indexed
by validator and not by delegator, and the recipient may/may not have
any delegations which we can reward. Even if one was guaranteed to
exist, it would mess up the math of the validator.
* set up mainnet recipient of emission split
* HIP-30: Emission split addresses for non mainnet
* update test
* update test
* Update mainnet.go
---------
Co-authored-by: Casey Gardiner <117784577+ONECasey@users.noreply.github.com>
* HIP-30: sharding configuration boilerplate
* update comments
* goimports
* HIP-30: minimum validator commission of 7%
Based on #4495, which must be merged before this PR. This PR should be
rebased with dev after #4495 is merged to retain atomicity of changes by
pull request.
* goimports
* update test
* update test
---------
Co-authored-by: Casey Gardiner <117784577+ONECasey@users.noreply.github.com>
* add error desc in validateNewBlock to help to identify validation issues
* let stream sync downloader continues loop even if error occured
* pass consensus to stream sync through downloader
* add last mile functions to stream sync
* add if to check invalid block revert
* add last mile stage to stream sync
* goimports
* improve stream sync downloader loop to block redundant calls
* move startSyncing out of the shortrange loop
* goimports
* fix sync loop go routine
* remove extra log
* add debug mode to stream sync help debugging syncing issues
* fix stream sync loop channels
* add streamFailed function to short range helper to avoid removing of the healthy streams
* remove execution of stage bodies, stage lastmile, stage short range and stage state for epoch chain
* refactor stage epoch
* add debug logs
* goimports
* add a few debug log to stage short range
* doSync returns estimated height as well
* only switch to short range if the current block number is very close to the chain current height
* stream sync gets UseMemDb from config file
* goimports
* only flag failed streams rather than removing them in stream sync
* if stage blocks progress behind current head, remove block cache
* add rollback to short range
* refactor stage last mile blocks, add roll back to this stage
* improve addConsensusLastMile
* goimports
* fix log spell error
* improve revert function, no need to revert if hashes are empty
* fix switch to short range by removing extra condition
* add donC chan size
* refactor downloader loop mechanism
* use atomic flag rather than done channel in downloader loop
* no need for fail stream in epoch sync
* ignore context timeout
* add mux lock to get access to last mile blocks
* remove atomic flag for downloader loop
* a few improvements on staged sync, check addedBn before switch to short range
* goimports
* fix consensus catchup issue
* fix panic issue from runnig sync loop while stream sync is runing
* goimports
* add two more logs to staged sync
* remove extra debug logs, add more file logs for stream sync
* add comment for DebugMode
* improve the byte comparison for getBlockFromLastMileBlocksByParentHash function
The function `state_object.CodeSize` erroneously returns the validator
code size to be 0. This functionality is only used by the EVM, where the
size is overridden by the `ValidatorCodeFixEpoch` hard fork to be 0
anyway.
However, for nodes currently syncing blocks before the hard fork was
effective, this causes an error. A transaction which attempts to mint an
NFT to a validator address prior to the hard fork would have failed,
because `onERC1155Received` is not implemented by a validator's code.
With the erroneous line, the transaction goes through - causing
a node with the unpatched binary to reject the block.
This [github document](https://help.github.com/articles/creating-a-pull-request/) provides some guidance on how to create a pull request in github.
This [github document](https://help.github.com/articles/creating-a-pull-request/) provides some guidance on how to create a pull request in github.
## PR requirement
## PR requirement
To pursue engineering excellence, we have insisted on the highest stardard on the quality of each PR.
To pursue engineering excellence, we have insisted on the highest standard for the quality of each PR.
* For each PR, please run [golint](https://github.com/golang/lint), [goimports](https://godoc.org/golang.org/x/tools/cmd/goimports), to fix the basic issues/warnings.
* For each PR, please run [golint](https://github.com/golang/lint), [goimports](https://godoc.org/golang.org/x/tools/cmd/goimports), to fix the basic issues/warnings.
* Make sure you understand [How to Write a Git Commit Message](https://chris.beams.io/posts/git-commit/).
* Make sure you understand [How to Write a Git Commit Message](https://chris.beams.io/posts/git-commit/).
@ -21,12 +21,12 @@ To pursue engineering excellence, we have insisted on the highest stardard on th
The best practice is to reorder and squash your local commits before the PR submission to create an atomic and self-contained PR.
The best practice is to reorder and squash your local commits before the PR submission to create an atomic and self-contained PR.
This [book chapter](https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History) provides detailed explanation and guidance on how to rewrite the local git history.
This [book chapter](https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History) provides detailed explanation and guidance on how to rewrite the local git history.
For exampple, a typical workflow is like the following.
For example, a typical workflow is like the following.
```bash
```bash
# assuming you are working on a fix of bug1, and use a local branch called "fixes_of_bug1".
# assuming you are working on a fix of bug1, and use a local branch called "fixes_of_bug1".
git clone https://github.com/harmony-one/harmony
git clone https://github.com/woop-chain/woop
cd harmony
cd woop
# create a local branch to keep track of the origin/master
# create a local branch to keep track of the origin/master
@ -47,27 +39,27 @@ For macOS, you can reference this [guide](http://tldrdevnotes.com/bash-upgrade-3
## Dev Environment
## Dev Environment
**Most repos from [harmony-one](https://github.com/harmony-one) assumes the GOPATH convention. More information [here](https://github.com/golang/go/wiki/GOPATH).**
**Most repos from [woop-chain](https://github.com/woop-chain) assumes the GOPATH convention. More information [here](https://github.com/golang/go/wiki/GOPATH).**
### First Install
### First Install
Clone and set up all of the repos with the following set of commands:
Clone and set up all of the repos with the following set of commands:
Included in this repo is a Dockerfile that has a full harmony development environment and
Included in this repo is a Dockerfile that has a full woop development environment and
comes with emacs, vim, ag, tig and other creature comforts. Most importantly, it already has the go environment
comes with emacs, vim, ag, tig and other creature comforts. Most importantly, it already has the go environment
with our C/C++ based library dependencies (`libbls` and `mcl`) set up correctly for you.
with our C/C++ based library dependencies (`libbls` and `mcl`) set up correctly for you.
You can build the docker image for yourself with the following commands:
You can build the docker image for yourself with the following commands:
```bash
```bash
cd $(go env GOPATH)/src/github.com/harmony-one/harmony
cd $(go env GOPATH)/src/github.com/woop-chain/woop
make clean
make clean
docker build -t harmony .
docker build -t woop .
```
```
> If your build machine has an ARM-based chip, like Apple silicon (M1), the image is built for `linux/arm64` by default. To build for `x86_64`, apply the `--platform` arg like so:
> If your build machine has an ARM-based chip, like Apple silicon (M1), the image is built for `linux/arm64` by default. To build for `x86_64`, apply the `--platform` arg like so:
> Learn more about the `--platform` arg and multi-CPU architecture support, [here](https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope) and [here](https://docs.docker.com/desktop/multi-arch/).
> Learn more about the `--platform` arg and multi-CPU architecture support, [here](https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope) and [here](https://docs.docker.com/desktop/multi-arch/).
@ -97,26 +89,26 @@ docker build -t harmony .
Then you can start your docker container with the following command:
Then you can start your docker container with the following command:
> Note that the harmony repo will be shared between your docker container and your host machine. However, everything else in the docker container will be ephemeral.
> Note that the woop repo will be shared between your docker container and your host machine. However, everything else in the docker container will be ephemeral.
If you need to open another shell, just do:
If you need to open another shell, just do:
```bash
```bash
docker exec -it harmony /bin/bash
docker exec -it woop /bin/bash
```
```
Learn more about docker [here](https://docker-curriculum.com/).
Learn more about docker [here](https://docker-curriculum.com/).
## Build
## Build
The `make` command should automatically build the Harmony binary & all dependent libs.
The `make` command should automatically build the Woop binary & all dependent libs.
However, if you wish to bypass the Makefile, first export the build flags:
However, if you wish to bypass the Makefile, first export the build flags:
To keep things consistent, we have a docker image to run all tests. **These are the same tests ran on the pull request checks**.
To keep things consistent, we have a docker image to run all tests. **These are the same tests ran on the pull request checks**.
Note that all testing docker container binds a couple of ports to the host machine for your convince. The ports are:
Note that all test Docker containers bind several ports to the host machine for your convenience. The ports are:
* `9500` - Shard 0 RPC for a validator
* `9500` - Shard 0 RPC for a validator
* `9501` - Shard 1 RPC for a validator
* `9501` - Shard 1 RPC for a validator
* `9599` - Shard 0 RPC for an explorer
* `9599` - Shard 0 RPC for an explorer
@ -159,7 +151,7 @@ Note that all testing docker container binds a couple of ports to the host machi
* `9798` - Shard 1 Rosetta (for an explorer)
* `9798` - Shard 1 Rosetta (for an explorer)
* `9899` - Shard 0 WS for an explorer
* `9899` - Shard 0 WS for an explorer
* `9898` - Shard 1 WS for an explorer
* `9898` - Shard 1 WS for an explorer
> This allows you to use curl, hmy CLI, postman, rosetta-cli, etc... on your host machine to play with or probe the localnet that was used for the test.
> This allows you to use curl, wiki CLI, postman, rosetta-cli, etc... on your host machine to play with or probe the localnet that was used for the test.
### Go tests
### Go tests
To run this test, do:
To run this test, do:
@ -174,8 +166,8 @@ To run this test, do:
make test-rpc
make test-rpc
```
```
This test starts a localnet (within the Docker container), **ensures it reaches a consensus**, and runs a series of tests to ensure correct RPC behavior.
This test starts a localnet (within the Docker container), **ensures it reaches a consensus**, and runs a series of tests to ensure correct RPC behavior.
This test also acts as a preliminary integration test (more through tests are done on the testnets).
This test also acts as a preliminary integration test (more thorough tests are done on the testnets).
> The tests ran by this command can be found [here](https://github.com/harmony-one/harmony-test/tree/master/localnet).
> The tests ran by this command can be found [here](https://github.com/woop-chain/woop-test/tree/master/localnet).
If you wish to debug further with the localnet after the tests are done, open a new shell and run:
If you wish to debug further with the localnet after the tests are done, open a new shell and run:
```bash
```bash
@ -183,9 +175,9 @@ make test-rpc-attach
```
```
> This will open a shell in the docker container that is running the Node API tests.
> This will open a shell in the docker container that is running the Node API tests.
>
>
> Note that the docker container has the [Harmony CLI](https://docs.harmony.one/home/wallets/harmony-cli) on path,
> Note that the docker container has the [Woop CLI](https://docs.wikiwoop.com/home/wallets/woop-cli) on path,
> therefore you can use that to debug if needed. For example, one could do `hmy blockchain latest-headers` to check
> therefore you can use that to debug if needed. For example, one could do `wiki blockchain latest-headers` to check
> the current block height of localnet. Reference the documentation for the CLI [here](https://docs.harmony.one/home/wallets/harmony-cli)
> the current block height of localnet. Reference the documentation for the CLI [here](https://docs.wikiwoop.com/home/wallets/woop-cli)
> for more details & commands.
> for more details & commands.
### Rosetta tests
### Rosetta tests
@ -194,8 +186,8 @@ To run this test, do:
make test-rosetta
make test-rosetta
```
```
This test starts a localnet (within the Docker container), **ensures it reaches a consensus**, and runs the Construction & Data API checks using the [rosetta-cli](https://github.com/coinbase/rosetta-cli).
This test starts a localnet (within the Docker container), **ensures it reaches a consensus**, and runs the Construction & Data API checks using the [rosetta-cli](https://github.com/coinbase/rosetta-cli).
This test also acts as a preliminary integration test (more through tests are done on the testnets).
This test also acts as a preliminary integration test (more thorough tests are done on the testnets).
> The config for this test can be found [here](https://github.com/harmony-one/harmony-test/blob/master/localnet/configs/localnet_rosetta_test_s0.json) & [here](https://github.com/harmony-one/harmony-test/blob/master/localnet/configs/localnet_rosetta_test_s1.json)
> The config for this test can be found [here](https://github.com/woop-chain/woop-test/blob/master/localnet/configs/localnet_rosetta_test_s0.json) & [here](https://github.com/woop-chain/woop-test/blob/master/localnet/configs/localnet_rosetta_test_s1.json)
Similar to the RPC tests, if you wish to debug further with the localnet after the tests are done, open a new shell and run:
Similar to the RPC tests, if you wish to debug further with the localnet after the tests are done, open a new shell and run:
```bash
```bash
@ -204,10 +196,10 @@ make test-rosetta-attach
## License
## License
Harmony is licensed under GNU Lesser General Public License v3.0. See [`LICENSE`](LICENSE) file for
Woop is licensed under GNU Lesser General Public License v3.0. See [`LICENSE`](LICENSE) file for
the terms and conditions.
the terms and conditions.
Harmony includes third-party open-source code. In general, a source subtree
Woop includes third-party open-source code. In general, a source subtree
with a `LICENSE` or `COPYRIGHT` file is from a third party, and our
with a `LICENSE` or `COPYRIGHT` file is from a third party, and our
modifications thereto are licensed under the same third-party open source
modifications thereto are licensed under the same third-party open source
license.
license.
@ -216,7 +208,7 @@ Also please see [our Fiduciary License Agreement](FLA.md) if you are
contributing to the project. By your submission of your contribution to us, you
contributing to the project. By your submission of your contribution to us, you
and we mutually agree to the terms and conditions of the agreement.
and we mutually agree to the terms and conditions of the agreement.
## Contributing To Harmony
## Contributing To Woop
See [`CONTRIBUTING`](CONTRIBUTING.md) for details.
See [`CONTRIBUTING`](CONTRIBUTING.md) for details.
@ -235,10 +227,3 @@ See [`CONTRIBUTING`](CONTRIBUTING.md) for details.