* Use binary search to eth_estimateGas more accurately
eth_estimateGas currently conservatively estimates gas this change
will take that estimate and use it as a hi bound and will use
gasUsed as lo bound to hone in on the most true gas limit required
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Refactor to optimize pivot block selector on PoS networks
On PoS network we use a pivot block sent by the Consensus Layer, so we do
not need peers, and so all the logic for selecting the pivot block from peers
has been moved from FastSyncActions to PivotSelectorFromPeers.
We do not need anymore the TransictionPeerSelector, and the --fast-sync-min-peers
applies only to PoW networks.
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Create new datatypes module
Create a new `datatypes` module to hold datatypes that are broadly used.
This will aid modularization by making sure the base types in the module
minimize the amount of unrelated support classes needed.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Add Address, Hash, and Wei to datatypes
Move the Address, Hash, and Wei to datatypes in as they are needed for
EVM modularization.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
The EthQL API spec specifies pulledStates and knownStates in the
`syncing` query. Previously we always returned null. This plumbs through
the needed data so that the synchronization states can report the
fast sync progress via EthQL, as well as the `eth_syncing` JSON-RPC.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* adding in spdx-license-identifier & updated check for the same; removing license check from spotless
Signed-off-by: Joshua Fernandes <joshua.fernandes@consensys.net>
* Change CheckSpdxHeader to a task.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Display World State Sync Progress in Logs
- compute estimated world state completion
- display estimated world state completion in the logs
- `CompleteTaskStep` now have access to a `LongSupplier` to retireve the number of pending requests
- use a `RunnableCounter` to trigger displaying every 1000 requests completed
- only show a new log when the estimation changes
- added test to check the estimation computation
* report using raw ratio rather than percentage
* resolve PR discussion
* rename getTotal to get
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Also increases the number of requests without progress before considering the download stalled.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Forces subclasses have to implement it so they specify minimum requirements for the peer to use.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Use a single thread to persist received world state nodes to avoid RocksDB timeouts due to contention on the write lock.
* Fix RocksDbTaskQueue so it doesn't read stale data when resuming a transfer.
* Skip downloading empty trie nodes.
* Log unhandled errors from world state download requests
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Instantiate individual metrics where they're used
* Cache prometheus metrics to allow "duplicate" creation of metrics
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
There's an issue with the world state downloader where the download process could stall. The sequence goes like:
1. Inside requestDataFromPeer thread A takes out the sendingRequests lock
2. Thread A checks shouldRequestNodeData which returns true
3. Thread A sends a request for data
4. Thread A checks shouldRequestNodeData which returns false so it exits the while loop
5. Thread B receives the response to the (only) outstanding request
6. Thread B enters shouldRequestNodeData but fails to get the sendingRequests lock so exits the method
7. Thread A releases the sendingRequests lock and exits the methods
There are now no threads checking if they should send new requests and no outstanding requests to trigger a check in the future so the download is stuck and will never make anymore progress.
The fix is to switch the order of taking out the sendingRequests lock and checking shouldRequestNodeData so we release the sendingRequests lock before we go back round the loop to check shouldRequestNodeData.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Skip requesting data we already have but continue to walk the tree to ensure we have all child nodes.
* Don't delete fast sync state on stop. Allow resuming world state downloads.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Return world states as an optional to allow code to handle world states not be available cleanly.
A world state is considered available if it's root node is available.
Delay storing the root hash when fast syncing a world state until the download completes so it isn't considered available and skip downloading world states that are already available.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Usman Saleem
Antony Denyer
Fabio Di Fabio
matkt
Danno Ferrin
Joshua Fernandes
mark-terry
Abdelhamid Bakhta
Adrian Sutton
Adrian Sutton
mbaxter
S. Matthew English