* Make the use of an existing fast sync state more explicit - passing it into the FastSyncDownloader.start method.
* Simplify FastSyncState constructors.
* Move PivotHeaderStorage to be FastSyncStateStorage and fully encapsulate loading and storing of FastSyncState instances.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Request 384 nodes per request instead of 200 since that's how many geth will allow returning.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Skip requesting data we already have but continue to walk the tree to ensure we have all child nodes.
* Don't delete fast sync state on stop. Allow resuming world state downloads.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Allow fast sync to resume if chain head is not genesis but a fast sync is still in progress.
If present, use the stored pivot block header when restarting.
Extract the fast sync logic out of DefaultSynchronizer into it's own class.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Modify EthScheduler to use shutdownNow immediately instead of trying shutdown first.
EthScheduler executes a lot of tasks which wait for responses from the network and may have a significant number of tasks queued. Using shutdown would wait for all network responses and all queued tasks to complete before exiting which almost always reaches the 2 minute timeout allowed before switching to shutdownNow. All tasks have to cope with being unexpectedly terminated (as would happen with a kill -9) so there's no reason to have this extra delay.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Return world states as an optional to allow code to handle world states not be available cleanly.
A world state is considered available if it's root node is available.
Delay storing the root hash when fast syncing a world state until the download completes so it isn't considered available and skip downloading world states that are already available.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* register sub tasks in ParalellImportChainSegmentTask
* rememger and shutdown eth services in EthScheduler
Fixes#768
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Exceptions that serve more as flow control than error reporting should not log stack traces.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
ParallelImportChainSegmentTask, an explicitly parallel re-implementation
of PipelinedImportChainSegmentTask. Data is passed between stages
via BlockingQueues.
Pipeline stages are implemented in AbstractPipelinePeerTask and the
parent task assembles and initiates the pipeline execution.
Other changes to support this:
* Move ethTaskTimer to abstract root
* Don't use deterministic scheduler for downloader tests, this depends on explicit parallelism
* Change download segment size = 200
* Increase timeout in recoversFromSyncTargetDisconnect, the chain downloader
may stall for 10 seconds looking for an alternative target.
* Use a blocking queue instead of fixed wait period for the test peers
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Introduce GetReceiptsForHeadersTask to retry requests for receipts until they have all been received.
* Avoid requesting empty receipts from the network. (#802)
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Two changes:
* Stream chains now take up less vertical lines, only breaking on
stream operations.
* Long annotations that span multiple lines no longer have a dangling
parentesis and indent 4 spaces.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Add the services thread pool and a computation thread pool to the
EthScheduler.
* Services are long running, sequential, and infrequently start tasks
such as Full Sync and Fast Sync.
* Computations are short and high CPU intensity tasks such as ECDSA
signature extractions and POW validation. The intent is that each
runnable represents one such extraction and the extractions from a
block are saturated across available processing power. These
computations should have zero dependencies outside their object and
thread.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Download receipts
* store blocks without processing transactions
* Only apply light validation during fast sync.
* Mark fast sync chain download as complete when pivot block has been reached.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* created fixed difficulty calculator.
* Use fixed calculator at the right spot.
* Use fixeddifficulty in the main dev.json file.
* Removed development difficulty calculators.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Separate the management of sync target and actual import from the rest of the Downloader logic in preparation for introducing a fast sync chain downloader.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Makes Synchronizer responsible for deciding if it has enough peers or not rather than ProtocolManager.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Add support for initiating fast sync to DefaultSynchronizer, starting full sync once that completes.
* Wait for a minimum number of peers to be available before starting fast sync.
* Select pivot block.
* Fetch the pivot block header.
* Ensure that a majority of peers (which have the pivot block) agree on the block.
* Pull isRetryingError and assignPeer up to AbstractRetryingPeerTask so it can be reused.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Adrian Sutton
mbaxter
S. Matthew English
Danno Ferrin
mark-terry
Rob Dawson