Implementation currently gets as far as managing the sync target and downloading checkpoint headers.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Also increases the number of requests without progress before considering the download stalled.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Number of metrics labels need to match up with constructor
* Number of labels must be consistant, so I split it into two metrics
* Also, naming best practices say that sum() and avg() of a metric
should be meaningful, separating into two metrics fixes that.
* fix style issues (finals, intellij warnings)
* Change NoOpMetrics to check label count.
* Cascading changes to support this in many support classes. Mostly places
we presumed all NoOpMetrics were equals.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Added ability to subscribe to dropped transactions from the transaction pending pool.
Implemented subscription webservice to support this.
* Added metrics to the pending transactions, tracking the number of local and remote transactions in the pool.
* Converted listener management in pending transactions to use the Subscribers util object.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
The 25% of maximum peers is fairly arbitrary but is deliberately above zero to ensure we contribute something to the overall network even while catching up.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
When chain downloading, the SyncTargetManager continues to
attempt to download headers/blocks from the current syncTarget,
even if the system is "insync". This is due to the fact that
the "insync" check is only performed at initial peer selection,
rather than on each loop of the chain downloader.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
This only happens when there are multiple chain downloads in progress so the common ancestor is above the last header we imported, but safety-first...
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Synchronizer returns false if it is in sync
* expand RunnerTest to test eth_syncing behaviour
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Biggest change is that UnusedVariable and UnusedMethod went to WARN by
default. Since our build is a no warning build this means we either need
to turn them off or fix them. I mostly opted for the latter. Test code
was mostly fixed, unused loggers were deleted, and other shipped code
was mostly suppressed.
Two less noisy fixes to not use `SortedSet` and to use zero based
comparable results instead of -1, 0, and 1. Also a compiler nit in
errorprone was suppressed, per the description it won't affect us.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Capture input and output metrics for each pipe rather than out input.
* Replace the batching processor with a BatchingReadPipe that wraps a normal pipe and creates batching on the fly.
Avoids needing an extra thread and synchronization overhead to create batches.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Improve the import log line to include relevant metrics like number
of transactions, ommers, and gas usage.
Also, this fixes a bug in that we get task times when metrics are not
turned on.
Sample line:
2019-03-11 21:03:50.245+00:00 | EthScheduler-Workers-3 | INFO | BlockPropagationManager | Imported #36 / 20 tx / 0 om / 420,000 (4.0%) gas / (0xecb366469d88d9d75b20721581797c1eb1118557b93db9a460dc0e000b9ec7fe) in 0.143s.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Store the instant that a pending transaction is added to the txpool, and expose this out through the api.
* Clock.systemUTC only used once.
* TestClock class added with a factory method for a fixed clock.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Forces subclasses have to implement it so they specify minimum requirements for the peer to use.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Start P2P network synchronously so the ports are guaranteed to be known before we write the ports file
Include the P2P TCP port in ports file even when peer discovery is disabled.
Load information from the advertised peer rather than the discovery listening socket.
Fix admin_nodeInfo to include the ?discport param in the enode URI when the discovery port differs from the P2P port.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* [PAN-2341] Created mechanism to publish sync status
* Refactoring WebSockets syncing subscription to respond to SyncStatus events
* Removing rpc-ws-refresh-delay option
* Removing ws-refresh-delay option from ATs
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Use a single thread to persist received world state nodes to avoid RocksDB timeouts due to contention on the write lock.
* Fix RocksDbTaskQueue so it doesn't read stale data when resuming a transfer.
* Skip downloading empty trie nodes.
* Log unhandled errors from world state download requests
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Instantiate individual metrics where they're used
* Cache prometheus metrics to allow "duplicate" creation of metrics
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Adds a PeerValidator that, when the Dao fork milestone is in use, checks that the Dao block is present on each peer when they connect and disconnects them if they are on the wrong chain.
Also:
* Make GetHeadersFromPeer task stricter in validating response matches.
* Update BlockHeadersMessage to return a list of headers
* Add more controls to DeterministicEthScheduler test util
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Do parallel extract signatures in the parallel block importer.
* remove the extraction from FullSyncBlockHandler
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Adrian Sutton
Danno Ferrin
Rob Dawson
Trent Mohay
mbaxter
Sally MacFarlane
Lucas Saldanha
S. Matthew English