Two changes:
* Stream chains now take up less vertical lines, only breaking on
stream operations.
* Long annotations that span multiple lines no longer have a dangling
parentesis and indent 4 spaces.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Add the services thread pool and a computation thread pool to the
EthScheduler.
* Services are long running, sequential, and infrequently start tasks
such as Full Sync and Fast Sync.
* Computations are short and high CPU intensity tasks such as ECDSA
signature extractions and POW validation. The intent is that each
runnable represents one such extraction and the extractions from a
block are saturated across available processing power. These
computations should have zero dependencies outside their object and
thread.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Download receipts
* store blocks without processing transactions
* Only apply light validation during fast sync.
* Mark fast sync chain download as complete when pivot block has been reached.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* created fixed difficulty calculator.
* Use fixed calculator at the right spot.
* Use fixeddifficulty in the main dev.json file.
* Removed development difficulty calculators.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Separate the management of sync target and actual import from the rest of the Downloader logic in preparation for introducing a fast sync chain downloader.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Makes Synchronizer responsible for deciding if it has enough peers or not rather than ProtocolManager.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Add support for initiating fast sync to DefaultSynchronizer, starting full sync once that completes.
* Wait for a minimum number of peers to be available before starting fast sync.
* Select pivot block.
* Fetch the pivot block header.
* Ensure that a majority of peers (which have the pivot block) agree on the block.
* Pull isRetryingError and assignPeer up to AbstractRetryingPeerTask so it can be reused.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* use Peer for enode not String
* moved NodeWhitelistController and PermissionConfig to p2p/permissioning
* reject messages if not from a whitelisted peer
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Time all tasks
This is fairly high touch consisting of 3 things:
* Moving to Prometheus's Summary for timers
* Timing at .2, .5, .8, .9, .99, and 1.0 (1.0 actually gets max I believe)
* Timing all abstract EthTasks
* The bulk of the changes: plumbing the timing context everywhere we need it
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* NC-1880 High TX volume swamps block processing
Move transaction processing into its own thread(s).
Size of txWorkerExecutor thread pool can be independently configured.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Plumb in three more metrics
* add blockchain_height gauge
* add blockchain_difficulty_total gauge
* add blockchain_announcedBlock_ingest histogram
This involved some deep pluming such that the metrics system needs to be
created in the PantheonCommand, along with trickle down effects into other
consensus engines. This is likely where it should live anyway.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Metrics being captured initially:
Total number of peers ever connected to
Total number of peers disconnected, by disconnect reason and whether the disconnect was initiated locally or remotely.
Current number of peers
Timing for processing JSON-RPC requests, broken down by method name.
Generic JVM and process metrics (memory used, heap size, thread count, time spent in GC, file descriptors opened, CPU time etc).
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Upgrade dependency versions
* All version information is merged into one place
* Upgrade most version to their latest non-test version.
* dependencies are now sorted
Two dependencies were not upgraded:
errorprone - There are new checks that require build or source changes.
vertex - The license to eclipse 2.0 which is not in our current approved license list.
The change is trivial but for tracability it should be done alone.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Remove release/retain requirement from MessageData.
MessageData now wraps a BytesValue rather than a Netty ByteBuf so we no longer need to call release/retain and pass that through to the underlying ByteBuf.
Entirely removed support for a BytesValue wrapping a Netty ByteBuf as it does not provide any way to release the underlying ByteBuf.
* Add BytesValue.copyTo(byte[]) to avoid the need to call getArrayUnsafe() to copy the data.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Some more logging level downgrades.
* At steady state we should get only one info level log per new block
* We don't need to know about peer stuff at info
* Caught up to best peer fires for every block
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Update value returned by web3_clientVersion to be ethstats friendly
* Version part starts with a v
* SNAPSHOT builds report 32 bits of the git hash
* OS and architecture are sniffed out and reported
* JVM version and branding are sniffed out and reported
Example values (not all real):
pantheon/v0.9.0-dev-f800a0b1/osx-x86_64/oracle-java-1.8
pantheon/v0.9.0-dev-27960b57/osx-x86_64/zulu-java-11
pantheon/v0.9.0/linux-arm64/openjdk-java-12
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Add tests that hit the retry failure case
Move "complete what we just got" semantic into the
DownloadHeaderSequenceTask. Previously it was completing everything
in the chunk it was assigned.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Danno Ferrin
Adrian Sutton
mbaxter
mark-terry
Rob Dawson
Michael Connor
Jason Frame
Sally MacFarlane