Fixes race condition between attaching the NodeAddedListener and some state from post marked block being persisted. We now prepare and cleanup the listener only once each, on start and stop respectively.
Other changes:
Start the Pruner before FullSyncDownloader. Luckily the downloader took enough time to start downloading that this wasn't an issue but it's safer this way.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
Add a new CLI flag `--required-block` that takes a block number and a
block hash. Before using a peer for syncing we validate that the block
exists with the spcified hash at the peer.
For example `--required-block=6485846=0x43f0cd1e5b1f9c4d5cda26c240b59ee4f1b510d0a185aa8fd476d091b0097a80`
deals with the current Istanbul Ropsten chainsplit.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
We should not send a sync status for every forking block state update.
Yes, we send status updates for detected forks as well as new canonical
heads.
Instead we should send a synching message for status changes as well as
when we reorg the chain.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* adding in spdx-license-identifier & updated check for the same; removing license check from spotless
Signed-off-by: Joshua Fernandes <joshua.fernandes@consensys.net>
* Change CheckSpdxHeader to a task.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Expose EthPeer validation state so that the Synchronizer can choose peers based on whether or not they have been fully validated. This allows us to use only fully validated peers when choosing a pivot block.
Signed-off-by: Meredith Baxter <meredith.baxter@consensys.net>
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Interally we use a `BigInteger` for ChainID, but currently use an `int` for
NetworkID. Because the default for NetworkID is the same value as ChainID there
are some chains where this will be very problematic, and there is at least
one other long-living chain outside the 32 bit int value range.
This is a large commit because the type is baked fairly deep into some of
the other APIs and very much in the test code testing the type.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
As part of the plugin events service we need to support adding and removing
listeners for events. TransactionPool only supports adding pending and dropped
transactions. To support removing we need to pass around the subscription id.
To make it consistent some APIs were renamed.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Mark Sweep Pruner
* add `unload` method on Node interface, which is a noop everywhere but on `StoredNode`
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
This reverts commit 814b36e4
The needed chantes to get rid of Instant.now (which is also needed to get rid
of the wall clock dependency) are too deep and intrusive into IBFT to try and
speed patch them in that some APIs require re-work, so in the interst of test
stability this gets sheleved until it is all ready.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
To allow us to reset the timestamp in the blockchain for Retesteth support
we need to pass a Clock to affected APIs and use that instead of the static method
System.currentTimeMillis(). The most consistent way to do this that will ensure
that the API does not sneak back in is to ban the method via ErrorProne.
TestClock.fixed() was altered to return the "now" time of the first time the fixed clock was requested, needed for many header validation tasks validating headers are not from the future.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* [PAN-1683] Limit the fraction of wire connections initiated by peers
To protect against eclipse attacks, we should not allow all of our wire connections to be initiated from network peers. Some fraction of wire connections should be initiated by our node by connecting to peers in our discovery peer table. This PR ensures the fraction abides the limit.
* change fraction default value and add tests
* make fraction of remote wire connections configurable
- add a cli option to configure the fraction: `--fraction-remote-connections-allowed`
- introduce `Fraction` class to handle the conversion of the CLI option to a double and check if the value is between 0.0 and 1.0
- add tests
- fix broken tests
* remove unused local variable
* fix RunnerTest
* set fraction to 1.0 in PantheonFactoryConfigurationBuilder
* update test
* Introduce --limit-remote-wire-connections-enabled
* fix unused field
* fix conflict
* Update ethereum/p2p/src/main/java/tech/pegasys/pantheon/ethereum/p2p/rlpx/RlpxAgent.java
Co-Authored-By: Danno Ferrin <danno.ferrin@shemnon.com>
* Update ethereum/p2p/src/main/java/tech/pegasys/pantheon/ethereum/p2p/rlpx/RlpxAgent.java
Co-Authored-By: Danno Ferrin <danno.ferrin@shemnon.com>
* Update ethereum/p2p/src/main/java/tech/pegasys/pantheon/ethereum/p2p/rlpx/RlpxAgent.java
Co-Authored-By: Danno Ferrin <danno.ferrin@shemnon.com>
* fix PR discussion and tests
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Add tolerance delta in test. Generated transaction data sizes are sometimes differents, the number of transactions in each batch may vary a bit, the tolerance delta helps dealing with such cases.
The most important thing in this test is the assertion verifying that the union of all batches equals to the original list of messages to send.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Display World State Sync Progress in Logs
- compute estimated world state completion
- display estimated world state completion in the logs
- `CompleteTaskStep` now have access to a `LongSupplier` to retireve the number of pending requests
- use a `RunnableCounter` to trigger displaying every 1000 requests completed
- only show a new log when the estimation changes
- added test to check the estimation computation
* report using raw ratio rather than percentage
* resolve PR discussion
* rename getTotal to get
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* [PIE-1736] Implement incoming transaction messages CLI option as an unstable command.
`--tx-pool-keep-alive-seconds` should be a `-X` option.
- `tx-pool-keep-alive-seconds` is renamed to `incoming-tx-messages-keep-alive-seconds`
- `incoming-tx-messages-keep-alive-seconds` is an unstable command not visible for the user (`--Xincoming-tx-messages-keep-alive-seconds`)
* fix unit test & spotless apply
* use TransactionPoolConfiguration to hold unstable option.
* add tests
* remove unecessary mock stubbing
* spotless apply
* use real builder instead of mock
* remove commented code
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Updated TransactionValidationParams with checkLocalPermissioning flag
* Added TransactionValidationParam to TxProcessor
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Columnated storage to allow for iteration over world state
* change MetricsCategory to PantheonMetricsCategory
* consistency renaming of kvstores
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* [PAN-2850] Create a transaction pool configuration object
* fix unit test by adding equals and hash code methods
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* [PIE-1710] Expose a CLI option to configure the life time of transaction messages.
- add a cli option `--tx-pool-message-keep-alive-seconds`
* rename cli option
* change cli option description
* spotless apply
* Update everything_config.toml
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* use MockitoJUnitRunner to run TransactionsMessageProcessor tests
* implement metrics counter and logging
* (format) replace statement by lambda expression
* (doc) fix javadoc
* spotless apply
* remove modulo and reset step counter to zero
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* [PIE-1707] Implement a timeout in TransactionMessageProcessor
- `processTransactionsMessage` now takes a `keepAlive` parameter
- don't process the message if expired
- add unit tests
- use a default timeout for transactions (1 minute)
* Update ethereum/eth/src/test/java/tech/pegasys/pantheon/ethereum/eth/transactions/TransactionsMessageProcessorTest.java
Co-Authored-By: Nicolas MASSART <NicolasMassart@users.noreply.github.com>
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* [PAN-2786] Stop Transaction Pool Queue from Growing Unbounded
- use a `ArrayBlockingQueue` with a fixed size to limit the transaction task queue
- expose a method in `MonitoredExecutors` to create a working queue with a maximum capacity
- update `EthScheduler` to use a limited working queue for the `txWorkerExecutor`
* [PAN-2786] Implement a bounded timed queue
- implement a custom bounded queue
- use a time based policy with keep alive configuration
- implement eviction process based on the policy
- add metrics
* use field instead of parameter
* fix PR pass 1
- use concrete class instead of interface
- change metric name to comply with global policy
- update unit test
- wrap `Runnable` into `scheduleTxWorkerTask`
* fix PR
- remove time based policy
- use raw `Runnable`
- make a room for a new element at full capacity
* Update BoundedQueueTest.java
invert condition
* fix PR comments
- remove Mock class
- make logic more thread safe, avoid race condition
- remove element until the new one is accepted
* Update ethereum/eth/src/main/java/tech/pegasys/pantheon/ethereum/eth/manager/EthScheduler.java
Co-Authored-By: Adrian Sutton <adrian@symphonious.net>
* spotless apply
* fix nit comments
- use assertj assertions for better readability
- improve unit test
* spotless apply
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* [PAN-2630] Synchronizer should disconnect the sync target peer on invalid block data
- check if headers are sequential
- if sequential, check if they form a chain, if not, disconnect the peer
* update behaviour and add tests
- change checks order
- add log message
- write tests
* disconnect sync target when InvalidBlockException
* remove power mockito
- remove powermockito
- write tests in subclasses of the `AbstractGetHeadersFromPeerTaskTest`
* fix Exception check bug
* fix PR discussion first pass
- add assertion to check when the peer is disconnected
- add assertion to check the peer has not been disonnected
- undo remove final on class
- remove sync target field
* remove spy invocation
* spotlessApply
* shouldDisconnectPeerIfInvalidBlockException
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Adrian Sutton
mbaxter
Ratan Rai Sur
Danno Ferrin
Joshua Fernandes
CJ Hare
mark-terry
Danno Ferrin
mbaxter
Karim T
Abdelhamid Bakhta
pinges
Lucas Saldanha