Add tolerance delta in test. Generated transaction data sizes are sometimes differents, the number of transactions in each batch may vary a bit, the tolerance delta helps dealing with such cases.
The most important thing in this test is the assertion verifying that the union of all batches equals to the original list of messages to send.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Display World State Sync Progress in Logs
- compute estimated world state completion
- display estimated world state completion in the logs
- `CompleteTaskStep` now have access to a `LongSupplier` to retireve the number of pending requests
- use a `RunnableCounter` to trigger displaying every 1000 requests completed
- only show a new log when the estimation changes
- added test to check the estimation computation
* report using raw ratio rather than percentage
* resolve PR discussion
* rename getTotal to get
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* [PIE-1736] Implement incoming transaction messages CLI option as an unstable command.
`--tx-pool-keep-alive-seconds` should be a `-X` option.
- `tx-pool-keep-alive-seconds` is renamed to `incoming-tx-messages-keep-alive-seconds`
- `incoming-tx-messages-keep-alive-seconds` is an unstable command not visible for the user (`--Xincoming-tx-messages-keep-alive-seconds`)
* fix unit test & spotless apply
* use TransactionPoolConfiguration to hold unstable option.
* add tests
* remove unecessary mock stubbing
* spotless apply
* use real builder instead of mock
* remove commented code
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Updated TransactionValidationParams with checkLocalPermissioning flag
* Added TransactionValidationParam to TxProcessor
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Columnated storage to allow for iteration over world state
* change MetricsCategory to PantheonMetricsCategory
* consistency renaming of kvstores
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* [PAN-2850] Create a transaction pool configuration object
* fix unit test by adding equals and hash code methods
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* [PIE-1710] Expose a CLI option to configure the life time of transaction messages.
- add a cli option `--tx-pool-message-keep-alive-seconds`
* rename cli option
* change cli option description
* spotless apply
* Update everything_config.toml
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* use MockitoJUnitRunner to run TransactionsMessageProcessor tests
* implement metrics counter and logging
* (format) replace statement by lambda expression
* (doc) fix javadoc
* spotless apply
* remove modulo and reset step counter to zero
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* [PIE-1707] Implement a timeout in TransactionMessageProcessor
- `processTransactionsMessage` now takes a `keepAlive` parameter
- don't process the message if expired
- add unit tests
- use a default timeout for transactions (1 minute)
* Update ethereum/eth/src/test/java/tech/pegasys/pantheon/ethereum/eth/transactions/TransactionsMessageProcessorTest.java
Co-Authored-By: Nicolas MASSART <NicolasMassart@users.noreply.github.com>
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* [PAN-2786] Stop Transaction Pool Queue from Growing Unbounded
- use a `ArrayBlockingQueue` with a fixed size to limit the transaction task queue
- expose a method in `MonitoredExecutors` to create a working queue with a maximum capacity
- update `EthScheduler` to use a limited working queue for the `txWorkerExecutor`
* [PAN-2786] Implement a bounded timed queue
- implement a custom bounded queue
- use a time based policy with keep alive configuration
- implement eviction process based on the policy
- add metrics
* use field instead of parameter
* fix PR pass 1
- use concrete class instead of interface
- change metric name to comply with global policy
- update unit test
- wrap `Runnable` into `scheduleTxWorkerTask`
* fix PR
- remove time based policy
- use raw `Runnable`
- make a room for a new element at full capacity
* Update BoundedQueueTest.java
invert condition
* fix PR comments
- remove Mock class
- make logic more thread safe, avoid race condition
- remove element until the new one is accepted
* Update ethereum/eth/src/main/java/tech/pegasys/pantheon/ethereum/eth/manager/EthScheduler.java
Co-Authored-By: Adrian Sutton <adrian@symphonious.net>
* spotless apply
* fix nit comments
- use assertj assertions for better readability
- improve unit test
* spotless apply
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* [PAN-2630] Synchronizer should disconnect the sync target peer on invalid block data
- check if headers are sequential
- if sequential, check if they form a chain, if not, disconnect the peer
* update behaviour and add tests
- change checks order
- add log message
- write tests
* disconnect sync target when InvalidBlockException
* remove power mockito
- remove powermockito
- write tests in subclasses of the `AbstractGetHeadersFromPeerTaskTest`
* fix Exception check bug
* fix PR discussion first pass
- add assertion to check when the peer is disconnected
- add assertion to check the peer has not been disonnected
- undo remove final on class
- remove sync target field
* remove spy invocation
* spotlessApply
* shouldDisconnectPeerIfInvalidBlockException
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Adds metrics to expose:
* height of best known block
* synchronizer in sync flag
* max peers
* Timestamp of current chain head
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* It returns for any block we would broadcast to other peers, when
we would broadcast them.
* It returns a JSON String containing hash, number, and timestamp
* This event data is not set in stone, it may change in type or content.
* Acceptance tests and unit tests got a re-work away from the assumption
that there is only one plugin type.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Reduce the amount of work done within synchronized blocks in PendingTransactions.
In particular getTransactionByHash and notifying listeners of added or removed transactions are no longer in synchronized blocks.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Bail out of addRemoteTransactions immediately if we're not in sync instead of after we sort by nonce and begin iterating.
* Don't sort transactions by nonce. This was only required because we previously rejected transactions where the nonce was too high.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Don't copy collections if we don't need to. Change types higher up if needed.
* Don't use Guava's Object.equal, use Java's Objects.equals.
** add errorprone test to enforce the banning of Guava's Objects class.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Change all Stream<?> getX() and Stream<?> x() methods to Stream<?> streanX methods, such as `Stream<Peer> streamIdlePeers()`
* Update coding conventions to reflect this.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Replaces the RocksDB based queue for pending world state download tasks with one that uses a simple file. Added tasks are appended to the file while the reader starts from the beginning of the file and reads forwards.
Periodically a new file is started to limit the disk space used. The reader deletes files it has completed reading.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Fix non-deterministic test caused by variable size of generated transactions.
Validate transactions against a size range rather than a fixed size
* Use assertj instead of own method.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Provides better control over the maximum number of concurrent requests to a peer.
Gives a fairer allocation of peers to requests being made.
Avoids a task failing because all peers are busy which then introduces a delay before the request is retried.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Abdelhamid Bakhta
pinges
mbaxter
Lucas Saldanha
Ratan Rai Sur
Adrian Sutton
Danno Ferrin
Trent Mohay