* Detect zombies backend query and abort useless processing.
The timeout handler populate the request context with a boolean value indicating whether or not the query is alive, i.e the HTTP request has not expired. Backend queries are now conditioned by this value and can be stopped if needed.
This PR experiments this mechanism on a reduced scope. Hence, only `eth_getLogs` backend queries are affected.
- Created `BackendQuery` utility class to run a process only if the query is alive, i.e timeout not expired.
- Put `AtomicBoolean` value in the `JsonRpcRequestContext`
- `TimeoutHandler` sets the alive value to `false` if the timeout handler is triggered.
- Updated `BlockchainQueries` to run steps depending on the value of the `AtomicBoolean` retrieved from the request context.
- Added unit tests.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
Way back in the pantheon days, I put the DATABASE_METADATA.json file in
the database directory. This was a bad idea because rocksdb owns that
directory. We changed to putting it in the directory we own, data, but
we needed to support the old way of doing things. The time to support
that old location has well and truly past so now we're safe to stop
looking in that directory.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* replace whitelist/blacklist with allowlist/denylist and support both versions in CLI options
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
- Implemented submitHashrate endpoint.
- Updated eth_hashrateEndpoint so that it returns the cumulative hashrate of all sealers if available. Otherwise it returns the local hashrate
- Added hashrate submission with Stratum1EthProxyProtocol and Stratum1Protocol
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
* Resolve inconsistent handling of [Bb]oolean config options by handling primitives the same as the boxed type.
Signed-off-by: Steven J Schroeder <steven.schroeder@consensys.net>
Ensures the "node" field on log messages is correct when node-clusters are restarted during AT.
Signed-off-by: Joshua Fernandes <joshua.fernandes@consensys.net>
Before this pull request Besu was using the latest known fork id to create status message for Ethereum P2P protocol handshake.
This latest known fork id was created based on a list of forks retrieved from the Genesis.
For private networks it is possible that all fork blocks number are set to 0.
The algorithm to compute the valid fork hashes excludes 0 values.
As a result, the list was empty and the `getLatestForkId` was returning `null`. This is an issue when you support capabilities >= to Eth/64 sub protocol because other peers expect the fork id value in the `RLP` encoded message.
Moreover, the algorithm to compute the fork id should be aware of the chain head number and update `CRC` value only for fork blocks below the current head.
This pull request fixes this issue by fetching the chain head number and update accordingly the `CRC` value.
Unit tests have been extended to cover an exhaustive list of possible combinations on named networks (`goerli`, `rinkeby`, `ropsten` and `mainnet`).
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Add onchain privacy group multi-tenancy support to priv_getPrivateTx and priv_getPrivRcpt
Signed-off-by: Lucas Saldanha <lucas.saldanha@consensys.net>
ProtocolContext uses a generic for the consensus state, which has a very
large footprint across the code to accomplish what it intends to
accomplish. For every call there are about 61 other lines per call that
need to be updated, over 1300 lines total.
Instead replace it with java.lang.Class#cast, which provides runtime
security, and use generics to provide the compile time sugar that
allows for chained methods of the appropriate type. Then remove its
(quite large) footprint from the rest of the code.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
The NAT manager's `MANUAL` method has been removed. This mode does not seem really useful because it does the same thing as a NONE mode.
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
* reduce number of validators, remove unnecessary checks for number of peers (this is already done in cluster.start) and reduce number of blocks we wait for
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
* check status of cluster not a single node
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
Performance measurements have indicated that the list of Bytes32 for
memory is not performant enough. Moving to a byte array reduces wrapper
object thrashing as well as simplifying cross-word boundary logic (there
are no boundaries).
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Add an option to run besu after blocks import. This is aimed to cut the
startup penalty in half for hive testing, combining the block import
with the node execution.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* feat: FastSync now stops on failure instead of falling back to FullSync
* feat: exiting on FastSync failed
* feat: FastSync continues on FullSync when no errors happened
Signed-off-by: Alexandre PARIS-VERGNE <alexpv14@gmail.com>
We consider the named network condition to be satisfied when --network
is explicitly used OR when it isn't supplied but neither are
--genesis-file nor --privacy-enabled
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
Extend the `blocks import` subcommand to accept multiple files in the
command line and import them in the same batch.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Improved JUMP and JUMPSUB destination check. Avoid having to browse the code if the destination == code size
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
* Handle nested TOML array to resolve TOML array decoding bug in certain plugin configs.
* Fix for picocli unable to determine if plugin options are multivalue.
Signed-off-by: Steven J Schroeder <steven.schroeder@consensys.net>
We are seeing the gradle daemon die when running the acceptance tests
```
The message received from the daemon indicates that the daemon has
disappeared.
```
Starting from Java 10, UseContainerSupport is turned on by default,
except in the case whereXmx is specified. I'll be honest in that I
don't know why the Xmx of 1gb was specified in the first place, so this
change is bad on the Chesterton's Fence front but if it has anything to
do with limiting the memory usage on containers, removing it would
probably be better.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* update getForks to ignore classicforkblock param
Signed-off-by: Edward Mack <ed@edwardmack.com>
* update gotham block fork number
Signed-off-by: Edward Mack <ed@edwardmack.com>
* add to entry to CHANGELOG.md
Signed-off-by: Edward Mack <ed@edwardmack.com>
* Use non-blocking randomness for acceptance tests
This addresses entropy draining during unit tests.
* wait 30 seconds instead of 2 when killing AT processes
* mark NodeSmartContractPermissioningIbftStallAcceptanceTest as @Ignore since it has become reliably and specifically flakey.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
The EthGetWork.shouldReturnSuccessResponseWhenMining was exhibiting intermittent failures, as a mining node would occasionally report "No mining work available yet" - rather than the inputs required to be solved for the current block.
From investigation, it was found that Besu's EthHash miner has a small time window after completing a block, in which the target for the next block has not yet been computed (eg while collating transactions) - during this interval, Besu will validly report "No mining work available yet".
To overcome this, the acceptance test has been updated to poll Besu for upto 5 seconds, waiting for a valid work target to be provided - rather than executing a single-attempt (which may fall within the window specified above).
Signed-off-by: Trent Mohay <trent.mohay@consensys.net>
DevP2P packets expiration fields should be in seconds past the epoch,
not milliseconds past the epoch.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
We cleared the mark storage upon starting the pruner in order to prevent the first sweep from being smaller than it needs to be.
```
// Optimization for the case where the previous cycle was interrupted (like the node was shut
// down). If the previous cycle was interrupted, there will be marks in the mark storage from
// last time, causing the first sweep to be smaller than it needs to be.
clearMarks();
```
Unfortunately, it's currently called in prepare, which is synchronous with the main thread. This means that we can delay startup by on the order of 10 minutes if the storage was especially full.
This commit makes the pruner `start` asynchronous.
This was reverted once before due to test intermittency and this commit is the second attempt :P
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
Rollback a Eth64 change intended to improve compatibility in consortium network. It resulted in many nodes not connecting on named networks such as goerli and mordor.
* Revert "Handle backward compatibility for EIP-2124 fork id management (#981)"
This reverts commit 1d8218ae
* Revert "[EIP-2124] Backward compatible fork id management implementation (#979)"
This reverts commit cb2c67d7
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Use seconds for timeout configuration
* Add units to variable names
* Change timeout to 5 minutes
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Sally MacFarlane
MadelineMurray
Abdelhamid Bakhta
Joshua Fernandes
Ratan (Rai) Sur
Karim T
Steven Schroeder
Trent Mohay
Lucas Saldanha
Danno Ferrin
Adrian Sutton
br0tchain
Edward Mack