* logging cleanup, giant new payload.
Signed-off-by: Matt Nelson <85905982+non-fungible-nelson@users.noreply.github.com>
---------
Signed-off-by: Matt Nelson <85905982+non-fungible-nelson@users.noreply.github.com>
Add support for withdrawals in GraphQL, including needed changes to
testing infrastructure for shanghai-era blocks.
Also align existing adapters with graphql schema optionality.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Allow scheme (ws or wss) to be specified in ethstats url
* Start ethstat service after main ethereum loop is up
Signed-off-by: Usman Saleem <usman@usmans.info>
* Fix the number of RocksDB log files to 5 with 100 MB size for each LOG file
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
* Fix build issue
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
* spotless and add a changelog entry
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
* keep 1 week log files
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
* Update change log and fix the number of files
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
---------
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
Signed-off-by: ahamlat <ameziane.hamlat@consensys.net>
Co-authored-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Introduce experimental layered transaction pool
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* new Xlayered-tx-pool flag to enabled the new tx pool
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Move pending transaction sorter tests in the sorter folder
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Unit tests for new and old transaction pool
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix: do not decrease size when promoting ready txs
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix: remove tx from orderByFee when replaced
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix: decrease size when removing confirmed txs
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix: always recreate orderByFee for London fee market
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix: transaction removal counter when txs added to block
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix: update expected nonce when demoting a prioritized transaction
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix: correctly remove expected nonce entry when removing the last prioritized transaction for the sender
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix NullPointerException when the replaced tx is not prioritized
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Replace postponed with spare transactions
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* WIP
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* WIP
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix merge from main
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fixed most tests
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix more tests
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Rename and reorg some classes
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* More renaming and code clean up
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Refactor transaction pool metrics
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Stats log refined
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Cleanup unit tests
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Improve stats log
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Remove unnecessary test parameters
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix unit test
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
# Conflicts:
# ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/Transaction.java
# ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidator.java
* Cancel older block creation tasks upon receiving a new one
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
* Fixes to expected next nonce for sender
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix promotion filter and use synchronized methods instead of blocks
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix metrics concurrent access issue
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fixes
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Configuration options for the layered txpool
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Use long instead of Instant for PendingTransaction add time
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fixes
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Move layered txpool clasess in a dedicated package
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fixes
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* WIP
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Remove confirmed transaction from sparse set too
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* WIP
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* WIP
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fill gap on added tx
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix eviction on sparse layer
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix remove from ready layer
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix remove from sparse layer
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix for block added and confirmed txs
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fixes to sparse layer
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix the filling of the gap when adding transactions
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Layered pending transactions test and fixes
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Distinguish between layer and comulative space used
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* unit tests for expected next nonce for sender
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Adding test for transaction selection
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Re-enable prioritized transaction tests and more fixes
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* log stats
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* syncronized some methods, metrics update and dump transactions for replay
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Test that replay tx and fix for tx replacement across layers
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Add missing copyright and fix replay test
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Add consistency check asserts
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix ready internalRemove
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Metrics tests improvements
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* WIP: Transaction memory size estimation
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Complete pending transaction memory used computation
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Improve metrics
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Rename to specify that the limit is per layer
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update metric names in tests
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Adjust tx layer max capacity according to new tx memory size calculation
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix legacy transaction expiration tests
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix IndexOutOfBoundsException in sparse layer
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Unique senders metric
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Ignore ReplayTest by default, fix logging of stats
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Log for replay renamings
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Document howto generate txpool replay
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Reduce max layer capacity
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* exclude transaction replay resource
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Improve compareByFee when effectivePriorityFee is 0
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* More debug logs during transaction selection for a block
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Use only one thread for building blocks so there is no risk of overlapping
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Improve transaction trace log making wei human readable
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Use List instead of Set when getting all pending transactions
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* More detailed log on adding remote txs
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Execute transaction broadcast aysnc after adding them to the txpool
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Log time taken to add remote txs before their broadcast
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix test
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Add missing header
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix unit tests
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Add CHANGELOG entry
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Delete unneeded file
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Rename some layered txpool metrics to avoid conflict with existing metrics
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix pushing to next layers txs following an invalid one
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* In case of an unexpected error, log more data and do a consistency check
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix null check on wrong var
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix some codeql alerts
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix sparse gap calculation when invalidating
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Apply suggestions from doce review
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Only trigger consistency check if trace log is enable in case of unexpected error
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix replay of blocks with no transactions
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix for negative gap when there is a reorg
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Implement code review suggestions
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix for a case when deleting tx with zero gap in sparse layer
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Delete redoundant tests
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update CHANGELOG.md
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPool.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPoolMetrics.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/ReadyTransactions.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/AbstractPrioritizedTransactionsTestBase.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/SparseTransactions.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update besu/src/main/java/org/hyperledger/besu/cli/options/unstable/TransactionPoolOptions.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update besu/src/main/java/org/hyperledger/besu/cli/options/unstable/TransactionPoolOptions.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Address code review suggestions
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Improve logSender when there are no sparse txs
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix off by one error
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/TransactionsLayer.java
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Address code review suggestions
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Rename fix
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Simplify the way reorgs are handled, by detecting a negative gap and deleting and readding all the txs for that sender
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Do not run consistency check on internal error since it is too expensive,
instead force a reorg of the sender txs
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Remove invalid txs after the selection is complete to avoid ConcurrentModificationException
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update txpool defaults
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Tune default
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix merge
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
---------
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: garyschulte <garyschulte@gmail.com>
* Transaction pool flag to disable special handling of locally submitted transactions
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update CHANGELOG
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix options and more tests
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update besu/src/main/java/org/hyperledger/besu/cli/options/unstable/TransactionPoolOptions.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
---------
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Move the implementation of "big" eof into Future EIPs, out of cancun.
Also, create space for Prague, Osaka, and Bogota forks in the EVM code.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
* Use binary search to eth_estimateGas more accurately
eth_estimateGas currently conservatively estimates gas this change
will take that estimate and use it as a hi bound and will use
gasUsed as lo bound to hone in on the most true gas limit required
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
Performance improvements
- Reduce the use of UInt256 in operations, replacing them with Bytes as
appropriate.
- Use Java17 expression switch
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
* Refactor to make RocksDBColumnarKeyValueStorage abstract and extend it to optimistic and pessimistic. Atm Forest has a concurrency level that does not cope well with the OptimisticTransactionDB and RocksDB ends up raising a lot of Busy when committing RocksDBTransactions.
A TransactionDB will do conflict checking for all write operations (Put, Delete and Merge), including writes performed outside a Transaction according to RocksDB. This does impact the times we see when syncing so likely not the final solution for Forest.
Bonsai should continue using OptimisticTransactionDB.
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
---------
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Changing bonsai-max-back-layers-to-load to --bonsai-historical-block-limit
Signed-off-by: Matt Nelson <monels11@gmail.com>
* remove a couple other places where snapshot option was configured
Signed-off-by: garyschulte <garyschulte@gmail.com>
* Update besu/src/main/java/org/hyperledger/besu/cli/options/stable/DataStorageOptions.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Matt Nelson <85905982+non-fungible-nelson@users.noreply.github.com>
---------
Signed-off-by: Matt Nelson <monels11@gmail.com>
Signed-off-by: Matt Nelson <85905982+non-fungible-nelson@users.noreply.github.com>
Signed-off-by: garyschulte <garyschulte@gmail.com>
Co-authored-by: garyschulte <garyschulte@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* Add new sepolia bootnodes from EF
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
---------
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Co-authored-by: Parithosh Jayanthi <parithosh@indenwolken.xyz>
* removed separate decoding logic
* run non-mainnet ATs to make sure
* remove goQuorum flag from everywhere
* remove GOQUORUM api group
* remove GoQuorum enclave, privacy params, RPCs and other related config
* removed Goquorum related error codes, rpcMethod codes, privateBloom
* removed v from transaction
* removed private GoQuorum storage provider
---------
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Add 8 metrics to evaluate the state of the flat database for accounts and storage slots
* Total number of calls (for both accounts and storage)
* Number of account or storage entries found in the flat database
* Number of storage slots located in the merkle trie but not found in the flat database
* Number of account or storage entries not found in either the flat database or the merkle trie
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* remove controller and config
* remove ibftlegacy module
* remove further config options
* run non-mainnet ATs just to be sure
* remove temp change to circle config
---------
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
* Fix ethstats retry logic. Attempt to connect with and without ssl. If --ethstats is specified without a port, use port 443 (for wss://) and port 80 (for ws://) as defaults instead of 3000.
* Introduced optional --ethstats-cacert-file to specify root CA for ethstats server.
Signed-off-by: Usman Saleem <usman@usmans.info>
While syncing, it is possible that we access the cached header's number before the block itself has had chance to be imported. In other words there is a mismatch between cached blockNumber and non-cached blockHeader that is being retrieved for the RPC.
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Override the transactionSimulator's default TransactionValidationParams with one that allows for exceeding the account balance (which effectively zeros the baseFee).
This mimics the way that eth_estimateGas and eth_call are implemented.
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Changes and config to support using GraalVM AOT to compile and
execute evmTool as an alternate configuration.
While not as long-term performant the startup time
makes filling reference tests with Besu reasonable.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Fabio Di Fabio
garyschulte
Matt Nelson
George Tebrean
Danno Ferrin
Usman Saleem
Simon Dudley
Adrian Sutton
Gabriel-Trintinalia
Jason Frame
ahamlat
Sebastian Bathke
Danno Ferrin
Justin Florentine
Antony Denyer
Diego López León
Nischal Sharma
Gabriel Fukushima
Sally MacFarlane
Håvard Anda Estensen