This API fulfils the basic requirements, but will probably be extended in the near future.
Signed-off-by: Stefan <stefan.pingel@consensys.net>
Signed-off-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* Log access denied error separately to other IO exceptions so cause is clearer
Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>
* Fix Java spotless errors
Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>
---------
Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>
Co-authored-by: Matthew Whitehead <matthew1001@gmail.com>
* Add info log to indicate Besu is waiting on CL to be synced
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
---------
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* Introduce experimental layered transaction pool
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* new Xlayered-tx-pool flag to enabled the new tx pool
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Move pending transaction sorter tests in the sorter folder
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Unit tests for new and old transaction pool
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix: do not decrease size when promoting ready txs
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix: remove tx from orderByFee when replaced
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix: decrease size when removing confirmed txs
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix: always recreate orderByFee for London fee market
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix: transaction removal counter when txs added to block
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix: update expected nonce when demoting a prioritized transaction
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix: correctly remove expected nonce entry when removing the last prioritized transaction for the sender
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix NullPointerException when the replaced tx is not prioritized
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Replace postponed with spare transactions
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* WIP
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* WIP
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix merge from main
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fixed most tests
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix more tests
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Rename and reorg some classes
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* More renaming and code clean up
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Refactor transaction pool metrics
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Stats log refined
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Cleanup unit tests
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Improve stats log
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Remove unnecessary test parameters
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix unit test
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
# Conflicts:
# ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/Transaction.java
# ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidator.java
* Cancel older block creation tasks upon receiving a new one
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
* Fixes to expected next nonce for sender
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix promotion filter and use synchronized methods instead of blocks
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix metrics concurrent access issue
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fixes
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Configuration options for the layered txpool
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Use long instead of Instant for PendingTransaction add time
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fixes
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Move layered txpool clasess in a dedicated package
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fixes
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* WIP
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Remove confirmed transaction from sparse set too
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* WIP
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* WIP
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fill gap on added tx
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix eviction on sparse layer
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix remove from ready layer
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix remove from sparse layer
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix for block added and confirmed txs
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fixes to sparse layer
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix the filling of the gap when adding transactions
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Layered pending transactions test and fixes
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Distinguish between layer and comulative space used
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* unit tests for expected next nonce for sender
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Adding test for transaction selection
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Re-enable prioritized transaction tests and more fixes
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* log stats
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* syncronized some methods, metrics update and dump transactions for replay
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Test that replay tx and fix for tx replacement across layers
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Add missing copyright and fix replay test
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Add consistency check asserts
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix ready internalRemove
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Metrics tests improvements
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* WIP: Transaction memory size estimation
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Complete pending transaction memory used computation
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Improve metrics
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Rename to specify that the limit is per layer
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update metric names in tests
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Adjust tx layer max capacity according to new tx memory size calculation
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix legacy transaction expiration tests
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix IndexOutOfBoundsException in sparse layer
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Unique senders metric
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Ignore ReplayTest by default, fix logging of stats
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Log for replay renamings
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Document howto generate txpool replay
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Reduce max layer capacity
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* exclude transaction replay resource
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Improve compareByFee when effectivePriorityFee is 0
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* More debug logs during transaction selection for a block
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Use only one thread for building blocks so there is no risk of overlapping
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Improve transaction trace log making wei human readable
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Use List instead of Set when getting all pending transactions
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* More detailed log on adding remote txs
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Execute transaction broadcast aysnc after adding them to the txpool
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Log time taken to add remote txs before their broadcast
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix test
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Add missing header
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix unit tests
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Add CHANGELOG entry
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Delete unneeded file
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Rename some layered txpool metrics to avoid conflict with existing metrics
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix pushing to next layers txs following an invalid one
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* In case of an unexpected error, log more data and do a consistency check
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix null check on wrong var
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix some codeql alerts
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix sparse gap calculation when invalidating
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Apply suggestions from doce review
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Only trigger consistency check if trace log is enable in case of unexpected error
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix replay of blocks with no transactions
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix for negative gap when there is a reorg
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Implement code review suggestions
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix for a case when deleting tx with zero gap in sparse layer
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Delete redoundant tests
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update CHANGELOG.md
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPool.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPoolMetrics.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/ReadyTransactions.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/AbstractPrioritizedTransactionsTestBase.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/SparseTransactions.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update besu/src/main/java/org/hyperledger/besu/cli/options/unstable/TransactionPoolOptions.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update besu/src/main/java/org/hyperledger/besu/cli/options/unstable/TransactionPoolOptions.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Address code review suggestions
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Improve logSender when there are no sparse txs
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix off by one error
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/TransactionsLayer.java
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Address code review suggestions
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Rename fix
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Simplify the way reorgs are handled, by detecting a negative gap and deleting and readding all the txs for that sender
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Do not run consistency check on internal error since it is too expensive,
instead force a reorg of the sender txs
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Remove invalid txs after the selection is complete to avoid ConcurrentModificationException
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update txpool defaults
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Tune default
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix merge
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
---------
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: garyschulte <garyschulte@gmail.com>
* Refactor to remove duplicated code that was handling RLP read/write. This logic doesn't change that ofter having the read and writes spread accross the code base has tripped some devs. We are now handling all the Body readfrom rlp and write to rlp using methods of the BlockBody class.
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
---------
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Transaction pool flag to disable special handling of locally submitted transactions
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update CHANGELOG
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix options and more tests
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update besu/src/main/java/org/hyperledger/besu/cli/options/unstable/TransactionPoolOptions.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
---------
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* add -validity option to -gencert commands
* update certs to be valide 100 years
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Stefan <stefan.pingel@consensys.net>
Co-authored-by: Stefan <stefan.pingel@consensys.net>
* Fold AbstractProtocolScheduleBuilder into ProtocolScheduleBuilder
* Tidy up WithdrawalsValidatorProvider
* Remove unnecessary createSchedule in JsonRpcMethodsFactory now timestamps are in the main protocolschedule
* Fold BlockNumberStreamingProtocolSchedule into MilestoneStreamingProtocolSchedule
* Rename UnifiedProtocolSchedule to DefaultProtocolSchedule
* inline ProtocolScheduleBuilder.postBuildStep
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
* add account zero reads, mark self-destructed accounts, code and storage as cleared
* refactor BonsaiWorldView and TrieLog to use StorageSlotKey record type rather than slotHash
* add isCleared to read/write of BonsaiValues
* for consistency between serialize and deserialize, use null directly in TrieLogLayer.addCodeChange rather than Bytes.EMPTY
Signed-off-by: garyschulte <garyschulte@gmail.com>
* Use binary search to eth_estimateGas more accurately
eth_estimateGas currently conservatively estimates gas this change
will take that estimate and use it as a hi bound and will use
gasUsed as lo bound to hone in on the most true gas limit required
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* dagger component
* pushed up MetricsSystemModule
* passes around the BesuComponent as an interim application context till more things are managed by Dagger
---------
Signed-off-by: Justin Florentine <justin+github@florentine.us>
- Combine MutableProtocolSchedule and DefaultTimestampSchedule into UnifiedProtocolSchedule.
- Implement getByBlockHeader taking into account both timestamp-based and blockNumber-based forks
- Unstitch timestampSchedule from TransitionProtocolSchedule
- BftProtocolSchedule extends UnifiedProtocolSchedule (instead of MutableProtocolSchedule)
- In MergeProtocolSchedule, unapply mergeSpecificModifications from Shanghai onwards
- Migrate getByBlockNumber modifications into getByBlockHeader for retesteth NoReward
- Temporarily fix reference tests by artificially increasing shanghaiTime
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* trielog save event, observer, test
* add zero read slots to Accumulator storageToUpdate, omit zero read slots from trielog generation
* add account zero reads, mark self-destructed accounts, code and storage as cleared
Signed-off-by: garyschulte <garyschulte@gmail.com>
* Changing bonsai-max-back-layers-to-load to --bonsai-historical-block-limit
Signed-off-by: Matt Nelson <monels11@gmail.com>
* remove a couple other places where snapshot option was configured
Signed-off-by: garyschulte <garyschulte@gmail.com>
* Update besu/src/main/java/org/hyperledger/besu/cli/options/stable/DataStorageOptions.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Matt Nelson <85905982+non-fungible-nelson@users.noreply.github.com>
---------
Signed-off-by: Matt Nelson <monels11@gmail.com>
Signed-off-by: Matt Nelson <85905982+non-fungible-nelson@users.noreply.github.com>
Signed-off-by: garyschulte <garyschulte@gmail.com>
Co-authored-by: garyschulte <garyschulte@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* Removing RocksDB option MaxBackgroundCompactions deprecated in version 8.0
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
---------
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Allow lenient hex param in eth_submitHashRate
* Migrate Stratum tests to Junit5
* Introduce MockitoExtension in Stratum tests
* Convert start/stop Stratum method to Vert.x future
* Use Stratum only with PowCoordinator
* Call JSON-RPC implementations from Stratum proxy
* Subscribe to newWork on proxy login
* Replace HTTP processing with Netty pipelines
---------
Signed-off-by: Diego López León <dieguitoll@gmail.com>
Move the decision making for connecting or not connecting to peers into the eth layer. Future changes will take advantage if this to improve peering.
Signed-off-by: Stefan <stefan.pingel@consensys.net>
---------
Signed-off-by: Stefan <stefan.pingel@consensys.net>
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* removed separate decoding logic
* run non-mainnet ATs to make sure
* remove goQuorum flag from everywhere
* remove GOQUORUM api group
* remove GoQuorum enclave, privacy params, RPCs and other related config
* removed Goquorum related error codes, rpcMethod codes, privateBloom
* removed v from transaction
* removed private GoQuorum storage provider
---------
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Fabio Di Fabio
Gabriel Fukushima
Simon Dudley
garyschulte
Stefan Pingel
Gabriel-Trintinalia
Adrian Sutton
Sally MacFarlane
Matt Whitehead
Jason Frame
Sebastian Bathke
Camilo Reis Dotto
George Tebrean
matkt
Antony Denyer
Diego López León
Nischal Sharma
Justin Florentine
Matt Nelson
Danno Ferrin