The idea behind this commit is to preload asynchronously account nodes and storage nodes from the database during the transaction processing to use these nodes during the calculate root hash step.
We've created two caches, one for account nodes and one for storage nodes. The size of these caches is 100k for accounts and 200k for storage. We've tested other values but this configuration is the one that works better.
We also use exporter cache metrics as Prometheus metrics to check cache efficiency.
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
Co-authored-by: Ameziane H <ameziane.hamlat@consensys.net>
* aggresively seeking out other places to defend from this
Signed-off-by: Justin Florentine <justin+github@florentine.us>
Co-authored-by: Justin Florentine <justin+github@florentine.us>
* Rename fastSyncEnabled to fullSyncDisabled and pass syncConfig down instead of boolean
Signed-off-by: Gabriel Trintinalia <gabriel.trintinalia@gmail.com>
* Fix tests references
Signed-off-by: Gabriel Trintinalia <gabriel.trintinalia@gmail.com>
* Fix class references
Signed-off-by: Gabriel Trintinalia <gabriel.trintinalia@gmail.com>
Signed-off-by: Gabriel Trintinalia <gabriel.trintinalia@gmail.com>
Signed-off-by: Gabriel Trintinalia <gabriel.trintinalia@consensys.net>
Signed-off-by: Gabriel Trintinalia <gabriel.trintinalia@consensys.net>
Co-authored-by: Gabriel Trintinalia <gabriel.trintinalia@consensys.net>
* Add Shandong network and fork
Add the transient Shandong fork and network definitions. For Shanghai
fork testing. Besu doesn't sync, but it does connect.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
* implementation of Bonsai snapshots based BonsaiWorldStateArchive
includes: try-with-resources and AutoCloseable WorldState in order to release snapshots when we are done with them
Signed-off-by: garyschulte <garyschulte@gmail.com>
Due to subtle differences between Bytes32 and UInt256 the changes failed
burn-in testing and are being reverted.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
* don't add to bad blocks manager on StorageException
* support for MerklePatriciaTrie exceptions
Signed-off-by: Justin Florentine <justin+github@florentine.us>
The current internal storage APIs implement UInt256 rather than Bytes32,
which is an accident of prior EVM design. This migrates all internal
handling of storage addresses and values to Bytes32.
The main performance gain is reduced GC pressure, in that UInt256
stores the data as an array of ints, whereas Bytes32 keeps a sliced or
wrapped array of bytes. Since database APIs are all byte array based
this should remove unneeded conversions when going to/from the DB.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Based on previous work of @garyschulte
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Refactor to optimize pivot block selector on PoS networks
On PoS network we use a pivot block sent by the Consensus Layer, so we do
not need peers, and so all the logic for selecting the pivot block from peers
has been moved from FastSyncActions to PivotSelectorFromPeers.
We do not need anymore the TransictionPeerSelector, and the --fast-sync-min-peers
applies only to PoW networks.
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Refactor unverified forkchoice event
Renaming to make clear everywhere that the forkchoice is not verified, so
it will be clear in case there will be a future event for a verified forkchoice.
Finalized block hash no more optional.
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* RequestManager: Lessen penalty for late responses with request ID
If the communication with a peer uses the eth/66 protocol, requests have
an accompanying request ID. If the response to any of these requests was
late, we would disconnect and ban the peer mmediately. This is too
excessive a punishment.
A late response is typically already punished with a timeout before (too
many of which and we disconnect). This commit changes the immediate
banning of the peer to just considering the late response useless
(again, too many of which and we disconnect).
Note that in eth/65, the lack of a request ID would mean we would just
consider the late response to be useless (or process it anyway). This
commit therefore brings the punishment of late responses more in line
with what it used to be before request IDs were introduced.
Closes#4320
Signed-off-by: Pietje Puk <pietje@pietjepuk.net>
* PeerDenylistManager: Do not ban static/maintained peers
These types of peers are added manually by the user, and have a certain
trusted status. We should therefore not ban them.
Note that we will still disconnect from these peers when they exhibit
undesirable behavior (e.g. repeated timeouts). We will however continue
to reconnect to them.
Signed-off-by: Pietje Puk <pietje@pietjepuk.net>
Signed-off-by: Pietje Puk <pietje@pietjepuk.net>
Co-authored-by: Pietje Puk <pietje@pietjepuk.net>
* Replace boolean return with BlockImportResult object
Signed-off-by: Gabriel Trintinalia <gabriel.trintinalia@consensys.net>
Co-authored-by: Gabriel Trintinalia <gabriel.trintinalia@consensys.net>
* make transaction pool limits for sender based on pool size. default by sender is 5
move config into TransactionPoolOptions class
Signed-off-by: garyschulte <garyschulte@gmail.com>
* Create a new flag on RocksDB (--Xplugin-rocksdb-high-spec-enabled) for high spec hardware to boost performance.
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
* Add more tracing logs to transaction pool
* Prevent adding non executable transactions and evict based on age
* Move sender account in TransactionsForSenderInfo
* Implement a size expiration cache for lowest invalid nonce for sender
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Justin Florentine <justin+github@florentine.us>
Co-authored-by: Justin Florentine <justin+github@florentine.us>
* Retry mechanism for block creation
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update CHANGELOG
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Always keep 1 thread active in the computation executors
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Surface StorageException when building a block for finer filtering
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Log successful block created at info
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Reformat block proposal logs
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Remove test code
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Simon Dudley
matkt
Fabio Di Fabio
Gabriel-Trintinalia
Danno Ferrin
Daniel Lehrner
Sally MacFarlane
Stefan Pingel
Gabriel-Trintinalia
Danno Ferrin
garyschulte
Justin Florentine
pietjepuk2
Justin Traglia
ahamlat
Matt Nelson