Implement shanghaiTime including TimestampSchedule and associated infrastructure code.
TimestampSchedule sits alongside the pre and post ProtocolSchedules in TransitionProtocolSchedule.
Introduces getByTimestamp, wrapped inside getByBlockHeader (to also support getByBlockNumber).
General call pattern followed is that if a given timestamp precedes the first timestamp in the schedule, i.e. a pre-shanghai block, then delegate to the appropriate pre or post merge ProtocolSchedule to get by block instead.
cancunTime and a placeholder cancunDefinition has also been implemented in order to effectively test fork order logic.
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Jiri Peinlich <jiri.peinlich@gmail.com>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Jason Frame <jason.frame@consensys.net>
* Add chain pruner
* Increase minimum blocks to retain
* Skip ancestor check in pruning mode
* Separate class for pruning storage
* Move pruning to separate thread
* Limit total pruning threads
Signed-off-by: wcgcyx <wcgcyx@gmail.com>
Signed-off-by: Zhenyang Shi <wcgcyx@gmail.com>
Co-authored-by: Simon Dudley <simon.l.dudley@hotmail.com>
Co-authored-by: Jason Frame <jason.frame@consensys.net>
* Add Shandong gas calculator to support EIP-3860
Signed-off-by: Diego López León <dieguitoll@gmail.com>
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* add unit tests for Shandong gas calculator
Signed-off-by: lukelee-sl <luke.lee@swirldslabs.com>
Signed-off-by: Diego López León <dieguitoll@gmail.com>
Signed-off-by: Diego López León <dieguitoll@gmail.com>
Signed-off-by: lukelee-sl <luke.lee@swirldslabs.com>
Co-authored-by: lukelee-sl <luke.lee@swirldslabs.com>
The idea behind this commit is to preload asynchronously account nodes and storage nodes from the database during the transaction processing to use these nodes during the calculate root hash step.
We've created two caches, one for account nodes and one for storage nodes. The size of these caches is 100k for accounts and 200k for storage. We've tested other values but this configuration is the one that works better.
We also use exporter cache metrics as Prometheus metrics to check cache efficiency.
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
Co-authored-by: Ameziane H <ameziane.hamlat@consensys.net>
* use snapshots for in-memory copies of layered worldstate
* ensure we close snapshot storage used for layered world state copies
* wrap access to CachedSnapshotWorldState in Optional.ofNullable to prevent NPE when trying to close from AbstractTrieLogManager
Signed-off-by: garyschulte <garyschulte@gmail.com>
* aggresively seeking out other places to defend from this
Signed-off-by: Justin Florentine <justin+github@florentine.us>
Co-authored-by: Justin Florentine <justin+github@florentine.us>
Implement "Small" EOF - EIP-3540 (container) and EIP-3670 (validation).
Make code an interface so EOF specific features are compartmentalized,
including an 'invalid' code type representing a code block that didn't
pass validation, CodeV1 for EOF1, and CodeV0 which represents pre-EOF
code. EVMs track a maximum supported EOF version (where 0 is pre-eof)and
code is generated from a CodeFactory taking in context (is it a CREATE
operation and max code size) for the validation.
Includes spec versions for "Shanghai" and transient testnet "Shandong".
"Small" EOF is only activated in Shandong.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
* add isPersistable(), copy worldstate in mainnetblockvalidator if it is not
* only copy the mutable worldstate if it is not persistable in abstract block creator
Signed-off-by: garyschulte <garyschulte@gmail.com>
* supplier workaround for Cached snapshots to defer snapshots until the block is added to the chain
* handle cache update when worldstate is fast-syncing
* add additional coverage for SnapshotTrieLogManager caching
Signed-off-by: garyschulte <garyschulte@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
WorldStateUpdater commit method is one of the most consuming methods during block processing (engine_newPayloadV1 call). This PR will focus on parallelizing some parts of this method to make it faster.
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
Add a flag to MainnetTransactionProcessor to add the miningBeneficiary
to the list of pre-warmed addresses.
Have shandong fork set this flag to true by default.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
* Add Shandong network and fork
Add the transient Shandong fork and network definitions. For Shanghai
fork testing. Besu doesn't sync, but it does connect.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
* Parallelize some steps in BonsaiPersistedWorldState.calculateRootHash method
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
* Add synchronized on storage flat database remove and update method
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
* Add synchronized on storage flat database remove and update method
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
* Fix this error org.rocksdb.RocksDBException: unknown WriteBatch tag
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
* create a specific calculateRootHash for BonsaiInMemoryWorldState class
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
* create a specific calculateRootHash for BonsaiInMemoryWorldState class
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
* Fix nullPointerException on Collections.synchronizedSet
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
* Use parallelStreams instead of CompletableFuture API
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
* Modify CHANGELOG.md
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
* spotless and synchronizedSet initializaton
Signed-off-by: garyschulte <garyschulte@gmail.com>
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
Signed-off-by: garyschulte <garyschulte@gmail.com>
Co-authored-by: garyschulte <garyschulte@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* implementation of Bonsai snapshots based BonsaiWorldStateArchive
includes: try-with-resources and AutoCloseable WorldState in order to release snapshots when we are done with them
Signed-off-by: garyschulte <garyschulte@gmail.com>
Due to subtle differences between Bytes32 and UInt256 the changes failed
burn-in testing and are being reverted.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
* don't add to bad blocks manager on StorageException
* support for MerklePatriciaTrie exceptions
Signed-off-by: Justin Florentine <justin+github@florentine.us>
The current internal storage APIs implement UInt256 rather than Bytes32,
which is an accident of prior EVM design. This migrates all internal
handling of storage addresses and values to Bytes32.
The main performance gain is reduced GC pressure, in that UInt256
stores the data as an array of ints, whereas Bytes32 keeps a sliced or
wrapped array of bytes. Since database APIs are all byte array based
this should remove unneeded conversions when going to/from the DB.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Overview of changes:
* Remove vestigial log tracing. We will never use it in production and
standard json tracing gets us what we need
* Reduce use of lambdas and optionals. Reads great, translates into a 10%
perf hit in a tight loop.
* Unroll operation loop in some cases. Those are (a) ops that haven't
changed in any way since Frontier (b) ops not overridden in downstream
uses and (c) operations that translate into short static executions.
This has the longest tendrils as it is enabled by operations exposing
static methods to do their work.
* Refactoring of the operationTracer. The single, lambda consuming
traceExecution method was a barrier to performance. It has been replaced
with tracePreExecution and tracePostExecution. Look at
DebugOperationTracer to see how traces that need to operate on both
sides of the operation can be handled with this API.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* use optimistictransactiondb for mutable isolated snapshots
* plumbing necessary to have a snapshot specific updater.
* snapshot rolling working
* implement AutoCloseable on BonsaiSnapshotWorldState to ensure we can correctly dispose of snapshots
* add snapshot transaction cloning, change snapshot based worldstate to extend persisted worldstate rather than in-memory worldstate
Signed-off-by: garyschulte <garyschulte@gmail.com>
* don't add to bad blocks manager on StorageException
* add bugfix to changelog
* adds test coverage
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* re-emits correct block added event when reminding
* uprevs version of log bloom cache metadata to force regeneration on startup
Signed-off-by: Justin Florentine <justin+github@florentine.us>
modified flexibleutil so it works with dynamic byte arrays, some logs to remove
smart contracts adapted, changed bytecodes
solidity contracts changed so they use bytes instead of bytes32
modified flexibleutil to support ec keys
flexible privacy tests adapted
Signed-off-by: Miguel Rojo <miguelangel.rojofernandez@mastercard.com>
Signed-off-by: Miguel Angel Rojo <miguelangel.rojofernandez@mastercard.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* Log index is counted per block, not per transaction
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: mark-terry <mark.terry@consensys.net>
* Replace boolean return with BlockImportResult object
Signed-off-by: Gabriel Trintinalia <gabriel.trintinalia@consensys.net>
Co-authored-by: Gabriel Trintinalia <gabriel.trintinalia@consensys.net>
Change the "minimum" gas in the TangerineWhistleGasCalculator to use an
unsigned comparison, to ensure a larger signed value is not selected.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Add more tracing logs to transaction pool
* Prevent adding non executable transactions and evict based on age
* Move sender account in TransactionsForSenderInfo
* Implement a size expiration cache for lowest invalid nonce for sender
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Justin Florentine <justin+github@florentine.us>
Co-authored-by: Justin Florentine <justin+github@florentine.us>
Fabio Di Fabio
Zhenyang Shi
Jiri Peinlich
Gabriel-Trintinalia
lukelee-sl
garyschulte
ahamlat
Diego López León
matkt
Sally MacFarlane
Stefan Pingel
Danno Ferrin
Justin Florentine
Pedro Novais
Miguel Angel Rojo
Daniel Lehrner
Gabriel-Trintinalia
Matt Nelson
Justin Traglia