- Combine MutableProtocolSchedule and DefaultTimestampSchedule into UnifiedProtocolSchedule.
- Implement getByBlockHeader taking into account both timestamp-based and blockNumber-based forks
- Unstitch timestampSchedule from TransitionProtocolSchedule
- BftProtocolSchedule extends UnifiedProtocolSchedule (instead of MutableProtocolSchedule)
- In MergeProtocolSchedule, unapply mergeSpecificModifications from Shanghai onwards
- Migrate getByBlockNumber modifications into getByBlockHeader for retesteth NoReward
- Temporarily fix reference tests by artificially increasing shanghaiTime
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* removed separate decoding logic
* run non-mainnet ATs to make sure
* remove goQuorum flag from everywhere
* remove GOQUORUM api group
* remove GoQuorum enclave, privacy params, RPCs and other related config
* removed Goquorum related error codes, rpcMethod codes, privateBloom
* removed v from transaction
* removed private GoQuorum storage provider
---------
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Refactor crypto, datatypes, and plugin-api to reduce plugin APIs in the EVM.
Split crypto into crypto services and crypto algorithms
Reverse the dependency between datatyps and plugin-api.
Remove plugin Hash and Address types (use datatypes)
Move PublicKey and Quantity into datatypes.
Lots of changes to imports and build files, and some fromPlugin calls removed.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@shemnon.com>
Co-authored-by: Justin Florentine <justin+github@florentine.us>
* Cancel older block creation tasks upon receiving a new one
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
* Use only one thread for building blocks so there is no risk of overlapping
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fix tests that fail when debug log is enabled
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
---------
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Add deposits to the Execution Layer block structure: EIP-6110. The scope of this commit is to add Deposit related info into BlockHeader and BlockBody. The rest of the EIP including RPC API and validating Deposit with be included in future PRs.
---------
Signed-off-by: Navie Chan <naviechan@gmail.com>
Signed-off-by: navie <naviechan@gmail.com>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
* Replace getByBlockNumber by getByBlockHeader
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Replace getByBlockNumber by getByBlockHeader
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Replace BlockDataGenerator by BlockHeaderTestFixture
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Replace BlockDataGenerator by BlockHeaderTestFixture
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add final to method param
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Replace BlockDataGEnerator by BlockHeaderTestFixture
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Replace BlockDataGenerator by BlockHeaderTestFixture
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Remove BlockDataGenerator
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
---------
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
It's now only used by the tests following removal of production code paths in 08b822a860
A simple decorator pattern would have been a nicer implementation but I didn't want to expose the protocolSpecs from within the schedule hence why I've used inheritance.
ForksIdsNetworkConfigTest has a particularly hairy construction since it relies on TransitionProtocolSchedule but I've isolated this as an inner class.
I am hopeful that we can refactor ForksIdsNetworkConfigTest out of existence eventually and avoid using streamMilestoneBlocks at the same time.
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Replace usages of streamMilestoneBlocks on production code paths: RunnerBuilder and TransactionPoolFactory.
These usages were also using getBlockByNumber which is ultimately what we're trying to remove.
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Reduce the number of places that expose Log4J classes as a part of the
interfaces for methods and classes. While Log4j remains the default we
still need to be able to function when the Log4J jars are removed from
the classpath.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
This is misleading when debugging because we return VALID in the API and there's not actually an INVALID issue to debug despite what the logs were saying.
Also rename and flip the isValid boolean to be shouldNotProceedToPayloadBuildProcess for clarity.
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
* Add flag to distinguish when bad block is a proposed block
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add flag to distinguish when bad block is a proposed block
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* javadoc
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add shouldRecordBadBlock flag to stop adding proposed bad blocks to the bad block manager
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add overload to validateAndProcessBlock with shouldRecordBadBlock flag
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add unit test to ensure flag is being used to add blocks to bad block manager
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add unit test to ensure we don't add our own proposed bad block to the bad block manager
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add debug log when badBlock is not added
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Undo change done in the first approach
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Improve log
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Improve test names
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Improve javadoc of validateProposedBlock method
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Remove method from interface
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Change validateProposedBlock access modifier to private
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
---------
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Adding the t8n and b11r tool allows Hyperledger besu to produce reference
tests from the https://github.com/ethereum/execution-spec-tests repository.
* Add t8n tool, or transition tool. Calculates the state changes from a set
of transactions against a given parent context
* Add b11r tool, or block-builder tool. Given block parts it can generate a
RLP block.
* Change some tracing CLI flags to work better with retesteth
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
* If a PoS block creation repetition takes less than a configurable duration, then waits before next repetition
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update CHANGELOG
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Add unit test
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update besu/src/main/java/org/hyperledger/besu/cli/options/unstable/MiningOptions.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
---------
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Update and fix forkId tests
Move timestamp forks from getForkBlockNumbers to getForkBlockTimestamps in JsonGenesisConfigOptions - this ultimately gets used to popoulate the ForkIdManager which handles lists of blocks and timestamps the same way so this hasn't changed any actual behaviour, but rather supports the test fixes.
Implement TransitionProtocolSchedule.streamMilestoneBlocks as a concatenation of blockNumbers++blockTimestamps. This may have been a latent bug since it's used to update the node record when a fork transition occurs.
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
* Replace getByBlockNumber by getByBlockHeader where trivial
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Fix tests
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Fix tests
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Rolledback change since tests will need non trivial changes
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Rollback replace
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Change other 2 trivial tests
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* spotless
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Pass ProcessableBlockHeader to the mocks
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Replace getByBlockNumber by getByBlockHeader
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add mock of blockHeader
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Use any() for the tests
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add unit test for headBlockHeader()
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
---------
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* Don't cache protocol spec in block creator
Don't cache the protocol spec in the block creator. With the new
shanghaiTimestamp the correct spec may be a function of the timestamp
not just the block number. So every time we are asked to build a block
re-query the spec.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
- Added missing javadocs so that javadoc doclint passes against JDK 17 (invoke by Besu gradle build).
- Exclude following packages from javadoc lint:
org.hyperledger.besu.privacy.contracts.generated
org.hyperledger.besu.tests.acceptance.*
- Temporarily exclude ethereum and evm submodule for doc lint checks.
- Run the javadoc task using GitHub actions (use Java 17) to report any javadoc errors during the PR builds
- Updating plugin-api build.gradle with new hash as javadoc comments caused it to change
Signed-off-by: Usman Saleem <usman@usmans.info>
There is a performance regression in the RC version 23.1.0-RC1 compared to 22.10.3 in both engine_getPayloadV1 and engine_forkchoiceUpdatedV1 calls. This regression is related to recent changes in the way we get the protocol specifications. We get the whole block by decoding each single transaction to use only the block header.
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
* Calculate block value & optimise block building
* Use block with receipts & keep block validation
* Add test on zero block value
* Use non-static method
* Update copyright
Signed-off-by: Zhenyang Shi <wcgcyx@gmail.com>
Signed-off-by: Zhenyang Shi <wcgcyx@gmail.com>
In order to avoid maintaining two versions of forkchoiceUpdated as much as possible, the validation works for both V1 and V2.
The JSON deserialization allows for withdrawals to be omitted which supports V1 requests, however if shanghai is enabled then we will expect withdrawals in a V1 request.
It would be a bug for a CL to send V1 with a post-shanghai timestamp but null withdrawals.
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
The previous evmBuilder override would mean that Shanghai evm configuration is reverted back to Paris
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Implement CALLF, RETF, and JUMPF operations. This introduces the
notion of "code sections" into the code. For legacy code section zero
consists of all of the code and is the only code section. Also
implements EIP specified code validation to ensure validated EOF code
won't use CALLF or JUMPF to a non-existent section.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Implement shanghaiTime including TimestampSchedule and associated infrastructure code.
TimestampSchedule sits alongside the pre and post ProtocolSchedules in TransitionProtocolSchedule.
Introduces getByTimestamp, wrapped inside getByBlockHeader (to also support getByBlockNumber).
General call pattern followed is that if a given timestamp precedes the first timestamp in the schedule, i.e. a pre-shanghai block, then delegate to the appropriate pre or post merge ProtocolSchedule to get by block instead.
cancunTime and a placeholder cancunDefinition has also been implemented in order to effectively test fork order logic.
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Jiri Peinlich <jiri.peinlich@gmail.com>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Jason Frame <jason.frame@consensys.net>
* Add chain pruner
* Increase minimum blocks to retain
* Skip ancestor check in pruning mode
* Separate class for pruning storage
* Move pruning to separate thread
* Limit total pruning threads
Signed-off-by: wcgcyx <wcgcyx@gmail.com>
Signed-off-by: Zhenyang Shi <wcgcyx@gmail.com>
Co-authored-by: Simon Dudley <simon.l.dudley@hotmail.com>
Co-authored-by: Jason Frame <jason.frame@consensys.net>
* aggresively seeking out other places to defend from this
Signed-off-by: Justin Florentine <justin+github@florentine.us>
Co-authored-by: Justin Florentine <justin+github@florentine.us>
* Remove log statements that are keeping references to objects for too much time
Improvement in terms of reducing java heap used,
since the logs were keeping reference to blocks sent by newPayload,
that causes high memory consumption during initial sync,
and could be one of the causes that prevent to complete snap sync on low spec machines.
Exceptions are also logged by the backward sync, so there is no loss of information.
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Implement "Small" EOF - EIP-3540 (container) and EIP-3670 (validation).
Make code an interface so EOF specific features are compartmentalized,
including an 'invalid' code type representing a code block that didn't
pass validation, CodeV1 for EOF1, and CodeV0 which represents pre-EOF
code. EVMs track a maximum supported EOF version (where 0 is pre-eof)and
code is generated from a CodeFactory taking in context (is it a CREATE
operation and max code size) for the validation.
Includes spec versions for "Shanghai" and transient testnet "Shandong".
"Small" EOF is only activated in Shandong.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Due to subtle differences between Bytes32 and UInt256 the changes failed
burn-in testing and are being reverted.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
* don't add to bad blocks manager on StorageException
* support for MerklePatriciaTrie exceptions
Signed-off-by: Justin Florentine <justin+github@florentine.us>
Simon Dudley
Jason Frame
Sally MacFarlane
Danno Ferrin
Fabio Di Fabio
naviechan
Gabriel Fukushima
Diego López León
Usman Saleem
matkt
Zhenyang Shi
garyschulte
Jiri Peinlich
Danno Ferrin
Justin Florentine
Pedro Novais