tracing endpoints keep open a worldstate for the duration of tracing and
response generation.
---------
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
Signed-off-by: garyschulte <garyschulte@gmail.com>
Co-authored-by: Karim TAAM <karim.t2am@gmail.com>
<!-- Thanks for sending a pull request! Please check out our
contribution guidelines: -->
<!-- https://github.com/hyperledger/besu/blob/main/CONTRIBUTING.md -->
## PR description
There is an issue when restarting Besu when a backward sync session is
running, since after the restart it is possible that the Consensus
client sends a FcU or a NewPayload for a block that is present in the
backward sync storage, but not yet imported, so not on the main chain,
but still the backward sync thinks it should not do anything with that
block, so it returns like it has completed the sync, but since the sync
is not done actually then the internal error that the finalize block is
not present.
The solution is to persist the backward sync status, so in case of a
restart, it can resume from where it was interrupted.
## Fixed Issue(s)
<!-- Please link to fixed issue(s) here using format: fixes #<issue
number> -->
<!-- Example: "fixes #2" -->
fixes#5053
## Documentation
- [x] I thought about documentation and added the `doc-change-required`
label to this PR if
[updates are
required](https://wiki.hyperledger.org/display/BESU/Documentation).
## Acceptance Tests (Non Mainnet)
- [x] I have considered running `./gradlew acceptanceTestNonMainnet`
locally if my PR affects non-mainnet modules.
## Changelog
- [x] I thought about the changelog and included a [changelog update if
required](https://wiki.hyperledger.org/display/BESU/Changelog).
---------
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
When receiving transactions from a file parse them as an RLP list.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Co-authored-by: Antoine Toulme <antoine@toulme.name>
Reduce the number of places that expose Log4J classes as a part of the
interfaces for methods and classes. While Log4j remains the default we
still need to be able to function when the Log4J jars are removed from
the classpath.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Pull up and simplify data passed to CheckpointSource, which then allows checkpointBlockHeader to be used by protocolSchedule.getByBlockHeader.
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
* Fix empty body concept after shanghai
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Check protocolSchedule to create empty block and add withdrawals to empty block check
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add unit tests
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Extract block creation into a private method
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Nit changes
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Refactor test names
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
---------
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Co-authored-by: garyschulte <garyschulte@gmail.com>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Added Ethereum execution spec test to be generated and executed by reference tests gradle task. Add `https://github.com/ethereum/execution-spec-tests/releases` as custom ivy repository to easily download and cache the test specs. Updated the reference test gradle file to generate the test Java classes.
Signed-off-by: Usman Saleem <usman@usmans.info>
Support two in-the-wild models of blockhash for testing. For state-tests
always use the keccak hash of the decimal text of the block number.
For t8n always use the blockHashes table in the env. No change to production
handling of blockhash. Refactored BlockHashLookup to split caching
implementation from API.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
In one case reading config files used parseLong instead of decode.
decode accepts `0x` and decimal, only a single radix for parseLong.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
This is misleading when debugging because we return VALID in the API and there's not actually an INVALID issue to debug despite what the logs were saying.
Also rename and flip the isValid boolean to be shouldNotProceedToPayloadBuildProcess for clarity.
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
* Add flag to distinguish when bad block is a proposed block
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add flag to distinguish when bad block is a proposed block
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* javadoc
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add shouldRecordBadBlock flag to stop adding proposed bad blocks to the bad block manager
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add overload to validateAndProcessBlock with shouldRecordBadBlock flag
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add unit test to ensure flag is being used to add blocks to bad block manager
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add unit test to ensure we don't add our own proposed bad block to the bad block manager
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add debug log when badBlock is not added
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Undo change done in the first approach
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Improve log
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Improve test names
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Improve javadoc of validateProposedBlock method
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Remove method from interface
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Change validateProposedBlock access modifier to private
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
---------
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Fix a formatting error in standard trace that produced non-parsable
JSON. Add unit tests to cover those cases.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Adding the t8n and b11r tool allows Hyperledger besu to produce reference
tests from the https://github.com/ethereum/execution-spec-tests repository.
* Add t8n tool, or transition tool. Calculates the state changes from a set
of transactions against a given parent context
* Add b11r tool, or block-builder tool. Given block parts it can generate a
RLP block.
* Change some tracing CLI flags to work better with retesteth
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
* Add gasUsed calculation
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Fix unit test
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Fix typo
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
---------
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* If a PoS block creation repetition takes less than a configurable duration, then waits before next repetition
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update CHANGELOG
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Add unit test
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update besu/src/main/java/org/hyperledger/besu/cli/options/unstable/MiningOptions.java
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
---------
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Update and fix forkId tests
Move timestamp forks from getForkBlockNumbers to getForkBlockTimestamps in JsonGenesisConfigOptions - this ultimately gets used to popoulate the ForkIdManager which handles lists of blocks and timestamps the same way so this hasn't changed any actual behaviour, but rather supports the test fixes.
Implement TransitionProtocolSchedule.streamMilestoneBlocks as a concatenation of blockNumbers++blockTimestamps. This may have been a latent bug since it's used to update the node record when a fork transition occurs.
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
* Use the copy during prepareTrieLog instead of saveTrieLog
* add final flag on BonsaiWorldStateUpdater
* Use a copy of BonsaiInMemoryWorldState inside prepareTrieLog
* add link to persisted worldstate storage
* fix tests
* Make a copy of the worldstate after committing changes to the trielog
* spotless + remove maybeUnSubscribe in setNextWorldView
* subscribe storage on layered worldstate
* fix null issue
* not close layered worldstate during getAccount
* clean code
* Add changelog entry
---------
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
Co-authored-by: Karim TAAM <karim.t2am@gmail.com>
* Add engine get payload body methods and test
Signed-off-by: Zhenyang Shi <wcgcyx@gmail.com>
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add header
Signed-off-by: Zhenyang Shi <wcgcyx@gmail.com>
* Update result struct & add test
Signed-off-by: Zhenyang Shi <wcgcyx@gmail.com>
* Change constant to use upper case
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add PayloadBody class and withdrawals to response of methods
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add unit tests
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add changelog
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* spotless
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add check to prevent returning trailing null results past the head
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add test to check trailing null post head scenario
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Split tests into pre and post shanghai
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* spotless
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Rename methods
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Use getName() to log method name
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* spotless
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Rename variable
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Call constructor directly
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Fix ByHash json parsing
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
* Fix json parsing again
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
* Add check to prevent unnecessary queries
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Refactor method
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add new error code for EngineGetPayloadBodies methods
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add return error for request above the API limit
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add constructor for empty response
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* add check for number of blocks requested and for requests of post head
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add test to check error code when request exceeds API limits
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* add constant for max blocks allowed per request
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* spotless
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Fix some nits
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add invalid params check
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add tests for invalid params check
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Fix test and spotless
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Revert "Fix json parsing again"
This reverts commit 558d325bf3.
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Revert "Fix ByHash json parsing Signed-off-by: Simon Dudley <simon.dudley@consensys.net>"
This reverts commit 663e11e2
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Use UnsignedLongParameter to cast params of the request
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add optional withdrawals to the NewPayload log (#5021)
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
* kubernetes and errorprone - update versions (#5013)
* update errorprone and kubernetes versions
* fixed errorprone issues in prod cod
* fixed errorprone issues in test code
---------
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
* Rename JsonRpcService to EngineJsonRpcService (#5036)
* rename JsonRpcService to EngineJsonRpcService
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
* Params should be single item of array type, not outer array of strings (#5037)
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
* Add EIP-2537 (BLS12-381 curve precompiles) to Cancun (#5017)
Add the BLS curve precompiles into the registry for cancun. All of the
curve precompiles have been here since berlin, so this is just wiring
them in.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* Use UnsignedLongParameter to cast params of the request
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add optional withdrawals to the NewPayload log (#5021)
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
* kubernetes and errorprone - update versions (#5013)
* update errorprone and kubernetes versions
* fixed errorprone issues in prod cod
* fixed errorprone issues in test code
---------
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
* Rename JsonRpcService to EngineJsonRpcService (#5036)
* rename JsonRpcService to EngineJsonRpcService
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
* Params should be single item of array type, not outer array of strings (#5037)
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
* Add EIP-2537 (BLS12-381 curve precompiles) to Cancun (#5017)
Add the BLS curve precompiles into the registry for cancun. All of the
curve precompiles have been here since berlin, so this is just wiring
them in.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* Convert start and count from hex to match JSON-RPC Spec standard
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
---------
Signed-off-by: Zhenyang Shi <wcgcyx@gmail.com>
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Co-authored-by: Zhenyang Shi <wcgcyx@gmail.com>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Co-authored-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
* Convert blockCount from hex to match JSON-RPC Specification
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Fix tests
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add changelog
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
---------
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Replace getByBlockNumber by getByBlockHeader where trivial
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Fix tests
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Fix tests
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Rolledback change since tests will need non trivial changes
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Rollback replace
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Change other 2 trivial tests
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* spotless
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Pass ProcessableBlockHeader to the mocks
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Replace getByBlockNumber by getByBlockHeader
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add mock of blockHeader
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Use any() for the tests
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add unit test for headBlockHeader()
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
---------
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* bonsai layered worldstate subscription
* unsubscribe from worldstatestorage on close of BonsaiLayeredWorldState
* minor txpool logging improvements
Signed-off-by: garyschulte <garyschulte@gmail.com>
Speedup initcode validation by
- optimizing the jumpdest loop
- not caching initcode in the code cache
(but keep codecache for normal contracts).
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Update following sections:
Gradle build, CircleCI, GitHub Actions, Docker
The devp2p TLS mutual authentication unit tests dealing with PKCS11+nss and QBFT+PKCS11+nss ATs were failing on Java 17 when dealing with RSA keys. The unit tests and AT are updated to use EC keys instead. The keystores, truststores, CRL etc. involved in those ATs are regenerated and detailed instructions are provided on how they are created.
Signed-off-by: Usman Saleem <usman@usmans.info>
* Improve withdrawals processing performance
* Add a changelog entry
* refactoring + remove the changelog entry as we have not change logs for the withdrawals
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
Add the BLS curve precompiles into the registry for cancun. All of the
curve precompiles have been here since berlin, so this is just wiring
them in.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
garyschulte
Fabio Di Fabio
Gabriel Fukushima
Danno Ferrin
Simon Dudley
Jason Frame
Gabriel-Trintinalia
Usman Saleem
Diego López León
matkt
Sally MacFarlane
Stefan Pingel
ahamlat
Justin Florentine
Cody Born