Support two in-the-wild models of blockhash for testing. For state-tests
always use the keccak hash of the decimal text of the block number.
For t8n always use the blockHashes table in the env. No change to production
handling of blockhash. Refactored BlockHashLookup to split caching
implementation from API.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
* Replace getByBlockNumber by getByBlockHeader where trivial
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Fix tests
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Fix tests
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Rolledback change since tests will need non trivial changes
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Rollback replace
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Change other 2 trivial tests
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* spotless
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Pass ProcessableBlockHeader to the mocks
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Replace getByBlockNumber by getByBlockHeader
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add mock of blockHeader
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Use any() for the tests
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add unit test for headBlockHeader()
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
---------
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* Don't cache protocol spec in block creator
Don't cache the protocol spec in the block creator. With the new
shanghaiTimestamp the correct spec may be a function of the timestamp
not just the block number. So every time we are asked to build a block
re-query the spec.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
* add withdrawalRoot computation
Signed-off-by: Jason Frame <jason.frame@consensys.net>
* include withdrawalsRoot and withdrawals when creating block
Signed-off-by: Jason Frame <jason.frame@consensys.net>
* validate withdrawals when validation mainnet blocks
Signed-off-by: Jason Frame <jason.frame@consensys.net>
* Fix invalid withdrawals root during backwards sync
Add withdrawalsRoot as part of BodyIdentifier
Add comment to avoid new issues when adding new fields to BlockBody
Signed-off-by: Jason Frame <jason.frame@consensys.net>
* Include withdrawalsRoot when creating new header in EngineNewPayload
Signed-off-by: Jason Frame <jason.frame@consensys.net>
* Use empty trie hash in genesis for withdrawalsRoot if shanghai is enabled at genesis
Signed-off-by: Jason Frame <jason.frame@consensys.net>
* Revert testing changes to AbstractEngineNewPayload
Signed-off-by: Jason Frame <jason.frame@consensys.net>
* Fix BesuController tests failing after GenesisState change
Signed-off-by: Jason Frame <jason.frame@consensys.net>
* Missing withdrawals wiring in BlockProcessor
Add empty withdrawals in GenesisState for shanghai at genesis
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Jason Frame <jason.frame@consensys.net>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
* subscribe snapshot worldstates to parent worldstate storage events like clear and clearFlatDatabase and close as appropriate to avoid segfaults
* fix for direct snapshot creation when using snapshot archive
* ensure we only prune bonsai worldstates if we have more than the configured number of retained states
Signed-off-by: garyschulte <garyschulte@gmail.com>
Implement shanghaiTime including TimestampSchedule and associated infrastructure code.
TimestampSchedule sits alongside the pre and post ProtocolSchedules in TransitionProtocolSchedule.
Introduces getByTimestamp, wrapped inside getByBlockHeader (to also support getByBlockNumber).
General call pattern followed is that if a given timestamp precedes the first timestamp in the schedule, i.e. a pre-shanghai block, then delegate to the appropriate pre or post merge ProtocolSchedule to get by block instead.
cancunTime and a placeholder cancunDefinition has also been implemented in order to effectively test fork order logic.
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Jiri Peinlich <jiri.peinlich@gmail.com>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Jason Frame <jason.frame@consensys.net>
* add isPersistable(), copy worldstate in mainnetblockvalidator if it is not
* only copy the mutable worldstate if it is not persistable in abstract block creator
Signed-off-by: garyschulte <garyschulte@gmail.com>
* implementation of Bonsai snapshots based BonsaiWorldStateArchive
includes: try-with-resources and AutoCloseable WorldState in order to release snapshots when we are done with them
Signed-off-by: garyschulte <garyschulte@gmail.com>
Due to subtle differences between Bytes32 and UInt256 the changes failed
burn-in testing and are being reverted.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
* Replace boolean return with BlockImportResult object
Signed-off-by: Gabriel Trintinalia <gabriel.trintinalia@consensys.net>
Co-authored-by: Gabriel Trintinalia <gabriel.trintinalia@consensys.net>
* make transaction pool limits for sender based on pool size. default by sender is 5
move config into TransactionPoolOptions class
Signed-off-by: garyschulte <garyschulte@gmail.com>
* Add more tracing logs to transaction pool
* Prevent adding non executable transactions and evict based on age
* Move sender account in TransactionsForSenderInfo
* Implement a size expiration cache for lowest invalid nonce for sender
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Justin Florentine <justin+github@florentine.us>
Co-authored-by: Justin Florentine <justin+github@florentine.us>
* Retry mechanism for block creation
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update CHANGELOG
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Always keep 1 thread active in the computation executors
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Surface StorageException when building a block for finer filtering
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Log successful block created at info
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Reformat block proposal logs
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Remove test code
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* evict transactions from tail of senderAndNonce rather than just lowest price
* make pendingTransactions secondary sort based on timestamp rather than sequence id (to better prevent collisions for same gas price and sequence id)
Signed-off-by: garyschulte <garyschulte@gmail.com>
* initial impl of engine-api override:
* remove reliance on isMergeEnabled in pre-merge block header validation rules
* create a merge-specific block header validtion stack rather than leveraging mainnet
* re-add --engine-api-enabled cli param
* make engine api usable without a merge config.
leaves the door open to supporting the engine api with other MiningCoordinators, but disables all but engine_exchangeTransitionConfiguration in the absence of mergeCoordinator
Signed-off-by: garyschulte <garyschulte@gmail.com>
* avoid the edge case of long minimum value which gives a negative value when passed into abs()
* changed some Longs to long where they were statically intialized
* replace Lists.emptyList() with Collections.emptyList()
* removed unnecessary eq() invocation and resulting code smell
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
Change the unit test execution to use the Junit5 JUnitPlatform. This
allows for a mix of junit 4 and junit 5 tests and for a gradual
migration to junit 5 instead of a big bang. One class depended on
junit 4 exceptions and was updated. Two tests depending on
native libraries fail gracefully on mac (and only mac).
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Currently Besu has a limited support for sending NewPooledTransactionHashes messages, and other aspect related to reduce transactions synchronization traffic, described in the Ethereum Wire Protocol version 66.
Specifically:
Besu only uses NewPooledTransactionHashes for new local transactions, while it could be extended to any transaction added to the transaction pool
Besu does not limit the sending of the full transaction messages to a small fraction of the connected peers, and sends the new transaction hashes to all the remaining peers
This PR, extends eth/66 support and does some code refactoring, to remove some reduntant code and rename some classes to identify they are related to the NewPooledTransactionHashes message.
The main changes are:
Do not have a separate tracker for transaction hashes, since for them we can reuse PeerTransactionTracker, that tracks full transactions exchange history and sending queue with a peer. So PeerPendingTransactionTracker has been removed. --tx-pool-hashes-max-size is now deprecated and has no more effect and it will be removed in a future release.
When a new peer connects, if it support eth/6[56] then we send all the transaction hashes we have in the pool, otherwise we send the full transactions.
When new transactions are added to the pool, we send full transactions to peers without eth/6[56] support, or to a small fractions of all peers, and then we send only transaction hashes to the remaining peer that support eth/6[56]. Both transactions and transaction hashes are only sent if not already exchanged with that specific peer.
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Merge: extend block creation and mining to support The Merge
Code originally written by garyschulte on the merge branch.
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Bump SLF4J version
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Replace log4j2 API with SLF4j API
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Replace usage of LogManager#getFormatterLogger
This is for keeping compatibility with SLF4J. If neccesary, a specific formatter can be created for the RlpBlockImporter class
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Unset the default logging value for the retesteth
This is because it's not possible to resolve the root logger level into a Log4J2 field
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Prevent creation of Logger context outside SLF4J
org.hyperledger.besu.cli.BesuCommand#setAllLevels was taken from
https://github.com/apache/logging-log4j2/blob/rel%2F2.17.1/log4j-core/src/main/java/org/apache/logging/log4j/core/config/Configurator.java#L309
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Add FATAL level deprecation message
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* [Sonar] Fix java:S2139
Exceptions should be either logged or rethrown but not both
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* [Sonar] Fix java:S3457
Printf-style format strings should be used correctly
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Add changelog
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Remove duplicate code (TODO) in FlexiblePrivacyPrecompiledContract
Signed-off-by: Romeet Puhar <Romeet27@outlook.com>
* Increasing visibility of gas fee being below the configured minimum message
Signed-off-by: Romeet Puhar <38156169+RP27@users.noreply.github.com>
* Increasing visibility of logging for when the gas fee of a transaction being sent is below the configured minimum
Signed-off-by: RP27 <38156169+RP27@users.noreply.github.com>
* Reverting unintended change
Signed-off-by: RP27 <38156169+RP27@users.noreply.github.com>
Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
* lots of errorprone fixes
* some license updates
* some mockito updates
* upgrade the rocksdb version
* Prometheus left at 0.9.0 as 0.10.0+ introduces OpenMetrics
related changes that break unit tests.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* compile issues sorted, some tests failing
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* closing runnerBehind closes the vertx shared with runnerAhead, which now throws an exception
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* checkpoint when 4/5 websocket login tests pass
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* exp moved to attribute from principal
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* fixed more tests
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* fixed more tests
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* exception handling test improvement
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* exception handling test improvement
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* static renamed
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* We want old implementation of the host()
Newly vertex handles the forward headers and modifies host(). In the
process vert.x loses track of port from Host header in case the port was
not a string.
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
* adding dependency on jackson-databind for tests
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
* making sure changes are spotless
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
* Dealing with regression
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
* fixing last failing vert.x test hopefully
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
* removed commented out code
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* reverts debugging adjustment
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* removed commented out code
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* minor whitespace cleanup
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* (internal) Refactor 'onchain' to 'flexible' where applicable (#3075)
* CLI option name change
Signed-off-by: Frank Li <b439988l@gmail.com>
* refactor privacyparameters.java and add deprecation warning
Signed-off-by: Frank Li <b439988l@gmail.com>
* more refactoring
Signed-off-by: Frank Li <b439988l@gmail.com>
* add to everything.toml
Signed-off-by: Frank Li <b439988l@gmail.com>
* bugs
Signed-off-by: Frank Li <b439988l@gmail.com>
* more missing variable names
Signed-off-by: Frank Li <b439988l@gmail.com>
* more classes
Signed-off-by: Frank Li <b439988l@gmail.com>
* more classes
Signed-off-by: Frank Li <b439988l@gmail.com>
* fix compile error
Signed-off-by: Frank Li <b439988l@gmail.com>
* add new test to invalidate passing both commands
Signed-off-by: Frank Li <b439988l@gmail.com>
* more refactoring + more tests
Signed-off-by: Frank Li <b439988l@gmail.com>
* new batch
Signed-off-by: Frank Li <b439988l@gmail.com>
* final batch?
Signed-off-by: Frank Li <b439988l@gmail.com>
* failing unit test
Signed-off-by: Frank Li <b439988l@gmail.com>
* revert incorrect refactoring back to onchain
Signed-off-by: Frank Li <b439988l@gmail.com>
* fix unit test
Signed-off-by: Frank Li <b439988l@gmail.com>
* comment
Signed-off-by: Frank Li <b439988l@gmail.com>
* comment
Signed-off-by: Frank Li <b439988l@gmail.com>
* support both privx methods
Signed-off-by: Frank Li <b439988l@gmail.com>
* add to changelog
Signed-off-by: Frank Li <b439988l@gmail.com>
* address comment
Signed-off-by: Frank Li <b439988l@gmail.com>
* add plugin privacy
Signed-off-by: Frank Li <b439988l@gmail.com>
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* checkpoint when 4/5 websocket login tests pass
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* trying to figure out how to decouple this test
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* spotless
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* removes Orion from integration test
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* spotless
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* Add jackson dependency to merge module
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Migrate JWTAuthOptions creations for public keys
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Check http client response status
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Replace Orian with Tessera in tests
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Change Tessera expected error messages in tests
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Change executor of integrationTests to allow spanning Docker processes
Signed-off-by: Diego López León <dieguitoll@gmail.com>
Co-authored-by: Justin Florentine <justin+github@florentine.us>
Co-authored-by: Jiri Peinlich <jiri.peinlich@gmail.com>
Co-authored-by: Frank Li <39414003+frankisawesome@users.noreply.github.com>
Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
* reduce use of global quorum config
The use of the org.hyperledger.besu.config.GoQuorumOptions static
variable is ugly. In particular does changing its value during tests
cause flaky tests. To ameliorate the issue this commit
- removes unused code that did rely on GoQuorumOptions
- removes access to GoQuorumOptions deep in the call stack by looking
it up in the transaction validator instead
- change the TransactionDecoder to explicitly take the
goQuorumCompatbility flag as an argument where access to the
transaction validator is not easy
- rewriting all tests that did change the GoQuorumOptions by mocking
the option change instead.
Signed-off-by: Taccat Isid <taccatisid@protonmail.com>
* protect access to GoQuorum flag by getter and write once setter
Signed-off-by: Taccat Isid <taccatisid@protonmail.com>
* deduce goQuorum flag from privacyOptions
Signed-off-by: Taccat Isid <taccatisid@protonmail.com>
* remove unnecessary mocks from TransactionPoolTest
Signed-off-by: Taccat Isid <taccatisid@protonmail.com>
Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
Move EVM to a standalone module
Move the EVM classes to a standalone module. This is mostly moves but
some API re-resign to peel out some features not essential to the EVM,
such as privacy support and ties to the data storage subsystem.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Create new datatypes module
Create a new `datatypes` module to hold datatypes that are broadly used.
This will aid modularization by making sure the base types in the module
minimize the amount of unrelated support classes needed.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Add Address, Hash, and Wei to datatypes
Move the Address, Hash, and Wei to datatypes in as they are needed for
EVM modularization.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Fabio Di Fabio
Danno Ferrin
Jason Frame
Gabriel Fukushima
Usman Saleem
garyschulte
Jiri Peinlich
Danno Ferrin
Pedro Novais
Gabriel-Trintinalia
Justin Florentine
Sally MacFarlane
mbaxter
Daniel Lehrner
fab-10
Diego López León
RP27
taccatisid
Justin Florentine
matkt