Currently Besu has a limited support for sending NewPooledTransactionHashes messages, and other aspect related to reduce transactions synchronization traffic, described in the Ethereum Wire Protocol version 66.
Specifically:
Besu only uses NewPooledTransactionHashes for new local transactions, while it could be extended to any transaction added to the transaction pool
Besu does not limit the sending of the full transaction messages to a small fraction of the connected peers, and sends the new transaction hashes to all the remaining peers
This PR, extends eth/66 support and does some code refactoring, to remove some reduntant code and rename some classes to identify they are related to the NewPooledTransactionHashes message.
The main changes are:
Do not have a separate tracker for transaction hashes, since for them we can reuse PeerTransactionTracker, that tracks full transactions exchange history and sending queue with a peer. So PeerPendingTransactionTracker has been removed. --tx-pool-hashes-max-size is now deprecated and has no more effect and it will be removed in a future release.
When a new peer connects, if it support eth/6[56] then we send all the transaction hashes we have in the pool, otherwise we send the full transactions.
When new transactions are added to the pool, we send full transactions to peers without eth/6[56] support, or to a small fractions of all peers, and then we send only transaction hashes to the remaining peer that support eth/6[56]. Both transactions and transaction hashes are only sent if not already exchanged with that specific peer.
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Merge: extend block creation and mining to support The Merge
Code originally written by garyschulte on the merge branch.
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Bump SLF4J version
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Replace log4j2 API with SLF4j API
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Replace usage of LogManager#getFormatterLogger
This is for keeping compatibility with SLF4J. If neccesary, a specific formatter can be created for the RlpBlockImporter class
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Unset the default logging value for the retesteth
This is because it's not possible to resolve the root logger level into a Log4J2 field
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Prevent creation of Logger context outside SLF4J
org.hyperledger.besu.cli.BesuCommand#setAllLevels was taken from
https://github.com/apache/logging-log4j2/blob/rel%2F2.17.1/log4j-core/src/main/java/org/apache/logging/log4j/core/config/Configurator.java#L309
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Add FATAL level deprecation message
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* [Sonar] Fix java:S2139
Exceptions should be either logged or rethrown but not both
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* [Sonar] Fix java:S3457
Printf-style format strings should be used correctly
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Add changelog
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Remove duplicate code (TODO) in FlexiblePrivacyPrecompiledContract
Signed-off-by: Romeet Puhar <Romeet27@outlook.com>
* Increasing visibility of gas fee being below the configured minimum message
Signed-off-by: Romeet Puhar <38156169+RP27@users.noreply.github.com>
* Increasing visibility of logging for when the gas fee of a transaction being sent is below the configured minimum
Signed-off-by: RP27 <38156169+RP27@users.noreply.github.com>
* Reverting unintended change
Signed-off-by: RP27 <38156169+RP27@users.noreply.github.com>
Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
* lots of errorprone fixes
* some license updates
* some mockito updates
* upgrade the rocksdb version
* Prometheus left at 0.9.0 as 0.10.0+ introduces OpenMetrics
related changes that break unit tests.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* compile issues sorted, some tests failing
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* closing runnerBehind closes the vertx shared with runnerAhead, which now throws an exception
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* checkpoint when 4/5 websocket login tests pass
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* exp moved to attribute from principal
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* fixed more tests
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* fixed more tests
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* exception handling test improvement
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* exception handling test improvement
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* static renamed
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* We want old implementation of the host()
Newly vertex handles the forward headers and modifies host(). In the
process vert.x loses track of port from Host header in case the port was
not a string.
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
* adding dependency on jackson-databind for tests
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
* making sure changes are spotless
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
* Dealing with regression
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
* fixing last failing vert.x test hopefully
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
* removed commented out code
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* reverts debugging adjustment
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* removed commented out code
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* minor whitespace cleanup
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* (internal) Refactor 'onchain' to 'flexible' where applicable (#3075)
* CLI option name change
Signed-off-by: Frank Li <b439988l@gmail.com>
* refactor privacyparameters.java and add deprecation warning
Signed-off-by: Frank Li <b439988l@gmail.com>
* more refactoring
Signed-off-by: Frank Li <b439988l@gmail.com>
* add to everything.toml
Signed-off-by: Frank Li <b439988l@gmail.com>
* bugs
Signed-off-by: Frank Li <b439988l@gmail.com>
* more missing variable names
Signed-off-by: Frank Li <b439988l@gmail.com>
* more classes
Signed-off-by: Frank Li <b439988l@gmail.com>
* more classes
Signed-off-by: Frank Li <b439988l@gmail.com>
* fix compile error
Signed-off-by: Frank Li <b439988l@gmail.com>
* add new test to invalidate passing both commands
Signed-off-by: Frank Li <b439988l@gmail.com>
* more refactoring + more tests
Signed-off-by: Frank Li <b439988l@gmail.com>
* new batch
Signed-off-by: Frank Li <b439988l@gmail.com>
* final batch?
Signed-off-by: Frank Li <b439988l@gmail.com>
* failing unit test
Signed-off-by: Frank Li <b439988l@gmail.com>
* revert incorrect refactoring back to onchain
Signed-off-by: Frank Li <b439988l@gmail.com>
* fix unit test
Signed-off-by: Frank Li <b439988l@gmail.com>
* comment
Signed-off-by: Frank Li <b439988l@gmail.com>
* comment
Signed-off-by: Frank Li <b439988l@gmail.com>
* support both privx methods
Signed-off-by: Frank Li <b439988l@gmail.com>
* add to changelog
Signed-off-by: Frank Li <b439988l@gmail.com>
* address comment
Signed-off-by: Frank Li <b439988l@gmail.com>
* add plugin privacy
Signed-off-by: Frank Li <b439988l@gmail.com>
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* checkpoint when 4/5 websocket login tests pass
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* trying to figure out how to decouple this test
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* spotless
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* removes Orion from integration test
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* spotless
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* Add jackson dependency to merge module
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Migrate JWTAuthOptions creations for public keys
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Check http client response status
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Replace Orian with Tessera in tests
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Change Tessera expected error messages in tests
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Change executor of integrationTests to allow spanning Docker processes
Signed-off-by: Diego López León <dieguitoll@gmail.com>
Co-authored-by: Justin Florentine <justin+github@florentine.us>
Co-authored-by: Jiri Peinlich <jiri.peinlich@gmail.com>
Co-authored-by: Frank Li <39414003+frankisawesome@users.noreply.github.com>
Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
* reduce use of global quorum config
The use of the org.hyperledger.besu.config.GoQuorumOptions static
variable is ugly. In particular does changing its value during tests
cause flaky tests. To ameliorate the issue this commit
- removes unused code that did rely on GoQuorumOptions
- removes access to GoQuorumOptions deep in the call stack by looking
it up in the transaction validator instead
- change the TransactionDecoder to explicitly take the
goQuorumCompatbility flag as an argument where access to the
transaction validator is not easy
- rewriting all tests that did change the GoQuorumOptions by mocking
the option change instead.
Signed-off-by: Taccat Isid <taccatisid@protonmail.com>
* protect access to GoQuorum flag by getter and write once setter
Signed-off-by: Taccat Isid <taccatisid@protonmail.com>
* deduce goQuorum flag from privacyOptions
Signed-off-by: Taccat Isid <taccatisid@protonmail.com>
* remove unnecessary mocks from TransactionPoolTest
Signed-off-by: Taccat Isid <taccatisid@protonmail.com>
Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
Move EVM to a standalone module
Move the EVM classes to a standalone module. This is mostly moves but
some API re-resign to peel out some features not essential to the EVM,
such as privacy support and ties to the data storage subsystem.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Create new datatypes module
Create a new `datatypes` module to hold datatypes that are broadly used.
This will aid modularization by making sure the base types in the module
minimize the amount of unrelated support classes needed.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Add Address, Hash, and Wei to datatypes
Move the Address, Hash, and Wei to datatypes in as they are needed for
EVM modularization.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Add unstable CLI option for max ommers depth
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Move to a builder pattern for mining parameters
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add support for mining ommers, up to 8 blocks behind the head job
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add a CLI option to set PoW jobs TTL
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Upgrade to Apache Tuweni 2.0
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Remove intermediate repository
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Remove all occurrences of toBytes
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Migrate to tuweni-bytes
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* add changelog
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* correct reference tests
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Initial API changes
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* more changes
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Change APIs for VM ops
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Use constant UInt256.ONE
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Optimize a bit address <> word transformation
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* spotless
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
Since there are no separate gas budgets for eip-1559 and non-eip-1559 txs,
we can return to the simpler method of just comparing against the gas limit.
This is a non-functional change.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
We were creating new in-memory storage segments each time we were
supposed to be retrieving it which prevented me from being able to test
what keys ended up being stored in that segment.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* Decouple PoW from ethash
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Address code review comments, create a dev network for ecip1049, prepare for keccak hasher
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add PoW function and a few simple tests as test vectors
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Make the PoWHasher hash function a bit easier to understand
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* simplify and call out the code of the keccak hash function
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* support fixed difficulty for keccak mining
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Fix the dev network config
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add comment to KeccakHasher
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Increase fixed difficulty for the ecip1049 dev network to produce hashes a bit less often
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* spotless
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* fix test expectations
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Fix javadoc issue
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* add acceptance test using keccak mining
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* add changelog entry
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Address code review comments
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
This is the first step in supporting switchable consensus mechanisms.
Specifically this allows additional protocol specs to be inserted to the protocol schedule at milestones other than that explicitly specified in the genesis config.
Signed-off-by: Trent Mohay <trent.mohay@consensys.net>
* hand craft receipt for goquorum private tx when creating a block
* validate goquorum private transaction and handle nonce issues
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
* set gas used to zero for goQuorum private tx
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
Co-authored-by: mark-terry <36909937+mark-terry@users.noreply.github.com>
Bonsai tries will require storing state by block hash rather than by
state root. To accommodate both forest mode and bonsai mode all state
queries will pass in both the block hash and state root. This also
permits parallel forest/bonsai modes for private state with bonsai
public state.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Since the EIP-1559 transition is going away, simply use a set of
accepted transactions for transaction validator that we'll check
against.
Don't assume that there are two transaction types and instead check
what type the transactions are.
Use guessType when we're deserializing from json.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
This implements the encoding/decoding logic and because it's backwards
compatible we can introduce it immediately. You can see that some of
the typed-transaction specific encoding/decoding logic is tested where
there are EIP-1559 encoding/decoding tests but there'll be more tests
included in the EIP-2930 PR.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* expose epochCalculator to MiningCoordinator
Signed-off-by: Edward Mack <ed@edwardmack.com>
* replace epochCalculator Function with interface
Signed-off-by: Edward Mack <ed@edwardmack.com>
* apply spotless, clean-up tests
Signed-off-by: Edward Mack <ed@edwardmack.com>
* cleanup comments, fix more tests
Signed-off-by: Edward Mack <ed@edwardmack.com>
* fixing failures from testing
Signed-off-by: Edward Mack <ed@edwardmack.com>
* modify dagSeed calculator to use epochCalculator
epochStartBlock to determine epoch start block based on current epoch
length.
Signed-off-by: Edward Mack <ed@edwardmack.com>
* add epochCalculator to StratumConnectionTest
Signed-off-by: Edward Mack <ed@edwardmack.com>
* apply spotless
Signed-off-by: Edward Mack <ed@edwardmack.com>
* remove unnecessary comments
Signed-off-by: Edward Mack <ed@edwardmack.com>
* remove activation block from Ecip1099EpochCalculator
Signed-off-by: Edward Mack <ed@edwardmack.com>
* add messages to IllegalArgumentExceptions, init epochCalculator
Signed-off-by: Edward Mack <ed@edwardmack.com>
* un-wrap private dagSeed method
Signed-off-by: Edward Mack <ed@edwardmack.com>
* fix typo
Signed-off-by: Edward Mack <ed@edwardmack.com>
* add unit tests for epochStartBlock functions
Signed-off-by: Edward Mack <ed@edwardmack.com>
* remove unused field
Signed-off-by: Edward Mack <ed@edwardmack.com>
For every block produced and imported directly log the same information
we would get from other import paths.
* If this node produced the block, note that with "Produced" instead of
"Imported"
* Log the size of the pending transaction pool
* Only log IBFT rounds at INFO if they go past round 0, DEBUG otherwise.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Moves TransactionInvalidReason to top level so it can be accessed by
privacy code while also allowing the TransactionValidator to be
collapsed as it's only used on the permissionless side.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* #1066 Switched to use unprefixed hex strings for memory and stack values
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Disable flaky tests per Ben Burns(Yeti) request
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Revert last commit and enable ignored tests.
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1157 - updated to create 2 agents so that proper bonding can occur
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1162 - Updated test to mock the local peer PING packet creation so that the hash can be managed.
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Added admin_logsRepairCache end point
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Added admin_logsRepairCache end point
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Remove p2p network code per PR comments
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Updates from PR comments
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Spotless Apply fixes
Signed-off-by: David Mechler <david.mechler@consensys.net>
* PR updates
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Admin force cache refresh when called through end point per PR comments
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Pr updates
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Update changelog for 1.5.1
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Remove check for 0x prefix on addresses to match expectations
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Update graphql pending to allow for sorting of transactions
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1408 Add Miner data endpoints
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1408 Add Miner data endpoints
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1408 Add Miner data endpoints
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1408 Added tests for new miner endpoints
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1408 - PR updates
Signed-off-by: David Mechler <david.mechler@consensys.net>
* SpotlessApply updtes
Signed-off-by: David Mechler <david.mechler@consensys.net>
* SpotlessApply updtes
Signed-off-by: David Mechler <david.mechler@consensys.net>
Co-authored-by: David Mechler <davemec@users.noreply.github.com>
Created a new JSON-RPC method miner_changeTargetGasLimit allowing users
to change the targetGasLimit option while besu is running.
Co-authored-by: Terrence Cooke <terrence.s.cooke@gmail.com>
Co-authored-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
Fabio Di Fabio
mbaxter
Daniel Lehrner
fab-10
Diego López León
RP27
Danno Ferrin
taccatisid
Justin Florentine
matkt
garyschulte
Antoine Toulme
Ratan (Rai) Sur
Trent Mohay
Daniel Lehrner
Sally MacFarlane
Edward Mack
David Mechler
mark-terry
Antoine Toulme
terrencecooke