This is a not-fully-functional prototype of Bonsai Tries.
Bonsai tries is a flat leaf storage, branch-by-location, and diff based reorgs
refactoring of the existing forest based trie storage mechanism aimed at
creating sustainable performance at mainnet loads.
* Since it is experimental a feature flag of --Xdata-storage-format=BONSAI
controls activation.
Some required changes have a long reach:
* To accommodate location based storage many Trie operations accept both a
location and hash value. Each data storage format is keyed off of only
one of the fields, so many tests will pass in null to the other field.
* MutableWorldStateUpdater.persist now takes an argument of a block hash.
If this is a natural progression of blocks the hash of the new block is
passed in. Otherwise null should be passed in.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
This implements the encoding/decoding logic and because it's backwards
compatible we can introduce it immediately. You can see that some of
the typed-transaction specific encoding/decoding logic is tested where
there are EIP-1559 encoding/decoding tests but there'll be more tests
included in the EIP-2930 PR.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* modifying transaction RLP encode/decode to handle GoQuorum private tx value
* added static flag to be read by TransactionRLPDecoder so that we can still allow chainId=1
* ensure chainId and v not both provided
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
With future storage needs in mind change `createPruningStorage` to a
more generic `getStorageBySegmentIdentifier` method where the segment
identifier is the argument. For now that's just pruning storage as
world state needs a specific return type.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* expose epochCalculator to MiningCoordinator
Signed-off-by: Edward Mack <ed@edwardmack.com>
* replace epochCalculator Function with interface
Signed-off-by: Edward Mack <ed@edwardmack.com>
* apply spotless, clean-up tests
Signed-off-by: Edward Mack <ed@edwardmack.com>
* cleanup comments, fix more tests
Signed-off-by: Edward Mack <ed@edwardmack.com>
* fixing failures from testing
Signed-off-by: Edward Mack <ed@edwardmack.com>
* modify dagSeed calculator to use epochCalculator
epochStartBlock to determine epoch start block based on current epoch
length.
Signed-off-by: Edward Mack <ed@edwardmack.com>
* add epochCalculator to StratumConnectionTest
Signed-off-by: Edward Mack <ed@edwardmack.com>
* apply spotless
Signed-off-by: Edward Mack <ed@edwardmack.com>
* remove unnecessary comments
Signed-off-by: Edward Mack <ed@edwardmack.com>
* remove activation block from Ecip1099EpochCalculator
Signed-off-by: Edward Mack <ed@edwardmack.com>
* add messages to IllegalArgumentExceptions, init epochCalculator
Signed-off-by: Edward Mack <ed@edwardmack.com>
* un-wrap private dagSeed method
Signed-off-by: Edward Mack <ed@edwardmack.com>
* fix typo
Signed-off-by: Edward Mack <ed@edwardmack.com>
* add unit tests for epochStartBlock functions
Signed-off-by: Edward Mack <ed@edwardmack.com>
* remove unused field
Signed-off-by: Edward Mack <ed@edwardmack.com>
* Add OTLP metrics support
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Don't automatically push to grpc. Only allow if pushEnabled is set to true
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Code review
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Missed refactoring
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add missing header
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Don't make otel depend on the push enabled flag
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Expose JUL logs to log4j2
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Code review fixes and make sure not to start push gateway if not set to prometheus
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* spotless
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Fix unit test
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* code review feedback
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add changelog entry
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
Moves TransactionInvalidReason to top level so it can be accessed by
privacy code while also allowing the TransactionValidator to be
collapsed as it's only used on the permissionless side.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* #1066 Switched to use unprefixed hex strings for memory and stack values
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Disable flaky tests per Ben Burns(Yeti) request
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Revert last commit and enable ignored tests.
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1157 - updated to create 2 agents so that proper bonding can occur
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1162 - Updated test to mock the local peer PING packet creation so that the hash can be managed.
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Added admin_logsRepairCache end point
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Added admin_logsRepairCache end point
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Remove p2p network code per PR comments
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Updates from PR comments
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Spotless Apply fixes
Signed-off-by: David Mechler <david.mechler@consensys.net>
* PR updates
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Admin force cache refresh when called through end point per PR comments
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Pr updates
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Update changelog for 1.5.1
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Remove check for 0x prefix on addresses to match expectations
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Update graphql pending to allow for sorting of transactions
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1408 Add Miner data endpoints
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1408 Add Miner data endpoints
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1408 Add Miner data endpoints
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1408 Added tests for new miner endpoints
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1086 Initial changes for EIP-2565
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1408 - PR updates
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1086 Initial changes for EIP-2565
Signed-off-by: David Mechler <david.mechler@consensys.net>
* SpotlessApply updtes
Signed-off-by: David Mechler <david.mechler@consensys.net>
* SpotlessApply updtes
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1086 Initial changes for EIP-2565
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1086 Initial changes for EIP-2565
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1086 Initial changes for EIP-2565
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1086 Initial changes for EIP-2565
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1086 - Added tests for precompiled contract and for byzantium and berlin gas values
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1086 Initial changes for EIP-2565
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1086 Initial changes for EIP-2565
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1086 - Added tests for precompiled contract and for byzantium and berlin gas values
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1086 Initial changes for EIP-2565
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1086 Initial changes for EIP-2565
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1086 - Added tests for precompiled contract and for byzantium and berlin gas values
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1086 - PR updates
Signed-off-by: David Mechler <david.mechler@consensys.net>
Co-authored-by: David Mechler <davemec@users.noreply.github.com>
* #1066 Switched to use unprefixed hex strings for memory and stack values
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Disable flaky tests per Ben Burns(Yeti) request
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Revert last commit and enable ignored tests.
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1157 - updated to create 2 agents so that proper bonding can occur
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1162 - Updated test to mock the local peer PING packet creation so that the hash can be managed.
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Added admin_logsRepairCache end point
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Added admin_logsRepairCache end point
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Remove p2p network code per PR comments
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Updates from PR comments
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Spotless Apply fixes
Signed-off-by: David Mechler <david.mechler@consensys.net>
* PR updates
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Admin force cache refresh when called through end point per PR comments
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Pr updates
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Update changelog for 1.5.1
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Remove check for 0x prefix on addresses to match expectations
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Update graphql pending to allow for sorting of transactions
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1408 Add Miner data endpoints
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1408 Add Miner data endpoints
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1408 Add Miner data endpoints
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1408 Added tests for new miner endpoints
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1408 - PR updates
Signed-off-by: David Mechler <david.mechler@consensys.net>
* SpotlessApply updtes
Signed-off-by: David Mechler <david.mechler@consensys.net>
* SpotlessApply updtes
Signed-off-by: David Mechler <david.mechler@consensys.net>
Co-authored-by: David Mechler <davemec@users.noreply.github.com>
Make `TargetingGasLimitCalculator` respect the rule that a gas _delta_ can be no more than floor(previous_gas/1024)
Signed-off-by: Atkins <atkinschang@gmail.com>
Update the besu-native libraries to 0.3.0 to gain access to faster BN128
and secp256k1 implementations.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
The parallel task production is by sub-trie, so calling `visitAll` on a
root node will eventually spawn up to 16 tasks (for a hexary trie).
If we marked each sub-trie in its own thread, with no common queue of
tasks, our mark speed would be limited by the sub-trie with the maximum
number of nodes. In practice for the Ethereum mainnet, we see a large
imbalance in sub-trie size so without a common task pool the time in
which there is only 1 thread left marking its big sub-trie would be
substantial.
If we were to leave all threads to produce mark tasks before starting
to mark, we would run out of memory quickly.
If we were to have a constant number of threads producing the mark
tasks with the others consuming them, we would have to optimize the
production/consumption balance.
To get the best of both worlds, the marking executor has a
ThreadPoolExecutor.CallerRunsPolicy which causes the producing tasks to
essentially consume their own mark task immediately when the task queue
is full. The resulting behavior is threads that mark their own sub-trie
until they finish that sub-trie, at which point they switch to marking
the sub-trie tasks produced by another thread.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
The warm address/slot data was not returned to the calling messageFrame
for CREATE[2] calls as it was for CALL series calls.
Also, improve performance by using more efficient copy constructors and
reusing objects were relevant.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
If a peer exceeds the authorized number of pending blocks, Besu will replace the lowest priority block in the cache from this peer by this new one until the local node sync a new block and maybe purges one of the blocks of this peer.
The highest priority blocks are those that are lowest in block height and then higher priority if they were sent more recently.
Other peers will not be impacted and will be able to continue sending pending blocks.
The cache size limit is the distance between the minimum and maximum value of the BlockPropagationRange parameter. Besu automatically purges blocks outside this range.
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
* Add genesis config parameter ecip1099Block
Signed-off-by: Edward Mack <ed@edwardmack.com>
* add stub for epoch activation config
Signed-off-by: Edward Mack <ed@edwardmack.com>
* add stubs for EtcBesuControllerBuilder
that will handle ECIP 1099 Colibrate Epoch Duration functionality
Signed-off-by: Edward Mack <ed@edwardmack.com>
* introduce EtcHashMinerExecutor
Signed-off-by: Edward Mack <ed@edwardmack.com>
* implement EtcHashMinerExecutor
Signed-off-by: Edward Mack <ed@edwardmack.com>
* remove ecip1099 genesis config option
Signed-off-by: Edward Mack <ed@edwardmack.com>
* apply spotless to code
Signed-off-by: Edward Mack <ed@edwardmack.com>
* add test for Etc Hash epoch calculation
Signed-off-by: Edward Mack <ed@edwardmack.com>
* cleanup comments, apply spotless
Signed-off-by: Edward Mack <ed@edwardmack.com>
* make needed variables final
Signed-off-by: Edward Mack <ed@edwardmack.com>
* update changelog
Signed-off-by: Edward Mack <ed@edwardmack.com>
* refactor code to add epochCalculator to EthHash
As suggested in comment removed EtcHash series of classes to keep
existing controller.
Signed-off-by: Edward Mack <ed@edwardmack.com>
* use gas limit calculator
Signed-off-by: Edward Mack <ed@edwardmack.com>
* fix imports
Signed-off-by: Edward Mack <ed@edwardmack.com>
* run spotless apply
Signed-off-by: Edward Mack <ed@edwardmack.com>
* fix imports
Signed-off-by: Edward Mack <ed@edwardmack.com>
* fix comments
Signed-off-by: Edward Mack <ed@edwardmack.com>
* add ecip1099Block option to asMap method
Signed-off-by: Edward Mack <ed@edwardmack.com>
* Prefer `EvmAccount` in interfaces and methods
* Rename `DefaultEvmAccount` to `WrappedEvmAccount`
* Move `UpdateTrackingAccount` to a top level class from an inner class
* Re-type `getTouchedAcounts()` to not use `UpdateTrackingAccount`
* Extract `WorldStateArchive` interface, rename old class
`DefaultWorldStateArchive`
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Add better invalid reason logging.
The EVMTool state-test will add a field in the json result object.
Block transaction processing will log at INFO why the block is invalid
and include the block hash and (if relevant) transaction hash.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* store warmed addresses and slots i the MessageFrame
* update gas calculations for berlin specific rules
* allow state tests in EVMTool to override the forks.
* retesteth should't broadcast re-orgs
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Add some Quality of Life changes to reference tests
* ShanghaiLove tests tend to OOM in parallel execution. Exclude them.
* LegacyTests, as generated code, are not subject to SPDX header checks.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* handle case where onchain groups do not exist
* added constructor so the tests can mock the onchain management contract
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
* Fix#1110: Privacy: Restart caused by insufficient memory can cause inconsistent private state
Signed-off-by: Stefan Pingel <stefan.pingel@consensys.net>
mark-terry
matkt
Danno Ferrin
Lucas Saldanha
Ratan (Rai) Sur
Sally MacFarlane
Edward Mack
David Mechler
Antoine Toulme
Atkins
Abdelhamid Bakhta
Stefan Pingel