Two updates to Bonsai Tries
* Log Rolling is implemented on top of the existing Persisted head. When
Besu is at chain head and the new best head makes the current head an
orphan branch, the Bonsai TrieLogs are used to roll back to a common
block and roll forward to the needed base block. Goerli is known to
maintain sync. There are still some issues with frontier era block
receipts.
* Non-mutable reads can be done off of the persisted block. These are
accurate for all reads that were performed in the block. If a read is
not known it proceeds through a fallback series of calls to prior
layers until it hits the persisted block. These layered reads are
driven off of the TrieLogs.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Two refactorings to make the next Bonsai PR less noisy.
Prefer WorldState when we used to ask for MutableWorldState.
Persist takes an optional block header instead of a block hash.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Bonsai tries will require storing state by block hash rather than by
state root. To accommodate both forest mode and bonsai mode all state
queries will pass in both the block hash and state root. This also
permits parallel forest/bonsai modes for private state with bonsai
public state.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Aspects of the consensus mechanism associated with block creation, and validation have been moved
from the IBFT package, into into consensus/common, such that they can be reused for the QBFT
implementation.
Signed-off-by: Trent Mohay <trent.mohay@consensys.net>
The tests were brittle when I started to change BlockDataGenerator to
randomly generate transactions of different types. This makes sure that
we're always under the limit so we can test it more resiliently.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
Remove all but 4 log4j2.xml config files
* The main config for the besu CLI app
* The config for the evmTool CLI app
* The config for acceptance tests
* A config in testUtil
If any tests depend on a log4j file they should import testutil.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Add tracing support for internals and JSON-RPC
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Remove rocksdb tracing as it slows down execution too much
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add B3 headers extraction on JSON-RPC requests
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Remove traces around trie tree as they slow down syncing significantly
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add tracing to fast sync pipeline
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add tracing for all pipelines
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Address code review
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add acceptance tests and break out the shaded dependency
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Fix tracer id
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Revert changes to trie
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Upgrade otel to latest, remove old tracing
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Code review comments
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* #1695 - Added support for EIP-868
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Fix broken test
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1695 - Added tests for new functionality
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1695 - spotlessApply
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Update changelog
Signed-off-by: David Mechler <david.mechler@consensys.net>
Fixes two bugs related to tracing APIs
- invalid trace when calling a precompiled contract
- invalid trace when a create fails with a depth > 0
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
Move the block header validation errors logging levels from trace and
debug to info. Also, include a standard prefix "Invalid block header: "
in each of the log lines.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Since the EIP-1559 transition is going away, simply use a set of
accepted transactions for transaction validator that we'll check
against.
Don't assume that there are two transaction types and instead check
what type the transactions are.
Use guessType when we're deserializing from json.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
The change from create to DEFAULT was something that I saw was needed
to keep the reference tests performant in the upcoming EIP-2718 changes.
The change from reading the whole mainnet genesis to reading just the
config portion came from OutOfMemoryErrors in the reference tests when
running normalizeKeys.
Also moved them to ProtocolTestFixture since they're only used by tests.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
Adding the Log4j "jul" (java.util.logging) adapter resulted in many
messages like this at startup:
`main INFO Registered Log4j as the java.util.logging.LogManager.`
These come from the Log4j status logger. We can get rid of those by
setting the status attribute on all configurations to a higher logging
level. WARN is the next higher level.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* #1561 - Create/store/update ENR when local node is created
Signed-off-by: David Mechler <david.mechler@consensys.net>
* #1561 - spotlessApply
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Update discovery library to prod version 0.4.1
Signed-off-by: David Mechler <david.mechler@consensys.net>
This is a not-fully-functional prototype of Bonsai Tries.
Bonsai tries is a flat leaf storage, branch-by-location, and diff based reorgs
refactoring of the existing forest based trie storage mechanism aimed at
creating sustainable performance at mainnet loads.
* Since it is experimental a feature flag of --Xdata-storage-format=BONSAI
controls activation.
Some required changes have a long reach:
* To accommodate location based storage many Trie operations accept both a
location and hash value. Each data storage format is keyed off of only
one of the fields, so many tests will pass in null to the other field.
* MutableWorldStateUpdater.persist now takes an argument of a block hash.
If this is a natural progression of blocks the hash of the new block is
passed in. Otherwise null should be passed in.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
This implements the encoding/decoding logic and because it's backwards
compatible we can introduce it immediately. You can see that some of
the typed-transaction specific encoding/decoding logic is tested where
there are EIP-1559 encoding/decoding tests but there'll be more tests
included in the EIP-2930 PR.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* modifying transaction RLP encode/decode to handle GoQuorum private tx value
* added static flag to be read by TransactionRLPDecoder so that we can still allow chainId=1
* ensure chainId and v not both provided
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
This PR adds a waiting list for NewPooledTransactionHashesMessage in order to group several hashes into a single GetPooledTransactionsFromPeerTask
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
Co-authored-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
With future storage needs in mind change `createPruningStorage` to a
more generic `getStorageBySegmentIdentifier` method where the segment
identifier is the argument. For now that's just pruning storage as
world state needs a specific return type.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Fixed issue where get methods were no longer public
Signed-off-by: David Mechler <david.mechler@consensys.net>
* Add tests for JSON
Signed-off-by: David Mechler <david.mechler@consensys.net>
* expose epochCalculator to MiningCoordinator
Signed-off-by: Edward Mack <ed@edwardmack.com>
* replace epochCalculator Function with interface
Signed-off-by: Edward Mack <ed@edwardmack.com>
* apply spotless, clean-up tests
Signed-off-by: Edward Mack <ed@edwardmack.com>
* cleanup comments, fix more tests
Signed-off-by: Edward Mack <ed@edwardmack.com>
* fixing failures from testing
Signed-off-by: Edward Mack <ed@edwardmack.com>
* modify dagSeed calculator to use epochCalculator
epochStartBlock to determine epoch start block based on current epoch
length.
Signed-off-by: Edward Mack <ed@edwardmack.com>
* add epochCalculator to StratumConnectionTest
Signed-off-by: Edward Mack <ed@edwardmack.com>
* apply spotless
Signed-off-by: Edward Mack <ed@edwardmack.com>
* remove unnecessary comments
Signed-off-by: Edward Mack <ed@edwardmack.com>
* remove activation block from Ecip1099EpochCalculator
Signed-off-by: Edward Mack <ed@edwardmack.com>
* add messages to IllegalArgumentExceptions, init epochCalculator
Signed-off-by: Edward Mack <ed@edwardmack.com>
* un-wrap private dagSeed method
Signed-off-by: Edward Mack <ed@edwardmack.com>
* fix typo
Signed-off-by: Edward Mack <ed@edwardmack.com>
* add unit tests for epochStartBlock functions
Signed-off-by: Edward Mack <ed@edwardmack.com>
* remove unused field
Signed-off-by: Edward Mack <ed@edwardmack.com>
* Add DNS discovery for goerli, rinkeby and mainnet to help speed up discovery of peers
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add final to constant fields
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
Co-authored-by: David Mechler <david.mechler@consensys.net>
Add the protocols section to admin_peers. This involved plumbing
EthPeers through where we were previously using the P2PNetwork to get
our data. Hence most of the PR is DI refactoring.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Update eth_gasPrice to return the median gas price of transactions in
the last several blocks (or whole chain if the chain is short). If there
are no TXes return the old value. Block window and percentile are CLI
configurable.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Danno Ferrin
David Mechler
Trent Mohay
Ratan (Rai) Sur
Antoine Toulme
Vladyslav Lupashevskyi
Stevan Lohja
Stefan Pingel
matkt
Sally MacFarlane
Abdelhamid Bakhta
mark-terry
Lucas Saldanha
Edward Mack