The proposed pull request introduces a feature that allows healing of the flat database by streaming the flat database data and validating it by generating a proof from the trie structure. If the proof is found to be invalid, the code traverses the trie to fix the invalid range. To optimize the process and avoid checking the entire flat database, the PR includes enhancements such as tracking the accounts that need to be repaired during SnapSync. By implementing these optimizations, the PR aims to significantly reduce the time and resources required for repairing the flat database.
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
* update to 2.4.1
* update use of DNS daemon with Vertx
* fix issue with Bytes.repeat
* update antlr version
* fix dns tests
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
---------
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
This API fulfils the basic requirements, but will probably be extended in the near future.
Signed-off-by: Stefan <stefan.pingel@consensys.net>
Signed-off-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Refactor crypto, datatypes, and plugin-api to reduce plugin APIs in the EVM.
Split crypto into crypto services and crypto algorithms
Reverse the dependency between datatyps and plugin-api.
Remove plugin Hash and Address types (use datatypes)
Move PublicKey and Quantity into datatypes.
Lots of changes to imports and build files, and some fromPlugin calls removed.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@shemnon.com>
Co-authored-by: Justin Florentine <justin+github@florentine.us>
Add deposits to the Execution Layer block structure: EIP-6110. The scope of this commit is to add Deposit related info into BlockHeader and BlockBody. The rest of the EIP including RPC API and validating Deposit with be included in future PRs.
---------
Signed-off-by: Navie Chan <naviechan@gmail.com>
Signed-off-by: navie <naviechan@gmail.com>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
- Added missing javadocs so that javadoc doclint passes against JDK 17 (invoke by Besu gradle build).
- Exclude following packages from javadoc lint:
org.hyperledger.besu.privacy.contracts.generated
org.hyperledger.besu.tests.acceptance.*
- Temporarily exclude ethereum and evm submodule for doc lint checks.
- Run the javadoc task using GitHub actions (use Java 17) to report any javadoc errors during the PR builds
- Updating plugin-api build.gradle with new hash as javadoc comments caused it to change
Signed-off-by: Usman Saleem <usman@usmans.info>
* new type and fix for BesuEventsImplTest
* replace iteration over tx type enums with explicit lists, fixes BesuEventsImplTest
* fixes more tests
* fixes BlockchainQueriesTest by changing the ORDER of transaction types, wtf
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* use debug rpc endpoint to resync worldstate
* Reset transaction pool state every time the initial sync is done
Signed-off-by: garyschulte <garyschulte@gmail.com>
Co-authored-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* implementation of Bonsai snapshots based BonsaiWorldStateArchive
includes: try-with-resources and AutoCloseable WorldState in order to release snapshots when we are done with them
Signed-off-by: garyschulte <garyschulte@gmail.com>
Due to subtle differences between Bytes32 and UInt256 the changes failed
burn-in testing and are being reverted.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
The current internal storage APIs implement UInt256 rather than Bytes32,
which is an accident of prior EVM design. This migrates all internal
handling of storage addresses and values to Bytes32.
The main performance gain is reduced GC pressure, in that UInt256
stores the data as an array of ints, whereas Bytes32 keeps a sliced or
wrapped array of bytes. Since database APIs are all byte array based
this should remove unneeded conversions when going to/from the DB.
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
* use optimistictransactiondb for mutable isolated snapshots
* plumbing necessary to have a snapshot specific updater.
* snapshot rolling working
* implement AutoCloseable on BonsaiSnapshotWorldState to ensure we can correctly dispose of snapshots
* add snapshot transaction cloning, change snapshot based worldstate to extend persisted worldstate rather than in-memory worldstate
Signed-off-by: garyschulte <garyschulte@gmail.com>
Add java module info to the manifests of the following libraries used by the Besu EVM
./crypto - org.hyperledger.besu.internal.Crypto
./ethereum/rlp - org.hyperledger.besu.internal.Rlp
./util - org.hyperledger.besu.internal.Util
./datatypes - org.hyperledger.besu.Datatypes
./evm - org.hyperledger.besu.Evm
./plugin-api - org.hyperledger.besu.PluginApi
Signed-off-by: Mustafa Uzun <mustafa.uzun@limechain.tech>
* Backward Sync is now remembering recent finalized blocks.
When syncing using BWS we now consider finalized blocks reported by consensus layer.
When no finalized block is specified, nothing strange will go on. When Finalized block is specified
But Besu is not aware of any finalized block yet, we will sync as usual until the finalized block is about to get imported.
A new invariant is introduced when importing blocks using BWS.
All imported blocks has to descent from latest finalized block or a TTD block if we did not finalize yet.
* Importing a finalized block updates the information in Besu Mutable Blockchain.
* It is no longer possible to import blocks into the blockchain below a previously finalized block when using BWS
* When a new finalized block gets announced while BWS is in progress and Besu already has it imported, the Mutable Blockchain gets updated, and the chain gets checked for possible pruning
* When trying to import blocks of equal height as newly announced finalized block, then only the new announced finalized block will be possible to import using BWS
* When importing a new block using BWS after finalized, we can now guarantee that the block descends from latest finalized block
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
* BWS now has explicit parametrized check for how deep to sync
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
* addressing review comments
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
* Addressing review comment
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
Co-authored-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Start external services before we start mining
Reasons:
- you may wish to query the blockchain using rpc before you start
processing
- you may wish to register a websocket e.g to listen for blocks
currently you will probably miss one because we start processing
before we start the ws service
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Add beforeExternalServices hook for plugins
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* move the acceptance test plugins
move the test plugins that are used in the acceptance tests into the same
package namespace as the acceptance tests so that they are ignored by
sonar for code coverage
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Fix typo in log
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
This is a re-implementation of the initial POC done in https://github.com/PegaSysEng/pantheon/pull/1909/ by Danno Ferrin <danno.ferrin@gmail.com>
* Only enable plugin rpc api when enabled on --rpc-http-api or --rpc-ws-apis
* Only allow new rpc endpoints to be defined
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
The labelled gauge functionality will allow monitoring of gauge type data without creating many gauges, which can be particularly handy in tracking related data.
Signed-off-by: Paul Harris <paul.harris@consensys.net>
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
EIP-1702 account versioning was a candidate for the Istanbul hard fork
but was removed prior to the first testnet. Other versioning techniques
have greater core dev mind share and one preparatory step landed in
London, EIP-3541, making EIP-1702 very unlikely to make it to Mainnet.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Refactor: PrivacyBlockProcessor to clarify intent
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Wire up privacy genesis options into PrivacyParameters
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Refactor private state genesis into it's own class
- pass through in privacyParameters
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Refactor: inject PrivateStateGenesis into PrivacyPrecompiles
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Private Genesis initialisation
- set code, balance and storage from private-genesis.json
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Check on-chain with private genesis in acceptance tests
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Remove unused EthGetCodeCall
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Use a plugin based aproach for privacy genesis
- if the plugin is registered it will be used to apply private genesis
state
- if onchain flexible privacy groups is enable that will be applied
after
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* PrivateGenesisAcceptanceTest::createPrivacyGroup can be private
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* PrivateStateGenesis add debug logs
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Warn if genesis account allocation is in reserved precompile range
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Add sender into unsignedPrivateMarkerTransaction for plugin to make descision
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Remove locking solution as may not be needed
- if this is required we should first evaluate actual use cases
and test scenarios
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Rename PrivateStateGenesis -> PrivateStateGenesisAllocator
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Tidy up naming for getPrivateStateGenesisAllocator
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Privacy Plugin javadocs
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
Update spotless to 5.14.2 and java-google-format to 1.10.0.
Both of these are needed to support Java 16 and Gradle 7.x.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Refactor: clarify intent of arg when testing permissioning plugin
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Refactor: remove duplicate createPrivateMarkerTransaction
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* PrivateMarkerTransactionFactory: add option to delegate to plugin
if defined the plugin will be responsible for creating the pmt
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Implement locking strategy for eea_sendRawTransaction
- this will lock per address being sent to prevent nonce
too low errors
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Refactor plugin integration to use a more explicit poco
- the plugin will need to sign the transaction with the data it's given
- it will not need to query anything e.g nonce etc
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* refactor: update naming and tidy up based on comments
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Isolate code for calculating gas limit when using privacy plugin
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
Co-authored-by: Vijay Michalik <vijay.michalik@consensys.net>
* Upgrade to Apache Tuweni 2.0
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Remove intermediate repository
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Remove all occurrences of toBytes
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Migrate to tuweni-bytes
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* add changelog
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* correct reference tests
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Initial API changes
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* more changes
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Change APIs for VM ops
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Use constant UInt256.ONE
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Optimize a bit address <> word transformation
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* spotless
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Hook up an empty plugin for future unrestricted privacy encryption
Skeleton implementation for PrivacyPayloadEncryptionProvider
Wire up unencrypted serialization for private transactions
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* wire up simple privacy plugin in acceptance tests
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Pass the marker transaction through to plugin
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Move getters/setters into consistent place
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Rename plugin methods and config to be more generic
- remove protections around transaction.restriction
- the plugin is responsible for this
- make plugin more generic so could be used for restricted/unrestricted
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Provide more meaningful error message
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Fix up test naming to match impl naming
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Rename UNRESTRICTED_PRIVACY to PLUGIN_PRIVACY
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* fix typo in PrivacyParmeters::toString
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Fix exception messages to be PrivacyPlugin
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
* Remove blockNumber from getPrivateTransactionFromPayload
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
matkt
Sally MacFarlane
Antoine Toulme
Fabio Di Fabio
garyschulte
Stefan Pingel
Simon Dudley
Camilo Reis Dotto
Gabriel-Trintinalia
Danno Ferrin
naviechan
Jason Frame
Usman Saleem
Justin Florentine
Danno Ferrin
mustafauzunn
Daniel Lehrner
Jiri Peinlich
Mohit Saxena
Antony Denyer
fab-10
Paul Harris