Add a new way to synchronize which is X_CHECKPOINT. This mode is experimental so use it at your own risk. This mode allows you to do like a snapsync but starting from a specific checkpoint instead of starting from the genesis.
This checkpoint will be in the genesis configuration of each network. To add the checkpoint mechanism in a network you just have to add the checkpoint section in the genesis.
Currently there is a checkpoint for ropten, goerli and mainnet.
Mainnet on i3.2xlarge <6 hours
Goerli on i3.2xlarge <1 hours
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
* first pass at building arm64 besu docker images
* add arch to docker publish
* add architecture envars to executors
* first pass at manifest docker
* setup docker in manifest step
* add base image to manifestDocker
* add changelog
Signed-off-by: garyschulte <garyschulte@gmail.com>
* added test file to match any topic
* add singleton list containing null if empty topic passed in
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
* Fix json-rpc HTTP tests [#535]
The `o.h.b.e.a.j.JsonRpcHttpServiceTest#exceptionallyHandleJsonSingleRequest` and `o.h.b.e.a.j.JsonRpcHttpServiceTest#exceptionallyHandleJsonBatchRequest` tests were throwing ClassCastException in `o.h.b.e.a.j.JsonRpcHttpService#validateMethodAvailability` which wasn't ever catched, returning status 500 by default, but that wasn't the use case aimed to test. Another test running an exceptional method is `o.h.b.t.a.p.EnclaveErrorAcceptanceTest#whenEnclaveIsDisconnectedGetReceiptReturnsInternalError` which validates an "Internal Error" proper json-rpc response. I changed the first two tests to be consistent with the later one.
* Extract json-rpc HTTP authentication to a handler [#535]
* Replace TimeoutHandler in GraphQLHttpService with Vert.x's impl [#535]
* Extract json-rpc HTTP parser to a handler [#535]
* Refactor json-rpc WS handler [#535]
* Add json-rpc IPC support [#535]
Signed-off-by: Diego López León <dieguitoll@gmail.com>
Signed-off-by: Diego López León <dieguitoll@gmail.com>
Rename SHA3 -> Keccak256
Ethereum doesn't do "official" sha3, we do keccak256, so rename as needed to reflect that reality.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Add download links
* update version number
* update changelog
* one critical change was missed (was not included in original pr)
* add note about Java 17 migration.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Change the unit test execution to use the Junit5 JUnitPlatform. This
allows for a mix of junit 4 and junit 5 tests and for a gradual
migration to junit 5 instead of a big bang. One class depended on
junit 4 exceptions and was updated. Two tests depending on
native libraries fail gracefully on mac (and only mac).
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
The currently way Besu has to spot if a transaction has already been process, is to look if the transaction is present in the transaction pool, that by default is 4K, while the amount of pending transactions on the mainnet, is much more, the order of hundred of thousands, so basically even if a transaction has been already processed, the chances that it gets reprocessed is very high, with the result of doing a lot of useless work, that affects Besu performance.
A trivial solution could be to just raise the transaction pool size, but that is not always advisable, because it is critical for block production to keep it fast, and incresing its size could negatively affect the perfomance of the strategy choosen to select transactions to include in the block.
A better option, implemented here, is to leverage data that we already have, and that keeps the history of the transactions exchanged with other peers. This data is just a collection of transaction hashes that we have received or seen, and in any case if a transaction is in that collection, it means that it has already been processed by Besu, so it is possible to directly skip it.
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Currently Besu has a limited support for sending NewPooledTransactionHashes messages, and other aspect related to reduce transactions synchronization traffic, described in the Ethereum Wire Protocol version 66.
Specifically:
Besu only uses NewPooledTransactionHashes for new local transactions, while it could be extended to any transaction added to the transaction pool
Besu does not limit the sending of the full transaction messages to a small fraction of the connected peers, and sends the new transaction hashes to all the remaining peers
This PR, extends eth/66 support and does some code refactoring, to remove some reduntant code and rename some classes to identify they are related to the NewPooledTransactionHashes message.
The main changes are:
Do not have a separate tracker for transaction hashes, since for them we can reuse PeerTransactionTracker, that tracks full transactions exchange history and sending queue with a peer. So PeerPendingTransactionTracker has been removed. --tx-pool-hashes-max-size is now deprecated and has no more effect and it will be removed in a future release.
When a new peer connects, if it support eth/6[56] then we send all the transaction hashes we have in the pool, otherwise we send the full transactions.
When new transactions are added to the pool, we send full transactions to peers without eth/6[56] support, or to a small fractions of all peers, and then we send only transaction hashes to the remaining peer that support eth/6[56]. Both transactions and transaction hashes are only sent if not already exchanged with that specific peer.
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Tune transaction worker queue capacity to adapt to mainnet
Currently Besu handles hundreds of transaction message per seconds,
so having a queue of 1M messages result in a lot of expired messages,
make sense to reduce the capacity to drop incoming messages that could not
be handled anyway.
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Tune the number transactions seen by peer to adapt to mainnet traffic
Currently on mainnet there are many thousand of pending transactions
exchanged between peers, but Besu has a short memory of what has been
exchanged with a specific peer, with the result that the same transaction
is often exchanged back and forth with the same peer, expecially when the
peer is another Besu.
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Improve trace log of transaction exchanges with other peers
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Update CHANGELOG
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Allow precompiled contracts to return richer information instead of
results/fail. System/precompile contracts can now revert, fail, refund
gas, etc. instead of just succeed or fail.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* refactored auth to allow variant JWT rules to be adopted by different stacks
* configures JWT secret from a file, or creates a temporary one
* jwt auth is optional for engine apis
* test coverage of new cli options
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* Json RPC endpoint and test files for trace_rawTransaction
* cache the block number to make sure we are using the same block
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
Co-authored-by: Frank Li <b439988l@gmail.com>
Co-authored-by: MadelineMurray <43356962+MadelineMurray@users.noreply.github.com>
* Refactor TransactionPool to accept MiningParameters
* Check for zero GasPrice Frontier Transactions
* if you are not mining your node could fill up with pending transactions.
* make low-or-no-gas transactions viable for local transactions
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
Co-authored-by: garyschulte <garyschulte@gmail.com>
matkt
garyschulte
Sally MacFarlane
Fabio Di Fabio
Lucas Saldanha
Diego López León
Danno Ferrin
Justin Florentine
Stefan Pingel
Simon Dudley
Frank Li
mbaxter
Antony Denyer