Fix some issues which caused some log to be missing when calling the eth_getLogs method
- Setting up a cache version
- Add a check integrity of the cache
- Fix a lock issue
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
We were attempting to use hashes from a queue we frequently evict from
and not handling that case in an iterator that walks over that
collection.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
This solves the same problem as 3a39fbb but better. The attempt
there has an issue where it still emulates sending the request and
therefore only 5 requests max can be aborted. Thus the problem of the
peer becoming stuck in a busy state remains. Here we do what we
should've done all along, which is proactively abort these requests
when a disconnect event occurs.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
Any process that depends on pre-assigned peer tasks is liable to grow
our pendingRequests out of control.
When we attempt to execute a pending transaction, we check for that
peers availability:
PendingPeerRequest.java
```
final Optional<EthPeer> selectedPeer =
leastBusySuitablePeer.filter(EthPeer::hasAvailableRequestCapacity);
selectedPeer.ifPresent(this::sendRequest);
return selectedPeer.isPresent();
}
}
```
However, if that peer was disconnected while there were max outstanding
requsts, we'll never reattempt it as selectedPeer will be empty.
Furthermore, we'll return false and keep that pending request in our
list of requests.
With this change we indicate that we aren't waiting for any outstanding
requests when we disconnect from a peer, allowing the existing logic
that triggers when peers are disconnected to be hit.
The getter I added just for testing purposes was to get around the
inadequacies of the MockPeerConnection. There's a whole lot of callback
plumbing that's needed for a simple ethPeer.disconnect to do the same
thing it does when we call it in our production code.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
Move key referenceTest classes relating to reading the JSON data into a
"main" java package so that it can be included in other production
classes. Some classes were renamed to make their intent clearer. Other
smaller changes that bring classes in line with current coding
standards were done.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
CompletableFuture already has its own atomicity under the hood that
allows for threadsafe cancelations.
Hides the AtomicReference that gives us idempotency in executing the
task so that inheritors don't need to care about it.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
This PR fixes an error when downloading chain (Async operation failed)
When there are several blocks which are very large (> 12M) and which are requested in the same segment (by default 200) we can have timeouts and never manage to synchronize. This modification will make it possible to gradually reduce the size of the segment with each attempt. Then the segment resumes its default size for the next blocks
If the reduction is not enough at the last attempt we try with a single block
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
- Added `--rpc-tx-feecap` command line flag.
- Maximum transaction fees (in Wei) accepted for transaction submitted through RPC.
- Defaulted to 1 ether.
- Updated `TransactionPool.addLocalTransaction` method: performs an additional check to verify if transaction fees don't exceed user defined fee cap (if cap is set to 0 then it is ignored and means no cap).
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* rename more whitelist occurrences; change allowlisted to allowed and reword where we ended up with allowlisting
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
Upgrade to ErrorProne 2.4.0
* public constructors on abstract classes are removed
* Javadoc must have meaningfull documentation
* lambdas should not be variables
* Added to the list of confusing inner class names (Entry and Type)
* no assert keyword in tests
* Obsolete JDK classes produce errors now
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Before this pull request Besu was using the latest known fork id to create status message for Ethereum P2P protocol handshake.
This latest known fork id was created based on a list of forks retrieved from the Genesis.
For private networks it is possible that all fork blocks number are set to 0.
The algorithm to compute the valid fork hashes excludes 0 values.
As a result, the list was empty and the `getLatestForkId` was returning `null`. This is an issue when you support capabilities >= to Eth/64 sub protocol because other peers expect the fork id value in the `RLP` encoded message.
Moreover, the algorithm to compute the fork id should be aware of the chain head number and update `CRC` value only for fork blocks below the current head.
This pull request fixes this issue by fetching the chain head number and update accordingly the `CRC` value.
Unit tests have been extended to cover an exhaustive list of possible combinations on named networks (`goerli`, `rinkeby`, `ropsten` and `mainnet`).
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
ProtocolContext uses a generic for the consensus state, which has a very
large footprint across the code to accomplish what it intends to
accomplish. For every call there are about 61 other lines per call that
need to be updated, over 1300 lines total.
Instead replace it with java.lang.Class#cast, which provides runtime
security, and use generics to provide the compile time sugar that
allows for chained methods of the appropriate type. Then remove its
(quite large) footprint from the rest of the code.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* feat: FastSync now stops on failure instead of falling back to FullSync
* feat: exiting on FastSync failed
* feat: FastSync continues on FullSync when no errors happened
Signed-off-by: Alexandre PARIS-VERGNE <alexpv14@gmail.com>
Rollback a Eth64 change intended to improve compatibility in consortium network. It resulted in many nodes not connecting on named networks such as goerli and mordor.
* Revert "Handle backward compatibility for EIP-2124 fork id management (#981)"
This reverts commit 1d8218ae
* Revert "[EIP-2124] Backward compatible fork id management implementation (#979)"
This reverts commit cb2c67d7
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Slight rework on some PendingTransaction synchronization
* sync on `prioritizedTransactions` instead of `pendingTransactions`
* `pendingTransactions` is already a `ConcurrentHashMap`, which
syncs on itself internally
* not all accesses to `pendingTransactions` were synced
* `prioritizedTransactions` is not exposed directly or indirectly,
making it a better lock
* Improve synchronization in TransactionsForSenderInfo
* `gaps` was returned without requiring synchronization
* `transacitonInfos` was not thread safe.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Handle zeros fork blocks when computing fork id.
- Updated `ForkIdManager`:
- Created `ForkIDChecker` interface with `boolean check(ForkId forkId)` method.
- Added a check in the constructor to use the proper `ForkIDChecker` based on the following conditions:
- Accept all peers if the list of forks contains only `zeros`.
- Use `EIP-2124` rules otherwise.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Handle backward compatibility
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
It was possible to run Besu on an incomplete fast-synced db with full sync enabled. This causes the node to stall out. This PR resolves this issue.
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
Because of where we provide info log messages in the
BlockPropagationManager we don't log out of order blocks. Moving to
task cleanup allows us to get the timing information too instead of
simply as part of the executeTask method
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Add checks on the replacement of a transaction in the pool:
- reject EIP-1559 for pre-fork blocks
- accept both frontier and EIP-1559 transactions during phase 1
- reject frontier transactions after phase 2 is finalized
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* The transaction gas price is computed when adding a transaction into the local pool using eth_sendRawTransaction JSON RPC endpoint. Transaction price must be computed properly depending on the type of the transaction.
For instance `shouldReplace` method of PendingTransactions must be updated to deal with EIP-1559 transactions.
- Updated `PendingTransactions` to add access to the chain header in order to retrieve the last base fee value.
- Updated `TransactionReplacementByPriceRule` to compute the transaction price depending on the type of the transaction (frontier or eip-1559).
- Added unit tests to cover all possible replacement scenarios.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Update transaction pool error handling.
Transactions added locally into the pool via `eth_sendRawTransaction` can now generate 2 new JSON RPC errors:
- `ETH_SEND_TX_ALREADY_KNOWN`: occurs when adding a transaction already present in the pool.
- code: -32000
- message: `Known Transaction`
- `ETH_SEND_TX_REPLACEMENT_UNDERPRICED`: occurs when adding a transaction already present in the pool.
- code: -32000
- message: `Replacement transaction underpriced`
### Changes summary
- Created `TransactionAddedStatus` enum.
- `ALREADY_KNOWN`
- `REJECTED_UNDERPRICED_REPLACEMENT`
- `ADDED`
- Created 2 new errors in `JsonRpcError`.
- Updated JsonRpcErrorConverter to handle newly created errors.
- Updated `addLocalTransaction` method of `PendingTransactions` to return `TransactionAddedStatus` instead of boolean.
- Updated unit tests accordingly.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
prior to Eth/65 we did not participate in transaction gossiping until we
were close to in sync. With this change we won't request pooled
transaction hashes until we are similarly close to sync.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Update BlockTransactionSelector transaction gas price computation.
BlockTransactionSelector should compare transaction gas price to the min transaction gas price parameter according to EIP-1559 rules.
- Added `TransactionPriceCalculator` to `ProtocolSpec` (defaulted to `TransactionPriceCalculator.frontier()`.
- Created `minTransactionPriceInNextBlock` method in `BaseFee`.
- Removed useless constructor in `BaseFee`.
- Updated EIP-1559 protocol spec definition to set the correct transaction price calculator.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Manage EIP-1559 transactions forging in local transaction pool.
- Updated `TransactionPool`
- Added `TransactionPriceCalculator` field for frontier transactions.
- Added `TransactionPriceCalculator` field for EIP-1559 transactions.
- Added `EIP1559` field.
- Added `minTransactionGasPrice` to compute transaction gas price according to EIP-1559 rules.
- Created `BaseFee` domain object.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
Make sure we remove pending transaction tracker when a peer disconnects
with a unit tests to test that disconnect callbacks are indeed called when a peer disconnects, for transaction and pending transaction tracking
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Fix support of peers <eth/65
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Fix unit test
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Update ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/PeerPendingTransactionTracker.java
Co-Authored-By: Danno Ferrin <danno.ferrin@shemnon.com>
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* make protocol version part of the function
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
Co-authored-by: Danno Ferrin <danno.ferrin@shemnon.com>
This change allows the newly defined "SecurityModule" to work against a generic API, while the NodeKey interface adapts these generic types to the types already used throughout Besu (i.e. those defined in Secp256k1).
Signed-off-by: Trent Mohay <trent.mohay@consensys.net>
* Added changelog entries for PR:
- https://github.com/hyperledger/besu/pull/430
- https://github.com/hyperledger/besu/pull/440
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Implement gas limit and gas price block header validation rules as per specified in EIP-1559
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* We should enforce the per transaction gas limit, as specified by the EIP, to 8,000,000 gas.
Add a `TransactionInvalidReason`
Update the `validate` method in `MainnetTransactionValidator` to enforce this check.
Update the validation process in `MainnetBlockBodyValidator` to enforce this check for all transactions within a block.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* fix unit test
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Removed blockNumber from TransactionValidator.validate
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
Update BesuController to use the NodeKey, rather than working with KeyPair - which in turn allows the crypto operations to be injected.
Signed-off-by: Trent Mohay <trent.mohay@consensys.net>
The EthQL API spec specifies pulledStates and knownStates in the
`syncing` query. Previously we always returned null. This plumbs through
the needed data so that the synchronization states can report the
fast sync progress via EthQL, as well as the `eth_syncing` JSON-RPC.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* remove max pivot block resets during fast sync
* increase max retry number and fix test
* change logs in the handleFailure method
* change logs related to suspicious number of retries
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
matkt
Ratan (Rai) Sur
Danno Ferrin
Abdelhamid Bakhta
Sally MacFarlane
Adrian Sutton
br0tchain
mark-terry
Danno Ferrin
Pierre Grimaud
Antoine Toulme
Trent Mohay
Raw Pong Ghmoa
David Mechler