Fix cases where the gas estimate is too low and causes the transaction to fail
Implementation of a strategy that will estimate the number of gas needed by a transaction
1 - Estimate the number of gas used by a transaction
2 - Estimate the number of gas necessary to add for each sub call present in a transaction (65/64^depth)
3 - Add the minimum gas required for operations that need it (SSTORE)
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
* Update transaction pool error handling.
Transactions added locally into the pool via `eth_sendRawTransaction` can now generate 2 new JSON RPC errors:
- `ETH_SEND_TX_ALREADY_KNOWN`: occurs when adding a transaction already present in the pool.
- code: -32000
- message: `Known Transaction`
- `ETH_SEND_TX_REPLACEMENT_UNDERPRICED`: occurs when adding a transaction already present in the pool.
- code: -32000
- message: `Replacement transaction underpriced`
### Changes summary
- Created `TransactionAddedStatus` enum.
- `ALREADY_KNOWN`
- `REJECTED_UNDERPRICED_REPLACEMENT`
- `ADDED`
- Created 2 new errors in `JsonRpcError`.
- Updated JsonRpcErrorConverter to handle newly created errors.
- Updated `addLocalTransaction` method of `PendingTransactions` to return `TransactionAddedStatus` instead of boolean.
- Updated unit tests accordingly.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
prior to Eth/65 we did not participate in transaction gossiping until we
were close to in sync. With this change we won't request pooled
transaction hashes until we are similarly close to sync.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Increase Netty threadpool size for handshake handling
It was found that all ECIES handhshaking is taking place in
an NioEventLoop of being serviced by one worker.
This in turn meant that, during periods of high demand,
many potential peers were timing out (as the latter peers
had to wait for earlier peers to complete prior to
being serviced).
This change grows the threadpool to 10 (arbitrarily selected),
which minimises the number of peers waiting for connectivity.
Signed-off-by: Trent Mohay <trent.mohay@consensys.net>
Co-authored-by: Jason Frame <jasonwframe@gmail.com>
* add doomed key check (busy-waiting for now)
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* optional and logging
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* remove logging
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* sleeping and hardening
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* rename segments
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* move away from atomic references to regular vars
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* remove hardened segment parameter
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* increase sleep
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* spotless
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* remove unnecessary interface
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* rename
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* move commit waiting outside of timer
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* set default lock timeout to 1ms
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* add default lock timeout to tests
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* Revert "rename segments"
This reverts commit 184eefaaa0ccc857b0caff2b382f8338ff225d5d.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* fix jmh compilation error
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* add documentation
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* bump up sleep to 1ms
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* (POC) Add lock to ensure that we don't prune while comitting
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* remove unnecessary persist (#569)
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* flesh out @mbaxter's idea and remove my code
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* iterator changes
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* hybridize with doomed key
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* comment
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* move doomed key unset to after node added listener
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* update instead of getting and setting doomedKeyRef in commit
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* comment
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* invert condition
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* remove locks
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* remove `removeAllKeysUnless`
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* more remove removeAllKeysUnless
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* reuse streamKeys
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* remove test
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* set default lock timeout to 1ms
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* add default lock timeout to tests
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* fix jmh compilation error
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* revert back to locks instead of doomedkey
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* change delete to not guarantee deletion
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* plugin hash
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* javadoc
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* Revert "change delete to not guarantee deletion"
This reverts commit 2289bb34cf.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* skip key deletion on timeout
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* clear in rollback
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* Revert "fix jmh compilation error"
This reverts commit b64ecf8656.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* Revert "add default lock timeout to tests"
This reverts commit aff6aa6065.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* Revert "set default lock timeout to 1ms"
This reverts commit 267fe0a642.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* use noSlowDown write option instead of global timeout
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* add back tests
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* close tryDeleteOptions
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* remove unnecessary lock
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* move increment inside try
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* use StorageException subclass instead of field
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* revert accidental deletion in javadoc
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* tryDelete javadoc
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* add trace for skipping key deletion
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* merge catch and finally try blocks
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* switch from exception to boolean return value
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* tweak
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* changelog changes
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* add api back
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* add back throws javadoc
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
Co-authored-by: Meredith Baxter <meredith.baxter@consensys.net>
Co-authored-by: MadelineMurray <43356962+MadelineMurray@users.noreply.github.com>
* When in retesteth return the coinbase in the eth_getBlockByHash, as
required by the retesteth tool.
* use genesis extraData when mining
* storage keys should be 0x00 instead of 0x when zero.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Compute transaction gas budget allocation according to EIP-1559 rules.
- Added `TransactionGasBudgetCalculator` with `hasBudget` method.
- Updated `AbstractBlockProcessor` to compute the gas budget using `TransactionGasBudgetCalculator`.
- Added unit tests in `TransactionGasBudgetCalculatorTest`.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Fixed unit test.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
IbftRound has been updated to accept Signing errors (eg no signature supplier available) and
continue operating if possible.
This also catches failures in signing and ECDH Key agreement
creation during discovery and handshaking.
Signed-off-by: Trent Mohay <trent.mohay@consensys.net>
* Fix EIP-1559 block mining.
- Updated `AbstractBlockCreator` to set base fee to initial value if block is the fork block number.
- Updated `BlockHeaderBuilder` to populate base fee field.
- Updated `EthHash` to include base fee in hash computation if present in header.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Set the gas block limit as a factor of the target gas usage instead of hard coded value.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
When updating an account we often calculate the address hash multiple
times. We also will always calculate it at least once when tracking an
accout update. Hence, just calculate it in the constructor and re-use
it.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Some of our loggers were not private, static, and final. In a few
cases these were non-static fields in classes that were repeatedly
instantiated in core transaction logic.
This is enforced via a new ErrorProne check, so the PR includes fixes
for all of the places this was a problem, not just the performance
impacting code.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Handle legacy gas pool and EIP-1559 gas pool when mining blocks.
- Added `EIP1559` manager class in `ProtocolSpec` (defaulted to `Optional.empty()`).
- Updated `MainnetProtocolSpecs` EIP-1559 definition to set the `EIP1559` manager class.
- Updated `BlockTransactionSelector`
- Added `eip1559CumulativeGasUsed` field in `TransactionSelectionResults`.
- Updated `updateTransactionResultTracking` method to compute cumulative gas used according to `EIP-1559` rules.
- Updated `transactionTooLargeForBlock` method to compute gas remaining according to `EIP-1559` rules.
- Updated `blockOccupancyAboveThreshold` method to compute minimum block occupancy ratio according to `EIP-1559` rules.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Update BlockTransactionSelector transaction gas price computation.
BlockTransactionSelector should compare transaction gas price to the min transaction gas price parameter according to EIP-1559 rules.
- Added `TransactionPriceCalculator` to `ProtocolSpec` (defaulted to `TransactionPriceCalculator.frontier()`.
- Created `minTransactionPriceInNextBlock` method in `BaseFee`.
- Removed useless constructor in `BaseFee`.
- Updated EIP-1559 protocol spec definition to set the correct transaction price calculator.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
@pinges correctly pointed out that, even though we have access to all the logs in the BlockAddedEvent, we still use BlockchainQueries to add the new logs from the event. Now we do the sane thing.
Unfortunately, it doesn't seem as easy with private logs so we still have to use privacyQueries as far as I can tell.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
Enable native encryption by default, but retain the feature flags. The problem with acceptance tests is that the new native libraries added some startup overhead, which when translated to starting up 12 nodes on the same vm resulted in longer time getting to a testable state. Start up timeout was increased to 60 seconds.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Via JNA use native scep256k1 and altbn128 libraries.
This is gated by two feature flags and is disabled by default.
* `--Xsecp256k1-native-enabled` enables native secp256k1 across all of besu
* `--Xaltbn128-native-enabled` enables native altbn128 in the precompiled contracts
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Manage EIP-1559 transactions forging in local transaction pool.
- Updated `TransactionPool`
- Added `TransactionPriceCalculator` field for frontier transactions.
- Added `TransactionPriceCalculator` field for EIP-1559 transactions.
- Added `EIP1559` field.
- Added `minTransactionGasPrice` to compute transaction gas price according to EIP-1559 rules.
- Created `BaseFee` domain object.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
## PR description
- Created `TransactionPriceCalculator` to compute the transaction gas price.
- Created `CoinbaseFeePriceCalculator` to compute the coinbase reward.
- Updated `MainnetTransactionProcessor` to apply new consensus rules:
- Updated transaction gas price for EIP-1559 transactions.
- Updated coinbase reward for EIP-1559 transactions.
- Updated `MainnetProtocolSpecs` to build the correct TransactionProcessor for EIP-1559 definition.
## Fixed Issue(s)
- #601
- #605
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Move baseFee from SealableBlockHeader to ProcessableBlockHeader
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Move baseFee from SealableBlockHeader to ProcessableBlockHeader
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Write null gas price in the RLP structure if this an EIP-1559 transaction. (#738)
Write null gas price in the RLP structure if this an EIP-1559 transaction.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
Make sure we remove pending transaction tracker when a peer disconnects
with a unit tests to test that disconnect callbacks are indeed called when a peer disconnects, for transaction and pending transaction tracking
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Added changelog entries for PR:
- https://github.com/hyperledger/besu/pull/430
- https://github.com/hyperledger/besu/pull/440
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* For post EIP-1559 blocks we must disable GasLimitRangeAndDeltaValidationRule.
We must also update ProofOfWorkValidationRule to include base fee field in the hash computation.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Rename logger
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Rename logger
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Fix base fee computation and add tests.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Address PR comments
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
Karim T
Abdelhamid Bakhta
Lucas Saldanha
Trent Mohay
Danno Ferrin
Ratan Rai Sur
Stefan Pingel
Pierre Grimaud
Antoine Toulme