We cleared the mark storage upon starting the pruner in order to prevent the first sweep from being smaller than it needs to be.
```
// Optimization for the case where the previous cycle was interrupted (like the node was shut
// down). If the previous cycle was interrupted, there will be marks in the mark storage from
// last time, causing the first sweep to be smaller than it needs to be.
clearMarks();
```
Unfortunately, it's currently called in prepare, which is synchronous with the main thread. This means that we can delay startup by on the order of 10 minutes if the storage was especially full.
This commit makes the pruner `start` asynchronous.
This was reverted once before due to test intermittency and this commit is the second attempt :P
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
We cleared the mark storage upon starting the pruner in order to prevent the first sweep from being smaller than it needs to be.
// Optimization for the case where the previous cycle was interrupted (like the node was shut
// down). If the previous cycle was interrupted, there will be marks in the mark storage from
// last time, causing the first sweep to be smaller than it needs to be.
clearMarks();
Unfortunately, it's currently called in prepare, which is synchronous with the main thread. This means that we can delay startup by on the order of 10 minutes if the storage was especially full.
This commit makes the pruner `start` asynchronous.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
Update implementation of the EIP-2315
- Execution of BEGINSUB causes exception and terminates execution.
- JUMPSUB sets the pc to location + 1 .
- In the edge case when BEGINSUB is the last instruction in code and this subroutine is jumped-to, the implementations should execute STOP .
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
* Add checks on the replacement of a transaction in the pool:
- reject EIP-1559 for pre-fork blocks
- accept both frontier and EIP-1559 transactions during phase 1
- reject frontier transactions after phase 2 is finalized
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
Fix cases where the gas estimate is too low and causes the transaction to fail
Implementation of a strategy that will estimate the number of gas needed by a transaction
1 - Estimate the number of gas used by a transaction
2 - Estimate the number of gas necessary to add for each sub call present in a transaction (65/64^depth)
3 - Add the minimum gas required for operations that need it (SSTORE)
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
* Update transaction pool error handling.
Transactions added locally into the pool via `eth_sendRawTransaction` can now generate 2 new JSON RPC errors:
- `ETH_SEND_TX_ALREADY_KNOWN`: occurs when adding a transaction already present in the pool.
- code: -32000
- message: `Known Transaction`
- `ETH_SEND_TX_REPLACEMENT_UNDERPRICED`: occurs when adding a transaction already present in the pool.
- code: -32000
- message: `Replacement transaction underpriced`
### Changes summary
- Created `TransactionAddedStatus` enum.
- `ALREADY_KNOWN`
- `REJECTED_UNDERPRICED_REPLACEMENT`
- `ADDED`
- Created 2 new errors in `JsonRpcError`.
- Updated JsonRpcErrorConverter to handle newly created errors.
- Updated `addLocalTransaction` method of `PendingTransactions` to return `TransactionAddedStatus` instead of boolean.
- Updated unit tests accordingly.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* add doomed key check (busy-waiting for now)
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* optional and logging
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* remove logging
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* sleeping and hardening
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* rename segments
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* move away from atomic references to regular vars
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* remove hardened segment parameter
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* increase sleep
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* spotless
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* remove unnecessary interface
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* rename
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* move commit waiting outside of timer
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* set default lock timeout to 1ms
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* add default lock timeout to tests
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* Revert "rename segments"
This reverts commit 184eefaaa0ccc857b0caff2b382f8338ff225d5d.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* fix jmh compilation error
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* add documentation
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* bump up sleep to 1ms
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* (POC) Add lock to ensure that we don't prune while comitting
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* remove unnecessary persist (#569)
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* flesh out @mbaxter's idea and remove my code
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* iterator changes
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* hybridize with doomed key
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* comment
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* move doomed key unset to after node added listener
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* update instead of getting and setting doomedKeyRef in commit
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* comment
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* invert condition
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* remove locks
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* remove `removeAllKeysUnless`
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* more remove removeAllKeysUnless
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* reuse streamKeys
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* remove test
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* set default lock timeout to 1ms
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* add default lock timeout to tests
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* fix jmh compilation error
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* revert back to locks instead of doomedkey
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* change delete to not guarantee deletion
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* plugin hash
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* javadoc
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* Revert "change delete to not guarantee deletion"
This reverts commit 2289bb34cf.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* skip key deletion on timeout
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* clear in rollback
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* Revert "fix jmh compilation error"
This reverts commit b64ecf8656.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* Revert "add default lock timeout to tests"
This reverts commit aff6aa6065.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* Revert "set default lock timeout to 1ms"
This reverts commit 267fe0a642.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* use noSlowDown write option instead of global timeout
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* add back tests
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* close tryDeleteOptions
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* remove unnecessary lock
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* move increment inside try
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* use StorageException subclass instead of field
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* revert accidental deletion in javadoc
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* tryDelete javadoc
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* add trace for skipping key deletion
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* merge catch and finally try blocks
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* switch from exception to boolean return value
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* tweak
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* changelog changes
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* add api back
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
* add back throws javadoc
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
Co-authored-by: Meredith Baxter <meredith.baxter@consensys.net>
Co-authored-by: MadelineMurray <43356962+MadelineMurray@users.noreply.github.com>
* Compute transaction gas budget allocation according to EIP-1559 rules.
- Added `TransactionGasBudgetCalculator` with `hasBudget` method.
- Updated `AbstractBlockProcessor` to compute the gas budget using `TransactionGasBudgetCalculator`.
- Added unit tests in `TransactionGasBudgetCalculatorTest`.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Fixed unit test.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Fix EIP-1559 block mining.
- Updated `AbstractBlockCreator` to set base fee to initial value if block is the fork block number.
- Updated `BlockHeaderBuilder` to populate base fee field.
- Updated `EthHash` to include base fee in hash computation if present in header.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Set the gas block limit as a factor of the target gas usage instead of hard coded value.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
When updating an account we often calculate the address hash multiple
times. We also will always calculate it at least once when tracking an
accout update. Hence, just calculate it in the constructor and re-use
it.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Some of our loggers were not private, static, and final. In a few
cases these were non-static fields in classes that were repeatedly
instantiated in core transaction logic.
This is enforced via a new ErrorProne check, so the PR includes fixes
for all of the places this was a problem, not just the performance
impacting code.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Handle legacy gas pool and EIP-1559 gas pool when mining blocks.
- Added `EIP1559` manager class in `ProtocolSpec` (defaulted to `Optional.empty()`).
- Updated `MainnetProtocolSpecs` EIP-1559 definition to set the `EIP1559` manager class.
- Updated `BlockTransactionSelector`
- Added `eip1559CumulativeGasUsed` field in `TransactionSelectionResults`.
- Updated `updateTransactionResultTracking` method to compute cumulative gas used according to `EIP-1559` rules.
- Updated `transactionTooLargeForBlock` method to compute gas remaining according to `EIP-1559` rules.
- Updated `blockOccupancyAboveThreshold` method to compute minimum block occupancy ratio according to `EIP-1559` rules.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Update BlockTransactionSelector transaction gas price computation.
BlockTransactionSelector should compare transaction gas price to the min transaction gas price parameter according to EIP-1559 rules.
- Added `TransactionPriceCalculator` to `ProtocolSpec` (defaulted to `TransactionPriceCalculator.frontier()`.
- Created `minTransactionPriceInNextBlock` method in `BaseFee`.
- Removed useless constructor in `BaseFee`.
- Updated EIP-1559 protocol spec definition to set the correct transaction price calculator.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
Enable native encryption by default, but retain the feature flags. The problem with acceptance tests is that the new native libraries added some startup overhead, which when translated to starting up 12 nodes on the same vm resulted in longer time getting to a testable state. Start up timeout was increased to 60 seconds.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Via JNA use native scep256k1 and altbn128 libraries.
This is gated by two feature flags and is disabled by default.
* `--Xsecp256k1-native-enabled` enables native secp256k1 across all of besu
* `--Xaltbn128-native-enabled` enables native altbn128 in the precompiled contracts
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Manage EIP-1559 transactions forging in local transaction pool.
- Updated `TransactionPool`
- Added `TransactionPriceCalculator` field for frontier transactions.
- Added `TransactionPriceCalculator` field for EIP-1559 transactions.
- Added `EIP1559` field.
- Added `minTransactionGasPrice` to compute transaction gas price according to EIP-1559 rules.
- Created `BaseFee` domain object.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
## PR description
- Created `TransactionPriceCalculator` to compute the transaction gas price.
- Created `CoinbaseFeePriceCalculator` to compute the coinbase reward.
- Updated `MainnetTransactionProcessor` to apply new consensus rules:
- Updated transaction gas price for EIP-1559 transactions.
- Updated coinbase reward for EIP-1559 transactions.
- Updated `MainnetProtocolSpecs` to build the correct TransactionProcessor for EIP-1559 definition.
## Fixed Issue(s)
- #601
- #605
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Move baseFee from SealableBlockHeader to ProcessableBlockHeader
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Move baseFee from SealableBlockHeader to ProcessableBlockHeader
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Write null gas price in the RLP structure if this an EIP-1559 transaction. (#738)
Write null gas price in the RLP structure if this an EIP-1559 transaction.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Added changelog entries for PR:
- https://github.com/hyperledger/besu/pull/430
- https://github.com/hyperledger/besu/pull/440
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* For post EIP-1559 blocks we must disable GasLimitRangeAndDeltaValidationRule.
We must also update ProofOfWorkValidationRule to include base fee field in the hash computation.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Rename logger
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Rename logger
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Fix base fee computation and add tests.
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
* Address PR comments
Signed-off-by: Abdelhamid Bakhta <abdelhamid.bakhta@consensys.net>
Pull two methods out of ProtocolSpec into the only class that uses them.
This is in preparation for future refactoring of the ProtocolSpec.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Add support for `berlinBlock` in genesis configs.
Until the final contents of Berlin are finalized this will be guarded by
a feature flag `--Xberlin-enabled`. Once Berlin is finalized and before
it is deployed on test nets the flag will be removed.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Speed up build time by removing some of the most time consuming tests
* CALLBlake2f_MaxRounds - runs Blake2f 4 billion rounds, something no
one ever does and there is never enough gas to do. We have a unit test
for this already.
* The "do this 50k times" tests. Each has a smaller "5k times" or less
variant. This drifts into performance testing instead of conformance
testing
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Karim T
Ratan Rai Sur
Adrian Sutton
Danno Ferrin
Danno Ferrin
Abdelhamid Bakhta
br0tchain
Stefan Pingel
Pierre Grimaud
Lucas Saldanha