The parallel task production is by sub-trie, so calling `visitAll` on a
root node will eventually spawn up to 16 tasks (for a hexary trie).
If we marked each sub-trie in its own thread, with no common queue of
tasks, our mark speed would be limited by the sub-trie with the maximum
number of nodes. In practice for the Ethereum mainnet, we see a large
imbalance in sub-trie size so without a common task pool the time in
which there is only 1 thread left marking its big sub-trie would be
substantial.
If we were to leave all threads to produce mark tasks before starting
to mark, we would run out of memory quickly.
If we were to have a constant number of threads producing the mark
tasks with the others consuming them, we would have to optimize the
production/consumption balance.
To get the best of both worlds, the marking executor has a
ThreadPoolExecutor.CallerRunsPolicy which causes the producing tasks to
essentially consume their own mark task immediately when the task queue
is full. The resulting behavior is threads that mark their own sub-trie
until they finish that sub-trie, at which point they switch to marking
the sub-trie tasks produced by another thread.
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
Upgrade to ErrorProne 2.4.0
* public constructors on abstract classes are removed
* Javadoc must have meaningfull documentation
* lambdas should not be variables
* Added to the list of confusing inner class names (Entry and Type)
* no assert keyword in tests
* Obsolete JDK classes produce errors now
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Some of our loggers were not private, static, and final. In a few
cases these were non-static fields in classes that were repeatedly
instantiated in core transaction logic.
This is enforced via a new ErrorProne check, so the PR includes fixes
for all of the places this was a problem, not just the performance
impacting code.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Removes as many Gradle 7.0 compatibility issues as possible
* `baseName` -> `archiveBaseName`
* `extension` -> `archiveExtension`
* `destinationDir` -> `destinationDirectory`
* `runtime` -> `runtimeOnly`
* Change some log4j-api and log4j-core dependencies
* Remove an unneeded and outdated plugin (`net.ltgt.apt`)
* tweak the plugin-api change detector's property annotations.
Warnings still exist with one external plugin used for license file
checking that we do not control the source code for.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Update dependencies to most current version
- except picocli which is a major version update
Alphabetize dependencies
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* adding in spdx-license-identifier & updated check for the same; removing license check from spotless
Signed-off-by: Joshua Fernandes <joshua.fernandes@consensys.net>
* Change CheckSpdxHeader to a task.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
This reverts commit 814b36e4
The needed chantes to get rid of Instant.now (which is also needed to get rid
of the wall clock dependency) are too deep and intrusive into IBFT to try and
speed patch them in that some APIs require re-work, so in the interst of test
stability this gets sheleved until it is all ready.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
To allow us to reset the timestamp in the blockchain for Retesteth support
we need to pass a Clock to affected APIs and use that instead of the static method
System.currentTimeMillis(). The most consistent way to do this that will ensure
that the API does not sneak back in is to ban the method via ErrorProne.
TestClock.fixed() was altered to return the "now" time of the first time the fixed clock was requested, needed for many header validation tasks validating headers are not from the future.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
EIP-1602 specifies a new validation step. Add hooks for those to occur.
Move contract size checking into the validation hooks.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Update the build to require and target Java 11
* Add JVM flags to suppress specific illegal access warnings
* Remove Java 8 accommodations
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Most libraries with newer stable versions were upgraded to their
newest stable version
* One exception is RocksDB, where a previous update caused a performance
regression. the 6.0.1 update will be tested in another PR
* GraphQL got stingier about null variable maps, so empty maps were passed instead
* errorprone plugin got stricter with it's variable access
* There was a license that was slightly different, creating 2 MIT licenses in the license report.
* Gradle wanted a new almost identical wrapper jar.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Don't copy collections if we don't need to. Change types higher up if needed.
* Don't use Guava's Object.equal, use Java's Objects.equals.
** add errorprone test to enforce the banning of Guava's Objects class.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Biggest change is that UnusedVariable and UnusedMethod went to WARN by
default. Since our build is a no warning build this means we either need
to turn them off or fix them. I mostly opted for the latter. Test code
was mostly fixed, unused loggers were deleted, and other shipped code
was mostly suppressed.
Two less noisy fixes to not use `SortedSet` and to use zero based
comparable results instead of -1, 0, and 1. Also a compiler nit in
errorprone was suppressed, per the description it won't affect us.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Two changes:
* Stream chains now take up less vertical lines, only breaking on
stream operations.
* Long annotations that span multiple lines no longer have a dangling
parentesis and indent 4 spaces.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* upgread all dependencies and gradle plugins
* did not upgrade net.researchgate.release gradle plugin as that is know to break correto JVM
* removed quickstart from module list. It's been gone a while now.
* sorted the modules
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Upgrade errorprone
* Upgrade errorprone from 2.3.1 to 2.3.2
* Upgrade Jenkinsfile so that CI will use Java 11
* Suppress these new rules:
* EqualsGetClass
* ImmutableEnumChecker
* UnnecessaryParentheses
* Change code to conform to these new rules:
* BadImport
* BadInstanceof
* InconsistentHashCode
* LockNotBeforeTry
* MathAbsoluteRandom
* ModifiedButNotUsed
* UndefinedEquals
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Upgrade dependency versions
* All version information is merged into one place
* Upgrade most version to their latest non-test version.
* dependencies are now sorted
Two dependencies were not upgraded:
errorprone - There are new checks that require build or source changes.
vertex - The license to eclipse 2.0 which is not in our current approved license list.
The change is trivial but for tracability it should be done alone.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* [MINOR] Only Test errorprone-checks on JDK8
The current version of errorprone causes errors when compiled with Java
11. The next version causes causes problems with Java 8. So for now
keep the old version and only test when compiled with Java 8 (which
is the current Jenkins version). When the test is skipped an info log is sent.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Ratan (Rai) Sur
matkt
Danno Ferrin
Antoine Toulme
Joshua Fernandes
Danno Ferrin
Danno Ferrin
mark-terry
Christopher Mckay
PegaSys Admin