* Consolidate all metric collectors into the metrics system
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Fixes
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Do not recreate the Prometheus metric system
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
---------
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* Include Commit-Hash in manifests
* Add commit method in BesuInfo
* Use 8 character hash in EngineGetClientVersionV1 and revert calculateVersion back to original spec
Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>
Co-authored-by: garyschulte <garyschulte@gmail.com>
* services to junit5
* removed some junit4 engine imports
* updated some plugins test since these extend from testutil KV storage
* one more form of EPL v2
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
---------
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
* dagger component
* pushed up MetricsSystemModule
* passes around the BesuComponent as an interim application context till more things are managed by Dagger
---------
Signed-off-by: Justin Florentine <justin+github@florentine.us>
* Revert "Revert "Upgrade OpenTelemetry (#3675)" (#4031)"
This reverts commit 17de636fe2.
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Make sure we don't initialize the OpenTelemetry global singleton by mistake
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* disable global otel singleton explicitly
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* make sure to set GlobalOpenTelemetry at most once to avoid test failures
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* reset for tests
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* fix changelog
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Upgrade OpenTelemetry
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* remove a single sleep, poll with a for loop instead
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* use the new approach to send trace requests
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
Change the unit test execution to use the Junit5 JUnitPlatform. This
allows for a mix of junit 4 and junit 5 tests and for a gradual
migration to junit 5 instead of a big bang. One class depended on
junit 4 exceptions and was updated. Two tests depending on
native libraries fail gracefully on mac (and only mac).
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* Bump SLF4J version
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Replace log4j2 API with SLF4j API
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Replace usage of LogManager#getFormatterLogger
This is for keeping compatibility with SLF4J. If neccesary, a specific formatter can be created for the RlpBlockImporter class
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Unset the default logging value for the retesteth
This is because it's not possible to resolve the root logger level into a Log4J2 field
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Prevent creation of Logger context outside SLF4J
org.hyperledger.besu.cli.BesuCommand#setAllLevels was taken from
https://github.com/apache/logging-log4j2/blob/rel%2F2.17.1/log4j-core/src/main/java/org/apache/logging/log4j/core/config/Configurator.java#L309
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Add FATAL level deprecation message
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* [Sonar] Fix java:S2139
Exceptions should be either logged or rethrown but not both
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* [Sonar] Fix java:S3457
Printf-style format strings should be used correctly
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Add changelog
Signed-off-by: Diego López León <dieguitoll@gmail.com>
* Add tracing support for internals and JSON-RPC
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Remove rocksdb tracing as it slows down execution too much
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add B3 headers extraction on JSON-RPC requests
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Remove traces around trie tree as they slow down syncing significantly
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add tracing to fast sync pipeline
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add tracing for all pipelines
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Address code review
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add acceptance tests and break out the shaded dependency
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Fix tracer id
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Revert changes to trie
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Upgrade otel to latest, remove old tracing
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Code review comments
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add OTLP metrics support
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Don't automatically push to grpc. Only allow if pushEnabled is set to true
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Code review
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Missed refactoring
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add missing header
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Don't make otel depend on the push enabled flag
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Expose JUL logs to log4j2
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Code review fixes and make sure not to start push gateway if not set to prometheus
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* spotless
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Fix unit test
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* code review feedback
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add changelog entry
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
Removes as many Gradle 7.0 compatibility issues as possible
* `baseName` -> `archiveBaseName`
* `extension` -> `archiveExtension`
* `destinationDir` -> `destinationDirectory`
* `runtime` -> `runtimeOnly`
* Change some log4j-api and log4j-core dependencies
* Remove an unneeded and outdated plugin (`net.ltgt.apt`)
* tweak the plugin-api change detector's property annotations.
Warnings still exist with one external plugin used for license file
checking that we do not control the source code for.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Update dependencies to most current version
- except picocli which is a major version update
Alphabetize dependencies
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* adding in spdx-license-identifier & updated check for the same; removing license check from spotless
Signed-off-by: Joshua Fernandes <joshua.fernandes@consensys.net>
* Change CheckSpdxHeader to a task.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Ensure `plugin-api` module gets published at the correct maven path
* Move `plugins` to `plugin-api`
Signed-off-by: Edward Evans <edward.evans@consensys.net>
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Return plugin-api to the main repo
* Spotless
* Migrate all external plugin-api references to the project in this repo
* Add licence header
* Update repo reference for publish, even if commented
* Use real configuration for publishing plugin-api
This was tested with the
`:plugins:publishMavenJavaPublicationToMavenLocal` task and checking the
local Maven repo to make sure it was using the correct paths
Signed-off-by: Edward Evans <edward.evans@consensys.net>
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Some auto-generated maven names cause collissions when used with the gradle
publish plugin, namely `core` and `util`. Rename two jars that have such
a colission so the colission doesn't occur.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Make MetricCategory an interface so it isn't locked into the pantheon specific categories.
* Split categories into StandardMetricCategory and PanteonMetricCategory to separate the common and Pantheon specific categories.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Move RocksDBStats to it's own module. This also brings metrics to
metrics:core since none of our other module have nested modules but
they have peer modules.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Sometimes metrics are hard to poll (docker containers with varying ip
addresses). Because of that the push gateway exists. This extends the
metrics system to support push or pull mode for metrics (but not both
at the same time).
Three new flags
`--metrics-mode=`<`push`|`pull`> - Whether we are in pull mode (the default) where
prometheus is expected to poll or push mode where pantheon pushes to
a push gateway.
`--metrics-push-interval=`<_integer_> the frequency, in seconds, between pushes to
the push gateway. Only relevant in push mode
`--metrics-prometheus-job=`<_string_> The name of the job to report in the push gateway
Also, `--metrics-host=` and `--metrics-port=` gain new meaning in push mode. Instead of the
server they are opening up it is the host and the port of the push gateway it should push to.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
A service like the JSON-RPC service is opened up, only serving /metrics
requests in a file format for prometheus.
New CLI flags are --metrics-enabled and --metrics-listen, just like the
--rpc and --ws variants of the same.
--host-whitelist is respected the same as the JSON-RPC endpoint.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Metrics being captured initially:
Total number of peers ever connected to
Total number of peers disconnected, by disconnect reason and whether the disconnect was initiated locally or remotely.
Current number of peers
Timing for processing JSON-RPC requests, broken down by method name.
Generic JVM and process metrics (memory used, heap size, thread count, time spent in GC, file descriptors opened, CPU time etc).
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Update value returned by web3_clientVersion to be ethstats friendly
* Version part starts with a v
* SNAPSHOT builds report 32 bits of the git hash
* OS and architecture are sniffed out and reported
* JVM version and branding are sniffed out and reported
Example values (not all real):
pantheon/v0.9.0-dev-f800a0b1/osx-x86_64/oracle-java-1.8
pantheon/v0.9.0-dev-27960b57/osx-x86_64/zulu-java-11
pantheon/v0.9.0/linux-arm64/openjdk-java-12
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Fabio Di Fabio
Matilda-Clerke
Sally MacFarlane
Nischal Sharma
Justin Florentine
Antoine Toulme
Sally MacFarlane
Danno Ferrin
Diego López León
Antoine Toulme
Joshua Fernandes
Edward
mark-terry
Ratan Rai Sur
Danno Ferrin
Adrian Sutton
Danno Ferrin
Adrian Sutton
Chris Mckay
PegaSys Admin