When the prometheus service restarts, connections can be left open and never closed. These connections should close after an idle timeout to avoid causing issues with running out of ports.
Signed-off-by: Paul Harris <paul.harris@consensys.net>
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
The labelled gauge functionality will allow monitoring of gauge type data without creating many gauges, which can be particularly handy in tracking related data.
Signed-off-by: Paul Harris <paul.harris@consensys.net>
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Add stratum metrics
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Use a more descriptive name for the target difficulty divisor
Use a double gauge so all difficulty values can be represented
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* rename field
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add tracing support for internals and JSON-RPC
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Remove rocksdb tracing as it slows down execution too much
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add B3 headers extraction on JSON-RPC requests
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Remove traces around trie tree as they slow down syncing significantly
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add tracing to fast sync pipeline
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add tracing for all pipelines
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Address code review
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add acceptance tests and break out the shaded dependency
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Fix tracer id
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Revert changes to trie
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Upgrade otel to latest, remove old tracing
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Code review comments
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add OTLP metrics support
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Don't automatically push to grpc. Only allow if pushEnabled is set to true
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Code review
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Missed refactoring
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add missing header
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Don't make otel depend on the push enabled flag
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Expose JUL logs to log4j2
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Code review fixes and make sure not to start push gateway if not set to prometheus
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* spotless
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Fix unit test
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* code review feedback
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* Add changelog entry
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
* rename more whitelist occurrences; change allowlisted to allowed and reword where we ended up with allowlisting
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
Upgrade to ErrorProne 2.4.0
* public constructors on abstract classes are removed
* Javadoc must have meaningfull documentation
* lambdas should not be variables
* Added to the list of confusing inner class names (Entry and Type)
* no assert keyword in tests
* Obsolete JDK classes produce errors now
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
* replace whitelist/blacklist with allowlist/denylist and support both versions in CLI options
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
When we deny a connection based on HTTP hostname log to trace the
rejected value.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Co-authored-by: Usman Saleem <usman@usmans.info>
Occasionally the other side of an HTTP connection will drop the
connection before we are done processing. This results in a harmless
but annoying exception showing up in the log. If we check before we
write we won't get that exception.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Removes as many Gradle 7.0 compatibility issues as possible
* `baseName` -> `archiveBaseName`
* `extension` -> `archiveExtension`
* `destinationDir` -> `destinationDirectory`
* `runtime` -> `runtimeOnly`
* Change some log4j-api and log4j-core dependencies
* Remove an unneeded and outdated plugin (`net.ltgt.apt`)
* tweak the plugin-api change detector's property annotations.
Warnings still exist with one external plugin used for license file
checking that we do not control the source code for.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
This was turned off a number of months ago because of long build times.
However it appears either because of code structure changes or fixed
errorprone that this finishes in a reasonable time now.
Code that violated this check is also fixed.
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Update dependencies to most current version
- except picocli which is a major version update
Alphabetize dependencies
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
It was identified that when both KV_ROCKSDB metrics and Privacy were enabled,
that Besu failed to start due to a naming collision in Metrics Collators - this was ultimately due to Besu having 2 keyvalue stores - one for public state, and another for private state - and _both_ using the same metrics.
To overcome this issue, the metrics used in the private kv store are prefixed with the word "private".
Signed-off-by: Trent Mohay <trent.mohay@consensys.net>
[PIE-1858] Added functionality to register custom metrics categories and exposed some PoA data for metrics.
Signed-off-by: Mark Terry <mark.terry@consensys.net>
* adding in spdx-license-identifier & updated check for the same; removing license check from spotless
Signed-off-by: Joshua Fernandes <joshua.fernandes@consensys.net>
* Change CheckSpdxHeader to a task.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Remove metrics from plugin registration. Doing it this way here always results in a
NoOpsMetrics being inited because CLI options have not been parsed.
Add an acceptance test for metrics. This is what should have been in place and
would have detected metrics not working earlier.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Ensure `plugin-api` module gets published at the correct maven path
* Move `plugins` to `plugin-api`
Signed-off-by: Edward Evans <edward.evans@consensys.net>
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Return plugin-api to the main repo
* Spotless
* Migrate all external plugin-api references to the project in this repo
* Add licence header
* Update repo reference for publish, even if commented
* Use real configuration for publishing plugin-api
This was tested with the
`:plugins:publishMavenJavaPublicationToMavenLocal` task and checking the
local Maven repo to make sure it was using the correct paths
Signed-off-by: Edward Evans <edward.evans@consensys.net>
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Some auto-generated maven names cause collissions when used with the gradle
publish plugin, namely `core` and `util`. Rename two jars that have such
a colission so the colission doesn't occur.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Mark Sweep Pruner
* add `unload` method on Node interface, which is a noop everywhere but on `StoredNode`
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Display World State Sync Progress in Logs
- compute estimated world state completion
- display estimated world state completion in the logs
- `CompleteTaskStep` now have access to a `LongSupplier` to retireve the number of pending requests
- use a `RunnableCounter` to trigger displaying every 1000 requests completed
- only show a new log when the estimation changes
- added test to check the estimation computation
* report using raw ratio rather than percentage
* resolve PR discussion
* rename getTotal to get
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* use MockitoJUnitRunner to run TransactionsMessageProcessor tests
* implement metrics counter and logging
* (format) replace statement by lambda expression
* (doc) fix javadoc
* spotless apply
* remove modulo and reset step counter to zero
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Make MetricCategory an interface so it isn't locked into the pantheon specific categories.
* Split categories into StandardMetricCategory and PanteonMetricCategory to separate the common and Pantheon specific categories.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Don't copy collections if we don't need to. Change types higher up if needed.
* Don't use Guava's Object.equal, use Java's Objects.equals.
** add errorprone test to enforce the banning of Guava's Objects class.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Change all Stream<?> getX() and Stream<?> x() methods to Stream<?> streanX methods, such as `Stream<Peer> streamIdlePeers()`
* Update coding conventions to reflect this.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Move RocksDBStats to it's own module. This also brings metrics to
metrics:core since none of our other module have nested modules but
they have peer modules.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Giuseppe Bertone
Paul Harris
Antoine Toulme
Danno Ferrin
Antoine Toulme
Adrian Sutton
Sally MacFarlane
Trent Mohay
Lucas Saldanha
mark-terry
Joshua Fernandes
CJ Hare
Danno Ferrin
Edward
Ratan Rai Sur
mbaxter
Abdelhamid Bakhta
Chris Mckay
Danno Ferrin