* [PRIV-41] Use metrics system for private state db
- Use PrivacyParametersBuilder to build PrivacyParameters
- refactor PrivacyParameters to expose default options
- refactor test builders to use PrivacyParameters.DEFAULT
- Use URI in Enclave
* Fix: enclave tests from bad merge
* Fix privacy acceptance tests after db configuration changes
* Switch to use nested class for PrivacyParametersBuilder
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Adds handling for options that need to be configurable for tuning and performance but in general are not long-term supported configuration options.
Options that are added via a mixin, are hidden, and that start with `--X` are considered 'Unstable Options.' These options do not show up via normal help facilities but are enumerated via a `XHelp` CLI invocation. These options are unstable, unsupported, and undocumented. They may be added, removed, or modified without prior warning or notice. Use them at your own risk.
For example, the `--Xrocksdb-max-open-files` flag can be set via CLI and can increase the number of max open files RocsDB uses. On a 8GB memory system the default is appropriate, but on a 128GB memory system a much larger number may improve performance. Normal operation scenarios will not need to modify this flag.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
- add `--fast-sync-min-peers` hidden cli option
- add `--fast-sync-max-wait-time` hidden cli option
- update `FastSyncActions` to handle no timeout behaviour (when timeout set to 0 or no timeout set then we just trigger asynchronously the `waitPeersTask`)
- update unit tests mock related to sync config builder
- add fast sync options in `everything_config.toml` and `partial_config.toml`
- re enable PantheonCommandTest.syncModeOptionMustBeUsed
- add new flags in `noOverrideDefaultValuesIfKeyIsNotPresentInConfigFile` and `overrideDefaultValuesIfKeyIsPresentInConfigFile` tests
fix PAN-2265
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Also increases the number of requests without progress before considering the download stalled.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Added ability to subscribe to dropped transactions from the transaction pending pool.
Implemented subscription webservice to support this.
* Added metrics to the pending transactions, tracking the number of local and remote transactions in the pool.
* Converted listener management in pending transactions to use the Subscribers util object.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
The 25% of maximum peers is fairly arbitrary but is deliberately above zero to ensure we contribute something to the overall network even while catching up.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
When receiving an AddPeer, or when loading the static-nodes.json file, pantheon
should not accept its own NodeID as a valid connection (however, this should not invalidate the static-nodes file).
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
To prevent unexpected behaviour, if a static node is not in the nodes whitelist
(when permissioning is enabled) pantheon will not start (rather than starting
with only whitelisted static nodes).
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Synchronizer returns false if it is in sync
* expand RunnerTest to test eth_syncing behaviour
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Maintain a staticnodes.json
The staticnodes.json file resides in the pantheon data directory,
is parsed at startup to determine which peers can be connected
to (aside from bootnodes), and is updated whenever addPeer/removePeer
RPC is invoked.
This file is not updated when finding neighbour peers (ONLY on RPC
call).
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Biggest change is that UnusedVariable and UnusedMethod went to WARN by
default. Since our build is a no warning build this means we either need
to turn them off or fix them. I mostly opted for the latter. Test code
was mostly fixed, unused loggers were deleted, and other shipped code
was mostly suppressed.
Two less noisy fixes to not use `SortedSet` and to use zero based
comparable results instead of -1, 0, and 1. Also a compiler nit in
errorprone was suppressed, per the description it won't affect us.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
change --bootnodes option to a method to be able to handle the check and error
fixes PAN-2387 and PAN-2015
Handling this as a method enables to keep the parameter as a string and not a
strongly typed value. It then is always accepted by the command line so it's
always considered as a param and doesn't raise an unknown param error but also
immediately checked against the enode pattern and intelligible matching error
is displayed.
Converter is removed and only a simple lambda is used in a try block
leaving the handling of the error message close to the option thus making code
easier to understand. Anyway, as we use a Collection of Strings as input no
converter is needed anymore.
Update and add tests to check non regression and handle the new behaviour.
Also this change can change the order in which options are handled in the spec.
So tests are updated to check for the presence of options but not specifying a
static order. verifyOptionsConstraintLoggerCall now loops through the options
to check they are contained in the error string but not in a specific order.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* refactor NodeWhitelistController -> NodeLocalConfigPermissioningController
* added smart contract permissioning config
* only use SyncStatusProvider if Smart Contract Permissioning enabled
* check we are not adding self enode
* local permissioning controller isPermitted() should return true if either source OR destination is permitted
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
start() guarantees ports are open before it returns but doesn't block until shutdown. Then awaitStop can be used by callers who want to block until Pantheon exits.
Fixes intermittency regression in RunnerTest with thanks to @usmansaleem for reporting via #1067.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* Store the instant that a pending transaction is added to the txpool, and expose this out through the api.
* Clock.systemUTC only used once.
* TestClock class added with a factory method for a fixed clock.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
With re-enabling IBFT 1.0, an error was introduced whereby the Eth/62
and ETH/63 protocols did not have a mapped ProtocolManager (resulting
in an eror at runtime).
This change ensures all protocols are handled by the Istanbul64
ProtocolManager.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
fixes PAN-2259 prevent public key export subcommands to create the database
Creation of the database is useless at this stage, so don't create it as it's
usually necessary to remove it if created as we want to use the output of
this subcommand to change the genesis that makes the created database useless.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
With the removal of the ottoman command line option, Pantheon lost the
ability to synchronise with Quorum.
This change allows Pantheon to synchronise with quorum via the
IbftLegacyPantheonController, which uses an Instanbul64 protocol
manager in place of the EthProtocolManager.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
The P2P and discovery ports are now always dynamically assigned (set to port 0 when starting pantheon) to guarantee they always select an available port.
This however means you can't determine the ports that will be used or the enode URL for a node before it has started. The main effect of this is that nodes have to use the peer discovery process to discover each other rather than having every node in the network in the boot nodes list. A single node is started first with no bootnodes, then other nodes are started with that node as their bootnode and use peer discovery to find the other nodes as they start.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Start P2P network synchronously so the ports are guaranteed to be known before we write the ports file
Include the P2P TCP port in ports file even when peer discovery is disabled.
Load information from the advertised peer rather than the discovery listening socket.
Fix admin_nodeInfo to include the ?discport param in the enode URI when the discovery port differs from the P2P port.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Implement admin_nodeInfo RPC call. This involved bringing data from the
ethNetworkConfig and genesis json into the rpc call, so it had a lot of
testing side effects.
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
* [PAN-2341] Created mechanism to publish sync status
* Refactoring WebSockets syncing subscription to respond to SyncStatus events
* Removing rpc-ws-refresh-delay option
* Removing ws-refresh-delay option from ATs
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
[PAN-2369] RunnerTest fail on Windows due to network startup timing issue
* Forces RunnerTest to wait for the P2PNetwork to start and actually configure it's ports before asking for them.
* Re-Enables RunnerTest.fullSyncFromGenesis()
* Re-Enables RunnerTest.fastSyncFromGenesis()
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Abdelhamid Bakhta
Antony Denyer
Danno Ferrin
Chris Mckay
Adrian Sutton
Rob Dawson
Trent Mohay
Puneetha Karamsetty
Lucas Saldanha
Sally MacFarlane
mbaxter
Nicolas MASSART
Eric Kellstrand