126 Commits (d1a018710c922a6ed01f8d102951284d5d041860)

Author	SHA1	Message	Date
Konstantin Zolotarev	c1d1385abb	Added docker files	6 years ago
Konstantin Zolotarev	a9fd475df4	Unified config variables for all JSON RPC variants	6 years ago
Luke Imhoff	cae25195e5	Indexer.Memory.Monitor ... `Indexer.Memory.Monitor` checks if the BEAM memory usage exceeds a set limit (defaults to 1 GiB) and if it does, it asks the process with the most memory that is registered as shrinkable to shrink.	6 years ago
Sebastian Abondano	1cbaafd894	Adds log rotation ... Why: * To limit the total size of the logs retained while still allowing analysis of recent events. * Issue link: n/a This change addresses the need by: * Editing the logger backends to do file rotation every 50mb and keep the last 19 log files per logger backend. 50mb * 20 = ~1gb	6 years ago
Sebastian Abondano	2b8ceb4e32	Update block reward contract beneficiary balances ... Why: * "Parity’s consensus engine allows using a smart contract for block reward calculation." - see https://wiki.parity.io/Block-Reward-Contract for details. This means that we need to use `trace_block` (https://wiki.parity.io/JSONRPC-trace-module#trace_block) to fetch the block reward contract beneficiaries and update their balances. Currently this doesn't happen so there are addresses (block contract reward beneficiaries) with inaccurate balances and missing rows in the `address_coin_balances` table. * Issue link: https://github.com/poanetwork/blockscout/issues/767 This change addresses the need by: * Adding `EthereumJSONRPC.fetch_beneficiaries/2` to help us fetch the block reward contract beneficiaries of a given block range. This only works with Parity. * Editing Indexer's dev and prod config by adding `trace_block` config to `:method_to_url`. * Adding `block_reward_contract_beneficiaries` to `Indexer.AddressExtraction` for it to know how to get addresses in this scenario. * Editing `Indexer.Block.Fetcher.fetch_and_import_range/2` to get block reward contract beneficiaries, create their addresses if they don't yet exist, and update their coin balances.	6 years ago
Konstantin Zolotarev	0604f7a234	Fixes for pr comments	6 years ago
Konstantin Zolotarev	f89a27adf4	Fixes for ganache integration	6 years ago
Nam Chu Hoai	d17917ce30	Allow unsecure WebSocket connections ... Some setup like ours will have the WebSocket connection in a private network to a local node and thus do not require a secure connection. This change allows to specify ws:// instead of wss:// endpoints as well as reading from the ENV var for the geth config of the indexer Co-authored-by: Luke Imhoff <luke.imhoff@dockyard.com>	6 years ago
Andrew Cravenho	9892d1bd17	Add env variables to the production parity files	6 years ago
Igor Florian	99012b2c4c	Add a new log file for token_balances fetching errors	6 years ago
Luke Imhoff	9a8cdc4031	Use Path.absname so that log paths are stable during reload ... Fixes #628 The Phoenix.CodeReloader causes the file loggers to restart and reload their config, but when they reload, the current working directory has shifted from the project root to the OTP app root, so it produces log files under each OTP app (`apps/APP_NAME/logs`). We don't want this, so compute the absolute path in the config files, so that the loaded `Application` environment has an absolute path in it that can't shift when the working directory changes.	6 years ago
Luke Imhoff	fdd1090c45	Re-enable Ecto loggers ... Allow Ecto loggers to log to the application-specific log **file**, `log/<Mix.env>/ecto.log` but raise `:console`'s level to `:info`, so that Ecto output does not flood `iex` and make it unusable.	6 years ago
Luke Imhoff	c032ea2291	Improved logging ... Have separate loggers for errors and per-application. * `:console` will continue to have all applications and levels. Level is - `:debug` for `dev` - `:warn` for `test` - `:info` for `prod` * `:error` is only for errors across all environments. It uses `LoggerFileBackend` and writes to `logs/<Mix.env>/error.log`. * `:block_scout_web` is only when `:application` metadata is `:block_scout_web`. Level is - `:debug` for `dev` - `:warn` for `test` - `:info` for `prod` It uses `LoggerFileBackend` and writes to `logs/<Mix.env>/block_scout_web.log`. * `:ethereum_jsonrpc` is only when `:application` metadata is `:ethereum_jsonrpc`. Level is - `:debug` for `dev` - `:warn` for `test` - `:info` for `prod` It uses `LoggerFileBackend` and writes to `logs/<Mix.env>/ethereum_jsonrpc.log`. * `:explorer` is only when `:application` metadata is `:exploer`. Level is - `:debug` for `dev` - `:warn` for `test` - `:info` for `prod` It uses `LoggerFileBackend` and writes to `logs/<Mix.env>/explorer.log`. * `:indexer` is only when `:application` metadata is `:indexer`. Level is - `:debug` for `dev` - `:warn` for `test` - `:info` for `prod` It uses `LoggerFileBackend` and writes to `logs/<Mix.env>/indexer.log`.	6 years ago
Luke Imhoff	cddd810a58	Use :websocket_client for EthereumJSONRPC.WebSocket ... :websocket_client can properly use :ssl_verify_hostname.verify_fun/3 for :verify_fun and allow for the handshake to both be secure and pass for infura.	6 years ago
Luke Imhoff	c8c53405fc	Subscribe to newHeads for realtime indexer ... Separate Catchup and Realtime indexer's fetching, so that Catchup continues to use the old polling, `EthereumJSONRPC.json_rpc` while Realtime subscribes with `EthereumJSONRPC.subscribe`. Because Realtime gets notified of new blocks through the subscription, it no longer has a Sequence, so Sequence has been removed from `%BlockFetcher{}`. There is also no block concurrency or block batch size for Realtime, so that was removed from `%BlockFetcher{}` too and moved to `%BlockFetcher.Catchup{}`. Because Realtime requires a websocket to be connected, there is now an `Indexer.BlockFetcher.Realtime.Supervisor` that supervises the `Indexer.BlockFetcher.Realtime.WebSocket` and `Indexer.BlockFetcher.Realtime`. `Indexer.BlockFetcher.Realtime` because it is waiting on notifications from is subscription becomes a `GenServer` insteado of a `Task`, so it can be supervised normally. BlockFetcher.Supervisor becomes a normal `Supervisor` of `Catchup.Supervisor` and `Realtime.Supervisor`. As Realtime is using `EthereumJSONRPC.subscribe`, `subscribe_named_arguments` has been added as a sibling of `json_rpc_named_arguments` in the `:indexer` `config`s.	6 years ago
Peter Romfeld	d9e37c44a6	fix formatting	6 years ago
Peter Romfeld	b473420df3	use ETHEREUM_JSONRPC_HTTP_URL instead	6 years ago
Peter Romfeld	2ddc1b9747	get geth_url from environment and fix `variant` for {dev,prod}.exs	6 years ago
Luke Imhoff	fe49a753d3	block_rate -> block_interval ... block_rate is measured in milliseconds / block, but a rate would blocks / unit-time, so rename block_rate to block_interval to reflect it is the interval between blocks.	6 years ago
Amanda Sposito	3335817d90	Fix prod config file to use prod config instead of dev	6 years ago
Luke Imhoff	8f827ecfa6	Use Mox by default for local integration tests ... How to run without Mox is in the root `README.md`. Run all combinations of Variant and Transport on CircleCI to ensure that Mox matches HTTP behavior, so Mox can be used for fast local testing: * Geth, HTTP * Geth, Mox * Parity, HTTP * Parity, Mox Using Mox locally also lessens load on Sokol and the Infura API key the tests use for Geth under the assumption that we do far more local testing than CI testing. May help with #340, but does not fix the root cause, which is now known as `Sequence` not chunking ranges correctly on retry.	6 years ago
Luke Imhoff	466c1e76f6	Request transport module and options for all JSONRPC calls ... To lessen load on Sokol chain nodes during testing and to allow substituting different transports than HTTP, such as IPC, in the future, pass the transport module and its options to any call to `EthereumJSONRPC.json_rpc`. This will allow us to use a `mox` module in local testing, so we only do true chain integration testing in CircleCI.	6 years ago
Luke Imhoff	6b3fd80916	Extract Explorer.Indexer to Indexer ... Resolves #304	7 years ago
Luke Imhoff	27b4bf62d5	mix new indexer --sup	7 years ago