95 Commits (7e395348b28776049cb6db6ca0c62bcbe85d65b3)

Author	SHA1	Message	Date
goodsoft	ab6111b466	Chunk JSON-RPC batches in case connection times out ... There exists code to ensure long-running batch requests are split into smaller batches in case we receive 504 (Gateway Timeout) errorfrom loadbalancer. This PR adds handling of batches in the same manner in case there either is no balancer, or its timeouts are configured larger than indexer's, and thus the HTTP client library itself times out.	6 years ago
goodsoft	ea3af5e299	Use transaction-level fetch_internal_transactions for Geth ... Internal transaction fetcher is split into two APIs: * `fetch_block_internal_transactions` is used for Parity * `fetch_internal_transactions` is used for the rest of variants `Indexer.InternalTransaction.Fetcher` is updated to take care of that.	6 years ago
Paul Tsupikoff	e13f6f8a8e	Use trace_replayBlockTransactions API for faster tracing	6 years ago
goodsoft	3060264bfb	Fix test coverage	6 years ago
Paul Tsupikoff	2a8def19e2	Allow executing smart contract functions in arbitrarily sized batches ... # Problem There are a lot of `getBalanceOf` token function calls during indexing blockchain. Each of them is sent as a separate JSONRPC request, which creates a lot of unnecessary overhead and clogs archive node workers, which could otherwise be busy performing other JSONRPC queries. Moreover, function name is used as the `id` parameter of JSONRPC queries, which doesn't allow us to simply put multiple requests into one batch. # Solution * Change `EthereumJSONRPC.execute_contract_functions` to accept a list of arbitrary function calls and return a list of results. * Handle `id` field generation in the `EthereumJSONRPC` package instead of leaking implementation details to `Explorer.SmartContract.Reader`. * Change `Token.BalanceReader.get_balance_of` to accept a list of parameter sets and return a list of balances. * Make one batch request instead of launching multiple concurrent tasks in `Indexer.TokenBalances.fetch_token_balances_from_blockchain` # Notes * Protocol details of calling smart contract functions is now fully encapsulated in `EthereumJSONRPC` package. * `SmartContract.Reader.query_contract` is left for backwards compatibility with other existing applications of smart contract requests, but can be fully deprecated at some later time.	6 years ago
Ayrat Badykov	d1ec793889	mix format	6 years ago
Ayrat Badykov	6f1a521ca7	do not try to convert nils in logs	6 years ago
Ayrat Badykov	f7456db04c	fix receipts for pending transactions	6 years ago
Luke Imhoff	229df065b4	Failing regression tests for #1337	6 years ago
Luke Imhoff	8135fcafbd	Allow lists of block numbers instead of ranges for beneficiares ... Simplifies calling as numbers don't need to grouped into ranges when getting numbers from database.	6 years ago
Luke Imhoff	5a576fcb04	Update to Phoenix 1.4 + Cowboy 2 for HTTP 2 ... Fixes #1322	6 years ago
Ayrat Badykov	fbe2e80c91	skip parity because it's not synced to the required block	6 years ago
Ayrat Badykov	3d6412664a	set actual code from mainnet	6 years ago
Ayrat Badykov	e1284326bf	fix credo	6 years ago
Ayrat Badykov	8792417110	fix parity and geth json rpc tests	6 years ago
Luke Imhoff	719ff778a8	Update apps/ethereum_jsonrpc/test/ethereum_jsonrpc_test.exs ... Co-Authored-By: ayrat555 <ayratin555@gmail.com>	6 years ago
Ayrat Badykov	e40e4dca27	fetch codes	6 years ago
Andrew Cravenho	f3ab5e4c58	Update Parity Sokol node tests	6 years ago
Luke Imhoff	bf50a03070	Change RollingWindowTest to async: false ... The same named process is used for all tests and they use the same key, so they interfere with each other.	6 years ago
William Sanches	512511aee6	Add better test coverage	6 years ago
William Sanches	a754a72390	Add block_reward import	6 years ago
Luke Imhoff	3499155cdb	Partial retries for fetch_blocks_by_range ... Means that both Block.Fetchers, realtime and catchup, now handle fetch errors. The Block.Catchup.Fetcher will retry those blocks that failed to fetch while importing the rest that did fetch instead of refetching the whole batch.	6 years ago
Luke Imhoff	d43a8c6133	Partial retry of batches of blocks by hash	6 years ago
Luke Imhoff	ad1119eb4c	Fetched Beneficiaries partial batch retries	6 years ago
Luke Imhoff	3efa2d9996	Partial retries for CoinBalance.Fetcher	6 years ago
Andrew Cravenho	a64a1dc6bb	Update sokol node tests	6 years ago
Andrew Cravenho	063c393281	Update documentation replacing old sokol nodes	6 years ago
Sebastian Abondano	345bec4a7a	fetch_beneficiaries/2 works with all reward types ... Why: * For the indexer to update block reward contract beneficiaries for all types (e.g. block, external, uncle, etc). For details on Parity's reward types please see: 0f90696528/rpc/src/v1/types/trace.rs (L299-L308) * Issue link: https://github.com/poanetwork/blockscout/issues/767	6 years ago
Luke Imhoff	e4b7a4ee5f	Geth internal transactions ... Get internal transactions using debug_traceTransaction and a custom tracer. The tracer was based on the callTracer that ships with Geth, but rewritten to break it into more functions with more explicit purpose and to patch some holes in callTracer: * All calls have gas and gasUsed, which patches the hole in callTracer for handling `DELEGATECALL`. * All calls have value, which patches the hole in callTracer for handling `DELEGATECALL` and `STATICALL`. Since the tracer needed to be resubmitted in whole anyway, it was further customized to match the format needed for internal transactions: * Instead of returning the `op` as `type`, the `type` and `callType` used for internal transactions is returned. * Instead of returning a call tree, the tree is walked in pre-order, displaying the parent call before its nested calls to match the flat order returned from parity's trace_transaction.	6 years ago
Alex Garibay	bd8b5d6864	Remove unused test functions	6 years ago
Alex Garibay	e23ebab069	Add transform functions for block indexing	6 years ago
Luke Imhoff	0ba5e692fc	Remove flaky test	6 years ago
Luke Imhoff	79dc13bf15	Remove flaky test	6 years ago
Luke Imhoff	2c12f62345	Remove flaky test	6 years ago
Luke Imhoff	396285f36f	Remove flaky test	6 years ago
Luke Imhoff	3bbe7b6e89	Remove flaky test	6 years ago
Gustavo Santos Ferreira	1b239dadcc	update tests	6 years ago
Alex Garibay	6d61420f32	Address code review feedback	6 years ago
Alex Garibay	b6bc6cc2f8	Add backoff for bad gateway errors	6 years ago
Alex Garibay	76970f6258	Finish documentation and tests for request backoffs	6 years ago
zachdaniel	75c88fac85	WIP: test rolling window and request coordinator	6 years ago
Alex Garibay	1f021a7e1f	Provide default 2nd arg for executing contract functions on Geth (#970)	6 years ago
Amanda Sposito	b86335cf34	Add 'transaction_index' in the fetchers to save at 'internal_transaction' ... Add 'transaction_index' in the fetchers to save at 'internal_transaction'	6 years ago
Sebastian Abondano	2b8ceb4e32	Update block reward contract beneficiary balances ... Why: * "Parity’s consensus engine allows using a smart contract for block reward calculation." - see https://wiki.parity.io/Block-Reward-Contract for details. This means that we need to use `trace_block` (https://wiki.parity.io/JSONRPC-trace-module#trace_block) to fetch the block reward contract beneficiaries and update their balances. Currently this doesn't happen so there are addresses (block contract reward beneficiaries) with inaccurate balances and missing rows in the `address_coin_balances` table. * Issue link: https://github.com/poanetwork/blockscout/issues/767 This change addresses the need by: * Adding `EthereumJSONRPC.fetch_beneficiaries/2` to help us fetch the block reward contract beneficiaries of a given block range. This only works with Parity. * Editing Indexer's dev and prod config by adding `trace_block` config to `:method_to_url`. * Adding `block_reward_contract_beneficiaries` to `Indexer.AddressExtraction` for it to know how to get addresses in this scenario. * Editing `Indexer.Block.Fetcher.fetch_and_import_range/2` to get block reward contract beneficiaries, create their addresses if they don't yet exist, and update their coin balances.	6 years ago
Luke Imhoff	1100f56fa1	Test WebSocketClient with cowboy_websocket_handler ... Allows to properly simulate TCP disconnect.	6 years ago
Luke Imhoff	b1f4696569	Reconnect, rerequest, and resubscribe when websocket disconnects ... * Use struct, `%EthereumJSONRPC.WebSocket.WebSocketClient.Options{}` to hold `web_socket_options`. * `%EthereumJSONRPC.WebSocket.Registration{}` * Record `t:EthereumJSONRPC.request/0` in `%EthereumJSONRPC.WebSocket.Registration{}`, so that the request can be re-requested if the websocket disconnects and reconnects. * Drop server-side `subscription_id` from `%EthereumJSONRPC.WebSocket.Registration{}` as it can retrieved from `request` now. * `EthereumJSONRPC.WebSocket.WebSocketClient` * Track whether the web socket is connected or not, so that requests aren't sent when in the disconnected state. This prevents double-sends as `handle_call` can be called before `onconnect`, which caused that first subscribe to be sent twice. * Instead of the subscriber knowing the server-side subscription id, an opaque `t:reference/0` is used, so that the reference is stable in the subscriber's `%Subscription{}`, but the subscription ID can change on reconnect, which means replacing `subscription_id_to_subscription` in `%WebSocketClient{}` with 3 maps 1. `subscription_id_to_subscription_reference` - used translate server-side, per-connection subscription to the stable reference in `subscription_reference_to_subscription` 2. `subscription_reference_to_subscription_id` - used to convert stable reference to subscription id to unsubscribe. 3. `subscription_reference_to_subscription` - used to get full Subscription, including the subscriber_pid for when subscription messages are delivered or the subscription needs to be resubscribed using the `web_socket_options`. * `ondisconnect`, `:reconnect` immediately. * When disconnecting, tell unsubscribers they were successful. * Hold any json_rpc or subscribe requests to retry on reconnect. * Hold any subscriptions to resubscribe on reconnect. * `onconnect` * re-request the saved json_rpc and subscribe requests. * issue new requests to re-establish the subscriptions that had already succeeded in their subscribe requests. * `%EthereumJSONRPC.Subscription{}` * Replace `id`, which did match the server created ID, with a `t:reference/0`, which allows for indirection and for the server created ID to change on reconnect while the subscriber does not need to care about the reconnections. * `EthereumJSONRPC.WebSocket` * `options` is switched from a Keyword.t to a struct, `t`, so that it is easier to pattern match for the reconnect logic in `WebSocketClient`. * `web_socket_options` is no longer a generic `term` because the code did require it to be a map with a `:web_socket` key for the `t:GenServer.server/0`.	6 years ago
Luke Imhoff	6b53d96075	_optons -> _options	6 years ago
Luke Imhoff	0b5729c144	correclty -> correctly	6 years ago
Luke Imhoff	96d5aea311	Block.SecondDegreeRelation ... The database table block_second_degree_relations contains links between the `uncle_hash` of the uncle and the `nephew_hash` of the block that said `uncle_hash` was an uncle. It also has a `uncle_fetched_at` that is `null` until the `uncle_hash` is confirmed as fetched in the database. This does not contain such a fetcher.	6 years ago
Luke Imhoff	48b302f935	Return uncles from EtherumJSONRPC.Block.elixir_to_params	6 years ago