288 Commits (8fd8ba62b70659dadbb08e4dd3a72c06b9169854)

Author	SHA1	Message	Date
William Sanches	a754a72390	Add block_reward import	6 years ago
zachdaniel	5d12613f70	feat: support tracing via spandex	6 years ago
Sebastian Abondano	1bdb8c1166	Ganache ignores internal transactions ... Why: * We were using the `EthereumJSONRPC.Geth` variant for Ganache but we recently learned that it doesn't support the `tracer` option for [debug_traceTransaction](https://github.com/ethereum/go-ethereum/wiki/Management-APIs#debug_tracetransaction). This `tracer` option is what we currently use to fetch internal transactions for Geth. This was the cause of errors as shown on issue 1152 (link below). * Issue link: https://github.com/poanetwork/blockscout/issues/1152 This change addresses the need by: * Creating `EthereumJSONRPC.Ganache`. It ignores fetching beneficiaries, internal transactions, and pending transactions. * Editing ganache config to use the `EthereumJSONRPC.Ganache` variant instead of `EthereumJSONRPC.Geth`. * Updating inaccurate doc in `EthereumJSONRPC.Geth`. It previously said fetching internal transactions was not supported for Geth which is inaccurate.	6 years ago
Sebastian Abondano	f0de6f4b09	Handles Ganache transaction input bug ... Why: * To be able to index transactions, we need to add special handling for a bug in Ganache in which transaction inputs are returned as invalid `DATA` types. For details please see: https://github.com/trufflesuite/ganache/issues/997 * Issue link: https://github.com/poanetwork/blockscout/issues/1068 This change addresses the need by: * Editing `EthereumJSONRPC.Transaction` to convert incorrectly formatted transaction `input`s of `0x0` to `0x`.	6 years ago
Luke Imhoff	a599aaff06	Extract EthereumJSONRPC.Block.ByTag	6 years ago
Luke Imhoff	3499155cdb	Partial retries for fetch_blocks_by_range ... Means that both Block.Fetchers, realtime and catchup, now handle fetch errors. The Block.Catchup.Fetcher will retry those blocks that failed to fetch while importing the rest that did fetch instead of refetching the whole batch.	6 years ago
Luke Imhoff	d43a8c6133	Partial retry of batches of blocks by hash	6 years ago
Luke Imhoff	ad1119eb4c	Fetched Beneficiaries partial batch retries	6 years ago
Luke Imhoff	3efa2d9996	Partial retries for CoinBalance.Fetcher	6 years ago
Andrew Cravenho	a64a1dc6bb	Update sokol node tests	6 years ago
Andrew Cravenho	063c393281	Update documentation replacing old sokol nodes	6 years ago
Sebastian Abondano	345bec4a7a	fetch_beneficiaries/2 works with all reward types ... Why: * For the indexer to update block reward contract beneficiaries for all types (e.g. block, external, uncle, etc). For details on Parity's reward types please see: 0f90696528/rpc/src/v1/types/trace.rs (L299-L308) * Issue link: https://github.com/poanetwork/blockscout/issues/767	6 years ago
Luke Imhoff	3973e6607d	Support Parity 2.1 pending tag output ... It returns `number: null` instead of `null` or an error.	6 years ago
Luke Imhoff	ea00df0cd1	Include input when converting elixir call traces	6 years ago
Luke Imhoff	60b02b4604	Add trace addresses to Geth tracer output ... Andrew explained that the trace address from Parity is the index of each nested call. I can compute that as I flatten the tree into a sequence.	6 years ago
Luke Imhoff	e4b7a4ee5f	Geth internal transactions ... Get internal transactions using debug_traceTransaction and a custom tracer. The tracer was based on the callTracer that ships with Geth, but rewritten to break it into more functions with more explicit purpose and to patch some holes in callTracer: * All calls have gas and gasUsed, which patches the hole in callTracer for handling `DELEGATECALL`. * All calls have value, which patches the hole in callTracer for handling `DELEGATECALL` and `STATICALL`. Since the tracer needed to be resubmitted in whole anyway, it was further customized to match the format needed for internal transactions: * Instead of returning the `op` as `type`, the `type` and `callType` used for internal transactions is returned. * Instead of returning a call tree, the tree is walked in pre-order, displaying the parent call before its nested calls to match the flat order returned from parity's trace_transaction.	6 years ago
Luke Imhoff	9cce7ab661	Adopt EIP-6 ... Geth has EIP-6, while Parity does not, but the Ethereum Yellow Paper includes EIP-6, so it is better to use EIP-6 when looking up community docs.	6 years ago
Alex Garibay	e23ebab069	Add transform functions for block indexing	6 years ago
Konstantin Zolotarev	802cb2d838	Cleanup	6 years ago
Konstantin Zolotarev	da47588cb9	Fix ping/pong from ganache ws implementation	6 years ago
Luke Imhoff	6134d5ad07	credo 0.9.2 => 0.10.2	6 years ago
Gustavo Santos Ferreira	1b239dadcc	update tests	6 years ago
Gustavo Santos Ferreira	e2b20f95bb	add new field to Receipt elixir_to_params	6 years ago
zachdaniel	8efb8610c5	fix: sleep when throttled to prevent stampedes	6 years ago
Sebastian Abondano	8ac6d8d0a4	Fix error when indexing Wanchain ... Why: * When attempting to index Wanchain we were seeing this error: ``` ** (FunctionClauseError) no function clause matching in EthereumJSONRPC.Transaction.entry_to_elixir/1 (ethereum_jsonrpc) lib/ethereum_jsonrpc/transaction.ex:260: EthereumJSONRPC.Transaction.entry_to_elixir({"txType", "0x1"}) ``` After looking into this it looks like Wanchain returns a "txType" of 1 for normal transactions and 6 for privacy transactions. It's not clear at this time if this is something we want BlockScout to keep track of so we'll be ignoring it for now. * Issue link: https://github.com/poanetwork/blockscout/issues/994 This change addresses the need by: * Editing `EthereumJSONRPC.Transaction.entry_to_elixir/1` to accept a key of "txType" without generating a `FunctionClauseError`.	6 years ago
Alex Garibay	bbf88ca947	Fix typo	6 years ago
Alex Garibay	6d61420f32	Address code review feedback	6 years ago
Alex Garibay	b6bc6cc2f8	Add backoff for bad gateway errors	6 years ago
Alex Garibay	76970f6258	Finish documentation and tests for request backoffs	6 years ago
zachdaniel	75c88fac85	WIP: test rolling window and request coordinator	6 years ago
zachdaniel	bd31796d40	WIP: Finish/clean up rolling window tracker	6 years ago
Alex Garibay	be7271a8d9	WIP: clean up request coordinator with functionality and docs	6 years ago
zachdaniel	9410a86605	WIP: application + request coordinator cleanup	6 years ago
zachdaniel	961a2fb03c	WIP: request coordinator + starting rolling window	6 years ago
zachdaniel	697be65e82	WIP: Rolling window event counter.	6 years ago
Alex Garibay	632981e16b	WIP: generalize rolling window	6 years ago
zachdaniel	0e3fdc3995	WIP: Add rolling window rate limiter + retry logic	6 years ago
Alex Garibay	1f021a7e1f	Provide default 2nd arg for executing contract functions on Geth (#970)	6 years ago
Konstantin Zolotarev	1674540779	Fix ping/pong from ganache ws implementation	6 years ago
Amanda Sposito	b86335cf34	Add 'transaction_index' in the fetchers to save at 'internal_transaction' ... Add 'transaction_index' in the fetchers to save at 'internal_transaction'	6 years ago
Sebastian Abondano	2b8ceb4e32	Update block reward contract beneficiary balances ... Why: * "Parity’s consensus engine allows using a smart contract for block reward calculation." - see https://wiki.parity.io/Block-Reward-Contract for details. This means that we need to use `trace_block` (https://wiki.parity.io/JSONRPC-trace-module#trace_block) to fetch the block reward contract beneficiaries and update their balances. Currently this doesn't happen so there are addresses (block contract reward beneficiaries) with inaccurate balances and missing rows in the `address_coin_balances` table. * Issue link: https://github.com/poanetwork/blockscout/issues/767 This change addresses the need by: * Adding `EthereumJSONRPC.fetch_beneficiaries/2` to help us fetch the block reward contract beneficiaries of a given block range. This only works with Parity. * Editing Indexer's dev and prod config by adding `trace_block` config to `:method_to_url`. * Adding `block_reward_contract_beneficiaries` to `Indexer.AddressExtraction` for it to know how to get addresses in this scenario. * Editing `Indexer.Block.Fetcher.fetch_and_import_range/2` to get block reward contract beneficiaries, create their addresses if they don't yet exist, and update their coin balances.	6 years ago
Luke Imhoff	1100f56fa1	Test WebSocketClient with cowboy_websocket_handler ... Allows to properly simulate TCP disconnect.	6 years ago
Luke Imhoff	b1f4696569	Reconnect, rerequest, and resubscribe when websocket disconnects ... * Use struct, `%EthereumJSONRPC.WebSocket.WebSocketClient.Options{}` to hold `web_socket_options`. * `%EthereumJSONRPC.WebSocket.Registration{}` * Record `t:EthereumJSONRPC.request/0` in `%EthereumJSONRPC.WebSocket.Registration{}`, so that the request can be re-requested if the websocket disconnects and reconnects. * Drop server-side `subscription_id` from `%EthereumJSONRPC.WebSocket.Registration{}` as it can retrieved from `request` now. * `EthereumJSONRPC.WebSocket.WebSocketClient` * Track whether the web socket is connected or not, so that requests aren't sent when in the disconnected state. This prevents double-sends as `handle_call` can be called before `onconnect`, which caused that first subscribe to be sent twice. * Instead of the subscriber knowing the server-side subscription id, an opaque `t:reference/0` is used, so that the reference is stable in the subscriber's `%Subscription{}`, but the subscription ID can change on reconnect, which means replacing `subscription_id_to_subscription` in `%WebSocketClient{}` with 3 maps 1. `subscription_id_to_subscription_reference` - used translate server-side, per-connection subscription to the stable reference in `subscription_reference_to_subscription` 2. `subscription_reference_to_subscription_id` - used to convert stable reference to subscription id to unsubscribe. 3. `subscription_reference_to_subscription` - used to get full Subscription, including the subscriber_pid for when subscription messages are delivered or the subscription needs to be resubscribed using the `web_socket_options`. * `ondisconnect`, `:reconnect` immediately. * When disconnecting, tell unsubscribers they were successful. * Hold any json_rpc or subscribe requests to retry on reconnect. * Hold any subscriptions to resubscribe on reconnect. * `onconnect` * re-request the saved json_rpc and subscribe requests. * issue new requests to re-establish the subscriptions that had already succeeded in their subscribe requests. * `%EthereumJSONRPC.Subscription{}` * Replace `id`, which did match the server created ID, with a `t:reference/0`, which allows for indirection and for the server created ID to change on reconnect while the subscriber does not need to care about the reconnections. * `EthereumJSONRPC.WebSocket` * `options` is switched from a Keyword.t to a struct, `t`, so that it is easier to pattern match for the reconnect logic in `WebSocketClient`. * `web_socket_options` is no longer a generic `term` because the code did require it to be a map with a `:web_socket` key for the `t:GenServer.server/0`.	6 years ago
Luke Imhoff	32535bd73c	Simplify WebSocketClient.respond_to_registration ... All clauses set request_id_to_registration in state to new_request_id_to_registration, so it does not need to be passed. The catchall cause does the equivalent because new_request_id_to_registration was request_id_to_registration already as Map.pop popped nothing.	6 years ago
Konstantin Zolotarev	ff8053d656	fix formatting	6 years ago
Konstantin Zolotarev	3eee0a2944	Fix spacing	6 years ago
Konstantin Zolotarev	0604f7a234	Fixes for pr comments	6 years ago
Konstantin Zolotarev	f89a27adf4	Fixes for ganache integration	6 years ago
Nam Chu Hoai	d17917ce30	Allow unsecure WebSocket connections ... Some setup like ours will have the WebSocket connection in a private network to a local node and thus do not require a secure connection. This change allows to specify ws:// instead of wss:// endpoints as well as reading from the ENV var for the geth config of the indexer Co-authored-by: Luke Imhoff <luke.imhoff@dockyard.com>	6 years ago
Luke Imhoff	5d0c1fc18a	succucessfully -> successfully	6 years ago