27 Commits (ee0f704c5fe21e41b595b0084e7bc73fe356a843)

Author	SHA1	Message	Date
Paul Tsupikoff	2a8def19e2	Allow executing smart contract functions in arbitrarily sized batches ... # Problem There are a lot of `getBalanceOf` token function calls during indexing blockchain. Each of them is sent as a separate JSONRPC request, which creates a lot of unnecessary overhead and clogs archive node workers, which could otherwise be busy performing other JSONRPC queries. Moreover, function name is used as the `id` parameter of JSONRPC queries, which doesn't allow us to simply put multiple requests into one batch. # Solution * Change `EthereumJSONRPC.execute_contract_functions` to accept a list of arbitrary function calls and return a list of results. * Handle `id` field generation in the `EthereumJSONRPC` package instead of leaking implementation details to `Explorer.SmartContract.Reader`. * Change `Token.BalanceReader.get_balance_of` to accept a list of parameter sets and return a list of balances. * Make one batch request instead of launching multiple concurrent tasks in `Indexer.TokenBalances.fetch_token_balances_from_blockchain` # Notes * Protocol details of calling smart contract functions is now fully encapsulated in `EthereumJSONRPC` package. * `SmartContract.Reader.query_contract` is left for backwards compatibility with other existing applications of smart contract requests, but can be fully deprecated at some later time.	6 years ago
Luke Imhoff	8135fcafbd	Allow lists of block numbers instead of ranges for beneficiares ... Simplifies calling as numbers don't need to grouped into ranges when getting numbers from database.	6 years ago
Ayrat Badykov	fbe2e80c91	skip parity because it's not synced to the required block	6 years ago
Ayrat Badykov	3d6412664a	set actual code from mainnet	6 years ago
Ayrat Badykov	e1284326bf	fix credo	6 years ago
Ayrat Badykov	8792417110	fix parity and geth json rpc tests	6 years ago
Luke Imhoff	719ff778a8	Update apps/ethereum_jsonrpc/test/ethereum_jsonrpc_test.exs ... Co-Authored-By: ayrat555 <ayratin555@gmail.com>	6 years ago
Ayrat Badykov	e40e4dca27	fetch codes	6 years ago
Luke Imhoff	3499155cdb	Partial retries for fetch_blocks_by_range ... Means that both Block.Fetchers, realtime and catchup, now handle fetch errors. The Block.Catchup.Fetcher will retry those blocks that failed to fetch while importing the rest that did fetch instead of refetching the whole batch.	6 years ago
Luke Imhoff	d43a8c6133	Partial retry of batches of blocks by hash	6 years ago
Luke Imhoff	ad1119eb4c	Fetched Beneficiaries partial batch retries	6 years ago
Luke Imhoff	3efa2d9996	Partial retries for CoinBalance.Fetcher	6 years ago
Alex Garibay	e23ebab069	Add transform functions for block indexing	6 years ago
Alex Garibay	1f021a7e1f	Provide default 2nd arg for executing contract functions on Geth (#970)	6 years ago
Sebastian Abondano	2b8ceb4e32	Update block reward contract beneficiary balances ... Why: * "Parity’s consensus engine allows using a smart contract for block reward calculation." - see https://wiki.parity.io/Block-Reward-Contract for details. This means that we need to use `trace_block` (https://wiki.parity.io/JSONRPC-trace-module#trace_block) to fetch the block reward contract beneficiaries and update their balances. Currently this doesn't happen so there are addresses (block contract reward beneficiaries) with inaccurate balances and missing rows in the `address_coin_balances` table. * Issue link: https://github.com/poanetwork/blockscout/issues/767 This change addresses the need by: * Adding `EthereumJSONRPC.fetch_beneficiaries/2` to help us fetch the block reward contract beneficiaries of a given block range. This only works with Parity. * Editing Indexer's dev and prod config by adding `trace_block` config to `:method_to_url`. * Adding `block_reward_contract_beneficiaries` to `Indexer.AddressExtraction` for it to know how to get addresses in this scenario. * Editing `Indexer.Block.Fetcher.fetch_and_import_range/2` to get block reward contract beneficiaries, create their addresses if they don't yet exist, and update their coin balances.	6 years ago
Luke Imhoff	b1f4696569	Reconnect, rerequest, and resubscribe when websocket disconnects ... * Use struct, `%EthereumJSONRPC.WebSocket.WebSocketClient.Options{}` to hold `web_socket_options`. * `%EthereumJSONRPC.WebSocket.Registration{}` * Record `t:EthereumJSONRPC.request/0` in `%EthereumJSONRPC.WebSocket.Registration{}`, so that the request can be re-requested if the websocket disconnects and reconnects. * Drop server-side `subscription_id` from `%EthereumJSONRPC.WebSocket.Registration{}` as it can retrieved from `request` now. * `EthereumJSONRPC.WebSocket.WebSocketClient` * Track whether the web socket is connected or not, so that requests aren't sent when in the disconnected state. This prevents double-sends as `handle_call` can be called before `onconnect`, which caused that first subscribe to be sent twice. * Instead of the subscriber knowing the server-side subscription id, an opaque `t:reference/0` is used, so that the reference is stable in the subscriber's `%Subscription{}`, but the subscription ID can change on reconnect, which means replacing `subscription_id_to_subscription` in `%WebSocketClient{}` with 3 maps 1. `subscription_id_to_subscription_reference` - used translate server-side, per-connection subscription to the stable reference in `subscription_reference_to_subscription` 2. `subscription_reference_to_subscription_id` - used to convert stable reference to subscription id to unsubscribe. 3. `subscription_reference_to_subscription` - used to get full Subscription, including the subscriber_pid for when subscription messages are delivered or the subscription needs to be resubscribed using the `web_socket_options`. * `ondisconnect`, `:reconnect` immediately. * When disconnecting, tell unsubscribers they were successful. * Hold any json_rpc or subscribe requests to retry on reconnect. * Hold any subscriptions to resubscribe on reconnect. * `onconnect` * re-request the saved json_rpc and subscribe requests. * issue new requests to re-establish the subscriptions that had already succeeded in their subscribe requests. * `%EthereumJSONRPC.Subscription{}` * Replace `id`, which did match the server created ID, with a `t:reference/0`, which allows for indirection and for the server created ID to change on reconnect while the subscriber does not need to care about the reconnections. * `EthereumJSONRPC.WebSocket` * `options` is switched from a Keyword.t to a struct, `t`, so that it is easier to pattern match for the reconnect logic in `WebSocketClient`. * `web_socket_options` is no longer a generic `term` because the code did require it to be a map with a `:web_socket` key for the `t:GenServer.server/0`.	6 years ago
Luke Imhoff	48b302f935	Return uncles from EtherumJSONRPC.Block.elixir_to_params	6 years ago
Luke Imhoff	c10b6c218b	Disable another test that times out on Infura	6 years ago
Luke Imhoff	ded4ec4246	Disable tests for Geth that timeout on Infura ... Infura experiencing timeouts with 50 second timeout set.	6 years ago
Luke Imhoff	9e6198da1b	Cover EthereumJSONRPC	6 years ago
Felipe Renan	e4a9463584	Add tests to EthereumJSONRPC.execute_contract_functions/3	6 years ago
Luke Imhoff	72bd8d7c68	Ether Mainnet needs a block interval of 25000ms for consistency ... Using the 5000ms for POA leads to timeouts for the subscribe tests.	6 years ago
Luke Imhoff	dc5ec15619	Support for Moxing WebSocket	6 years ago
Luke Imhoff	6036dea12d	Websocket json_rpc, subscribe, and unsubscribe against Sokol	6 years ago
Luke Imhoff	3427873986	{:ok, nil} -> {:error, :not_found} in handle_get_block_by_tag ... Fixes #490 Compatibility with paritytech/parity-ethereum#8281, which fixed paritytech/parity-ethereum#8028 and made pending return be compliant with Ethereum docs.	6 years ago
Luke Imhoff	e3f7581572	Historical Address Balances	6 years ago
Lucas Narciso	e81281edf6	Fix typo in etheream_jsonrpc_test.exs ... Rename it to ethereum_jsonrpc_test.exs	6 years ago
Luke Imhoff	8f827ecfa6	Use Mox by default for local integration tests ... How to run without Mox is in the root `README.md`. Run all combinations of Variant and Transport on CircleCI to ensure that Mox matches HTTP behavior, so Mox can be used for fast local testing: * Geth, HTTP * Geth, Mox * Parity, HTTP * Parity, Mox Using Mox locally also lessens load on Sokol and the Infura API key the tests use for Geth under the assumption that we do far more local testing than CI testing. May help with #340, but does not fix the root cause, which is now known as `Sequence` not chunking ranges correctly on retry.	6 years ago
Luke Imhoff	834ad23f26	Standardize JSONRPC response ... Restrict response to only those fields supported by the JSON-RPC 2.0 standard, which means that level of keys is validated, so we can indicate that switch to atom keys.	6 years ago
Luke Imhoff	466c1e76f6	Request transport module and options for all JSONRPC calls ... To lessen load on Sokol chain nodes during testing and to allow substituting different transports than HTTP, such as IPC, in the future, pass the transport module and its options to any call to `EthereumJSONRPC.json_rpc`. This will allow us to use a `mox` module in local testing, so we only do true chain integration testing in CircleCI.	6 years ago
Luke Imhoff	d9d9e54dac	Remove problematic test ... Test is rewritten a separate branch, but it is breaking CI tests now, so remove it from this branch and restore with rewrite on other branch.	6 years ago
Luke Imhoff	3c680ff953	Log, but do not fail tests, when 502 Bad Gateway occurs ... 502 Bad Gateway is an intermittent problem that needs to be fixed by ops when the JSONRPC servers are under heavy load, so they should not fail the test runs.	6 years ago
Luke Imhoff	4ecfb3b409	CircleCI tests for Geth	6 years ago
Luke Imhoff	e44a617a55	Rechunk batch JSONRPC request when 413 is returned ... When status 413 Request Entity Too Large is returned from JSONRPC, we can split the batch in half and try again. Continuing to break up the chunks of the batch until we suceed or hit 1 request in the chunk, in which case we still need to error.	7 years ago
Luke Imhoff	c7a80fb75b	Convert eth_getBalance error responses to error tuple	7 years ago
Luke Imhoff	805afcd536	Start realtime index immediately from latest block on boot.	7 years ago