261 Commits (73cde46bb59b20cfb4d23b49d34033e5a81362ab)

Author	SHA1	Message	Date
Luke Imhoff	cddd810a58	Use :websocket_client for EthereumJSONRPC.WebSocket ... :websocket_client can properly use :ssl_verify_hostname.verify_fun/3 for :verify_fun and allow for the handshake to both be secure and pass for infura.	6 years ago
Luke Imhoff	727459ec92	Use `socket`'s Socket.Web for another implementation of WebSockets	6 years ago
Luke Imhoff	c8c53405fc	Subscribe to newHeads for realtime indexer ... Separate Catchup and Realtime indexer's fetching, so that Catchup continues to use the old polling, `EthereumJSONRPC.json_rpc` while Realtime subscribes with `EthereumJSONRPC.subscribe`. Because Realtime gets notified of new blocks through the subscription, it no longer has a Sequence, so Sequence has been removed from `%BlockFetcher{}`. There is also no block concurrency or block batch size for Realtime, so that was removed from `%BlockFetcher{}` too and moved to `%BlockFetcher.Catchup{}`. Because Realtime requires a websocket to be connected, there is now an `Indexer.BlockFetcher.Realtime.Supervisor` that supervises the `Indexer.BlockFetcher.Realtime.WebSocket` and `Indexer.BlockFetcher.Realtime`. `Indexer.BlockFetcher.Realtime` because it is waiting on notifications from is subscription becomes a `GenServer` insteado of a `Task`, so it can be supervised normally. BlockFetcher.Supervisor becomes a normal `Supervisor` of `Catchup.Supervisor` and `Realtime.Supervisor`. As Realtime is using `EthereumJSONRPC.subscribe`, `subscribe_named_arguments` has been added as a sibling of `json_rpc_named_arguments` in the `:indexer` `config`s.	6 years ago
Luke Imhoff	dc5ec15619	Support for Moxing WebSocket	6 years ago
Luke Imhoff	6036dea12d	Websocket json_rpc, subscribe, and unsubscribe against Sokol	6 years ago
Alex Garibay	f81d20fc24	Add support for multiple return types in contracts (#562)	6 years ago
Alex Garibay	8b16f81b73	Catalog token transfers and tokens during indexing (#484) ... * Fetch token information asynchronously during indexing * Include token transfers imports in main importer * Extract token transfers when importing blocks while indexing * Move ecto to explorer deps * Update References to the Indexer namespace * Add token insertions with main importer * Import token address/type during block indexing	6 years ago
Luke Imhoff	f2884992a7	Fetch realtime internal transactions and balances synchronously ... Fixes #366 Queuing the internal transactions and balances to be fetched asynchronously for realtime blocks lead to upto 15 minute delay between when a block appeared and the internal transaction was drained from the queue. It is a better user experience to not see the realtime blocks at all than to see them incomplete, so fetch the internal transactions and then the balances before importing the whole tree of data instead of using the catchup asynchronous queues.	6 years ago
Luke Imhoff	3427873986	{:ok, nil} -> {:error, :not_found} in handle_get_block_by_tag ... Fixes #490 Compatibility with paritytech/parity-ethereum#8281, which fixed paritytech/parity-ethereum#8028 and made pending return be compliant with Ethereum docs.	6 years ago
Luke Imhoff	e3f7581572	Historical Address Balances	6 years ago
Amanda Sposito	dae9ff2f10	Add support to Pluggable Transport and HTTP transport modules	6 years ago
Amanda Sposito	b8a8f083d2	Add parser function to handle addresses arguments when calling the blockchain	6 years ago
Amanda Sposito	421b8df68c	Add treatment to decode string values from blockchain	6 years ago
Lucas Narciso	537f090597	Create Explorer.SmartContract.Reader ... This new module is the main entry point for getting results of a smart contract function. It accepts a smart contract hash and a list of functions names and its arguments (when applicable) in order to execute those functions and return their results. Before actually making the call to the blockchain, the function name is encrypted and the arguments are encoded. The EthereumJSONRPC deals with this, encrypting the name of the function using the Keccak256 algorithm and encoding the arguments based on their type. The call to the blockchain is also made by EthereumJSONRPC. The result is then decoded - as it comes in the same type of encoding that has been sent - by the EthereumJSONRPC lib as well, and returned. The return is a list of results of calling the functions in the blockchain. Note that each function may return more than one result, so for each functions there is a list of results.	6 years ago
Lucas Narciso	4b9bc4a69f	Create EthereumJSONRPC.Encoder ... It deals with encoding and decoding data to be sent to, or that is received from, the blockchain.	6 years ago
Lucas Narciso	acdbed2ba7	Support eth_call on EthereumJSONRPC ... It just makes the call, so the smart contract function name should already be encrypted and its arguments already encoded. It supports multiple function calls, by receiving a list of hashs and data, e.g. `[{contract_hash_1, call_data_1}, {contract_hash_2, call_data_2}]`.	6 years ago
Luke Imhoff	ac841af29f	Split batches on 504 Gateway Timeout ... Under the assumption that gateway timeouts occur because it is taking too long for Sokol nodes to respond to batch requests, 504 Gateway Timeout should be treated the same as 413 Request Entity Too Large: the batch should be split in half and retried. If we still get 504 Gateway Timeouts after this change, we'll need to add a delay before retry.	6 years ago
Luke Imhoff	834ad23f26	Standardize JSONRPC response ... Restrict response to only those fields supported by the JSON-RPC 2.0 standard, which means that level of keys is validated, so we can indicate that switch to atom keys.	6 years ago
Luke Imhoff	3d1aa20287	Convert empty batch to empty response list ... JSONRPC 2.0 standard says that an empty batch (`[]`) return an empty response (`""`), but an empty response isn't valid JSON, so instead act like it returns an empty list (`[]`).	6 years ago
Luke Imhoff	466c1e76f6	Request transport module and options for all JSONRPC calls ... To lessen load on Sokol chain nodes during testing and to allow substituting different transports than HTTP, such as IPC, in the future, pass the transport module and its options to any call to `EthereumJSONRPC.json_rpc`. This will allow us to use a `mox` module in local testing, so we only do true chain integration testing in CircleCI.	6 years ago
Luke Imhoff	3c680ff953	Log, but do not fail tests, when 502 Bad Gateway occurs ... 502 Bad Gateway is an intermittent problem that needs to be fixed by ops when the JSONRPC servers are under heavy load, so they should not fail the test runs.	6 years ago
Luke Imhoff	4ecfb3b409	CircleCI tests for Geth	6 years ago
Luke Imhoff	cac1cab39c	Geth receipts compatibility ... When testing transaction receipt processing using Geth against Ethereum Mainnet, a number of missing features were discovered: * Support for `to` and `from` addresses in receipts * The need to derive status for pre-Byzantium transactions by using the supplied gas from the transaction and the gasUsed in the receipt to check for out-of-gas.	6 years ago
Luke Imhoff	72373f7770	method_to_url config ... Instead of making up symbolic names for the different types of URLs we need, have the configuration be a base `url` for a fallback and then entries under `method_to_url` which maps a JSONRPC method name to a the URL to use. This allows the devops running the JSONRPC nodes to say which nodes support which methods specifically, which is easier than knowing which symbolic name uses which methods. It also means we don't need separate variant specific symbolic names since the variants will be able to use their own method names in this new format for the config.	6 years ago
Luke Imhoff	88bb8a3b55	log type must be optional ... type is not supplied by Geth	6 years ago
Luke Imhoff	d946b0d073	EthereumJSONRPC.Variant behaviour ... Define which callbacks and the types needed for each variant of the JSONRPC, so it is easier to add support for new variants in the future.	6 years ago
Luke Imhoff	8b5327ad65	Remove public_key and standard_v from transactions ... public_key and standard_v is only exposed by Parity, so to allow for Geth compatibility, public_key and standard_v should be removed. If they are needed, they can be recovered from r, s, and v.	6 years ago
Luke Imhoff	7d8d09a370	Show HTTP status code when Jason decode fails	6 years ago
Luke Imhoff	53d34141ee	Convert blocks nonce to bytea ... A block nonce is an 8-byte hash. Only triggered for Ethereum Mainnet where nonce is used for Proof-of-Work. Nonce was unused and 0 for POA networks like Sokol.	6 years ago
Luke Imhoff	2701d1fd20	Support mixHash from Geth	6 years ago
Luke Imhoff	e44a617a55	Rechunk batch JSONRPC request when 413 is returned ... When status 413 Request Entity Too Large is returned from JSONRPC, we can split the batch in half and try again. Continuing to break up the chunks of the batch until we suceed or hit 1 request in the chunk, in which case we still need to error.	7 years ago
Luke Imhoff	c7a80fb75b	Convert eth_getBalance error responses to error tuple	7 years ago
Luke Imhoff	3e5e6c95c4	Pass through removed flag for logs ... Fixes #314 Parity 1.11.4 sends a `removed` flag of whether a transaction was removed or not in the transaction receipt that was not sent in 1.10.6. As the flag value is already `true` or `false`, no conversion is necessary when converting to Elixir.	7 years ago
Luke Imhoff	f7d79b666a	Handle error responses from trace_replayTransaction ... Conver error responses to error tuple with multiple reasons.	7 years ago
Luke Imhoff	0680973e64	fetched_balance_block_number instead of fetched_balance_at ... Fixes #243 Instead of getting the address balance at the latest block and recording the timestamp of the insert, which is only loosely correlated with even the latest block's timestamp on-chain, use the block number for the last time an address was used in the indexed address foreign keys in the chain: * blocks * miner_hash * internal_transactions * created_contract_address_hash * from_address_hash * to_address_hash * logs * address_hash * transactions * from_address_hash * to_address_hash To gather `fetched_balance_block_number` on reboot, `Explorer.Chain.stream_unfetched_addresses` queries across the above columns for the `MAX(blocks.number)`, which is used in `Explorer.Indexer.AddressBalanceFetcher.init/2`. During indexing, `Explorer.Indexer.AddressBalanceFetcher.async_fetch_balances/1` now requires a list of `%{block_number: Block.block_number(), hash: Hash.Truncated.t()}` instead of list of `Hash.Truncated.t()`. Having a `block_number` available for all importable entities's addresses required changing `Explorer.Indexer.AddressExtraction` to extract the `block_number` too. The use of `insert` in the factories was making tests that show the count not make sense, so the factories were switched to `build` because it means only the exact count is created when a default value is overridden. This does, unfortunately, mean that `params_for(factory)` no longer generates foreign keys and they must be set explicitly OR the association passed instead of the foreign key. Allow nil extracted fetched_balance_block_number when pending transaction *NOTE*: This is a database drop and reindex change.	7 years ago
Luke Imhoff	d3ea7c251a	Fetch pending transactions ... Use `parity_pendingTransactions` JSONRPC to get the pending transactions and import them. If they conflict with pre-existing transactions, then the previous transactions win, under the assumption that sometimes pending transactions repository transaction will COMMIT after the the realtime index has COMMITed that same transaction being validated as the timeout for transactions is 60 seconds while the block rate is faster than that.	7 years ago
Luke Imhoff	c98afd5bbb	Allow pending transactions in Transaction.to_elixir ... Pending transactions have nil for "blockNumber" and "blockHash", which broke `quantity_to_integer`, so for those keys, check if `nil` and pass through. For others, check that the value is not `nil` as it gives a better error message with both the key and value instead of just the nil value passed to quantity_to_integer as happened before.	7 years ago
Luke Imhoff	394df24ea2	Use EthereumJSONRPC.request in EthereumJSONRPC.Parity	7 years ago
Luke Imhoff	308c899379	Index from latest to genesis ... Reverse the genesis index order as most users will care about recent blocks as opposed to the very beginning of the chain. This should shorten the Time-to-Usefulness.	7 years ago
Luke Imhoff	805afcd536	Start realtime index immediately from latest block on boot.	7 years ago
Luke Imhoff	a025091af1	Extract common request code	7 years ago
Luke Imhoff	d2c1dc3696	Remove unused eth_*Filter functions	7 years ago
Luke Imhoff	240be361b8	handle_get_block_by_number -> handle_get_block ... `handle_get_block_by_number` handles the result from both `fetch_blocks_by_number` AND `fetch_blocks_by_number`, so it isn't handling just the `_by_number` result and is a misnomer.	7 years ago
Luke Imhoff	beea8f5a60	Denormalize blocks.number into transasctions.block_number ... Allows to use a sorted index for `Explorer.Chain.recent_collated_transactions`, which speeds up the query 600x with 1000 blocks with 100 transaction each and the speed up appears to only grow as more blocks and transactions per block is used.	7 years ago
Luke Imhoff	f9c026202d	Fold receipts into transactions ... `Explorer.Chain.Receipt`s are never shown in isolation from `Explorer.Chain.Transaction`s and `Explorer.Chain.Transaction`s are used for pending and collated transactions, so move all collated-only fields from `Explorer.Chain.Receipt`s to `Explorer.Chain.Transaction`s. Remove `receipts` table completely: its migration is removed. **THIS IS A DB RESET AND REINDEX CHANGE.**	7 years ago
Luke Imhoff	6749e15cb8	Transaction v as integer ... Documentation I can find say v should be between 27 and 30 with 29 and 30 being rare, but there are some values of 189 and 190 from Sokol, so we can't constrain the value now.	7 years ago
Luke Imhoff	3f7a737238	standard_v is a smallint (2 bytes) that must be between 0 and 3	7 years ago
Luke Imhoff	9e24b3b0d3	Convert transactions r and s to decimal	7 years ago
Luke Imhoff	c496e5f43a	Remove double backticks not flagged by earmark	7 years ago
Luke Imhoff	11bcecad96	Explorer.JSONRPC -> EthereumJSONRPC ... Extract Explorer.JSONRPC to its own OTP application, apps/ethereum_jsonrpc. It is not ready to be a completely separate library because it knowns the internal params format used for Explorer.Chain.	7 years ago