425 Commits (b12c70d050c998c56330f25c132fd66084790a4c)

Author	SHA1	Message	Date
Luke Imhoff	c8c53405fc	Subscribe to newHeads for realtime indexer ... Separate Catchup and Realtime indexer's fetching, so that Catchup continues to use the old polling, `EthereumJSONRPC.json_rpc` while Realtime subscribes with `EthereumJSONRPC.subscribe`. Because Realtime gets notified of new blocks through the subscription, it no longer has a Sequence, so Sequence has been removed from `%BlockFetcher{}`. There is also no block concurrency or block batch size for Realtime, so that was removed from `%BlockFetcher{}` too and moved to `%BlockFetcher.Catchup{}`. Because Realtime requires a websocket to be connected, there is now an `Indexer.BlockFetcher.Realtime.Supervisor` that supervises the `Indexer.BlockFetcher.Realtime.WebSocket` and `Indexer.BlockFetcher.Realtime`. `Indexer.BlockFetcher.Realtime` because it is waiting on notifications from is subscription becomes a `GenServer` insteado of a `Task`, so it can be supervised normally. BlockFetcher.Supervisor becomes a normal `Supervisor` of `Catchup.Supervisor` and `Realtime.Supervisor`. As Realtime is using `EthereumJSONRPC.subscribe`, `subscribe_named_arguments` has been added as a sibling of `json_rpc_named_arguments` in the `:indexer` `config`s.	6 years ago
Luke Imhoff	ae21b7e8b0	Separate sequence from BlockFetcher.t ... Move code dealing with the Sequence to the outer most layer, so that Realtime can stop using it when switching to WebSocket.	6 years ago
Luke Imhoff	dc5ec15619	Support for Moxing WebSocket	6 years ago
Luke Imhoff	0873bcdc28	Disable BlockFetcher.SupervisorTest for Geth ... Opened https://github.com/poanetwork/blockscout/issues/597 to re-enable later. This is blocking too many PR merges.	6 years ago
Igor Florian	b05196df0f	Create records for TokenBalances in the database ... The TokenBalances will be built in the BlockFetcher and will be stored with value and value_fetch_at as nil. When the TokenBalanceFetcher is plugued this info will be fetched and updated.	6 years ago
Igor Florian	57b6f7487e	Stop indexing Balances from TokenTransfers ... A TokenTransfer doesn't transfer Coin. So we don't need to store Chain.Balance for it since nothing will change in the amount of coin a Address has after this particular event.	6 years ago
Luke Imhoff	ad7e15ca3b	Handle :ignore from EthereumJSONRPC.fetch_internal_transactions ... `c:EthereumJSONRPC.fetch_internal_transactions/2` can return `:ignore` as it does for `EthereumJSONRPC.Geth` because it doesn't currently support internal transactions, but `Indexer.BlockFetcher.Realtime.internal_transactions/2` did not handle `:ignore` and forwarded it along instead. Switch the `with` to a `case` so we don't forward unexpected returns in the future. Make `:ignore` return empty `:internal_transaction_params`.	6 years ago
Alex Garibay	f81d20fc24	Add support for multiple return types in contracts (#562)	6 years ago
Andrew Cravenho	3e8c5b84d6	Changes from POA Explorer to BlockScout	6 years ago
Andrew Cravenho	28e485a5c1	Replace poanetwork/poa-explorer urls with poanetwork/blockscout	6 years ago
Luke Imhoff	88db1e776f	They mocked behaviours -> The mocked behaviours	6 years ago
Luke Imhoff	2dee97119f	Replicate Variant and Chain testing instructions to other READMEs ... Replicate the information about how to test against Geth/Parity and Sokol/Ethereum Mainnet to the other READMEs that use the Mox infrastructure for EthereumJSONRPC, so that they are not missed by developers that don't need to look to EthereumJSONRPC.	6 years ago
Luke Imhoff	f24d2a8d0c	Sleep before checking for DOWN	6 years ago
Luke Imhoff	bf201f1a52	Elixir 1.7.2 / Erlang 21.0.4	6 years ago
Luke Imhoff	24c2d8e24e	Use nil value to indicate unfetched Balance ... Balances will be created when a given (address_hash, block_number) is first seen, but with a `nil` value. The BalanceFetcher will stream this value is `NULL` rows on restart to initialize the queue.	6 years ago
Alex Garibay	8b16f81b73	Catalog token transfers and tokens during indexing (#484) ... * Fetch token information asynchronously during indexing * Include token transfers imports in main importer * Extract token transfers when importing blocks while indexing * Move ecto to explorer deps * Update References to the Indexer namespace * Add token insertions with main importer * Import token address/type during block indexing	6 years ago
Luke Imhoff	06475f1275	Don't broadcast from catchup ... Fixes rebase picking wrong version with broadcasting turned on for catchup index, which we don't want	6 years ago
Luke Imhoff	81ba3a5b7b	Only use address_hash_to_block_number if block_number is missing	6 years ago
Luke Imhoff	1225bc2f74	Regression test for https://github.com/poanetwork/poa-explorer/pull/489#issuecomment-409637329	6 years ago
Luke Imhoff	f2884992a7	Fetch realtime internal transactions and balances synchronously ... Fixes #366 Queuing the internal transactions and balances to be fetched asynchronously for realtime blocks lead to upto 15 minute delay between when a block appeared and the internal transaction was drained from the queue. It is a better user experience to not see the realtime blocks at all than to see them incomplete, so fetch the internal transactions and then the balances before importing the whole tree of data instead of using the catchup asynchronous queues.	6 years ago
Luke Imhoff	12f70945da	Switch Chain.import options to maps ... 1. Allows for easier composing with put_in than Keyword.t 2. Allows more precise specs as individal options can be marked as optional or required while the Keyword.t pattern makes it seem like everything is optional.	6 years ago
Luke Imhoff	68229420b4	Move Receipts functions out of BlockFetcher	6 years ago
Luke Imhoff	11735b1289	Separate Catch, Realtime, Supervisor, and BlockFetcher states ... BlockFetcher.Catchup and BlockFetcher.Realtime get their own structs, which use BlockFetcher struct. This better emphasizes that Catchup and Realtime are both block fetchers. The BlockFetcher.Catchup and BlockFetcher.Realtime structs move into Supervisor's struct.	6 years ago
Luke Imhoff	2333cc6cf5	Separate supervision and fetching parts of BlockFetcher ... Move supervision aspect to BlockFetcher.Supervisor and leave only the actual fetching code in BlockFetcher.	6 years ago
Luke Imhoff	e0fed23fad	Fix catchup_block_number not being set ... catchup_block_number can't be set in state because it is set in a separate task, so have the Task return it instead of storing it in a state field.	6 years ago
Luke Imhoff	807f5e6d7c	BlockFetcher.realtime_task -> BlockFetcher.Realtime.task	6 years ago
Luke Imhoff	2afa934225	BlockFetcher.catchup_task -> BlockFetcher.Catchup.task ... Move `catchup_task` out of `BlockFetcher`, so it is easier to spot catchup vs realtime vs common code.	6 years ago
Luke Imhoff	062ebd9463	Remove unreliable test	6 years ago
Luke Imhoff	2f3af51741	Unified Explorer.Chain.import ... * Merge `Explorer.Chain.import_blocks` and `Explorer.Chain.import_internal_transactions` into `Explorer.Chain.import`. * Move body of `Explorer.Chain.import` and its private helper functions to `Explorer.Chain.Import` as an independent context that `Explorer.Chain` can use. `Explorer.Chain.Import` must be under `Explorer.Chain` because `Explorer.Chain.Import` still needs direct access to the `Explorer.Chain` schemas.	6 years ago
jimmay5469	e5e695331f	Subscribe to only realtime-indexer events for channel broadcasts	6 years ago
Peter Romfeld	d9e37c44a6	fix formatting	6 years ago
Peter Romfeld	b473420df3	use ETHEREUM_JSONRPC_HTTP_URL instead	6 years ago
Peter Romfeld	2ddc1b9747	get geth_url from environment and fix `variant` for {dev,prod}.exs	6 years ago
Luke Imhoff	9d66f4b84b	Handle result from realtime tasks ... Previously, BlockFetcher only had handle_info for the DOWN message from realtime tasks, but when the reason is :normal, the task would have succeeded and therefore, the BlockFetcher would also have received a `{ref, :ok}` that was not being matched.	6 years ago
Luke Imhoff	6abe66f7b1	Catch up index based on whether previous catch up found missing blocks ... When a catchup_index task completes without error, increase the interval until the next check by 2 if no blocks were missing and decrease it by 2 if blocks were missing. Minimum interval is the same as the realtime interval while the maximum interval is arbitarily chosen to be 10x the realtime interval.	6 years ago
Luke Imhoff	8f63f2a87c	Rename genesis task to catchup task ... It matches message (:catchup_index) used to trigger the task and reflects that the task to to catch-up the index to what the realtime task handles.	6 years ago
Luke Imhoff	b79d361909	Fix phrasing when genesis task completes	6 years ago
Luke Imhoff	d368159a09	Remove unused start_block_number in BlockFetcher state	6 years ago
Luke Imhoff	a6be2924b2	Running realtime task on interval ... Instead of scheduling the next realtime index in block_interval / 2 only after the previous one complete, run the realtime index every block_interval / 2 using `:timer.send_interval`. This ensures that the time it takes to complete the realtime index doesn't slowly build up, which is currently leading a missed block approximately every 15 blocks.	6 years ago
Luke Imhoff	3e09353d62	Fix typo	6 years ago
Luke Imhoff	f1e3c9e376	Use Task.Supervisor.async_nolink ... Task.Supervisor.async_nolink monitors the Task pid instead of linking, so it does the same thing as our Indexer.start_monitor.	6 years ago
Luke Imhoff	75e2bfccc0	realtime_interval -> block_interval / 2 ... The realtime_interval should be half of the block_interval to ensure that we don't get stuck in a state where we just miss a new block because using precisely the block rate doesn't account for processing time. The exact value of half the block_interval was chosen for the Nyquist rules.	6 years ago
Luke Imhoff	fe49a753d3	block_rate -> block_interval ... block_rate is measured in milliseconds / block, but a rate would blocks / unit-time, so rename block_rate to block_interval to reflect it is the interval between blocks.	6 years ago
Luke Imhoff	69a6bb65c8	Increase timeout for flaky test	6 years ago
Luke Imhoff	e3f7581572	Historical Address Balances	6 years ago
Amanda Sposito	3335817d90	Fix prod config file to use prod config instead of dev	6 years ago
Tim Mecklem	ee0e726fb4	Return addresses from bulk insert instead of address_hashes ... Co-authored-by: stamates <stamates@hotmail.com>	6 years ago
Tim Mecklem	9ce3f3d9e4	Hook up block confirmations to subscribed indexer update	6 years ago
Luke Imhoff	3432fe4c51	Disable logging debug_count at regular intervals ... Fixes #422 BlockFetcher's debug_count support used the slow `Explorer.Chain.*_count` functions. We want those counts to be completely accurate when needed, so we can't move to the `Explorer.Chain.*_estimated_count`. Since, the current, accurate counts are slowing down the indexing by causing a timeout to have to be handled, completely remove the debug count support and have developers call the `Explorer.Chain.*_count` when they need them to monitor progress.	6 years ago
Luke Imhoff	5b01cd396b	Check that ranges are in same direction as step	6 years ago