blockscout

Commit Graph

Author	SHA1	Message	Date
Luke Imhoff	64585c4938	Insert all BufferedTaskTest assert_receive timeouts to 200ms	6 years ago
Luke Imhoff	c8c53405fc	Subscribe to newHeads for realtime indexer Separate Catchup and Realtime indexer's fetching, so that Catchup continues to use the old polling, `EthereumJSONRPC.json_rpc` while Realtime subscribes with `EthereumJSONRPC.subscribe`. Because Realtime gets notified of new blocks through the subscription, it no longer has a Sequence, so Sequence has been removed from `%BlockFetcher{}`. There is also no block concurrency or block batch size for Realtime, so that was removed from `%BlockFetcher{}` too and moved to `%BlockFetcher.Catchup{}`. Because Realtime requires a websocket to be connected, there is now an `Indexer.BlockFetcher.Realtime.Supervisor` that supervises the `Indexer.BlockFetcher.Realtime.WebSocket` and `Indexer.BlockFetcher.Realtime`. `Indexer.BlockFetcher.Realtime` because it is waiting on notifications from is subscription becomes a `GenServer` insteado of a `Task`, so it can be supervised normally. BlockFetcher.Supervisor becomes a normal `Supervisor` of `Catchup.Supervisor` and `Realtime.Supervisor`. As Realtime is using `EthereumJSONRPC.subscribe`, `subscribe_named_arguments` has been added as a sibling of `json_rpc_named_arguments` in the `:indexer` `config`s.	6 years ago
Luke Imhoff	ae21b7e8b0	Separate sequence from BlockFetcher.t Move code dealing with the Sequence to the outer most layer, so that Realtime can stop using it when switching to WebSocket.	6 years ago
Luke Imhoff	0873bcdc28	Disable BlockFetcher.SupervisorTest for Geth Opened https://github.com/poanetwork/blockscout/issues/597 to re-enable later. This is blocking too many PR merges.	6 years ago
Andrew Cravenho	28e485a5c1	Replace poanetwork/poa-explorer urls with poanetwork/blockscout	6 years ago
Luke Imhoff	f24d2a8d0c	Sleep before checking for DOWN	6 years ago
Luke Imhoff	24c2d8e24e	Use nil value to indicate unfetched Balance Balances will be created when a given (address_hash, block_number) is first seen, but with a `nil` value. The BalanceFetcher will stream this value is `NULL` rows on restart to initialize the queue.	6 years ago
Alex Garibay	8b16f81b73	Catalog token transfers and tokens during indexing (#484 ) * Fetch token information asynchronously during indexing * Include token transfers imports in main importer * Extract token transfers when importing blocks while indexing * Move ecto to explorer deps * Update References to the Indexer namespace * Add token insertions with main importer * Import token address/type during block indexing	6 years ago
Luke Imhoff	81ba3a5b7b	Only use address_hash_to_block_number if block_number is missing	6 years ago
Luke Imhoff	1225bc2f74	Regression test for https://github.com/poanetwork/poa-explorer/pull/489#issuecomment-409637329	6 years ago
Luke Imhoff	f2884992a7	Fetch realtime internal transactions and balances synchronously Fixes #366 Queuing the internal transactions and balances to be fetched asynchronously for realtime blocks lead to upto 15 minute delay between when a block appeared and the internal transaction was drained from the queue. It is a better user experience to not see the realtime blocks at all than to see them incomplete, so fetch the internal transactions and then the balances before importing the whole tree of data instead of using the catchup asynchronous queues.	6 years ago
Luke Imhoff	11735b1289	Separate Catch, Realtime, Supervisor, and BlockFetcher states BlockFetcher.Catchup and BlockFetcher.Realtime get their own structs, which use BlockFetcher struct. This better emphasizes that Catchup and Realtime are both block fetchers. The BlockFetcher.Catchup and BlockFetcher.Realtime structs move into Supervisor's struct.	6 years ago
Luke Imhoff	2333cc6cf5	Separate supervision and fetching parts of BlockFetcher Move supervision aspect to BlockFetcher.Supervisor and leave only the actual fetching code in BlockFetcher.	6 years ago
Luke Imhoff	e0fed23fad	Fix catchup_block_number not being set catchup_block_number can't be set in state because it is set in a separate task, so have the Task return it instead of storing it in a state field.	6 years ago
Luke Imhoff	062ebd9463	Remove unreliable test	6 years ago
jimmay5469	e5e695331f	Subscribe to only realtime-indexer events for channel broadcasts	6 years ago
Luke Imhoff	6abe66f7b1	Catch up index based on whether previous catch up found missing blocks When a catchup_index task completes without error, increase the interval until the next check by 2 if no blocks were missing and decrease it by 2 if blocks were missing. Minimum interval is the same as the realtime interval while the maximum interval is arbitarily chosen to be 10x the realtime interval.	6 years ago
Luke Imhoff	69a6bb65c8	Increase timeout for flaky test	6 years ago
Luke Imhoff	e3f7581572	Historical Address Balances	6 years ago
Tim Mecklem	ee0e726fb4	Return addresses from bulk insert instead of address_hashes Co-authored-by: stamates <stamates@hotmail.com>	6 years ago
Tim Mecklem	9ce3f3d9e4	Hook up block confirmations to subscribed indexer update	6 years ago
Luke Imhoff	3432fe4c51	Disable logging debug_count at regular intervals Fixes #422 BlockFetcher's debug_count support used the slow `Explorer.Chain._count` functions. We want those counts to be completely accurate when needed, so we can't move to the `Explorer.Chain._estimated_count`. Since, the current, accurate counts are slowing down the indexing by causing a timeout to have to be handled, completely remove the debug count support and have developers call the `Explorer.Chain.*_count` when they need them to monitor progress.	6 years ago
Luke Imhoff	5b01cd396b	Check that ranges are in same direction as step	6 years ago
Luke Imhoff	105ce9b613	Only allow positive step for infinite sequences	6 years ago
Luke Imhoff	94d8267799	Don't allow Sequence to be started with a mix of ranges and infinite start We either use Sequence with a fixed set of ranges in finite mode when getting the missing blocks down to genesis or in infinite mode when using it for the realtime index. We can't completely separate the support for queuing ranges for the realtime index as that handles retries, but we can simplify the validation considerably by disallowing ranges and first to `start_link`.	6 years ago
Luke Imhoff	a3d51f5e1a	Raise error if `:first` would cause retraversal	6 years ago
Luke Imhoff	15be505219	Move range chunking from BlockFetcher to Sequence Ensures that all callers benefit from chunking bug fixes, not just BlockFetcher.	6 years ago
Luke Imhoff	ff11541bde	Failing regression tests for #340	6 years ago
Luke Imhoff	8f827ecfa6	Use Mox by default for local integration tests How to run without Mox is in the root `README.md`. Run all combinations of Variant and Transport on CircleCI to ensure that Mox matches HTTP behavior, so Mox can be used for fast local testing: * Geth, HTTP * Geth, Mox * Parity, HTTP * Parity, Mox Using Mox locally also lessens load on Sokol and the Infura API key the tests use for Geth under the assumption that we do far more local testing than CI testing. May help with #340, but does not fix the root cause, which is now known as `Sequence` not chunking ranges correctly on retry.	6 years ago
Luke Imhoff	466c1e76f6	Request transport module and options for all JSONRPC calls To lessen load on Sokol chain nodes during testing and to allow substituting different transports than HTTP, such as IPC, in the future, pass the transport module and its options to any call to `EthereumJSONRPC.json_rpc`. This will allow us to use a `mox` module in local testing, so we only do true chain integration testing in CircleCI.	6 years ago
Luke Imhoff	a41cfaa4e8	Trap exits, so that Indexer.Sequence cleans up	6 years ago
Luke Imhoff	382406b347	Indexer.Sequence as GenServer	6 years ago
Luke Imhoff	bb10c02292	Failing regression test for #387	6 years ago
Luke Imhoff	5ad0ed4571	Use tags to exclude Variant running incompatible tests	6 years ago
Luke Imhoff	1b06363eba	Handle partially-synced infura nodes in tests Some Geth nodes behind https://mainnet.infura.io are not fully-synced and so don't have historical address balances. Thankfully, the check that the max block number in a batch can still be done by checking the retry data.	6 years ago
Luke Imhoff	3c680ff953	Log, but do not fail tests, when 502 Bad Gateway occurs 502 Bad Gateway is an intermittent problem that needs to be fixed by ops when the JSONRPC servers are under heavy load, so they should not fail the test runs.	6 years ago
Luke Imhoff	4ecfb3b409	CircleCI tests for Geth	6 years ago
Luke Imhoff	8b5327ad65	Remove public_key and standard_v from transactions public_key and standard_v is only exposed by Parity, so to allow for Geth compatibility, public_key and standard_v should be removed. If they are needed, they can be recovered from r, s, and v.	6 years ago
Luke Imhoff	6b3fd80916	Extract Explorer.Indexer to Indexer Resolves #304	7 years ago
Luke Imhoff	1c0bfe5eb0	Replacements	7 years ago
Luke Imhoff	768f2c14d5	Renames and delete Kept separate, so that git history remains continuous for changes and replacements.	7 years ago
Luke Imhoff	27b4bf62d5	mix new indexer --sup	7 years ago

43 Commits (360e224b19ce3c8a9c03c5779cf9d4e465574942)