636 Commits (191e8a66c5c4c9ac4470a7573cc2241af10907ec)

Author	SHA1	Message	Date
Luke Imhoff	5ffb54129f	Extract Explorer.Chain.Import.Logs	6 years ago
Luke Imhoff	4842462667	Extract Explorer.Chain.Import.Block.SecondDegreeRelations	6 years ago
Luke Imhoff	56f0365239	Extract Explorer.Chain.Import.Blocks	6 years ago
Luke Imhoff	03557f4200	Extract Explorer.Chain.Address.CoinBalances ... Standardize on `:address_coin_balances` instead of `:balances` for option names.	6 years ago
Nam Chu Hoai	d17917ce30	Allow unsecure WebSocket connections ... Some setup like ours will have the WebSocket connection in a private network to a local node and thus do not require a secure connection. This change allows to specify ws:// instead of wss:// endpoints as well as reading from the ENV var for the geth config of the indexer Co-authored-by: Luke Imhoff <luke.imhoff@dockyard.com>	6 years ago
Luke Imhoff	147a7e14fb	Don't do async imports with empty lists	6 years ago
Luke Imhoff	2e3627d536	Don't run imports for empty params	6 years ago
Andrew Cravenho	9892d1bd17	Add env variables to the production parity files	6 years ago
Luke Imhoff	bf7e8f11c5	Unsupport -> Unsupported	6 years ago
Luke Imhoff	9264a92db3	remaning -> remaining	6 years ago
Luke Imhoff	c60ae9467f	upto -> up to	6 years ago
Luke Imhoff	088589b11a	transaactions -> transactions	6 years ago
Luke Imhoff	65cbce71b4	registred -> registered	6 years ago
Luke Imhoff	1ba2e01bfa	lastest -> latest	6 years ago
Luke Imhoff	f87235c79a	strucutured -> structured	6 years ago
Alex Garibay	d7e38a223e	Missed token transfer cataloger (#807) ... * Rename token transfer params parser * Rename token transfer parsing function * Requeue blocks with incomplete token transfer indexing on start up * Add additional test for worker * Add missed token transfers to the front of the catchup queue * Address code review concerns * Remove unused alias * Add process names for uncataloged token transfer processes * Improved handling of when sequence isn't avaialbe to queue * Move Uncataloged.Supervisor name out of init and into start_link call Name is not a valid option to init as it is a GenServer.option, which means it is passed to the 3rd argument to Supervisor.start_link.	6 years ago
Felipe Renan	cc89b1f6db	Ignore burn address token balance for Tokens ERC-721 ... As we always get errors trying to fetch token balances when the address is the burn addres for Tokens ERC-721, we are gonna ignore them when we are processing the token balances from the token transfer.	6 years ago
Luke Imhoff	b8d9edf97d	Indexer.Block.Catchup.BoundIntervalSupervisor is proper supervisor ... Fixes #592 Ensure that Indexer.Block.Catchup.BoundIntervalSupervisor acts as a proper Supervisor by ensure it supports all Supervisor and :supervisor functions.	6 years ago
Luke Imhoff	2f7be3fb7d	OTP application descriptions ... The Application selector in Wobserver shows the descriptions, so it is useful to be set.	6 years ago
Sebastian Abondano	fdd0c84721	Realtime fetcher retries for unknown block number ... Why: * We want to add a retry mechanism, instead of waiting half a second for every block the realtime fetcher attempts to fetch and index. We had this half a second wait in place in an attempt to avoid "Unknown block number" errors. The half a second wait was effective at reducing these errors but it also meant we were awaiting unnecessarily for most fetches. Retrying after half a second, when Parity says it doesn't yet know of a given block number, makes more sense. * Issue link: n/a This change addresses the need by: * Editing `Indexer.Block.Realtime.Fetcher.fetch_and_import_block/3` to retry failed fetch/import attempts because of "Unknown block number" errors. It waits half a second before retrying and retries a maximum of three times.	6 years ago
Luke Imhoff	8e8b4b5508	Indexer.Block.Uncle.Fetcher ... * Use `Explorer.Chain.stream_unfetched_uncle_hashes` to initialize queue. * Have `Indexer.Block.Catchup.Fetcher` and `Indexer.Block.Realtime.Fetcher` queue uncles to fetch by calling `Indexer.Block.Uncle.Fetcher.async_fetch_blocks`.	6 years ago
Felipe Renan	e74bfb9d3b	Fix token balance log error ... We inverted the value of address_hash with the contract_address_hash. This commit fixes that.	6 years ago
Luke Imhoff	4b140c87df	Block in the consensus chain marked consensus true	6 years ago
Luke Imhoff	96d5aea311	Block.SecondDegreeRelation ... The database table block_second_degree_relations contains links between the `uncle_hash` of the uncle and the `nephew_hash` of the block that said `uncle_hash` was an uncle. It also has a `uncle_fetched_at` that is `null` until the `uncle_hash` is confirmed as fetched in the database. This does not contain such a fetcher.	6 years ago
Luke Imhoff	48b302f935	Return uncles from EtherumJSONRPC.Block.elixir_to_params	6 years ago
Stamates	ff5d4d0163	Wireup live update of pending transactions	6 years ago
Sebastian Abondano	8e3d80a6db	Realtime indexer waits half a second before fetch ... Why: * For Sokol, on staging, we noticed many errors like this one: ``` 00:45:05.627 application=indexer [error] realtime indexer failed to validate for block 4668478: [%{code: -32602, data: %{"blockNumber" => "0x473C3E", "hash" => "0x532e069e200be3e372da9680872826bb08caef91"}, message: "Unknown block number"}, %{code: -32602, data: %{"blockNumber" => "0x473C3E", "hash" => "0x95426f2bc716022fcf1def006dbc4bb81f5b5164"}, message: "Unknown block number"}, %{code: -32602, data: %{"blockNumber" => "0x473C3E", "hash" => "0xe797a1da01eb0f951e0e400f9343de9d17a06bac"}, message: "Unknown block number"}]. Block will be retried by catchup indexer. ``` * Issue link: https://github.com/poanetwork/blockscout/issues/808 This change addresses the need by: * Editing `Indexer.Block.Realtime.Fetcher.fetch_and_import_block/2` for it to wait half a second before starting to fetch and import. The rationale here is that we're hoping that half a second will be enough for Parity to sync and not return an "Unknown block number" error.	6 years ago
Sebastian Abondano	14b916f9e8	Realtime.Fetcher async fetching and no skips ... Why: * For the realtime fetcher to keep up on ETH we want it to fetch blocks asynchronously. We noticed that the realtime fetcher was falling behind on ETH mainnet. Currently the realtime fetcher fetches and imports one block at a time. This commit changes that, so that whenever we're notified of a new block number, through websockets, we start a new process to fetch and import each block asynchronously. * Also, whenever the websocket subscription, through which we're notified of new block numbers, skips block numbers, we want to make sure the skipped blocks are also fetched and imported. * Issue link: n/a This change addresses the need by: * Adding a new `Task.Supervisor` called `Indexer.Block.Realtime.TaskSupervisor` under the `Indexer.Block.Realtime.Supervisor` supervisor. This supervisor was created to supervise fetching and importing processes started by the realtime fetcher every time a new block number is received through the websocket subscription. * Editing `Indexer.Block.Realtime.Fetcher` to start a new process, under the new supervisor mentioned above, to fetch the latest block per the websocket `newHeads` subscription. It also determines if any block numbers have been skipped and starts individual processes to fetch and import the skipped blocks also.	6 years ago
Igor Florian	52e861fa73	Include and Log token_balances error attribute	6 years ago
Igor Florian	99012b2c4c	Add a new log file for token_balances fetching errors	6 years ago
Lucas Narciso	d8b22f6988	Handle large strings when fetching tokens	6 years ago
Amanda Sposito	32ad1214b6	Increase timeout at the `Chain.import` at the 'Indexer.InternalTransaction.Fetcher'	6 years ago
Amanda Sposito	3244113be4	Increase timeout at the `Repo.transaction` level at the 'Indexer.TokenBalance.Fetcher'	6 years ago
Amanda Sposito	eea81817f7	Increase 'Indexer.TokenBalance.Fetcher' timeout	6 years ago
Amanda Sposito	b4a66ad7f0	Add treatment to TokenBalance.Fetcher when there is an error with the import data ... * The TokenBalanceFetcher was not considering empty lists and errors when trying to import the data into the database.	6 years ago
Alex Garibay	56299e308b	Add support for ERC-721 token transfer logs with addresses in `data`	6 years ago
Luke Imhoff	1a8bb6cbed	Restructure Indexer.BlockFetcher to match other Fetchers ... Move the "Fetcher" part below the "time" part of "Realtime" or "Catchup", so that the "Fetcher" and "Supervisor" are siblings like in the fetchers that don't have a "time" part.	6 years ago
Luke Imhoff	5b7872f030	Don't use closure for BufferedTask Task.Supervisor.async_nolink call ... Fixes #676 For some reason, which I haven't tracked down yet, a closure inside of `BufferedTask.spawn_next_batch/1` continually grows in size. When this closure is passed to `Task.Supervisor.async_nolink`, the `Task.Supervisor`, for #676, this was `Indexer.TaskSupervisor` and is now `Indexer.InternalTransaction.TaskSupervisor`, it also grows in memory size. I think this is related to the way that functions and closures are serialized when sent between processes, but I haven't figured out how to verify that. The change also cleared the MsgQ for `Indexer.TransactionFetcher`, probably because its not blocked by the call to `Indexer.InternalTransaction.Fetcher`, which was blocked due to something to do with the memory usage of the closures. The `Indexer.InternalTransaction.TaskSupervisor` consistently shows multiple tasks under it in `:observer`, which it did not previously either.	6 years ago
Luke Imhoff	6aaa300de0	Setup excoveralls for indexer in isolation ... `mix coveralls.html` only works from the root project for indexer and not the `app/indexer`, which made it difficult to generate the indexer-only report.	6 years ago
Luke Imhoff	279c6987b1	Sort Indexer project keys	6 years ago
Luke Imhoff	09051ceb8a	Separate usage of Task.Supervisor ... Have a separate subtree for each fetcher with a supervisor supervising the fetcher and a Task.Supervisor that is only used by the fetcher, so that looking at the Task.Supervisor in that subtree will show which fetcher is causing any memory, message, or blocking problems.	6 years ago
Luke Imhoff	9b3ed74507	Allow all genserver options to be passed to BufferedTasks ... Previously, only name could be passed, but now, by separating callback init options from genserver options, we can pass any options that GenServer supports, including :debug options, which is helpful for tracing performance issues.	6 years ago
Felipe Renan	6148658f61	Store tokens with invalid strings in Token indexer. ... It's a temp fix for https://github.com/poanetwork/blockscout/issues/669. For now, we are going to store those tokens that have names and/or symbols invalids considering: - the first 6 characters from the contract address hash for the name. - nil for the symbol.	6 years ago
Stamates	18380efe0a	fix broken tests ... Co-authored-by: Luke Imhoff <luke.imhoff@dockyard.com>	6 years ago
Felipe Renan	e86c5c79f3	Handle timeout when fetching token balances from blockchain	6 years ago
Igor Florian	7077a41576	Change Address fetched_balance's columns to fetched_coin_balace	6 years ago
Igor Florian	acf14d6cf3	Rename Balance references to Address.CoinBalance	6 years ago
Igor Florian	c640bf2de1	Rename AddressBalanceFetcherCase to CoinBalanceCase	6 years ago
Igor Florian	8e0f8b2e92	Rename BalanceFetcher to CoinBalanceFetcher	6 years ago
Igor Florian	2a7c2feac2	Rename Balances module to CoinBalances	6 years ago