3103 Commits (4f1254b1be90094d84b97aa274bf12f3fff170e8)

 

All Branches   

Search

Author	SHA1	Message	Date
Luke Imhoff	cbe3308c3d	fetch_and_import_range_from_sequence log exceptions instead of raise ... Fixes #1234 Raising the exception will kill all Tasks, the sequence and the fetcher because of the linking from `Task.async_stream`. This leads to when 1 Task times out in a DB transaction, they all are affected, leading to slowdowns when the sequence has to be rebuilt when the fetcher restarts.	6 years ago
Luke Imhoff	285d8f410c	Merge pull request #1233 from poanetwork/1232 ... Label all steps in Block.Fetcher implementations	6 years ago
Luke Imhoff	adc03fd08b	Label all steps in Block.Fetcher implementations ... Fixes #1232 `{:error, :timeout}` should have been caught, but additionally, it should never have been allowed to propagate out unassigned to a step as it is not possible to track the timeout to a specific fetch.	6 years ago
Luke Imhoff	37c06ff7ce	Merge pull request #1230 from poanetwork/missing-block-number-ranges-lag ... Use lag windowing function to reduce missing block time	6 years ago
Luke Imhoff	75f9af7f56	Use lag windowing function to reduce missing block time ... Ecto 3.0 gets us easy access to partitions and the windowing function `lag`. `lag` is able to keep track of the previous row without a self join. Importantly, the new query with `lag` and no more `generate_series` completes in ~2.5 seconds on Ethereum mainnet using `:timer.tc(Explorer.Chain, :missing_block_number_ranges, [6889126..0])` on ETH Mainnet Test DB 40 while the old version doesn't return after 5+ minutes. I've heavily commented the new version of the query since this took awhile to figure out and most importantly finding the proper terminology: "gaps and islands" was key to find the starting query using `lag`. The query in SQL is as follows: ```sql SELECT island.last_number + 1, island.next_number - 1 FROM ( SELECT lag(land.number) OVER "w" AS last_number, land.number AS next_number FROM ( SELECT blocks.number AS number FROM blocks WHERE blocks.consensus = TRUE UNION ALL ( SELECT LEAST(min_blocks.number, before_range_min) FROM ( SELECT min(blocks.number) AS number FROM blocks WHERE blocks.consensus = TRUE) ) AS min_blocks WHERE min_blocks.number IS NULL OR min_blocks.number != before_range_min ) UNION ALL ( SELECT GREATEST(max_blocks.number, after_range_max) FROM ( SELECT max(blocks.number) AS number FROM blocks WHERE blocks.consensus = TRUE ) AS max_blocks WHERE max_blocks.number IS NULL OR max_blocks.number != after_range_max ) AS land WINDOW "w" AS (ORDER BY land.number) ) AS islands WHERE islands.last_number != islands.next_number - 1 ORDER BY islands.last_number ```	6 years ago
Luke Imhoff	24e81e28df	Merge pull request #1229 from poanetwork/ecto-3.0 ... Update to Ecto 3.0	6 years ago
Luke Imhoff	ea58dad12b	Update to Ecto 3.0	6 years ago
William Sanches	48fe3a450c	Merge pull request #1184 from poanetwork/wsa-save-emission-reward-data-from-blockchain-call ... Save reward data from blockchain call	6 years ago
William Sanches	e1b3d436f3	Add doc to FetchedBeneficiaries	6 years ago
William Sanches	512511aee6	Add better test coverage	6 years ago
William Sanches	0a430d9309	Update FetchedBeneficiary typespec	6 years ago
William Sanches	8d3d26d140	Add missing dependencies to ethereum_jsonrpc	6 years ago
William Sanches	a754a72390	Add block_reward import	6 years ago
William Sanches	23bd2b0049	Rename block_rewards to emission_rewards	6 years ago
Luke Imhoff	17292ad89b	Merge pull request #1224 from poanetwork/1223 ... Don't log when queue is unavailable in Indexer.Block.InvalidConsensus.Worker	6 years ago
Luke Imhoff	6a0f625da7	Port fix for InvalidConsensus to Uncataloged ... They have a similar style where Uncataloged could also receive :queue_unavailable.	6 years ago
Luke Imhoff	c978b6168b	Don't log when queue is unavailable ... Fixes #1223 For large chains like ETH Mainnet, it is fairly common for the catchup fetcher to take a long time to calculate the missing block numbers and start the named Sequence used by `push_front(block)`. Since this is normal and not exceptional, `InvalidConsensus.Worker` shouldn't log this and retry silently.	6 years ago
Luke Imhoff	f7b89123d6	Merge pull request #1217 from poanetwork/1215 ... Fix picking wrong clause for fetch_token_balances_from_blockchain	6 years ago
Luke Imhoff	e4902fc479	Fix picking wrong clause for fetch_token_balances_from_blockchain ... Fixes #1215 The short-circuiting `[]` was not being used.	6 years ago
Luke Imhoff	54f9e1b3ba	Merge pull request #1222 from poanetwork/1221 ... Don't log errors when uncles aren't found	6 years ago
Luke Imhoff	5a0f02019b	Don't log errors when uncles aren't found ... Fixes #1221 It happens too often to be worth treating as an error as we should only log things that are exceptional and need human attention as errors.	6 years ago
Luke Imhoff	14c17cb2ee	Retry retried_entries for uncles ... `Indexer.Block.Uncle.Fetcher.retry/1` when there are errors did not actually retry the calculated `retried_errors` and instead only logged them. This didn't cause an unused variable warning because `retried_errors` is used in the Logger metadata. The `Task` for the `run/2` callback did not error because `Logger.error` returns `:ok`. To fix this problem, `{:retry, retried_errors}` need to be returned after the `Logger.error` call.	6 years ago
Luke Imhoff	44e4f17d14	Merge pull request #1218 from poanetwork/1216 ... Log timeout in PendingTransaction fetcher instead of crashing	6 years ago
Luke Imhoff	30e0ac4498	Log timeout in PendingTransaction fetcher instead of crashing ... Fixes #1216	6 years ago
Andrew Cravenho	a247b47275	Merge pull request #1206 from poanetwork/fix-displayed-address-length ... Fix displayed address length when a primary name presents	6 years ago
Andrew Cravenho	a919502d67	Merge branch 'master' into fix-displayed-address-length	6 years ago
Andrew Cravenho	2fdd5bb7df	Merge pull request #1214 from poanetwork/metadata-instead-of-prose ... Metadata instead of prose	6 years ago
Luke Imhoff	65197f5913	Add shrunk to metadata ... Allow separating shrunk vs unshrunk to monitor how often shrinking occurs.	6 years ago
Luke Imhoff	2569d2595e	Move missing_block_range_count and missing_block_count out of prose ... Being in metadata will allow it to be graphed by tools.	6 years ago
Luke Imhoff	1af8f89001	Move step out of prose and into metadata	6 years ago
Luke Imhoff	ac35c4680f	Move error_count out of prose and into metadata ... Having count and error_count in the metadata would allow us to graph one vs the other to see how often we achieve partial retries vs full retries.	6 years ago
Luke Imhoff	63e646a5ff	Move count out of prose and into metadata ... Record the number of things being fetched in metadata instead of in text so that the count can be graphed.	6 years ago
Luke Imhoff	2010e90a71	Move block range out of prose and into metadata ... The catchup block range `first_block_number..last_block_number` will set `Logger.metadata(first_block_number: first_block_number, last_block_number: last_block_number)` for errors in `BoundIntervalSupervisor` and tasks in `Catchup.Fetcher`.	6 years ago
Luke Imhoff	0243726f9b	Move block number out of prose and into metadata ... Realtime block number will set `Logger.metadata(block_number: ...)` for each block number is fetches separately after getting the `newHead` websocket message.	6 years ago
Felipe Renan	9153d0767b	Merge pull request #1207 from poanetwork/frg-add-docs-to-redux-helpers ... Improve docs in the application	6 years ago
Felipe Renan	24c5e27f2d	Update README including frond-end section	6 years ago
Felipe Renan	3ce83278fb	Improve docs in Address.TokenBalance and Address.CurrentTokenBalance schemas	6 years ago
Felipe Renan	86b9a6a2d7	Add docs to redux helpers	6 years ago
Andrew Cravenho	ef5ad0df8a	Merge branch 'master' into fix-displayed-address-length	6 years ago
Amanda	0eafa6a49a	Merge pull request #1186 from poanetwork/ams-fix-address-transactions-query ... Fix Address Trasaction query	6 years ago
Amanda Sposito	4833c6c962	Fix Address Trasaction query ... * When we removed the Transaction dynamic query in favor of a union approach, we removed the guarantee we had with the joins and the subquery. * The bug was that only the first transaction that had token transfers was being shown.	6 years ago
Amanda Sposito	607be27612	Remove unused function from Transaction module.	6 years ago
Andrew Cravenho	decf8dafc7	Merge pull request #1197 from poanetwork/1026-improve-token-balances ... Improve token balances fetching for specific addresses	6 years ago
fvictorio	2a795af42a	Avoid repeated request for token balances	6 years ago
fvictorio	be36adbc8e	Improve token balances fetching for specific addresses	6 years ago
Luke Imhoff	778ed31073	Merge pull request #1211 from poanetwork/logger-metadata ... Include fetcher in logger metadata	6 years ago
Luke Imhoff	fed7d304e2	Include fetcher in logger metadata ... For each fetcher set the `Logger.metadata(fetcher: ...)` to a short semantic name. For Fetchers that use `BufferedTask`, `BufferedTask` now has an option to set `metadata` passed to `Logger.metadata/1` in 1. GenServer process 2. The `c:init/3` Task 3. The `c:run/2` Tasks	6 years ago
Sebastian Abondano	47169d1813	Merge pull request #1194 from poanetwork/sa-1181-block-consensus-ensurer ... Ensures consensus is valid for parent blocks	6 years ago
Sebastian Abondano	c762719ba0	Ensures consensus is valid for parent blocks ... Why: * We noticed some blocks with invalid consensus values on Goerli. * Issue link: https://github.com/poanetwork/blockscout/issues/1181 This change addresses the need by: * Editing the realtime fetcher to check, after insertion to DB, a block's parent consensus value and trigger a refetch for that block number if consensus is `FALSE`. * Creating `Indexer.Block.Realtime.ConsensusEnsurer` module, where logic to ensure a given block has consensus lives. * Adding `list_block_number_with_invalid_consensus/0` to `Explorer.Chain` to get block numbers with invalid consensus from DB. * Adding `Indexer.Block.InvalidConsensus.Worker` to find blocks with invalid consensus and queue them for refetching.	6 years ago
Sebastian Abondano	532a0a97bb	Fix realtime indexer's 'no skips prevention' bug ... Why: * For the realtime indexer's logic that fills in skipped block numbers to account for reorgs. See issue for details and examples. * Issue link: https://github.com/poanetwork/blockscout/issues/1189 This change addresses the need by: * Editing the realtime fetcher to keep track of the max block number seen. * Editing realtime fetcher's `start_fetch_and_import` function to only fill in for skipped block numbers when the block number received from the `newHeads` subscription is not a reorg. A reorg in this case is defined as a block number that is less than or equal to the max block number the realtime fetcher has seen. * Editing realtime fetcher's `fetch_and_import_block` function to wait 5 seconds if the block number given for it to fetch is a reorg. This is useful to improve the chance that the latest fetched block for a given number gets consensus. E.g. when goerli's `newHeads` sends two identical block numbers at virtually the same time: `100, 101, 101, 102, 103`, we want to increase the chance taht the second attempt at fetching 101 gets consensus.	6 years ago