2164 Commits (7a3dce7c8dbd896227743d136b45a44ddf1e2bc1)

Author	SHA1	Message	Date
Luke Imhoff	d36dac0dc2	Import stages ... Stages will be between Import and the Import.Runner. Unlike Runners, which use 1 common Ecto.Multi, a Stage can produce 1 or more independent Ecto.Multi that will run in separate transactions. This allows addresses to be chunked into separate transactions, which releases the locks between transactions unlike the INSERT-level chunking that was all in 1 transaction.	6 years ago
Luke Imhoff	29d945b9b6	Use reraise and __STACKTRACE__ to reraise safe_insert_all errors ... Fixes #1238 `reraise` is the current accepted idiom for reraising, not `raise` alone as it preserves the original stacktrace.	6 years ago
Luke Imhoff	3bfc342941	Missed error pattern change in #1233	6 years ago
Luke Imhoff	cbe3308c3d	fetch_and_import_range_from_sequence log exceptions instead of raise ... Fixes #1234 Raising the exception will kill all Tasks, the sequence and the fetcher because of the linking from `Task.async_stream`. This leads to when 1 Task times out in a DB transaction, they all are affected, leading to slowdowns when the sequence has to be rebuilt when the fetcher restarts.	6 years ago
Luke Imhoff	adc03fd08b	Label all steps in Block.Fetcher implementations ... Fixes #1232 `{:error, :timeout}` should have been caught, but additionally, it should never have been allowed to propagate out unassigned to a step as it is not possible to track the timeout to a specific fetch.	6 years ago
Luke Imhoff	75f9af7f56	Use lag windowing function to reduce missing block time ... Ecto 3.0 gets us easy access to partitions and the windowing function `lag`. `lag` is able to keep track of the previous row without a self join. Importantly, the new query with `lag` and no more `generate_series` completes in ~2.5 seconds on Ethereum mainnet using `:timer.tc(Explorer.Chain, :missing_block_number_ranges, [6889126..0])` on ETH Mainnet Test DB 40 while the old version doesn't return after 5+ minutes. I've heavily commented the new version of the query since this took awhile to figure out and most importantly finding the proper terminology: "gaps and islands" was key to find the starting query using `lag`. The query in SQL is as follows: ```sql SELECT island.last_number + 1, island.next_number - 1 FROM ( SELECT lag(land.number) OVER "w" AS last_number, land.number AS next_number FROM ( SELECT blocks.number AS number FROM blocks WHERE blocks.consensus = TRUE UNION ALL ( SELECT LEAST(min_blocks.number, before_range_min) FROM ( SELECT min(blocks.number) AS number FROM blocks WHERE blocks.consensus = TRUE) ) AS min_blocks WHERE min_blocks.number IS NULL OR min_blocks.number != before_range_min ) UNION ALL ( SELECT GREATEST(max_blocks.number, after_range_max) FROM ( SELECT max(blocks.number) AS number FROM blocks WHERE blocks.consensus = TRUE ) AS max_blocks WHERE max_blocks.number IS NULL OR max_blocks.number != after_range_max ) AS land WINDOW "w" AS (ORDER BY land.number) ) AS islands WHERE islands.last_number != islands.next_number - 1 ORDER BY islands.last_number ```	6 years ago
Luke Imhoff	ea58dad12b	Update to Ecto 3.0	6 years ago
William Sanches	e1b3d436f3	Add doc to FetchedBeneficiaries	6 years ago
William Sanches	512511aee6	Add better test coverage	6 years ago
William Sanches	0a430d9309	Update FetchedBeneficiary typespec	6 years ago
William Sanches	8d3d26d140	Add missing dependencies to ethereum_jsonrpc	6 years ago
William Sanches	a754a72390	Add block_reward import	6 years ago
William Sanches	23bd2b0049	Rename block_rewards to emission_rewards	6 years ago
Luke Imhoff	6a0f625da7	Port fix for InvalidConsensus to Uncataloged ... They have a similar style where Uncataloged could also receive :queue_unavailable.	6 years ago
Luke Imhoff	c978b6168b	Don't log when queue is unavailable ... Fixes #1223 For large chains like ETH Mainnet, it is fairly common for the catchup fetcher to take a long time to calculate the missing block numbers and start the named Sequence used by `push_front(block)`. Since this is normal and not exceptional, `InvalidConsensus.Worker` shouldn't log this and retry silently.	6 years ago
Luke Imhoff	e4902fc479	Fix picking wrong clause for fetch_token_balances_from_blockchain ... Fixes #1215 The short-circuiting `[]` was not being used.	6 years ago
Luke Imhoff	5a0f02019b	Don't log errors when uncles aren't found ... Fixes #1221 It happens too often to be worth treating as an error as we should only log things that are exceptional and need human attention as errors.	6 years ago
Luke Imhoff	14c17cb2ee	Retry retried_entries for uncles ... `Indexer.Block.Uncle.Fetcher.retry/1` when there are errors did not actually retry the calculated `retried_errors` and instead only logged them. This didn't cause an unused variable warning because `retried_errors` is used in the Logger metadata. The `Task` for the `run/2` callback did not error because `Logger.error` returns `:ok`. To fix this problem, `{:retry, retried_errors}` need to be returned after the `Logger.error` call.	6 years ago
Luke Imhoff	30e0ac4498	Log timeout in PendingTransaction fetcher instead of crashing ... Fixes #1216	6 years ago
Luke Imhoff	65197f5913	Add shrunk to metadata ... Allow separating shrunk vs unshrunk to monitor how often shrinking occurs.	6 years ago
Luke Imhoff	2569d2595e	Move missing_block_range_count and missing_block_count out of prose ... Being in metadata will allow it to be graphed by tools.	6 years ago
Luke Imhoff	1af8f89001	Move step out of prose and into metadata	6 years ago
Luke Imhoff	ac35c4680f	Move error_count out of prose and into metadata ... Having count and error_count in the metadata would allow us to graph one vs the other to see how often we achieve partial retries vs full retries.	6 years ago
Luke Imhoff	63e646a5ff	Move count out of prose and into metadata ... Record the number of things being fetched in metadata instead of in text so that the count can be graphed.	6 years ago
Luke Imhoff	2010e90a71	Move block range out of prose and into metadata ... The catchup block range `first_block_number..last_block_number` will set `Logger.metadata(first_block_number: first_block_number, last_block_number: last_block_number)` for errors in `BoundIntervalSupervisor` and tasks in `Catchup.Fetcher`.	6 years ago
Luke Imhoff	0243726f9b	Move block number out of prose and into metadata ... Realtime block number will set `Logger.metadata(block_number: ...)` for each block number is fetches separately after getting the `newHead` websocket message.	6 years ago
Felipe Renan	3ce83278fb	Improve docs in Address.TokenBalance and Address.CurrentTokenBalance schemas	6 years ago
Felipe Renan	86b9a6a2d7	Add docs to redux helpers	6 years ago
Amanda Sposito	4833c6c962	Fix Address Trasaction query ... * When we removed the Transaction dynamic query in favor of a union approach, we removed the guarantee we had with the joins and the subquery. * The bug was that only the first transaction that had token transfers was being shown.	6 years ago
Amanda Sposito	607be27612	Remove unused function from Transaction module.	6 years ago
fvictorio	2a795af42a	Avoid repeated request for token balances	6 years ago
fvictorio	be36adbc8e	Improve token balances fetching for specific addresses	6 years ago
Luke Imhoff	fed7d304e2	Include fetcher in logger metadata ... For each fetcher set the `Logger.metadata(fetcher: ...)` to a short semantic name. For Fetchers that use `BufferedTask`, `BufferedTask` now has an option to set `metadata` passed to `Logger.metadata/1` in 1. GenServer process 2. The `c:init/3` Task 3. The `c:run/2` Tasks	6 years ago
Sebastian Abondano	c762719ba0	Ensures consensus is valid for parent blocks ... Why: * We noticed some blocks with invalid consensus values on Goerli. * Issue link: https://github.com/poanetwork/blockscout/issues/1181 This change addresses the need by: * Editing the realtime fetcher to check, after insertion to DB, a block's parent consensus value and trigger a refetch for that block number if consensus is `FALSE`. * Creating `Indexer.Block.Realtime.ConsensusEnsurer` module, where logic to ensure a given block has consensus lives. * Adding `list_block_number_with_invalid_consensus/0` to `Explorer.Chain` to get block numbers with invalid consensus from DB. * Adding `Indexer.Block.InvalidConsensus.Worker` to find blocks with invalid consensus and queue them for refetching.	6 years ago
Sebastian Abondano	532a0a97bb	Fix realtime indexer's 'no skips prevention' bug ... Why: * For the realtime indexer's logic that fills in skipped block numbers to account for reorgs. See issue for details and examples. * Issue link: https://github.com/poanetwork/blockscout/issues/1189 This change addresses the need by: * Editing the realtime fetcher to keep track of the max block number seen. * Editing realtime fetcher's `start_fetch_and_import` function to only fill in for skipped block numbers when the block number received from the `newHeads` subscription is not a reorg. A reorg in this case is defined as a block number that is less than or equal to the max block number the realtime fetcher has seen. * Editing realtime fetcher's `fetch_and_import_block` function to wait 5 seconds if the block number given for it to fetch is a reorg. This is useful to improve the chance that the latest fetched block for a given number gets consensus. E.g. when goerli's `newHeads` sends two identical block numbers at virtually the same time: `100, 101, 101, 102, 103`, we want to increase the chance taht the second attempt at fetching 101 gets consensus.	6 years ago
Luke Imhoff	c7d614e2ec	Include date in addition to time in log format	6 years ago
Luke Imhoff	4d778f032d	Stream token holders into ETS table	6 years ago
Luke Imhoff	e0921ff3a9	Stream block validations into ETS table	6 years ago
Luke Imhoff	e99850c65d	Stream token transfers into ETS table	6 years ago
Luke Imhoff	a1f3f1f3b7	Explorer.Repo.stream_in_transaction ... Have all uses of Repo.stream use the same function, so that none of them miss the infinite timeouts if they are copied for a new one later.	6 years ago
natlg	fca1f15dc3	Fix displayed address length	6 years ago
Felipe Renan	04a0100db8	Generate gettext	6 years ago
Felipe Renan	e7624b7a46	Decrease tabs padding to have more space for tabs	6 years ago
Felipe Renan	b18b491839	Fix blocks validated page to show all tabs ... * Add to blocks validated use the "address/_tabs.html" partial * Add to "address/_tabs.html" considers if block validated page is the current page * Add tests to ensure all behavior above.	6 years ago
Gustavo Santos Ferreira	fc2985d50a	class to truncate excessivelly long token names	6 years ago
Sebastian Abondano	0204d08fba	Fix realtime indexer's 'no skips prevention' bug ... Why: * For the realtime indexer's logic that fills in skipped block numbers to account for reorgs. See issue for details and examples. * Issue link: https://github.com/poanetwork/blockscout/issues/1189 This change addresses the need by: * Editing the realtime fetcher to keep track of the max block number seen. * Editing realtime fetcher's `start_fetch_and_import` function to only fill in for skipped block numbers when the block number received from the `newHeads` subscription is not a reorg. A reorg in this case is defined as a block number that is less than or equal to the max block number the realtime fetcher has seen. * Editing realtime fetcher's `fetch_and_import_block` function to wait 5 seconds if the block number given for it to fetch is a reorg. This is useful to improve the chance that the latest fetched block for a given number gets consensus. E.g. when goerli's `newHeads` sends two identical block numbers at virtually the same time: `100, 101, 101, 102, 103`, we want to increase the chance taht the second attempt at fetching 101 gets consensus.	6 years ago
Felipe Renan	f8797a2058	Fix realtime address balance update ... The balance update stopped working because the client was trying to subscribe twice in the channel topic. When the server receives a duplicated subscription, he closes the first connection to open a new one. In our case, the client subscribed in the address channel in address.js and again address/transactions.js, then the server was keeping only the last subscription.	6 years ago
fvictorio	3f3b0646d7	Generate gettext	6 years ago
fvictorio	6966f01097	Add coin balance history tab to address page	6 years ago
fvictorio	a31c41fad0	Generate gettext	6 years ago