3265 Commits (c1c30c4298ae5b533418cc9298ebec08a472aaf7)

 

All Branches   

Search

Author	SHA1	Message	Date
Luke Imhoff	e88fb5c999	Merge pull request #1296 from poanetwork/1295 ... Order current token balance inserts to prevent deadlock	6 years ago
Luke Imhoff	df8673242e	Order current token balance inserts to prevent deadlock ... Fixes #1295	6 years ago
Luke Imhoff	b69d437b06	Include block number when ordering token balances ... The block number matters for ordering as a token can be updated for an address multiple times across different blocks in one insert.	6 years ago
Luke Imhoff	6165f6757f	Move uniquifiation of address current token balances out of runner ... For all other runners, it is the responsibility of the caller to ensure unique entries. This ensures that the least amount of data as possible is sent to `Explorer.Chain.Import.all`, which speeds up the whole import process.	6 years ago
Luke Imhoff	cb2e17e373	Merge pull request #1291 from poanetwork/token-holders-counter ... Use COUNT(*) and index of current_token_balances to eliminate TokenHoldersCounter	6 years ago
Luke Imhoff	8ffbe6418d	Use COUNT(*) of current_token_balances to eliminate TokenHoldersCounter ... `TokenHolderCounter` was created 20 days before the current_token_balances table and wasn't redesigned to make use of `current_token_balances`. With `current_token_balances` in place, the problematic query over `token_balances` that necessitated `TokenHoldersCounter` can use `current_token_balances`. Using `current_token_balances` with `COUNT(*)` is when tested on eth60-test, the new query for `Explorer.Chain.count_token_holders_from_token_hash/1` still took 11 seconds. ``` sql> SELECT COUNT(*) FROM address_current_token_balances WHERE token_contract_address_hash = decode('f230b790e05390fc8295f4d3f60332c93bed42e2', 'hex') AND address_hash != decode('0000000000000000000000000000000000000000', 'hex') AND value > 0 [2019-01-02 08:04:26] 1 row retrieved starting from 1 in 11 s 554 ms (execution: 11 s 463 ms, fetching: 91 ms) ``` Using `EXPLAIN`, it should this was using a full sequential scan of table, so like in the other `COUNT(*)` optimizations PRs, I added an index to enable an index scan. Running the migration to add the index took only 1 minute on eth60-test: ``` 2019-01-02T08:40:16.220 application=ecto_sql [info] == Running 20190102141900 Explorer.Repo.Migrations.IndexCurrentTokenHolders.change/0 forward 2019-01-02T08:40:16.220 application=ecto_sql [info] create index address_current_token_balances_token_contract_address_hash_index 2019-01-02T08:41:13.746 application=ecto_sql [info] == Migrated 20190102141900 in 57.5s ``` After the migration ran, Index Scan was enabled and the query took only 837ms. ``` sql> SELECT COUNT(*) FROM address_current_token_balances WHERE token_contract_address_hash = decode('f230b790e05390fc8295f4d3f60332c93bed42e2', 'hex') AND address_hash != decode('0000000000000000000000000000000000000000', 'hex') AND value > 0 [2019-01-02 08:42:44] 1 row retrieved starting from 1 in 866 ms (execution: 837 ms, fetching: 29 ms) ```	6 years ago
Andrew Cravenho	7b3ca991ff	Merge pull request #1284 from poanetwork/ac-update-sokol-tests ... Update Parity Sokol node tests	6 years ago
Andrew Cravenho	f3ab5e4c58	Update Parity Sokol node tests	6 years ago
Andrew Cravenho	0d9e6ab2cc	Merge pull request #1282 from poanetwork/remove-show-logs-button ... Remove buttons to show raw input/logs	6 years ago
fvictorio	f2bc30b5ce	Generate gettext	6 years ago
fvictorio	27c57b62d3	Show raw data directly	6 years ago
Gustavo	e9e6d3444d	Merge pull request #1278 from poanetwork/gsf-market-history-chart-load-async ... market history chart load async	6 years ago
Gustavo Santos Ferreira	f945ea53d8	gettext	6 years ago
Gustavo Santos Ferreira	34dd660c1f	deal with coin market cap's pagination situation	6 years ago
Gustavo Santos Ferreira	ae7bd4aefc	make the market history chart load async	6 years ago
Gustavo Santos Ferreira	bc48b8ea23	new endpoint for the market chart data	6 years ago
Gustavo Santos Ferreira	47886ae50d	keep the chart space always occupied	6 years ago
Amanda	1e00990166	Merge pull request #1281 from poanetwork/ams-home-blocks-async ... Async loading to block list in the home page	6 years ago
Amanda Sposito	b3c04beaa2	Move transaction error bind to its own function in chain.js	6 years ago
Amanda Sposito	99bf28f0c0	Add gettext	6 years ago
Amanda Sposito	9cf7e8397e	Add async loading to block list in the home page.	6 years ago
William Sanches	d81c5dd765	Merge pull request #1279 from poanetwork/wsa-async-load-home-page-transactions ... Async load home page transactions	6 years ago
William Sanches	024ef5ecd8	Fix frontend tests	6 years ago
William Sanches	ff27fb3ebb	Gettext	6 years ago
William Sanches	32f24664f2	Add async load to home page transactions	6 years ago
Andrew Cravenho	927ffdfdfc	Merge pull request #1264 from poanetwork/1237-tokens-price ... Show token price in USD	6 years ago
fvictorio	58e8a0c1b8	Generate gettext	6 years ago
fvictorio	616658adbf	Show token price in USD	6 years ago
Andrew Cravenho	04d734fb1f	Merge pull request #1280 from poanetwork/1158-verify-contract-message-info ... Use info style in verify contract message	6 years ago
fvictorio	18c1001918	Use info style in verify contract message	6 years ago
William Sanches	8cc95e9f78	Merge pull request #1268 from poanetwork/wsa-implement-emission-reward-tile ... Implement emission reward tile	6 years ago
William Sanches	b06d81d891	Gettext	6 years ago
William Sanches	dfc600b1c9	Add tests	6 years ago
William Sanches	57fab18a39	Add realtime to emission rewards	6 years ago
William Sanches	1e6e2251b2	Add configuration for enabling emission tiles	6 years ago
William Sanches	33b06b228b	Refactor transaction tile identifier ... Now both transactions and emission rewards must be in the same list, so I changed the identifier to a more generic name to contemplate both of cases.	6 years ago
William Sanches	590f81004c	Add emission tiles to address transactions	6 years ago
William Sanches	b42f09cdc0	Add emission reward tile	6 years ago
Andrew Cravenho	37121c1d25	Merge pull request #1267 from poanetwork/ams-event-handler-messages ... Decrease BlockScoutWeb.EventHandler message queue	6 years ago
Amanda Sposito	bd58afff74	Add blocks indexed counter at BlockScoutWeb	6 years ago
Amanda Sposito	bf46d3ee51	Add BlockScoutWeb.RealtimeEventHandler and Explorer.Chain.Events.Publisher ... * Separate EventHandler according to broadcast_type event, so the frontend realtime does not need to ignore the catchup events.	6 years ago
Andrew Cravenho	8a926c1b05	Merge pull request #1269 from poanetwork/gsf-block-rewards-for-older-blocks ... get rewards of blocks already indexed without rewards	6 years ago
Gustavo Santos Ferreira	0b3e7f987d	genserver to find blocks without rewards and send them to the importer module	6 years ago
Gustavo Santos Ferreira	3a6751c3cc	importer module to import batches of block rewards	6 years ago
Gustavo Santos Ferreira	af8b92aa0d	new queries for blocks and transactions	6 years ago
Gustavo Santos Ferreira	f139317f88	multiply two wei values	6 years ago
Luke Imhoff	e5e32d693a	Merge pull request #1275 from poanetwork/counters-queries ... Counter the need for counter queries by using COUNT(*) and adding indexes	6 years ago
Luke Imhoff	903d322553	Use index and COUNT(*) for address_to_validation_count ... BlockValidationCounter is not needed once the query is switched to `COUNT(*)` and an index is added for `blocks.miner_hash`. Creating the index on eth60-test took only 12 seconds. ``` 2018-12-21T08:32:46.375 application=ecto_sql [info] == Running 20181221143000 Explorer.Repo.Migrations.CreateBlocksMinerHashIndex.change/0 forward 2018-12-21T08:32:46.375 application=ecto_sql [info] create index blocks_miner_hash_index 2018-12-21T08:32:59.211 application=ecto_sql [info] == Migrated 20181221143000 in 12.8s ``` Before the index both `COUNT(*)` and `COUNT(hash)` (the old usage before 76fc8e0377 in 92f99cac22) take > 800ms, which would have a noticable impact on the UI. ``` sql> SELECT COUNT(*) FROM blocks WHERE blocks.miner_hash = ( SELECT blocks.miner_hash FROM blocks LIMIT 1 ) [2018-12-21 08:27:43] 1 row retrieved starting from 1 in 906 ms (execution: 889 ms, fetching: 17 ms) sql> SELECT COUNT(hash) FROM blocks WHERE blocks.miner_hash = ( SELECT blocks.miner_hash FROM blocks LIMIT 1 ) [2018-12-21 08:28:04] 1 row retrieved starting from 1 in 819 ms (execution: 811 ms, fetching: 8 ms) ``` After the index both queries benefited because `COUNT(hash)` was able to use Bitmap Index Scan, but `COUNT(*)` was much better using a normal Index Scan: ``` sql> SELECT COUNT(hash) FROM blocks WHERE blocks.miner_hash = ( SELECT blocks.miner_hash FROM blocks LIMIT 1 ) [2018-12-21 08:33:43] 1 row retrieved starting from 1 in 776 ms (execution: 768 ms, fetching: 8 ms) sql> SELECT COUNT(*) FROM blocks WHERE blocks.miner_hash = ( SELECT blocks.miner_hash FROM blocks LIMIT 1 ) [2018-12-21 08:33:55] 1 row retrieved starting from 1 in 130 ms (execution: 120 ms, fetching: 10 ms) ``` The `SELECT` to get a miner_hash takes 54ms alone, so the `COUNT(*)` takes ~70ms, but EXPLAINs take just as long so ~50ms is my latency to eth60-test, meaning the `SELECT COUNT(*)` takes about 20ms plus latency.	6 years ago
Luke Imhoff	f67241cfc0	Use COUNT(*) for count_token_transfers_from_token_hash ... By using `COUNT(*)` instead of `COUNT(column)`, Postgres optimizes the count to use the row count recorded in the index. The ETS counter appeared necessary because in 95de1ebcb0, `COUNT(id)` was being used (95de1ebcb0/apps/explorer/lib/explorer/chain/token_transfer.ex (L138)), which seems reasonable in Ecto as you need some column to count, but in Postgres, it is more efficient to count everything with `*` as it implies "count rows". `COUNT(column)` requires Postgres check if any of the values are `NULL` and it does not optimize the query even if the column is `NOT NULL` in most cases like when the `id` was used. Testing on eth60-test database, `COUNT(*)` took 62ms while `COUNT(column)` (in this case `amount` since the table no longer has `id) took 16s demonstrating the vast speed improvement of `*` counting. ``` sql> SELECT COUNT(*) FROM token_transfers WHERE token_contract_address_hash = ( SELECT token_contract_address_hash FROM token_transfers LIMIT 1) [2018-12-21 07:51:02] 1 row retrieved starting from 1 in 70 ms (execution: 62 ms, fetching: 8 ms) sql> SELECT COUNT(amount) FROM token_transfers WHERE token_contract_address_hash = ( SELECT token_contract_address_hash FROM token_transfers LIMIT 1) [2018-12-21 07:51:43] 1 row retrieved starting from 1 in 16 s 862 ms (execution: 16 s 855 ms, fetching: 7 ms) ```	6 years ago
Andrew Cravenho	5999db7fab	Merge pull request #1272 from poanetwork/fix-compile-time-spandex-config ... fix: update spandex for run time config	6 years ago