106 Commits (e7b3c3a4525e2e352c9d284a5738129de2e6e0b7)

Author	SHA1	Message	Date
Victor Baranov	455d6d647f	Addresses counter	5 years ago
pasqu4le	af7782f2fe	Add ETS-based cache for accounts page	5 years ago
Victor Baranov	5d9f7d0c20	Exclude nonconsensus blocks from avg block time calculation by default	5 years ago
saneery	c856a59260	Add caches updating independently from Indexer	5 years ago
Ayrat Badykov	213395ec32	remove config entry	5 years ago
Yegor San	772bc92e6c	Changes staging	5 years ago
Ayrat Badykov	372fb0d51d	remove CoinMarketCap logic	5 years ago
Ayrat Badykov	0da2afb1ca	use CoinGecko instead of CoinMarketcap for exchange rates ... CoinMarketcap deprecated and took offline its v1 API. They have v2 API which has free plan and requires API keys. But as part of https://github.com/poanetwork/blockscout/issues/234 we alresy implemented fetching from CoinGecko. Now, coin id should be set for CoinGecko for the required coin. ``` config :explorer, Explorer.ExchangeRates.Source.CoinGecko, coin_id: System.get_env("COIN_GECKO_ID", "poa-network") ``` Or the `COIN_GECKO_ID` env var should be set. For example, COIN_GECKO_ID=poa-network	5 years ago
Ayrat Badykov	2b6f7eff78	do not start genesis data fetching periodically ... From this point genesis data fetching will start and will be retried on a task failure. Also, this PR disables genesis data fetcher process by default because most our users won't know that they should set chain spec for it.	5 years ago
Ayrat Badykov	f9676bc048	integrate poa emission rewards to supervision tree	5 years ago
Ayrat Badykov	598db94f8b	fix GenesisData process startup	5 years ago
Ayrat Badykov	cfaca5726e	add genesis data importer to supervisor tree	5 years ago
saneery	c09766e792	Disable MetadataProcessor with indexer	5 years ago
saneery	8584a7fc28	Disable market history cataloger with indexer	5 years ago
saneery	e93b71c0f3	Rename cache ttl to cache period	5 years ago
saneery	c2dec2ad27	update env vars parsing in the config.exs	5 years ago
saneery	c03f6ac91c	Add env var to manage price chart cache frequency update	5 years ago
saneery	8b06da3ccc	Add env var to manage refreshing period of average block time	5 years ago
Ayrat Badykov	eff839f1c4	add default evm version to smart contract verification	5 years ago
pasqu4le	da775b07b7	Group Explorer caches ... Problem: a bunch of different caches are under the Explorer.Chain namespaces, cluttering the codebase Solution: create a subdirectory and collect them all under the Explorer.Chain.Cache namespace	5 years ago
Ayrat Badykov	0b17f71ad7	fix name for xDai source	5 years ago
Ayrat Badykov	39f19401c3	add healthy block period checking query	5 years ago
Ayrat Badykov	9c33fe7179	allow to exclude uncles from average block time calculation	5 years ago
zachdaniel	4a63b978fd	feat: calculate RSK market cap	6 years ago
Ayrat Badykov	43f88e364b	use fetched net version in rsk checksum	6 years ago
Ayrat Badykov	3f3705c40e	add env vars for config	6 years ago
maxgrapps	5892fa6de6	content issues, test runned	6 years ago
Victor Baranov	f9d4480a77	add test for not trimmed evm versions input	6 years ago
Victor Baranov	aa9c2c22dd	Process reviewers comments	6 years ago
Victor Baranov	813b0d80ae	ALLOWED_EVM_VERIONS ENV VAR	6 years ago
Ayrat Badykov	390bb24d64	fix tests	6 years ago
saneery	57f924f440	edit config	6 years ago
saneery	7c9fe671a4	fetching staking epoch info	6 years ago
saneery	f46730c999	format	6 years ago
Ayrat Badykov	be1623b07e	fix System.get_env("BLOCK_COUNT_CACHE_TTL") type	6 years ago
saneery	7aafc8914f	fix test	6 years ago
Victor Baranov	df2228e7ae	Fix mix format test	6 years ago
saneery	7bf6759646	add cache for block counter	6 years ago
Ayrat Badykov	da2c79de7c	make coinmarketcap's number of pages configurable	6 years ago
saneery	5acda9df98	add staking pools importer	6 years ago
Ayrat Badykov	02f3d922fe	use seconds	6 years ago
Ayrat Badykov	6d21e8529e	fix converting to integer	6 years ago
Ayrat Badykov	dd2dbbfddb	allow setting update interval for addresses with balances counter	6 years ago
Ayrat Badykov	245a19121f	replace source_module	6 years ago
zachdaniel	b02383a80b	feat: store average block time in a genserver	6 years ago
Luke Imhoff	8ffbe6418d	Use COUNT(*) of current_token_balances to eliminate TokenHoldersCounter ... `TokenHolderCounter` was created 20 days before the current_token_balances table and wasn't redesigned to make use of `current_token_balances`. With `current_token_balances` in place, the problematic query over `token_balances` that necessitated `TokenHoldersCounter` can use `current_token_balances`. Using `current_token_balances` with `COUNT(*)` is when tested on eth60-test, the new query for `Explorer.Chain.count_token_holders_from_token_hash/1` still took 11 seconds. ``` sql> SELECT COUNT(*) FROM address_current_token_balances WHERE token_contract_address_hash = decode('f230b790e05390fc8295f4d3f60332c93bed42e2', 'hex') AND address_hash != decode('0000000000000000000000000000000000000000', 'hex') AND value > 0 [2019-01-02 08:04:26] 1 row retrieved starting from 1 in 11 s 554 ms (execution: 11 s 463 ms, fetching: 91 ms) ``` Using `EXPLAIN`, it should this was using a full sequential scan of table, so like in the other `COUNT(*)` optimizations PRs, I added an index to enable an index scan. Running the migration to add the index took only 1 minute on eth60-test: ``` 2019-01-02T08:40:16.220 application=ecto_sql [info] == Running 20190102141900 Explorer.Repo.Migrations.IndexCurrentTokenHolders.change/0 forward 2019-01-02T08:40:16.220 application=ecto_sql [info] create index address_current_token_balances_token_contract_address_hash_index 2019-01-02T08:41:13.746 application=ecto_sql [info] == Migrated 20190102141900 in 57.5s ``` After the migration ran, Index Scan was enabled and the query took only 837ms. ``` sql> SELECT COUNT(*) FROM address_current_token_balances WHERE token_contract_address_hash = decode('f230b790e05390fc8295f4d3f60332c93bed42e2', 'hex') AND address_hash != decode('0000000000000000000000000000000000000000', 'hex') AND value > 0 [2019-01-02 08:42:44] 1 row retrieved starting from 1 in 866 ms (execution: 837 ms, fetching: 29 ms) ```	6 years ago
fvictorio	616658adbf	Show token price in USD	6 years ago
Luke Imhoff	903d322553	Use index and COUNT(*) for address_to_validation_count ... BlockValidationCounter is not needed once the query is switched to `COUNT(*)` and an index is added for `blocks.miner_hash`. Creating the index on eth60-test took only 12 seconds. ``` 2018-12-21T08:32:46.375 application=ecto_sql [info] == Running 20181221143000 Explorer.Repo.Migrations.CreateBlocksMinerHashIndex.change/0 forward 2018-12-21T08:32:46.375 application=ecto_sql [info] create index blocks_miner_hash_index 2018-12-21T08:32:59.211 application=ecto_sql [info] == Migrated 20181221143000 in 12.8s ``` Before the index both `COUNT(*)` and `COUNT(hash)` (the old usage before 76fc8e0377 in 92f99cac22) take > 800ms, which would have a noticable impact on the UI. ``` sql> SELECT COUNT(*) FROM blocks WHERE blocks.miner_hash = ( SELECT blocks.miner_hash FROM blocks LIMIT 1 ) [2018-12-21 08:27:43] 1 row retrieved starting from 1 in 906 ms (execution: 889 ms, fetching: 17 ms) sql> SELECT COUNT(hash) FROM blocks WHERE blocks.miner_hash = ( SELECT blocks.miner_hash FROM blocks LIMIT 1 ) [2018-12-21 08:28:04] 1 row retrieved starting from 1 in 819 ms (execution: 811 ms, fetching: 8 ms) ``` After the index both queries benefited because `COUNT(hash)` was able to use Bitmap Index Scan, but `COUNT(*)` was much better using a normal Index Scan: ``` sql> SELECT COUNT(hash) FROM blocks WHERE blocks.miner_hash = ( SELECT blocks.miner_hash FROM blocks LIMIT 1 ) [2018-12-21 08:33:43] 1 row retrieved starting from 1 in 776 ms (execution: 768 ms, fetching: 8 ms) sql> SELECT COUNT(*) FROM blocks WHERE blocks.miner_hash = ( SELECT blocks.miner_hash FROM blocks LIMIT 1 ) [2018-12-21 08:33:55] 1 row retrieved starting from 1 in 130 ms (execution: 120 ms, fetching: 10 ms) ``` The `SELECT` to get a miner_hash takes 54ms alone, so the `COUNT(*)` takes ~70ms, but EXPLAINs take just as long so ~50ms is my latency to eth60-test, meaning the `SELECT COUNT(*)` takes about 20ms plus latency.	6 years ago
Luke Imhoff	f67241cfc0	Use COUNT(*) for count_token_transfers_from_token_hash ... By using `COUNT(*)` instead of `COUNT(column)`, Postgres optimizes the count to use the row count recorded in the index. The ETS counter appeared necessary because in 95de1ebcb0, `COUNT(id)` was being used (95de1ebcb0/apps/explorer/lib/explorer/chain/token_transfer.ex (L138)), which seems reasonable in Ecto as you need some column to count, but in Postgres, it is more efficient to count everything with `*` as it implies "count rows". `COUNT(column)` requires Postgres check if any of the values are `NULL` and it does not optimize the query even if the column is `NOT NULL` in most cases like when the `id` was used. Testing on eth60-test database, `COUNT(*)` took 62ms while `COUNT(column)` (in this case `amount` since the table no longer has `id) took 16s demonstrating the vast speed improvement of `*` counting. ``` sql> SELECT COUNT(*) FROM token_transfers WHERE token_contract_address_hash = ( SELECT token_contract_address_hash FROM token_transfers LIMIT 1) [2018-12-21 07:51:02] 1 row retrieved starting from 1 in 70 ms (execution: 62 ms, fetching: 8 ms) sql> SELECT COUNT(amount) FROM token_transfers WHERE token_contract_address_hash = ( SELECT token_contract_address_hash FROM token_transfers LIMIT 1) [2018-12-21 07:51:43] 1 row retrieved starting from 1 in 16 s 862 ms (execution: 16 s 855 ms, fetching: 7 ms) ```	6 years ago
Luke Imhoff	81179f9459	Use :timer functions to make timeout values clearer	6 years ago