297 Commits (be9b1fcb0551504dfc2eb237f26206e77934dbd2)

Author	SHA1	Message	Date
slightlycyborg	6bdbfc16f1	Add init_lag to historians to ensure historians start after indexer	5 years ago
Ayrat Badykov	598db94f8b	fix GenesisData process startup	5 years ago
Ayrat Badykov	cfaca5726e	add genesis data importer to supervisor tree	5 years ago
saneery	ea0d47cfa4	Add mock to send events directly to listener	5 years ago
slightlycyborg	8d36fe8787	Fixed explorer/config/config.exs problems	5 years ago
slightlycyborg	323c592b01	Added ENV_VAR lookup for configs relivant to stat charts	5 years ago
saneery	c09766e792	Disable MetadataProcessor with indexer	5 years ago
saneery	8584a7fc28	Disable market history cataloger with indexer	5 years ago
slightlycyborg	6af33827d8	fixed formatting	5 years ago
saneery	e93b71c0f3	Rename cache ttl to cache period	5 years ago
saneery	c2dec2ad27	update env vars parsing in the config.exs	5 years ago
saneery	c03f6ac91c	Add env var to manage price chart cache frequency update	5 years ago
saneery	8b06da3ccc	Add env var to manage refreshing period of average block time	5 years ago
Ayrat Badykov	eff839f1c4	add default evm version to smart contract verification	5 years ago
slightlycyborg	49dde3e239	Enbl historians in config.exs, disbl in test.exs; add to Application.ex	5 years ago
pasqu4le	da775b07b7	Group Explorer caches ... Problem: a bunch of different caches are under the Explorer.Chain namespaces, cluttering the codebase Solution: create a subdirectory and collect them all under the Explorer.Chain.Cache namespace	5 years ago
Ayrat Badykov	0b17f71ad7	fix name for xDai source	5 years ago
Ayrat Badykov	39f19401c3	add healthy block period checking query	5 years ago
Ayrat Badykov	9c33fe7179	allow to exclude uncles from average block time calculation	6 years ago
zachdaniel	4a63b978fd	feat: calculate RSK market cap	6 years ago
Ayrat Badykov	43f88e364b	use fetched net version in rsk checksum	6 years ago
Ayrat Badykov	3f3705c40e	add env vars for config	6 years ago
maxgrapps	5892fa6de6	content issues, test runned	6 years ago
Victor Baranov	f9d4480a77	add test for not trimmed evm versions input	6 years ago
Victor Baranov	aa9c2c22dd	Process reviewers comments	6 years ago
Victor Baranov	813b0d80ae	ALLOWED_EVM_VERIONS ENV VAR	6 years ago
Ayrat Badykov	390bb24d64	fix tests	6 years ago
saneery	57f924f440	edit config	6 years ago
saneery	7c9fe671a4	fetching staking epoch info	6 years ago
saneery	f46730c999	format	6 years ago
Ayrat Badykov	be1623b07e	fix System.get_env("BLOCK_COUNT_CACHE_TTL") type	6 years ago
saneery	7aafc8914f	fix test	6 years ago
Victor Baranov	df2228e7ae	Fix mix format test	6 years ago
saneery	7bf6759646	add cache for block counter	6 years ago
Ayrat Badykov	da2c79de7c	make coinmarketcap's number of pages configurable	6 years ago
saneery	5acda9df98	add staking pools importer	6 years ago
zachdaniel	8c1bb01fb1	feat: Add RSK support	6 years ago
Ayrat Badykov	02f3d922fe	use seconds	6 years ago
Ayrat Badykov	6d21e8529e	fix converting to integer	6 years ago
Ayrat Badykov	dd2dbbfddb	allow setting update interval for addresses with balances counter	6 years ago
Ayrat Badykov	245a19121f	replace source_module	6 years ago
zachdaniel	b02383a80b	feat: store average block time in a genserver	6 years ago
Luke Imhoff	8ffbe6418d	Use COUNT(*) of current_token_balances to eliminate TokenHoldersCounter ... `TokenHolderCounter` was created 20 days before the current_token_balances table and wasn't redesigned to make use of `current_token_balances`. With `current_token_balances` in place, the problematic query over `token_balances` that necessitated `TokenHoldersCounter` can use `current_token_balances`. Using `current_token_balances` with `COUNT(*)` is when tested on eth60-test, the new query for `Explorer.Chain.count_token_holders_from_token_hash/1` still took 11 seconds. ``` sql> SELECT COUNT(*) FROM address_current_token_balances WHERE token_contract_address_hash = decode('f230b790e05390fc8295f4d3f60332c93bed42e2', 'hex') AND address_hash != decode('0000000000000000000000000000000000000000', 'hex') AND value > 0 [2019-01-02 08:04:26] 1 row retrieved starting from 1 in 11 s 554 ms (execution: 11 s 463 ms, fetching: 91 ms) ``` Using `EXPLAIN`, it should this was using a full sequential scan of table, so like in the other `COUNT(*)` optimizations PRs, I added an index to enable an index scan. Running the migration to add the index took only 1 minute on eth60-test: ``` 2019-01-02T08:40:16.220 application=ecto_sql [info] == Running 20190102141900 Explorer.Repo.Migrations.IndexCurrentTokenHolders.change/0 forward 2019-01-02T08:40:16.220 application=ecto_sql [info] create index address_current_token_balances_token_contract_address_hash_index 2019-01-02T08:41:13.746 application=ecto_sql [info] == Migrated 20190102141900 in 57.5s ``` After the migration ran, Index Scan was enabled and the query took only 837ms. ``` sql> SELECT COUNT(*) FROM address_current_token_balances WHERE token_contract_address_hash = decode('f230b790e05390fc8295f4d3f60332c93bed42e2', 'hex') AND address_hash != decode('0000000000000000000000000000000000000000', 'hex') AND value > 0 [2019-01-02 08:42:44] 1 row retrieved starting from 1 in 866 ms (execution: 837 ms, fetching: 29 ms) ```	6 years ago
fvictorio	616658adbf	Show token price in USD	6 years ago
Luke Imhoff	903d322553	Use index and COUNT(*) for address_to_validation_count ... BlockValidationCounter is not needed once the query is switched to `COUNT(*)` and an index is added for `blocks.miner_hash`. Creating the index on eth60-test took only 12 seconds. ``` 2018-12-21T08:32:46.375 application=ecto_sql [info] == Running 20181221143000 Explorer.Repo.Migrations.CreateBlocksMinerHashIndex.change/0 forward 2018-12-21T08:32:46.375 application=ecto_sql [info] create index blocks_miner_hash_index 2018-12-21T08:32:59.211 application=ecto_sql [info] == Migrated 20181221143000 in 12.8s ``` Before the index both `COUNT(*)` and `COUNT(hash)` (the old usage before 76fc8e0377 in 92f99cac22) take > 800ms, which would have a noticable impact on the UI. ``` sql> SELECT COUNT(*) FROM blocks WHERE blocks.miner_hash = ( SELECT blocks.miner_hash FROM blocks LIMIT 1 ) [2018-12-21 08:27:43] 1 row retrieved starting from 1 in 906 ms (execution: 889 ms, fetching: 17 ms) sql> SELECT COUNT(hash) FROM blocks WHERE blocks.miner_hash = ( SELECT blocks.miner_hash FROM blocks LIMIT 1 ) [2018-12-21 08:28:04] 1 row retrieved starting from 1 in 819 ms (execution: 811 ms, fetching: 8 ms) ``` After the index both queries benefited because `COUNT(hash)` was able to use Bitmap Index Scan, but `COUNT(*)` was much better using a normal Index Scan: ``` sql> SELECT COUNT(hash) FROM blocks WHERE blocks.miner_hash = ( SELECT blocks.miner_hash FROM blocks LIMIT 1 ) [2018-12-21 08:33:43] 1 row retrieved starting from 1 in 776 ms (execution: 768 ms, fetching: 8 ms) sql> SELECT COUNT(*) FROM blocks WHERE blocks.miner_hash = ( SELECT blocks.miner_hash FROM blocks LIMIT 1 ) [2018-12-21 08:33:55] 1 row retrieved starting from 1 in 130 ms (execution: 120 ms, fetching: 10 ms) ``` The `SELECT` to get a miner_hash takes 54ms alone, so the `COUNT(*)` takes ~70ms, but EXPLAINs take just as long so ~50ms is my latency to eth60-test, meaning the `SELECT COUNT(*)` takes about 20ms plus latency.	6 years ago
Luke Imhoff	f67241cfc0	Use COUNT(*) for count_token_transfers_from_token_hash ... By using `COUNT(*)` instead of `COUNT(column)`, Postgres optimizes the count to use the row count recorded in the index. The ETS counter appeared necessary because in 95de1ebcb0, `COUNT(id)` was being used (95de1ebcb0/apps/explorer/lib/explorer/chain/token_transfer.ex (L138)), which seems reasonable in Ecto as you need some column to count, but in Postgres, it is more efficient to count everything with `*` as it implies "count rows". `COUNT(column)` requires Postgres check if any of the values are `NULL` and it does not optimize the query even if the column is `NOT NULL` in most cases like when the `id` was used. Testing on eth60-test database, `COUNT(*)` took 62ms while `COUNT(column)` (in this case `amount` since the table no longer has `id) took 16s demonstrating the vast speed improvement of `*` counting. ``` sql> SELECT COUNT(*) FROM token_transfers WHERE token_contract_address_hash = ( SELECT token_contract_address_hash FROM token_transfers LIMIT 1) [2018-12-21 07:51:02] 1 row retrieved starting from 1 in 70 ms (execution: 62 ms, fetching: 8 ms) sql> SELECT COUNT(amount) FROM token_transfers WHERE token_contract_address_hash = ( SELECT token_contract_address_hash FROM token_transfers LIMIT 1) [2018-12-21 07:51:43] 1 row retrieved starting from 1 in 16 s 862 ms (execution: 16 s 855 ms, fetching: 7 ms) ```	6 years ago
Luke Imhoff	81179f9459	Use :timer functions to make timeout values clearer	6 years ago
Luke Imhoff	3c0adace16	Include import_id and transaction_id in logger metadata ... When `import_id` and `transaction_id` was added in Logger calls, they were not added to the metadata allowed to be shown and so weren't appearing in the logs.	6 years ago
Gustavo Santos Ferreira	f2cbf37200	genserver that executes validator data imports every 24 hours	6 years ago
Gustavo Santos Ferreira	7549b8f8c8	prepare configs and new dependencies for the retriever	6 years ago