401 Commits (e88fb5c999e8ad5953e77c498428c1c8e3b39754)

Author	SHA1	Message	Date
Luke Imhoff	6165f6757f	Move uniquifiation of address current token balances out of runner ... For all other runners, it is the responsibility of the caller to ensure unique entries. This ensures that the least amount of data as possible is sent to `Explorer.Chain.Import.all`, which speeds up the whole import process.	6 years ago
Luke Imhoff	8ffbe6418d	Use COUNT(*) of current_token_balances to eliminate TokenHoldersCounter ... `TokenHolderCounter` was created 20 days before the current_token_balances table and wasn't redesigned to make use of `current_token_balances`. With `current_token_balances` in place, the problematic query over `token_balances` that necessitated `TokenHoldersCounter` can use `current_token_balances`. Using `current_token_balances` with `COUNT(*)` is when tested on eth60-test, the new query for `Explorer.Chain.count_token_holders_from_token_hash/1` still took 11 seconds. ``` sql> SELECT COUNT(*) FROM address_current_token_balances WHERE token_contract_address_hash = decode('f230b790e05390fc8295f4d3f60332c93bed42e2', 'hex') AND address_hash != decode('0000000000000000000000000000000000000000', 'hex') AND value > 0 [2019-01-02 08:04:26] 1 row retrieved starting from 1 in 11 s 554 ms (execution: 11 s 463 ms, fetching: 91 ms) ``` Using `EXPLAIN`, it should this was using a full sequential scan of table, so like in the other `COUNT(*)` optimizations PRs, I added an index to enable an index scan. Running the migration to add the index took only 1 minute on eth60-test: ``` 2019-01-02T08:40:16.220 application=ecto_sql [info] == Running 20190102141900 Explorer.Repo.Migrations.IndexCurrentTokenHolders.change/0 forward 2019-01-02T08:40:16.220 application=ecto_sql [info] create index address_current_token_balances_token_contract_address_hash_index 2019-01-02T08:41:13.746 application=ecto_sql [info] == Migrated 20190102141900 in 57.5s ``` After the migration ran, Index Scan was enabled and the query took only 837ms. ``` sql> SELECT COUNT(*) FROM address_current_token_balances WHERE token_contract_address_hash = decode('f230b790e05390fc8295f4d3f60332c93bed42e2', 'hex') AND address_hash != decode('0000000000000000000000000000000000000000', 'hex') AND value > 0 [2019-01-02 08:42:44] 1 row retrieved starting from 1 in 866 ms (execution: 837 ms, fetching: 29 ms) ```	6 years ago
Andrew Cravenho	f3ab5e4c58	Update Parity Sokol node tests	6 years ago
fvictorio	616658adbf	Show token price in USD	6 years ago
William Sanches	dfc600b1c9	Add tests	6 years ago
William Sanches	590f81004c	Add emission tiles to address transactions	6 years ago
Amanda Sposito	bf46d3ee51	Add BlockScoutWeb.RealtimeEventHandler and Explorer.Chain.Events.Publisher ... * Separate EventHandler according to broadcast_type event, so the frontend realtime does not need to ignore the catchup events.	6 years ago
Gustavo Santos Ferreira	af8b92aa0d	new queries for blocks and transactions	6 years ago
Gustavo Santos Ferreira	f139317f88	multiply two wei values	6 years ago
Luke Imhoff	903d322553	Use index and COUNT(*) for address_to_validation_count ... BlockValidationCounter is not needed once the query is switched to `COUNT(*)` and an index is added for `blocks.miner_hash`. Creating the index on eth60-test took only 12 seconds. ``` 2018-12-21T08:32:46.375 application=ecto_sql [info] == Running 20181221143000 Explorer.Repo.Migrations.CreateBlocksMinerHashIndex.change/0 forward 2018-12-21T08:32:46.375 application=ecto_sql [info] create index blocks_miner_hash_index 2018-12-21T08:32:59.211 application=ecto_sql [info] == Migrated 20181221143000 in 12.8s ``` Before the index both `COUNT(*)` and `COUNT(hash)` (the old usage before 76fc8e0377 in 92f99cac22) take > 800ms, which would have a noticable impact on the UI. ``` sql> SELECT COUNT(*) FROM blocks WHERE blocks.miner_hash = ( SELECT blocks.miner_hash FROM blocks LIMIT 1 ) [2018-12-21 08:27:43] 1 row retrieved starting from 1 in 906 ms (execution: 889 ms, fetching: 17 ms) sql> SELECT COUNT(hash) FROM blocks WHERE blocks.miner_hash = ( SELECT blocks.miner_hash FROM blocks LIMIT 1 ) [2018-12-21 08:28:04] 1 row retrieved starting from 1 in 819 ms (execution: 811 ms, fetching: 8 ms) ``` After the index both queries benefited because `COUNT(hash)` was able to use Bitmap Index Scan, but `COUNT(*)` was much better using a normal Index Scan: ``` sql> SELECT COUNT(hash) FROM blocks WHERE blocks.miner_hash = ( SELECT blocks.miner_hash FROM blocks LIMIT 1 ) [2018-12-21 08:33:43] 1 row retrieved starting from 1 in 776 ms (execution: 768 ms, fetching: 8 ms) sql> SELECT COUNT(*) FROM blocks WHERE blocks.miner_hash = ( SELECT blocks.miner_hash FROM blocks LIMIT 1 ) [2018-12-21 08:33:55] 1 row retrieved starting from 1 in 130 ms (execution: 120 ms, fetching: 10 ms) ``` The `SELECT` to get a miner_hash takes 54ms alone, so the `COUNT(*)` takes ~70ms, but EXPLAINs take just as long so ~50ms is my latency to eth60-test, meaning the `SELECT COUNT(*)` takes about 20ms plus latency.	6 years ago
Luke Imhoff	f67241cfc0	Use COUNT(*) for count_token_transfers_from_token_hash ... By using `COUNT(*)` instead of `COUNT(column)`, Postgres optimizes the count to use the row count recorded in the index. The ETS counter appeared necessary because in 95de1ebcb0, `COUNT(id)` was being used (95de1ebcb0/apps/explorer/lib/explorer/chain/token_transfer.ex (L138)), which seems reasonable in Ecto as you need some column to count, but in Postgres, it is more efficient to count everything with `*` as it implies "count rows". `COUNT(column)` requires Postgres check if any of the values are `NULL` and it does not optimize the query even if the column is `NOT NULL` in most cases like when the `id` was used. Testing on eth60-test database, `COUNT(*)` took 62ms while `COUNT(column)` (in this case `amount` since the table no longer has `id) took 16s demonstrating the vast speed improvement of `*` counting. ``` sql> SELECT COUNT(*) FROM token_transfers WHERE token_contract_address_hash = ( SELECT token_contract_address_hash FROM token_transfers LIMIT 1) [2018-12-21 07:51:02] 1 row retrieved starting from 1 in 70 ms (execution: 62 ms, fetching: 8 ms) sql> SELECT COUNT(amount) FROM token_transfers WHERE token_contract_address_hash = ( SELECT token_contract_address_hash FROM token_transfers LIMIT 1) [2018-12-21 07:51:43] 1 row retrieved starting from 1 in 16 s 862 ms (execution: 16 s 855 ms, fetching: 7 ms) ```	6 years ago
Luke Imhoff	07cb543948	Failing regression test for #1266 ... When the max block number exceeds the search range by more than 1 it causes extra ranges to be calculated and they depend on the max block number instead of the search range.	6 years ago
Luke Imhoff	786f89164b	Failing regression test for #1262	6 years ago
Luke Imhoff	2b60cba8a1	Remove unused import	6 years ago
Luke Imhoff	ef02ae0504	Fix missed pinning ... Showed up as unused variables	6 years ago
Amanda Sposito	de3520667f	Add reward value of a block in the tile and overview section	6 years ago
William Sanches	2b48235898	Remove opcodes from CodeCompiler ... Really old versions of solidity compiler, e.g. v0.1.3, have different output schemas. This data was never used by our application, so instead of having to deal with different outputs, we just don't try to get this item anymore.	6 years ago
William Sanches	c1e582e85c	Add better error messages	6 years ago
Gustavo Santos Ferreira	ebf0aa6860	module that imports validators metadata into the DataBase	6 years ago
Gustavo Santos Ferreira	be397fbd7b	retriever module that queries the contracts to generate validator metadata	6 years ago
fvictorio	cd51e7cb29	Use address_current_token_balances in address tokens tab	6 years ago
Luke Imhoff	d36dac0dc2	Import stages ... Stages will be between Import and the Import.Runner. Unlike Runners, which use 1 common Ecto.Multi, a Stage can produce 1 or more independent Ecto.Multi that will run in separate transactions. This allows addresses to be chunked into separate transactions, which releases the locks between transactions unlike the INSERT-level chunking that was all in 1 transaction.	6 years ago
Luke Imhoff	ea58dad12b	Update to Ecto 3.0	6 years ago
William Sanches	23bd2b0049	Rename block_rewards to emission_rewards	6 years ago
Amanda Sposito	4833c6c962	Fix Address Trasaction query ... * When we removed the Transaction dynamic query in favor of a union approach, we removed the guarantee we had with the joins and the subquery. * The bug was that only the first transaction that had token transfers was being shown.	6 years ago
fvictorio	be36adbc8e	Improve token balances fetching for specific addresses	6 years ago
Sebastian Abondano	c762719ba0	Ensures consensus is valid for parent blocks ... Why: * We noticed some blocks with invalid consensus values on Goerli. * Issue link: https://github.com/poanetwork/blockscout/issues/1181 This change addresses the need by: * Editing the realtime fetcher to check, after insertion to DB, a block's parent consensus value and trigger a refetch for that block number if consensus is `FALSE`. * Creating `Indexer.Block.Realtime.ConsensusEnsurer` module, where logic to ensure a given block has consensus lives. * Adding `list_block_number_with_invalid_consensus/0` to `Explorer.Chain` to get block numbers with invalid consensus from DB. * Adding `Indexer.Block.InvalidConsensus.Worker` to find blocks with invalid consensus and queue them for refetching.	6 years ago
Luke Imhoff	e0921ff3a9	Stream block validations into ETS table	6 years ago
Luke Imhoff	e99850c65d	Stream token transfers into ETS table	6 years ago
fvictorio	6966f01097	Add coin balance history tab to address page	6 years ago
fvictorio	65e2592a9d	Only assert that last token name is null	6 years ago
Sebastian Abondano	fa09f399d8	GraphQL API token_transfers query ... Why: * For GraphQL API users to be able to fetch token transfer events. RPC API users could use this instead of the `tokentx` RPC API action. Example usage: 1. ``` query ($token_contract_address_hash: AddressHash!, $first: Int!) { token_transfers(token_contract_address_hash: $token_contract_address_hash, first: $first) { edges { node { amount block_number log_index token_id from_address_hash to_address_hash token_contract_address_hash transaction_hash } } } } ``` 2. with pagination support via [Relay Cursor Connections](https://facebook.github.io/relay/graphql/connections.htm) ``` query ($token_contract_address_hash: AddressHash!, $first: Int!, $after: ID!) { token_transfers(token_contract_address_hash: $token_contract_address_hash, first: $first, after: $after) { page_info { has_next_page has_previous_page } edges { node { amount from_address_hash to_address_hash } cursor } } } ``` * Issue link: n/a This change addresses the need by: * Editing the node interface and node field in `BlockScoutWeb.Schema` to support token transfers. * Adding `token_transfers` connection field to root query object in `BlockScoutWeb.Schema` * Making `token_transfer` object types a node type and adding a few new fields to it, in `BlockScoutWeb.Schema.Types`. * Creating `BlockScoutWeb.Resolvers.TokenTransfer` with resolver functions for getting token transfers. * Adding `get_token_transfer/1` and `list_token_transfers_query/1` to `Explorer.GraphQL`. These two functions are used by the new resolver functions mentioned above. * Editing RPC API docs to reference the new GraphQL query field added in this commit.	6 years ago
Amanda Sposito	845b399281	Add a cache to count the addresses with balance > 0.	6 years ago
Lokraan	582345cfb7	Create tests for new dependency injection model.	6 years ago
Lokraan	2a603be370	Merge with master.	6 years ago
Lokraan	42504acd44	Create tests for CoinGecko api.	6 years ago
fvictorio	ed93bfbb55	Fix randomly failing test	6 years ago
Sebastian Abondano	24079bb741	GraphQL support to get internal transactions ... Why: * For GraphQL API users to be able to get internal transactions. RPC API users could use this instead of the `txlistinternal` RPC API action. Example usage: 1. ``` query ($hash: FullHash!, $first: Int!) { transaction(hash: $hash) { internal_transactions(first: $first) { edges { node { index value block_number transaction_index from_address_hash to_address_hash } } } } } ``` 2. with pagiantion support via [Relay Cursor Connections](https://facebook.github.io/relay/graphql/connections.htm) ``` query ($hash: AddressHash!, $first: Int!, $after: ID!) { transaction(hash: $hash) { internal_transactions(first: $first, after: $after) { page_info { has_next_page has_previous_page } edges { node { index transaction_hash } cursor } } } } ``` * Issue link: n/a This change addresses the need by: * Editing the node interface and node field in `BlockScoutWeb.Schema` to support internal transactions. * Adding `internal_transaction` object type to `BlockScoutWeb.Schema.Types`. * Adding an `internal_transaction` connection field to the `transaction` object type in `BlockScoutWeb.Schema.Types`. * Adding `:call_type` and `:type` enums to `BlockScoutWEb.Schema.Scalars`. Needed by `internal_transaction` GraphQL object type. * Creating `BlockScoutWeb.Resolvers.InternalTransaction` with resolver functions for getting internal transactions. * Adding `get_internal_transactions/1` and `transaction_to_internal_transactions_query/1` to `Explorer.GraphQL`. These are used by the new resolver functions mentioned above.	6 years ago
Felipe Renan	d4feeff7b1	Fix pagination on inventory tab at Token's page ... This code was removed and it shouldn't :(. Besides bringing this code back, I'm adding tests in the controller layer to make sure that it will not happen anymore.	6 years ago
Felipe Renan	da72f83b35	Refactor token transfers' pagination ... We changed the token transfer's pagination uses the block_number instead of the inserted_at. It will make the pagination consistent since the list is sorted by the block_number.	6 years ago
Felipe Renan	c486837ba8	Fix token transfers' tests	6 years ago
Andrew Cravenho	a64a1dc6bb	Update sokol node tests	6 years ago
Andrew Cravenho	063c393281	Update documentation replacing old sokol nodes	6 years ago
William Sanches	421262a2da	Fix credo and format issues	6 years ago
William Sanches	2df87c2cd1	Refactor metadata_updater to get interval on start	6 years ago
William Sanches	8ee4e62087	Refactor cataloged token query to Chain.Token	6 years ago
William Sanches	06ce9ab5c8	Add chain function to get cataloged tokens	6 years ago
zachdaniel	63f7ca7f1d	chore: improve test coverage	6 years ago
Sebastian Abondano	195f7d39ee	GraphQL support to get transactions by address ... Why: * We'd like to add support for GraphQL queries to get transactions by address hash. RPC API users could use this instead of the `txlist` RPC API action. Example usage: 1. ``` query($hash: AddressHash!, $first: Int!) { address(hash: $hash) { transactions(first: $first) { edges { node { hash blockNumber } cursor } } } ``` 2. with pagination support via [Relay Cursor Connections](https://facebook.github.io/relay/graphql/connections.htm) ``` query($hash: AddressHash!, $first: Int!, $after: String!) { address(hash: $hash) { transactions(first: $first, after: $after) { pageInfo { hasNextPage hasPreviousPage } edges { node { hash blockNumber } cursor } } } } ``` * Issue link: n/a This change addresses the need by: * Renaming `BlockScoutWeb.Schema.Query.AddressTest` to `...AddressesTest` for the name to match the query field it is testing. * Adding `:absinthe_relay` dependency to `BlockScoutWeb` app for Absinthe support for the [Relay framework](https://facebook.github.io/relay/graphql/connections.htm). * Editing `BlockScoutWeb.Schema` to use `Absinthe.Relay.Schema`, define a node interface, and define a node field. This was necessary to support relay connections. * Adding an `address` field to `BlockScoutWeb.Schema`, to get a single address by hash. * Adding a `TransactionConnection` field to the `address` object type in `BlockScoutWeb.Schema.Types` * Configuring the `transaction` object type as a node, in `BlockScoutWeb.Schema.Types`. * Adding a resolver function in `BlockScoutWeb.Resolvers.Address`, to get a single address. * Adding a resolver function in `BlockScoutWeb.Resolvers.Transaction`, to get transactions for an address. * Creating `Explorer.GraphQL` for GraphQL API specific Ecto queries. The only function in this module now is `address_to_transactions_query/1`. This function is necessary for `Absinthe.Relay.Connection.from_query/4` to load transactions in the new transaction resolver mentioned above. * Editing RPC API docs to include reference to the GraphQL "transactions" field.	6 years ago
Felipe Renan	10821e6b0b	Rename FunctionsReader to MetadaRetriever ... Also: * Renaming `fetch_functions_from_contract` to `fetch_functions_with_retries` * Using pattern matching instead of guard in `fetch_functions_with_retries`	6 years ago