365 Commits (e1c7ef8c739c8c0017d8db1e5c48c7f48505f5bb)

Author	SHA1	Message	Date
pasqu4le	64acef7b71	Add temporary indexer worker to remove consensus on blocks with mismatching transactions ... After the solution for #1991 it is necessary to remove consensus from blocks whose number of transactions mismatch compared to a node, so that these blocks are refetched.	6 years ago
Ayrat Badykov	88a2a528f7	fix build	6 years ago
Ayrat Badykov	e49412f389	remove temporary worker modules	6 years ago
saneery	32498b5b29	fix typo	6 years ago
goodsoft	8e6ee4ceb1	Use eth_getUncleByBlockHashAndIndex for uncle block fetching ... Parity returns `null` for many uncle blocks, if they are fetched using `eth_getBlockByHash`. This is most probably caused by the fact that Parity only keeps full block data for recent non-consensus blocks. This causes uncle_block fetcher to loop endlessly trying to fetch "missing" blocks, hogging both app server and Parity resources. Instead, we use `eth_getUncleByBlockHashAndIndex` method, which works for all uncle blocks. As we didn't previously store index of an uncle block within a nephew block, a new field and a temporary fixup fetcher is added to get the index from nephew blocks.	6 years ago
saneery	fc79dfb9eb	fix credo warnings	6 years ago
saneery	b158d71219	add staking pools fetcher test	6 years ago
saneery	009a54466e	fix staking pools fetcher	6 years ago
saneery	e9c61945e6	add staking pools fetcher	6 years ago
zachdaniel	e4d0fef374	feat: set a ping_interval from config, defaulting to 300 seconds	6 years ago
Paul Tsupikoff	b8a8a008ec	Make websocket optional for realtime fetcher (#1778) ... Realtime fetcher already uses both websocket and JSONRPC polling for more reliable new block fetching. Though, if no websocket was available, the realtime fetcher wouldn't start, and no new blocks arrived at all. As we already have a polling fallback, we now make websocket optional. This will also help with supporting nodes without websocket API. To disable websocket API, pass `nil` in `subscribe_named_arguments.transport`.	6 years ago
Ayrat Badykov	cb920e29f4	fix first block parameter	6 years ago
zachdaniel	8c1bb01fb1	feat: Add RSK support	6 years ago
Paul Tsupikoff	f02e2d8841	Remove internal tx and token balance fetching from realtime fetcher (#1724) ... Internal transactions and token balances are the slowest to fetch and import. We'd rather have almost instant import of almost all block-related info than frequent timeouts in realtime fetcher and handing blocks over to catchup fetcher which imports this data asynchronously anyway.	6 years ago
Ayrat Badykov	8839e18290	fix empty block time	6 years ago
Paul Tsupikoff	c423a41e49	Flatten indexer module hierarchy and supervisor tree (#1718) ... * Add a defsupervisor macro with common supervisor functionality This macro allows removing more than a dozen of copy-pasted `supervisor.ex` modules. The `Indexer.Fetcher` module will be expanded with even more common fetcher functionality in upcoming PRs. * Flatten indexer module hierarchy and remove generated supervisors Modules are organized into 3 directories: `fetcher`, `transform` and `temporary`. Supervisors are replaced with `use Indexer.Fetcher` clause. `token_balances.ex` and block fetchers are left as-is for now, as they require more invasive code refactoring, which will be done in upcoming PRs. * Flatten supervisor tree There is no need in separate `Block.Supervisor` hierarchy, so its children are placed in the root supervisor tree. The tree itself is moved to `Indexer.Supervisor` module to reduce amount of directories. * Add CHANGELOG.md entry	6 years ago
Ayrat Badykov	198ebcd140	do not set contract code from transactio input	6 years ago
goodsoft	0e15873fbc	Remove obsolete InvalidConsensus.Worker ... This worker only handled one of invalid consensus cases, i.e. when a parent block lost consensus, but the child one didn't. Even then it worked only when all blocks are reliably and sequentially imported (certainly not our case), and only once on indexer launch. Now this particular case is covered by previous commit, and we don't need this worker at all.	6 years ago
goodsoft	9dbf51db94	Remove obsolete ConsensusEnsurer ... As a result of #1657 out-of-place parent blocks are marked as non-consensus more reliably, i.e. they will eventually be fetched even if first attempt is unsuccessful. The `ConsensusEnsurer` module, on the contrary, only tried refetching block once, and left it as-is if the refetch failed. Now it is not needed anymore.	6 years ago
Ayrat Badykov	76fee9241c	mix format	6 years ago
Ayrat Badykov	a6a0d4c5a9	fix test	6 years ago
Ayrat Badykov	318dbae4a9	update gettext	6 years ago
Ayrat Badykov	f34e88d8ce	remove fetching token 2 times	6 years ago
Ayrat Badykov	e5f4ba9b8e	update tokens in fetcher	6 years ago
goodsoft	8cbafd2ced	Set internal_transactions_indexed_at for empty blocks ... As there exist blocks without transactions we now pass the list of blocks to import routine instead of deriving this list from the list of fetched internal transactions.	6 years ago
Ayrat Badykov	25564c7fac	add infinite timeout for Repo.transaction	6 years ago
Ayrat Badykov	1ce53363e5	set infinity timeout for addresses without code	6 years ago
Ayrat Badykov	a8b108f299	modify query to fetch failed contract creations	6 years ago
Ayrat Badykov	c2804dbfbf	add more logging to code fixer process	6 years ago
Ayrat Badykov	456c5549a5	do not fetch burn address balance	6 years ago
Ayrat Badykov	946b596abc	allow to set first indexing block	6 years ago
Ayrat Badykov	99e30c5a52	select queries by batches	6 years ago
Ayrat Badykov	c42f81f5ca	fix credo	6 years ago
Paul Tsupikoff	5ed8a9590d	Reduce block_rewards in the catchup fetcher ... Fixes #1584 If one address is rewarded for two different uncles in one block, cardinality violation occured due to rewards not being reduced. The reducing function from Block.Rewards.Fetcher is now reused in Block.Fetcher.	6 years ago
saneery	5c2e2f2309	Added guard for check block number before fetch it	6 years ago
Ayrat Badykov	316a07663f	add second query	6 years ago
Ayrat Badykov	0e6d2d4d0d	fix merge conflicts	6 years ago
Ayrat Badykov	b03558b755	fix tests	6 years ago
Ayrat Badykov	8f0222efa3	mix format	6 years ago
Ayrat Badykov	2d51a7f76c	add logging	6 years ago
Ayrat Badykov	06544e089a	check if code is set	6 years ago
Ayrat Badykov	cffd555137	add process to fix contract with code	6 years ago
nervehammer	d9387ae495	add option to disable block rewards	6 years ago
zachdaniel	75b558b500	feat: allow setting different configuration just for realtime fetcher	6 years ago
goodsoft	7f43ad5850	Don't refetch internal transactions in FailedCreatedAddress fetcher ... We refetch all internal transactions anyway.	6 years ago
goodsoft	ea3af5e299	Use transaction-level fetch_internal_transactions for Geth ... Internal transaction fetcher is split into two APIs: * `fetch_block_internal_transactions` is used for Parity * `fetch_internal_transactions` is used for the rest of variants `Indexer.InternalTransaction.Fetcher` is updated to take care of that.	6 years ago
Paul Tsupikoff	e13f6f8a8e	Use trace_replayBlockTransactions API for faster tracing	6 years ago
Paul Tsupikoff	2a8def19e2	Allow executing smart contract functions in arbitrarily sized batches ... # Problem There are a lot of `getBalanceOf` token function calls during indexing blockchain. Each of them is sent as a separate JSONRPC request, which creates a lot of unnecessary overhead and clogs archive node workers, which could otherwise be busy performing other JSONRPC queries. Moreover, function name is used as the `id` parameter of JSONRPC queries, which doesn't allow us to simply put multiple requests into one batch. # Solution * Change `EthereumJSONRPC.execute_contract_functions` to accept a list of arbitrary function calls and return a list of results. * Handle `id` field generation in the `EthereumJSONRPC` package instead of leaking implementation details to `Explorer.SmartContract.Reader`. * Change `Token.BalanceReader.get_balance_of` to accept a list of parameter sets and return a list of balances. * Make one batch request instead of launching multiple concurrent tasks in `Indexer.TokenBalances.fetch_token_balances_from_blockchain` # Notes * Protocol details of calling smart contract functions is now fully encapsulated in `EthereumJSONRPC` package. * `SmartContract.Reader.query_contract` is left for backwards compatibility with other existing applications of smart contract requests, but can be fully deprecated at some later time.	6 years ago
goodsoft	f06787fd7f	Fix scheduling of latest block polling in Realtime Fetcher ... Timer is set using Elixir's `Process.send_after/3` method, which is incompatible with Erlang's `:timer` module. Thus, `:timer.cancel` doesn't have any effect on it, and in fact returns `{:error, :badarg}`. This causes more and more polling being scheduled after every websocket event, which might cause excessive flooding of a node with `eth_getBlockByNumber` requests after some running time.	6 years ago
Ayrat Badykov	714db43a14	fetch ERC721 token balances if senser is '0x0..0'	6 years ago