Blockchain explorer for Ethereum based network and a tool for inspecting and analyzing EVM based blockchains.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
blockscout/apps/ethereum_jsonrpc/lib/ethereum_jsonrpc.ex

744 lines
27 KiB

defmodule EthereumJSONRPC do
@moduledoc """
Ethereum JSONRPC client.
## Configuration
Configuration for Nethermind URLs can be provided with the following mix config:
config :ethereum_jsonrpc,
url: "http://localhost:8545",
trace_url: "http://localhost:8545",
http: [recv_timeout: 60_000, timeout: 60_000, hackney: [pool: :ethereum_jsonrpc]]
Note: the tracing node URL is provided separately from `:url`, via `:trace_url`. The trace URL and is used for
`fetch_internal_transactions`, which is only a supported method on tracing nodes. The `:http` option is passed
directly to the HTTP library (`HTTPoison`), which forwards the options down to `:hackney`.
## Throttling
Requests for fetching blockchain can put a lot of CPU pressure on JSON RPC
nodes. EthereumJSONRPC will check for request timeouts as well as bad-gateway
responses and add delay between requests until the JSON RPC nodes reach
6 years ago
stability. For finer tuning and configuration of throttling, read the
documentation for `EthereumJSONRPC.RequestCoordinator`.
"""
require Logger
alias EthereumJSONRPC.{
Block,
Blocks,
Contract,
FetchedBalances,
FetchedBeneficiaries,
6 years ago
FetchedCodes,
Receipts,
RequestCoordinator,
Subscription,
Transport,
Utility.CommonHelper,
Utility.EndpointAvailabilityObserver,
Utility.RangesHelper,
Variant
}
@default_throttle_timeout :timer.minutes(2)
@typedoc """
Truncated 20-byte [KECCAK-256](https://en.wikipedia.org/wiki/SHA-3) hash encoded as a hexadecimal number in a
`String.t`.
"""
@type address :: String.t()
@typedoc """
A block number as an Elixir `t:non_neg_integer/0` instead of `t:data/0`.
"""
@type block_number :: non_neg_integer()
@typedoc """
Reference to an uncle block by nephew block's `hash` and `index` in it.
"""
@type nephew_index :: %{required(:nephew_hash) => String.t(), required(:index) => non_neg_integer()}
@typedoc """
Binary data encoded as a single hexadecimal number in a `String.t`
"""
@type data :: String.t()
6 years ago
@typedoc """
Contract code encoded as a single hexadecimal number in a `String.t`
"""
@type code :: String.t()
@typedoc """
A full 32-byte [KECCAK-256](https://en.wikipedia.org/wiki/SHA-3) hash encoded as a hexadecimal number in a `String.t`
## Example
"0xe670ec64341771606e55d6b4ca35a1a6b75ee3d5145a99d05921026d1527331"
"""
@type hash :: String.t()
@typedoc """
Named arguments to `json_rpc/2`.
* `:transport` - the `t:EthereumJSONRPC.Transport.t/0` callback module
* `:transport_options` - options passed to `c:EthereumJSONRPC.Transport.json_rpc/2`
* `:variant` - the `t:EthereumJSONRPC.Variant.t/0` callback module
* `:throttle_timeout` - the maximum amount of time in milliseconds to throttle
before automatically returning a timeout. Defaults to #{@default_throttle_timeout} milliseconds.
"""
@type json_rpc_named_arguments :: [
{:transport, Transport.t()}
| {:transport_options, Transport.options()}
| {:variant, Variant.t()}
| {:throttle_timeout, non_neg_integer()}
]
@typedoc """
Named arguments to `subscribe/2`.
* `:transport` - the `t:EthereumJSONRPC.Transport.t/0` callback module
* `:transport_options` - options passed to `c:EthereumJSONRPC.Transport.json_rpc/2`
* `:variant` - the `t:EthereumJSONRPC.Variant.t/0` callback module
"""
@type subscribe_named_arguments :: [
{:transport, Transport.t()} | {:transport_options, Transport.options()} | {:variant, Variant.t()}
]
@typedoc """
8 byte [KECCAK-256](https://en.wikipedia.org/wiki/SHA-3) hash of the proof-of-work.
"""
@type nonce :: String.t()
@typedoc """
A number encoded as a hexadecimal number in a `String.t`
## Example
"0x1b4"
"""
@type quantity :: String.t()
@typedoc """
A logic block tag that can be used in place of a block number.
| Tag | Description |
|--------------|--------------------------------|
| `"earliest"` | The first block in the chain |
| `"latest"` | The latest collated block. |
| `"pending"` | The next block to be collated. |
"""
@type tag :: String.t()
@typedoc """
Unix timestamp encoded as a hexadecimal number in a `String.t`
"""
@type timestamp :: String.t()
@typedoc """
JSONRPC request id can be a `String.t` or Integer
"""
@type request_id :: String.t() | non_neg_integer()
@doc """
Execute smart contract functions.
Receives a list of smart contract functions to execute. Each function is
represented by a map. The contract_address key is the address of the smart
contract being queried, the data key indicates which function should be
executed, as well as what are their arguments, and the id key is the id that
is going to be sent with the JSON-RPC call.
## Examples
Execute the "sum" function that receives two arguments (20 and 22) and returns their sum (42):
iex> EthereumJSONRPC.execute_contract_functions([%{
...> contract_address: "0x7e50612682b8ee2a8bb94774d50d6c2955726526",
...> data: "0xcad0899b00000000000000000000000000000000000000000000000000000000000000140000000000000000000000000000000000000000000000000000000000000016",
...> id: "sum"
...> }])
{:ok,
[
%{
"id" => "sum",
"jsonrpc" => "2.0",
"result" => "0x000000000000000000000000000000000000000000000000000000000000002a"
}
]}
"""
@spec execute_contract_functions([Contract.call()], [map()], json_rpc_named_arguments) :: [Contract.call_result()]
def execute_contract_functions(functions, abi, json_rpc_named_arguments, leave_error_as_map \\ false) do
if Enum.empty?(functions) do
[]
else
Contract.execute_contract_functions(functions, abi, json_rpc_named_arguments, leave_error_as_map)
end
end
@doc """
fetched_balance_block_number instead of fetched_balance_at Fixes #243 Instead of getting the address balance at the latest block and recording the timestamp of the insert, which is only loosely correlated with even the latest block's timestamp on-chain, use the block number for the last time an address was used in the indexed address foreign keys in the chain: * blocks * miner_hash * internal_transactions * created_contract_address_hash * from_address_hash * to_address_hash * logs * address_hash * transactions * from_address_hash * to_address_hash To gather `fetched_balance_block_number` on reboot, `Explorer.Chain.stream_unfetched_addresses` queries across the above columns for the `MAX(blocks.number)`, which is used in `Explorer.Indexer.AddressBalanceFetcher.init/2`. During indexing, `Explorer.Indexer.AddressBalanceFetcher.async_fetch_balances/1` now requires a list of `%{block_number: Block.block_number(), hash: Hash.Truncated.t()}` instead of list of `Hash.Truncated.t()`. Having a `block_number` available for all importable entities's addresses required changing `Explorer.Indexer.AddressExtraction` to extract the `block_number` too. The use of `insert` in the factories was making tests that show the count not make sense, so the factories were switched to `build` because it means only the exact count is created when a default value is overridden. This does, unfortunately, mean that `params_for(factory)` no longer generates foreign keys and they must be set explicitly OR the association passed instead of the foreign key. Allow nil extracted fetched_balance_block_number when pending transaction *NOTE*: This is a database drop and reindex change.
7 years ago
Fetches balance for each address `hash` at the `block_number`
"""
@spec fetch_balances(
[%{required(:block_quantity) => quantity, required(:hash_data) => data()}],
json_rpc_named_arguments
) :: {:ok, FetchedBalances.t()} | {:error, reason :: term}
def fetch_balances(params_list, json_rpc_named_arguments, latest_block_number \\ 0, chunk_size \\ nil)
when is_list(params_list) and is_list(json_rpc_named_arguments) do
latest_block_number_params =
case latest_block_number do
0 -> fetch_block_number_by_tag("latest", json_rpc_named_arguments)
number -> {:ok, number}
end
params_in_range =
params_list
|> Enum.filter(fn
%{block_quantity: block_quantity} ->
block_quantity |> quantity_to_integer() |> RangesHelper.traceable_block_number?()
end)
trace_url_used? = !is_nil(json_rpc_named_arguments[:transport_options][:method_to_url][:eth_getBalance])
archive_disabled? = Application.get_env(:ethereum_jsonrpc, :disable_archive_balances?)
{latest_balances_params, archive_balance_params} =
with true <- not trace_url_used? or archive_disabled?,
{:ok, max_block_number} <- latest_block_number_params do
window = Application.get_env(:ethereum_jsonrpc, :archive_balances_window)
Enum.split_with(params_in_range, fn
%{block_quantity: "latest"} -> true
%{block_quantity: block_quantity} -> quantity_to_integer(block_quantity) > max_block_number - window
_ -> false
end)
else
_ -> {params_in_range, []}
end
latest_id_to_params = id_to_params(latest_balances_params)
archive_id_to_params = id_to_params(archive_balance_params)
with {:ok, latest_responses} <- do_balances_request(latest_id_to_params, chunk_size, json_rpc_named_arguments),
{:ok, archive_responses} <-
maybe_request_archive_balances(
archive_id_to_params,
trace_url_used?,
archive_disabled?,
chunk_size,
json_rpc_named_arguments
) do
latest_fetched_balances = FetchedBalances.from_responses(latest_responses, latest_id_to_params)
archive_fetched_balances = FetchedBalances.from_responses(archive_responses, archive_id_to_params)
{:ok, FetchedBalances.merge(latest_fetched_balances, archive_fetched_balances)}
fetched_balance_block_number instead of fetched_balance_at Fixes #243 Instead of getting the address balance at the latest block and recording the timestamp of the insert, which is only loosely correlated with even the latest block's timestamp on-chain, use the block number for the last time an address was used in the indexed address foreign keys in the chain: * blocks * miner_hash * internal_transactions * created_contract_address_hash * from_address_hash * to_address_hash * logs * address_hash * transactions * from_address_hash * to_address_hash To gather `fetched_balance_block_number` on reboot, `Explorer.Chain.stream_unfetched_addresses` queries across the above columns for the `MAX(blocks.number)`, which is used in `Explorer.Indexer.AddressBalanceFetcher.init/2`. During indexing, `Explorer.Indexer.AddressBalanceFetcher.async_fetch_balances/1` now requires a list of `%{block_number: Block.block_number(), hash: Hash.Truncated.t()}` instead of list of `Hash.Truncated.t()`. Having a `block_number` available for all importable entities's addresses required changing `Explorer.Indexer.AddressExtraction` to extract the `block_number` too. The use of `insert` in the factories was making tests that show the count not make sense, so the factories were switched to `build` because it means only the exact count is created when a default value is overridden. This does, unfortunately, mean that `params_for(factory)` no longer generates foreign keys and they must be set explicitly OR the association passed instead of the foreign key. Allow nil extracted fetched_balance_block_number when pending transaction *NOTE*: This is a database drop and reindex change.
7 years ago
end
end
6 years ago
@doc """
feat: EIP-7702 support (#10870) * feat: return `authorizationList` for EIP-7702 transactions in `/transactions/:tx_hash` response (#10776) * feat: support EIP-7702 transactions * fix: handle invalid signatures * fix: save authority * Update apps/block_scout_web/lib/block_scout_web/views/api/v2/transaction_view.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * Update apps/explorer/lib/explorer/chain/signed_authorization.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * Update apps/indexer/lib/indexer/block/fetcher.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * fix: remove set_code_transaction from @allowed_type_labels * Update apps/explorer/lib/explorer/chain/import/runner/signed_authorizations.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * fix: move signed_authorization to a separate module * add todo --------- Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * feat: support EIP-7702 in `/address/:address` endpoint (#10799) * feat: support EIP-7702 transactions * fix: handle invalid signatures * fix: save authority * Update apps/block_scout_web/lib/block_scout_web/views/api/v2/transaction_view.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * Update apps/explorer/lib/explorer/chain/signed_authorization.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * Update apps/indexer/lib/indexer/block/fetcher.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * fix: remove set_code_transaction from @allowed_type_labels * feat: add EIP-7702 support to /address/:address endpoint * fix: refactor fetch? * fix: move get_implementation_address_hash_string_eip7702 * fix: remove EIP-7702 flag from response, modify transactions filter to handle EOA with code correctly * fix: minor refactoring * fix: remove unused alias * fix: review comments * Update apps/explorer/priv/repo/migrations/20240904161254_create_signed_authorizations.exs Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> --------- Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * fix: remove unused code * fix: refactor code * Update apps/ethereum_jsonrpc/lib/ethereum_jsonrpc/signed_authorization.ex Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com> * chore: documentation improvement * fix typo * fix: move spec and doc * fix: authorization_list spec * fix: wrap address and authority with checksum * fix: invalid typespec Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com> * Apply suggestions from code review Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com> * fix typo * Update apps/explorer/lib/explorer/chain/smart_contract/proxy/eip_7702.ex Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com> --------- Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com>
1 month ago
Fetches contract code for multiple addresses at specified block numbers.
This function takes a list of parameters, each containing an address and a
block number, and retrieves the contract code for each address at the
specified block.
## Parameters
- `params_list`: A list of maps, each containing:
- `:block_quantity`: The block number (as a quantity string) at which to fetch the code.
- `:address`: The address of the contract to fetch the code for.
- `json_rpc_named_arguments`: A keyword list of JSON-RPC configuration options.
## Returns
- `{:ok, fetched_codes}`, where `fetched_codes` is a `FetchedCodes.t()` struct containing:
- `params_list`: A list of successfully fetched code parameters, each containing:
- `address`: The contract address.
- `block_number`: The block number at which the code was fetched.
- `code`: The fetched contract code in hexadecimal format.
- `errors`: A list of errors encountered during the fetch operation.
- `{:error, reason}`: An error occurred during the fetch operation.
6 years ago
"""
@spec fetch_codes(
[%{required(:block_quantity) => quantity, required(:address) => address()}],
json_rpc_named_arguments
) :: {:ok, FetchedCodes.t()} | {:error, reason :: term}
def fetch_codes(params_list, json_rpc_named_arguments)
when is_list(params_list) and is_list(json_rpc_named_arguments) do
id_to_params = id_to_params(params_list)
with {:ok, responses} <-
id_to_params
|> FetchedCodes.requests()
|> json_rpc(json_rpc_named_arguments) do
{:ok, FetchedCodes.from_responses(responses, id_to_params)}
end
end
@doc """
Fetches block reward contract beneficiaries from variant API.
"""
@spec fetch_beneficiaries([block_number], json_rpc_named_arguments) ::
{:ok, FetchedBeneficiaries.t()} | {:error, reason :: term} | :ignore
def fetch_beneficiaries(block_numbers, json_rpc_named_arguments) when is_list(block_numbers) do
filtered_block_numbers = RangesHelper.filter_traceable_block_numbers(block_numbers)
Keyword.fetch!(json_rpc_named_arguments, :variant).fetch_beneficiaries(
filtered_block_numbers,
json_rpc_named_arguments
)
end
@doc """
Fetches blocks by block hashes.
Transaction data is included for each block.
"""
@spec fetch_blocks_by_hash([hash()], json_rpc_named_arguments) :: {:ok, Blocks.t()} | {:error, reason :: term}
def fetch_blocks_by_hash(block_hashes, json_rpc_named_arguments) do
block_hashes
|> Enum.map(fn block_hash -> %{hash: block_hash} end)
|> fetch_blocks_by_params(&Block.ByHash.request/1, json_rpc_named_arguments)
end
@doc """
Fetches blocks by block number range.
"""
@spec fetch_blocks_by_range(Range.t(), json_rpc_named_arguments) :: {:ok, Blocks.t()} | {:error, reason :: term}
def fetch_blocks_by_range(_first.._last//_ = range, json_rpc_named_arguments) do
range
|> Enum.map(fn number -> %{number: number} end)
|> fetch_blocks_by_params(&Block.ByNumber.request/1, json_rpc_named_arguments)
end
@doc """
Fetches blocks by block number list.
"""
@spec fetch_blocks_by_numbers([block_number()], json_rpc_named_arguments, boolean()) ::
{:ok, Blocks.t()} | {:error, reason :: term}
def fetch_blocks_by_numbers(block_numbers, json_rpc_named_arguments, with_transactions? \\ true) do
block_numbers
|> Enum.map(fn number -> %{number: number} end)
|> fetch_blocks_by_params(&Block.ByNumber.request(&1, with_transactions?), json_rpc_named_arguments)
end
@doc """
Fetches block by "t:tag/0".
"""
@spec fetch_block_by_tag(tag(), json_rpc_named_arguments) ::
{:ok, Blocks.t()} | {:error, reason :: :invalid_tag | :not_found | term()}
Deposits and Withdrawals for Polygon Edge (#8180) * Prepare tables for Polygon Supernet Withdrawals module * Prepare Indexer.Fetcher.PolygonSupernetWithdrawal * Prepare Indexer.Fetcher.PolygonSupernetWithdrawalExit * Prepare tables for Polygon Supernet Deposits module * Partially add Polygon Supernet Deposits module * Add Polygon Supernet Deposits module * Extend transaction data in API v2 for Polygon Supernet Deposits and Withdrawals * Extend API v2 for list of Polygon Supernet Deposits * Extend API v2 for list of Polygon Supernet Withdrawals * Modify collated_gas_price constraint to support Immutable test chain * Update number of retries * Fix Indexer.Fetcher.PolygonSupernetWithdrawal * Fix API v2 for Indexer.Fetcher.PolygonSupernetWithdrawal * Return page_size to 50 * Fix API v2 for Indexer.Fetcher.PolygonSupernetDeposit * Fix Explorer.Validator.MetadataRetriever to prevent sending request to nil address * Refactor PolygonSupernet* modules * Refactor PolygonSupernet* modules * Small refactoring of PolygonSupernet* modules * Update changelog * Fixes for dialyzer and cspell * Fix of Explorer tests * Add env vars for docker * Add env variable for eth_getLogs * Share event signatures between modules * Refactoring for fill_block_range function * Small refactoring of transaction actions module * Move polygon_supernet_* modules to a separate folder * Add specs for public functions * Remove redundant function * Reuse decode_data function * Move polygon_supernet_* files into a separate folder * Move polygon_supernet_* files into a separate folder * Partially disband Explorer.Chain module * Define chain type * Remove infinite waiting for safe block * max format * Rename Polygon Supernet to Polygon Edge * Set INDEXER_INTERNAL_TRANSACTIONS_TRACER_TYPE to polygon_edge if CHAIN_TYPE is polygon_edge * Fix chain type definition * Fix repos list definition in releases_tasks * INDEXER_POLYGON_SUPERNET_ -> INDEXER_POLYGON_EDGE_ --------- Co-authored-by: POA <33550681+poa@users.noreply.github.com> Co-authored-by: Viktor Baranov <baranov.viktor.27@gmail.com>
1 year ago
def fetch_block_by_tag(tag, json_rpc_named_arguments) when tag in ~w(earliest latest pending safe) do
[%{tag: tag}]
|> fetch_blocks_by_params(&Block.ByTag.request/1, json_rpc_named_arguments)
end
@doc """
Fetches uncle blocks by nephew hashes and indices.
"""
@spec fetch_uncle_blocks([nephew_index()], json_rpc_named_arguments) :: {:ok, Blocks.t()} | {:error, reason :: term}
def fetch_uncle_blocks(blocks, json_rpc_named_arguments) do
blocks
|> fetch_blocks_by_params(&Block.ByNephew.request/1, json_rpc_named_arguments)
end
@spec fetch_net_version(json_rpc_named_arguments) :: {:ok, non_neg_integer()} | {:error, reason :: term}
def fetch_net_version(json_rpc_named_arguments) do
result =
%{id: 0, method: "net_version", params: []}
|> request()
|> json_rpc(json_rpc_named_arguments)
case result do
{:ok, bin_number} -> {:ok, String.to_integer(bin_number)}
other -> other
end
end
@doc """
Fetches block number by `t:tag/0`.
## Returns
* `{:ok, number}` - the block number for the given `tag`.
* `{:error, :invalid_tag}` - When `tag` is not a valid `t:tag/0`.
* `{:error, reason}` - other JSONRPC error.
"""
Optimism chain type (#9460) * Transaction page L1 fields * Path fix * Reduce the number of files from 19 to 5 in logs rotate config * Customize optimism-goerli deployment * Optimism branding * Remove testnet logo text. OG uses customized label * Fix Circles theme * L1 tx fields fix for Optimism BedRock update * Remove redundant line * Add gas_price handling for elixir_to_params and change function ordering * Remove l1TxOrigin handling for another version of RPC * Add GA * Fix realtime fetcher test * Update Changelog * Fix internal transactions processing for non-consensus blocks * Lose consensus only for consensus=true blocks * Fix handling transaction RPC responses without some fields * Fix tests except for indexer module * Add Optimism BedRock support (Txn Batches, Output Roots, Deposits, Withdrawals) (#6980) * Add op_output_roots table * Add OptimismOutputRoots runner * Add initial code for output roots fetcher * Add checks to init function * Partially add logs and L1 reorgs handling * Add reorgs handling * Add RPC retries * Write output roots to database * Log output roots handling * Update indexer README * Add API v2 for Optimism Output Roots * Add op_withdrawals table * Add OptimismWithdrawals runner * Prepare realtime optimism withdrawals fetching * Add realtime optimism withdrawals fetching * Define checks in init function * log.first_topic can be nil * Show total count of output roots in API v2 * Add msg_nonce gaps filler * Refactoring * Intermediate refactoring * Add historical withdrawals handling and refactor * Finish op_withdrawals table filling * Small refactoring * Add op_withdrawal_events table * Add OptimismWithdrawalEvents runner * Add OptimismWithdrawalEvent fetcher * Update indexer README * Add API v2 for Optimism Withdrawals * Add env variables to common-blockscout.env and Makefile * Set `from` as address object instead of just address hash for withdrawal * mix format * Add op_transaction_batches table * Add OptimismTxnBatches runner * Add a draft for OptimismTxnBatch fetcher * Add a draft for OptimismTxnBatch * Extend a draft for OptimismTxnBatch * Extend OptimismTxnBatch * Finish OptimismTxnBatch (without reorgs handling yet) * Optimize OptimismTxnBatch fetcher * Remove duplicated txn batches * Add zlib_inflate_handler for empty case * Add reorgs handling for txn batches * Fix reorgs handling for txn batches * Small refactor * Finish Indexer.Fetcher.OptimismTxnBatch (without refactoring yet) * Apply new ex_rlp version * Add API v2 for Optimism Txn Batches * Add env variables to common-blockscout.env and Makefile * Refactor OptimismTxnBatch fetcher for mix credo * Replace binary_slice with binary_part function to run with Elixir 1.13 * Update changelog * Update indexer readme * Rename op_withdrawals.l2_tx_hash field to l2_transaction_hash * Rename l1_tx_hash fields to l1_transaction_hash * Rename *tx* fields to *transaction* fields * Rename env variables * Rename env variables * Add an indexer helper * Add an indexer helper * Small refactoring * Fix tx_count for txn batches view * Use EthereumJSONRPC.Block.ByHash instead of the raw call * Infinity timeout for blocks query * Small refactoring * Refactor init function for two modules * Small refactoring * Rename l1_transaction_timestamp field to l1_timestamp * Rename withdrawal_hash field to hash * Refactor for decode_data function * Refactor for mix credo * Add INDEXER_OPTIMISM_L1_BATCH_BLOCKS_CHUNK_SIZE env and small refactoring * Add INDEXER_OPTIMISM_L1_BATCH_BLOCKS_CHUNK_SIZE env to other files * Add an index for l1_block_number field * Add an index for l1_block_number field * Remove redundant :ok * Use faster way to count rows in a table * Refactor reorgs monitor functions * Clarify frame structure * Reduce storage consumption for optimism transaction batches * Reuse CacheHelper.estimated_count_from function * Bedrock optimism deposits (#6993) * Create `op_deposits` table * Add OptimismDeposit runner * WIP Fetcher * Finish fetcher * Integrate deposits into APIv2 * Add envs * Fix requests * Remove debug * Update envs names * Rename `tx` -> `transaction` * Reuse `decode_data/2` * Fix review * Add `uninstall_filter` * Fix formatting * Switch to realtime mode more carefully * Fix review Allow nil in timestamp Add progress logging Improve check_interval calculation * Fix logging and env * Fix Association.NotLoaded error * Replace switching to realtime mode log * Remove excess start_block * Fix reorg logging * Fix `from_block` > `to_block` and add realtime logging * Fix block boundaries --------- Co-authored-by: POA <33550681+poa@users.noreply.github.com> * mix format * Return total count of L2 entity by a separate API request * Filter by consensus blocks * Parallelize tx count operation and small refactoring * Use read replica for L2 entities in API * Parse block_number and tx_hash for Optimism Deposits module * Return page_size back to 50 * Small fixes and refactoring * Update apps/block_scout_web/lib/block_scout_web/api_router.ex Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com> * Small optimization * Use ecto association instead of explicit join for txn batches * Refactoring * Use Stream inspead of Enum * Small refactoring * Add assoc for transaction batches in OptimismFrameSequence * Use common reorg monitor for Optimism modules * Rename Explorer.Helpers to Explorer.Helper * Don't start an optimism module unless the main optimism module is not started * Don't start reorg monitor for optimism modules when it is not needed * Small refactoring * Remove debug broadcasting * Add Optimism BedRock Deposits to the main page in API (#7200) * Add Optimism BedRock Deposits to the main page in API * Update changelog * Pass the number of deposits instead of only one item per once --------- Co-authored-by: POA <33550681+poa@users.noreply.github.com> * Refactor for credo * Output L1 fields in API v2 for transaction page * Update changelog * Use helper * Refactor Indexer.Fetcher.Optimism * Fix l1_timestamp issue in OptimismTxnBatch fetcher * Reset Logger metadata before Indexer.Transform.OptimismWithdrawals.parse function finishes * Fix IDs ordering in remove_duplicates function of Indexer.Fetcher.OptimismTxnBatch * Consider rewriting of the first frame in Indexer.Fetcher.OptimismTxnBatch * Fix Indexer.Fetcher.OptimismTxnBatch (consider chunking) * Fix Indexer.Fetcher.OptimismTxnBatch * Fix handling invalid frame sequences in Indexer.Fetcher.OptimismTxnBatch * Read Optimism finalization period from a smart contract * Fixes for dialyzer * Fix for EthereumJSONRPC tests * Fixes for Explorer tests * Fixes for Explorer tests * Fix of block/realtime/fetcher_test.exs * mix format and small fixes for block_scout_web tests * Reset GA cache * Fix handling nil in PendingBlockOperation.estimated_count() --------- Co-authored-by: POA <33550681+poa@users.noreply.github.com> Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com> * Fix autocomplete * Fix merging conflicts * Add exit handler to Indexer.Fetcher.OptimismWithdrawal * Fix transactions ordering in Indexer.Fetcher.OptimismTxnBatch * Update changelog * Refactor to fix credo * Mix credo fix * Fix transaction batches module for L2 OP stack (#7827) * Fix mixed transactions handling in Indexer.Fetcher.OptimismTxnBatch * Ignore duplicated frame * Update changelog * Add sorting to the future frames list * Change list order --------- Co-authored-by: POA <33550681+poa@users.noreply.github.com> * Remove unused aliases * Ignore previously handled frame by OP transaction batches module (#8122) * Ignore duplicated frame * Update changelog --------- Co-authored-by: POA <33550681+poa@users.noreply.github.com> * Return alias for Explorer.Chain.Cache.Helper in chain.ex * Ignore invalid frame by OP transaction batches module (#8208) * Update changelog * Ignore invalid frame * Update changelog --------- Co-authored-by: POA <33550681+poa@users.noreply.github.com> * Fix Indexer.Fetcher.OptimismTxnBatch * Fix API v2 for OP Withdrawals * Refactor optimism fetchers init * Add log for switching from fallback url * Fix for Indexer.Fetcher.OptimismTxnBatch * Add OP withdrawal status to transaction page in API (#8702) * Add OP withdrawal status to transaction page in API * Update changelog * Small refactoring * Update .dialyzer-ignore --------- Co-authored-by: POA <33550681+poa@users.noreply.github.com> * Add start pause to `Indexer.Fetcher.OptimismTxnBatch` * Small refactor of `Indexer.Fetcher.OptimismTxnBatch` * Consider consensus block only when retrieving OP withdrawal transaction status (#8811) * Consider consensus block only when retrieving OP withdrawal transaction status * Update changelog * Clear GA cache --------- Co-authored-by: POA <33550681+poa@users.noreply.github.com> * Hotfix for optimism_withdrawal_transaction_status function * Return all OP Withdrawals bound to L2 transaction * Try to import config * Remove unused functions from Explorer.Chain * Refactor for mix credo * Fix order of proxy standards: 1167, 1967 * Fixes in Optimism due to changed log topics type * Fix for EthereumJSONRPC tests * Clear GA cache and update cspell.json * Fix indexer tests * Return current exchange rate in api/v2/stats * Fix log decoding bug * Temp disable build of image for arm64 * Rewrite Indexer.Fetcher.OptimismTxnBatch module * Add handling of span batches * Add support of latest block for Optimism modules * Update changelog and spelling * Rewrite Indexer.Fetcher.OptimismTxnBatch module * Add handling of span batches * Add support of latest block for Optimism modules * Refactoring * Partially add specs and docs for public functions * Refactoring * add an entry to CHANEGELOG.md * apply review (use origin entity instead of joined entity in with tx status) * Fixes after rebase * Remove old UI sustomizations * Optimism chain type * Change structure of folders * Fixes after review * Fix CHANGELOG * Fixes after 2nd review * Process 3d review: add tests for fee/2 function * Process 4th review * Review fix: move Op related functions from chain.ex * Review fix: make OptimismFinalizationPeriod configurable * Process review comment * System.get_env("CHAIN_TYPE") => Application.get_env(:explorer, :chain_type) --------- Co-authored-by: POA <33550681+poa@users.noreply.github.com> Co-authored-by: Qwerty5Uiop <alex000010@bk.ru> Co-authored-by: varasev <33550681+varasev@users.noreply.github.com> Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com> Co-authored-by: rlgns98kr <rlgns98kr@gmail.com>
9 months ago
@spec fetch_block_number_by_tag_op_version(tag(), json_rpc_named_arguments) ::
{:ok, non_neg_integer()} | {:error, reason :: :invalid_tag | :not_found | term()}
def fetch_block_number_by_tag_op_version(tag, json_rpc_named_arguments)
when tag in ~w(earliest latest pending safe) do
%{id: 0, tag: tag}
|> Block.ByTag.request()
|> json_rpc(json_rpc_named_arguments)
|> Block.ByTag.number_from_result()
end
@spec fetch_block_number_by_tag(tag(), json_rpc_named_arguments) ::
{:ok, non_neg_integer()} | {:error, reason :: :invalid_tag | :not_found | term()}
Deposits and Withdrawals for Polygon Edge (#8180) * Prepare tables for Polygon Supernet Withdrawals module * Prepare Indexer.Fetcher.PolygonSupernetWithdrawal * Prepare Indexer.Fetcher.PolygonSupernetWithdrawalExit * Prepare tables for Polygon Supernet Deposits module * Partially add Polygon Supernet Deposits module * Add Polygon Supernet Deposits module * Extend transaction data in API v2 for Polygon Supernet Deposits and Withdrawals * Extend API v2 for list of Polygon Supernet Deposits * Extend API v2 for list of Polygon Supernet Withdrawals * Modify collated_gas_price constraint to support Immutable test chain * Update number of retries * Fix Indexer.Fetcher.PolygonSupernetWithdrawal * Fix API v2 for Indexer.Fetcher.PolygonSupernetWithdrawal * Return page_size to 50 * Fix API v2 for Indexer.Fetcher.PolygonSupernetDeposit * Fix Explorer.Validator.MetadataRetriever to prevent sending request to nil address * Refactor PolygonSupernet* modules * Refactor PolygonSupernet* modules * Small refactoring of PolygonSupernet* modules * Update changelog * Fixes for dialyzer and cspell * Fix of Explorer tests * Add env vars for docker * Add env variable for eth_getLogs * Share event signatures between modules * Refactoring for fill_block_range function * Small refactoring of transaction actions module * Move polygon_supernet_* modules to a separate folder * Add specs for public functions * Remove redundant function * Reuse decode_data function * Move polygon_supernet_* files into a separate folder * Move polygon_supernet_* files into a separate folder * Partially disband Explorer.Chain module * Define chain type * Remove infinite waiting for safe block * max format * Rename Polygon Supernet to Polygon Edge * Set INDEXER_INTERNAL_TRANSACTIONS_TRACER_TYPE to polygon_edge if CHAIN_TYPE is polygon_edge * Fix chain type definition * Fix repos list definition in releases_tasks * INDEXER_POLYGON_SUPERNET_ -> INDEXER_POLYGON_EDGE_ --------- Co-authored-by: POA <33550681+poa@users.noreply.github.com> Co-authored-by: Viktor Baranov <baranov.viktor.27@gmail.com>
1 year ago
def fetch_block_number_by_tag(tag, json_rpc_named_arguments) when tag in ~w(earliest latest pending safe) do
tag
|> fetch_block_by_tag(json_rpc_named_arguments)
|> Block.ByTag.number_from_result()
end
@doc """
Fetches internal transactions from variant API.
"""
def fetch_internal_transactions(params_list, json_rpc_named_arguments) when is_list(params_list) do
Keyword.fetch!(json_rpc_named_arguments, :variant).fetch_internal_transactions(
params_list,
json_rpc_named_arguments
)
end
@doc """
Fetches internal transactions for entire blocks from variant API.
"""
def fetch_block_internal_transactions(block_numbers, json_rpc_named_arguments) when is_list(block_numbers) do
filtered_block_numbers = RangesHelper.filter_traceable_block_numbers(block_numbers)
Keyword.fetch!(json_rpc_named_arguments, :variant).fetch_block_internal_transactions(
filtered_block_numbers,
json_rpc_named_arguments
)
end
@doc """
Retrieves traces from variant API.
"""
def fetch_first_trace(params_list, json_rpc_named_arguments) when is_list(params_list) do
Keyword.fetch!(json_rpc_named_arguments, :variant).fetch_first_trace(
params_list,
json_rpc_named_arguments
)
end
@doc """
Fetches pending transactions from variant API.
"""
def fetch_pending_transactions(json_rpc_named_arguments) do
Keyword.fetch!(json_rpc_named_arguments, :variant).fetch_pending_transactions(json_rpc_named_arguments)
end
@doc """
Retrieves raw traces from Ethereum JSON RPC variant API.
"""
def fetch_transaction_raw_traces(transaction_params, json_rpc_named_arguments) do
Keyword.fetch!(json_rpc_named_arguments, :variant).fetch_transaction_raw_traces(
transaction_params,
json_rpc_named_arguments
)
end
@spec fetch_transaction_receipts(
[
%{required(:gas) => non_neg_integer(), required(:hash) => hash, optional(atom) => any}
],
json_rpc_named_arguments
) :: {:ok, %{logs: list(), receipts: list()}} | {:error, reason :: term}
def fetch_transaction_receipts(transactions_params, json_rpc_named_arguments) when is_list(transactions_params) do
Receipts.fetch(transactions_params, json_rpc_named_arguments)
end
fetched_balance_block_number instead of fetched_balance_at Fixes #243 Instead of getting the address balance at the latest block and recording the timestamp of the insert, which is only loosely correlated with even the latest block's timestamp on-chain, use the block number for the last time an address was used in the indexed address foreign keys in the chain: * blocks * miner_hash * internal_transactions * created_contract_address_hash * from_address_hash * to_address_hash * logs * address_hash * transactions * from_address_hash * to_address_hash To gather `fetched_balance_block_number` on reboot, `Explorer.Chain.stream_unfetched_addresses` queries across the above columns for the `MAX(blocks.number)`, which is used in `Explorer.Indexer.AddressBalanceFetcher.init/2`. During indexing, `Explorer.Indexer.AddressBalanceFetcher.async_fetch_balances/1` now requires a list of `%{block_number: Block.block_number(), hash: Hash.Truncated.t()}` instead of list of `Hash.Truncated.t()`. Having a `block_number` available for all importable entities's addresses required changing `Explorer.Indexer.AddressExtraction` to extract the `block_number` too. The use of `insert` in the factories was making tests that show the count not make sense, so the factories were switched to `build` because it means only the exact count is created when a default value is overridden. This does, unfortunately, mean that `params_for(factory)` no longer generates foreign keys and they must be set explicitly OR the association passed instead of the foreign key. Allow nil extracted fetched_balance_block_number when pending transaction *NOTE*: This is a database drop and reindex change.
7 years ago
@doc """
feat: EIP-7702 support (#10870) * feat: return `authorizationList` for EIP-7702 transactions in `/transactions/:tx_hash` response (#10776) * feat: support EIP-7702 transactions * fix: handle invalid signatures * fix: save authority * Update apps/block_scout_web/lib/block_scout_web/views/api/v2/transaction_view.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * Update apps/explorer/lib/explorer/chain/signed_authorization.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * Update apps/indexer/lib/indexer/block/fetcher.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * fix: remove set_code_transaction from @allowed_type_labels * Update apps/explorer/lib/explorer/chain/import/runner/signed_authorizations.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * fix: move signed_authorization to a separate module * add todo --------- Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * feat: support EIP-7702 in `/address/:address` endpoint (#10799) * feat: support EIP-7702 transactions * fix: handle invalid signatures * fix: save authority * Update apps/block_scout_web/lib/block_scout_web/views/api/v2/transaction_view.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * Update apps/explorer/lib/explorer/chain/signed_authorization.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * Update apps/indexer/lib/indexer/block/fetcher.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * fix: remove set_code_transaction from @allowed_type_labels * feat: add EIP-7702 support to /address/:address endpoint * fix: refactor fetch? * fix: move get_implementation_address_hash_string_eip7702 * fix: remove EIP-7702 flag from response, modify transactions filter to handle EOA with code correctly * fix: minor refactoring * fix: remove unused alias * fix: review comments * Update apps/explorer/priv/repo/migrations/20240904161254_create_signed_authorizations.exs Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> --------- Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * fix: remove unused code * fix: refactor code * Update apps/ethereum_jsonrpc/lib/ethereum_jsonrpc/signed_authorization.ex Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com> * chore: documentation improvement * fix typo * fix: move spec and doc * fix: authorization_list spec * fix: wrap address and authority with checksum * fix: invalid typespec Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com> * Apply suggestions from code review Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com> * fix typo * Update apps/explorer/lib/explorer/chain/smart_contract/proxy/eip_7702.ex Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com> --------- Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com>
1 month ago
Assigns a unique integer ID to each set of parameters in the given list.
This function is used to prepare parameters for batch request-response
correlation in JSON-RPC calls.
## Parameters
- `params_list`: A list of parameter sets, where each set can be of any type.
## Returns
A map where the keys are integer IDs (starting from 0) and the values are
the corresponding parameter sets from the input list.
## Example
iex> id_to_params([%{block: 1}, %{block: 2}])
%{0 => %{block: 1}, 1 => %{block: 2}}
fetched_balance_block_number instead of fetched_balance_at Fixes #243 Instead of getting the address balance at the latest block and recording the timestamp of the insert, which is only loosely correlated with even the latest block's timestamp on-chain, use the block number for the last time an address was used in the indexed address foreign keys in the chain: * blocks * miner_hash * internal_transactions * created_contract_address_hash * from_address_hash * to_address_hash * logs * address_hash * transactions * from_address_hash * to_address_hash To gather `fetched_balance_block_number` on reboot, `Explorer.Chain.stream_unfetched_addresses` queries across the above columns for the `MAX(blocks.number)`, which is used in `Explorer.Indexer.AddressBalanceFetcher.init/2`. During indexing, `Explorer.Indexer.AddressBalanceFetcher.async_fetch_balances/1` now requires a list of `%{block_number: Block.block_number(), hash: Hash.Truncated.t()}` instead of list of `Hash.Truncated.t()`. Having a `block_number` available for all importable entities's addresses required changing `Explorer.Indexer.AddressExtraction` to extract the `block_number` too. The use of `insert` in the factories was making tests that show the count not make sense, so the factories were switched to `build` because it means only the exact count is created when a default value is overridden. This does, unfortunately, mean that `params_for(factory)` no longer generates foreign keys and they must be set explicitly OR the association passed instead of the foreign key. Allow nil extracted fetched_balance_block_number when pending transaction *NOTE*: This is a database drop and reindex change.
7 years ago
"""
@spec id_to_params([params]) :: %{id => params} when id: non_neg_integer(), params: any()
fetched_balance_block_number instead of fetched_balance_at Fixes #243 Instead of getting the address balance at the latest block and recording the timestamp of the insert, which is only loosely correlated with even the latest block's timestamp on-chain, use the block number for the last time an address was used in the indexed address foreign keys in the chain: * blocks * miner_hash * internal_transactions * created_contract_address_hash * from_address_hash * to_address_hash * logs * address_hash * transactions * from_address_hash * to_address_hash To gather `fetched_balance_block_number` on reboot, `Explorer.Chain.stream_unfetched_addresses` queries across the above columns for the `MAX(blocks.number)`, which is used in `Explorer.Indexer.AddressBalanceFetcher.init/2`. During indexing, `Explorer.Indexer.AddressBalanceFetcher.async_fetch_balances/1` now requires a list of `%{block_number: Block.block_number(), hash: Hash.Truncated.t()}` instead of list of `Hash.Truncated.t()`. Having a `block_number` available for all importable entities's addresses required changing `Explorer.Indexer.AddressExtraction` to extract the `block_number` too. The use of `insert` in the factories was making tests that show the count not make sense, so the factories were switched to `build` because it means only the exact count is created when a default value is overridden. This does, unfortunately, mean that `params_for(factory)` no longer generates foreign keys and they must be set explicitly OR the association passed instead of the foreign key. Allow nil extracted fetched_balance_block_number when pending transaction *NOTE*: This is a database drop and reindex change.
7 years ago
def id_to_params(params_list) do
params_list
|> Stream.with_index()
|> Enum.into(%{}, fn {params, id} -> {id, params} end)
end
@doc """
feat: EIP-7702 support (#10870) * feat: return `authorizationList` for EIP-7702 transactions in `/transactions/:tx_hash` response (#10776) * feat: support EIP-7702 transactions * fix: handle invalid signatures * fix: save authority * Update apps/block_scout_web/lib/block_scout_web/views/api/v2/transaction_view.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * Update apps/explorer/lib/explorer/chain/signed_authorization.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * Update apps/indexer/lib/indexer/block/fetcher.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * fix: remove set_code_transaction from @allowed_type_labels * Update apps/explorer/lib/explorer/chain/import/runner/signed_authorizations.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * fix: move signed_authorization to a separate module * add todo --------- Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * feat: support EIP-7702 in `/address/:address` endpoint (#10799) * feat: support EIP-7702 transactions * fix: handle invalid signatures * fix: save authority * Update apps/block_scout_web/lib/block_scout_web/views/api/v2/transaction_view.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * Update apps/explorer/lib/explorer/chain/signed_authorization.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * Update apps/indexer/lib/indexer/block/fetcher.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * fix: remove set_code_transaction from @allowed_type_labels * feat: add EIP-7702 support to /address/:address endpoint * fix: refactor fetch? * fix: move get_implementation_address_hash_string_eip7702 * fix: remove EIP-7702 flag from response, modify transactions filter to handle EOA with code correctly * fix: minor refactoring * fix: remove unused alias * fix: review comments * Update apps/explorer/priv/repo/migrations/20240904161254_create_signed_authorizations.exs Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> --------- Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * fix: remove unused code * fix: refactor code * Update apps/ethereum_jsonrpc/lib/ethereum_jsonrpc/signed_authorization.ex Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com> * chore: documentation improvement * fix typo * fix: move spec and doc * fix: authorization_list spec * fix: wrap address and authority with checksum * fix: invalid typespec Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com> * Apply suggestions from code review Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com> * fix typo * Update apps/explorer/lib/explorer/chain/smart_contract/proxy/eip_7702.ex Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com> --------- Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com>
1 month ago
Sanitizes responses by assigning unmatched IDs to responses with missing IDs.
This function processes a list of responses and a map of expected IDs to
parameters. It handles cases where responses have missing (nil) IDs by
assigning them unmatched IDs from the id_to_params map.
## Parameters
- `responses`: A list of response maps from a batch JSON-RPC call.
- `id_to_params`: A map of request IDs to their corresponding parameters.
## Returns
A list of sanitized response maps where each response has a valid ID.
## Example
iex> responses = [%{id: 1, result: "ok"}, %{id: nil, result: "error"}]
iex> id_to_params = %{1 => %{}, 2 => %{}, 3 => %{}}
iex> EthereumJSONRPC.sanitize_responses(responses, id_to_params)
[%{id: 1, result: "ok"}, %{id: 2, result: "error"}]
"""
feat: EIP-7702 support (#10870) * feat: return `authorizationList` for EIP-7702 transactions in `/transactions/:tx_hash` response (#10776) * feat: support EIP-7702 transactions * fix: handle invalid signatures * fix: save authority * Update apps/block_scout_web/lib/block_scout_web/views/api/v2/transaction_view.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * Update apps/explorer/lib/explorer/chain/signed_authorization.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * Update apps/indexer/lib/indexer/block/fetcher.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * fix: remove set_code_transaction from @allowed_type_labels * Update apps/explorer/lib/explorer/chain/import/runner/signed_authorizations.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * fix: move signed_authorization to a separate module * add todo --------- Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * feat: support EIP-7702 in `/address/:address` endpoint (#10799) * feat: support EIP-7702 transactions * fix: handle invalid signatures * fix: save authority * Update apps/block_scout_web/lib/block_scout_web/views/api/v2/transaction_view.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * Update apps/explorer/lib/explorer/chain/signed_authorization.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * Update apps/indexer/lib/indexer/block/fetcher.ex Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * fix: remove set_code_transaction from @allowed_type_labels * feat: add EIP-7702 support to /address/:address endpoint * fix: refactor fetch? * fix: move get_implementation_address_hash_string_eip7702 * fix: remove EIP-7702 flag from response, modify transactions filter to handle EOA with code correctly * fix: minor refactoring * fix: remove unused alias * fix: review comments * Update apps/explorer/priv/repo/migrations/20240904161254_create_signed_authorizations.exs Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> --------- Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> * fix: remove unused code * fix: refactor code * Update apps/ethereum_jsonrpc/lib/ethereum_jsonrpc/signed_authorization.ex Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com> * chore: documentation improvement * fix typo * fix: move spec and doc * fix: authorization_list spec * fix: wrap address and authority with checksum * fix: invalid typespec Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com> * Apply suggestions from code review Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com> * fix typo * Update apps/explorer/lib/explorer/chain/smart_contract/proxy/eip_7702.ex Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com> --------- Co-authored-by: Kirill Fedoseev <kirill@blockscout.com> Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com>
1 month ago
@spec sanitize_responses(Transport.batch_response(), %{id => params}) :: Transport.batch_response()
when id: EthereumJSONRPC.request_id(), params: any()
def sanitize_responses(responses, id_to_params) do
responses
|> Enum.reduce(
{[], Map.keys(id_to_params) -- Enum.map(responses, & &1.id)},
fn
%{id: nil} = res, {result_res, [id | rest]} ->
Logger.error(
"Empty id in response: #{inspect(res)}, stacktrace: #{inspect(Process.info(self(), :current_stacktrace))}"
)
{[%{res | id: id} | result_res], rest}
res, {result_res, non_matched} ->
{[res | result_res], non_matched}
end
)
|> elem(0)
|> Enum.reverse()
end
@doc """
1. POSTs JSON `payload` to `url`
2. Decodes the response
3. Handles the response
## Returns
* Handled response
6 years ago
* `{:error, reason}` if POST fails
"""
@spec json_rpc(Transport.request(), json_rpc_named_arguments) ::
{:ok, Transport.result()} | {:error, reason :: term()}
@spec json_rpc(Transport.batch_request(), json_rpc_named_arguments) ::
{:ok, Transport.batch_response()} | {:error, reason :: term()}
def json_rpc(request, named_arguments) when (is_map(request) or is_list(request)) and is_list(named_arguments) do
transport = Keyword.fetch!(named_arguments, :transport)
transport_options = Keyword.fetch!(named_arguments, :transport_options)
throttle_timeout = Keyword.get(named_arguments, :throttle_timeout, @default_throttle_timeout)
url = maybe_replace_url(transport_options[:url], transport_options[:fallback_url], transport)
corrected_transport_options = Keyword.replace(transport_options, :url, url)
case RequestCoordinator.perform(request, transport, corrected_transport_options, throttle_timeout) do
{:ok, result} ->
{:ok, result}
{:error, reason} ->
maybe_inc_error_count(corrected_transport_options[:url], named_arguments, transport)
{:error, reason}
end
end
defp do_balances_request(id_to_params, _chunk_size, _args) when id_to_params == %{}, do: {:ok, []}
defp do_balances_request(id_to_params, chunk_size, json_rpc_named_arguments) do
id_to_params
|> FetchedBalances.requests()
|> chunk_requests(chunk_size)
|> json_rpc(json_rpc_named_arguments)
end
defp archive_json_rpc_named_arguments(json_rpc_named_arguments) do
CommonHelper.put_in_keyword_nested(
json_rpc_named_arguments,
[:transport_options, :method_to_url, :eth_getBalance],
System.get_env("ETHEREUM_JSONRPC_TRACE_URL")
)
end
defp maybe_request_archive_balances(id_to_params, trace_url_used?, disabled?, chunk_size, json_rpc_named_arguments) do
if not trace_url_used? and not disabled? do
do_balances_request(id_to_params, chunk_size, archive_json_rpc_named_arguments(json_rpc_named_arguments))
else
{:ok, []}
end
end
defp maybe_replace_url(url, _replace_url, EthereumJSONRPC.HTTP), do: url
defp maybe_replace_url(url, replace_url, _), do: EndpointAvailabilityObserver.maybe_replace_url(url, replace_url, :ws)
defp maybe_inc_error_count(_url, _arguments, EthereumJSONRPC.HTTP), do: :ok
defp maybe_inc_error_count(url, arguments, _), do: EndpointAvailabilityObserver.inc_error_count(url, arguments, :ws)
@doc """
Converts `t:quantity/0` to `t:non_neg_integer/0`.
"""
@spec quantity_to_integer(quantity) :: non_neg_integer() | nil
fetched_balance_block_number instead of fetched_balance_at Fixes #243 Instead of getting the address balance at the latest block and recording the timestamp of the insert, which is only loosely correlated with even the latest block's timestamp on-chain, use the block number for the last time an address was used in the indexed address foreign keys in the chain: * blocks * miner_hash * internal_transactions * created_contract_address_hash * from_address_hash * to_address_hash * logs * address_hash * transactions * from_address_hash * to_address_hash To gather `fetched_balance_block_number` on reboot, `Explorer.Chain.stream_unfetched_addresses` queries across the above columns for the `MAX(blocks.number)`, which is used in `Explorer.Indexer.AddressBalanceFetcher.init/2`. During indexing, `Explorer.Indexer.AddressBalanceFetcher.async_fetch_balances/1` now requires a list of `%{block_number: Block.block_number(), hash: Hash.Truncated.t()}` instead of list of `Hash.Truncated.t()`. Having a `block_number` available for all importable entities's addresses required changing `Explorer.Indexer.AddressExtraction` to extract the `block_number` too. The use of `insert` in the factories was making tests that show the count not make sense, so the factories were switched to `build` because it means only the exact count is created when a default value is overridden. This does, unfortunately, mean that `params_for(factory)` no longer generates foreign keys and they must be set explicitly OR the association passed instead of the foreign key. Allow nil extracted fetched_balance_block_number when pending transaction *NOTE*: This is a database drop and reindex change.
7 years ago
def quantity_to_integer("0x" <> hexadecimal_digits) do
String.to_integer(hexadecimal_digits, 16)
end
def quantity_to_integer(integer) when is_integer(integer), do: integer
def quantity_to_integer(string) when is_binary(string) do
case Integer.parse(string) do
{integer, ""} -> integer
_ -> nil
end
end
def quantity_to_integer(_), do: nil
@doc """
Sanitizes ID in JSON RPC request following JSON RPC [spec](https://www.jsonrpc.org/specification#request_object:~:text=An%20identifier%20established%20by%20the%20Client%20that%20MUST%20contain%20a%20String%2C%20Number%2C%20or%20NULL%20value%20if%20included.%20If%20it%20is%20not%20included%20it%20is%20assumed%20to%20be%20a%20notification.%20The%20value%20SHOULD%20normally%20not%20be%20Null%20%5B1%5D%20and%20Numbers%20SHOULD%20NOT%20contain%20fractional%20parts%20%5B2%5D).
"""
@spec sanitize_id(quantity) :: non_neg_integer() | String.t() | nil
def sanitize_id(integer) when is_integer(integer), do: integer
def sanitize_id(string) when is_binary(string) do
# match ID string and ID string without non-ASCII characters
if string == for(<<c <- string>>, c < 128, into: "", do: <<c>>) do
string
else
nil
end
end
def sanitize_id(_), do: nil
fetched_balance_block_number instead of fetched_balance_at Fixes #243 Instead of getting the address balance at the latest block and recording the timestamp of the insert, which is only loosely correlated with even the latest block's timestamp on-chain, use the block number for the last time an address was used in the indexed address foreign keys in the chain: * blocks * miner_hash * internal_transactions * created_contract_address_hash * from_address_hash * to_address_hash * logs * address_hash * transactions * from_address_hash * to_address_hash To gather `fetched_balance_block_number` on reboot, `Explorer.Chain.stream_unfetched_addresses` queries across the above columns for the `MAX(blocks.number)`, which is used in `Explorer.Indexer.AddressBalanceFetcher.init/2`. During indexing, `Explorer.Indexer.AddressBalanceFetcher.async_fetch_balances/1` now requires a list of `%{block_number: Block.block_number(), hash: Hash.Truncated.t()}` instead of list of `Hash.Truncated.t()`. Having a `block_number` available for all importable entities's addresses required changing `Explorer.Indexer.AddressExtraction` to extract the `block_number` too. The use of `insert` in the factories was making tests that show the count not make sense, so the factories were switched to `build` because it means only the exact count is created when a default value is overridden. This does, unfortunately, mean that `params_for(factory)` no longer generates foreign keys and they must be set explicitly OR the association passed instead of the foreign key. Allow nil extracted fetched_balance_block_number when pending transaction *NOTE*: This is a database drop and reindex change.
7 years ago
@doc """
Converts `t:non_neg_integer/0` to `t:quantity/0`
"""
@spec integer_to_quantity(non_neg_integer | binary) :: quantity
fetched_balance_block_number instead of fetched_balance_at Fixes #243 Instead of getting the address balance at the latest block and recording the timestamp of the insert, which is only loosely correlated with even the latest block's timestamp on-chain, use the block number for the last time an address was used in the indexed address foreign keys in the chain: * blocks * miner_hash * internal_transactions * created_contract_address_hash * from_address_hash * to_address_hash * logs * address_hash * transactions * from_address_hash * to_address_hash To gather `fetched_balance_block_number` on reboot, `Explorer.Chain.stream_unfetched_addresses` queries across the above columns for the `MAX(blocks.number)`, which is used in `Explorer.Indexer.AddressBalanceFetcher.init/2`. During indexing, `Explorer.Indexer.AddressBalanceFetcher.async_fetch_balances/1` now requires a list of `%{block_number: Block.block_number(), hash: Hash.Truncated.t()}` instead of list of `Hash.Truncated.t()`. Having a `block_number` available for all importable entities's addresses required changing `Explorer.Indexer.AddressExtraction` to extract the `block_number` too. The use of `insert` in the factories was making tests that show the count not make sense, so the factories were switched to `build` because it means only the exact count is created when a default value is overridden. This does, unfortunately, mean that `params_for(factory)` no longer generates foreign keys and they must be set explicitly OR the association passed instead of the foreign key. Allow nil extracted fetched_balance_block_number when pending transaction *NOTE*: This is a database drop and reindex change.
7 years ago
def integer_to_quantity(integer) when is_integer(integer) and integer >= 0 do
"0x" <> Integer.to_string(integer, 16)
end
def integer_to_quantity(integer) when is_binary(integer) do
integer
end
@doc """
A request payload for a JSONRPC.
"""
@spec request(%{id: request_id, method: String.t(), params: list()}) :: Transport.request()
def request(%{method: method, params: params} = map)
when is_binary(method) and is_list(params) do
Map.put(map, :jsonrpc, "2.0")
end
@doc """
Subscribes to `t:EthereumJSONRPC.Subscription.event/0` with `t:EthereumJSONRPC.Subscription.params/0`.
Events are delivered in a tuple tagged with the `t:EthereumJSONRPC.Subscription.t/0` and containing the same output
as the single-request form of `json_rpc/2`.
| Message | Description |
|-----------------------------------------------------------------------------------|----------------------------------------|
| `{EthereumJSONRPC.Subscription.t(), {:ok, EthereumJSONRPC.Transport.result.t()}}` | New result in subscription |
| `{EthereumJSONRPC.Subscription.t(), {:error, reason :: term()}}` | There was an error in the subscription |
Subscription can be canceled by calling `unsubscribe/1` with the returned `t:EthereumJSONRPC.Subscription.t/0`.
"""
@spec subscribe(event :: Subscription.event(), params :: Subscription.params(), subscribe_named_arguments) ::
{:ok, Subscription.t()} | {:error, reason :: term()}
def subscribe(event, params \\ [], named_arguments) when is_list(params) do
transport = Keyword.fetch!(named_arguments, :transport)
transport_options = Keyword.fetch!(named_arguments, :transport_options)
transport.subscribe(event, params, transport_options)
end
@doc """
Unsubscribes to `t:EthereumJSONRPC.Subscription.t/0` created with `subscribe/2`.
## Returns
* `:ok` - subscription was canceled
* `{:error, :not_found}` - subscription could not be canceled. It did not exist because either the server already
canceled it, it never existed, or `unsubscribe/1 ` was called on it before.
* `{:error, reason :: term}` - other error cancelling subscription.
"""
@spec unsubscribe(Subscription.t()) :: :ok | {:error, reason :: term()}
def unsubscribe(%Subscription{transport: transport} = subscription) do
transport.unsubscribe(subscription)
end
# We can only depend on implementations supporting 64-bit integers:
# * Ganache only supports u32 (https://github.com/trufflesuite/ganache-core/issues/190)
def unique_request_id do
<<unique_request_id::big-integer-size(4)-unit(8)>> = :crypto.strong_rand_bytes(4)
unique_request_id
end
@doc """
Converts `t:timestamp/0` to `t:DateTime.t/0`
"""
def timestamp_to_datetime(timestamp) do
case quantity_to_integer(timestamp) do
nil ->
nil
quantity ->
Timex.from_unix(quantity)
end
end
defp fetch_blocks_by_params(params, request, json_rpc_named_arguments)
when is_list(params) and is_function(request, 1) do
id_to_params = id_to_params(params)
with {:ok, responses} <-
id_to_params
|> Blocks.requests(request)
|> json_rpc(json_rpc_named_arguments) do
{:ok, Blocks.from_responses(responses, id_to_params)}
end
end
defp chunk_requests(requests, nil), do: requests
defp chunk_requests(requests, chunk_size), do: Enum.chunk_every(requests, chunk_size)
feat: support for `:celo` chain type (#10564) * feat: indexing of Celo transactions (#10275) * feat(json-rpc): support celo-specific fields in transactions * chore: add new chain type to matrix builder * feat: add celo-specific fields to transactions schema * chore: add "celo" to `cspell.json` * chore: fix formatting * refactor: improve naming for celo fields and ignore `ethCompatible` field * chore: add block fetcher test for celo fields * feat: add celo core contract abis * feat: add a cache for core celo contracts * feat: treat celo native token transfers as token transfers * fix: update `cspell.json` with celo words * fix(dialyzer): pattern can never match the type * fix: update `cspell.json` with celo words * feat: move `CELO_NETWORK` env to config * feat: parse native Celo token transfers from internal transactions * feat: add token balances fetching * refactor: remove `dbg` and organize code better * fix: start TokenBalance fetcher * fix: start token balance fetcher * fix: typo in log message * fix: missing fields in celo transaction token transfer * refactor: remove `AbiHandler` in favour of hardcoded pieces of abi * refactor: native token transfers from internal transactions fether * feat: implement parsing of token transfers for traceable rpc node * feat: fetch address coin balances after erc20 celo token transfers * feat: add `gas_token` to transaction response in api v2 * feat: index tokens specified as `feeCurrency` * chore: fix credo warnings * chore: decrease log level to debug * chore: fix cspell warnings * refactor: apply suggestions * fix: handle case when `gas_token` is not loaded * dbg * chore: expect contract calls in celo fetcher test * fix: compile errors for celo view on default chain type * chore: remove unused import * refactor: move `celo_gas_tokens` to chain type imports * refactor: apply suggestions * fix: swap arguments in `put_if_present` * fix: indexer tests * refactor: always include `gas_token` to the response * feat: add `gas_token` to all transaction-related endpoints * feat: celo core contracts with historical data * fix: `uncles` field is expected but not present in RPC node response * fix: credo & format * fix: define `CELO_CORE_CONTRACTS` in tests * fix: put `gas_token` under `celo` section * refactor: move token transfers filtering from `TokenTotalSupplyUpdater` to `Transform.TokenTransfers` * feat: add workflow to publish docker image * refactor: apply suggestions by @nikitosing * chore: add docs and specs * fix: malformed import * fix: publish image for celo * fix: ci build for celo * feat: index celo epoch transactions (#9944) * feat(json-rpc): support celo-specific fields in transactions * chore: add new chain type to matrix builder * feat: add celo-specific fields to transactions schema * chore: add "celo" to `cspell.json` * chore: fix formatting * refactor: improve naming for celo fields and ignore `ethCompatible` field * chore: add block fetcher test for celo fields * feat: add celo core contract abis * feat: add a cache for core celo contracts * feat: treat celo native token transfers as token transfers * fix: update `cspell.json` with celo words * fix(dialyzer): pattern can never match the type * fix: update `cspell.json` with celo words * feat: move `CELO_NETWORK` env to config * feat: parse native Celo token transfers from internal transactions * feat: add token balances fetching * refactor: remove `dbg` and organize code better * fix: start TokenBalance fetcher * fix: start token balance fetcher * fix: typo in log message * fix: missing fields in celo transaction token transfer * refactor: remove `AbiHandler` in favour of hardcoded pieces of abi * refactor: native token transfers from internal transactions fether * feat: implement parsing of token transfers for traceable rpc node * feat: fetch address coin balances after erc20 celo token transfers * feat: add `gas_token` to transaction response in api v2 * feat: index tokens specified as `feeCurrency` * chore: fix credo warnings * chore: decrease log level to debug * chore: fix cspell warnings * refactor: apply suggestions * fix: handle case when `gas_token` is not loaded * dbg * chore: expect contract calls in celo fetcher test * fix: compile errors for celo view on default chain type * chore: remove unused import * refactor: move `celo_gas_tokens` to chain type imports * refactor: apply suggestions * fix: swap arguments in `put_if_present` * fix: indexer tests * refactor: always include `gas_token` to the response * feat: add `gas_token` to all transaction-related endpoints * feat: celo core contracts with historical data * fix: `uncles` field is expected but not present in RPC node response * fix: credo & format * fix: define `CELO_CORE_CONTRACTS` in tests * fix: put `gas_token` under `celo` section * refactor: move token transfers filtering from `TokenTotalSupplyUpdater` to `Transform.TokenTransfers` * feat: add workflow to publish docker image * refactor: apply suggestions by @nikitosing * chore: add docs and specs * fix: malformed import * feat: add pending epoch operations table * refactor: fix format * feat: add transformers for epoch events * feat: add query to stream pending epoch block operations * fix: call to renamed function * fix: add factory method for pending epoch block operation * feat: add utils for logs parsing * feat: add schemas for epoch rewards and election rewards * fix: rename transformer according to event name * feat: implement epoch rewards fetcher * feat: fetch and import epoch logs * feat: improve epoch rewards fetcher 1. Do not fetch logs -- use the ones stored in DB 2. Import epochs to the database 3. Configure fetcher in runtime.exs * feat: add epoch helper functions * activated_validator_group_vote.ex * feat: add task to fetch core contracts * feat: fetch core contract events in the task * fix: merge artifacts * refactor: logs requests and reduce scope of epoch logs request * chore: fix formatting, credo warning, etc. * feat: fetch epoch rewards in one sql query * feat: fetch validator group votes (historical data and on demand) * refactor: rename fields and replace all entries on conflict * feat: validator group votes fetcher * fix: put each topic in a separate request * feat: fetch voter rewards * refactor: split epoch fetcher to multiple modules * feat: send epoch blocks for async fetching from block fetcher * chore: fix credo and formatting issues * fix: dialyzer warnings * fix: add on demand fetch of group votes * fix: failed explorer and indexer tests * fix: block fetcher tests * fix: match error in epoch logs * feat: add env to manage logs batch size when fetching validator group votes * fix: Add `ssl_opts` for Celo repo * fix: add `disabled?` predicate to supervisor config * fix: return empty list when `getPaymentDelegation` is not available * fix: validate the case when there is no voter rewards for an epoch * fix: formatting * fix: import `put_if_present/3` * fix: do not treat genesis block as an epoch block * fix: some of the fetcher tests * dbg * chore: remove commented code * chore: canonical disable flag env name * fix: run test only for celo chain type * fix: add explicit `wait_ for_tasks` in token instance realtime test * fix: set missing `CELO_CORE_CONTRACTS` env * fix: rollback token instance realtime test * chore: remove debug artifact * chore: add docs for new tables * perf: remove unused fields from `celo_validator_group_votes` table * fix: add on_exit clause * refactor: use `remove/3` function for migration rollback possibility * refactor: extract celo core contracts environment variable setup into a separate function * chore: add new vars to `common-blockscout.env` file * chore: add specs and docs for new modules and functions * refactor: eliminate unused import warnings * fix: credo warning * feat: API for celo epoch rewards and fees breakdown (#10308) * feat(json-rpc): support celo-specific fields in transactions * chore: add new chain type to matrix builder * feat: add celo-specific fields to transactions schema * chore: add "celo" to `cspell.json` * chore: fix formatting * refactor: improve naming for celo fields and ignore `ethCompatible` field * chore: add block fetcher test for celo fields * feat: add celo core contract abis * feat: add a cache for core celo contracts * feat: treat celo native token transfers as token transfers * fix: update `cspell.json` with celo words * fix(dialyzer): pattern can never match the type * fix: update `cspell.json` with celo words * feat: move `CELO_NETWORK` env to config * feat: parse native Celo token transfers from internal transactions * feat: add token balances fetching * refactor: remove `dbg` and organize code better * fix: start TokenBalance fetcher * fix: start token balance fetcher * fix: typo in log message * fix: missing fields in celo transaction token transfer * refactor: remove `AbiHandler` in favour of hardcoded pieces of abi * refactor: native token transfers from internal transactions fether * feat: implement parsing of token transfers for traceable rpc node * feat: fetch address coin balances after erc20 celo token transfers * feat: add `gas_token` to transaction response in api v2 * feat: index tokens specified as `feeCurrency` * chore: fix credo warnings * chore: decrease log level to debug * chore: fix cspell warnings * refactor: apply suggestions * fix: handle case when `gas_token` is not loaded * dbg * chore: expect contract calls in celo fetcher test * fix: compile errors for celo view on default chain type * chore: remove unused import * refactor: move `celo_gas_tokens` to chain type imports * refactor: apply suggestions * fix: swap arguments in `put_if_present` * fix: indexer tests * feat: add pending epoch operations table * feat: add several celo core contracts * refactor: fix format * feat: add new core contract default addresses * feat: add transformers for epoch events * feat: add query to stream pending epoch block operations * fix: call to renamed function * fix: add factory method for pending epoch block operation * feat: add utils for logs parsing * feat: add schemas for epoch rewards and election rewards * fix: rename transformer according to event name * feat: implement epoch rewards fetcher * feat: fetch and import epoch logs * feat: improve epoch rewards fetcher 1. Do not fetch logs -- use the ones stored in DB 2. Import epochs to the database 3. Configure fetcher in runtime.exs * fix: remove duplicated module attribute * feat: add epoch helper functions * activated_validator_group_vote.ex * feat: tmp * refactor: always include `gas_token` to the response * feat: add `gas_token` to all transaction-related endpoints * feat: celo core contracts with historical data * fix: `uncles` field is expected but not present in RPC node response * fix: credo & format * fix: define `CELO_CORE_CONTRACTS` in tests * fix: put `gas_token` under `celo` section * refactor: move token transfers filtering from `TokenTotalSupplyUpdater` to `Transform.TokenTransfers` * feat: add workflow to publish docker image * refactor: apply suggestions by @nikitosing * chore: add docs and specs * fix: malformed import * feat: add pending epoch operations table * refactor: fix format * feat: add transformers for epoch events * feat: add query to stream pending epoch block operations * fix: call to renamed function * fix: add factory method for pending epoch block operation * feat: add utils for logs parsing * feat: add schemas for epoch rewards and election rewards * fix: rename transformer according to event name * feat: implement epoch rewards fetcher * feat: fetch and import epoch logs * feat: improve epoch rewards fetcher 1. Do not fetch logs -- use the ones stored in DB 2. Import epochs to the database 3. Configure fetcher in runtime.exs * feat: add epoch helper functions * activated_validator_group_vote.ex * feat: add task to fetch core contracts * feat: fetch core contract events in the task * fix: merge artifacts * refactor: logs requests and reduce scope of epoch logs request * chore: fix formatting, credo warning, etc. * feat: fetch epoch rewards in one sql query * feat: fetch validator group votes (historical data and on demand) * refactor: rename fields and replace all entries on conflict * feat: validator group votes fetcher * fix: put each topic in a separate request * feat: fetch voter rewards * fix: merge artifacts * feat: add epoch rewards in block api response * fix: merge artifacts * feat: add base fee breakdown in block api response * feat: add aggregated election rewards to block api response * perf: use replica when querying epoch rewards * feat: endpoint for paginated election rewards * feat: endpoint for paginated election rewards for address * chore: rename `rewards` to `distributions` * fix: add missing reward type to API error message * refactor: split epoch fetcher to multiple modules * feat: send epoch blocks for async fetching from block fetcher * chore: fix credo and formatting issues * fix: dialyzer warnings * fix: add on demand fetch of group votes * fix: failed explorer and indexer tests * fix: block fetcher tests * chore: remove unused module * fix: sort election rewards by block number * fix: match error in epoch logs * feat: add env to manage logs batch size when fetching validator group votes * fix: credo and formatting warnings * fix: cspell errors * fix: dialyzer errors * fix: Add `ssl_opts` for Celo repo * fix: add missing preloads and make more explicit api response for block * fix: add `disabled?` predicate to supervisor config * fix: return empty list when `getPaymentDelegation` is not available * fix: validate the case when there is no voter rewards for an epoch * fix: formatting * fix: add missing preload * refactor: more robust fees breakdown logic for the case of fee handler * fix: formatting * refactor: move epoch information to the separate endpoint * fix: import `put_if_present/3` * fix: formatting * fix: do not treat genesis block as an epoch block * fix: some of the fetcher tests * dbg * chore: remove commented code * chore: canonical disable flag env name * fix: run test only for celo chain type * fix: add explicit `wait_ for_tasks` in token instance realtime test * fix: set missing `CELO_CORE_CONTRACTS` env * fix: rollback token instance realtime test * chore: remove debug artifact * chore: add `@docs` and `@specs` * fix: missing core contracts var in tests * feat: extend `tabs-counters` endpoint with election rewards count * refactor: move dead address to `Explorer.Chain.SmartContract` * perf: clause with simplified queries in the case of `amount == 0` and/or `block_number == 0` * fix: it comes that `NotLoaded` clause is not redundant actually... * fix: add missing preload in `BlockScoutWeb.AddressChannel` * chore: unset `CELO_CORE_CONTRACTS` in tests * chore: add missing specs * fix: `paginate` clause when `amount == 0` * refactor: move `paging_options/2` to `Explorer.Chain.Celo.ElectionReward` * refactor: avoid using virtual field for block number. * fix: remove redundant condition in the query * chore: clarify `@spec` * chore: fix credo warning * chore: remove `fi-celo` from branches that trigger ci * fix: merge artifacts * fix: remove todo comment
4 months ago
def put_if_present(result, transaction, keys) do
Enum.reduce(keys, result, fn key, acc ->
key_list = key |> Tuple.to_list()
from_key = Enum.at(key_list, 0)
to_key = Enum.at(key_list, 1)
opts = if Enum.count(key_list) > 2, do: Enum.at(key_list, 2), else: %{}
feat: support for `:celo` chain type (#10564) * feat: indexing of Celo transactions (#10275) * feat(json-rpc): support celo-specific fields in transactions * chore: add new chain type to matrix builder * feat: add celo-specific fields to transactions schema * chore: add "celo" to `cspell.json` * chore: fix formatting * refactor: improve naming for celo fields and ignore `ethCompatible` field * chore: add block fetcher test for celo fields * feat: add celo core contract abis * feat: add a cache for core celo contracts * feat: treat celo native token transfers as token transfers * fix: update `cspell.json` with celo words * fix(dialyzer): pattern can never match the type * fix: update `cspell.json` with celo words * feat: move `CELO_NETWORK` env to config * feat: parse native Celo token transfers from internal transactions * feat: add token balances fetching * refactor: remove `dbg` and organize code better * fix: start TokenBalance fetcher * fix: start token balance fetcher * fix: typo in log message * fix: missing fields in celo transaction token transfer * refactor: remove `AbiHandler` in favour of hardcoded pieces of abi * refactor: native token transfers from internal transactions fether * feat: implement parsing of token transfers for traceable rpc node * feat: fetch address coin balances after erc20 celo token transfers * feat: add `gas_token` to transaction response in api v2 * feat: index tokens specified as `feeCurrency` * chore: fix credo warnings * chore: decrease log level to debug * chore: fix cspell warnings * refactor: apply suggestions * fix: handle case when `gas_token` is not loaded * dbg * chore: expect contract calls in celo fetcher test * fix: compile errors for celo view on default chain type * chore: remove unused import * refactor: move `celo_gas_tokens` to chain type imports * refactor: apply suggestions * fix: swap arguments in `put_if_present` * fix: indexer tests * refactor: always include `gas_token` to the response * feat: add `gas_token` to all transaction-related endpoints * feat: celo core contracts with historical data * fix: `uncles` field is expected but not present in RPC node response * fix: credo & format * fix: define `CELO_CORE_CONTRACTS` in tests * fix: put `gas_token` under `celo` section * refactor: move token transfers filtering from `TokenTotalSupplyUpdater` to `Transform.TokenTransfers` * feat: add workflow to publish docker image * refactor: apply suggestions by @nikitosing * chore: add docs and specs * fix: malformed import * fix: publish image for celo * fix: ci build for celo * feat: index celo epoch transactions (#9944) * feat(json-rpc): support celo-specific fields in transactions * chore: add new chain type to matrix builder * feat: add celo-specific fields to transactions schema * chore: add "celo" to `cspell.json` * chore: fix formatting * refactor: improve naming for celo fields and ignore `ethCompatible` field * chore: add block fetcher test for celo fields * feat: add celo core contract abis * feat: add a cache for core celo contracts * feat: treat celo native token transfers as token transfers * fix: update `cspell.json` with celo words * fix(dialyzer): pattern can never match the type * fix: update `cspell.json` with celo words * feat: move `CELO_NETWORK` env to config * feat: parse native Celo token transfers from internal transactions * feat: add token balances fetching * refactor: remove `dbg` and organize code better * fix: start TokenBalance fetcher * fix: start token balance fetcher * fix: typo in log message * fix: missing fields in celo transaction token transfer * refactor: remove `AbiHandler` in favour of hardcoded pieces of abi * refactor: native token transfers from internal transactions fether * feat: implement parsing of token transfers for traceable rpc node * feat: fetch address coin balances after erc20 celo token transfers * feat: add `gas_token` to transaction response in api v2 * feat: index tokens specified as `feeCurrency` * chore: fix credo warnings * chore: decrease log level to debug * chore: fix cspell warnings * refactor: apply suggestions * fix: handle case when `gas_token` is not loaded * dbg * chore: expect contract calls in celo fetcher test * fix: compile errors for celo view on default chain type * chore: remove unused import * refactor: move `celo_gas_tokens` to chain type imports * refactor: apply suggestions * fix: swap arguments in `put_if_present` * fix: indexer tests * refactor: always include `gas_token` to the response * feat: add `gas_token` to all transaction-related endpoints * feat: celo core contracts with historical data * fix: `uncles` field is expected but not present in RPC node response * fix: credo & format * fix: define `CELO_CORE_CONTRACTS` in tests * fix: put `gas_token` under `celo` section * refactor: move token transfers filtering from `TokenTotalSupplyUpdater` to `Transform.TokenTransfers` * feat: add workflow to publish docker image * refactor: apply suggestions by @nikitosing * chore: add docs and specs * fix: malformed import * feat: add pending epoch operations table * refactor: fix format * feat: add transformers for epoch events * feat: add query to stream pending epoch block operations * fix: call to renamed function * fix: add factory method for pending epoch block operation * feat: add utils for logs parsing * feat: add schemas for epoch rewards and election rewards * fix: rename transformer according to event name * feat: implement epoch rewards fetcher * feat: fetch and import epoch logs * feat: improve epoch rewards fetcher 1. Do not fetch logs -- use the ones stored in DB 2. Import epochs to the database 3. Configure fetcher in runtime.exs * feat: add epoch helper functions * activated_validator_group_vote.ex * feat: add task to fetch core contracts * feat: fetch core contract events in the task * fix: merge artifacts * refactor: logs requests and reduce scope of epoch logs request * chore: fix formatting, credo warning, etc. * feat: fetch epoch rewards in one sql query * feat: fetch validator group votes (historical data and on demand) * refactor: rename fields and replace all entries on conflict * feat: validator group votes fetcher * fix: put each topic in a separate request * feat: fetch voter rewards * refactor: split epoch fetcher to multiple modules * feat: send epoch blocks for async fetching from block fetcher * chore: fix credo and formatting issues * fix: dialyzer warnings * fix: add on demand fetch of group votes * fix: failed explorer and indexer tests * fix: block fetcher tests * fix: match error in epoch logs * feat: add env to manage logs batch size when fetching validator group votes * fix: Add `ssl_opts` for Celo repo * fix: add `disabled?` predicate to supervisor config * fix: return empty list when `getPaymentDelegation` is not available * fix: validate the case when there is no voter rewards for an epoch * fix: formatting * fix: import `put_if_present/3` * fix: do not treat genesis block as an epoch block * fix: some of the fetcher tests * dbg * chore: remove commented code * chore: canonical disable flag env name * fix: run test only for celo chain type * fix: add explicit `wait_ for_tasks` in token instance realtime test * fix: set missing `CELO_CORE_CONTRACTS` env * fix: rollback token instance realtime test * chore: remove debug artifact * chore: add docs for new tables * perf: remove unused fields from `celo_validator_group_votes` table * fix: add on_exit clause * refactor: use `remove/3` function for migration rollback possibility * refactor: extract celo core contracts environment variable setup into a separate function * chore: add new vars to `common-blockscout.env` file * chore: add specs and docs for new modules and functions * refactor: eliminate unused import warnings * fix: credo warning * feat: API for celo epoch rewards and fees breakdown (#10308) * feat(json-rpc): support celo-specific fields in transactions * chore: add new chain type to matrix builder * feat: add celo-specific fields to transactions schema * chore: add "celo" to `cspell.json` * chore: fix formatting * refactor: improve naming for celo fields and ignore `ethCompatible` field * chore: add block fetcher test for celo fields * feat: add celo core contract abis * feat: add a cache for core celo contracts * feat: treat celo native token transfers as token transfers * fix: update `cspell.json` with celo words * fix(dialyzer): pattern can never match the type * fix: update `cspell.json` with celo words * feat: move `CELO_NETWORK` env to config * feat: parse native Celo token transfers from internal transactions * feat: add token balances fetching * refactor: remove `dbg` and organize code better * fix: start TokenBalance fetcher * fix: start token balance fetcher * fix: typo in log message * fix: missing fields in celo transaction token transfer * refactor: remove `AbiHandler` in favour of hardcoded pieces of abi * refactor: native token transfers from internal transactions fether * feat: implement parsing of token transfers for traceable rpc node * feat: fetch address coin balances after erc20 celo token transfers * feat: add `gas_token` to transaction response in api v2 * feat: index tokens specified as `feeCurrency` * chore: fix credo warnings * chore: decrease log level to debug * chore: fix cspell warnings * refactor: apply suggestions * fix: handle case when `gas_token` is not loaded * dbg * chore: expect contract calls in celo fetcher test * fix: compile errors for celo view on default chain type * chore: remove unused import * refactor: move `celo_gas_tokens` to chain type imports * refactor: apply suggestions * fix: swap arguments in `put_if_present` * fix: indexer tests * feat: add pending epoch operations table * feat: add several celo core contracts * refactor: fix format * feat: add new core contract default addresses * feat: add transformers for epoch events * feat: add query to stream pending epoch block operations * fix: call to renamed function * fix: add factory method for pending epoch block operation * feat: add utils for logs parsing * feat: add schemas for epoch rewards and election rewards * fix: rename transformer according to event name * feat: implement epoch rewards fetcher * feat: fetch and import epoch logs * feat: improve epoch rewards fetcher 1. Do not fetch logs -- use the ones stored in DB 2. Import epochs to the database 3. Configure fetcher in runtime.exs * fix: remove duplicated module attribute * feat: add epoch helper functions * activated_validator_group_vote.ex * feat: tmp * refactor: always include `gas_token` to the response * feat: add `gas_token` to all transaction-related endpoints * feat: celo core contracts with historical data * fix: `uncles` field is expected but not present in RPC node response * fix: credo & format * fix: define `CELO_CORE_CONTRACTS` in tests * fix: put `gas_token` under `celo` section * refactor: move token transfers filtering from `TokenTotalSupplyUpdater` to `Transform.TokenTransfers` * feat: add workflow to publish docker image * refactor: apply suggestions by @nikitosing * chore: add docs and specs * fix: malformed import * feat: add pending epoch operations table * refactor: fix format * feat: add transformers for epoch events * feat: add query to stream pending epoch block operations * fix: call to renamed function * fix: add factory method for pending epoch block operation * feat: add utils for logs parsing * feat: add schemas for epoch rewards and election rewards * fix: rename transformer according to event name * feat: implement epoch rewards fetcher * feat: fetch and import epoch logs * feat: improve epoch rewards fetcher 1. Do not fetch logs -- use the ones stored in DB 2. Import epochs to the database 3. Configure fetcher in runtime.exs * feat: add epoch helper functions * activated_validator_group_vote.ex * feat: add task to fetch core contracts * feat: fetch core contract events in the task * fix: merge artifacts * refactor: logs requests and reduce scope of epoch logs request * chore: fix formatting, credo warning, etc. * feat: fetch epoch rewards in one sql query * feat: fetch validator group votes (historical data and on demand) * refactor: rename fields and replace all entries on conflict * feat: validator group votes fetcher * fix: put each topic in a separate request * feat: fetch voter rewards * fix: merge artifacts * feat: add epoch rewards in block api response * fix: merge artifacts * feat: add base fee breakdown in block api response * feat: add aggregated election rewards to block api response * perf: use replica when querying epoch rewards * feat: endpoint for paginated election rewards * feat: endpoint for paginated election rewards for address * chore: rename `rewards` to `distributions` * fix: add missing reward type to API error message * refactor: split epoch fetcher to multiple modules * feat: send epoch blocks for async fetching from block fetcher * chore: fix credo and formatting issues * fix: dialyzer warnings * fix: add on demand fetch of group votes * fix: failed explorer and indexer tests * fix: block fetcher tests * chore: remove unused module * fix: sort election rewards by block number * fix: match error in epoch logs * feat: add env to manage logs batch size when fetching validator group votes * fix: credo and formatting warnings * fix: cspell errors * fix: dialyzer errors * fix: Add `ssl_opts` for Celo repo * fix: add missing preloads and make more explicit api response for block * fix: add `disabled?` predicate to supervisor config * fix: return empty list when `getPaymentDelegation` is not available * fix: validate the case when there is no voter rewards for an epoch * fix: formatting * fix: add missing preload * refactor: more robust fees breakdown logic for the case of fee handler * fix: formatting * refactor: move epoch information to the separate endpoint * fix: import `put_if_present/3` * fix: formatting * fix: do not treat genesis block as an epoch block * fix: some of the fetcher tests * dbg * chore: remove commented code * chore: canonical disable flag env name * fix: run test only for celo chain type * fix: add explicit `wait_ for_tasks` in token instance realtime test * fix: set missing `CELO_CORE_CONTRACTS` env * fix: rollback token instance realtime test * chore: remove debug artifact * chore: add `@docs` and `@specs` * fix: missing core contracts var in tests * feat: extend `tabs-counters` endpoint with election rewards count * refactor: move dead address to `Explorer.Chain.SmartContract` * perf: clause with simplified queries in the case of `amount == 0` and/or `block_number == 0` * fix: it comes that `NotLoaded` clause is not redundant actually... * fix: add missing preload in `BlockScoutWeb.AddressChannel` * chore: unset `CELO_CORE_CONTRACTS` in tests * chore: add missing specs * fix: `paginate` clause when `amount == 0` * refactor: move `paging_options/2` to `Explorer.Chain.Celo.ElectionReward` * refactor: avoid using virtual field for block number. * fix: remove redundant condition in the query * chore: clarify `@spec` * chore: fix credo warning * chore: remove `fi-celo` from branches that trigger ci * fix: merge artifacts * fix: remove todo comment
4 months ago
value = transaction[from_key]
validate_key(acc, to_key, value, opts)
feat: support for `:celo` chain type (#10564) * feat: indexing of Celo transactions (#10275) * feat(json-rpc): support celo-specific fields in transactions * chore: add new chain type to matrix builder * feat: add celo-specific fields to transactions schema * chore: add "celo" to `cspell.json` * chore: fix formatting * refactor: improve naming for celo fields and ignore `ethCompatible` field * chore: add block fetcher test for celo fields * feat: add celo core contract abis * feat: add a cache for core celo contracts * feat: treat celo native token transfers as token transfers * fix: update `cspell.json` with celo words * fix(dialyzer): pattern can never match the type * fix: update `cspell.json` with celo words * feat: move `CELO_NETWORK` env to config * feat: parse native Celo token transfers from internal transactions * feat: add token balances fetching * refactor: remove `dbg` and organize code better * fix: start TokenBalance fetcher * fix: start token balance fetcher * fix: typo in log message * fix: missing fields in celo transaction token transfer * refactor: remove `AbiHandler` in favour of hardcoded pieces of abi * refactor: native token transfers from internal transactions fether * feat: implement parsing of token transfers for traceable rpc node * feat: fetch address coin balances after erc20 celo token transfers * feat: add `gas_token` to transaction response in api v2 * feat: index tokens specified as `feeCurrency` * chore: fix credo warnings * chore: decrease log level to debug * chore: fix cspell warnings * refactor: apply suggestions * fix: handle case when `gas_token` is not loaded * dbg * chore: expect contract calls in celo fetcher test * fix: compile errors for celo view on default chain type * chore: remove unused import * refactor: move `celo_gas_tokens` to chain type imports * refactor: apply suggestions * fix: swap arguments in `put_if_present` * fix: indexer tests * refactor: always include `gas_token` to the response * feat: add `gas_token` to all transaction-related endpoints * feat: celo core contracts with historical data * fix: `uncles` field is expected but not present in RPC node response * fix: credo & format * fix: define `CELO_CORE_CONTRACTS` in tests * fix: put `gas_token` under `celo` section * refactor: move token transfers filtering from `TokenTotalSupplyUpdater` to `Transform.TokenTransfers` * feat: add workflow to publish docker image * refactor: apply suggestions by @nikitosing * chore: add docs and specs * fix: malformed import * fix: publish image for celo * fix: ci build for celo * feat: index celo epoch transactions (#9944) * feat(json-rpc): support celo-specific fields in transactions * chore: add new chain type to matrix builder * feat: add celo-specific fields to transactions schema * chore: add "celo" to `cspell.json` * chore: fix formatting * refactor: improve naming for celo fields and ignore `ethCompatible` field * chore: add block fetcher test for celo fields * feat: add celo core contract abis * feat: add a cache for core celo contracts * feat: treat celo native token transfers as token transfers * fix: update `cspell.json` with celo words * fix(dialyzer): pattern can never match the type * fix: update `cspell.json` with celo words * feat: move `CELO_NETWORK` env to config * feat: parse native Celo token transfers from internal transactions * feat: add token balances fetching * refactor: remove `dbg` and organize code better * fix: start TokenBalance fetcher * fix: start token balance fetcher * fix: typo in log message * fix: missing fields in celo transaction token transfer * refactor: remove `AbiHandler` in favour of hardcoded pieces of abi * refactor: native token transfers from internal transactions fether * feat: implement parsing of token transfers for traceable rpc node * feat: fetch address coin balances after erc20 celo token transfers * feat: add `gas_token` to transaction response in api v2 * feat: index tokens specified as `feeCurrency` * chore: fix credo warnings * chore: decrease log level to debug * chore: fix cspell warnings * refactor: apply suggestions * fix: handle case when `gas_token` is not loaded * dbg * chore: expect contract calls in celo fetcher test * fix: compile errors for celo view on default chain type * chore: remove unused import * refactor: move `celo_gas_tokens` to chain type imports * refactor: apply suggestions * fix: swap arguments in `put_if_present` * fix: indexer tests * refactor: always include `gas_token` to the response * feat: add `gas_token` to all transaction-related endpoints * feat: celo core contracts with historical data * fix: `uncles` field is expected but not present in RPC node response * fix: credo & format * fix: define `CELO_CORE_CONTRACTS` in tests * fix: put `gas_token` under `celo` section * refactor: move token transfers filtering from `TokenTotalSupplyUpdater` to `Transform.TokenTransfers` * feat: add workflow to publish docker image * refactor: apply suggestions by @nikitosing * chore: add docs and specs * fix: malformed import * feat: add pending epoch operations table * refactor: fix format * feat: add transformers for epoch events * feat: add query to stream pending epoch block operations * fix: call to renamed function * fix: add factory method for pending epoch block operation * feat: add utils for logs parsing * feat: add schemas for epoch rewards and election rewards * fix: rename transformer according to event name * feat: implement epoch rewards fetcher * feat: fetch and import epoch logs * feat: improve epoch rewards fetcher 1. Do not fetch logs -- use the ones stored in DB 2. Import epochs to the database 3. Configure fetcher in runtime.exs * feat: add epoch helper functions * activated_validator_group_vote.ex * feat: add task to fetch core contracts * feat: fetch core contract events in the task * fix: merge artifacts * refactor: logs requests and reduce scope of epoch logs request * chore: fix formatting, credo warning, etc. * feat: fetch epoch rewards in one sql query * feat: fetch validator group votes (historical data and on demand) * refactor: rename fields and replace all entries on conflict * feat: validator group votes fetcher * fix: put each topic in a separate request * feat: fetch voter rewards * refactor: split epoch fetcher to multiple modules * feat: send epoch blocks for async fetching from block fetcher * chore: fix credo and formatting issues * fix: dialyzer warnings * fix: add on demand fetch of group votes * fix: failed explorer and indexer tests * fix: block fetcher tests * fix: match error in epoch logs * feat: add env to manage logs batch size when fetching validator group votes * fix: Add `ssl_opts` for Celo repo * fix: add `disabled?` predicate to supervisor config * fix: return empty list when `getPaymentDelegation` is not available * fix: validate the case when there is no voter rewards for an epoch * fix: formatting * fix: import `put_if_present/3` * fix: do not treat genesis block as an epoch block * fix: some of the fetcher tests * dbg * chore: remove commented code * chore: canonical disable flag env name * fix: run test only for celo chain type * fix: add explicit `wait_ for_tasks` in token instance realtime test * fix: set missing `CELO_CORE_CONTRACTS` env * fix: rollback token instance realtime test * chore: remove debug artifact * chore: add docs for new tables * perf: remove unused fields from `celo_validator_group_votes` table * fix: add on_exit clause * refactor: use `remove/3` function for migration rollback possibility * refactor: extract celo core contracts environment variable setup into a separate function * chore: add new vars to `common-blockscout.env` file * chore: add specs and docs for new modules and functions * refactor: eliminate unused import warnings * fix: credo warning * feat: API for celo epoch rewards and fees breakdown (#10308) * feat(json-rpc): support celo-specific fields in transactions * chore: add new chain type to matrix builder * feat: add celo-specific fields to transactions schema * chore: add "celo" to `cspell.json` * chore: fix formatting * refactor: improve naming for celo fields and ignore `ethCompatible` field * chore: add block fetcher test for celo fields * feat: add celo core contract abis * feat: add a cache for core celo contracts * feat: treat celo native token transfers as token transfers * fix: update `cspell.json` with celo words * fix(dialyzer): pattern can never match the type * fix: update `cspell.json` with celo words * feat: move `CELO_NETWORK` env to config * feat: parse native Celo token transfers from internal transactions * feat: add token balances fetching * refactor: remove `dbg` and organize code better * fix: start TokenBalance fetcher * fix: start token balance fetcher * fix: typo in log message * fix: missing fields in celo transaction token transfer * refactor: remove `AbiHandler` in favour of hardcoded pieces of abi * refactor: native token transfers from internal transactions fether * feat: implement parsing of token transfers for traceable rpc node * feat: fetch address coin balances after erc20 celo token transfers * feat: add `gas_token` to transaction response in api v2 * feat: index tokens specified as `feeCurrency` * chore: fix credo warnings * chore: decrease log level to debug * chore: fix cspell warnings * refactor: apply suggestions * fix: handle case when `gas_token` is not loaded * dbg * chore: expect contract calls in celo fetcher test * fix: compile errors for celo view on default chain type * chore: remove unused import * refactor: move `celo_gas_tokens` to chain type imports * refactor: apply suggestions * fix: swap arguments in `put_if_present` * fix: indexer tests * feat: add pending epoch operations table * feat: add several celo core contracts * refactor: fix format * feat: add new core contract default addresses * feat: add transformers for epoch events * feat: add query to stream pending epoch block operations * fix: call to renamed function * fix: add factory method for pending epoch block operation * feat: add utils for logs parsing * feat: add schemas for epoch rewards and election rewards * fix: rename transformer according to event name * feat: implement epoch rewards fetcher * feat: fetch and import epoch logs * feat: improve epoch rewards fetcher 1. Do not fetch logs -- use the ones stored in DB 2. Import epochs to the database 3. Configure fetcher in runtime.exs * fix: remove duplicated module attribute * feat: add epoch helper functions * activated_validator_group_vote.ex * feat: tmp * refactor: always include `gas_token` to the response * feat: add `gas_token` to all transaction-related endpoints * feat: celo core contracts with historical data * fix: `uncles` field is expected but not present in RPC node response * fix: credo & format * fix: define `CELO_CORE_CONTRACTS` in tests * fix: put `gas_token` under `celo` section * refactor: move token transfers filtering from `TokenTotalSupplyUpdater` to `Transform.TokenTransfers` * feat: add workflow to publish docker image * refactor: apply suggestions by @nikitosing * chore: add docs and specs * fix: malformed import * feat: add pending epoch operations table * refactor: fix format * feat: add transformers for epoch events * feat: add query to stream pending epoch block operations * fix: call to renamed function * fix: add factory method for pending epoch block operation * feat: add utils for logs parsing * feat: add schemas for epoch rewards and election rewards * fix: rename transformer according to event name * feat: implement epoch rewards fetcher * feat: fetch and import epoch logs * feat: improve epoch rewards fetcher 1. Do not fetch logs -- use the ones stored in DB 2. Import epochs to the database 3. Configure fetcher in runtime.exs * feat: add epoch helper functions * activated_validator_group_vote.ex * feat: add task to fetch core contracts * feat: fetch core contract events in the task * fix: merge artifacts * refactor: logs requests and reduce scope of epoch logs request * chore: fix formatting, credo warning, etc. * feat: fetch epoch rewards in one sql query * feat: fetch validator group votes (historical data and on demand) * refactor: rename fields and replace all entries on conflict * feat: validator group votes fetcher * fix: put each topic in a separate request * feat: fetch voter rewards * fix: merge artifacts * feat: add epoch rewards in block api response * fix: merge artifacts * feat: add base fee breakdown in block api response * feat: add aggregated election rewards to block api response * perf: use replica when querying epoch rewards * feat: endpoint for paginated election rewards * feat: endpoint for paginated election rewards for address * chore: rename `rewards` to `distributions` * fix: add missing reward type to API error message * refactor: split epoch fetcher to multiple modules * feat: send epoch blocks for async fetching from block fetcher * chore: fix credo and formatting issues * fix: dialyzer warnings * fix: add on demand fetch of group votes * fix: failed explorer and indexer tests * fix: block fetcher tests * chore: remove unused module * fix: sort election rewards by block number * fix: match error in epoch logs * feat: add env to manage logs batch size when fetching validator group votes * fix: credo and formatting warnings * fix: cspell errors * fix: dialyzer errors * fix: Add `ssl_opts` for Celo repo * fix: add missing preloads and make more explicit api response for block * fix: add `disabled?` predicate to supervisor config * fix: return empty list when `getPaymentDelegation` is not available * fix: validate the case when there is no voter rewards for an epoch * fix: formatting * fix: add missing preload * refactor: more robust fees breakdown logic for the case of fee handler * fix: formatting * refactor: move epoch information to the separate endpoint * fix: import `put_if_present/3` * fix: formatting * fix: do not treat genesis block as an epoch block * fix: some of the fetcher tests * dbg * chore: remove commented code * chore: canonical disable flag env name * fix: run test only for celo chain type * fix: add explicit `wait_ for_tasks` in token instance realtime test * fix: set missing `CELO_CORE_CONTRACTS` env * fix: rollback token instance realtime test * chore: remove debug artifact * chore: add `@docs` and `@specs` * fix: missing core contracts var in tests * feat: extend `tabs-counters` endpoint with election rewards count * refactor: move dead address to `Explorer.Chain.SmartContract` * perf: clause with simplified queries in the case of `amount == 0` and/or `block_number == 0` * fix: it comes that `NotLoaded` clause is not redundant actually... * fix: add missing preload in `BlockScoutWeb.AddressChannel` * chore: unset `CELO_CORE_CONTRACTS` in tests * chore: add missing specs * fix: `paginate` clause when `amount == 0` * refactor: move `paging_options/2` to `Explorer.Chain.Celo.ElectionReward` * refactor: avoid using virtual field for block number. * fix: remove redundant condition in the query * chore: clarify `@spec` * chore: fix credo warning * chore: remove `fi-celo` from branches that trigger ci * fix: merge artifacts * fix: remove todo comment
4 months ago
end)
end
defp validate_key(acc, _to_key, nil, _opts), do: acc
defp validate_key(acc, to_key, value, %{:validation => validation}) do
case validation do
:address_hash ->
if address_correct?(value), do: Map.put(acc, to_key, value), else: acc
_ ->
Map.put(acc, to_key, value)
end
end
defp validate_key(acc, to_key, value, _validation) do
Map.put(acc, to_key, value)
end
# todo: The similar function exists in Indexer application:
# Here is the room for future refactoring to keep a single function.
@spec address_correct?(binary()) :: boolean()
defp address_correct?(address) when is_binary(address) do
String.match?(address, ~r/^0x[[:xdigit:]]{40}$/i)
end
defp address_correct?(_address) do
false
end
end