Blockchain explorer for Ethereum based network and a tool for inspecting and analyzing EVM based blockchains.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
blockscout/config/config_helper.exs

306 lines
9.6 KiB

defmodule ConfigHelper do
require Logger
import Bitwise
alias Explorer.ExchangeRates.Source
alias Explorer.Market.History.Source.{MarketCap, Price, TVL}
alias Indexer.Transform.Blocks
Deposits and Withdrawals for Polygon Edge (#8180) * Prepare tables for Polygon Supernet Withdrawals module * Prepare Indexer.Fetcher.PolygonSupernetWithdrawal * Prepare Indexer.Fetcher.PolygonSupernetWithdrawalExit * Prepare tables for Polygon Supernet Deposits module * Partially add Polygon Supernet Deposits module * Add Polygon Supernet Deposits module * Extend transaction data in API v2 for Polygon Supernet Deposits and Withdrawals * Extend API v2 for list of Polygon Supernet Deposits * Extend API v2 for list of Polygon Supernet Withdrawals * Modify collated_gas_price constraint to support Immutable test chain * Update number of retries * Fix Indexer.Fetcher.PolygonSupernetWithdrawal * Fix API v2 for Indexer.Fetcher.PolygonSupernetWithdrawal * Return page_size to 50 * Fix API v2 for Indexer.Fetcher.PolygonSupernetDeposit * Fix Explorer.Validator.MetadataRetriever to prevent sending request to nil address * Refactor PolygonSupernet* modules * Refactor PolygonSupernet* modules * Small refactoring of PolygonSupernet* modules * Update changelog * Fixes for dialyzer and cspell * Fix of Explorer tests * Add env vars for docker * Add env variable for eth_getLogs * Share event signatures between modules * Refactoring for fill_block_range function * Small refactoring of transaction actions module * Move polygon_supernet_* modules to a separate folder * Add specs for public functions * Remove redundant function * Reuse decode_data function * Move polygon_supernet_* files into a separate folder * Move polygon_supernet_* files into a separate folder * Partially disband Explorer.Chain module * Define chain type * Remove infinite waiting for safe block * max format * Rename Polygon Supernet to Polygon Edge * Set INDEXER_INTERNAL_TRANSACTIONS_TRACER_TYPE to polygon_edge if CHAIN_TYPE is polygon_edge * Fix chain type definition * Fix repos list definition in releases_tasks * INDEXER_POLYGON_SUPERNET_ -> INDEXER_POLYGON_EDGE_ --------- Co-authored-by: POA <33550681+poa@users.noreply.github.com> Co-authored-by: Viktor Baranov <baranov.viktor.27@gmail.com>
1 year ago
def repos do
base_repos = [Explorer.Repo, Explorer.Repo.Account]
repos =
case chain_type() do
:ethereum -> base_repos ++ [Explorer.Repo.Beacon]
:optimism -> base_repos ++ [Explorer.Repo.Optimism]
:polygon_edge -> base_repos ++ [Explorer.Repo.PolygonEdge]
:polygon_zkevm -> base_repos ++ [Explorer.Repo.PolygonZkevm]
:rsk -> base_repos ++ [Explorer.Repo.RSK]
:shibarium -> base_repos ++ [Explorer.Repo.Shibarium]
:suave -> base_repos ++ [Explorer.Repo.Suave]
:filecoin -> base_repos ++ [Explorer.Repo.Filecoin]
:stability -> base_repos ++ [Explorer.Repo.Stability]
:zksync -> base_repos ++ [Explorer.Repo.ZkSync]
feat: indexer for cross level messages on Arbitrum (#9312) * Initial version of x-level messages indexer * fixes for cspell and credo * new state of x-level messages * Monitoring of new L1-to-L2 messages on L1 * new batches discovery * fetcher workers in separate modules * proper name * Fix for responses without "id", e.g. "Too Many Requests" * update DB with new batches and corresponding data * update DB with confirmed blocks * fixes for cspell and credo * tracking commitments confirmations for L1 to L2 messages * Proper usign of max function * tracking completion of L2 to L1 messages * catchup historical messages to L2 * incorrect version of committed file * catchup historical messages from L2 and completion of L1-to-L2 messages * historical batches catchup * status for historical l2-to-l1 messages * address matching issue * catchup historical executions of L2-to-L1 messages * db query to find unconfirmed blocks gaps * first changes to catchup historical confirmations * finalized catchup of historical confirmations * 4844 blobs support * fix for the issue with multiple confirmations * limit amount of batches to handle at once * Use latest L1 block by fetchers if start block is not configured * merge issue fix * missed file * historical messages discovery * reduce logs severity * first iteration to improve documentation for new functionality * second iteration to improve documentation for new functionality * third iteration to improve documentation for new functionality * fourth iteration to improve documentation for new functionality * fifth iteration to improve documentation for new functionality * final iteration to improve documentation for new functionality * merge issues addressed * code review issues addressed * code review issues addressed * fix merge issue * raising exception in the case of DB inconsistency * fix formatting issue * termination case for RollupMessagesCatchup * code review comments addressed * code review comments addressed * consistency in primary keys * dialyzer fix * code review comments addressed * missed doc comment * code review comments addressed * updated indices creation as per code review comments * fix merge issue * configuration of intervals as time variables * TODO added to reflect improvement ability * database fields refactoring * association renaming * feat: APIv2 endpoints for Arbitrum messages and batches (#9963) * Arbitrum related info in Transaction and Block views * Views to get info about batches and messages * usage of committed for batches instead of confirmed * merge issues addressed * changes after merge * formatting issue fix * code review comment addressed * associations and fields in api response renamed * format issue addressed * feat: Arbitrum-specific fields in the block and transaction API endpoints (#10067) * Arbitrum related info in Transaction and Block views * Views to get info about batches and messages * usage of committed for batches instead of confirmed * merge issues addressed * changes after merge * formatting issue fix * block and transaction views extended * code review comment addressed * associations and fields in api response renamed * format issue addressed * fix credo issue * fix tests issues * ethereumjsonrpc test fail investigation * test issues fixes
6 months ago
:arbitrum -> base_repos ++ [Explorer.Repo.Arbitrum]
_ -> base_repos
end
ext_repos =
[
{parse_bool_env_var("BRIDGED_TOKENS_ENABLED"), Explorer.Repo.BridgedTokens},
{parse_bool_env_var("MUD_INDEXER_ENABLED"), Explorer.Repo.Mud}
]
|> Enum.filter(&elem(&1, 0))
|> Enum.map(&elem(&1, 1))
repos ++ ext_repos
Deposits and Withdrawals for Polygon Edge (#8180) * Prepare tables for Polygon Supernet Withdrawals module * Prepare Indexer.Fetcher.PolygonSupernetWithdrawal * Prepare Indexer.Fetcher.PolygonSupernetWithdrawalExit * Prepare tables for Polygon Supernet Deposits module * Partially add Polygon Supernet Deposits module * Add Polygon Supernet Deposits module * Extend transaction data in API v2 for Polygon Supernet Deposits and Withdrawals * Extend API v2 for list of Polygon Supernet Deposits * Extend API v2 for list of Polygon Supernet Withdrawals * Modify collated_gas_price constraint to support Immutable test chain * Update number of retries * Fix Indexer.Fetcher.PolygonSupernetWithdrawal * Fix API v2 for Indexer.Fetcher.PolygonSupernetWithdrawal * Return page_size to 50 * Fix API v2 for Indexer.Fetcher.PolygonSupernetDeposit * Fix Explorer.Validator.MetadataRetriever to prevent sending request to nil address * Refactor PolygonSupernet* modules * Refactor PolygonSupernet* modules * Small refactoring of PolygonSupernet* modules * Update changelog * Fixes for dialyzer and cspell * Fix of Explorer tests * Add env vars for docker * Add env variable for eth_getLogs * Share event signatures between modules * Refactoring for fill_block_range function * Small refactoring of transaction actions module * Move polygon_supernet_* modules to a separate folder * Add specs for public functions * Remove redundant function * Reuse decode_data function * Move polygon_supernet_* files into a separate folder * Move polygon_supernet_* files into a separate folder * Partially disband Explorer.Chain module * Define chain type * Remove infinite waiting for safe block * max format * Rename Polygon Supernet to Polygon Edge * Set INDEXER_INTERNAL_TRANSACTIONS_TRACER_TYPE to polygon_edge if CHAIN_TYPE is polygon_edge * Fix chain type definition * Fix repos list definition in releases_tasks * INDEXER_POLYGON_SUPERNET_ -> INDEXER_POLYGON_EDGE_ --------- Co-authored-by: POA <33550681+poa@users.noreply.github.com> Co-authored-by: Viktor Baranov <baranov.viktor.27@gmail.com>
1 year ago
end
@spec hackney_options() :: any()
def hackney_options() do
basic_auth_user = System.get_env("ETHEREUM_JSONRPC_USER", "")
basic_auth_pass = System.get_env("ETHEREUM_JSONRPC_PASSWORD", nil)
[pool: :ethereum_jsonrpc]
|> (&if(System.get_env("ETHEREUM_JSONRPC_HTTP_INSECURE", "") == "true", do: [:insecure] ++ &1, else: &1)).()
|> (&if(basic_auth_user != "" && !is_nil(basic_auth_pass),
do: [basic_auth: {basic_auth_user, basic_auth_pass}] ++ &1,
else: &1
)).()
end
@spec timeout(non_neg_integer()) :: non_neg_integer()
def timeout(default_minutes \\ 1) do
case Integer.parse(safe_get_env("ETHEREUM_JSONRPC_HTTP_TIMEOUT", "#{default_minutes * 60}")) do
{seconds, ""} -> seconds
_ -> default_minutes * 60
end
|> :timer.seconds()
end
@spec parse_integer_env_var(String.t(), integer()) :: non_neg_integer()
def parse_integer_env_var(env_var, default_value) do
env_var
|> safe_get_env(to_string(default_value))
|> Integer.parse()
|> case do
{integer, _} -> integer
_ -> 0
end
end
@spec parse_float_env_var(String.t(), float()) :: float()
def parse_float_env_var(env_var, default_value) do
env_var
|> safe_get_env(to_string(default_value))
|> Float.parse()
|> case do
{float, _} -> float
_ -> 0
end
end
@spec parse_integer_or_nil_env_var(String.t()) :: non_neg_integer() | nil
def parse_integer_or_nil_env_var(env_var) do
env_var
|> System.get_env("")
|> Integer.parse()
|> case do
{integer, _} -> integer
_ -> nil
end
end
@spec parse_time_env_var(String.t(), String.t() | nil) :: non_neg_integer()
def parse_time_env_var(env_var, default_value) do
case env_var |> safe_get_env(default_value) |> String.downcase() |> Integer.parse() do
{milliseconds, "ms"} -> milliseconds
{hours, "h"} -> :timer.hours(hours)
{minutes, "m"} -> :timer.minutes(minutes)
{seconds, s} when s in ["s", ""] -> :timer.seconds(seconds)
_ -> 0
end
end
@doc """
Parses value of env var through catalogued values list. If a value is not in the list, nil is returned.
Also, the application shutdown option is supported, if a value is wrong.
"""
@spec parse_catalog_value(String.t(), List.t(), bool(), String.t() | nil) :: atom() | nil
def parse_catalog_value(env_var, catalog, shutdown_on_wrong_value?, default_value \\ nil) do
value = env_var |> safe_get_env(default_value)
if value !== "" do
if value in catalog do
String.to_atom(value)
else
if shutdown_on_wrong_value? do
Logger.error(wrong_value_error(value, env_var, catalog))
exit(:shutdown)
else
Logger.warning(wrong_value_error(value, env_var, catalog))
nil
end
end
else
nil
end
end
defp wrong_value_error(value, env_var, catalog) do
"Invalid value \"#{value}\" of #{env_var} environment variable is provided. Supported values are #{inspect(catalog)}"
end
def safe_get_env(env_var, default_value) do
env_var
|> System.get_env(default_value)
|> case do
"" -> default_value
value -> value
end
|> to_string()
end
@spec parse_bool_env_var(String.t(), String.t()) :: boolean()
def parse_bool_env_var(env_var, default_value \\ "false"),
do: String.downcase(safe_get_env(env_var, default_value)) == "true"
@spec cache_ttl_check_interval(boolean()) :: non_neg_integer() | false
def cache_ttl_check_interval(disable_indexer?) do
if(disable_indexer?, do: :timer.seconds(1), else: false)
end
@spec cache_global_ttl(boolean()) :: non_neg_integer()
def cache_global_ttl(disable_indexer?) do
if(disable_indexer?, do: :timer.seconds(5))
end
@spec indexer_memory_limit() :: integer()
def indexer_memory_limit do
indexer_memory_limit_default = 1
"INDEXER_MEMORY_LIMIT"
|> safe_get_env(to_string(indexer_memory_limit_default))
|> String.downcase()
|> Integer.parse()
|> case do
{integer, g} when g in ["g", "gb", ""] -> integer <<< 30
{integer, m} when m in ["m", "mb"] -> integer <<< 20
_ -> indexer_memory_limit_default <<< 30
end
end
@spec exchange_rates_source() :: Source.CoinGecko | Source.CoinMarketCap | Source.Mobula
def exchange_rates_source do
case System.get_env("EXCHANGE_RATES_MARKET_CAP_SOURCE") do
"coin_gecko" -> Source.CoinGecko
"coin_market_cap" -> Source.CoinMarketCap
"mobula" -> Source.Mobula
_ -> Source.CoinGecko
end
end
@spec exchange_rates_market_cap_source() :: MarketCap.CoinGecko | MarketCap.CoinMarketCap | MarketCap.Mobula
def exchange_rates_market_cap_source do
case System.get_env("EXCHANGE_RATES_MARKET_CAP_SOURCE") do
"coin_gecko" -> MarketCap.CoinGecko
"coin_market_cap" -> MarketCap.CoinMarketCap
"mobula" -> MarketCap.Mobula
_ -> MarketCap.CoinGecko
end
end
@spec exchange_rates_tvl_source() :: TVL.DefiLlama
def exchange_rates_tvl_source do
case System.get_env("EXCHANGE_RATES_TVL_SOURCE") do
"defillama" -> TVL.DefiLlama
_ -> TVL.DefiLlama
end
end
@spec exchange_rates_price_source() :: Price.CoinGecko | Price.CoinMarketCap | Price.CryptoCompare | Price.Mobula
def exchange_rates_price_source do
case System.get_env("EXCHANGE_RATES_PRICE_SOURCE") do
"coin_gecko" -> Price.CoinGecko
"coin_market_cap" -> Price.CoinMarketCap
"crypto_compare" -> Price.CryptoCompare
"mobula" -> Price.Mobula
_ -> Price.CryptoCompare
end
end
@spec exchange_rates_secondary_coin_price_source() ::
Price.CoinGecko | Price.CoinMarketCap | Price.CryptoCompare | Price.Mobula
def exchange_rates_secondary_coin_price_source do
cmc_secondary_coin_id = System.get_env("EXCHANGE_RATES_COINMARKETCAP_SECONDARY_COIN_ID")
cg_secondary_coin_id = System.get_env("EXCHANGE_RATES_COINGECKO_SECONDARY_COIN_ID")
cc_secondary_coin_symbol = System.get_env("EXCHANGE_RATES_CRYPTOCOMPARE_SECONDARY_COIN_SYMBOL")
mobula_secondary_coin_id = System.get_env("EXCHANGE_RATES_MOBULA_SECONDARY_COIN_ID")
cond do
cg_secondary_coin_id && cg_secondary_coin_id !== "" -> Price.CoinGecko
cmc_secondary_coin_id && cmc_secondary_coin_id !== "" -> Price.CoinMarketCap
cc_secondary_coin_symbol && cc_secondary_coin_symbol !== "" -> Price.CryptoCompare
mobula_secondary_coin_id && mobula_secondary_coin_id !== "" -> Price.Mobula
true -> Price.CryptoCompare
end
end
def block_transformer do
block_transformers = %{
"clique" => Blocks.Clique,
"base" => Blocks.Base
}
# Compile time environment variable access requires recompilation.
configured_transformer = safe_get_env("BLOCK_TRANSFORMER", "base")
case Map.get(block_transformers, configured_transformer) do
nil ->
raise """
No such block transformer: #{configured_transformer}.
Valid values are:
#{Enum.join(Map.keys(block_transformers), "\n")}
Please update environment variable BLOCK_TRANSFORMER accordingly.
"""
transformer ->
transformer
end
end
@spec parse_json_env_var(String.t(), String.t()) :: any()
def parse_json_env_var(env_var, default_value) do
env_var
|> safe_get_env(default_value)
|> Jason.decode!()
rescue
err -> raise "Invalid JSON in environment variable #{env_var}: #{inspect(err)}"
end
@spec parse_list_env_var(String.t(), String.t() | nil) :: list()
def parse_list_env_var(env_var, default_value \\ nil) do
addresses_var = safe_get_env(env_var, default_value)
if addresses_var !== "" do
addresses_list = (addresses_var && String.split(addresses_var, ",")) || []
formatted_addresses_list =
addresses_list
|> Enum.map(fn addr ->
String.downcase(addr)
end)
formatted_addresses_list
else
[]
end
end
@supported_chain_types [
"default",
"arbitrum",
"ethereum",
"filecoin",
"optimism",
"polygon_edge",
"polygon_zkevm",
"rsk",
"shibarium",
"stability",
"suave",
"zetachain",
"zksync"
]
@spec chain_type() :: atom() | nil
def chain_type, do: parse_catalog_value("CHAIN_TYPE", @supported_chain_types, true, "default")
@spec eth_call_url(String.t() | nil) :: String.t() | nil
def eth_call_url(default \\ nil) do
System.get_env("ETHEREUM_JSONRPC_ETH_CALL_URL") || System.get_env("ETHEREUM_JSONRPC_HTTP_URL") || default
end
end