Merge branch 'master' into circle-ci-confg-update

pull/1988/head
Victor Baranov 6 years ago committed by GitHub
commit 2cebce70b0
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
  1. 1
      CHANGELOG.md
  2. 179
      README.md
  3. 5
      apps/block_scout_web/lib/block_scout_web/etherscan.ex
  4. 3
      apps/block_scout_web/lib/block_scout_web/views/api/rpc/transaction_view.ex
  5. 3
      apps/block_scout_web/test/block_scout_web/controllers/api/rpc/transaction_controller_test.exs
  6. 8
      apps/explorer/config/config.exs
  7. 3
      apps/explorer/lib/explorer/application.ex
  8. 70
      apps/explorer/lib/explorer/chain.ex
  9. 29
      apps/explorer/lib/explorer/chain/import/runner/staking_pools.ex
  10. 5
      apps/explorer/lib/explorer/paging_options.ex
  11. 124
      apps/explorer/lib/explorer/staking/epoch_counter.ex
  12. 23
      apps/explorer/lib/explorer/staking/pools_reader.ex
  13. 802
      apps/explorer/priv/contracts_abi/pos/staking.json
  14. 55
      apps/explorer/test/explorer/chain_test.exs
  15. 97
      apps/explorer/test/explorer/staking/epoch_counter_test.exs
  16. 21
      apps/explorer/test/explorer/staking/pools_reader_test.exs
  17. 1
      apps/indexer/lib/indexer/fetcher/staking_pools.ex
  18. 19
      apps/indexer/test/indexer/fetcher/staking_pools_test.exs

@ -57,6 +57,7 @@
- [#1900](https://github.com/poanetwork/blockscout/pull/1900) - SUPPORTED_CHAINS ENV var - [#1900](https://github.com/poanetwork/blockscout/pull/1900) - SUPPORTED_CHAINS ENV var
- [#1892](https://github.com/poanetwork/blockscout/pull/1892) - Remove temporary worker modules - [#1892](https://github.com/poanetwork/blockscout/pull/1892) - Remove temporary worker modules
- [#1958](https://github.com/poanetwork/blockscout/pull/1958) - Default value for release link env var - [#1958](https://github.com/poanetwork/blockscout/pull/1958) - Default value for release link env var
- [#1975](https://github.com/poanetwork/blockscout/pull/1975) - add log index to transaction view
- [#1988](https://github.com/poanetwork/blockscout/pull/1988) - Fix wrong parity tasks names in Circle CI - [#1988](https://github.com/poanetwork/blockscout/pull/1988) - Fix wrong parity tasks names in Circle CI
## 1.3.10-beta ## 1.3.10-beta

@ -16,7 +16,9 @@ BlockScout provides a comprehensive, easy-to-use interface for users to view, co
Following is an overview of the project and instructions for [getting started](#getting-started). Following is an overview of the project and instructions for [getting started](#getting-started).
Visit the [POA BlockScout forum](https://forum.poa.network/c/blockscout) or the [Gitter Channel](https://gitter.im/poanetwork/blockscout) to access additional information or post questions. Visit the [POA BlockScout forum](https://forum.poa.network/c/blockscout) for additional deployment instructions, FAQs, troubleshooting, and other BlockScout related items. You can also post and answer questions here.
You can also access the dev chatroom on our [Gitter Channel](https://gitter.im/poanetwork/blockscout).
## About BlockScout ## About BlockScout
@ -41,31 +43,16 @@ Currently available block explorers (i.e. Etherscan and Etherchain) are closed s
### Supported Projects ### Supported Projects
#### Hosted Chains | **Hosted Mainnets** | **Hosted Testnets** | **Additional Chains using BlockScout** |
|--------------------------------------------------------|-------------------------------------------------------|----------------------------------------------------|
* [POA Core Network](https://blockscout.com/poa/core) | [Aerum](https://blockscout.com/aerum/mainnet) | [Goerli Testnet](https://blockscout.com/eth/goerli) | [ARTIS](https://explorer.sigma1.artis.network) |
* [POA Sokol Testnet](https://blockscout.com/poa/sokol) | [Callisto](https://blockscout.com/callisto/mainnet) | [Kovan Testnet](https://blockscout.com/eth/kovan) | [Ether-1](https://blocks.ether1.wattpool.net/) |
* [xDai Chain](https://blockscout.com/poa/dai) | [Ethereum Classic](https://blockscout.com/etc/mainnet) | [POA Sokol Testnet](https://blockscout.com/poa/sokol) | [Fuse Network](https://explorer.fuse.io/) |
* [Ethereum Mainnet](https://blockscout.com/eth/mainnet) | [Ethereum Mainnet](https://blockscout.com/eth/mainnet) | [Rinkeby Testnet](https://blockscout.com/eth/rinkeby) | [Oasis Labs](https://blockexplorer.oasiscloud.io/) |
* [Kovan Testnet](https://blockscout.com/eth/kovan) | [POA Core Network](https://blockscout.com/poa/core) | [Ropsten Testnet](https://blockscout.com/eth/ropsten) | [Petrichor](https://explorer.petrachor.com/) |
* [Ropsten Testnet](https://blockscout.com/eth/ropsten) | [RSK](https://blockscout.com/rsk/mainnet) | | [PIRL](http://pirl.es/) |
* [Goerli Testnet](https://blockscout.com/eth/goerli) | [xDai Chain](https://blockscout.com/poa/dai) | | [SafeChain](https://explorer.safechain.io) |
* [Rinkeby Testnet](https://blockscout.com/eth/rinkeby) | | | [SpringChain](https://explorer.springrole.com/) |
* [Ethereum Classic](https://blockscout.com/etc/mainnet)
* [Aerum](https://blockscout.com/aerum/mainnet)
* [Callisto](https://blockscout.com/callisto/mainnet)
* [RSK](https://blockscout.com/rsk/mainnet)
#### Additional Chains Utilizing BlockScout
* [Oasis Labs](https://blockexplorer.oasiscloud.io/)
* [Fuse Network](https://explorer.fuse.io/)
* [ARTIS](https://explorer.sigma1.artis.network)
* [SafeChain](https://explorer.safechain.io)
* [SpringChain](https://explorer.springrole.com/)
* [PIRL](http://pirl.es/)
* [Petrichor](https://explorer.petrachor.com/)
* [Ether-1](https://blocks.ether1.wattpool.net/)
### Visual Interface ### Visual Interface
@ -74,13 +61,24 @@ Interface for the POA network _updated 02/2019_
![BlockScout Example](explorer_example_2_2019.gif) ![BlockScout Example](explorer_example_2_2019.gif)
## Getting Started
We use [Terraform](https://www.terraform.io/intro/getting-started/install.html) to build the correct infrastructure to run BlockScout. See [https://github.com/poanetwork/blockscout-terraform](https://github.com/poanetwork/blockscout-terraform) for details. ### Umbrella Project Organization
### Requirements This repository is an [umbrella project](https://elixir-lang.org/getting-started/mix-otp/dependencies-and-umbrella-projects.html). Each directory under `apps/` is a separate [Mix](https://hexdocs.pm/mix/Mix.html) project and [OTP application](https://hexdocs.pm/elixir/Application.html), but the projects can use each other as a dependency in their `mix.exs`.
The [development stack page](https://github.com/poanetwork/blockscout/wiki/Development-Stack) contains more information about these frameworks. Each OTP application has a restricted domain.
| Directory | OTP Application | Namespace | Purpose |
|:------------------------|:--------------------|:------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `apps/ethereum_jsonrpc` | `:ethereum_jsonrpc` | `EthereumJSONRPC` | Ethereum JSONRPC client. It is allowed to know `Explorer`'s param format, but it cannot directly depend on `:explorer` |
| `apps/explorer` | `:explorer` | `Explorer` | Storage for the indexed chain. Can read and write to the backing storage. MUST be able to boot in a read-only mode when run independently from `:indexer`, so cannot depend on `:indexer` as that would start `:indexer` indexing. |
| `apps/block_scout_web` | `:block_scout_web` | `BlockScoutWeb` | Phoenix interface to `:explorer`. The minimum interface to allow web access should go in `:block_scout_web`. Any business rules or interface not tied directly to `Phoenix` or `Plug` should go in `:explorer`. MUST be able to boot in a read-only mode when run independently from `:indexer`, so cannot depend on `:indexer` as that would start `:indexer` indexing. |
| `apps/indexer` | `:indexer` | `Indexer` | Uses `:ethereum_jsonrpc` to index chain and batch import data into `:explorer`. Any process, `Task`, or `GenServer` that automatically reads from the chain and writes to `:explorer` should be in `:indexer`. This restricts automatic writes to `:indexer` and read-only mode can be achieved by not running `:indexer`. |
## Getting Started
### Requirements
| Dependency | Mac | Linux | | Dependency | Mac | Linux |
|-------------|-----|-------| |-------------|-----|-------|
@ -96,121 +94,42 @@ The [development stack page](https://github.com/poanetwork/blockscout/wiki/Devel
### Build and Run ### Build and Run
1. Clone the repository. #### Playbook Deployment
`git clone https://github.com/poanetwork/blockscout`
2. Go to the explorer subdirectory.
`cd blockscout`
3. Set up default configurations.
`cp apps/explorer/config/dev.secret.exs.example apps/explorer/config/dev.secret.exs`
`cp apps/block_scout_web/config/dev.secret.exs.example apps/block_scout_web/config/dev.secret.exs`
<br />Linux: Update the database username and password configuration in `apps/explorer/config/dev.secret.exs`
<br />Mac: Remove the `username` and `password` fields from `apps/explorer/config/dev.secret.exs`
<br />Optional: Set up default configuration for testing.
`cp apps/explorer/config/test.secret.exs.example apps/explorer/config/test.secret.exs`
Example usage: Changing the default Postgres port from localhost:15432 if [Boxen](https://github.com/boxen/boxen) is installed.
4. Install dependencies.
`mix do deps.get, local.rebar --force, deps.compile, compile`
5. Create and migrate database.
`mix ecto.create && mix ecto.migrate`
<br />_Note:_ If you have run previously, drop the previous database
`mix do ecto.drop, ecto.create, ecto.migrate`
6. Install Node.js dependencies.
`cd apps/block_scout_web/assets && npm install; cd -`
`cd apps/explorer && npm install; cd -`
7. Update your JSON RPC Variant in `apps/explorer/config/dev.exs` and `apps/indexer/config/dev.exs`. We use [Ansible](https://docs.ansible.com/ansible/latest/index.html) & [Terraform](https://www.terraform.io/intro/getting-started/install.html) to build the correct infrastructure to run BlockScout. See [https://github.com/poanetwork/blockscout-terraform](https://github.com/poanetwork/blockscout-terraform) for details and instructions.
For `variant`, enter `ganache`, `geth`, `parity`, or `rsk`
8. Update your JSON RPC Endpoint in `apps/explorer/config/dev/` and `apps/indexer/config/dev/` #### Manual Deployment
For the `variant` chosen in step 7, enter the correct information for the corresponding JSON RPC Endpoint in `parity.exs`, `geth.exs`, or `ganache.exs`
9. Enable HTTPS in development. The Phoenix server only runs with HTTPS. See [Manual BlockScout Deployment](https://forum.poa.network/t/manual-blockscout-deployment/2458) for instructions.
* `cd apps/block_scout_web`
* `mix phx.gen.cert blockscout blockscout.local; cd -`
* Add blockscout and blockscout.local to your `/etc/hosts`
```
127.0.0.1 localhost blockscout blockscout.local
255.255.255.255 broadcasthost
::1 localhost blockscout blockscout.local
```
* If using Chrome, Enable `chrome://flags/#allow-insecure-localhost`.
9. Run the Phoenix Server from the root directory of your application. #### Environment Variables
`mix phx.server`
Now you can visit [`localhost:4000`](http://localhost:4000) from your browser. Our forum contains a [full list of BlockScout environment variables](https://forum.poa.network/t/faq-blockscout-environment-variables/1814).
_Additional runtime options:_ #### Configuring EVM Chains
* Run Phoenix Server with IEx (Interactive Elixer) * **CSS:** Update the import instruction in `apps/block_scout_web/assets/css/theme/_variables.scss` to select a preset css file. This is reflected in the `production-${chain}` branch for each instance. For example, in the `production-xdai` branch, it is set to `@import "dai-variables"`.
`iex -S mix phx.server`
* Run Phoenix Server with real time indexer * **ENV:** Update the [environment variables](https://forum.poa.network/t/faq-blockscout-environment-variables/1814) to match the chain specs.
`iex -S mix phx.server`
### Automating Restarts #### Automating Restarts
By default `blockscout` does not restart if it crashes. To enable automated By default `blockscout` does not restart if it crashes. To enable automated
restarts, set the environment variable `HEART_COMMAND` to whatever you run to restarts, set the environment variable `HEART_COMMAND` to whatever command you run to start `blockscout`. Configure the heart beat timeout to change how long it waits before considering the application unresponsive. At that point, it will kill the current blockscout instance and execute the `HEART_COMMAND`. By default a crash dump is not written unless you set `ERL_CRASH_DUMP_SECONDS` to a positive or negative integer. See the [heart](http://erlang.org/doc/man/heart.html) documentation for more information.
start `blockscout`. You can configure the heart beat timeout, which will change
how long it will wait before considering the application to be unresponsive. At
that point, it will kill the current blockscout and execute `HEART_COMMAND`.
By default a crash dump is not written unless you set `ERL_CRASH_DUMP_SECONDS`
to a positive or negative integer. See the documentation for
[heart](http://erlang.org/doc/man/heart.html) for more information.
### Configuring Ethereum Classic and other EVM Chains #### CircleCI Updates
**Note: Most of these modifications will be consolidated into a single file in the future.**
1. Update the import file in `apps/block_scout_web/assets/css/theme/_variables.scss`. There are several preset css files for our supported chains which include Ethereum Classic, Ethereum Mainnet, Ropsten Testnet, Kovan Testnet, POA Core, and POA Sokol. To deploy Ethereum Classic, change the import to `ethereum_classic_variables`.
2. Update the logo file in `apps/block_scout_web/config/config.exs`. To deploy Ethereum Classic, change this file to `classic_ethereum_logo.svg`.
3. Update the `check_origin` configuration in `apps/block_scout_web/config/prod.exs`. This allows realtime events to occur on your endpoint.
4. Update the node configuration. You will need a full tracing node with WebSockets enabled. Make the changes in the following files (dev/prod):
* `apps/explorer/config/dev/parity.exs`
* `apps/explorer/config/prod/parity.exs`
* `apps/indexer/config/dev/parity.exs`
* `apps/indexer/config/prod/parity.exs`
5. Update the dropdown menu in the main navigation `apps/block_scout_web/lib/block_scout_web/templates/layout/_topnav.html.eex`
6. Update the coin in `apps/explorer/config/config.exs`. This will pull relevant information from Coinmarketcap.com.
### Umbrella Project Organization
This repository is an [umbrella project](https://elixir-lang.org/getting-started/mix-otp/dependencies-and-umbrella-projects.html). Each directory under `apps/` is a separate [Mix](https://hexdocs.pm/mix/Mix.html) project and [OTP application](https://hexdocs.pm/elixir/Application.html), but the projects can use each other as a dependency in their `mix.exs`.
Each OTP application has a restricted domain.
| Directory | OTP Application | Namespace | Purpose |
|:------------------------|:--------------------|:------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `apps/ethereum_jsonrpc` | `:ethereum_jsonrpc` | `EthereumJSONRPC` | Ethereum JSONRPC client. It is allowed to know `Explorer`'s param format, but it cannot directly depend on `:explorer` |
| `apps/explorer` | `:explorer` | `Explorer` | Storage for the indexed chain. Can read and write to the backing storage. MUST be able to boot in a read-only mode when run independently from `:indexer`, so cannot depend on `:indexer` as that would start `:indexer` indexing. |
| `apps/block_scout_web` | `:block_scout_web` | `BlockScoutWeb` | Phoenix interface to `:explorer`. The minimum interface to allow web access should go in `:block_scout_web`. Any business rules or interface not tied directly to `Phoenix` or `Plug` should go in `:explorer`. MUST be able to boot in a read-only mode when run independently from `:indexer`, so cannot depend on `:indexer` as that would start `:indexer` indexing. |
| `apps/indexer` | `:indexer` | `Indexer` | Uses `:ethereum_jsonrpc` to index chain and batch import data into `:explorer`. Any process, `Task`, or `GenServer` that automatically reads from the chain and writes to `:explorer` should be in `:indexer`. This restricts automatic writes to `:indexer` and read-only mode can be achieved by not running `:indexer`. |
### CircleCI Updates
To monitor build status, configure your local [CCMenu](http://ccmenu.org/) with the following url: [`https://circleci.com/gh/poanetwork/blockscout.cc.xml?circle-token=f8823a3d0090407c11f87028c73015a331dbf604`](https://circleci.com/gh/poanetwork/blockscout.cc.xml?circle-token=f8823a3d0090407c11f87028c73015a331dbf604) To monitor build status, configure your local [CCMenu](http://ccmenu.org/) with the following url: [`https://circleci.com/gh/poanetwork/blockscout.cc.xml?circle-token=f8823a3d0090407c11f87028c73015a331dbf604`](https://circleci.com/gh/poanetwork/blockscout.cc.xml?circle-token=f8823a3d0090407c11f87028c73015a331dbf604)
### Testing ## Testing
#### Requirements ### Requirements
* PhantomJS (for wallaby) * PhantomJS (for wallaby)
#### Running the tests ### Running the tests
1. Build the assets. 1. Build the assets.
`cd apps/block_scout_web/assets && npm run build; cd -` `cd apps/block_scout_web/assets && npm run build; cd -`
@ -237,9 +156,9 @@ To monitor build status, configure your local [CCMenu](http://ccmenu.org/) with
8. Test the JavaScript code. 8. Test the JavaScript code.
`cd apps/block_scout_web/assets && npm run test; cd -` `cd apps/block_scout_web/assets && npm run test; cd -`
##### Parity #### Parity
###### Mox ##### Mox
**This is the default setup. `mix coveralls.html --umbrella` will work on its own, but to be explicit, use the following setup**: **This is the default setup. `mix coveralls.html --umbrella` will work on its own, but to be explicit, use the following setup**:
@ -249,7 +168,7 @@ export ETHEREUM_JSONRPC_WEB_SOCKET_CASE=EthereumJSONRPC.WebSocket.Case.Mox
mix coveralls.html --umbrella --exclude no_parity mix coveralls.html --umbrella --exclude no_parity
``` ```
###### HTTP / WebSocket ##### HTTP / WebSocket
```shell ```shell
export ETHEREUM_JSONRPC_CASE=EthereumJSONRPC.Case.Parity.HTTPWebSocket export ETHEREUM_JSONRPC_CASE=EthereumJSONRPC.Case.Parity.HTTPWebSocket
@ -262,9 +181,9 @@ mix coveralls.html --umbrella --exclude no_parity
| HTTP | `http://localhost:8545` | | HTTP | `http://localhost:8545` |
| WebSocket | `ws://localhost:8546` | | WebSocket | `ws://localhost:8546` |
##### Geth #### Geth
###### Mox ##### Mox
```shell ```shell
export ETHEREUM_JSONRPC_CASE=EthereumJSONRPC.Case.Geth.Mox export ETHEREUM_JSONRPC_CASE=EthereumJSONRPC.Case.Geth.Mox
@ -272,7 +191,7 @@ export ETHEREUM_JSONRPC_WEB_SOCKET_CASE=EthereumJSONRPC.WebSocket.Case.Mox
mix coveralls.html --umbrella --exclude no_geth mix coveralls.html --umbrella --exclude no_geth
``` ```
###### HTTP / WebSocket ##### HTTP / WebSocket
```shell ```shell
export ETHEREUM_JSONRPC_CASE=EthereumJSONRPC.Case.Geth.HTTPWebSocket export ETHEREUM_JSONRPC_CASE=EthereumJSONRPC.Case.Geth.HTTPWebSocket

@ -594,6 +594,11 @@ defmodule BlockScoutWeb.Etherscan do
type: "block number", type: "block number",
definition: "A nonnegative number used to identify blocks.", definition: "A nonnegative number used to identify blocks.",
example: ~s("0x5c958") example: ~s("0x5c958")
},
index: %{
type: "log index",
definition: "A nonnegative number used to identify logs.",
example: ~s("1")
} }
} }
} }

@ -77,7 +77,8 @@ defmodule BlockScoutWeb.API.RPC.TransactionView do
%{ %{
"address" => "#{log.address_hash}", "address" => "#{log.address_hash}",
"topics" => get_topics(log), "topics" => get_topics(log),
"data" => "#{log.data}" "data" => "#{log.data}",
"index" => "#{log.index}"
} }
end end

@ -460,7 +460,8 @@ defmodule BlockScoutWeb.API.RPC.TransactionControllerTest do
%{ %{
"address" => "#{address.hash}", "address" => "#{address.hash}",
"data" => "#{log.data}", "data" => "#{log.data}",
"topics" => ["first topic", "second topic", nil, nil] "topics" => ["first topic", "second topic", nil, nil],
"index" => "#{log.index}"
} }
], ],
"next_page_params" => nil "next_page_params" => nil

@ -60,6 +60,14 @@ config :explorer, Explorer.Staking.PoolsReader,
validators_contract_address: System.get_env("POS_VALIDATORS_CONTRACT"), validators_contract_address: System.get_env("POS_VALIDATORS_CONTRACT"),
staking_contract_address: System.get_env("POS_STAKING_CONTRACT") staking_contract_address: System.get_env("POS_STAKING_CONTRACT")
if System.get_env("POS_STAKING_CONTRACT") do
config :explorer, Explorer.Staking.EpochCounter,
enabled: true,
staking_contract_address: System.get_env("POS_STAKING_CONTRACT")
else
config :explorer, Explorer.Staking.EpochCounter, enabled: false
end
if System.get_env("SUPPLY_MODULE") == "TokenBridge" do if System.get_env("SUPPLY_MODULE") == "TokenBridge" do
config :explorer, supply: Explorer.Chain.Supply.TokenBridge config :explorer, supply: Explorer.Chain.Supply.TokenBridge
end end

@ -51,7 +51,8 @@ defmodule Explorer.Application do
configure(Explorer.Market.History.Cataloger), configure(Explorer.Market.History.Cataloger),
configure(Explorer.Counters.AddressesWithBalanceCounter), configure(Explorer.Counters.AddressesWithBalanceCounter),
configure(Explorer.Counters.AverageBlockTime), configure(Explorer.Counters.AverageBlockTime),
configure(Explorer.Validator.MetadataProcessor) configure(Explorer.Validator.MetadataProcessor),
configure(Explorer.Staking.EpochCounter)
] ]
|> List.flatten() |> List.flatten()
end end

@ -10,6 +10,7 @@ defmodule Explorer.Chain do
limit: 2, limit: 2,
order_by: 2, order_by: 2,
order_by: 3, order_by: 3,
offset: 2,
preload: 2, preload: 2,
select: 2, select: 2,
subquery: 1, subquery: 1,
@ -2903,6 +2904,75 @@ defmodule Explorer.Chain do
value value
end end
@doc "Get staking pools from the DB"
@spec staking_pools(filter :: :validator | :active | :inactive, options :: PagingOptions.t()) :: [map()]
def staking_pools(filter, %PagingOptions{page_size: page_size, page_number: page_number} \\ @default_paging_options) do
off = page_size * (page_number - 1)
Address.Name
|> staking_pool_filter(filter)
|> limit(^page_size)
|> offset(^off)
|> Repo.all()
end
@doc "Get count of staking pools from the DB"
@spec staking_pools_count(filter :: :validator | :active | :inactive) :: integer
def staking_pools_count(filter) do
Address.Name
|> staking_pool_filter(filter)
|> Repo.aggregate(:count, :address_hash)
end
defp staking_pool_filter(query, :validator) do
where(
query,
[address],
fragment(
"""
(?->>'is_active')::boolean = true and
(?->>'deleted')::boolean is not true and
(?->>'is_validator')::boolean = true
""",
address.metadata,
address.metadata,
address.metadata
)
)
end
defp staking_pool_filter(query, :active) do
where(
query,
[address],
fragment(
"""
(?->>'is_active')::boolean = true and
(?->>'deleted')::boolean is not true
""",
address.metadata,
address.metadata
)
)
end
defp staking_pool_filter(query, :inactive) do
where(
query,
[address],
fragment(
"""
(?->>'is_active')::boolean = false and
(?->>'deleted')::boolean is not true
""",
address.metadata,
address.metadata
)
)
end
defp staking_pool_filter(query, _), do: query
defp with_decompiled_code_flag(query, hash) do defp with_decompiled_code_flag(query, hash) do
has_decompiled_code_query = has_decompiled_code_query =
from(decompiled_contract in DecompiledSmartContract, from(decompiled_contract in DecompiledSmartContract,

@ -41,6 +41,9 @@ defmodule Explorer.Chain.Import.Runner.StakingPools do
|> Map.put(:timestamps, timestamps) |> Map.put(:timestamps, timestamps)
multi multi
|> Multi.run(:mark_as_deleted, fn repo, _ ->
mark_as_deleted(repo, changes_list, insert_options)
end)
|> Multi.run(:insert_staking_pools, fn repo, _ -> |> Multi.run(:insert_staking_pools, fn repo, _ ->
insert(repo, changes_list, insert_options) insert(repo, changes_list, insert_options)
end) end)
@ -49,6 +52,32 @@ defmodule Explorer.Chain.Import.Runner.StakingPools do
@impl Import.Runner @impl Import.Runner
def timeout, do: @timeout def timeout, do: @timeout
defp mark_as_deleted(repo, changes_list, %{timeout: timeout}) when is_list(changes_list) do
addresses = Enum.map(changes_list, & &1.address_hash)
query =
from(
address_name in Address.Name,
where:
address_name.address_hash not in ^addresses and
fragment("(?->>'is_pool')::boolean = true", address_name.metadata),
update: [
set: [
metadata: fragment("? || '{\"deleted\": true}'::jsonb", address_name.metadata)
]
]
)
try do
{_, result} = repo.update_all(query, [], timeout: timeout)
{:ok, result}
rescue
postgrex_error in Postgrex.Error ->
{:error, %{exception: postgrex_error}}
end
end
@spec insert(Repo.t(), [map()], %{ @spec insert(Repo.t(), [map()], %{
optional(:on_conflict) => Import.Runner.on_conflict(), optional(:on_conflict) => Import.Runner.on_conflict(),
required(:timeout) => timeout, required(:timeout) => timeout,

@ -4,10 +4,11 @@ defmodule Explorer.PagingOptions do
number and index. number and index.
""" """
@type t :: %__MODULE__{key: key, page_size: page_size} @type t :: %__MODULE__{key: key, page_size: page_size, page_number: page_number}
@typep key :: any() @typep key :: any()
@typep page_size :: non_neg_integer() @typep page_size :: non_neg_integer()
@typep page_number :: pos_integer()
defstruct [:key, :page_size] defstruct [:key, :page_size, page_number: 1]
end end

@ -0,0 +1,124 @@
defmodule Explorer.Staking.EpochCounter do
@moduledoc """
Fetches current staking epoch number and the epoch end block number.
It subscribes to handle new blocks and conclude whether the epoch is over.
"""
use GenServer
alias Explorer.Chain.Events.Subscriber
alias Explorer.SmartContract.Reader
@table_name __MODULE__
@epoch_key "epoch_num"
@epoch_end_key "epoch_end_block"
@doc "Current staking epoch number"
def epoch_number do
if :ets.info(@table_name) != :undefined do
case :ets.lookup(@table_name, @epoch_key) do
[{_, epoch_num}] ->
epoch_num
_ ->
0
end
end
end
@doc "Block number on which will start new epoch"
def epoch_end_block do
if :ets.info(@table_name) != :undefined do
case :ets.lookup(@table_name, @epoch_end_key) do
[{_, epoch_end}] ->
epoch_end
_ ->
0
end
end
end
def start_link([]) do
GenServer.start_link(__MODULE__, [], name: __MODULE__)
end
def init([]) do
:ets.new(@table_name, [
:set,
:named_table,
:public,
write_concurrency: true
])
Subscriber.to(:blocks, :realtime)
{:ok, [], {:continue, :epoch_info}}
end
def handle_continue(:epoch_info, state) do
fetch_epoch_info()
{:noreply, state}
end
@doc "Handles new blocks and decides to fetch new epoch info"
def handle_info({:chain_event, :blocks, :realtime, blocks}, state) do
new_block_number =
blocks
|> Enum.map(&Map.get(&1, :number, 0))
|> Enum.max(fn -> 0 end)
case :ets.lookup(@table_name, @epoch_end_key) do
[] ->
fetch_epoch_info()
[{_, epoch_end_block}] when epoch_end_block < new_block_number ->
fetch_epoch_info()
_ ->
:ok
end
{:noreply, state}
end
defp fetch_epoch_info do
with data <- get_epoch_info(),
{:ok, [epoch_num]} <- data["stakingEpoch"],
{:ok, [epoch_end_block]} <- data["stakingEpochEndBlock"] do
:ets.insert(@table_name, {@epoch_key, epoch_num})
:ets.insert(@table_name, {@epoch_end_key, epoch_end_block})
end
end
defp get_epoch_info do
contract_abi = abi("staking.json")
functions = ["stakingEpoch", "stakingEpochEndBlock"]
functions
|> Enum.map(fn function ->
%{
contract_address: staking_address(),
function_name: function,
args: []
}
end)
|> Reader.query_contracts(contract_abi)
|> Enum.zip(functions)
|> Enum.into(%{}, fn {response, function} ->
{function, response}
end)
end
defp staking_address do
Application.get_env(:explorer, __MODULE__, [])[:staking_contract_address]
end
# sobelow_skip ["Traversal"]
defp abi(file_name) do
:explorer
|> Application.app_dir("priv/contracts_abi/pos/#{file_name}")
|> File.read!()
|> Jason.decode!()
end
end

@ -29,6 +29,7 @@ defmodule Explorer.Staking.PoolsReader do
{:ok, [delegator_addresses]} <- data["poolDelegators"], {:ok, [delegator_addresses]} <- data["poolDelegators"],
delegators_count = Enum.count(delegator_addresses), delegators_count = Enum.count(delegator_addresses),
{:ok, [staked_amount]} <- data["stakeAmountTotalMinusOrderedWithdraw"], {:ok, [staked_amount]} <- data["stakeAmountTotalMinusOrderedWithdraw"],
{:ok, [self_staked_amount]} <- data["stakeAmountMinusOrderedWithdraw"],
{:ok, [is_validator]} <- data["isValidator"], {:ok, [is_validator]} <- data["isValidator"],
{:ok, [was_validator_count]} <- data["validatorCounter"], {:ok, [was_validator_count]} <- data["validatorCounter"],
{:ok, [is_banned]} <- data["isValidatorBanned"], {:ok, [is_banned]} <- data["isValidatorBanned"],
@ -42,6 +43,7 @@ defmodule Explorer.Staking.PoolsReader do
is_active: is_active, is_active: is_active,
delegators_count: delegators_count, delegators_count: delegators_count,
staked_amount: staked_amount, staked_amount: staked_amount,
self_staked_amount: self_staked_amount,
is_validator: is_validator, is_validator: is_validator,
was_validator_count: was_validator_count, was_validator_count: was_validator_count,
is_banned: is_banned, is_banned: is_banned,
@ -77,14 +79,15 @@ defmodule Explorer.Staking.PoolsReader do
contract_abi = abi("staking.json") ++ abi("validators.json") contract_abi = abi("staking.json") ++ abi("validators.json")
methods = [ methods = [
{:staking, "isPoolActive", staking_address}, {:staking, "isPoolActive", [staking_address]},
{:staking, "poolDelegators", staking_address}, {:staking, "poolDelegators", [staking_address]},
{:staking, "stakeAmountTotalMinusOrderedWithdraw", staking_address}, {:staking, "stakeAmountTotalMinusOrderedWithdraw", [staking_address]},
{:validators, "isValidator", mining_address}, {:staking, "stakeAmountMinusOrderedWithdraw", [staking_address, staking_address]},
{:validators, "validatorCounter", mining_address}, {:validators, "isValidator", [mining_address]},
{:validators, "isValidatorBanned", mining_address}, {:validators, "validatorCounter", [mining_address]},
{:validators, "bannedUntil", mining_address}, {:validators, "isValidatorBanned", [mining_address]},
{:validators, "banCounter", mining_address} {:validators, "bannedUntil", [mining_address]},
{:validators, "banCounter", [mining_address]}
] ]
methods methods
@ -96,11 +99,11 @@ defmodule Explorer.Staking.PoolsReader do
end) end)
end end
defp format_request({contract_name, function_name, param}) do defp format_request({contract_name, function_name, params}) do
%{ %{
contract_address: contract(contract_name), contract_address: contract(contract_name),
function_name: function_name, function_name: function_name,
args: [param] args: params
} }
end end

File diff suppressed because it is too large Load Diff

@ -3948,4 +3948,59 @@ defmodule Explorer.ChainTest do
refute Chain.contract_address?(to_string(hash), 1, json_rpc_named_arguments) refute Chain.contract_address?(to_string(hash), 1, json_rpc_named_arguments)
end end
end end
describe "staking_pools/3" do
test "validators staking pools" do
inserted_validator = insert(:address_name, primary: true, metadata: %{is_active: true, is_validator: true})
insert(:address_name, primary: true, metadata: %{is_active: true, is_validator: false})
options = %PagingOptions{page_size: 20, page_number: 1}
assert [gotten_validator] = Chain.staking_pools(:validator, options)
assert inserted_validator.address_hash == gotten_validator.address_hash
end
test "active staking pools" do
inserted_validator = insert(:address_name, primary: true, metadata: %{is_active: true})
insert(:address_name, primary: true, metadata: %{is_active: false})
options = %PagingOptions{page_size: 20, page_number: 1}
assert [gotten_validator] = Chain.staking_pools(:active, options)
assert inserted_validator.address_hash == gotten_validator.address_hash
end
test "inactive staking pools" do
insert(:address_name, primary: true, metadata: %{is_active: true})
inserted_validator = insert(:address_name, primary: true, metadata: %{is_active: false})
options = %PagingOptions{page_size: 20, page_number: 1}
assert [gotten_validator] = Chain.staking_pools(:inactive, options)
assert inserted_validator.address_hash == gotten_validator.address_hash
end
end
describe "staking_pools_count/1" do
test "validators staking pools" do
insert(:address_name, primary: true, metadata: %{is_active: true, is_validator: true})
insert(:address_name, primary: true, metadata: %{is_active: true, is_validator: false})
assert Chain.staking_pools_count(:validator) == 1
end
test "active staking pools" do
insert(:address_name, primary: true, metadata: %{is_active: true})
insert(:address_name, primary: true, metadata: %{is_active: false})
assert Chain.staking_pools_count(:active) == 1
end
test "inactive staking pools" do
insert(:address_name, primary: true, metadata: %{is_active: true})
insert(:address_name, primary: true, metadata: %{is_active: false})
assert Chain.staking_pools_count(:inactive) == 1
end
end
end end

@ -0,0 +1,97 @@
defmodule Explorer.Staking.EpochCounterTest do
use ExUnit.Case, async: false
import Mox
alias Explorer.Staking.EpochCounter
alias Explorer.Chain.Events.Publisher
setup :verify_on_exit!
setup :set_mox_global
test "when disabled, it returns nil" do
assert EpochCounter.epoch_number() == nil
assert EpochCounter.epoch_end_block() == nil
end
test "fetch epoch data" do
set_mox(10, 880)
Application.put_env(:explorer, EpochCounter, enabled: true)
start_supervised!(EpochCounter)
Process.sleep(1_000)
assert EpochCounter.epoch_number() == 10
assert EpochCounter.epoch_end_block() == 880
end
test "fetch new epoch data" do
set_mox(10, 880)
Application.put_env(:explorer, EpochCounter, enabled: true)
start_supervised!(EpochCounter)
Process.sleep(1_000)
assert EpochCounter.epoch_number() == 10
assert EpochCounter.epoch_end_block() == 880
event_type = :blocks
broadcast_type = :realtime
event_data = [%Explorer.Chain.Block{number: 881}]
set_mox(11, 960)
Publisher.broadcast([{event_type, event_data}], broadcast_type)
Process.sleep(1_000)
assert EpochCounter.epoch_number() == 11
assert EpochCounter.epoch_end_block() == 960
end
defp set_mox(epoch_num, end_block_num) do
expect(
EthereumJSONRPC.Mox,
:json_rpc,
fn [
%{
id: 0,
jsonrpc: "2.0",
method: "eth_call",
params: _
},
%{
id: 1,
jsonrpc: "2.0",
method: "eth_call",
params: _
}
],
_options ->
{:ok,
[
%{
id: 0,
jsonrpc: "2.0",
result: encode_num(epoch_num)
},
%{
id: 1,
jsonrpc: "2.0",
result: encode_num(end_block_num)
}
]}
end
)
end
defp encode_num(num) do
selector = %ABI.FunctionSelector{function: nil, types: [uint: 32]}
encoded_num =
[num]
|> ABI.TypeEncoder.encode(selector)
|> Base.encode16(case: :lower)
"0x" <> encoded_num
end
end

@ -1,6 +1,5 @@
defmodule Explorer.Token.PoolsReaderTest do defmodule Explorer.Token.PoolsReaderTest do
use EthereumJSONRPC.Case use EthereumJSONRPC.Case
use Explorer.DataCase
alias Explorer.Staking.PoolsReader alias Explorer.Staking.PoolsReader
@ -44,6 +43,7 @@ defmodule Explorer.Token.PoolsReaderTest do
mining_address: mining_address:
<<187, 202, 168, 212, 130, 137, 187, 31, 252, 249, 128, 141, 154, 164, 177, 210, 21, 5, 76, 120>>, <<187, 202, 168, 212, 130, 137, 187, 31, 252, 249, 128, 141, 154, 164, 177, 210, 21, 5, 76, 120>>,
staked_amount: 0, staked_amount: 0,
self_staked_amount: 0,
staking_address: <<11, 47, 94, 47, 60, 189, 134, 78, 170, 44, 100, 46, 55, 105, 193, 88, 35, 97, 202, 246>>, staking_address: <<11, 47, 94, 47, 60, 189, 134, 78, 170, 44, 100, 46, 55, 105, 193, 88, 35, 97, 202, 246>>,
was_banned_count: 0, was_banned_count: 0,
was_validator_count: 2 was_validator_count: 2
@ -162,6 +162,25 @@ defmodule Explorer.Token.PoolsReaderTest do
result: "0x0000000000000000000000000000000000000000000000000000000000000000" result: "0x0000000000000000000000000000000000000000000000000000000000000000"
} }
# stakeAmountMinusOrderedWithdraw
%{
id: id,
jsonrpc: "2.0",
method: "eth_call",
params: [
%{
data:
"0x58daab6a0000000000000000000000000b2f5e2f3cbd864eaa2c642e3769c1582361caf60000000000000000000000000b2f5e2f3cbd864eaa2c642e3769c1582361caf6",
to: _
},
"latest"
]
} ->
%{
id: id,
result: "0x0000000000000000000000000000000000000000000000000000000000000000"
}
# isValidator # isValidator
%{ %{
id: id, id: id,

@ -125,6 +125,7 @@ defmodule Indexer.Fetcher.StakingPools do
pool pool
|> Map.delete(:staking_address) |> Map.delete(:staking_address)
|> Map.put(:mining_address, mining_address) |> Map.put(:mining_address, mining_address)
|> Map.put(:is_pool, true)
%{ %{
name: "anonymous", name: "anonymous",

@ -129,6 +129,25 @@ defmodule Indexer.Fetcher.StakingPoolsTest do
result: "0x0000000000000000000000000000000000000000000000000000000000000000" result: "0x0000000000000000000000000000000000000000000000000000000000000000"
} }
# stakeAmountMinusOrderedWithdraw
%{
id: id,
jsonrpc: "2.0",
method: "eth_call",
params: [
%{
data:
"0x58daab6a0000000000000000000000000b2f5e2f3cbd864eaa2c642e3769c1582361caf60000000000000000000000000b2f5e2f3cbd864eaa2c642e3769c1582361caf6",
to: _
},
"latest"
]
} ->
%{
id: id,
result: "0x0000000000000000000000000000000000000000000000000000000000000000"
}
# isValidator # isValidator
%{ %{
id: id, id: id,

Loading…
Cancel
Save