Merge pull request #4542 from blockscout/vb-tt-transform-tokens-dedup

Indexer performance update: Deduplicate tokens in the indexer token transfers transformer

CHANGELOG.md

@@ -8,6 +8,7 @@
 - [#4452](https://github.com/blockscout/blockscout/pull/4452) - Add names for smart-conrtact's function response
 
 ### Fixes
+- [#4542](https://github.com/blockscout/blockscout/pull/4542) - Indexer performance update: Deduplicate tokens in the indexer token transfers transformer
 - [#4535](https://github.com/blockscout/blockscout/pull/4535) - Indexer performance update:: Eliminate multiple updates of the same token while parsing mint/burn token transfers batch
 - [#4527](https://github.com/blockscout/blockscout/pull/4527) - Indexer performance update: refactor coin balance daily fetcher
 - [#4525](https://github.com/blockscout/blockscout/pull/4525) - Uncataloged token transfers query performance improvement

apps/indexer/lib/indexer/transform/token_transfers.ex

@@ -35,7 +35,14 @@ defmodule Indexer.Transform.TokenTransfers do
     |> Enum.dedup()
     |> Enum.each(&update_token/1)
 
-    token_transfers_from_logs
+    tokens_dedup = token_transfers_from_logs.tokens |> Enum.dedup()
+
+    token_transfers_from_logs_dedup = %{
+      tokens: tokens_dedup,
+      token_transfers: token_transfers_from_logs.token_transfers
+    }
+
+    token_transfers_from_logs_dedup
   end
 
   defp do_parse(log, %{tokens: tokens, token_transfers: token_transfers} = acc) do