17 Commits (b2ec14fa6c159c7124c485bcca7d9f057b690859)

Author	SHA1	Message	Date
pasqu4le	1f5bf98a06	Add script to validate internal_transactions constraint for large DBs ... Problem: executing the existing sql script to validate internal_transactions constraint is too slow (or even gets stuck) for large DBs and its progress cannot be checked. Solution: add an alternative implementation of the same script that can be monitored from outside and has better performance.	5 years ago
goodsoft	f5268fc450	Address review comments	6 years ago
goodsoft	fe25bf7b44	Add a migration to mark all invalid blocks as non-consensus (#1644)	6 years ago
Andrew Cravenho	4a5bf27060	Only add confirmed dropped/replaced txs to temp table	6 years ago
Andrew Cravenho	c0ac367960	Fix block_hash is null statement	6 years ago
Andrew Cravenho	f71228884c	Update migration script for dropped/replaced transactions	6 years ago
Ayrat Badykov	d3c7c8636f	add replaced_transaction update script	6 years ago
Luke Imhoff	a89fe692a7	Make insert_address_current_token_balances_in_batches only insert new ... The previous version of `insert_address_current_token_balances_in_batches` could only be run once because it assumed that `address_current_token_balances` rows did not exist. I could have added an `ON CONFLICT` to to allow `insert_address_current`insert_address_current` to be rerun, but that would have attempted to rebuild the whole table, which would be very costly as the the complex join and aggregate calculations would need to be done for all rows even if they eventually conflict and we do `ON CONFLICT DO NOTHING`. Instead, `insert_address_current_token_balances_in_batches` searched for `address_token_balances` with `(address_hash, token_contract_address_hash)` pairs that should have an `address_current_token_balance` because the `address_token_balances` `value` is non-`NULL`. From this set of `address_token_balances_requiring_current`, the set of `address_current_token_balances` is removed, leaving only those `address_token_balances_without_required_current`, which are those affected by #1345. The rest of the script is rewritten to follow the convention from the new scripts like `update_new_tokens_holder_count_in_batches`, which include more logging with timing informations and estimates.	6 years ago
Luke Imhoff	28695f3d75	Data migration to populate tokens holder_count ... `update_new_tokens_holder_count_in_batches` will `UPDATE` any `tokens` where `holders_count` is `NULL` will have it set using the same query as `Explorer.Chain.count_token_holders_from_token_hash/1`, so after `holders_count` is filled in, it will just be a faster version of `Explorer.Chain.count_token_holders_from_token_hash/1`. `holder_counts` is updated with deltas by `Explorer.Chain.Import.Runner.{Block, Address.CurrentTokenBalance}` as `address_current_token_balances` is changed in a way that would change `Explorer.Chain.count_token_holders_from_token_hash/1` without the need to run the expensive `COUNT` query again.	6 years ago
Luke Imhoff	8a1c526cf8	Optimize update_address_current_token_balances_in_batches ... Fixes #1335 1. Calculate only the `address_current_token_balances` that need to be updated in the temporary table instead of all `address_current_token_balances`. 2. Add lot of `RAISE NOTICE` to print 1. Start Times 2. End Times 3. Elapsed Times 4. Estimate Time of Completion 3. Only update address_current_token_balances if the block number is less than the temporary tables value, in case other transactions occurred to fix the value while the script is running.	6 years ago
Luke Imhoff	99e6947d62	Update `address_current_token_balances` that are out-of-date ... Fixes #1317 Some bug in the past causes `address_current_token_balances` to not always be up-to-date. The verifier below show the number of incorrect `address_current_token_balances` not growing, so the bug itself is fixed, but the data is still wrong and need to be updated. Run `apps/explorer/priv/repo/migrations/scripts/update_address_current_token_balances_in_batches.sql` to fix any `address_current_token_balances` that have the incorrect `block_number` or `value`. Run the following to verify everything is fixed: ```sql SELECT COUNT(*) FROM (SELECT 1 FROM address_token_balances INNER JOIN address_current_token_balances ON address_current_token_balances.address_hash = address_token_balances.address_hash AND address_current_token_balances.token_contract_address_hash = address_token_balances.token_contract_address_hash GROUP BY address_token_balances.address_hash, address_token_balances.token_contract_address_hash, address_current_token_balances.block_number HAVING MAX(address_token_balances.block_number) != address_current_token_balances.block_number) AS address_incorrect_token_balances; ``` Which should return `0`. after the script is run. WARNING: the check SQL above will take 15-20 minutes on Eth Mainnet.	6 years ago
Luke Imhoff	6ec9afb4ba	Gradual migration support for call_has_call_type, call_has_input, and create_has_init ... Fixes #1112	6 years ago
Luke Imhoff	39b6eccfda	Separate schema and data migration for EIP-6 ... Fixes #1109 Use `NOT VALID` on constraint and don't put data updates in schema migration. Instead, migrate data using script based on @amandasposito's scripts.	6 years ago
Amanda Sposito	40a9da3022	Add script to migrate nonce to addresses ... This script copy nonce from the transactions' table to addresses' table in batches.	6 years ago
Felipe Renan	2ad1042de2	Add script to migrate block_number to token transfers ... This script copy block_number from the transactions' table to token transfers' table in batches.	6 years ago
Amanda Sposito	9843f2bba6	Add migration scripts that we've been using to migrate data	6 years ago