besu/tasks at e7ccb330610dc3b46ebd5cc79407f1527ce2250b - besu

An enterprise-grade Java-based, Apache 2.0 licensed Ethereum client https://wiki.hyperledger.org/display/besu

History

Jiri Peinlich 97a3494085 When downloading the world state, persist children before parents (#3202 ) * Fast sync should traverse the world state depth first 1. The pending requests queue in the world state downloader is now a different data structure. We now use a list of priority queues. 2. The node requests now know about the parent request that was responsible for spawning them. 3. When a node has data available and is asked to persist before its children are persisted, the node will not do anything. Instead, it will wait for all its children to persist first and will persist together with the last child. Storing children before parents gives us the following benefits: * We don't need to store pending requests on disk any more. * When restarting download from a new pivot, we do not need to walk and check the whole tree any more. And the following drawbacks: * We now have pending nodes in memory for which we already downloaded data, but we do not store them in the database yet. Overall expectations on performance: We still need to download every single state node at least once, so there is no improvement there. We will save a significant amount of time in case we change pivots. And we save lots of read/writes on filesystem because tasks are not needed to be written to disk any more. We want to avoid having too many pending unsaved nodes in memory, not to run out of it. If we were always handling only one request to our peers at the same time, we would not need to be worried, and we would just use a simple depth first search. Because we batch our requests, we might produce too many pending unprocessed nodes in memory if we are not careful about the order of processing requests. That is where the priority on node request comes from. We want to always process nodes lower in the tree before nodes higher in the tree, and preferably we want to first process children from the same parent so that we can save the current unsaved parent as soon as possible. At the moment, I still left in the code several artefacts that I use for debugging the behaviour. I am planning to get rid of most of these counters, feel free to point them out in the review. There is for instance a weird counter in the NodeDataRequest class that I am using to monitor the total amount of unsaved nodes. In case pending unsaved node count rises too high, there is a warning printed into the logs. At the moment of writing, I would expect the counter to stay below 10 000 generally and not rise above 20 000 nodes. If you saw the number rise to for instance 100 000 that would signify a bug. Similarly, because of the order of processing of the nodes, we do not need to store huge number of requests on the disk any more and the whole list fits comfortably into the memory. Without batching, we would not have more than a thousand requests around waiting. Because of the batching, we can see the number of requests occasionally rises all the way up to 300 000, but usually should be under 200 000. Note that at any time there should not be more pending unsaved blocks than pending requests. Such a situation would be a bug to be reported. Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com> * Addressing review comments Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com> * Fixed failing test Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com> * Improving test coverage Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com> * Addressing review comments Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>	3 years ago
..
src	When downloading the world state, persist children before parents (#3202 )	3 years ago
build.gradle	Upgrade to Apache Tuweni 2.0 (#2376 )	3 years ago

When downloading the world state, persist children before parents (#3202 )

* Fast sync should traverse the world state depth first

1. The pending requests queue in the world state downloader is now a different data structure. We now use a list of priority queues.
2. The node requests now know about the parent request that was responsible for spawning them.
3. When a node has data available and is asked to persist before its children are persisted, the node will not do anything. Instead, it will wait for all its children to persist first and will persist together with the last child.

Storing children before parents gives us the following benefits:
* We don't need to store pending requests on disk any more.
* When restarting download from a new pivot, we do not need to walk and check the whole tree any more.

And the following drawbacks:
* We now have pending nodes in memory for which we already downloaded data, but we do not store them in the database yet.

Overall expectations on performance:
We still need to download every single state node at least once, so there is no improvement there. We will save a significant amount of time in case we change pivots. And we save lots of read/writes on filesystem because tasks are not needed to be written to disk any more.

We want to avoid having too many pending unsaved nodes in memory, not to run out of it. If we were always handling only one request to our peers at the same time, we would not need to be worried, and we would just use a simple depth first search. Because we batch our requests, we might produce too many pending unprocessed nodes in memory if we are not careful about the order of processing requests. That is where the priority on node request comes from. We want to always process nodes lower in the tree before nodes higher in the tree, and preferably we want to first process children from the same parent so that we can save the current unsaved parent as soon as possible.

At the moment, I still left in the code several artefacts that I use for debugging the behaviour. I am planning to get rid of most of these counters, feel free to point them out in the review. There is for instance a weird counter in the NodeDataRequest class that I am using to monitor the total amount of unsaved nodes. In case pending unsaved node count rises too high, there is a warning printed into the logs. At the moment of writing, I would expect the counter to stay below 10 000 generally and not rise above 20 000 nodes. If you saw the number rise to for instance 100 000 that would signify a bug.

Similarly, because of the order of processing of the nodes, we do not need to store huge number of requests on the disk any more and the whole list fits comfortably into the memory. Without batching, we would not have more than a thousand requests around waiting. Because of the batching, we can see the number of requests occasionally rises all the way up to 300 000, but usually should be under 200 000.

Note that at any time there should not be more pending unsaved blocks than pending requests. Such a situation would be a bug to be reported.