alternative malloc implementation - jemalloc (#4126)

* add malloc arena params to besu startup
* add jemalloc preload to besu startup
* add jemalloc lib download to docker jvm images
* use 5.* for jemalloc-dev version
* add jemalloc to circleci containers for acceptance tests

Signed-off-by: garyschulte <garyschulte@gmail.com>

.circleci/config.yml

@@ -66,7 +66,7 @@ commands:
           name: Install Packages - LibSodium, nssdb
           command: |
             sudo apt-get update
-            sudo apt-get install -y libsodium23 libsodium-dev apt-transport-https haveged libnss3-tools
+            sudo apt-get install -y libsodium23 libsodium-dev libjemalloc-dev apt-transport-https haveged libnss3-tools
             sudo service haveged restart
       - restore_gradle_cache
   restore_gradle_cache:

CHANGELOG.md

@@ -7,6 +7,7 @@
 - Backward sync exception improvements [#4092](https://github.com/hyperledger/besu/pull/4092)
 - Remove block header checks during backward sync, since they will be always performed during block import phase [#4098](https://github.com/hyperledger/besu/pull/4098)
 - Optimize the backward sync retry strategy [#4095](https://github.com/hyperledger/besu/pull/4095)
+- Add support for jemalloc library to better handle rocksdb memory consumption [#4126](https://github.com/hyperledger/besu/pull/4126)
 
 ### Bug Fixes
 - Return the correct latest valid hash in case of bad block when calling engine methods [#4056](https://github.com/hyperledger/besu/pull/4056)

besu/src/main/scripts/unixStartScript.txt

@@ -182,4 +182,8 @@ APP_ARGS=`save "\$@"`
 # Collect all arguments for the java command, following the shell quoting and substitution rules
 eval set -- \$DEFAULT_JVM_OPTS \$JAVA_OPTS \$${optsEnvironmentVar} <% if ( appNameSystemProperty ) { %>"\"-D${appNameSystemProperty}=\$APP_BASE_NAME\"" <% } %>-classpath "\"\$CLASSPATH\"" <% if ( mainClassName.startsWith('--module ') ) { %>--module-path "\"\$MODULE_PATH\"" <% } %>${mainClassName} "\$APP_ARGS"
 
+# limit malloc to 2 arenas, and use jemalloc if available
+export MALLOC_ARENA_MAX=2
+export LD_PRELOAD=libjemalloc.so
+
 exec "\$JAVACMD" "\$@"

docker/openjdk-11-debug/Dockerfile

@@ -3,7 +3,7 @@ FROM ubuntu:20.04
 
 ARG VERSION="dev"
 RUN apt-get update  && \
- apt-get install --no-install-recommends -q --assume-yes curl=7* wget=1.20* jq=1.6* net-tools=1.60* openjdk-11-jdk-headless=11* && \
+ apt-get install --no-install-recommends -q --assume-yes curl=7* wget=1.20* jq=1.6* net-tools=1.60* openjdk-11-jdk-headless=11* libjemalloc-dev=5.* && \
  apt-get clean  && \
  rm -rf /var/lib/apt/lists/*  && \
  adduser --disabled-password --gecos "" --home /opt/besu besu && \

docker/openjdk-11/Dockerfile

@@ -3,7 +3,7 @@ FROM ubuntu:20.04
 ARG VERSION="dev"
 
 RUN apt-get update && \
- apt-get install --no-install-recommends -q --assume-yes openjdk-11-jre-headless=11* && \
+ apt-get install --no-install-recommends -q --assume-yes openjdk-11-jre-headless=11* libjemalloc-dev=5.* && \
  apt-get clean  && \
  rm -rf /var/lib/apt/lists/*  && \
  adduser --disabled-password --gecos "" --home /opt/besu besu && \

docker/openjdk-latest/Dockerfile

@@ -3,7 +3,7 @@ FROM ubuntu:20.04
 ARG VERSION="dev"
 
 RUN apt-get update && \
- apt-get install --no-install-recommends -q --assume-yes openjdk-17-jre-headless=17* && \
+ apt-get install --no-install-recommends -q --assume-yes openjdk-17-jre-headless=17* libjemalloc-dev=5.* && \
  apt-get clean  && \
  rm -rf /var/lib/apt/lists/*  && \
  adduser --disabled-password --gecos "" --home /opt/besu besu && \