chore: merge `main` into `dev` (#4415)

* Mainnet Release Candidate 2023.1.2 (#4376)

* remove default timeouts

* store the evm call timeout in rosetta object

* [cmd] actually apply ToRPCServerConfig

* Removed unused method.

* Rotate external leaders on non-beacon chains.

* Fix nil panic.

* Bump github.com/aws/aws-sdk-go from 1.33.0 to 1.34.0

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.33.0 to 1.34.0.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/v1.34.0/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.33.0...v1.34.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump github.com/ipld/go-ipld-prime from 0.9.0 to 0.19.0

Bumps [github.com/ipld/go-ipld-prime](https://github.com/ipld/go-ipld-prime) from 0.9.0 to 0.19.0.
- [Release notes](https://github.com/ipld/go-ipld-prime/releases)
- [Changelog](https://github.com/ipld/go-ipld-prime/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ipld/go-ipld-prime/compare/v0.9.0...v0.19.0)

---
updated-dependencies:
- dependency-name: github.com/ipld/go-ipld-prime
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump golang.org/x/net from 0.3.0 to 0.7.0

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.3.0 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.3.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* Small fixes.

* in progress.

* in progress.

* in progress.

* consensus check is forked

* update master

* fix leader

* check leader for N blocks

* fix

* fix

* Cleanup and fix update pub keys.

* Rotate leader.

* fix

fix

fix

fix

fix

* Cleaned.

* Cache for `GetLeaderPubKeyFromCoinbase`, removed `NthNextHmyExt`.

* activate epoch

* comment activation

* 295 epoch

* Fix failed tests.

* Fixed code review.

* Fix review "--port flag".

* Fix review comments.

* Returned locks in rotateLeader.

* Rebased onto dev.

* Commented golangci.

* staged stream sync v1.0

* fix protocol tests

* fix spell

* remove unused struct

* fix rosetta test

* add comments and refactor verify sig

* add comments, remove extra function

* add comment

* refactor errors, rename metrics

* refactor p2p host creation

* fix initsync and host creation

* fix short range hash chain

* fix beacon node detection for p2p protocol

* refactor stream peer cooldown and fix protocol beacon node field

* refactor p2p host and routing

* fix p2p discovery test issue

* add MaxAdvertiseWaitTime to handle advertisements interval and address stream connection issue

* terminal print the peer id and proto id

* fix boot complete message when node is shut down

* add new config option ( ForceReachabilityPublic ) to fix local-net consensus issue

* fix self query issue

* fix test NewDNSSyncingPeerProvider

* [testnet] disable leader rotation

* fix discovery issue for legacy sync

* add watermark low/high options for p2p connection manager

* add test for new conn manager flags

* fix dedent

* add comment to inform about p2p connection manager options

* fix max height issue

* add a separate log for get max height error

* fix log

* feat: triesInMemory flag

* fix: panic if TriesInMemory is 1 to 2

* in progress.

* consensus check is forked

* fix

* Cleanup and fix update pub keys.

* fix

fix

fix

fix

fix

* activate epoch

* EpochTBD for leader rotation epoch.

* 295 epoch

* Decider no longer requires public keys as a dependency. (#4289)

* Consensus doesn't require anymore `Node` as a circular dependency.

* Proper blockchain initialization.

* Rwlock consensus.

* Removed channels.

* Removed view change locks.

* Removed timers locks.

* Removed fbft locks.

* Removed multiSigMutex locks.

* Removed leader locks.

* Removed additional locks and isViewChange.

* Added locks detected by race.

* Added locks detected by race.

* Locks for start.

* Removed additional logs.

* Removed additional locks.

* Removed additional locks.

* Make func private.

* Make VerifyBlock private.

* Make IsLeader private.

* Make ParseFBFTMessage private.

* Fix remove locks.

* Added additional locks.

* Added additional locks.

* Added readSignatureBitmapPayload locks.

* Added HandleMessageUpdate locks.

* Added LastMile locks.

* Locks for IsValidatorInCommittee.

* Fixed locks.

* Fixed tests.

* Fixed tests.

* Fixed lock.

* Rebased over leader rotation.

* Fix formatting.

* Rebased onto dev.

* in progress.

* consensus check is forked

* update master

* fix leader

* check leader for N blocks

* fix

* fix

* Cleanup and fix update pub keys.

* Rotate leader.

* fix

fix

fix

fix

fix

* Cleaned.

* Cache for `GetLeaderPubKeyFromCoinbase`, removed `NthNextHmyExt`.

* comment activation

* 295 epoch

* Fix failed tests.

* Fixed code review.

* Fix review comments.

* Merged leader rotation.

* Rebased on dev.

* Rebased on dev.

* Fix usage of private methods.

* Fix usage of private methods.

* Fix usage of private methods.

* Removed deadcode, LockedFBFTPhase.

* Fix review comment.

* Fix review comment.

* Go mod tidy.

* Set to EpochTBD.

* Fix tests.

* [core] fix state handling of self destruct

If a contract self destructs to self and then receives funds within the
same transaction, it is possible for its stale state to be saved. This
change removes that possibility by checking for deleted state objects
before returning them.

* Fixed race error.

* rpc: add configurable http and `eth_call` timeout

* remove default timeouts

* store the evm call timeout in rosetta object

* [cmd] actually apply ToRPCServerConfig

* Removed unused method.

* Rotate external leaders on non-beacon chains.

* Fix nil panic.

* in progress.

* in progress.

* in progress.

* consensus check is forked

* update master

* fix leader

* check leader for N blocks

* fix

* fix

* Cleanup and fix update pub keys.

* Rotate leader.

* fix

fix

fix

fix

fix

* Cleaned.

* Cache for `GetLeaderPubKeyFromCoinbase`, removed `NthNextHmyExt`.

* Fixed code review.

* Fix review comments.

* Returned locks in rotateLeader.

* Rebased onto dev.

* staged stream sync v1.0

* refactor errors, rename metrics

* fix p2p discovery test issue

* add watermark low/high options for p2p connection manager

* fix dedent

* in progress.

* consensus check is forked

* fix

* Cleanup and fix update pub keys.

* fix

fix

fix

fix

fix

* activate epoch

* EpochTBD for leader rotation epoch.

* 295 epoch

* Decider no longer requires public keys as a dependency. (#4289)

* Consensus doesn't require anymore `Node` as a circular dependency.

* Proper blockchain initialization.

* Rwlock consensus.

* Removed channels.

* Removed view change locks.

* Removed multiSigMutex locks.

* Removed leader locks.

* Removed additional locks and isViewChange.

* Added locks detected by race.

* Added locks detected by race.

* Locks for start.

* Removed additional locks.

* Removed additional locks.

* Make func private.

* Make VerifyBlock private.

* Make IsLeader private.

* Make ParseFBFTMessage private.

* Fix remove locks.

* Added additional locks.

* Added additional locks.

* Added readSignatureBitmapPayload locks.

* Added HandleMessageUpdate locks.

* Added LastMile locks.

* Locks for IsValidatorInCommittee.

* Fixed locks.

* Fixed tests.

* Fixed lock.

* Rebased over leader rotation.

* in progress.

* consensus check is forked

* update master

* fix leader

* check leader for N blocks

* fix

* fix

* Cleanup and fix update pub keys.

* Rotate leader.

* fix

fix

fix

fix

fix

* Cleaned.

* Cache for `GetLeaderPubKeyFromCoinbase`, removed `NthNextHmyExt`.

* Fix failed tests.

* Fixed code review.

* Fix review comments.

* Merged leader rotation.

* Rebased on dev.

* Rebased on dev.

* Fix usage of private methods.

* Fix usage of private methods.

* Fix usage of private methods.

* Removed deadcode, LockedFBFTPhase.

* Fix review comment.

* Go mod tidy.

* remove default timeouts

* Rotate external leaders on non-beacon chains.

* Fix nil panic.

* Fixes.

* Update singleton.go

* evm: don't return extcode for validators

Due to technical debt, validator information is stored in the code field
of the address. The code field can be accessed in Solidity for an
arbitrary address using `extcodesize`, `extcodehash`, and `extcodecopy`
or helper commands (such as `address.code.Length`). The presence of this
field is used by contract developers to (erroneously) deny smart
contract access to other smart contracts (and therefore, validators).
This PR fixes that oversight by returning the same values as other EOAs
for known validator addresses. Obviously, it needs a hard fork that will
be scheduled separately.

* Fix context passing.

* Clean up code.

* Removed engine dependency.

* Fix possible panic.

* Clean up code.

* Network type.

* Fix tests.

* Revert "Removed engine dependency." (#4392)

* Revert "Fix tests."

This reverts commit 597ba2d6f1.

* Revert "Network type."

This reverts commit 5e1878aedc.

* Revert "Clean up code."

This reverts commit 15885f4c9b.

* Revert "Fix possible panic."

This reverts commit 1a70d5eb66.

* Revert "Removed engine dependency."

This reverts commit 8c2ff803f7.

* gitignore the cache folder (#4389)

* stable localnet with external validator (#4388)

* stable localnet with external validator

* ignore deploy config file comments

* reduce node launched in localnet

* update makefile

* localnet configuration - add more fn

* fix validator information command typo

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: MaxMustermann2 <82761650+MaxMustermann2@users.noreply.github.com>
Co-authored-by: frozen <355847+Frozen@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”>
Co-authored-by: “GheisMohammadi” <36589218+GheisMohammadi@users.noreply.github.com>
Co-authored-by: Sun Hyuk Ahn <sunhyukahn@Suns-MacBook-Pro.local>
Co-authored-by: Soph <35721420+sophoah@users.noreply.github.com>

* build: update pinned curl version (#4394)

Per the Alpine Linux package repositories, the version for cURL included
with v3.16 has changed to revision 6

* consensus: replace type assert with test (#4398)

If `consensus.finalityCounter` does not have anything stored (for
example in Syncing mode), the `Load()` returns an interface that cannot
be automatically asserted to an `int64`. This results in the node
crashing. This commit fixes that.

* Turn pprof default on with local saved files (#3894)

* Turn pprof default on with local saved files

* [pprof] change interval from 600s to 3600s

* Revert "Turn pprof default on with local saved files (#3894)" (#4400)

This reverts commit 78d26d7910.

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Casey Gardiner <117784577+ONECasey@users.noreply.github.com>
Co-authored-by: frozen <355847+Frozen@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”>
Co-authored-by: “GheisMohammadi” <36589218+GheisMohammadi@users.noreply.github.com>
Co-authored-by: Sun Hyuk Ahn <sunhyukahn@Suns-MacBook-Pro.local>
Co-authored-by: Soph <35721420+sophoah@users.noreply.github.com>
Co-authored-by: Jacky Wang <jackyw.se@gmail.com>

consensus/consensus_service.go

@@ -538,8 +538,10 @@ func (consensus *Consensus) StartFinalityCount() {
 // FinishFinalityCount calculate the current finality
 func (consensus *Consensus) FinishFinalityCount() {
 	d := time.Now().UnixNano()
-	consensus.finality = (d - consensus.finalityCounter.Load().(int64)) / 1000000
+	if prior, ok := consensus.finalityCounter.Load().(int64); ok {
-	consensusFinalityHistogram.Observe(float64(consensus.finality))
+		consensus.finality = (d - prior) / 1000000
+		consensusFinalityHistogram.Observe(float64(consensus.finality))
+	}
 }
 
 // GetFinality returns the finality time in milliseconds of previous consensus

scripts/docker/Dockerfile

@@ -39,7 +39,7 @@ ARG HARMONY_USER_GID=1000
 ENV HARMONY_HOME=/harmony
 ENV HOME=${HARMONY_HOME}
 
-RUN apk add --no-cache bash~=5.1.16-r2 bind-tools~=9.16.37-r0 tini~=0.19.0 curl==7.83.1-r5 sed~=4.8-r0 \
+RUN apk add --no-cache bash~=5.1.16-r2 bind-tools~=9.16.37-r0 tini~=0.19.0 curl==7.83.1-r6 sed~=4.8-r0 \
     && rm -rf /var/cache/apk/* \
     && addgroup -g ${HARMONY_USER_GID} ${HARMONY_USER} \
     && adduser -u ${HARMONY_USER_UID} -G ${HARMONY_USER} --shell /sbin/nologin --no-create-home -D ${HARMONY_USER} \