* added bloom filter
* upgrade rawdb and statedb
* change var name and remove extra comments
* return back fake storage in case if we need it for test later
* add the previous change back
* remove some extra entries from go.mod
* fix WritePreimages to use batch
* mark unused functions which are ported over from eth
---------
Co-authored-by: Casey Gardiner <117784577+ONECasey@users.noreply.github.com>
Due to technical debt, validator information is stored in the code field
of the address. The code field can be accessed in Solidity for an
arbitrary address using `extcodesize`, `extcodehash`, and `extcodecopy`
or helper commands (such as `address.code.Length`). The presence of this
field is used by contract developers to (erroneously) deny smart
contract access to other smart contracts (and therefore, validators).
This PR fixes that oversight by returning the same values as other EOAs
for known validator addresses. Obviously, it needs a hard fork that will
be scheduled separately.
* chore(build): upgrade golang to 1.19
* chore(build): run `go mod tidy`
* chore(build): run `goimports -w -e ${file}`
* chore(build): revert github ci changes
* chore(build): pin golang version to 1.19.5
* chore(build): fix protoc version on gen files
...and ensure transfers from the precompile address can be made before
executing them. Since we are now blocking multiple `CXReceipts` per
transaction explicitly, it doesn't make sense to be wary of contracts
subsidizing cross shard transfers for end users via `delegatecall`. It
is then safe to remove the gas charge for the precompile. Secondly,
ensure that there are enough funds on the precompile via `CanTransfer`
as part of the sanity check.
* fix unexpected chain id (https://github.com/harmony-one/harmony/issues/4129)
* update epoch numbers for ChainId fix
* updated ChainID Epoch for test net
* align comment with other comments
* fix the build using goimports
Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”>
...for native tokens only, and not for smart contracts (which will be
added later). Resolves#4132 and requires a hard fork, currently
scheduled at EpochTBD on main and test nets, but epoch 1 (after
AcceptsCrossTx) on other nets.
The precompile is at address 249 and the method has the signature
`crossShardTransfer(uint256 amount, address to, uint32 toShardID)`. It
requires a transfer of the `amount` to the precompile address first, in
the form of `msg.value`, and cascades a cross shard receipt to the
network when executed. At this stage, it is blocked for use by smart
contracts by checking that there is no code at the address, and that the
address is not a validator. Documentation for Python integration tests
to follow.
* Resolve harmony-one/bounties#77: Staking precompiles
Create write capable precompiles that can perform staking transactions
Add hard fork logic (EpochTBD) for these precompiles
Tests for new code with at least 80% unit test coverage
Staking library + tests in MaxMustermann2/harmony-staking-precompiles
* Fix small typo in comment
* Run goimports on files to fix Travis
* Do not activate staking precompile on shard 0
* Cascade readOnly to WriteCapableContract
* No overlap in readOnly + writeCapable precompiles
* Use function selector instead of directive
From Solidity, use abi.encodeWithSelector and match it against the
exact ABI of the functions. This allows us to remove the need for
a directive (32) being encoded, and thus saves 28 bytes of data.
* Do not allow contracts to become validators
As discussed with Jacky on #3906
* Merge harmony-one/harmony/main properly this time
* Run goimports
* Update gas calculation for staking precompile
Please see comment in core/vm/contracts_write.go RequiredGas
* Do not allow contract to become validator (2/2)
* Cache StakeMsgs from precompiled transactions
Add the StakeMsgs to ProcessorResult and cascade them in insertChain
* Remove ContractCode fields from validators
Since smart contracts can no longer beecome validators,
this field is superfluous. Remove it from the Wrapper
structure, and do not assign it a value when creating
a validator. Build and goimports checked
* Update comments in response to feedback
(1) Comments to start with function names
(2) Comments for public variables
(3) Comment to match function name RunPrecompiledContract
(4) Clarify that CreateValidatorFunc + EditValidatorFunc are still used
* Fix Travis build by reverting rosetta change
* Add revert capability to 3 staking tx types
- Delegate
- Undelegate
- CollectRewards
* Fix build: Update evm_test for ValidatorWrapper
* Merge main into harmony-staking-precompiles
* Add gas for precompile calls and allow EOA usage
- Each time the precompile is called, charge the base gas fee plus data
cost (if data can be parsed successfully). A gas fee is added to
prevent benevolent contract deployers from subsidizing the staking
transactions for EOAs through repeated assembly `delegatecall`.
- Allow EOAs to use the staking precompile directly. Some changes to
the Solidity library are associated with this change.
- Remove bytes from parsing address, since the ABI unpacks it into an
address format correctly.
- Add or update tests. Test coverage report to be attached to the PR
shortly.
* Run goimports
* Check read only and write capable for overlap
* Handle precompile stakeMsgs for block proposer
The staking precompile generates staking messages which are cascaded to
the block via the EVM in `state_processor.go`. This change cascades them
in `worker.go` to allow block proposers and block verifiers to keep the
same state.
* Run goimports for cf2dfac4081444e36a120c9432f4e..
* Update staking precompile epoch to 2 for localnet
Bring it in line with staking epoch. Change effects all configurations
except mainnet and testnet. `goimports` included.
* Add read only precompile to fetch the epoch num
* Move epoch precompile to 250
* precompiles: left pad the returned epoch number
* chainConfig: check epochs for precompiles
panic if staking precompile epoch < pre staking epoch
* Add staking migration precompile
- Lives at address 251
- Migrates delegations + pending undelegations from address A to B
- Useful if address A is hacked
- Charges gas of 21k + cost of bytes for two addresses
- Does not remove existing delegations, just sets them to zero.
Replicates current undelegate setup
- Unit tests and `goimports` included. Integration test following
shortly in MaxMustermann2/harmony-staking-precompiles
* Migration precompile: merge into staking
Merge the two precompiles into one, add gas calculation for migration
precompile. Move epoch precompile to 251 as a result. When migrating,
add undelegations to `To`'s existing undelegations, if any match the
epoch.
* Add migration gas test, remove panic, add check
In response to review comments, add tests for migration gas wherein
there are 0/1/2 delegations to migrate. Add the index out of bound check
to migration gas calculator and remove panics. Lastly, re-sort
migrated undelegations if no existing undelegation in the same epoch was
found on `To`.
* Move undelegations sorting to end of loop
* Add revert mechanism for UpdateValidatorWrapper
Closesharmony-one/bounties#90
(1) Use LRU for ValidatorWrapper objects in stateDB to plug a potential
memory leak
(2) Merge ValidatorWrapper and ValidatorWrapperCopy to let callers ask
for either a copy, or a pointer to the cached object. Additionally,
give callers the option to not deep copy delegations (which is a
heavy process). Copies need to be explicitly committed (and thus
can be reverted), while the pointers are committed when Finalise
is called.
(3) Add a UpdateValidatorWrapperWithRevert function, which is used by
staking txs `Delegate`, `Undelegate`, and `CollectRewards`. Other
2 types of staking txs and `db.Finalize` continue to use
UpdateValidateWrapper without revert, again, to save memoery
(4) Add unit tests which check
a) Revert goes through
b) Wrapper is as expected after revert
c) State is as expected after revert
* Change back to dictionary for stateValidators
Since the memory / CPU usage saved is not significantly different when
using an LRU + map structure, go back to the original dictionary
structure to keep code easy to read and have limited modifications.
* Add tests for validator wrapper reverts
As requested by @rlan35, add tests beyond just adding and reverting a
delegation. The tests are successive in the sense that we do multiple
modifications to the wrapper, save a snapshot before each modification
and revert to each of them to confirm everything works well. This change
improves test coverage of statedb.go to 66.7% from 64.8% and that of
core/state to 71.9% from 70.8%, and covers all the code that has been
modified by this PR in statedb.go.
For clarity, the modifications to the wrapper include (1) creation of
wrapper in state, (2) adding a delegation to the wrapper, (3)
increasing the blocks signed, and (4) a change in the validator Name and
the BlockReward. Two additional tests have been added to cover the
`panic` and the `GetCode` cases.
* SHA3-256 FIPS-202 precompiled contract implementation
* go fmt
* Added precompiles address under VRF
* added sha3 prcompiles to Istanbul
* fix goimports error
* Updated the sha3 & ecreceover pk contract address to 253 & 254 respectively
* Added a new PrecompiledContractsSHA3FIPS and add forking epoch & Changed the gas prices for sha3
Co-authored-by: Ganesha Upadhyaya <ganeshrvce@gmail.com>
* Add ethereum transaction support
* fix fmt
* fix lint
* add new block body fork logic
* add back deleted code
* lower case block version const
* add eth txn into new version of block
* refactor back
* refactor more
* fix test code
* Fix build
* Fix build
* revert eth txn in block and block proposal
* fix build
* fix build
* fix build
* Add ethereum transaction support
* fix fmt
* fix lint
* add new block body fork logic
* add back deleted code
* lower case block version const
* add eth txn into new version of block
* refactor back
* refactor more
* fix test code
* Fix build
* Fix build
* [rosetta] Refactor operations & prep for internal tx exposure
* Remove gas op relation for tx operations. Gas is for submission & processing
the tx, thus not really related to the amount being transferred
* Make optional starting op a ptr to a uint to keep consistent
* Reorg file for consistency of fn placement
* Rename functions for clarity
* Make getContractCreationNativeOperations consume getBasicTransferOperations for consistency
* Remove invariant doc as it does not apply anymore
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add framework for parsing traced txs
* Define ContractInfo struct for FormatTransaction
* Add tx trace helper function defs & propagate type defs
* Add a GetTransactionStatus helper fn for Operation formatting/creation
* Add wrapper function, getContractTransferNativeOperations, to get internal operations
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Implement transaction tracer
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [tracer] Add CallerAddress & CodeAddress to tracer logs
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [tracer] Remove ptr to slice & map in StructLogRes
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Implement getContractInternalTransferNativeOperations
* Add ContractAddress to ContractInfo for future usages (i.e: erc20 parsing)
* Only check for contract address if there is tx data in BlockTransaction
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix status report for contract related txs
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [tracer] Expose contract address instead of code address
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add trace cache & update TODO
* Trace any PLAIN transaction instead of only transactions with data.
This is to account for fall back contract fn calls & ignore staking data.
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Make internal tx formatter not return err on nil exec result
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* Fix lint
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix tests
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Add internal tx unit tests
* Fix tx data len check for contract related transfers as a transaction with len 0
data can happen for a contract (and fail) due to fall back operations.
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Update invariant comment
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Expose mutually exclusive ops + update docs & tests
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* [rosetta] Fix docs and err msgs
Signed-off-by: Daniel Van Der Maden <dvandermaden0@berkeley.edu>
* Rollup evm to geth v1.9.9 Muir Glacier
* fix go gen
* update intrinsic gas for istanbul
* Update statedb.Commit
* revert go gen result
* revert protobuf version
* update protobuf
* update go gen files
* set testnet epoch for evm upgrade
* Add cache to state validator access; refactor snapshot read/write
* make validator update happen at commit
* fix lint
* make sure slashing applies on validator wrapper in state
* Revert live update of validtor wrapper in staking txn
* add init population of validator cache logic
* Fix validator cache commit logic
* Separate validatorWrapper func
* fix build
* Fix lint
* [double-sign] Commit changes in consensus needed for double-sign
* [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain
* [slash] Add quick iteration tool for testing double-signing
* [slash] Add webhook example
* [slash] Add http server for hook to trigger double sign behavior
* [double-sign] Use bin/trigger-double-sign to cause a double-sign
* [double-sign] Full feedback loop working
* [slash] Thread through the slash records in the block proposal step
* [slash] Compute the slashing rate
* [double-sign] Generalize yaml malicious for many keys
* [double-sign][slash] Modify data structures, verify via webhook handler
* [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures
* [slash] Apply to state slashing for double signing
* [slash][double-sign] Checkpoint for working code that slashes on beaconchain
* [slash] Keep track of the total slash and total reporters reward
* [slash] Dump account state before and after the slash
* [slash] Satisfy Travis
* [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates
* [slash] Capture the unique new delegations since snapshot as well
* [slash] Filter undelegation by epoch of double sign
* [slash] Add TODO of correctness needed in slash needs on off-chain data
* [rpc] Fix closure issue on shardID
* [slash] Add delegator to double-sign testing script
* [slash] Expand crt-validator.sh with commenting printfs and make delegation
* [slash] Finish track payment of leftover slash debt after undelegation runs out
* [slash] Now be explicit about error wrt delegatorSlashApply
* [slash] Capture specific sanity check on slash paidoff
* [slash] Track slash from undelegation piecemeal
* [slash][delegation] Named slice types, .String()
* [slash] Do no RLP encode twice, once is enough
* [slash] Remove special case of validators own delegation
* [slash] Refactor approach to slash state application
* [slash] Begin expanding out Verify
* [slash] Slash on snapshot delegations, not current
* [slash] Fix Epoch Cmp
* [slash] Third iteration on slash logic
* [slash] Use full slash amount
* [slash] More log, whitespace
* [slash] Remove Println, add log
* [slash] Remove debug Println
* [slash] Add record in unit test
* [slash] Build Validator snapshot, current. Fill out slash record
* [slash] Need to get RLP dump of a header to use in test
* [slash] Factor out double sign test constants
* [slash] Factor out common for validator, stub out slash application, finish out deserialization setup
* [slash] Factor out data structure creation because of var lexical scoping
* [slash] Seem to have pipeline of unit test e2e executing
* [slash] Add expected snitch, slash amounts
* [slash] Checkpoint
* [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing
* [config] add double-sign testnet config (#1)
Signed-off-by: Leo Chen <leo@harmony.one>
* [slash] Commit for as is code & data of current dump.json
* [slash] Order of state operation not correct in test, hence bad results, thank you dlv
* [slash] Add snapshot state dump
* [slash] Pay off slash of validator own delegation correctly
* [slash] Pay off slash debt with special case for min-self
* [slash] Pass first scenario conclusively
* [slash] 2% slash passes unit test for own delegation and external
* [slash] Parameterize unit test to easily test .02 vs .80 slash
* [slash] Handle own delegation correctly at 80% slash
* [slash] Have 80% slash working with external delegator
* [slash] Remove debug code from slash
* [slash] Adjust Apply signature, test again for 2% slash
* [slash] Factor out scenario in testing so can test 2% and 80% at same time
* [slash] Correct balance deduction on plan delegation
* [slash] Mock out ChainReader for TestVerify
* [slash] Small surface area interface, now feedback loop for verify
* [slash] Remove development json
* [slash] trigger-double-sign consumes yaml
* [slash] Remove dead code
* [slash][test] Factor ValidatorWrapper into scenario
* [slash][test] Add example from local-testing dump - caution might be off
* [slash] Factor out mutation of slashDebt
* [slash][test] Factor out tests so can easily load test-case from bytes
* [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected
* [slash] Satisfy Travis
* [slash] Begin cleanup of PR
* [slash] Apply slash from header to Finalize via state processor
* [slash] Productionize code, Println => logs; adjust slash picked in newblock
* [slash] Need pointer for rlp.Decode
* [slash] ValidatorInformation use full wrapper
* Fix median stake
* [staking] Adjust MarshalJSON for Validator, Wrapper
* Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)
* Refactor offchain data; Add epoch to ValidatorSnapshot
* Make block onchain/offchain data commit atomically
* [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment
* [effective] VC eligible.go
* [consensus] Redundant field in printf
* [docker] import-ks for a dev account
* [slash] Create BLS key for dockerfile and crt-validator.sh
* [slash][docker] Easy deployment of double-sign testing
* [docker] Have slash work as single docker command
* [rpc] Fix median-stake RPC
* [slash] Update webhook with default docker BLS key
* [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go
* [slash] Remove helper binary, commented out code, change to local config
* [params] Factor out test genesis value
* Add shard checking to Tx-Pool & correct blacklist (#2301)
* [core] Fix blacklist & add shardID check
* [staking + node + cmd] Fix blacklist & add shardID check
* [slash] Adjust to PR comments part 1
* [docker] Use different throw away funded account
* [docker] Create easier testing for delegation with private keys
* [docker] Update yaml
* [slash] Remove special case for slashing validator own delegation wrt min-self-delegate
* [docker] Install nano as well
* [slash] Early error if banned
* [quorum] Expose earning account in decider marshal json
* Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (#2279)"
This reverts commit 9ffbf682c0.
* [slash] Add non-sanity check way to update validator
* [reward] Increase percision on percentage in schedule
* [slash] Adjust logs
* [committee] Check eligibility of validator before doing sanity check
* [slash] Update docker
* [slash] Move create validator script to test
* [slash] More log
* [param] Make things faster
* [slash][off-chain] Clear out slashes from pending in writeblockwithstate
* [cross-link] Log is not error, just info
* [blockchain] Not necessary to guard DeletePendingSlashingCandidates
* [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that
* [slash][test] Use faucet as sender, assume user imported
* [slash] Test setup
* [slash] reserve error for real error in logs
* [slash][availability] Apply availability correct, bump signing count each block
* [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed
* [slash] Pay as much as can
* [slash] use right nowAmt
* [slash] Take away from rewards as well
* [slash] iterate faster
* [slash] Remove dev based timing
* [slash] Add more log, sanity check incoming slash records, only count external for slash rate
* [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters
* [availability] More log
* [slash] Simply pre slash erra slashing
* [slash] Remove development code
* [slash] Use height from recvMsg, todo on epoch
* [staking] Not necessary to touch LastEpochInCommittee in staking_verifier
* [slash] Undo ds in endpoint pattern config
* [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment
* [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count
Co-authored-by: Leo Chen <leo@harmony.one>
Co-authored-by: flicker-harmony <52401354+flicker-harmony@users.noreply.github.com>
Co-authored-by: Rongjian Lan <rongjian@harmony.one>
Co-authored-by: Daniel Van Der Maden <daniel@harmony.one>
Gheis Mohammadi
Casey Gardiner
MaxMustermann2
Konstantin
Gheis
Konstantin
haodi
lutty
Rongjian Lan
peekpi
Simsonraj Easvarasakthi
Sebastian Johnsson
Daniel Van Der Maden
Edgar Aroutiounian
flicker-harmony
chao