Ethereum Protocol Attackathon Client Testing 🛠️


Client
How to run a PoC?
Documentation
Consensus Layer
The Executable Python Spec (PySpec) is derived directly from the  consensus specifications  and is enhanced by helper functions for hashing, BLS, and other core operations. This spec allows test generators to create test vectors for client implementations and validates its coherence through pytest-based sanity tests.
  •  https://github.com/ethereum/consensus-specs/tree/dev/tests/core/pyspec 
Execution Layer
The Execution Specs serve as the Python-based executable specification for Ethereum’s execution layer, built to model core protocol mechanics and ensure consistency across clients. Tests are run via tox and pytest, validating the spec’s behavior and generating test vectors for client implementations. Additionally, development tools facilitate code formatting, type-checking, and error reporting, ensuring that the specification remains robust and aligns with Ethereum’s protocol requirements.
  •  https://github.com/ethereum/execution-specs/blob/master/CONTRIBUTING.md#development 
ethPandaOps
 Kurtosis  is our local testing environment of choice. The network definition and tooling definition is stored in the  ethereum-package . We have some test definitions codified in a test assertion tool called  assertoor , the tests for assertoor are defined  here . While the assertoor tests are mostly used for finding bugs in network upgrades and for orchestrating more complicated interactions - it might be useful in the bug hunting process to put the network in a certain state. 

 Attacknet  is a chaos engineering framework that is integrated with kurtosis. This allows us to cause chaos on almost any layer of the linux stack: packet drops, network splits, cpu load, ram load, disk faults and so on. This might also be a useful tool to bring the network into a certain condition before a bug can be triggered. 
  •  https://ethpandaops.io/posts/kurtosis-deep-dive/ 
  •  https://ethpandaops.io/posts/assertoor-introduction/ 
  •  https://ethpandaops.io/posts/attacknet-introduction/ 
  •  https://github.com/ethpandaops/assertoor-test/ 
Ethereum Execution Specifications Tests
Ethereum Execution Spec Tests (EEST) is a framework designed to validate Ethereum client implementations by generating standardized JSON test cases based on pre-state, transaction, and post-state definitions. It uses Python tooling to create tests aligned with Ethereum's specifications and supports execution against clients like Geth or Hive environments. To run tests, you need Python, uv for dependency management, an Ethereum client binary (e.g., Geth's EVM), and optionally Hive for advanced testing. Create tests in Python, generate JSON fixtures with uv fill, and execute them with uv consume-direct or Hive for verification.
  •  EEST Documentation 
  •  Overview of Spec Tests 
  •  Overall Testing Details 
Prysm
Prerequisites:
  •  Docker Engine 
  •  Kurtosis 
  •  Go 
  •  Bazel 
Test:
  • bazel test //beacon-chain/node:go_default_test
  • bazel test //validator/client/...
  • See for Kurtosis integration testing  https://gist.github.com/potuz/56d3c05163e7ce24ddba4f7a452a325a 
  •  https://gist.github.com/potuz/56d3c05163e7ce24ddba4f7a452a325a 
  •  https://docs.prylabs.network/docs/install/install-with-bazel 
  •  https://docs.prylabs.network/docs/contribute/contribution-guidelines#building-and-testing-prysm-with-bazel 
Geth
Prerequisites:
  •  Go 
Build:
  • go install -v ./...
Test:
  • go test -v ./eth -run TestMethod
  •  https://geth.ethereum.org/docs/developers/geth-developer/dev-guide#testing 
Lighthouse
Prerequisites:
  •  Rust 
Test:
  • cargo test --release -p [crate_name] -- [test_name]
  •  https://hackmd.io/@sproul/testing-lighthouse-2024 
Nethermind
Prerequisites:
  •  .NET SDK 
Test:
  • cd nethermind/src/Nethermind
  • dotnet test Nethermind.sln -c release --filter "TestName~TestMethod"
  •  https://github.com/NethermindEth/nethermind#building-from-source 
Teku
Prerequisites:
  •  Java 21+ 
Build:
  • ./gradlew
Test:
  • ./gradlew test
  •  https://github.com/ConsenSys/teku?tab=readme-ov-file#build-and-test 
  •  https://docs.teku.consensys.io/get-started/connect/testnet 
  •  https://hackmd.io/@mehdi-aouadi/SJmnv5UFR 
Besu
Prerequisites:
  •  Java 21 
  • libsodium >1.0.16
  • libnss >3.35
Test:
  • ./gradlew test
  •  https://lf-hyperledger.atlassian.net/wiki/spaces/BESU/pages/22154264/Building+from+source 
  •  https://lf-hyperledger.atlassian.net/wiki/spaces/BESU/pages/22154264/Building+from+source#Dependencies 
  •  https://lf-hyperledger.atlassian.net/wiki/spaces/BESU/pages/22154265/Testing 
Nimbus ETH2
Prerequisites:
  •  git-lfs 
Build:
  • make j4
Test:
  • make test
  •  https://nimbus.guide/developers.html 
Erigon
Prerequisites:
  •  Docker Engine 
  •  Kurtosis 
  •  Node.js ,   npx , yarn 
Test:
  • kurtosis --enclave local-eth-testnet run github.com/ethpandaops/ethereum-package
  •  https://docs.kurtosis.com/how-to-local-eth-testnet/ 
  •  https://github.com/ethpandaops/ethereum-package 
  • In general the most effective way of producing a POC for a vulnerability will be to either produce a kurtosis test, or to provide an external network client that produces the eronious network behaviour, this is somewhat dependent on the network API for which the vulerenability is found.
  • The Erigon repo has 2 'live branches':
  • release/2.60: Which is the stable version of Erigon 2
  • main: Which is the current version of Erigon 3. This is alpha code and subject to continuous change
Reth
Prerequisites:
  •  Rust 
Test:
  • cargo test --workspace -p [package_name] [test_name]
  •  https://github.com/paradigmxyz/reth/?tab=readme-ov-file#building-and-testing 
Lodestar
Prerequisites:
  •  Node.js ,   npx , yarn 
Test:
  • yarn test:unit
  •  https://chainsafe.github.io/lodestar/contribution/getting-started/ 
  •  https://chainsafe.github.io/lodestar/contribution/testing/ 
Solidity Compiler
Prerequisites:
  •  evmone ,  z3 ,  Eldarica ,  cvc5 
Test:
  • /scripts/soltest.sh -t TestSuite/TestName
  •  https://docs.soliditylang.org/en/latest/contributing.html#running-the-compiler-tests 
Vyper Compiler
Prerequisites:
  •  Python 3.10 
  •  pytest 
  •  Vyper 
Build:
  • make dev-init
Test:
  • python setup.py test

  • Running Vyper's test suite:  https://github.com/vyperlang/vyper?tab=readme-ov-file#testing-using-pytest - pip install --editable:  https://pip-python3.readthedocs.io/en/stable/reference/pip_install.html#install-editable - optional, but useful for quick testing- useful little script for testing:  https://github.com/vyperlang/vyper/blob/master/quicktest.sh - another useful script for testing:  https://github.com/vyperlang/vyper?tab=readme-ov-file#developing-working-on-the-compiler "
  •  https://github.com/vyperlang/vyper?tab=readme-ov-file#testing-using-pytest 
  • Foundry Vyper docs:  https://book.getfoundry.sh/config/vyper?highlight=vyper#vyper-support 
  • Foundry test example:  https://github.com/pcaversaccio/snekmate/blob/dbd09b9b6ec33de80ec5439dff6cb8958a39ab8f/test/tokens/ERC20.t.sol 
  • Titanoboa testing framework docs:  https://titanoboa.readthedocs.io/en/latest/# 
  • Titanoboa test example:  https://github.com/curvefi/curve-stablecoin/tree/968e930f5fe709d0b77b6a497cf29f5318f7b44f/tests/swap