Code quality, DX, and documentation improvements

Cargo.toml

@@ -13,4 +13,4 @@ resolver = "2"
 
 [workspace.package]
 version = "0.1.0"
-edition = "2024"
+edition = "2021"

Makefile

@@ -0,0 +1,85 @@
+.PHONY: help dev dev-build dev-down test-unit test-sync test-forge fmt fmt-check clippy build build-release build-contracts key-gen clean docker-clean
+
+# Default target: show help
+help:
+	@echo "Nightfall 4 CE - Development Commands"
+	@echo ""
+	@echo "  make dev              Start the full development stack"
+	@echo "  make dev-build        Build and start the development stack"
+	@echo "  make dev-down         Stop all development services"
+	@echo ""
+	@echo "  make build            Build all Rust crates"
+	@echo "  make build-release    Build all Rust crates (release mode)"
+	@echo "  make build-contracts  Build Solidity contracts with Foundry"
+	@echo ""
+	@echo "  make test-unit        Run Rust unit tests"
+	@echo "  make test-sync        Run synchronization tests via Docker"
+	@echo "  make test-forge       Run Solidity contract tests"
+	@echo ""
+	@echo "  make fmt              Format Rust code"
+	@echo "  make fmt-check        Check Rust code formatting"
+	@echo "  make clippy           Run clippy linter"
+	@echo ""
+	@echo "  make key-gen          Generate ZK proving keys (heavy)"
+	@echo "  make clean            Clean Rust and Solidity build artifacts"
+	@echo "  make docker-clean     Remove Docker containers, volumes, and images"
+
+# Start the full development stack
+dev:
+	docker compose --profile development --env-file .env up
+
+# Build and start the development stack
+dev-build:
+	docker compose --profile development --env-file .env up --build
+
+# Stop all development services
+dev-down:
+	docker compose --profile development down
+
+# Run Rust unit tests
+test-unit:
+	cargo test
+
+# Run synchronization tests via Docker
+test-sync:
+	docker compose --profile sync_test --env-file .env up
+
+# Run Solidity contract tests
+test-forge:
+	forge test
+
+# Format Rust code (requires nightly)
+fmt:
+	cargo +nightly fmt
+
+# Check Rust code formatting
+fmt-check:
+	cargo +nightly fmt -- --check
+
+# Run clippy linter (strict mode)
+clippy:
+	cargo clippy --all-targets -- -D warnings
+
+# Build all Rust crates
+build:
+	cargo build
+
+# Build all Rust crates in release mode
+build-release:
+	cargo build --release
+
+# Build Solidity contracts with Foundry
+build-contracts:
+	forge clean && forge build
+
+# Generate ZK proving keys (resource-intensive)
+key-gen:
+	NF4_MOCK_PROVER=false cargo run --release --bin key_generation
+
+# Clean Rust and Solidity build artifacts
+clean:
+	cargo clean && forge clean
+
+# Remove Docker containers, volumes, and locally-built images
+docker-clean:
+	docker compose down -v --rmi local

README.md

@@ -1,9 +1,108 @@
-# nightfall_4_CE
-Community edition of Nightfall_4
+# Nightfall 4 CE
+
+Community Edition of Nightfall\_4
+
+Nightfall\_4 is a Zero-Knowledge Proof (ZKP)-based Layer 2 ZK-ZK rollup for transferring ERC20, ERC721, ERC1155, and ERC3525 tokens privately on Ethereum. Unlike Nightfall\_3's optimistic rollup, Nightfall\_4 uses cryptographic proofs for near-instant finality. A private transfer typically costs around 6000 gas.
 
 _This code is not owned by EY and EY provides no warranty and disclaims any and all liability for use of this code. Users must conduct their own diligence with respect to use for their purposes and any and all usage is on an as-is basis and at your own risk._
 
-Nightfall_4 is a ZK rollup build around the ZK Privacy of Nightfall. It enables one to transfer ERC20, ERC721, ERC1155 and ERC3525 tokens in privacy. Full details can be found in the /doc folder of this repository.
+**This software is experimental. It should not be used to make significant value transactions.**
+
+## Architecture
+
+The project is a Rust workspace with the following crates:
+
+| Crate | Description |
+|---|---|
+| `nightfall_client` | Client service for creating private transactions (deposit, transfer, withdraw) |
+| `nightfall_proposer` | Block proposer that assembles L2 blocks and generates rollup proofs |
+| `nightfall_deployer` | Smart contract deployment and ZK proving key generation |
+| `nightfall_bindings` | Auto-generated Rust bindings for Solidity contracts |
+| `lib` | Shared cryptographic and blockchain utilities (Merkle trees, Poseidon hashing, PLONK proofs) |
+| `configuration` | Configuration management via TOML files and environment variables |
+
+Smart contracts are in `blockchain_assets/contracts/` and use the UUPS upgradeable proxy pattern.
+
+## Prerequisites
+
+- [Docker](https://docs.docker.com/get-docker/) and Docker Compose
+- [Rust](https://rustup.rs/) 1.88.0+ (pinned via `rust-toolchain.toml`)
+- [Foundry](https://book.getfoundry.sh/getting-started/installation) (forge, anvil)
+
+## Quick Start
+
+Start the full local development stack (Anvil chain, deployer, proposer, two clients, and MongoDB):
+
+```bash
+make dev
+```
+
+This uses the `development` profile with a local Anvil chain and mock provers, so no heavy ZK key generation is required.
+
+Once running:
+- Client 1 API: `http://localhost:3000`
+- Client 2 API: `http://localhost:3002`
+- Proposer API: `http://localhost:3001`
+- Anvil RPC: `http://localhost:8545`
+
+## Development Commands
+
+Run `make help` to see all available targets:
+
+| Command | Description |
+|---|---|
+| `make dev` | Start the full development stack |
+| `make dev-build` | Build and start the development stack |
+| `make dev-down` | Stop all development services |
+| `make build` | Build all Rust crates |
+| `make build-contracts` | Build Solidity contracts with Foundry |
+| `make test-unit` | Run Rust unit tests |
+| `make test-sync` | Run synchronization tests via Docker |
+| `make test-forge` | Run Solidity contract tests |
+| `make fmt` | Format Rust code (requires nightly) |
+| `make clippy` | Run clippy linter (strict mode) |
+| `make key-gen` | Generate ZK proving keys (resource-intensive) |
+| `make clean` | Clean build artifacts |
+
+## API Endpoints
+
+### Client (port 3000)
+
+| Method | Path | Description |
+|---|---|---|
+| POST | `/v1/deposit` | Deposit tokens into Nightfall |
+| POST | `/v1/transfer` | Private token transfer |
+| POST | `/v1/withdraw` | Withdraw tokens from Nightfall |
+| GET | `/v1/commitments` | List commitments (supports `?limit=&offset=`) |
+| GET | `/v1/commitment/{key}` | Get a single commitment |
+| GET | `/v1/balance/{token}/{owner}` | Get ERC token balance |
+| GET | `/v1/fee_balance` | Get fee balance |
+| GET | `/v1/l1_balance` | Get Layer 1 balance |
+| GET | `/v1/synchronisation` | Check sync status |
+| POST | `/v1/certification` | Submit X.509 certificate |
+| GET | `/v1/health` | Health check |
+
+### Proposer (port 3001)
+
+| Method | Path | Description |
+|---|---|---|
+| POST | `/v1/transaction` | Submit client transaction |
+| POST | `/v1/register` | Register as a proposer |
+| POST | `/v1/deregister` | Deregister a proposer |
+| GET | `/v1/rotate` | Rotate active proposer |
+| POST | `/v1/pause` | Pause block assembly |
+| POST | `/v1/resume` | Resume block assembly |
+| GET | `/v1/blockdata` | Get current block data |
+| POST | `/v1/certification` | Submit X.509 certificate |
+| GET | `/v1/health` | Health check |
+
+## Documentation
+
+- [Architecture and API Reference](doc/nf_4.md) - Full documentation
+- [Testnet Setup Guide](doc/Setup%20Testnet%20Guide.md) - Deploy to a host chain
+- [Upgradable Contracts Guide](doc/Upgradable%20Contracts%20Guide.md) - Contract upgrade procedures
+- [Changelog](doc/CHANGELOG.md) - Version history
 
-Please note that this software should be treated as experimental. It should not be used to make significant value transactions.
+## License
 
+See [LICENSE](LICENSE).

nightfall_client/src/driven/db/mongo.rs

@@ -431,12 +431,20 @@ impl RequestCommitmentMappingDB for Client {
 impl CommitmentDB<Fr254, CommitmentEntry> for Client {
     async fn get_all_commitments(
         &self,
+        limit: Option<u64>,
+        offset: Option<u64>,
     ) -> Result<Vec<(Fr254, CommitmentEntry)>, mongodb::error::Error> {
-        let mut cursor = self
+        let mut find = self
             .database(DB)
             .collection::<CommitmentEntry>("commitments")
-            .find(doc! {})
-            .await?;
+            .find(doc! {});
+        if let Some(offset) = offset {
+            find = find.skip(offset);
+        }
+        if let Some(limit) = limit {
+            find = find.limit(limit.min(i64::MAX as u64) as i64);
+        }
+        let mut cursor = find.await?;
         let mut result: Vec<(Fr254, CommitmentEntry)> = Vec::new();
         while cursor.advance().await? {
             let v = cursor.deserialize_current()?;
@@ -553,7 +561,7 @@ impl CommitmentDB<Fr254, CommitmentEntry> for Client {
         let k_string = k.to_hex_string();
         debug!("Getting commitment with key: {k_string}");
         let commitment_1 = self
-            .get_all_commitments()
+            .get_all_commitments(None, None)
             .await
             .expect("Database error")
             .into_iter()