Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ commands:
CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16 \
CARGO_PROFILE_RELEASE_OPT_LEVEL=2 \
CARGO_PROFILE_RELEASE_DEBUG=0 \
cargo install --locked --path . --features test_network
cargo install --locked --path . --features devnet

setup_environment:
description: "Setup testing environment"
Expand Down Expand Up @@ -760,7 +760,7 @@ jobs:
- run:
name: Lint scripts
command: |
shellcheck -x .ci/*.sh
shellcheck -x .ci/*.sh scripts/*.sh
- clear_environment:
cache_key: v4.2.0-rust-1.88.0-lint-scripts-cache

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/benchmarks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,8 @@ jobs:
gcloud storage cp gs://ci_testdata/sync-ledger-val40-250-9ec2291c57.zip ledger.zip
unzip ledger.zip

- name: Install snarkOS (test_network)
run: cargo install --path=. --locked --features=test_network
- name: Install snarkOS (devnet)
run: cargo install --path=. --locked --features=devnet

# Download previous benchmark result from cache (if exists)
- name: Download previous benchmark data
Expand Down
2 changes: 1 addition & 1 deletion .rusty-hook.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[hooks]
pre-commit = "cargo clippy --workspace --all-targets --all-features -- -D warnings && cargo +nightly fmt --all -- --check"
pre-commit = "./scripts/lint.sh"

[logging]
verbose = true
4 changes: 3 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -309,8 +309,10 @@ serial = [
]
test_targets = [ "snarkos-cli/test_targets" ]
test_consensus_heights = [ "snarkos-cli/test_consensus_heights" ]
test_network = [ "snarkos-cli/test_network", "snarkos-node/test_network" ]
devnet = [ "snarkos-cli/devnet", "snarkos-node/devnet" ]
tokio_console = [ "snarkos-cli/tokio_console" ]
# Deprecated alias for "devnet".
test_network = [ "devnet" ]

[dependencies.clap]
workspace = true
Expand Down
87 changes: 62 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,10 @@
* [6.1 Quick Start](#61-quick-start)
* [6.2 Operations](#62-operations)
* [6.3 Local Devnet](#63-local-devnet)
* [6.4 Feature Flags](#64-feature-flags)
* [6.5 Local Backups](#65-local-backups)
* [7. Contributors](#7-contributors)
* [8. License](#8-license)
* [6.4 Local Backups](#64-local-backups)
* [7. Cargo Features](#7-cargo-features)
* [8. Contributors](#8-contributors)
* [9. License](#9-license)

[comment]: <> (* [4. JSON-RPC Interface]&#40;#4-json-rpc-interface&#41;)
[comment]: <> (* [5. Additional Information]&#40;#5-additional-information&#41;)
Expand Down Expand Up @@ -292,7 +292,7 @@ The following is an overview of all files that may be needed to be migrated.

To run a node with custom settings, refer to the options and flags available in the `snarkOS` CLI.

The full list of CLI flags and options can be viewed with `snarkos --help`:
The available CLI flags and options can be viewed using `snarkos --help`:
```
snarkOS
The Aleo Team <hello@aleo.org>
Expand All @@ -312,7 +312,8 @@ SUBCOMMANDS:
update Update snarkOS
```

The following are the options for the `snarkos start` command:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made this argument list collapsible because it is massive

<details><summary>Full <code>snarkos start</code> list of command-line options/summary>

```
--network <NETWORK>
Specify the network ID of this node [options: 0 = mainnet, 1 = testnet, 2 = canary]
Expand Down Expand Up @@ -455,6 +456,8 @@ The following are the options for the `snarkos start` command:
Print help (see a summary with '-h')
```

</details>

## 6. Development Guide

### 6.1 Quick Start
Expand Down Expand Up @@ -587,22 +590,7 @@ To clean up the node storage, run:
cargo run --release -- clean --dev <NODE_ID>
```

## 6.4 Feature Flags

By default, the metrics feature is turned on for some internal crates.

* **history** -
Enables a /history REST endpoint.
* **telemetry** -
Allows the node to upload telemetry data.
* **cuda** -
Allows some operations to run on the (NVidia) GPU, instead of on the CPU. See [CUDA acceleration for provers](#optional-cuda-acceleration-for-provers) for install tips and current puzzle status.
* **locktick** -
This feature turns on code for detecting deadlocks.
* **test_targets** -
This feature allows the lowering of coinbase and proof targets for testing.

## 6.5 Local Backups
## 6.4 Local Backups

The snarkOS node implementation uses rocksdb under the hood. By using its native checkpointing mechanism, you can create backups locally and efficiently. The backups leverage hard links on your filesystem, thereby incurring only a marginal amount of extra space. The aim of these local backups is for you to be able to recover quickly in case your node were to halt.

Expand All @@ -616,7 +604,56 @@ You can find a basic sample script in `scripts/backup.sh` which you can run as a

You may want to change the `NETWORK`, `BASE_DIR`, `ENDPOINT` and `JWT` variables.

## 7. Contributors
## 7. Cargo Features

snarkOS exposes a number of cargo features that toggle optional functionality at build time. They
can be enabled by passing `--features <name>` (comma-separated) to `cargo build`, `cargo install`,
or `cargo run`.

By default, the `metrics` feature is enabled for some internal crates; all other features below
are opt-in.

### Production features

* **history** —
Enables the `/history` REST endpoint, which exposes historical chain data.
* **history-staking-rewards** —
Extends `history` with per-validator staking-reward records. Implies `history`.
* **telemetry** —
Allows validators to upload participation telemetry. See
[Enable Validator Telemetry Metrics](#321-enable-validator-telemetry-metrics-optional).
* **cuda** —
Runs select cryptographic operations on an NVIDIA GPU instead of the CPU. See
[CUDA acceleration for provers](#optional-cuda-acceleration-for-provers) for installation notes
and the current puzzle status.

### Development and debugging features

* **locktick** —
Compiles in instrumentation for detecting lock contention and potential deadlocks.
* **tokio_console** —
Enables a [`tokio-console`](https://github.com/tokio-rs/console) subscriber for inspecting
async tasks at runtime.
* **serial** —
Forces single-threaded execution in components that would otherwise use rayon/tokio
parallelism. Useful for deterministic debugging.

### Testing features

* **test_targets** —
Lowers the coinbase and proof targets so that puzzles can be solved quickly in tests.
* **test_consensus_heights** —
Allows overriding consensus version heights via environment variables, so tests can exercise
upgrade paths without waiting for real mainnet heights.
* **devnet** —
Enables development-only code paths used by local devnets: the `--dev` flag and dev-committee
hotswap support, deterministic account derivation, and snarkVM's `dev-print` output. Implies
`test_targets` and `test_consensus_heights`, so it is the most convenient way to build a node
for local testing.
* **test_network** —
*Deprecated.* Alias for `devnet`.

## 8. Contributors
Thank you for helping make snarkOS better!
[🧐 What do the emojis mean?](https://allcontributors.org/docs/en/emoji-key)

Expand Down Expand Up @@ -716,8 +753,8 @@ Thank you for helping make snarkOS better!

This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind are welcome!

## 8. License
## 9. License

We welcome all contributions to `snarkOS`. Please refer to the [license](#7-license) for the terms of contributions.
We welcome all contributions to `snarkOS`. Please refer to the [license](#9-license) for the terms of contributions.

[![License: GPL v3](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](./LICENSE.md)
4 changes: 2 additions & 2 deletions cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,8 @@ cuda = [
]
test_targets = [ "snarkvm/test_targets" ]
test_consensus_heights = [ "snarkvm/test_consensus_heights" ]
test_network = [
"test_targets",
devnet = [
"test_targets",
"test_consensus_heights",
"snarkvm/dev-print"
]
Expand Down
6 changes: 3 additions & 3 deletions cli/src/commands/start.rs
Original file line number Diff line number Diff line change
Expand Up @@ -689,9 +689,9 @@ impl Start {
println!("{}", crate::helpers::welcome_message());
}

// Only allow dev mode if we built with the 'test_network' feature.
if self.dev.is_some() && cfg!(not(feature = "test_network")) {
bail!("The 'dev' flag is set, but the 'test_network' feature is not enabled");
// Only allow dev mode if we built with the 'devnet' feature.
if self.dev.is_some() && cfg!(not(feature = "devnet")) {
bail!("The 'dev' flag is set, but the 'devnet' feature is not enabled");
}

// Parse the trusted peers to connect to.
Expand Down
2 changes: 1 addition & 1 deletion node/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ serial = [
"snarkos-node-bft/serial"
]
test = []
test_network = [ "snarkos-node-bft/test_network", "snarkos-node-consensus/test_network", "snarkos-utilities/test_network" ]
devnet = [ "snarkos-node-bft/devnet", "snarkos-node-consensus/devnet", "snarkos-utilities/devnet" ]

[dependencies.aleo-std]
workspace = true
Expand Down
4 changes: 2 additions & 2 deletions node/bft/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,8 @@ test = [
"snarkos-node-bft-ledger-service/test",
"snarkos-node-bft-storage-service/test"
]
test_network = [
"snarkos-node-bft-ledger-service/test_network",
devnet = [
"snarkos-node-bft-ledger-service/devnet",
]
serial = [
"snarkos-node-metrics/serial",
Expand Down
2 changes: 1 addition & 1 deletion node/bft/ledger-service/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ serial = [
"snarkvm/serial"
]
test = [ "mock", "translucent" ]
test_network = [ ]
devnet = [ ]
translucent = [ "ledger" ]

[dependencies.anyhow]
Expand Down
22 changes: 11 additions & 11 deletions node/bft/ledger-service/src/ledger.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

use crate::{BeginLedgerUpdateError, LedgerService, LedgerUpdateService, fmt_id, spawn_blocking};

#[cfg(feature = "test_network")]
#[cfg(feature = "devnet")]
use snarkos_utilities::NodeDataDir;
use snarkos_utilities::Stoppable;

Expand Down Expand Up @@ -70,7 +70,7 @@ pub struct CoreLedgerService<N: Network, C: ConsensusStorage<N>> {
latest_leader: Arc<RwLock<Option<(u64, Address<N>)>>>,
stoppable: Arc<dyn Stoppable>,
update_lock: Arc<Mutex<()>>,
#[cfg(feature = "test_network")]
#[cfg(feature = "devnet")]
dev_committee: Option<Committee<N>>,
}

Expand Down Expand Up @@ -149,7 +149,7 @@ impl<N: Network, C: ConsensusStorage<N>> CoreLedgerService<N, C> {
///
/// This variant should be used by long-running nodes (e.g. validators) that may be restarted,
/// so that the deterministic dev committee's starting round remains stable across runs.
#[cfg(feature = "test_network")]
#[cfg(feature = "devnet")]
pub fn new_dev(
ledger: Ledger<N, C>,
stoppable: Arc<dyn Stoppable>,
Expand All @@ -173,7 +173,7 @@ impl<N: Network, C: ConsensusStorage<N>> CoreLedgerService<N, C> {
latest_leader: Default::default(),
stoppable,
update_lock: Default::default(),
#[cfg(feature = "test_network")]
#[cfg(feature = "devnet")]
dev_committee: _dev_committee,
}
}
Expand All @@ -188,7 +188,7 @@ impl<N: Network, C: ConsensusStorage<N>> CoreLedgerService<N, C> {
/// on subsequent invocations. This keeps the committee's identity (and
/// therefore the certificates that reference it) consistent across
/// restarts.
#[cfg(feature = "test_network")]
#[cfg(feature = "devnet")]
fn build_dev_committee(
default_start_round: u64,
node_data_dir: NodeDataDir,
Expand All @@ -213,7 +213,7 @@ impl<N: Network, C: ConsensusStorage<N>> CoreLedgerService<N, C> {

/// Reads the persisted dev committee starting round from disk if it exists and is consistent
/// with `default_start_round`; otherwise writes the default to disk and returns it.
#[cfg(feature = "test_network")]
#[cfg(feature = "devnet")]
fn load_or_init_dev_committee_start_round(node_data_dir: NodeDataDir, default_start_round: u64) -> Result<u64> {
let path = node_data_dir.dev_committee_state_path();
let path_str = path.display();
Expand Down Expand Up @@ -243,7 +243,7 @@ impl<N: Network, C: ConsensusStorage<N>> CoreLedgerService<N, C> {

/// Writes the given `start_round` to the dev committee state file at `path`, creating the
/// parent directory if needed.
#[cfg(feature = "test_network")]
#[cfg(feature = "devnet")]
fn write_dev_committee_start_round(path: &std::path::Path, start_round: u64) -> Result<()> {
if let Some(parent) = path.parent()
&& !parent.exists()
Expand All @@ -258,7 +258,7 @@ impl<N: Network, C: ConsensusStorage<N>> CoreLedgerService<N, C> {
}

/// Returns the deterministic dev committee for rounds at or after the hotswap start.
#[cfg(feature = "test_network")]
#[cfg(feature = "devnet")]
fn dev_committee_for_round(&self, round: u64) -> Result<Option<Committee<N>>> {
let Some(dev_committee) = self.dev_committee.as_ref() else {
return Ok(None);
Expand Down Expand Up @@ -361,7 +361,7 @@ impl<N: Network, C: ConsensusStorage<N>> LedgerService<N> for CoreLedgerService<

/// Returns the current committee.
fn current_committee(&self) -> Result<Committee<N>> {
#[cfg(feature = "test_network")]
#[cfg(feature = "devnet")]
{
if let Some(dev_committee) = self.dev_committee.as_ref() {
return Ok(dev_committee.clone());
Expand All @@ -381,7 +381,7 @@ impl<N: Network, C: ConsensusStorage<N>> LedgerService<N> for CoreLedgerService<

/// Returns the committee lookback for the given round.
fn get_committee_lookback_for_round(&self, round: u64) -> Result<Committee<N>> {
#[cfg(feature = "test_network")]
#[cfg(feature = "devnet")]
{
if let Some(dev_committee) = self.dev_committee_for_round(round)? {
return Ok(dev_committee);
Expand All @@ -403,7 +403,7 @@ impl<N: Network, C: ConsensusStorage<N>> LedgerService<N> for CoreLedgerService<
}

/// Returns the deterministic hotswapped dev committee for the given round, if active.
#[cfg(feature = "test_network")]
#[cfg(feature = "devnet")]
fn dev_committee_for_round(&self, round: u64) -> Result<Option<Committee<N>>> {
CoreLedgerService::dev_committee_for_round(self, round)
}
Expand Down
2 changes: 1 addition & 1 deletion node/bft/ledger-service/src/traits.rs
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ pub trait LedgerService<N: Network>: std::fmt::Debug + Send + Sync {
fn get_committee_lookback_for_round(&self, round: u64) -> Result<Committee<N>>;

/// Returns the deterministic hotswapped dev committee for the given round, if active.
#[cfg(feature = "test_network")]
#[cfg(feature = "devnet")]
fn dev_committee_for_round(&self, _round: u64) -> Result<Option<Committee<N>>> {
Ok(None)
}
Expand Down
2 changes: 1 addition & 1 deletion node/bft/src/primary.rs
Original file line number Diff line number Diff line change
Expand Up @@ -627,7 +627,7 @@ impl<N: Network> proposal_task::BatchPropose for Primary<N> {
if previous_committee_lookback.is_quorum_threshold_reached(&authors) {
is_ready = true;
}
#[cfg(feature = "test_network")]
#[cfg(feature = "devnet")]
{
// If we are using a hotswapped dev committee, use simplified checks to more easily advance.
if let Some(dev_committee) = self.ledger.dev_committee_for_round(previous_round)? {
Expand Down
2 changes: 1 addition & 1 deletion node/consensus/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ serial = [
"snarkos-node-metrics/serial",
"snarkvm/serial"
]
test_network = [ ]
devnet = [ ]

[dependencies.aleo-std]
workspace = true
Expand Down
2 changes: 1 addition & 1 deletion node/consensus/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -595,7 +595,7 @@ impl<N: Network> Consensus<N> {

let check_instant = std::time::Instant::now();
cfg_if! {
if #[cfg(feature = "test_network")] {
if #[cfg(feature = "devnet")] {
// If we are using a hotswapped dev committee, skip checking the block.
let result = if self.ledger.dev_committee_for_round(block.round())?.is_some() {
Ok(block)
Expand Down
8 changes: 4 additions & 4 deletions node/src/validator/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -93,8 +93,8 @@ impl<N: Network, C: ConsensusStorage<N>> Validator<N, C> {
dev_txs: bool,
dev: Option<u16>,
_slipstream_configs: &[std::path::PathBuf],
#[cfg(feature = "test_network")] dev_num_validators_for_committee_hotswap: Option<u16>,
#[cfg(not(feature = "test_network"))] _dev_num_validators_for_committee_hotswap: Option<u16>,
#[cfg(feature = "devnet")] dev_num_validators_for_committee_hotswap: Option<u16>,
#[cfg(not(feature = "devnet"))] _dev_num_validators_for_committee_hotswap: Option<u16>,
signal_handler: Arc<SignalHandler>,
) -> Result<Self> {
// Initialize the ledger.
Expand All @@ -118,9 +118,9 @@ impl<N: Network, C: ConsensusStorage<N>> Validator<N, C> {
}

// Initialize the ledger service.
#[cfg(not(feature = "test_network"))]
#[cfg(not(feature = "devnet"))]
let ledger_service = Arc::new(CoreLedgerService::new(ledger.clone(), signal_handler.clone()));
#[cfg(feature = "test_network")]
#[cfg(feature = "devnet")]
// Initialize the ledger service with a deterministic dev committee.
let ledger_service = if let Some(dev_num_validators) = dev_num_validators_for_committee_hotswap {
Arc::new(CoreLedgerService::new_dev(
Expand Down
Loading