Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions docs/ecs/command-buffer.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

`CommandBuffer` is how game code performs structural changes — creating and destroying entities, adding and removing components — from inside a system tick. Recording an operation is cheap and immediate; the actual World mutation is deferred to the **stage barrier**, after every system in the current stage has finished iterating. Inside a system you never mutate the World structurally yourself: you call `ctx.cmd.*`, and the scheduler plays your buffer back at a safe, deterministic point.

Defined in [src/openvic-simulation/ecs/CommandBuffer.hpp](../../src/openvic-simulation/ecs/CommandBuffer.hpp) (recording) and [src/openvic-simulation/ecs/CommandBuffer.cpp](../../src/openvic-simulation/ecs/CommandBuffer.cpp) (playback).
Defined in [src/openvic-simulation/core/ecs/CommandBuffer.hpp](../../src/openvic-simulation/core/ecs/CommandBuffer.hpp) (recording) and [src/openvic-simulation/core/ecs/CommandBuffer.cpp](../../src/openvic-simulation/core/ecs/CommandBuffer.cpp) (playback).

## Where you get one

Expand Down Expand Up @@ -323,9 +323,9 @@ You could call `world.add_component` directly between ticks instead (the in-tick

## Source files

- [src/openvic-simulation/ecs/CommandBuffer.hpp](../../src/openvic-simulation/ecs/CommandBuffer.hpp) — recording API, op storage, payload holders
- [src/openvic-simulation/ecs/CommandBuffer.cpp](../../src/openvic-simulation/ecs/CommandBuffer.cpp) — `apply` / `clear` / `merge_from` playback
- [src/openvic-simulation/ecs/World.hpp](../../src/openvic-simulation/ecs/World.hpp) — in-tick mutation guard, reserved-slot lifecycle, `is_immutable`
- [src/openvic-simulation/ecs/System.hpp](../../src/openvic-simulation/ecs/System.hpp) — `TickContext` (the `cmd` member), per-chunk buffer pool on `SystemThreaded`
- [src/openvic-simulation/ecs/EntityID.hpp](../../src/openvic-simulation/ecs/EntityID.hpp) — `is_deferred()`, `DEFERRED_GENERATION_BIT`, `ImmutableEntityID`
- [src/openvic-simulation/core/ecs/CommandBuffer.hpp](../../src/openvic-simulation/core/ecs/CommandBuffer.hpp) — recording API, op storage, payload holders
- [src/openvic-simulation/core/ecs/CommandBuffer.cpp](../../src/openvic-simulation/core/ecs/CommandBuffer.cpp) — `apply` / `clear` / `merge_from` playback
- [src/openvic-simulation/core/ecs/World.hpp](../../src/openvic-simulation/core/ecs/World.hpp) — in-tick mutation guard, reserved-slot lifecycle, `is_immutable`
- [src/openvic-simulation/core/ecs/System.hpp](../../src/openvic-simulation/core/ecs/System.hpp) — `TickContext` (the `cmd` member), per-chunk buffer pool on `SystemThreaded`
- [src/openvic-simulation/core/ecs/EntityID.hpp](../../src/openvic-simulation/core/ecs/EntityID.hpp) — `is_deferred()`, `DEFERRED_GENERATION_BIT`, `ImmutableEntityID`
- Tests: [tests/src/ecs/CommandBuffer.cpp](../../tests/src/ecs/CommandBuffer.cpp), [tests/src/ecs/InTickMutationGuard.cpp](../../tests/src/ecs/InTickMutationGuard.cpp), [tests/src/ecs/SystemThreadedSpawn.cpp](../../tests/src/ecs/SystemThreadedSpawn.cpp)
20 changes: 10 additions & 10 deletions docs/ecs/components.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Forgetting to register is a clear compile error, not a silent misbehaviour: the
What the type system actually enforces:

- **Move-constructible and destructible.** Storage moves components between rows during archetype migrations and swap-pop compaction, and destroys them on `destroy_entity` / `remove_component`. Non-trivial members (e.g. a `std::string`) are handled correctly — destructors run, nothing leaks (see `tests/src/ecs/Component.cpp`, "Components with non-trivial dtor are destroyed properly").
- **Checksummable.** The first use of a component type with a `World` instantiates its column vtable, which `static_assert`s the universal hashing rule from `src/openvic-simulation/ecs/ChecksumTraits.hpp` (`is_checksummable_v<C>`). Every component (and singleton) type must hash one of two ways:
- **Checksummable.** The first use of a component type with a `World` instantiates its column vtable, which `static_assert`s the universal hashing rule from `src/openvic-simulation/core/ecs/ChecksumTraits.hpp` (`is_checksummable_v<C>`). Every component (and singleton) type must hash one of two ways:
1. **Raw bytes** — allowed only if `std::has_unique_object_representations_v<C>`, i.e. the compiler inserted no padding and there are no `float`/`double` members. Padding bytes are indeterminate garbage; fill gaps with explicit `_pad` members, zeroed at construction.
2. **A custom hash** — a free function `uint64_t ecs_checksum(C const&, uint64_t seed)` declared at `C`'s scope (found by ADL), before `C`'s first `World`/`CommandBuffer` use in the translation unit. Mandatory for anything holding heap data (`std::string`, vector members, ...): walk sizes + elements in index order, never capacities or addresses.

Expand Down Expand Up @@ -67,7 +67,7 @@ ECS_COMPONENT(Velocity, "test_Component::Velocity")

## ComponentTypeID — how types get IDs

You will mostly never touch this machinery directly; it is what `ECS_COMPONENT` plugs into. From `src/openvic-simulation/ecs/ComponentTypeID.hpp`:
You will mostly never touch this machinery directly; it is what `ECS_COMPONENT` plugs into. From `src/openvic-simulation/core/ecs/ComponentTypeID.hpp`:

```cpp
using component_type_id_t = uint64_t;
Expand Down Expand Up @@ -140,7 +140,7 @@ Notes:
- Only the *numeric ordering for one column within one run* is meaningful. Bulk creation bumps versions once per touched chunk instead of once per row, so never compare version values across runs or use them as data — they only signal "this column changed".
- An `ImmutableEntityID` overload exists with identical behaviour.

This is the primitive `CachedRef<C>::get(World&)` is built on (`src/openvic-simulation/ecs/CachedRef.hpp`): it re-resolves the pointer only when the live version differs from the cached one.
This is the primitive `CachedRef<C>::get(World&)` is built on (`src/openvic-simulation/core/ecs/CachedRef.hpp`): it re-resolves the pointer only when the live version differs from the cached one.

### Example: reads, writes, and what `nullptr` means

Expand Down Expand Up @@ -301,7 +301,7 @@ GameClock* c = ctx.world.get_singleton<GameClock>();

## `DenseSlotAllocator` — deterministic rows for singleton side tables

Some singleton-owned state is per-entity but doesn't fit a component column — variable-width or shared tables ("side tables") indexed by a dense row number that the owning entity stores. `DenseSlotAllocator` (`src/openvic-simulation/ecs/DenseSlotAllocator.hpp`) hands out those rows deterministically.
Some singleton-owned state is per-entity but doesn't fit a component column — variable-width or shared tables ("side tables") indexed by a dense row number that the owning entity stores. `DenseSlotAllocator` (`src/openvic-simulation/core/ecs/DenseSlotAllocator.hpp`) hands out those rows deterministically.

```cpp
inline constexpr uint32_t INVALID_DENSE_SLOT = static_cast<uint32_t>(-1);
Expand Down Expand Up @@ -380,10 +380,10 @@ if (!restored.restore(snap)) {

## Source files

- src/openvic-simulation/ecs/ComponentTypeID.hpp — `component_type_id_t`, `fnv1a_64`, `ComponentName`, `component_type_id_of`, `ECS_COMPONENT`
- src/openvic-simulation/ecs/World.hpp — `create_entity`, `add_component`, `remove_component`, `get_component`, `has_component`, `component_version_in`, `set_singleton`, `get_singleton`, `clear_singleton`
- src/openvic-simulation/ecs/ChecksumTraits.hpp — the checksum contract: `is_checksummable_v`, `ecs_checksum` convention, `ECS_CHECKSUM_BYTES`
- src/openvic-simulation/ecs/Archetype.hpp — `ColumnVTable` (the move/destroy/hash operations a component type must support)
- src/openvic-simulation/ecs/CachedRef.hpp — version-validated cross-tick component pointer
- src/openvic-simulation/ecs/DenseSlotAllocator.hpp / src/openvic-simulation/ecs/DenseSlotAllocator.cpp — deterministic dense rows for singleton side tables
- src/openvic-simulation/core/ecs/ComponentTypeID.hpp — `component_type_id_t`, `fnv1a_64`, `ComponentName`, `component_type_id_of`, `ECS_COMPONENT`
- src/openvic-simulation/core/ecs/World.hpp — `create_entity`, `add_component`, `remove_component`, `get_component`, `has_component`, `component_version_in`, `set_singleton`, `get_singleton`, `clear_singleton`
- src/openvic-simulation/core/ecs/ChecksumTraits.hpp — the checksum contract: `is_checksummable_v`, `ecs_checksum` convention, `ECS_CHECKSUM_BYTES`
- src/openvic-simulation/core/ecs/Archetype.hpp — `ColumnVTable` (the move/destroy/hash operations a component type must support)
- src/openvic-simulation/core/ecs/CachedRef.hpp — version-validated cross-tick component pointer
- src/openvic-simulation/core/ecs/DenseSlotAllocator.hpp / src/openvic-simulation/core/ecs/DenseSlotAllocator.cpp — deterministic dense rows for singleton side tables
- tests/src/ecs/Component.cpp, tests/src/ecs/Tag.cpp, tests/src/ecs/Singleton.cpp, tests/src/ecs/DenseSlotAllocator.cpp, tests/src/ecs/FNVHash.cpp — executable examples of everything above
36 changes: 18 additions & 18 deletions docs/ecs/determinism.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Determinism and checksums

OpenVic targets lockstep multiplayer: every peer runs the same simulation and must produce **bit-identical state** every tick, regardless of CPU, worker-thread count, or whether the session was freshly started or loaded from a save. This page is the contract that makes that work — the rules your game code must follow, the ordering guarantees the ECS gives you in return, and the checksum machinery (`src/openvic-simulation/ecs/Checksum.hpp`) that measures whether two worlds are actually in the same state.
OpenVic targets lockstep multiplayer: every peer runs the same simulation and must produce **bit-identical state** every tick, regardless of CPU, worker-thread count, or whether the session was freshly started or loaded from a save. This page is the contract that makes that work — the rules your game code must follow, the ordering guarantees the ECS gives you in return, and the checksum machinery (`src/openvic-simulation/core/ecs/Checksum.hpp`) that measures whether two worlds are actually in the same state.

The short version: write per-row integer/fixed-point arithmetic, declare everything you touch, never let a thread id or a memory address influence a result — and the worker-count-invariance tests will hold you to it.

Expand All @@ -12,7 +12,7 @@ Floating-point results vary with compiler, optimization flags, FMA contraction,

### 2. No thread-schedule-dependent results — use `reductions::*`

Any fold whose accumulation order depends on which worker ran first (a shared accumulator, a per-`worker_id` partial array folded in completion order) produces different results at different worker counts. Use the helpers in `src/openvic-simulation/ecs/Reductions.hpp` — `parallel_sum`, `parallel_min`, `parallel_max`, `parallel_keyed_sum`. They buffer per-chunk results keyed by `chunk_idx` and fold them **sequentially in `chunk_idx` ascending order** after the parallel section joins, so the result is bit-identical regardless of worker count. See threading-and-reductions.md for usage.
Any fold whose accumulation order depends on which worker ran first (a shared accumulator, a per-`worker_id` partial array folded in completion order) produces different results at different worker counts. Use the helpers in `src/openvic-simulation/core/ecs/Reductions.hpp` — `parallel_sum`, `parallel_min`, `parallel_max`, `parallel_keyed_sum`. They buffer per-chunk results keyed by `chunk_idx` and fold them **sequentially in `chunk_idx` ascending order** after the parallel section joins, so the result is bit-identical regardless of worker count. See threading-and-reductions.md for usage.

For the same reason, never key anything off the `worker_id` your code can observe, and never read wall-clock time, thread ids, or process-local RNG inside a tick.

Expand Down Expand Up @@ -44,7 +44,7 @@ A raw pointer is uniquely representable, so a byte-hashed component containing o

### 6. Keep `should_run` pure over deterministic state

The optional `static bool should_run(TickContext const&)` cadence gate must be a pure function of `ctx.today` and singletons read via `ctx.world`. It runs on every peer every tick; if it reads anything per-machine, peers diverge on *whether a system ran at all*. The skip is dispatch-time only — `schedule_hash()` is untouched — so gating never perturbs the schedule. Full contract in `src/openvic-simulation/ecs/System.hpp` and systems.md.
The optional `static bool should_run(TickContext const&)` cadence gate must be a pure function of `ctx.today` and singletons read via `ctx.world`. It runs on every peer every tick; if it reads anything per-machine, peers diverge on *whether a system ran at all*. The skip is dispatch-time only — `schedule_hash()` is untouched — so gating never perturbs the schedule. Full contract in `src/openvic-simulation/core/ecs/System.hpp` and systems.md.

### 7. Don't make logic sensitive to packing order across save/load

Expand All @@ -65,11 +65,11 @@ Given identical inputs and game code that follows the rules above, the ECS guara
uint64_t schedule_hash();
```

FNV-1a over the `(stage_index, system_type_id_t)` pairs of the current schedule (`src/openvic-simulation/ecs/World.hpp`). Multiplayer peers compute this at session-start handshake; a mismatch rejects the join. Registration *within* a conflict-free stage is order-insensitive — `tests/src/ecs/SystemFiltersWorkerCountInvariance.cpp` asserts that registering four co-staged systems in reverse order produces the same hash. See scheduling.md.
FNV-1a over the `(stage_index, system_type_id_t)` pairs of the current schedule (`src/openvic-simulation/core/ecs/World.hpp`). Multiplayer peers compute this at session-start handshake; a mismatch rejects the join. Registration *within* a conflict-free stage is order-insensitive — `tests/src/ecs/SystemFiltersWorkerCountInvariance.cpp` asserts that registering four co-staged systems in reverse order produces the same hash. See scheduling.md.

## EntityID stability across save/load

`EntityID` is `{ uint32_t index, uint32_t generation }` (`src/openvic-simulation/ecs/EntityID.hpp`). Ids are **save-stable**: the identity layer (slot generations, immutability flags, free-list order) can be snapshotted and restored exactly, so an `EntityID` stored inside a component means the same thing after a load as it did in the never-saved run. This is why checksumming hashes `EntityID` fields raw, and why components reference other entities by id rather than pointer.
`EntityID` is `{ uint32_t index, uint32_t generation }` (`src/openvic-simulation/core/ecs/EntityID.hpp`). Ids are **save-stable**: the identity layer (slot generations, immutability flags, free-list order) can be snapshotted and restored exactly, so an `EntityID` stored inside a component means the same thing after a load as it did in the never-saved run. This is why checksumming hashes `EntityID` fields raw, and why components reference other entities by id rather than pointer.

```cpp
bool snapshot_identity(WorldIdentitySnapshot& out) const;
Expand All @@ -79,7 +79,7 @@ template<typename... Cs>
bool restore_entity(EntityID eid, Cs&&... values);
```

- `snapshot_identity` captures the identity layer **only** — per-slot generations, per-slot immutability, free-list order (`WorldIdentitySnapshot` in `src/openvic-simulation/ecs/World.hpp`). Archetypes, packing, singletons, and systems are deliberately not captured; the loader rebuilds them. Refuses (error log + `false`) mid-tick or while any reserved-but-unfinalised slot exists (a `CommandBuffer` holding un-applied creates) — snapshot only between ticks, after every buffer has applied. It also validates the free chain and refuses to save a corrupt one.
- `snapshot_identity` captures the identity layer **only** — per-slot generations, per-slot immutability, free-list order (`WorldIdentitySnapshot` in `src/openvic-simulation/core/ecs/World.hpp`). Archetypes, packing, singletons, and systems are deliberately not captured; the loader rebuilds them. Refuses (error log + `false`) mid-tick or while any reserved-but-unfinalised slot exists (a `CommandBuffer` holding un-applied creates) — snapshot only between ticks, after every buffer has applied. It also validates the free chain and refuses to save a corrupt one.
- `restore_identity` requires a **fresh** World (no entity slot ever allocated), outside any tick. The snapshot is fully validated before any mutation; on failure the World is untouched. Afterward every live slot is reserved-but-unfinalised: addressable at its original `(index, generation)` but `is_alive == false` until finalised.
- `restore_entity` finalises one restored slot with its components (same component rules as `create_entity`). Between `restore_identity` and the last `restore_entity`, the **only** legal entity operations are `restore_entity` calls — in particular, `destroy_entity` on a not-yet-finalised id would push the slot onto the free list and silently corrupt the restored order. Recreate live entities in **slot-index ascending order**: identity correctness is order-independent, but packing is not, and the canonical order is what makes packing reproducible across loads.

Expand Down Expand Up @@ -193,8 +193,8 @@ Enforcement happens automatically at the two registration points — instantiati
Adapted from `tests/src/ecs/Checksum.cpp`:

```cpp
#include "openvic-simulation/ecs/ChecksumTraits.hpp"
#include "openvic-simulation/ecs/ComponentTypeID.hpp"
#include "openvic-simulation/core/ecs/ChecksumTraits.hpp"
#include "openvic-simulation/core/ecs/ComponentTypeID.hpp"

#include <vector>

Expand Down Expand Up @@ -253,9 +253,9 @@ The contract test: same starting World + same input → bit-identical post-tick
The full-state-checksum variant, adapted from `tests/src/ecs/Checksum.cpp` — use this as the template when adding a gate for new game systems:

```cpp
#include "openvic-simulation/ecs/Checksum.hpp"
#include "openvic-simulation/ecs/SystemImpl.hpp"
#include "openvic-simulation/ecs/World.hpp"
#include "openvic-simulation/core/ecs/Checksum.hpp"
#include "openvic-simulation/core/ecs/SystemImpl.hpp"
#include "openvic-simulation/core/ecs/World.hpp"

namespace {
// Threaded spawner: every CkSeed entity spawns one CkSpawned with a deterministic value.
Expand Down Expand Up @@ -317,7 +317,7 @@ TEST_CASE("Full-state checksum is identical across worker counts and serial mode
}
```

The two World knobs the harness uses (`src/openvic-simulation/ecs/World.hpp`):
The two World knobs the harness uses (`src/openvic-simulation/core/ecs/World.hpp`):

```cpp
// Override the ECS worker count. Call before the first `tick_systems` invocation.
Expand Down Expand Up @@ -352,10 +352,10 @@ Run them with `ctest --preset <preset>-debug` (after building with `cmake --buil

## Source files

- src/openvic-simulation/ecs/Checksum.hpp — `world_checksum`, `world_checksum_breakdown`, `fold_checksum_breakdown`, breakdown structs
- src/openvic-simulation/ecs/Checksum.cpp — the canonical walk implementation
- src/openvic-simulation/ecs/ChecksumTraits.hpp — the per-type hashing contract, traits, primitives, `ECS_CHECKSUM_BYTES`
- src/openvic-simulation/ecs/World.hpp — `schedule_hash`, `set_ecs_worker_count`, `set_serial_mode`, `snapshot_identity` / `restore_identity` / `restore_entity`, `WorldIdentitySnapshot`
- src/openvic-simulation/ecs/Reductions.hpp — deterministic parallel folds
- src/openvic-simulation/ecs/EntityID.hpp — `EntityID` / `ImmutableEntityID`
- src/openvic-simulation/core/ecs/Checksum.hpp — `world_checksum`, `world_checksum_breakdown`, `fold_checksum_breakdown`, breakdown structs
- src/openvic-simulation/core/ecs/Checksum.cpp — the canonical walk implementation
- src/openvic-simulation/core/ecs/ChecksumTraits.hpp — the per-type hashing contract, traits, primitives, `ECS_CHECKSUM_BYTES`
- src/openvic-simulation/core/ecs/World.hpp — `schedule_hash`, `set_ecs_worker_count`, `set_serial_mode`, `snapshot_identity` / `restore_identity` / `restore_entity`, `WorldIdentitySnapshot`
- src/openvic-simulation/core/ecs/Reductions.hpp — deterministic parallel folds
- src/openvic-simulation/core/ecs/EntityID.hpp — `EntityID` / `ImmutableEntityID`
- tests/src/ecs/WorkerCountInvariance.cpp, tests/src/ecs/SystemFiltersWorkerCountInvariance.cpp, tests/src/ecs/Checksum.cpp, tests/src/ecs/IdentitySnapshotInvariance.cpp — the determinism gates
Loading
Loading