Repository navigation
Make the simulator generic and safe as a subproject - #1
Open
GabrielCosme wants to merge 52 commits into
Open
GabrielCosme wants to merge 52 commits into
GabrielCosme wants to merge 52 commits into
Conversation
…er a host HAL The simulator knows no robot any more. engine/ holds the world, the clock, the run loop, the firmware thread, the devices, the arenas, the robot description and its model, scenarios and recording; view/ the window, the declarative panel and overlay and the video; bridge/ the monitor bridge; app/ the command line and the Target interface a robot implements. hal_host/ implements micras_hal on a PC and knows no physics. Everything about the Micras lives in targets/micras: its bindings, the proxies of its two SPI chips, the hand written Cube layer, the robot description in robot.toml, its tests, tools and baselines, and the firmware submodule, which now follows high-level-review. The rule everything serves is that a run is reproducible: the same binary with the same arguments writes byte identical data.csv and meta.json, headless or watched, recorded or not, with a bridge nobody connects to or without one. Noise is seeded. Baselines are summaries with tolerances instead of recordings. The ten contest mazes join the two there were, and the old models, the Python maze generator, the firmware build and the recorded runs are gone. SPEC_1_WORK.md and SPEC_2_DECISIONS.md record the work and the decisions.
… walls The maze arena reads the usual ASCII drawing, with the start and the goal cells, and generates its MJCF. The walls and posts are white with a red top, as they were in the Python generator, except around the start and the goal cells, which are all white. The floor checker has one square per cell and falls on the middle of the walls: the plane reaches a whole number of cells past the maze, since MuJoCo lays its texture from the corner, and the top camera is orthographic, which the old generator's was too, so the tops of the walls do not drift out of line with the floor towards the edges. The red tops are boxes of their own, in a paint group that collides with nothing and that the rays of the wall sensors pass through, like the robot's own group. With them the runs stay byte identical to runs without them; only the hash of the model changes.
--video-trail draws the path the robot has traveled behind it, sampled every tick a point every 3 mm so that it follows the curves however far apart the frames are; --video-trail-color sets its color and --video-tint draws the robot in the same color, to tell two recordings of one maze apart when they are composited. The robot model gets an onboard camera at the front of the board, below the top of the walls, and a near clipping plane close enough for it to see the floor in front of it. All of it only reads the state, so a recorded run writes the same data.csv as one without a video.
The firmware moves to its high-level-review branch with four commits: the sensing, estimate and control errors the simulation found, the planner of labels with the turns of two bends, turns braked and accelerated through as their curvature allows, and a racing line through the cells of the route on the second switch, which used to allow diagonals and now names the racing line. The simulated switch follows, and the scenarios with every switch on turn it on. FIRMWARE_CHANGES.md records every change the simulator made to the firmware, with its evidence and what it means on the robot. On the ten mazes the whole contest runs clean with the fan alone and with every switch on, where the fast runs take 3.6 to 8.6 s.
…ng the firmware notes SPEC_1_WORK.md was the plan of the cleanup and SPEC_2_DECISIONS.md the record of its decisions; what still holds of them is in CLAUDE.md, and the rest is history. tools/gen_maze.py wrote a standalone MJCF that Maze::mjcf replaced, and nothing ran it. third_party/ only held a README for the download directory, which is back in the build directory, where the builds already put it. FIRMWARE_CHANGES.md stays on disk as the owner's notes, but out of the repository.
…found The firmware moves three commits on: the turn tables designed once, everything watched through the variable pool, and comments that describe the current state. The Micras target compiles without the job pool it needed when every source designed the turns, and the turn designer links the firmware alone. Fixed: - wall-calibration traced its rays through the whole maze, since the geom group of the paint was left at 0; the robot and the paint now share one group the range sensors skip, and the tool prints the gains robot.toml holds back. - Tab reached every other camera; it now cycles through the free one and each camera of the model. - A step asked for while the run was running fired once it paused; stepping now only works while paused. - Touching the panel marked the run interactive but left the scenario driving the inputs; the application now hands the board over. Removed, as nothing used them: push, whose force a window cleared after one tick; --video-trail-color and --video-tint; the provenance record of the robot description, max_motor_speed, imu.chip_z and the battery's load and resistance; the forward axis and three probes of the ground truth; Target::check_options and default_scenario, Wiring::inputs, observers and monitor_inbound; the listener that named unbound ports, which the target now does at the end of the run; counters of the host HAL nothing read, the unused vendor constants and the clock's quantum parameter; Noise::enabled and SerialBus::sent_bytes. Merged: the recorder is created in one step with its overlay and trail; the window and the recorder share the camera lookup; the three loaders share one file reader; the scenario hooks and the event log share one type of state names; the joints are looked up by the world; DigitalInput has its own header; the panel spec moves to the view, its only user; the tests share their helpers. The switch columns are named dip_fan, dip_racing_line, dip_boost and dip_risky, and the build checks that the target names every firmware state. Every checked run keeps its bytes but for those columns and the new pool variables.
just micras contest runs a scenario in all ten mazes at once, a row every 10 ms, and checks their health: explore_solve with the fan, explore_solve_all with every switch. check_run.py now also fails a run with a collision, and computes its own count of non-finite samples instead of reading a report.json built only for it. The gate runs less: the window, the video and the bridge are compared with one plain run instead of three, and the flash is checked from the map the checked exploration saves instead of a 45 s run of its own. compare_run.py compares every meta.json field but the command line, so it names no robot, and drops the gzip and column subset support left from the recorded baselines. The baseline's unbound_ports comes from the Micras plugin, which also no longer plots the motor voltages twice; collisions are in the engine's report. The baselines restart at v1, which is v5; v3 and v4 are in the history. The scenarios no longer claim a clearance they only have with the firmware's walls.
CLAUDE.md and the README drop the references to the specs, the history of how the port got where it is and the claims that no longer held: unbound ports are named on stderr, the target lists the state names, the listeners and the Wiring are what they are now, and push, the two video flags and the maze generator are gone. The state of the port says what the firmware does, and that with every switch on the robot grazes a post on apec2016 in about half the seeds, as it already did before these changes.
The firmware's high-level-review is rebased onto feature/bluetooth-link, which follows main with the low level review merged, and builds, tests and lints as the firmware's CI does. The host Mcu answers the two questions the initialization now asks: no reset came from the watchdog, since a simulated run always starts cold and counts a watchdog expiry instead of acting on it, and the core runs within its datasheet. The turns are designed in the firmware's config/dynamics_config.cpp, which the target now compiles. The checked runs drive exactly as before; the battery voltage column follows the 4 Hz filter of the low level review.
The firmware now restarts an ADC that an overrun stopped (Team-Micras/MicrasFirmware#57). The host AdcDma follows its header: every converter registers itself when it is constructed, the error callback's mark and recover() exist, and get_restarts() reads the count, which no host converter ever moves. adc_restarts is a new variable column, and announcing it on the link costs 15 bytes of credit, so link_credit starts at 51 instead of 66; every other column of the reference runs is byte identical.
The firmware's branches were rebased onto its rewritten main. The files are the same as at 61a8e14 except the README, which lost its CI badge.
The firmware answered the review of Team-Micras/MicrasFirmware#54 in one commit (9e307de). The host AdcDma publishes nothing while a converter is stopped, like the firmware's, and the current sense amplifiers are bidirectional, as the board's are: a channel reads half its range with no current and moves from there with the sign of the current, which the firmware now reads. It changes behaviour, so the baselines are a new version, v2. Everything but four changes was checked byte identical first: the reference runs differ only in the two new variable columns, failed_saves and localizer_recoveries, in link_credit, which announcing them costs, and in run_profile for the 0.1 s before a scenario sets the switches, since it is no longer restored from the flash. The four are the turn factor of the planner, the explorer round that runs to its end, the search speed cap into a cell with an unknown front wall, and the tick that restarts after a save and the boot. The contest stays clean on the ten mazes, with every switch on too. The fast runs are the same but for maze 1 with every switch, 11 ms faster, alljapan with the fan, 6 ms faster, and apec2019 with every switch, 127 ms slower: the turn factor changes the route there, and the racing line drawn through it comes out slower. The search takes 77 to 123 s, against 62 to 109 s: the cap costs about 10 s a maze, while the explorer and the turn factor together save 5 s over the ten.
… as v1 The firmware takes the turn factor of the planner out again (e3aafea): it discounted only the turns of two bends, and on apec2019 it cost a fast run 127 ms. The fast runs are the ones before it in every maze, and the contest is byte identical to the build that had the factor disabled. The baseline is recorded again as the only version, v1, from this behaviour. The search takes 77 to 113 s: the speed cap into a cell with an unknown front wall costs about 10 s a maze, and the explorer round that runs to its end saves some of it back.
… again The firmware's planner now prices a turn entered below its speed at a blend of the two, and a turn that leaves slower for the next one at nothing more (682373a). That changes the routes, so the baseline v1 is recorded again. The contest stays clean with the fan: every fast run equal or faster, by up to 100 ms on japan2017ef, and the search at 65 to 118 s. With every switch on, the racing line is drawn through different routes: japan2017ef is 70 ms faster but its board now grazes two walls at about 1.5 m/s on every seed, while apec2016, which grazed on about half the seeds, does on one in eight. Four other mazes are 10 to 30 ms slower there.
…ne recorded again The firmware answers the second review of #54 (72651b0): the racing line keeps 13 mm whatever the switch, a straight with the wall ahead unknown is held to the search speed only up to the point the robot can still stop from, the planner's preparation is charged for its real cost, and the time scale, the localizer's range check, the IMU status and blinded wall sensors are fixed. That changes the search, so the baseline v1 is recorded again. The contest stays clean with the fan, every fast run the same, and the search takes 867 s over the ten mazes instead of 911 s, none slower. With every switch on, japan2017ef no longer grazes, 70 ms slower, and it and apec2016 are clean on eight seeds of the fast run; the other mazes run as before.
… the baseline recorded again The firmware answers the third review of #54 (82585b5): the time scale rises with the right sign and not while the motors saturate, the planner's preparation is charged only for the edges it times, and with risky the racing line goes only through the route planned without the risky turns. Then it brakes no harder than the motors can (3489bde), which leaves today's robot byte identical. The search changes, so the baseline v1 is recorded again. The contest stays clean with the fan and with every switch, with the same fast runs; the search takes 859 s over the ten mazes instead of 867 s, from 64 to 118 s per maze, and the planning with every switch half as long.
…e recorded again The firmware answers the fourth review of #54 (4deedba): a braking that does not fit its distance scales the tire and motor limits together, the speed that can be braked from takes a closed form or eight Newton steps, the motor limited braking distance never comes out negative, and the dark reading of each wall sensor is in the pool. Today's robot brakes harder with its motors than its tires allow, so it runs as before: the states and every value of the baseline are unchanged, and only the data hash moves with the new columns, so the baseline v1 is recorded again. The stack of #55 and #54 is merged, so the submodule now follows the firmware's main (4dac6e2, whose tree is 4deedba's) instead of high-level-review, and the three accessors the simulator reads are upstream. The gate passes on main with both checked runs byte identical to the recorded ones.
Every preset and the minimal configuration of check-options name gcc-15 and g++-15, so a run is built with a chosen compiler rather than whatever c++ the machine has. Both checked runs built with GCC 15.2 are byte identical to baselines/v1, recorded with GCC 13.4, so v1 stays the gate's baseline.
MICRAS_EXACT=1 makes compare-baseline pass --exact to baseline.py, so a checked run whose data.csv moved by a single byte fails the gate. It is the check for a change that must not alter behavior, such as a refactoring.
A chip behind an SPI chip select is a SpiDevice: select, exchange and deselect, and the SPI mode it answers in. Board::spi_device attaches one by the bus handle and the chip select, since hspi3 carries the IMU and both encoders. The host Spi routes every transfer between select_device and unselect_device to the selected device, so a register access split over several HAL calls is one transaction. start_transfer completes when the host clock passes the time its bytes take at the bus's bit rate, which the fake Cube layer now gives SPI3 (125 MHz kernel clock, prescaler 32), and ends through on_transfer_end as the DMA interrupt does. A device addressed in another mode, or a chip select with no device, answers all ones.
micras_hal_host_models (micras::hal_host_models, namespace micras::models) holds the two SPI chips of Micras as devices of the host SPI slot, knowing nothing but the slot. Lsm6dsvModel is the 128 registers of the main page with the datasheet defaults, software power-on reset, auto-increment, data ready bits set by a new sample and cleared by the read of its output, samples encoded with the full scale in CTRL6 and CTRL8, and the output data rates of CTRL1, CTRL2 and HAODR_CFG, in SPI mode 3. As5047uModel is the 24-bit frames with their CRC-8 checked on every frame, answers pipelined by one frame, ERRFL with the CRC and framing error bits, and the volatile registers the firmware writes, in SPI mode 1. Their unit tests address the LSM6DSV's registers by ST's driver header, so a register the model puts elsewhere than the driver fails.
A column source may name its columns only once the run has gone some way: the firmware's variables exist once its robot is constructed, which a real chip driver's start-up waits delay by several ticks. ColumnSource::ready says so, the recorder writes no row before every source is ready, and the Micras variables are ready once Micras::get_instance() returns the robot. A source that is ready from the start records as before.
The Micras target now compiles the firmware's imu.cpp and rotary_sensor.cpp and ST's lsm6dsv_reg.c, and its own replacements are gone. The bindings attach an Lsm6dsvModel and two As5047uModels to hspi3 by the chip selects target.hpp names, and turn each sample of the IMU device back into rad/s and m/s^2 for Lsm6dsvModel::push_sample; the encoders' position stays on the timer encoders. The sample port had no other user and is deleted. The firmware's objects outlive the target, so unwire forgets the ports and with them the chips. Recorded as baselines/v2, the gate's baseline now, because the behavior moved: - the real Imu reads the burst it started one update() earlier, so a sample reaches the firmware one loop later; - its constructor waits 10 ms, then 30 ms after the software power-on reset, so the robot exists 40 ms into the run: INIT and IDLE move from 0.000125 s and 0.00025 s to 0.040125 s and 0.04025 s, and the CSV starts at tick 320; - the scale of every IMU reading comes from the full scale the firmware wrote into CTRL6 and CTRL8. Every value stays within v1's tolerances: the explore reaches the goal at 26.891 s instead of 26.880 s, with a maximum pose error of 12.90 mm instead of 12.78 mm.
CLAUDE.md explains the slot (keyed by bus and chip select, one transaction per selection, the mode check, DMA transfers timed on the host clock, why unwire forgets the ports), the two chip models, the CSV starting once the firmware's robot exists, and the timings the real drivers bring. The README's layout lists hal_host/models/.
A project that adds the simulator with add_subdirectory keeps its own targets, cache and language settings: the style targets are prefixed micras_sim_ and exist only at top level; the build type is defaulted only at top level and never forced; the dependencies' switches are normal variables; paths are anchored to the simulator's project; the engine libraries require C++23 of their consumers. The options are named MICRAS_SIM_*, and the tests and the robot targets default to on only at top level. MICRAS_SIM_TOOLS_DIR and MICRAS_SIM_MAZES_DIR tell a consumer where the tools and the mazes are. ImGui's target is micras_sim_imgui. Every dependency is pinned by commit and fetched as that commit's archive with its SHA-256, which needs no history. ST's LSM6DSV driver is declared by the Micras target, which also adds the chip models, since only it uses them.
A target that lives in another repository keeps its analysis plugin next to its code, so meta.json records the target's folder (Target::directory) as target_dir, and analyze.py and baseline.py look for the plugin in --plugin, then $MICRAS_SIM_PLUGIN, then <target_dir>/tools/analysis.py, then targets/<target>/tools/analysis.py. baseline.py takes --plugin on record and compare; without its plugin a summary would lose the robot's values and the comparison would fail. compare_run.py ignores the fields that name paths, which differ between two checkouts of the same code.
A target's program is called with the FirmwareThread it runs on, so a target that drives the handover itself yields on it instead of keeping a pointer from wire(). CliOption, Wiring and WorldInfo move into micras/sim/app/wiring.hpp, and the README lists the headers a target may include, which are the public interface.
The style and build files are micras-lib's canonical copies, byte for byte: .clang-format, .clang-tidy, tests/.clang-tidy, cmake/micras_warnings.cmake and the lint script's template. The style targets run clang 22's tools by their versioned names and fail the configure without them, and micras_sim_lint_fix applies the fixes. Everything is formatted again with clang-format 22. The Micras target's .clang-tidy leaves the firmware's headers and the host HAL's to the repositories that lint them, and hal_host/ is formatted here but linted in micras-lib, which holds its copy.
The three repositories test with one framework, doctest, at the commit micras-lib uses. The engine's tests, the Micras target's and the chip models' keep their cases and their names (Suite.Case), each discovered by CTest through doctest_discover_tests: fixtures set up in their constructors, EXPECT and ASSERT became CHECK and REQUIRE, a near comparison checks the distance, and a double comparison allows a relative 1e-12. GoogleTest leaves the dependencies.
Every target gets micras_apply_warnings, the list the firmware and micras-lib compile with, instead of -Wall -Wextra, and MICRAS_SIM_WERROR makes the warnings errors. What they found is fixed without changing a result: the float to double promotions and the sign conversions are written out, the flash port initializes every member, and a cast that changed nothing is gone. The fetched dependencies are SYSTEM, so their headers raise no warning in the simulator's code.
The shared configuration of clang-tidy 22 finds what version 18 did not. Every file includes what it uses, with the conditional headers inside the branch that uses them; containers are indexed with at(), and spans through micras::sim::at, a checked access, since std::span has none before C++26; helper functions are static instead of living in an anonymous namespace; iterator arithmetic goes through std::next and std::to_address; the float to double promotions clang reports are written out; the state names are built on first use; the tools report an exception instead of terminating. toml++ is included through toml.hpp, and a TOML table's operator[] is a lookup, which engine/src/.clang-tidy tells the checks. Every run and every tool output is byte identical.
The simulator needs a reference that is not a real robot: targets/toy is the smallest one that uses every device, the firmware thread's handover, a command line option, the panel, the overlay and the scenario hooks, with no HAL. Its program waits, drives on a press of the button or a go over the link, turns at the wall its front sensor sees by integrating its gyro, and stops when the link says stop. Its robot is the engine tests' tiny robot with a second wall sensor, since the sensors fire in two groups, and a timestep that divides its 1 ms loop. Its two scenarios, idle and drive, have the baseline v1.
The simulator has no task runner: each check and each Micras recipe is a CMake target that calls a bash script with every path as an argument, so no script guesses a build layout. micras_sim_check builds everything and runs the unit tests, micras_sim_check_options, micras_sim_check_generic and the toy target's micras_sim_toy_check, which proves on the toy that a window, a video and an idle bridge change nothing and compares its runs with its baseline. The Micras recipes keep the names the firmware will give them (sim_run_idle, sim_run_explore, sim_contest, sim_contest_all, sim_compare_baseline, sim_record_baseline, sim_check_flash, sim_check, sim_turn_designs, sim_robot_report, sim_wall_calibration, sim_serve, sim_watch), with MICRAS_SIM_EXACT, MICRAS_SIM_BASELINE and MICRAS_SIM_CONTEST_MAZES as cache variables, and their scripts live in targets/micras/scripts. The checks and the recipes exist only at top level. The presets are host (Debug with the address and undefined behavior sanitizers), host-release (RelWithDebInfo) and host-ci. The justfiles are gone.
The simulator is public, so every push and every pull request, forks' included, runs its CI: the host-ci build, the unit tests, the format check, the lint and micras_sim_check, in the image of .docker/Dockerfile. Its host stage is the one micras-lib and the firmware use, byte for byte; the simulator stage adds GLFW, OpenGL, EGL and Mesa's software renderer, Xvfb, ffmpeg, and clang's sanitizer headers, which clang-tidy needs to read a sanitized build. The fetched dependencies are cached. The firmware submodule is cloned over HTTPS, which needs no key.
std::stod reported only its own name; std::from_chars lets the option say what it expects and what it got.
They are the simulator's public interface, the headers both targets include, so a new target knows what it may rely on and what changes only with notice.
The README and robot.toml name sim_turn_designs and sim_robot_report, the targets that run the tools.
Comments, documentation, messages, test names and the panel's Color type. The MJCF material names stay as they are, so the composed model does not change.
The new source files carry only their @file block, as the firmware's and micras-lib's do.
The toy's and Micras's baseline checks both read the root option.
ctest runs each case in its own process, several at once, so cases that wrote the same file in the temporary directory overwrote each other's output. The file now carries the process id and is removed when the case ends.
The Micras target lives in the firmware's sim/ now, with its history, and the host HAL and the chip models in micras-lib. The simulator keeps the engine, the tools, the mazes and the toy target, and no longer has a submodule.
The Micras target, its scenarios, its recipes, its robot description and its findings are documented in the firmware's sim/README.md, and the host HAL, the SPI device slot and the chip models in micras-lib's README. The simulator's documents keep the engine, the tools, the toy target and the contract a robot's project follows, with the toy as the example.
With the Micras target and the host HAL gone, nothing outside targets/ has a reason to name a robot: the documentation, the build files and the CI are scanned too. Only the files micras-lib shares with every repository, which name the repositories that carry them, the Dockerfile's shared host stage and the script itself are left out.
The switch is the way to leave the window out of a check, so it wins over an installed xvfb-run and over a display.
It has none: the robot targets that needed a firmware live in the robots' own repositories.
The material only colors the wheels, so every run stays byte identical.
A restored _deps without its stamps is downloaded again. No target pins a dependency any more, so the key reads only the files that do.
A port found free is released before the server binds it, so under a parallel ctest another process can take it in between. The tests then try another port and connect to the one the server holds; IXWebSocket cannot report the port the system picks for a server started on port 0.
GabrielCosme
commented
Sep 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The simulator becomes a generic micromouse simulator that another project can add as a subproject. The Micras robot target and the host HAL leave it: the target moves to the firmware's
sim/and the host HAL to micras-lib, both with their history. The toy target takes over as the simulator's reference.Subproject safety
A parent project with its own
formatandlinttargets can add the simulator withadd_subdirectory(... EXCLUDE_FROM_ALL), and it configures and builds:micras_sim_and exist only at top level. The build type is defaulted only at top level, and no cache variable is forced.CMAKE_SOURCE_DIR.MICRAS_SIM_*, and tests and targets default to top level only.MICRAS_SIM_TOOLS_DIRandMICRAS_SIM_MAZES_DIRlet a consumer's targets call the tools and name the mazes.The Target interface
FirmwareThread::Programtakes the thread it runs on.Wiring,WorldInfoandCliOptionhave their own header.--plugin,$MICRAS_SIM_PLUGINor the run'smeta.json, so a target can live outside the simulator.The toy target
targets/toy/is the smallest robot that uses every engine device and every part of theTargetinterface: motors, encoders, IMU, wall sensors, fan, battery, current sense, a serial link, a button, a CLI option, a panel and overlay, and scenario hooks. Its checks run asmicras_sim_toy_check:Standards
scripts/(micras_sim_check,micras_sim_format, …); the justfile is gone.micras_sim_check.targets/.Before merging
This pull request goes in with Team-Micras/MicrasFirmware#59, which pins this branch.