Repository navigation
Take the libraries and the simulator as submodules - #59
Open
GabrielCosme wants to merge 63 commits into
Open
GabrielCosme wants to merge 63 commits into
GabrielCosme wants to merge 63 commits into
Conversation
…er a host HAL The simulator knows no robot any more. engine/ holds the world, the clock, the run loop, the firmware thread, the devices, the arenas, the robot description and its model, scenarios and recording; view/ the window, the declarative panel and overlay and the video; bridge/ the monitor bridge; app/ the command line and the Target interface a robot implements. hal_host/ implements micras_hal on a PC and knows no physics. Everything about the Micras lives in targets/micras: its bindings, the proxies of its two SPI chips, the hand written Cube layer, the robot description in robot.toml, its tests, tools and baselines, and the firmware submodule, which now follows high-level-review. The rule everything serves is that a run is reproducible: the same binary with the same arguments writes byte identical data.csv and meta.json, headless or watched, recorded or not, with a bridge nobody connects to or without one. Noise is seeded. Baselines are summaries with tolerances instead of recordings. The ten contest mazes join the two there were, and the old models, the Python maze generator, the firmware build and the recorded runs are gone. SPEC_1_WORK.md and SPEC_2_DECISIONS.md record the work and the decisions.
… walls The maze arena reads the usual ASCII drawing, with the start and the goal cells, and generates its MJCF. The walls and posts are white with a red top, as they were in the Python generator, except around the start and the goal cells, which are all white. The floor checker has one square per cell and falls on the middle of the walls: the plane reaches a whole number of cells past the maze, since MuJoCo lays its texture from the corner, and the top camera is orthographic, which the old generator's was too, so the tops of the walls do not drift out of line with the floor towards the edges. The red tops are boxes of their own, in a paint group that collides with nothing and that the rays of the wall sensors pass through, like the robot's own group. With them the runs stay byte identical to runs without them; only the hash of the model changes.
The firmware moves to its high-level-review branch with four commits: the sensing, estimate and control errors the simulation found, the planner of labels with the turns of two bends, turns braked and accelerated through as their curvature allows, and a racing line through the cells of the route on the second switch, which used to allow diagonals and now names the racing line. The simulated switch follows, and the scenarios with every switch on turn it on. FIRMWARE_CHANGES.md records every change the simulator made to the firmware, with its evidence and what it means on the robot. On the ten mazes the whole contest runs clean with the fan alone and with every switch on, where the fast runs take 3.6 to 8.6 s.
…ng the firmware notes SPEC_1_WORK.md was the plan of the cleanup and SPEC_2_DECISIONS.md the record of its decisions; what still holds of them is in CLAUDE.md, and the rest is history. tools/gen_maze.py wrote a standalone MJCF that Maze::mjcf replaced, and nothing ran it. third_party/ only held a README for the download directory, which is back in the build directory, where the builds already put it. FIRMWARE_CHANGES.md stays on disk as the owner's notes, but out of the repository.
…found The firmware moves three commits on: the turn tables designed once, everything watched through the variable pool, and comments that describe the current state. The Micras target compiles without the job pool it needed when every source designed the turns, and the turn designer links the firmware alone. Fixed: - wall-calibration traced its rays through the whole maze, since the geom group of the paint was left at 0; the robot and the paint now share one group the range sensors skip, and the tool prints the gains robot.toml holds back. - Tab reached every other camera; it now cycles through the free one and each camera of the model. - A step asked for while the run was running fired once it paused; stepping now only works while paused. - Touching the panel marked the run interactive but left the scenario driving the inputs; the application now hands the board over. Removed, as nothing used them: push, whose force a window cleared after one tick; --video-trail-color and --video-tint; the provenance record of the robot description, max_motor_speed, imu.chip_z and the battery's load and resistance; the forward axis and three probes of the ground truth; Target::check_options and default_scenario, Wiring::inputs, observers and monitor_inbound; the listener that named unbound ports, which the target now does at the end of the run; counters of the host HAL nothing read, the unused vendor constants and the clock's quantum parameter; Noise::enabled and SerialBus::sent_bytes. Merged: the recorder is created in one step with its overlay and trail; the window and the recorder share the camera lookup; the three loaders share one file reader; the scenario hooks and the event log share one type of state names; the joints are looked up by the world; DigitalInput has its own header; the panel spec moves to the view, its only user; the tests share their helpers. The switch columns are named dip_fan, dip_racing_line, dip_boost and dip_risky, and the build checks that the target names every firmware state. Every checked run keeps its bytes but for those columns and the new pool variables.
just micras contest runs a scenario in all ten mazes at once, a row every 10 ms, and checks their health: explore_solve with the fan, explore_solve_all with every switch. check_run.py now also fails a run with a collision, and computes its own count of non-finite samples instead of reading a report.json built only for it. The gate runs less: the window, the video and the bridge are compared with one plain run instead of three, and the flash is checked from the map the checked exploration saves instead of a 45 s run of its own. compare_run.py compares every meta.json field but the command line, so it names no robot, and drops the gzip and column subset support left from the recorded baselines. The baseline's unbound_ports comes from the Micras plugin, which also no longer plots the motor voltages twice; collisions are in the engine's report. The baselines restart at v1, which is v5; v3 and v4 are in the history. The scenarios no longer claim a clearance they only have with the firmware's walls.
The firmware's high-level-review is rebased onto feature/bluetooth-link, which follows main with the low level review merged, and builds, tests and lints as the firmware's CI does. The host Mcu answers the two questions the initialization now asks: no reset came from the watchdog, since a simulated run always starts cold and counts a watchdog expiry instead of acting on it, and the core runs within its datasheet. The turns are designed in the firmware's config/dynamics_config.cpp, which the target now compiles. The checked runs drive exactly as before; the battery voltage column follows the 4 Hz filter of the low level review.
The firmware answered the review of #54 in one commit (9e307de). The host AdcDma publishes nothing while a converter is stopped, like the firmware's, and the current sense amplifiers are bidirectional, as the board's are: a channel reads half its range with no current and moves from there with the sign of the current, which the firmware now reads. It changes behaviour, so the baselines are a new version, v2. Everything but four changes was checked byte identical first: the reference runs differ only in the two new variable columns, failed_saves and localizer_recoveries, in link_credit, which announcing them costs, and in run_profile for the 0.1 s before a scenario sets the switches, since it is no longer restored from the flash. The four are the turn factor of the planner, the explorer round that runs to its end, the search speed cap into a cell with an unknown front wall, and the tick that restarts after a save and the boot. The contest stays clean on the ten mazes, with every switch on too. The fast runs are the same but for maze 1 with every switch, 11 ms faster, alljapan with the fan, 6 ms faster, and apec2019 with every switch, 127 ms slower: the turn factor changes the route there, and the racing line drawn through it comes out slower. The search takes 77 to 123 s, against 62 to 109 s: the cap costs about 10 s a maze, while the explorer and the turn factor together save 5 s over the ten.
… as v1 The firmware takes the turn factor of the planner out again (e3aafea): it discounted only the turns of two bends, and on apec2019 it cost a fast run 127 ms. The fast runs are the ones before it in every maze, and the contest is byte identical to the build that had the factor disabled. The baseline is recorded again as the only version, v1, from this behaviour. The search takes 77 to 113 s: the speed cap into a cell with an unknown front wall costs about 10 s a maze, and the explorer round that runs to its end saves some of it back.
… again The firmware's planner now prices a turn entered below its speed at a blend of the two, and a turn that leaves slower for the next one at nothing more (682373a). That changes the routes, so the baseline v1 is recorded again. The contest stays clean with the fan: every fast run equal or faster, by up to 100 ms on japan2017ef, and the search at 65 to 118 s. With every switch on, the racing line is drawn through different routes: japan2017ef is 70 ms faster but its board now grazes two walls at about 1.5 m/s on every seed, while apec2016, which grazed on about half the seeds, does on one in eight. Four other mazes are 10 to 30 ms slower there.
…ne recorded again The firmware answers the second review of #54 (72651b0): the racing line keeps 13 mm whatever the switch, a straight with the wall ahead unknown is held to the search speed only up to the point the robot can still stop from, the planner's preparation is charged for its real cost, and the time scale, the localizer's range check, the IMU status and blinded wall sensors are fixed. That changes the search, so the baseline v1 is recorded again. The contest stays clean with the fan, every fast run the same, and the search takes 867 s over the ten mazes instead of 911 s, none slower. With every switch on, japan2017ef no longer grazes, 70 ms slower, and it and apec2016 are clean on eight seeds of the fast run; the other mazes run as before.
… the baseline recorded again The firmware answers the third review of #54 (82585b5): the time scale rises with the right sign and not while the motors saturate, the planner's preparation is charged only for the edges it times, and with risky the racing line goes only through the route planned without the risky turns. Then it brakes no harder than the motors can (3489bde), which leaves today's robot byte identical. The search changes, so the baseline v1 is recorded again. The contest stays clean with the fan and with every switch, with the same fast runs; the search takes 859 s over the ten mazes instead of 867 s, from 64 to 118 s per maze, and the planning with every switch half as long.
…e recorded again The firmware answers the fourth review of #54 (4deedba): a braking that does not fit its distance scales the tire and motor limits together, the speed that can be braked from takes a closed form or eight Newton steps, the motor limited braking distance never comes out negative, and the dark reading of each wall sensor is in the pool. Today's robot brakes harder with its motors than its tires allow, so it runs as before: the states and every value of the baseline are unchanged, and only the data hash moves with the new columns, so the baseline v1 is recorded again. The stack of #55 and #54 is merged, so the submodule now follows the firmware's main (4dac6e2, whose tree is 4deedba's) instead of high-level-review, and the three accessors the simulator reads are upstream. The gate passes on main with both checked runs byte identical to the recorded ones.
MICRAS_EXACT=1 makes compare-baseline pass --exact to baseline.py, so a checked run whose data.csv moved by a single byte fails the gate. It is the check for a change that must not alter behavior, such as a refactoring.
A chip behind an SPI chip select is a SpiDevice: select, exchange and deselect, and the SPI mode it answers in. Board::spi_device attaches one by the bus handle and the chip select, since hspi3 carries the IMU and both encoders. The host Spi routes every transfer between select_device and unselect_device to the selected device, so a register access split over several HAL calls is one transaction. start_transfer completes when the host clock passes the time its bytes take at the bus's bit rate, which the fake Cube layer now gives SPI3 (125 MHz kernel clock, prescaler 32), and ends through on_transfer_end as the DMA interrupt does. A device addressed in another mode, or a chip select with no device, answers all ones.
A column source may name its columns only once the run has gone some way: the firmware's variables exist once its robot is constructed, which a real chip driver's start-up waits delay by several ticks. ColumnSource::ready says so, the recorder writes no row before every source is ready, and the Micras variables are ready once Micras::get_instance() returns the robot. A source that is ready from the start records as before.
The Micras target now compiles the firmware's imu.cpp and rotary_sensor.cpp and ST's lsm6dsv_reg.c, and its own replacements are gone. The bindings attach an Lsm6dsvModel and two As5047uModels to hspi3 by the chip selects target.hpp names, and turn each sample of the IMU device back into rad/s and m/s^2 for Lsm6dsvModel::push_sample; the encoders' position stays on the timer encoders. The sample port had no other user and is deleted. The firmware's objects outlive the target, so unwire forgets the ports and with them the chips. Recorded as baselines/v2, the gate's baseline now, because the behavior moved: - the real Imu reads the burst it started one update() earlier, so a sample reaches the firmware one loop later; - its constructor waits 10 ms, then 30 ms after the software power-on reset, so the robot exists 40 ms into the run: INIT and IDLE move from 0.000125 s and 0.00025 s to 0.040125 s and 0.04025 s, and the CSV starts at tick 320; - the scale of every IMU reading comes from the full scale the firmware wrote into CTRL6 and CTRL8. Every value stays within v1's tolerances: the explore reaches the goal at 26.891 s instead of 26.880 s, with a maximum pose error of 12.90 mm instead of 12.78 mm.
A project that adds the simulator with add_subdirectory keeps its own targets, cache and language settings: the style targets are prefixed micras_sim_ and exist only at top level; the build type is defaulted only at top level and never forced; the dependencies' switches are normal variables; paths are anchored to the simulator's project; the engine libraries require C++23 of their consumers. The options are named MICRAS_SIM_*, and the tests and the robot targets default to on only at top level. MICRAS_SIM_TOOLS_DIR and MICRAS_SIM_MAZES_DIR tell a consumer where the tools and the mazes are. ImGui's target is micras_sim_imgui. Every dependency is pinned by commit and fetched as that commit's archive with its SHA-256, which needs no history. ST's LSM6DSV driver is declared by the Micras target, which also adds the chip models, since only it uses them.
A target that lives in another repository keeps its analysis plugin next to its code, so meta.json records the target's folder (Target::directory) as target_dir, and analyze.py and baseline.py look for the plugin in --plugin, then $MICRAS_SIM_PLUGIN, then <target_dir>/tools/analysis.py, then targets/<target>/tools/analysis.py. baseline.py takes --plugin on record and compare; without its plugin a summary would lose the robot's values and the comparison would fail. compare_run.py ignores the fields that name paths, which differ between two checkouts of the same code.
A target's program is called with the FirmwareThread it runs on, so a target that drives the handover itself yields on it instead of keeping a pointer from wire(). CliOption, Wiring and WorldInfo move into micras/sim/app/wiring.hpp, and the README lists the headers a target may include, which are the public interface.
The style and build files are micras-lib's canonical copies, byte for byte: .clang-format, .clang-tidy, tests/.clang-tidy, cmake/micras_warnings.cmake and the lint script's template. The style targets run clang 22's tools by their versioned names and fail the configure without them, and micras_sim_lint_fix applies the fixes. Everything is formatted again with clang-format 22. The Micras target's .clang-tidy leaves the firmware's headers and the host HAL's to the repositories that lint them, and hal_host/ is formatted here but linted in micras-lib, which holds its copy.
The three repositories test with one framework, doctest, at the commit micras-lib uses. The engine's tests, the Micras target's and the chip models' keep their cases and their names (Suite.Case), each discovered by CTest through doctest_discover_tests: fixtures set up in their constructors, EXPECT and ASSERT became CHECK and REQUIRE, a near comparison checks the distance, and a double comparison allows a relative 1e-12. GoogleTest leaves the dependencies.
Every target gets micras_apply_warnings, the list the firmware and micras-lib compile with, instead of -Wall -Wextra, and MICRAS_SIM_WERROR makes the warnings errors. What they found is fixed without changing a result: the float to double promotions and the sign conversions are written out, the flash port initializes every member, and a cast that changed nothing is gone. The fetched dependencies are SYSTEM, so their headers raise no warning in the simulator's code.
The shared configuration of clang-tidy 22 finds what version 18 did not. Every file includes what it uses, with the conditional headers inside the branch that uses them; containers are indexed with at(), and spans through micras::sim::at, a checked access, since std::span has none before C++26; helper functions are static instead of living in an anonymous namespace; iterator arithmetic goes through std::next and std::to_address; the float to double promotions clang reports are written out; the state names are built on first use; the tools report an exception instead of terminating. toml++ is included through toml.hpp, and a TOML table's operator[] is a lookup, which engine/src/.clang-tidy tells the checks. Every run and every tool output is byte identical.
The simulator has no task runner: each check and each Micras recipe is a CMake target that calls a bash script with every path as an argument, so no script guesses a build layout. micras_sim_check builds everything and runs the unit tests, micras_sim_check_options, micras_sim_check_generic and the toy target's micras_sim_toy_check, which proves on the toy that a window, a video and an idle bridge change nothing and compares its runs with its baseline. The Micras recipes keep the names the firmware will give them (sim_run_idle, sim_run_explore, sim_contest, sim_contest_all, sim_compare_baseline, sim_record_baseline, sim_check_flash, sim_check, sim_turn_designs, sim_robot_report, sim_wall_calibration, sim_serve, sim_watch), with MICRAS_SIM_EXACT, MICRAS_SIM_BASELINE and MICRAS_SIM_CONTEST_MAZES as cache variables, and their scripts live in targets/micras/scripts. The checks and the recipes exist only at top level. The presets are host (Debug with the address and undefined behavior sanitizers), host-release (RelWithDebInfo) and host-ci. The justfiles are gone.
The five libraries and their host tests live in micras-lib, which the firmware carries as a submodule in external/micras-lib and adds after the Cube project, whose board target the HAL links. micras-lib applies its warnings to its own targets; the firmware includes the same warnings module and applies it to micras_config, the executable and the tests under its own MICRAS_WERROR. The link line keeps micras_config, micras::nav, micras::comm, micras::proxy and the Cube objects in that order, and cmake/app_sources.cmake lists the application's sources and include directories once, so the robot image is byte for byte the same. The wall sensor layout is the robot's, so WallSensorsIndex is defined in config/constants.hpp.
.clang-format, .clang-tidy, tests/.clang-tidy and the lint script template are micras-lib's copies, byte for byte. clang-format-22, clang-tidy-22, run-clang-tidy-22 and clang-apply-replacements-22 are found by those names, so that another version on the PATH can move neither the formatting rules nor the enabled check set, and configuring fails clearly without them. The script receives the ARM sysroot and C++ library directories as extra arguments, and lints the firmware's own sources: the libraries' headers under external/ are micras-lib's to lint.
The build stage starts from the host stage micras-lib, micras-simulation and the firmware share line for line (Ubuntu 26.04, GCC 15, clang 22, CMake, Ninja, Doxygen), adds the ARM toolchain, and configures with MICRAS_WERROR and MICRAS_LIB_WERROR on. The robot build needs only micras-lib's sources, so the context leaves out micras-simulation and the submodules' git links and build trees, and keeps the notes in docs/.
Every workflow checks out micras-lib and runs on a push to any branch and on pull requests, so the restructure branches are checked before anything is merged; the documentation is published and the binaries attached to a release only from main and from a published release. The libraries' host tests run in micras-lib's CI. The verified key covers the submodule commits through their gitlinks, which needs no submodule checkout. The standards job compares the shared style files and the host stage of the Dockerfile with micras-lib's.
micras-lib's stm32 backend now keeps the chip-specific code in one folder per family, H7 and G4, behind family functions. The v0 record no longer lists the instruction cache among what reviving the board needs, since the G4 family covers the part, and names the FMAC its STM32CubeMX project still lacks.
The simulation records these names in its runs and baselines, so a static_assert pins each one where the states are declared, and sim/ reads them from there.
The .cpp files of sim/ carry no comments, so Doxygen reads the simulation's headers and README only.
Third-party code is pinned by commit, as micras-lib pins the same theme.
…the family header micras-lib derives the H7 storage address from the family flash header, which removes its start-up read of the flash size register; micras-simulation names the tire material in American English and caches its fetch stamps. The simulation's runs are unchanged.
micras::state_names is sized by NUMBER_OF_STATES, so a state added without a name leaves an empty entry that no per-name static_assert catches; one static_assert over the whole table does.
…ions and race-free socket tests
GabrielCosme
commented
Sep 30, 2026
Comment on lines
+211
to
+221
| struct WallSensorsIndex { | ||
| /** | ||
| * @brief Index of the sensor named after where it looks. | ||
| */ | ||
| ///@{ | ||
| uint8_t left_front{}; | ||
| uint8_t left{}; | ||
| uint8_t right{}; | ||
| uint8_t right_front{}; | ||
| ///@} | ||
| }; |
Member
Author
There was a problem hiding this comment.
Estranho isso declarado aqui
Member
Author
There was a problem hiding this comment.
entendi agora, esse tipo só existe pro micras, talvez poderia ser só um enum
GabrielCosme
commented
Sep 30, 2026
| * @brief Designs of the turns of two bends, for each margin. | ||
| * | ||
| * @note Written by the turn designer of the simulator (`just micras turn-designs`), which searches | ||
| * @note Written by the turn designer of the simulation (the `sim_turn_designs` target), which searches |
Member
Author
There was a problem hiding this comment.
Estranho esse arquivo existir, achava que isso era gerado em tempo de compilação
GabrielCosme
commented
Sep 30, 2026
GabrielCosme
commented
Sep 30, 2026
| @@ -0,0 +1,42 @@ | |||
| /** | |||
Member
Author
There was a problem hiding this comment.
Talvez faz mais sentido um target custom pra simulação ao invés desses mocks, mas vale a discussão
test_bluetooth finds the module's baud rate, reads its settings, configures it on request and answers text lines over the radio, and tools/bluetooth_check.html is the other end. The documentation gains the HM-19 baud index, the settings that keep the serial stream clean and how the module handles writes.
The PIN lives in config/bluetooth_pin.hpp, which is not under version control, and the module bonds with each computer that enters it.
GabrielCosme
force-pushed
the
refactor/restructure
branch
from
October 4, 2026 05:50
b4c5e07 to
2e9641b
Compare
The DMA streams of ADC2, ADC3 and UART4 and the interrupts of the converters are disabled in the board's project, and micras-lib observes those transfers by polling. The wall sensors' ADC1, the addressable LEDs' TIM8 and the IMU's SPI3 keep their interrupts. test_wall_sensors also exposes the dark readings, the update count and whether the sensors started.
GabrielCosme
force-pushed
the
refactor/restructure
branch
from
October 4, 2026 06:28
2e9641b to
d31cdda
Compare
The STSPIN958 accepts up to 500 kHz.
The VBAT pin of the microcontroller does not reach the pack on this board, so the measurement is commented out where the robot uses it, and battery_config stays for a board that connects it.
GabrielCosme
force-pushed
the
refactor/restructure
branch
from
October 4, 2026 08:05
80a041a to
71684f7
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The firmware becomes the integrator. The libraries come from the new micras-lib repository and the simulator from micras-simulation, both as submodules under
external/. The Micras simulation target lives here insim/, with its history, next to the code whose behavior its baselines record.Building
git submodule update --init external/micras-lib, thenmkdir build && cd build && cmake .. && make, as before, orcmake --preset stm32.git submodule update --init, thencmake --preset hostandcmake --build --preset host. The host preset takes a dispatch at the top of the root CMake and buildssim/on micras-lib's host HAL and chip models. It never includes the ARM toolchain, so exceptions stay on for the simulator's run control.sim_run_idle,sim_run_explore,sim_contest,sim_check,sim_compare_baselineandsim_turn_designs.sim/README.mdlists them, the contract betweensim/and the app, and the common invocations ofbuild/host/sim/micras_sim.What moves
tests/host/andcmake/warnings.cmakeleave; the unit tests are in micras-lib now.WallSensorsIndexis inconfig/constants.hpp, and the HM-19 wiring is indocs/bluetooth.md.micras::Stateininclude/micras/states/names.hpp. A state without a name, or a renamed one, fails the build, because the baselines record those names.ImuandRotarySensorover the LSM6DSV and AS5047U models.sim/baselines/v2records what that changed: the IMU's 40 ms of start-up, and one extra update of latency on the IMU sample.Same behavior
CI
sim_check, and runs the contest on two mazes, or on all ten onmain.Before merging
main: switch the twobranch =lines of.gitmodulestomainand bump the submodules.Goes with Team-Micras/micras-lib#2 and Team-Micras/micras-simulation#1.