Conversation
Command.run() times every module and measures how much the process's peak resident set size grew while it ran. Both are logged at INFO in one line per module and written to info.json under module_stats, in run order, so any run says which module cost what and CI can compare a branch against main. ru_maxrss is the process peak, so a module's growth is how much it raised the high-water mark, which is what matters on a host without swap; a module running after a bigger one reports 0. Where the resource module is unavailable the growth is reported as 0.
Contributor
Coverage Report •
|
||||||||||||||||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Command.run()records, for every module it runs, the wall time and how much the process's peak resident set size grew while the module ran. Each is logged at INFO in one line and the list is written toinfo.jsonundermodule_stats, in run order:Why
An analysis run on a memory-constrained host grew to the host's whole RAM and stalled it, and nothing in the run said which module cost what. With the numbers in every run's info file, a regression in a new or changed module shows up in CI (comparing
module_statsagainst the previous run on main) and in any case run.ru_maxrssis the process peak, so a module's number is how much it raised the high-water mark, which is what matters on a host without swap; a module running after a bigger one reports 0. On Windowsresourceis unavailable and the growth is reported as 0.Not in this change
A CI gate comparing the numbers against main, a per-module budget with a warning when exceeded, and a way to bound a run (thread count for parallel parsers, a memory limit a command applies to itself). Those need baselines from real runs first; this change produces them.
Tried on a real run
A full
check-sysdiagnoseover a 545 MB test-device sysdiagnose with private modules loaded, 74 modules: the stats put 2087 MB of the 2268 MB total peak growth on the one module that triggers the shared unified-log parse, and no other module above 145 MB. That is the attribution the issue wanted; the overhead of the accounting itself is not measurable against the run's 55 s.