[doc] Update stages doc with top-level stages by martin-velay · Pull Request #597 · lowRISC/mocha

martin-velay · 2026-06-03T16:47:04Z

No description provided.

martin-velay · 2026-06-03T16:47:32Z

marnovandermaas

Thank you so much for this PR. I really enjoyed the motivation sections you added. I put some of my thoughts in here, which I hope are useful.

marnovandermaas · 2026-06-05T12:21:55Z

+The two key documents governing chip-level verification are:
+
+- **Verification plan** (primary): [`hw/top_chip/dv/data/top_mocha_vplan.hjson`](../../hw/top_chip/dv/data/top_mocha_vplan.hjson) - defines the coverage metrics and their mapping to tests.
+- **Testplan**: [`hw/top_chip/data/chip_testplan.hjson`](../../hw/top_chip/data/chip_testplan.hjson) - captures individual testpoints and their associated tests.


This also doesn't exist yet right?

Not yet indeed, I am also already working on it

It would be good not to have a dead link so maybe you can say it will live at that path eventually.

marnovandermaas · 2026-06-05T12:30:21Z

+|-----------|----------|----------------|
+| V0 | Initial Work | <ul> <li> Chip-level testbench being set up </li> <li> Chip-level verification plan being written </li> <li> Chip-level testplan being written </li> </ul> |
+| V1 | Smoke Passing | <ul> <li> All IPs smoke-tested at chip level </li> <li> Testbench infrastructure validated </li> <li> CI smoke regression running </li> </ul> |
+| V2 | Integration Complete | <ul> <li> All planned chip-level tests passing </li> <li> All chip interfaces connected to an active agent and exercised end-to-end </li> <li> End-to-end interrupt routing confirmed for all interrupt-capable IPs </li> <li> Cross-IP integration paths and reset sequences exercised </li> <li> Chip-level coverage targets met </li> </ul> |


Can we say that the tests are passing with at least 90% of random seeds?

Here I wanted to distinguish tests according to their Milestone. If they are mapped as V1 in the testplan then we should have 100% of them to signoff V1. For V2 we should have all V1 and all V2.

One of the things we try to do with V2 is get to a very high level of verification but not necessarily go into the long tail of trying to hit 100% everywhere. This is why I suggest making it 90% especially in the case of full randomisation. This also aligns with our block-level definitions.

I think it differs from the block-level philosophy in the sense that top-level tests are directed and the randomization has barely no effect. So I think the tail is not that long and also, here we talk only about test passing rates, not about any other coverage metrics. Do you still think a wording adjustment is preferable?

marnovandermaas · 2026-06-05T12:35:00Z

+| **Item name** | **Description** |
+|---------------|-----------------|
+| TOP_DV_DOC_DRAFTED | DV document drafted covering testbench architecture, agent topology, firmware-driven stimulus model, and chip-level coverage intent. |
+| TOP_VPLAN_COMPLETED | Verification plan (`top_mocha_vplan.hjson`) complete with the metric-to-test mapping for each coverage item and milestone specified. Reviewed by designers, a peer DV engineer, firmware author, and chip architect. |


I think completed is too ambitious here. I would like drafted here. That doesn't mean it cannot be complete but I think it might prevent us from doing the sign-off with a vplan that is good enough.

It is maybe ambitious, but it should be among the first task we complete as the DV is then driven by that. I'd like to keep it as it is more or less. Also, a point to raise, having a vPlan written and reviewed doesn't mean that it shouldn't move at all. Of course in reality we'll have spec updates or we'll notice that we were wrong in the vPlan. Other I could reformulate slightly in that sense, but i feel that if we write "draft" people will assume that they only need to create the file with few titles, IMO it should be more than 90% done and a review with the stakeholders should have been done already. I don't want to end up driving the DV in a wrong direction or miss really major points. Does it sounds OK for you?

I think drafted may actually not be the worst word here, but I agree we want more than just a file with a few titles. Maybe you can describe what needs to be achieved at this stage in the description?

I have edited the text in my last push into:

Verification plan (top_mocha_vplan.hjson) substantively complete (>90% of intended scope) with the metric-to-test mapping for each coverage item and milestone specified. Reviewed and approved by designers, a peer DV engineer, firmware author, and chip architect. Further evolution is expected as the design matures; the review ensures no major coverage gaps or methodological misalignments remain before DV execution begins.

Is it OK for you?

The wording is much better. We may also want to reduce the number of people this needs to be reviewed by. I would make it be 3 reviewers and not necessarily specify the roles those people need to have.

OK, I'll still add a note to say that it's preferable to get review from people from the different teams such as DV, DD and SW. This is a common perquisite as we need to make sure we're all aligned and nobody will complain about an obvious hole from their perspective later when it's too late

I've pushed my changes, I let you take a look

marnovandermaas · 2026-06-05T12:35:08Z

+|---------------|-----------------|
+| TOP_DV_DOC_DRAFTED | DV document drafted covering testbench architecture, agent topology, firmware-driven stimulus model, and chip-level coverage intent. |
+| TOP_VPLAN_COMPLETED | Verification plan (`top_mocha_vplan.hjson`) complete with the metric-to-test mapping for each coverage item and milestone specified. Reviewed by designers, a peer DV engineer, firmware author, and chip architect. |
+| TOP_TESTPLAN_COMPLETED | Chip-level testplan (`chip_testplan.hjson`) complete with at least one testpoint per integrated IP. Reviewed by designers, a peer DV engineer, firmware author, and chip architect. |


Same as for vplan.

I have edited the text in my last push into:

Chip-level testplan (chip_testplan.hjson) substantively complete (>90% of intended scope) with at least one testpoint per integrated IP. Reviewed and approved by designers, a peer DV engineer, firmware author, and chip architect. Further evolution is expected as the design matures; the review ensures no major testpoint gaps remain before DV execution begins.

Is it OK for you?

marnovandermaas · 2026-06-05T12:37:15Z

+| TOP_BOOT_INFRA_PASSING | The SW-to-DV pass/fail signalling mechanism is confirmed working before any other firmware-driven test result is trusted. |
+| TOP_ALL_TESTS_PASSING_V1 | All V1 testpoints in the testplan passing. |
+| TOP_VPLAN_COVERAGE_V1 | All V1 items defined in the verification plan achieved. |
+| TOP_SMOKE_REGRESSION_IN_CI | V1 smoke suite runs automatically on PRs touching top-level RTL or testbench and failures block merge. |


Can't we do the same with out current nightly setup?

You mean without it being part of the CI? I think it'd be good to have UVM top tests running for CI at some point (maybe in a private one). Don't you think so?

Yes, we want to run CI, but currently they only run in the nightly and not on each PR. I just don't want to say that we have to run it on every PR when we can probably make due with the CI we currently have right?

martin-velay · 2026-06-15T11:47:45Z

@marnovandermaas,

I have rebased on the main in the 1st push.
The 2nd and 3rd push address some comments. Let me know if you are happy with it? In that case could you mark the related comments as resolved?

Signed-off-by: martin-velay <mvelay@lowrisc.org>

martin-velay force-pushed the top_level_stages branch from 2d50ddb to f094b94 Compare June 4, 2026 06:20

marnovandermaas reviewed Jun 5, 2026

View reviewed changes

martin-velay force-pushed the top_level_stages branch from f094b94 to 0bf8c92 Compare June 12, 2026 15:30

martin-velay marked this pull request as ready for review June 12, 2026 15:42

martin-velay force-pushed the top_level_stages branch 3 times, most recently from 3a5f103 to 274ac44 Compare June 15, 2026 11:46

martin-velay force-pushed the top_level_stages branch from 274ac44 to 0cfecc9 Compare July 9, 2026 14:38

martin-velay requested a review from marnovandermaas July 9, 2026 14:40

martin-velay force-pushed the top_level_stages branch from 0cfecc9 to 4c31f0b Compare July 9, 2026 15:51

[doc] Update stages doc with top-level stages

548a6df

Signed-off-by: martin-velay <mvelay@lowrisc.org>

martin-velay force-pushed the top_level_stages branch from 4c31f0b to 548a6df Compare July 10, 2026 15:46

Uh oh!

Conversation

martin-velay commented Jun 3, 2026

Uh oh!

martin-velay commented Jun 3, 2026

Uh oh!

marnovandermaas left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

martin-velay Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

martin-velay commented Jun 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

martin-velay Jul 9, 2026 •

edited

Loading