Skip to content

CI: Write the AAS schemas where the tests read them - #639

Open
LGUIUX wants to merge 2 commits into
eclipse-basyx:developfrom
LGUIUX:fix/ci-schema-path
Open

LGUIUX wants to merge 2 commits into
eclipse-basyx:developfrom
LGUIUX:fix/ci-schema-path

Conversation

@LGUIUX

@LGUIUX LGUIUX commented Sep 20, 2026 •

Copy link
Copy Markdown

Fixes #637.

pr.yml downloads the schema files into ./test/adapter/schema, while the tests read ../schemas/, that is test/adapter/schemas, the path .gitignore already covers (L27). The files never reach the tests, so JsonSerializationSchemaTest and XMLSerializationSchemaTest skip their whole class on every run: that is the skipped=2 in a green build.

Renaming the directory in the workflow is enough to make them run.

Testing

Reproduced the CI layout locally, with the schemas in test/adapter/schema, and both classes skip:

setUpClass (…JsonSerializationSchemaTest) ... skipped 'JSON Schema does not exist at …/test/adapter/json/../schemas/aasJSONSchema.json'
setUpClass (…XMLSerializationSchemaTest) ... skipped 'XSD schema does not exist at …/test/adapter/xml/../schemas/aasXMLSchema.xsd'
OK (skipped=2)

With the files in test/adapter/schemas the 13 tests in those classes run. Two of them fail on develop, which is #636, fixed separately; with both changes the suite is green (282 tests).

Two things I left out on purpose, happy to add either if you want them here rather than in a follow-up: making the absence of the schema fatal in CI (a missing file currently turns into a green build), and pointing the workflow and the tests at one shared constant so the two paths can't drift again.

@hpoeche

hpoeche commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Thank you for spotting and fixing this mistake. As you pointed out in #637, the root cause here is that the test are skipped silently in the CI.
We are currently implementing unittests for the server package, where we focused a similar situation with the integration tests: It is okay to skip them in local dev environments, but the CI must not skip them but fail. We solved this by introducing an environment variable that prevents skipping the tests when set.
You can take a look at our implementation here:
https://github.com/rwth-iat/basyx-python-sdk/blob/aeaa5063ff214012529c4fcc25d0f83930189a6b/server/test/_helper/test_helpers.py#L27

If @s-heppner agrees, it would be very helpful to implement this approach here too.

The workflow downloads them into `test/adapter/schema`, while the tests read
`test/adapter/schemas`, the path `.gitignore` already covers. The files
therefore never reach the tests, `JsonSerializationSchemaTest` and
`XMLSerializationSchemaTest` skip their whole class on every run, and CI has
been reporting `OK (skipped=2)` while validating nothing against the schema.

Signed-off-by: LGUIUX <phdl@outlook.es>
The schema validation tests skip their whole class when the schema files
are missing. That is right for a local run, but it is also what let the
misplaced download in CI go unnoticed.

Follow the approach of the server's Docker integration tests: with
`REQUIRE_SCHEMA_TESTS` set, a missing schema file is an error instead of a
skip. `pr.yml` sets it for the unittest step; local runs are unaffected.

Signed-off-by: LGUIUX <phdl@outlook.es>
@LGUIUX

LGUIUX commented Sep 22, 2026

Copy link
Copy Markdown
Author

Thanks, that addresses the actual root cause better than the path fix alone. I've added it here as a separate commit, so it's easy to drop if @s-heppner would rather keep this PR to the path:

  • test/_helper/test_helpers.py gets REQUIRE_SCHEMAS, read from REQUIRE_SCHEMA_TESTS, same shape as your REQUIRE_SERVER_INTEGRATION_TESTS.
  • JsonSerializationSchemaTest and XMLSerializationSchemaTest raise in setUpClass when it's set and the schema file is missing. Otherwise they skip as before.
  • pr.yml sets it on the unittest step.

Checked from sdk/: without the schema files and with the variable set, both classes error out (FAILED (errors=2)); unset or 0, they skip as before. With the files in place this branch still shows the two #636 failures, so #638 needs to go in first; with both, the suite is green (282 tests).

The CouchDB tests skip the same way when the service isn't reachable. I left them alone to keep this about the schemas, but the same switch would work there if you want it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants