Development: Reuse compliance analysis across languages via snippet mapping - #2470
Development: Reuse compliance analysis across languages via snippet mapping #2470ge94zec wants to merge 31 commits into
Development: Reuse compliance analysis across languages via snippet mapping #2470Conversation
…rget text - added endpoint POST that maps source issues to a target language - added prompt that returns one mapped text snippet per line in source complianceIssues order, other fields carry over - wired compliance mapping through client → server API - separated compliance analysis from score calculation - extracted save flow in JobService into Consumer - improved performance by eliminating redundant analysis steps in job creation
github-actions
Bot
added
server
Not up to standards ⛔🔴 Issues
|
| Category | Results |
|---|---|
| Complexity | 1 medium |
🟢 Metrics 8 complexity
Metric Results Complexity 8
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
|
📊 Server Test Coverage Too Low 🔍 View coverage locally: ./gradlew test jacocoTestReport
open build/reports/jacoco/test/html/index.html🌐 View coverage from GitHub: |
|
📊 Server Test Coverage Too Low 🔍 View coverage locally: ./gradlew test jacocoTestReport
open build/reports/jacoco/test/html/index.html🌐 View coverage from GitHub: |
|
🤖 No OpenAPI or client changes needed. |
ge94zec
changed the title
Chore: Reuse compliance analysis across languages via snippet mapping Development: Reuse compliance analysis across languages via snippet mapping
- updated openapi - added comment to AiService - simplified AiService mapping - updated prompt
ge94zec
changed the title
Development: Reuse compliance analysis across languages via snippet mapping Development: Reuse compliance analysis across languages via snippet mapping
|
📊 Server Test Coverage Too Low 🔍 View coverage locally: ./gradlew test jacocoTestReport
open build/reports/jacoco/test/html/index.html🌐 View coverage from GitHub: |
|
📊 Client Test Coverage Too Low 🔍 View coverage locally: npm run test:ci
open build/test-results/vitest/coverage/index.html🌐 View coverage from GitHub: |
|
🤖 No OpenAPI or client changes needed. |
|
📊 Server Test Coverage Too Low 🔍 View coverage locally: ./gradlew test jacocoTestReport
open build/reports/jacoco/test/html/index.html🌐 View coverage from GitHub: |
|
🤖 No OpenAPI or client changes needed. |
|
📊 Client Test Coverage Too Low 🔍 View coverage locally: npm run test:ci
open build/test-results/vitest/coverage/index.html🌐 View coverage from GitHub: |
…or-compliance' into chore/2346-enhance-performance-for-compliance
|
📊 Server Test Coverage Too Low 🔍 View coverage locally: ./gradlew test jacocoTestReport
open build/reports/jacoco/test/html/index.html🌐 View coverage from GitHub: |
|
🤖 No OpenAPI or client changes needed. |
Not up to standards ⛔🔴 Issues
|
| Category | Results |
|---|---|
| Complexity | 3 medium |
🟢 Metrics 10 complexity
Metric Results Complexity 10
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
|
📊 Server Test Coverage Too Low 🔍 View coverage locally: ./gradlew test jacocoTestReport
open build/reports/jacoco/test/html/index.html🌐 View coverage from GitHub: |
|
🤖 No OpenAPI or client changes needed. |
# Conflicts: # src/main/java/de/tum/cit/aet/ai/service/AiService.java # src/main/webapp/app/job/job-creation-form/job-creation-form.component.ts
|
🤖 No OpenAPI or client changes needed. |
|
📊 Server Test Coverage Too Low 🔍 View coverage locally: ./gradlew test jacocoTestReport
open build/reports/jacoco/test/html/index.html🌐 View coverage from GitHub: |
|
📊 Client Test Coverage Too Low 🔍 View coverage locally: pnpm run test:ci
open build/test-results/vitest/coverage/index.html🌐 View coverage from GitHub: |
…nslated descriptions - remove the second compliance analysis for the translated description - reuse source-language compliance issues for snippet mapping - update target-language highlights from mapped issues
Not up to standards ⛔🔴 Issues
|
| Category | Results |
|---|---|
| Security | 1 high |
| Complexity | 1 medium |
🟢 Metrics 18 complexity
Metric Results Complexity 18
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
|
📊 Server Test Coverage Too Low 🔍 View coverage locally: ./gradlew test jacocoTestReport
open build/reports/jacoco/test/html/index.html🌐 View coverage from GitHub: |
|
🤖 No OpenAPI or client changes needed. |
|
🤖 No OpenAPI or client changes needed. |
|
There hasn't been any activity on this pull request recently. Therefore, this pull request has been automatically marked as stale and will be closed if no further activity occurs within seven days. Thank you for your contributions. |
|
🤖 No OpenAPI or client changes needed. |
- map source findings to exact translated snippets - preserve issue metadata and persist mapped findings - reject invalid mapping responses - refine compliance and snippet-mapping prompts - ignore empty mapped snippets during editor highlighting - simplify focused mapping tests
|
🤖 OpenAPI spec and client code auto-updated and committed. |
|
📊 Client Test Coverage Too Low 🔍 View coverage locally: pnpm run test:ci
open build/test-results/vitest/coverage/index.html🌐 View coverage from GitHub: |
|
🤖 No OpenAPI or client changes needed. |
…or-compliance' into chore/2346-enhance-performance-for-compliance
|
🤖 No OpenAPI or client changes needed. |
|
🤖 No OpenAPI or client changes needed. |
az108
left a comment
az108
left a comment
There was a problem hiding this comment.
Reviewed on top of Cathy's existing review — none of the points below overlap with hers.
The core idea is right: snippet mapping instead of a second full-text analysis is exactly the lever worth pulling here, and applyJobChangeForAnalysis quietly adds the first ownership check on this path — updateAiAnalysis had none on main. Raw List → List<ComplianceIssue> is a good catch too.
The prompt rewrite that ships with it is broken, though.
Blocking
1. {format} breaks every compliance analysis (AnalyzeComplianceText.st:59) — unresolved template variable, StTemplateRenderer defaults to ValidationMode.THROW. Reproduced against the real Spring AI 2.0.0-M4 jars; details inline.
2. Output contract no longer matches ComplianceIssue (AnalyzeComplianceText.st:52) — the prompt asks for text, category, suggestion, but there is no suggestion field anywhere in the project, and article / explanation / action are no longer requested at all. The compliance popover renders nothing but article and explanation.
3. {userLang} silently dropped (AnalyzeComplianceText.st:9) — the parameter is still passed from client through resource to service, and the JavaDoc still promises it controls the explanation language. It no longer does anything.
4. jobId has no Bean Validation (MapComplianceIssuesRequestDTO.java:13) — a request without it runs the LLM mapping, returns 200, and persists nothing without any error or log.
A general note on 1–3: the prompt rewrite looks unrelated to "reuse compliance analysis across languages via snippet mapping". Splitting it into its own PR would have surfaced these — the mapping logic itself is sound.
Test coverage
62be17d06 "removed tests" deletes AiServiceTest.java (418 lines) including five tests for mapComplianceIssues, with no resource-level replacement. That follows the no-*ServiceTest rule, but it leaves the empty-source-issues early return, the LLM catch branch, the size/null mismatch branch, and — most importantly — the silent dropping of non-verbatim snippets (continue after SnippetMatcher.isVerbatim) completely uncovered. SnippetMatcherTest only covers the trivial helper.
Things I checked and found fine
Listing these so they don't come up in a later round:
- LLM call before the ownership check in
mapComplianceIssues— consistent withanalyzeJobDescriptionandextractPdfData, and this PR improves the situation rather than worsening it.isAdminOrMemberOfreturnsvoidand throwsAccessDeniedException, so nothing is being silently ignored. let finalContent: string | null = null— the type is forced by the pre-existingextractTranslatedTextFromStream(): string | null; no lint rule againstnull, and it appears in 77 of 267 client files.- The
if (!jobId) returnmoved to the top oftranslateAndStoreOtherLanguage— both callers run after an awaitedsaveDraftwith no await in between, so it is unreachable defence, not a behaviour change. void-ed analysis promise inprocessDescriptionWithAi—analyzeAndUpdateScorehas nothing throwable outside itstry, andmainhad the same shape withvoid Promise.all([...]).- Fully-qualified
java.util.stream.IntStream— 19 comparable spots insrc/main/java, no checkstyle or Spotless gate on imports. postAndRead(..., Void.class, 400)withoutassertThat—postInvalidasserts the status itself; 81 test methods in the repo do it this way.toHaveBeenCalledWithinstead oftoHaveBeenCalledOnce— the rule targets call-count assertions; 468 of 546toHaveBeenCalledWithin the repo carry no count assertion.
| - `suggestion`: exact fix in {descriptionLanguage}; REPLACE = safe alternative, ADD = sentence to append, REMOVE = "". | ||
| - If compliant, return exactly []. | ||
|
|
||
| {format} |
There was a problem hiding this comment.
{format} has no matching .param(...). AiService.analyzeJobDescription passes only descriptionLanguage, userLang, jobDescription and title, and this PR does not touch that call site.
Spring AI's default renderer is StTemplateRenderer (DefaultChatClient.java:88 — DEFAULT_TEMPLATE_RENDERER = StTemplateRenderer.builder().build()) with DEFAULT_VALIDATION_MODE = ValidationMode.THROW (StTemplateRenderer.java:67), and SpringAIConfiguration.java:44 is a plain ChatClient.builder(chatModel).build() — no custom renderer, no relaxed validation. .entity() does not create a format template variable: doSingleWithBeanOutputConverter only puts the format into the request context under ChatClientAttributes.OUTPUT_FORMAT, and the user text was already rendered back at .call(). format is not an ST built-in either, so it is not skipped.
Reproduced against the real 2.0.0-M4 jars with both prompt versions and the four actual params:
PR -> IllegalStateException: Not all variables were replaced in the template.
Missing variable names are: [format].
MAIN -> RENDER OK, length=2895
The exception is caught by analyzeJobDescription, calls aiFeatureToggleService.recordFailure() (which also trips the AI circuit breaker) and surfaces as InternalServerException("Compliance analysis parsing failed"). It is not data-dependent — every invocation fails while the AI toggle is on. CI does not catch it: AiResourceTest mocks AiService, and nothing in the suite renders a real prompt.
Fix: drop this line. Spring AI appends the format instruction on its own, which is why no other prompt in the repo has {format}.
| {jobDescription} | ||
|
|
||
| OUTPUT | ||
| Return only a minified one-line JSON array; no markdown or prose. Use exactly these fields: |
There was a problem hiding this comment.
The output contract no longer matches the type this is deserialized into. analyzeJobDescription still parses List<ComplianceIssue>, and ComplianceIssue has id, category, text, article, explanation, action, language — there is no suggestion field anywhere in src/main/java or the generated client. So suggestion is silently discarded by the converter (FAIL_ON_UNKNOWN_PROPERTIES is off), while article, explanation and action are no longer requested at all. main asked for them explicitly: "Object fields must be exactly: id, text, category, article, explanation, action".
This is not cosmetic. ai-compliance-popover.component.html:7-8 renders nothing but {{ issue()?.article }} and {{ issue()?.explanation }}, and job-creation-form.component.ts:338 uses issue.explanation for titleComplianceError. Both end up empty or filled with whatever the model happens to invent from the JSON schema.
Either keep the previous field list, or add suggestion to ComplianceIssue and migrate the UI in the same PR.
| - Match meaning case-insensitively; examples are non-exhaustive. | ||
| - Check every occurrence against all four categories and return every match. Do not narrate the analysis. | ||
|
|
||
| LANGUAGES |
There was a problem hiding this comment.
{userLang} is gone from the template — main had The explanation must be written in: {userLang}. here — but the parameter is still threaded through the entire chain: the client sends it (job-creation-form.component.ts:1874), AiResource.analyzeJobDescription takes @RequestParam(defaultValue = "en") String userLanguage, and AiService still calls .param("userLang", userLang). The JavaDoc still states it "controls the language of explanation texts in the returned issues".
StTemplateRenderer.validate only reports missing variables, never surplus ones, so this fails silently — the user-language feature is simply gone while the API signature and docs still promise it. Either wire the variable back into the prompt or remove the parameter from client, resource, service and JavaDoc.
| @JsonInclude | ||
| public record MapComplianceIssuesRequestDTO( | ||
| String toLang, | ||
| UUID jobId, |
There was a problem hiding this comment.
jobId is the only field here without validation, while translatedText and complianceIssues carry @NotBlank / @NotNull. applyJobChangeForAnalysis opens with if (jobId == null) { return; } — no log, no exception. A request without it runs the expensive LLM mapping, returns 200 with the mapped issues, and persists nothing. The empty-source-issues early return is affected too. The generated OpenAPI model has readonly jobId?: string, so a spec-compliant caller is allowed to omit it.
@NotNull UUID jobId is what the server guidelines document — the input-DTO example in server-development.mdx (Request Validation) is literally public record CreateApplicationDTO(@NotBlank String motivation, @NotNull UUID jobId) {} — and what AssignSlotRequestDTO, BookSlotRequestDTO and UpdateApplicationDTO already do.
3 participants
Checklist
General
Server
Client
Motivation and Context
Closes: #2346
Description
This change improves the bilingual AI compliance workflow for job descriptions by replacing the second target-language compliance analysis with a snippet mapping. The editor now analyzes the source language once in
analyzeJobDescription, streams and stores the translated text intranslateTextStream, and then maps the detected compliance snippets onto the translated version automaticallymapComplianceIssues. This reduces duplicate AI work, keeps translated highlights in sync with sourceIssues.The duplicated compliance issue persistence logic in JobService was extracted into a shared helper. Since the Score is currently a combined value derived from both gender and legal analysis, this ensures the score remains consistent and is only updated when a full, valid recalculation is possible.
LLM compliance calls per autosave cycle is now reduced from 2 to 1. The translation with second analysis is reduced from 11s to approximately 2.30s.
Steps for Testing
Prerequisites:
Review Progress
Code Review
Manual Tests
Screenshots
Test Coverage
Client
Server
Last updated: 2026-08-17 21:48:40 UTC