feat: update Translator and AnalyzeText API support - #2696
Conversation
AB#2693 ## Summary`nAdd opt-in Translator Text API 2026-06-06 support for Translate, Transliterate, and Languages while preserving API 3.0 defaults and behavior. Update AnalyzeText to default to the 2024-11-01 GA API and add offline regression, schema, validation, fuzzing, and documentation coverage. ## Prompting Intent`nFix microsoft#2693 and microsoft#2694 in a new worktree. Preserve existing serialized and JVM behavior, implement the official version-specific wire contracts and removed-operation validation, avoid live-service tests, and prepare a merge-ready pull request against master. ## Linked Sources`n- Requirements: https://github.com/microsoft/SynapseML/issues/2693`n- Requirements: https://github.com/microsoft/SynapseML/issues/2694`n- Migration guide: https://learn.microsoft.com/azure/ai-services/translator/text-translation/how-to/migrate-to-2026-06-06`n- REST specification: https://github.com/Azure/azure-rest-api-specs/tree/main/specification/translation/data-plane/TextTranslation/stable/2026-06-06 ## Rationale`nKeep API 3.0 as the Translator default because 2026-06-06 is not wire-compatible. Use a scalar persisted API-version parameter so Spark output schemas remain deterministic, branch request and response handling only for supported operations, and reject removed operations before network execution. Model Languages scopes and schemas per API version and retain the existing Translate controls where the new target schema has direct equivalents. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>`nCopilot-Session: 7a122689-457b-4d0d-b387-2fac644ee89b
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
|
Hey Rana Singh (@ranadeepsingh) 👋! We use semantic commit messages to streamline the release process. Examples of commit messages with semantic prefixes:
To test your commit locally, please follow our guild on building from source. |
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The Translator request-building logic can throw null-related runtime exceptions on common array-column inputs and should be corrected before approval.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Lite
Findings: 1
New issues introduced by this change (2)
| Severity | Finding |
|---|---|
 |
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/translate/TextTranslator.scala — TextAsOnlyEntity always constructs inputs using JsString(s) even when running against API… |
 |
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/translate/TextTranslator.scala — supportedApiVersions is built from a Set.mkString, which is not guaranteed to be stable across… |
What changed in this PR
Adds opt-in support for Azure Translator Text API 2026-06-06 across SynapseML’s Translator transformers while preserving API 3.0 as the default contract, and updates Azure Language AnalyzeText to default to the latest GA API version.
Changes:
- Introduces Translator API version selection (
3.0default, opt-in2026-06-06) with version-specific request/response handling forTranslateandTransliterate, plus a newLanguagestransformer. - Adds offline/core tests validating deterministic defaults, version gating for unsupported operations, and 2026 payload/response schema shaping.
- Updates
AnalyzeTextdefault API version to2024-11-01and documents Translator migration guidance.
| File | Description |
|---|---|
| docs/Quick Examples/transformers/cognitive/_Translator.md | Documents Translator API 3.0 default and opt-in 2026-06-06 behavior/compatibility constraints. |
| cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/translate/TextTranslator.scala | Implements version-aware Translator request building, response schema selection, and adds Languages transformer. |
| cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/translate/TranslatorSchemas.scala | Adds 2026 response wrapper schemas and language schema modeling. |
| cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/language/AnalyzeText.scala | Updates AnalyzeText default API version to 2024-11-01. |
| cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/translate/TextTranslatorCoreSuite.scala | Adds deterministic/default/version-gating tests and 2026 request/response schema tests. |
| cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/translate/TranslatorSuite.scala | Adds fuzzing suite coverage for the new Languages transformer. |
| cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/language/AnalyzeTextCoreOfflineSuite.scala | Verifies AnalyzeText default API version selection. |
Suppressed comments (1)
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/translate/TextTranslator.scala:317
v2026PayloadbuildstargetsandinputswithJsString(language)/JsString(textValue)without filtering nulls. IftoLanguageortextcomes from an array column containing nulls, this will throw at request-build time. Filter out null/blank entries (and fail fast if no valid targets remain).
val targets = getValue(row, toLanguage).map { language =>
JsObject(Map("language" -> JsString(language)) ++
targetScript.map("script" -> _) ++
deploymentName.map("deploymentName" -> _) ++
fallback.map("allowFallback" -> _) ++
action.map("profanityAction" -> _) ++
marker.map("profanityMarker" -> _))
}
val inputs = texts.map { textValue =>
JsObject(Map("text" -> JsString(textValue), "targets" -> JsArray(targets.toVector)) ++
sourceLanguage.map("language" -> _) ++
sourceScript.map("script" -> _) ++
sourceTextType.filterNot(_ == JsString("Plain")).map("textType" -> _))
}
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Rana Singh (ranadeepsingh)
changed the title
AB#2693 ## Summary Filter null text entries before building Translator request bodies, filter null and blank inputs for the 2026 API, reject 2026 requests without a valid target language, and make supported-version errors deterministic. ## Prompting Intent Review every unresolved and suppressed PR finding, address valid feedback without changing supported public contracts, add regression coverage, and keep the PR metadata aligned with both Translator and AnalyzeText changes. ## Linked Sources - Requirements: microsoft#2693 - Requirements: microsoft#2694 - Null request-body review: microsoft#2696 (comment) - Deterministic error review: microsoft#2696 (comment) - Suppressed null-target review: microsoft#2696 (review) ## Rationale Filtering invalid array entries at request construction prevents null-related exceptions while preserving valid text exactly. The v3 path keeps its established payload contract, while the 2026 path applies the stricter non-blank requirements of its nested input and target schema. Sorting supported versions avoids unstable error text across Scala and JVM implementations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7a122689-457b-4d0d-b387-2fac644ee89b
|
Addressed the suppressed null/blank-array finding in 2698459. Translator 2026 requests now filter null/blank text and target entries and reject rows with no valid target language; targeted regression tests cover mixed and all-invalid arrays. |
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
The implementation appears consistent with the stated compatibility goals and is backed by deterministic offline tests; the remaining review note is a minor determinism improvement to an error message.
Review tier: Lite
Findings: None
Issues resolved since last review (2)
| Severity | Finding |
|---|---|
|
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/translate/TextTranslator.scala — supportedApiVersions is built from a Set.mkString, which is not guaranteed to be stable across… View resolved comment |
|
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/translate/TextTranslator.scala — TextAsOnlyEntity always constructs inputs using JsString(s) even when running against API… View resolved comment |
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/translate/TextTranslator.scala:718
validateScopebuilds the allowed scopes as aSetand then interpolatesallowed.mkString(", ")into the error message. Iteration order forSetis not guaranteed, so this can make the exception message non-deterministic across Scala/JVM versions (similar to the earliersupportedApiVersionsordering fix). Use an ordered collection for message formatting (or sort before mkString) while still validating membership efficiently.
AB#2693 ## Summary Sort the allowed Translator Languages scopes before including them in validation errors and cover both API-version-specific messages with deterministic assertions. ## Prompting Intent Re-audit automated review feedback on the latest PR head, including suppressed findings, and address every valid issue before declaring the review complete. ## Linked Sources - Requirements: microsoft#2693 - Pull request review: microsoft#2696 (review) ## Rationale Languages scope membership remains set-based for efficient validation, while sorting only at message formatting preserves behavior and guarantees stable diagnostics across Scala and JVM implementations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7a122689-457b-4d0d-b387-2fac644ee89b
|
Addressed the latest suppressed review finding in d13d373. |
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
Translate’s 2026 payload builder rejects v3-only params based on whether they are set rather than whether they are meaningfully enabled, which can cause unnecessary failures for neutral/default values.
Review tier: Lite
Findings: None
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/translate/TextTranslator.scala:292
- In the 2026 payload path, this check rejects
includeAlignment/includeSentenceLength/suggestedFromwhenever the params are set, even if they’re set to their neutral defaults (e.g.,setIncludeAlignment(false)). That can cause an unnecessary failure during migration/config standardization. Prefer rejecting only when a non-default value is actually requested (true / non-blank).
AB#2693 ## Summary Allow explicitly configured neutral values for v3-only Translate controls when using API `2026-06-06`, while continuing to reject enabled alignment, sentence-length, or suggested-source behavior. ## Prompting Intent Continue auditing current-head automated review feedback, including suppressed findings, and resolve compatibility issues before finalizing the pull request. ## Linked Sources - Requirements: microsoft#2693 - Pull request review: microsoft#2696 (review) ## Rationale Checking resolved parameter values rather than only whether a parameter was explicitly set preserves migration and configuration-standardization workflows. Neutral false or blank values do not request removed API behavior, while true or non-blank values still fail before network execution. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7a122689-457b-4d0d-b387-2fac644ee89b
|
Addressed the latest suppressed compatibility finding in f8329dd. Translator 2026 now rejects v3-only controls only when they request non-default behavior; explicitly set |
|
/azp run |
1 similar comment
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
1 similar comment
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
It introduces version-dependent request/response schemas and a new public transformer in externally-facing Azure service integrations, which warrants final human validation.
Review tier: Lite
Findings: None
Summary
Modernize the Azure AI language service integrations covered by #2693 and #2694:
2026-06-06support forTranslateandTransliterateLanguagestransformer with API-version-specific scopes and schemasAnalyzeTextto use the GA2024-11-01API by defaultTranslator compatibility
Translator API
3.0remains the default so existing pipelines keep their serialized parameters, request bodies, and output schemas.When
apiVersionis set to2026-06-06:inputspayload and wrappedvalueresponsesValidation
UnitTests translate: 59 passed, 0 failedUnitTests language: 98 passed, 0 failed234539362: all checks passedTranslateloaded from the cognitive jar built from this PRThe Translator request and response contracts were checked against the stable Azure Translator
2026-06-06OpenAPI specification and migration documentation. Fabric validation proves managed-runtime and exact-jar compatibility; live Translator2026-06-06service behavior is covered by deterministic protocol tests rather than a credentialed Fabric service call.Closes #2693
Closes #2694