Skip to content

Remove trigger kwargs from the REST API response#67868

Open
potiuk wants to merge 1 commit into
apache:mainfrom
potiuk:redact-trigger-kwargs-rest-api
Open

Remove trigger kwargs from the REST API response#67868
potiuk wants to merge 1 commit into
apache:mainfrom
potiuk:redact-trigger-kwargs-rest-api

Conversation

@potiuk
Copy link
Copy Markdown
Member

@potiuk potiuk commented Jun 1, 2026

Why

TriggerResponse.kwargs returned the decrypted trigger keyword arguments verbatim — and only as a stringified Python dict, not a usable JSON value. Those kwargs can carry credentials a deferred operator hands to its trigger (an API key, a token, …), so the field both leaked secrets and wasn't machine-readable. It isn't consumed by the UI.

What

Per review, rather than masking the value, the kwargs field is removed from TriggerResponse entirely. The triggerer still decrypts and uses the real kwargs at runtime, so trigger execution is unaffected — only the API response no longer exposes them.

  • Regenerated the v2 REST + private-UI OpenAPI specs and the UI TypeScript client.
  • Updated the task-instance API tests (the deferred-state response no longer carries trigger.kwargs).

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 4.8)

Generated-by: Claude Code (Opus 4.8) following the guidelines

@boring-cyborg boring-cyborg Bot added the area:API Airflow's REST/HTTP API label Jun 1, 2026
potiuk added a commit to potiuk/airflow that referenced this pull request Jun 1, 2026
sensitive keys are masked here for consistency with how connection extras, variables
and rendered fields are already redacted.
"""
return str(redact(value))
Copy link
Copy Markdown
Member

@ashb ashb Jun 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will fundamentally break any trigger that gets caught by this. This is not merely display. It is changing the value that the Trigger class sees.

You say:

the triggerer still decrypts and
uses the real kwargs at runtime, so trigger execution is unaffected.

But how/where is that done?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But how/where is that done?

The redact_kwargs is only called explicitly and only in one place - in the "TriggerResponse" that is passed as the regular API response (not TaskSDK).

We are running it in BeforeValidator, which means that this redaction takes place before the response is prepared to be wired to be sent as response of API call.

https://pydantic.dev/docs/validation/latest/api/pydantic/functional_validators/#pydantic.functional_validators.BeforeValidator

I looked where TriggerResponse - and seems that this class is only used in responses in regular REST API.

or am I wrong @ashb ?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah no I was wrong.

So my other comment (about is this even worth returning stands) but if it does make sense to keep this:

This shouldn't have str() on it -- value is almost certainly a dict, which means this is relying on the stringification of a python dict, rather than keeping it as a json value.

@ashb
Copy link
Copy Markdown
Member

ashb commented Jun 2, 2026

Thinking about it, if this is part of the normal public API, not something used by the triggerer directly, perhaps we should just remove the trigger_kwargs from the response entirely?

Comment on lines +437 to +440
# The sensitive value is masked; the key name and non-sensitive values are preserved.
assert "super-secret-value-123" not in kwargs
assert "'gemini_api_key': '***'" in kwargs
assert "'polling_interval': 30" in kwargs
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# The sensitive value is masked; the key name and non-sensitive values are preserved.
assert "super-secret-value-123" not in kwargs
assert "'gemini_api_key': '***'" in kwargs
assert "'polling_interval': 30" in kwargs
# The sensitive value is masked; the key name and non-sensitive values are preserved.
assert kwargs == {"gemini_api_key": "***", "polling_interval": 30}

@potiuk potiuk force-pushed the redact-trigger-kwargs-rest-api branch from c851984 to bb487d3 Compare June 3, 2026 22:40
@potiuk potiuk changed the title Redact sensitive values in trigger kwargs returned by the REST API Remove trigger kwargs from the REST API response Jun 3, 2026
@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented Jun 3, 2026

Good call — dropped kwargs from TriggerResponse entirely rather than masking it. It only ever exposed the decrypted trigger kwargs (potential credentials) as a stringified dict, and it isn't read by the UI; the triggerer keeps decrypting and using the real kwargs at runtime, so nothing downstream changes. Regenerated the v2 + private-UI OpenAPI specs and the TS client, and updated the task-instance tests. Force-pushed the rewrite.


Drafted-by: Claude Code (Opus 4.8); reviewed by @potiuk before posting

required:
- id
- classpath
- kwargs
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmmm, though removing this field means it's a breaking change on the API, and could possibly cause down stream consumers to fail with "no such property" type errors

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. Thought about it - what we could do instead is to leave it in but make it always empty. That is far less of a breaking change - the schema would remain and we could simply state in a documentation that it is here for historical reasons but kwargs are never populated.

I think this is. a bit of "Have cake and eat it too".

TriggerResponse.kwargs returned the decrypted trigger keyword arguments
verbatim (as a stringified Python dict). Those kwargs can contain
credentials a deferred operator hands to its trigger (an API key, a token),
so the field leaked secrets.

Rather than removing the field (a breaking schema change for API consumers),
keep it but always return it empty as "{}" -- the same value an empty-kwargs
trigger already produced under the previous str() serialization, so the field
type and OpenAPI schema are unchanged. The triggerer still decrypts and uses
the real kwargs at runtime; only the API representation is emptied.
@potiuk potiuk force-pushed the redact-trigger-kwargs-rest-api branch from bb487d3 to 974f0a9 Compare June 4, 2026 23:23
@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented Jun 5, 2026

Thanks @ashb — reworked based on your feedback. Rather than removing the field (which, as you noted, would be a breaking schema change for downstream consumers), TriggerResponse.kwargs is now retained in the schema but always returned empty as "{}".

Addressing your specific points:

  • Not breaking: the field stays and keeps its existing str type, so the OpenAPI schema is unchanged — no missing-property errors for consumers, and no regenerated spec/UI client.
  • On the str() concern: I kept it a string deliberately (it has always been one) rather than switching to an object, to avoid a schema type change. Since the value is now a constant "{}", it no longer leaks the real kwargs and no longer depends on stringifying a meaningful dict — "{}" is exactly what an empty-kwargs trigger already serialized to under the old str() behavior.
  • The triggerer still decrypts and uses the real kwargs at runtime; only the API representation is emptied.

Added a unit test asserting sensitive kwargs are redacted to "{}", and updated the newsfragment. Could you take another look when you get a chance?


Drafted-by: Claude Code (Opus 4.8); reviewed by @potiuk before posting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants