feat(platform): add X/Twitter Ads client (OAuth 1.0a)#19
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds the first Phase-2 platform client: a standalone internal/platform/twitter/ package that ports the Express BFF's x-ads.service.ts to Go. It implements OAuth 1.0a (HMAC-SHA1) request signing with stdlib only and drives the X Ads campaign → line_item → promoted_tweet creation flow with name-based idempotency. Credentials and account config are injected via NewClient (never read from env/DB), keeping the package decoupled from the orchestrator until the wiring PR (#11) lands.
Changes:
- New X Ads client with OAuth 1.0a signing, micro-currency/ISO8601 helpers, 1 req/sec write delay, and 429 exponential-backoff retry that honors context cancellation.
- Public API
NewClient(...)→CreateCampaign(ctx, CampaignInput)returning a step-loggedCampaignResult, reusing existing campaigns/line items by name for idempotency. - 14 deterministic unit tests (OAuth signature golden value, currency round-trip, date formatting, UTM building, 429 retry/exhaustion, context cancellation, full create flow, idempotency).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| internal/platform/twitter/client.go | New OAuth 1.0a signer + X Ads campaign creation flow, retry/rate-limit handling, and conversion/idempotency helpers. |
| internal/platform/twitter/client_test.go | Unit tests covering signing determinism, validation, retry behavior, and the end-to-end create/idempotency flows via httptest. |
Notes for reviewers: One moderate correctness issue is flagged inline — X-Rate-Limit-Reset is a Unix epoch timestamp but is treated as a delay in seconds, so 429 retries relying on that header always wait the full 60s cap. This is also security-sensitive OAuth signing code whose golden signature value could not be independently verified during this review.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
1238a1d to
88fdcbb
Compare
88fdcbb to
7d5e9a8
Compare
7d5e9a8 to
060c9d4
Compare
060c9d4 to
93e6930
Compare
Address David's resilience finding on PR #20: doRequest treated a 429 the same as any other non-2xx and returned immediately, so a transient rate limit during CreateCampaign's sequential Graph API calls (campaign -> ad set -> ad) aborted the whole flow. - Retry a 429 up to retryMax (3) times with bounded backoff, honoring Retry-After (delay-seconds or HTTP-date) and capping any single wait at maxRetryWait (60s); fall back to exponential backoff otherwise. - Add an injectable clock (WithClock) for the HTTP-date path and an unexported withRetryBaseDelay so tests exercise retries without real wall-clock waits. - sleepCtx honors context cancellation during backoff. - Tests: retry-then-succeed, exhaust-after-retryMax, parseRetryAfter header matrix, and context-cancel-during-backoff (go test -race clean). Mirrors the retry/backoff approach in the Twitter client (#19). Signed-off-by: Misha Rautela <mrautela@linuxfoundation.org>
Address David's resilience finding on PR #22: doRequest mapped every non-2xx status, including 429, straight to a returned error with no retry. CreateCampaign drives several sequential Marketing API calls (campaign group, campaign, dark post, creative) - exactly the burst that trips a per-account rate limit - so a single transient 429 aborted the whole flow. - Retry a 429 up to retryMax (3) times with bounded backoff, honoring Retry-After (delay-seconds or HTTP-date) and capping any single wait at maxRetryWait (60s); fall back to exponential backoff otherwise. The final attempt returns the standard non-2xx error rather than looping. - Reuse the existing injectable clock (WithClock) for the HTTP-date path; add an unexported withRetryBaseDelay so tests exercise retries without real wall-clock waits. - sleepCtx honors context cancellation during backoff. The response body is now closed explicitly on every path (the retry loop precludes defer). - Tests: retry-then-succeed, exhaust-after-retryMax, parseRetryAfter header matrix, and context-cancel-during-backoff (go test -race clean). Mirrors the retry/backoff approach in the Twitter client (#19) and the matching fix in the Meta client (#20). Signed-off-by: Misha Rautela <mrautela@linuxfoundation.org>
First Phase-2 platform client: port x-ads.service.ts from the Express BFF to Go. - OAuth 1.0a HMAC-SHA1 request signing (stdlib crypto/hmac + crypto/sha1) - Campaign -> line item -> promoted tweet creation flow with name-based idempotency (findCampaignByName / findLineItemByName) - Micro-currency conversion, ISO8601 UTC dates, 1 req/sec write delay + 429 exponential-backoff retry (honors context cancellation) - Credentials are INJECTED (Credentials + AccountConfig structs) rather than read from env — in production they come from the decrypted stored connection (D7). Standalone package; the orchestrator adapter lands with the wiring PR. - Unit tests: OAuth1 signature (deterministic nonce/timestamp), currency round-trip, date formatting, 429-retry via httptest. Ports lfx-v2-ui apps/lfx-one/src/server/services/x-ads.service.ts. LFXV2-2640 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Misha Rautela <mrautela@linuxfoundation.org>
93e6930 to
7d78954
Compare
Address David's resilience finding on PR #20: doRequest treated a 429 the same as any other non-2xx and returned immediately, so a transient rate limit during CreateCampaign's sequential Graph API calls (campaign -> ad set -> ad) aborted the whole flow. - Retry a 429 up to retryMax (3) times with bounded backoff, honoring Retry-After (delay-seconds or HTTP-date) and capping any single wait at maxRetryWait (60s); fall back to exponential backoff otherwise. - Add an injectable clock (WithClock) for the HTTP-date path and an unexported withRetryBaseDelay so tests exercise retries without real wall-clock waits. - sleepCtx honors context cancellation during backoff. - Tests: retry-then-succeed, exhaust-after-retryMax, parseRetryAfter header matrix, and context-cancel-during-backoff (go test -race clean). Mirrors the retry/backoff approach in the Twitter client (#19). Signed-off-by: Misha Rautela <mrautela@linuxfoundation.org>
Address David's resilience finding on PR #22: doRequest mapped every non-2xx status, including 429, straight to a returned error with no retry. CreateCampaign drives several sequential Marketing API calls (campaign group, campaign, dark post, creative) - exactly the burst that trips a per-account rate limit - so a single transient 429 aborted the whole flow. - Retry a 429 up to retryMax (3) times with bounded backoff, honoring Retry-After (delay-seconds or HTTP-date) and capping any single wait at maxRetryWait (60s); fall back to exponential backoff otherwise. The final attempt returns the standard non-2xx error rather than looping. - Reuse the existing injectable clock (WithClock) for the HTTP-date path; add an unexported withRetryBaseDelay so tests exercise retries without real wall-clock waits. - sleepCtx honors context cancellation during backoff. The response body is now closed explicitly on every path (the retry loop precludes defer). - Tests: retry-then-succeed, exhaust-after-retryMax, parseRetryAfter header matrix, and context-cancel-during-backoff (go test -race clean). Mirrors the retry/backoff approach in the Twitter client (#19) and the matching fix in the Meta client (#20). Signed-off-by: Misha Rautela <mrautela@linuxfoundation.org>
MRashad26
left a comment
There was a problem hiding this comment.
Two findings on the X/Twitter Ads client.
…off, validate RegistrationURL, fix doc (PR #19 round 11) - verifyAccount now goes through doRequest so it gets the same OAuth signing and 429 retry/backoff as every other call (was firing httpClient.Do directly); verification stays non-fatal - clamp the no-Retry-After computed exponential backoff to maxRetryWait, matching the header path - validate RegistrationURL (absolute http/https, real host) up front before any mutating call - OKF doc: a DUPLICATE_PROMOTABLE_ENTITY response is surfaced as a warning (may refer to another line item), not idempotent success Signed-off-by: Misha Rautela <mrautela@linuxfoundation.org>
…rename dup test (PR #19 round 12) - parseRetryAfter checks X-Account-Rate-Limit-Reset first (SDK precedence), then X-Rate-Limit-Reset, then Retry-After, so an account-scoped 429 waits the real reset instead of the short exponential fallback - reject an empty/whitespace Project before any mutating call and drop the 'tlf' name-builder fallback (avoids misattributing a non-TLF campaign) - rename the promoted-tweet duplicate test to describe the asserted warning behavior Signed-off-by: Misha Rautela <mrautela@linuxfoundation.org>
…c (PR #19) After Project validation moved ahead of budget/date/account-config checks, several fixtures with an empty Project short-circuited on "invalid project" and never exercised the guard they assert. Add a valid Project to: - TestCreateCampaignValidation budget/date/calendar cases (event-name cases keep no Project — EventName is validated first — and explicit empty-project cases were added). - the whitespace-account-config sub-cases in TestAccountConfigTrimmedInRequests, so they reach the account-id guard rather than the project guard. Also move the TestTweetIDRejectsNonSnowflake doc paragraph off TestTweetIDInt64OverflowRejected so each test's doc matches its declaration. Signed-off-by: Misha Rautela <mrautela@linuxfoundation.org>
readAll silently swallowed all read errors and always returned nil, and doRequest discarded that error — so a truncated/failed body read produced a misleading JSON decode error or hid the transport failure. Rewrite readAll with io.ReadAll(io.LimitReader(body, maxResponseBody+1)) to surface both read and truncation errors, and handle it at the call site: include it in the non-2xx error, and on a 2xx don't decode a partial body — return the I/O failure. Mirrors the reddit/meta readResponseBody helpers. Signed-off-by: Misha Rautela <mrautela@linuxfoundation.org>
findByName walked the campaign/line-item list with no count param and a 25-page cap (~5k records), so name-based idempotency failed on large accounts (X allows ~8k active campaigns). Request count=1000 (X Ads v12 max) so the lookup covers realistic large accounts (25*1000=25,000 >> 8,000 active); the inconclusive-cap still returns an error rather than risking a duplicate create. Test asserts the count param and a deep-page match. Signed-off-by: Misha Rautela <mrautela@linuxfoundation.org>
Per Copilot: count=1000 alone still scans the whole account. Both the campaigns and line_items list endpoints support the server-side `q` name filter, so add q=<name> before paging — the lookup is now O(matches), not O(account), so name-based idempotency holds even on accounts with 8,000+ campaigns. q does substring/prefix matching, so findByName still enforces the EXACT-name match locally; count=1000 keeps any residual paging cheap. Tests assert q=<name> on both the campaign and line-item lookups. Signed-off-by: Misha Rautela <mrautela@linuxfoundation.org>
dealako
left a comment
There was a problem hiding this comment.
@mrautela365 thanks for the persistence through this many review rounds — the OAuth1 signing, idempotent find-or-create flow, pagination, rate-limit header precedence, and input validation are all in solid shape at this point, and the test suite is thorough (deterministic signature goldens, 429-retry via httptest, guard-branch coverage). Nice work getting this porting job this tight.
How this review was done: I ran a security-focused subagent and a general code-quality subagent in parallel against the current diff (gh pr diff 19) and the full client.go/client_test.go, cross-checked their output against the existing Copilot review threads on this PR, and traced the two still-open findings myself in the code before posting.
Revision history: This PR has been through 13 rounds of fixes already (see commit history: rate-limit header precedence, pagination with q= filter, whitespace/format validation on IDs, response-body read-error surfacing, naming-convention date segment, etc.) — all confirmed resolved, no regressions found. The two items below are new, from the Copilot pass that landed after your most recent fix commit (1282828), so they haven't been addressed yet.
Issue count:
- 🔴 Blocking: 2 — Retry-After integer overflow bypassing
maxRetryWaitabort; confirmed campaign/line-item IDs discarded on downstream failure paths - 🟡 Minor: 0
- ⚪ Nit: 0
- ❔ Question: 0
Both blocking items were independently surfaced by Copilot's latest pass and confirmed by both my own trace and the code-reviewer-pro subagent — no false positives found, security-auditor subagent found nothing further (credential handling, injection risks, and defaults all check out clean).
Decision: 🔴 Needs changes before approval — both blocking items are inline above with proof and suggested fixes.
…nstream failure (PR #19) - parseRetryAfter guards the seconds->Duration conversion (mirroring reddit): a value above the maxRetryWait ceiling returns the over-cap sentinel so the abort fires, instead of overflowing/wrapping to a small delay that bypasses the cap - once the campaign (and later the line item) is created, downstream failures return a partial CampaignResult carrying the created IDs + steps alongside the error, so an orphaned paid resource is identifiable for reconcile (mirrors the meta/reddit clients); true retry-safe idempotency tracked in LFXV2-2665 Signed-off-by: Misha Rautela <mrautela@linuxfoundation.org>
Both blocking findings addressed — ready for re-review @dealakoBoth
Both threads resolved. CI green on HEAD (CodeQL / MegaLinter / License / DCO / OKF; Build in progress). Re-requesting your review. |
OKF v0.1 §4.1 recommends `tags` (cross-cutting categorization) and `timestamp` (ISO 8601 last-meaningful-change) as queryable frontmatter fields, alongside the type/title/description/resource we already carry. The twitter concept doc had neither. Add both to internal/platform/twitter so the bundle's queryable surface matches the format's recommendation. okfvalidate stays green (it requires only `type` and preserves additional keys). Signed-off-by: Misha Rautela <mrautela@linuxfoundation.org>
The OAuth 1.0a signature base string must use a normalized URI (RFC 5849 §3.4.1.2): scheme and host lowercased, default port (:80/:443) omitted. Since WithBaseURL accepts arbitrary URLs, a value like "HTTPS://ADS-API.X.COM:443" would otherwise be signed verbatim and X would reject the signature. Add normalizeSigningURL (lowercase scheme/host, drop default port, keep non-default, exclude query) and use it for the base string only; the request still targets the real URL. Signed-off-by: Misha Rautela <mrautela@linuxfoundation.org>
…drift (PR #19) - add tests for normalizeSigningURL (via matching OAuth signatures for a normalized vs raw base URL, plus direct unit cases) - surface a warning step when an existing campaign or line item is reused by name, since its budget/status/dates are NOT updated to match the request; authoritative reconcile is the orchestrator's job (LFXV2-2665) Signed-off-by: Misha Rautela <mrautela@linuxfoundation.org>
…ring (PR #19) normalizeSigningURL used the decoded u.Path and u.Hostname(), so two base-URL shapes produced a signature that didn't match the request URI: - an escaped path (e.g. base "/proxy%2Ftwitter") was signed decoded ("/proxy/ twitter") while the request sends the escaped form — every request through such a proxy base failed signature validation. Use u.EscapedPath(). - an IPv6 literal host lost its brackets (Hostname() strips them), so "http://[::1]:8080" became "http://::1:8080". Re-bracket a host containing ':' before adding/omitting the port. Added escaped-path and IPv6 (with-port + default-port-dropped) cases to TestNormalizeSigningURL. Signed-off-by: Misha Rautela <mrautela@linuxfoundation.org>
… counters (PR #19) Address remaining Copilot review items on the X/Twitter client: - client.go: partial-failure error messages hardcoded "campaign %s created, PAUSED" / "line item %s created" even when the resource was REUSED (found by name) rather than created by this call. Added campaignStatus()/lineItemStatus() helpers driven by campaignReused/ lineItemReused flags so cleanup/reconcile sees accurate provenance ("reused, PRE-EXISTING" vs "created, PAUSED"). (per copilot[bot]) - client_test.go: TestRetryOn429 and the other httptest-handler call counters incremented `var calls int` in the server goroutine and read it from the test goroutine — a data race under -go test -race. Converted all handler counters to int32 with atomic.AddInt32/LoadInt32/StoreInt32, matching the pattern already used elsewhere in this test file. (per copilot[bot]) Verified: gofmt, go build, go vet, golangci-lint (0 issues), go test -race ./internal/platform/twitter (pass, no races). Signed-off-by: Misha Rautela <mrautela@linuxfoundation.org>
…ng slash (PR #19) Address the latest Copilot review items on the X/Twitter client: - client.go: buildOAuthHeader folded query params into the signature via a map[string]string, keeping only the first value per key (vs[0]). A repeated query parameter (a=1&a=2) would therefore be signed with only one value, producing an invalid signature (RFC 5849 §3.4.1.3.2 requires every value). generateOAuthSignature now takes an extra []oauthParam slice; buildOAuthHeader collects EVERY (name,value) query pair into it so all values are signed. (per copilot[bot]) - client.go: WithBaseURL now trims trailing slashes, so a base URL like "https://ads-api.x.com/" no longer yields a double-slash account path ("//12/accounts/...") that would be signed and sent verbatim and could break signature verification. (per copilot[bot]) - client_test.go: added TestBuildOAuthHeaderMultiValuedQuery (both values of a repeated key are signed, with a collapse-detection guard) and TestWithBaseURLTrimsTrailingSlash. Updated the 3 direct generateOAuthSignature callers for the new nil extraPairs arg. Verified: gofmt, go build, go vet, golangci-lint (0 issues), go test -race ./internal/platform/twitter (pass, no races). Signed-off-by: Misha Rautela <mrautela@linuxfoundation.org>
dealako
left a comment
There was a problem hiding this comment.
@mrautela365 great turnaround — both blocking items from my last review landed correctly in 19774f8, and I re-verified rather than trusting the commit message.
How this re-review was done: fetched the branch tip (64184c1, 6 commits past what I last reviewed at 1282828), ran security-auditor and code-reviewer-pro subagents in parallel against the full diff, traced the two fixes myself in the code, cross-checked every Copilot thread opened since my last pass, and ran go build/go vet/go test locally.
👏 Nice work:
parseRetryAfternow caps the Retry-After seconds value againstmaxRetryWaitbefore multiplying bytime.Second, returning an explicit over-cap sentinel rather than letting the multiply wrap — exactly closes the overflow.CreateCampaignnow threads apartialResult()closure through every downstream failure point aftercampaignID(and laterlineItemID) is known — all 5 failure paths verified to return the partial result, not just the two I flagged.- The RFC 5849 URI-normalization work (lowercase scheme/host, default-port stripping, IPv6 re-bracketing,
EscapedPath()for the signature base string) and the multi-valued query-param signing fix are both correct and non-trivial to get right — verified by tracingnormalizeSigningURLandgenerateOAuthSignaturedirectly, not just from the tests. - Every Copilot thread opened since my last review has a reply pointing to the resolving commit, and the timestamps line up cleanly with the fix commits — no dangling threads.
Revision tracking:
- ✅ Retry-After int64 overflow bypassing
maxRetryWait— resolved in19774f8 - ✅ Confirmed campaignID/lineItemID discarded on downstream failure — resolved in
19774f8 - ✅ All Copilot findings since (naming-date-segment reuse divergence wording, line-item-reuse status drift, race in test counters, OAuth base-string normalization/IPv6/escaped-path, multi-valued query signing, base-URL trailing slash) — resolved across
daf75fa..64184c1, each with a linked reply
Issue count:
- 🔴 Blocking: 0
- 🟡 Minor: 1 — HTTP-date branch of
parseRetryAfterisn't symmetrically guarded against the same overflow class (theoretical, requires a ~292-years-out date; inline comment has the fix) - ⚪ Nit: 1 — no test exercises the "reused, PRE-EXISTING" wording on a failure path (only the create-then-fail and reuse-then-success paths are covered); not required, just a coverage gap for the next person debugging a reused-resource failure
Decision: ✅ Approved with minor comments — the one minor item is optional polish, not a blocker; the two prior blocking issues are correctly and thoroughly fixed.
| if t, err := http.ParseTime(v); err == nil { | ||
| if d := t.Sub(c.timeFn()); d > 0 { | ||
| return d | ||
| } |
There was a problem hiding this comment.
[minor] HTTP-date branch of parseRetryAfter isn't guarded against the same overflow class you just fixed
Issue: The seconds-form branch above (line ~526-527) now caps n against maxRetryWait before multiplying, to avoid time.Duration wrapping. The HTTP-date branch (t.Sub(c.timeFn())) has no equivalent guard.
Proof: time.Duration is an int64 count of nanoseconds, so a Retry-After HTTP-date more than ~292 years out would make t.Sub(...) overflow and wrap to a negative/small value, which — like the seconds-form bug — would slip past the caller's waitDur > maxRetryWait abort and trigger an immediate retry instead of aborting.
Why it matters: this is the same bug class as the one you just fixed, just reached via the HTTP-date format instead of the seconds format. Low likelihood (requires an adversarial/badly-misconfigured upstream sending a far-future date), but cheap to close for consistency now that the seconds path is hardened.
Fix: apply the same cap:
if t, err := http.ParseTime(v); err == nil {
if d := t.Sub(c.timeFn()); d > 0 {
if d > maxRetryWait {
return maxRetryWait + time.Second
}
return d
}
}Not blocking — flagging for consistency; up to you whether it's worth a follow-up.
First Phase-2 platform client. Ports
x-ads.service.tsfrom the Express BFF to Go as a standalone packageinternal/platform/twitter/.What it does: OAuth 1.0a HMAC-SHA1 signing (stdlib), campaign→line-item→promoted-tweet creation with name-based idempotency, micro-currency conversion, ISO8601 UTC dates, 1 req/sec write delay + 429 exponential-backoff retry (context-aware).
Design: credentials are injected (
Credentials+AccountConfig), not read from env — in production they come from the decrypted stored connection (D7). No new deps (stdlib only). Does not import the orchestrator; a thinPlatformDispatcheradapter lands with the wiring PR once #11 merges.Public API:
NewClient(creds, account, opts...)→CreateCampaign(ctx, CampaignInput) (*CampaignResult, error).Tests: 14 unit tests (deterministic OAuth1 signature, currency round-trip, date formatting, 429-retry via httptest) — no network/real creds.
Not ported (separate later PR): status-toggle + analytics/monitoring read path.
Ports
lfx-v2-ui/apps/lfx-one/src/server/services/x-ads.service.ts. LFXV2-2640🤖 Generated with Claude Code