Repository navigation
[pull] main from firecrawl:main - #470
Merged
Merged
Conversation
… retention (#4817) * fix(crawl): keep crawl prompt LLM calls out of traces under zero data retention Crawl option generation from a natural-language prompt ran in the API process with AI SDK telemetry always on and outside any zero data retention tracing context, so the prompt and the site URLs added to it could be exported for ZDR crawls. - crawl and crawl params preview wrap the prompt step (site lookup and LLM call) in withZeroDataRetention and pass the ZDR flag to generateCrawlerOptionsFromPrompt. The preview applies the same team rule as a crawl (forced ZDR or Safe Mode lockdown) and tags its logger. - generateCompletions also turns telemetry off whenever it runs in a ZDR context, so callers that omit the flag no longer export prompts. - otel-tracer exposes isZeroDataRetentionActive for that check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(crawl): honor a per-request zeroDataRetention flag in the params preview The preview schema dropped zeroDataRetention, so a preview sent with it still traced the prompt step. Accept the flag and include it in the ZDR rule; it only affects the preview's own tracing, so no permission check is needed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
analyzeSchemaAndPrompt_F0 passed its metadata to generateCompletions_F0 without a functionId, so its gpt-4.1 call ran with an empty ai.telemetry.functionId and could not be attributed to a feature. - performExtraction_F0 now passes functionId "performExtraction_F0", and analyzeSchemaAndPrompt_F0 extends it to "performExtraction_F0/analyzeSchemaAndPrompt_F0" (falling back to "analyzeSchemaAndPrompt_F0"), like the other F0 completion helpers. - functionId is now required on generateCompletions_F0's metadata, so a new caller cannot leave it out. - Add a unit test for the telemetry settings on the analysis call and the schema generation call it can make. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
… use (#4818) * fix(cost-tracking): record branding LLM calls and price all models in use Branding called generateObject without recording the call in the job's cost tracking, so its LLM spend never showed up in cost_tracking. It now takes the scrape's CostTracking and records each call (model, tokens, cost), including calls whose output failed to parse, and lets a cost limit error propagate instead of treating it as an LLM failure. calculateCost only knew a short hard-coded list of ids and silently returned $0 for anything else, including gpt-4.1, gpt-4.1-mini, bare gpt-4o, bare o3-mini and bare gemini-2.5-flash-lite. It now strips a provider prefix, falls back to the model-prices table, and warns once per model when it still finds no price. The Fireworks query fine-tune is priced at its base model's serverless rate. Also fixes operator precedence in totalTokens for no-object extractions, which reported only the input tokens. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(branding): record the resolved model and surface cost limits Record the model id getModel resolved (it honors a MODEL_NAME override) instead of the requested name, and rethrow CostLimitExceededError from the branding transformer's fallback catch so a cost limit is not reported as a failed LLM enhancement. Tests pin branding model selection with a screenshot rather than relying on prompt length, cover the resolved model id and a cost limit hit while recording a parse failure, and check both gemini-2.5-pro price tiers around the 200k-token boundary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…e spans (#4816) * feat(telemetry): record cached and reasoning tokens on AI SDK generate spans On ai@6.0.86 the `ai.generateObject.doGenerate` and `ai.generateText.doGenerate` spans only carry prompt and completion token totals; the provider's cache-read, cache-write and reasoning token counts are dropped. Only the streaming spans record them. Wrap every model returned by getModel() in a small language model middleware that runs inside the SDK's doGenerate span and sets `ai.usage.cachedInputTokens`, `ai.usage.reasoningTokens` and the `ai.usage.inputTokenDetails.*` / `ai.usage.outputTokenDetails.*` breakdown, using the attribute names newer ai@6 releases emit for generateText. When SDK telemetry is disabled no doGenerate span exists, so the middleware leaves the caller's span alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(telemetry): reset module registry after the getModel usage test Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(telemetry): assert the stubbed OpenAI call hits the Responses endpoint Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
… 503 code (#4819) * feat(api): count every FirePDF transport decision and keep fire-pdf's 503 code Sync /ocr decisions were invisible: only async routing was logged, so ZDR, short-deadline and out-of-cohort traffic could only be inferred. Every first FirePDF attempt now increments firecrawl_fire_pdf_route_decisions_total {source_kind,path,reason,features,zdr} and observes the caller's remaining time in firecrawl_fire_pdf_route_remaining_seconds; sync decisions also log fire_pdf_sync_routed. Images record reason=no_async_route, large PDFs reason=by_reference. POST /jobs 503s now keep fire-pdf's error code (page_markdown_not_ready, admission_rejected, ...) in the failAsync log and in firecrawl_fire_pdf_async_submit_503_total{code}. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(api): count a by-reference attempt only once it used async; log the 503 code only - by-reference: a null return (input never placed) falls through to the inline chain, which records its own decision, so the by_reference decision is now recorded only after a result or a post-placement throw. The remaining time is captured before the attempt so the histogram still reflects the routing moment. - submit 503: log { code } instead of the whole response body. - tests: the remaining-time assertion filters source_kind and compares the le=15 and le=10 buckets against their pre-recording values. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(api): assert the histogram bucket for a captured remaining time Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(telemetry): record usage spans for x-twitter Grok calls The x-twitter engine calls Grok through `@ai-sdk/xai` directly with no `experimental_telemetry`, so its calls emit no `ai.generateText.doGenerate` span and none of their token usage is traced. Enable AI SDK telemetry on both calls with a function id per lookup kind (`xTwitter/profile`, `xTwitter/post`) and the scrape id, team id and `feature: "x-twitter"` metadata other LLM callers set, and wrap the model in the usage middleware so the spans also carry cached and reasoning token counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(telemetry): skip x-twitter LLM spans for zero-data-retention scrapes Gate the Grok calls' AI SDK telemetry on zeroDataRetention like the other scrape-path LLM callers, and extend the tests: the post lookup checks the same span attributes as the profile lookup, both check the recorded prompt, and a ZDR scrape records no AI SDK spans. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…t LLM spans with job ids (#4814) * feat(telemetry): tag branding, prompt-injection-guard and crawl-prompt LLM spans with job ids AI SDK spans for these calls carried only teamId in their telemetry metadata, so they could not be joined to the job that made them. - Branding: add scrapeId and feature: "branding". - Prompt injection guard: extractData now passes its scrapeId and extractId to the guard, which adds them to each classifier span. - generateCompletions: build span metadata in one helper and add crawlId, which the crawl controller already passed but was dropped. Telemetry enablement, recordInputs and ZDR handling are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(telemetry): assert branding span is enabled and crawl/scrape trace ids Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(telemetry): pass costTracking to branding telemetry test BrandingLLMInput now requires costTracking (#4818). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(telemetry): drop langfuseTraceId from crawl spans Only crawlId is needed to join crawl-prompt spans to their crawl. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…4822) enhanceBrandingWithLLM always enabled AI SDK telemetry, so spans with recorded outputs and the scrape id were exported even when the scrape requested zero data retention. Gate isEnabled on zeroDataRetention like the other LLM call sites do. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A team with the largePdfs flag gets PDF_BY_REFERENCE_MAX_BYTES_PRIVILEGED, the same cap the PDF_BY_REFERENCE_PRIVILEGED_TEAM_IDS allowlist grants, so enabling a team no longer needs a config change and deploy. The env allowlist keeps working. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )