OTLP profiles export foundation (experimental)#1934
Draft
adriangb wants to merge 3 commits into
Draft
Conversation
Convert Python 3.15 `profiling.sampling` (Tachyon) collapsed-stack output into the OpenTelemetry profiles signal and export it over HTTP. - collapsed.py: parse Tachyon `--collapsed` output into folded stacks - otlp.py: build an OTLP `ExportProfilesServiceRequest` - exporter.py: gzip + POST the request, failing soft - _proto/: vendored OTLP profiles protobuf bindings (see _proto/README.md) Validated end-to-end against Grafana Pyroscope. Still to come: the subprocess supervisor that runs the profiler, wiring into `logfire.configure()`, and per-span correlation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Deploying logfire-docs with
|
| Latest commit: |
612383d
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://bd9ed2cf.logfire-docs.pages.dev |
| Branch Preview URL: | https://profiling-otlp-spike.logfire-docs.pages.dev |
`ProfilingSupervisor` runs the Python 3.15 `profiling.sampling` profiler as a child process that attaches back to this process, in repeated fixed-duration chunks, converting each chunk to OTLP profiles and exporting it. - `prctl(PR_SET_PTRACER)` lets the child ptrace us without root on Linux - degrades gracefully: disabled with a warning on Python < 3.15 or when the platform / permissions don't allow attaching; never raises - judges chunk success by output presence, since the profiler exits 0 even when it fails to read the target Still to come: wiring into `logfire.configure()`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add an experimental `profiling: bool` option to `logfire.configure()`. When enabled (and sending to Logfire) it builds a `ProfilesExporter` from the same region URL / token / session as the other signals, posting to `/v1development/profiles`, and runs a `ProfilingSupervisor` for the process lifetime. The supervisor is stopped on `logfire.shutdown()` and on reconfiguration. Off by default; a no-op with a warning on Python < 3.15. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Experimental support for shipping Python 3.15
profiling.sampling(Tachyon)CPU profiles to the OpenTelemetry profiles signal.
Opening as a draft — it builds on two alpha foundations (the Python 3.15
profiler and the OTel profiles signal) and is gated to Python 3.15+, so it
ships experimental, not stable.
Usage
Off by default. On Python < 3.15, or where the platform / permissions don't
let the profiler attach, it's a no-op with a warning — the rest of Logfire is
unaffected. Nothing in the profiling path raises.
What's here
profiling/collapsed.py— parse the profiler's--collapsedoutputprofiling/otlp.py— build an OTLPExportProfilesServiceRequestprofiling/exporter.py— gzip + POST, fails softprofiling/supervisor.py— runs the profiler as a child process (chunkedattach), converts + exports each chunk;prctl(PR_SET_PTRACER)lets itattach without root on Linux
configure(profiling=True)wiring — builds the exporter from the sameregion URL / token / session as the other signals (
/v1development/profiles),and starts / stops the supervisor with Logfire's lifecycle
Validated end-to-end against a local Grafana Pyroscope: real Tachyon output →
OTLP profiles → flamegraph.
Follow-ups (not in this PR)
Sample.link_index→Link(trace_id, span_id)) — v2logfire/_internal/profiling/_proto/contains vendored, generated OTLPprofiles protobuf bindings instead of importing from
opentelemetry-proto.Why: the profiles signal is alpha (
v1development) and theopentelemetry-protoPyPI package ships a stale snapshot — its
Samplefield numbers predate arenumbering in proto
v1.10.0and are wire-incompatible with current consumers(Pyroscope silently drops the profile). These bindings are generated from
canonical proto
v1.10.0, re-namespaced tolf_otlp.*so they don't collidewith the stale copy in protobuf's descriptor pool.
This directory should be deleted once
opentelemetry-protoships bindingsfrom proto ≥
v1.10.0. The upstream fix is open atopen-telemetry/opentelemetry-python#5223. See
_proto/README.md.🤖 Generated with Claude Code