feat(detections): add recorded_at field for capture-time sequence linking#617
Open
Acruve15 wants to merge 5 commits into
Open
feat(detections): add recorded_at field for capture-time sequence linking#617Acruve15 wants to merge 5 commits into
Acruve15 wants to merge 5 commits into
Conversation
Add a recorded_at timestamp to the detections table, carrying the on-device capture time (UTC), distinct from created_at which stays as the trusted DB insertion clock. New rows default recorded_at to utcnow(). The Alembic migration adds the column as nullable, backfills it from created_at for existing rows, then tightens it to NOT NULL. Refs #510
Sequence bucketing now keys on each detection's recorded_at instead of
created_at, so detections cached on the engine and uploaded in a burst
group by when they were captured rather than when they reached the API.
The POST /detections payload accepts an optional recorded_at (defaults
to server now when the engine omits it), and the candidate-sequence and
overlap time-window queries anchor on the new detection's capture time
instead of utcnow(). Sequence started_at / last_seen_at and the
GET /sequences/{id}/detections ordering follow recorded_at too.
Refs #510
Timezone-aware capture timestamps (for example a France-local +02:00 value) are converted to UTC before being stored and before the sequence time-window comparisons, matching the naive-UTC convention used by the rest of the schema. Naive timestamps are assumed to already be UTC. Adds a to_utc_naive helper next to utcnow(). Refs #510
Resolve conflicts in the sequence detections endpoint and the detection tests, where both branches appended independent code (crop image support from #575 alongside recorded_at). The sequence endpoint now orders by recorded_at while keeping the new offset / with_crop params. Re-point the recorded_at migration onto the crop_bucket_key migration so the Alembic history stays a single linear head.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #617 +/- ##
==========================================
+ Coverage 92.30% 92.32% +0.02%
==========================================
Files 56 56
Lines 2585 2593 +8
==========================================
+ Hits 2386 2394 +8
Misses 199 199 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
The sequence started_at now derives from the detection recorded_at rather than created_at, which come from separate utcnow() calls. Compare against recorded_at so the end-to-end check matches the new behavior. Refs #510
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #510
Motivation
The
POST /detectionsendpoint imposedcreated_at = utcnow(), so detections cached on the engine and uploaded in a burst all landed at virtually the same DB timestamp, even though they were captured minutes or hours apart. Sequence linking buckets detections by time, so a burst upload either grouped unrelated captures or split a real sequence depending on upload timing.This adds a
recorded_atfield carrying the on-device capture time (UTC) and switches all sequence linking and time-window logic to use it.created_atstays as the trusted DB insertion clock.Per the issue thread (frgfm / fe51 / MateoLostanlen): both fields are kept on purpose.
created_atis the timestamp we can always trust; thecreated_at - recorded_atgap is a useful signal for spotting cameras with connection lag. The EXIF-extraction idea was considered and deferred (not all cameras expose it, TZ handling, latency) in favor of the engine sendingrecorded_atexplicitly.What changed
Detection.recorded_at(naive-UTC,default_factory=utcnow), optionalrecorded_atonDetectionCreate. Read endpoints (/detections/{id},/detections/,/sequences/{id}/detections) expose it for free via the inherited read schemas.src/app/api/api_v1/endpoints/detections.py):recorded_atis an optional form field defaulting to server now when the engine omits it (so un-updated engines keep working, and the envdenv "alerts in the past" demo just needs a pastrecorded_at). Sequence candidate lookup, the overlap window,started_at/last_seen_at, and the last-bbox lookup all key onrecorded_at; the time-window comparisons anchor on the new detection's capture time instead ofutcnow(), so a late upload attaches to the sequence that was active at capture time.src/app/api/api_v1/endpoints/sequences.py):GET /sequences/{id}/detectionsorders byrecorded_at.to_utc_naivehelper (next toutcnow()) converts timezone-aware payloads (e.g. a France-local+02:00value) to UTC before storing and before the window comparisons; naive values are assumed UTC. Coercion rather than rejection, so a tz-aware timestamp does the right thing instead of erroring.recorded_at = created_atfor existing rows, then setsNOT NULL.alerts.pyand thefromdateendpoints need no change: they filter onstarted_at/last_seen_at, which now derive fromrecorded_attransitively.Tests
NOT NULL.