Optimize inference preprocessing pipeline - #787
Draft
xav-ie wants to merge 4 commits into
Draft
Conversation
royshil
reviewed
Feb 27, 2026
royshil
left a comment
royshil
left a comment
Owner
There was a problem hiding this comment.
Did you test this with all models?
| } | ||
| } | ||
| tf->lastImageBGRA = imageBGRA.clone(); | ||
| tf->lastImageBGRA = thumbnail; |
Owner
There was a problem hiding this comment.
Is the last image only used for PSNR calculations?
If so I suppose it's fine
Author
No, sorry. Let me try to figure this out. Sorry for the noise. |
xav-ie
marked this pull request as draft
February 27, 2026 04:16
umireon
previously requested changes
Feb 27, 2026
umireon
left a comment
umireon
left a comment
Collaborator
There was a problem hiding this comment.
Working on the CONTRIBUTING.md now. Sorry but wait a moment, please.
Extract shared pipeline helper functions (render_write, tick_read, check_similarity, preprocess_for_inference) into src/pipeline-helpers.h for use by both the plugin and a standalone benchmark. The benchmark exercises the same code paths as the plugin: 1. Render thread writes frame to shared buffer (clone) 2. Tick thread reads frame from shared buffer (clone) 3. Similarity check via full-frame PSNR 4. Preprocessing: cvtColor(BGRA→RGB) then resize Reports per-frame timing (mean/median/p95), hardware perf counters (cache-misses, LLC-store-misses via perf_event_open), and optionally measures CPU utilization with a real ONNX Runtime thread pool at 30fps. Signed-off-by: Xavier Ruiz <github@xav.ie>
Set session.intra_op.allow_spinning=0 and session.inter_op.allow_spinning=0 to prevent ORT worker threads from busy-waiting between inference calls. This eliminates idle CPU burn without affecting inference latency. Signed-off-by: Xavier Ruiz <github@xav.ie>
xav-ie
force-pushed
the
pr/optimize-inference-preprocessing
branch
from
March 8, 2026 05:07
e90246d to
af7c862
Compare
Signed-off-by: Xavier Ruiz <github@xav.ie>
…eline Signed-off-by: Xavier Ruiz <github@xav.ie>
xav-ie
force-pushed
the
pr/optimize-inference-preprocessing
branch
from
March 8, 2026 05:12
af7c862 to
7fde9c5
Compare
Owner
|
@xav-ie do you want to bring this up so we can incorporate in the plugin? this is a very good contribution |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cvtColoroperates on the smaller network-input-sized image instead of the full frameINTER_LINEARtoINTER_NEARESTsince the result feeds directly into a neural network that is invariant to interpolation qualityBenchmark results (300 iterations, best of 3 runs)
Pull Request Checklist
Please read our latest CONTRIBUTING.md.