Skip to content

Fix leak linked to event listeners on quality change - #1781

Merged
peaBerberian merged 2 commits into
devfrom
fix/apap-stream-listener-leak
Jan 23, 2026
Merged

Fix leak linked to event listeners on quality change#1781
peaBerberian merged 2 commits into
devfrom
fix/apap-stream-listener-leak

Conversation

@peaBerberian

@peaBerberian peaBerberian commented Jan 19, 2026

Copy link
Copy Markdown
Collaborator

The #1778 and #1779 issues / PR noticed a leak that seem to arise when multiple quality switches happen.

I'm still unsure of the severity (looking at it what this fixes seems very minimal, and we did not notice this yet on production at Canal+ including on low-memory devices for what seems to be a change that has been here for 2 years - but external contributors actually did notice a leak so maybe a set of conditions amplify the issue), but looking closely at the code in question, there does seem to be an improper event listener clean-up on a quality switch.

The issue is rooted in the complexity behind how quality switches happen:

  • depending on heuristics, we may either perform an "urgent" quality switch (where we directly cancel the requests linked to the older quality) or a non-urgent one (where we will wait for the current requests to finish and only after load the new quality).

  • If non-urgent, we want to still do the requests for the new quality as soon as we can, thus once the older requests are finished we parallelize its pushing operations with the new requests.

Thus when a "non-urgent" quality switch happen, there might be a short time where several quality-linked modules are running at the same time (the old one to push older segments, the new one to load newer ones), whereas at first glance they could seem conflicting (one loads and push one quality, the other loads and push another quality of the same thing).

This lead to an awkward architecture where the clean-up process of those modules is subtly different than in other RxPlayer modules - this one has actually 2 means to terminate:

  • its terminate parameter, kind of like a SIGTERM: just finish what you're doing (e.g. finish loading segments and/or pushing them then stop).

    Once the RepresentationStream (the module in question) has finished loading segments, it sends a terminating event - but it might still be pushing segments.

    It however has no event to indicate that segments have been pushed, for now.

  • its cancelSignal parameter, more akin to a SIGKILL: terminate everything now without delay.

    This one is e.g. triggered when stopping the content, changing the track etc.

The leaking event listener was wrongly linked to that "SIGKILL" signal, even if it was intended to be cleaned up when the module is not needed anymore. When the module was only "SIGTERMed", it was not cleaned up.


I chose to clean it up not right when "SIGTERMed", but when the module itself anounced that it is "terminating" (it is done loading and is now pushing segments).

I found it to be more appropriate for the logic in question and a corresponding CancellationSignal was already used for other similar logic linked to the same lifetime.

@peaBerberian peaBerberian added the Priority: 0 (Very high) This issue or PR has a very high priority. Efforts should be concentrated on it first. label Jan 19, 2026
@peaBerberian
peaBerberian force-pushed the fix/apap-stream-listener-leak branch 3 times, most recently from 40be01f to 423a02a Compare January 19, 2026 13:26
@peaBerberian
peaBerberian force-pushed the fix/apap-stream-listener-leak branch 2 times, most recently from ef53a24 to 405b338 Compare January 19, 2026 13:54
@peaBerberian peaBerberian added this to the 4.5.0 milestone Jan 19, 2026
@github-actions

Copy link
Copy Markdown

✅ Automated performance checks have passed on commit b805cadcef9ac18d662d5262d050b360ae85d3f1 with the base branch dev.

Details

Performance tests 1st run output

No significative change in performance for tests:

Name Mean Median
loading 20.66ms -> 20.97ms (-0.305ms, z: 1.25275) 29.25ms -> 29.40ms
seeking 12.11ms -> 10.09ms (2.019ms, z: 0.33630) 12.15ms -> 12.30ms
audio-track-reload 28.22ms -> 28.27ms (-0.053ms, z: 0.13353) 41.55ms -> 41.55ms
cold loading multithread 47.31ms -> 46.76ms (0.549ms, z: 9.55205) 69.75ms -> 68.70ms
seeking multithread 77.34ms -> 75.32ms (2.025ms, z: 0.83940) 10.80ms -> 10.80ms
audio-track-reload multithread 27.75ms -> 27.51ms (0.234ms, z: 2.38653) 40.95ms -> 40.65ms
hot loading multithread 15.69ms -> 15.53ms (0.164ms, z: 4.90546) 22.95ms -> 22.65ms

for (const rep of updated.removedRepresentations) {
if (rep === representation.id) {
if (fnCancelSignal.isCancelled()) {
if (terminatingCanceller.isUsed()) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH, I'm not so sure how this change solve the issues, the other changes seems more like renames.

@github-actions

Copy link
Copy Markdown

✅ Automated performance checks have passed on commit 32573cf84889a139b05374112f8529ca231884db with the base branch dev.

Details

Performance tests 1st run output

No significative change in performance for tests:

Name Mean Median
loading 21.53ms -> 22.04ms (-0.505ms, z: 0.03598) 30.30ms -> 30.30ms
seeking 9.59ms -> 14.29ms (-4.697ms, z: 0.55263) 12.45ms -> 12.60ms
audio-track-reload 29.15ms -> 29.06ms (0.089ms, z: 0.44193) 42.75ms -> 42.75ms
cold loading multithread 48.37ms -> 47.99ms (0.383ms, z: 7.05517) 71.40ms -> 70.65ms
seeking multithread 97.64ms -> 96.83ms (0.808ms, z: 1.20321) 10.80ms -> 10.95ms
audio-track-reload multithread 28.47ms -> 28.39ms (0.087ms, z: 0.86474) 42.15ms -> 42.00ms
hot loading multithread 16.32ms -> 16.20ms (0.123ms, z: 3.87213) 24.15ms -> 23.85ms

@peaBerberian

peaBerberian commented Jan 21, 2026

Copy link
Copy Markdown
Collaborator Author

That second SegmentQueue linked leak seen by @KunXi-Fox seem to have an observed effect: https://github.com/canalplus/rx-player/actions/runs/21204957791/job/60999139813?pr=1781#step:7:60 (~1.7MB after now 1000 quality switches as opposed to the ~4MB we've been seeing generally after 500 switches)

On (new) "reload" quality switching memory-tests we even go from ~3.7MB to reliably a few negative kB for some reason that I didn't look at yet (meaning: using a little less memory than when starting)!

Anyway results from our memory tests are still currently fuzzy, I'm trying to extract more value from them.

The #1778 and #1779 issues / PR noticed a leak that seem to arise when
multiple quality switches happen.

I'm still unsure of the severity (looking at it what this fixes seems
very minimal, and we did not notice this yet on production at Canal+
including on low-memory devices for what seems to be a change that has
been here for 2 years - but external contributors actually did notice a
leak so maybe a set of conditions amplify the issue), but looking closely
at the code in question, there does seem to be an improper event listener
clean-up on a quality switch.

The issue is rooted in the complexity behind how quality switch happen:

- depending on heuristics, we may either perform an "urgent" quality
  switch (where we directly cancel the requests linked to the older
  quality) or a non-urgent one (where we will wait for the current
  requests to finish and only after load the new quality).

- If non-urgent, we want to still do the requests for the new quality as
  soon as we can, thus we parallelize it with the pushing operations of
  the segments we just loaded from the previous quality.

Thus when a "non-urgent" quality switch happen, there might be a short
time where several quality-linked modules are running at the same time
(the old one to push segments, the new one to load them), whereas at
first glance they seemed conflicting (one loads and push one quality,
the other loads and push another quality of the same thing).

This lead to an awkward architecture where the clean-up process of those
modules is subtly different than in other RxPlayer modules - this one
has actually 2 means to terminate:

- its `terminate` parameter, kind of like a SIGTERM: just finish what
  you're doing (e.g. finish loading segments and/or pushing them then
  stop).

  Once the `RepresentationStream` (the module in question) has finished
  loading segments, it sends a `terminating` event - but it might still
  be pushing segments.

  It however has no event to indicate that segments have been pushed,
  for now.

- its `cancelSignal` parameter, more akin to a SIGKILL: terminate
  everything now without delay.

  This one is e.g. triggered when stopping the content, changing the
  track etc.

The leaking event listener was wrongly linked to that "SIGKILL" signal,
even if it was intended to be cleaned up when the module is not needed
anymore. When the module was only "SIGTERMed", it was not cleaned up.

---

I chose to clean it up not right when "SIGTERMed", but when the module
itself anounced that it is "terminating" (it is done loading and is now
pushing segments).

I found it to be more appropriate for the logic in question and a
corresponding `CancellationSignal` was already used for other similar
logic linked to the same lifetime.
@peaBerberian
peaBerberian force-pushed the fix/apap-stream-listener-leak branch from d90ca0a to b20b8bf Compare January 23, 2026 11:07
@peaBerberian
peaBerberian merged commit 0918aa5 into dev Jan 23, 2026
14 checks passed
@github-actions

Copy link
Copy Markdown

✅ Automated performance checks have passed on commit ff8e8e53454fd2d03a4dc58c80d79a20aafa6cf2 with the base branch dev.

Details

Performance tests 1st run output

No significative change in performance for tests:

Name Mean Median
loading 21.05ms -> 21.45ms (-0.401ms, z: 2.20342) 29.85ms -> 29.85ms
seeking 15.07ms -> 13.70ms (1.365ms, z: 0.91088) 12.45ms -> 12.30ms
audio-track-reload 29.30ms -> 29.12ms (0.179ms, z: 1.73232) 42.90ms -> 42.75ms
cold loading multithread 48.06ms -> 48.08ms (-0.021ms, z: 11.17578) 70.80ms -> 69.90ms
seeking multithread 276.75ms -> 258.73ms (18.018ms, z: 0.98904) 10.05ms -> 10.05ms
audio-track-reload multithread 28.67ms -> 28.53ms (0.139ms, z: 2.72174) 42.45ms -> 42.00ms
hot loading multithread 16.07ms -> 15.95ms (0.112ms, z: 3.99858) 23.70ms -> 23.55ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Priority: 0 (Very high) This issue or PR has a very high priority. Efforts should be concentrated on it first.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants