Skip to content

fix(ssl): release connections when their context terminates - #8541

Merged
oneandonlydean merged 2 commits into
boykopovar:mainfrom
Ondra97:codex/fix-ssl-context-connections
Oct 25, 2026
Merged

oneandonlydean merged 2 commits into
boykopovar:mainfrom
Ondra97:codex/fix-ssl-context-connections

Conversation

@Ondra97

@Ondra97 Ondra97 commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

What

Terminating an SSL context removes the context but retains its connection records. Later connection lookup rejects their missing parent, so sceSslDeleteConnection returns 0x8095F006 and cannot release the orphaned registry entries. Three Init/Create/Term/Delete cycles leave three records on the tested main.

Remove the terminating context's connections under the existing registry mutex before releasing the context. Existing shared session ownership lets operations already in progress finish and keeps their memory pool alive until the last operation releases it. Calls made after termination keep returning the existing invalid-ID result. Record removal uses std::erase_if without allocating a temporary container. Idle sessions, including a last native socket owner, can be destroyed while the registry mutex is held.

Remove the two obsolete ownership clauses from docs/dev/debt/libSceSsl.md while preserving the console-measured invalid-ID returns and the remaining open gaps.

Add a lifecycle regression that compiles the actual SSL implementation into its test translation unit to observe private ownership. It checks both cleanup orders, multiple contexts, repeated concurrent create/delete/term operations and a paused verification callback with initialized TLS backend state. Related to #8005, finding C10.

Tested

Rebase validation on Windows 11 Pro 10.0.26300, WinLibs MinGW-w64 GCC 15.2.0r7 UCRT, Release/O3, BUILD_TESTING=ON, full runtime configuration. CPU: AMD Ryzen 9 7900X; RAM: 32 GB; GPU: NVIDIA GeForce RTX 4070 Ti SUPER.

Validated base: a6dcea9279b53b3a39c13695eac66f7c8b5b503c. Rebased head: f2173bba34562299f9d90583120ade2205ae12ef. Both original commits are retained. All production build targets and selected fixtures pass without an overlay. All 20 selected CTests pass without skips in 8.42 s, including all six SSL tests, eleven transfer regressions, guest_sce_net, guest_tcp and patched_libraries.

The rebase preserves current-main SSL transport and certificate behavior. The production change, lifecycle regression and CMake registration are unchanged from the previous PR head. The documentation conflict preserves all current-main entries and removes exactly the two obsolete ownership clauses.

Validation commands (build is the configured full Windows build directory):

cmake --build build --target libs relinker patched_libraries_tests guest_ssl_tests guest_ssl_load_tests guest_ssl_ps5_readers_tests guest_ssl_transport_tests guest_ssl_context_lifetime_tests net_dontwait_tests net_blocking_send_tests guest_sce_net_tests guest_tcp_tests --parallel 8
ctest --test-dir build -R "^(guest_ssl($|_)|patched_libraries$|net_dontwait_|guest_sce_net$|guest_tcp$)" -V --output-on-failure --no-tests=error
python -B tools/check_conventions.py --base origin/main
python -B tools/check_main_sync.py --base origin/main --strict
python -B tools/pr_overlap.py --branch codex/pr8541-rebase-20261022 --repo boykopovar/AnyPS5 --jobs 3
git diff --check origin/main...HEAD

Conventions has zero errors and one warning about the existing 7,600-character debt entry. Main-sync and whitespace checks pass. The overlap tool lists this PR itself because its hosted head differs from the rebased local branch. Manual review of uncompared shared files confirms that #6039 edits a separate debt entry, #6044 moves existing test paths, #8951 declares a separate export, and #9173 changes separate asymmetric-key cleanup exports. I also checked related #8658 and #9407; none implements context-termination cleanup or adds a functional prerequisite.

Original regression evidence is pinned to base d073fe6a1da82441d33f4d3da72cc5f7d3496259 and fix 15afb86473371f57bb2c6fa0d80bd3e4d7d049b8:

Metric Before (base) After (fix)
Retained connection records after 3 Term-before-Delete cycles 3 0
Retained connection records after 3 Delete-before-Term cycles 0 0
Delete result after context termination 0x8095F006 0x8095F006

The same regression failed on that unchanged base with termination retained orphan connection records. The original five baseline SSL tests and six fixed SSL tests passed three executions each. One initial fixed run failed the existing net transport handshake assertion with a 200 ms receive timeout; an isolated rerun, those repeated checks and six further diagnostic handshakes passed. The timeout and assertion were not changed, and the exact transient cause was not reproduced. These are historical results for the pinned revisions above.

The lifecycle regression checks 32 concurrent create/delete/term cycles, separate context ownership and an in-flight verification callback retaining its session and pool until completion. The callback case initializes TLS backend state but performs no wire I/O; the transport tests exercise loopback TLS through the kernel and libSceNet socket tables.

No complete CTest suite, GPU execution or title/gameplay run is claimed. This fixes host registry ownership; it does not establish a PRAGMATA OOM or crash cause.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 6039 docs/dev/debt/libSceSsl.md
PR 9822 core/libs/prx/libSceSsl/Export.cpp
PR 9826 core/libs/prx/libSceSsl/Export.cpp
PR 10167 core/libs/prx/libSceSsl/Export.cpp
PR 10255 core/libs/prx/libSceSsl/Export.cpp
docs/dev/debt/libSceSsl.md

@nikowolf123

Copy link
Copy Markdown
Contributor

This PR does not change docs/dev/debt/libSceSsl.md line 5, which still describes the old sceSslTerm:

  • "a connection keeps the pool alive after sceSslTerm, whose connections return SSL_ERROR_INVALID_ID from then on"
  • "sceSslTerm leaves the context's connections to sceSslDeleteConnection, which then returns SSL_ERROR_INVALID_ID"

With this PR, sceSslInit → sceSslCreateConnection → sceSslConnect → sceSslTerm erases the connection inside sceSslTerm (Export.cpp#L625). Its session and the pool are freed right there. If a call on that connection is still running, they are freed when that call returns. The new test checks both cases (SslContextLifetime.cpp#L70, #L163). After the merge, the entry would still describe the orphaned records that this PR removes.

Of this, only the SSL_ERROR_INVALID_ID returns were measured on 13.40 (#7756). The entry already states them earlier in the same line ("every connection call ... on a connection whose context was terminated returns SSL_ERROR_INVALID_ID"). The "leaves the context's connections to sceSslDeleteConnection" and "keeps the pool alive" parts described how this build was designed, not console results. d569439 added the first one, saying "none of this is verified on the console", and e371372 (#6828) added the second. CONTRIBUTING.md asks a PR that closes a gap to cut the entry down to what is still open, so I would remove both clauses and keep the rest:

Connection state is allocated from the context's pool and returned to it by `sceSslDeleteConnection`.
...
`sceSslDeleteConnection` sends no close_notify and returns `SSL_ERROR_INVALID_ID` when repeated. A second `sceSslTerm` returns `SSL_ERROR_INVALID_ID` and `sceSslInit` with a pool size of 0 returns `SCE_KERNEL_ERROR_EINVAL` (0x80020016).

@Ondra97

Ondra97 commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

@nikowolf123 Thanks for catching this. I removed both obsolete ownership clauses from docs/dev/debt/libSceSsl.md in 321e6de791, preserving the measured SSL_ERROR_INVALID_ID behavior and the rest of the entry as you suggested.

This update only changes documentation. The conventions check passes with zero errors and one warning about the existing entry length; the whitespace check also passes.

@github-actions github-actions Bot added the docs label Oct 20, 2026
@xXPBSXx

xXPBSXx commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Independent Windows run of this PR (data only).

Environment: Windows 10 Pro 10.0.19045, Ryzen 7 5800X3D, WinLibs GCC 15.2.0 r7 (MinGW-w64 UCRT), Release, BUILD_TESTING=ON. Merge-tree of the PR head against main is clean. I built only guest_ssl_context_lifetime_tests; ctest -N -R guest_ssl_context_lifetime lists exactly one test.

Results (3 runs each):

  • PR head: guest_ssl_context_lifetime passes 3/3.
  • Baseline: I kept the PR's test and replaced only core/libs/prx/libSceSsl/Export.cpp with the version from the PR's merge base (d073fe6a1d), then rebuilt. The test fails 3/3 (ctest reports Failed, a non-zero exit). I restored the PR's file afterwards.

Not run: other platforms, the other tests in the same family, the full repository CTest, and any title or console run.

AI-assisted (Claude Code).

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 21, 2026
@Ondra97
Ondra97 force-pushed the codex/fix-ssl-context-connections branch from 321e6de to d450e0c Compare October 21, 2026 18:46
@github-actions github-actions Bot added needs rebase Conflicts with main, rebase needed and removed needs rebase Conflicts with main, rebase needed labels Oct 21, 2026
@Ondra97
Ondra97 force-pushed the codex/fix-ssl-context-connections branch from d450e0c to f2173bb Compare October 21, 2026 23:40
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 22, 2026
@DoubleAAGuy

Copy link
Copy Markdown

My draft #9834 adds a libSceHttp HTTPS integration test in the existing SSL transport suite and adjusts its Windows PATH list, which overlaps this CMake hunk. I plan to sequence #9834 after #8541 and preserve your new test/PATH changes on rebase. Please let me know if you prefer another order.

@DoubleAAGuy

Copy link
Copy Markdown

I moved the HTTPS integration test into the new guest_http_transport test, so #9834 no longer changes libSceSsl/CMakeLists.txt or GuestSslTransport.cpp. The transport test links the existing libSceSsl and runs against its TLS verifier; your CMake/PATH change no longer conflicts. Thanks for the earlier overlap review.

@oneandonlydean oneandonlydean self-assigned this Oct 22, 2026

@harisibrar1 harisibrar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked against main 522e32e and the libSceSsl Export.cpp / libkernel Udp.cpp sources on main:

  • It still merges cleanly onto current main. Main's sceSslTerm still only erases the context, and FindConnection rejects the orphan afterwards, so the leak this fixes is still on main.
  • I traced the ownership: Session holds shared_ptr<Context>, and Establish copies the session under the registry lock, so an operation in progress keeps both alive after Term; the handshake-failure path looks the connection up again by id; and nothing destroyed under state.mutex takes that mutex again. I found no use-after-free, double-free or deadlock.
  • Optional nit: sceSslDeleteConnection declares session before the lock_guard, so the last Session reference is released after unlock. In the new Term, erase_if drops idle sessions inside the lock, including socketOwner, which can be the last reference to the libkernel Socket (its destructor closes the host socket) if the title already closed the descriptor. Moving the erased sessions into a vector declared before the lock would match DeleteConnection. From reading the code, I don't think it can deadlock; it's only extra work under the global SSL mutex.

Reviewed with AI assistance (Claude Code); the checks above were run locally on my machine.

@Ondra97

Ondra97 commented Oct 24, 2026

Copy link
Copy Markdown
Contributor Author

@harisibrar1 Thanks for tracing the ownership and destructor paths. I am keeping the current erase_if for this patch because registry removal requires no temporary container allocation. Collecting sessions in a vector would add an allocation-failure point to context termination.

The lock-hold trade-off is real: destroying an idle session can also release the last native socket owner while state.mutex is held. I made that explicit in the PR description. In-flight operations retain their shared session/context ownership as reviewed; the source remains at f2173bba34562299f9d90583120ade2205ae12ef.

AI-assisted: yes.

@oneandonlydean
oneandonlydean merged commit d2b3ee3 into boykopovar:main Oct 25, 2026
13 of 14 checks passed
@Ondra97
Ondra97 deleted the codex/fix-ssl-context-connections branch October 25, 2026 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants