Skip to content

fix(http): trust the system CA list for HTTPS requests - #11134

Open
p3n1cillin wants to merge 18 commits into
boykopovar:mainfrom
p3n1cillin:claude/http-system-roots-v2
Open

p3n1cillin wants to merge 18 commits into
boykopovar:mainfrom
p3n1cillin:claude/http-system-roots-v2

Conversation

@p3n1cillin

@p3n1cillin p3n1cillin commented Oct 25, 2026 •

Copy link
Copy Markdown

What

Depends on #9834, which adds libSceHttp's request transport. The libSceSsl helper this PR calls came from #11145, now merged. The stack, from the bottom:

The libSceSsl code is the one merged with #11145, as MarcZX10 asked on #11145.

With #9834, every HTTPS request that doesn't install a verify callback fails with 0x80431063, last error 0x8095F00C (unknown CA). The cause is that the libSceSsl context the title passes to sceHttpInit starts with no trust anchors unless the title loads some itself. Titles that rely on the console's built-in trust can't reach any public server.

  • PokeMMO Prospero (PPSA98001) makes its Google sign-in and client download through libSceHttp. It calls no sceHttps* function, loads no certificate and installs no verify callback. Its README reports the Google Drive backup working on firmware 12.40.
  • ProsperoStore (PPSA99000) also loads no certificate, but explicitly enables the known-CA check with sceHttpsEnableOption before fetching its public catalog.

On the console, both requests pass the known-CA check against public roots, so the system must trust them by default. That is inferred from the titles, not measured.

The change (the libSceSsl parts came with #11145):

  • libSceSsl reads the system CA list, /<sandbox word>/common/cert/CA_LIST.cer, once per process. libkernel already serves that file from the host trust store (feat(libkernel): serve the sandbox CA bundle from host TLS roots #5280). The roots are parsed into a separate allocation pool, so they aren't charged to the title's sceSslInit pool; PokeMMO's is 512 KiB.
  • A new internal call, SslTrustSystemRoots_nid_no_patch(connection), makes one connection trust those roots after the context's own authorities. The session links them into the chain instead of copying them, and unlinks them before freeing.
  • libSceHttp calls it for every HTTPS connection.
  • A missing list fails loudly: if the sandbox word is unavailable, or the list can't be opened, is empty or holds no certificate, the call throws and names the path. Nothing is cached after a failure, so a later call tries again (from the reviews on fix(http2): trust the system CA list for HTTPS requests #11145).
  • Unchanged: a bare sceSslCreateConnection, and what sceSslGetCaList and sceSslGetCaCerts report. Whether a direct libSceSsl connection on the console trusts the system roots is not measured.
  • Debt entries in both libraries say what is inferred and what isn't measured.

Tested

Windows 11 Pro (26300), Ryzen 5 7600X, WinLibs GCC 15.2.0, Release, full build of head 35c09d332f (merges cleanly into main e49d43d625). 164 of 164 pass: the 98 tests that libSceHttp, libSceHttp2, libSceSsl and libSceNet register (including the two trailer cases in guest_http_transport and #11145's five system CA list tests), public_headers with its 64 compile tests, and guest_offline_imports. The libSceHttp code of this PR is unchanged from the earlier stack; the title runs below were made on that stack:

  • All tests that libSceHttp, libSceNet and libSceSsl register pass, including guest_http_transport with a new no-callback case.
  • PokeMMO Prospero (PPSA98001): before this change it logs cloud: POST send failed (0x80431063). With it:
    • it shows Google's "Back up to Google Drive" screen with a device code (sign-in code shown (… at https://www.google.com/device));
    • after "not now", its updater checks the release over HTTPS and downloads and installs its 92 MB client (client update: download done, 92 MB in 2 s, RESULT client.update PASS);
    • it then stops at its ROM screen, which needs the user's own ROMs.
  • ProsperoStore (PPSA99000), with feat(libSceAppInstUtil): answer for an installer service that never installs #8346 also merged because it stops on sceAppInstUtilInitialize without it, from a cleared /data/prosperostore, 60 s each:
    • without this change: "The catalog is on its way";
    • with it: the catalog loads, "New in the catalog 12", and the store's signed update check offers 01.000.106.
  • Prospero Themes (PPSA90210), which installs its own accept-all callback, still loads its community store.

The new guest_http_transport case makes an HTTPS request with no callback to the fixture TLS server:

  • WSL Ubuntu 26.04.1, GCC 15.2.0: the test sets SSL_CERT_FILE to the fixture root, so the request succeeds. With the libSceHttp call removed, the test fails. The transport and libSceSsl tests pass.
  • Windows: the roots come from the user's real store, so the case checks for 0x8095F00C.

tools/check_conventions.py --base origin/main passes.

Checklist

🤖 Generated with Claude Code

@github-actions github-actions Bot added the unblocks title Names a title ID on the Unblocks line label Oct 25, 2026
@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Progress report (bff0289 → 35c09d3)

📈 System libraries: 26.34% (+0.01%, +1 implemented functions)

✅ 1 implemented
Library Name
libSceHttp sceHttpAddRequestHeaderRaw

Changed libraries (implemented exports)

Library Before After Change
libSceHttp 86/115 (74.78%) 87/115 (75.65%) +0.87%

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found one blocking input-bound issue in the chunked response reader; details are attached to the terminal-chunk loop below. I did not complete a full review of the 2,194-line diff, so this request is limited to that finding.

if (const int parseResult = ParseChunkSize(line, request.chunkRemaining); parseResult != 0) return parseResult;
if (request.chunkRemaining == 0) {
for (;;) {
if (ReadLine(*request.stream, request.pending, request.pendingOffset, line, kMaximumChunkLine) != 0) return ERROR_NETWORK;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please cap the aggregate trailer section and reclaim consumed input as the parser advances. ReadLine limits each line to 8192 bytes but appends each received block to pending while only advancing pendingOffset; this loop accepts an unlimited number of nonempty trailer lines before the blank terminator, with no aggregate byte/count limit. A peer can therefore keep the synchronous request in this loop and grow the retained buffer without bound. The same retained-prefix issue can accumulate chunk framing on long streamed responses. Please enforce a total trailer limit (for example, based on the configured response-header maximum), compact or otherwise bound consumed pending data, and add regression coverage for an over-limit trailer block and sustained chunked input.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. The trailer loop at Request.cpp:720 is in #9834's transport commit (feat(libSceHttp): add synchronous HTTP request transport). This PR's own commits don't touch ReadLine, pending or the chunk reader, and the loop is unchanged at #9834's current head f658abbf1e. So I've asked for it on #9834, so that it's fixed where the code lives, and I'll restack this PR on whatever lands there.

#9834 has also been rebased since I opened this, so I'm moving these commits onto its new head. That applies cleanly, and I'll post the test results once the build finishes.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restacked on #9834's head f658abbf1e (new head 0e548ae912). The four commits applied cleanly, and all 80 tests registered by libSceHttp, libSceSsl and libSceNet pass on Windows. The trailer bound is still waiting on #9834.

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 10535 core/libs/prx/libSceHttp/CMakeLists.txt
core/libs/prx/libSceHttp/Export.cpp
core/libs/tests/GuestHttp.cpp
docs/dev/debt/libSceHttp.md
PR 12056 core/libs/tests/GuestHttp.cpp
docs/dev/debt/libSceHttp.md
PR 14212 docs/dev/debt/libSceHttp.md
PR 15965 docs/dev/debt/libSceHttp.md
PR 16117 core/libs/prx/libSceHttp/Export.cpp
docs/dev/debt/libSceHttp.md

@p3n1cillin
p3n1cillin marked this pull request as ready for review October 25, 2026 20:30
@p3n1cillin

Copy link
Copy Markdown
Author

A related console observation for the scope question in this PR's debt entries, from PS5SX2 (PPSA99203). Its HTTPS client header, fe_https.h, says:

  • After the jailbreak, the console's libSceSsl can't reach its certificate store: sceSslGetCaList answers 0x80020002.
  • So libSceSsl trusts no site unless it is handed roots.

The author marks the client "AI-assisted; needs proper testing", so this is weaker evidence than a measurement.

Read together with this PR's two titles:

  • A normal process: libSceSsl presumably has a system certificate store behind sceSslGetCaList. That suggests a retail title's bare libSceSsl connection may trust the system roots by default too, which this PR deliberately leaves unchanged.
  • A jailbroken homebrew process: PokeMMO and ProsperoStore are both homebrew, yet their HTTPS through libSceHttp works on hardware. So libSceHttp gets trust in a way that doesn't depend on libSceSsl's store.

I've kept the change to libSceHttp, where the titles show it. Making bare libSceSsl contexts trust the system roots, and having sceSslGetCaList report them, would need a retail-process measurement first. I can add that to the debt entry if you want it recorded.

AI-assisted: yes (Claude Code)

@p3n1cillin

Copy link
Copy Markdown
Author

A precision on the ProsperoStore evidence in the description. I re-read its transport selection and found a second homebrew note on this.

  • ProsperoStore (store_http.cpp) uses libcurl with /system/common/cert/CA_LIST.cer only when it is elevated and that file is readable. Otherwise it uses libSceHttp with no certificate of its own. On the console the libSceHttp path is the sandboxed one. Under AnyPS5 it logs transport elevated=1 ca_list=0 -> sceHttp, because nothing is served at /system/common/cert/CA_LIST.cer.
  • VITA5 (PPSA39410), docs/CURL.md, lists this symptom: sceHttp/sceSsl fail with 0x8095f00c on every HTTPS site in an elevated app, because the system TLS stack rejects public certificates once elevated. PS5SX2's note in my previous comment says the same.

So the evidence supports default trust for a sandboxed process, which is what a retail title is. It doesn't apply to an elevated homebrew process: there the console's system TLS stack fails with unknown CA, and these homebrew titles fall back to their own TLS. AnyPS5 reports a root identity (libkernel debt: getuid and the others return 0), so a homebrew title may take its elevated branch here. The debt entry says the trust follows the sandboxed behaviour; I'll add the elevated case to it.

AI-assisted: yes (Claude Code)

@p3n1cillin
p3n1cillin force-pushed the claude/http-system-roots-v2 branch from cc3a1f0 to db2335e Compare October 25, 2026 20:38
@p3n1cillin

Copy link
Copy Markdown
Author

One more sandboxed title for the default trust. DOOM Native Port (PPSA99666) creates its libSceHttp template with no certificate, callback or option (http.c). Its DEVELOPMENT.md records from hardware that "sceHttp with sceSsl works in the sandboxed title, plain and HTTPS".

AI-assisted: yes (Claude Code)

@p3n1cillin
p3n1cillin force-pushed the claude/http-system-roots-v2 branch from db2335e to 0e548ae Compare October 25, 2026 21:56
@github-actions github-actions Bot added build docs prx needs rebase Conflicts with main, rebase needed labels Oct 25, 2026
@p3n1cillin
p3n1cillin force-pushed the claude/http-system-roots-v2 branch from 0e548ae to 17b2aef Compare October 26, 2026 12:41
@p3n1cillin

Copy link
Copy Markdown
Author

Restacked on #9834's new head bc8320aec4 (new head 17b2aefb4a; it applied cleanly). I also added one commit from a review of the same code on #11145: a null sandbox word or a system CA list that can't be opened now throws, naming the path, instead of caching an empty chain. All 82 tests that libSceHttp, libSceSsl and libSceNet register pass on Windows. The trailer bound MarcZX10 asked for is still in #9834's code, unchanged at bc8320aec4.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 26, 2026
@p3n1cillin
p3n1cillin force-pushed the claude/http-system-roots-v2 branch from 17b2aef to aa5fd7b Compare October 26, 2026 22:43
@p3n1cillin

Copy link
Copy Markdown
Author

@MarcZX10 I restacked this PR at head aa5fd7b719.

@p3n1cillin

Copy link
Copy Markdown
Author

@jotitadev Thanks, confirmed on Windows: public_headers fails on 0618c3d9c1 with the same duplicate declarations. Fixed in f9f85640fd, now the head.

What it changes:

  • Includes instead of redeclarations: Request.cpp, GuestHttpTransport.cpp and HttpTransportContextAllocation.cpp include Http.hpp, Net.hpp and Ssl.hpp instead of redeclaring the exports.
  • Request.cpp bridges: it keeps only its four *_nid_no_patch lines (NetSend, NetRecv, SslRead and this PR's SslTrustSystemRoots).
  • Pool stats test: rather than casting to ::HttpMemoryPoolStats next to a same-named local struct, the allocation test drops its private copy and uses the header's struct. It has the same 24-byte layout, so only the field names changed (peakUsedSize, usedSize), and the static_assert stays.

Rerun on that exact head (Windows 11 Pro 26300, Ryzen 5 7600X, WinLibs GCC 15.2.0, Release, full build). This covers the Windows side you hadn't checked, with Net.hpp included after winsock2.h:

  • 163 of 163 pass: all 97 HTTP/HTTP2/SSL/net tests, public_headers, the 64 public_headers_compile_* tests, and guest_offline_imports.
  • guest_http_transport passes 3 of 3 runs.
  • tools/check_conventions.py --base origin/main passes.
  • The head merges cleanly into current main 9f748f4d8e.

The duplicates come from #9834's files, so I've also added the same commit to the fix branch I offered on #9834.

@jotitadev jotitadev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked f9f85640fd on Linux: public_headers passes, so both sides are covered now.

Ubuntu 26.04, i7-8700, GCC 15.2, Release, BUILD_TESTING=ON. Head merged (--no-commit) onto current main 16345cad6a; clean merge, full build OK.

  • public_headers and guest_http_transport*: pass 3/3.
  • HTTP/SSL/net tests, public_headers* and guest_offline_imports: 160/160.
  • Control: reverting only f9f85640fd on the merge brings back the same 43 duplicate declarations and public_headers fails.

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

p3nicillin and others added 6 commits October 31, 2026 00:50
The trailer lines after the last chunk now count against the response
header maximum, and a trailer block over it fails the read with
ERROR_PARSE_HTTP_INVALID_RESPONSE as the last error. Consumed bytes of
the pending buffer are dropped once it is empty or the consumed prefix
reaches the I/O block size, so a long chunked body no longer keeps all
earlier input in memory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
public_headers requires each export to be declared once, in its library's
public header. Request.cpp and the two transport tests redeclared the
sceHttp, sceNet and sceSsl exports they call in their own extern "C"
blocks. Include Http.hpp, Net.hpp and Ssl.hpp instead. The allocation
test now uses the header's HttpMemoryPoolStats instead of its own copy of
the same 24-byte layout.

Reported by jotitadev on boykopovar#11134.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d check

d4e20e7 brought back the NpEntitlementAccess abort/delete loop that
expects both calls to throw. main removed that loop when 2456bef made
them idempotent, so guest_offline_imports aborted at it. Drop the loop
again; only the sceHttpSendRequest expectation differs from main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
libSceHttp's TLS connections trusted only certificates the title had
loaded, so an HTTPS request without a verify callback always failed with
an unknown CA. PokeMMO Prospero (PPSA98001) makes its Google sign-in that
way, and its README reports it working on firmware 12.40.

libSceHttp now calls SslTrustSystemRoots_nid_no_patch, added by boykopovar#11145,
for every HTTPS connection, so the system CA list served by libkernel is
trusted after the context's own authorities. Bare libSceSsl connections
are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@p3n1cillin
p3n1cillin force-pushed the claude/http-system-roots-v2 branch from f9f8564 to 35c09d3 Compare October 31, 2026 01:05
@p3n1cillin

Copy link
Copy Markdown
Author

Restacked onto #9834's new head 974dfd5ade; the head is now 35c09d332f. #9834 now links ws2_32 itself, and on Windows its head builds and passes the HTTP tests without the NID bridges. So I've dropped my Windows commit (ws2_32 link, *_nid_no_patch bridges, test PATH). The only bridge declaration left in Request.cpp is this PR's own SslTrustSystemRoots_nid_no_patch.

#9834's new test commit also brought back the NpEntitlementAccess abort/delete loop that main had removed, which made guest_offline_imports abort. That's fixed here and offered on #9834 (comment).

Against the previous head f9f85640fd, the tree differs only by those removals (4 files, +3/-36). Rerun on the exact head (Windows 11 Pro 26300, Ryzen 5 7600X, WinLibs GCC 15.2.0, Release, full build):

  • Tests: 164 of 164 pass. That's the 98 tests registered by libSceHttp, libSceHttp2, libSceSsl and libSceNet, public_headers, the 64 public_headers_compile_* tests and guest_offline_imports.
  • Repeats: guest_http_transport and guest_offline_imports each pass 3 of 3 runs.
  • Conventions: tools/check_conventions.py --base origin/main passes.
  • Main: the head merges cleanly into main e49d43d625.

@XairRrR XairRrR left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — changes requested

This pull request does not merge cleanly into main (5cb015c): head 35c09d3 conflicts in

  • core/libs/prx/libSceHttp/CMakeLists.txt

Please rebase it onto current main. I will review the change once it merges cleanly.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build docs needs rebase Conflicts with main, rebase needed prx unblocks title Names a title ID on the Unblocks line

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants