Skip to content

fix(libkernel): refuse a loopback TCP connect at once on Windows - #5141

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
0xRickx:fix/refused-loopback-connect
Oct 16, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
0xRickx:fix/refused-loopback-connect

Conversation

@0xRickx

@0xRickx 0xRickx commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

What

On FreeBSD, a TCP connect fails with ECONNREFUSED as soon as the peer answers the SYN with a reset (POSIX connect: "The target address was not listening for connections or refused the connection request"). Windows instead retransmits the SYN twice after a reset. On this machine a blocking connect to a loopback port with no listener takes 2.04 s, against 0.2 ms once the retransmissions are off.

Homebrew that looks for a local service by connecting to 127.0.0.1 waits that long on every attempt. PS5VR (PPSA99177) probes its helper service on port 17600 up to 21 times before its library screen (main.cpp, app_bridge.c). On main its log shows ps5vr: sandbox open at 23.7 s and ps5vr: dev service absent at 68.6 s: 45 s of refused connects. With this change the two lines are 2.0 s apart, which is PS5VR's own 20 × 100 ms sleep.

Before a stream socket connects to a loopback address (127.0.0.0/8 or ::1), libkernel's connect and libSceNet's sceNetConnect now check whether any socket listens on the destination port (GetExtendedTcpTable with TCP_TABLE_OWNER_PID_LISTENER, IPv4 and IPv6). When none does, they set SIO_TCP_INITIAL_RTO on the host socket with TCP_INITIAL_RTO_NO_SYN_RETRANSMISSIONS, which sets the SYN retransmissions to 0, so the reset fails the connect at once, as on FreeBSD.

  • A port with a listener keeps Windows' retransmissions. Windows also answers a SYN to a listener whose accept backlog is full with a reset, while FreeBSD drops that SYN; in both cases the retransmission connects once the server accepts.
  • Connects to other hosts are unchanged: they need the retransmissions when a SYN is lost. A remote host's reset still costs the Windows wait.
  • If the table cannot be read, or the control is refused, the connect behaves as before.
  • Linux already fails at once and is unchanged.

The check is RefuseClosedLoopbackAtOnce_nid_no_patch in a new libkernel/Socket/src/SocketConnect.cpp; libSceNet calls it from libkernel, and libkernel links iphlpapi on Windows.

Tested

Windows 11 (26200), WinLibs GCC 15.2.0 r7, Release, main ff23657 (rebased onto 0e6a5e7 afterwards, no conflicts):

  • guest_refused_connect, with libkernel's connect and with sceNetConnect:

    • binds a TCP socket to 127.0.0.1:0, reads its port, closes it, connects to that port and requires ECONNREFUSED (61 / 0x8041013D) in under one second;
    • listens with a backlog of 1, queues one connection that the server accepts after 100 ms, and requires a second connect to succeed.

    It passes in 1.05 s. With main's behaviour it aborts in the closed-port case (2.34 s), and without the listener check in the backlog case (0.40 s).

  • A probe calling libkernel's socket, listen, connect and accept, 3 rounds per case:

    Case main without the listener check this PR
    Closed port ECONNREFUSED after 2047-2050 ms after 0.12-0.19 ms after 0.45-0.71 ms
    Backlog 1 with one queued connection, server accepts from 100 ms, second connect connects after 510-512 ms ECONNREFUSED at once connects after 511-515 ms
    8 concurrent connects, server accepts from 100 ms 8 of 8 not run 8 of 8
  • PS5VR (PPSA99177, catalog 1.2.0), relinked with this main, second run (warm shader cache): ps5vr: sandbox open at 7.11 s, ps5vr: dev service absent at 9.25 s, its home screen's library: list Library -> ok, 7 items at 10.26 s. On main the first two lines are about 45 s apart.

  • All 27 socket tests pass: guest_refused_connect, guest_sce_net, the four guest_datagram_*, the guest_socket_* tests and patched_libraries. tools/check_conventions.py --base upstream/main passes.

  • Linux (Ubuntu 24.04 in WSL, GCC 13.3, Release, the CI's configure options, lavapipe), merged onto main 92e5041 together with fix(libSceSystemService): report the host time zone and summer time #5084: guest_refused_connect, guest_sce_net, the four guest_datagram_*, the guest_socket_* tests and patched_libraries pass, except guest_socket_read_write. Full ctest: 1263/1273; the ten failures (nine agc_* tests and guest_socket_read_write) fail the same way on main 92e5041 alone in this setup.

  • git merge-tree: merges cleanly with main, fix(libkernel): report SO_ERROR and the other SOL_SOCKET options like FreeBSD #4796, fix(libSceNet): let a datagram receive on an unbound socket wait on Windows #5966, fix(libSceNet): close a libkernel socket through sceNetSocketClose #6140 and fix(libSceNet): set an option on a libkernel socket through sceNetSetsockopt #6142. feat(libkernel): carry kqueue read and write events on pipes and terminals #4910 conflicts only in files this PR does not touch. The call in sceNetConnect sits before the address-family check, and libSceNet declares the helper instead of including a header, so the change stays off the lines those PRs edit.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: none
  • AI-assisted: yes

🤖 Generated with Claude Code

@CnRJay CnRJay left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The diagnosis is right: Winsock retries a SYN after a RST, and FreeBSD doesn't. One case this change makes worse, though: a loopback listener whose accept backlog is full.

Windows answers a SYN to a listener with a full backlog with a RST, the same as for a closed port. FreeBSD instead drops the SYN when the listen queue is full (or answers through its syncache), and the client's retransmission connects once the server has accepted the earlier connections. On main, Windows' SYN retries happen to cover that case. With TCP_INITIAL_RTO_NO_SYN_RETRANSMISSIONS the connect fails with ECONNREFUSED at once. A title or homebrew that opens a burst of connections to a local server, for example several HTTP requests to an in-process service, or a server that calls listen with a small backlog, would then see refusals that neither FreeBSD nor main produce.

Ways to keep the fast refusal for closed ports without breaking busy ones:

  • Keep one SYN retransmission with a short initial RTO (TCP_INITIAL_RTO_PARAMETERS{Rtt = a few ms, MaxSynRetransmissions = 1}) rather than none. A closed port then fails in a few milliseconds, and a full backlog gets one more chance.
  • Or retry the connect once in connect itself after a short delay when it is refused on loopback.

A test with a loopback listener of backlog 1 and two quick connects would pin the behaviour either way.

@LeandroLP23

Copy link
Copy Markdown
Contributor

Windows/AMD check with PS5VR (PPSA99177 v1.2.0, catalog release): Windows 11 (26300), Radeon 860M (RDNA 3.5, driver 32.0.13058.8001), WinLibs GCC 15.2.0, Release, relinked with --windows, same relinked executable for both builds, only the .prx files swapped.

Time from launch until the title logs library: list Library -> ok (right before it draws its home screen), polled from its log every 200 ms:

Build Run 1 Run 2
main 66f8947 46.7 s 46.7 s
main 66f8947 + this PR (f87099f) 4.0 s 4.0 s

On main, gdb attached at 25 s shows the main thread inside connect_nid_postfix → ws2_32!connect, and a breakpoint on connect prints 127.0.0.1:17600 every time (main thread and one more thread, 33 calls in about 37 s). A plain Python connect to that port on this machine takes 2.04 s to be refused, the same figure as in the description.

guest_refused_connect passes.

With the PR the title shows its home screen ("Put on your PlayStation VR2", the phone-link QR code for port 8090):

PS5VR home

That's the only frame it presents without a headset: its main loop draws the home screen once and then only presents while a VR session is active (main.cpp), so "FPS 0.00 (1)" in the window title is expected here.

AI-assisted: yes (Claude Code)

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the diagnosis and the PS5VR numbers hold up. One point to resolve before merging, the one CnRJay raised and that is still open:

  • core/libs/prx/libkernel/Socket/include/SocketConnect.hpp:17: TCP_INITIAL_RTO_NO_SYN_RETRANSMISSIONS also turns a loopback listener with a full accept backlog into an immediate ECONNREFUSED, because Windows answers that SYN with a RST just like a closed port. FreeBSD drops the SYN (or answers through the syncache) and the client's retransmission connects once the server accepts, and main currently gets the same result from Windows' retries. A burst of connections to a local server with a small listen backlog would regress. Keeping one SYN retransmission with a short initial RTO (MaxSynRetransmissions = 1, Rtt of a few ms) keeps the closed-port case fast and leaves the full-backlog case one more chance.
  • core/libs/tests/GuestRefusedConnect.cpp: add a case with a loopback listener of backlog 1 and a second quick connect that must succeed once the first is accepted, so the behaviour is pinned either way, and keep the closed-port timing check.

Please also report the closed-port time with whatever setting you choose (it should still be well under the 1 s limit).

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 5280 core/libs/prx/libkernel/CMakeLists.txt L83

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 4972 core/libs/prx/libkernel/CMakeLists.txt
PR 5021 core/libs/prx/libkernel/CMakeLists.txt
core/libs/prx/libkernel/Socket/src/Udp.cpp
PR 6044 core/libs/prx/libSceNet/CMakeLists.txt
core/libs/prx/libkernel/CMakeLists.txt
PR 6222 core/libs/prx/libkernel/CMakeLists.txt
core/libs/prx/libkernel/Socket/src/Udp.cpp
PR 6447 core/libs/prx/libSceNet/Export.cpp

@screamm

screamm commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

NVIDIA check with PS5VR (PPSA99177 v1.2.0, catalog release) as a second vendor next to the AMD run above, plus the backlog-full case from the review.

  • Head f87099f6f merged onto main 2559e9623: clean, no conflicts (merge 06a1d9dda). Windows 11 Pro 26220, WinLibs MinGW-w64 GCC 15.2.0 r7, Ninja, Release, NVIDIA GeForce RTX 4070 (driver 596.21).
  • Build: full default build and libs pass, no new warnings.
  • --repeat until-fail:3: guest_refused_connect, guest_sce_net, the four guest_datagram_* and the nineteen guest_socket_* tests: 25/25, 3 of 3.
  • Negative control: guest_refused_connect_tests.exe with main's libraries first on PATH aborts after 2.2-2.3 s (3 of 3).
  • ctest -j8 --timeout 120 -E '^agc_driver_|^video_out_flip': 812/813. The one failure is hardware_oracle_compare (20 s limit), which also times out on main under -j8 and passes 3 of 3 when run alone.
  • PS5VR relinked with relinker --windows --registry unused-filter=0. Both builds were seeded with the same warm shader cache, because a cold run spends about 95 s compiling before sandbox open. Times are from launch, read from the log every 100 ms:
Build ps5vr: sandbox open ps5vr: dev service absent library: list Library -> ok, 7 items Gap Frames
main 2559e96 3.3 s 48.2 s 48.3 s 44.9 s 1
main (2nd run, machine under heavier load) 41.1 s 85.8 s 86.7 s 44.7 s 1
main + #5141 4.1 s 6.1 s 6.3 s 2.0 s 1
main + #5141 3.4 s 5.4 s 5.6 s 2.0 s 1

Every run reaches the home screen ("Put on your PlayStation VR2") with PS5VR | FPS: 0.00 (1), as expected without a headset. There is no stop line; the log ends with vr: headset setup failed.

  • Backlog-full probe (not part of the PR). It is guest code calling libkernel socket/listen/connect/poll/accept against listen(fd, 1) on 127.0.0.1. One binary was built, then run with main's or this PR's libkernel.prx first on PATH. Each row has 3 rounds:
Case main main + #5141
Closed port ECONNREFUSED after 2046-2056 ms ECONNREFUSED after 0.2-0.3 ms
First connect queued (not accepted), server starts accepting at 100 ms, then a second and third blocking connect second connects after 512-516 ms, third after 0.5 ms second and third fail with ECONNREFUSED (61) after 0.1-0.2 ms
8 concurrent connects, server accepts from 100 ms 8/8 connected (slowest 509-514 ms) 6/8 connected, 2 refused
32 concurrent connects, server accepts from 0 ms 32/32 connected (slowest 0.5-1.5 s) 6-8/32 connected, 24-26 refused

So the regression raised in review reproduces on Windows 11 26220. A SYN to a full backlog gets a RST. On main the SYN retry about 500 ms later connects; with TCP_INITIAL_RTO_NO_SYN_RETRANSMISSIONS the connect fails at once.

Verdict: the PS5VR speed-up holds on NVIDIA: the library screen comes about 42 s sooner, with the gap falling from 44.9 s to 2.0 s. The backlog-full regression is real, though, so the requested change still applies (keep a retransmission or retry for loopback).

AI-assisted: yes (Claude Code)

Windows answers a SYN to a closed port with a RST and then retransmits
the SYN, so connect to a closed loopback port takes about 2 s before
ECONNREFUSED; FreeBSD fails at once. PS5VR (PPSA99177) makes about 33
such connects to 127.0.0.1:17600 and reaches its home screen 45 s late.

When no socket listens on the destination port (GetExtendedTcpTable),
a loopback connect now sets SIO_TCP_INITIAL_RTO with no SYN
retransmissions and fails at once. A port with a listener keeps the
retransmissions: Windows also answers a SYN to a full accept backlog
with a RST, and the retransmission connects once the server accepts,
as the console's does after FreeBSD drops the SYN.

guest_refused_connect requires ECONNREFUSED in under a second for a
closed port through libkernel connect and sceNetConnect, and a second
connect to a listener with backlog 1 to succeed once the server accepts
the first, through both.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@0xRickx
0xRickx force-pushed the fix/refused-loopback-connect branch from f87099f to 831c583 Compare October 15, 2026 15:04
@0xRickx

0xRickx commented Oct 15, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @CnRJay for the backlog case and @adriadam10 for the review; @screamm's probe confirmed the regression.

I changed the approach instead of keeping one retransmission: the fast refusal now applies only when nothing listens on the destination port. Before a loopback connect, libkernel reads the listening TCP sockets (GetExtendedTcpTable, IPv4 and IPv6). If any listens on that port, the connect keeps Windows' retransmissions exactly as on main, so a full backlog connects once the server accepts. Only a port with no listener gets TCP_INITIAL_RTO_NO_SYN_RETRANSMISSIONS; that is the case where FreeBSD also answers with a reset and the connect fails at once.

One retransmission with a short initial RTO would still refuse whenever the server accepts later than that RTO. With a backlog of 1 and a server that accepts after 100 ms, a retry a few milliseconds later still finds the backlog full.

guest_refused_connect now also listens with a backlog of 1, queues one connection that the server accepts after 100 ms, and requires a second connect to succeed, through libkernel's connect and sceNetConnect. The closed-port check stays.

Windows 11 (26200), WinLibs GCC 15.2.0 r7, Release, head 831c583 on main 0e6a5e7; a probe calling libkernel's socket, listen, connect and accept, 3 rounds per case:

Case main without the listener check this head
Closed port ECONNREFUSED after 2047-2050 ms after 0.12-0.19 ms after 0.45-0.71 ms
Backlog 1 with one queued connection, server accepts from 100 ms, second connect connects after 510-512 ms ECONNREFUSED at once connects after 511-515 ms
8 concurrent connects, server accepts from 100 ms 8 of 8 not run 8 of 8
  • guest_refused_connect passes in 1.05 s. It aborts in the closed-port case with main's behaviour (2.34 s), and in the backlog case without the listener check (0.40 s).
  • PS5VR (PPSA99177), relinked, warm shader cache: ps5vr: sandbox open at 7.11 s, ps5vr: dev service absent at 9.25 s, library: list Library -> ok, 7 items at 10.26 s.
  • All 27 socket tests pass.
  • Linux (Ubuntu 24.04 in WSL, GCC 13.3, Release, the CI's configure options, lavapipe), merged onto main 92e5041 together with fix(libSceSystemService): report the host time zone and summer time #5084: guest_refused_connect, guest_sce_net, the four guest_datagram_*, the guest_socket_* tests and patched_libraries pass, except guest_socket_read_write. Full ctest: 1263/1273; the ten failures (nine agc_* tests and guest_socket_read_write) fail the same way on main 92e5041 alone in this setup.

The call in sceNetConnect now sits before the address-family check, and libSceNet declares the helper instead of including the header, so the change merges cleanly with #5966, #6140 and #6142. The description is updated.

AI-assisted (Claude Code).

@screamm

screamm commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Re-ran my backlog probe on head 831c58300 (merged onto main 44df21632, no conflicts). Windows 11 Pro 26220, WinLibs MinGW-w64 GCC 15.2.0 r7, Release, NVIDIA RTX 4070 (driver 596.21). The probe is the same guest code as before plus sceNetConnect variants, run against main's or this head's libraries, 6 samples per cell:

Case (127.0.0.1, listen(fd, 1)) main this head
Closed port, connect / sceNetConnect ECONNREFUSED after 2040-2057 ms after 0.6-2.6 ms
One queued connection, server accepts from 100 ms, second connect (connect and sceNetConnect) connects after 501-516 ms connects after 504-516 ms
One queued connection, server never accepts, second and third connect refused after 2037-2059 ms refused after 2032-2070 ms
8 / 32 concurrent connects 8/8 and 32/32 8/8 and 32/32

So @adriadam10's point is resolved: a full backlog behaves as on main, and the previous head's refusals (2 of 8 and 24-26 of 32) are gone. The listener lookup adds about 0.2-0.6 ms to a loopback connect to a listening port (per-round minimum 0.53-0.98 ms vs 0.30-0.39 ms, 6 rounds of 50 connects each).

  • guest_refused_connect, 26 more socket/net tests and patched_libraries pass 3 of 3. guest_sce_net aborted once in the 3x run. In 10 direct runs it aborts 3/10 with this head's libs and 2/10 with main's under the same host load, so I don't attribute it to this PR.
  • PS5VR (PPSA99177), 120 s, warm shader cache: main sandbox open 4.9 s → dev service absent 49.8 s; this head 2.6 s → 5.2 s, library: list Library -> ok, 7 items at 5.8 s. 1 frame each, no stop line.
  • check_conventions.py passes, git diff --check is clean.

Not checked: full build and non-GPU suite, IPv6 ::1, Linux, the CI runner.

Written with Claude Code assistance (validation by a subagent, reviewed in this session).

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the new head against the open point: the fast refusal now applies only when GetExtendedTcpTable shows no listener on the destination port (IPv4 and IPv6), so a full backlog keeps Windows' retransmissions; guest_refused_connect covers both the closed port and backlog 1 through connect and sceNetConnect. Local build and ctest pass, and the Windows results from @screamm and @LeandroLP23 on this head cover the Windows behaviour; merging once the Windows build is green.

@adriadam10
adriadam10 merged commit 4ff1e88 into boykopovar:main Oct 16, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants