Skip to content

fix(libc): reopen a closed stream in freopen instead of aborting - #9763

Merged
oneandonlydean merged 2 commits into
boykopovar:mainfrom
rooklz:fix/libc-freopen-closed-stream
Oct 28, 2026
Merged

oneandonlydean merged 2 commits into
boykopovar:mainfrom
rooklz:fix/libc-freopen-closed-stream

Conversation

@rooklz

@rooklz rooklz commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

What

freopen on a stream that had been closed aborted the process: FileStream::Reopen starts with GetHandle(), which throws FileStream: closed stream when the handle is gone. PPSA99630 (OpenMW, the Morrowind port in the homebrew catalog, v1.0.1, bring-up thread #9762) does that when /download0 cannot be used: its first freopen of stdout fails and leaves the stream closed, and a later freopen of it dies on main before the title prints anything:

[libc] unhandled exception 'St13runtime_error' thrown from 0x7ffff5d4ddda
terminate called after throwing an instance of 'std::runtime_error'
  what():  FileStream: closed stream

(0x35dda into libc.prx, freopen_nid_postfix.) FreeBSD's freopen keeps a closed FILE and reopens it: with fp->_flags == 0 it "holds on to it", opens the path on a new descriptor and makes fp that stream (freopen.c, releng/11.4). A failed open leaves it closed, with the open's errno.

Reopen now does the same for a closed stream: it opens the path through the caller's opener and adopts the handle, or fails with the open's error (ENOENT for an empty path, as open("") gives there). A path the resolver rejects before any open (an overlong name, an unavailable mount) used to reach Close() on the closed stream and throw the same way; freopen now closes only a stream that is open there and returns the resolver's error. A null filename on a closed stream answers EBADF, as a PS5 does (an open stream keeps answering ENOTSUP, since mode-only reopening is not supported here). The guest state reset and descriptor bookkeeping that the open-stream path did at its end moved into Adopt, which both paths use. A closed stream whose reopen fails keeps answering EBADF, as before. Only the standard streams can reach this, since a dynamic stream is freed by its fclose.

The debt entry records the differences from the console: the new descriptor is whatever the host hands out, where a PS5 takes the lowest free one, and a write to the closed stream answers EBADF here (as on main), where the console's fputs/fputc leave errno unchanged and its fprintf gives EINVAL.

Tested

  • PS5 (payload on the console's libc): after fclose(stdout) or fclose(stderr), freopen into a missing directory and of "" gives ENOENT, of a 320-byte path ENAMETOOLONG, of a null filename EBADF; a good path returns the same FILE* on the lowest free descriptor and reads back what was written. Verify run on 144298bfca: guest_standard_streams* pass; its AGC failures are main's on that runner.
  • Linux x86-64 container (Ubuntu 24.04, GCC 13.3, Release, llvmpipe): PPSA99630 on main a6dcea9279 with no usable /download0 dies with the lines above; with this change it runs through to its own data check and exits 0, as it does when /download0 works (details on Morrowind (OpenMW) (PPSA99630) #9762). With a complete package the title doesn't reach this path, so this is Unblocks: none; the trace is the title's, the behaviour is FreeBSD's.
  • guest_standard_streams gains CheckClosedStreamReopen (a closed tmpfile stream: a failed reopen into a missing directory gives ENOENT, one with a 300-byte name gives ENAMETOOLONG from the resolver, a null filename gives EINVAL, the stream stays EBADF after each; then a reopen for writing, a write, a reopen for reading and a read, and the file holds the text) and the --reopen-closed-stdout case (fclose(stdout), printf fails with EBADF, the overlong reopen fails with ENAMETOOLONG and printf still EBADF, freopen returns stdout, printf and puts land in the file). Hosted Linux runner: the first verify run passed those tests on the diff before the resolver fix; the run on 1c2eed30a2 passes them too (Ubuntu 24.04, GCC, Release).
  • macOS 26.2, Apple M5, Apple clang 21, x86-64 under Rosetta 2, on @mugurc's ci/macos-series-2 integration tree: --reopen-closed-stdout (with the overlong-path check), --closed-stdout and --exclusive-stdout pass; the plain case runs CheckClosedStreamReopen and then stops at that tree's pre-existing CheckFlushWhileReading failure (an fflush on a reading stream changes the next read there, before and after this change; noted for macOS support #1).
  • Windows 11, MinGW-w64 GCC 15.2 (UCRT), Release, by @Dom3005 on 1c2eed30a2 merged onto main 11f976b27c: the four guest_standard_streams* tests pass 5 of 5, including the new case and CheckClosedStreamReopen; with FileStream.hpp and FileAndHeap.cpp restored from main the two closed-stream cases abort with what(): FileStream: closed stream. The full suite passes 1887 of 1895, the 8 failures being AGC tests that fail on that machine's AMD GPU on main (AGC: f64 and division tests crash in amdvlk64 on RX 9060 XT (RDNA 4) since AMD driver 32.0.32015.2008 #7451, test(agc): read the sample order tests' depth through a sampled copy #9056, test(agc): align the array window test's texels at run time #8310).
  • check_conventions.py passes. pr_overlap.py reports no conflicting lines; the shared files it could not compare belong to PRs that conflict with main.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

@rooklz
rooklz force-pushed the fix/libc-freopen-closed-stream branch from 9194191 to a783c27 Compare October 22, 2026 13:33
@rooklz

rooklz commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (8b3e39832a): head a783c27f41, same diff.

@harisibrar1 harisibrar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked against main 8b3e398 and FreeBSD 11.4 lib/libc/stdio/freopen.c:

  • This merges cleanly, and main still throws from Reopen → GetHandle() on a closed stream, so it isn't duplicated. The new branch matches FreeBSD's _flags == 0 path: it opens on a new descriptor, and on failure the stream stays free with the open's errno.
  • One abort remains on the same path. When ResolvePathError_nid_no_patch fails, freopen_nid_postfix (FileAndHeap.cpp:365-367) calls stream->Close() for anything that isn't IsHostStandard(). A closed stream has a null handle, so it lands there, and Close() → GetHandle() throws FileStream: closed stream. For example, after fclose(stdout), freopen(<1024-byte path>, "w", stdout) still terminates, and so does a path under a blocked mount such as an unavailable /download0. FreeBSD returns NULL with ENAMETOOLONG/ENOENT and leaves fp free. Guarding that branch with stream->IsOpen() and adding the case to CheckClosedStreamReopen would close it.
  • Optional, pre-existing: for a NULL filename, FreeBSD answers EINVAL when fp is closed, where main returns 45 before looking at the stream.

Reviewed with AI assistance (Claude Code); the checks above were run locally on my machine.

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 5021 core/libs/prx/libc/CMakeLists.txt
core/libs/prx/libc/include/FileStream.hpp
docs/dev/debt/libc.md
PR 6085 core/libs/prx/libc/CMakeLists.txt
PR 6713 core/libs/prx/libc/CMakeLists.txt
docs/dev/debt/libc.md
PR 7081 core/libs/prx/libc/CMakeLists.txt
core/libs/prx/libc/src/FileAndHeap.cpp
docs/dev/debt/libc.md
PR 7386 docs/dev/debt/libc.md
PR 7447 core/libs/prx/libc/CMakeLists.txt
core/libs/prx/libc/include/FileStream.hpp
core/libs/prx/libc/src/FileAndHeap.cpp
docs/dev/debt/libc.md
PR 7648 docs/dev/debt/libc.md
PR 7715 core/libs/prx/libc/CMakeLists.txt
docs/dev/debt/libc.md
PR 8150 core/libs/prx/libc/CMakeLists.txt
docs/dev/debt/libc.md
PR 8937 core/libs/prx/libc/CMakeLists.txt
docs/dev/debt/libc.md
PR 9371 docs/dev/debt/libc.md
PR 9447 docs/dev/debt/libc.md
PR 9526 core/libs/prx/libc/src/FileAndHeap.cpp
docs/dev/debt/libc.md
PR 9677 docs/dev/debt/libc.md
PR 9775 core/libs/prx/libc/CMakeLists.txt
PR 9867 docs/dev/debt/libc.md
PR 9929 core/libs/prx/libc/CMakeLists.txt
docs/dev/debt/libc.md
PR 10113 docs/dev/debt/libc.md
PR 10222 docs/dev/debt/libc.md
PR 10226 docs/dev/debt/libc.md
PR 10229 docs/dev/debt/libc.md
PR 10245 docs/dev/debt/libc.md
PR 10247 docs/dev/debt/libc.md
PR 10250 docs/dev/debt/libc.md
PR 10253 docs/dev/debt/libc.md
PR 10265 docs/dev/debt/libc.md
PR 10324 core/libs/prx/libc/CMakeLists.txt
core/libs/prx/libc/include/FileStream.hpp
core/libs/prx/libc/src/FileAndHeap.cpp
docs/dev/debt/libc.md
PR 10349 docs/dev/debt/libc.md
PR 10433 core/libs/prx/libc/src/FileAndHeap.cpp
docs/dev/debt/libc.md
PR 10620 core/libs/prx/libc/src/FileAndHeap.cpp
docs/dev/debt/libc.md
PR 11332 docs/dev/debt/libc.md
PR 11737 docs/dev/debt/libc.md
PR 11848 core/libs/prx/libc/src/FileAndHeap.cpp
docs/dev/debt/libc.md
PR 11850 docs/dev/debt/libc.md
PR 11870 docs/dev/debt/libc.md
PR 11895 docs/dev/debt/libc.md
PR 12053 docs/dev/debt/libc.md
PR 12122 docs/dev/debt/libc.md

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The closed-stream reopen path still has a failure case on the current head. I verified the control flow against this commit; the existing test covers an open failure after path resolution, but not a path-resolution error.

const auto filename = "anyps5-closed-reopen-" + std::to_string(std::chrono::steady_clock::now().time_since_epoch().count());
FileStream stream(std::tmpfile());
Require(fclose_nid_postfix(&stream) == 0);
Require(freopen_nid_postfix("missing-closed-stream-dir/out.log", "w", &stream) == nullptr && *__error_nid_postfix() == 2);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This failure case reaches FileStream::Reopen and tests the open failure, but it does not cover an error from ResolvePathError_nid_no_patch before Reopen is called. That path still aborts for a closed stream: freopen_nid_postfix at core/libs/prx/libc/src/FileAndHeap.cpp:365-368 calls Close() when the resolver returns an error; IsHostStandard() is false once _handle is null, and Close() calls GetHandle(), which throws. After fclose(stdout), a freopen using an overlong path or unavailable mount therefore still terminates instead of returning nullptr with the resolver error. Please add this resolver-error case for a closed standard stream and guard the caller’s close with stream->IsOpen() while retaining the standard-stream detach handling. I could not run guest_standard_streams locally because there is no configured build and less than 700 MB free; git diff --check and current-main conventions pass.

@rooklz
rooklz force-pushed the fix/libc-freopen-closed-stream branch from a783c27 to 4142bfa Compare October 22, 2026 15:37
@rooklz

rooklz commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Right, the resolver-error branch still called Close() on the closed stream. Head 4142bfaca5: freopen_nid_postfix now closes only a stream that is open there (else if (stream->IsOpen()) stream->Close();) and returns the resolver's error, and both test cases gained an overlong-name reopen on the closed stream: nullptr with ENAMETOOLONG (63), the stream still EBADF, and the later reopen still works. On the macOS integration tree the --reopen-closed-stdout case passes with it and the plain case runs CheckClosedStreamReopen before stopping at that tree's pre-existing CheckFlushWhileReading failure (#8920). The hosted Linux run on this head is https://github.com/rooklz/AnyPS5/actions/runs/39806541901; I'll note the result.

@rooklz
rooklz force-pushed the fix/libc-freopen-closed-stream branch from 4142bfa to 1c2eed3 Compare October 22, 2026 15:39
@rooklz

rooklz commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

@harisibrar1 thanks, the resolver branch is the same one @MarcZX10 found and is fixed as above. I took the optional point too, since it is the same closed-stream path: head 1c2eed30a2 answers EINVAL to a null filename on a closed stream (FreeBSD's _flags == 0 check) and keeps ENOTSUP for an open one, with a check in CheckClosedStreamReopen. The hosted run on this head is https://github.com/rooklz/AnyPS5/actions/runs/39806954271.

@rooklz

rooklz commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

The hosted Linux run on 1c2eed30a2 passed: guest_standard_streams with all its cases, guest_fopen_errors, file_streams, guest_host_standard_error, guest_append_streams, guest_file_lock and guest_filesystem (https://github.com/rooklz/AnyPS5/actions/runs/39806954271).

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The latest head 1c2eed30a2d18862352c02b79f6fc7046349fb47 addresses both issues I raised: path-resolution failure only closes an open stream, and a null filename on a closed stream returns EINVAL. The new ordinary-stream and stdout tests cover resolver errors, ENAMETOOLONG, failed writes, successful subsequent reopen, and retained host stdout behavior. The author reports the focused Linux suite passing on this head; current hosted relinker/register-context/write-watch checks also pass, while product Windows is skipped. Current-main conventions, diff-check and merge-tree pass. This branch is based on 8b3e39832a, 32 commits behind current main (11f976b27c16ed91633f00039dba5dfd97dd7cc1); please refresh and rerun the stream tests before merge. I did not rerun them locally.

@Dom3005

Dom3005 commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Windows run of 1c2eed3 merged onto main 11f976b (clean merge). Windows is not in the description's list of tested hosts.

Environment: Windows 11 Home 10.0.26200, WinLibs MinGW-w64 GCC 15.2.0 r7 (UCRT), Ninja, Release, BUILD_TESTING=ON, AMD GPU.

With this PR: all four guest_standard_streams* tests pass 5 of 5 (--repeat until-fail:5), including the new guest_standard_streams_reopen_closed_stdout and CheckClosedStreamReopen in the plain case.

Baseline: I kept the PR's test and CMakeLists.txt and restored FileStream.hpp and FileAndHeap.cpp from main. guest_standard_streams and guest_standard_streams_reopen_closed_stdout then abort with 0xc0000409, what(): FileStream: closed stream, the crash in the description. --closed-stdout and --exclusive-stdout pass either way.

Full suite with this PR: 1887 of 1895 pass. The 8 failures are AGC tests that fail on this machine's AMD GPU on main too: the 4 f64 and division tests (#7451), the 3 sample_order tests (#9056) and image_array_window_past_layer_limit (#8310). Nothing in libc or libkernel fails.

Code read: no findings. On Windows the closed-stream branch opens through OpenNative, the opener fopen uses, so it gets the same handling. Close already clears directory and forgets the descriptor before Adopt runs, and a failed open leaves a static stream closed, as in FreeBSD's freopen.c.

AI-assisted: yes

@rooklz

rooklz commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

@Dom3005 thanks for the Windows run and the baseline comparison; the description's Tested section now lists it with your numbers. The OpenNative reading is right: on Windows the closed-stream branch takes the same opener as fopen, so the two hosts differ only in how the handle is produced.

@github-actions github-actions Bot added libc build docs needs rebase Conflicts with main, rebase needed labels Oct 22, 2026
@rooklz
rooklz force-pushed the fix/libc-freopen-closed-stream branch from 1c2eed3 to 68eb26b Compare October 23, 2026 15:55
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 23, 2026
@rooklz

rooklz commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (15e71dbe11): head 68eb26b4de. https://github.com/rooklz/AnyPS5/actions/runs/39982388734 (Ubuntu 24.04, GCC, Release, lavapipe, fails the job on any ctest failure): the conflict was freopen_nid_postfix, where #9768 on main folds the empty-name case into the resolver branch; this change keeps that structure and still closes the stream there only when it is open. guest_standard_streams with all its cases, guest_fopen_errors, file_streams, guest_host_standard_error, guest_append_streams, guest_file_lock and guest_filesystem pass; the run's two failures, agc_driver_queued_index_read_order and agc_driver_sgpr_pattern_fill_keys, are main's on that runner.

@rooklz

rooklz commented Oct 24, 2026

Copy link
Copy Markdown
Contributor Author

Measured on a PS5: the error values for a missing, empty or overlong path match, and the reopen takes the lowest free descriptor. A null filename on a closed stream gives EBADF rather than EINVAL, so 144298bfca changes that and its test. A write to the closed stream behaves differently there too (fputs/fputc leave errno unchanged and fprintf gives EINVAL), but that comes from main, so it is recorded in the debt entry rather than changed here.

@rooklz

rooklz commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

@MarcZX10 two loose ends here, both yours to close when you have a moment:

  1. Your inline thread on the ResolvePathError_nid_no_patch path (2026-10-22) is still unresolved, but it has been addressed since 4142bfaca5: the resolver-error branch now closes only a stream that is open there and returns the resolver's error, and both test cases gained an overlong-name reopen on the closed stream. Your re-review of 1c2eed30a2 confirmed both points — if it still reads right, could you resolve the thread?

  2. That re-review also signed off on EINVAL for a null filename on a closed stream. A later console measurement found the PS5 answers EBADF there instead, so head 144298bfca flips that behavior and its test (the missing/empty/overlong-path cases were measured alongside it and match). Since this reverses something you signed off on, it deserves a fresh look; the full relinker/register-context/write-watch matrix is green on that head.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 27, 2026
@rooklz
rooklz force-pushed the fix/libc-freopen-closed-stream branch from 144298b to 4c28647 Compare October 27, 2026 08:11
@rooklz

rooklz commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (12686b52de) after the merges at 07:53Z conflicted with the debt-doc bullet — resolution keeps main's measured fopen/freopen wording and our closed-stream bullet unchanged. No code changes; the two commits are now 9618254977 and 4c28647f0a (was 68eb26b4d/144298bfc), and the matrix is rerunning on the new head.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 27, 2026
@oneandonlydean
oneandonlydean merged commit 4c6fcfc into boykopovar:main Oct 28, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants