Skip to content

fix(libkernel): run sceKernelRename through the checks rename uses - #5377

Closed
DavidSamir wants to merge 6 commits into
boykopovar:mainfrom
DavidSamir:fix/libkernel-rename-directory-windows
Closed

DavidSamir wants to merge 6 commits into
boykopovar:mainfrom
DavidSamir:fix/libkernel-rename-directory-windows

Conversation

@DavidSamir

Copy link
Copy Markdown
Contributor

What

On Windows, sceKernelRename cannot rename a directory onto an existing empty directory: it returns 0x80020005 (EIO) and nothing moves. FreeBSD's rename(2) replaces an empty directory, the Linux build does, and rename in this tree already does on both hosts. A program that prepares a directory under a temporary name and renames it over the old, emptied one fails only through the sceKernel export.

sceKernelRename also reports 0x80020005 for every failure on both hosts, where rename and FreeBSD give the specific error (ENOTDIR, EISDIR, ENOTEMPTY, EINVAL, ENOENT).

Cause. rename runs RenamePath_nid_no_patch, which checks the two paths, removes an empty destination directory on Windows before the host rename, and maps each case to its error. sceKernelRename has its own shorter body that calls the host rename directly and turns any error into EIO, so on Windows the host refuses a directory onto an existing directory.

Observed on main 0214069, Windows 11 (10.0.26300), WinLibs GCC 15.2.0 r7, Release:

sceKernelRename(from, to) main on Windows main on Linux this PR, both hosts
directory onto an empty directory 0x80020005 0 0
directory onto a non-empty directory 0x80020005 0x80020005 0x80020042 (ENOTEMPTY)
directory onto a file 0x80020005 0x80020005 0x80020014 (ENOTDIR)
file onto a directory 0x80020005 0x80020005 0x80020015 (EISDIR)
directory into its own subdirectory 0x80020005 0x80020005 0x80020016 (EINVAL)
destination's parent is missing / is a file 0x80020005 0x80020005 0x80020002 / 0x80020014
file onto other/, other being a file 0, other replaced 0x80020005 0x80020014, both kept
file onto new/, new missing 0, new created 0x80020005 0x80020002

I have not found a title that stops on this. It is the directory case #4532 left as "not changed", and the error codes came out of a probe run on both hosts.

Six commits:

  1. fix(libc): rename a directory onto a name written with a trailing slash: Rename.cpp. rename("dir", "new/") fails with ENOENT on both hosts today; FreeBSD allows a trailing slash on the destination when the source is a directory (kern_renameat sets WILLBEDIR). sceKernelRename accepts it today, so this comes first: without it the next commit would take that away.
  2. fix(libkernel): run sceKernelRename through the checks rename uses: Stdio.cpp. sceKernelRename becomes the null check it had plus one call to RenameGuestPath_nid_no_patch, the function rename and renameat already use, with its error returned as the sceKernel code.
  3. test(libkernel): rename a directory onto an empty directory with sceKernelRename.
  4. test(libkernel): check the errors sceKernelRename reports.
  5. test(libc): rename a directory onto a name written with a trailing slash.
  6. docs(debt): describe what rename and sceKernelRename share and still differ in on Windows.

After this, sceKernelRename and rename cannot drift apart again: one path, one set of errors. The file-onto-file replacement from #4532 is unchanged; sceKernelRename reaches it through the same RenameNative_nid_no_patch call as before.

Other results that change, all in sceKernelRename: a file another program holds without delete sharing gives 0x80020010 (EBUSY) instead of EIO on Windows, as rename does; and the last two rows of the table, where main on Windows replaced or created a file through a name with a trailing slash.

Not changed, and recorded in the debt file: on Windows the empty destination directory is removed and then the source is renamed, so the replacement is not atomic (this is what rename does today); and a regular file named with a trailing slash is still accepted as the source on Windows, where FreeBSD fails with ENOTDIR. Also unchanged and not in the debt file: a source path that runs through a regular file gives ENOENT on Windows where FreeBSD gives ENOTDIR, as in the other path functions.

Overlap: #5008 adds a check at the top of both sceKernelRename and RenameGuestPath_nid_no_patch. Commit 2 rewrites the body of sceKernelRename, so the two conflict in those lines of Stdio.cpp. They do not conflict in behaviour: with this PR the check #5008 puts in RenameGuestPath_nid_no_patch covers sceKernelRename too. Whichever lands second needs a small rebase; I will do mine if #5008 goes first.

guest_filesystem gains three groups of checks next to the existing sceKernelRename ones: a directory with a file in it renamed onto an empty directory (the file must arrive and the source be gone); the six error cases in the table, with both directories unchanged afterwards; and a directory renamed onto name/ through sceKernelRename and through rename, plus a file onto name/, which must fail with ENOENT and leave the file.

Tested

Windows (28b9f2d on main 0214069, Windows 11 10.0.26300, WinLibs GCC 15.2.0 r7, Release, full builds). The runs were made on 96ad4438b, which has the same code and tests; the two differ only in where the debt note of commit 6 is placed (it is in docs/dev/debt/libc.md, as the conventions check requires for a note that links a libc file):

  • With Stdio.cpp and Rename.cpp as on main and the tests kept, guest_filesystem fails (exit 0xc0000409, Filesystem check failed at line 517 (guest errno 5, Win32 error 87)). With the change it passes 5/5 (--repeat until-fail:5), and guest_kernel_rename, guest_posix_filesystem, guest_path_case, guest_kernel_fcntl, guest_truncate, guest_dup, file_streams, guest_standard_streams and guest_fdopendir pass 3/3.
  • A probe of 102 lines, with main's libraries, the branch's, and on the Linux build before and after: rename and sceKernelRename over every pair of six kinds of source and destination (missing, file, 0444 file, file held open, empty directory, directory with a file), then fifteen named cases (same name, into itself, onto its parent, missing parent, parent is a file, source under a file, trailing slashes on either side, . and .. sources, a change of case), each printing the result and what is left at both names. On Windows 23 lines change. Five change what happens: the two directory-onto-empty-directory renames now succeed, rename of a directory onto new/ now succeeds, and the two trailing-slash rows of the table now fail. The other 18 are sceKernelRename error codes. On Linux 23 lines change: one is rename of a directory onto new/, which now succeeds, and 22 are sceKernelRename error codes; nothing that succeeded on Linux fails. Afterwards the two hosts differ in four lines: a file named with a trailing slash as the source, and a source under a regular file, each for rename and for sceKernelRename. Both are listed under "Not changed".
  • The Windows matrix from fix(libc): replace an open or read-only destination in rename on Windows #4532, 202 lines, both functions over ten kinds of source and destination including files held by another process: no rename line changes. 56 sceKernelRename lines change: six directory-onto-directory renames that now succeed, and 50 error codes (0x80020014 18, 0x80020015 18, 0x80020010 11, 0x80020042 3). Its stress part, with both sets: 20,000 renames to a free name, 10,000 replacements of a closed file and 10,000 of an open one, half through each function, with no failure and the handle count unchanged at 140; eight threads replacing an open file 2000 times each with no failure and the expected ten directory entries left.
  • Probes of 122 lines over the descriptor functions, 866 over streams, 188 over the file, directory, vector and mapping functions, 431 over time, number, character and string functions, 145 over threads, 95 and 54 over flock, 44 over pipe streams, 15 over fcntl status flags, 127 over freopen, 130 over directory descriptors, 117 over streams used the other way, 144 over truncating opens, 164 over unlink, 156 over opens of names ending in a slash, 313 over fdopen and 198 over paths that run through a regular file, run with both sets of libraries: the lines that differ are five sceKernelRename lines in the third probe and eight in the last (the cases above), and two counters in the fdopen probe that change from run to run with either set.
  • Timing per call, three alternating runs of three passes (the probe build prints a call log, which is included): sceKernelRename to a free name 426 to 657 µs with main's libraries and 473 to 665 µs with the branch's; rename to a free name 475 to 685 and 480 to 696; creating a file and sceKernelRename onto an existing one 451 to 774 and 569 to 882. sceKernelRename now makes the status queries rename makes before the host call, which is where the last pair comes from.
  • Titles from the homebrew catalog, 35 s each with both sets of libraries, windows kept off-screen: ccleste (PPSA59322), RommPS (PPSA76677), SwanStationPS5 (PPSA98510), ProsperoTV (PPSA99003), DOOM (PPSA99666), ScreenTester5 (PPSA69420), Prospero Vibrate (PPSA99920), SymphonyStation5 (PPSA17641), ProsperoStore (PPSA99000), OpenNOW (PPSA99082), ProsperoRadio (PPSA99001) and PSXS5 (PPSA97510) are all still running at 35 s with both, with the same number of stderr lines for each title and the same last kind of log line.
  • Full ctest --parallel 4: 1095 of 1095 pass, as with main 0214069 built without this change.
  • Each commit alone: each of the six was checked out and built in order, the first two with full builds of the libraries, and guest_filesystem, guest_kernel_rename, guest_posix_filesystem, guest_path_case and file_streams pass 3/3 at every one.
  • tools/check_conventions.py --base origin/main passes. tools/pr_overlap.py --branch reports one overlap, fix(libc): fail a rename of a path that ends in a dot component with EINVAL #5008 in Stdio.cpp, described above.

Linux (96ad4438b, WSL2 Ubuntu 24.04, GCC 13.3.0, Release): libs builds, and guest_filesystem, guest_kernel_rename and guest_posix_filesystem pass 3/3. The probe results for Linux are in the second item above.

Not checked on a console or with a commercial title.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: none
  • AI-assisted: yes

@oneandonlydean

Copy link
Copy Markdown
Collaborator

Closing: no title is known to reach this. With the queue at 480+ open PRs, please hold off on new PRs that don't name a title needing the change until it's down. If a title stops on this on current main, reopen with the title ID and the log line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants