diff --git a/docs/dev/debt/libkernel.md b/docs/dev/debt/libkernel.md index ead756f295c..6885a377f0f 100644 --- a/docs/dev/debt/libkernel.md +++ b/docs/dev/debt/libkernel.md @@ -86,7 +86,7 @@ - [tcgetattr, tcsetattr](../../../core/libs/prx/libkernel/File/src/Terminal.cpp) (libkernel) - every file and pipe descriptor is taken as not a terminal (`ENOTTY`), and a socket gets `EADDRNOTAVAIL` for `AF_INET` and `EOPNOTSUPP` otherwise, as FreeBSD's `in_control` and `pru_control_notsupp` answer a termios ioctl; what descriptors 0 to 2 of a title are on the console, and whether any of them is a terminal there, is unknown - [cfsetspeed](../../../core/libs/prx/libkernel/File/src/Terminal.cpp) (libkernel, NID `C1MQ6OaXRjo`) - console behavior and module/library versions are unverified. The prototype, uint32 speed and 44-byte termios layout (input speed at 36, output speed at 40) follow the public PS5 payload SDK [termios.h](https://github.com/ps5-payload-dev/sdk/blob/a7685ce8e97ce1f8a4f38253a41d54241c9124f1/include/freebsd/termios.h) and [sys/_termios.h](https://github.com/ps5-payload-dev/sdk/blob/a7685ce8e97ce1f8a4f38253a41d54241c9124f1/include/freebsd/sys/_termios.h). The module association follows its libkernel stub table and the NID rehashes to the aerolib name. Both speed fields accept every uint32 value and no flags or control characters change, as [FreeBSD 11.4 termios.c](https://github.com/freebsd/freebsd-src/blob/d106002967eff712894ab8b5337bbbff106ce5e0/lib/libc/gen/termios.c) does. No terminal I/O is performed. Null pointers throw where the reference dereferences them. - [select](../../../core/libs/prx/libkernel/Socket/src/Socket.cpp) (libkernel) - descriptor 0 is never reported readable and descriptors 1 and 2 are always writable, as for a console title with no input; reads from descriptor 0 still go to the host stdin. `/dev/random` and `/dev/urandom` follow `randomdev_poll` for a seeded device under [`selscan`](https://github.com/freebsd/freebsd-src/blob/release/11.4.0/sys/kern/sys_generic.c#L1178-L1236), which asks once with the flags of every set the descriptor is in: in the read set they are readable and reported in no other set, and otherwise they are reported in the write and exception sets they were asked in. A regular file, a directory or a file read from app0.zar is always readable and writable and never in the exception set, as `vop_stdpoll` answers, on every host. Guest sockets and libSceNet sockets are polled. A kqueue is readable while one of its events is triggered and is never writable or in the exception set, as FreeBSD 11.4's [`kqueue_poll`](https://github.com/freebsd/freebsd-src/blob/release/11.4.0/sys/kern/kern_event.c#L1900-L1924) answers, on every host. Other descriptors are polled through the host on Linux, while on Windows pipes are checked every millisecond with the readiness test `poll` uses and any other descriptor throws. Dungeons (PPSA16064) selects on descriptor 0, and RetroArch (PPSA99169) selects on its web server's wake-up pipe next to its listening socket -- [select](../../../core/libs/prx/libkernel/Socket/src/Socket.cpp) (libkernel) - a descriptor the socket poller reports as unknown that is below `GuestSockets::FirstDescriptor` is polled on the host instead of throwing (on Windows only a pipe, as above); an invalid host descriptor fails with `EBADF`, and mixed host and socket sets are polled in short alternating slices so neither side waits out the full slice alone. A descriptor that stops being a socket, a kqueue or a pipe during the wait fails the call with `EBADF` when it is closed and is selected again with the rest of the timeout when its number was reused, as FreeBSD 11.4's [`kern_select`](https://github.com/freebsd/freebsd-src/blob/release/11.4.0/sys/kern/sys_generic.c#L1129-L1140) looks every descriptor up again in each pass. A close that has already removed the descriptor from the guest's kqueue, pipe or file tables but not yet closed its host descriptor is seen by that new look-up as a descriptor of unknown kind, which throws on Windows and is polled through the host on Linux. Wobbly Life (PPSA29413) selects on file descriptors while streaming its bundles +- [select](../../../core/libs/prx/libkernel/Socket/src/Socket.cpp) (libkernel) - a descriptor the socket poller reports as unknown is polled on the host instead of throwing (on Windows only a pipe, as above); an invalid host descriptor fails with `EBADF`, and mixed host and socket sets are polled in short alternating slices so neither side waits out the full slice alone. A descriptor that stops being a socket, a kqueue or a pipe during the wait fails the call with `EBADF` when it is closed and is selected again with the rest of the timeout when its number was reused, as FreeBSD 11.4's [`kern_select`](https://github.com/freebsd/freebsd-src/blob/release/11.4.0/sys/kern/sys_generic.c#L1129-L1140) looks every descriptor up again in each pass. A close that has already removed the descriptor from the guest's kqueue, pipe or file tables but not yet closed its host descriptor is seen by that new look-up as a descriptor of unknown kind, which throws on Windows and is polled through the host on Linux. Wobbly Life (PPSA29413) selects on file descriptors while streaming its bundles - [__progname](../../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - unknown data export - [environ](../../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - data export pointing to a fixed empty environment so titles that import it load; it is not the environment libc's `getenv` reads, so it lists none of those variables and `setenv`/`putenv` changes do not appear in it - [seteuid, setegid, setuid, setgid, setreuid, setregid, setresuid, setresgid, setgroups, getgroups](../../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - the guest process is assumed to run as root, with real, effective and saved user and group ids 0. As in FreeBSD's `sys_seteuid`, `sys_setegid`, `sys_setuid`, `sys_setgid`, `sys_setreuid`, `sys_setregid`, `sys_setresuid` and `sys_setresgid`, id 0 (or -1 for an unchanged `setreuid`/`setregid`/`setresuid`/`setresgid` id) is then accepted without a privilege check and changes no credential, so 0 is returned; any other id would change an id, which is not modelled, and throws. `setgroups` keeps the supplementary group list as `kern_setgroups` does (`NGROUPS_MAX` 1023, an empty list leaves the effective group as the only entry, a first entry other than 0 would change the effective group and throws) and `getgroups` reports it. The ids and privileges a title has on a console are unverified @@ -160,7 +160,7 @@ - [sceKernelAddReadEvent and sceKernelAddWriteEvent](../../../core/libs/prx/libkernel/Equeue/Equeue.cpp) (libkernel): a read event on a listening socket, here and in `kevent`, reports the number of completed connections waiting to be accepted in `data`, as FreeBSD 11.4's [`filt_solisten`](https://github.com/freebsd/freebsd-src/blob/release/11.4.0/sys/kern/uipc_socket.c#L3384-L3392) reports `so_qlen`: on Linux a TCP listener's accept queue from `TCP_INFO` (`tcpi_unacked`, which Linux fills with the accept backlog for a listening socket), and elsewhere, or for a listener that is not TCP, 1 while a connection is waiting, as no host call reports that count. A write event reports the free space of the socket's send buffer in `data`, as FreeBSD 11.4's [`filt_sowrite`](https://github.com/freebsd/freebsd-src/blob/release/11.4.0/sys/kern/uipc_socket.c#L3358-L3382) reports `sbspace`: on Linux the host's `SO_SNDBUF` less the bytes `SIOCOUTQ` counts as queued, which include the host's per-packet overhead, so the space shrinks by somewhat more than the bytes sent. Windows has no query for that space and macOS is not wired up, so both report the whole `SO_SNDBUF`, which does not shrink as data is queued. Only guest sockets are supported; other descriptors and a low watermark above 1 throw - [sceKernelAddReadEvent on pipes](../../../core/libs/prx/libkernel/Equeue/Equeue.cpp) (libkernel) is an exception to the sockets-only rule above: it also takes the read end of a pipe, as `kevent` does through FreeBSD's `filt_piperead`, and ignores the size argument there. Host streams, other descriptors and a write event on a pipe throw - [sceKernelAddReadEvent and sceKernelAddWriteEvent on regular files](../../../core/libs/prx/libkernel/Equeue/Equeue.cpp) (libkernel) are the other exception: a size argument above 1, directories and other vnode types throw, and whether the console's file systems answer as `vfs_kqfilter` does was not measured. A wait holding one of these events polls it every millisecond, as no host call reports every file change, so a write through another descriptor shows up to 1 ms late. Unlike FreeBSD, a seek can make a read event fire again: FreeBSD 11.4's [`kqueue_scan`](https://github.com/freebsd/freebsd-src/blob/release/11.4.0/sys/kern/kern_event.c) deactivates a level-triggered read knote once `filt_vfsread` reports `data` 0, and only a vnode `KNOTE` (a write, an extend or a truncate) activates it again; `lseek` posts none, so after a file read to its end is seeked back, the 1 ms poll here reports the new `data` (10 for a 10-byte file seeked to 0) where FreeBSD keeps waiting until the file changes. Otherwise they follow FreeBSD 11.4's [`filt_vfsread` and `filt_vfswrite`](https://github.com/freebsd/freebsd-src/blob/release/11.4.0/sys/kern/vfs_subr.c#L5122-L5168): a read event reports the size less the descriptor's offset in `data` and fires while that is not 0; a write event always fires with `data` 0 -- [close](../../../core/libs/prx/libkernel/File/src/Stdio.cpp) and [setsockopt](../../../core/libs/prx/libkernel/Socket/src/Udp.cpp) (libkernel) reach a [libSceNet](../../../core/libs/prx/libSceNet/Export.cpp) socket through hooks libSceNet registers, as `select` polls them; the other libkernel socket calls do not. libSceNet numbers its sockets from 32 in its own table, so a CRT file descriptor and a socket can share a number; `close` then closes the socket. On the console both are one kernel descriptor space +- [close](../../../core/libs/prx/libkernel/File/src/Stdio.cpp) and [setsockopt](../../../core/libs/prx/libkernel/Socket/src/Udp.cpp) (libkernel) reach a [libSceNet](../../../core/libs/prx/libSceNet/Export.cpp) socket through hooks libSceNet registers, as `select` and `poll` poll them and `fcntl` answers `F_GETFL` and `F_SETFL`, failing other commands with `EINVAL`; the other libkernel socket calls do not. libSceNet numbers its sockets from 32 in its own table, so a CRT file descriptor and a socket can share a number; `close` then closes the socket. On the console both are one kernel descriptor space - [sceKernelPread, sceKernelPwrite, sceKernelPreadv, sceKernelPwritev](../../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) on Windows transfer with an overlapped `ReadFile`/`WriteFile` and restore the descriptor's file pointer afterwards. Guest scalar transfers, positioned transfers, vectored transfers and seeks share a position lock across duplicated descriptors; a vector holds it for the whole operation. Independently opened descriptors and other processes do not share that lock, so vector transfers over their overlapping file ranges are not coordinated. Handles that are not disk files return `ESPIPE`. A pipe or console `readv` can return a short read when no more input is available; each iovec is transferred separately, so a failure after the first transfer returns the bytes already done - [sceKernelAioSubmitReadCommands, sceKernelAioSubmitWriteCommands and their Multiple variants](../../../core/libs/prx/libkernel/Aio/src/Aio.cpp) (libkernel) on Windows transfer through the same positioned path as `sceKernelPread`/`sceKernelPwrite` and restore the descriptor's file pointer afterwards; positioned transfers, plain transfers and seeks of one descriptor are serialized per descriptor (shared across duplicated descriptors), so a concurrent transfer observes a stable file pointer - [sceKernelAioSetParam](../../../core/libs/prx/libkernel/Aio/src/Aio.cpp) (libkernel) - the limits and `sceKernelAioInitializeParam`'s defaults (three 20-byte scheduling structs, the last without split) are measured on a PS5 (firmware 13.60), where a null struct crashes both calls; here both throw. The scheduling settings are not used: `sceKernelAioInitializeImpl` ignores the struct and requests run synchronously on submit @@ -218,7 +218,7 @@ - [send, sendto](../../../core/libs/prx/libkernel/Socket/src/Udp.cpp) (libkernel) - send and sendto without an address on an unconnected datagram socket map Windows `WSAENOTCONN` to FreeBSD's `EDESTADDRREQ` (39). - [bind, connect, sendto](../../../core/libs/prx/libkernel/Socket/src/Udp.cpp) (libkernel) check a socket address length as FreeBSD 11.4's [`getsockaddr`](https://github.com/freebsd/freebsd-src/blob/release/11.4.0/sys/kern/uipc_syscalls.c) and the `sa_len` test of `tcp_usr_connect` and `in_pcbconnect` do: below 2 or an `AF_INET` address below 16 or an `AF_INET6` address below 28 bytes fails with `EINVAL`, above 255 with `ENAMETOOLONG`. `sendto` does not check an address longer than `sockaddr_in` on an `AF_INET` datagram socket, where `udp_output` answers `EINVAL` unless the socket has a local address and port and the destination is neither `INADDR_ANY` nor `INADDR_BROADCAST`, nor on an unconnected stream socket, which FreeBSD connects implicitly through `tcp_connect` and its length test. An address of another family still fails with `EAFNOSUPPORT`, as the console's behaviour for it is not known. None of this was measured on a console. - [read, write and vector I/O](../../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) on a guest socket use `recv` and `send` without flags; `sceKernelReadv` and `sceKernelWritev` gather or scatter through one socket operation, preserving stream short reads and datagram boundaries. The positioned `pread`, `pwrite`, `preadv` and `pwritev` forms, and their `sceKernel` forms, return `ESPIPE` for an open guest socket and `EBADF` for a closed one. `EPIPE` from a write raises the host's `SIGPIPE` on Linux unless the socket has `SO_NOSIGPIPE` set, and the guest cannot ignore it: `signal(SIGPIPE, SIG_IGN)` and `sigaction` record the disposition, as they do for every number from 1 to 128, but only the six signals with host equivalents (2, 4, 6, 8, 11 and 15) take effect on the host or are delivered, so the host `SIGPIPE` is not stopped; nothing is raised on Windows; a `write` longer than `INT_MAX` bytes is `EMSGSIZE` where the console sends it. AIO on a guest socket still goes to the host, which has no such descriptor -- [Socket descriptor numbers](../../../core/libs/prx/libkernel/Socket/src/Udp.cpp) (libkernel, libSceNet) - `socket`, `accept`, `socketpair`, `dup`, `sceNetSocket` and `sceNetAccept` take the lowest free number of the host descriptor table that files use, as FreeBSD 11.4 `fdalloc` takes the lowest free entry of the one process table (`kern_descrip.c`), so every number stays below `FD_SETSIZE` (1024) while fewer than 1024 descriptors are open. The number holds a zero-access `NUL` handle on Windows and a duplicate of the host socket on Linux, and the socket tables map it to the socket. The host table also holds the runtime's own files, so the numbers differ from the console's; the first number `socket` and `sceNetSocket` return on a console and its `getdtablesize` are not measured. Paths that bypass the socket tables see the placeholder: libc `fdopen` on a socket opens it, and libkernel `close` of a libSceNet socket releases the number but leaves the libSceNet socket open. Palworld (PPSA23126), The Mound (PPSA26223) and Dying Light 2 (PPSA02262) set socket bits in a 1024-bit `fd_set` +- [Socket descriptor numbers](../../../core/libs/prx/libkernel/Socket/src/Udp.cpp) (libkernel, libSceNet) - `socket`, `accept`, `socketpair`, `dup`, `sceNetSocket` and `sceNetAccept` take the lowest free number of the host descriptor table that files use, as FreeBSD 11.4 `fdalloc` takes the lowest free entry of the one process table (`kern_descrip.c`), so every number stays below `FD_SETSIZE` (1024) while fewer than 1024 descriptors are open. The number holds a zero-access `NUL` handle on Windows and a duplicate of the host socket on Linux, and the socket tables map it to the socket. The host table also holds the runtime's own files, so the numbers differ from the console's; the first number `socket` and `sceNetSocket` return on a console and its `getdtablesize` are not measured. Paths that bypass the socket tables see the placeholder: libc `fdopen` on a socket opens it, and libkernel `sceKernelClose` of a libSceNet socket releases the number but leaves the libSceNet socket open. Palworld (PPSA23126), The Mound (PPSA26223) and Dying Light 2 (PPSA02262) set socket bits in a 1024-bit `fd_set` - [fstat on a guest socket](../../../core/libs/prx/libkernel/Socket/src/Udp.cpp) (libkernel) fills the fields FreeBSD 11's soo_stat and pru_sense_null fill and zeroes the rest: S_IFSOCK, the read bits while the receive side is open or data is queued, the write bits while the send side is open, st_size and st_blksize. Only this process's shutdown closes a side here; a peer's shutdown, close or reset does not clear the bits as FreeBSD's socantrcvmore and socantsendmore do. st_size is the host's FIONREAD, which for a Linux datagram socket is the size of the next datagram where FreeBSD counts every queued byte, and st_blksize is the host's SO_SNDBUF, which Linux reports as twice the size that was set. A socketpair end reports st_dev and st_ino 0, where FreeBSD's uipc_sense reports NODEV and a per-socket number - [poll](../../../core/libs/prx/libkernel/Socket/src/Udp.cpp) (libkernel) waits on guest sockets and pipes, through `WSAPoll` on Windows and `poll` on Linux; the `pollfd` layout and flag values are FreeBSD's. A libSceNet socket joins the same host wait through the native socket libSceNet lends to libkernel. `WSAPoll` rejects `POLLPRI`, so on Windows a socket's `POLLPRI` is asked as `POLLRDBAND` and `POLLIN` as `POLLRDNORM`; Winsock reports urgent data until it is read with `MSG_OOB`, where FreeBSD 11.4's `sopoll_generic` stops once the normal stream passes the mark. Guest `send` and `recv` refuse `MSG_OOB` with `EOPNOTSUPP`, so urgent data can only come from an outside peer and Windows keeps reporting `POLLPRI` after the stream passes the mark. A pipe follows FreeBSD's `pipe_poll`: the read end is readable when data is buffered, either end reports `POLLHUP` once the other end is closed, with the requested `POLLIN` and `POLLRDNORM` bits and without `POLLOUT`, which `pollscan` clears, and the write end is writable when `PIPE_BUF` (512) bytes are free on Windows, or when the host kernel says so on Linux. `POLLOUT` on a read end throws: FreeBSD opens both ends for reading and writing and reports it writable, and host pipes are one-way. Windows pipes cannot be waited on, so while a pipe is polled `poll` checks it with `NtQueryInformationFile` and waits on the sockets in slices of 1 ms. A kqueue is ready for `POLLIN` and `POLLRDNORM` while one of its events is triggered and for no other event, as FreeBSD 11.4's [`kqueue_poll`](https://github.com/freebsd/freebsd-src/blob/release/11.4.0/sys/kern/kern_event.c#L1900-L1924) answers. A kqueue cannot be waited on either, so on every host `poll` checks it every millisecond and waits on the other descriptors in slices of 1 ms. A kqueue closed during the wait reports `POLLNVAL`, and one whose number was reused is polled again with the rest of the timeout, as FreeBSD 11.4's [`pollrescan`](https://github.com/freebsd/freebsd-src/blob/release/11.4.0/sys/kern/sys_generic.c#L1499-L1519) looks the descriptor up again. A close that has already removed the kqueue but not yet closed its host descriptor is seen as a descriptor of unknown kind, which throws. A regular file, a directory or a file read from app0.zar is always ready for `POLLIN`, `POLLRDNORM` and `POLLOUT` and never for `POLLPRI`, `POLLRDBAND` or `POLLWRBAND`, as FreeBSD 11.4's [`vop_stdpoll`](https://github.com/freebsd/freebsd-src/blob/release/11.4.0/sys/kern/vfs_default.c#L563-L574) and `vop_nopoll` answer every event `poll` accepts; which of the two the console's filesystems use is not known, and they agree on these events. `/dev/random` and `/dev/urandom` follow FreeBSD 11.4's [`randomdev_poll`](https://github.com/freebsd/freebsd-src/blob/release/11.4.0/sys/dev/random/randomdev.c#L274-L285) for a seeded device: a request with `POLLIN` or `POLLRDNORM` gets only those bits back, and any other request gets every requested bit. Open descriptors 0 to 2 follow the console model `select` uses: descriptor 0 never reports an event and descriptors 1 and 2 report `POLLOUT`, whatever the host attaches to them. A closed descriptor reports `POLLNVAL`, and an open descriptor that is neither a guest socket, a libSceNet socket, a pipe, a file, a directory, the random device nor descriptor 0 to 2 (other devices) throws, as do event bits other than `POLLIN`, `POLLPRI`, `POLLOUT`, `POLLRDNORM`, `POLLRDBAND` and `POLLWRBAND` (the `POLLERR`, `POLLHUP` and `POLLNVAL` bits are ignored in `events`), an infinite timeout with no socket, pipe or kqueue to wait on. A closed socket descriptor reports `POLLNVAL`; the console's limit on the descriptor count is unknown and not checked - [poll](../../../core/libs/prx/libkernel/Socket/src/Udp.cpp) (libkernel) on Windows reports a guest stream socket whose peer closed as readable and adds `POLLHUP` only once the socket has shut its own sending side, as FreeBSD 11.4's [`sopoll_generic`](https://github.com/freebsd/freebsd-src/blob/release/11.4.0/sys/kern/uipc_socket.c#L3080-L3086) does; `WSAPoll` reports `POLLHUP` alone in both cases. A reset or refused connection (`POLLHUP` with `POLLERR`) keeps `WSAPoll`'s flags, a libSceNet socket keeps them too, and a call that asks for no event still returns early on the peer's close with nothing reported. Not measured on a console.