Repository navigation
fix(libc): append stream writes atomically on Windows - #6287
Conversation
|
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
|
Cross-validation on Windows 11 Pro 26220, NVIDIA RTX 4070 (596.21), MinGW GCC 15.2, PR head
No concerns. Concurrent Written with Claude Code assistance. |
fopen and freopen in an append mode gave the C runtime a descriptor with _O_APPEND, which seeks to the end of the file and then writes. Streams on separate descriptors overwrote each other's data. FreeBSD opens these streams with O_APPEND, so every flush lands at the end of the file. Open append streams with FILE_APPEND_DATA but without FILE_WRITE_DATA, so Windows writes each flush at the end of the file. ftruncate and flock reopen the file to get the access such a handle lacks.
a431a12 to
0b24f92
Compare
|
Rebased onto main 3e3e8b9. The only conflict was in Windows 11 Pro, WinLibs GCC 15.2.0 r7, Release: full build passes. Full Thanks @screamm for the cross-validation, and for the extra variable-length and unflushed runs. |
What
On Windows,
fopenandfreopenin an append mode (a,ab,a+, ...) give the C runtime a descriptor with_O_APPEND. The runtime appends by seeking to the end and then writing, two separate steps. When two streams on separate descriptors flush at once, both seek to the same end and the later write overwrites the earlier one. FreeBSD'sfopenandfreopenopen these modes withO_APPEND, andvn_writewrites eachwrite(2)at the end of the file under the vnode lock, so flushes never overwrite each other. This is the stream side of #5831, which covers descriptors opened withsceKernelOpen. @screamm pointed it out there.The change:
fopenopens the append modes withFILE_GENERIC_WRITEminusFILE_WRITE_DATA, plusGENERIC_READfora+. Windows documents that withFILE_APPEND_DATAand withoutFILE_WRITE_DATA, "write operations will not overwrite existing data": every write on such a handle goes to the end of the file, wherever the file pointer is, and the pointer ends up after the data.freopenin an append mode moves an append-only duplicate of the handle onto the stream's descriptor, the same wayfdopenalready moves an_O_APPENDduplicate (now one sharedMoveAppendingHandle). For descriptors 0 to 2 it keeps the host standard handle, asFileStream::Reopendoes.fdopenis unchanged. FreeBSD'sfdopendoes not setO_APPENDeither; it only seeks before each write (__SAPP), which is what the existing code does.FILE_WRITE_DATAcannot truncate, and without read access it cannot take a byte-range lock. FreeBSD allows both on these descriptors.ftruncateon an append-only handle now truncates throughReOpenFile(GENERIC_WRITE)(asfsyncalready reopens), andflocklocks through a reopened handle kept in the lock state, which closes when the last descriptor sharing that state is forgotten. Without this,flockon anastream would throw, becauseERROR_ACCESS_DENIEDis not a lock conflict. All other descriptors take the existing paths. File-backedmmapis not supported on Windows, so it is not affected.guest_append_streams: four threads append 1,500 fixed-length lines each to one file throughfopen("a"),fopen("a+"),freopen("a")andfreopen("ab"), flushing after each line. The file must hold exactly 6,000 intact lines, each once. Then:a+stream reads from the start, and a write afterfseek(0, SEEK_SET)lands at the end withftellafter it;ftruncateon anastream's descriptor truncates, and the next write appends after the new end;flockon anastream's descriptor excludes a second open of the file, andfclosereleases it.With main's libc and libkernel it fails at the size check in 10 of 10 runs; with the change it passes 10 of 10.
Tested
Windows 11 Pro (26200), i5-13400, WinLibs GCC 15.2.0 r7, Release,
BUILD_TESTING=ON, on main 721db55:ctest -j8 --timeout 120: 1,255 of 1,255 pass (15 skipped by their own environment checks).tools/check_conventions.py --base origin/mainandgit diff --check: pass.HOME. Its stdout and stderr, and AnyPS5's own trace lines, all go to/download0/ps5-opengl.logthrough separate append descriptors. I counted the places where one writer's output landed inside another's line:[debug:0] [kodi] 2026-10-15 [src/Open.1pp:215 sceKernelOpen] ...followed by0 T:100001 ...[debug:0] [kodi] [0s5-cache] clflushopt=1 ...With the change, a line can still be split around another writer's line, for example
[debug:0] [kodi] 2026-10-1+[ps5-cache] clflushopt=1 cacheline=64+5 11:57:03.480 T:100002 .... That happens because the title writes one log line in more than onewrite, and FreeBSD splits it the same way. No bytes are lost.GENERIC_WRITEhandle and_O_APPEND, 3,124 and 3,179 of 8,000 lines arrive; with the append-only handle, 8,000 of 8,000. The same program showsfsyncworks on such a handle,ftellis at the end after each write, and_chsize_sandLockFileExfail on it, which is whyftruncateandflockreopen.Linux not tested. Nothing changes in the Linux build; the new test uses only guest functions, so it runs there too.
Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)