Skip to content

feat(libkernel): watch a kqueue with EVFILT_READ - #8523

Merged
adriadam10 merged 3 commits into
boykopovar:mainfrom
MarrasFederico:feat/kqueue-read-on-kqueue
Oct 21, 2026
Merged

adriadam10 merged 3 commits into
boykopovar:mainfrom
MarrasFederico:feat/kqueue-read-on-kqueue

Conversation

@MarrasFederico

@MarrasFederico MarrasFederico commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

What

kevent threw not implemented for EVFILT_READ on a kqueue descriptor. FreeBSD 11 lets a kqueue watch another one (kqueue_kqfilter and filt_kqueue): the watched queue is readable while it has pending events, and data is their count. This is now supported:

  • Readiness: the note is ready while the watched queue has at least one enabled, triggered note, and data is that count. It is evaluated on every poll or wait, after the watched queue's timers and descriptors are refreshed.
  • Waking: a blocking kevent on the outer queue wakes when the inner one gets an event. The waiter registers with every watched queue, recursively, and rechecks after registering so no wake is lost. It also takes on their nearest timer and their descriptor interests, so an inner timer or socket wakes it too.
  • Unchanged generic handling: EV_DISABLE/EV_ENABLE, EV_DISPATCH, EV_ONESHOT, EV_DELETE, and closing the watched queue, which drops the note.
  • Errors: other filters on a kqueue return EINVAL, as kqueue_kqfilter does. A note on the queue itself stays EINVAL (L1300-L1303).
  • What throws: EV_CLEAR on such a note, since FreeBSD re-arms it only when the watched queue queues a note, which is not tracked here. Also a note that would close a cycle (A watches B, B watches A), which no source defines and which would otherwise deadlock.

The debt file drops this case from the kqueue entry's throw list and gets a kevent on a kqueue entry with the throwing cases and how the count differs from FreeBSD. Here data is the watched queue's enabled notes that are ready when the note is evaluated; FreeBSD's kq_count keeps counting a queued note until that kqueue is scanned, even once it is disabled or no longer ready, and reports a queued EV_ONESHOT note with its last count before deleting it, where here it is evaluated again and, if the watched queue is empty by then, neither reported nor deleted.

Tested

Ubuntu 26.04.1, kernel 7.0.0-38, GCC 15.2.0, Release, full build with BUILD_TESTING=ON. Windows was not compiled. No title was run.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 7492 core/libs/tests/GuestKqueue.cpp L1014
PR 7653 docs/dev/debt/libkernel.md L124
PR 8547 docs/dev/debt/libkernel.md L198-199
PR 9099 core/libs/prx/libkernel/Equeue/Equeue.cpp L100-102, L107-109, L113-114, L222-223
core/libs/prx/libkernel/Equeue/Equeue.hpp L58

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 4630 docs/dev/debt/libkernel.md
PR 5021 docs/dev/debt/libkernel.md
PR 6044 core/libs/tests/GuestKqueue.cpp
PR 6149 docs/dev/debt/libkernel.md
PR 6469 docs/dev/debt/libkernel.md
PR 6740 docs/dev/debt/libkernel.md
PR 6879 docs/dev/debt/libkernel.md
PR 6936 docs/dev/debt/libkernel.md
PR 6977 docs/dev/debt/libkernel.md
PR 7049 core/libs/prx/libkernel/Equeue/Equeue.cpp
core/libs/prx/libkernel/Equeue/src/Kqueue.cpp
core/libs/tests/GuestKqueue.cpp
docs/dev/debt/libkernel.md
PR 7088 core/libs/prx/libkernel/Equeue/Equeue.hpp
core/libs/prx/libkernel/Equeue/src/Kqueue.cpp
core/libs/tests/GuestKqueue.cpp
docs/dev/debt/libkernel.md
PR 7105 docs/dev/debt/libkernel.md
PR 7121 core/libs/prx/libkernel/Equeue/Equeue.hpp
core/libs/prx/libkernel/Equeue/src/Kqueue.cpp
core/libs/tests/GuestKqueue.cpp
docs/dev/debt/libkernel.md
PR 7149 docs/dev/debt/libkernel.md
PR 7192 docs/dev/debt/libkernel.md
PR 7447 docs/dev/debt/libkernel.md
PR 7509 core/libs/prx/libkernel/Equeue/Equeue.cpp
core/libs/prx/libkernel/Equeue/Equeue.hpp
docs/dev/debt/libkernel.md
PR 8520 docs/dev/debt/libkernel.md
PR 8726 docs/dev/debt/libkernel.md
PR 9137 docs/dev/debt/libkernel.md

@nikowolf123

Copy link
Copy Markdown
Contributor

The kevent on a kqueue entry and the PR body list two differences from FreeBSD. There is a broader one, and the new test pins it.

PollKqueue polls the watched queue again on every evaluation, so a note there stops counting as soon as it is no longer ready. FreeBSD's filt_kqueue reports kq_count, which is the number of queued notes. knote only ever activates a note. A queued note leaves the queue only when that kqueue is scanned: kqueue_scan drops it once its filter returns 0 and puts a reported level-triggered note back at the tail. Until that scan, a note that is no longer ready still counts.

The socket case in WatchKqueue shows this:

  • kevent(outer) at L444 reports inner with data 1 and puts outer's note back in the queue. Then recv empties pair[0].
  • inner was not scanned, so its socket note is still queued. On FreeBSD Poll(outer) then returns 1 with data 1, but L449 expects 0.
  • L465 is the same case one level up. Poll(outer) at L463 puts its note back, so after Poll(inner) consumes note 5, the kq_count of outer is still 1 and Poll(top) returns 1.

This also affects EV_ONESHOT in a way the entry does not cover. kqueue_scan reports a queued EV_ONESHOT note from kn_kevent without calling the filter, then deletes it (L1761-L1775). Take a note added while note 5 of inner is triggered, then call Poll(inner) and then Poll(outer). On FreeBSD the note is reported with data 1 and deleted. Here PollEvents evaluates it again and gets 0. So it is not reported and stays registered, and a later EV_DELETE returns 0 instead of ENOENT.

If the live count is meant to stay, I think widening the entry is enough, for example:

EVFILT_READ on a kqueue counts the watched queue's enabled notes that are ready when it is evaluated, where filt_kqueue reports kq_count, the queued notes. A queued note keeps counting until the watched kqueue is scanned, even after it was disabled or stopped being ready (a socket or pipe read empty, a user note cleared, a watched kqueue drained), so FreeBSD can report the kqueue readable where this reports nothing. A queued EV_ONESHOT note is reported there with the count of its last evaluation and deleted; here it is evaluated again, and if the watched queue is empty by then it is neither reported nor deleted.

The other option is a per-note queued flag so the count follows kq_count. L449 and L465 would then expect 1.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 20, 2026
@oneandonlydean

Copy link
Copy Markdown
Collaborator

@MarrasFederico Thanks, this is a careful piece of work. Two things before the full review:

  • main has moved under docs/dev/debt/libkernel.md, so the PR now conflicts there (the code itself still merges cleanly). Please rebase.
  • nikowolf123's point about kq_count decides what the code should be, so it's worth settling first: either keep the live count and widen the debt entry (their suggested text covers the socket and EV_ONESHOT cases), or track a per-note queued flag so data follows FreeBSD's kq_count (then the checks at GuestKqueue.cpp L449 and L465 would expect 1). Either is fine; the debt entry just needs to match what the code does.

Once that's in, I'll review the waiter registration (lock order across nested queues) and check the Windows build, which hasn't been compiled yet.

@ycbuzi

ycbuzi commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Windows run of 5f6c305 merged onto main b77bd58 (05fafdd7d1), the build the description lists as not done. Windows 11 Home 10.0.26200, MinGW-w64 GCC 15.2.0 (ucrt-posix-seh r7), Release, BUILD_TESTING=ON.

The merge conflicts only in docs/dev/debt/libkernel.md, as GitHub shows; for this run I took this PR's side of that, and the code merged without conflicts. Built libkernel and guest_kqueue_tests with no errors or warnings; ctest --repeat until-fail:2 on guest_kqueue: passes on both repeats. Not run: the negative control against main's library, a title.

…-kqueue

# Conflicts:
#	docs/dev/debt/libkernel.md
@adriadam10 adriadam10 self-assigned this Oct 21, 2026
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 21, 2026
@MarrasFederico

Copy link
Copy Markdown
Contributor Author

@oneandonlydean @nikowolf123 thanks. Merged current main (the conflict was only in docs/dev/debt/libkernel.md) and kept the live count. The kevent on a kqueue entry now states the difference as nikowolf123 described:

  • data counts the watched queue's enabled notes that are ready when it is evaluated;
  • FreeBSD's kq_count keeps a queued note until that kqueue is scanned, even once it is disabled or no longer ready;
  • a queued EV_ONESHOT note is reported there with its last count and deleted, while here it is evaluated again and, if the watched queue is empty by then, neither reported nor deleted.

The code and the test are unchanged. guest_kqueue passes three runs in a row and the 80 equeue/kqueue/pipe/socket/poll tests pass. The PR text says the same.

@ycbuzi thanks for the Windows run.

@adriadam10
adriadam10 merged commit f90a3f7 into boykopovar:main Oct 21, 2026
11 of 12 checks passed
@adriadam10

Copy link
Copy Markdown
Collaborator

Checked the nested wait registration (lock order is always outer to inner, cycles are refused at registration), the kq_count difference now written in the debt entry, and that WatchKqueue pins the counts and the timer/socket/chain wakeups and fails on main; Windows build confirmed by @ycbuzi; merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants