Skip to content

Add options --rawfd and --slurpfd#3546

Open
howtonotwin wants to merge 5 commits into
jqlang:masterfrom
howtonotwin:arg-from-fd
Open

Add options --rawfd and --slurpfd#3546
howtonotwin wants to merge 5 commits into
jqlang:masterfrom
howtonotwin:arg-from-fd

Conversation

@howtonotwin

Copy link
Copy Markdown

These allow jq to read data from files that might not have names, make it easier for shell scripts that use jq to avoid races, etc., without resorting to tricks like /dev/fd or /proc/self/fd that might not always be available (and don't quite behave the same). These options are otherwise very similar to --rawfile and --slurpfile and share most of their code.

I'm using this on (GNU/)Linux. I'd appreciate feedback about/testing on other platforms. I didn't find a contributors' guidelines file or a style guide, so please tell me if I should do something differently!

These allow jq to read data from files that might not have names, make
it easier for shell scripts that use jq to avoid races, etc., without
resorting to tricks like /dev/fd or /proc/self/fd that might not always
be available (and don't quite behave the same). These options are
otherwise very similar to --rawfile and --slurpfile and share most of
their code.
@howtonotwin

howtonotwin commented May 18, 2026

Copy link
Copy Markdown
Author

These options do work on Windows, but Windows programs that use FDs are subject to the restriction that programs with incompatible C runtimes can't inherit FDs from each other. The sh that runs the test script in CI is using the Cygwin/MSYS2 runtime but is calling a jq that uses the MSVCRT/UCRT runtime. So the FDs set up by the new test in sh don't quite make it to jq.

If I use a native Windows (MSVCRT/UCRT) binary like busybox-w32 to call jq, it's more apparent that this PR works:

busybox> ./jq -n '{$foo, $bar}' 8<tests/modules/data.json 9<>tests/modules/data.json --slurpfd foo 8 --rawfd bar 9
{
  "foo": [
    {
      "this": "is a test",
      "that": "is too"
    }
  ],
  "bar": "{\n  \"this\": \"is a test\",\n  \"that\": \"is too\"\n}\n"
}

So my questions are

  • Should these options just take handles on Windows from the start?
  • Should the test just be skipped on Windows, or should the test be moved to a compiled C program (which will work for the same reason busybox-w32 works)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant