brintos

brintos / hardwarejs public Read only

0
0

fcntl(F_DUPFD) returns ENOSYS (-38) for non-console kernel fds (dup/dup2 work) #1

Closed vlad opened this issue · 1 comment
vlad vlad commented

fcntl(fd, F_DUPFD, n) (and F_DUPFD_CLOEXEC) returns ENOSYS (-38) for a file descriptor that refers to a non-console kernel device — e.g. a /dev/pts/* slave or /dev/zero — while dup(fd) and dup2(fd, n) on the same descriptor work. F_DUPFD works only when fd is the console.

In stock Linux F_DUPFD is handled generically in do_fcntl() (f_dupfd()alloc_fd()), independent of the backing file, so any real fd should support it.

Reproduction

A small wasm32-musl probe (built with the platform toolchain), run from the interactive init shell:

fcntl(2, F_DUPFD, 7)              = 7     errno=0     (fd 2 = console)
fcntl(0, F_DUPFD, 7)              = 7     errno=0     (fd 0 = console)
fcntl(0, F_DUPFD, 1)              = 4     errno=0     (fd 0 = console)
open("/dev/zero")                 = 0
fcntl(0, F_DUPFD, 7)  [zero fd]   = -1    errno=38 (ENOSYS)   <-- BUG
fcntl(0, F_DUPFD, 1)  [zero fd]   = -1    errno=38 (ENOSYS)   <-- BUG
dup2(0, 1)            [zero fd]   = 1     errno=0
dup(0)               [zero fd]    = 1     errno=0

The same holds for a /dev/pts/* slave opened onto a low fd: fcntl(F_DUPFD) → ENOSYS, dup/dup2 → OK.

Root cause

The worker's fcntl routing (worker.mjs, the "D2-FCNTL" block) routes fcntl kernel-side only for fd >= 3 or a low fd already recorded in kernelRoutedLowFds (populated when a kernel fd is dup2'd onto a low fd). A descriptor obtained by a fresh open() that lands on a low fd (0/1/2) — e.g. after a process closes 0/1/2 and opens a pts slave, which lands on fd 0 — is a real kernel fd but is not in kernelRoutedLowFds, so its fcntl falls to the SAB-RPC fallback, which has no fcntl handler and returns -ENOSYS. dup/dup2 are separate syscalls that are routed correctly, which is why they work on the identical fd.

Impact

This breaks Expect (spawn/expect) on musl. Expect's child (exp_getptyslave) closes fds 0/1/2, open()s the pts slave (which lands on fd 0), then wires it onto stdout/stderr with fcntl(0, F_DUPFD, 1) / fcntl(0, F_DUPFD, 2). Both return ENOSYS, so the exec'd child's fd 1 is never wired to the pty and it dies immediately (cat: standard output: Bad file descriptor). More generally, any program that fcntl(fd, F_DUPFD, n)s a freshly-opened non-console fd is affected.

Suggested fix

Route the generic fcntl commands (at least F_DUPFD / F_DUPFD_CLOEXEC, and arguably F_GETFL/F_SETFL/F_GETFD/F_SETFD) kernel-side whenever perProcessTaskAddr !== 0, regardless of the fd number — the kernel sys_fcntl ([72], HWJS_NATIVE) handles all fds including the console tty. That removes the dependency on kernelRoutedLowFds tracking a fresh open() and mirrors how dup/dup2 already behave. (I did not attach a worker.mjs branch — I don't have a pushable host checkout, and the routing heuristic exists for reasons I did not want to change untested.)

Workaround (client-side)

For Expect specifically, rewriting the two child-side pty-slave dups from fcntl(0, F_DUPFD, 1/2) to dup2(0, 1) / dup2(0, 2) (semantically identical, since the target fds were just closed) fully unblocks spawn/expect; dup2 routes kernel-side. This is a viable per-application workaround but does not fix the underlying F_DUPFD gap.

jdbrinton jdbrinton commented

(agent, reporting on joel's behalf)

Fixed on joel-dev — verified headlessly. Your root-cause analysis was exactly right, and your suggested fix is what landed.

Root cause (confirmed)

The worker's D2-FCNTL routing gate (fd >= 3 || kernelRoutedLowFds.has(fd)) was the gap — host layer, not the kernel. Kernel sys_fcntl ([72], generic do_fcntlf_dupfd) always handled every fd correctly, including the console tty. A real kernel fd sitting on an untracked low slot (your close(0); open("/dev/zero") → fd 0 shape) fell through to the SAB-RPC fallback, which has no fcntl handler → -ENOSYS (-38). dup/dup2 route kernel-side unconditionally, hence the asymmetry; the console "worked" only because the boot fdActions had registered fds 0/1/2 in kernelRoutedLowFds. The gate also mis-answered fcntl(bad fd) with ENOSYS instead of EBADF.

While reproducing we found a second shape even the recent low-fd open re-add (5d3f9fb, #2) can't cover: a pipe() end landing on a freed low fd (pipe fds are written into user memory, so no post-syscall bookkeeping ever sees them). Same ENOSYS. Both shapes are fixed by removing the fd-number dependence entirely.

Fix

fcntl now routes kernel-side unconditionally whenever the Worker has a real task (perProcessTaskAddr !== 0) — exactly mirroring dup/dup2, as you suggested. All commands, all fds; the JS-direct low-fd split remains only for read/write/ioctl data routing, which fcntl doesn't touch. 5d3f9fb's F_DUPFD low-fd re-add bookkeeping is unchanged and stays coherent.

  • hardwarejs cddfce5worker: route fcntl kernel-side unconditionally, like dup/dup2
  • superproject b333fd1 — permanent gate + gitlink advance

Gate evidence (headless, musl-bash-coreutils image)

New permanent gate distros/linux-6.12-musl-bash-coreutils/tools/run-fcntl-dupfd-gate.mjs (probe: console pts, /dev/zero→fd 0, /dev/ptmx→fd 0, pipe()→fd 0, F_DUPFD_CLOEXEC flag check, EBADF arms).

GREEN (post-fix):

FCNTL-console-dupfd: ret=7 errno=0 OK
FCNTL-zerolow-dupfd: ret=7 errno=0 OK
FCNTL-zerolow-dupfd-cloexec: ret=7 errno=0 OK
FCNTL-ptmxlow-dupfd: ret=7 errno=0 OK
FCNTL-ptmxlow-dupfd-cloexec: ret=7 errno=0 OK
FCNTL-pipelow-dupfd: ret=7 errno=0 OK
FCNTL-pipelow-dupfd-cloexec: ret=7 errno=0 OK
FCNTL-ebadf-dupfd: ret=-1 errno=9 OK
FCNTL-ebadf-getfd: ret=-1 errno=9 OK
FCNTLPROBE: PASS (0 failing legs)
[fcntl-gate] RESULT: GREEN (F_DUPFD/F_DUPFD_CLOEXEC work on console, low-fd /dev/zero, low-fd ptmx, low-fd pipe; bad fd says EBADF)

RED arm (FCNTL_GATE_RED=1 restores the pre-fix fd-number gate via HWJS_RED_FCNTL_FDGATE; genuinely flipping):

FCNTL-console-dupfd: ret=7 errno=0 OK
FCNTL-zerolow-dupfd: ret=-1 errno=38 BUG
FCNTL-zerolow-dupfd-cloexec: ret=-1 errno=38 BUG
FCNTL-ptmxlow-dupfd: ret=-1 errno=38 BUG
FCNTL-ptmxlow-dupfd-cloexec: ret=-1 errno=38 BUG
FCNTL-pipelow-dupfd: ret=-1 errno=38 BUG
FCNTL-pipelow-dupfd-cloexec: ret=-1 errno=38 BUG
FCNTL-ebadf-dupfd: ret=-1 errno=9 OK
FCNTL-ebadf-getfd: ret=-1 errno=38 BUG
[fcntl-gate] RESULT: RED CONFIRMED — the fd-number gate reproduces vlad's asymmetry (console OK, non-console kernel fds ENOSYS; legs genuinely flip)

Regressions all green: run-hw2-expect-shape-gate.mjs (the expect-shape pty spawn adjacency), run-a2-jobcontrol.mjs, run-poll-timeout-gate.mjs, hardwarejs vitest (no new failures vs. pre-fix baseline — same 14 pre-existing environment-dependent failures with and without the change).

Your dup2 workaround in Expect can be dropped whenever convenient — fcntl(0, F_DUPFD, 1/2) now works as filed. Ships with the next nightly.

jdbrinton closed this as completed