.. SPDX-License-Identifier: GPL-2.0 ================================================================================ arch/wasm32 memory model ================================================================================ :Status: K0 (build glue). Stable for K0–K2. Revisited at K3 (mm_init). This document is the source of truth for how ``arch/wasm32`` maps Linux's kernel/user split onto WebAssembly. Two related questions are answered here: 1. *Where does kernel memory live, and where does user memory live, at the wasm-module level?* (Section 1.) 2. *What page-table shape do we claim to have, given that wasm has no hardware MMU?* (Section 2.) Both questions are answered deliberately, not as a side-effect of which ``asm-generic`` headers shut clang up fastest. Changing either model is a breaking change to the arch ABI. -------------------------------------------------------------------------------- 1. Two-memory model -------------------------------------------------------------------------------- The cross-module contract — the arch ABI proper — is expressed in **names**, not in memory indices. Indices are local to each module's import order and will differ between vmlinux and userspace binaries by design. There are exactly two named ``WebAssembly.Memory`` objects in the system: .. list-table:: :header-rows: 1 :widths: 22 18 14 46 * - Import name - Kind - Cardinality - Description * - ``env.kernel_memory`` - shared (``WebAssembly.Memory({shared: true})``) - **1** for the entire system - The kernel's address space. Holds kernel ``.text`` / ``.rodata`` / ``.data`` / ``.bss``, the kernel heap (slab, vmalloc area), per-task kernel stacks, page-table arrays, virtq rings, signal-pending words, and every other Linux datum that crosses Worker boundaries. Imported by every wasm module instance in the system — every userspace ``.wasm`` instance AND every per-Worker instance of ``vmlinux.wasm``. * - ``env.user_memory`` - shared (``WebAssembly.Memory({shared: true})``) - **1 per process Worker** (plus a 1-page sentinel for the standalone vmlinux Worker, see §1.1) - That Worker's userspace pages: user ``.text`` / ``.rodata`` / ``.data`` / ``.bss``, user heap, user stack, mapped files. Imported by the userspace ``.wasm`` running in that Worker, AND by the kernel ``.wasm`` instance running in that Worker (so that kernel code executing on behalf of this process can do uaccess against the calling process's user memory). **Why ``shared: true`` for what the model calls a "per-process private" memory.** "Private" here is an address-space property (no other process has a window into this Worker's user pages), not a SharedArrayBuffer-sharability claim. The backing SAB has to be visible to (a) the running userspace ``.wasm`` instance, (b) the kernel ``vmlinux.wasm`` instance in the same Worker, and (c) the main-thread syscall router that resolves syscall buffers without having to bounce through a wasm export. (a) and (b) live in the same Worker so they would share a non-shared Memory too; (c) requires SAB-backing. Additionally, V8 (and SpiderMonkey) reject ``WebAssembly.instantiate`` with "mismatch in shared state of memory" if the wasm module's import declaration says ``shared`` and the JS-side Memory is not — and since the post-link splice (``scripts/wasm-add-memory.py``) declares this import as ``shared``, every JS-side allocation MUST be ``shared: true`` too. K1 discovered this the hard way; the spec now matches reality. Two consequences worth pinning here, before talking about indices: * HardwareJS hands the **same** ``env.kernel_memory`` JS object to every Worker at instantiation. That is what gives Linux real SMP semantics against shared memory in this port: per-CPU variables, spinlocks, RCU, IPIs all carry their full upstream meaning because the wasm engine enforces the cross-thread memory model on the underlying ``SharedArrayBuffer``. * HardwareJS allocates a **fresh** ``env.user_memory`` per process Worker. The kernel never directly addresses *another* process's user memory; a ``copy_to_user`` happens only while the kernel is executing in *this* Worker, where ``env.user_memory`` is bound to *this* process's userspace pages. What each module's local memory indices end up being is a derived fact: .. list-table:: :header-rows: 1 :widths: 32 34 34 * - Module - Memory index 0 - Memory index 1 * - Userspace ``.wasm`` - ``env.user_memory`` (own) - ``env.kernel_memory`` * - Kernel ``vmlinux.wasm`` instance running in a process Worker - ``env.kernel_memory`` (own) - ``env.user_memory`` (that Worker's user pages) * - Kernel ``vmlinux.wasm`` instance running in the vmlinux Worker (kthreads only — see §1.1) - ``env.kernel_memory`` (own) - ``env.user_memory`` bound to a 1-page sentinel — see §1.1 This pattern is just "``memory 0`` is whichever memory the module's own compiled ``.data``/``.bss``/stack land in (because that is the memory ``wasm-ld`` emits default loads/stores against), ``memory 1`` is the other side." It is not a special rule; it is what falls out once you accept that **names are the ABI, indices are an artefact of link order**. Cross-module portability of source code follows directly. The address-space attributes used in ``uaccess.S`` are written from the *kernel* module's index space (because they describe kernel→user / user→kernel copies); the same copies *cannot* be expressed from userspace because userspace never crosses into kernel memory in the source language — only via syscall traps which HardwareJS dispatches without a ``memory.copy``. 1.1 Per-Worker kernel instantiation and the kthread sentinel ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Every Worker — every process Worker and the dedicated vmlinux Worker — holds its **own** ``WebAssembly.Instance`` of ``vmlinux.wasm``. They share exactly what they import. So: * All instances import the same ``env.kernel_memory`` and therefore see identical ``current``, ``init_mm``, runqueues, slab arenas, kernel ``__bss``, etc., subject to the wasm memory model's atomic-access rules. * Each instance has its own ``__stack_pointer`` global, its own non-shared globals, its own Asyncify control state. This is what makes the "a process Worker IS a CPU" identity literally true at the wasm level: each Worker is a distinct execution context with its own register-state analogue, executing against shared kernel memory. The **vmlinux Worker** is the sole Worker without a userspace process in it. It hosts only kthreads (``kthreadd``, ``ksoftirqd``, ``kworker``, the idle thread, and so on). It must still satisfy the ``env.user_memory`` import to keep ``vmlinux.wasm``'s import shape uniform; calling into the kernel from kthread context with no user memory bound is otherwise an invariant that the wasm loader cannot express. HardwareJS therefore hands this Worker a **1-page, ``shared:true``-backed sentinel Memory** under the same ``env.user_memory`` import field name. Its only purpose is to fault loudly if anything ever tries to use it. Any ``copy_to_user`` / ``copy_from_user`` issued from kthread context is a kernel bug; the sentinel ensures the bug materialises as a wasm trap (out-of-bounds at offset > 65535) on the first cross-memory ``memory.copy`` rather than as silent corruption of an unrelated process's pages. **Why the same import name instead of ``env.user_memory_sentinel``?** A wasm module declares each import once at compile time; we cannot swap import names per Worker class. The kernel's ``vmlinux.wasm`` binary has a single ``env.user_memory`` import slot, and HardwareJS chooses *what to bind to it* at instantiation time: a real process's user pages for process Workers, the 1-page sentinel for the vmlinux Worker. The distinction is in the binding (a HardwareJS-side decision keyed off "is this Worker hosting a userspace process or running kthreads?"), not in the import name. **Why ``shared:true`` for the sentinel?** Same reason as for real ``env.user_memory`` (see §1 table): V8/SpiderMonkey require the JS-side Memory's ``shared`` flag to match the wasm module's import declaration, and that declaration is fixed to ``shared`` by the ``scripts/wasm-add-memory.py`` splice. The sentinel is 1 page, owned by exactly one Worker, never handed elsewhere — its SAB-backing is inert. 1.2 Syscall entry sequence (kernel-stack swap) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A kernel stack is per ``task_struct``, slab-allocated out of ``env.kernel_memory`` exactly like upstream Linux. The wasm runtime's notion of "the stack" is a single ``__stack_pointer`` global per instance; the kernel must therefore swap that pointer at every kernel-mode entry and exit. The required dance on a syscall trap from userspace into the kernel module instance running in the same Worker: 1. Userspace ``.wasm`` calls the ``linux.syscall`` import. 2. HardwareJS calls the kernel instance's exported ``wasm_syscall(nr, a0..a5)``. This crosses no Worker boundary; the kernel instance lives in the same Worker as the userspace instance. 3. ``wasm_syscall`` is implemented in ``arch/wasm32/kernel/entry.c``. Before any C code runs, it reads ``current->stack`` from ``env.kernel_memory`` and writes it into the instance's ``__stack_pointer`` global; saves the old (userspace) value as a per-task field. From this point all C locals and saved registers land in this task's kernel stack in ``env.kernel_memory``. 4. ``do_syscall(nr, a0..a5)`` runs as in upstream Linux. 5. On return, the saved userspace ``__stack_pointer`` is restored and control returns through the import to the userspace ``.wasm``. Without that swap, two Workers concurrently entering the kernel would each write C locals into the *userspace* stack of their own process — which is private memory, so they wouldn't clobber each other directly, but they also wouldn't be writing anywhere the kernel can later find them. Locks, ``schedule()``'s stack-snapshotting, the entire kernel stack-walk and unwind machinery, and Asyncify's saved-state pointer all assume the running C code is using a slab-allocated stack in ``env.kernel_memory``. The swap is non-negotiable. The kthread-context path (vmlinux Worker) does not swap from userspace into kernel; it always runs against the kernel stack of whichever kthread is currently scheduled. The Asyncify-driven ``switch_to`` (see ``docs/ARCHITECTURE.md`` §17.3) saves the outgoing kthread's stack pointer into its ``task_struct`` and loads the incoming one. 1.3 Cross-memory boundary copies ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Every ``copy_to_user`` / ``copy_from_user`` (and the ``get_user`` / ``put_user`` family) is implemented as a wasm multi-memory ``memory.copy`` from the kernel module's index space. From the kernel instance's perspective (``memory 0 = env.kernel_memory``, ``memory 1 = env.user_memory``): * ``copy_to_user(dst_user, src_kernel, n)`` → ``memory.copy 1, 0`` (dst=user, src=kernel) * ``copy_from_user(dst_kernel, src_user, n)`` → ``memory.copy 0, 1`` (dst=kernel, src=user) * ``clear_user(dst_user, n)`` → ``memory.fill 1`` (target=user) Each translates to one wasm instruction (opcode prefix ``0xfc``, opcode ``0x0a`` for copy, ``0x0b`` for fill, then ULEB128 memory indices). They are NOT ``memcpy``. A ``memcpy`` that pretends both pointers live in the same linear memory will silently read garbage the first time a real syscall argument arrives. The primitives live in ``arch/wasm32/lib/uaccess.S`` because clang's C front-end cannot today lower address-space-annotated pointer copies to multi-memory ``memory.copy`` (LLVM 22 crashes in instruction selection on ``addrspacecast``). The LLVM **assembler**, however, accepts ``memory.copy , `` and ``memory.fill `` syntax and emits the correct opcode bytes; ``wasm-ld`` preserves those bytes unchanged through linking. Userspace binaries never emit cross-memory copies. A userspace process that needs to ``read()`` from the kernel does so via the syscall trap, which is structurally a ``call`` to the ``linux.syscall`` import; the kernel side does the cross-memory copy on the userspace's behalf. Toolchain note: wasm-ld single-memory limitation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ LLVM 22's ``wasm-ld`` does not (yet) emit multi-memory modules: it preserves ``memory.copy 0, 1`` opcode bytes in the CODE section but only writes a single entry in the MEMORY/IMPORT section. The resulting module is structurally invalid (``[parse exception: invalid memory index]``). ``linux-wasm/scripts/link-vmlinux-wasm.sh`` therefore runs a small post-link rewriter (``linux-wasm/scripts/wasm-add-memory.py``) on every produced wasm artifact to splice in the second memory as an imported memory entry. The rewriter is ~280 lines of self-contained wasm-format code; the wasm spec is final, V8 / SpiderMonkey / Wasmtime all accept the result, and the rewriter goes away when ``wasm-ld`` grows native multi-memory linking. Nothing about the kernel source has to change at that point. This is NOT shortcut debt. The same rewriter is reused by ``wackywasm-tools/bin/wasmld`` for userspace binaries — they import ``env.kernel_memory`` as their second memory. -------------------------------------------------------------------------------- 2. Page-table model -------------------------------------------------------------------------------- WebAssembly has no hardware MMU. A wasm linear memory is a flat, byte-addressed array of bytes; there are no page tables, no TLB, no faults, no PROT_* bits the hardware enforces. From the kernel's point of view, calling ``set_pte()`` on wasm32 cannot stop ``memory.load`` from succeeding at that address. What Linux still needs from the arch even on flat memory is a *bookkeeping data structure* per address space, so that ``do_mmap``, ``do_munmap``, ``vm_area_struct`` linkage, ``copy_page_range`` on fork, and the rmap / OOM killer accounting all have something to walk. The wasm32 page tables are exactly that: a degenerate single-level array of bookkeeping ``pte_t`` entries per process, with PUD and PMD folded away. Levels claimed ~~~~~~~~~~~~~~ :: PGD ──> (folded PUD) ──> (folded PMD) ──> PTE ──> physical page CONFIG_PGTABLE_LEVELS = 2 pgtable-nopud.h — folds PUD into PGD pgtable-nopmd.h — folds PMD into PUD (== PGD because of nopud) Both fold headers are pulled in via ``arch/wasm32/include/asm/pgtable.h`` explicitly, NOT via ``generic-y``, so the level shape is reviewable here in one place. Page granularity ~~~~~~~~~~~~~~~~ :: PAGE_SHIFT = 12 (4 KiB, matches Linux's everywhere-default) PAGE_SIZE = 4096 PAGE_MASK = ~(PAGE_SIZE - 1) A wasm linear memory page is 64 KiB. We do not export the wasm page size to the rest of the kernel; we use 4 KiB everywhere and let ``memory.grow`` allocate sixteen Linux pages per wasm page. ``mm_init`` will see this as "physical memory comes in 16-page contiguous chunks" and the buddy allocator handles that case natively. PTE shape ~~~~~~~~~ ``pte_t`` is a single ``unsigned long``. It encodes: * bits 12+ — page-frame number into the per-process ``env.user_memory`` of this VMA's owning Worker. Because ``env.user_memory`` is flat, PFN == byte offset >> 12. * bits 0..11 — software-only flags: ``_PAGE_PRESENT``, ``_PAGE_USER``, ``_PAGE_RW``, ``_PAGE_DIRTY``, ``_PAGE_ACCESSED``. These are advisory; the wasm runtime cannot enforce them. Real protection comes from the Worker boundary (a process Worker cannot reach into another Worker's ``env.user_memory`` at all, regardless of PTE flags). The fact that protection is enforced by the Worker boundary rather than by PTE bits is what makes "everything in flat memory is technically accessible to the running wasm code" not a security hole: the running wasm code is one process, and there is no other process in the same address space to protect. Address space size ~~~~~~~~~~~~~~~~~~ ``env.user_memory`` is a wasm32 linear memory: max 4 GiB. We cap ``TASK_SIZE`` at 3 GiB, leaving the top 1 GiB of the wasm32 address range as a guard / future kernel-virtual-mapping window. ``PAGE_OFFSET`` is therefore ``0xc0000000``; addresses ≥ ``PAGE_OFFSET`` in a user pointer mean "this is a kernel pointer that someone passed to userspace by mistake, EFAULT." (Inside the kernel, kernel data structures are addressed by their natural offsets into ``env.kernel_memory``. The "kernel virtual address space" Linux normally talks about is collapsed into "indices into env.kernel_memory", which has its own 4 GiB range; we never have to translate between the two.) Fork / COW ~~~~~~~~~~ ``fork`` cannot use copy-on-write because wasm has no page faults: the child Worker's ``env.user_memory`` is allocated fresh and the parent's data is bulk-copied into it at clone time. The PTE bookkeeping in ``copy_page_range`` still runs so that the child's mm accounting agrees with the buddy allocator, but ``pte_mkwrite`` / ``pte_wrprotect`` are advisory. This is the only place the model loses anything significant compared to a real MMU arch. K3 will revisit whether to expose a "snapshot+lazy-copy" primitive from HardwareJS that gets the wasm engine to deduplicate physical pages until first write; if so, COW becomes a HardwareJS-level optimization under an unchanged PTE bookkeeping layer. Acceptance of this model ~~~~~~~~~~~~~~~~~~~~~~~~ This model is sufficient for K0–K2 (build, head.c, setup_arch). It commits to a single-level real PTE table at K3 (mm_init). Changes to the level count, page size, PTE bit layout, or PAGE_OFFSET after v0.1.0 are breaking changes to the arch ABI and require a corresponding bump of the ``wasm.linux.exec`` ``abi_version``. -------------------------------------------------------------------------------- 3. Per-CPU data layout -------------------------------------------------------------------------------- Linux's per-CPU machinery uses two abstractions that this port has to reproduce against ``wasm-ld``'s linker-script-less layout: 1. **Static per-CPU image:** a contiguous block of bytes in the linked binary, bracketed by linker symbols ``__per_cpu_start`` and ``__per_cpu_end``, containing the initial values of every variable declared with ``DEFINE_PER_CPU``. ``__per_cpu_load`` is the address from which ``pcpu_embed_first_chunk`` reads when copying the image into each CPU's runtime slot — on every other arch, the linker script aliases ``__per_cpu_load`` onto ``__per_cpu_start``. 2. **Dynamic per-CPU allocator:** ``__alloc_percpu`` requests are served out of a runtime chunk allocated by ``setup_per_cpu_areas`` at boot. The dynamic area starts immediately after the static area within each CPU's slot, sized per CPU as ``unit_size = roundup(static_size + reserved_size + dyn_size, PAGE_SIZE)``. The generic ``mm/percpu.c::pcpu_build_alloc_info()`` derives ``static_size = __per_cpu_end - __per_cpu_start``. On wasm32, that subtraction does NOT yield the static extent — see §17.7 EIGHTH HISTORICAL NOTE for the empirical refutation and the cross-tool layout interaction that breaks it. Quick summary of the mechanism: * ``wasm-ld`` with ``--no-merge-data-segments`` emits data segments in input order (across input objects, and within an object in source declaration order). It does NOT sort by section name. * ``arch/wasm32/built-in.a`` is extracted alphabetically first from ``vmlinux.a``; within it, ``percpu_anchors.o`` is the first member to define ``.data..percpu*`` segments. * ``percpu_anchors.c`` declares ``__per_cpu_start`` (section ``.data..percpu..aa_start``) and ``__per_cpu_end`` (section ``.data..percpu..zz_end``). Both emit FIRST and adjacent, BEFORE any other compilation unit's ``.data..percpu`` data. So ``&__per_cpu_end - &__per_cpu_start`` is one byte (or padded thereto by alignment), wildly understating the actual extent. 3.1 Wasm32-specific solution (K4) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Two pieces, each slotted against an upstream-documented extension point: (a) ``__per_cpu_load`` aliased to ``__per_cpu_start`` via ``__attribute__((alias("__per_cpu_start")))`` in ``percpu_anchors.c``. This matches what every other arch's linker script does with the ``__per_cpu_load = .`` line just before ``__per_cpu_start = .``. (b) ``CONFIG_HAVE_SETUP_PER_CPU_AREA=y`` (set in ``arch/wasm32/Kconfig``) replaces the generic ``mm/percpu.c::setup_per_cpu_areas`` with a wasm32-private implementation in ``arch/wasm32/kernel/percpu.c``. The wasm32 version constructs ``pcpu_alloc_info`` directly via ``pcpu_alloc_alloc_info()`` and hardcodes ``static_size`` to ``WASM32_STATIC_PCPU_SIZE = 128 KiB``. The actual defconfig extent is on the order of a few KiB; 128 KiB is generous head-room. ``__per_cpu_offset[]`` is defined locally in this same file (the generic definition in ``mm/percpu.c`` is suppressed when ``HAVE_SETUP_PER_CPU_AREA=y``). The two pieces compose: ``__per_cpu_load`` aliasing fixes the memcpy source address; the custom ``setup_per_cpu_areas`` fixes the static-size and offset computation. 3.2 Cost ~~~~~~~~ Each per-CPU slot has ``128 KiB - actual_extent`` of unused memory in its static-image area. With ``NR_CPUS=8`` and the defconfig static extent (~few KiB), worst case is roughly 1 MiB of unused-but-reserved RAM. We have 64 MiB of kernel RAM at K4. Acceptable. 3.3 Future cleanup ~~~~~~~~~~~~~~~~~~ The proper fix is a post-link Python pass that walks the linked wasm binary's ``.data..percpu*`` data segments, computes the actual highest address reached, and patches ``__per_cpu_end``'s exported value (or rewrites ``setup_per_cpu_areas`` to read the computed size from a wasm global emitted by the pass). At that point: * ``percpu_anchors.c`` can drop its custom anchor sections and the ``__per_cpu_load`` alias. * ``arch/wasm32/kernel/percpu.c`` can be deleted; the generic ``mm/percpu.c::setup_per_cpu_areas`` takes over. * ``CONFIG_HAVE_SETUP_PER_CPU_AREA`` can be unset. This is captured as a §15 row in the same family as the ``vmlinux.lds.S`` post-link Python pass that covers ``__setup_start`` / ``__initcall_start`` / etc. — same root cause (wasm-ld doesn't consume linker scripts) and the same fix retires multiple symptomatic rows simultaneously. -------------------------------------------------------------------------------- Appendix A — Per-module index tables (derived, non-normative) -------------------------------------------------------------------------------- The names in §1 are the ABI. The indices below are what each module ends up with after ``wasm-ld`` + the ``wasm-add-memory.py`` post-link splice; they are reviewable here as a debugging aid but they are NOT the spec — if a future toolchain or splice order changes them, the spec is unchanged so long as the import NAMES still match §1. Userspace ``.wasm`` (e.g. ``/init``, ``/bin/bash``, ``/bin/ls``):: memory 0 ← env.user_memory (wasm-ld default, declared by --import-memory=env,user_memory) memory 1 ← env.kernel_memory (spliced in by wackywasm-tools/bin/wasmld via scripts/wasm-add-memory.py) Kernel ``vmlinux.wasm`` (one binary, instantiated per Worker):: memory 0 ← env.kernel_memory (wasm-ld default, declared by --import-memory=env,kernel_memory in scripts/link-vmlinux-wasm.sh) memory 1 ← env.user_memory (spliced in by scripts/wasm-add-memory.py during the kernel link; same import name in every Worker. HardwareJS binds this name at Worker instantiation time: process Workers get the process's own user pages; the vmlinux Worker gets the 1-page sentinel.) Changing the splice order — for example, splicing the second memory *before* wasm-ld's default in some future refactor — flips the indices but does not change the ABI. ``uaccess.S`` would need to be updated to match, the doc would not.