245 lines · plain
1xdrgen - Linux Kernel XDR code generator2 3Introduction4------------5 6SunRPC programs are typically specified using a language defined by7RFC 4506. In fact, all IETF-published NFS specifications provide a8description of the specified protocol using this language.9 10Since the 1990's, user space consumers of SunRPC have had access to11a tool that could read such XDR specifications and then generate C12code that implements the RPC portions of that protocol. This tool is13called rpcgen.14 15This RPC-level code is code that handles input directly from the16network, and thus a high degree of memory safety and sanity checking17is needed to help ensure proper levels of security. Bugs in this18code can have significant impact on security and performance.19 20However, it is code that is repetitive and tedious to write by hand.21 22The C code generated by rpcgen makes extensive use of the facilities23of the user space TI-RPC library and libc. Furthermore, the dialect24of the generated code is very traditional K&R C.25 26The Linux kernel's implementation of SunRPC-based protocols hand-roll27their XDR implementation. There are two main reasons for this:28 291. libtirpc (and its predecessors) operate only in user space. The30 kernel's RPC implementation and its API are significantly31 different than libtirpc.32 332. rpcgen-generated code is believed to be less efficient than code34 that is hand-written.35 36These days, gcc and its kin are capable of optimizing code better37than human authors. There are only a few instances where writing38XDR code by hand will make a measurable performance different.39 40In addition, the current hand-written code in the Linux kernel is41difficult to audit and prove that it implements exactly what is in42the protocol specification.43 44In order to accrue the benefits of machine-generated XDR code in the45kernel, a tool is needed that will output C code that works against46the kernel's SunRPC implementation rather than libtirpc.47 48Enter xdrgen.49 50 51Dependencies52------------53 54These dependencies are typically packaged by Linux distributions:55 56- python357- python3-lark58- python3-jinja259 60These dependencies are available via PyPi:61 62- pip install 'lark[interegular]'63 64 65XDR Specifications66------------------67 68When adding a new protocol implementation to the kernel, the XDR69specification can be derived by feeding a .txt copy of the RFC to70the script located in tools/net/sunrpc/extract.sh.71 72 $ extract.sh < rfc0001.txt > new2.x73 74 75Operation76---------77 78Once a .x file is available, use xdrgen to generate source and79header files containing an implementation of XDR encoding and80decoding functions for the specified protocol.81 82 $ ./xdrgen definitions new2.x > include/linux/sunrpc/xdrgen/new2.h83 $ ./xdrgen declarations new2.x > new2xdr_gen.h84 85and86 87 $ ./xdrgen source new2.x > new2xdr_gen.c88 89The files are ready to use for a server-side protocol implementation,90or may be used as a guide for implementing these routines by hand.91 92By default, the only comments added to this code are kdoc comments93that appear directly in front of the public per-procedure APIs. For94deeper introspection, specifying the "--annotate" flag will insert95additional comments in the generated code to help readers match the96generated code to specific parts of the XDR specification.97 98Because the generated code is targeted for the Linux kernel, it99is tagged with a GPLv2-only license.100 101The xdrgen tool can also provide lexical and syntax checking of102an XDR specification:103 104 $ ./xdrgen lint xdr/new.x105 106 107How It Works108------------109 110xdrgen does not use machine learning to generate source code. The111translation is entirely deterministic.112 113RFC 4506 Section 6 contains a BNF grammar of the XDR specification114language. The grammar has been adapted for use by the Python Lark115module.116 117The xdr.ebnf file in this directory contains the grammar used to118parse XDR specifications. xdrgen configures Lark using the grammar119in xdr.ebnf. Lark parses the target XDR specification using this120grammar, creating a parse tree.121 122xdrgen then transforms the parse tree into an abstract syntax tree.123This tree is passed to a series of code generators.124 125The generators are implemented as Python classes residing in the126generators/ directory. Each generator emits code created from Jinja2127templates stored in the templates/ directory.128 129The source code is generated in the same order in which they appear130in the specification to ensure the generated code compiles. This131conforms with the behavior of rpcgen.132 133xdrgen assumes that the generated source code is further compiled by134a compiler that can optimize in a number of ways, including:135 136 - Unused functions are discarded (ie, not added to the executable)137 138 - Aggressive function inlining removes unnecessary stack frames139 140 - Single-arm switch statements are replaced by a single conditional141 branch142 143And so on.144 145 146Pragmas147-------148 149Pragma directives specify exceptions to the normal generation of150encoding and decoding functions. Currently one directive is151implemented: "public".152 153Pragma exclude154------ -------155 156 pragma exclude <RPC procedure> ;157 158In some cases, a procedure encoder or decoder function might need159special processing that cannot be automatically generated. The160automatically-generated functions might conflict or interfere with161the hand-rolled function. To avoid editing the generated source code162by hand, a pragma can specify that the procedure's encoder and163decoder functions are not included in the generated header and164source.165 166For example:167 168 pragma exclude NFSPROC3_READDIRPLUS;169 170Excludes the decoder function for the READDIRPLUS argument and the171encoder function for the READDIRPLUS result.172 173Note that because data item encoder and decoder functions are174defined "static __maybe_unused", subsequent compilation175automatically excludes data item encoder and decoder functions that176are used only by excluded procedure.177 178Pragma header179------ ------180 181 pragma header <string> ;182 183Provide a name to use for the header file. For example:184 185 pragma header nlm4;186 187Adds188 189 #include "nlm4xdr_gen.h"190 191to the generated source file.192 193Pragma public194------ ------195 196 pragma public <XDR data item> ;197 198Normally XDR encoder and decoder functions are "static". In case an199implementer wants to call these functions from other source code,200s/he can add a public pragma in the input .x file to indicate a set201of functions that should get a prototype in the generated header,202and the function definitions will not be declared static.203 204For example:205 206 pragma public nfsstat3;207 208Adds these prototypes in the generated header:209 210 bool xdrgen_decode_nfsstat3(struct xdr_stream *xdr, enum nfsstat3 *ptr);211 bool xdrgen_encode_nfsstat3(struct xdr_stream *xdr, enum nfsstat3 value);212 213And, in the generated source code, both of these functions appear214without the "static __maybe_unused" modifiers.215 216 217Future Work218-----------219 220Finish implementing XDR pointer and list types.221 222Generate client-side procedure functions223 224Expand the README into a user guide similar to rpcgen(1)225 226Add more pragma directives:227 228 * @pages -- use xdr_read/write_pages() for the specified opaque229 field230 * @skip -- do not decode, but rather skip, the specified argument231 field232 233Enable something like a #include to dynamically insert the content234of other specification files235 236Properly support line-by-line pass-through via the "%" decorator237 238Build a unit test suite for verifying translation of XDR language239into compilable code240 241Add a command-line option to insert trace_printk call sites in the242generated source code, for improved (temporary) observability243 244Generate kernel Rust code as well as C code245