316 lines · plain
1=================2TableGen Overview3=================4 5.. contents::6 :local:7 8.. toctree::9 :hidden:10 11 BackEnds12 BackGuide13 ProgRef14 15Introduction16============17 18TableGen's purpose is to help a human develop and maintain records of19domain-specific information. Because there may be a large number of these20records, it is specifically designed to allow writing flexible descriptions and21for common features of these records to be factored out. This reduces the22amount of duplication in the description, reduces the chance of error, and makes23it easier to structure domain-specific information.24 25The TableGen front end parses a file, instantiates the declarations, and26hands the result off to a domain-specific `backend`_ for processing. See27the :doc:`TableGen Programmer's Reference <./ProgRef>` for an in-depth28description of TableGen. See :doc:`tblgen - Description to C++ Code29<../CommandGuide/tblgen>` for details on the ``*-tblgen`` commands30that run the various flavors of TableGen.31 32The current major users of TableGen are :doc:`The LLVM Target-Independent33Code Generator <../CodeGenerator>` and the `Clang diagnostics and attributes34<https://clang.llvm.org/docs/UsersManual.html#controlling-errors-and-warnings>`_.35 36Note that if you work with TableGen frequently and use emacs or vim,37you can find an emacs "TableGen mode" and a vim language file in the38``llvm/utils/emacs`` and ``llvm/utils/vim`` directories of your LLVM39distribution, respectively.40 41.. _intro:42 43 44The TableGen program45====================46 47TableGen files are interpreted by the TableGen program: ``llvm-tblgen`` available48in your build directory under ``bin``. It is not installed in the system (or where49your sysroot is set to), since it has no use beyond LLVM's build process.50 51Running TableGen52----------------53 54TableGen runs just like any other LLVM tool. The first (optional) argument55specifies the file to read. If a filename is not specified, ``llvm-tblgen``56reads from standard input.57 58The ``-o`` option specifies the output file or ``-`` to output to59stdout. Where TableGen produces multiple output files, the option60specifies the name of the main output file, which also works as the61name prefix for other output files.62 63To be useful, one of the `backends`_ must be used. These backends are64selectable on the command line (type '``llvm-tblgen -help``' for a list). For65example, to get a list of all of the definitions that subclass a particular type66(which can be useful for building up an enum list of these records), use the67``-print-enums`` option:68 69.. code-block:: bash70 71 $ llvm-tblgen X86.td -print-enums -class=Register72 AH, AL, AX, BH, BL, BP, BPL, BX, CH, CL, CX, DH, DI, DIL, DL, DX, EAX, EBP, EBX,73 ECX, EDI, EDX, EFLAGS, EIP, ESI, ESP, FP0, FP1, FP2, FP3, FP4, FP5, FP6, IP,74 MM0, MM1, MM2, MM3, MM4, MM5, MM6, MM7, R10, R10B, R10D, R10W, R11, R11B, R11D,75 R11W, R12, R12B, R12D, R12W, R13, R13B, R13D, R13W, R14, R14B, R14D, R14W, R15,76 R15B, R15D, R15W, R8, R8B, R8D, R8W, R9, R9B, R9D, R9W, RAX, RBP, RBX, RCX, RDI,77 RDX, RIP, RSI, RSP, SI, SIL, SP, SPL, ST0, ST1, ST2, ST3, ST4, ST5, ST6, ST7,78 XMM0, XMM1, XMM10, XMM11, XMM12, XMM13, XMM14, XMM15, XMM2, XMM3, XMM4, XMM5,79 XMM6, XMM7, XMM8, XMM9,80 81 $ llvm-tblgen X86.td -print-enums -class=Instruction82 ABS_F, ABS_Fp32, ABS_Fp64, ABS_Fp80, ADC32mi, ADC32mi8, ADC32mr, ADC32ri,83 ADC32ri8, ADC32rm, ADC32rr, ADC64mi32, ADC64mi8, ADC64mr, ADC64ri32, ADC64ri8,84 ADC64rm, ADC64rr, ADD16mi, ADD16mi8, ADD16mr, ADD16ri, ADD16ri8, ADD16rm,85 ADD16rr, ADD32mi, ADD32mi8, ADD32mr, ADD32ri, ADD32ri8, ADD32rm, ADD32rr,86 ADD64mi32, ADD64mi8, ADD64mr, ADD64ri32, ...87 88The default backend prints out all of the records. There is also a general89backend which outputs all the records as a JSON data structure, enabled using90the `-dump-json` option.91 92If you plan to use TableGen, you will most likely have to write a `backend`_93that extracts the information specific to what you need and formats it in the94appropriate way. You can do this by extending TableGen itself in C++ or by95writing a script in any language that can consume the JSON output.96 97Example98-------99 100With no other arguments, `llvm-tblgen` parses the specified file and prints out all101of the classes, then all of the definitions. This is a good way to see what the102various definitions expand to fully. Running this on the ``X86.td`` file prints103this (at the time of this writing):104 105.. code-block:: text106 107 ...108 def ADD32rr { // Instruction X86Inst I109 string Namespace = "X86";110 dag OutOperandList = (outs GR32:$dst);111 dag InOperandList = (ins GR32:$src1, GR32:$src2);112 string AsmString = "add{l}\t{$src2, $dst|$dst, $src2}";113 list<dag> Pattern = [(set GR32:$dst, (add GR32:$src1, GR32:$src2))];114 list<Register> Uses = [];115 list<Register> Defs = [EFLAGS];116 list<Predicate> Predicates = [];117 int CodeSize = 3;118 int AddedComplexity = 0;119 bit isReturn = 0;120 bit isBranch = 0;121 bit isIndirectBranch = 0;122 bit isBarrier = 0;123 bit isCall = 0;124 bit canFoldAsLoad = 0;125 bit mayLoad = 0;126 bit mayStore = 0;127 bit isImplicitDef = 0;128 bit isConvertibleToThreeAddress = 1;129 bit isCommutable = 1;130 bit isTerminator = 0;131 bit isReMaterializable = 0;132 bit isPredicable = 0;133 bit hasDelaySlot = 0;134 bit usesCustomInserter = 0;135 bit hasCtrlDep = 0;136 bit isNotDuplicable = 0;137 bit hasSideEffects = 0;138 InstrItinClass Itinerary = NoItinerary;139 string Constraints = "";140 string DisableEncoding = "";141 bits<8> Opcode = { 0, 0, 0, 0, 0, 0, 0, 1 };142 Format Form = MRMDestReg;143 bits<6> FormBits = { 0, 0, 0, 0, 1, 1 };144 ImmType ImmT = NoImm;145 bits<3> ImmTypeBits = { 0, 0, 0 };146 bit hasOpSizePrefix = 0;147 bit hasAdSizePrefix = 0;148 bits<4> Prefix = { 0, 0, 0, 0 };149 bit hasREX_WPrefix = 0;150 FPFormat FPForm = ?;151 bits<3> FPFormBits = { 0, 0, 0 };152 }153 ...154 155This definition corresponds to the 32-bit register-register ``add`` instruction156of the x86 architecture. ``def ADD32rr`` defines a record named157``ADD32rr``, and the comment at the end of the line indicates the superclasses158of the definition. The body of the record contains all of the data that159TableGen assembled for the record, indicating that the instruction is part of160the ``X86`` namespace, the pattern indicating how the instruction is selected by161the code generator, that it is a two-address instruction, has a particular162encoding, etc. The contents and semantics of the information in the record are163specific to the needs of the X86 backend, and are only shown as an example.164 165As you can see, a lot of information is needed for every instruction supported166by the code generator, and specifying it all manually would be unmaintainable,167prone to bugs, and tiring to do in the first place. Because we are using168TableGen, all of the information was derived from the following definition:169 170.. code-block:: text171 172 let Defs = [EFLAGS],173 isCommutable = 1, // X = ADD Y,Z --> X = ADD Z,Y174 isConvertibleToThreeAddress = 1 in // Can transform into LEA.175 def ADD32rr : I<0x01, MRMDestReg, (outs GR32:$dst),176 (ins GR32:$src1, GR32:$src2),177 "add{l}\t{$src2, $dst|$dst, $src2}",178 [(set GR32:$dst, (add GR32:$src1, GR32:$src2))]>;179 180This definition makes use of the custom class ``I`` (extended from the custom181class ``X86Inst``), which is defined in the X86-specific TableGen file, to182factor out the common features that instructions of its class share. A key183feature of TableGen is that it allows the end user to define the abstractions184they prefer to use when describing their information.185 186Syntax187======188 189TableGen has a syntax loosely based on C++ templates, with built-in190types and specification. In addition, TableGen's syntax introduces some191automation concepts like multiclass, foreach, let, etc.192 193Basic concepts194--------------195 196TableGen files consist of two key parts: 'classes' and 'definitions', both of197which are considered 'records'.198 199**TableGen records** have a unique name, a list of values, and a list of200superclasses. The list of values is the main data that TableGen builds for each201record; it is this that holds the domain-specific information for the202application. The interpretation of this data is left to a specific `backend`_,203but the structure and format rules are taken care of and fixed by204TableGen.205 206**TableGen definitions** are the concrete form of 'records'. These generally do207not have any undefined values and are marked with the '``def``' keyword.208 209.. code-block:: text210 211 def FeatureFPARMv8 : SubtargetFeature<"fp-armv8", "HasFPARMv8", "true",212 "Enable ARMv8 FP">;213 214In this example, ``FeatureFPARMv8`` is ``SubtargetFeature`` record initialised215with some values. The names of the classes are defined via the216keyword `class` either in the same file or some other included. Most target217TableGen files include the generic ones in ``include/llvm/Target``.218 219**TableGen classes** are abstract records that are used to build and describe220other records. These classes allow the end-user to build abstractions for221either the domain they are targeting (such as ``Register``, ``RegisterClass``, and222``Instruction`` in the LLVM code generator) or for the implementor to help factor223out common properties of records (such as ``FPInst``, which is used to represent224floating point instructions in the X86 backend). TableGen keeps track of all of225the classes that are used to build up a definition, so the backend can find all226definitions of a particular class, such as ``Instruction``.227 228.. code-block:: text229 230 class ProcNoItin<string Name, list<SubtargetFeature> Features>231 : Processor<Name, NoItineraries, Features>;232 233Here, the class ``ProcNoItin``, receiving parameters ``Name`` of type ``string`` and234a list of target features is specializing the class ``Processor`` by passing the235arguments down as well as hard-coding ``NoItineraries``.236 237**TableGen multiclasses** are groups of abstract records that are instantiated238all at once. Each instantiation can result in multiple TableGen definitions.239If a multiclass inherits from another multiclass, the definitions in the240sub-multiclass become part of the current multiclass, as if they were declared241in the current multiclass.242 243.. code-block:: text244 245 multiclass ro_signed_pats<string T, string Rm, dag Base, dag Offset, dag Extend,246 dag address, ValueType sty> {247 def : Pat<(i32 (!cast<SDNode>("sextload" # sty) address)),248 (!cast<Instruction>("LDRS" # T # "w_" # Rm # "_RegOffset")249 Base, Offset, Extend)>;250 251 def : Pat<(i64 (!cast<SDNode>("sextload" # sty) address)),252 (!cast<Instruction>("LDRS" # T # "x_" # Rm # "_RegOffset")253 Base, Offset, Extend)>;254 }255 256 defm : ro_signed_pats<"B", Rm, Base, Offset, Extend,257 !foreach(decls.pattern, address,258 !subst(SHIFT, imm_eq0, decls.pattern)),259 i8>;260 261See the :doc:`TableGen Programmer's Reference <./ProgRef>` for an in-depth262description of TableGen.263 264 265.. _backend:266.. _backends:267 268TableGen backends269=================270 271TableGen files have no real meaning without a backend. The default operation272when running ``*-tblgen`` is to print the information in a textual format, but273that's only useful for debugging the TableGen files themselves. The power274in TableGen is, however, to interpret the source files into an internal275representation that can be generated into anything you want.276 277Current usage of TableGen is to create huge include files with tables that you278can either include directly (if the output is in the language you're coding),279or be used in pre-processing via macros surrounding the include of the file.280 281Direct output can be used if the backend already prints a table in C format282or if the output is just a list of strings (for error and warning messages).283Pre-processed output should be used if the same information needs to be used284in different contexts (like Instruction names), so your backend should print285a meta-information list that can be shaped into different compile-time formats.286 287See :doc:`TableGen BackEnds <./BackEnds>` for a list of available288backends, and see the :doc:`TableGen Backend Developer's Guide <./BackGuide>`289for information on how to write and debug a new backend.290 291Tools and Resources292===================293 294In addition to this documentation, a list of tools and resources for TableGen295can be found in TableGen's296`README <https://github.com/llvm/llvm-project/blob/main/llvm/utils/TableGen/README.md>`_.297 298TableGen Deficiencies299=====================300 301Despite being very generic, TableGen has some deficiencies that have been302pointed out numerous times. The common theme is that, while TableGen allows303you to build domain-specific languages, the final languages that you create304lack the power of other DSLs, which in turn considerably increases the size305and complexity of TableGen files.306 307At the same time, TableGen allows you to create virtually any meaning of308the basic concepts via custom-made backends, which can pervert the original309design and make it very hard for newcomers to understand the evil TableGen310file.311 312There are some in favor of extending the semantics even more, but making sure313backends adhere to strict rules. Others suggest moving to fewer,314more powerful DSLs designed with specific purposes, or even reusing existing315DSLs.316