466 lines · plain
1=====================================2The PDB DBI (Debug Info) Stream3=====================================4 5.. contents::6 :local:7 8.. _dbi_intro:9 10Introduction11============12 13The PDB DBI Stream (Index 3) is one of the largest and most important streams14in a PDB file. It contains information about how the program was compiled,15(e.g. compilation flags, etc), the compilands (e.g. object files) that16were used to link together the program, the source files which were used17to build the program, as well as references to other streams that contain more18detailed information about each compiland, such as the CodeView symbol records19contained within each compiland and the source and line information for20functions and other symbols within each compiland.21 22 23.. _dbi_header:24 25Stream Header26=============27At offset 0 of the DBI Stream is a header with the following layout:28 29 30.. code-block:: c++31 32 struct DbiStreamHeader {33 int32_t VersionSignature;34 uint32_t VersionHeader;35 uint32_t Age;36 uint16_t GlobalStreamIndex;37 uint16_t BuildNumber;38 uint16_t PublicStreamIndex;39 uint16_t PdbDllVersion;40 uint16_t SymRecordStream;41 uint16_t PdbDllRbld;42 int32_t ModInfoSize;43 int32_t SectionContributionSize;44 int32_t SectionMapSize;45 int32_t SourceInfoSize;46 int32_t TypeServerMapSize;47 uint32_t MFCTypeServerIndex;48 int32_t OptionalDbgHeaderSize;49 int32_t ECSubstreamSize;50 uint16_t Flags;51 uint16_t Machine;52 uint32_t Padding;53 };54 55- **VersionSignature** - Unknown meaning. Appears to always be ``-1``.56 57- **VersionHeader** - A value from the following enum.58 59.. code-block:: c++60 61 enum class DbiStreamVersion : uint32_t {62 VC41 = 930803,63 V50 = 19960307,64 V60 = 19970606,65 V70 = 19990903,66 V110 = 2009120167 };68 69Similar to the :doc:`PDB Stream <PdbStream>`, this value always appears to be70``V70``, and it is not clear what the other values are for.71 72- **Age** - The number of times the PDB has been written. Equal to the same73 field from the :ref:`PDB Stream header <pdb_stream_header>`.74 75- **GlobalStreamIndex** - The index of the :doc:`Global Symbol Stream <GlobalStream>`,76 which contains CodeView symbol records for all global symbols. Actual records77 are stored in the symbol record stream, and are referenced from this stream.78 79- **BuildNumber** - A bitfield containing values representing the major and minor80 version number of the toolchain (e.g. 12.0 for MSVC 2013) used to build the81 program, with the following layout:82 83.. code-block:: c++84 85 uint16_t MinorVersion : 8;86 uint16_t MajorVersion : 7;87 uint16_t NewVersionFormat : 1;88 89For the purposes of LLVM, we assume ``NewVersionFormat`` to be always ``true``.90If it is ``false``, the layout above does not apply and the reader should consult91the `Microsoft Source Code <https://github.com/Microsoft/microsoft-pdb>`__ for92further guidance.93 94- **PublicStreamIndex** - The index of the :doc:`Public Symbol Stream <PublicStream>`,95 which contains CodeView symbol records for all public symbols. Actual records96 are stored in the symbol record stream, and are referenced from this stream.97 98- **PdbDllVersion** - The version number of ``mspdbXXXX.dll`` used to produce this99 PDB. Note this obviously does not apply for LLVM as LLVM does not use ``mspdb.dll``.100 101- **SymRecordStream** - The stream containing all CodeView symbol records used102 by the program. This is used for deduplication, so that many different103 compilands can refer to the same symbols without having to include the full record104 content inside of each module stream.105 106- **PdbDllRbld** - Unknown107 108- **MFCTypeServerIndex** - The index of the MFC type server in the109 :ref:`dbi_type_server_map_substream`.110 111- **Flags** - A bitfield with the following layout, containing various112 information about how the program was built:113 114.. code-block:: c++115 116 uint16_t WasIncrementallyLinked : 1;117 uint16_t ArePrivateSymbolsStripped : 1;118 uint16_t HasConflictingTypes : 1;119 uint16_t Reserved : 13;120 121The only one of these that is not self-explanatory is ``HasConflictingTypes``.122Although undocumented, ``link.exe`` contains a hidden flag ``/DEBUG:CTYPES``.123If it is passed to ``link.exe``, this field will be set. Otherwise it will124not be set. It is unclear what this flag does, although it seems to have125subtle implications on the algorithm used to look up type records.126 127- **Machine** - A value from the `CV_CPU_TYPE_e <https://msdn.microsoft.com/en-us/library/b2fc64ek.aspx>`__128 enumeration. Common values are ``0x8664`` (x86-64) and ``0x14C`` (x86).129 130Immediately after the fixed-size DBI Stream header are ``7`` variable-length131`substreams`. The following ``7`` fields of the DBI Stream header specify the132number of bytes of the corresponding substream. Each substream's contents will133be described in detail :ref:`below <dbi_substreams>`. The length of the entire134DBI Stream should equal ``64`` (the length of the header above) plus the value135of each of the following ``7`` fields.136 137- **ModInfoSize** - The length of the :ref:`dbi_mod_info_substream`.138 139- **SectionContributionSize** - The length of the :ref:`dbi_sec_contr_substream`.140 141- **SectionMapSize** - The length of the :ref:`dbi_section_map_substream`.142 143- **SourceInfoSize** - The length of the :ref:`dbi_file_info_substream`.144 145- **TypeServerMapSize** - The length of the :ref:`dbi_type_server_map_substream`.146 147- **OptionalDbgHeaderSize** - The length of the :ref:`dbi_optional_dbg_stream`.148 149- **ECSubstreamSize** - The length of the :ref:`dbi_ec_substream`.150 151.. _dbi_substreams:152 153Substreams154==========155 156.. _dbi_mod_info_substream:157 158Module Info Substream159^^^^^^^^^^^^^^^^^^^^^160 161Begins at offset ``0`` immediately after the :ref:`header <dbi_header>`. The162module info substream is an array of variable-length records, each one163describing a single module (e.g. object file) linked into the program. Each164record in the array has the format:165 166.. code-block:: c++167 168 struct ModInfo {169 uint32_t Unused1;170 struct SectionContribEntry {171 uint16_t Section;172 char Padding1[2];173 int32_t Offset;174 int32_t Size;175 uint32_t Characteristics;176 uint16_t ModuleIndex;177 char Padding2[2];178 uint32_t DataCrc;179 uint32_t RelocCrc;180 } SectionContr;181 uint16_t Flags;182 uint16_t ModuleSymStream;183 uint32_t SymByteSize;184 uint32_t C11ByteSize;185 uint32_t C13ByteSize;186 uint16_t SourceFileCount;187 char Padding[2];188 uint32_t Unused2;189 uint32_t SourceFileNameIndex;190 uint32_t PdbFilePathNameIndex;191 char ModuleName[];192 char ObjFileName[];193 };194 195- **SectionContr** - Describes the properties of the section in the final binary196 which contain the code and data from this module.197 198 ``SectionContr.Characteristics`` corresponds to the ``Characteristics`` field199 of the `IMAGE_SECTION_HEADER <https://msdn.microsoft.com/en-us/library/windows/desktop/ms680341(v=vs.85).aspx>`__200 structure.201 202 203- **Flags** - A bitfield with the following format:204 205.. code-block:: c++206 207 // ``true`` if this ModInfo has been written since reading the PDB. This is208 // likely used to support incremental linking, so that the linker can decide209 // if it needs to commit changes to disk.210 uint16_t Dirty : 1;211 // ``true`` if EC information is present for this module. EC is presumed to212 // stand for "Edit & Continue", which LLVM does not support. So this flag213 // will always be false.214 uint16_t EC : 1;215 uint16_t Unused : 6;216 // Type Server Index for this module. This is assumed to be related to /Zi,217 // but as LLVM treats /Zi as /Z7, this field will always be invalid for LLVM218 // generated PDBs.219 uint16_t TSM : 8;220 221 222- **ModuleSymStream** - The index of the stream that contains symbol information223 for this module. This includes CodeView symbol information as well as source224 and line information. If this field is -1, then no additional debug info will225 be present for this module (for example, this is what happens when you strip226 private symbols from a PDB).227 228- **SymByteSize** - The number of bytes of data from the stream identified by229 ``ModuleSymStream`` that represent CodeView symbol records.230 231- **C11ByteSize** - The number of bytes of data from the stream identified by232 ``ModuleSymStream`` that represent C11-style CodeView line information.233 234- **C13ByteSize** - The number of bytes of data from the stream identified by235 ``ModuleSymStream`` that represent C13-style CodeView line information. At236 most one of ``C11ByteSize`` and ``C13ByteSize`` will be non-zero. Modern PDBs237 always use C13 instead of C11.238 239- **SourceFileCount** - The number of source files that contributed to this240 module during compilation.241 242- **SourceFileNameIndex** - The offset in the names buffer of the primary243 translation unit used to build this module. All PDB files observed to date244 always have this value equal to 0.245 246- **PdbFilePathNameIndex** - The offset in the names buffer of the PDB file247 containing this module's symbol information. This has only been observed248 to be non-zero for the special ``* Linker *`` module.249 250- **ModuleName** - The module name. This is usually either a full path to an251 object file (either directly passed to ``link.exe`` or from an archive) or252 a string of the form ``Import:<dll name>``.253 254- **ObjFileName** - The object file name. In the case of an module that is255 linked directly passed to ``link.exe``, this is the same as **ModuleName**.256 In the case of a module that comes from an archive, this is usually the full257 path to the archive.258 259.. _dbi_sec_contr_substream:260 261Section Contribution Substream262^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^263Begins at offset ``0`` immediately after the :ref:`dbi_mod_info_substream` ends,264and consumes ``Header->SectionContributionSize`` bytes. This substream begins265with a single ``uint32_t`` which will be one of the following values:266 267.. code-block:: c++268 269 enum class SectionContrSubstreamVersion : uint32_t {270 Ver60 = 0xeffe0000 + 19970605,271 V2 = 0xeffe0000 + 20140516272 };273 274``Ver60`` is the only value which has been observed in a PDB so far. Following275this is an array of fixed-length structures. If the version is ``Ver60``,276it is an array of ``SectionContribEntry`` structures (this is the nested structure277from the ``ModInfo`` type. If the version is ``V2``, it is an array of278``SectionContribEntry2`` structures, defined as follows:279 280.. code-block:: c++281 282 struct SectionContribEntry2 {283 SectionContribEntry SC;284 uint32_t ISectCoff;285 };286 287The purpose of the second field is not well understood. The name implies that288is the index of the COFF section, but this also describes the existing field289``SectionContribEntry::Section``.290 291 292.. _dbi_section_map_substream:293 294Section Map Substream295^^^^^^^^^^^^^^^^^^^^^296Begins at offset ``0`` immediately after the :ref:`dbi_sec_contr_substream` ends,297and consumes ``Header->SectionMapSize`` bytes. This substream begins with an ``4``298byte header followed by an array of fixed-length records. The header and records299have the following layout:300 301.. code-block:: c++302 303 struct SectionMapHeader {304 uint16_t Count; // Number of segment descriptors305 uint16_t LogCount; // Number of logical segment descriptors306 };307 308 struct SectionMapEntry {309 uint16_t Flags; // See the SectionMapEntryFlags enum below.310 uint16_t Ovl; // Logical overlay number311 uint16_t Group; // Group index into descriptor array.312 uint16_t Frame;313 uint16_t SectionName; // Byte index of segment / group name in string table, or 0xFFFF.314 uint16_t ClassName; // Byte index of class in string table, or 0xFFFF.315 uint32_t Offset; // Byte offset of the logical segment within physical segment. If group is set in flags, this is the offset of the group.316 uint32_t SectionLength; // Byte count of the segment or group.317 };318 319 enum class SectionMapEntryFlags : uint16_t {320 Read = 1 << 0, // Segment is readable.321 Write = 1 << 1, // Segment is writable.322 Execute = 1 << 2, // Segment is executable.323 AddressIs32Bit = 1 << 3, // Descriptor describes a 32-bit linear address.324 IsSelector = 1 << 8, // Frame represents a selector.325 IsAbsoluteAddress = 1 << 9, // Frame represents an absolute address.326 IsGroup = 1 << 10 // If set, descriptor represents a group.327 };328 329Many of these fields are not well understood, so will not be discussed further.330 331.. _dbi_file_info_substream:332 333File Info Substream334^^^^^^^^^^^^^^^^^^^335Begins at offset ``0`` immediately after the :ref:`dbi_section_map_substream` ends,336and consumes ``Header->SourceInfoSize`` bytes. This substream defines the mapping337from module to the source files that contribute to that module. Since multiple338modules can use the same source file (for example, a header file), this substream339uses a string table to store each unique file name only once, and then have each340module use offsets into the string table rather than embedding the string's value341directly. The format of this substream is as follows:342 343.. code-block:: c++344 345 struct FileInfoSubstream {346 uint16_t NumModules;347 uint16_t NumSourceFiles;348 349 uint16_t ModIndices[NumModules];350 uint16_t ModFileCounts[NumModules];351 uint32_t FileNameOffsets[NumSourceFiles];352 char NamesBuffer[][NumSourceFiles];353 };354 355**NumModules** - The number of modules for which source file information is356contained within this substream. Should match the corresponding value from the357ref:`dbi_header`.358 359**NumSourceFiles**: In theory this is supposed to contain the number of source360files for which this substream contains information. But that would present a361problem in that the width of this field being ``16``-bits would prevent one from362having more than 64K source files in a program. In early versions of the file363format, this seems to have been the case. In order to support more than this, this364field of the is simply ignored, and computed dynamically by summing up the values of365the ``ModFileCounts`` array (discussed below). In short, this value should be366ignored.367 368**ModIndices** - This array is present, but does not appear to be useful.369 370**ModFileCountArray** - An array of ``NumModules`` integers, each one containing371the number of source files which contribute to the module at the specified index.372While each individual module is limited to 64K contributing source files, the373union of all modules' source files may be greater than 64K. The real number of374source files is thus computed by summing this array. Note that summing this array375does not give the number of `unique` source files, only the total number of source376file contributions to modules.377 378**FileNameOffsets** - An array of **NumSourceFiles** integers (where **NumSourceFiles**379here refers to the 32-bit value obtained from summing **ModFileCountArray**), where380each integer is an offset into **NamesBuffer** pointing to a null terminated string.381 382**NamesBuffer** - An array of null terminated strings containing the actual source383file names.384 385.. _dbi_type_server_map_substream:386 387Type Server Map Substream388^^^^^^^^^^^^^^^^^^^^^^^^^389Begins at offset ``0`` immediately after the :ref:`dbi_file_info_substream`390ends, and consumes ``Header->TypeServerMapSize`` bytes. Neither the purpose391nor the layout of this substream is understood, although it is assumed to392related somehow to the usage of ``/Zi`` and ``mspdbsrv.exe``. This substream393will not be discussed further.394 395.. _dbi_ec_substream:396 397EC Substream398^^^^^^^^^^^^399Begins at offset ``0`` immediately after the400:ref:`dbi_type_server_map_substream` ends, and consumes401``Header->ECSubstreamSize`` bytes. This is presumed to be related to Edit &402Continue support in MSVC. LLVM does not support Edit & Continue, so this403stream will not be discussed further.404 405.. _dbi_optional_dbg_stream:406 407Optional Debug Header Stream408^^^^^^^^^^^^^^^^^^^^^^^^^^^^409Begins at offset ``0`` immediately after the :ref:`dbi_ec_substream` ends, and410consumes ``Header->OptionalDbgHeaderSize`` bytes. This field is an array of411stream indices (e.g. ``uint16_t``'s), each of which identifies a stream412index in the larger MSF file which contains some additional debug information.413Each position of this array has a special meaning, allowing one to determine414what kind of debug information is at the referenced stream. ``11`` indices415are currently understood, although it's possible there may be more. The416layout of each stream generally corresponds exactly to a particular type417of debug data directory from the PE/COFF file. The format of these fields418can be found in the `Microsoft PE/COFF Specification <https://www.microsoft.com/en-us/download/details.aspx?id=19509>`__.419If any of these fields is -1, it means the corresponding type of debug info is420not present in the PDB.421 422**FPO Data** - ``DbgStreamArray[0]``. The data in the referenced stream is an423array of ``FPO_DATA`` structures. This contains the relocated contents of424any ``.debug$F`` section from any of the linker inputs.425 426**Exception Data** - ``DbgStreamArray[1]``. The data in the referenced stream427is a debug data directory of type ``IMAGE_DEBUG_TYPE_EXCEPTION``.428 429**Fixup Data** - ``DbgStreamArray[2]``. The data in the referenced stream is a430debug data directory of type ``IMAGE_DEBUG_TYPE_FIXUP``.431 432**Omap To Src Data** - ``DbgStreamArray[3]``. The data in the referenced stream433is a debug data directory of type ``IMAGE_DEBUG_TYPE_OMAP_TO_SRC``. This434is used for mapping addresses between instrumented and uninstrumented code.435 436**Omap From Src Data** - ``DbgStreamArray[4]``. The data in the referenced stream437is a debug data directory of type ``IMAGE_DEBUG_TYPE_OMAP_FROM_SRC``. This438is used for mapping addresses between instrumented and uninstrumented code.439 440**Section Header Data** - ``DbgStreamArray[5]``. A dump of all section headers from441the original executable.442 443**Token / RID Map** - ``DbgStreamArray[6]``. The layout of this stream is not444understood, but it is assumed to be a mapping from ``CLR Token`` to445``CLR Record ID``. Refer to `ECMA 335 <http://www.ecma-international.org/publications/standards/Ecma-335.htm>`__446for more information.447 448**Xdata** - ``DbgStreamArray[7]``. A copy of the ``.xdata`` section from the449executable.450 451**Pdata** - ``DbgStreamArray[8]``. This is assumed to be a copy of the ``.pdata``452section from the executable, but that would make it identical to453``DbgStreamArray[1]``. The difference between these two indices is not well454understood.455 456**New FPO Data** - ``DbgStreamArray[9]``. The data in the referenced stream is a457debug data directory of type ``IMAGE_DEBUG_TYPE_FPO``. Note that this is different458from ``DbgStreamArray[0]`` in that ``.debug$F`` sections are only emitted by MASM.459Thus, it is possible for both to appear in the same PDB if both MASM object files460and cl object files are linked into the same program.461 462**Original Section Header Data** - ``DbgStreamArray[10]``. Similar to463``DbgStreamArray[5]``, but contains the section headers before any binary translation464has been performed. This can be used in conjunction with ``DebugStreamArray[3]``465and ``DbgStreamArray[4]`` to map instrumented and uninstrumented addresses.466