762 lines · plain
1.. SPDX-License-Identifier: GPL-2.02 3Journal (jbd2)4--------------5 6Introduced in ext3, the ext4 filesystem employs a journal to protect the7filesystem against metadata inconsistencies in the case of a system crash. Up8to 10,240,000 file system blocks (see man mke2fs(8) for more details on journal9size limits) can be reserved inside the filesystem as a place to land10“important” data writes on-disk as quickly as possible. Once the important11data transaction is fully written to the disk and flushed from the disk write12cache, a record of the data being committed is also written to the journal. At13some later point in time, the journal code writes the transactions to their14final locations on disk (this could involve a lot of seeking or a lot of small15read-write-erases) before erasing the commit record. Should the system16crash during the second slow write, the journal can be replayed all the17way to the latest commit record, guaranteeing the atomicity of whatever18gets written through the journal to the disk. The effect of this is to19guarantee that the filesystem does not become stuck midway through a20metadata update.21 22For performance reasons, ext4 by default only writes filesystem metadata23through the journal. This means that file data blocks are /not/24guaranteed to be in any consistent state after a crash. If this default25guarantee level (``data=ordered``) is not satisfactory, there is a mount26option to control journal behavior. If ``data=journal``, all data and27metadata are written to disk through the journal. This is slower but28safest. If ``data=writeback``, dirty data blocks are not flushed to the29disk before the metadata are written to disk through the journal.30 31In case of ``data=ordered`` mode, Ext4 also supports fast commits which32help reduce commit latency significantly. The default ``data=ordered``33mode works by logging metadata blocks to the journal. In fast commit34mode, Ext4 only stores the minimal delta needed to recreate the35affected metadata in fast commit space that is shared with JBD2.36Once the fast commit area fills in or if fast commit is not possible37or if JBD2 commit timer goes off, Ext4 performs a traditional full commit.38A full commit invalidates all the fast commits that happened before39it and thus it makes the fast commit area empty for further fast40commits. This feature needs to be enabled at mkfs time.41 42The journal inode is typically inode 8. The first 68 bytes of the43journal inode are replicated in the ext4 superblock. The journal itself44is normal (but hidden) file within the filesystem. The file usually45consumes an entire block group, though mke2fs tries to put it in the46middle of the disk.47 48All fields in jbd2 are written to disk in big-endian order. This is the49opposite of ext4.50 51NOTE: Both ext4 and ocfs2 use jbd2.52 53The maximum size of a journal embedded in an ext4 filesystem is 2^3254blocks. jbd2 itself does not seem to care.55 56Layout57~~~~~~58 59Generally speaking, the journal has this format:60 61.. list-table::62 :widths: 16 48 1663 :header-rows: 164 65 * - Superblock66 - descriptor_block (data_blocks or revocation_block) [more data or67 revocations] commmit_block68 - [more transactions...]69 * - 70 - One transaction71 -72 73Notice that a transaction begins with either a descriptor and some data,74or a block revocation list. A finished transaction always ends with a75commit. If there is no commit record (or the checksums don't match), the76transaction will be discarded during replay.77 78External Journal79~~~~~~~~~~~~~~~~80 81Optionally, an ext4 filesystem can be created with an external journal82device (as opposed to an internal journal, which uses a reserved inode).83In this case, on the filesystem device, ``s_journal_inum`` should be84zero and ``s_journal_uuid`` should be set. On the journal device there85will be an ext4 super block in the usual place, with a matching UUID.86The journal superblock will be in the next full block after the87superblock.88 89.. list-table::90 :widths: 12 12 12 32 1291 :header-rows: 192 93 * - 1024 bytes of padding94 - ext4 Superblock95 - Journal Superblock96 - descriptor_block (data_blocks or revocation_block) [more data or97 revocations] commmit_block98 - [more transactions...]99 * - 100 -101 -102 - One transaction103 -104 105Block Header106~~~~~~~~~~~~107 108Every block in the journal starts with a common 12-byte header109``struct journal_header_s``:110 111.. list-table::112 :widths: 8 8 24 40113 :header-rows: 1114 115 * - Offset116 - Type117 - Name118 - Description119 * - 0x0120 - __be32121 - h_magic122 - jbd2 magic number, 0xC03B3998.123 * - 0x4124 - __be32125 - h_blocktype126 - Description of what this block contains. See the jbd2_blocktype_ table127 below.128 * - 0x8129 - __be32130 - h_sequence131 - The transaction ID that goes with this block.132 133.. _jbd2_blocktype:134 135The journal block type can be any one of:136 137.. list-table::138 :widths: 16 64139 :header-rows: 1140 141 * - Value142 - Description143 * - 1144 - Descriptor. This block precedes a series of data blocks that were145 written through the journal during a transaction.146 * - 2147 - Block commit record. This block signifies the completion of a148 transaction.149 * - 3150 - Journal superblock, v1.151 * - 4152 - Journal superblock, v2.153 * - 5154 - Block revocation records. This speeds up recovery by enabling the155 journal to skip writing blocks that were subsequently rewritten.156 157Super Block158~~~~~~~~~~~159 160The super block for the journal is much simpler as compared to ext4's.161The key data kept within are size of the journal, and where to find the162start of the log of transactions.163 164The journal superblock is recorded as ``struct journal_superblock_s``,165which is 1024 bytes long:166 167.. list-table::168 :widths: 8 8 24 40169 :header-rows: 1170 171 * - Offset172 - Type173 - Name174 - Description175 * -176 -177 -178 - Static information describing the journal.179 * - 0x0180 - journal_header_t (12 bytes)181 - s_header182 - Common header identifying this as a superblock.183 * - 0xC184 - __be32185 - s_blocksize186 - Journal device block size.187 * - 0x10188 - __be32189 - s_maxlen190 - Total number of blocks in this journal.191 * - 0x14192 - __be32193 - s_first194 - First block of log information.195 * -196 -197 -198 - Dynamic information describing the current state of the log.199 * - 0x18200 - __be32201 - s_sequence202 - First commit ID expected in log.203 * - 0x1C204 - __be32205 - s_start206 - Block number of the start of log. Contrary to the comments, this field207 being zero does not imply that the journal is clean!208 * - 0x20209 - __be32210 - s_errno211 - Error value, as set by jbd2_journal_abort().212 * -213 -214 -215 - The remaining fields are only valid in a v2 superblock.216 * - 0x24217 - __be32218 - s_feature_compat;219 - Compatible feature set. See the table jbd2_compat_ below.220 * - 0x28221 - __be32222 - s_feature_incompat223 - Incompatible feature set. See the table jbd2_incompat_ below.224 * - 0x2C225 - __be32226 - s_feature_ro_compat227 - Read-only compatible feature set. There aren't any of these currently.228 * - 0x30229 - __u8230 - s_uuid[16]231 - 128-bit uuid for journal. This is compared against the copy in the ext4232 super block at mount time.233 * - 0x40234 - __be32235 - s_nr_users236 - Number of file systems sharing this journal.237 * - 0x44238 - __be32239 - s_dynsuper240 - Location of dynamic super block copy. (Not used?)241 * - 0x48242 - __be32243 - s_max_transaction244 - Limit of journal blocks per transaction. (Not used?)245 * - 0x4C246 - __be32247 - s_max_trans_data248 - Limit of data blocks per transaction. (Not used?)249 * - 0x50250 - __u8251 - s_checksum_type252 - Checksum algorithm used for the journal. See jbd2_checksum_type_ for253 more info.254 * - 0x51255 - __u8[3]256 - s_padding2257 -258 * - 0x54259 - __be32260 - s_num_fc_blocks261 - Number of fast commit blocks in the journal.262 * - 0x58263 - __be32264 - s_head265 - Block number of the head (first unused block) of the journal, only266 up-to-date when the journal is empty.267 * - 0x5C268 - __u32269 - s_padding[40]270 -271 * - 0xFC272 - __be32273 - s_checksum274 - Checksum of the entire superblock, with this field set to zero.275 * - 0x100276 - __u8277 - s_users[16*48]278 - ids of all file systems sharing the log. e2fsprogs/Linux don't allow279 shared external journals, but I imagine Lustre (or ocfs2?), which use280 the jbd2 code, might.281 282.. _jbd2_compat:283 284The journal compat features are any combination of the following:285 286.. list-table::287 :widths: 16 64288 :header-rows: 1289 290 * - Value291 - Description292 * - 0x1293 - Journal maintains checksums on the data blocks.294 (JBD2_FEATURE_COMPAT_CHECKSUM)295 296.. _jbd2_incompat:297 298The journal incompat features are any combination of the following:299 300.. list-table::301 :widths: 16 64302 :header-rows: 1303 304 * - Value305 - Description306 * - 0x1307 - Journal has block revocation records. (JBD2_FEATURE_INCOMPAT_REVOKE)308 * - 0x2309 - Journal can deal with 64-bit block numbers.310 (JBD2_FEATURE_INCOMPAT_64BIT)311 * - 0x4312 - Journal commits asynchronously. (JBD2_FEATURE_INCOMPAT_ASYNC_COMMIT)313 * - 0x8314 - This journal uses v2 of the checksum on-disk format. Each journal315 metadata block gets its own checksum, and the block tags in the316 descriptor table contain checksums for each of the data blocks in the317 journal. (JBD2_FEATURE_INCOMPAT_CSUM_V2)318 * - 0x10319 - This journal uses v3 of the checksum on-disk format. This is the same as320 v2, but the journal block tag size is fixed regardless of the size of321 block numbers. (JBD2_FEATURE_INCOMPAT_CSUM_V3)322 * - 0x20323 - Journal has fast commit blocks. (JBD2_FEATURE_INCOMPAT_FAST_COMMIT)324 325.. _jbd2_checksum_type:326 327Journal checksum type codes are one of the following. crc32 or crc32c are the328most likely choices.329 330.. list-table::331 :widths: 16 64332 :header-rows: 1333 334 * - Value335 - Description336 * - 1337 - CRC32338 * - 2339 - MD5340 * - 3341 - SHA1342 * - 4343 - CRC32C344 345Descriptor Block346~~~~~~~~~~~~~~~~347 348The descriptor block contains an array of journal block tags that349describe the final locations of the data blocks that follow in the350journal. Descriptor blocks are open-coded instead of being completely351described by a data structure, but here is the block structure anyway.352Descriptor blocks consume at least 36 bytes, but use a full block:353 354.. list-table::355 :widths: 8 8 24 40356 :header-rows: 1357 358 * - Offset359 - Type360 - Name361 - Descriptor362 * - 0x0363 - journal_header_t364 - (open coded)365 - Common block header.366 * - 0xC367 - struct journal_block_tag_s368 - open coded array[]369 - Enough tags either to fill up the block or to describe all the data370 blocks that follow this descriptor block.371 372Journal block tags have any of the following formats, depending on which373journal feature and block tag flags are set.374 375If JBD2_FEATURE_INCOMPAT_CSUM_V3 is set, the journal block tag is376defined as ``struct journal_block_tag3_s``, which looks like the377following. The size is 16 or 32 bytes.378 379.. list-table::380 :widths: 8 8 24 40381 :header-rows: 1382 383 * - Offset384 - Type385 - Name386 - Descriptor387 * - 0x0388 - __be32389 - t_blocknr390 - Lower 32-bits of the location of where the corresponding data block391 should end up on disk.392 * - 0x4393 - __be32394 - t_flags395 - Flags that go with the descriptor. See the table jbd2_tag_flags_ for396 more info.397 * - 0x8398 - __be32399 - t_blocknr_high400 - Upper 32-bits of the location of where the corresponding data block401 should end up on disk. This is zero if JBD2_FEATURE_INCOMPAT_64BIT is402 not enabled.403 * - 0xC404 - __be32405 - t_checksum406 - Checksum of the journal UUID, the sequence number, and the data block.407 * -408 -409 -410 - This field appears to be open coded. It always comes at the end of the411 tag, after t_checksum. This field is not present if the "same UUID" flag412 is set.413 * - 0x8 or 0xC414 - char415 - uuid[16]416 - A UUID to go with this tag. This field appears to be copied from the417 ``j_uuid`` field in ``struct journal_s``, but only tune2fs touches that418 field.419 420.. _jbd2_tag_flags:421 422The journal tag flags are any combination of the following:423 424.. list-table::425 :widths: 16 64426 :header-rows: 1427 428 * - Value429 - Description430 * - 0x1431 - On-disk block is escaped. The first four bytes of the data block just432 happened to match the jbd2 magic number.433 * - 0x2434 - This block has the same UUID as previous, therefore the UUID field is435 omitted.436 * - 0x4437 - The data block was deleted by the transaction. (Not used?)438 * - 0x8439 - This is the last tag in this descriptor block.440 441If JBD2_FEATURE_INCOMPAT_CSUM_V3 is NOT set, the journal block tag442is defined as ``struct journal_block_tag_s``, which looks like the443following. The size is 8, 12, 24, or 28 bytes:444 445.. list-table::446 :widths: 8 8 24 40447 :header-rows: 1448 449 * - Offset450 - Type451 - Name452 - Descriptor453 * - 0x0454 - __be32455 - t_blocknr456 - Lower 32-bits of the location of where the corresponding data block457 should end up on disk.458 * - 0x4459 - __be16460 - t_checksum461 - Checksum of the journal UUID, the sequence number, and the data block.462 Note that only the lower 16 bits are stored.463 * - 0x6464 - __be16465 - t_flags466 - Flags that go with the descriptor. See the table jbd2_tag_flags_ for467 more info.468 * -469 -470 -471 - This next field is only present if the super block indicates support for472 64-bit block numbers.473 * - 0x8474 - __be32475 - t_blocknr_high476 - Upper 32-bits of the location of where the corresponding data block477 should end up on disk.478 * -479 -480 -481 - This field appears to be open coded. It always comes at the end of the482 tag, after t_flags or t_blocknr_high. This field is not present if the483 "same UUID" flag is set.484 * - 0x8 or 0xC485 - char486 - uuid[16]487 - A UUID to go with this tag. This field appears to be copied from the488 ``j_uuid`` field in ``struct journal_s``, but only tune2fs touches that489 field.490 491If JBD2_FEATURE_INCOMPAT_CSUM_V2 or492JBD2_FEATURE_INCOMPAT_CSUM_V3 are set, the end of the block is a493``struct jbd2_journal_block_tail``, which looks like this:494 495.. list-table::496 :widths: 8 8 24 40497 :header-rows: 1498 499 * - Offset500 - Type501 - Name502 - Descriptor503 * - 0x0504 - __be32505 - t_checksum506 - Checksum of the journal UUID + the descriptor block, with this field set507 to zero.508 509Data Block510~~~~~~~~~~511 512In general, the data blocks being written to disk through the journal513are written verbatim into the journal file after the descriptor block.514However, if the first four bytes of the block match the jbd2 magic515number then those four bytes are replaced with zeroes and the “escaped”516flag is set in the descriptor block tag.517 518Revocation Block519~~~~~~~~~~~~~~~~520 521A revocation block is used to prevent replay of a block in an earlier522transaction. This is used to mark blocks that were journalled at one523time but are no longer journalled. Typically this happens if a metadata524block is freed and re-allocated as a file data block; in this case, a525journal replay after the file block was written to disk will cause526corruption.527 528**NOTE**: This mechanism is NOT used to express “this journal block is529superseded by this other journal block”, as the author (djwong)530mistakenly thought. Any block being added to a transaction will cause531the removal of all existing revocation records for that block.532 533Revocation blocks are described in534``struct jbd2_journal_revoke_header_s``, are at least 16 bytes in535length, but use a full block:536 537.. list-table::538 :widths: 8 8 24 40539 :header-rows: 1540 541 * - Offset542 - Type543 - Name544 - Description545 * - 0x0546 - journal_header_t547 - r_header548 - Common block header.549 * - 0xC550 - __be32551 - r_count552 - Number of bytes used in this block.553 * - 0x10554 - __be32 or __be64555 - blocks[0]556 - Blocks to revoke.557 558After r_count is a linear array of block numbers that are effectively559revoked by this transaction. The size of each block number is 8 bytes if560the superblock advertises 64-bit block number support, or 4 bytes561otherwise.562 563If JBD2_FEATURE_INCOMPAT_CSUM_V2 or564JBD2_FEATURE_INCOMPAT_CSUM_V3 are set, the end of the revocation565block is a ``struct jbd2_journal_revoke_tail``, which has this format:566 567.. list-table::568 :widths: 8 8 24 40569 :header-rows: 1570 571 * - Offset572 - Type573 - Name574 - Description575 * - 0x0576 - __be32577 - r_checksum578 - Checksum of the journal UUID + revocation block579 580Commit Block581~~~~~~~~~~~~582 583The commit block is a sentry that indicates that a transaction has been584completely written to the journal. Once this commit block reaches the585journal, the data stored with this transaction can be written to their586final locations on disk.587 588The commit block is described by ``struct commit_header``, which is 32589bytes long (but uses a full block):590 591.. list-table::592 :widths: 8 8 24 40593 :header-rows: 1594 595 * - Offset596 - Type597 - Name598 - Descriptor599 * - 0x0600 - journal_header_s601 - (open coded)602 - Common block header.603 * - 0xC604 - unsigned char605 - h_chksum_type606 - The type of checksum to use to verify the integrity of the data blocks607 in the transaction. See jbd2_checksum_type_ for more info.608 * - 0xD609 - unsigned char610 - h_chksum_size611 - The number of bytes used by the checksum. Most likely 4.612 * - 0xE613 - unsigned char614 - h_padding[2]615 -616 * - 0x10617 - __be32618 - h_chksum[JBD2_CHECKSUM_BYTES]619 - 32 bytes of space to store checksums. If620 JBD2_FEATURE_INCOMPAT_CSUM_V2 or JBD2_FEATURE_INCOMPAT_CSUM_V3621 are set, the first ``__be32`` is the checksum of the journal UUID and622 the entire commit block, with this field zeroed. If623 JBD2_FEATURE_COMPAT_CHECKSUM is set, the first ``__be32`` is the624 crc32 of all the blocks already written to the transaction.625 * - 0x30626 - __be64627 - h_commit_sec628 - The time that the transaction was committed, in seconds since the epoch.629 * - 0x38630 - __be32631 - h_commit_nsec632 - Nanoseconds component of the above timestamp.633 634Fast commits635~~~~~~~~~~~~636 637Fast commit area is organized as a log of tag length values. Each TLV has638a ``struct ext4_fc_tl`` in the beginning which stores the tag and the length639of the entire field. It is followed by variable length tag specific value.640Here is the list of supported tags and their meanings:641 642.. list-table::643 :widths: 8 20 20 32644 :header-rows: 1645 646 * - Tag647 - Meaning648 - Value struct649 - Description650 * - EXT4_FC_TAG_HEAD651 - Fast commit area header652 - ``struct ext4_fc_head``653 - Stores the TID of the transaction after which these fast commits should654 be applied.655 * - EXT4_FC_TAG_ADD_RANGE656 - Add extent to inode657 - ``struct ext4_fc_add_range``658 - Stores the inode number and extent to be added in this inode659 * - EXT4_FC_TAG_DEL_RANGE660 - Remove logical offsets to inode661 - ``struct ext4_fc_del_range``662 - Stores the inode number and the logical offset range that needs to be663 removed664 * - EXT4_FC_TAG_CREAT665 - Create directory entry for a newly created file666 - ``struct ext4_fc_dentry_info``667 - Stores the parent inode number, inode number and directory entry of the668 newly created file669 * - EXT4_FC_TAG_LINK670 - Link a directory entry to an inode671 - ``struct ext4_fc_dentry_info``672 - Stores the parent inode number, inode number and directory entry673 * - EXT4_FC_TAG_UNLINK674 - Unlink a directory entry of an inode675 - ``struct ext4_fc_dentry_info``676 - Stores the parent inode number, inode number and directory entry677 678 * - EXT4_FC_TAG_PAD679 - Padding (unused area)680 - None681 - Unused bytes in the fast commit area.682 683 * - EXT4_FC_TAG_TAIL684 - Mark the end of a fast commit685 - ``struct ext4_fc_tail``686 - Stores the TID of the commit, CRC of the fast commit of which this tag687 represents the end of688 689Fast Commit Replay Idempotence690~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~691 692Fast commits tags are idempotent in nature provided the recovery code follows693certain rules. The guiding principle that the commit path follows while694committing is that it stores the result of a particular operation instead of695storing the procedure.696 697Let's consider this rename operation: 'mv /a /b'. Let's assume dirent '/a'698was associated with inode 10. During fast commit, instead of storing this699operation as a procedure "rename a to b", we store the resulting file system700state as a "series" of outcomes:701 702- Link dirent b to inode 10703- Unlink dirent a704- Inode 10 with valid refcount705 706Now when recovery code runs, it needs "enforce" this state on the file707system. This is what guarantees idempotence of fast commit replay.708 709Let's take an example of a procedure that is not idempotent and see how fast710commits make it idempotent. Consider following sequence of operations:711 7121) rm A7132) mv B A7143) read A715 716If we store this sequence of operations as is then the replay is not idempotent.717Let's say while in replay, we crash after (2). During the second replay,718file A (which was actually created as a result of "mv B A" operation) would get719deleted. Thus, file named A would be absent when we try to read A. So, this720sequence of operations is not idempotent. However, as mentioned above, instead721of storing the procedure fast commits store the outcome of each procedure. Thus722the fast commit log for above procedure would be as follows:723 724(Let's assume dirent A was linked to inode 10 and dirent B was linked to725inode 11 before the replay)726 7271) Unlink A7282) Link A to inode 117293) Unlink B7304) Inode 11731 732If we crash after (3) we will have file A linked to inode 11. During the second733replay, we will remove file A (inode 11). But we will create it back and make734it point to inode 11. We won't find B, so we'll just skip that step. At this735point, the refcount for inode 11 is not reliable, but that gets fixed by the736replay of last inode 11 tag. Thus, by converting a non-idempotent procedure737into a series of idempotent outcomes, fast commits ensured idempotence during738the replay.739 740Journal Checkpoint741~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~742 743Checkpointing the journal ensures all transactions and their associated buffers744are submitted to the disk. In-progress transactions are waited upon and included745in the checkpoint. Checkpointing is used internally during critical updates to746the filesystem including journal recovery, filesystem resizing, and freeing of747the journal_t structure.748 749A journal checkpoint can be triggered from userspace via the ioctl750EXT4_IOC_CHECKPOINT. This ioctl takes a single, u64 argument for flags.751Currently, three flags are supported. First, EXT4_IOC_CHECKPOINT_FLAG_DRY_RUN752can be used to verify input to the ioctl. It returns error if there is any753invalid input, otherwise it returns success without performing754any checkpointing. This can be used to check whether the ioctl exists on a755system and to verify there are no issues with arguments or flags. The756other two flags are EXT4_IOC_CHECKPOINT_FLAG_DISCARD and757EXT4_IOC_CHECKPOINT_FLAG_ZEROOUT. These flags cause the journal blocks to be758discarded or zero-filled, respectively, after the journal checkpoint is759complete. EXT4_IOC_CHECKPOINT_FLAG_DISCARD and EXT4_IOC_CHECKPOINT_FLAG_ZEROOUT760cannot both be set. The ioctl may be useful when snapshotting a system or for761complying with content deletion SLOs.762