152 lines · plain
1======================================2Immutable biovecs and biovec iterators3======================================4 5Kent Overstreet <kmo@daterainc.com>6 7As of 3.13, biovecs should never be modified after a bio has been submitted.8Instead, we have a new struct bvec_iter which represents a range of a biovec -9the iterator will be modified as the bio is completed, not the biovec.10 11More specifically, old code that needed to partially complete a bio would12update bi_sector and bi_size, and advance bi_idx to the next biovec. If it13ended up partway through a biovec, it would increment bv_offset and decrement14bv_len by the number of bytes completed in that biovec.15 16In the new scheme of things, everything that must be mutated in order to17partially complete a bio is segregated into struct bvec_iter: bi_sector,18bi_size and bi_idx have been moved there; and instead of modifying bv_offset19and bv_len, struct bvec_iter has bi_bvec_done, which represents the number of20bytes completed in the current bvec.21 22There are a bunch of new helper macros for hiding the gory details - in23particular, presenting the illusion of partially completed biovecs so that24normal code doesn't have to deal with bi_bvec_done.25 26 * Driver code should no longer refer to biovecs directly; we now have27 bio_iovec() and bio_iter_iovec() macros that return literal struct biovecs,28 constructed from the raw biovecs but taking into account bi_bvec_done and29 bi_size.30 31 bio_for_each_segment() has been updated to take a bvec_iter argument32 instead of an integer (that corresponded to bi_idx); for a lot of code the33 conversion just required changing the types of the arguments to34 bio_for_each_segment().35 36 * Advancing a bvec_iter is done with bio_advance_iter(); bio_advance() is a37 wrapper around bio_advance_iter() that operates on bio->bi_iter, and also38 advances the bio integrity's iter if present.39 40 There is a lower level advance function - bvec_iter_advance() - which takes41 a pointer to a biovec, not a bio; this is used by the bio integrity code.42 43As of 5.12 bvec segments with zero bv_len are not supported.44 45What's all this get us?46=======================47 48Having a real iterator, and making biovecs immutable, has a number of49advantages:50 51 * Before, iterating over bios was very awkward when you weren't processing52 exactly one bvec at a time - for example, bio_copy_data() in block/bio.c,53 which copies the contents of one bio into another. Because the biovecs54 wouldn't necessarily be the same size, the old code was tricky convoluted -55 it had to walk two different bios at the same time, keeping both bi_idx and56 and offset into the current biovec for each.57 58 The new code is much more straightforward - have a look. This sort of59 pattern comes up in a lot of places; a lot of drivers were essentially open60 coding bvec iterators before, and having common implementation considerably61 simplifies a lot of code.62 63 * Before, any code that might need to use the biovec after the bio had been64 completed (perhaps to copy the data somewhere else, or perhaps to resubmit65 it somewhere else if there was an error) had to save the entire bvec array66 - again, this was being done in a fair number of places.67 68 * Biovecs can be shared between multiple bios - a bvec iter can represent an69 arbitrary range of an existing biovec, both starting and ending midway70 through biovecs. This is what enables efficient splitting of arbitrary71 bios. Note that this means we _only_ use bi_size to determine when we've72 reached the end of a bio, not bi_vcnt - and the bio_iovec() macro takes73 bi_size into account when constructing biovecs.74 75 * Splitting bios is now much simpler. The old bio_split() didn't even work on76 bios with more than a single bvec! Now, we can efficiently split arbitrary77 size bios - because the new bio can share the old bio's biovec.78 79 Care must be taken to ensure the biovec isn't freed while the split bio is80 still using it, in case the original bio completes first, though. Using81 bio_chain() when splitting bios helps with this.82 83 * Submitting partially completed bios is now perfectly fine - this comes up84 occasionally in stacking block drivers and various code (e.g. md and85 bcache) had some ugly workarounds for this.86 87 It used to be the case that submitting a partially completed bio would work88 fine to _most_ devices, but since accessing the raw bvec array was the89 norm, not all drivers would respect bi_idx and those would break. Now,90 since all drivers _must_ go through the bvec iterator - and have been91 audited to make sure they are - submitting partially completed bios is92 perfectly fine.93 94Other implications:95===================96 97 * Almost all usage of bi_idx is now incorrect and has been removed; instead,98 where previously you would have used bi_idx you'd now use a bvec_iter,99 probably passing it to one of the helper macros.100 101 I.e. instead of using bio_iovec_idx() (or bio->bi_iovec[bio->bi_idx]), you102 now use bio_iter_iovec(), which takes a bvec_iter and returns a103 literal struct bio_vec - constructed on the fly from the raw biovec but104 taking into account bi_bvec_done (and bi_size).105 106 * bi_vcnt can't be trusted or relied upon by driver code - i.e. anything that107 doesn't actually own the bio. The reason is twofold: firstly, it's not108 actually needed for iterating over the bio anymore - we only use bi_size.109 Secondly, when cloning a bio and reusing (a portion of) the original bio's110 biovec, in order to calculate bi_vcnt for the new bio we'd have to iterate111 over all the biovecs in the new bio - which is silly as it's not needed.112 113 So, don't use bi_vcnt anymore.114 115 * The current interface allows the block layer to split bios as needed, so we116 could eliminate a lot of complexity particularly in stacked drivers. Code117 that creates bios can then create whatever size bios are convenient, and118 more importantly stacked drivers don't have to deal with both their own bio119 size limitations and the limitations of the underlying devices. Thus120 there's no need to define ->merge_bvec_fn() callbacks for individual block121 drivers.122 123Usage of helpers:124=================125 126* The following helpers whose names have the suffix of `_all` can only be used127 on non-BIO_CLONED bio. They are usually used by filesystem code. Drivers128 shouldn't use them because the bio may have been split before it reached the129 driver.130 131::132 133 bio_for_each_segment_all()134 bio_for_each_bvec_all()135 bio_first_bvec_all()136 bio_first_page_all()137 bio_first_folio_all()138 bio_last_bvec_all()139 140* The following helpers iterate over single-page segment. The passed 'struct141 bio_vec' will contain a single-page IO vector during the iteration::142 143 bio_for_each_segment()144 bio_for_each_segment_all()145 146* The following helpers iterate over multi-page bvec. The passed 'struct147 bio_vec' will contain a multi-page IO vector during the iteration::148 149 bio_for_each_bvec()150 bio_for_each_bvec_all()151 rq_for_each_bvec()152