79 lines · plain
1==========================2Reference counting in pnfs3==========================4 5The are several inter-related caches. We have layouts which can6reference multiple devices, each of which can reference multiple data servers.7Each data server can be referenced by multiple devices. Each device8can be referenced by multiple layouts. To keep all of this straight,9we need to reference count.10 11 12struct pnfs_layout_hdr13======================14 15The on-the-wire command LAYOUTGET corresponds to struct16pnfs_layout_segment, usually referred to by the variable name lseg.17Each nfs_inode may hold a pointer to a cache of these layout18segments in nfsi->layout, of type struct pnfs_layout_hdr.19 20We reference the header for the inode pointing to it, across each21outstanding RPC call that references it (LAYOUTGET, LAYOUTRETURN,22LAYOUTCOMMIT), and for each lseg held within.23 24Each header is also (when non-empty) put on a list associated with25struct nfs_client (cl_layouts). Being put on this list does not bump26the reference count, as the layout is kept around by the lseg that27keeps it in the list.28 29deviceid_cache30==============31 32lsegs reference device ids, which are resolved per nfs_client and33layout driver type. The device ids are held in a RCU cache (struct34nfs4_deviceid_cache). The cache itself is referenced across each35mount. The entries (struct nfs4_deviceid) themselves are held across36the lifetime of each lseg referencing them.37 38RCU is used because the deviceid is basically a write once, read many39data structure. The hlist size of 32 buckets needs better40justification, but seems reasonable given that we can have multiple41deviceid's per filesystem, and multiple filesystems per nfs_client.42 43The hash code is copied from the nfsd code base. A discussion of44hashing and variations of this algorithm can be found `here.45<http://groups.google.com/group/comp.lang.c/browse_thread/thread/9522965e2b8d3809>`_46 47data server cache48=================49 50file driver devices refer to data servers, which are kept in a module51level cache. Its reference is held over the lifetime of the deviceid52pointing to it.53 54lseg55====56 57lseg maintains an extra reference corresponding to the NFS_LSEG_VALID58bit which holds it in the pnfs_layout_hdr's list. When the final lseg59is removed from the pnfs_layout_hdr's list, the NFS_LAYOUT_DESTROYED60bit is set, preventing any new lsegs from being added.61 62layout drivers63==============64 65PNFS utilizes what is called layout drivers. The STD defines 4 basic66layout types: "files", "objects", "blocks", and "flexfiles". For each67of these types there is a layout-driver with a common function-vectors68table which are called by the nfs-client pnfs-core to implement the69different layout types.70 71Files-layout-driver code is in: fs/nfs/filelayout/.. directory72Blocks-layout-driver code is in: fs/nfs/blocklayout/.. directory73Flexfiles-layout-driver code is in: fs/nfs/flexfilelayout/.. directory74 75blocks-layout setup76===================77 78TODO: Document the setup needs of the blocks layout driver79