254 lines · plain
1.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later2.. c:namespace:: MC3 4.. _media-request-api:5 6Request API7===========8 9The Request API has been designed to allow V4L2 to deal with requirements of10modern devices (stateless codecs, complex camera pipelines, ...) and APIs11(Android Codec v2). One such requirement is the ability for devices belonging to12the same pipeline to reconfigure and collaborate closely on a per-frame basis.13Another is support of stateless codecs, which require controls to be applied14to specific frames (aka 'per-frame controls') in order to be used efficiently.15 16While the initial use-case was V4L2, it can be extended to other subsystems17as well, as long as they use the media controller.18 19Supporting these features without the Request API is not always possible and if20it is, it is terribly inefficient: user-space would have to flush all activity21on the media pipeline, reconfigure it for the next frame, queue the buffers to22be processed with that configuration, and wait until they are all available for23dequeuing before considering the next frame. This defeats the purpose of having24buffer queues since in practice only one buffer would be queued at a time.25 26The Request API allows a specific configuration of the pipeline (media27controller topology + configuration for each media entity) to be associated with28specific buffers. This allows user-space to schedule several tasks ("requests")29with different configurations in advance, knowing that the configuration will be30applied when needed to get the expected result. Configuration values at the time31of request completion are also available for reading.32 33General Usage34-------------35 36The Request API extends the Media Controller API and cooperates with37subsystem-specific APIs to support request usage. At the Media Controller38level, requests are allocated from the supporting Media Controller device39node. Their life cycle is then managed through the request file descriptors in40an opaque way. Configuration data, buffer handles and processing results41stored in requests are accessed through subsystem-specific APIs extended for42request support, such as V4L2 APIs that take an explicit ``request_fd``43parameter.44 45Request Allocation46------------------47 48User-space allocates requests using :ref:`MEDIA_IOC_REQUEST_ALLOC`49for the media device node. This returns a file descriptor representing the50request. Typically, several such requests will be allocated.51 52Request Preparation53-------------------54 55Standard V4L2 ioctls can then receive a request file descriptor to express the56fact that the ioctl is part of said request, and is not to be applied57immediately. See :ref:`MEDIA_IOC_REQUEST_ALLOC` for a list of ioctls that58support this. Configurations set with a ``request_fd`` parameter are stored59instead of being immediately applied, and buffers queued to a request do not60enter the regular buffer queue until the request itself is queued.61 62Request Submission63------------------64 65Once the configuration and buffers of the request are specified, it can be66queued by calling :ref:`MEDIA_REQUEST_IOC_QUEUE` on the request file descriptor.67A request must contain at least one buffer, otherwise ``ENOENT`` is returned.68A queued request cannot be modified anymore.69 70.. caution::71 For :ref:`memory-to-memory devices <mem2mem>` you can use requests only for72 output buffers, not for capture buffers. Attempting to add a capture buffer73 to a request will result in an ``EBADR`` error.74 75If the request contains configurations for multiple entities, individual drivers76may synchronize so the requested pipeline's topology is applied before the77buffers are processed. Media controller drivers do a best effort implementation78since perfect atomicity may not be possible due to hardware limitations.79 80.. caution::81 82 It is not allowed to mix queuing requests with directly queuing buffers:83 whichever method is used first locks this in place until84 :ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>` is called or the device is85 :ref:`closed <func-close>`. Attempts to directly queue a buffer when earlier86 a buffer was queued via a request or vice versa will result in an ``EBUSY``87 error.88 89Controls can still be set without a request and are applied immediately,90regardless of whether a request is in use or not.91 92.. caution::93 94 Setting the same control through a request and also directly can lead to95 undefined behavior!96 97User-space can :c:func:`poll()` a request file descriptor in98order to wait until the request completes. A request is considered complete99once all its associated buffers are available for dequeuing and all the100associated controls have been updated with the values at the time of completion.101Note that user-space does not need to wait for the request to complete to102dequeue its buffers: buffers that are available halfway through a request can103be dequeued independently of the request's state.104 105A completed request contains the state of the device after the request was106executed. User-space can query that state by calling107:ref:`ioctl VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` with the request file108descriptor. Calling :ref:`ioctl VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` for a109request that has been queued but not yet completed will return ``EBUSY``110since the control values might be changed at any time by the driver while the111request is in flight.112 113.. _media-request-life-time:114 115Recycling and Destruction116-------------------------117 118Finally, a completed request can either be discarded or be reused. Calling119:c:func:`close()` on a request file descriptor will make120that file descriptor unusable and the request will be freed once it is no121longer in use by the kernel. That is, if the request is queued and then the122file descriptor is closed, then it won't be freed until the driver completed123the request.124 125The :ref:`MEDIA_REQUEST_IOC_REINIT` will clear a request's state and make it126available again. No state is retained by this operation: the request is as127if it had just been allocated.128 129Example for a Codec Device130--------------------------131 132For use-cases such as :ref:`codecs <mem2mem>`, the request API can be used133to associate specific controls to134be applied by the driver for the OUTPUT buffer, allowing user-space135to queue many such buffers in advance. It can also take advantage of requests'136ability to capture the state of controls when the request completes to read back137information that may be subject to change.138 139Put into code, after obtaining a request, user-space can assign controls and one140OUTPUT buffer to it:141 142.. code-block:: c143 144 struct v4l2_buffer buf;145 struct v4l2_ext_controls ctrls;146 int req_fd;147 ...148 if (ioctl(media_fd, MEDIA_IOC_REQUEST_ALLOC, &req_fd))149 return errno;150 ...151 ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL;152 ctrls.request_fd = req_fd;153 if (ioctl(codec_fd, VIDIOC_S_EXT_CTRLS, &ctrls))154 return errno;155 ...156 buf.type = V4L2_BUF_TYPE_VIDEO_OUTPUT;157 buf.flags |= V4L2_BUF_FLAG_REQUEST_FD;158 buf.request_fd = req_fd;159 if (ioctl(codec_fd, VIDIOC_QBUF, &buf))160 return errno;161 162Note that it is not allowed to use the Request API for CAPTURE buffers163since there are no per-frame settings to report there.164 165Once the request is fully prepared, it can be queued to the driver:166 167.. code-block:: c168 169 if (ioctl(req_fd, MEDIA_REQUEST_IOC_QUEUE))170 return errno;171 172User-space can then either wait for the request to complete by calling poll() on173its file descriptor, or start dequeuing CAPTURE buffers. Most likely, it will174want to get CAPTURE buffers as soon as possible and this can be done using a175regular :ref:`VIDIOC_DQBUF <VIDIOC_QBUF>`:176 177.. code-block:: c178 179 struct v4l2_buffer buf;180 181 memset(&buf, 0, sizeof(buf));182 buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;183 if (ioctl(codec_fd, VIDIOC_DQBUF, &buf))184 return errno;185 186Note that this example assumes for simplicity that for every OUTPUT buffer187there will be one CAPTURE buffer, but this does not have to be the case.188 189We can then, after ensuring that the request is completed via polling the190request file descriptor, query control values at the time of its completion via191a call to :ref:`VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>`.192This is particularly useful for volatile controls for which we want to193query values as soon as the capture buffer is produced.194 195.. code-block:: c196 197 struct pollfd pfd = { .events = POLLPRI, .fd = req_fd };198 poll(&pfd, 1, -1);199 ...200 ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL;201 ctrls.request_fd = req_fd;202 if (ioctl(codec_fd, VIDIOC_G_EXT_CTRLS, &ctrls))203 return errno;204 205Once we don't need the request anymore, we can either recycle it for reuse with206:ref:`MEDIA_REQUEST_IOC_REINIT`...207 208.. code-block:: c209 210 if (ioctl(req_fd, MEDIA_REQUEST_IOC_REINIT))211 return errno;212 213... or close its file descriptor to completely dispose of it.214 215.. code-block:: c216 217 close(req_fd);218 219Example for a Simple Capture Device220-----------------------------------221 222With a simple capture device, requests can be used to specify controls to apply223for a given CAPTURE buffer.224 225.. code-block:: c226 227 struct v4l2_buffer buf;228 struct v4l2_ext_controls ctrls;229 int req_fd;230 ...231 if (ioctl(media_fd, MEDIA_IOC_REQUEST_ALLOC, &req_fd))232 return errno;233 ...234 ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL;235 ctrls.request_fd = req_fd;236 if (ioctl(camera_fd, VIDIOC_S_EXT_CTRLS, &ctrls))237 return errno;238 ...239 buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;240 buf.flags |= V4L2_BUF_FLAG_REQUEST_FD;241 buf.request_fd = req_fd;242 if (ioctl(camera_fd, VIDIOC_QBUF, &buf))243 return errno;244 245Once the request is fully prepared, it can be queued to the driver:246 247.. code-block:: c248 249 if (ioctl(req_fd, MEDIA_REQUEST_IOC_QUEUE))250 return errno;251 252User-space can then dequeue buffers, wait for the request completion, query253controls and recycle the request as in the M2M example above.254