205 lines · plain
1.. SPDX-License-Identifier: GPL-2.02 3.. include:: <isonum.txt>4 5=========================================6Rockchip Image Signal Processor (rkisp1)7=========================================8 9Introduction10============11 12This file documents the driver for the Rockchip ISP1 that is part of RK328813and RK3399 SoCs. The driver is located under drivers/media/platform/rockchip/14rkisp1 and uses the Media-Controller API.15 16Revisions17=========18 19There exist multiple smaller revisions to this ISP that got introduced in20later SoCs. Revisions can be found in the enum :c:type:`rkisp1_cif_isp_version`21in the UAPI and the revision of the ISP inside the running SoC can be read22in the field hw_revision of struct media_device_info as returned by23ioctl MEDIA_IOC_DEVICE_INFO.24 25Versions in use are:26 27- RKISP1_V10: used at least in rk3288 and rk339928- RKISP1_V11: declared in the original vendor code, but not used29- RKISP1_V12: used at least in rk3326 and px3030- RKISP1_V13: used at least in rk180831 32Topology33========34.. _rkisp1_topology_graph:35 36.. kernel-figure:: rkisp1.dot37 :alt: Diagram of the default media pipeline topology38 :align: center39 40 41The driver has 4 video devices:42 43- rkisp1_mainpath: capture device for retrieving images, usually in higher44 resolution.45- rkisp1_selfpath: capture device for retrieving images.46- rkisp1_stats: a metadata capture device that sends statistics.47- rkisp1_params: a metadata output device that receives parameters48 configurations from userspace.49 50The driver has 3 subdevices:51 52- rkisp1_resizer_mainpath: used to resize and downsample frames for the53 mainpath capture device.54- rkisp1_resizer_selfpath: used to resize and downsample frames for the55 selfpath capture device.56- rkisp1_isp: is connected to the sensor and is responsible for all the isp57 operations.58 59 60rkisp1_mainpath, rkisp1_selfpath - Frames Capture Video Nodes61-------------------------------------------------------------62Those are the `mainpath` and `selfpath` capture devices to capture frames.63Those entities are the DMA engines that write the frames to memory.64The selfpath video device can capture YUV/RGB formats. Its input is YUV encoded65stream and it is able to convert it to RGB. The selfpath is not able to66capture bayer formats.67The mainpath can capture both bayer and YUV formats but it is not able to68capture RGB formats.69Both capture videos support70the ``V4L2_CAP_IO_MC`` :ref:`capability <device-capabilities>`.71 72 73rkisp1_resizer_mainpath, rkisp1_resizer_selfpath - Resizers Subdevices Nodes74----------------------------------------------------------------------------75Those are resizer entities for the mainpath and the selfpath. Those entities76can scale the frames up and down and also change the YUV sampling (for example77YUV4:2:2 -> YUV4:2:0). They also have cropping capability on the sink pad.78The resizers entities can only operate on YUV:4:2:2 format79(MEDIA_BUS_FMT_YUYV8_2X8).80The mainpath capture device supports capturing video in bayer formats. In that81case the resizer of the mainpath is set to 'bypass' mode - it just forward the82frame without operating on it.83 84rkisp1_isp - Image Signal Processing Subdevice Node85---------------------------------------------------86This is the isp entity. It is connected to the sensor on sink pad 0 and87receives the frames using the CSI-2 protocol. It is responsible of configuring88the CSI-2 protocol. It has a cropping capability on sink pad 0 that is89connected to the sensor and on source pad 2 connected to the resizer entities.90Cropping on sink pad 0 defines the image region from the sensor.91Cropping on source pad 2 defines the region for the Image Stabilizer (IS).92 93.. _rkisp1_stats:94 95rkisp1_stats - Statistics Video Node96------------------------------------97The statistics video node outputs the 3A (auto focus, auto exposure and auto98white balance) statistics, and also histogram statistics for the frames that99are being processed by the rkisp1 to userspace applications.100Using these data, applications can implement algorithms and re-parameterize101the driver through the rkisp_params node to improve image quality during a102video stream.103The buffer format is defined by struct :c:type:`rkisp1_stat_buffer`, and104userspace should set105:ref:`V4L2_META_FMT_RK_ISP1_STAT_3A <v4l2-meta-fmt-rk-isp1-stat-3a>` as the106dataformat.107 108.. _rkisp1_params:109 110rkisp1_params - Parameters Video Node111-------------------------------------112The rkisp1_params video node receives a set of parameters from userspace113to be applied to the hardware during a video stream, allowing userspace114to dynamically modify values such as black level, cross talk corrections115and others.116 117The ISP driver supports two different parameters configuration methods, the118`fixed parameters format` or the `extensible parameters format`.119 120When using the `fixed parameters` method the buffer format is defined by struct121:c:type:`rkisp1_params_cfg`, and userspace should set122:ref:`V4L2_META_FMT_RK_ISP1_PARAMS <v4l2-meta-fmt-rk-isp1-params>` as the123dataformat.124 125When using the `extensible parameters` method the buffer format is defined by126struct :c:type:`rkisp1_ext_params_cfg`, and userspace should set127:ref:`V4L2_META_FMT_RK_ISP1_EXT_PARAMS <v4l2-meta-fmt-rk-isp1-ext-params>` as128the dataformat.129 130Capturing Video Frames Example131==============================132 133In the following example, the sensor connected to pad 0 of 'rkisp1_isp' is134imx219.135 136The following commands can be used to capture video from the selfpath video137node with dimension 900x800 planar format YUV 4:2:2. It uses all cropping138capabilities possible, (see explanation right below)139 140.. code-block:: bash141 142 # set the links143 "media-ctl" "-d" "platform:rkisp1" "-r"144 "media-ctl" "-d" "platform:rkisp1" "-l" "'imx219 4-0010':0 -> 'rkisp1_isp':0 [1]"145 "media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"146 "media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_mainpath':0 [0]"147 148 # set format for imx219 4-0010:0149 "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"imx219 4-0010":0 [fmt:SRGGB10_1X10/1640x1232]'150 151 # set format for rkisp1_isp pads:152 "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0 [fmt:SRGGB10_1X10/1640x1232 crop: (0,0)/1600x1200]'153 "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":2 [fmt:YUYV8_2X8/1600x1200 crop: (0,0)/1500x1100]'154 155 # set format for rkisp1_resizer_selfpath pads:156 "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0 [fmt:YUYV8_2X8/1500x1100 crop: (300,400)/1400x1000]'157 "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":1 [fmt:YUYV8_2X8/900x800]'158 159 # set format for rkisp1_selfpath:160 "v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=900,height=800,"161 "v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "pixelformat=422P"162 163 # start streaming:164 v4l2-ctl "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "10"165 166 167In the above example the sensor is configured to bayer format:168`SRGGB10_1X10/1640x1232`. The rkisp1_isp:0 pad should be configured to the169same mbus format and dimensions as the sensor, otherwise streaming will fail170with 'EPIPE' error. So it is also configured to `SRGGB10_1X10/1640x1232`.171In addition, the rkisp1_isp:0 pad is configured to cropping `(0,0)/1600x1200`.172 173The cropping dimensions are automatically propagated to be the format of the174isp source pad `rkisp1_isp:2`. Another cropping operation is configured on175the isp source pad: `(0,0)/1500x1100`.176 177The resizer's sink pad `rkisp1_resizer_selfpath` should be configured to format178`YUYV8_2X8/1500x1100` in order to match the format on the other side of the179link. In addition a cropping `(300,400)/1400x1000` is configured on it.180 181The source pad of the resizer, `rkisp1_resizer_selfpath:1` is configured to182format `YUYV8_2X8/900x800`. That means that the resizer first crop a window183of `(300,400)/1400x100` from the received frame and then scales this window184to dimension `900x800`.185 186Note that the above example does not uses the stats-params control loop.187Therefore the capture frames will not go through the 3A algorithms and188probably won't have a good quality, and can even look dark and greenish.189 190Configuring Quantization191========================192 193The driver supports limited and full range quantization on YUV formats,194where limited is the default.195To switch between one or the other, userspace should use the Colorspace196Conversion API (CSC) for subdevices on source pad 2 of the197isp (`rkisp1_isp:2`). The quantization configured on this pad is the198quantization of the captured video frames on the mainpath and selfpath199video nodes.200Note that the resizer and capture entities will always report201``V4L2_QUANTIZATION_DEFAULT`` even if the quantization is configured to full202range on `rkisp1_isp:2`. So in order to get the configured quantization,203application should get it from pad `rkisp1_isp:2`.204 205