110 lines · plain
1.. SPDX-License-Identifier: GPL-2.02 3=========================================================4Raspberry Pi PiSP Back End Memory-to-Memory ISP (pisp-be)5=========================================================6 7The PiSP Back End8=================9 10The PiSP Back End is a memory-to-memory Image Signal Processor (ISP) which reads11image data from DRAM memory and performs image processing as specified by the12application through the parameters in a configuration buffer, before writing13pixel data back to memory through two distinct output channels.14 15The ISP registers and programming model are documented in the `Raspberry Pi16Image Signal Processor (PiSP) Specification document`_17 18The PiSP Back End ISP processes images in tiles. The handling of image19tessellation and the computation of low-level configuration parameters is20realized by a free software library called `libpisp21<https://github.com/raspberrypi/libpisp>`_.22 23The full image processing pipeline, which involves capturing RAW Bayer data from24an image sensor through a MIPI CSI-2 compatible capture interface, storing them25in DRAM memory and processing them in the PiSP Back End to obtain images usable26by an application is implemented in `libcamera <https://libcamera.org>`_ as27part of the Raspberry Pi platform support.28 29The pisp-be driver30==================31 32The Raspberry Pi PiSP Back End (pisp-be) driver is located under33drivers/media/platform/raspberrypi/pisp-be. It uses the `V4L2 API` to register34a number of video capture and output devices, the `V4L2 subdev API` to register35a subdevice for the ISP that connects the video devices in a single media graph36realized using the `Media Controller (MC) API`.37 38The media topology registered by the `pisp-be` driver is represented below:39 40.. _pips-be-topology:41 42.. kernel-figure:: raspberrypi-pisp-be.dot43 :alt: Diagram of the default media pipeline topology44 :align: center45 46 47The media graph registers the following video device nodes:48 49- pispbe-input: output device for images to be submitted to the ISP for50 processing.51- pispbe-tdn_input: output device for temporal denoise.52- pispbe-stitch_input: output device for image stitching (HDR).53- pispbe-output0: first capture device for processed images.54- pispbe-output1: second capture device for processed images.55- pispbe-tdn_output: capture device for temporal denoise.56- pispbe-stitch_output: capture device for image stitching (HDR).57- pispbe-config: output device for ISP configuration parameters.58 59pispbe-input60------------61 62Images to be processed by the ISP are queued to the `pispbe-input` output device63node. For a list of image formats supported as input to the ISP refer to the64`Raspberry Pi Image Signal Processor (PiSP) Specification document`_.65 66pispbe-tdn_input, pispbe-tdn_output67-----------------------------------68 69The `pispbe-tdn_input` output video device receives images to be processed by70the temporal denoise block which are captured from the `pispbe-tdn_output`71capture video device. Userspace is responsible for maintaining queues on both72devices, and ensuring that buffers completed on the output are queued to the73input.74 75pispbe-stitch_input, pispbe-stitch_output76-----------------------------------------77 78To realize HDR (high dynamic range) image processing the image stitching and79tonemapping blocks are used. The `pispbe-stitch_output` writes images to memory80and the `pispbe-stitch_input` receives the previously written frame to process81it along with the current input image. Userspace is responsible for maintaining82queues on both devices, and ensuring that buffers completed on the output are83queued to the input.84 85pispbe-output0, pispbe-output186------------------------------87 88The two capture devices write to memory the pixel data as processed by the ISP.89 90pispbe-config91-------------92 93The `pispbe-config` output video devices receives a buffer of configuration94parameters that define the desired image processing to be performed by the ISP.95 96The format of the ISP configuration parameter is defined by97:c:type:`pisp_be_tiles_config` C structure and the meaning of each parameter is98described in the `Raspberry Pi Image Signal Processor (PiSP) Specification99document`_.100 101ISP configuration102=================103 104The ISP configuration is described solely by the content of the parameters105buffer. The only parameter that userspace needs to configure using the V4L2 API106is the image format on the output and capture video devices for validation of107the content of the parameters buffer.108 109.. _Raspberry Pi Image Signal Processor (PiSP) Specification document: https://datasheets.raspberrypi.com/camera/raspberry-pi-image-signal-processor-specification.pdf110