381 lines · plain
1=======================2Linux UVC Gadget Driver3=======================4 5Overview6--------7The UVC Gadget driver is a driver for hardware on the *device* side of a USB8connection. It is intended to run on a Linux system that has USB device-side9hardware such as boards with an OTG port.10 11On the device system, once the driver is bound it appears as a V4L2 device with12the output capability.13 14On the host side (once connected via USB cable), a device running the UVC Gadget15driver *and controlled by an appropriate userspace program* should appear as a UVC16specification compliant camera, and function appropriately with any program17designed to handle them. The userspace program running on the device system can18queue image buffers from a variety of sources to be transmitted via the USB19connection. Typically this would mean forwarding the buffers from a camera sensor20peripheral, but the source of the buffer is entirely dependent on the userspace21companion program.22 23Configuring the device kernel24-----------------------------25The Kconfig options USB_CONFIGFS, USB_LIBCOMPOSITE, USB_CONFIGFS_F_UVC and26USB_F_UVC must be selected to enable support for the UVC gadget.27 28Configuring the gadget through configfs29---------------------------------------30The UVC Gadget expects to be configured through configfs using the UVC function.31This allows a significant degree of flexibility, as many of a UVC device's32settings can be controlled this way.33 34Not all of the available attributes are described here. For a complete enumeration35see Documentation/ABI/testing/configfs-usb-gadget-uvc36 37Assumptions38~~~~~~~~~~~39This section assumes that you have mounted configfs at `/sys/kernel/config` and40created a gadget as `/sys/kernel/config/usb_gadget/g1`.41 42The UVC Function43~~~~~~~~~~~~~~~~44 45The first step is to create the UVC function:46 47.. code-block:: bash48 49 # These variables will be assumed throughout the rest of the document50 CONFIGFS="/sys/kernel/config"51 GADGET="$CONFIGFS/usb_gadget/g1"52 FUNCTION="$GADGET/functions/uvc.0"53 54 mkdir -p $FUNCTION55 56Formats and Frames57~~~~~~~~~~~~~~~~~~58 59You must configure the gadget by telling it which formats you support, as well60as the frame sizes and frame intervals that are supported for each format. In61the current implementation there is no way for the gadget to refuse to set a62format that the host instructs it to set, so it is important that this step is63completed *accurately* to ensure that the host never asks for a format that64can't be provided.65 66Formats are created under the streaming/uncompressed and streaming/mjpeg configfs67groups, with the framesizes created under the formats in the following68structure:69 70::71 72 uvc.0 +73 |74 + streaming +75 |76 + mjpeg +77 | |78 | + mjpeg +79 | |80 | + 720p81 | |82 | + 1080p83 |84 + uncompressed +85 |86 + yuyv +87 |88 + 720p89 |90 + 1080p91 92Each frame can then be configured with a width and height, plus the maximum93buffer size required to store a single frame, and finally with the supported94frame intervals for that format and framesize. Width and height are enumerated in95units of pixels, frame interval in units of 100ns. To create the structure96above with 2, 15 and 100 fps frameintervals for each framesize for example you97might do:98 99.. code-block:: bash100 101 create_frame() {102 # Example usage:103 # create_frame <width> <height> <group> <format name>104 105 WIDTH=$1106 HEIGHT=$2107 FORMAT=$3108 NAME=$4109 110 wdir=$FUNCTION/streaming/$FORMAT/$NAME/${HEIGHT}p111 112 mkdir -p $wdir113 echo $WIDTH > $wdir/wWidth114 echo $HEIGHT > $wdir/wHeight115 echo $(( $WIDTH * $HEIGHT * 2 )) > $wdir/dwMaxVideoFrameBufferSize116 cat <<EOF > $wdir/dwFrameInterval117 666666118 100000119 5000000120 EOF121 }122 123 create_frame 1280 720 mjpeg mjpeg124 create_frame 1920 1080 mjpeg mjpeg125 create_frame 1280 720 uncompressed yuyv126 create_frame 1920 1080 uncompressed yuyv127 128The only uncompressed format currently supported is YUYV, which is detailed at129Documentation/userspace-api/media/v4l/pixfmt-packed-yuv.rst.130 131Color Matching Descriptors132~~~~~~~~~~~~~~~~~~~~~~~~~~133It's possible to specify some colometry information for each format you create.134This step is optional, and default information will be included if this step is135skipped; those default values follow those defined in the Color Matching Descriptor136section of the UVC specification.137 138To create a Color Matching Descriptor, create a configfs item and set its three139attributes to your desired settings and then link to it from the format you wish140it to be associated with:141 142.. code-block:: bash143 144 # Create a new Color Matching Descriptor145 146 mkdir $FUNCTION/streaming/color_matching/yuyv147 pushd $FUNCTION/streaming/color_matching/yuyv148 149 echo 1 > bColorPrimaries150 echo 1 > bTransferCharacteristics151 echo 4 > bMatrixCoefficients152 153 popd154 155 # Create a symlink to the Color Matching Descriptor from the format's config item156 ln -s $FUNCTION/streaming/color_matching/yuyv $FUNCTION/streaming/uncompressed/yuyv157 158For details about the valid values, consult the UVC specification. Note that a159default color matching descriptor exists and is used by any format which does160not have a link to a different Color Matching Descriptor. It's possible to161change the attribute settings for the default descriptor, so bear in mind that if162you do that you are altering the defaults for any format that does not link to163a different one.164 165 166Header linking167~~~~~~~~~~~~~~168 169The UVC specification requires that Format and Frame descriptors be preceded by170Headers detailing things such as the number and cumulative size of the different171Format descriptors that follow. This and similar operations are achieved in172configfs by linking between the configfs item representing the header and the173config items representing those other descriptors, in this manner:174 175.. code-block:: bash176 177 mkdir $FUNCTION/streaming/header/h178 179 # This section links the format descriptors and their associated frames180 # to the header181 cd $FUNCTION/streaming/header/h182 ln -s ../../uncompressed/yuyv183 ln -s ../../mjpeg/mjpeg184 185 # This section ensures that the header will be transmitted for each186 # speed's set of descriptors. If support for a particular speed is not187 # needed then it can be skipped here.188 cd ../../class/fs189 ln -s ../../header/h190 cd ../../class/hs191 ln -s ../../header/h192 cd ../../class/ss193 ln -s ../../header/h194 cd ../../../control195 mkdir header/h196 ln -s header/h class/fs197 ln -s header/h class/ss198 199 200Extension Unit Support201~~~~~~~~~~~~~~~~~~~~~~202 203A UVC Extension Unit (XU) basically provides a distinct unit to which control set204and get requests can be addressed. The meaning of those control requests is205entirely implementation dependent, but may be used to control settings outside206of the UVC specification (for example enabling or disabling video effects). An207XU can be inserted into the UVC unit chain or left free-hanging.208 209Configuring an extension unit involves creating an entry in the appropriate210directory and setting its attributes appropriately, like so:211 212.. code-block:: bash213 214 mkdir $FUNCTION/control/extensions/xu.0215 pushd $FUNCTION/control/extensions/xu.0216 217 # Set the bUnitID of the Processing Unit as the source for this218 # Extension Unit219 echo 2 > baSourceID220 221 # Set this XU as the source of the default output terminal. This inserts222 # the XU into the UVC chain between the PU and OT such that the final223 # chain is IT > PU > XU.0 > OT224 cat bUnitID > ../../terminal/output/default/baSourceID225 226 # Flag some controls as being available for use. The bmControl field is227 # a bitmap with each bit denoting the availability of a particular228 # control. For example to flag the 0th, 2nd and 3rd controls available:229 echo 0x0d > bmControls230 231 # Set the GUID; this is a vendor-specific code identifying the XU.232 echo -e -n "\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10" > guidExtensionCode233 234 popd235 236The bmControls attribute and the baSourceID attribute are multi-value attributes.237This means that you may write multiple newline separated values to them. For238example to flag the 1st, 2nd, 9th and 10th controls as being available you would239need to write two values to bmControls, like so:240 241.. code-block:: bash242 243 cat << EOF > bmControls244 0x03245 0x03246 EOF247 248The multi-value nature of the baSourceID attribute belies the fact that XUs can249be multiple-input, though note that this currently has no significant effect.250 251The bControlSize attribute reflects the size of the bmControls attribute, and252similarly bNrInPins reflects the size of the baSourceID attributes. Both253attributes are automatically increased / decreased as you set bmControls and254baSourceID. It is also possible to manually increase or decrease bControlSize255which has the effect of truncating entries to the new size, or padding entries256out with 0x00, for example:257 258::259 260 $ cat bmControls261 0x03262 0x05263 264 $ cat bControlSize265 2266 267 $ echo 1 > bControlSize268 $ cat bmControls269 0x03270 271 $ echo 2 > bControlSize272 $ cat bmControls273 0x03274 0x00275 276bNrInPins and baSourceID function in the same way.277 278Configuring Supported Controls for Camera Terminal and Processing Unit279~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~280 281The Camera Terminal and Processing Units in the UVC chain also have bmControls282attributes which function similarly to the same field in an Extension Unit.283Unlike XUs however, the meaning of the bitflag for these units is defined in284the UVC specification; you should consult the "Camera Terminal Descriptor" and285"Processing Unit Descriptor" sections for an enumeration of the flags.286 287.. code-block:: bash288 289 # Set the Processing Unit's bmControls, flagging Brightness, Contrast290 # and Hue as available controls:291 echo 0x05 > $FUNCTION/control/processing/default/bmControls292 293 # Set the Camera Terminal's bmControls, flagging Focus Absolute and294 # Focus Relative as available controls:295 echo 0x60 > $FUNCTION/control/terminal/camera/default/bmControls296 297If you do not set these fields then by default the Auto-Exposure Mode control298for the Camera Terminal and the Brightness control for the Processing Unit will299be flagged as available; if they are not supported you should set the field to3000x00.301 302Note that the size of the bmControls field for a Camera Terminal or Processing303Unit is fixed by the UVC specification, and so the bControlSize attribute is304read-only here.305 306Custom Strings Support307~~~~~~~~~~~~~~~~~~~~~~308 309String descriptors that provide a textual description for various parts of a310USB device can be defined in the usual place within USB configfs, and may then311be linked to from the UVC function root or from Extension Unit directories to312assign those strings as descriptors:313 314.. code-block:: bash315 316 # Create a string descriptor in us-EN and link to it from the function317 # root. The name of the link is significant here, as it declares this318 # descriptor to be intended for the Interface Association Descriptor.319 # Other significant link names at function root are vs0_desc and vs1_desc320 # For the VideoStreaming Interface 0/1 Descriptors.321 322 mkdir -p $GADGET/strings/0x409/iad_desc323 echo -n "Interface Associaton Descriptor" > $GADGET/strings/0x409/iad_desc/s324 ln -s $GADGET/strings/0x409/iad_desc $FUNCTION/iad_desc325 326 # Because the link to a String Descriptor from an Extension Unit clearly327 # associates the two, the name of this link is not significant and may328 # be set freely.329 330 mkdir -p $GADGET/strings/0x409/xu.0331 echo -n "A Very Useful Extension Unit" > $GADGET/strings/0x409/xu.0/s332 ln -s $GADGET/strings/0x409/xu.0 $FUNCTION/control/extensions/xu.0333 334The interrupt endpoint335~~~~~~~~~~~~~~~~~~~~~~336 337The VideoControl interface has an optional interrupt endpoint which is by default338disabled. This is intended to support delayed response control set requests for339UVC (which should respond through the interrupt endpoint rather than tying up340endpoint 0). At present support for sending data through this endpoint is missing341and so it is left disabled to avoid confusion. If you wish to enable it you can342do so through the configfs attribute:343 344.. code-block:: bash345 346 echo 1 > $FUNCTION/control/enable_interrupt_ep347 348Bandwidth configuration349~~~~~~~~~~~~~~~~~~~~~~~350 351There are three attributes which control the bandwidth of the USB connection.352These live in the function root and can be set within limits:353 354.. code-block:: bash355 356 # streaming_interval sets bInterval. Values range from 1..255357 echo 1 > $FUNCTION/streaming_interval358 359 # streaming_maxpacket sets wMaxPacketSize. Valid values are 1024/2048/3072360 echo 3072 > $FUNCTION/streaming_maxpacket361 362 # streaming_maxburst sets bMaxBurst. Valid values are 1..15363 echo 1 > $FUNCTION/streaming_maxburst364 365 366The values passed here will be clamped to valid values according to the UVC367specification (which depend on the speed of the USB connection). To understand368how the settings influence bandwidth you should consult the UVC specifications,369but a rule of thumb is that increasing the streaming_maxpacket setting will370improve bandwidth (and thus the maximum possible framerate), whilst the same is371true for streaming_maxburst provided the USB connection is running at SuperSpeed.372Increasing streaming_interval will reduce bandwidth and framerate.373 374The userspace application375-------------------------376By itself, the UVC Gadget driver cannot do anything particularly interesting. It377must be paired with a userspace program that responds to UVC control requests and378fills buffers to be queued to the V4L2 device that the driver creates. How those379things are achieved is implementation dependent and beyond the scope of this380document, but a reference application can be found at https://gitlab.freedesktop.org/camera/uvc-gadget381