177 lines · plain
1.. SPDX-License-Identifier: GPL-2.02 3Introduction of Uacce4---------------------5 6Uacce (Unified/User-space-access-intended Accelerator Framework) targets to7provide Shared Virtual Addressing (SVA) between accelerators and processes.8So accelerator can access any data structure of the main cpu.9This differs from the data sharing between cpu and io device, which share10only data content rather than address.11Because of the unified address, hardware and user space of process can12share the same virtual address in the communication.13Uacce takes the hardware accelerator as a heterogeneous processor, while14IOMMU share the same CPU page tables and as a result the same translation15from va to pa.16 17::18 19 __________________________ __________________________20 | | | |21 | User application (CPU) | | Hardware Accelerator |22 |__________________________| |__________________________|23 24 | |25 | va | va26 V V27 __________ __________28 | | | |29 | MMU | | IOMMU |30 |__________| |__________|31 | |32 | |33 V pa V pa34 _______________________________________35 | |36 | Memory |37 |_______________________________________|38 39 40 41Architecture42------------43 44Uacce is the kernel module, taking charge of iommu and address sharing.45The user drivers and libraries are called WarpDrive.46 47The uacce device, built around the IOMMU SVA API, can access multiple48address spaces, including the one without PASID.49 50A virtual concept, queue, is used for the communication. It provides a51FIFO-like interface. And it maintains a unified address space between the52application and all involved hardware.53 54::55 56 ___________________ ________________57 | | user API | |58 | WarpDrive library | ------------> | user driver |59 |___________________| |________________|60 | |61 | |62 | queue fd |63 | |64 | |65 v |66 ___________________ _________ |67 | | | | | mmap memory68 | Other framework | | uacce | | r/w interface69 | crypto/nic/others | |_________| |70 |___________________| |71 | | |72 | register | register |73 | | |74 | | |75 | _________________ __________ |76 | | | | | |77 ------------- | Device Driver | | IOMMU | |78 |_________________| |__________| |79 | |80 | V81 | ___________________82 | | |83 -------------------------- | Device(Hardware) |84 |___________________|85 86 87How does it work88----------------89 90Uacce uses mmap and IOMMU to play the trick.91 92Uacce creates a chrdev for every device registered to it. New queue is93created when user application open the chrdev. The file descriptor is used94as the user handle of the queue.95The accelerator device present itself as an Uacce object, which exports as96a chrdev to the user space. The user application communicates with the97hardware by ioctl (as control path) or share memory (as data path).98 99The control path to the hardware is via file operation, while data path is100via mmap space of the queue fd.101 102The queue file address space:103 104::105 106 /**107 * enum uacce_qfrt: qfrt type108 * @UACCE_QFRT_MMIO: device mmio region109 * @UACCE_QFRT_DUS: device user share region110 */111 enum uacce_qfrt {112 UACCE_QFRT_MMIO = 0,113 UACCE_QFRT_DUS = 1,114 };115 116All regions are optional and differ from device type to type.117Each region can be mmapped only once, otherwise -EEXIST returns.118 119The device mmio region is mapped to the hardware mmio space. It is generally120used for doorbell or other notification to the hardware. It is not fast enough121as data channel.122 123The device user share region is used for share data buffer between user process124and device.125 126 127The Uacce register API128----------------------129 130The register API is defined in uacce.h.131 132::133 134 struct uacce_interface {135 char name[UACCE_MAX_NAME_SIZE];136 unsigned int flags;137 const struct uacce_ops *ops;138 };139 140According to the IOMMU capability, uacce_interface flags can be:141 142::143 144 /**145 * UACCE Device flags:146 * UACCE_DEV_SVA: Shared Virtual Addresses147 * Support PASID148 * Support device page faults (PCI PRI or SMMU Stall)149 */150 #define UACCE_DEV_SVA BIT(0)151 152 struct uacce_device *uacce_alloc(struct device *parent,153 struct uacce_interface *interface);154 int uacce_register(struct uacce_device *uacce);155 void uacce_remove(struct uacce_device *uacce);156 157uacce_register results can be:158 159a. If uacce module is not compiled, ERR_PTR(-ENODEV)160 161b. Succeed with the desired flags162 163c. Succeed with the negotiated flags, for example164 165 uacce_interface.flags = UACCE_DEV_SVA but uacce->flags = ~UACCE_DEV_SVA166 167 So user driver need check return value as well as the negotiated uacce->flags.168 169 170The user driver171---------------172 173The queue file mmap space will need a user driver to wrap the communication174protocol. Uacce provides some attributes in sysfs for the user driver to175match the right accelerator accordingly.176More details in Documentation/ABI/testing/sysfs-driver-uacce.177