531 lines · plain
1===========================2Linux for S/390 and zSeries3===========================4 5Common Device Support (CDS)6Device Driver I/O Support Routines7 8Authors:9 - Ingo Adlung10 - Cornelia Huck11 12Copyright, IBM Corp. 1999-200213 14Introduction15============16 17This document describes the common device support routines for Linux/390.18Different than other hardware architectures, ESA/390 has defined a unified19I/O access method. This gives relief to the device drivers as they don't20have to deal with different bus types, polling versus interrupt21processing, shared versus non-shared interrupt processing, DMA versus port22I/O (PIO), and other hardware features more. However, this implies that23either every single device driver needs to implement the hardware I/O24attachment functionality itself, or the operating system provides for a25unified method to access the hardware, providing all the functionality that26every single device driver would have to provide itself.27 28The document does not intend to explain the ESA/390 hardware architecture in29every detail.This information can be obtained from the ESA/390 Principles of30Operation manual (IBM Form. No. SA22-7201).31 32In order to build common device support for ESA/390 I/O interfaces, a33functional layer was introduced that provides generic I/O access methods to34the hardware.35 36The common device support layer comprises the I/O support routines defined37below. Some of them implement common Linux device driver interfaces, while38some of them are ESA/390 platform specific.39 40Note:41 In order to write a driver for S/390, you also need to look into the interface42 described in Documentation/arch/s390/driver-model.rst.43 44Note for porting drivers from 2.4:45 46The major changes are:47 48* The functions use a ccw_device instead of an irq (subchannel).49* All drivers must define a ccw_driver (see driver-model.txt) and the associated50 functions.51* request_irq() and free_irq() are no longer done by the driver.52* The oper_handler is (kindof) replaced by the probe() and set_online() functions53 of the ccw_driver.54* The not_oper_handler is (kindof) replaced by the remove() and set_offline()55 functions of the ccw_driver.56* The channel device layer is gone.57* The interrupt handlers must be adapted to use a ccw_device as argument.58 Moreover, they don't return a devstat, but an irb.59* Before initiating an io, the options must be set via ccw_device_set_options().60* Instead of calling read_dev_chars()/read_conf_data(), the driver issues61 the channel program and handles the interrupt itself.62 63ccw_device_get_ciw()64 get commands from extended sense data.65 66ccw_device_start(), ccw_device_start_timeout(), ccw_device_start_key(), ccw_device_start_key_timeout()67 initiate an I/O request.68 69ccw_device_resume()70 resume channel program execution.71 72ccw_device_halt()73 terminate the current I/O request processed on the device.74 75do_IRQ()76 generic interrupt routine. This function is called by the interrupt entry77 routine whenever an I/O interrupt is presented to the system. The do_IRQ()78 routine determines the interrupt status and calls the device specific79 interrupt handler according to the rules (flags) defined during I/O request80 initiation with do_IO().81 82The next chapters describe the functions other than do_IRQ() in more details.83The do_IRQ() interface is not described, as it is called from the Linux/39084first level interrupt handler only and does not comprise a device driver85callable interface. Instead, the functional description of do_IO() also86describes the input to the device specific interrupt handler.87 88Note:89 All explanations apply also to the 64 bit architecture s390x.90 91 92Common Device Support (CDS) for Linux/390 Device Drivers93========================================================94 95General Information96-------------------97 98The following chapters describe the I/O related interface routines the99Linux/390 common device support (CDS) provides to allow for device specific100driver implementations on the IBM ESA/390 hardware platform. Those interfaces101intend to provide the functionality required by every device driver102implementation to allow to drive a specific hardware device on the ESA/390103platform. Some of the interface routines are specific to Linux/390 and some104of them can be found on other Linux platforms implementations too.105Miscellaneous function prototypes, data declarations, and macro definitions106can be found in the architecture specific C header file107linux/arch/s390/include/asm/irq.h.108 109Overview of CDS interface concepts110----------------------------------111 112Different to other hardware platforms, the ESA/390 architecture doesn't define113interrupt lines managed by a specific interrupt controller and bus systems114that may or may not allow for shared interrupts, DMA processing, etc.. Instead,115the ESA/390 architecture has implemented a so called channel subsystem, that116provides a unified view of the devices physically attached to the systems.117Though the ESA/390 hardware platform knows about a huge variety of different118peripheral attachments like disk devices (aka. DASDs), tapes, communication119controllers, etc. they can all be accessed by a well defined access method and120they are presenting I/O completion a unified way : I/O interruptions. Every121single device is uniquely identified to the system by a so called subchannel,122where the ESA/390 architecture allows for 64k devices be attached.123 124Linux, however, was first built on the Intel PC architecture, with its two125cascaded 8259 programmable interrupt controllers (PICs), that allow for a126maximum of 15 different interrupt lines. All devices attached to such a system127share those 15 interrupt levels. Devices attached to the ISA bus system must128not share interrupt levels (aka. IRQs), as the ISA bus bases on edge triggered129interrupts. MCA, EISA, PCI and other bus systems base on level triggered130interrupts, and therewith allow for shared IRQs. However, if multiple devices131present their hardware status by the same (shared) IRQ, the operating system132has to call every single device driver registered on this IRQ in order to133determine the device driver owning the device that raised the interrupt.134 135Up to kernel 2.4, Linux/390 used to provide interfaces via the IRQ (subchannel).136For internal use of the common I/O layer, these are still there. However,137device drivers should use the new calling interface via the ccw_device only.138 139During its startup the Linux/390 system checks for peripheral devices. Each140of those devices is uniquely defined by a so called subchannel by the ESA/390141channel subsystem. While the subchannel numbers are system generated, each142subchannel also takes a user defined attribute, the so called device number.143Both subchannel number and device number cannot exceed 65535. During sysfs144initialisation, the information about control unit type and device types that145imply specific I/O commands (channel command words - CCWs) in order to operate146the device are gathered. Device drivers can retrieve this set of hardware147information during their initialization step to recognize the devices they148support using the information saved in the struct ccw_device given to them.149This methods implies that Linux/390 doesn't require to probe for free (not150armed) interrupt request lines (IRQs) to drive its devices with. Where151applicable, the device drivers can use issue the READ DEVICE CHARACTERISTICS152ccw to retrieve device characteristics in its online routine.153 154In order to allow for easy I/O initiation the CDS layer provides a155ccw_device_start() interface that takes a device specific channel program (one156or more CCWs) as input sets up the required architecture specific control blocks157and initiates an I/O request on behalf of the device driver. The158ccw_device_start() routine allows to specify whether it expects the CDS layer159to notify the device driver for every interrupt it observes, or with final status160only. See ccw_device_start() for more details. A device driver must never issue161ESA/390 I/O commands itself, but must use the Linux/390 CDS interfaces instead.162 163For long running I/O request to be canceled, the CDS layer provides the164ccw_device_halt() function. Some devices require to initially issue a HALT165SUBCHANNEL (HSCH) command without having pending I/O requests. This function is166also covered by ccw_device_halt().167 168 169get_ciw() - get command information word170 171This call enables a device driver to get information about supported commands172from the extended SenseID data.173 174::175 176 struct ciw *177 ccw_device_get_ciw(struct ccw_device *cdev, __u32 cmd);178 179==== ========================================================180cdev The ccw_device for which the command is to be retrieved.181cmd The command type to be retrieved.182==== ========================================================183 184ccw_device_get_ciw() returns:185 186===== ================================================================187 NULL No extended data available, invalid device or command not found.188!NULL The command requested.189===== ================================================================190 191::192 193 ccw_device_start() - Initiate I/O Request194 195The ccw_device_start() routines is the I/O request front-end processor. All196device driver I/O requests must be issued using this routine. A device driver197must not issue ESA/390 I/O commands itself. Instead the ccw_device_start()198routine provides all interfaces required to drive arbitrary devices.199 200This description also covers the status information passed to the device201driver's interrupt handler as this is related to the rules (flags) defined202with the associated I/O request when calling ccw_device_start().203 204::205 206 int ccw_device_start(struct ccw_device *cdev,207 struct ccw1 *cpa,208 unsigned long intparm,209 __u8 lpm,210 unsigned long flags);211 int ccw_device_start_timeout(struct ccw_device *cdev,212 struct ccw1 *cpa,213 unsigned long intparm,214 __u8 lpm,215 unsigned long flags,216 int expires);217 int ccw_device_start_key(struct ccw_device *cdev,218 struct ccw1 *cpa,219 unsigned long intparm,220 __u8 lpm,221 __u8 key,222 unsigned long flags);223 int ccw_device_start_key_timeout(struct ccw_device *cdev,224 struct ccw1 *cpa,225 unsigned long intparm,226 __u8 lpm,227 __u8 key,228 unsigned long flags,229 int expires);230 231============= =============================================================232cdev ccw_device the I/O is destined for233cpa logical start address of channel program234user_intparm user specific interrupt information; will be presented235 back to the device driver's interrupt handler. Allows a236 device driver to associate the interrupt with a237 particular I/O request.238lpm defines the channel path to be used for a specific I/O239 request. A value of 0 will make cio use the opm.240key the storage key to use for the I/O (useful for operating on a241 storage with a storage key != default key)242flag defines the action to be performed for I/O processing243expires timeout value in jiffies. The common I/O layer will terminate244 the running program after this and call the interrupt handler245 with ERR_PTR(-ETIMEDOUT) as irb.246============= =============================================================247 248Possible flag values are:249 250========================= =============================================251DOIO_ALLOW_SUSPEND channel program may become suspended252DOIO_DENY_PREFETCH don't allow for CCW prefetch; usually253 this implies the channel program might254 become modified255DOIO_SUPPRESS_INTER don't call the handler on intermediate status256========================= =============================================257 258The cpa parameter points to the first format 1 CCW of a channel program::259 260 struct ccw1 {261 __u8 cmd_code;/* command code */262 __u8 flags; /* flags, like IDA addressing, etc. */263 __u16 count; /* byte count */264 __u32 cda; /* data address */265 } __attribute__ ((packed,aligned(8)));266 267with the following CCW flags values defined:268 269=================== =========================270CCW_FLAG_DC data chaining271CCW_FLAG_CC command chaining272CCW_FLAG_SLI suppress incorrect length273CCW_FLAG_SKIP skip274CCW_FLAG_PCI PCI275CCW_FLAG_IDA indirect addressing276CCW_FLAG_SUSPEND suspend277=================== =========================278 279 280Via ccw_device_set_options(), the device driver may specify the following281options for the device:282 283========================= ======================================284DOIO_EARLY_NOTIFICATION allow for early interrupt notification285DOIO_REPORT_ALL report all interrupt conditions286========================= ======================================287 288 289The ccw_device_start() function returns:290 291======== ======================================================================292 0 successful completion or request successfully initiated293 -EBUSY The device is currently processing a previous I/O request, or there is294 a status pending at the device.295-ENODEV cdev is invalid, the device is not operational or the ccw_device is296 not online.297======== ======================================================================298 299When the I/O request completes, the CDS first level interrupt handler will300accumulate the status in a struct irb and then call the device interrupt handler.301The intparm field will contain the value the device driver has associated with a302particular I/O request. If a pending device status was recognized,303intparm will be set to 0 (zero). This may happen during I/O initiation or delayed304by an alert status notification. In any case this status is not related to the305current (last) I/O request. In case of a delayed status notification no special306interrupt will be presented to indicate I/O completion as the I/O request was307never started, even though ccw_device_start() returned with successful completion.308 309The irb may contain an error value, and the device driver should check for this310first:311 312========== =================================================================313-ETIMEDOUT the common I/O layer terminated the request after the specified314 timeout value315-EIO the common I/O layer terminated the request due to an error state316========== =================================================================317 318If the concurrent sense flag in the extended status word (esw) in the irb is319set, the field erw.scnt in the esw describes the number of device specific320sense bytes available in the extended control word irb->scsw.ecw[]. No device321sensing by the device driver itself is required.322 323The device interrupt handler can use the following definitions to investigate324the primary unit check source coded in sense byte 0 :325 326======================= ====327SNS0_CMD_REJECT 0x80328SNS0_INTERVENTION_REQ 0x40329SNS0_BUS_OUT_CHECK 0x20330SNS0_EQUIPMENT_CHECK 0x10331SNS0_DATA_CHECK 0x08332SNS0_OVERRUN 0x04333SNS0_INCOMPL_DOMAIN 0x01334======================= ====335 336Depending on the device status, multiple of those values may be set together.337Please refer to the device specific documentation for details.338 339The irb->scsw.cstat field provides the (accumulated) subchannel status :340 341========================= ============================342SCHN_STAT_PCI program controlled interrupt343SCHN_STAT_INCORR_LEN incorrect length344SCHN_STAT_PROG_CHECK program check345SCHN_STAT_PROT_CHECK protection check346SCHN_STAT_CHN_DATA_CHK channel data check347SCHN_STAT_CHN_CTRL_CHK channel control check348SCHN_STAT_INTF_CTRL_CHK interface control check349SCHN_STAT_CHAIN_CHECK chaining check350========================= ============================351 352The irb->scsw.dstat field provides the (accumulated) device status :353 354===================== =================355DEV_STAT_ATTENTION attention356DEV_STAT_STAT_MOD status modifier357DEV_STAT_CU_END control unit end358DEV_STAT_BUSY busy359DEV_STAT_CHN_END channel end360DEV_STAT_DEV_END device end361DEV_STAT_UNIT_CHECK unit check362DEV_STAT_UNIT_EXCEP unit exception363===================== =================364 365Please see the ESA/390 Principles of Operation manual for details on the366individual flag meanings.367 368Usage Notes:369 370ccw_device_start() must be called disabled and with the ccw device lock held.371 372The device driver is allowed to issue the next ccw_device_start() call from373within its interrupt handler already. It is not required to schedule a374bottom-half, unless a non deterministically long running error recovery procedure375or similar needs to be scheduled. During I/O processing the Linux/390 generic376I/O device driver support has already obtained the IRQ lock, i.e. the handler377must not try to obtain it again when calling ccw_device_start() or we end in a378deadlock situation!379 380If a device driver relies on an I/O request to be completed prior to start the381next it can reduce I/O processing overhead by chaining a NoOp I/O command382CCW_CMD_NOOP to the end of the submitted CCW chain. This will force Channel-End383and Device-End status to be presented together, with a single interrupt.384However, this should be used with care as it implies the channel will remain385busy, not being able to process I/O requests for other devices on the same386channel. Therefore e.g. read commands should never use this technique, as the387result will be presented by a single interrupt anyway.388 389In order to minimize I/O overhead, a device driver should use the390DOIO_REPORT_ALL only if the device can report intermediate interrupt391information prior to device-end the device driver urgently relies on. In this392case all I/O interruptions are presented to the device driver until final393status is recognized.394 395If a device is able to recover from asynchronously presented I/O errors, it can396perform overlapping I/O using the DOIO_EARLY_NOTIFICATION flag. While some397devices always report channel-end and device-end together, with a single398interrupt, others present primary status (channel-end) when the channel is399ready for the next I/O request and secondary status (device-end) when the data400transmission has been completed at the device.401 402Above flag allows to exploit this feature, e.g. for communication devices that403can handle lost data on the network to allow for enhanced I/O processing.404 405Unless the channel subsystem at any time presents a secondary status interrupt,406exploiting this feature will cause only primary status interrupts to be407presented to the device driver while overlapping I/O is performed. When a408secondary status without error (alert status) is presented, this indicates409successful completion for all overlapping ccw_device_start() requests that have410been issued since the last secondary (final) status.411 412Channel programs that intend to set the suspend flag on a channel command word413(CCW) must start the I/O operation with the DOIO_ALLOW_SUSPEND option or the414suspend flag will cause a channel program check. At the time the channel program415becomes suspended an intermediate interrupt will be generated by the channel416subsystem.417 418ccw_device_resume() - Resume Channel Program Execution419 420If a device driver chooses to suspend the current channel program execution by421setting the CCW suspend flag on a particular CCW, the channel program execution422is suspended. In order to resume channel program execution the CIO layer423provides the ccw_device_resume() routine.424 425::426 427 int ccw_device_resume(struct ccw_device *cdev);428 429==== ================================================430cdev ccw_device the resume operation is requested for431==== ================================================432 433The ccw_device_resume() function returns:434 435========= ==============================================436 0 suspended channel program is resumed437 -EBUSY status pending438 -ENODEV cdev invalid or not-operational subchannel439 -EINVAL resume function not applicable440-ENOTCONN there is no I/O request pending for completion441========= ==============================================442 443Usage Notes:444 445Please have a look at the ccw_device_start() usage notes for more details on446suspended channel programs.447 448ccw_device_halt() - Halt I/O Request Processing449 450Sometimes a device driver might need a possibility to stop the processing of451a long-running channel program or the device might require to initially issue452a halt subchannel (HSCH) I/O command. For those purposes the ccw_device_halt()453command is provided.454 455ccw_device_halt() must be called disabled and with the ccw device lock held.456 457::458 459 int ccw_device_halt(struct ccw_device *cdev,460 unsigned long intparm);461 462======= =====================================================463cdev ccw_device the halt operation is requested for464intparm interruption parameter; value is only used if no I/O465 is outstanding, otherwise the intparm associated with466 the I/O request is returned467======= =====================================================468 469The ccw_device_halt() function returns:470 471======= ==============================================================472 0 request successfully initiated473-EBUSY the device is currently busy, or status pending.474-ENODEV cdev invalid.475-EINVAL The device is not operational or the ccw device is not online.476======= ==============================================================477 478Usage Notes:479 480A device driver may write a never-ending channel program by writing a channel481program that at its end loops back to its beginning by means of a transfer in482channel (TIC) command (CCW_CMD_TIC). Usually this is performed by network483device drivers by setting the PCI CCW flag (CCW_FLAG_PCI). Once this CCW is484executed a program controlled interrupt (PCI) is generated. The device driver485can then perform an appropriate action. Prior to interrupt of an outstanding486read to a network device (with or without PCI flag) a ccw_device_halt()487is required to end the pending operation.488 489::490 491 ccw_device_clear() - Terminage I/O Request Processing492 493In order to terminate all I/O processing at the subchannel, the clear subchannel494(CSCH) command is used. It can be issued via ccw_device_clear().495 496ccw_device_clear() must be called disabled and with the ccw device lock held.497 498::499 500 int ccw_device_clear(struct ccw_device *cdev, unsigned long intparm);501 502======= ===============================================503cdev ccw_device the clear operation is requested for504intparm interruption parameter (see ccw_device_halt())505======= ===============================================506 507The ccw_device_clear() function returns:508 509======= ==============================================================510 0 request successfully initiated511-ENODEV cdev invalid512-EINVAL The device is not operational or the ccw device is not online.513======= ==============================================================514 515Miscellaneous Support Routines516------------------------------517 518This chapter describes various routines to be used in a Linux/390 device519driver programming environment.520 521get_ccwdev_lock()522 523Get the address of the device specific lock. This is then used in524spin_lock() / spin_unlock() calls.525 526::527 528 __u8 ccw_device_get_path_mask(struct ccw_device *cdev);529 530Get the mask of the path currently available for cdev.531