From sac-list-owner Wed Nov 29 21:20:24 2000 From: Shudong.Z.@eng.sun.com Date: Wed, 29 Nov 2000 21:21:20 -0800 (PST) To: psarc@eng.sun.com Cc: solaris-io@devi.eng.sun.com Subject: PSARC/2000/532 -- Devinfo State Model (devfs #1) Content-Length: 8246 I am sponsoring the following fasttrack for myself. The timer is set to expire on Dec. 6, 2000. Shudong #ident @(#)devinfo.txt 1.2 00/11/29 State Model for Device Information Nodes Shudong Z. 1. Background ------------- Device information (devinfo) nodes are a fundamental entity in Solaris I/O configuration. In the current implementation, the principle devinfo node states are "proto", CF1, and CF2, where CF stands for Canonical Form. A proto node is a newly allocated node. CF1 indicates that parent nexus has completed implementation specific initialization and assigned a bus address to the node. CF2 means that a driver is attached. 2. The Problem -------------- The CF1 and CF2 states are no longer sufficient for describing devinfo nodes, whose states are changed by DR. It is not possible to tell from the current state if a driver's attach(9e) routine returned success or if the post-attach processing, such as DACF processing (PSARC 1998/212), has completed. In addition, the state information is not explicitly maintained by the DDI framework. It is only implied by applying DDI_CF1() and DDI_CF2() macros. Such a treatment results in bugs which are difficult to debug (see 4390495 and 4366721). 3. The Proposal --------------- We propose a new devinfo state model with the following states: o DS_PROTO The node has just been allocated. o DS_LINKED The node has been linked into the device tree. o DS_BOUND The node is bound to a driver and linked into the per-driver list. o DS_INITIALIZED The parent nexus has assigned a bus address and completed implementation specific initializations. o DS_PROBED The driver's probe(9e) routine has been invoked and it returned success. o DS_ATTACHED The driver's attach(9e) routine has been invoked and it returned success. o DS_READY Post attach processing, including DACF, has been completed. Device is ready to be opened by application. This proposal covers interfaces listed in the following table. Several private interfaces are proposed to fix problems in the existing implementation and to export the state model to userland. See section 4 for details of how these interfaces are used. -------------------------------------------------------------------- Interface Name Comm. Level Comments -------------------------------------------------------------------- typedef enum Cons. Private enumerated devinfo state ddi_node_state_t { DS_PROTO, DS_LINKED, DS_BOUND, DS_INITIALIZED, DS_PROBED, DS_ATTACHED, DS_READY }; di_node_state Cons. Private get devinfo state via libdevinfo i_ddi_node_state "" get devinfo state from kernel modctl(MODREMMAJBIND) Proj. Private Invalidate a driver -------------------------------------------------------------------- 4. Technical Details -------------------- 4.1 The dev_info node life cycle The devinfo node life cycles are different for persistent nodes and non-persistent nodes. Persistent nodes include OBP nodes and SID (self-identifying) nodes created by DR drivers. Non-persistent nodes are nodes created from driver.conf files and are destroyed when the driver module is unloaded. 4.1.1 Node creation A node may be created in one of three ways: o by boot code, which copies the OBP device tree to the kernel o by nexus drivers, which allocates SID node for devices discovered o by framework, which instantiates pseudo nodes from driver.conf entries. Once created (DS_PROTO), the node is always linked into the device tree (DS_LINKED) and bound to a driver (DS_BOUND), if a driver binding is available. 4.1.2 Node configuration Nodes in DS_LINKED state cannot be configured until a driver for the node is added to the system. Nodes in DS_BOUND state may be configured in response to a device open or a device configuration command such as devfsadm(1M) and cfgadm(1M). A successful configuration will transition the node to DS_READY state. The node state upon a configuration failure depends on the node persistence and type of failure: o persistent nodes are placed in DS_BOUND state on any failure o non-persistent nexus nodes are always removed on any failure o non-persistent leaf nodes are removed if they fail to initialize; otherwise, they are left in DS_INITIALIZED state so deferred attach may reconfigure the node. 4.1.3 Node unconfiguration A node in DS_READY state may be unconfigured as a result of module unloading or DR. If any part of unconfigure fails, the node is putback in the DS_READY state. If unconfigure succeeds, the final state depends on the type of node and the operation: o If unconfigure occurs as part of module unloading, persistent nodes are placed in DS_BOUND state and non-persistent nodes are destroyed. o If unconfigure occurs as part of DR removal operation, all nodes are destroyed. 4.2 Driver addition and removal When a driver is added to the system via add_drv(1M), nodes binding to the new driver are transitioned from DS_LINKED state to DS_BOUND state. This is accomplished by modifying the existing system call modctl(MODADDMAJBIND). When a driver is removed from the system by rem_drv(1M), non-persistent nodes are destroyed as part of unloading driver module. Persistent nodes are transitioned to DS_LINKED state. The latter is accomplished by adding a new modctl command int modctl(MODREMMAJBIND, major_t); and modify rem_drv(1M) to call this interface. 4.3 Relationship with hotplug device state PSARC 1993/687 introduced device hotplug states. The hotplug states are orthogonal to devinfo node state, except for the following interaction: A device in OFFLINE state may not have a devinfo state above DS_INITIALIZED. This interaction is managed by the framework as follows: - attempting to configure a devinfo node beyond DS_INITIALIZED will fail if the device marked OFFLINE. - ndi_devi_offline() will not mark device OFFLINE unless the devinfo node state is brought down to DS_INITIALIZED or lower. 4.2 Exposure of devinfo state model The devinfo state model is primarily consumed by the DDI framework. Nexus drivers supporting hotplug operations may need to be aware of the state model. Leaf drivers are not exposed to the state model. The state of a devinfo node is retrieved from the kernel by a new interface: ddi_node_state_t i_ddi_node_state(dev_info_t *); From userland, the state can be obtained from a new libdevinfo libdevinfo interface: ddi_node_state_t di_node_state(di_node_t); 5. Manpages ----------- 5.1 i_ddi_node_state(9f) SYNOPSIS #include ddi_node_state_t i_ddi_node_state(dev_info_t *dip); DESCRIPTION i_ddi_node_state() returns the state of a devinfo node. The return value is one of the following: DS_PROTO The node has just been allocated DS_LINKED The node has been linked into the device tree and, if a persistent node, the orphanlist as well. DS_BOUND The node is bound to a driver and linked into the per-driver list. DS_INITIALIZED The parent nexus has assigned a bus address and completed implementation specific initializations. DS_PROBED The driver's probe routine has been invoked and it returned success. DS_ATTACHED The driver's attach routine has been invoked and it returned success. DS_READY Post attach processing, including DACF, has been completed. Device is ready to be opened. 5.2 di_node_state(3LIBDEVINFO) SYNOPSIS #include ddi_node_state_t di_node_state(di_node_t node); DESCRIPTION di_node_state() returns the state of a devinfo node in the libdevinfo snapshot. See 5.1 for a list of possible states. 5.5 modctl(MODREMMAJBIND) SYNOPSIS #include int modctl(MODREMMAJBIND, major_t major); DESCRIPTION modctl(MODREMMAJBIND) invalidates driver major and the associated driver.conf entries cached in the kernel. This interface is used by rem_drv(1M) to prevent devices from binding to a removed driver. From sac-list-owner Thu Nov 30 15:57:19 2000 Date: Thu, 30 Nov 2000 15:58:14 -0800 (PST) From: Barry G. Subject: Re: PSARC/2000/532 -- Devinfo State Model (devfs #1) To: psarc@eng.sun.com, Shudong.Z.@eng.sun.com Cc: solaris-io@devi.eng.sun.com Content-Type: text X-Sun-Text-Type: ascii Content-Length: 1087 I still don't see the need for the DS_PROTO state as separate from DS_LINKED. This issue was raised in a previous round of reviews and I don't recall an answer to it. When the node is allocated, no drivers or other code can access it until it has been linked into the device tree. Therefore it seems to me that only the creator of the node can ever see it in the DS_PROTO state, and the creator already knows that it hasn't been linked yet. Comments? Shudong wrote: > -------------------------------------------------------------------- > Interface Name Comm. Level Comments > -------------------------------------------------------------------- > typedef enum Cons. Private enumerated devinfo state > ddi_node_state_t { > DS_PROTO, > DS_LINKED, > DS_BOUND, > DS_INITIALIZED, > DS_PROBED, > DS_ATTACHED, > DS_READY > }; -------- Barry G. Public Key signature: BA 2A 50 53 BC 6E 9A DB 9F 25 5B 13 1A B2 47 12 Filk Hall of Fame, 1997. Interfilk guest, Ohio Valley Filk Fest, 2000. Nominee, Johnstone award, 2000. I don't make policy for Sun. From sac-list-owner Thu Nov 30 16:29:36 2000 Date: Thu, 30 Nov 2000 16:30:31 -0800 (PST) From: Shudong Z. Subject: Re: PSARC/2000/532 -- Devinfo State Model (devfs #1) To: psarc@eng.sun.com, Shudong.Z.@eng.sun.com, Barry.G.@west.sun.com Cc: solaris-io@devi.eng.sun.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: JmBPrBGNlfcr596obvGkBA== Content-Length: 1186 > > I still don't see the need for the DS_PROTO state as separate from DS_LINKED. > This issue was raised in a previous round of reviews and I don't recall an > answer to it. > > When the node is allocated, no drivers or other code can access it until it has > been linked into the device tree. Therefore it seems to me that only the > creator of the node can ever see it in the DS_PROTO state, and the creator > already knows that it hasn't been linked yet. > > Comments? > First, you have set the state of newly allocated nodes to something. It doesn't make sense to set it to DS_LINKED when it really isn't linked. The framework does see DS_PROTO state when it is asked to link a node into the tree, although other drivers won't. Second, some drivers may link DS_PROTO nodes into the device tree without going through the framework (e.g. pci configurator and efcode introduced private functions for inserting nodes into the device tree, bad bad...). Having a separate state will allow us to catch such cases. Third, even if all drivers behave well, you can still find unlinked nodes in core dumps. Having a DS_PROTO state will make core dump analysis easier. Shudong From sac-list-owner Wed Dec 13 10:38:02 2000 Date: Wed, 13 Dec 2000 10:39:02 -0800 (PST) From: Shudong Z. Subject: Re: PSARC/2000/532 -- Devinfo State Model (devfs #1) To: psarc@eng.sun.com Cc: solaris-io@devi.eng.sun.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: oblLXl8DHRSeddo0uosATw== Content-Length: 59 This case was approved at today's PSARC meeting. Shudong From sacadmin Mon Nov 4 15:29:11 2002 Date: Mon, 04 Nov 2002 16:22:52 -0700 From: Chris Horne User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.0.1) Gecko/20020920 Netscape/7.0 X-Accept-Language: en-US, en-GB, en MIME-Version: 1.0 To: psarc@sac.eng.sun.com, io-frameworks@sun.com Subject: PSARC/2000/532 -- Devinfo State Model (S10 implementation adendum) Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Length: 346 There is a clarification needed to cover the state model operation with devfs in S10: In S10_13 and beyond if you successfully unload a driver then all devinfo nodes associated with the driver, both persistent and non-persistent, are taken to DS_INITIALIZED. Non-persistent nodes taken below DS_INITIALIZED will be destroyed.