next up previous
Next: Performance of User-Space Devices Up: FUSD: A Linux Framework Previous: Implementing Blocking System Calls

Subsections


Implementing selectable Devices

One important feature that almost every sdevice driver in a system should have is support for the select(2) system call. select allows clients to assemble a set of file descriptors and ask to be notified when one of them becomes readable or writable. This simple feature is deceptively powerful--it allows clients to wait for any number of a set of possible events to occur. This is fundamentally different than (for example) a blocking read, which only unblocks on one kind of event. In this section, we'll describe how FUSD can be used to create a device whose state can be queried by a client's call to select(2).

This section is limited to a discussion what a FUSD driver writer needs to know to implement a selectable device. Details of the FUSD implementation required to support this feature are described in Section 11.1

Poll state and the poll_diff callback

FUSD's implementation of selectable devices depends on the concept of poll state. A file descriptor's poll state is a bitmask that describes its current properties--readable, writable, or exception raised. These three states correspond to select(2)'s three fd_sets. FUSD has constants used to describe these states:

These constants can be combined with C's bitwise-or operator. For example, a descriptor that is both readable and writable is expressed as FUSD_NOTIFY_INPUT | FUSD_NOTIFY_OUTPUT. 0 means a file descriptor is not readable, not writable, and not in the exception set.

For a FUSD device to be selectable, its driver must implement a callback called poll_diff. This callback is very different than the others; it is not a ``direct line'' between the client and the driver as is the case with a call such as ioctl. A driver's response to poll_diff is not the return value seen by a client's call to select. When a client tries to select on a set of file descriptors, the kernel collects the responses from all the appropriate callbacks--poll for file descriptors managed by kernel drivers, and poll_diff callbacks those managed by FUSD drivers--and synthesizes all of that information into the return value seen by the client.

FUSD keeps a cache of the poll state it has most recently received from each FUSD device driver, initially assumed to be 0. This state is returned to clients trying to select() on devices managed by those drivers. Under certain conditions, FUSD sends a query to the driver in order to ensure that the kernel's poll state cache is up to date. This query takes the form of a poll_diff callback activation, which is given a single argument: the poll state that FUSD currently has cached. The driver should consult its internal data structures to determine the actual, current poll state (i.e., whether or not buffers have readable data). Then:

In other words, when a driver's poll_diff callback is activated, the kernel is effectively saying to the driver, ``Here is what I think the current poll state of this file descriptor is; let me know when that state changes.'' The driver can either respond immediately (if the kernel's cache is already known to be out of date), or return -FUSD_NOREPLY if no update is immediately necessary. Later, when the poll state changes (for example, if new data arrives that makes a device readable), the driver can used its saved fusd_file_info pointer to send a poll state update to the kernel.

When a FUSD driver sends a poll state update, it might (or might not) have the effect of waking up a client that was blocked in select(2). On the same note, it's worth reiterating that a -FUSD_NOREPLY response to a poll_diff callback does not necessarily block the client--other descriptors in the client's select set might be readable, for example.


Receiving a poll_diff request when the previous one has not been returned yet

Calls such as read and write are synchronous from the standpoint of an individual client--a request is made, and the requester blocks until a reply is received. This means that there can't ever be more than a single read request outstanding for a single client at a time. (The driver as a whole may be keeping track of many outstanding read requests in parallel, but no two of them will be from the same client file descriptor.)

As we mentioned in the previous section, the poll_diff callback is different from other callbacks. It is not part of a synchronous request/reply sequence that causes the client to block. It is also an interface to the kernel, not directly to the client. So, it is possible to receive a poll_diff request while there is already one outstanding. This happens if the kernel's poll state cache changes, causing it to notify the driver that it has a new cached value.

This is easy to handle; the client should simply

  1. Destroy the old (now out-of-date) poll_diff request using the fusd_destroy function we saw in Section 8.2.3.
  2. Either respond to or save the new poll_diff request, exactly as described in the previous section.

The next section will show an example of this technique.

Adding select support to pager.c

Given the explanation of poll_diff in the previous sections, it might seem that implementing a selectable device is a daunting task. It's actually not as bad as it sounds--the example code may well be shorter than its explanation!


\begin{Program}
% latex2html id marker 889\listinginput[5]{1}{pager-polldiff.c...
...orting {\tt select(2)} by implementing a
{\tt poll\_diff} callback}\end{Program}

Program 14 shows the implementation of poll_diff in pager.c, which makes its notification interface (/dev/pager/notify) selectable. It is decomposed into a ``top half'' and ``bottom half'' function, exactly as we did for the blocking read implementation in Program 12. First, on lines 1-20, we see the the callback for poll_diff callback itself. It is virtually identical to the read callback in Program 12. The main difference is that it first checks (on line 12) to see if a poll_diff request is already outstanding when a new request comes in. If so, the out-of-date request is destroyed using fusd_destroy, as we described in Section 9.2.

The bottom half is shown on lines 22-46. First, on lines 32-35, it computes the current poll state--if a page has arrived that the client hasn't seen yet, the file is readable; otherwise, it isn't. Next, the driver compares the current poll state with the poll state that the kernel has cached. If the kernel's cache is out of date, the current state is returned to the kernel. Otherwise, it does nothing.

As with the read callback we saw previously, notice that pager_notify_complete_polldiff is called in two different cases:

  1. It is called immediately from the pager_notify_polldiff callback itself. This causes the current poll state to be returned to the kernel immediately when the request arrives if the driver already knows the kernel's state needs to be updated.
  2. It is called when new data arrives that causes the poll state to change. Refer back to Program 12 on page [*]; in the callback that receives new pages, notice on line 45 that the poll_diff completion function is called alongside the read completion function.

With this poll_diff implementation, it is possible for a client to open /dev/pager/notify, and block in a select(2) system call. If another client writes page to /dev/pager/input, the first client's select will unblock, indicating the file has become readable.

For additional example code, take a look at the logring example program we first mentioned in Section 8.3. It also has supports select by implementing a poll_diff callback.


next up previous
Next: Performance of User-Space Devices Up: FUSD: A Linux Framework Previous: Implementing Blocking System Calls