All of the example drivers that we've seen until now have had an important feature missing: they never had to wait for anything. So far, a driver's response to a system call has always been immediately available--allowing the driver to response immediately. However, real devices are often not that lucky: they usually have to wait for something to happen before completing a client's system call. For example, a driver might be waiting for data to arrive from the serial port or over the network, or even waiting for a user action.
In situations like this, a basic capability most device drivers must have is the ability to block the caller. Blocking operations are important because they provide a simple interface to user programs that does flow control, rather than something more expensive like continuous polling. For example, user programs expect to be able to execute a statement like read(fd, buf, sizeof(buf)), and expect the read call to block (stop the flow of the calling program) until data is available. This is much simpler and more efficient than polling repeatedly.
In the following sections, we'll describe how to block and unblock system calls for devices that use FUSD.
The easiest way to block a client's system call is simply to block the driver, too. For example, consider Program 10, which implements a device called /dev/console-read. Whenever a process tries to read from this device, the driver prints a prompt to standard input, asking for a reply. (The prompt appears in the shell the driver was run in, not the shell that's trying to read from the device.) When the user enters a line of text, the response is returned to the client that did the original read(). By blocking the driver waiting for the reply, the client that issued the system call is blocked as well.
Blocking the driver this way is safe--unlike programming in the kernel proper, where doing something like this would block the entire system. It's also easy to implement, as seen from the example above. However, it makes the driver unresponsive to system call requests that might be coming from other clients. This limitation makes blocking drivers inappropriate for any device driver that expects to service more than one client at a time.
If a device driver expects more than one client at a time--as is often the case--a slightly different programming model is needed for system calls that can potentially block. Instead of blocking, the driver immediately sends a message to the FUSD framework that says, in essence, ``Don't unblock the client that issued this system call, but continue sending additional system call requests that might be coming from other clients.'' Driver callbacks can send this message to FUSD by returning the special value -FUSD_NOREPLY instead of a normal system call return value.
Before a callback blocks the caller by returning -FUSD_NOREPLY, it must save the fusd_file_info pointer that was provided to the callback as its first argument. Later, when an event occurs which allows the client's blocked system call to complete, the driver should call fusd_return(), which will unblock the calling process and complete its system call. fusd_return() takes two arguments:
Drivers should never call fusd_return more than once on a single fusd_file_info pointer. Doing so will have undefined results, similar to calling free() twice on the same pointer.
It also bears repeating that a callback can call either call fusd_return() explicitly or return a normal return value (i.e., not -FUSD_NOREPLY), but not both.
-FUSD_NOREPLY and fusd_return() make it easy for a driver to block a process, then unblock it later when data becomes available. When the callback returns -FUSD_NOREPLY, the driver is freed up to wait for other events, even though the process making the system call is still blocked. The driver can then wait for something to happen that unblocks the original caller--for example, another FUSD event, data from a serial port, or data from the network. (Recall from Section 7 that a FUSD driver can simultaneously wait for both FUSD and non-FUSD events.)
FUSD includes an example program, pager.c, which demonstrates these techniques. The pager driver implements a simple notification interface which lets any number of ``waiters'' wait for a signal from a ``notifier.'' All the waiters wait by trying to read from /dev/pager/notify. Those reads will block until a notifier writes the string page to /dev/pager/input. It's easy to try the application out--run the driver, and then open three other shells. In two of them, type cat /dev/pager/notify. The reads will block. Then, in the third shell, type echo page > /dev/pager/notify--the other two shells should become unblocked.
Let's take a look at how this application is implemented, step by step.
The first thing to notice about pager.c is that it keeps per-client state. That is, for every file descriptor open to the driver, a structure is allocated that has information relating to that file descriptor. Previous driver examples were, for the most part, reactive--they received requests, and immediately generated responses. Since there was never more than one request outstanding, there was no need to keep a list of them. The pager application is the first one that must keep track of an arbitrary number of requests that might be outstanding at the same time. The first excerpt of pager.c, which appears in Program 11, shows the code which creates this per-client state. Lines 1-6 define a structure, pager_client, which keeps all the information we need about each client attached to the driver. The open callback for /dev/pager/notify, shown on lines 12-31, allocates memory for an instance of this structure and adds it to a linked list. (If the memory allocation fails, an error is returned to the client on line 18; this will prevent the file from opening.) Note on line 25 that we use the private_data field to store a pointer to the client state; this allows the structure to be retrieved when later callbacks on this file descriptor arrive. The memory is deallocated when the file is closed; we'll see that in a later section.
Another thing to notice about the open callback is the use of the last_page_seen variable. The driver gives a sequence number to every page it receives; last_page_seen stores the number of the most recent page seen by a client. When a new client arrives (i.e., it opens /dev/pager/notify), its last_page_seen state is set equal to the page that has most recently arrived; this forces a new client to wait for the next page, rather than immediately being notified of a page that has arrived in the past.
The next part of pager.c is shown in Program 12. The pager_notify_read function seen on line 1 is registered as the read callback for the /dev/pager/notify device. It blocks the read request using the technique we described earlier: it stores the fusd_file_info pointer in that client's state structure, and returns -FUSD_NOREPLY. (Note that the pointer to the client's state structure comes from the private_data field of fusd_file_info, where the open callback stored it.)
pager_notify_complete_read unblocks previously blocked reads. This function first checks to see that there is, in fact, a blocked read (line 19). It then checks to see if a page has arrived that the client hasn't seen yet (line 23). Finally, it updates the client state and unblocks the blocked read by calling fusd_return. Note the second argument to fusd_return is a 0; as we saw in Section 4.3, a 0 return value to a read system call means EOF. (The system call will be unblocked regardless of the return value.)
pager_notify_complete_read is called every time a new page arrives. New pages are processed by pager_input_write (line 34), which is the write callback for /dev/pager/input. After recording the fact that a new page has arrived, it calls pager_notify_complete_read for each client that has an open file descriptor. This will complete the reads of any clients who have not yet seen this new data, and have no effect on clients that don't have outstanding reads.
There is another interesting point to notice about pager_notify_read. On line 12, after it stores the blocked system call's pointer, but before we return -FUSD_NOREPLY, it calls the completion function. This has the effect of returning any data that might already be available back to the caller immediately. If that happens, we will end up calling fusd_return before we return -FUSD_NOREPLY. This probably seems strange, but it's legal. Recall that a callback can call fusd_return() explicitly or return a normal (not -FUSD_NOREPLY) return value, but not both; the order doesn't matter.
Finally, let's take a look at one last aspect of the pager program: how it cleans up the per-client state when a client leaves. This is mostly straightforward, with one exception: a client may have an outstanding read request out when a close request comes in. Normally, clients can't make system call request while a previous system call is still blocked. However, the close system call is an exception: it gets called when a client dies (for example, if it receives an interrupt signal). If a close comes in while another system call is still outstanding, the state associated with the outstanding request should be freed to avoid a memory leak. The fusd_destroy function is used to do this, seen on linen 12-14 of Program 13.
In the previous section, we showed how the fusd_return function can be used to specify the return value of a system call that was previously blocked. However, many system calls have side effects in addition to returning a value--for example, in a read() request, the data being returned has to be copied into the caller's buffer. To facilitate this, FUSD provides accessor functions that let drivers retrieve the arguments that had been passed to its callbacks at the time the call was originally issued. For example, the fusd_get_read_buffer() function will return a pointer to the data buffer that is provided with read() callbacks. Drivers can use these accessor functions to affect change to a client before calling fusd_return().
The following accessor functions are available, all of which take a single fusd_file_info * argument:
We got away without using these accessor functions in our pager.c example because the pager doesn't actually return data--it just blocks and unblocks read calls. However, the FUSD distribution contains another example program, logring, that demonstrates their use.
logring makes it easy to access the most recent (and only the most recent) output from a process. It works just like tail -f on a log file, except that the storage required never grows. This can be useful in embedded systems where there isn't enough memory or disk space for keeping complete log files, but the most recent debugging messages are sometimes needed (e.g., after an error is observed).
logring uses FUSD to implement a character device, /dev/logring, that acts like a named pipe that has a finite, circular buffer. The size of the buffer is given as a command-line argument. As more data is written into the buffer, the oldest data is discarded. A process that reads from the logring device will first read the existing buffer, then block and see new data as it's written, similar to monitoring a log file using tail -f.
You can run this example program by typing logring <logsize>, where logsize is the size of the circular buffer in bytes. Then, type cat /dev/logring in a shell. The cat process will block, waiting for data. From another shell, write to the logring (e.g., echo Hi there > /dev/logring). The cat process will see the message appear.
(This example program is based on emlog, a (real) Linux kernel module with identical functionality. If you find logring useful, but want to use it on a system that does not have FUSD, check out the original emlog.)