Using Named Pipes (FIFOs)


Written 4/2006 by Wayne Pollock, Tampa Florida USA

One of best innovations of Unix was the ease in which two processes could communicate.  Normally, a process would use a complex method to pass data to another process, such as using a file on the disk, shared memory, or IPC methods (which vary from one flavor of Unix to another).  But with Unix another way was provided, the pipe.

Using a pipe is simple.  To have process A send data to process B, you merely start the two processes at the same time like this:

program-A | program-B

This starts process-A and process-B at the same time.  Process-B waits for process-A to send it data, which it then reads.  When process-A ends, process-B has reached the end of the data to read and finishes.

While this work very well there are two cases where you can't use a pipe:

  1. If two or more processes want to send data to a single process.
  2. If the processes must be started at different times (or by different users).

To solve these problems Unix (and of course Linux) provide a named pipe.  This is a pipe that sits on the disk and has a regular file name.  Using a named pipe is easy, you can redirect output to it and input from it, as if it were an ordinary file.  You can start the process that reads from the named pipe before you start the process that writes to it.  You can also have multiple processes write to the same named pipe. 

The data is read from the named pipe in the order it is written.  This behavior is known as first in, first out order.  Because of this named pipes have another name, a FIFO.  In fact the command to create one is called mkfifo.  Here's a trivial example:

[user@localhost ~] who | sort
auser    pts/1        Jun 17 12:20 (
user     pts/0        Jun 17 11:24 (

[user@localhost ~] mkfifo fifo
[user@localhost ~] ls -l fifo
prw-r-----  1 user user 0 Jun 17 12:20 fifo

[user@localhost ~] who > fifo &
[1] 12153
[user@localhost ~] sort < fifo
auser    pts/1        Jun 17 12:20 (
user     pts/0        Jun 17 11:24 (
[1]+  Done                    who >fifo

[user@localhost ~] sort < fifo &
[1] 12154
[user@localhost ~] who > fifo &
auser    pts/1        Jun 17 12:20 (
user     pts/0        Jun 17 11:24 (

A FIFO doesn't buffer or store the data sent to it, so it takes very little RAM.  Instead, when a reader process attempts to read from a FIFO before another process is connected to the sending side, the reader process will block.  Similarly if a sending process attempts to send output to the FIFO before a reader process is connected to the receiving side, it will block.  Only when both sides of the named pipe are connected to processes can any data flow.  You can see this behavior in this example:

[user@localhost ~] date
Sat Jun 17 12:51:47 EDT 2006
[user@localhost ~] date > fifo &
[1] 12329
[user@localhost ~] sleep 5
[user@localhost ~] date
Sat Jun 17 12:52:03 EDT 2006
[user@localhost ~] cat < fifo
Sat Jun 17 12:52:06 EDT 2006
[1]+  Done                    date >fifo

Here you can clearly see that the date command didn't run (that is, it was blocked) until the cat command was run.

Consider one last example showing several processes sending data to a single reader process using a FIFO:

[user@localhost ~] cal > fifo &
[1] 12209
[user@localhost ~] date > fifo &
[2] 12210
[user@localhost ~] cat < fifo
Sat Jun 17 12:32:16 EDT 2006
     June 2006
Su Mo Tu We Th Fr Sa
             1  2  3
 4  5  6  7  8  9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30

[1]-  Done                    cal >fifo
[2]+  Done                    date >fifo
[user@localhost ~] 

The last example above shows an unexpected behavior for something called a FIFO: the output isn't in the order it was sent!  The problem is the two sending processes are both blocked initially.  Once the reader process is connected to the named pipe, all sender processes wake up.  However the kernel does not run processes in any particular order, so there is no guarantee of FIFO behavior with multiple senders.  Indeed the output of two (or more) sender processes can become mixed together in some cases.

Named pipes have a number of system administration uses.  The most common example is to allow server processes (daemons) to send log messages to the system logger process.  During boot up it isn't feasible to use a regular pipe to connect each daemon to syslogd so a named pipe is used, often called /dev/log.  First syslogd is started to read from this FIFO.  Then the daemons redirect their log messages to that same FIFO.