Create a temporary FIFO (named pipe) in Python?

Question

How can you create a temporary FIFO (named pipe) in Python? This should work:

import tempfile
temp_file_name = mktemp()
os.mkfifo(temp_file_name)
open(temp_file_name, os.O_WRONLY)
# ... some process, somewhere, will read it ...

However, I'm hesitant because of the big warning in Python Docs 11.6 and potential removal because it's deprecated.

EDIT: It's noteworthy that I've tried tempfile.NamedTemporaryFile (and by extension tempfile.mkstemp), but os.mkfifo throws:

OSError -17: File already exists

when you run it on the files that mkstemp/NamedTemporaryFile have created.

• user79758: As you mention, since os.mkfifo throws an error if the file exists, there is no security hole using mkstemp or even a totally deterministic filename (besides a bad feeling and a risk of someone naively copying your code).
• Steven Huwig: Question: How do you plan to communicate the name of the pipe to the other processes? How is this channel secured?
• Brian M. Hunt: @Joe: You're right. I hadn't thought of that. @Steven Huwig: Both processes are spawned by the same parent, though I might at some point move to a new fifo for that, too. Why do you ask?

Answer 1

os.mkfifo() will fail with exception OSError: [Errno 17] File exists if the file already exists, so there is no security issue here. The security issue with using tempfile.mktemp() is the race condition where it is possible for an attacker to create a file with the same name before you open it yourself, but since os.mkfifo() fails if the file already exists this is not a problem.

However, since mktemp() is deprecated you shouldn't use it. You can use tempfile.mkdtemp() instead:

import os, tempfile
tmpdir = tempfile.mkdtemp()
filename = os.path.join(tmpdir, 'myfifo')
print filename
try:
 os.mkfifo(filename)
except OSError, e:
 print "Failed to create FIFO: %s" % e
else:
 fifo = open(filename, 'w')
 # write stuff to fifo
 print >> fifo, "hello"
 fifo.close()
 os.remove(filename)
 os.rmdir(tmpdir)

EDIT: I should make it clear that, just because the mktemp() vulnerability is averted by this, there are still the other usual security issues that need to be considered; e.g. an attacker could create the fifo (if they had suitable permissions) before your program did which could cause your program to crash if errors/exceptions are not properly handled.

• Artelius: Should the os.rmdir(tmpdir) not be outside the the try-else block?
• Tim Dierks: You mean except OSError as e.
• Tobias: @TimDierks: It is true that the new syntax is more readable and the only one supported in Python 3, but the old one is valid for all Python 2.x versions, and the new one needs at least Python 2.6.

Answer 2

You may find it handy to use the following context manager, which creates and removes the temporary file for you:

import os
import tempfile
from contextlib import contextmanager
@contextmanager
def temp_fifo():
 """Context Manager for creating named pipes with temporary names."""
 tmpdir = tempfile.mkdtemp()
 filename = os.path.join(tmpdir, 'fifo') # Temporary filename
 os.mkfifo(filename) # Create FIFO
 try:
 yield filename
 finally:
 os.unlink(filename) # Remove file
 os.rmdir(tmpdir) # Remove directory

You can use it, for example, like this:

with temp_fifo() as fifo_file:
 # Pass the fifo_file filename e.g. to some other process to read from.
 # Write something to the pipe 
 with open(fifo_file, 'w') as f:
 f.write("Hello\n")

Answer 3

How about using

d = mkdtemp()
t = os.path.join(d, 'fifo')

• Brian M. Hunt: That suffers the same security issues as mktemp. Thanks though, Brendan.
• user79758: It only has the same security issue if the user has already compromised the account running the script (the directory is 0700). In which case, they can probably do much worse.
• Brian M. Hunt: Oop - I was wrong. This doesn't have the same security issue as mktemp because mkfifo() balks if the file exists (i.e. there's no chance for a MiM attack). This would work.

Answer 4

If it's for use within your program, and not with any externals, have a look at the Queue module. As an added benefit, python queues are thread-safe.

• Brian M. Hunt: Thanks Nilamo. Unfortunately it's not for internal communication. I'm using Queues elsewhere, though. Thanks for the suggestion.
• nilamo: Not a problem, just throwing the 'use the simplest possible thing that could work' flag. If you actually need the fifo, then this answer doesn't help at all.

Answer 5

Effectively, all that mkstemp does is run mktemp in a loop and keeps attempting to exclusively create until it succeeds (see stdlib source code here). You can do the same with os.mkfifo:

import os, errno, tempfile
def mkftemp(*args, **kwargs):
 for attempt in xrange(1024):
 tpath = tempfile.mktemp(*args, **kwargs)
 try:
 os.mkfifo(tpath, 0600)
 except OSError as e:
 if e.errno == errno.EEXIST:
 # lets try again
 continue
 else:
 raise
 else:
 # NOTE: we only return the path because opening with
 # os.open here would block indefinitely since there 
 # isn't anyone on the other end of the fifo.
 return tpath
 else:
 raise IOError(errno.EEXIST, "No usable temporary file name found")

Answer 6

Why not just use mkstemp()?

For example:

import tempfile
import os
handle, filename = tempfile.mkstemp()
os.mkfifo(filename)
writer = open(filename, os.O_WRONLY)
reader = open(filename, os.O_RDONLY)
os.close(handle)

• Brian M. Hunt: os.mkfifo throws an OSError 17: File Exists. Thanks though.