On-line Guides

How To Guides






Advanced Bash-Scripting Guide:
Prev Next

Chapter 22. Process Substitution

Process substitution is the counterpart to command substitution. Command substitution sets a variable to the result of a command, as in dir_contents=`ls -al` or xref=$( grep word datafile). Process substitution feeds the output of a process to another process (in other words, it sends the results of a command to another command).

Command substitution template

command within parentheses

>(command)

<(command)

These initiate process substitution. This uses /dev/fd/<n> files to send the results of the process within parentheses to another process. [1]

Note

There is no space between the the "<" or ">" and the parentheses. Space there would give an error message.

bash$ echo >(true)
/dev/fd/63
bash$ echo <(true)
/dev/fd/63
	 
Bash creates a pipe with two file descriptors, --fIn and fOut--. The stdin of true connects to fOut (dup2(fOut, 0)), then Bash passes a /dev/fd/fIn argument to echo. On systems lacking /dev/fd/<n> files, Bash may use temporary files. (Thanks, S.C.)

Process substitution can compare the output of two different commands, or even the output of different options to the same command.

bash$ comm <(ls -l) <(ls -al)
total 12
-rw-rw-r-- 1 bozo bozo 78 Mar 10 12:58 File0
-rw-rw-r-- 1 bozo bozo 42 Mar 10 12:58 File2
-rw-rw-r-- 1 bozo bozo 103 Mar 10 12:58 t2.sh
 total 20
 drwxrwxrwx 2 bozo bozo 4096 Mar 10 18:10 .
 drwx------ 72 bozo bozo 4096 Mar 10 17:58 ..
 -rw-rw-r-- 1 bozo bozo 78 Mar 10 12:58 File0
 -rw-rw-r-- 1 bozo bozo 42 Mar 10 12:58 File2
 -rw-rw-r-- 1 bozo bozo 103 Mar 10 12:58 t2.sh

Using process substitution to compare the contents of two directories (to see which filenames are in one, but not the other):

diff <(ls $first_directory) <(ls $second_directory)

Some other usages and uses of process substitution:

cat <(ls -l)
# Same as ls -l | cat
sort -k 9 <(ls -l /bin) <(ls -l /usr/bin) <(ls -l /usr/X11R6/bin)
# Lists all the files in the 3 main 'bin' directories, and sorts by filename.
# Note that three (count 'em) distinct commands are fed to 'sort'.
 
diff <(command1) <(command2) # Gives difference in command output.
tar cf >(bzip2 -c > file.tar.bz2) $directory_name
# Calls "tar cf /dev/fd/?? $directory_name", and "bzip2 -c > file.tar.bz2".
#
# Because of the /dev/fd/<n> system feature,
# the pipe between both commands does not need to be named.
#
# This can be emulated.
#
bzip2 -c < pipe > file.tar.bz2&
tar cf pipe $directory_name
rm pipe
# or
exec 3>&1
tar cf /dev/fd/4 $directory_name 4>&1 >&3 3>&- | bzip2 -c > file.tar.bz2 3>&-
exec 3>&-
# Thanks, St�phane Chazelas

A reader sent in the following interesting example of process substitution.

# Script fragment taken from SuSE distribution:
while read des what mask iface; do
# Some commands ...
done < <(route -n) 
# To test it, let's make it do something.
while read des what mask iface; do
 echo $des $what $mask $iface
done < <(route -n) 
# Output:
# Kernel IP routing table
# Destination Gateway Genmask Flags Metric Ref Use Iface
# 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
# As St�phane Chazelas points out, an easier-to-understand equivalent is:
route -n |
 while read des what mask iface; do # Variables set from output of pipe.
 echo $des $what $mask $iface
 done # This yields the same output as above.
 # However, as Ulrich Gayer points out . . .
 #+ this simplified equivalent uses a subshell for the while loop,
 #+ and therefore the variables disappear when the pipe terminates.
	
	
# However, Filip Moritz comments that there is a subtle difference
#+ between the above two examples, as the following shows.
(
route -n | while read x; do ((y++)); done
echo $y # $y is still unset
while read x; do ((y++)); done < <(route -n)
echo $y # $y has the number of lines of output of route -n
)
More generally spoken
(
: | x=x
# seems to start a subshell like
: | ( x=x )
# while
x=x < <(:)
# does not
)
# This is useful, when parsing csv and the like.
# That is, in effect, what the original SuSE code fragment does.

Notes

[1]

This has the same effect as a named pipe (temp file), and, in fact, named pipes were at one time used in process substitution.


Prev Home Next
Restricted ShellsUp Functions
Published under the terms of the GNU General Public License Design by Interspire

AltStyle によって変換されたページ (->オリジナル) /