Linux Classes
Linux Classes
Share This With a Friend
LINUX CLASSES - LINUX BASICS

Will Linux Pipelines Make Me Filthy Rich?

Pumping a Program's Output into Another Program Linux provides you with a wide array of utilities to manipulate data. You can search, sort, slice, dice, and transform data stored in files in many different ways. A pipe (also called a pipeline) is a powerful shell feature that allows you to p ump the output of one program directly into another.

For example, say you have a file with information about a group of people, including a name, age, zip code, and phone number for each person, that looks like this:

Roosevelt Tommy 38 54579 555-1212
Nixon Edward 19 37583 246-3457
Roosevelt Freddie 47 11745 674-6972
Lincoln Albert 26 26452 916-5763

If you wanted to find all the Roosevelts and sort them by zip code, you could do it like this:

grep Roosevelt people.txt > grep.out
sort +3 grep.out
rm grep.out

Since I haven't introduced the grep and sort commands yet, here's an English translation of what's happening above:

Look for lines that contain Roosevelt in the people.txt file and put them in a file named grep.out. Then sort the grep.out file on the fourth column and display the results on the console befor e deleting the grep.out file. (Yes, it is odd that the +3 flag tells sort to use the fourth column!)

But you could avoid creating and deleting the intermediate file (grep.out) by combining the operation into one command like this:

grep Roosevelt people.txt | sort +3

The vertical bar tells the shell that the output of the program on the left (grep) should become the input for the program on the right (sort). Behind the scenes, the shell may be issuing the exact same three commands as in the previous example, but you don't really care--you've combined three commands into one.

You can have any number of steps in a pipeline, and you can even combine pipes with redirection, as shown here:

grep Roosevelt people.txt | sort +3 > sort-results

Here, the same thing happens, except that the end result is stored in a file called sort-results.

Previous Lesson: Redirection
Next Lesson: Processes

[ RETURN TO INDEX ]



Comments - most recent first
(Please feel free to answer questions posted by others!)

Don Kalyan (18 Nov 2012, 12:49)
Thanks a Ton...Bob..Nice little explanation..
Oniex (02 Mar 2012, 06:44)
thanks a lot...... avoided weight gain just by absorbing low fat linux tuts..........gr8 work.....hope you can come up with many more linux/unix based applications and services....
Mike (19 Feb 2012, 21:45)
Just wanted to say thanks for putting in what must have been a HUGE amount of time and work to make this. I am just starting with Linux/Unix and you have made it nearly painless!
cyril (17 Feb 2012, 01:48)
try grep Roosevelt people.txt | sort -k 3
Felipe (13 Feb 2012, 05:03)
This command is not working
open failed: +3 No such file or directory

Thanks for the Tutorial, is great
ram (06 Jul 2011, 07:38)
this command is not working and giving error open failed. kindly post the reason for this pls!
oasg (07 Apr 2011, 23:10)
Yes! Now I am understanding.
Thank you very much Dr. Bob!
Ulrich (31 Mar 2011, 13:32)
Hi,
I've got the same error:

open failed: +3: No such file or directory
Jelloso (19 Oct 2010, 11:07)
You're the best Dr. Bob!!! Everything I ever need to pass my Linux subject is right here!!!
aldiazo (10 Sep 2010, 16:53)
I am trying to do a telnet, run a shell to extrac data from an informix DB, and redirect de output to a text file...

Who can I do that?
Kevin (27 Jun 2010, 16:40)
When piped the sort command (sort +3) gave me the error message:
open failed: +3: No such file or directory
When you add a -4 (as the second position flag) the command works. Do you Bob still use this syntax or do you now use the -k flag?
umakanta (25 Jun 2010, 08:33)
thanks .. information is everything...
kapil pateria (08 Apr 2010, 04:40)
nformation is really very useful and in a very concise manner. best for a novice begginer.
Dennis (25 Feb 2010, 08:57)
thanks.
I just enter the linux world, and the information is very accessible.
Bob Rankin (25 Feb 2010, 08:30)
@Dennis, those commands are equivalent. Here's the only difference: In the first case, grep is passed the name of the input file, and grep opens the file. In the second, grep is not passed a filename, and reads from the standard input stream. Not really important from a user perspective, but that's the scoop.
Dennis (25 Feb 2010, 08:12)
grep Roosevelt people.txt | sort +3

does it overrides ?
grep Roosevelt < people.txt | sort +3

< - is missing

I welcome your comments. However... I am puzzled by many people who say "Please send me the Linux tutorial." This website *is* your Linux Tutorial! Read everything here, learn all you can, ask questions if you like. But don't ask me to send what you already have. :-)

NO SPAM! If you post garbage, it will be deleted, and you will be banned.
*Name:
Email:
Notify me about new comments on this page
Hide my email
*Text:




Copyright © by - Privacy Policy
All rights reserved - Redistribution is allowed only with permission.

Popular Linux Topics

Linux Intro
Linux Files
Linux Commands
Change Password
Copy Files
Linux Shell Basics

Linux Tutorial

Who is Doctor Bob?
What is Linux?
History of Unix
Operating Systems
What's Next?

Linux Basics

Living in a Shell
Root and Other Users
Virtual Consoles
Logoff and Shutdown
Choosing a Shell
The Command Prompt
Wildcards
Command History
Aliases
Redirection
Pipelines
Processes
Stopping a Program
Environment Variables
Help!

Linux Files

The Linux File System
Linux File Names
Linux Directories
Directory Terminology
Navigating the File System
Listing Linux Files
Displaying Linux Files
Copying and Renaming Files
Creating Files and Directories
Deleting Files and Directories
Linux Files - Wildcards
The Nine Deadly Keystrokes
Linux File Permissions
Changing File Permissions

Linux Commands

Important Linux Commands
Changing Your Password
Switching Users
Who is Logged In?
Date and Time
The Echo Command
Spell Checking
Printing Linux Files
Joining Files
Searching for Files
Comparing Files
Task Scheduling
Linking Files

Linux Editors

The Vi Editor
The Emacs Editor
The Pico Editor

Linux Data Manipulation

Slicing & Dicing
Heads or Tails?
Sorting Data
Eliminating Duplicates
Selecting Columns
Selecting Records
Search & Replace
Crunching Data
Finding Files
Pipe Fitting

Linux Shell Programming

Linux Shell Scripts
Executing a Script
Shell Script Variables
Shell Script Logic
Shell Script Looping
Shell Script Debugging

Perl Programming

Perl Basics
Perl Variables
Perl Arguments
Perl Logic
Perl Looping
Perl and Files
Perl Pattern Matching

Linux and Email

Sending Email
Reading Email
Other Mail Commands
Using Pine for Email
The Pine Inbox
Pine Email Basics
Pine Email Folders
Pine for Power Users

Compression and Encoding

Linux File Compression
Archiving With Tar
Compression With Gzip
Compress and Zcat
Zmore and Zless
Zip and Unzip
Encoding and Decoding
Encryption

Linux Does DOS

Accesing DOS Files
Accesing DOS Partitions
Running DOS Programs

Managing Linux

Updating Your Linux System
Installing Packages with RPM
Uninstalling Packages w/ RPM
Upgrading Packages with RPM
Querying Packages with RPM

AltStyle によって変換されたページ (->オリジナル) /