IO REDIRECTION:

Introduction

For those of you’ll who have no idea what Redirection means, let me explain it in a few words. Whenever you run a program you get some output at the shell prompt. In case you don’t want that output to appear in the shell window, you can redirect it elsewhere. you can make the output go into a file…or maybe go directly to the printer.. or you could make it disappear

This is known as Redirection. Not only can the output of programs be redirected, you can also redirect the input for programs. I shall be explaining all this in detail in this article. Lets begin…

File Descriptors

One important thing you have to know to understand Redirection is file descriptors. In Unix every file has a no. associated with it called the file descriptor. And in Unix everything is a file. Right from your devices connected to your machine to the normal text files storing some information – all of these are looked at, as files by the Operating System.

Similarly even your screen on which your programs display their output are files for Unix. These have file descriptors associated with it. So when a program actually executes it sends its output to this file descriptor and since this particular file descriptor happens to be pointing to the screen, the output gets displayed on the screen. Had it been the file descriptor of the printer, the output would have been printed by the printer. (There are ofcourse other factors which come into play, but I guess you got the idea of how everything is a file and you send whatver you want to particular files descriptors)

Whenever any program is executed (i.e. when the user types a command) the program has 3 important files to work with. They are standard input, standard output, and standard error. These are 3 files that are always open when a program runs. You could kind of consider them to be inherently present for all programs (For the techies.. basically when a child process is forked from a parent process, these 3 files are made available to the child process). For the rest, just remember that you always have these 3 files with you whenever you type any command at the prompt. As explained before a file descriptor, is associated with each of these files -

You could redirect any of these files to other files. In short if you redirect 1 (standard output) to the printer, your programs output would start getting printed instead of being displayed on the screen.

What is the standard input? That would be your keyboard. Most of the times since you enter commands with your keyboard, you could consider 0 to be your keyboard. Since you get the output of your command on the screen, 1 would be the screen (display) and the errors as well are shown on the screen to you, so 2 would also be the screen.

For those of you’ll who like to think ahead of what is being discussed… you’ll must have already understood that you can now avoid all those irritating, irrelevant error messages you often get while executing some programs. You could just redirect the standard error (2) to some file and avoid seeing the error messages on the screen!!

Output Redirection

The most common use of Redirection is to redirect the output (that normally goes to the terminal) from a command to a file instead. This is known as Output Redirection. This is generally used when you get a lot of output when you execute your program. Often you see that screens scroll past very rapidly. You could get all the output in a file and then even transfer that file elsewhere or mail it to someone.

The way to redirect the output is by using the ‘ > ‘ operator in shell command you enter. This is shown below. The ‘ > ‘ symbol is known as the output redirection operator. Any command that outputs its results to the screen can have its output sent to a file.

$ ls > listing

The ‘ ls ‘ command would normally give you a directory listing. Since you have the ‘ > ‘ operator after the ‘ ls ‘ command, redirection would take place. What follows the ‘ > ‘ tells Unix where to redirect the output. In our case it would create a file named ‘ listing ‘ and write the directory listing in that file. You could view this file using any text editor or by using the cat command.

Note: If the file mentioned already exists, it is overwritten. So care should be taken to enter a proper name. In case you want to append to an existing file, then instead of the ‘ > ‘ operator you should use the ‘ >> ‘ operator. This would append to the file if it already exists, else it would create a new file by that name and then add the output to that newly created file.

Input Redirection

Input Redirection is not as popular as Output Redirection. Since most of the times you would expect the input to be typed at the keyboard. But when it is used effectively, Input Redirection can be of great use. The general use of Input Redirection is when you have some kind of file, which you have ready and now you would like to use some command on that file.

You can use Input Redirection by typing the ‘ < ‘ operator. An excellent example of Input Redirection has been shown below.

$ mail cousin < my_typed_letter

The above command would start the mail program with contents of the file named ‘ my_typed_letter ‘ as the input since the Input Redirection operator was used.

Note: You can’t have Input Redirection with any program/command. Only those commands that accept input from keyboard could be redirected to use some kind of text files as their input. Similarly Output Redirection is also useful only when the program sends its output to the terminal. In case you are redirecting the output of a program that runs under X, it would be of no use to you.

Error Redirection

This is a very popular feature that many Unix users are happy to learn. In case you have worked with Unix for some time, you must have realised that for a lot of commands you type you get a lot of error messages. And you are not really bothered about those error messages. For example whenever I perform a search for a file, I always get a lot of permission denied error messages. There may be ways to fix those things. But the simplest way is to redirect the error messages elsewhere so that it doesn’t bother me. In my case I know that errors I get while searching for files would be of no use to me.

Here is a way to redirect the error messages

$ myprogram 2>errorsfile

This above command would execute a program named ‘ myprogram ‘ and whatever errors are generated while executing that program would all be added to a file named ‘ errorsfile ‘ rather than be displayed on the screen. Remember that 2 is the error output file descriptor. Thus ‘ 2> ‘ means redirect the error output.

$ myprogram 2>>all_errors_till_now

The above command would be useful in case you have been saving all the error messages for some later use. This time the error messages would append to the file rather than create a new file.

You might realize that in the above case since I wasn’t interested in the error messages generated by the program I redirected the output to a file. But since those error messages don’t interest me I would have to go and delete that file created every time I run that command. Else I would have several such files created all over whenever I redirect my unwanted error output. An excellent way around is shown below

$ find / -name s*.jpg 2>/dev/null

What’s /dev/null ????? That something like a black hole. Whatever is sent to the ‘ /dev/null ‘ never returns. Neither does one know where it goes. It simple disappears. Isn’t that fantastic !! So remember.. whenever you want to remove something.. something that you don’t want …you could just send it to /dev/null

Isnt Unix wonderful !!!

Different ways to use Redirection Operators

Suppose you want to create a text file quickly

$ cat > filename
This is some text that I want in this file
^D

That’s it!! Once you type the ‘ cat ‘ command, use the Redirection operator and add a name for a file. Then start typing your line. And finally press Ctrl+D. You will have a file named ‘ filename ‘ in the same directory.

Suppose you want to add a single line to an existing file.

$ echo “this is a new line” >> exsisting_file

That would add the new line to the file named ‘ existing_file ‘ . Remember to use ‘ >> ‘ instead of ‘ > ‘ else you would overwrite the file.

Suppose you wanted to join 2 files

$ cat file2 >> file1

Wow!! That a much neater way then to open a text editor and copy paste. The contents of ‘ file2 ‘ would be added to ‘ file1 ‘ .

Suppose you want to join a couple of files

$ cat file1 file2 > file3

This would add the contents of ‘ file1 ‘ and ‘ file2 ‘ and then write these contents into a new file named ‘ file3 ‘ .

Redirection works with many commands besides normal ones such as ‘ cat ‘ or ‘ ls ‘ . One example I could give you is in case you are programming using any language you could redirect the output messages of the compilation of your code so that you can view them later on. There are lots of commands where you can use Redirection. The more you use Unix the more you will come to know.
command1 < infile > outfile
* combines the two capabilities: command1 reads from infile and writes to outfile.
Piping

Programs can be run together such that one program reads the output from another with no need for an explicit intermediate file:
A pipeline of three programs run on a text terminal

command1 | command2

executes command1, using its output as the input for command2 (commonly called piping, since the “|” character is known as a “pipe”).

This is equivalent to using two redirects and a temporary file:

command1 > tempfile
command2 < tempfile
rm tempfile

A good example for command piping is combining echo with another command to achieve something interactive in a non-interactive shell, e.g.

echo -e “user\npass” | ftp localhost

This runs the ftp client with input user, press return, then pass.

Chained pipelines

The redirection and piping tokens can be chained together to create complex commands. For example:

ls | grep '\.sh' | sort > shlist

lists the contents of the current directory, where this output is filtered to only contain lines which contain .sh, sort this resultant output lexicographically, and place the final output in shlist. This type of construction is used very commonly in shell scripts and batch files.

There are lots of commands where you can use Redirection. The more you use Unix the more you will come to know.
References:

http://www.codecoffee.com/tipsforlinux/articles2/042.html

www.wikipedia.com