Sunday, April 10, 2005

Bash - Reading line-by-line in a loop

This is something that many people would know. Let us take an example:
$ cat ~/file
If you take every crisis as an opportunity
Life will not only be successful,
But will also be satisfying.
Let us assume that you want to display all lines which have "will" in them. One way to do this would be by reading the file line by line and looking for "will". To do this use the following loop:
#!/bin/bash

while read line
do
 echo $line | grep "will"
done<~/file
The same cannot be directly done with for loops: (Thanks to Mark Clarkson for pointing it out).
#!/bin/bash

for line in `cat ~/file`
do
 echo $line | grep "will"
done

However, this can be done with for loops by combining it with an IFS change:
#!/bin/bash

OLD_IFS=$IFS
IFS='\r';

for line in `cat ~/file`
do
 echo $line | grep "will"
done

IFS=$OLD_IFS
In the third snippet, we can have any valid command in the place of cat ~/file. Thus, this can be a very flexible construct.

It is important to note that in the first snippet, read is a built-in and hence does not spawn a new process. However in the third snippet, the cat command causes an additional process to be spawned.

Another way to do this is by
#!/bin/bash

exec 3<file.txt # Open file.txt as FD#3

while read line  
do
 echo $line | grep "will"
done <&3

exec 3>&- # Close fd #3
In the fourth snippet, we use a functionality of the exec command which opens a file and uses the number (3 in this case) as the file descriptor. This file descriptor is used at the end of the while. It is important to add the & in front of the file descriptor.

All the cases will work generally. There is one case in which it will not work: when a value is read inside the loop.
#!/bin/bash

while read line
do
 echo $line
 read -p "Enter a value: " value
done<~/file
The fifth snippet would read alternate lines into the variable value. You can use a construct similar to snippet 3 for this purpose:
#!/bin/bash

exec 3<file.txt # Open file.txt as FD#3

while read -u 3 line  
do
 echo $line
 read -p "Enter a value: " -u 0 value
done

exec 3>&- # Close fd #3
In the sixth snippet, we use the -u option which is for specifying the file descriptor that needs to be used for the corresponding read. We specify that the first read is for reading from descriptor 3, while the second read is for reading from descriptor 0. Note that the redirection at the end of the loop is not necessary.