Can I make this shell script faster?

[backrow]

Hi guys,

I use nmh for my mail; plain text mails are displayed with less(1). Unfortunately, since my xterm uses UTF-8 by default, any ISO8859 mails will tend to have �’s everywhere.

I thought I’d fix this by piping output through a simple script first:

#!/bin/sh

while read input
do
echo $input | file - | grep 8859 > /dev/null
if [[ $? = 0 ]]; then
        echo $input | iconv -f ISO-8859-1 -t UTF-8
else
        echo $input
fi
done

This works, but makes reading my mail noticeably slower:

$ time u8conv.sh  < /etc/hosts >/dev/null
    0m0.22s real     0m0.11s user     0m0.11s system
$ time cat /etc/hosts >/dev/null         
    0m0.00s real     0m0.00s user     0m0.00s system

Is there an obvious way to make the script faster? Perhaps there’s a better way to solve my problem that I’ve missed.

[IdOp]

I'm not familiar with the conversion issues, and this is just a quick idea of the top of my head, but it seems likely your script may be slow because it's reading and processing the input a line at a time. So, can you work with the input as a whole file?

E.g., copy input to a temp file, apply the file command to that to determine what type it is, then cat the tempfile through the conversion filter or not, as you've done with lines. Then delete the tempfile.

There's probably a neater version of this, which might be worth looking for if this method works at all and gives a good speedup. Hope that helps.

[backrow]

Tried this, it was much faster, thanks. The mail is already saved to a temporary file, so this is easy.

#!/bin/sh

file $1 | grep 8859 > /dev/null
if [[ $? = 0 ]]; then
        iconv -f ISO-8859-1 -t UTF-8 < $1
else
        cat $1
fi

[TerryP]

Since your case is already solved, I'll just leave a note for the next person to find it.


If you're processing a stream line by line with a while read loop in sh script, you're probably doing it wrong.