BTTB: looping for shell script under embedded linux

November 17th, 2010 mysurface Posted in cut, for, if, seq, Text Manipulation | Hits: 164207 | 9 Comments »

You may already realized Linux happened to appear at many places, such as web server, storage server, desktop, kiosk machine, mobile devices. Yes, more and more devices running embedded Linux. Yeah, Android is a modified version of Linux kernel too!

Scarcity is still an issue, embedded Linux can be very different to Linux that hosted on desktop or high-end servers. You may be provided with simple shell (sh but not bash, not ksh and not csh) and limited commands, it bundles into a BusyBox, The Swiss Army Knife of Embedded Linux.

I am a bit upset when I holding this so called Swiss Army Knife ( probably a customized one, super tiny one), why this command also don’t have, that command also not available. Well, I am still quite new to embedded Linux perhaps.

My busybox does not have bash, therefore a lots of bash syntax can’t be applied to my script, ouch! I do wrote some post about bash. Such as if … then … fi and download multiple files from a site using for loop like c in bash. The if…then…fi examples may still valid for shell script, because that is the fundamental one.

The very basic for loop for shell script is like this


for p in 1 2 3 4 5
do
     echo $p
done

You can do simplified it by using command seq ( if seq is available in your busybox)


for p in `seq 1 5`
do
     echo $p
done

Well, my swiss army knife doesn’t contain me seq command, it doesn’t have bash, no top command, giving me a fake ps with limited info. I can’t check my how busy is my processors and thread easily (because fake ps do not tells you that), I need to access it through /proc/[PID] and thank God, it does provide me the cut command.

Yes I able to cut by using my swiss army knife.

Due to the scarcity, I learn more about /proc. /proc is a pseudo-filesystem that provides tones of process information. Check out the proc 5 manual for detail information.

man 5 proc

What I am interested is the file /proc/[PID]/stat, the status information about the process. stat provides a long array of values, I am interested on column 1,2,14 and 15 which is PID, Command, utime and stime.

Ok, list me all the process’s utime and stime. I wrote a simple script call probe.sh.


#!/bin/sh
for p in /proc/[0-9]*;
do
    cat $p/stat | cut -d" " -f1,2,14,15
done

The output looks decent, time takes less then 1 second.


...
6 (cpuset) 0 0
608 (scsi_eh_0) 0 1
612 (scsi_eh_1) 0 0
615 (scsi_eh_2) 0 1
661 (kpsmoused) 0 0
671 (kstriped) 0 0
677 (kondemand/0) 0 0
7 (khelper) 0 8
707 (usbhid_resumer) 0 0
854 (udevd) 0 11

real    0m0.965s
user    0m0.138s
sys     0m0.752s

But the list is long and overwhelming. What if I have targeted pid range?


#!/bin/sh
START=2000
END=3000
until [ $START -gt $END ]
do
    p="/proc/$START/stat"
    if [ -f $p ] ;
    then
        cat $p | cut -d" " -f1,2,14,15
    fi
    START=`expr $START + 1`
done

The output is promising, but it took too long to produce that.


2391 (syslog-ng) 0 0
2392 (syslog-ng) 1 11
2451 (gpm) 0 0
2882 (dhcpcd) 0 0

real    0m7.145s
user    0m0.959s
sys     0m5.640s

The expr for arithmetic function seems to consume a lots of processing time.

Another approach, which amazingly gives correct result and fast.


START=2000
END=3000
for p in /proc/[0-9]*;
do
    pid=`basename $p`
    # omg no basename? replace with the line below
    # pid=`cat $p | cut -d"/" -f3`
    if [ $pid -ge $START ]
    then
        if [ $pid -le $END ]
        then
            cat $p/stat | cut -d" " -f1,2,14,15
        fi
    fi
done

Check out the result. Outstanding isn’t it?


2391 (syslog-ng) 0 0
2392 (syslog-ng) 1 11
2451 (gpm) 0 0
2882 (dhcpcd) 0 0

real    0m0.550s
user    0m0.069s
sys     0m0.447s

I had briefly illustrate the looping mechanism on shell script, I hope that gives you some ideas while you working under embedded Linux.

9 Responses to “BTTB: looping for shell script under embedded linux”

  1. Hi, when you mentioned some optimization…
    (System is running on 2 CPUs.)

    NOT BAD:
    time {
    for i in /proc/[0-9]*; do pid=`basename $i`; done
    }

    real 0m0.706s
    user 0m0.292s
    sys 0m0.464s

    ??
    BETTER:
    time {
    cd /proc
    for i in [0-9]*; do pid=$i; done
    }

    real 0m0.010s
    user 0m0.004s
    sys 0m0.004s

    NOT BAD:
    time {
    cd /proc
    for i in [0-9]*
    do
    cat $i/stat | cut -d” ” -f1,2,14,15
    done
    }

    real 0m0.729s
    user 0m0.744s
    sys 0m0.688s

    BETTER:
    time {
    cd /proc
    for i in [0-9]*; do cat $i/stat; done | cut -d” ” -f1,2,14,15
    }

    real 0m0.460s
    user 0m0.444s
    sys 0m0.336s

    EVEN BETTER (sorry, no loop):
    time {
    cat /proc/[0-9]*/stat | cut -d” ” -f1,2,14,15
    }

    real 0m0.022s
    user 0m0.008s
    sys 0m0.020s

    I home, this can help somebody just to have another angle of view (time and CPU consumption is small anyway).

    Lukas

  2. If test (“[") is not shell built-in, then one test is faster than two tests...

    [ $pid -ge $START -a $pid -le $END ]

    Lukas

  3. Thanks Lukas, your optimization is awesome. :)

  4. First of all, thanks for the great tips on using /proc for getting process info, this post was very helpful in helping me understand the internals of Linux.

    I tested the command line you proposed for those systems that don’t have the basename utility and got this:

    cat: /proc/1: Is a directory
    ../Code/bash/pid-from-proc.sh: line 10: [: -ge: unary operator expected

    for each running process.

    I changed that line to this:

    pid=$(cat $p/stat | cut -d” ” -f1)

    and it worked. I used the $() construct for command substitution in place of backticks “ as it is the preferred method on modern systems.

    Thanks again,
    Carl

  5. @Carl, you can try the one suggested by Lukas too.

    simply:
    cd /proc
    cat [0-9]*/stat | cut -d” ” -f1,2,14,15

    if range is not important for you.

    or

    cd /proc
    for i in [0-9]*;
    do
    if [ $i -ge $START -a $i -le $END ]; then
    cat $i/stat;
    fi;
    done | cut -d” ” -f1,2,14,15

    for the range version.

  6. Just a heads up on how the website handles code. What appears to be double quotes in the last line of code:

    done | cut -d” ” -f1,2,14,15

    is something else (“smart” quotes??), thus making cut and paste unworkable without further editing.

    Rock on!
    Carl

  7. @Carl, must be the html unicode, unless the code put into

    <pre><code></code></pre>

  8. Hey very nice blog!! Man .. Excellent .. Amazing .. I will bookmark your web site and take the feeds also…I am happy to find numerous useful information here in the post, we need work out more techniques in this regard, thanks for sharing. . . . . .

  9. Really nice blog, and helpful discussions. thanks for posting.

Leave a Reply