python: handle string from pipelines and list of param

November 13th, 2007 mysurface Posted in Developer, python | Hits: 63692 | 10 Comments »

While I was searching ways to implement pipeline input for my python apps, I read an article from linuxjournal.com that mention about how easy that python can works with pipeline.

Here is the code quote from that article.

#! /usr/local/bin/python
import sys
sys.stdout.write(sys.stdin.read())

Well, I understand that served as a simple example to show how easy python can works with pipeline. But it is misleading because it doesn’t works well. People will think python sucks, because it doesn’t handle the pipeline properly. I find the code mined with two problems.

1. The shabang#!
Python scripts usually do not put the shabang directly point to the binaries like bash

#! /usr/local/bin/python

Because python might be install at other location, therefore it is a correct practice to put in this way:

#! /usr/bin/env python

Which it tells the env to search for python binaries.

2. The script cannot run without feeding something from pipelines.
Okay lets makes the code above as toosimple.py and execute it as following lines

echo "hello python" | ./toosimple.py

Hey! this work perfectly.

But what if the user do not specified anything from the pipe?

./toosimple.py

This will cause the script do nothing but keep waiting for user input forever.

So, how should I detects whether is there any inputs from pipes?

Let us just look at a simple code that cater for pipelines as well as parameters.


#! /usr/bin/env python
import sys

def main(argv = sys.argv):
    # detects whether have pipe line parsing in
    if not sys.stdin.isatty():
        for s in sys.stdin.readlines():
            print "pipe: " + s,
    # param
    L = sys.argv[1:]
    if len(L)!=0:
        print "param: " + str(L)
    sys.exit(0)
    pass

if __name__ == "__main__":
    try:
        main(sys.argv)
    except KeyboardInterrupt:
        pass

The important part is here if not sys.stdin.isatty(): which it detects whether any inputs from pipelines.

So, lets try this simple.py now:

./simple.py

Nothing happen, expected.

echo "hello python" | ./simple.py
pipe: hello python

Its fine. Now lets try to put in parameters.

./simple.py hello world
param: ['hello', 'world']

It seems that param is in the form of list.

How about pass in the entire file?

./simple.py < myloveletter.txt

You can try it out yourself. Have fun :)

10 Responses to “python: handle string from pipelines and list of param”

  1. Thanks that was extremely helpful. Thanks to you I have a new idea I am working on.

    Thanks!-Patrick

  2. echo “hello python” | ./simple.py
    is not worked
    Traceback (most recent call last):
    File “D:\python\pipe\simple.py”, line 3, in
    sys.stdout.write(sys.stdin.read())
    IOError: [Errno 9] Bad file descriptor

    echo “hello python” | ./toosimple.py
    is not worked
    D:\python\pipe>echo “wefwef” | toosimply.py
    Traceback (most recent call last):
    File “D:\python\pipe\toosimply.py”, line 18, in
    main(sys.argv)
    File “D:\python\pipe\toosimply.py”, line 7, in main
    for s in sys.stdin.readlines():
    IOError: [Errno 9] Bad file descriptor

    M$ windows

  3. @qman, My implementation will only works on Linux.

  4. Thanks for this trick. It is exactly what I needed for a current project.

  5. AprilCoolsDay Says:

    I believe the first code is supposed to implement a simple cat command.

    The cat command does wait for user input if the user does not specify anything. So the original code is more in line with the cat command. And this is useful. For example, if you run:

    $ cat > foo.txt

    or

    $ ./toosimple.py > foo.txt

    and write multiple lines of some text and then press ENTER followed by Ctrl-D, then the cat command finishes and the text is written to foo.txt.

    I wouldn’t describe this behavior as something unusual. The same thing is happening when you run

    $ python

    or

    $ bash

    On another note, there is a minor difference between the cat command and the original code: cat takes stdin input line by line and pushes them to stdout immediately per each line (at least on my system), while the python code takes everything (until Ctrl-D) at once. To see this, simply run

    $ cat

    and write more than one line.

  6. I\\\’ve been looking to the answer as to how to successfully make a python script part of a pipeline for so long. For some reason no one assumes there can be no input.

    #this line saves my life
    if not sys.stdin.isatty():

    Thank you,
    Thank you,
    Thanks

  7. Thank you so much. I just spent most of the day trying to make my command line linux scripts work like a command line program is SUPPOSED to work on linux! I found the same code scrap you started with and like you found it didn’t work correctly.

    As a bonus now I get why the “isatty” boolean function is in there!

  8. Is there any way to pipe in text on Windows? Would downloading something like Cygwin help at all?

  9. Great!

  10. As a bonus now I get why the

Leave a Reply