python: handle string from pipelines and list of param
November 13th, 2007 mysurface Posted in Developer, python | Hits: 55506 | 10 Comments »
While I was searching ways to implement pipeline input for my python apps, I read an article from linuxjournal.com that mention about how easy that python can works with pipeline.
Here is the code quote from that article.
#! /usr/local/bin/python
import sys
sys.stdout.write(sys.stdin.read())
Well, I understand that served as a simple example to show how easy python can works with pipeline. But it is misleading because it doesn’t works well. People will think python sucks, because it doesn’t handle the pipeline properly. I find the code mined with two problems.
1. The shabang#!
Python scripts usually do not put the shabang directly point to the binaries like bash
#! /usr/local/bin/python
Because python might be install at other location, therefore it is a correct practice to put in this way:
#! /usr/bin/env python
Which it tells the env to search for python binaries.
2. The script cannot run without feeding something from pipelines.
Okay lets makes the code above as toosimple.py and execute it as following lines
echo "hello python" | ./toosimple.py
Hey! this work perfectly.
But what if the user do not specified anything from the pipe?
./toosimple.py
This will cause the script do nothing but keep waiting for user input forever.
So, how should I detects whether is there any inputs from pipes?
Let us just look at a simple code that cater for pipelines as well as parameters.
#! /usr/bin/env python
import sys
def main(argv = sys.argv):
# detects whether have pipe line parsing in
if not sys.stdin.isatty():
for s in sys.stdin.readlines():
print "pipe: " + s,
# param
L = sys.argv[1:]
if len(L)!=0:
print "param: " + str(L)
sys.exit(0)
pass
if __name__ == "__main__":
try:
main(sys.argv)
except KeyboardInterrupt:
pass
The important part is here if not sys.stdin.isatty(): which it detects whether any inputs from pipelines.
So, lets try this simple.py now:
./simple.py
Nothing happen, expected.
echo "hello python" | ./simple.py
pipe: hello python
Its fine. Now lets try to put in parameters.
./simple.py hello world
param: ['hello', 'world']
It seems that param is in the form of list.
How about pass in the entire file?
./simple.py < myloveletter.txt
You can try it out yourself. Have fun :)







December 9th, 2007 at 9:47 am
Thanks that was extremely helpful. Thanks to you I have a new idea I am working on.
Thanks!-Patrick
July 31st, 2008 at 6:20 pm
echo “hello python” | ./simple.py
is not worked
Traceback (most recent call last):
File “D:\python\pipe\simple.py”, line 3, in
sys.stdout.write(sys.stdin.read())
IOError: [Errno 9] Bad file descriptor
echo “hello python” | ./toosimple.py
is not worked
D:\python\pipe>echo “wefwef” | toosimply.py
Traceback (most recent call last):
File “D:\python\pipe\toosimply.py”, line 18, in
main(sys.argv)
File “D:\python\pipe\toosimply.py”, line 7, in main
for s in sys.stdin.readlines():
IOError: [Errno 9] Bad file descriptor
M$ windows
August 1st, 2008 at 8:54 am
@qman, My implementation will only works on Linux.
October 19th, 2008 at 3:11 am
Thanks for this trick. It is exactly what I needed for a current project.
July 6th, 2009 at 4:34 am
I believe the first code is supposed to implement a simple cat command.
The cat command does wait for user input if the user does not specify anything. So the original code is more in line with the cat command. And this is useful. For example, if you run:
$ cat > foo.txt
or
$ ./toosimple.py > foo.txt
and write multiple lines of some text and then press ENTER followed by Ctrl-D, then the cat command finishes and the text is written to foo.txt.
I wouldn’t describe this behavior as something unusual. The same thing is happening when you run
$ python
or
$ bash
On another note, there is a minor difference between the cat command and the original code: cat takes stdin input line by line and pushes them to stdout immediately per each line (at least on my system), while the python code takes everything (until Ctrl-D) at once. To see this, simply run
$ cat
and write more than one line.
October 31st, 2009 at 3:22 am
I\\\’ve been looking to the answer as to how to successfully make a python script part of a pipeline for so long. For some reason no one assumes there can be no input.
#this line saves my life
if not sys.stdin.isatty():
Thank you,
Thank you,
Thanks
November 25th, 2009 at 1:50 pm
Thank you so much. I just spent most of the day trying to make my command line linux scripts work like a command line program is SUPPOSED to work on linux! I found the same code scrap you started with and like you found it didn’t work correctly.
As a bonus now I get why the “isatty” boolean function is in there!
January 27th, 2010 at 4:31 am
Is there any way to pipe in text on Windows? Would downloading something like Cygwin help at all?
September 7th, 2012 at 1:24 am
Great!
December 15th, 2012 at 7:21 am
As a bonus now I get why the