Python: How to run a command line within python?

March 4th, 2008 mysurface Posted in Developer, python | Hits: 203662 | 15 Comments »

I always got this question in my mind. How to run a command line within python, get the output and manipulates it. Before I learn python, I was doing bash scripts all the while to helps me manipulates text which I get it from log files, or pipes out from some certain command line.

To do it in bash script is straight forward, where I just puts my command line into text like I put in my bash prompt. But to run a command line in python is not like bash script, so what options do I have?

The first function I discover was system in os library.

import os
os.system("ls -l")

By passing the command line to the system function will cause the command executed in subshell and print it straight away to standard output. I don’t have chance to keep the outcomes for further manipulations.

The second function that drags my attention is popen, which the same way I pass my command line in, and this time it returns the result into a file object which allows me to manipulate it consequently.

import os
f=os.popen("ls -l")
for i in f.readlines():
     print "myresult:",i,

What it returns for me is a the result of standard output, what if I purposely make a wrong command line? let say …

import os
f=os.popen("ls -z")

The error will be print to screen instead of store in file object f,

ls: invalid option -- z
Try `ls --help' for more information.

The error is actually print to standard error, to store the standard error, we need to call popen3().

fi,fo,fe=os.popen3("ls -z")
for i in fe.readlines():
     print "error:",i

popen3() returns stdin, stdout and stderr which are standard input, standard output and the standard error. The examples above, I extract the error description from the stderr file object.

How can I tell the command line is success or fail? I have no idea too, the only thing I can do is run readlines() to extract the output from respective file object.

There is another Popen function in subprocess library, I will covers it after I get my hands dirty on that lib more.

15 Responses to “Python: How to run a command line within python?”

  1. if you are using os.system, the exit code is returned by the function itself.

    eg: p = os.system(‘ls’) .. p will store the exit code.

    os.popen returns a file object, so i’m not sure about its exit code.

    the subprocess.Popen module in the other hand, is designed to replace all those messy os.popen* into a one, flexible, feature-packed, easy-to-use function for executing external commands. You can basically duplicate os.system and os.popen* do by manipulating the parameters of subprocess.Popen. You can get PID, exit codes, etc, easier through subprocess.Popen.

    subprocess.Popen is more convenient and intuitive imo.

  2. Kage Sama : Agree ^ ^ You got the point. But it also depend on the context. IMHO, for command like ls -l, it is better to use os.popen because the command returns straight, but for command like find / -name hello, then it will be better to run through subprocess.Popen.

    But again, it is preferences.

  3. I guess there is a module called “commands” which can do this operation and return the status and the output of any command.

    http://docs.python.org/lib/module-commands.html

    This is quite simple than the popen functions.

  4. sebsauvage: Your are right, the advantages of subprocess is the process are carry on without blocking. So you can carry on your main process without need to wait for the execution of the subprocess.

  5. Subprocess is the best choice. Using a list with Popen() you can run command line commands without blocking IO.

    Some of you suggested using commands.getoutput, that is wrong. Don’t do it.

    Os.system is even worse. Don’t ever touch it. EVER.

  6. If you are looking for the exit code of that command, it is returned when you are close the file handle.

    f.close() will return 512 if it fails and it won’t return anything if it is a success.

  7. [...] It's reference from here. [...]

  8. [...] Python vs Lua, data structure In python, we have various type of data structure, such as list, set, tuple, dictionary etc, but in Lua, we only have ta… Python: How to run a command line within python? » Linux by Examples [...]

  9. how will work on commands of the os like linux

  10. You can use echo $? to tell the command is success or fail

    import os
    f=os.popen(“ls -z; echo $?”)
    for i in f.readlines():
    print i,

  11. I adore this blog site layout ! How do you make it!? It is so cool!

  12. [...] out I only needed 3 hours. I learned a lot about Facebook’s api, and handling files and executing shell commands with Python. I also learned about the proper way of handling cron jobs in [...]

  13. Having and issue with this… any thoughts?

    Example of the issue for arguments sake:

    Platform Ubuntu server 12.04LTS, python 2.7

    Say file1.txt has “hello world” in it.

    subprocess.Popen(“cat file2″, shell = True)
    subprocess.call(“cat file2″, shell = True)
    os.system(“cat file2″)

    I’m finding that file2 IS created, but with 0bytes in it, this happens when I try any sort of cmd to the system of the nature where I’m putting the output into a file.

    I’ve made sure it isn’t a permission issue. The command runs fine from the cmd line and python is being run with super user privileges. Strait from the terminal I get a hello world copy as file2… as expected.

    I would like python to simply exec the cmd and move on…. I don’t want to read and write the stdout ect into python and write it to a file. Any thoughts as to why this creates file2, but no data appears? Is there a better way to do this?

  14. cat file2″, shell = True

Leave a Reply