Redirect output to multiple processes

November 8th, 2006 toydi Posted in Common, grep, process substitution, tee | Hits: 143942 | 42 Comments »

Since tee can read the standard input, and write to multiple files, we may leverage this feature so that it writes to multiple processes (instead of files).

tee >(process1) >(process2) >(process3) | process4

>( ) (see process substitution) pretends itself as a write-only file. Everything you write into it, will be passed to the command (as standard input) within its parenthesis.

ls -A | tee >(grep ^[.] > hidden-files) >(grep -v ^[.] > normal-files) | less

So in this example (rather useless though), it will list all the filenames, then send the listing to 3 processes:

  • grep ^[.] > hidden-files – save hidden filenames to a file.
  • grep -v ^[.] > normal-files – save non-hidden filenames to another file.
  • less – display all filenames in a pager.

42 Responses to “Redirect output to multiple processes”

  1. How to redirect the o/p to a file in runtime.
    For example Here Run time means.. i’m trying to compile a project, in that compilation it takes input parameters in the runtime i.e after executing some steps it asks the user input to proceed further for compiling.. In this case how to get the output of the compilation into a logfile for postpartum purpose.

  2. Are you using make to do the compilation? or running a bash script?

    For normal compilation, usually come with ./configure;make;make install;

    I assume the compilation of your project is by running bash script such as install.sh, so that It is possible to ask for user input.

    If that is the case, I suggest you to read the scripts and modify that line that you are interested to log.

    Or, simple way, log entire process from start to the end, such as

    install.sh | tee >log.sh
  3. Ambika Prasad Says:

    Hi folks,

    Please help me to find a command so that i can redirect the output of below command to a file.

    # top -d 1 | grep “init”

  4. Ambika Prasad:
    Interesting, you can do this

    top -b -d 1 | tee >(grep "init" > test ) >/dev/null
  5. how to get the kernal functions addresses in symbol table?

  6. [...] Redirect output to multiple processes » Linux by Examples (tags: linux) [...]

  7. Hi, could you help me how to redirect the first 5 lines of the “top” command into a file….

  8. This will get you the first 5 lines out of top, then top will end.

    top -n 1 -b -d 1 | tee >(sed -n ’1,5 p’ > test ) >/dev/null

  9. Can you tell me the command in LS which lists only directories…

  10. I get this error in buildroot but not on my native system:

    wget -O – ftp://192.168.3.200/something.tgz |tee >(/bin/tar xzf – ) |md5sum

    -sh: syntax error near unexpected token `(‘

    I tried to change from Ash to Bash with no result.

    Now I’m searching for the sources of tee and will real tee instead of the busybox tee.

  11. Looks like busybox ash has troubles with process substitution like >() and wants $() instead. But then the data piped to the process is not valid…

    Aargh…

    I also tried it with bash, then I can use >() but data piped to the process is also invalid…

  12. JeroenSt: heh! tee >() sometimes is really hard to play with, it sometimes just doesn’t act what I ask for. I will suggest you to look into python. There is a os.popen() which can run any commands, where the result will be stored into file object, so you can get the results from .readlines() and manipulate as much as you can.

    Recently, I look into python, that is really convenient, will probably post about python later on when I have experience more.

  13. We don’t have python at our embedded device, but I figured out another way:

    mknod /tmp/pipe1 p
    mknod /tmp/pipe2 p
    wget -q -O – ftp://192.168.3.200/somefile.tgz |tee /tmp/pipe1 /tmp/pipe2 |>/dev/null&
    md5sum /tmp/checksum&
    tar -xzf – -C /tmp/somedir

  14. Hmmz, the rest of my article is not saved!?

    But the pipe output of tee is trashed because it is not valid somehow.

    The other problem is that md5sum and tar will wait forever if tee doesn’t start (out of memory or corrupt)

  15. JeroenSt:
    Actually, I am not sure what are you trying to do.
    Anyway, How about split it to few steps?
    wget -q -O /tmp/tmp.tgz ftp://192.168.3.200/somefile.tgz;md5sum /tmp/tmp.tgz; tar -zxvf /tmp/tmp.tgz -C /tmp/somedir;

    Not sure why you wanna checksum…

  16. Can I increase the run levels more than 6?If it is possible how can we do that and how many maximum run levels can we assign?

  17. Well, the reason why I do this is because I only have a limited space of flash on the embedded device. When I download the file on flash first, I can’t fill the flash (because 1/3 is allready used by the tgz file). The md5sum is also on the FTP server, so I can be sure that the transfer was correct and also was the right file.

  18. JeroenSt:
    Emm… You may get help, if you ask at irc://irc.freenode.net/bash.

    A lots of expert over there.

  19. Think I understand how it works, at least for Bash:
    # echo hello >(cat >testfile)
    Returns: hello /dev/fd/63
    This is a temporary file descriptor, that is connected to the STDIN of a process started executing ‘cat >testfile’.
    If the echo command ends, the file descriptor is removed.

    So the construction >(command) is replaced by a filename.

    If you execute:

    echo ‘hello’ | tee >(wc >wordcount) >(md5sum >sum)

    A file descriptor is created for the ‘wc >wordcount’, for exampe ‘/dev/fd/63′, and a file descriptor is created for the ‘md5sum >sum’, for example ‘/dev/fd/62′.

    The command that is executed after the creation of the file descriptors is:

    echo ‘hello’ | tee /dev/fd/63 /dev/fd/62

    And I see it works, I get a file named ‘wordcount’ and a file named ‘sum’.

  20. Jeroen:
    Thanks for the great explanation. I find tee sometimes is very hard to understand, but once you got it right, it will be very useful.

  21. Can anyone help me with this?
    I have one script which is running from last 15 days in the background.Now My project manager wants the few output of that running process without stopping it.
    Pls Help

  22. Ajeet: You may try fg to bring it to front, and ctrl+z to bring it back to background. But I am not sure whether it works for your case.

  23. Hai to all!!

    I want to install the drivers of video card on fedora but I am unable to find the graphics card installed in my pc .FYI ,PC is not booting into graphical mode..Help me out in this issue.

    Thanks

  24. Hi,

    I have to redirect the output of a time function of a command which also redirects also its out output to a log file:

    time athena.py jobOptions.pythia.py | tee athena_gen.out

    Output of athena.py is already going to athena_gen.out but I would like to redirect the output of time to a file too..

    Thanks,
    Omer

  25. Maxdamantus Says:

    Since time is actually a built in bash function, it handles it differently to usual commands. Bash, however, is quite an intelligent shell, allowing for redirection of its own functions (although time doesn’t work like the others (echo for example) for some reason). The solution is to parenthesize the time function and its arguments, then have that redirected to a file:

    (time do_something) 2>result.time

  26. [...] Redirect output to multiple processes Since tee can read the standard input, and write to multiple files, we may leverage this feature so that it writes to mu… [...]

  27. Hi,
    I want to tee into two process but keep fail.
    My linux system might be very old.
    tee –version -> 2.0 (C) 1999
    When I tried to
    echo “345″ | tee >(grep 5)
    it show a syntax error “(” unexpected.
    But I could do (…) | ….
    So is it because of the tee?
    Any suggestion or work around?

  28. [...] This example comes from this article. [...]

  29. [...] simple AND visually appealing. Beautiful. See Wiki link and by examples link. This entry was posted in 8/10, bash shell by admin. Bookmark the [...]

  30. It should be noted that this ability is quite specific to bash. Most other shells do not support this ability. Thanks for the great article

  31. Great article! Thanks~

  32. [...] simple AND visually appealing. Beautiful. See Wiki link and by examples link. Share this:TwitterFacebookLike this:Like Loading… This entry was posted in 8/10, bash shell by [...]

  33. [...] source Answered by Ben [...]

  34. Having read this I thought it was extremely informative.
    I appreciate you spending some time and energy to put
    this content together. I once again find myself spending a lot of time both reading and posting
    comments. But so what, it was still worthwhile!

  35. Very good post! We will be linking to this particularly
    great article on our website. Keep up the great writing.

    Here iss my homepage; Nitro Focus No3 free trial

  36. Hello There.I found your blog using msn. This
    is aan extremely wesll written article. I’ll make sure to bookmark it and
    come back to rad more of your usefjl information. Thanks
    for the post. I will definitely comeback.

  37. Hi there to every one, the contents existing at this site are really remarkable for people experience, well, keep up the good work fellows.

  38. Great post. I’m experiencing some of these issues as well..

  39. Howdy very cool website!! Man .. Excellent .. Wonderful ..

    I’ll bookmark your web site and take the feeds additionally?

    I am satisfied to find a lot of useful information right here
    in the post, we need work out extra techniques on this
    regard, thanks for sharing. . . . . .

    ???????????????????????????:???????

  40. Hi, every time i used to check web site posts here early in the
    morning, since i enjoy to find out more
    and more.

    ?????-????????

  41. Other Icaros call the spirit of Ayahuasca to open visions as though
    exposing the optic nerve to light’. If I had those two in my life, I settled much faster and I felt happy.
    The acai berries must be loaded into baskets and onto boats soon after picking.
    Moonstone Soothes and balances the emotions; helps eliminate
    fear of “feeling”; encourages inner growth and strength; aids peace
    and harmony and psychic abilities; aligns vertebrae; digestive
    aid. Inhalation: You can either apply the oil directly to
    your hands, rub them together and then cup your hands over your
    nose and inhale deeply.

Leave a Reply