xargs use stardard output as parameter for another command

0 votos

October 2nd, 2006 mysurface Posted in Text Manipulation, cat, pipeline, wget, xargs | Hits: 11268 |

xargs is a command line of findutils package. It is not so common for new user, but it is a very useful tools, let me pick an example to show the usage. Bare in mind, xargs is a kind of combo command use together with other command through pipeline.

I have a list of links to download, to download all, i put those links to a file. When i cat, i puts it line by line to standard output. Then I can pipeline and ask xargs to pass it one by one to wget.

cat list.txt | xargs wget -c

~~It is recommended to put -0 for xargs for filename, coz sometimes the filename consist of symbol.So~~

cat list.txt | xargs -0 wget -c

It doesn’t seems to be working, thanks durian for commenting this out. If your links consist of symbol like single quote ( ‘ ) or other reserved symbols, I ll suggest you to edit your link manually by putting escape (\).

Let say my original link look like this

http://sorastar.com/You're%20My%20Friend.mp3

Observered the “You’re”, change it to “You\’re”

http://sorastar.com/You\'re%20My%20Friend.mp3

Let say I store all my links into a file call url2dl.txt and separate each link by newline. I can do this wget -i url2d...

Remove all .svn directories at once

When you check out a project code base from a svn repository, each downloaded directory (from top to the deepest) contai...

rm complains argument list too long

There is a limitation of rm command, where you can't delete a large groups of files with *. For examples, rm -rf someth...

Categories

toydi Says:
October 2nd, 2006 at 5:17 pm

*MINOR UPDATES*
xargs does not call ‘wget’ several times and pass a list of whitespace-delimited items one by one as argument each time.

In fact, it calls ‘wget’ just once, and passes the list as argument list, similar to this:

wget -c `cat list.txt`

——-
To pass several string one by one to a command, here’s a one-liner by mysurface:

cat list.txt | while read line; do wget $line; done

durian Says:
October 3rd, 2006 at 12:25 am

Dude, your comment is not right. wget may be called more than once. A program can only receive a certain amount of arguments. 512 is the maximum number of arguments for any program, if my memory serves me right. Anyway, that’s not important. The point is xargs will invoke the command repeatedly until all the arguments are consumed.

wouldn’t work properly unless all the lines in
list.txt terminate with a null. That’s not normally
done in a text file because lines are usually terminated with a newline character, not null character. Have you actually tried the command before posting?

mysurface Says:
October 3rd, 2006 at 2:34 am

Yes, you are right! Thankyou for correcting my mistake. Let me edit my post.

durian Says:
October 3rd, 2006 at 11:18 am

Another example, you can use delimiter option of xargs.

cat list.txt|xargs –delimiter=”\n” wget -c

That way you won’t have to worry about escaping the blank spaces with %20.

toydi Says:
October 3rd, 2006 at 1:14 pm

durian, thanks for pointing out the error. Now I learnt. :-)

But one thing to emphesis is, xargs does not pass 10 items “one by one” to the command by launching the command 10 times.

So, don’t ever think that xargs works as a magical loop (as I did trying to loop a script with xargs.. lol).

durian Says:
October 3rd, 2006 at 2:29 pm

toydi, I think you are right about that. Although, I can’t be too sure about the exact mechanism used by xargs. You’re welcome :). We learn through mistakes.

wget files based on the links in a file. at Linux by Examples Says:
October 3rd, 2006 at 5:10 pm

[...] This is a short example, actually I think of put it in as comment of post xargs use stardard output as parameter for another command [...]

mysurface Says:
October 3rd, 2006 at 5:53 pm

To do the job, this is shorter: