Compare two directory listings

November 8th, 2006 toydi Posted in comm, process substitution, Text Manipulation | Hits: 92541 | 10 Comments »

Given two directories: ~/dir-new/ and ~/dir/,

$ ls ~/dir-new/
file-01.txt
file-02.txt
file-new.txt
$ ls ~/dir/
file-01.txt
file-02.txt
file-old.txt

To compare them:

comm <(ls ~/dir-new/) <(ls ~/dir)

comm compares two files line-by-line and displays 3 columns : 1) lines unique to first file, 2) lines unique to second file, 3) lines appear in both files.

                           file-01.txt
                           file-02.txt
file-new.txt
              file-old.txt

To remove Column 1, try 'comm -1', and 'comm -23' will remove Column 2 and 3.

And this, '<(ls ~/dir-new/)', they call it as process substitution. Since comm only reads from files, '<( )' pretends itself as a file by returning a filename, which is read-only, containing the output of 'ls ~/dir-new/'.

In short, we may think '<(commands)' as a pseudo, read-only file:

$ ls -l <(true)
lr-x------ 1 toydi toydi 64 2006-11-08 14:30 /dev/fd/63 -> pipe:[28600]

10 Responses to “Compare two directory listings”

  1. diff dir dir-new

    give er a try!

  2. i like the ‘comm’ command, diff was giving me grief trying to find unique files between 2 dirs so i could copy/sync things up.

    cheers for this tip!

  3. Good one!! Google shall rank this page as 1st result instead of 5th result :P

  4. diff is pretty easy although i’m finding comm to be neat. I love new things.

    Solve your diff unique record issues using ‘uniq -u’ after a ‘sort’

  5. Nice one, been looking for this.

    The problem with diff is that it will compare every file which can be very slow depending an the # and size of files. If you only want to compare dir listings, this is the way.

  6. Cool! This even works in a recursive way:

    comm <(cd DIRONE && find |sort) <(cd DIRTWO && find |sort) |less

  7. Oh!!! Thank you. Diff is slow. Sync is weird. This is simple.

  8. This is very fascinating, You are an overly skilled blogger. I have joined your rss feed and look ahead to seeking more of your wonderful post. Also, I have shared your site in my social networks

  9. Valuable information. Fortunate me I found your website by accident, and I’m surprised why this twist of fate did not took place in advance! I bookmarked it.

  10. [...] in DIR1, 2 – files only in DIR2, 3 – files only in DIR3For more details look at this blog post.use “diff -qr” to get the different files and then filter out the file comparison with [...]

Leave a Reply