Compare two directory listings

November 8th, 2006 toydi Posted in comm, process substitution, Text Manipulation | Hits: 92541 | 10 Comments »

Given two directories: ~/dir-new/ and ~/dir/,

$ ls ~/dir-new/
file-01.txt
file-02.txt
file-new.txt
$ ls ~/dir/
file-01.txt
file-02.txt
file-old.txt

To compare them:

comm <(ls ~/dir-new/) <(ls ~/dir)

comm compares two files line-by-line and displays 3 columns : 1) lines unique to first file, 2) lines unique to second file, 3) lines appear in both files.

                           file-01.txt
                           file-02.txt
file-new.txt
              file-old.txt

To remove Column 1, try 'comm -1', and 'comm -23' will remove Column 2 and 3.

And this, '<(ls ~/dir-new/)', they call it as process substitution. Since comm only reads from files, '<( )' pretends itself as a file by returning a filename, which is read-only, containing the output of 'ls ~/dir-new/'.

In short, we may think '<(commands)' as a pseudo, read-only file:

$ ls -l <(true)
lr-x------ 1 toydi toydi 64 2006-11-08 14:30 /dev/fd/63 -> pipe:[28600]
Related Posts
some tips of change directory
Don't think that you knows everything about change directory (cd). There might have something you do not know. Let say I...

A better svndiff, allow comparision between different revision
Refers back to svn command line tutorial for beginners 2, I introduces a bash script that uses svn cat and vimdiff to co...

directory stack
We can push our directories into a directory stack, and pop when we needed. Sometimes we need to access many long path o...

10 Responses to “Compare two directory listings”

  1. linux god Says:
    October 23rd, 2007 at 8:46 am

    diff dir dir-new

    give er a try!

  2. rxavier Says:
    March 6th, 2008 at 10:17 am

    i like the ‘comm’ command, diff was giving me grief trying to find unique files between 2 dirs so i could copy/sync things up.

    cheers for this tip!

  3. thiamteck Says:
    June 23rd, 2010 at 4:41 pm

    Good one!! Google shall rank this page as 1st result instead of 5th result :P

  4. asdfasdf Says:
    December 17th, 2010 at 9:53 pm

    diff is pretty easy although i’m finding comm to be neat. I love new things.

    Solve your diff unique record issues using ‘uniq -u’ after a ‘sort’

  5. Ram-Z Says:
    February 16th, 2011 at 4:11 am

    Nice one, been looking for this.

    The problem with diff is that it will compare every file which can be very slow depending an the # and size of files. If you only want to compare dir listings, this is the way.

  6. Granjow Says:
    March 3rd, 2011 at 5:36 am

    Cool! This even works in a recursive way:

    comm <(cd DIRONE && find |sort) <(cd DIRTWO && find |sort) |less

  7. cj Says:
    May 4th, 2011 at 12:00 pm

    Oh!!! Thank you. Diff is slow. Sync is weird. This is simple.

  8. Thank you for puting the time in to publish this info. I found it very useful. If you are ever interested in link building directory list then please contact me. Says:
    December 3rd, 2011 at 11:58 am

    This is very fascinating, You are an overly skilled blogger. I have joined your rss feed and look ahead to seeking more of your wonderful post. Also, I have shared your site in my social networks

  9. Link bids Says:
    December 20th, 2011 at 9:49 pm

    Valuable information. Fortunate me I found your website by accident, and I’m surprised why this twist of fate did not took place in advance! I bookmarked it.

  10. Linux: Compare Directory Structure Without Comparing Files | Click & Find Answer ! Says:
    April 22nd, 2013 at 5:40 am

    [...] in DIR1, 2 – files only in DIR2, 3 – files only in DIR3For more details look at this blog post.use “diff -qr” to get the different files and then filter out the file comparison with [...]

Leave a Reply