The Hunt for Duplicates Continues

February 17, 2007 at 11:26 pm 1 comment

A while back I wrote about having to sort through several backup DVD’s to find a ton of duplicate files. I decided to give fdupes a try. fdupes uses a combination of file size, md5 hash, and bit by bit comparison, so it should be fairly safe to trust that the results are exact duplicates. (I will double check the first couple just to make sure)

I moved the data over from 9 DVD’s (but there’s more 🙄 ) to an external hard drive and sent fdupes off to do its work. It seems to be fairly fast. It compared about 141,000 files with sizes between 0 bits to over a GB in about an hour and a half. Now I need to find out how to pass the results to rm leaving only 1 copy. (I think fdupes has an option for this)

I’ll try to post a tutorial if I get everything working ok. I’ll try running it with a couple of different options and run diff on the outputs to see if running with omit first (-f) will preserve only 1 file.

Advertisements

Entry filed under: Linux. Tags: , , , , .

How Will Vista Make My Life Easier? fdupes Tutorial

1 Comment Add your own

  • 1. J  |  February 4, 2009 at 2:19 am

    So… How did this turn out?

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


Feeds


%d bloggers like this: