Apr 21, 2010

De-duplicate with Fslint (little review)

When free space starts to shrink - you can either erase some files, or compress them or archive them on cd, dvd - or use fslint (Author: Padraig Brady) - a very cool tool (for Linux) that will scan your hd and track duplicate files.

I was a bit suspicious before using this tool - because usually I know where my files are etc ... but I decide to make a test with my external HD (used for backups)

To make the test more interesting I noted the free space before launching fslint and it shows 8.1 GB free.

Here below is the "fslint-gui" main screen - on the top is shown the search path - and below are the various options - by default it looks for "duplicates" now press the Find button (below)


The search process is pretty long, checking all the files with various methods - to make sure there are really duplicates files.

It took around 20-30 mn in my example.








When all "duplicate files" are finally displayed - you can delete extra files, right click and open them - to have a look, save the list of duplicate files and ... the cool feature "Merge" them - which mean fslint will keep multiple version of the same files - but hardlink them - so the space is used only once.


Tip : Now the funny thing with Merge is that the feature is a bit counter intuitive - because if you select 2 files to be "Merged" - these 2 files will actually be excluded for the Merge ... so make sure you toggle your selection before "Merging"









Result is pretty impressive, after "Merging" all my duplicates - see below my Free space jumped to almost 19GB (remember I had only 8GB to begin with) - pretty cool stuff!



You can find other reviews for fslint here and there.

2 comments:

Unknown said...

suppose i have 10 identical files, then after running fslint i have one source file and 9 links.

-- What happens if I erase the source file to which the 9 links point to ?

Did I then in effect just t erase 10 files?

eromana

Alakazam said...

I think fslint does hard link - http://www.cyberciti.biz/faq/unix-creating-symbolic-link-ln-command/

Popular Posts