[erlang-questions] disk merging

[Robert Raschke]

> Does anybody know of a good algorithm to consolidate/merge all this
> data or do I have
> to write my own? One immediate thought is to compute the MD5 sums of
> all files on all
> disk and thus find all duplicates - then create a master copy of all
> unique files
> but the file names will be wrong and this might result in a big mess.
> 
> This cannot be an uncommon problem - any ideas how to solve it?
> 
> /Joe

For a low-level (i.e., file system) approach, have a look at Plan 9's Venti (consolidates on block level, not file):

http://cm.bell-labs.com/sys/doc/venti/venti.html

I believe there's an implementation that can run as a user level program under Unix in http://swtch.com/plan9port/

Robby