[erlang-questions] disk merging

Robert Raschke <>
Fri Oct 26 11:57:46 CEST 2007

> Does anybody know of a good algorithm to consolidate/merge all this
> data or do I have
> to write my own? One immediate thought is to compute the MD5 sums of
> all files on all
> disk and thus find all duplicates - then create a master copy of all
> unique files
> but the file names will be wrong and this might result in a big mess.
> This cannot be an uncommon problem - any ideas how to solve it?
> /Joe

For a low-level (i.e., file system) approach, have a look at Plan 9's Venti (consolidates on block level, not file):


I believe there's an implementation that can run as a user level program under Unix in http://swtch.com/plan9port/


More information about the erlang-questions mailing list