[erlang-questions] disk merging

[Joe Armstrong]

I have an interesting? problem.

Over the last ? years (> 10) I have been upgrading my home system
this usually involved buying a bigger disk and copying most (or all)
of the files
from the old disk to the new disk or disks.

I've also been backing up the family photos etc on USB disks.

Now I have > 1 Tera bytes of files spread over c. 10 computers and
3 pluggable USB disks. Having made a "backup" both the original and the
copy live lives of their own.

Does anybody know of a good algorithm to consolidate/merge all this
data or do I have
to write my own? One immediate thought is to compute the MD5 sums of
all files on all
disk and thus find all duplicates - then create a master copy of all
unique files
but the file names will be wrong and this might result in a big mess.

This cannot be an uncommon problem - any ideas how to solve it?

/Joe