[erlang-questions] Application granularity (was: Parallel Shootout & a style question)
Fri Sep 5 16:35:16 CEST 2008
> the problem is that it doesn't really matter how many processes
> you have. to make use of a 1,000 core machine you'll need to have
> 1,000 *runnable* processes at any one time.
> E.g. a large phone switch, that handles 10,000 ongoing calls, will
> typically have ~10 runnable processes (given that you model a call
> as a process). I.e. for such a system it will buy you nothing to
> use more than 10 cores.
Quite right, but that's where the architecture challenge is. There
has to be some areas of the problem where it can be split differently
if you assume the hardware is constructed differently. For example,
why not one process per subscriber for services lookup rather than a
I guess the packet switching is done by other hardware than the
erlang router. Surely there are more than 10 packets to route at a
time? The granularity of the router process could be reduced by
spreading packets across 100s of router processes providing the
synchronization of order can be maintained. I don't know enough
about your application domain to comment. I can see ways that
webservers can be sped up, not by parsing requests and documents into
trees, but by treating their data transfers as streams that can be
chopped into lots of tiny pieces.
I agree that some applications are just sequential in nature. But I
can still imagine ways to carve Microsoft Word and the browser into
100s of processes. You may not get complete speed up, but you will
get fine-grained responsiveness. We've just gotten used to
architecting large applications (and here I am mainly referring to PC
applications and traditional IT services), so it is difficult to
imagine a radical alternative to design and execution of software.
More information about the erlang-questions