[erlang-questions] Application granularity (was: Parallel Shootout & a style question)

[Jay Nelson]

>  the problem is that it doesn't really matter how many processes  
> you have. to make use of a 1,000 core machine you'll need to have  
> 1,000 *runnable* processes at any one time.
>  E.g. a large phone switch, that handles 10,000 ongoing calls, will  
> typically have ~10 runnable processes (given that you model a call  
> as a process). I.e. for such a system it will buy you nothing to  
> use more than 10 cores.
>
>  mats
>

Quite right, but that's where the architecture challenge is.  There  
has to be some areas of the problem where it can be split differently  
if you assume the hardware is constructed differently.  For example,  
why not one process per subscriber for services lookup rather than a  
central database?

I guess the packet switching is done by other hardware than the  
erlang router.  Surely there are more than 10 packets to route at a  
time?  The granularity of the router process could be reduced by  
spreading packets across 100s of router processes providing the  
synchronization of order can be maintained.  I don't know enough  
about your application domain to comment.  I can see ways that  
webservers can be sped up, not by parsing requests and documents into  
trees, but by treating their data transfers as streams that can be  
chopped into lots of tiny pieces.

I agree that some applications are just sequential in nature.  But I  
can still imagine ways to carve Microsoft Word and the browser into  
100s of processes.  You may not get complete speed up, but you will  
get fine-grained responsiveness.  We've just gotten used to  
architecting large applications (and here I am mainly referring to PC  
applications and traditional IT services), so it is difficult to  
imagine a radical alternative to design and execution of software.

jay