[erlang-questions] Representation of a map in bytecode?

[Hugh Perkins]

Wow, Joe has replied in my thread.  That's cool :-)

I guess really we're never going to know for sure until we've got some
64-cores to play with.

I kindof tend to use real-world analogies to solve programming
problems.  Maybe this is why I always hated quantum physics where this
isnt possible.

In the real-world, a manager will assign tasks to people, then follow
up how the tasks are going.  Any tasks that are taking a while will be
shared among several people.  People who finish their tasks sooner
will be given new tasks.

The manager doesnt need to know how the tasks are actually done, or
need any prior knowledge of how long a task will take.

So, switching back to threading, I kindof see it like you have a
supervisor process which watches the processes around it.  When there
are cores free, it looks at which processes are (a) taking a while,
and (b) running some kind of parallelisable task (map, fold of
associative function, etc ...), and shares that task out among the
available cores.

In a later version, one could imagine that the supervisor process can
memorize (cache) which bits of the program tended to need
parallelizing out, so that in the future it proactively parallelizes
them out right off the bat.

"but everything ends up being choked by a serial bottleneck".  Well,
in the real-world tasks can be shared among thousands of people
successfully, in various ways.

"what about books, theyre always written by one person right?"  In the
case of books, we take a "mud sticks" strategy in the real world.
Many people write books, and a few of them turn out to be really good.

Parallelizing is only hard when you seek to run at close to 100% of
the single-core efficiency.  If one is willing to allow for a lot of
wastage, maybe each thread is only 10% as efficient as in a
single-threaded application, it's maybe easy to share among thousands
of cores?