[erlang-questions] Process pool map/3 implementation

[David Mercer]

I was curious about that, too.  Hoping you'll get a response...

> -----Original Message-----
> From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-
> bounces@REDACTED] On Behalf Of Robert Virding
> Sent: Wednesday, July 20, 2011 8:42 PM
> To: Parnell Springmeyer
> Cc: erlang-questions
> Subject: Re: [erlang-questions] Process pool map/3 implementation
> 
> One quick question: what was wrong with the straightforward solution of
> just spawning one process for each element in the list? Did this break
> or do you actually need more control?
> 
> Robert
> 
> ----- "Parnell Springmeyer" <ixmatus@REDACTED> wrote:
> 
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > For a work project I have a large list (thousands of items) to
> > process
> > and at first built a "pmap" implementation as per Joe's book until I
> > found the plists module (which is awesome btw).
> >
> > There is one glaring issue with the list -> subdivide -> spawn x
> > processes for n sublist items strategy; if an item in the sublist
> > takes
> > longer than all the other items it blocks the entire resource
> > allotment
> > until it is done.
> >
> > In most cases, the plists/pmap implementation works just fine because
> > the items in the list probably don't take more than a few
> > milliseconds
> > to map the fun over. However, it does become an issue when that is
> > not
> > the case.
> >
> > So, I figured the next best strategy would be to implement a process
> > pool since it would allow for slow running processes to continue
> > their
> > work while finished processes can die and new processes spawned into
> > the
> > pool ready for work - so none of the resources are sitting idle.
> >
> > Right now, my module isn't nearly as feature-complete as the plists
> > module is - this is only a drop in replacement for map. Please submit
> > your criticisms and comments to me at this address.
> >
> > You may find the code on BitBucket:
> > https://bitbucket.org/ixmatus/ppool
> >
> > - --
> > Parnell "ixmatus" Springmeyer (http://ixmat.us)
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> > Comment: GPGTools - http://gpgtools.org
> >
> > iQEcBAEBAgAGBQJOJmKwAAoJEPvtlbpI1POL+asIAKPcR0SOw67hFwwIbmkf89sS
> > 4+Zx9hx1V/+86OVtXcqcOY+yxNcHezNEKkw8z2XHmDAWbeOl3bbINFySRXbQVydV
> > 854lArqCHRG+ZlJ6ZrgecXKf9mG8ldbK1InwEZWOVZBj63rhmloMaGiyTzmxA88S
> > 7mDNS4uhhpvRT2znpnsWt1x12IAzeayV0hf5/BLjp+b5FMZPc9oSa4n5uzyA9AVW
> > +av6hyuFfK32lhxUb4u3bVMaHOf2n/YwJexS25+NODcpkI3BLXNkrmKwgz8Lv/sA
> > omKzKTiuhpa0vTM+TLI9pn82GCJLdD+ON9DDOFN4ww+BnmXjhykiicBQCg7yhtQ=
> > =GP7K
> > -----END PGP SIGNATURE-----
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://erlang.org/mailman/listinfo/erlang-questions
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions