[erlang-questions] how to design a scalable 'scheduler' app in erlang way
Thu Aug 26 18:20:04 CEST 2010
Your question is a little vague, so I'll just share an Erlang idiom that
- Spawn a new worker process for each request and have it handle the whole
request from beginning to end. Passing a partially completed request from
scheduler process to worker process is slow[er].
- Use a "server" process to serialize access to the resource.
- The scheduler sounds like a bottleneck. Remove if you can. Consider a
worker process that runs like this:
1. Receive request from file, socket or whatever.
2. Spawn a new process, identical to this one, to handle the next request.
3. Process request, using resource as needed.
4. Store/send result.
5. Terminate. (heap gets dropped, no garbage collection needed)
Does that give you some direction?
On Thu, Aug 19, 2010 at 2:53 AM, Oscar <> wrote:
> hi all,
> Here are the scheduler's requirements:
> 1. Scheduler should receive a lot of requests from other apps.
> 2. Scheduler should send the received requests to 'worker' apps.
> 3. Scheduler should determine when to send the received requests to
> 'worker' apps.
> The strategy is:
> a. A request depends on something, named *resource*.
> b. When *resource* is idle, scheduler sends the request to one worker.
> c. When *resource* is busy, scheduler stores the request.
> 4. Scheduler should be scalable, supporting about 1million tps.
> I've implemented it in the shared-state way. TPS is about 100,000, and
> 'lock' consumes the most CPU time.
> Now, I want to implement an erlang version, but have no idea about it.
> Shall I store the resource status, pending request to ETS? Will ETS be
> a bottleneck?
1 204 272 6149 (home/office)
1 204 470 8360 (mobile)
More information about the erlang-questions