gen_server locked for some time

[Guilherme Andrade]

Hello Roberto,

If copying the data to a second process is not (too) costly, you can do
just that: have a second process responsible for writing the data - the I/O
bound component - and the original one act as a coordinator, directing the
second one asynchronously but always conscious of what the secondary
process is doing.
This means it will remain available to handle incoming calls (unless
seriously overloaded) and, if need be, reject incoming requests
preemptively for back pressure, whether instantaneously or after a timeout
measured on its own (rather than on the caller) by replying through
`gen_server:reply/2` rather than through `{reply, Reply, State}`.

Alternatively, you can also switch the problem around and use some sort of
broker to match write requests with the writer process - this has the
advantage of making it trivial to scale to multiple writer processes unless
there are hard serialization constraints.
With this approach, and in the simplest setup, a single broker process
never blocks and can forward the write requests to an available writer
process which has explicitly declared itself available for writing (and it
only does this between write requests), as well as manage timeouts. The
`sbroker` library[1], although no longer maintained, is a true wonder for
implementing this sort of pattern.

[1]: https://github.com/fishcakez/sbroker

On Fri, 29 Nov 2019 at 22:47, Roberto Ostinelli <ostinelli@REDACTED> wrote:

> All,
> I have a gen_server that in periodic intervals becomes busy, eventually
> over 10 seconds, while writing bulk incoming data. This gen_server also
> receives smaller individual data updates.
>
> I could offload the bulk writing routine to separate processes but the
> smaller individual data updates would then be processed before the bulk
> processing is over, hence generating an incorrect scenario where smaller
> more recent data gets overwritten by the bulk processing.
>
> I'm trying to see how to solve the fact that all the gen_server calls
> during the bulk update would timeout.
>
> Any ideas of best practices?
>
> Thank you,
> r.
>


-- 
Guilherme
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20191129/1c4e1f7a/attachment.htm>