exception exit: timeout in gen_server:call

[Tristan Sloughter]

For metrics and tracing the pgo (https://github.com/erleans/pgo) library is a Postgres lib with built in pool that is currently instrumented with telemetry (https://github.com/beam-telemetry/telemetry) for metrics and can optionally create spans with opencensus (https://github.com/census-instrumentation/opencensus-erlang).

Hm, looks like I haven't yet added the `queue_time`to the metrics, I'll try to remember to do that soon when I replace OpenCensus with OpenTelemetry (https://github.com/open-telemetry/opentelemetry-erlang) https://opentelemetry.io/. At the same time I'll be expanding the available metrics to include information about the pool.

So only the query time is sent in the telemetry event at this time.

Tristan

On Wed, Dec 4, 2019, at 01:33, Roger Lipscombe wrote:
> On Tue, 3 Dec 2019 at 19:32, Jesper Louis Andersen
> <jesper.louis.andersen@REDACTED> wrote:
> > * Measure the maximal latency over the equery call you've made for a 10-15 second period. Plot it.
> 
> We wrap our database calls (and pool checkouts) in
> folsom_metrics:histogram_timed_update, which reports percentiles to
> graphite. Since folsom is (afaict) abandoned, you could see if (e.g.)
> exometer has something similar. We also report connections-in-use,
> etc., which is helpful.
> 
> > * Pipeline the queries. A query which waits for an answer affects every sibling query as well.
> 
> Other pooling models are available. Pull, rather than push, might help
> -- because it could pull N pending requests and pipeline them. Can't
> find the relevant links right now, though.
> 
> 
> >
> > Down the line:
> >
> > * Postgres can log slow queries. Turn that on.
> > * Postgres can log whenever it holds a lock for more than a certain time window. Turn that on.
> >
> > Narrow down where the problem can occur by having systems provide facts to you. Don't go for "what is wrong?" Go for "What would I like to know?". This helps observability (In the Control Theory / Charity Majors sense).
> >
>