[erlang-questions] [ANN] Asynchronous PostgreSQL driver, second release

[Anton Lebedevich]

Hello!

Publicly available benchmarks are in my todo list, haven't done them yet.
At my workplace I have write intensive application that does about 7
inserts/updates per client action and can tolerate some latency (seconds).
My first attempt was to parse all statements and bind/execute them. It
was terribly slow (~20 blocking network roundtrips per client action).
Second attempt was to store all incoming action into long list of
semicolon separated queries while waiting for previous query to finish
then run squery. Performance was acceptable (less than 1 non-blocking
network roundtrip per client action) but there was a room for
improvement (backend parsed same queries over and over).
Third attempt was to parse all statements and run apgsql:execute_batch
per client action. It was 2 times faster than squery approach.

Regards,
Anton Lebedevich.

On 02/24/2012 03:41 AM, Tim Watson wrote:
> Anton, this is very cool nice work.
> 
> Have you done much benchmarking with it? I have some reasonably sized
> schemas so I'll go play when I get a bit of time, but I can't publish
> them externally. I've been trying to figure out a good way to get hold
> of publicly available data sets in order to automate this, but just
> haven't got around to it.
> 
> Anyway I will definitely go play with the ipgsql API. If someone ever
> gets around to writing an EDBC API (which I've been dying to get
> around to but am just too busy for) then streaming delivery of rows is
> a must! :)
> 
> Cheers,
> 
> Tim
> 
> On 23 February 2012 13:28, Anton Lebedevich <mabrek@REDACTED> wrote:
>> Hello!
>>
>> https://github.com/mabrek/epgsql
>> branch named 'async'
>>
>> It's a fork of Will Glozer's epgsql.
>>
>> * Motivation
>>
>>  When you need to execute several queries it involves several network
>>  round-trips between your application and database.
>>  PostgreSQL frontend/backend protocol supports request pipelining.
>>  It means that you don't need to wait for previous command to finish
>>  before sending next command. This version of driver makes full use
>>  of the protocol feature allowing faster execution.
>>
>>
>> * Difference highlights
>>
>>  + 3 API sets: pgsql, apgsql and ipgsql:
>>    pgsql maintains backwards compatibility with original driver API,
>>    apgsql delivers complete results as regular erlang messages,
>>    ipgsql delivers results as messages incrementally (row by row)
>>  + internal queue of client requests, so you don't need to wait for
>> response to send next request
>>  + single process to hold driver state and receive socket data
>>  + execute several prepared statements as a batch
>>  + bind timestamps in erlang:now() format
>>  see CHANGES for full list.
>>
>>
>> Regards,
>> Anton Lebedevich.
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions