[erlang-questions] Erlang DataBase Connectivity (was trouble with erlang or erlang is a ghetto)

[Tim Watson]

On 1 August 2011 09:20, Alex Arnon <alex.arnon@REDACTED> wrote:
>
> I agree, it does feel unnatural. However, in this case:
> 1. It's rather simple, easy to make consistent and stable.
> 2. It enables us to deliver immediate value to the customer... HELP I'VE GOT
> ENTERPRISITIS!!!

Ok fine, but I need to see your code in order to know how to plug it
into any API we dream up. :)

> But seriously:
> 2. It makes any and all RDBMS's available, immediately.
> 3. It's a fallback.
>

Fine let's do it.

>
> Right.
> One beneficial side effect of streaming result sets is that it enables
> processing enormous sets without blowing up your process/node.
> Transformation/map/reduce/whichever is probably a good starting point.
>

I agree that streaming result sets is useful and we'll put something
in for that. I also want to do a basic "transform the result set using
this function" API - let's have both.

>>
>> Is your JDBC thing open source? I might have a stab at this just for
>> fun - maybe fronting your library and the ESL postgres API as the
>> first cut.
>>
>
> It was a prototype, got to the
> usable-but-hell-no-you're-not-seeing-this-pile stage. :)
> I don't think I've even updated my main repo with the last changes in a few
> months.
>
> Basically what I did was:
> 1. register a global controller (edbc_master). This is optional, meant to be
> used for general monitoring of the connection processes.
> 2. Each connection is an {Erlang process, JNode} pair, linked to its
> creator/user and registered with 'edbc_master'.
>     The connection is started with a configuration such as:
>     { { proxy_dir, "c:/dev/edbc/java/deploy" },
>       { driver_class, "com.oracle.xxx.yyy.OracleDriver" },
>       { hostname, "localhost" },
>       { port, 4444 },
>       ...
>       other connection-stringy stuff
>     }
> 3. The Controller process (the linked Erlang process) gets a "unique" node
> name for the JNode from the edbc_master, and spawns it.
> 4. A basic handshake with X-second timeout between the Controller process
> and a counterpart on the JNode (registered as 'edbc_proxy' on the JNode I
> think) is done, including adding monitors on success.
> 5. From this point on, operations are forwarded via the Controller to the
> JNode, which executes them, packages up the results and sends them back.
>
> What I've done is somewhat crude - a process per connection etc., no proper
> support for most datatypes - but it was just a couple of days' work to get a
> simple SELECT/UPDATE/INSERT working. And it would work on Oracle, MSSql,
> Postgres, MySQL, Sybase, what have you.
>
>

Yeah I'm terribly well versed in JDBC and have a vague inkling of how
JNodes work, so that's all fine. Like I said, let's glue together a
basic API and plug it into your thing and one other popular API -
perhaps the OTP odbc application would actually be the best thing
alongside yours - and then we'll go from there. Once we have something
working we can come back here and ask the community for input.