[erlang-questions] [HEADS UP] Emysql driver
Garrett Smith
g@REDACTED
Fri Jul 5 18:52:43 CEST 2013
I offhand can't think of an example of where the answer "it depends"
is more applicable :)
Here's how I approach this problem:
- Start by limiting the number of connections to the smallest
reasonable number - this usually falls out as the number of client
threads (sequential processing, so e.g. Erlang processes, or posix
threads in other environments)
- As your concurrency requirements increase -- and you start running
into disk IO bottlenecks on MySQL -- look to add memory and modify
your my.cnf to take advantage of it
As Loic said, this requires measuring and understanding performance
bottlenecks. In loaded scenarios this unfortunately gets quite
complicated. There's a small industry around MySQL performance
optimization.
At the 30K foot level, you're dealing with a shared disk problem --
and all else held constant, throwing more concurrent access at a disk
is bad idea. Memory/buffers will insulate you from the IO bottleneck,
but with trade offs (CPU cost, cache warming, db crash recovery). Move
down 15K feet and you're looking at the problems of sharing data --
databases, tables, rows, etc. and the complexity of transactions --
and that gets nasty complicated.
On Fri, Jul 5, 2013 at 10:42 AM, Barco You <barcojie@REDACTED> wrote:
> Then what should be a good number of acceptors and max_connections?
> depending on hardware? Can have 1 acceptor and thousands of
> max_connections? Is it only max_connections that determines the concurrency?
>
> On 5 Jul 2013 23:26, "Loïc Hoguin" <essen@REDACTED> wrote:
>>
>> On 07/05/2013 03:34 PM, Barco You wrote:
>>>
>>> Thank you all.
>>>
>>> Whats the maximum connections can it bear when add a pool? If I use
>>> Cowboy to provide a REST API for data query and I start up the http
>>> listener with 10000 acceptors and 10000 emysql connections, does that
>>> mean i can support 10000 concurrency?
>>
>>
>> Not answering your question.
>>
>> Acceptors aren't the maximum number of connections, it's the processes
>> doing the actual accept. They're only configurable to allow tweaking
>> performance, and with 10000 you will accept slower, that's too much. The
>> setting you probably meant is max_connections.
>>
>>> On 5 Jul 2013 14:35, "Mike Oxford" <moxford@REDACTED
>>> <mailto:moxford@REDACTED>> wrote:
>>>
>>> emysql_util also has some other methods ... as_record/1,
>>> as_proplist/1, as_dict/1
>>> Those will clean up the raw output into something far more
>>> usable/accessible.
>>>
>>> -mox
>>>
>>> On Thu, Jul 4, 2013 at 2:12 AM, Barco You <barcojie@REDACTED
>>> <mailto:barcojie@REDACTED>> wrote:
>>>
>>> Where can I find the detailed description about the fields for
>>> result_packet?
>>>
>>> The result_pakcet looks like {result_packet, Number, List1,
>>> List2, <<>>}, and seemingly only List2 is meaningful. Why is
>>> there Number, List1 and <<>> ?
>>>
>>>
>>> On Thu, Jul 4, 2013 at 4:37 PM, Barco You <barcojie@REDACTED
>>> <mailto:barcojie@REDACTED>> wrote:
>>>
>>> What is this part as following for?
>>>
>>> [{field,2,<<"def">>,<<"fangchan">>,<<"user">>,<<"user">>,
>>>
>>> <<"userid">>,<<"userid">>,8,<<>>,63,20,16899,0},
>>>
>>> {field,3,<<"def">>,<<"fangchan">>,<<"user">>,<<"user">>,
>>>
>>> <<"name">>,<<"name">>,254,<<>>,33,36,20489,0},
>>>
>>> {field,4,<<"def">>,<<"fangchan">>,<<"user">>,<<"user">>,
>>>
>>> <<"id_no">>,<<"id_no">>,254,<<>>,33,54,20481,0},
>>>
>>> {field,5,<<"def">>,<<"fangchan">>,<<"user">>,<<"user">>,
>>> <<"sex">>,<<"sex">>,1,<<>>,63,1,0,0},
>>>
>>> {field,6,<<"def">>,<<"fangchan">>,<<"user">>,<<"user">>,
>>> <<"age">>,<<"age">>,2,<<>>,63,6,0,0},
>>>
>>> {field,7,<<"def">>,<<"fangchan">>,<<"user">>,<<"user">>,
>>>
>>> <<"tel">>,<<"tel">>,254,<<>>,33,33,0,0},
>>>
>>> {field,8,<<"def">>,<<"fangchan">>,<<"user">>,<<"user">>,
>>>
>>> <<"email">>,<<"email">>,253,<<>>,33,132,0,0}],
>>>
>>>
>>> On Thu, Jul 4, 2013 at 4:35 PM, Barco You
>>> <barcojie@REDACTED <mailto:barcojie@REDACTED>> wrote:
>>>
>>> Solved, the bind-address of mhysql should be changed to
>>> 0.0.0.0
>>>
>>> But the result packet of emysql:execute looks very
>>> verbose and fuzzy. How can I get only the clean field
>>> value or a row, like {field, value} ?
>>>
>>> emysql:execute(db_pool, <<"select * from user where tel
>>> = 13761975289">>).
>>> {result_packet,11,
>>>
>>>
>>> [{field,2,<<"def">>,<<"fangchan">>,<<"user">>,<<"user">>,
>>>
>>> <<"userid">>,<<"userid">>,8,<<>>,63,20,16899,0},
>>>
>>> {field,3,<<"def">>,<<"fangchan">>,<<"user">>,<<"user">>,
>>>
>>> <<"name">>,<<"name">>,254,<<>>,33,36,20489,0},
>>>
>>> {field,4,<<"def">>,<<"fangchan">>,<<"user">>,<<"user">>,
>>>
>>> <<"id_no">>,<<"id_no">>,254,<<>>,33,54,20481,0},
>>>
>>> {field,5,<<"def">>,<<"fangchan">>,<<"user">>,<<"user">>,
>>>
>>> <<"sex">>,<<"sex">>,1,<<>>,63,1,0,0},
>>>
>>> {field,6,<<"def">>,<<"fangchan">>,<<"user">>,<<"user">>,
>>>
>>> <<"age">>,<<"age">>,2,<<>>,63,6,0,0},
>>>
>>> {field,7,<<"def">>,<<"fangchan">>,<<"user">>,<<"user">>,
>>>
>>> <<"tel">>,<<"tel">>,254,<<>>,33,33,0,0},
>>>
>>> {field,8,<<"def">>,<<"fangchan">>,<<"user">>,<<"user">>,
>>>
>>> <<"email">>,<<"email">>,253,<<>>,33,132,0,0}],
>>> [[1,
>>> <<231,138,185,230,157,176>>,
>>>
>>> <<"237689098736789870">>,1,32,<<"13761975289">>,
>>> <<"you.jie@REDACTED
>>> <mailto:you.jie@REDACTED>">>]],
>>> <<>>}
>>>
>>>
>>>
>>> On Thu, Jul 4, 2013 at 4:21 PM, Max Lapshin
>>> <max.lapshin@REDACTED <mailto:max.lapshin@REDACTED>>
>>> wrote:
>>>
>>> Barco, it means that you need to read manual for
>>> mysql server and
>>> enable socket listener.
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>
>>
>> --
>> Loďc Hoguin
>>
>> Erlang Cowboy
>> Nine Nines
>> http://ninenines.eu
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
More information about the erlang-questions
mailing list