[erlang-questions] Erlang and slow dns resolver

Alexander Petrovsky askjuise@REDACTED
Thu Oct 23 09:02:30 CEST 2014


Hi Gokhan.

I've see you test results. The first test result make sense (populate
resolver cache):

3> timer:tc(fun() -> inet:gethostbyname("yahoo.com") end).
> {143889,
>  {ok,{hostent,"yahoo.com",[],inet,4,
>               [{98,138,253,109},{206,190,36,45},{98,139,183,24}]}}}


but the second, third, etc... I can explant to my self why you get a good
(us) results and I'm get bad (ms) results?

BTW the last two tests is wrong. You got in all values (min, max,median,
etc) zeros. Looks like something goes wrong. Could you please recheck and
try again?

2014-10-22 19:06 GMT+04:00 Gokhan Boranalp <kunthar@REDACTED>:

> Hi Alexander,
>
> I am sharing my results.
>
> https://gist.github.com/kunthar/cede1ccb72dcfe694f71
>
> I think this is related to your machine's dns settings.
> - Did you try to change dns servers and run tests again?
> - Did you check resolv.conf and hosts files for syntax etc.
>
> Here is my dns settings for reference:
>
> λ kunthar : cat /etc/resolv.conf
> #
> # Mac OS X Notice
> #
> # This file is not used by the host name and address resolution
> # or the DNS query routing mechanisms used by most processes on
> # this Mac OS X system.
> #
> # This file is automatically generated.
> #
> nameserver 156.154.70.1
> nameserver 156.154.71.1
> nameserver 8.8.4.4
>
>
>
>
>
>
>
> On Wed, Oct 22, 2014 at 12:59 PM, Alexander Petrovsky <askjuise@REDACTED>
> wrote:
>
>> Hi!
>>
>> tl;dr?!
>>
>> In a couple days ago I'm stumble upon bad performance erlang dns resolver
>> subsystem. I noticed that erlang resolver not so fast.
>>
>> By example:
>>
>> 1. via native method, via inet_gethost:
>>
>> (dns_test@REDACTED)1> inet:get_rc().
>> [{domain,"local"},
>>  {nameservers,{8,8,8,8}},
>>  {nameservers,{8,8,4,4}},
>>  {search,["local"]},
>>  {resolv_conf,"/etc/resolv.conf"},
>>  {hosts_file,"/etc/hosts"},
>>  {lookup,[native]}]
>>
>> (dns_test@REDACTED)2> timer:tc(fun() -> inet:gethostbyname("yahoo.com")
>> end).
>> {78302, *<--- request to inet_gethost via port*
>>  {ok,{hostent,"yahoo.com",[],inet,4,
>>               [{206,190,36,45},{98,139,183,24},{98,138,253,109}]}}}
>>
>> (dns_test@REDACTED)3> timer:tc(fun() -> inet:gethostbyname("yahoo.com")
>> end).
>> {74727, *<--- request to inet_gethost via port*
>>  {ok,{hostent,"yahoo.com",[],inet,4,
>>               [{206,190,36,45},{98,139,183,24},{98,138,253,109}]}}}
>>
>> 2. via dns method:
>>
>> (dns_test@REDACTED)1> inet:get_rc().
>> [{domain,"local"},
>>  {nameservers,{8,8,8,8}},
>>  {nameservers,{8,8,4,4}},
>>  {search,["local"]},
>>  {resolv_conf,"/etc/resolv.conf"},
>>  {hosts_file,"/etc/hosts"},
>>  {cache_size,1000},
>>  {lookup,[file,dns]}]
>>
>> (dns_test@REDACTED)2> timer:tc(fun() -> inet:gethostbyname("yahoo.com")
>> end).
>> {79489, *<--- request to remote DNS server*
>>  {ok,{hostent,"yahoo.com",[],inet,4,
>>               [{98,139,183,24},{98,138,253,109},{206,190,36,45}]}}}
>>
>> (dns_test@REDACTED)3> timer:tc(fun() -> inet:gethostbyname("yahoo.com")
>> end).
>> {143, *<--- request local cache*
>>  {ok,{hostent,"yahoo.com",[],inet,4,
>>               [{98,139,183,24},{98,138,253,109},{206,190,36,45}]}}}
>>
>> (dns_test@REDACTED)4> timer:tc(fun() -> inet:gethostbyname("yahoo.com")
>> end).
>> {143, *<--- request local cache*
>>  {ok,{hostent,"yahoo.com",[],inet,4,
>>               [{98,139,183,24},{98,138,253,109},{206,190,36,45}]}}}
>>
>>
>> But, when I make stress test like spawn 1000 procs that make 1000
>> simultaneous calls inet:gethostbyname/1, I get a performance degradation:
>>
>> S = self().
>> R = fun(Y, Acc) -> receive {time, X} ->  Y(Y, [X | Acc]) after 5000 ->
>> Acc end end.
>> W = fun(X) -> Itr = lists:seq(1, X), [spawn(fun() -> {T, _} =
>> timer:tc(fun() -> inet:gethostbyname("yahoo.com") end), S ! {time, T}
>> end) || _ <- Itr] end.
>> inet:gethostbyname("yahoo.com").
>>
>> 1. via native method, via inet_gethost:
>>
>> (dns_test@REDACTED)3> rp(begin spawn(fun() -> W(1000) end),
>> bear:get_statistics(R(R, [])) end).
>> [{min,478},
>>  {max,79351},
>>  {arithmetic_mean,45360.174},
>>  {geometric_mean,36733.733209560276},
>>  {harmonic_mean,19980.545407674203},
>>  {median,50804},
>>  {variance,470036105.12885255},
>>  {standard_deviation,21680.31607539089},
>>  {skewness,-0.4003204928175034},
>>  {kurtosis,-1.0786416653034996},
>>  {percentile,[{50,50804},
>>               {75,62816},
>>               {90,71738},
>>               {95,74148},
>>               {99,77927},
>>               {999,79285}]},
>>  {histogram,[{8478,56},
>>              {16478,78},
>>              {23478,83},
>>              {31478,96},
>>              {40478,73},
>>              {50478,108},
>>              {60478,198},
>>              {70478,187},
>>              {80478,121},
>>              {90478,0}]},
>>  {n,1000}]
>> ok
>>
>> 2. via dns method:
>>
>> (dns_test@REDACTED)3> rp(begin spawn(fun() -> W(1000) end),
>> bear:get_statistics(R(R, [])) end).
>> [{min,35},
>>  {max,22254},
>>  {arithmetic_mean,2481.493},
>>  {geometric_mean,578.8992039619226},
>>  {harmonic_mean,175.0856422999963},
>>  {median,396}, *<--- not so fast, why? resource contention?*
>>  {variance,13536424.050001001},
>>  {standard_deviation,3679.187960678416},
>>  {skewness,1.6528371477689106},
>>  {kurtosis,2.036594779004444},
>>  {percentile,[{50,396},
>>               {75,3827}, *<--- request from local cache, or cache
>> invalidation? why so slow?*
>>               {90,8586}, *<--- request from local cache, or cache
>> invalidation? why so slow?*
>>               {95,10793}, *<--- request from local cache, or cache
>> invalidation? why so slow?*
>>               {99,13498}, *<--- request from local cache, or cache
>> invalidation? why so slow?*
>>               {999,17155}]}, *<--- request from local cache, or cache
>> invalidation? why so slow?*
>>  {histogram,[{1335,645},
>>              {2635,57},
>>              {4035,57},
>>              {6035,63},
>>              {7035,35},
>>              {8035,25},
>>              {10035,52},
>>              {11035,19},
>>              {12035,17},
>>              {13035,15},
>>              {15035,12},
>>              {16035,0},
>>              {17035,0},
>>              {19035,2},
>>              {20035,0},
>>              {21035,0},
>>              {22035,0},
>>              {24035,1}]},
>>  {n,1000}]
>> ok
>>
>> --
>> Петровский Александр / Alexander Petrovsky,
>>
>> Skype: askjuise
>> Phone: +7 914 8 820 815
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
>
> --
> BR,
> \|/ Kunthar
>



-- 
Петровский Александр / Alexander Petrovsky,

Skype: askjuise
Phone: +7 914 8 820 815
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20141023/cd5aee27/attachment.htm>


More information about the erlang-questions mailing list