[erlang-questions] Erlang and slow dns resolver

[Alexander Petrovsky]

Hi!

tl;dr?!

In a couple days ago I'm stumble upon bad performance erlang dns resolver
subsystem. I noticed that erlang resolver not so fast.

By example:

1. via native method, via inet_gethost:

(dns_test@REDACTED)1> inet:get_rc().
[{domain,"local"},
 {nameservers,{8,8,8,8}},
 {nameservers,{8,8,4,4}},
 {search,["local"]},
 {resolv_conf,"/etc/resolv.conf"},
 {hosts_file,"/etc/hosts"},
 {lookup,[native]}]

(dns_test@REDACTED)2> timer:tc(fun() -> inet:gethostbyname("yahoo.com")
end).
{78302, *<--- request to inet_gethost via port*
 {ok,{hostent,"yahoo.com",[],inet,4,
              [{206,190,36,45},{98,139,183,24},{98,138,253,109}]}}}

(dns_test@REDACTED)3> timer:tc(fun() -> inet:gethostbyname("yahoo.com")
end).
{74727, *<--- request to inet_gethost via port*
 {ok,{hostent,"yahoo.com",[],inet,4,
              [{206,190,36,45},{98,139,183,24},{98,138,253,109}]}}}

2. via dns method:

(dns_test@REDACTED)1> inet:get_rc().
[{domain,"local"},
 {nameservers,{8,8,8,8}},
 {nameservers,{8,8,4,4}},
 {search,["local"]},
 {resolv_conf,"/etc/resolv.conf"},
 {hosts_file,"/etc/hosts"},
 {cache_size,1000},
 {lookup,[file,dns]}]

(dns_test@REDACTED)2> timer:tc(fun() -> inet:gethostbyname("yahoo.com")
end).
{79489, *<--- request to remote DNS server*
 {ok,{hostent,"yahoo.com",[],inet,4,
              [{98,139,183,24},{98,138,253,109},{206,190,36,45}]}}}

(dns_test@REDACTED)3> timer:tc(fun() -> inet:gethostbyname("yahoo.com")
end).
{143, *<--- request local cache*
 {ok,{hostent,"yahoo.com",[],inet,4,
              [{98,139,183,24},{98,138,253,109},{206,190,36,45}]}}}

(dns_test@REDACTED)4> timer:tc(fun() -> inet:gethostbyname("yahoo.com")
end).
{143, *<--- request local cache*
 {ok,{hostent,"yahoo.com",[],inet,4,
              [{98,139,183,24},{98,138,253,109},{206,190,36,45}]}}}


But, when I make stress test like spawn 1000 procs that make 1000
simultaneous calls inet:gethostbyname/1, I get a performance degradation:

S = self().
R = fun(Y, Acc) -> receive {time, X} ->  Y(Y, [X | Acc]) after 5000 -> Acc
end end.
W = fun(X) -> Itr = lists:seq(1, X), [spawn(fun() -> {T, _} =
timer:tc(fun() -> inet:gethostbyname("yahoo.com") end), S ! {time, T} end)
|| _ <- Itr] end.
inet:gethostbyname("yahoo.com").

1. via native method, via inet_gethost:

(dns_test@REDACTED)3> rp(begin spawn(fun() -> W(1000) end),
bear:get_statistics(R(R, [])) end).
[{min,478},
 {max,79351},
 {arithmetic_mean,45360.174},
 {geometric_mean,36733.733209560276},
 {harmonic_mean,19980.545407674203},
 {median,50804},
 {variance,470036105.12885255},
 {standard_deviation,21680.31607539089},
 {skewness,-0.4003204928175034},
 {kurtosis,-1.0786416653034996},
 {percentile,[{50,50804},
              {75,62816},
              {90,71738},
              {95,74148},
              {99,77927},
              {999,79285}]},
 {histogram,[{8478,56},
             {16478,78},
             {23478,83},
             {31478,96},
             {40478,73},
             {50478,108},
             {60478,198},
             {70478,187},
             {80478,121},
             {90478,0}]},
 {n,1000}]
ok

2. via dns method:

(dns_test@REDACTED)3> rp(begin spawn(fun() -> W(1000) end),
bear:get_statistics(R(R, [])) end).
[{min,35},
 {max,22254},
 {arithmetic_mean,2481.493},
 {geometric_mean,578.8992039619226},
 {harmonic_mean,175.0856422999963},
 {median,396}, *<--- not so fast, why? resource contention?*
 {variance,13536424.050001001},
 {standard_deviation,3679.187960678416},
 {skewness,1.6528371477689106},
 {kurtosis,2.036594779004444},
 {percentile,[{50,396},
              {75,3827}, *<--- request from local cache, or cache
invalidation? why so slow?*
              {90,8586}, *<--- request from local cache, or cache
invalidation? why so slow?*
              {95,10793}, *<--- request from local cache, or cache
invalidation? why so slow?*
              {99,13498}, *<--- request from local cache, or cache
invalidation? why so slow?*
              {999,17155}]}, *<--- request from local cache, or cache
invalidation? why so slow?*
 {histogram,[{1335,645},
             {2635,57},
             {4035,57},
             {6035,63},
             {7035,35},
             {8035,25},
             {10035,52},
             {11035,19},
             {12035,17},
             {13035,15},
             {15035,12},
             {16035,0},
             {17035,0},
             {19035,2},
             {20035,0},
             {21035,0},
             {22035,0},
             {24035,1}]},
 {n,1000}]
ok

-- 
Петровский Александр / Alexander Petrovsky,

Skype: askjuise
Phone: +7 914 8 820 815
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20141022/fd9d39ff/attachment.htm>