[erlang-questions] Erlang and slow dns resolver
Gokhan Boranalp
kunthar@REDACTED
Wed Oct 22 17:06:41 CEST 2014
Hi Alexander,
I am sharing my results.
https://gist.github.com/kunthar/cede1ccb72dcfe694f71
I think this is related to your machine's dns settings.
- Did you try to change dns servers and run tests again?
- Did you check resolv.conf and hosts files for syntax etc.
Here is my dns settings for reference:
λ kunthar : cat /etc/resolv.conf
#
# Mac OS X Notice
#
# This file is not used by the host name and address resolution
# or the DNS query routing mechanisms used by most processes on
# this Mac OS X system.
#
# This file is automatically generated.
#
nameserver 156.154.70.1
nameserver 156.154.71.1
nameserver 8.8.4.4
On Wed, Oct 22, 2014 at 12:59 PM, Alexander Petrovsky <askjuise@REDACTED>
wrote:
> Hi!
>
> tl;dr?!
>
> In a couple days ago I'm stumble upon bad performance erlang dns resolver
> subsystem. I noticed that erlang resolver not so fast.
>
> By example:
>
> 1. via native method, via inet_gethost:
>
> (dns_test@REDACTED)1> inet:get_rc().
> [{domain,"local"},
> {nameservers,{8,8,8,8}},
> {nameservers,{8,8,4,4}},
> {search,["local"]},
> {resolv_conf,"/etc/resolv.conf"},
> {hosts_file,"/etc/hosts"},
> {lookup,[native]}]
>
> (dns_test@REDACTED)2> timer:tc(fun() -> inet:gethostbyname("yahoo.com")
> end).
> {78302, *<--- request to inet_gethost via port*
> {ok,{hostent,"yahoo.com",[],inet,4,
> [{206,190,36,45},{98,139,183,24},{98,138,253,109}]}}}
>
> (dns_test@REDACTED)3> timer:tc(fun() -> inet:gethostbyname("yahoo.com")
> end).
> {74727, *<--- request to inet_gethost via port*
> {ok,{hostent,"yahoo.com",[],inet,4,
> [{206,190,36,45},{98,139,183,24},{98,138,253,109}]}}}
>
> 2. via dns method:
>
> (dns_test@REDACTED)1> inet:get_rc().
> [{domain,"local"},
> {nameservers,{8,8,8,8}},
> {nameservers,{8,8,4,4}},
> {search,["local"]},
> {resolv_conf,"/etc/resolv.conf"},
> {hosts_file,"/etc/hosts"},
> {cache_size,1000},
> {lookup,[file,dns]}]
>
> (dns_test@REDACTED)2> timer:tc(fun() -> inet:gethostbyname("yahoo.com")
> end).
> {79489, *<--- request to remote DNS server*
> {ok,{hostent,"yahoo.com",[],inet,4,
> [{98,139,183,24},{98,138,253,109},{206,190,36,45}]}}}
>
> (dns_test@REDACTED)3> timer:tc(fun() -> inet:gethostbyname("yahoo.com")
> end).
> {143, *<--- request local cache*
> {ok,{hostent,"yahoo.com",[],inet,4,
> [{98,139,183,24},{98,138,253,109},{206,190,36,45}]}}}
>
> (dns_test@REDACTED)4> timer:tc(fun() -> inet:gethostbyname("yahoo.com")
> end).
> {143, *<--- request local cache*
> {ok,{hostent,"yahoo.com",[],inet,4,
> [{98,139,183,24},{98,138,253,109},{206,190,36,45}]}}}
>
>
> But, when I make stress test like spawn 1000 procs that make 1000
> simultaneous calls inet:gethostbyname/1, I get a performance degradation:
>
> S = self().
> R = fun(Y, Acc) -> receive {time, X} -> Y(Y, [X | Acc]) after 5000 -> Acc
> end end.
> W = fun(X) -> Itr = lists:seq(1, X), [spawn(fun() -> {T, _} =
> timer:tc(fun() -> inet:gethostbyname("yahoo.com") end), S ! {time, T}
> end) || _ <- Itr] end.
> inet:gethostbyname("yahoo.com").
>
> 1. via native method, via inet_gethost:
>
> (dns_test@REDACTED)3> rp(begin spawn(fun() -> W(1000) end),
> bear:get_statistics(R(R, [])) end).
> [{min,478},
> {max,79351},
> {arithmetic_mean,45360.174},
> {geometric_mean,36733.733209560276},
> {harmonic_mean,19980.545407674203},
> {median,50804},
> {variance,470036105.12885255},
> {standard_deviation,21680.31607539089},
> {skewness,-0.4003204928175034},
> {kurtosis,-1.0786416653034996},
> {percentile,[{50,50804},
> {75,62816},
> {90,71738},
> {95,74148},
> {99,77927},
> {999,79285}]},
> {histogram,[{8478,56},
> {16478,78},
> {23478,83},
> {31478,96},
> {40478,73},
> {50478,108},
> {60478,198},
> {70478,187},
> {80478,121},
> {90478,0}]},
> {n,1000}]
> ok
>
> 2. via dns method:
>
> (dns_test@REDACTED)3> rp(begin spawn(fun() -> W(1000) end),
> bear:get_statistics(R(R, [])) end).
> [{min,35},
> {max,22254},
> {arithmetic_mean,2481.493},
> {geometric_mean,578.8992039619226},
> {harmonic_mean,175.0856422999963},
> {median,396}, *<--- not so fast, why? resource contention?*
> {variance,13536424.050001001},
> {standard_deviation,3679.187960678416},
> {skewness,1.6528371477689106},
> {kurtosis,2.036594779004444},
> {percentile,[{50,396},
> {75,3827}, *<--- request from local cache, or cache
> invalidation? why so slow?*
> {90,8586}, *<--- request from local cache, or cache
> invalidation? why so slow?*
> {95,10793}, *<--- request from local cache, or cache
> invalidation? why so slow?*
> {99,13498}, *<--- request from local cache, or cache
> invalidation? why so slow?*
> {999,17155}]}, *<--- request from local cache, or cache
> invalidation? why so slow?*
> {histogram,[{1335,645},
> {2635,57},
> {4035,57},
> {6035,63},
> {7035,35},
> {8035,25},
> {10035,52},
> {11035,19},
> {12035,17},
> {13035,15},
> {15035,12},
> {16035,0},
> {17035,0},
> {19035,2},
> {20035,0},
> {21035,0},
> {22035,0},
> {24035,1}]},
> {n,1000}]
> ok
>
> --
> Петровский Александр / Alexander Petrovsky,
>
> Skype: askjuise
> Phone: +7 914 8 820 815
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
--
BR,
\|/ Kunthar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20141022/6e02e95a/attachment.htm>
More information about the erlang-questions
mailing list