From unexplained@REDACTED Sat Jun 1 13:14:10 2019 From: unexplained@REDACTED (Siliwangi Softworks) Date: Sat, 1 Jun 2019 18:14:10 +0700 Subject: [erlang-questions] Compiling erlang with cygwin64 and msvc 2019? Message-ID: Hello, I am trying to compile latest maint-22 branch from the github with cygwin64 and msvc 2019, on compilation it always stopped at : make[3]: Entering directory '/home/echelon/src/otp/lib/public_key/src' ERLC ../ebin/public_key.beam ERLC ../ebin/pubkey_pem.beam ERLC ../ebin/pubkey_ssh.beam ERLC ../ebin/pubkey_pbe.beam ERLC ../ebin/pubkey_cert.beam ERLC ../ebin/pubkey_cert_records.beam ERLC ../ebin/pubkey_crl.beam VSN ../ebin/public_key.app VSN ../ebin/public_key.appup make[3]: Leaving directory '/home/echelon/src/otp/lib/public_key/src' make[3]: Entering directory '/home/echelon/src/otp/lib/public_key/doc/src' make[3]: Nothing to be done for 'opt'. make[3]: Leaving directory '/home/echelon/src/otp/lib/public_key/doc/src' === Leaving application public_key make[2]: Leaving directory '/home/echelon/src/otp/lib/public_key' make[2]: Entering directory '/home/echelon/src/otp/lib/ssl' === Entering application ssl make[3]: Entering directory '/home/echelon/src/otp/lib/ssl/src' GEN /home/echelon/src/otp/lib/ssl/src/deps/ssl.d ERLC ../ebin/ssl_crl_cache_api.beam ERLC ../ebin/ssl_session_cache_api.beam make[3]: *** No rule to make target 'c:/cygwin64../../../BOOTST~1/lib/kernel/include/logger.hrl', needed by '../ebin/dtls_connection.beam'. Stop. make[3]: Leaving directory '/home/echelon/src/otp/lib/ssl/src' make[2]: *** [/home/echelon/src/otp/make/otp_subdir.mk:29: opt] Error 2 make[2]: Leaving directory '/home/echelon/src/otp/lib/ssl' make[1]: *** [/home/echelon/src/otp/make/otp_subdir.mk:29: opt] Error 2 make[1]: Leaving directory '/home/echelon/src/otp/lib' make: *** [Makefile:490: libs] Error 2 I have modified my .bash_profile, this setup is managed to compile the maint-21 branch cleanly, but getting error when compiling maint-22. Can anybody help me?. Thanks, -------------- next part -------------- An HTML attachment was scrubbed... URL: From unexplained@REDACTED Sat Jun 1 16:11:31 2019 From: unexplained@REDACTED (Siliwangi Softworks) Date: Sat, 1 Jun 2019 21:11:31 +0700 Subject: [erlang-questions] Compiling erlang with cygwin64 and msvc 2019? In-Reply-To: References: Message-ID: Found out the fix, in ..\src\otp\lib\ssl\src\Makefile file, replace line 156 the : | sed "s@$(ERL_TOP)@../../..@REDACTED" \ with | sed "s@REDACTED/../..@REDACTED" \ This is remove cygwin absolute path in the ssl.d dependencies file. Should i make a pull request?, i am not sure on why we need absolute path in the ssl.d file. On Sat, Jun 1, 2019 at 6:14 PM Siliwangi Softworks wrote: > Hello, > > I am trying to compile latest maint-22 branch from the github with > cygwin64 and msvc 2019, on compilation it always stopped at : > make[3]: Entering directory '/home/echelon/src/otp/lib/public_key/src' > ERLC ../ebin/public_key.beam > ERLC ../ebin/pubkey_pem.beam > ERLC ../ebin/pubkey_ssh.beam > ERLC ../ebin/pubkey_pbe.beam > ERLC ../ebin/pubkey_cert.beam > ERLC ../ebin/pubkey_cert_records.beam > ERLC ../ebin/pubkey_crl.beam > VSN ../ebin/public_key.app > VSN ../ebin/public_key.appup > make[3]: Leaving directory '/home/echelon/src/otp/lib/public_key/src' > make[3]: Entering directory '/home/echelon/src/otp/lib/public_key/doc/src' > make[3]: Nothing to be done for 'opt'. > make[3]: Leaving directory '/home/echelon/src/otp/lib/public_key/doc/src' > === Leaving application public_key > make[2]: Leaving directory '/home/echelon/src/otp/lib/public_key' > make[2]: Entering directory '/home/echelon/src/otp/lib/ssl' > === Entering application ssl > make[3]: Entering directory '/home/echelon/src/otp/lib/ssl/src' > GEN /home/echelon/src/otp/lib/ssl/src/deps/ssl.d > ERLC ../ebin/ssl_crl_cache_api.beam > ERLC ../ebin/ssl_session_cache_api.beam > make[3]: *** No rule to make target > 'c:/cygwin64../../../BOOTST~1/lib/kernel/include/logger.hrl', needed by > '../ebin/dtls_connection.beam'. Stop. > make[3]: Leaving directory '/home/echelon/src/otp/lib/ssl/src' > make[2]: *** [/home/echelon/src/otp/make/otp_subdir.mk:29: opt] Error 2 > make[2]: Leaving directory '/home/echelon/src/otp/lib/ssl' > make[1]: *** [/home/echelon/src/otp/make/otp_subdir.mk:29: opt] Error 2 > make[1]: Leaving directory '/home/echelon/src/otp/lib' > make: *** [Makefile:490: libs] Error 2 > > I have modified my .bash_profile, this setup is managed to compile the > maint-21 branch cleanly, but getting error when compiling maint-22. > > Can anybody help me?. > > Thanks, > -------------- next part -------------- An HTML attachment was scrubbed... URL: From okaprinarjaya@REDACTED Sun Jun 2 12:25:22 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Sun, 2 Jun 2019 17:25:22 +0700 Subject: [erlang-questions] Why use -record() ? Message-ID: Hi, I still learn Erlang OTP from learnyousomeerlang.com especially part https://learnyousomeerlang.com/building-applications-with-otp . Let's see code below: -module(ppool_serv). -behaviour(gen_server). .... .... -record(state, {limit=0, sup, refs, queue=queue:new()}). .... .... init({Limit, MFA, Sup}) -> self() ! {start_worker_supervisor, Sup, MFA}, %% why using -record here for state holder? {ok, #state{limit=Limit, refs=gb_sets:empty()}}. >From the code above, why using record to hold state? Other than using record we also can use tuple right? Let's see alternative code below -module(ppool_serv). -behaviour(gen_server). .... .... .... init({Limit, MFA, Sup}) -> self() ! {start_worker_supervisor, Sup, MFA}, Refs = gb_sets:empty(), Queue = queue:new(), Spv = undefined, State = {Limit, Spv, Refs, Queue}, {ok, State}. ... ... ... handle_call({run, Args}, _From, {Limit, Spv, Refs, Queue}) when N > 0 -> {ok, Pid} = supervisor:start_child(Spv, Args), Ref = erlang:monitor(process, Pid), NewRefs = gb_sets:add(Ref, Refs), NewState = {Limit-1, Spv, NewRefs, Queue}, {reply, {ok, Pid}, NewState}; We also can use the above alternative code right? Please enlightenment Thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: From hellkvist@REDACTED Sun Jun 2 13:25:32 2019 From: hellkvist@REDACTED (Stefan Hellkvist) Date: Sun, 2 Jun 2019 13:25:32 +0200 Subject: [erlang-questions] Why use -record() ? In-Reply-To: References: Message-ID: > ... > handle_call({run, Args}, _From, {Limit, Spv, Refs, Queue}) when N > 0 -> > {ok, Pid} = supervisor:start_child(Spv, Args), > Ref = erlang:monitor(process, Pid), > NewRefs = gb_sets:add(Ref, Refs), > NewState = {Limit-1, Spv, NewRefs, Queue}, > {reply, {ok, Pid}, NewState}; > > We also can use the above alternative code right? Yes, you can use whatever Erlang term you prefer - record, tuple, map, a list of maps containing records of tuples... What you choose is a matter of taste and depends on your requirements. A record for instance has the advantage over a tuple, that you access elements by name and the order of elements therefore becomes irrelevant. This could give advantages if you one day decide to add another field to your state. With the tuple approach this would likely break every access to the tuple everywhere in the code, but with a record where you access and match by names most of your code might still work...plus you never need to remember what meaning the third or fourth element in your tuple has l, because it has a descriptive name. Stefan -------------- next part -------------- An HTML attachment was scrubbed... URL: From himars@REDACTED Sun Jun 2 13:33:57 2019 From: himars@REDACTED (Jack Tang) Date: Sun, 2 Jun 2019 19:33:57 +0800 Subject: [erlang-questions] Why use -record() ? In-Reply-To: References: Message-ID: Yes, `state` can be any data structure of erlang, and `record` is another form of `tuple`. `-record(state....` equals to `{state, ....}` I prefer record to tuple in some cases because the element can be matched/retrieved without considering the order. the example: `#state{refs = Refs} = State`, do the same thing using tuple, you should take care of the order and must fully matched {_, _, Ref, _, _} = State -Jack On Sun, Jun 2, 2019 at 6:25 PM I Gusti Ngurah Oka Prinarjaya < okaprinarjaya@REDACTED> wrote: > Hi, > > I still learn Erlang OTP from learnyousomeerlang.com especially part > https://learnyousomeerlang.com/building-applications-with-otp . > > Let's see code below: > > -module(ppool_serv). > -behaviour(gen_server). > .... > .... > -record(state, {limit=0, sup, refs, queue=queue:new()}). > .... > .... > init({Limit, MFA, Sup}) -> > self() ! {start_worker_supervisor, Sup, MFA}, > > %% why using -record here for state holder? > {ok, #state{limit=Limit, refs=gb_sets:empty()}}. > > From the code above, why using record to hold state? > > Other than using record we also can use tuple right? Let's see alternative > code below > > -module(ppool_serv). > -behaviour(gen_server). > .... > .... > .... > init({Limit, MFA, Sup}) -> > self() ! {start_worker_supervisor, Sup, MFA}, > > Refs = gb_sets:empty(), > Queue = queue:new(), > Spv = undefined, > State = {Limit, Spv, Refs, Queue}, > > {ok, State}. > ... > ... > ... > handle_call({run, Args}, _From, {Limit, Spv, Refs, Queue}) when N > 0 -> > {ok, Pid} = supervisor:start_child(Spv, Args), > Ref = erlang:monitor(process, Pid), > NewRefs = gb_sets:add(Ref, Refs), > NewState = {Limit-1, Spv, NewRefs, Queue}, > {reply, {ok, Pid}, NewState}; > > We also can use the above alternative code right? > > Please enlightenment > > > Thank you > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Jack Tang http://www.linkedin.com/in/jacktang -------------- next part -------------- An HTML attachment was scrubbed... URL: From cean.ebengt@REDACTED Sun Jun 2 13:34:57 2019 From: cean.ebengt@REDACTED (bengt) Date: Sun, 2 Jun 2019 13:34:57 +0200 Subject: [erlang-questions] Why use -record() ? In-Reply-To: References: Message-ID: <12F6D64F-82E1-4733-94C3-C064D07B823D@gmail.com> Greetings, It is true that it is a matter of taste but remember that some Erlang containers can not be used to pattern match. Using one of those handle_call/3 would only be one function clause. And while maps can be pattern matched the compiler will not help you with spelling mistakes, as it does with records. Best Wishes, bengt > On 2 Jun 2019, at 13:25, Stefan Hellkvist > wrote: > > >> ... >> handle_call({run, Args}, _From, {Limit, Spv, Refs, Queue}) when N > 0 -> >> {ok, Pid} = supervisor:start_child(Spv, Args), >> Ref = erlang:monitor(process, Pid), >> NewRefs = gb_sets:add(Ref, Refs), >> NewState = {Limit-1, Spv, NewRefs, Queue}, >> {reply, {ok, Pid}, NewState}; >> >> We also can use the above alternative code right? > > Yes, you can use whatever Erlang term you prefer - record, tuple, map, a list of maps containing records of tuples... > > What you choose is a matter of taste and depends on your requirements. A record for instance has the advantage over a tuple, that you access elements by name and the order of elements therefore becomes irrelevant. This could give advantages if you one day decide to add another field to your state. With the tuple approach this would likely break every access to the tuple everywhere in the code, but with a record where you access and match by names most of your code might still work...plus you never need to remember what meaning the third or fourth element in your tuple has l, because it has a descriptive name. > > Stefan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Sun Jun 2 13:57:30 2019 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Sun, 2 Jun 2019 14:57:30 +0300 Subject: [erlang-questions] Why use -record() ? In-Reply-To: <12F6D64F-82E1-4733-94C3-C064D07B823D@gmail.com> References: <12F6D64F-82E1-4733-94C3-C064D07B823D@gmail.com> Message-ID: Hello, There was quite good reasoning already about records. I just want to make a quotation from my resent post about records. I hope it helps you to catch the reason why? Type theory and a functional programming operates with [algebraic data types](https://en.wikipedia.org/wiki/Algebraic_data_type). They are known as a composition of other types. The theory defines two classes of compositions: product types (tuples, records) and co-product types (sum, enumeration or variant types). Product types are strongly expressed by [records](http://erlang.org/doc/reference_manual/records.html) in Erlang. ```erlang -type fullname() :: binary(). -type address() :: binary(). -type city() :: binary(). -record(person, { name :: fullname(), address :: address(), city :: city() }). ``` The beauty of Erlang records (product type) is that they definitions are only available at compile time. The compiler has complete knowledge of defined "algebra" and catches misuse errors. The usage of records in your code benefits to write correct, maintainable code and support refactoring. Use them to define your domain models! ```erlang #person{birthday = "18810509"}. %% Compiling src/person.erl failed %% src/person.erl:18: field birthday undefined in record person ``` There are few other benefits of records over other data types: type testing and pattern matching. These subject has been widely covered at [official documentation](http://erlang.org/doc/programming_examples/records.html): ```erlang %% %% type testing myfun(X) when is_record(X, person) -> ... %% %% pattern matching myfun(#person{name = Name}) -> ... ``` As a summary, I would strongly advertise the usage of records to reflect your domain. Other types will work too... Best Regards, Dmitry > On 2 Jun 2019, at 14.34, bengt wrote: > > Greetings, > > It is true that it is a matter of taste but remember that some Erlang containers can not be used to pattern match. Using one of those handle_call/3 would only be one function clause. And while maps can be pattern matched the compiler will not help you with spelling mistakes, as it does with records. > > > Best Wishes, > bengt > >> On 2 Jun 2019, at 13:25, Stefan Hellkvist > wrote: >> >> >>> ... >>> handle_call({run, Args}, _From, {Limit, Spv, Refs, Queue}) when N > 0 -> >>> {ok, Pid} = supervisor:start_child(Spv, Args), >>> Ref = erlang:monitor(process, Pid), >>> NewRefs = gb_sets:add(Ref, Refs), >>> NewState = {Limit-1, Spv, NewRefs, Queue}, >>> {reply, {ok, Pid}, NewState}; >>> >>> We also can use the above alternative code right? >> >> Yes, you can use whatever Erlang term you prefer - record, tuple, map, a list of maps containing records of tuples... >> >> What you choose is a matter of taste and depends on your requirements. A record for instance has the advantage over a tuple, that you access elements by name and the order of elements therefore becomes irrelevant. This could give advantages if you one day decide to add another field to your state. With the tuple approach this would likely break every access to the tuple everywhere in the code, but with a record where you access and match by names most of your code might still work...plus you never need to remember what meaning the third or fourth element in your tuple has l, because it has a descriptive name. >> >> Stefan >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From okaprinarjaya@REDACTED Sun Jun 2 15:24:36 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Sun, 2 Jun 2019 20:24:36 +0700 Subject: [erlang-questions] Why use -record() ? In-Reply-To: References: <12F6D64F-82E1-4733-94C3-C064D07B823D@gmail.com> Message-ID: Hi, Thank you for all of you guys. It's very very clear now. Wow.. very enlightening! thank you. The key is: Using tuple will ruin your life when you need to add a new element in your state. Because tuple do really really need to match the order of elements. Because yes i absolutely lazy to remember the order of tuple's elements. Thank you for the additional knowledge about type theory algebra data types, it's a new knowledge for me. Pada tanggal Min, 2 Jun 2019 pukul 18.57 Dmitry Kolesnikov < dmkolesnikov@REDACTED> menulis: > Hello, > > There was quite good reasoning already about records. I just want to make > a quotation from my resent post about records. I hope it helps you to catch > the reason why? > > Type theory and a functional programming operates with [algebraic data > types](https://en.wikipedia.org/wiki/Algebraic_data_type). They are known > as a composition of other types. The theory defines two classes of > compositions: product types (tuples, records) and co-product types (sum, > enumeration or variant types). Product types are strongly expressed by > [records](http://erlang.org/doc/reference_manual/records.html) in Erlang. > > ```erlang > -type fullname() :: binary(). > -type address() :: binary(). > -type city() :: binary(). > > -record(person, { > name :: fullname(), > address :: address(), > city :: city() > }). > ``` > > The beauty of Erlang records (product type) is that they definitions are > only available at compile time. The compiler has complete knowledge of > defined "algebra" and catches misuse errors. The usage of records in your > code benefits to write correct, maintainable code and support refactoring. > Use them to define your domain models! > > ```erlang > #person{birthday = "18810509"}. > > %% Compiling src/person.erl failed > %% src/person.erl:18: field birthday undefined in record person > ``` > > There are few other benefits of records over other data types: type > testing and pattern matching. These subject has been widely covered at > [official documentation]( > http://erlang.org/doc/programming_examples/records.html): > > ```erlang > %% > %% type testing > myfun(X) when is_record(X, person) -> ... > > %% > %% pattern matching > myfun(#person{name = Name}) -> ... > ``` > > As a summary, I would strongly advertise the usage of records to reflect > your domain. Other types will work too... > > Best Regards, > Dmitry > > On 2 Jun 2019, at 14.34, bengt wrote: > > Greetings, > > It is true that it is a matter of taste but remember that some Erlang > containers can not be used to pattern match. Using one of those > handle_call/3 would only be one function clause. And while maps can be > pattern matched the compiler will not help you with spelling mistakes, as > it does with records. > > > Best Wishes, > bengt > > On 2 Jun 2019, at 13:25, Stefan Hellkvist wrote: > > > ... > handle_call({run, Args}, _From, {Limit, Spv, Refs, Queue}) when N > 0 -> > {ok, Pid} = supervisor:start_child(Spv, Args), > Ref = erlang:monitor(process, Pid), > NewRefs = gb_sets:add(Ref, Refs), > NewState = {Limit-1, Spv, NewRefs, Queue}, > {reply, {ok, Pid}, NewState}; > > We also can use the above alternative code right? > > > Yes, you can use whatever Erlang term you prefer - record, tuple, map, a > list of maps containing records of tuples... > > What you choose is a matter of taste and depends on your requirements. A > record for instance has the advantage over a tuple, that you access > elements by name and the order of elements therefore becomes irrelevant. > This could give advantages if you one day decide to add another field to > your state. With the tuple approach this would likely break every access to > the tuple everywhere in the code, but with a record where you access and > match by names most of your code might still work...plus you never need to > remember what meaning the third or fourth element in your tuple has l, > because it has a descriptive name. > > Stefan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From okaprinarjaya@REDACTED Sun Jun 2 20:00:56 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Mon, 3 Jun 2019 01:00:56 +0700 Subject: [erlang-questions] X gen_server also died when monitored Y gen_server died. why? Message-ID: Hi, I create two gen_server. Let's say X gen_server, and Y gen_server. X gen_server do monitoring using erlang:monitor/2 to Y gen_server. When i do Y:stop(). X gen_server also died. I want to *X gen_server still alive* even monitored Y gen_server is died. This is X gen_server https://pastebin.com/G2xDNxMf This is Y gen_server https://pastebin.com/hWUfUPmn This is the result when i run my code 1> oka_oprek_agensrv:start_link(). {ok,<0.86.0>} 2> oka_oprek_agensrv:start_bgensrv(). {ok,start_bgensrv, {<0.84.0>,#Ref<0.1555235413.310640643.180158>}} 3> oka_oprek_agensrv:hello(). {ok,hello,{<0.84.0>,#Ref<0.1555235413.310640643.180166>}} 4> oka_oprek_bgensrv:hello(). {ok,hello_from_b, {<0.84.0>,#Ref<0.1555235413.310640643.180173>}} 5> oka_oprek_bgensrv:stop(). Monitored process died. Ref=#Ref<0.1555235413.310640643.180161>, Pid=<0.88.0> ** exception exit: {normal,{gen_server,call,[bgensrv,stop]}} in function gen_server:call/2 (gen_server.erl, line 215) 6> oka_oprek_agensrv:hello(). ** exception exit: {noproc,{gen_server,call,[agensrv,hello]}} in function gen_server:call/2 (gen_server.erl, line 215) 7> Please enlightenment Thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: From tty.erlang@REDACTED Sun Jun 2 20:33:32 2019 From: tty.erlang@REDACTED (T Ty) Date: Sun, 2 Jun 2019 19:33:32 +0100 Subject: [erlang-questions] Why use -record() ? In-Reply-To: References: <12F6D64F-82E1-4733-94C3-C064D07B823D@gmail.com> Message-ID: I recommend using record for a very different reason and not in the form you showed in your example. My recommendation is to prefix the module name to the record. e.g ``` -module(ppool_serv). -record(ppool_serv_state, ...). ``` My reasoning is during an error report having just a list or tuple or map would result in an unintelligible data structure displayed. With a record, as above, you can determine at a quick glance the record in question. It is also easier to determine the attributes in the record in this manner. Cheers On Sun, Jun 2, 2019 at 2:24 PM I Gusti Ngurah Oka Prinarjaya < okaprinarjaya@REDACTED> wrote: > Hi, > > Thank you for all of you guys. It's very very clear now. Wow.. very > enlightening! thank you. > > The key is: > Using tuple will ruin your life when you need to add a new element in your > state. Because tuple do really really need to match the order of elements. > Because yes i absolutely lazy to remember the order of tuple's elements. > > Thank you for the additional knowledge about type theory algebra data > types, it's a new knowledge for me. > > > > > Pada tanggal Min, 2 Jun 2019 pukul 18.57 Dmitry Kolesnikov < > dmkolesnikov@REDACTED> menulis: > >> Hello, >> >> There was quite good reasoning already about records. I just want to make >> a quotation from my resent post about records. I hope it helps you to catch >> the reason why? >> >> Type theory and a functional programming operates with [algebraic data >> types](https://en.wikipedia.org/wiki/Algebraic_data_type). They are >> known as a composition of other types. The theory defines two classes of >> compositions: product types (tuples, records) and co-product types (sum, >> enumeration or variant types). Product types are strongly expressed by >> [records](http://erlang.org/doc/reference_manual/records.html) in >> Erlang. >> >> ```erlang >> -type fullname() :: binary(). >> -type address() :: binary(). >> -type city() :: binary(). >> >> -record(person, { >> name :: fullname(), >> address :: address(), >> city :: city() >> }). >> ``` >> >> The beauty of Erlang records (product type) is that they definitions are >> only available at compile time. The compiler has complete knowledge of >> defined "algebra" and catches misuse errors. The usage of records in your >> code benefits to write correct, maintainable code and support refactoring. >> Use them to define your domain models! >> >> ```erlang >> #person{birthday = "18810509"}. >> >> %% Compiling src/person.erl failed >> %% src/person.erl:18: field birthday undefined in record person >> ``` >> >> There are few other benefits of records over other data types: type >> testing and pattern matching. These subject has been widely covered at >> [official documentation]( >> http://erlang.org/doc/programming_examples/records.html): >> >> ```erlang >> %% >> %% type testing >> myfun(X) when is_record(X, person) -> ... >> >> %% >> %% pattern matching >> myfun(#person{name = Name}) -> ... >> ``` >> >> As a summary, I would strongly advertise the usage of records to reflect >> your domain. Other types will work too... >> >> Best Regards, >> Dmitry >> >> On 2 Jun 2019, at 14.34, bengt wrote: >> >> Greetings, >> >> It is true that it is a matter of taste but remember that some Erlang >> containers can not be used to pattern match. Using one of those >> handle_call/3 would only be one function clause. And while maps can be >> pattern matched the compiler will not help you with spelling mistakes, as >> it does with records. >> >> >> Best Wishes, >> bengt >> >> On 2 Jun 2019, at 13:25, Stefan Hellkvist wrote: >> >> >> ... >> handle_call({run, Args}, _From, {Limit, Spv, Refs, Queue}) when N > 0 -> >> {ok, Pid} = supervisor:start_child(Spv, Args), >> Ref = erlang:monitor(process, Pid), >> NewRefs = gb_sets:add(Ref, Refs), >> NewState = {Limit-1, Spv, NewRefs, Queue}, >> {reply, {ok, Pid}, NewState}; >> >> We also can use the above alternative code right? >> >> >> Yes, you can use whatever Erlang term you prefer - record, tuple, map, a >> list of maps containing records of tuples... >> >> What you choose is a matter of taste and depends on your requirements. A >> record for instance has the advantage over a tuple, that you access >> elements by name and the order of elements therefore becomes irrelevant. >> This could give advantages if you one day decide to add another field to >> your state. With the tuple approach this would likely break every access to >> the tuple everywhere in the code, but with a record where you access and >> match by names most of your code might still work...plus you never need to >> remember what meaning the third or fourth element in your tuple has l, >> because it has a descriptive name. >> >> Stefan >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alin.popa@REDACTED Sun Jun 2 21:52:12 2019 From: alin.popa@REDACTED (Alin Popa) Date: Sun, 2 Jun 2019 20:52:12 +0100 Subject: [erlang-questions] X gen_server also died when monitored Y gen_server died. why? In-Reply-To: References: Message-ID: Looks like you've linked the Y process to the X process, therefore when Y exists it'll take down X as well. What you could do is to make process X to trap exit: ``` process_flat(trap_exit, true) ``` Usually you could add that within the `init` function of process X. When you do that, if any linked process exists, it'll send a message to the process X instead of taking it down. Hope that helps, Alin On Sun, Jun 2, 2019 at 7:01 PM I Gusti Ngurah Oka Prinarjaya < okaprinarjaya@REDACTED> wrote: > Hi, > > I create two gen_server. Let's say X gen_server, and Y gen_server. > > X gen_server do monitoring using erlang:monitor/2 to Y gen_server. > When i do Y:stop(). X gen_server also died. > > I want to *X gen_server still alive* even monitored Y gen_server is died. > > This is X gen_server https://pastebin.com/G2xDNxMf > This is Y gen_server https://pastebin.com/hWUfUPmn > > This is the result when i run my code > > 1> oka_oprek_agensrv:start_link(). > > {ok,<0.86.0>} > > 2> oka_oprek_agensrv:start_bgensrv(). > > {ok,start_bgensrv, > > {<0.84.0>,#Ref<0.1555235413.310640643.180158>}} > > 3> oka_oprek_agensrv:hello(). > > {ok,hello,{<0.84.0>,#Ref<0.1555235413.310640643.180166>}} > > 4> oka_oprek_bgensrv:hello(). > > {ok,hello_from_b, > > {<0.84.0>,#Ref<0.1555235413.310640643.180173>}} > > 5> oka_oprek_bgensrv:stop(). > > Monitored process died. Ref=#Ref<0.1555235413.310640643.180161>, > Pid=<0.88.0> > > ** exception exit: {normal,{gen_server,call,[bgensrv,stop]}} > > in function gen_server:call/2 (gen_server.erl, line 215) > > 6> oka_oprek_agensrv:hello(). > > ** exception exit: {noproc,{gen_server,call,[agensrv,hello]}} > > in function gen_server:call/2 (gen_server.erl, line 215) > > 7> > > > Please enlightenment > > > > Thank you > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joaohf@REDACTED Sun Jun 2 22:21:41 2019 From: joaohf@REDACTED (=?UTF-8?Q?Jo=C3=A3o_Henrique_Freitas?=) Date: Sun, 2 Jun 2019 22:21:41 +0200 Subject: [erlang-questions] EMQ available in Yocto Project Message-ID: Hi, I'm the author of a Yocto Project layer called meta-erlang. I would like to announce that now is possible to use Yocto Project which build a embedded Linux distro and on top of that install the EMQ (an mqtt broker) or EMQ edge. I've created the recipe and all the patches necessary to do that. My main target is to bring software made with Erlang/Elixir possible to run in Yocto/Openembedded ecosystem. Just in case, if you have a embedded project and are planing to run emq edge, for example. This could be feasible for you. Check it: - https://layers.openembedded.org/layerindex/branch/master/recipes/?q=emq - http://joaohf.github.io/meta-erlang/#/guides-emqx - https://github.com/joaohf/meta-erlang - https://www.yoctoproject.org/ Best regards, -- Jo?o Henrique Ferreira de Freitas - joaohf_at_gmail.com Campinas-SP-Brasil -------------- next part -------------- An HTML attachment was scrubbed... URL: From okaprinarjaya@REDACTED Mon Jun 3 02:17:57 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Mon, 3 Jun 2019 07:17:57 +0700 Subject: [erlang-questions] X gen_server also died when monitored Y gen_server died. why? In-Reply-To: References: Message-ID: Hi, I've use process_flag(trap_exit, true) in X:init() but still failed. X still died. mmm.. if i look closely to the error message, the erlang shell is also died Thank you Pada tanggal Sen, 3 Jun 2019 pukul 02.53 Alin Popa menulis: > Looks like you've linked the Y process to the X process, therefore when Y > exists it'll take down X as well. > What you could do is to make process X to trap exit: > > ``` > process_flat(trap_exit, true) > ``` > > Usually you could add that within the `init` function of process X. > When you do that, if any linked process exists, it'll send a message to > the process X instead of taking it down. > > Hope that helps, > Alin > > On Sun, Jun 2, 2019 at 7:01 PM I Gusti Ngurah Oka Prinarjaya < > okaprinarjaya@REDACTED> wrote: > >> Hi, >> >> I create two gen_server. Let's say X gen_server, and Y gen_server. >> >> X gen_server do monitoring using erlang:monitor/2 to Y gen_server. >> When i do Y:stop(). X gen_server also died. >> >> I want to *X gen_server still alive* even monitored Y gen_server is >> died. >> >> This is X gen_server https://pastebin.com/G2xDNxMf >> This is Y gen_server https://pastebin.com/hWUfUPmn >> >> This is the result when i run my code >> >> 1> oka_oprek_agensrv:start_link(). >> >> {ok,<0.86.0>} >> >> 2> oka_oprek_agensrv:start_bgensrv(). >> >> {ok,start_bgensrv, >> >> {<0.84.0>,#Ref<0.1555235413.310640643.180158>}} >> >> 3> oka_oprek_agensrv:hello(). >> >> {ok,hello,{<0.84.0>,#Ref<0.1555235413.310640643.180166>}} >> >> 4> oka_oprek_bgensrv:hello(). >> >> {ok,hello_from_b, >> >> {<0.84.0>,#Ref<0.1555235413.310640643.180173>}} >> >> 5> oka_oprek_bgensrv:stop(). >> >> Monitored process died. Ref=#Ref<0.1555235413.310640643.180161>, >> Pid=<0.88.0> >> >> ** exception exit: {normal,{gen_server,call,[bgensrv,stop]}} >> >> in function gen_server:call/2 (gen_server.erl, line 215) >> >> 6> oka_oprek_agensrv:hello(). >> >> ** exception exit: {noproc,{gen_server,call,[agensrv,hello]}} >> >> in function gen_server:call/2 (gen_server.erl, line 215) >> >> 7> >> >> >> Please enlightenment >> >> >> >> Thank you >> >> >> >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alin.popa@REDACTED Mon Jun 3 10:51:34 2019 From: alin.popa@REDACTED (Alin Popa) Date: Mon, 3 Jun 2019 09:51:34 +0100 Subject: [erlang-questions] X gen_server also died when monitored Y gen_server died. why? In-Reply-To: References: Message-ID: Looks like since you're doing the stop through the gen_server:call, you'll need to return {stop, Reason, Reply, State}, and in your code I can see you're doing {stop, Reason, State}. Now, why is this happening, I don't know. Alin On Mon, Jun 3, 2019 at 1:18 AM I Gusti Ngurah Oka Prinarjaya < okaprinarjaya@REDACTED> wrote: > Hi, > > I've use process_flag(trap_exit, true) in X:init() but still failed. X > still died. mmm.. if i look closely to the error message, the erlang shell > is also died > > > Thank you > > > > Pada tanggal Sen, 3 Jun 2019 pukul 02.53 Alin Popa > menulis: > >> Looks like you've linked the Y process to the X process, therefore when Y >> exists it'll take down X as well. >> What you could do is to make process X to trap exit: >> >> ``` >> process_flat(trap_exit, true) >> ``` >> >> Usually you could add that within the `init` function of process X. >> When you do that, if any linked process exists, it'll send a message to >> the process X instead of taking it down. >> >> Hope that helps, >> Alin >> >> On Sun, Jun 2, 2019 at 7:01 PM I Gusti Ngurah Oka Prinarjaya < >> okaprinarjaya@REDACTED> wrote: >> >>> Hi, >>> >>> I create two gen_server. Let's say X gen_server, and Y gen_server. >>> >>> X gen_server do monitoring using erlang:monitor/2 to Y gen_server. >>> When i do Y:stop(). X gen_server also died. >>> >>> I want to *X gen_server still alive* even monitored Y gen_server is >>> died. >>> >>> This is X gen_server https://pastebin.com/G2xDNxMf >>> This is Y gen_server https://pastebin.com/hWUfUPmn >>> >>> This is the result when i run my code >>> >>> 1> oka_oprek_agensrv:start_link(). >>> >>> {ok,<0.86.0>} >>> >>> 2> oka_oprek_agensrv:start_bgensrv(). >>> >>> {ok,start_bgensrv, >>> >>> {<0.84.0>,#Ref<0.1555235413.310640643.180158>}} >>> >>> 3> oka_oprek_agensrv:hello(). >>> >>> {ok,hello,{<0.84.0>,#Ref<0.1555235413.310640643.180166>}} >>> >>> 4> oka_oprek_bgensrv:hello(). >>> >>> {ok,hello_from_b, >>> >>> {<0.84.0>,#Ref<0.1555235413.310640643.180173>}} >>> >>> 5> oka_oprek_bgensrv:stop(). >>> >>> Monitored process died. Ref=#Ref<0.1555235413.310640643.180161>, >>> Pid=<0.88.0> >>> >>> ** exception exit: {normal,{gen_server,call,[bgensrv,stop]}} >>> >>> in function gen_server:call/2 (gen_server.erl, line 215) >>> >>> 6> oka_oprek_agensrv:hello(). >>> >>> ** exception exit: {noproc,{gen_server,call,[agensrv,hello]}} >>> >>> in function gen_server:call/2 (gen_server.erl, line 215) >>> >>> 7> >>> >>> >>> Please enlightenment >>> >>> >>> >>> Thank you >>> >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From dieter@REDACTED Mon Jun 3 11:42:32 2019 From: dieter@REDACTED (dieter@REDACTED) Date: Mon, 03 Jun 2019 09:42:32 +0000 Subject: [erlang-questions] Why use -record() ? In-Reply-To: References: <12F6D64F-82E1-4733-94C3-C064D07B823D@gmail.com> Message-ID: <612d5bb6115a05547aff084462743ba5@afterlogic.edis.at> That's a good one, thank you for sharing! dieter Am So., Jun. 2, 2019 20:34 schrieb T Ty : I recommend using record for a very different reason and not in the form you showed in your example. My recommendation is to prefix the module name to the record. e.g ``` -module(ppool_serv). -record(ppool_serv_state, ...). ``` My reasoning is during an error report having just a list or tuple or map would result in an unintelligible data structure displayed. With a record, as above, you can determine at a quick glance the record in question. It is also easier to determine the attributes in the record in this manner. Cheers On Sun, Jun 2, 2019 at 2:24 PM I Gusti Ngurah Oka Prinarjaya wrote: Hi, Thank you for all of you guys. It's very very clear now. Wow.. very enlightening! thank you. The key is: Using tuple will ruin your life when you need to add a new element in your state. Because tuple do really really need to match the order of elements. Because yes i absolutely lazy to remember the order of tuple's elements. Thank you for the additional knowledge about type theory algebra data types, it's a new knowledge for me. Pada tanggal Min, 2 Jun 2019 pukul 18.57 Dmitry Kolesnikov menulis: Hello, There was quite good reasoning already about records. I just want to make a quotation from my resent post about records. I hope it helps you to catch the reason why? Type theory and a functional programming operates with [algebraic data types](https://en.wikipedia.org/wiki/Algebraic_data_type (https://en.wikipedia.org/wiki/Algebraic_data_type)). They are known as a composition of other types. The theory defines two classes of compositions: product types (tuples, records) and co-product types (sum, enumeration or variant types). Product types are strongly expressed by [records](http://erlang.org/doc/reference_manual/records.html (http://erlang.org/doc/reference_manual/records.html)) in Erlang. ```erlang -type fullname() :: binary(). -type address() :: binary(). -type city() :: binary(). -record(person, { name :: fullname(), address :: address(), city :: city() }). ``` The beauty of Erlang records (product type) is that they definitions are only available at compile time. The compiler has complete knowledge of defined "algebra" and catches misuse errors. The usage of records in your code benefits to write correct, maintainable code and support refactoring. Use them to define your domain models! ```erlang #person{birthday = "18810509"}. %% Compiling src/person.erl failed %% src/person.erl:18: field birthday undefined in record person ``` There are few other benefits of records over other data types: type testing and pattern matching. These subject has been widely covered at [official documentation](http://erlang.org/doc/programming_examples/records.html (http://erlang.org/doc/programming_examples/records.html)): ```erlang %% %% type testing myfun(X) when is_record(X, person) -> ... %% %% pattern matching myfun(#person{name = Name}) -> ... ``` As a summary, I would strongly advertise the usage of records to reflect your domain. Other types will work too... Best Regards, Dmitry On 2 Jun 2019, at 14.34, bengt wrote: Greetings, It is true that it is a matter of taste but remember that some Erlang containers can not be used to pattern match. Using one of those handle_call/3 would only be one function clause. And while maps can be pattern matched the compiler will not help you with spelling mistakes, as it does with records. Best Wishes, bengt On 2 Jun 2019, at 13:25, Stefan Hellkvist wrote: ... handle_call({run, Args}, _From, {Limit, Spv, Refs, Queue}) when N > 0 -> {ok, Pid} = supervisor:start_child(Spv, Args), Ref = erlang:monitor(process, Pid), NewRefs = gb_sets:add(Ref, Refs), NewState = {Limit-1, Spv, NewRefs, Queue}, {reply, {ok, Pid}, NewState}; We also can use the above alternative code right? Yes, you can use whatever Erlang term you prefer - record, tuple, map, a list of maps containing records of tuples... What you choose is a matter of taste and depends on your requirements. A record for instance has the advantage over a tuple, that you access elements by name and the order of elements therefore becomes irrelevant. This could give advantages if you one day decide to add another field to your state. With the tuple approach this would likely break every access to the tuple everywhere in the code, but with a record where you access and match by names most of your code might still work...plus you never need to remember what meaning the third or fourth element in your tuple has l, because it has a descriptive name. Stefan _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) http://erlang.org/mailman/listinfo/erlang-questions (http://erlang.org/mailman/listinfo/erlang-questions) _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) http://erlang.org/mailman/listinfo/erlang-questions (http://erlang.org/mailman/listinfo/erlang-questions) _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) http://erlang.org/mailman/listinfo/erlang-questions (http://erlang.org/mailman/listinfo/erlang-questions) -------------- next part -------------- An HTML attachment was scrubbed... URL: From okaprinarjaya@REDACTED Mon Jun 3 14:06:06 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Mon, 3 Jun 2019 19:06:06 +0700 Subject: [erlang-questions] X gen_server also died when monitored Y gen_server died. why? In-Reply-To: References: Message-ID: Hi, It's ok.. i will try another way. i think it must be under a supervisor. Thank you Pada tanggal Sen, 3 Jun 2019 15.52, Alin Popa menulis: > Looks like since you're doing the stop through the gen_server:call, you'll > need to return {stop, Reason, Reply, State}, and in your code I can see > you're doing {stop, Reason, State}. > Now, why is this happening, I don't know. > > Alin > > On Mon, Jun 3, 2019 at 1:18 AM I Gusti Ngurah Oka Prinarjaya < > okaprinarjaya@REDACTED> wrote: > >> Hi, >> >> I've use process_flag(trap_exit, true) in X:init() but still failed. X >> still died. mmm.. if i look closely to the error message, the erlang shell >> is also died >> >> >> Thank you >> >> >> >> Pada tanggal Sen, 3 Jun 2019 pukul 02.53 Alin Popa >> menulis: >> >>> Looks like you've linked the Y process to the X process, therefore when >>> Y exists it'll take down X as well. >>> What you could do is to make process X to trap exit: >>> >>> ``` >>> process_flat(trap_exit, true) >>> ``` >>> >>> Usually you could add that within the `init` function of process X. >>> When you do that, if any linked process exists, it'll send a message to >>> the process X instead of taking it down. >>> >>> Hope that helps, >>> Alin >>> >>> On Sun, Jun 2, 2019 at 7:01 PM I Gusti Ngurah Oka Prinarjaya < >>> okaprinarjaya@REDACTED> wrote: >>> >>>> Hi, >>>> >>>> I create two gen_server. Let's say X gen_server, and Y gen_server. >>>> >>>> X gen_server do monitoring using erlang:monitor/2 to Y gen_server. >>>> When i do Y:stop(). X gen_server also died. >>>> >>>> I want to *X gen_server still alive* even monitored Y gen_server is >>>> died. >>>> >>>> This is X gen_server https://pastebin.com/G2xDNxMf >>>> This is Y gen_server https://pastebin.com/hWUfUPmn >>>> >>>> This is the result when i run my code >>>> >>>> 1> oka_oprek_agensrv:start_link(). >>>> >>>> {ok,<0.86.0>} >>>> >>>> 2> oka_oprek_agensrv:start_bgensrv(). >>>> >>>> {ok,start_bgensrv, >>>> >>>> {<0.84.0>,#Ref<0.1555235413.310640643.180158>}} >>>> >>>> 3> oka_oprek_agensrv:hello(). >>>> >>>> {ok,hello,{<0.84.0>,#Ref<0.1555235413.310640643.180166>}} >>>> >>>> 4> oka_oprek_bgensrv:hello(). >>>> >>>> {ok,hello_from_b, >>>> >>>> {<0.84.0>,#Ref<0.1555235413.310640643.180173>}} >>>> >>>> 5> oka_oprek_bgensrv:stop(). >>>> >>>> Monitored process died. Ref=#Ref<0.1555235413.310640643.180161>, >>>> Pid=<0.88.0> >>>> >>>> ** exception exit: {normal,{gen_server,call,[bgensrv,stop]}} >>>> >>>> in function gen_server:call/2 (gen_server.erl, line 215) >>>> >>>> 6> oka_oprek_agensrv:hello(). >>>> >>>> ** exception exit: {noproc,{gen_server,call,[agensrv,hello]}} >>>> >>>> in function gen_server:call/2 (gen_server.erl, line 215) >>>> >>>> 7> >>>> >>>> >>>> Please enlightenment >>>> >>>> >>>> >>>> Thank you >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From otp@REDACTED Mon Jun 3 14:55:20 2019 From: otp@REDACTED (Erlang/OTP) Date: Mon, 3 Jun 2019 14:55:20 +0200 Subject: [erlang-questions] Patch Package OTP 22.0.2 Released Message-ID: <20190603125520.GA17362@erix.ericsson.se> Patch Package: OTP 22.0.2 Git Tag: OTP-22.0.2 Date: 2019-06-03 Trouble Report Id: OTP-15828, OTP-15829, OTP-15832, OTP-15834, OTP-15838, OTP-15839, OTP-15841, OTP-15845, OTP-15846, OTP-15847, OTP-15848 Seq num: ERIERL-367, ERL-948, ERL-950, ERL-954, ERL-955, ERL-956, ERL-957 System: OTP Release: 22 Application: compiler-7.4.1, crypto-4.5.1, erts-10.4.1, stdlib-3.9.1 Predecessor: OTP 22.0.1 Check out the git tag OTP-22.0.2, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- compiler-7.4.1 -------------------------------------------------- --------------------------------------------------------------------- The compiler-7.4.1 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-15828 Application(s): compiler Related Id(s): ERL-948 The type optimization pass of the compiler could hang or loop for a long time when analyzing a setelement/3 call with a varible position. OTP-15832 Application(s): compiler Related Id(s): ERL-950 Certain complex receive statements would result in an internal compiler failure. OTP-15838 Application(s): compiler Fixed an unsafe type optimization. OTP-15839 Application(s): compiler Related Id(s): ERL-954 Fixed a crash when optimizing compiler-generated exceptions (like badmatch) whose offending term was a constructed binary. OTP-15841 Application(s): compiler Fixed a bad optimization related to the ++/2 operator, where the compiler assumed that it always produced a list ([] ++ RHS returns RHS verbatim, even if it's not a list). OTP-15845 Application(s): compiler An is_binary/1 test followed by is_bitstring/1 (or vice versa) could fail because of an usafe optimization. OTP-15846 Application(s): compiler Related Id(s): ERL-955 A Core Erlang module where the last clause in a case matched a map would fail to load. OTP-15848 Application(s): compiler Related Id(s): ERL-956 Fixed a bug that could cause the compiler to crash when compiling complex nested case expressions. Full runtime dependencies of compiler-7.4.1: crypto-3.6, erts-9.0, hipe-3.12, kernel-4.0, stdlib-2.5 --------------------------------------------------------------------- --- crypto-4.5.1 ---------------------------------------------------- --------------------------------------------------------------------- The crypto-4.5.1 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-15829 Application(s): crypto The cipher aes-ctr was disabled by misstake in crypto:supports for cryptolibs before 1.0.1. It worked however in the encrypt and decrypt functions. Full runtime dependencies of crypto-4.5.1: erts-9.0, kernel-5.3, stdlib-3.4 --------------------------------------------------------------------- --- erts-10.4.1 ----------------------------------------------------- --------------------------------------------------------------------- The erts-10.4.1 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-15834 Application(s): erts Related Id(s): ERIERL-367 In nested use of try/catch, rethrowing an exception using erlang:raise/3 with a different class would not always be able to change the class of the exception. Full runtime dependencies of erts-10.4.1: kernel-6.1, sasl-3.3, stdlib-3.5 --------------------------------------------------------------------- --- stdlib-3.9.1 ---------------------------------------------------- --------------------------------------------------------------------- The stdlib-3.9.1 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-15847 Application(s): stdlib Related Id(s): ERL-957 Fix a bug that could cause a failure when formatting binaries using the control sequences p or P and limiting the output with the option chars_limit. Full runtime dependencies of stdlib-3.9.1: compiler-5.0, crypto-3.3, erts-10.4, kernel-6.0, sasl-3.0 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From roger@REDACTED Mon Jun 3 16:06:29 2019 From: roger@REDACTED (Roger Lipscombe) Date: Mon, 3 Jun 2019 15:06:29 +0100 Subject: [erlang-questions] Cover compilation breaks running node Message-ID: I'm attempting to use ct_run -cover (per http://erlang.org/doc/apps/common_test/cover_chapter.html) to get coverage information from an Erlang node. I've got a coverspec file that looks like this: {nodes, ['the_server@REDACTED']}. {incl_dirs_r, ["path/to/_rel/the_server/lib"]}. {export, "cover.coverdata"}. ...and I'm running it like this (abridged): ct_run -cover cover.spec -erl_args -pa path/to/_rel/the_server/lib/*/ebin After a long wait while it cover compiles 900 modules, it finally runs my CT test, but it fails because the remote node has been screwed up. I get a bunch of function_clause and badrecord errors and everything's entirely broken. The remote node finally gets stuck in a loop continually complaining about {badarg,[{ets,update_count,[cover_internal_data_table etc... Does anyone have a working example of getting coverage from a remote Erlang node while running ct_run? Potentially related: can I cover-compile the modules ahead of time, when I build the release (I'm using relx)? From erlangsiri@REDACTED Mon Jun 3 16:46:28 2019 From: erlangsiri@REDACTED (Siri Hansen) Date: Mon, 3 Jun 2019 16:46:28 +0200 Subject: [erlang-questions] Cover compilation breaks running node In-Reply-To: References: Message-ID: First, it is not possible to cover compile to file, so I guess compiling ahead of time is not straight forward. How do you start/stop your remote node? I think that specifying the node in the cover spec means that ct tries to start cover on the node when the test starts. I don't think it is clever enough to do the stuff if the node is started later on. Unless you start it with ct_slave:start, which automatically starts cover on any node, no matter if it is specified in the cover spec or not. When cover is started on a remote node, it loads the same cover compiled beam files as are running on the local node, so if your nodes are running different versions of your release, then you have a potential problem (e.g. function_clause and badrecords). >From the badarg with the ets table, it seems that you have cover compiled code running on the node without the cover server being running there (guessing the badarg means the table does not exist, that is). Is there any indication of the cover server crashing, or of unloading of cover compiled code failing? Does this happen early in the test run, or at the end of it? A problem I often see is that processes crash when cover is stopped. This has to do with code loading and occurs if a process is still running the old (non cover-compiled) code when cover is stopped. To avoid this, the cover_stop flag is added - see Code Coverage Analysis section in the user's guide. I don't know if your problems are in any way related to this though. So, this was a bunch of more or less related thoughts - don't know if anything may get you on the right track, but if you want to discuss further please let me know. /siri man. 3. jun. 2019 kl. 16:06 skrev Roger Lipscombe : > I'm attempting to use ct_run -cover (per > http://erlang.org/doc/apps/common_test/cover_chapter.html) to get > coverage information from an Erlang node. > > I've got a coverspec file that looks like this: > > {nodes, ['the_server@REDACTED']}. > {incl_dirs_r, ["path/to/_rel/the_server/lib"]}. > {export, "cover.coverdata"}. > > ...and I'm running it like this (abridged): > > ct_run -cover cover.spec -erl_args -pa path/to/_rel/the_server/lib/*/ebin > > After a long wait while it cover compiles 900 modules, it finally runs > my CT test, but it fails because the remote node has been screwed up. > I get a bunch of function_clause and badrecord errors and everything's > entirely broken. > > The remote node finally gets stuck in a loop continually complaining > about {badarg,[{ets,update_count,[cover_internal_data_table etc... > > Does anyone have a working example of getting coverage from a remote > Erlang node while running ct_run? > > Potentially related: can I cover-compile the modules ahead of time, > when I build the release (I'm using relx)? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Mon Jun 3 17:04:46 2019 From: roger@REDACTED (Roger Lipscombe) Date: Mon, 3 Jun 2019 16:04:46 +0100 Subject: [erlang-questions] Cover compilation breaks running node In-Reply-To: References: Message-ID: On Mon, 3 Jun 2019 at 15:46, Siri Hansen wrote: > First, it is not possible to cover compile to file, so I guess compiling ahead of time is not straight forward. Feature request for OTP-23? ;-) > How do you start/stop your remote node? In this particular case, the node is already running. > When cover is started on a remote node, it loads the same cover compiled beam files as are running on the local node, so if your nodes are running different versions of your release, then you have a potential problem (e.g. function_clause and badrecords). The node running ct_run is a separate node and isn't running the release, per se, but I did use -pa to point ct_run at the release's files. It's cover-compiling the source/beam files from the release directory, which was only just built, so I don't *think* this is the case... Moreover, one of the affected modules is in the RabbitMQ AMQP client library, which hasn't changed. Obviously, for efficiency, I should probably exclude third party dependencies from cover-compilation, but I'm still poking around and not at that stage yet. > From the badarg with the ets table, it seems that you have cover compiled code running on the node without the cover server being running there (guessing the badarg means the table does not exist, that is). Is there any indication of the cover server crashing, or of unloading of cover compiled code failing? I've closed the terminal window; I'll check again later. > Does this happen early in the test run, or at the end of it? A problem I often see is that processes crash when cover is stopped. This has to do with code loading and occurs if a process is still running the old (non cover-compiled) code when cover is stopped. The looping thing happens at the end of the test, but the test already failed because of badrecord, etc., by this point. > To avoid this, the cover_stop flag is added - see Code Coverage Analysis section in the user's guide. I'll try this later and report back. From icfp.publicity@REDACTED Mon Jun 3 18:03:45 2019 From: icfp.publicity@REDACTED (Sam Tobin-Hochstadt) Date: Mon, 03 Jun 2019 12:03:45 -0400 Subject: [erlang-questions] Second Call for Submissions: ICFP Student Research Competition Message-ID: <5cf544e123555_78b42ab427c125b89064@homer.mail> ICFP 2019 Student Research Competition Call for Submissions ICFP invites students to participate in the Student Research Competition in order to present their research and get feedback from prominent members of the programming language research community. Please submit your extended abstracts through the submission website. ### Important dates Submissions due: 14 Jun 2019 (Friday) https://icfp19src.hotcrp.com Notification: 28 Jun 2019 (Friday) Conference: 18 August (Sunday) - 23 August (Friday) Each submission (referred to as "abstract" below) should include the student author?s name and e-mail address; institutional affiliation; research advisor?s name; ACM student member number; category (undergraduate or graduate); research title; and an extended abstract addressing the following: * Problem and Motivation: Clearly state the problem being addressed and explain the reasons for seeking a solution to this problem. * Background and Related Work: Describe the specialized (but pertinent) background necessary to appreciate the work in the context of ICFP areas of interest. Include references to the literature where appropriate, and briefly explain where your work departs from that done by others. * Approach and Uniqueness: Describe your approach in addressing the problem and clearly state how your approach is novel. * Results and Contributions: Clearly show how the results of your work contribute to programming language design and implementation in particular and to computer science in general; explain the significance of those results. * Submissions must be original research that is not already published at ICFP or another conference or journal. One of the goals of the SRC is to give students feedback on ongoing, unpublished work. Furthermore, the abstract must be authored solely by the student. If the work is collaborative with others and*or part of a larger group project, the abstract should make clear what the student?s role was and should focus on that portion of the work. * Formatting: Submissions must be in PDF format, printable in black and white on US Letter sized paper, and interpretable by common PDF tools. All submissions must adhere to the "ACM Small" template that is available (in both LaTeX and Word formats) from https://www.acm.org/publications/authors/submissions. For authors using LaTeX, a lighter-weight package, including only the essential files, is available from http://sigplan.org/Resources/Author/#acmart-format. The submission must not exceed 3 pages in PDF format. Reference lists do not count towards the 3-page limit. Further information is available at the ICFP SRC website: https://icfp19.sigplan.org/track/icfp-2019-Student-Research-Competition ICFP Student Research Competition Chair: William J. Bowman (University of British Columbia) From okaprinarjaya@REDACTED Mon Jun 3 19:07:53 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Tue, 4 Jun 2019 00:07:53 +0700 Subject: [erlang-questions] X gen_server also died when monitored Y gen_server died. why? In-Reply-To: References: Message-ID: Hi, Solved!. Yes i run a X supervisor process, run a X gen_server process as adjoining process / sibling process with X supervisor process. Then under X supervisor i add / run P1, P2, P3 gen_server process, and each process under X supervisor is monitored by X gen_server process. and it works! Thank you Pada tanggal Sen, 3 Jun 2019 pukul 19.06 I Gusti Ngurah Oka Prinarjaya < okaprinarjaya@REDACTED> menulis: > Hi, > > It's ok.. i will try another way. i think it must be under a supervisor. > > Thank you > > Pada tanggal Sen, 3 Jun 2019 15.52, Alin Popa > menulis: > >> Looks like since you're doing the stop through the gen_server:call, >> you'll need to return {stop, Reason, Reply, State}, and in your code I can >> see you're doing {stop, Reason, State}. >> Now, why is this happening, I don't know. >> >> Alin >> >> On Mon, Jun 3, 2019 at 1:18 AM I Gusti Ngurah Oka Prinarjaya < >> okaprinarjaya@REDACTED> wrote: >> >>> Hi, >>> >>> I've use process_flag(trap_exit, true) in X:init() but still failed. X >>> still died. mmm.. if i look closely to the error message, the erlang shell >>> is also died >>> >>> >>> Thank you >>> >>> >>> >>> Pada tanggal Sen, 3 Jun 2019 pukul 02.53 Alin Popa >>> menulis: >>> >>>> Looks like you've linked the Y process to the X process, therefore when >>>> Y exists it'll take down X as well. >>>> What you could do is to make process X to trap exit: >>>> >>>> ``` >>>> process_flat(trap_exit, true) >>>> ``` >>>> >>>> Usually you could add that within the `init` function of process X. >>>> When you do that, if any linked process exists, it'll send a message to >>>> the process X instead of taking it down. >>>> >>>> Hope that helps, >>>> Alin >>>> >>>> On Sun, Jun 2, 2019 at 7:01 PM I Gusti Ngurah Oka Prinarjaya < >>>> okaprinarjaya@REDACTED> wrote: >>>> >>>>> Hi, >>>>> >>>>> I create two gen_server. Let's say X gen_server, and Y gen_server. >>>>> >>>>> X gen_server do monitoring using erlang:monitor/2 to Y gen_server. >>>>> When i do Y:stop(). X gen_server also died. >>>>> >>>>> I want to *X gen_server still alive* even monitored Y gen_server is >>>>> died. >>>>> >>>>> This is X gen_server https://pastebin.com/G2xDNxMf >>>>> This is Y gen_server https://pastebin.com/hWUfUPmn >>>>> >>>>> This is the result when i run my code >>>>> >>>>> 1> oka_oprek_agensrv:start_link(). >>>>> >>>>> {ok,<0.86.0>} >>>>> >>>>> 2> oka_oprek_agensrv:start_bgensrv(). >>>>> >>>>> {ok,start_bgensrv, >>>>> >>>>> {<0.84.0>,#Ref<0.1555235413.310640643.180158>}} >>>>> >>>>> 3> oka_oprek_agensrv:hello(). >>>>> >>>>> {ok,hello,{<0.84.0>,#Ref<0.1555235413.310640643.180166>}} >>>>> >>>>> 4> oka_oprek_bgensrv:hello(). >>>>> >>>>> {ok,hello_from_b, >>>>> >>>>> {<0.84.0>,#Ref<0.1555235413.310640643.180173>}} >>>>> >>>>> 5> oka_oprek_bgensrv:stop(). >>>>> >>>>> Monitored process died. Ref=#Ref<0.1555235413.310640643.180161>, >>>>> Pid=<0.88.0> >>>>> >>>>> ** exception exit: {normal,{gen_server,call,[bgensrv,stop]}} >>>>> >>>>> in function gen_server:call/2 (gen_server.erl, line 215) >>>>> >>>>> 6> oka_oprek_agensrv:hello(). >>>>> >>>>> ** exception exit: {noproc,{gen_server,call,[agensrv,hello]}} >>>>> >>>>> in function gen_server:call/2 (gen_server.erl, line 215) >>>>> >>>>> 7> >>>>> >>>>> >>>>> Please enlightenment >>>>> >>>>> >>>>> >>>>> Thank you >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From otp@REDACTED Tue Jun 4 09:54:50 2019 From: otp@REDACTED (Erlang/OTP) Date: Tue, 4 Jun 2019 09:54:50 +0200 Subject: [erlang-questions] Patch Package OTP 21.3.8.3 Released Message-ID: <20190604075450.GA35517@erix.ericsson.se> Patch Package: OTP 21.3.8.3 Git Tag: OTP-21.3.8.3 Date: 2019-06-04 Trouble Report Id: OTP-15834, OTP-15844, OTP-15847, OTP-15849, OTP-15858, OTP-15859 Seq num: ERIERL-367, ERL-700, ERL-957 System: OTP Release: 21 Application: erts-10.3.5.2, kernel-6.3.1.1, ssl-9.2.3.2, stdlib-3.8.2.1 Predecessor: OTP 21.3.8.2 Check out the git tag OTP-21.3.8.3, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- erts-10.3.5.2 --------------------------------------------------- --------------------------------------------------------------------- Note! The erts-10.3.5.2 application *cannot* be applied independently of other applications on an arbitrary OTP 21 installation. On a full OTP 21 installation, also the following runtime dependencies have to be satisfied: -- kernel-6.1 (first satisfied in OTP 21.1) -- sasl-3.3 (first satisfied in OTP 21.2) --- Fixed Bugs and Malfunctions --- OTP-15834 Application(s): erts Related Id(s): ERIERL-367 In nested use of try/catch, rethrowing an exception using erlang:raise/3 with a different class would not always be able to change the class of the exception. OTP-15849 Application(s): erts Related Id(s): ERL-700 Fixed bug in seq_trace:set_token(label,Term) which could cause VM crash if Term was heap allocated (not an atom, small integer, local pid or port). Bug exists since OTP 21.0 when terms other than small integers were first allowed as labels. OTP-15859 Application(s): erts Related Id(s): ERL-700 Fix seq_trace:print/2 not to raise badarg exception if label is not a small integer. Bug exists since OTP 21.0. Full runtime dependencies of erts-10.3.5.2: kernel-6.1, sasl-3.3, stdlib-3.5 --------------------------------------------------------------------- --- kernel-6.3.1.1 -------------------------------------------------- --------------------------------------------------------------------- Note! The kernel-6.3.1.1 application *cannot* be applied independently of other applications on an arbitrary OTP 21 installation. On a full OTP 21 installation, also the following runtime dependency has to be satisfied: -- erts-10.2.5 (first satisfied in OTP 21.2.7) --- Fixed Bugs and Malfunctions --- OTP-15858 Application(s): kernel Related Id(s): ERL-700 Fix type spec for seq_trace:set_token/2. Full runtime dependencies of kernel-6.3.1.1: erts-10.2.5, sasl-3.0, stdlib-3.5 --------------------------------------------------------------------- --- ssl-9.2.3.2 ----------------------------------------------------- --------------------------------------------------------------------- The ssl-9.2.3.2 application can be applied independently of other applications on a full OTP 21 installation. --- Fixed Bugs and Malfunctions --- OTP-15844 Application(s): ssl Returned "alert error string" is now same as logged alert string Full runtime dependencies of ssl-9.2.3.2: crypto-4.2, erts-10.0, inets-5.10.7, kernel-6.0, public_key-1.5, stdlib-3.5 --------------------------------------------------------------------- --- stdlib-3.8.2.1 -------------------------------------------------- --------------------------------------------------------------------- The stdlib-3.8.2.1 application can be applied independently of other applications on a full OTP 21 installation. --- Fixed Bugs and Malfunctions --- OTP-15847 Application(s): stdlib Related Id(s): ERL-957 Fix a bug that could cause a failure when formatting binaries using the control sequences p or P and limiting the output with the option chars_limit. Full runtime dependencies of stdlib-3.8.2.1: compiler-5.0, crypto-3.3, erts-10.0, kernel-6.0, sasl-3.0 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From fchschneider@REDACTED Wed Jun 5 12:48:06 2019 From: fchschneider@REDACTED (Frans Schneider) Date: Wed, 5 Jun 2019 12:48:06 +0200 Subject: [erlang-questions] error message formating Message-ID: <262dd139-af26-8286-f349-34cbd5ccf372@gmail.com> Hi all, With Erlang/OTP 22, while in the rebar3 (auto) shell, error messages now have become single line messages which are hard to read. How can I get them to look like the old messages? Guess this is really easy and somewhere in the docs... TIA, Frans From roger@REDACTED Wed Jun 5 14:10:55 2019 From: roger@REDACTED (Roger Lipscombe) Date: Wed, 5 Jun 2019 13:10:55 +0100 Subject: [erlang-questions] Can't connect to Erlang node in docker Message-ID: I've got an Erlang node running in docker[1]. How do I connect to it from the host? I've tried every combination of -name, -sname, etc., that I can think of, and all I'm getting is 'pang'. I can telnet to the distribution port on the container; if I use erl_epmd:names/1, passing the DNS name of the docker container, it returns the expected Erlang nodes. But I can't get Erlang distribution to connect from the host to the node in the docker container. Interestingly, if I connect from the node inside docker to a node on the host, it all starts working. But that's not useful for what I'm trying to do. [1] actually several Erlang nodes in several docker containers. From juanjo@REDACTED Wed Jun 5 14:27:41 2019 From: juanjo@REDACTED (Juan Jose Comellas) Date: Wed, 5 Jun 2019 09:27:41 -0300 Subject: [erlang-questions] Can't connect to Erlang node in docker In-Reply-To: References: Message-ID: This is most probably not a problem with Erlang but missing configuration settings in Docker. Have you published or exposed the ports outside of the Docker container [1]? Are you starting the containers individually or are you using docker-compose or something like that? I would recommend reading up on the different options that Docker provides for networking. If you send some sample Dockerfile or docker-compose.yml file with the arguments you're using to start the nodes, we could provide better feedback. [1] https://docs.docker.com/config/containers/container-networking/ On Wed, Jun 5, 2019 at 9:11 AM Roger Lipscombe wrote: > I've got an Erlang node running in docker[1]. How do I connect to it > from the host? > > I've tried every combination of -name, -sname, etc., that I can think > of, and all I'm getting is 'pang'. > > I can telnet to the distribution port on the container; if I use > erl_epmd:names/1, passing the DNS name of the docker container, it > returns the expected Erlang nodes. > > But I can't get Erlang distribution to connect from the host to the > node in the docker container. > > Interestingly, if I connect from the node inside docker to a node on > the host, it all starts working. But that's not useful for what I'm > trying to do. > > [1] actually several Erlang nodes in several docker containers. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Wed Jun 5 15:22:51 2019 From: roger@REDACTED (Roger Lipscombe) Date: Wed, 5 Jun 2019 14:22:51 +0100 Subject: [erlang-questions] Can't connect to Erlang node in docker In-Reply-To: References: Message-ID: On Wed, 5 Jun 2019 at 13:28, Juan Jose Comellas wrote: > This is most probably not a problem with Erlang but missing configuration settings in Docker. Have you published or exposed the ports outside of the Docker container [1]? Are you starting the containers individually or are you using docker-compose or something like that? Using docker-compose. The ports are not forwarded to localhost, because I've got a whole bunch of Erlang nodes in docker containers, and that approach won't scale. > I would recommend reading up on the different options that Docker provides for networking. Docker networking is fine; using the IP address of the container (or DNS name; I've got dnsmasq forwarding a domain suffix), I can telnet to the epmd port and to the distribution port (and to various HTTP ports, etc.). It's Erlang's node discovery/connection process that's not working. erl_epmd:names works fine if I run it on the host, and specify one of the containers. So I'm guessing that Erlang doesn't like the lashed-up DNS resolution, or something, but it won't connect using the IP address in the node name either. As I pointed out, if I connect from a node in the container to a node on the host, it works fine, which also implies that it's not a networking problem, per se. But rather it's a discovery/resolution problem. But I don't understand how that actually works in OTP, so... > If you send some sample Dockerfile or docker-compose.yml file with the arguments you're using to start the nodes, we could provide better feedback. The nodes are using relx-generated startup scripts, identical to the ones we're using in production (where they're not running in containers). We start them using s6-overlay. From roger@REDACTED Wed Jun 5 16:04:43 2019 From: roger@REDACTED (Roger Lipscombe) Date: Wed, 5 Jun 2019 15:04:43 +0100 Subject: [erlang-questions] Can't connect to Erlang node in docker In-Reply-To: References: Message-ID: OK. I think I discovered the relevant caveat in the documentation: "A node with a long node name cannot communicate with a node with a short node name." The node I'd like to connect to is using short names, but I'm having to use a long name on the host (because of the differing domain suffix). *But* when I set the remote node to use long name mode, I *still* can't connect. *However*, I found various places (e.g. [1]) which state that you can't mix short and long names at all. This is obviously incorrect, 'cos I successfully connected from the short-name node to the long-name node. Also, in that documentation, "communicate with" implies (to me, at least) both directions. Did something accidentally get "fixed", allowing short-to-long, but not long-to-short? [1] https://stackoverflow.com/questions/26474591/connecting-erlang-nodes-when-an-internal-and-external-ip-address-are-at-play On Wed, 5 Jun 2019 at 14:22, Roger Lipscombe wrote: > > On Wed, 5 Jun 2019 at 13:28, Juan Jose Comellas wrote: > > This is most probably not a problem with Erlang but missing configuration settings in Docker. Have you published or exposed the ports outside of the Docker container [1]? Are you starting the containers individually or are you using docker-compose or something like that? > > Using docker-compose. The ports are not forwarded to localhost, > because I've got a whole bunch of Erlang nodes in docker containers, > and that approach won't scale. > > > I would recommend reading up on the different options that Docker provides for networking. > > Docker networking is fine; using the IP address of the container (or > DNS name; I've got dnsmasq forwarding a domain suffix), I can telnet > to the epmd port and to the distribution port (and to various HTTP > ports, etc.). It's Erlang's node discovery/connection process that's > not working. erl_epmd:names works fine if I run it on the host, and > specify one of the containers. > > So I'm guessing that Erlang doesn't like the lashed-up DNS resolution, > or something, but it won't connect using the IP address in the node > name either. > > As I pointed out, if I connect from a node in the container to a node > on the host, it works fine, which also implies that it's not a > networking problem, per se. But rather it's a discovery/resolution > problem. But I don't understand how that actually works in OTP, so... > > > If you send some sample Dockerfile or docker-compose.yml file with the arguments you're using to start the nodes, we could provide better feedback. > > The nodes are using relx-generated startup scripts, identical to the > ones we're using in production (where they're not running in > containers). We start them using s6-overlay. From magnus@REDACTED Wed Jun 5 16:06:14 2019 From: magnus@REDACTED (Magnus Henoch) Date: Wed, 5 Jun 2019 15:06:14 +0100 Subject: [erlang-questions] Can't connect to Erlang node in docker In-Reply-To: References: Message-ID: <1d353ec0-0d6f-7638-75f1-964c8ab9ab5f@gameanalytics.com> On 05/06/2019 13:10, Roger Lipscombe wrote: > I've got an Erlang node running in docker[1]. How do I connect to it > from the host? > > I've tried every combination of -name, -sname, etc., that I can think > of, and all I'm getting is 'pang'. You need a couple of things to succeed: * Both nodes must have the same cookie * The nodes must be using either both -name or both -sname; no mixing long and short names * The local node must use the exact same node name (including host name) that the remote node thinks it has * The local node must be able to resolve the host name part of the node name of the remote node So if the remote node is running with node name foo@REDACTED, you should be able to connect to it with erl -sname whatever -remsh foo@REDACTED You can check the node name of the remote node by looking at the prompt in the Erlang shell. Note that epmd only displays the "local" part of the node name, not the host name, so you may be able to open a connection to the remote node and still have it rejected and closed if the host name part doesn't match. If mycontainer doesn't resolve to the correct IP address, you can use an inet configuration file: http://erlang.org/doc/apps/erts/inet_cfg.html For example, to hard-code an IP address for a host name and force the Erlang node to use this file instead of native name resolution: {host, {10,1,2,3}, ["mycontainer"]}. {lookup, [file]}. Then pass the name of this file like this: erl -sname whatever -remsh foo@REDACTED -kernel inetrc '"./myinetrc"' Hope this helps, Magnus -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Wed Jun 5 16:43:35 2019 From: roger@REDACTED (Roger Lipscombe) Date: Wed, 5 Jun 2019 15:43:35 +0100 Subject: [erlang-questions] Can't connect to Erlang node in docker In-Reply-To: <1d353ec0-0d6f-7638-75f1-964c8ab9ab5f@gameanalytics.com> References: <1d353ec0-0d6f-7638-75f1-964c8ab9ab5f@gameanalytics.com> Message-ID: On Wed, 5 Jun 2019 at 15:06, Magnus Henoch wrote: > Both nodes must have the same cookie They are. > The nodes must be using either both -name or both -sname; no mixing long and short names Except that you can connect from a short name to a long name; see my later post. > The local node must use the exact same node name (including host name) that the remote node thinks it has Does "host name" include the domain name suffix for this purpose? > The local node must be able to resolve the host name part of the node name of the remote node It can. > So if the remote node is running with node name foo@REDACTED, you should be able to connect to it with erl -sname whatever -remsh foo@REDACTED This works fine provided I put 'mycontainer' in /etc/hosts; what doesn't seem to work is 'mycontainer.docker_network', where I've got local dnsmasq forwarding '.docker_network' to the docker resolver. > If mycontainer doesn't resolve to the correct IP address, you can use an inet configuration file: http://erlang.org/doc/apps/erts/inet_cfg.html > For example, to hard-code an IP address for a host name and force the Erlang node to use this file instead of native name resolution: That's not a particularly scalable solution, given that docker hostnames and IP addresses change all the time. I guess I can script it, though. > Hope this helps, > Magnus Thanks, Roger. From mattevans123@REDACTED Wed Jun 5 20:23:56 2019 From: mattevans123@REDACTED (Matthew Evans) Date: Wed, 5 Jun 2019 18:23:56 +0000 Subject: [erlang-questions] Is there a way to stop the shell from doing special formatting? Message-ID: For example: 5> <<224,57,215,169,183,6>>. <<224,57,215,169,183,6>> 6> <<224,57,215,169,183,7>>. <<224,57,215,169,183,7>> 7> <<224,57,215,169,183,8>>. <<"?9???\b">> -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Wed Jun 5 20:52:11 2019 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 5 Jun 2019 14:52:11 -0400 Subject: [erlang-questions] Is there a way to stop the shell from doing special formatting? In-Reply-To: References: Message-ID: On Wed, Jun 5, 2019 at 2:24 PM Matthew Evans wrote: > For example: > > 5> <<224,57,215,169,183,6>>. > <<224,57,215,169,183,6>> > > 6> <<224,57,215,169,183,7>>. > <<224,57,215,169,183,7>> > > 7> <<224,57,215,169,183,8>>. > <<"?9???\b">> > Try turning off shell strings: 1> <<224,57,215,169,183,8>>. <<"\3409\327\251\267\b">> 2> shell:strings(false). true 3> <<224,57,215,169,183,8>>. <<224,57,215,169,183,8>> http://erlang.org/doc/man/shell.html#strings-1 --steve > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattevans123@REDACTED Wed Jun 5 21:25:16 2019 From: mattevans123@REDACTED (Matthew Evans) Date: Wed, 5 Jun 2019 19:25:16 +0000 Subject: [erlang-questions] Is there a way to stop the shell from doing special formatting? In-Reply-To: References: , Message-ID: Thanks Steve ?? ________________________________ From: Steve Vinoski Sent: Wednesday, June 5, 2019 2:52 PM To: Matthew Evans Cc: Erlang Questions discussions Subject: Re: [erlang-questions] Is there a way to stop the shell from doing special formatting? On Wed, Jun 5, 2019 at 2:24 PM Matthew Evans > wrote: For example: 5> <<224,57,215,169,183,6>>. <<224,57,215,169,183,6>> 6> <<224,57,215,169,183,7>>. <<224,57,215,169,183,7>> 7> <<224,57,215,169,183,8>>. <<"?9???\b">> Try turning off shell strings: 1> <<224,57,215,169,183,8>>. <<"\3409\327\251\267\b">> 2> shell:strings(false). true 3> <<224,57,215,169,183,8>>. <<224,57,215,169,183,8>> http://erlang.org/doc/man/shell.html#strings-1 --steve _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From hello@REDACTED Thu Jun 6 13:09:45 2019 From: hello@REDACTED (Adam Lindberg) Date: Thu, 6 Jun 2019 13:09:45 +0200 Subject: [erlang-questions] Trouble building SSL in 22.0.2 Message-ID: Hi, When building 22.0.2 using asdf and kerl, I get the following build error: Build failed. ERLC ../ebin/pubkey_crl.beam VSN ../ebin/public_key.app VSN ../ebin/public_key.appup make[3]: Nothing to be done for `opt'. === Leaving application public_key === Entering application ssl ~/.asdf/plugins/erlang/kerl-home/builds/asdf_22.0.2/otp_src_22.0.2/lib/ssl/src/deps/ssl.d:1: *** missing separator. Stop. make[2]: *** [opt] Error 2 make[1]: *** [opt] Error 2 make: *** [libs] Error 2 The contents of ssl.d: No such command in 22.0.2 of erlang Full log here: https://gist.github.com/eproxus/5a4ab985e4187edc6b5778f186fb4c1b Any ideas how this happened and what to do to fix it? Cheers, Adam From adam@REDACTED Thu Jun 6 15:10:59 2019 From: adam@REDACTED (Adam Kittelson) Date: Thu, 6 Jun 2019 09:10:59 -0400 Subject: [erlang-questions] Trouble building SSL in 22.0.2 In-Reply-To: References: Message-ID: I had this issue as well, I was on a very old version of asdf. Upgrading asdf fixed it for me. On Thu, Jun 6, 2019 at 7:09 AM Adam Lindberg wrote: > Hi, > > When building 22.0.2 using asdf and kerl, I get the following build error: > > Build failed. > ERLC ../ebin/pubkey_crl.beam > VSN ../ebin/public_key.app > VSN ../ebin/public_key.appup > make[3]: Nothing to be done for `opt'. > === Leaving application public_key > === Entering application ssl > > ~/.asdf/plugins/erlang/kerl-home/builds/asdf_22.0.2/otp_src_22.0.2/lib/ssl/src/deps/ssl.d:1: > *** missing separator. Stop. > make[2]: *** [opt] Error 2 > make[1]: *** [opt] Error 2 > make: *** [libs] Error 2 > > The contents of ssl.d: > > No such command in 22.0.2 of erlang > > Full log here: > https://gist.github.com/eproxus/5a4ab985e4187edc6b5778f186fb4c1b > > Any ideas how this happened and what to do to fix it? > > Cheers, > Adam > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hello@REDACTED Thu Jun 6 16:43:15 2019 From: hello@REDACTED (Adam Lindberg) Date: Thu, 6 Jun 2019 16:43:15 +0200 Subject: [erlang-questions] Trouble building SSL in 22.0.2 In-Reply-To: References: Message-ID: That?s it, thanks! Updating from 0.6.0 to 0.7.2 and upgrading asdf-erlang to latest makes the build pass out of the box (including using wxmac 3.0.4 from Homebrew). Cheers, Adam > On 6. Jun 2019, at 15:10, Adam Kittelson wrote: > > I had this issue as well, I was on a very old version of asdf. Upgrading asdf fixed it for me. > > On Thu, Jun 6, 2019 at 7:09 AM Adam Lindberg wrote: > Hi, > > When building 22.0.2 using asdf and kerl, I get the following build error: > > Build failed. > ERLC ../ebin/pubkey_crl.beam > VSN ../ebin/public_key.app > VSN ../ebin/public_key.appup > make[3]: Nothing to be done for `opt'. > === Leaving application public_key > === Entering application ssl > ~/.asdf/plugins/erlang/kerl-home/builds/asdf_22.0.2/otp_src_22.0.2/lib/ssl/src/deps/ssl.d:1: *** missing separator. Stop. > make[2]: *** [opt] Error 2 > make[1]: *** [opt] Error 2 > make: *** [libs] Error 2 > > The contents of ssl.d: > > No such command in 22.0.2 of erlang > > Full log here: https://gist.github.com/eproxus/5a4ab985e4187edc6b5778f186fb4c1b > > Any ideas how this happened and what to do to fix it? > > Cheers, > Adam > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From frank.muller.erl@REDACTED Thu Jun 6 17:25:48 2019 From: frank.muller.erl@REDACTED (Frank Muller) Date: Thu, 6 Jun 2019 17:25:48 +0200 Subject: [erlang-questions] Trouble building SSL in 22.0.2 In-Reply-To: References: Message-ID: Adam Can you put the steps here for future reference? /Frank That?s it, thanks! Updating from 0.6.0 to 0.7.2 and upgrading asdf-erlang > to latest makes the build pass out of the box (including using wxmac 3.0.4 > from Homebrew). > > Cheers, > Adam > > > On 6. Jun 2019, at 15:10, Adam Kittelson wrote: > > > > I had this issue as well, I was on a very old version of asdf. Upgrading > asdf fixed it for me. > > > > On Thu, Jun 6, 2019 at 7:09 AM Adam Lindberg wrote: > > Hi, > > > > When building 22.0.2 using asdf and kerl, I get the following build > error: > > > > Build failed. > > ERLC ../ebin/pubkey_crl.beam > > VSN ../ebin/public_key.app > > VSN ../ebin/public_key.appup > > make[3]: Nothing to be done for `opt'. > > === Leaving application public_key > > === Entering application ssl > > > ~/.asdf/plugins/erlang/kerl-home/builds/asdf_22.0.2/otp_src_22.0.2/lib/ssl/src/deps/ssl.d:1: > *** missing separator. Stop. > > make[2]: *** [opt] Error 2 > > make[1]: *** [opt] Error 2 > > make: *** [libs] Error 2 > > > > The contents of ssl.d: > > > > No such command in 22.0.2 of erlang > > > > Full log here: > https://gist.github.com/eproxus/5a4ab985e4187edc6b5778f186fb4c1b > > > > Any ideas how this happened and what to do to fix it? > > > > Cheers, > > Adam > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From publicityifl@REDACTED Fri Jun 7 11:12:05 2019 From: publicityifl@REDACTED (Jurriaan Hage) Date: Fri, 7 Jun 2019 02:12:05 -0700 Subject: [erlang-questions] Final call for regular papers for IFL 2019 (Implementation and Application of Functional Languages) Message-ID: Hello, Please, find below the call for papers for IFL 2019. With respect to the previous call, the deadline for submitting regular papers has been changed to June 15th. Please forward these to anyone you think may be interested. Apologies for any duplicates you may receive. best regards, Jurriaan Hage Publicity Chair of IFL --- ================================================================================ IFL 2019 31st Symposium on Implementation and Application of Functional Languages National University of Singapore September 25th-27th, 2019 http://2019.iflconference.org ================================================================================ ### Scope The goal of the IFL symposia is to bring together researchers actively engaged in the implementation and application of functional and function-based programming languages. IFL 2019 will be a venue for researchers to present and discuss new ideas and concepts, work in progress, and publication-ripe results related to the implementation and application of functional languages and function-based programming. Topics of interest to IFL include, but are not limited to: - language concepts - type systems, type checking, type inferencing - compilation techniques - staged compilation - run-time function specialization - run-time code generation - partial evaluation - (abstract) interpretation - metaprogramming - generic programming - automatic program generation - array processing - concurrent/parallel programming - concurrent/parallel program execution - embedded systems - web applications - (embedded) domain specific languages - security - novel memory management techniques - run-time profiling performance measurements - debugging and tracing - virtual/abstract machine architectures - validation, verification of functional programs - tools and programming techniques - (industrial) applications ### Keynote Speaker * Olivier Danvy, Yale-NUS College ### Submissions and peer-review Differently from previous editions of IFL, IFL 2019 solicits two kinds of submissions: * Regular papers (12 pages including references) * Draft papers for presentations ('weak' limit between 8 and 15 pages) Regular papers will undergo a rigorous review by the program committee, and will be evaluated according to their correctness, novelty, originality, relevance, significance, and clarity. A set of regular papers will be conditionally accepted for publication. Authors of conditionally accepted papers will be provided with committee reviews along with a set of mandatory revisions. Regular papers not accepted for publication will be considered as draft papers, at the request of the author. Draft papers will be screened to make sure that they are within the scope of IFL, and will be accepted for presentation or rejected accordingly. Prior to the symposium: Authors of conditionally accepted papers and accepted presentations will submit a pre-proceedings version of their work that will appear in the draft proceedings distributed at the symposium. The draft proceedings does not constitute a formal publication. We require that at least one of the authors present the work at IFL 2019. After the symposium: Authors of conditionally accepted papers will submit a revised versions of their paper for the formal post-proceedings. The program committee will assess whether the mandatory revisions have been adequately addressed by the authors and thereby determines the final accept/reject status of the paper. Our interest is to ultimately accept all conditionally accepted papers. If you are an author of a conditionally accepted paper, please make sure that you address all the concerns of the reviewers. Authors of accepted presentations will be given the opportunity to incorporate the feedback from discussions at the symposium and will be invited to submit a revised full article for the formal post-proceedings. The program committee will evaluate these submissions according to their correctness, novelty, originality, relevance, significance, and clarity, and will thereby determine whether the paper is accepted or rejected. ### Publication The formal proceedings will appear in the International Conference Proceedings Series of the ACM Digital Library. At no time may work submitted to IFL be simultaneously submitted to other venues; submissions must adhere to ACM SIGPLAN's republication policy: http://www.sigplan.org/Resources/Policies/Republication ### Important dates Submission of regular papers: June 15, 2019 Submission of draft papers: July 15, 2019 Regular and draft papers notification: August 1, 2019 Deadline for early registration: August 15, 2019 Submission of pre-proceedings version: September 15, 2019 IFL Symposium: September 25-27, 2019 Submission of papers for post-proceedings: November 30, 2019 Notification of acceptance: January 31, 2020 Camera-ready version: February 29, 2020 ### Submission details All contributions must be written in English. Papers must use the ACM two columns conference format, which can be found at: http://www.acm.org/publications/proceedings-template Authors submit through EasyChair: https://easychair.org/conferences/?conf=ifl2019 ### Peter Landin Prize The Peter Landin Prize is awarded to the best paper presented at the symposium every year. The honored article is selected by the program committee based on the submissions received for the formal review process. The prize carries a cash award equivalent to 150 Euros. ### Organization and Program committee Chairs: Jurrien Stutterheim (Standard Chartered Bank Singapore), Wei Ngan Chin (National University of Singapore) Program Committee: - Olaf Chitil, University of Kent - Clemens Grelck, University of Amsterdam - Daisuke Kimura, Toho University - Pieter Koopman, Radboud University - Tamas Kozsik, Eotvos Lorand University - Roman Leschinskiy, Facebook - Ben Lippmeier, The University of New South Wales - Marco T. Morazan, Seton Hall University - Sven-Bodo Scholz, Heriot-Watt University - Tom Schrijvers, Katholieke Universiteit Leuven - Alejandro Serrano, Utrecht University - Tony Sloane, Macquarie University - Simon Thompson, University of Kent - Marcos Viera, Universidad de la Republica - Wei Ngan Chin, NUS - Jurrien Stutterheim, Standard Chartered Bank ### Venue The 31st IFL is organized by the National University of Singapore. Singapore is located in the heart of South-East Asia, and the city itself is extremely well connected by trains and taxis. See the website for more information on the venue. ### Acknowledgments This call-for-papers is an adaptation and evolution of content from previous instances of IFL. We are grateful to prior organizers for their work, which is reused here. A part of IFL 2019 format and CFP language that describes conditionally accepted papers has been adapted from call-for-papers of OOPSLA conferences. -------------- next part -------------- An HTML attachment was scrubbed... URL: From chessvegas@REDACTED Fri Jun 7 13:03:35 2019 From: chessvegas@REDACTED (DOBRO Sergei) Date: Fri, 07 Jun 2019 14:03:35 +0300 Subject: [erlang-questions] Defined macros and logical bitwise operations Message-ID: <4516651559905415@iva4-6593cae50902.qloud-c.yandex.net> An HTML attachment was scrubbed... URL: From elbrujohalcon@REDACTED Fri Jun 7 13:14:05 2019 From: elbrujohalcon@REDACTED (Brujo Benavides) Date: Fri, 7 Jun 2019 08:14:05 -0300 Subject: [erlang-questions] Defined macros and logical bitwise operations In-Reply-To: <4516651559905415@iva4-6593cae50902.qloud-c.yandex.net> References: <4516651559905415@iva4-6593cae50902.qloud-c.yandex.net> Message-ID: <108C3E6F-6D52-4C83-B3B0-5B0262E6F3A5@gmail.com> Hi Sergei, Just to be clear, your issue does not depend on the usage of macros. See below? ????????????????????????????????????????????????????????? -module test. -export [print/0]. print() -> io:format("NUM1: ~p~n", [2#01]), io:format("NUM2: ~p~n", [2#01 bsl 1]), io:format("NUM_BOR_PAR: ~p~n", [(2#01) bor (2#01 bsl 1)]), io:format("NUM_BOR_NO_PAR: ~p~n", [2#01 bor 2#01 bsl 1]), io:format("NUM_BAND_PAR: ~p~n", [(2#01) band (2#01 bsl 1)]), io:format("NUM_BAND_NO_PAR: ~p~n", [2#01 band 2#01 bsl 1]), io:format("NUM_BXOR_PAR: ~p~n", [(2#01) bxor (2#01 bsl 1)]), io:format("NUM_BXOR_NO_PAR: ~p~n", [2#01 bxor 2#01 bsl 1]), ok. ????????????????????????????????????????????????????????? 1> c(test). {ok,test} 2> test:print(). NUM1: 1 NUM2: 2 NUM_BOR_PAR: 3 NUM_BOR_NO_PAR: 2 NUM_BAND_PAR: 0 NUM_BAND_NO_PAR: 2 NUM_BXOR_PAR: 3 NUM_BXOR_NO_PAR: 0 ok ????????????????????????????????????????????????????????? Your issue is predicated entirely on the operator precedence and you can find documentation about it here: http://erlang.org/doc/reference_manual/expressions.html#operator-precedence The important bit there is: + - bor bxor bsl bsr or xor Left associative That means that X band Y bsl Z is not the same as (X band (Y bsl Z)) but ((X band Y) bsl Z). Hope this helps :) Cheers! Brujo Benavides > On 7 Jun 2019, at 08:03, DOBRO Sergei wrote: > > -module(test). > > -export([print/0]). > > -define(NUM1, 2#01). > -define(NUM2, ?NUM1 bsl 1). > > -define(NUM_BOR_PAR, (?NUM1) bor (?NUM2)). > -define(NUM_BOR_NO_PAR, ?NUM1 bor ?NUM2). > > -define(NUM_BAND_PAR, (?NUM1) band (?NUM2)). > -define(NUM_BAND_NO_PAR, ?NUM1 band ?NUM2). > > -define(NUM_BXOR_PAR, (?NUM1) bxor (?NUM2)). > -define(NUM_BXOR_NO_PAR, ?NUM1 bxor ?NUM2). > > print() -> > io:format("NUM1: ~p~n", [?NUM1]), > io:format("NUM2: ~p~n", [?NUM2]), > io:format("NUM_BOR_PAR: ~p~n", [?NUM_BOR_PAR]), > io:format("NUM_BOR_NO_PAR: ~p~n", [?NUM_BOR_NO_PAR]), > io:format("NUM_BAND_PAR: ~p~n", [?NUM_BAND_PAR]), > io:format("NUM_BAND_NO_PAR: ~p~n", [?NUM_BAND_NO_PAR]), > io:format("NUM_BXOR_PAR: ~p~n", [?NUM_BXOR_PAR]), > io:format("NUM_BXOR_NO_PAR: ~p~n", [?NUM_BXOR_NO_PAR]), > ok. -------------- next part -------------- An HTML attachment was scrubbed... URL: From chessvegas@REDACTED Fri Jun 7 14:14:33 2019 From: chessvegas@REDACTED (DOBRO Sergei) Date: Fri, 07 Jun 2019 15:14:33 +0300 Subject: [erlang-questions] Defined macros and logical bitwise operations In-Reply-To: <108C3E6F-6D52-4C83-B3B0-5B0262E6F3A5@gmail.com> References: <4516651559905415@iva4-6593cae50902.qloud-c.yandex.net> <108C3E6F-6D52-4C83-B3B0-5B0262E6F3A5@gmail.com> Message-ID: <5584481559909673@sas2-80c42a3bfdfe.qloud-c.yandex.net> An HTML attachment was scrubbed... URL: From chessvegas@REDACTED Fri Jun 7 14:16:39 2019 From: chessvegas@REDACTED (DOBRO Sergei) Date: Fri, 07 Jun 2019 15:16:39 +0300 Subject: [erlang-questions] Defined macros and logical bitwise operations In-Reply-To: <108C3E6F-6D52-4C83-B3B0-5B0262E6F3A5@gmail.com> References: <4516651559905415@iva4-6593cae50902.qloud-c.yandex.net> <108C3E6F-6D52-4C83-B3B0-5B0262E6F3A5@gmail.com> Message-ID: <5561461559909799@sas2-0106f63be698.qloud-c.yandex.net> An HTML attachment was scrubbed... URL: From krzysztof.jurewicz@REDACTED Sat Jun 8 12:26:31 2019 From: krzysztof.jurewicz@REDACTED (Krzysztof Jurewicz) Date: Sat, 08 Jun 2019 12:26:31 +0200 Subject: [erlang-questions] Why use -record() ? In-Reply-To: References: <12F6D64F-82E1-4733-94C3-C064D07B823D@gmail.com> Message-ID: <87y32cnzjc.fsf@gmail.com> > There are few other benefits of records over other data types: type testing and pattern matching. These subject has been widely covered at [official documentation](http://erlang.org/doc/programming_examples/records.html): > > ```erlang > %% > %% type testing > myfun(X) when is_record(X, person) -> ... It is however worth to mention that ?type testing? of records doesn?t actually perform type testing of record fields. is_record(X, person) checks only that X is a tuple of correct size and that its first element is 'person'. Therefore is_record(#person{name=foo}, person) will return true despite the name field being type annotated as binary. Dialyzer too wil not always catch type errors in record construction. From okaprinarjaya@REDACTED Sat Jun 8 20:19:58 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Sun, 9 Jun 2019 01:19:58 +0700 Subject: [erlang-questions] Library similar with Elixir's GenStage in Erlang? Message-ID: Hi, What library similar with Elixir's GenStage in Erlang? Thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathaniel@REDACTED Sat Jun 8 21:03:27 2019 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Sat, 8 Jun 2019 15:03:27 -0400 Subject: [erlang-questions] Library similar with Elixir's GenStage in Erlang? In-Reply-To: References: Message-ID: <935E78FE-7C67-4934-9A35-8652C774D3DC@waisbrot.net> I?ve used Elixir?s GenStage in Erlang. It?s a good library. > On Jun 8, 2019, at 2:19 PM, I Gusti Ngurah Oka Prinarjaya wrote: > > Hi, > > What library similar with Elixir's GenStage in Erlang? > > Thank you > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From okaprinarjaya@REDACTED Sat Jun 8 23:08:56 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Sun, 9 Jun 2019 04:08:56 +0700 Subject: [erlang-questions] Library similar with Elixir's GenStage in Erlang? In-Reply-To: <935E78FE-7C67-4934-9A35-8652C774D3DC@waisbrot.net> References: <935E78FE-7C67-4934-9A35-8652C774D3DC@waisbrot.net> Message-ID: Hi, How to import Elixir's GenStage from Erlang? I've read this https://joearms.github.io/published/2017-12-18-Calling-Elixir-From-Erlang.html , but is that tutorial still relevant? any newer way? Thank you Pada tanggal Min, 9 Jun 2019 pukul 02.03 Nathaniel Waisbrot < nathaniel@REDACTED> menulis: > I?ve used Elixir?s GenStage in Erlang. It?s a good library. > > > > On Jun 8, 2019, at 2:19 PM, I Gusti Ngurah Oka Prinarjaya < > okaprinarjaya@REDACTED> wrote: > > > > Hi, > > > > What library similar with Elixir's GenStage in Erlang? > > > > Thank you > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jamie+erlang-questions@REDACTED Mon Jun 10 03:29:50 2019 From: jamie+erlang-questions@REDACTED (Jamie McClymont) Date: Mon, 10 Jun 2019 13:29:50 +1200 Subject: [erlang-questions] SSL Client memory usage Message-ID: <66aaddff-2985-47f7-a9bf-3821345e6962@www.fastmail.com> Hi all I'm building an app which needs to be able to make a good number of concurrent HTTPS requests (ideally I could keep high-hundreds/low-thousands of idle keepalive connections open at a time with modest memory usage). Poking around (by making concurrent requests to 50 domains), I find that holding HTTP connections is effectively free memory-wise, while holding HTTPS connections costs around 2.5MB per connection! All this memory usage is in the tls_connection:init/1 processes, according to observer. Looking at the state of these processes, they each hold a lot of information: the extracted/decoded certificate files for all the CA certs on my system (of which there are 134!). From some cursory googling about erlang's memory model, it seems that because this data is not a binary blob but a big tree of terms, it's going to be copied to every tls_connection process. Any pointers on if there is an existing way to mitigate this (while keeping a full set of root certs available)? Otherwise, it seems to me the reasonable solution would be to keep the CA store in a central process (ETS?) and query that for the specific root cert that the server says it is signed by. If I'll be implementing it, could someone sanity-check this approach: * provide an empty list of CAs to the ssl module * provide a verify_fun which matches on {bad_cert, unknown_ca} and then gets the correct CA and performs all the remaining necessary checks (whatever those are...) before returning {valid, UserState} (or {valid_peer, UserState}?) Thanks - Jamie P.s. sorry if this email is a duplicate: I think my last one was silently dropped since I sent it from a different address to the one I subscribed with, but not certain. Appendix - one of the 134 decoded certs stored by every process: {decoded, {{#Ref<0.577909663.1295777797.255240>, 106100277556486529736699587978573607008, {rdnSequence, [[{'AttributeTypeAndValue',{2,5,4,6},"CN"}], [{'AttributeTypeAndValue', {2,5,4,10}, {utf8String,<<"UniTrust">>}}], [{'AttributeTypeAndValue', {2,5,4,3}, {utf8String, <<"UCA Extend"...(28 B)>>}}]]}}, {<<48,130,5,90,48,130,3,66,160,3...(1 kB)>>, {'OTPCertificate', {'OTPTBSCertificate',v3,106100277556486529736699587978573607008, {'SignatureAlgorithm',{1,2,840,113549,1,1,11},'NULL'}, {rdnSequence, [[{'AttributeTypeAndValue',{2,5,4,6},"CN"}], [{'AttributeTypeAndValue', {2,5,4,10}, {utf8String,<<"UniTrust">>}}], [{'AttributeTypeAndValue', {2,5,4,3}, {utf8String, <<"UCA Extend"...(28 B)>>}}]]}, {'Validity',{utcTime,"150313000000Z"},{utcTime,"381231000000Z"}}, {rdnSequence, [[{'AttributeTypeAndValue',{2,5,4,6},"CN"}], [{'AttributeTypeAndValue', {2,5,4,10}, {utf8String,<<"UniTrust">>}}], [{'AttributeTypeAndValue', {2,5,4,3}, {utf8String, <<"UCA Extend"...(28 B)>>}}]]}, {'OTPSubjectPublicKeyInfo', {'PublicKeyAlgorithm',{1,2,840,113549,1,1,1},'NULL'}, {'RSAPublicKey', 689603717979852636831081426524261160465705539154429034072886964586226830541206711367761253770112725458465044497361739659482435934368052465546153546798794021727234724250491430601994902725573258440903769572247434719710136763019389835974660379560646189661431052777862620577580627735499519427983731365873646101116220703443112119094192330629089538925644308340292126955780669276375525797477380635249258541277353787220035677250408659616695990328739419944306797836041926101961617154636604738870383637100044978431121318943912528365357866046388476749437076957867974020718613998841641003305315809368105957269212277478816744375233680463674812053558530520195020911324127390436183137612053158831455696931169855471108960460199610930021503675868841756093916690737925568854144895439113023618139270733541586805299082563244530623575372402602522085830629158312024768857183218701826386841425816972120556588392270337545276474712543306659887836405319075385033422367187135702233624763183702955559196400695312060899684142798408078723091214153237781607055313585627048184929762700635100359534649156945631489915475941635189136663455081054904713071171644045263631869342706234148072080520374089220391876551645591673277270360119934713372188503089344323356273673087, 65537}}, asn1_NOVALUE,asn1_NOVALUE, [{'Extension', {2,5,29,14}, false, <<217,116,58,228,48,61,13,247,18,220...(20 B)>>}, {'Extension', {2,5,29,19}, true, {'BasicConstraints',true,asn1_NOVALUE}}, {'Extension', {2,5,29,15}, true, [digitalSignature,keyCertSign,cRLSign]}]}, {'SignatureAlgorithm',{1,2,840,113549,1,1,11},'NULL'}, <<54,141,151,204,66,21,100,41,55,155...(512 B)>>}}}}, From ingela.andin@REDACTED Mon Jun 10 11:51:00 2019 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 10 Jun 2019 11:51:00 +0200 Subject: [erlang-questions] SSL Client memory usage In-Reply-To: <66aaddff-2985-47f7-a9bf-3821345e6962@www.fastmail.com> References: <66aaddff-2985-47f7-a9bf-3821345e6962@www.fastmail.com> Message-ID: Hi! How do you provide your certificates to ssl? If you provide your CA-certs as binary DER blobs to the connection they will be stored by the connection and be local to the connection. If you provide them through a file they will be stored in a ETS table and maybe referenced by other connections as there is a way to refer to them. Regards Ingela Erlang/OTP Team Den m?n 10 juni 2019 kl 05:10 skrev Jamie McClymont < jamie+erlang-questions@REDACTED>: > Hi all > > I'm building an app which needs to be able to make a good number of > concurrent HTTPS requests (ideally I could keep high-hundreds/low-thousands > of idle keepalive connections open at a time with modest memory usage). > > Poking around (by making concurrent requests to 50 domains), I find that > holding HTTP connections is effectively free memory-wise, while holding > HTTPS connections costs around 2.5MB per connection! All this memory usage > is in the tls_connection:init/1 processes, according to observer. > > Looking at the state of these processes, they each hold a lot of > information: the extracted/decoded certificate files for all the CA certs > on my system (of which there are 134!). From some cursory googling about > erlang's memory model, it seems that because this data is not a binary blob > but a big tree of terms, it's going to be copied to every tls_connection > process. > > Any pointers on if there is an existing way to mitigate this (while > keeping a full set of root certs available)? Otherwise, it seems to me the > reasonable solution would be to keep the CA store in a central process > (ETS?) and query that for the specific root cert that the server says it is > signed by. If I'll be implementing it, could someone sanity-check this > approach: > * provide an empty list of CAs to the ssl module > * provide a verify_fun which matches on {bad_cert, unknown_ca} and then > gets the correct CA and performs all the remaining necessary checks > (whatever those are...) before returning {valid, UserState} (or > {valid_peer, UserState}?) > > > Thanks > - Jamie > > P.s. sorry if this email is a duplicate: I think my last one was silently > dropped since I sent it from a different address to the one I subscribed > with, but not certain. > > Appendix - one of the 134 decoded certs stored by every process: > > {decoded, > {{#Ref<0.577909663.1295777797.255240>, > 106100277556486529736699587978573607008, > {rdnSequence, > [[{'AttributeTypeAndValue',{2,5,4,6},"CN"}], > [{'AttributeTypeAndValue', > {2,5,4,10}, > {utf8String,<<"UniTrust">>}}], > [{'AttributeTypeAndValue', > {2,5,4,3}, > {utf8String, > <<"UCA Extend"...(28 B)>>}}]]}}, > {<<48,130,5,90,48,130,3,66,160,3...(1 kB)>>, > {'OTPCertificate', > {'OTPTBSCertificate',v3,106100277556486529736699587978573607008, > {'SignatureAlgorithm',{1,2,840,113549,1,1,11},'NULL'}, > {rdnSequence, > [[{'AttributeTypeAndValue',{2,5,4,6},"CN"}], > [{'AttributeTypeAndValue', > {2,5,4,10}, > {utf8String,<<"UniTrust">>}}], > [{'AttributeTypeAndValue', > {2,5,4,3}, > {utf8String, > <<"UCA Extend"...(28 B)>>}}]]}, > {'Validity',{utcTime,"150313000000Z"},{utcTime,"381231000000Z"}}, > {rdnSequence, > [[{'AttributeTypeAndValue',{2,5,4,6},"CN"}], > [{'AttributeTypeAndValue', > {2,5,4,10}, > {utf8String,<<"UniTrust">>}}], > [{'AttributeTypeAndValue', > {2,5,4,3}, > {utf8String, > <<"UCA Extend"...(28 B)>>}}]]}, > {'OTPSubjectPublicKeyInfo', > {'PublicKeyAlgorithm',{1,2,840,113549,1,1,1},'NULL'}, > {'RSAPublicKey', > > 689603717979852636831081426524261160465705539154429034072886964586226830541206711367761253770112725458465044497361739659482435934368052465546153546798794021727234724250491430601994902725573258440903769572247434719710136763019389835974660379560646189661431052777862620577580627735499519427983731365873646101116220703443112119094192330629089538925644308340292126955780669276375525797477380635249258541277353787220035677250408659616695990328739419944306797836041926101961617154636604738870383637100044978431121318943912528365357866046388476749437076957867974020718613998841641003305315809368105957269212277478816744375233680463674812053558530520195020911324127390436183137612053158831455696931169855471108960460199610930021503675868841756093916690737925568854144895439113023618139270733541586805299082563244530623575372402602522085830629158312024768857183218701826386841425816972120556588392270337545276474712543306659887836405319075385033422367187135702233624763183702955559196400 > > 695312060899684142798408078723091214153237781607055313585627048184929762700635100359534649156945631489915475941635189136663455081054904713071171644045263631869342706234148072080520374089220391876551645591673277270360119934713372188503089344323356273673087, > 65537}}, > asn1_NOVALUE,asn1_NOVALUE, > [{'Extension', > {2,5,29,14}, > false, > <<217,116,58,228,48,61,13,247,18,220...(20 B)>>}, > {'Extension', > {2,5,29,19}, > true, > {'BasicConstraints',true,asn1_NOVALUE}}, > {'Extension', > {2,5,29,15}, > true, > [digitalSignature,keyCertSign,cRLSign]}]}, > {'SignatureAlgorithm',{1,2,840,113549,1,1,11},'NULL'}, > <<54,141,151,204,66,21,100,41,55,155...(512 B)>>}}}}, > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathaniel@REDACTED Mon Jun 10 15:10:38 2019 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Mon, 10 Jun 2019 09:10:38 -0400 Subject: [erlang-questions] =?utf-8?q?Library_similar_with_Elixir=27s_GenS?= =?utf-8?q?tage_in_Erlang=3F?= In-Reply-To: References: <935E78FE-7C67-4934-9A35-8652C774D3DC@waisbrot.net> Message-ID: <72558b90-94b2-4039-8bdc-eccb10268c9f@www.fastmail.com> The last time I did it was with rebar3_elixir_compile, but I see there's a new suggestion at https://www.rebar3.org/docs/using-available-plugins#elixir-dependencies The readme at https://github.com/tsloughter/rebar_mix seems pretty clear and easy. Give it a try! On Sat, Jun 8, 2019, at 5:09 PM, I Gusti Ngurah Oka Prinarjaya wrote: > Hi, > > How to import Elixir's GenStage from Erlang? > I've read this https://joearms.github.io/published/2017-12-18-Calling-Elixir-From-Erlang.html , but is that tutorial still relevant? any newer way? > > Thank you > > > Pada tanggal Min, 9 Jun 2019 pukul 02.03 Nathaniel Waisbrot menulis: >> I?ve used Elixir?s GenStage in Erlang. It?s a good library. >> >> >> > On Jun 8, 2019, at 2:19 PM, I Gusti Ngurah Oka Prinarjaya wrote: >> > >> > Hi, >> > >> > What library similar with Elixir's GenStage in Erlang? >> > >> > Thank you >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From kuna.prime@REDACTED Mon Jun 10 15:16:07 2019 From: kuna.prime@REDACTED (Karlo Kuna) Date: Mon, 10 Jun 2019 15:16:07 +0200 Subject: [erlang-questions] pagination and mnesia Message-ID: Hello, i was wandering was ti the best way to paginate mnesia table. Problem is hot to get N record from/or up to record X ? I'm interested in ways of doing that without qlc. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Mon Jun 10 15:52:16 2019 From: vances@REDACTED (Vance Shipley) Date: Mon, 10 Jun 2019 19:22:16 +0530 Subject: [erlang-questions] pagination and mnesia In-Reply-To: References: Message-ID: On Mon, Jun 10, 2019 at 6:46 PM Karlo Kuna wrote: > i was wandering was ti the best way to paginate mnesia table. Problem is hot to get N record from/or up to record X ? I'm interested in ways of doing that without qlc. Use mnesia:select./4 which takes a Size argument and returns a Continuity. The trick is to use an ets context, not a transaction. Here's an example from one of our projects: https://github.com/sigscale/rim/blob/master/src/im.erl#L167 -- -Vance From kuna.prime@REDACTED Mon Jun 10 16:59:22 2019 From: kuna.prime@REDACTED (Karlo Kuna) Date: Mon, 10 Jun 2019 16:59:22 +0200 Subject: [erlang-questions] pagination and mnesia In-Reply-To: References: Message-ID: Thank you Vance this is valuable example, how does Continuity holds up against changes in table between mnesia:select calls? On Mon, Jun 10, 2019 at 3:52 PM Vance Shipley wrote: > On Mon, Jun 10, 2019 at 6:46 PM Karlo Kuna wrote: > > i was wandering was ti the best way to paginate mnesia table. Problem is > hot to get N record from/or up to record X ? I'm interested in ways of > doing that without qlc. > > Use mnesia:select./4 which takes a Size argument and returns a Continuity. > The trick is to use an ets context, not a transaction. > > Here's an example from one of our projects: > https://github.com/sigscale/rim/blob/master/src/im.erl#L167 > > -- > -Vance > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Mon Jun 10 17:17:59 2019 From: vances@REDACTED (Vance Shipley) Date: Mon, 10 Jun 2019 20:47:59 +0530 Subject: [erlang-questions] pagination and mnesia In-Reply-To: References: Message-ID: On Mon, Jun 10, 2019 at 8:29 PM Karlo Kuna wrote: > how does Continuity holds up against changes in table between mnesia:select calls? It continues to work however don't expect the results to include new data. -- -Vance From kuna.prime@REDACTED Mon Jun 10 18:58:53 2019 From: kuna.prime@REDACTED (Karlo Kuna) Date: Mon, 10 Jun 2019 18:58:53 +0200 Subject: [erlang-questions] pagination and mnesia In-Reply-To: References: Message-ID: thank you very much Vance On Mon, Jun 10, 2019 at 5:18 PM Vance Shipley wrote: > On Mon, Jun 10, 2019 at 8:29 PM Karlo Kuna wrote: > > how does Continuity holds up against changes in table between > mnesia:select calls? > > It continues to work however don't expect the results to include new data. > > -- > -Vance > -------------- next part -------------- An HTML attachment was scrubbed... URL: From olivier.boudeville@REDACTED Mon Jun 10 20:57:23 2019 From: olivier.boudeville@REDACTED (Olivier Boudeville) Date: Mon, 10 Jun 2019 20:57:23 +0200 Subject: [erlang-questions] [ANN] Ceylan-Seaplus 1.0 Message-ID: <43a5bb6d-4d97-e042-9c87-81194f9313f8@online.fr> Hi, Just wanting to share the release of the version 1.0 of Seaplus. This is an attempt (certainly not rocket science!) at streamlining a simpler, possibly safer/more efficient integration of Erlang with C/C++ code when it is based on a C node (hence not with a NIF) and ei (now that Erl_Interface is obsolete). A goal was to automate/generate most of the corresponding bridge (by default, nothing is to do on the Erlang side except defining the type specification of the targeted API, while on the C side a template of the driver is generated, and library-based facilities are provided for the actual two-way conversions; the service integrator just has to fill the boiler-plate accordingly). At the end, an Erlang module is available, allowing to access as wanted, and seamlessly, to the original C/C++ library. Besides the internal tests, Seaplus has been applied to a more complete example, Ceylan-Mobile (see http://mobile.esperide.org/), based on the Gammu C library. All information to be found in http://seaplus.esperide.org/ ; hope this helps, all feedback welcome! Best regards, Olivier. -- Olivier Boudeville From jamie+erlang-questions@REDACTED Mon Jun 10 16:23:34 2019 From: jamie+erlang-questions@REDACTED (Jamie McClymont) Date: Tue, 11 Jun 2019 02:23:34 +1200 Subject: [erlang-questions] SSL Client memory usage In-Reply-To: References: <66aaddff-2985-47f7-a9bf-3821345e6962@www.fastmail.com> Message-ID: <73033d50-caf9-4954-8fec-309ca26e58a2@www.fastmail.com> Ahh that did it, thanks! I'm using hackney, which in turn uses certifi:cacerts() (binary blobs of cert data). Using hackney's ssl options to instead pass certifi:cacertfile() as an argument brings the memory usage of each keepalive'd tls connection (once the process is GCed) down to <30KB -- that's excellent :) On Mon, 10 Jun 2019, at 9:51 PM, Ingela Andin wrote: > Hi! > > How do you provide your certificates to ssl? If you provide your > CA-certs as binary DER blobs to the connection they > will be stored by the connection and be local to the connection. If you > provide them through a file they will be stored > in a ETS table and maybe referenced by other connections as there is a > way to refer to them. > > Regards Ingela Erlang/OTP Team > > Den m?n 10 juni 2019 kl 05:10 skrev Jamie McClymont > >: > > Hi all > > > > I'm building an app which needs to be able to make a good number of concurrent HTTPS requests (ideally I could keep high-hundreds/low-thousands of idle keepalive connections open at a time with modest memory usage). > > > > Poking around (by making concurrent requests to 50 domains), I find that holding HTTP connections is effectively free memory-wise, while holding HTTPS connections costs around 2.5MB per connection! All this memory usage is in the tls_connection:init/1 processes, according to observer. > > > > Looking at the state of these processes, they each hold a lot of information: the extracted/decoded certificate files for all the CA certs on my system (of which there are 134!). From some cursory googling about erlang's memory model, it seems that because this data is not a binary blob but a big tree of terms, it's going to be copied to every tls_connection process. > > > > Any pointers on if there is an existing way to mitigate this (while keeping a full set of root certs available)? Otherwise, it seems to me the reasonable solution would be to keep the CA store in a central process (ETS?) and query that for the specific root cert that the server says it is signed by. If I'll be implementing it, could someone sanity-check this approach: > > * provide an empty list of CAs to the ssl module > > * provide a verify_fun which matches on {bad_cert, unknown_ca} and then gets the correct CA and performs all the remaining necessary checks (whatever those are...) before returning {valid, UserState} (or {valid_peer, UserState}?) > > > > > > Thanks > > - Jamie > > > > P.s. sorry if this email is a duplicate: I think my last one was silently dropped since I sent it from a different address to the one I subscribed with, but not certain. > > > > Appendix - one of the 134 decoded certs stored by every process: > > > > {decoded, > > {{#Ref<0.577909663.1295777797.255240>, > > 106100277556486529736699587978573607008, > > {rdnSequence, > > [[{'AttributeTypeAndValue',{2,5,4,6},"CN"}], > > [{'AttributeTypeAndValue', > > {2,5,4,10}, > > {utf8String,<<"UniTrust">>}}], > > [{'AttributeTypeAndValue', > > {2,5,4,3}, > > {utf8String, > > <<"UCA Extend"...(28 B)>>}}]]}}, > > {<<48,130,5,90,48,130,3,66,160,3...(1 kB)>>, > > {'OTPCertificate', > > {'OTPTBSCertificate',v3,106100277556486529736699587978573607008, > > {'SignatureAlgorithm',{1,2,840,113549,1,1,11},'NULL'}, > > {rdnSequence, > > [[{'AttributeTypeAndValue',{2,5,4,6},"CN"}], > > [{'AttributeTypeAndValue', > > {2,5,4,10}, > > {utf8String,<<"UniTrust">>}}], > > [{'AttributeTypeAndValue', > > {2,5,4,3}, > > {utf8String, > > <<"UCA Extend"...(28 B)>>}}]]}, > > {'Validity',{utcTime,"150313000000Z"},{utcTime,"381231000000Z"}}, > > {rdnSequence, > > [[{'AttributeTypeAndValue',{2,5,4,6},"CN"}], > > [{'AttributeTypeAndValue', > > {2,5,4,10}, > > {utf8String,<<"UniTrust">>}}], > > [{'AttributeTypeAndValue', > > {2,5,4,3}, > > {utf8String, > > <<"UCA Extend"...(28 B)>>}}]]}, > > {'OTPSubjectPublicKeyInfo', > > {'PublicKeyAlgorithm',{1,2,840,113549,1,1,1},'NULL'}, > > {'RSAPublicKey', > > 689603717979852636831081426524261160465705539154429034072886964586226830541206711367761253770112725458465044497361739659482435934368052465546153546798794021727234724250491430601994902725573258440903769572247434719710136763019389835974660379560646189661431052777862620577580627735499519427983731365873646101116220703443112119094192330629089538925644308340292126955780669276375525797477380635249258541277353787220035677250408659616695990328739419944306797836041926101961617154636604738870383637100044978431121318943912528365357866046388476749437076957867974020718613998841641003305315809368105957269212277478816744375233680463674812053558530520195020911324127390436183137612053158831455696931169855471108960460199610930021503675868841756093916690737925568854144895439113023618139270733541586805299082563244530623575372402602522085830629158312024768857183218701826386841425816972120556588392270337545276474712543306659887836405319075385033422367187135702233624763183702955559196400 > > 695312060899684142798408078723091214153237781607055313585627048184929762700635100359534649156945631489915475941635189136663455081054904713071171644045263631869342706234148072080520374089220391876551645591673277270360119934713372188503089344323356273673087, > > 65537}}, > > asn1_NOVALUE,asn1_NOVALUE, > > [{'Extension', > > {2,5,29,14}, > > false, > > <<217,116,58,228,48,61,13,247,18,220...(20 B)>>}, > > {'Extension', > > {2,5,29,19}, > > true, > > {'BasicConstraints',true,asn1_NOVALUE}}, > > {'Extension', > > {2,5,29,15}, > > true, > > [digitalSignature,keyCertSign,cRLSign]}]}, > > {'SignatureAlgorithm',{1,2,840,113549,1,1,11},'NULL'}, > > <<54,141,151,204,66,21,100,41,55,155...(512 B)>>}}}}, > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions From bram.verburg@REDACTED Tue Jun 11 12:00:37 2019 From: bram.verburg@REDACTED (Bram Verburg) Date: Tue, 11 Jun 2019 13:00:37 +0300 Subject: [erlang-questions] SSL Client memory usage In-Reply-To: <73033d50-caf9-4954-8fec-309ca26e58a2@www.fastmail.com> References: <66aaddff-2985-47f7-a9bf-3821345e6962@www.fastmail.com> <73033d50-caf9-4954-8fec-309ca26e58a2@www.fastmail.com> Message-ID: Please make sure to also pass {verify, verify_peer}, since any ssl_options you pass will overwrite the hackney defaults, and therefore pick up the ssl default of verify_none. Eshell V10.2.5 (abort with ^G) 1> hackney:request(get, <<"https://self-signed.badssl.com/">>, [], <<>>, [{ssl_options, [{cacertfile, certifi:cacertfile()}]}]). {ok,200,[...],#Ref<...>} // Not good! 2> hackney:request(get, <<"https://self-signed.badssl.com/">>, [], <<>>, [{ssl_options, [{cacertfile, certifi:cacertfile()}, {verify, verify_peer}]}]). {error,{tls_alert,"bad certificate"}} Bram On Tue, 11 Jun 2019 at 09:06, Jamie McClymont < jamie+erlang-questions@REDACTED> wrote: > Ahh that did it, thanks! > > I'm using hackney, which in turn uses certifi:cacerts() (binary blobs of > cert data). > > Using hackney's ssl options to instead pass certifi:cacertfile() as an > argument brings the memory usage of each keepalive'd tls connection (once > the process is GCed) down to <30KB -- that's excellent :) > > > > On Mon, 10 Jun 2019, at 9:51 PM, Ingela Andin wrote: > > Hi! > > > > How do you provide your certificates to ssl? If you provide your > > CA-certs as binary DER blobs to the connection they > > will be stored by the connection and be local to the connection. If you > > provide them through a file they will be stored > > in a ETS table and maybe referenced by other connections as there is a > > way to refer to them. > > > > Regards Ingela Erlang/OTP Team > > > > Den m?n 10 juni 2019 kl 05:10 skrev Jamie McClymont > > > >: > > > Hi all > > > > > > I'm building an app which needs to be able to make a good number of > concurrent HTTPS requests (ideally I could keep high-hundreds/low-thousands > of idle keepalive connections open at a time with modest memory usage). > > > > > > Poking around (by making concurrent requests to 50 domains), I find > that holding HTTP connections is effectively free memory-wise, while > holding HTTPS connections costs around 2.5MB per connection! All this > memory usage is in the tls_connection:init/1 processes, according to > observer. > > > > > > Looking at the state of these processes, they each hold a lot of > information: the extracted/decoded certificate files for all the CA certs > on my system (of which there are 134!). From some cursory googling about > erlang's memory model, it seems that because this data is not a binary blob > but a big tree of terms, it's going to be copied to every tls_connection > process. > > > > > > Any pointers on if there is an existing way to mitigate this (while > keeping a full set of root certs available)? Otherwise, it seems to me the > reasonable solution would be to keep the CA store in a central process > (ETS?) and query that for the specific root cert that the server says it is > signed by. If I'll be implementing it, could someone sanity-check this > approach: > > > * provide an empty list of CAs to the ssl module > > > * provide a verify_fun which matches on {bad_cert, unknown_ca} and > then gets the correct CA and performs all the remaining necessary checks > (whatever those are...) before returning {valid, UserState} (or > {valid_peer, UserState}?) > > > > > > > > > Thanks > > > - Jamie > > > > > > P.s. sorry if this email is a duplicate: I think my last one was > silently dropped since I sent it from a different address to the one I > subscribed with, but not certain. > > > > > > Appendix - one of the 134 decoded certs stored by every process: > > > > > > {decoded, > > > {{#Ref<0.577909663.1295777797.255240>, > > > 106100277556486529736699587978573607008, > > > {rdnSequence, > > > [[{'AttributeTypeAndValue',{2,5,4,6},"CN"}], > > > [{'AttributeTypeAndValue', > > > {2,5,4,10}, > > > {utf8String,<<"UniTrust">>}}], > > > [{'AttributeTypeAndValue', > > > {2,5,4,3}, > > > {utf8String, > > > <<"UCA Extend"...(28 B)>>}}]]}}, > > > {<<48,130,5,90,48,130,3,66,160,3...(1 kB)>>, > > > {'OTPCertificate', > > > {'OTPTBSCertificate',v3,106100277556486529736699587978573607008, > > > {'SignatureAlgorithm',{1,2,840,113549,1,1,11},'NULL'}, > > > {rdnSequence, > > > [[{'AttributeTypeAndValue',{2,5,4,6},"CN"}], > > > [{'AttributeTypeAndValue', > > > {2,5,4,10}, > > > {utf8String,<<"UniTrust">>}}], > > > [{'AttributeTypeAndValue', > > > {2,5,4,3}, > > > {utf8String, > > > <<"UCA Extend"...(28 B)>>}}]]}, > > > {'Validity',{utcTime,"150313000000Z"},{utcTime,"381231000000Z"}}, > > > {rdnSequence, > > > [[{'AttributeTypeAndValue',{2,5,4,6},"CN"}], > > > [{'AttributeTypeAndValue', > > > {2,5,4,10}, > > > {utf8String,<<"UniTrust">>}}], > > > [{'AttributeTypeAndValue', > > > {2,5,4,3}, > > > {utf8String, > > > <<"UCA Extend"...(28 B)>>}}]]}, > > > {'OTPSubjectPublicKeyInfo', > > > {'PublicKeyAlgorithm',{1,2,840,113549,1,1,1},'NULL'}, > > > {'RSAPublicKey', > > > > 689603717979852636831081426524261160465705539154429034072886964586226830541206711367761253770112725458465044497361739659482435934368052465546153546798794021727234724250491430601994902725573258440903769572247434719710136763019389835974660379560646189661431052777862620577580627735499519427983731365873646101116220703443112119094192330629089538925644308340292126955780669276375525797477380635249258541277353787220035677250408659616695990328739419944306797836041926101961617154636604738870383637100044978431121318943912528365357866046388476749437076957867974020718613998841641003305315809368105957269212277478816744375233680463674812053558530520195020911324127390436183137612053158831455696931169855471108960460199610930021503675868841756093916690737925568854144895439113023618139270733541586805299082563244530623575372402602522085830629158312024768857183218701826386841425816972120556588392270337545276474712543306659887836405319075385033422367187135702233624763183702955559196400 > > > > 695312060899684142798408078723091214153237781607055313585627048184929762700635100359534649156945631489915475941635189136663455081054904713071171644045263631869342706234148072080520374089220391876551645591673277270360119934713372188503089344323356273673087, > > > 65537}}, > > > asn1_NOVALUE,asn1_NOVALUE, > > > [{'Extension', > > > {2,5,29,14}, > > > false, > > > <<217,116,58,228,48,61,13,247,18,220...(20 B)>>}, > > > {'Extension', > > > {2,5,29,19}, > > > true, > > > {'BasicConstraints',true,asn1_NOVALUE}}, > > > {'Extension', > > > {2,5,29,15}, > > > true, > > > [digitalSignature,keyCertSign,cRLSign]}]}, > > > {'SignatureAlgorithm',{1,2,840,113549,1,1,11},'NULL'}, > > > <<54,141,151,204,66,21,100,41,55,155...(512 B)>>}}}}, > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aj.shirvani@REDACTED Wed Jun 12 15:39:23 2019 From: aj.shirvani@REDACTED (Ali Shirvani) Date: Wed, 12 Jun 2019 18:09:23 +0430 Subject: [erlang-questions] SIP protocol stack in Erlang/OTP Message-ID: Hi all, I'm new to Erlang/OTP. According to Erlang history, the main purpose of this language is telephony systems. I found that there are lots of related modules such as ASN.1 parser and Diameter. But I didn't find any SIP module in Erlang/OTP official libraries. It's hard to imagine a programming language that developed for telecom purposes and doesn't offer a module for the most useful telecom protocol such as SIP. So my question is why Erlang/OTP doesn't provide an official SIP stack? I found NkSIP but it seems it doesn't maintain anymore. Regards, Ali -------------- next part -------------- An HTML attachment was scrubbed... URL: From Oliver.Korpilla@REDACTED Thu Jun 13 08:20:42 2019 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Thu, 13 Jun 2019 08:20:42 +0200 Subject: [erlang-questions] SIP protocol stack in Erlang/OTP In-Reply-To: References: Message-ID: Hello, Ali. ? Erlang as a language was open-sourced at some point but previous to that was simply proprietary to Ericsson. Ericsson pursued Erlang for its own purposes. ? They might or might not have their own implementations of various telco protocols but they simply are under no obligation to share them with the public. After all, actual telco protocols usually would be most useful to their direct competitors... ? In comparison, open source developers seem less interested in the telco protocols and focus on Internet protocols. And projects like OpenBTS are written in the usual C derivatives... All the basic tools (like asn1ct) are there to build pretty much anything telco-related robust and reliably, but one often has to create their own (my guess). ? Oliver ? Gesendet:?Mittwoch, 12. Juni 2019 um 15:39 Uhr Von:?"Ali Shirvani" An:?erlang-questions@REDACTED Betreff:?[erlang-questions] SIP protocol stack in Erlang/OTP Hi all, ? I'm new to Erlang/OTP. According to Erlang history, the main purpose of this language is telephony systems. I found that there are lots of related modules such as ASN.1 parser and Diameter. But I didn't find any SIP module in Erlang/OTP official libraries. It's hard to imagine a programming language that developed for telecom purposes and doesn't offer? a module for the most useful telecom protocol such as SIP.? ? So my question is why Erlang/OTP doesn't provide an official SIP stack? I found NkSIP but it seems it doesn't maintain anymore. ? Regards, Ali_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From thomas.elsgaard@REDACTED Thu Jun 13 08:29:05 2019 From: thomas.elsgaard@REDACTED (Thomas Elsgaard) Date: Thu, 13 Jun 2019 08:29:05 +0200 Subject: [erlang-questions] SIP protocol stack in Erlang/OTP In-Reply-To: References: Message-ID: Hi Ali Look at ersip https://github.com/poroh/ersip Thomas tor. 13. jun. 2019 kl. 06.54 skrev Ali Shirvani : > Hi all, > > I'm new to Erlang/OTP. According to Erlang history, the main purpose of > this language is telephony systems. I found that there are lots of related > modules such as ASN.1 parser and Diameter. But I didn't find any SIP module > in Erlang/OTP official libraries. It's hard to imagine a programming > language that developed for telecom purposes and doesn't offer a module > for the most useful telecom protocol such as SIP. > > So my question is why Erlang/OTP doesn't provide an official SIP stack? I > found NkSIP but it seems it doesn't maintain anymore. > > Regards, > Ali > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From otp@REDACTED Thu Jun 13 10:45:15 2019 From: otp@REDACTED (Erlang/OTP) Date: Thu, 13 Jun 2019 10:45:15 +0200 Subject: [erlang-questions] Patch Package OTP 22.0.3 Released Message-ID: <20190613084515.GA27473@erix.ericsson.se> Patch Package: OTP 22.0.3 Git Tag: OTP-22.0.3 Date: 2019-06-13 Trouble Report Id: OTP-15844, OTP-15861, OTP-15862, OTP-15864, OTP-15865, OTP-15871, OTP-15872, OTP-15873, OTP-15875 Seq num: ERIERL-374, ERL-951, ERL-953, ERL-962, ERL-964, ERL-965, ERL-967 System: OTP Release: 22 Application: compiler-7.4.2, dialyzer-4.0.1, erts-10.4.2, ssl-9.3.2, stdlib-3.9.2 Predecessor: OTP 22.0.2 Check out the git tag OTP-22.0.3, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- compiler-7.4.2 -------------------------------------------------- --------------------------------------------------------------------- The compiler-7.4.2 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-15872 Application(s): compiler Fixed an incorrect type determination for constructed binaries, which could cause is_binary checks to succeed when they shouldn't have. Full runtime dependencies of compiler-7.4.2: crypto-3.6, erts-9.0, hipe-3.12, kernel-4.0, stdlib-2.5 --------------------------------------------------------------------- --- dialyzer-4.0.1 -------------------------------------------------- --------------------------------------------------------------------- The dialyzer-4.0.1 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-15861 Application(s): dialyzer Related Id(s): ERL-953 Fix a bug that caused a crash when indenting a Dialyzer warning mentioning more than one record field. Full runtime dependencies of dialyzer-4.0.1: compiler-7.0, erts-9.0, hipe-3.16.1, kernel-5.3, stdlib-3.4, syntax_tools-2.0, wx-1.2 --------------------------------------------------------------------- --- erts-10.4.2 ----------------------------------------------------- --------------------------------------------------------------------- The erts-10.4.2 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-15865 Application(s): erts Related Id(s): ERL-964 Fixed process_info(Pid,reductions) to not categorically increase reduction count of the measured process Pid. Repeated reduction measure of an idle process will most often (but not guaranteed) return the same value, like it behaved before OTP 21.3.8. OTP-15871 Application(s): erts Related Id(s): ERIERL-374 Fixed an incorrect load-time optimization that could cause a crash when extracting deeply nested tuple elements. OTP-15873 Application(s): erts Related Id(s): ERL-965 Fix bug causing VM crash when pressing P for "proc info" in Erlang shell break menu. Bug exists since OTP 22.0. Full runtime dependencies of erts-10.4.2: kernel-6.1, sasl-3.3, stdlib-3.5 --------------------------------------------------------------------- --- ssl-9.3.2 ------------------------------------------------------- --------------------------------------------------------------------- The ssl-9.3.2 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-15844 Application(s): ssl Returned "alert error string" is now same as logged alert string OTP-15862 Application(s): ssl Related Id(s): ERL-951 Fix returned extension map fields to follow the documentation. OTP-15864 Application(s): ssl Related Id(s): ERL-962 Avoid DTLS crash due to missing gen_server return value in DTLS packet demux process. Full runtime dependencies of ssl-9.3.2: crypto-4.2, erts-10.0, inets-5.10.7, kernel-6.0, public_key-1.5, stdlib-3.5 --------------------------------------------------------------------- --- stdlib-3.9.2 ---------------------------------------------------- --------------------------------------------------------------------- The stdlib-3.9.2 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-15875 Application(s): stdlib Related Id(s): ERL-967 Fix a bug that could cause a loop when formatting terms using the control sequences p or P and limiting the output with the option chars_limit. Full runtime dependencies of stdlib-3.9.2: compiler-5.0, crypto-3.3, erts-10.4, kernel-6.0, sasl-3.0 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From okaprinarjaya@REDACTED Fri Jun 14 06:45:17 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Fri, 14 Jun 2019 11:45:17 +0700 Subject: [erlang-questions] Why this code not being infinite recursive? Message-ID: Hi, I learn gen_tcp from learnyousomeerlangg book. I try to understand flow of code below. And i got confused with acceptor/1 . Because acceptor/1 call itself but have no base case and just execute once. It's not usual. Why? >From my understanding it should be an infinity recursive. -module(naive_tcp). -compile(export_all). start_server(Port) -> Pid = spawn_link( fun() -> io:format("Spawned at start_server()~n"), {ok, ListenSocket} = gen_tcp:listen(Port, [binary, {active, false}]), spawn(fun() -> acceptor(ListenSocket) end), timer:sleep(infinity) end ), {ok, Pid}. acceptor(ListenSocket) -> io:format("I am acceptor~n"), {ok, AcceptorSocket} = gen_tcp:accept(ListenSocket), spawn(fun() -> acceptor(ListenSocket) end), handle(AcceptorSocket). handle(AcceptorSocket) -> inet:setopts(AcceptorSocket, [{active, once}]), receive {tcp, AcceptorSocket, <<"quit", _/binary>>} -> gen_tcp:close(AcceptorSocket); {tcp, AcceptorSocket, Message} -> gen_tcp:send(AcceptorSocket, Message), handle(AcceptorSocket) end. Please enlightenment Thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: From by@REDACTED Fri Jun 14 06:56:29 2019 From: by@REDACTED (by) Date: Fri, 14 Jun 2019 12:56:29 +0800 Subject: [erlang-questions] Why this code not being infinite recursive? In-Reply-To: References: Message-ID: <1892CEE4-E99C-437C-95F3-89647113BF4E@meetlost.com> Hi, The acceptor is waiting for new TCP connection. If a new connection arrives, it will spawn a new process waiting for future new connection first, then the current process will handle communication on the connected connection. Best Regards, Yao > ? 2019?6?14??12:45?I Gusti Ngurah Oka Prinarjaya ??? > > Hi, > > I learn gen_tcp from learnyousomeerlangg book. I try to understand flow of code below. > And i got confused with acceptor/1 . Because acceptor/1 call itself but have no base case and just execute once. It's not usual. Why? > From my understanding it should be an infinity recursive. > > -module(naive_tcp). > -compile(export_all). > > start_server(Port) -> > Pid = spawn_link( > fun() -> > io:format("Spawned at start_server()~n"), > {ok, ListenSocket} = gen_tcp:listen(Port, [binary, {active, false}]), > spawn(fun() -> acceptor(ListenSocket) end), > timer:sleep(infinity) > end > ), > {ok, Pid}. > > acceptor(ListenSocket) -> > io:format("I am acceptor~n"), > {ok, AcceptorSocket} = gen_tcp:accept(ListenSocket), > spawn(fun() -> acceptor(ListenSocket) end), > handle(AcceptorSocket). > > handle(AcceptorSocket) -> > inet:setopts(AcceptorSocket, [{active, once}]), > receive > {tcp, AcceptorSocket, <<"quit", _/binary>>} -> > gen_tcp:close(AcceptorSocket); > {tcp, AcceptorSocket, Message} -> > gen_tcp:send(AcceptorSocket, Message), > handle(AcceptorSocket) > end. > > Please enlightenment > > Thank you > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From okaprinarjaya@REDACTED Fri Jun 14 08:31:26 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Fri, 14 Jun 2019 13:31:26 +0700 Subject: [erlang-questions] Library similar with Elixir's GenStage in Erlang? In-Reply-To: <72558b90-94b2-4039-8bdc-eccb10268c9f@www.fastmail.com> References: <935E78FE-7C67-4934-9A35-8652C774D3DC@waisbrot.net> <72558b90-94b2-4039-8bdc-eccb10268c9f@www.fastmail.com> Message-ID: Hi Nathaniel, Thank you very much! :) Pada tanggal Sen, 10 Jun 2019 pukul 20.10 Nathaniel Waisbrot < nathaniel@REDACTED> menulis: > The last time I did it was with rebar3_elixir_compile, but I see there's a > new suggestion at > > https://www.rebar3.org/docs/using-available-plugins#elixir-dependencies > > The readme at https://github.com/tsloughter/rebar_mix seems pretty clear > and easy. Give it a try! > > > > On Sat, Jun 8, 2019, at 5:09 PM, I Gusti Ngurah Oka Prinarjaya wrote: > > Hi, > > How to import Elixir's GenStage from Erlang? > I've read this > https://joearms.github.io/published/2017-12-18-Calling-Elixir-From-Erlang.html , > but is that tutorial still relevant? any newer way? > > Thank you > > > Pada tanggal Min, 9 Jun 2019 pukul 02.03 Nathaniel Waisbrot < > nathaniel@REDACTED> menulis: > > I?ve used Elixir?s GenStage in Erlang. It?s a good library. > > > > On Jun 8, 2019, at 2:19 PM, I Gusti Ngurah Oka Prinarjaya < > okaprinarjaya@REDACTED> wrote: > > > > Hi, > > > > What library similar with Elixir's GenStage in Erlang? > > > > Thank you > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Fri Jun 14 10:55:03 2019 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 14 Jun 2019 10:55:03 +0200 Subject: [erlang-questions] SIP protocol stack in Erlang/OTP In-Reply-To: References: Message-ID: It's not part of Ericsson's Erlang/OTP distribution, but YXA has been around since 2002: https://www.stacken.kth.se/project/yxa/ https://en.wikipedia.org/wiki/YXA https://github.com/fredrikt/yxa /Richard Den tors 13 juni 2019 kl 06:54 skrev Ali Shirvani : > Hi all, > > I'm new to Erlang/OTP. According to Erlang history, the main purpose of > this language is telephony systems. I found that there are lots of related > modules such as ASN.1 parser and Diameter. But I didn't find any SIP module > in Erlang/OTP official libraries. It's hard to imagine a programming > language that developed for telecom purposes and doesn't offer a module > for the most useful telecom protocol such as SIP. > > So my question is why Erlang/OTP doesn't provide an official SIP stack? I > found NkSIP but it seems it doesn't maintain anymore. > > Regards, > Ali > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ebegumisa@REDACTED Sun Jun 16 10:27:58 2019 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sun, 16 Jun 2019 18:27:58 +1000 Subject: [erlang-questions] Pattern matching of literal small/big/bignum integers and binary search Message-ID: Hello all, I have some questions about how pattern matches on integer literals are evaluated by the emulator. As an example, say I have the following function which expects an integer that's a power of 2 and returns corresponding position of the lone "on" bit... single_cardinalty_to_pos(2#1) -> 1; single_cardinalty_to_pos(2#10) -> 2; single_cardinalty_to_pos(2#100) -> 3; ... single_cardinalty_to_pos(2#100...0) -> 128. I'm trying to gauge whether or not I've correctly understood the resultant select_val BEAM assembly instruction w.r.t gen_select_val [1] in beam_load.c (without even being at all certain that gen_select_val is even relevant). {label,133}. {test,is_integer,{f,132},[{x,0}]}. {select_val,{x,0}, {f,132}, {list,[{integer,170141183460469231731687303715884105728}, {f,134}, .... {integer,4}, {f,139}, {integer,2}, {f,140}, {integer,1}, {f,140}]}}. I could use some help to determine if the following assumptions are correct... A) That rather than brute forcing, the emulator will perform a binary search [2] for the function argument against all the literals from all the clauses which is the purpose of quick-sorting the select_val list, and as such, B) If I want to increase the upper limit that the function can handle, I can simply add more clauses rather than trying to implement a binary search for the "on" bit myself using various masks and bit shifts. C) That the emulator will be smart about whether on not the function argument is an 8 bit, 32/64 bit integer versus a bignum integer and thus know what candidates can be excluded from the binary search so there's no benefit to splitting the function to handle different ranges. D) That it's not unreasonable in this case to have even say, 1024 clauses in order to handle bignum integers of up to 1024 bits. The only downside will be longer compile times and possibly longer loading of the module. If these assumptions are correct, then I could have the emulator do a lot of work for me with a number of similar functions and avoid writing a lot code. Thanks in advance. - Edmond - [1] https://github.com/erlang/otp/blob/48e1cc6a6f7bebacd5fb060bfd65ececbabfa6a1/erts/emulator/beam/beam_load.c#L4097 [2] https://github.com/erlang/otp/blob/48e1cc6a6f7bebacd5fb060bfd65ececbabfa6a1/erts/emulator/beam/beam_load.c#L4159 -- Using Opera's mail client: http://www.opera.com/mail/ From ebegumisa@REDACTED Sun Jun 16 10:39:40 2019 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sun, 16 Jun 2019 18:39:40 +1000 Subject: [erlang-questions] Pattern matching of literal small/big/bignum integers and binary search In-Reply-To: References: Message-ID: On Sun, 16 Jun 2019 18:27:58 +1000, Edmond Begumisa wrote: > Hello all, > > I have some questions about how pattern matches on integer literals are > evaluated by the emulator. > > As an example, say I have the following function which expects an > integer that's a power of 2 and returns corresponding position of the > lone "on" bit... > > > single_cardinalty_to_pos(2#1) -> 1; > single_cardinalty_to_pos(2#10) -> 2; > single_cardinalty_to_pos(2#100) -> 3; > ... > single_cardinalty_to_pos(2#100...0) -> 128. > > I'm trying to gauge whether or not I've correctly understood the > resultant select_val BEAM assembly instruction w.r.t gen_select_val [1] > in beam_load.c (without even being at all certain that gen_select_val is > even relevant). > > {label,133}. > {test,is_integer,{f,132},[{x,0}]}. > {select_val,{x,0}, > {f,132}, > {list,[{integer,170141183460469231731687303715884105728}, > {f,134}, > .... > {integer,4}, > {f,139}, > {integer,2}, > {f,140}, > {integer,1}, > {f,140}]}}. > > > I could use some help to determine if the following assumptions are > correct... > > A) That rather than brute forcing, the emulator will perform a > binary search [2] for the function argument against all the literals > from all the clauses which is the purpose of quick-sorting the > select_val list, and as such, > > B) If I want to increase the upper limit that the function can > handle, I can simply add more clauses rather than trying to implement a > binary search for the "on" bit myself using various masks and bit shifts. > > C) That the emulator will be smart about whether on not the function > argument is an 8 bit, 32/64 bit integer versus a bignum integer and thus > know what candidates can be excluded from the binary search so there's > no benefit to splitting the function to handle different ranges. > > D) That it's not unreasonable in this case to have even say, 1024 > clauses in order to handle bignum integers of up to 1024 bits. The only > downside will be longer compile times and possibly longer loading of the > module. Provided I don't go overboard with the size of the bignums due to the potential impact on runtime responsiveness (in addition to pattern matches, I believe the bignum arith operations are not broken-up for rescheduling?) > If these assumptions are correct, then I could have the emulator do a > lot of work for me with a number of similar functions and avoid writing > a lot code. > > Thanks in advance. > > - Edmond - > > [1] > https://github.com/erlang/otp/blob/48e1cc6a6f7bebacd5fb060bfd65ececbabfa6a1/erts/emulator/beam/beam_load.c#L4097 > > [2] > https://github.com/erlang/otp/blob/48e1cc6a6f7bebacd5fb060bfd65ececbabfa6a1/erts/emulator/beam/beam_load.c#L4159 > > -- > Using Opera's mail client: http://www.opera.com/mail/ -- Using Opera's mail client: http://www.opera.com/mail/ From bjorn@REDACTED Mon Jun 17 07:45:24 2019 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 17 Jun 2019 07:45:24 +0200 Subject: [erlang-questions] Pattern matching of literal small/big/bignum integers and binary search In-Reply-To: References: Message-ID: On Sun, Jun 16, 2019 at 10:28 AM Edmond Begumisa wrote: > > Hello all, > > I have some questions about how pattern matches on integer literals are > evaluated by the emulator. > > As an example, say I have the following function which expects an integer > that's a power of 2 and returns corresponding position of the lone "on" > bit... > > > single_cardinalty_to_pos(2#1) -> 1; > single_cardinalty_to_pos(2#10) -> 2; > single_cardinalty_to_pos(2#100) -> 3; > ... > single_cardinalty_to_pos(2#100...0) -> 128. The easiest way to determine what the loader will do with the code is to disassemble the loaded code by calling erts_debug:df/1. If your module is called 'foo', erts_debug:df('foo') will disassemble the loaded coded for the module into the file foo.dis. In the case of your example, the loader will split up the select_val instruction into multiple instructions. There will be a select_val_bins instruction that will do a binary search of all integers that fit into one word. If that instruction fails, there will be one i_is_ne_exact_literal instruction for comparing each of the bignum values. Here are the three last instructions from your example (matching 2^126, 2^127, 2^128): 0000000018E150B8: i_is_ne_exact_literal_fxc f(0000000018E15160) x(0) 85070591730234615865843651857942052864 0000000018E150D0: i_is_ne_exact_literal_fxc f(0000000018E15150) x(0) 42535295865117307932921825928971026432 0000000018E150E8: i_is_ne_exact_literal_fxc f(0000000018E15140) x(0) 21267647932558653966460912964485513216 . . . 0000000018E15160: move_return_c 126 0000000018E15170: move_return_c 127 0000000018E15180: move_return_c 128 /Bj?rn P.S. In OTP 22 documentation, there is internal documentation for the beam_makeops script and the loader: http://erlang.org/doc/apps/erts/beam_makeops.html -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bjorn@REDACTED Mon Jun 17 07:47:04 2019 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 17 Jun 2019 07:47:04 +0200 Subject: [erlang-questions] Pattern matching of literal small/big/bignum integers and binary search In-Reply-To: References: Message-ID: On Sun, Jun 16, 2019 at 10:39 AM Edmond Begumisa wrote: > Provided I don't go overboard with the size of the bignums due to the > potential impact on runtime responsiveness (in addition to pattern > matches, I believe the bignum arith operations are not broken-up for > rescheduling?) > Correct. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From otp@REDACTED Mon Jun 17 10:06:58 2019 From: otp@REDACTED (Erlang/OTP) Date: Mon, 17 Jun 2019 10:06:58 +0200 Subject: [erlang-questions] Patch Package OTP 20.3.8.22 Released Message-ID: <20190617080658.GA39480@erix.ericsson.se> Patch Package: OTP 20.3.8.22 Git Tag: OTP-20.3.8.22 Date: 2019-06-17 Trouble Report Id: OTP-15813, OTP-15819, OTP-15863, OTP-15869 Seq num: ERIERL-350, ERIERL-370, ERL-943, ERL-944 System: OTP Release: 20 Application: common_test-1.15.4.3, erts-9.3.3.11, tools-2.11.2.2 Predecessor: OTP 20.3.8.21 Check out the git tag OTP-20.3.8.22, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- common_test-1.15.4.3 -------------------------------------------- --------------------------------------------------------------------- The common_test-1.15.4.3 application can be applied independently of other applications on a full OTP 20 installation. --- Fixed Bugs and Malfunctions --- OTP-15863 Application(s): common_test Related Id(s): ERIERL-370 If a ct hook is installed in the suite/0 function in a test suite, then the hook's terminate/1 function would be called several times without it's init/2 function being called first. This is now corrected. OTP-15869 Application(s): common_test Related Id(s): ERIERL-350 If init_per_testcase fails, the test itself is skipped. According to the documentation, it should be possible to change the result to failed in a hook function. The only available hook function in this case is post_init_per_testcase, but changing the return value there did not affect the test case result. This is now corrected. Full runtime dependencies of common_test-1.15.4.3: compiler-6.0, crypto-3.6, debugger-4.1, erts-7.0, inets-6.0, kernel-4.0, observer-2.1, runtime_tools-1.8.16, sasl-2.4.2, snmp-5.1.2, ssh-4.0, stdlib-3.4, syntax_tools-1.7, tools-2.8, xmerl-1.3.8 --------------------------------------------------------------------- --- erts-9.3.3.11 --------------------------------------------------- --------------------------------------------------------------------- The erts-9.3.3.11 application can be applied independently of other applications on a full OTP 20 installation. --- Fixed Bugs and Malfunctions --- OTP-15819 Application(s): erts Related Id(s): ERL-944 Fixed a buffer overflow when binary_to_existing_atom/2 and list_to_existing_atom/2 was used with the latin1 encoding. Full runtime dependencies of erts-9.3.3.11: kernel-5.0, sasl-3.0.1, stdlib-3.0 --------------------------------------------------------------------- --- tools-2.11.2.2 -------------------------------------------------- --------------------------------------------------------------------- Note! The tools-2.11.2.2 application *cannot* be applied independently of other applications on an arbitrary OTP 20 installation. On a full OTP 20 installation, also the following runtime dependencies have to be satisfied: -- erts-9.1 (first satisfied in OTP 20.1) -- kernel-5.4 (first satisfied in OTP 20.1) --- Fixed Bugs and Malfunctions --- OTP-15813 Application(s): tools Related Id(s): ERL-943 cover would fail to start if two processes tried to start it at the exact same time. Full runtime dependencies of tools-2.11.2.2: compiler-5.0, erts-9.1, kernel-5.4, runtime_tools-1.8.14, stdlib-3.4 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From ebegumisa@REDACTED Mon Jun 17 13:49:57 2019 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Mon, 17 Jun 2019 21:49:57 +1000 Subject: [erlang-questions] Pattern matching of literal small/big/bignum integers and binary search In-Reply-To: References: Message-ID: Thank you Bj?rn, This answers all my questions precisely, especially revealing how bignum literals are treated differently w.r.t binary search. And erts_debug:df/1 is pure gold. Far more useful than erlc -S. Even made it easy to locate the relevant instruction implementation in beam_emu.c Much appreciated. - Edmond - On Mon, 17 Jun 2019 15:45:24 +1000, Bj?rn Gustavsson wrote: > On Sun, Jun 16, 2019 at 10:28 AM Edmond Begumisa > wrote: >> >> Hello all, >> >> I have some questions about how pattern matches on integer literals are >> evaluated by the emulator. >> >> As an example, say I have the following function which expects an >> integer >> that's a power of 2 and returns corresponding position of the lone "on" >> bit... >> >> >> single_cardinalty_to_pos(2#1) -> 1; >> single_cardinalty_to_pos(2#10) -> 2; >> single_cardinalty_to_pos(2#100) -> 3; >> ... >> single_cardinalty_to_pos(2#100...0) -> 128. > > The easiest way to determine what the loader will do with the code is > to disassemble the loaded code by calling erts_debug:df/1. If your > module is called 'foo', erts_debug:df('foo') will disassemble the > loaded coded for the module into the file foo.dis. > > In the case of your example, the loader will split up the select_val > instruction into multiple instructions. There will be a > select_val_bins instruction that will do a binary search of all > integers that fit into one word. If that instruction fails, there will > be one i_is_ne_exact_literal instruction for comparing each of the > bignum values. Here are the three last instructions from your example > (matching 2^126, 2^127, 2^128): > > 0000000018E150B8: i_is_ne_exact_literal_fxc f(0000000018E15160) x(0) > 85070591730234615865843651857942052864 > 0000000018E150D0: i_is_ne_exact_literal_fxc f(0000000018E15150) x(0) > 42535295865117307932921825928971026432 > 0000000018E150E8: i_is_ne_exact_literal_fxc f(0000000018E15140) x(0) > 21267647932558653966460912964485513216 > . > . > . > 0000000018E15160: move_return_c 126 > 0000000018E15170: move_return_c 127 > 0000000018E15180: move_return_c 128 > > /Bj?rn > > P.S. In OTP 22 documentation, there is internal documentation for the > beam_makeops script and the loader: > > http://erlang.org/doc/apps/erts/beam_makeops.html > -- Using Opera's mail client: http://www.opera.com/mail/ From gerhard@REDACTED Mon Jun 17 16:48:45 2019 From: gerhard@REDACTED (Gerhard Lazu) Date: Mon, 17 Jun 2019 15:48:45 +0100 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? Message-ID: Hi, We are trying to understand what prevents the Erlang distribution link from saturating the network. Even though there is plenty of CPU, memory & network bandwidth, the Erlang distribution doesn't fully utilise available resources. Can you help us figure out why? We have a 3-node Erlang 22.0.2 cluster running on Ubuntu 16.04 x86 64bit. This is the maximum network throughput between node-a & node-b, as measured by iperf: iperf -t 30 -c node-b ------------------------------------------------------------ Client connecting to 10.0.1.37, TCP port 5001 TCP window size: 45.0 KByte (default) ------------------------------------------------------------ [ 3] local 10.0.1.36 port 43576 connected with 10.0.1.37 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-30.0 sec 78.8 GBytes 22.6 Gbits/sec We ran this multiple times, in different directions & with different degree of parallelism, the maximum network throughput is roughly 22 Gbit/s. We run the following command on node-a: B = fun F() -> rpc:cast('foo@REDACTED', erlang, is_binary, [<<0:10000000/unit:8>>]), F() end. [spawn(fun() -> B() end) || _ <- lists:seq(1, 100)]. This is what the network reports on node-a: dstat -n 1 10 -net/total- recv send 0 0 676k 756M 643k 767M 584k 679M 693k 777M 648k 745M 660k 745M 667k 772M 651k 709M 675k 782M 688k 819M That roughly translates to 6 Gbit/s. In other words, the Erlang distribution link between node-a & node-b is maxing out at around ~6 Gbit/s. Erlang distribution is limited to 27% of what we are measuring consistently and repeatedly outside of Erlang. In other words, iperf is 3.6x faster than an Erlang distribution link. It gets better. If we start another 100 processes pumping 10Mbyte messages from node-a to node-c, we see the network throughput double: dstat -n 1 10 -net/total- recv send 0 0 1303k 1463M 1248k 1360M 1332k 1458M 1480k 1569M 1339k 1455M 1413k 1494M 1395k 1431M 1359k 1514M 1438k 1564M 1379k 1489M So 2 distribution links - each to a separate node - utilise 12Gbit/s out of the 22Gbit/s available on node-a. What is preventing the distribution links pushing more data through? There is plenty of CPU & memory available (all nodes have 16 CPUs & 104GB MEM - n1-highmem-16): dstat -cm 1 10 ----total-cpu-usage---- ------memory-usage----- usr sys idl wai hiq siq| used buff cach free 10 6 84 0 0 1|16.3G 118M 284M 85.6G 20 6 73 0 0 1|16.3G 118M 284M 85.6G 20 6 74 0 0 0|16.3G 118M 284M 85.6G 18 6 76 0 0 0|16.4G 118M 284M 85.5G 19 6 74 0 0 1|16.4G 118M 284M 85.4G 17 4 78 0 0 0|16.5G 118M 284M 85.4G 20 6 74 0 0 0|16.5G 118M 284M 85.4G 19 6 74 0 0 0|16.5G 118M 284M 85.4G 19 5 76 0 0 1|16.5G 118M 284M 85.4G 18 6 75 0 0 0|16.5G 118M 284M 85.4G 18 6 75 0 0 0|16.6G 118M 284M 85.3G The only smoking gun is the distribution output queue buffer: https://grafana.gcp.rabbitmq.com/dashboard/snapshot/H329EfN3SFhsveA20ei7jC7JMFHAm8Ru?orgId=1&fullscreen&panelId=62 Speaking of which, we look forward to erlang/otp#2270 being merged: https://github.com/erlang/otp/pull/2270 All distribution metrics are available here: https://grafana.gcp.rabbitmq.com/dashboard/snapshot/H329EfN3SFhsveA20ei7jC7JMFHAm8Ru?orgId=1 If you want to see the state of distribution links & dist process state (they are all green btw), check the point-in-time metrics (they will expire in 15 days from today): https://grafana.gcp.rabbitmq.com/d/d-SFCCmZz/erlang-distribution?from=1560775955127&to=1560779424482 How can we tell what is preventing the distribution link from using all available bandwidth? Are we missing a configuration flag? These are all the relevant beam.smp flags that we are using: https://github.com/erlang/otp/pull/2270#issuecomment-500953352 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmytro.lytovchenko@REDACTED Mon Jun 17 17:02:06 2019 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Mon, 17 Jun 2019 17:02:06 +0200 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? In-Reply-To: References: Message-ID: I believe the Erlang distribution is the wrong thing to use if you want to saturate the network. There is plenty of overhead for each incoming message, the data gets copied, then encoded (copied again) then sent, then received (copied), then decoded (copied again) and sent to the destination process (copied again). Then the receiving processes might be slow to fetch the incoming data, they aren't running in hard real time and sometimes go to sleep. Something about Linux tuning can be googled, like thing here https://medium.com/@_wmconsulting/tuning-linux-to-reach-maximum-performance-on-10-gbps-network-card-with-http-streaming-8599c9b4389d I remember there were suggestions to use regular TCP connections, consider using user-mode driver (kernel calls have a cost) and low level NIF driver for that, with the intent of delivering highest gigabits from your hardware. On Mon, 17 Jun 2019 at 16:49, Gerhard Lazu wrote: > Hi, > > We are trying to understand what prevents the Erlang distribution link > from saturating the network. Even though there is plenty of CPU, memory & > network bandwidth, the Erlang distribution doesn't fully utilise available > resources. Can you help us figure out why? > > We have a 3-node Erlang 22.0.2 cluster running on Ubuntu 16.04 x86 64bit. > > This is the maximum network throughput between node-a & node-b, as > measured by iperf: > > iperf -t 30 -c node-b > ------------------------------------------------------------ > Client connecting to 10.0.1.37, TCP port 5001 > TCP window size: 45.0 KByte (default) > ------------------------------------------------------------ > [ 3] local 10.0.1.36 port 43576 connected with 10.0.1.37 port 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-30.0 sec 78.8 GBytes 22.6 Gbits/sec > > We ran this multiple times, in different directions & with different > degree of parallelism, the maximum network throughput is roughly 22 Gbit/s. > > We run the following command on node-a: > > B = fun F() -> rpc:cast('foo@REDACTED', erlang, is_binary, [<<0:10000000/unit:8>>]), F() end. > [spawn(fun() -> B() end) || _ <- lists:seq(1, 100)]. > > This is what the network reports on node-a: > > dstat -n 1 10 > -net/total- > recv send > 0 0 > 676k 756M > 643k 767M > 584k 679M > 693k 777M > 648k 745M > 660k 745M > 667k 772M > 651k 709M > 675k 782M > 688k 819M > > That roughly translates to 6 Gbit/s. In other words, the Erlang > distribution link between node-a & node-b is maxing out at around ~6 > Gbit/s. Erlang distribution is limited to 27% of what we are measuring > consistently and repeatedly outside of Erlang. In other words, iperf is > 3.6x faster than an Erlang distribution link. It gets better. > > If we start another 100 processes pumping 10Mbyte messages from node-a to > node-c, we see the network throughput double: > > dstat -n 1 10 > -net/total- > recv send > 0 0 > 1303k 1463M > 1248k 1360M > 1332k 1458M > 1480k 1569M > 1339k 1455M > 1413k 1494M > 1395k 1431M > 1359k 1514M > 1438k 1564M > 1379k 1489M > > So 2 distribution links - each to a separate node - utilise 12Gbit/s out > of the 22Gbit/s available on node-a. > > What is preventing the distribution links pushing more data through? There > is plenty of CPU & memory available (all nodes have 16 CPUs & 104GB MEM - > n1-highmem-16): > > dstat -cm 1 10 > ----total-cpu-usage---- ------memory-usage----- > usr sys idl wai hiq siq| used buff cach free > 10 6 84 0 0 1|16.3G 118M 284M 85.6G > 20 6 73 0 0 1|16.3G 118M 284M 85.6G > 20 6 74 0 0 0|16.3G 118M 284M 85.6G > 18 6 76 0 0 0|16.4G 118M 284M 85.5G > 19 6 74 0 0 1|16.4G 118M 284M 85.4G > 17 4 78 0 0 0|16.5G 118M 284M 85.4G > 20 6 74 0 0 0|16.5G 118M 284M 85.4G > 19 6 74 0 0 0|16.5G 118M 284M 85.4G > 19 5 76 0 0 1|16.5G 118M 284M 85.4G > 18 6 75 0 0 0|16.5G 118M 284M 85.4G > 18 6 75 0 0 0|16.6G 118M 284M 85.3G > > The only smoking gun is the distribution output queue buffer: > https://grafana.gcp.rabbitmq.com/dashboard/snapshot/H329EfN3SFhsveA20ei7jC7JMFHAm8Ru?orgId=1&fullscreen&panelId=62 > > Speaking of which, we look forward to erlang/otp#2270 being merged: > https://github.com/erlang/otp/pull/2270 > > All distribution metrics are available here: > https://grafana.gcp.rabbitmq.com/dashboard/snapshot/H329EfN3SFhsveA20ei7jC7JMFHAm8Ru?orgId=1 > > If you want to see the state of distribution links & dist process state > (they are all green btw), check the point-in-time metrics (they will expire > in 15 days from today): > https://grafana.gcp.rabbitmq.com/d/d-SFCCmZz/erlang-distribution?from=1560775955127&to=1560779424482 > > How can we tell what is preventing the distribution link from using all > available bandwidth? > > Are we missing a configuration flag? These are all the relevant beam.smp > flags that we are using: > https://github.com/erlang/otp/pull/2270#issuecomment-500953352 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gerhard@REDACTED Mon Jun 17 17:07:41 2019 From: gerhard@REDACTED (Gerhard Lazu) Date: Mon, 17 Jun 2019 16:07:41 +0100 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? In-Reply-To: References: Message-ID: I wouldn't expect the Erlang distribution to reach the same network performance as iperf, but I would expect it to be within 70%-80% of maximum. In our measurements it's within 27% of maximum, which makes me believe that something is misconfigured or inefficient. The goal is to figure out which component/components are responsible for this significant network throughput loss. Thanks for the quick response! On Mon, Jun 17, 2019 at 4:02 PM Dmytro Lytovchenko < dmytro.lytovchenko@REDACTED> wrote: > I believe the Erlang distribution is the wrong thing to use if you want to > saturate the network. > There is plenty of overhead for each incoming message, the data gets > copied, then encoded (copied again) then sent, then received (copied), then > decoded (copied again) and sent to the destination process (copied again). > Then the receiving processes might be slow to fetch the incoming data, they > aren't running in hard real time and sometimes go to sleep. > > Something about Linux tuning can be googled, like thing here > https://medium.com/@_wmconsulting/tuning-linux-to-reach-maximum-performance-on-10-gbps-network-card-with-http-streaming-8599c9b4389d > > I remember there were suggestions to use regular TCP connections, consider > using user-mode driver (kernel calls have a cost) and low level NIF driver > for that, with the intent of delivering highest gigabits from your > hardware. > > On Mon, 17 Jun 2019 at 16:49, Gerhard Lazu wrote: > >> Hi, >> >> We are trying to understand what prevents the Erlang distribution link >> from saturating the network. Even though there is plenty of CPU, memory & >> network bandwidth, the Erlang distribution doesn't fully utilise available >> resources. Can you help us figure out why? >> >> We have a 3-node Erlang 22.0.2 cluster running on Ubuntu 16.04 x86 64bit. >> >> This is the maximum network throughput between node-a & node-b, as >> measured by iperf: >> >> iperf -t 30 -c node-b >> ------------------------------------------------------------ >> Client connecting to 10.0.1.37, TCP port 5001 >> TCP window size: 45.0 KByte (default) >> ------------------------------------------------------------ >> [ 3] local 10.0.1.36 port 43576 connected with 10.0.1.37 port 5001 >> [ ID] Interval Transfer Bandwidth >> [ 3] 0.0-30.0 sec 78.8 GBytes 22.6 Gbits/sec >> >> We ran this multiple times, in different directions & with different >> degree of parallelism, the maximum network throughput is roughly 22 Gbit/s. >> >> We run the following command on node-a: >> >> B = fun F() -> rpc:cast('foo@REDACTED', erlang, is_binary, [<<0:10000000/unit:8>>]), F() end. >> [spawn(fun() -> B() end) || _ <- lists:seq(1, 100)]. >> >> This is what the network reports on node-a: >> >> dstat -n 1 10 >> -net/total- >> recv send >> 0 0 >> 676k 756M >> 643k 767M >> 584k 679M >> 693k 777M >> 648k 745M >> 660k 745M >> 667k 772M >> 651k 709M >> 675k 782M >> 688k 819M >> >> That roughly translates to 6 Gbit/s. In other words, the Erlang >> distribution link between node-a & node-b is maxing out at around ~6 >> Gbit/s. Erlang distribution is limited to 27% of what we are measuring >> consistently and repeatedly outside of Erlang. In other words, iperf is >> 3.6x faster than an Erlang distribution link. It gets better. >> >> If we start another 100 processes pumping 10Mbyte messages from node-a to >> node-c, we see the network throughput double: >> >> dstat -n 1 10 >> -net/total- >> recv send >> 0 0 >> 1303k 1463M >> 1248k 1360M >> 1332k 1458M >> 1480k 1569M >> 1339k 1455M >> 1413k 1494M >> 1395k 1431M >> 1359k 1514M >> 1438k 1564M >> 1379k 1489M >> >> So 2 distribution links - each to a separate node - utilise 12Gbit/s out >> of the 22Gbit/s available on node-a. >> >> What is preventing the distribution links pushing more data through? >> There is plenty of CPU & memory available (all nodes have 16 CPUs & 104GB >> MEM - n1-highmem-16): >> >> dstat -cm 1 10 >> ----total-cpu-usage---- ------memory-usage----- >> usr sys idl wai hiq siq| used buff cach free >> 10 6 84 0 0 1|16.3G 118M 284M 85.6G >> 20 6 73 0 0 1|16.3G 118M 284M 85.6G >> 20 6 74 0 0 0|16.3G 118M 284M 85.6G >> 18 6 76 0 0 0|16.4G 118M 284M 85.5G >> 19 6 74 0 0 1|16.4G 118M 284M 85.4G >> 17 4 78 0 0 0|16.5G 118M 284M 85.4G >> 20 6 74 0 0 0|16.5G 118M 284M 85.4G >> 19 6 74 0 0 0|16.5G 118M 284M 85.4G >> 19 5 76 0 0 1|16.5G 118M 284M 85.4G >> 18 6 75 0 0 0|16.5G 118M 284M 85.4G >> 18 6 75 0 0 0|16.6G 118M 284M 85.3G >> >> The only smoking gun is the distribution output queue buffer: >> https://grafana.gcp.rabbitmq.com/dashboard/snapshot/H329EfN3SFhsveA20ei7jC7JMFHAm8Ru?orgId=1&fullscreen&panelId=62 >> >> Speaking of which, we look forward to erlang/otp#2270 being merged: >> https://github.com/erlang/otp/pull/2270 >> >> All distribution metrics are available here: >> https://grafana.gcp.rabbitmq.com/dashboard/snapshot/H329EfN3SFhsveA20ei7jC7JMFHAm8Ru?orgId=1 >> >> If you want to see the state of distribution links & dist process state >> (they are all green btw), check the point-in-time metrics (they will expire >> in 15 days from today): >> https://grafana.gcp.rabbitmq.com/d/d-SFCCmZz/erlang-distribution?from=1560775955127&to=1560779424482 >> >> How can we tell what is preventing the distribution link from using all >> available bandwidth? >> >> Are we missing a configuration flag? These are all the relevant beam.smp >> flags that we are using: >> https://github.com/erlang/otp/pull/2270#issuecomment-500953352 >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmytro.lytovchenko@REDACTED Mon Jun 17 17:10:55 2019 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Mon, 17 Jun 2019 17:10:55 +0200 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? In-Reply-To: References: Message-ID: Consider that the VM copies the data at least 4 or 5 times, and compare that with Gbit/s of your RAM on both servers too. Plus eventual garbage collection, which can be minimal if your VM memory footprint is small and you're only doing this perf testing. On Mon, 17 Jun 2019 at 17:08, Gerhard Lazu wrote: > I wouldn't expect the Erlang distribution to reach the same network > performance as iperf, but I would expect it to be within 70%-80% of maximum. > > In our measurements it's within 27% of maximum, which makes me believe > that something is misconfigured or inefficient. > > The goal is to figure out which component/components are responsible for > this significant network throughput loss. > > Thanks for the quick response! > > On Mon, Jun 17, 2019 at 4:02 PM Dmytro Lytovchenko < > dmytro.lytovchenko@REDACTED> wrote: > >> I believe the Erlang distribution is the wrong thing to use if you want >> to saturate the network. >> There is plenty of overhead for each incoming message, the data gets >> copied, then encoded (copied again) then sent, then received (copied), then >> decoded (copied again) and sent to the destination process (copied again). >> Then the receiving processes might be slow to fetch the incoming data, they >> aren't running in hard real time and sometimes go to sleep. >> >> Something about Linux tuning can be googled, like thing here >> https://medium.com/@_wmconsulting/tuning-linux-to-reach-maximum-performance-on-10-gbps-network-card-with-http-streaming-8599c9b4389d >> >> I remember there were suggestions to use regular TCP connections, >> consider using user-mode driver (kernel calls have a cost) and low level >> NIF driver for that, with the intent of delivering highest gigabits from >> your hardware. >> >> On Mon, 17 Jun 2019 at 16:49, Gerhard Lazu wrote: >> >>> Hi, >>> >>> We are trying to understand what prevents the Erlang distribution link >>> from saturating the network. Even though there is plenty of CPU, memory & >>> network bandwidth, the Erlang distribution doesn't fully utilise available >>> resources. Can you help us figure out why? >>> >>> We have a 3-node Erlang 22.0.2 cluster running on Ubuntu 16.04 x86 64bit. >>> >>> This is the maximum network throughput between node-a & node-b, as >>> measured by iperf: >>> >>> iperf -t 30 -c node-b >>> ------------------------------------------------------------ >>> Client connecting to 10.0.1.37, TCP port 5001 >>> TCP window size: 45.0 KByte (default) >>> ------------------------------------------------------------ >>> [ 3] local 10.0.1.36 port 43576 connected with 10.0.1.37 port 5001 >>> [ ID] Interval Transfer Bandwidth >>> [ 3] 0.0-30.0 sec 78.8 GBytes 22.6 Gbits/sec >>> >>> We ran this multiple times, in different directions & with different >>> degree of parallelism, the maximum network throughput is roughly 22 Gbit/s. >>> >>> We run the following command on node-a: >>> >>> B = fun F() -> rpc:cast('foo@REDACTED', erlang, is_binary, [<<0:10000000/unit:8>>]), F() end. >>> [spawn(fun() -> B() end) || _ <- lists:seq(1, 100)]. >>> >>> This is what the network reports on node-a: >>> >>> dstat -n 1 10 >>> -net/total- >>> recv send >>> 0 0 >>> 676k 756M >>> 643k 767M >>> 584k 679M >>> 693k 777M >>> 648k 745M >>> 660k 745M >>> 667k 772M >>> 651k 709M >>> 675k 782M >>> 688k 819M >>> >>> That roughly translates to 6 Gbit/s. In other words, the Erlang >>> distribution link between node-a & node-b is maxing out at around ~6 >>> Gbit/s. Erlang distribution is limited to 27% of what we are measuring >>> consistently and repeatedly outside of Erlang. In other words, iperf is >>> 3.6x faster than an Erlang distribution link. It gets better. >>> >>> If we start another 100 processes pumping 10Mbyte messages from node-a >>> to node-c, we see the network throughput double: >>> >>> dstat -n 1 10 >>> -net/total- >>> recv send >>> 0 0 >>> 1303k 1463M >>> 1248k 1360M >>> 1332k 1458M >>> 1480k 1569M >>> 1339k 1455M >>> 1413k 1494M >>> 1395k 1431M >>> 1359k 1514M >>> 1438k 1564M >>> 1379k 1489M >>> >>> So 2 distribution links - each to a separate node - utilise 12Gbit/s out >>> of the 22Gbit/s available on node-a. >>> >>> What is preventing the distribution links pushing more data through? >>> There is plenty of CPU & memory available (all nodes have 16 CPUs & 104GB >>> MEM - n1-highmem-16): >>> >>> dstat -cm 1 10 >>> ----total-cpu-usage---- ------memory-usage----- >>> usr sys idl wai hiq siq| used buff cach free >>> 10 6 84 0 0 1|16.3G 118M 284M 85.6G >>> 20 6 73 0 0 1|16.3G 118M 284M 85.6G >>> 20 6 74 0 0 0|16.3G 118M 284M 85.6G >>> 18 6 76 0 0 0|16.4G 118M 284M 85.5G >>> 19 6 74 0 0 1|16.4G 118M 284M 85.4G >>> 17 4 78 0 0 0|16.5G 118M 284M 85.4G >>> 20 6 74 0 0 0|16.5G 118M 284M 85.4G >>> 19 6 74 0 0 0|16.5G 118M 284M 85.4G >>> 19 5 76 0 0 1|16.5G 118M 284M 85.4G >>> 18 6 75 0 0 0|16.5G 118M 284M 85.4G >>> 18 6 75 0 0 0|16.6G 118M 284M 85.3G >>> >>> The only smoking gun is the distribution output queue buffer: >>> https://grafana.gcp.rabbitmq.com/dashboard/snapshot/H329EfN3SFhsveA20ei7jC7JMFHAm8Ru?orgId=1&fullscreen&panelId=62 >>> >>> Speaking of which, we look forward to erlang/otp#2270 being merged: >>> https://github.com/erlang/otp/pull/2270 >>> >>> All distribution metrics are available here: >>> https://grafana.gcp.rabbitmq.com/dashboard/snapshot/H329EfN3SFhsveA20ei7jC7JMFHAm8Ru?orgId=1 >>> >>> If you want to see the state of distribution links & dist process state >>> (they are all green btw), check the point-in-time metrics (they will expire >>> in 15 days from today): >>> https://grafana.gcp.rabbitmq.com/d/d-SFCCmZz/erlang-distribution?from=1560775955127&to=1560779424482 >>> >>> How can we tell what is preventing the distribution link from using all >>> available bandwidth? >>> >>> Are we missing a configuration flag? These are all the relevant beam.smp >>> flags that we are using: >>> https://github.com/erlang/otp/pull/2270#issuecomment-500953352 >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From gerhard@REDACTED Mon Jun 17 17:21:09 2019 From: gerhard@REDACTED (Gerhard Lazu) Date: Mon, 17 Jun 2019 16:21:09 +0100 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? In-Reply-To: References: Message-ID: Are you saying that due to the overhead associated with sending data over the distribution we are hitting the distribution link limit? We have measured this to be 6 Gbit/s. Can others confirm? On Mon, Jun 17, 2019 at 4:11 PM Dmytro Lytovchenko < dmytro.lytovchenko@REDACTED> wrote: > Consider that the VM copies the data at least 4 or 5 times, and compare > that with Gbit/s of your RAM on both servers too. > Plus eventual garbage collection, which can be minimal if your VM memory > footprint is small and you're only doing this perf testing. > > On Mon, 17 Jun 2019 at 17:08, Gerhard Lazu wrote: > >> I wouldn't expect the Erlang distribution to reach the same network >> performance as iperf, but I would expect it to be within 70%-80% of maximum. >> >> In our measurements it's within 27% of maximum, which makes me believe >> that something is misconfigured or inefficient. >> >> The goal is to figure out which component/components are responsible for >> this significant network throughput loss. >> >> Thanks for the quick response! >> >> On Mon, Jun 17, 2019 at 4:02 PM Dmytro Lytovchenko < >> dmytro.lytovchenko@REDACTED> wrote: >> >>> I believe the Erlang distribution is the wrong thing to use if you want >>> to saturate the network. >>> There is plenty of overhead for each incoming message, the data gets >>> copied, then encoded (copied again) then sent, then received (copied), then >>> decoded (copied again) and sent to the destination process (copied again). >>> Then the receiving processes might be slow to fetch the incoming data, they >>> aren't running in hard real time and sometimes go to sleep. >>> >>> Something about Linux tuning can be googled, like thing here >>> https://medium.com/@_wmconsulting/tuning-linux-to-reach-maximum-performance-on-10-gbps-network-card-with-http-streaming-8599c9b4389d >>> >>> I remember there were suggestions to use regular TCP connections, >>> consider using user-mode driver (kernel calls have a cost) and low level >>> NIF driver for that, with the intent of delivering highest gigabits from >>> your hardware. >>> >>> On Mon, 17 Jun 2019 at 16:49, Gerhard Lazu wrote: >>> >>>> Hi, >>>> >>>> We are trying to understand what prevents the Erlang distribution link >>>> from saturating the network. Even though there is plenty of CPU, memory & >>>> network bandwidth, the Erlang distribution doesn't fully utilise available >>>> resources. Can you help us figure out why? >>>> >>>> We have a 3-node Erlang 22.0.2 cluster running on Ubuntu 16.04 x86 >>>> 64bit. >>>> >>>> This is the maximum network throughput between node-a & node-b, as >>>> measured by iperf: >>>> >>>> iperf -t 30 -c node-b >>>> ------------------------------------------------------------ >>>> Client connecting to 10.0.1.37, TCP port 5001 >>>> TCP window size: 45.0 KByte (default) >>>> ------------------------------------------------------------ >>>> [ 3] local 10.0.1.36 port 43576 connected with 10.0.1.37 port 5001 >>>> [ ID] Interval Transfer Bandwidth >>>> [ 3] 0.0-30.0 sec 78.8 GBytes 22.6 Gbits/sec >>>> >>>> We ran this multiple times, in different directions & with different >>>> degree of parallelism, the maximum network throughput is roughly 22 Gbit/s. >>>> >>>> We run the following command on node-a: >>>> >>>> B = fun F() -> rpc:cast('foo@REDACTED', erlang, is_binary, [<<0:10000000/unit:8>>]), F() end. >>>> [spawn(fun() -> B() end) || _ <- lists:seq(1, 100)]. >>>> >>>> This is what the network reports on node-a: >>>> >>>> dstat -n 1 10 >>>> -net/total- >>>> recv send >>>> 0 0 >>>> 676k 756M >>>> 643k 767M >>>> 584k 679M >>>> 693k 777M >>>> 648k 745M >>>> 660k 745M >>>> 667k 772M >>>> 651k 709M >>>> 675k 782M >>>> 688k 819M >>>> >>>> That roughly translates to 6 Gbit/s. In other words, the Erlang >>>> distribution link between node-a & node-b is maxing out at around ~6 >>>> Gbit/s. Erlang distribution is limited to 27% of what we are measuring >>>> consistently and repeatedly outside of Erlang. In other words, iperf is >>>> 3.6x faster than an Erlang distribution link. It gets better. >>>> >>>> If we start another 100 processes pumping 10Mbyte messages from node-a >>>> to node-c, we see the network throughput double: >>>> >>>> dstat -n 1 10 >>>> -net/total- >>>> recv send >>>> 0 0 >>>> 1303k 1463M >>>> 1248k 1360M >>>> 1332k 1458M >>>> 1480k 1569M >>>> 1339k 1455M >>>> 1413k 1494M >>>> 1395k 1431M >>>> 1359k 1514M >>>> 1438k 1564M >>>> 1379k 1489M >>>> >>>> So 2 distribution links - each to a separate node - utilise 12Gbit/s >>>> out of the 22Gbit/s available on node-a. >>>> >>>> What is preventing the distribution links pushing more data through? >>>> There is plenty of CPU & memory available (all nodes have 16 CPUs & 104GB >>>> MEM - n1-highmem-16): >>>> >>>> dstat -cm 1 10 >>>> ----total-cpu-usage---- ------memory-usage----- >>>> usr sys idl wai hiq siq| used buff cach free >>>> 10 6 84 0 0 1|16.3G 118M 284M 85.6G >>>> 20 6 73 0 0 1|16.3G 118M 284M 85.6G >>>> 20 6 74 0 0 0|16.3G 118M 284M 85.6G >>>> 18 6 76 0 0 0|16.4G 118M 284M 85.5G >>>> 19 6 74 0 0 1|16.4G 118M 284M 85.4G >>>> 17 4 78 0 0 0|16.5G 118M 284M 85.4G >>>> 20 6 74 0 0 0|16.5G 118M 284M 85.4G >>>> 19 6 74 0 0 0|16.5G 118M 284M 85.4G >>>> 19 5 76 0 0 1|16.5G 118M 284M 85.4G >>>> 18 6 75 0 0 0|16.5G 118M 284M 85.4G >>>> 18 6 75 0 0 0|16.6G 118M 284M 85.3G >>>> >>>> The only smoking gun is the distribution output queue buffer: >>>> https://grafana.gcp.rabbitmq.com/dashboard/snapshot/H329EfN3SFhsveA20ei7jC7JMFHAm8Ru?orgId=1&fullscreen&panelId=62 >>>> >>>> Speaking of which, we look forward to erlang/otp#2270 being merged: >>>> https://github.com/erlang/otp/pull/2270 >>>> >>>> All distribution metrics are available here: >>>> https://grafana.gcp.rabbitmq.com/dashboard/snapshot/H329EfN3SFhsveA20ei7jC7JMFHAm8Ru?orgId=1 >>>> >>>> If you want to see the state of distribution links & dist process state >>>> (they are all green btw), check the point-in-time metrics (they will expire >>>> in 15 days from today): >>>> https://grafana.gcp.rabbitmq.com/d/d-SFCCmZz/erlang-distribution?from=1560775955127&to=1560779424482 >>>> >>>> How can we tell what is preventing the distribution link from using all >>>> available bandwidth? >>>> >>>> Are we missing a configuration flag? These are all the relevant >>>> beam.smp flags that we are using: >>>> https://github.com/erlang/otp/pull/2270#issuecomment-500953352 >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From okaprinarjaya@REDACTED Mon Jun 17 18:06:23 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Mon, 17 Jun 2019 23:06:23 +0700 Subject: [erlang-questions] Why this code not being infinite recursive? In-Reply-To: References: Message-ID: Hi, OMG. Finally..... I know and understand how this code flowing. It takes days for me to understand. It turns out this code below: {ok, AcceptorSocket} = gen_tcp:accept(ListenSocket), is locking / blocking the execution of these lines: 1. spawn(fun() -> acceptor(ListenSocket) end), 2. handle(AcceptorSocket). Then, after we doing a connection to the port via telnet, the locking / blocking become released / unlocked / unblocked, then executing lines: 1 and create new process, lines: 2 listen / waiting the incoming message. Then make a new locking at the new created process for the next telnet connection. Yes this is an infinity recursive. The base case is killing main process that started from start_server/1 . exit(Pid, killed) Thank you :) Pada tanggal Jum, 14 Jun 2019 pukul 11.45 I Gusti Ngurah Oka Prinarjaya < okaprinarjaya@REDACTED> menulis: > Hi, > > I learn gen_tcp from learnyousomeerlangg book. I try to understand flow of > code below. > And i got confused with acceptor/1 . Because acceptor/1 call itself but > have no base case and just execute once. It's not usual. Why? > From my understanding it should be an infinity recursive. > > -module(naive_tcp). > -compile(export_all). > > start_server(Port) -> > Pid = spawn_link( > fun() -> > io:format("Spawned at start_server()~n"), > {ok, ListenSocket} = gen_tcp:listen(Port, [binary, {active, false}]), > spawn(fun() -> acceptor(ListenSocket) end), > timer:sleep(infinity) > end > ), > {ok, Pid}. > > acceptor(ListenSocket) -> > io:format("I am acceptor~n"), > {ok, AcceptorSocket} = gen_tcp:accept(ListenSocket), > spawn(fun() -> acceptor(ListenSocket) end), > handle(AcceptorSocket). > > handle(AcceptorSocket) -> > inet:setopts(AcceptorSocket, [{active, once}]), > receive > {tcp, AcceptorSocket, <<"quit", _/binary>>} -> > gen_tcp:close(AcceptorSocket); > {tcp, AcceptorSocket, Message} -> > gen_tcp:send(AcceptorSocket, Message), > handle(AcceptorSocket) > end. > > Please enlightenment > > Thank you > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From the.artiegold@REDACTED Mon Jun 17 18:45:24 2019 From: the.artiegold@REDACTED (Artie Gold) Date: Mon, 17 Jun 2019 11:45:24 -0500 Subject: [erlang-questions] Why this code not being infinite recursive? In-Reply-To: References: Message-ID: But, wait! Infinite (well, unbounded) recursion is a bad thing, right? At the very least it will blow up the stack, right? Well, yes. It would. Except for *tail recursion*. When the recursive step is at the end of the function (and there's no lingering state to maintain), the compiler can optimize the recursion into a simple loop. As a result you get the cleanliness of recursive solution without having to worry about the space needed by the calculation being unbounded. Cheers, --ag On Mon, Jun 17, 2019 at 11:06 AM I Gusti Ngurah Oka Prinarjaya < okaprinarjaya@REDACTED> wrote: > Hi, > > OMG. Finally..... I know and understand how this code flowing. It takes > days for me to understand. > It turns out this code below: > > {ok, AcceptorSocket} = gen_tcp:accept(ListenSocket), > > is locking / blocking the execution of these lines: > > 1. spawn(fun() -> acceptor(ListenSocket) end), > 2. handle(AcceptorSocket). > > Then, after we doing a connection to the port via telnet, the locking / > blocking become released / unlocked / unblocked, then executing lines: 1 > and create new process, > lines: 2 listen / waiting the incoming message. Then make a new locking at > the new created process for the next telnet connection. Yes this is an > infinity recursive. > The base case is killing main process that started from start_server/1 . exit(Pid, > killed) > > Thank you :) > > > > > Pada tanggal Jum, 14 Jun 2019 pukul 11.45 I Gusti Ngurah Oka Prinarjaya < > okaprinarjaya@REDACTED> menulis: > >> Hi, >> >> I learn gen_tcp from learnyousomeerlangg book. I try to understand flow >> of code below. >> And i got confused with acceptor/1 . Because acceptor/1 call itself but >> have no base case and just execute once. It's not usual. Why? >> From my understanding it should be an infinity recursive. >> >> -module(naive_tcp). >> -compile(export_all). >> >> start_server(Port) -> >> Pid = spawn_link( >> fun() -> >> io:format("Spawned at start_server()~n"), >> {ok, ListenSocket} = gen_tcp:listen(Port, [binary, {active, >> false}]), >> spawn(fun() -> acceptor(ListenSocket) end), >> timer:sleep(infinity) >> end >> ), >> {ok, Pid}. >> >> acceptor(ListenSocket) -> >> io:format("I am acceptor~n"), >> {ok, AcceptorSocket} = gen_tcp:accept(ListenSocket), >> spawn(fun() -> acceptor(ListenSocket) end), >> handle(AcceptorSocket). >> >> handle(AcceptorSocket) -> >> inet:setopts(AcceptorSocket, [{active, once}]), >> receive >> {tcp, AcceptorSocket, <<"quit", _/binary>>} -> >> gen_tcp:close(AcceptorSocket); >> {tcp, AcceptorSocket, Message} -> >> gen_tcp:send(AcceptorSocket, Message), >> handle(AcceptorSocket) >> end. >> >> Please enlightenment >> >> Thank you >> >> >> _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Artie Gold, Austin, Texas -- http://makeitsimpler.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Jun 17 18:53:43 2019 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 17 Jun 2019 19:53:43 +0300 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? In-Reply-To: References: Message-ID: > Consider that the VM copies the data at least 4 or 5 times This is not good. I'm designing multi-node hardware appliance for our Flussonic and thought that it would be a good idea to interconnect nodes via erlang distribution. Seems that it would be better to create 2 channels: control and data? On Mon, Jun 17, 2019 at 6:11 PM Dmytro Lytovchenko < dmytro.lytovchenko@REDACTED> wrote: > Consider that the VM copies the data at least 4 or 5 times, and compare > that with Gbit/s of your RAM on both servers too. > Plus eventual garbage collection, which can be minimal if your VM memory > footprint is small and you're only doing this perf testing. > > On Mon, 17 Jun 2019 at 17:08, Gerhard Lazu wrote: > >> I wouldn't expect the Erlang distribution to reach the same network >> performance as iperf, but I would expect it to be within 70%-80% of maximum. >> >> In our measurements it's within 27% of maximum, which makes me believe >> that something is misconfigured or inefficient. >> >> The goal is to figure out which component/components are responsible for >> this significant network throughput loss. >> >> Thanks for the quick response! >> >> On Mon, Jun 17, 2019 at 4:02 PM Dmytro Lytovchenko < >> dmytro.lytovchenko@REDACTED> wrote: >> >>> I believe the Erlang distribution is the wrong thing to use if you want >>> to saturate the network. >>> There is plenty of overhead for each incoming message, the data gets >>> copied, then encoded (copied again) then sent, then received (copied), then >>> decoded (copied again) and sent to the destination process (copied again). >>> Then the receiving processes might be slow to fetch the incoming data, they >>> aren't running in hard real time and sometimes go to sleep. >>> >>> Something about Linux tuning can be googled, like thing here >>> https://medium.com/@_wmconsulting/tuning-linux-to-reach-maximum-performance-on-10-gbps-network-card-with-http-streaming-8599c9b4389d >>> >>> I remember there were suggestions to use regular TCP connections, >>> consider using user-mode driver (kernel calls have a cost) and low level >>> NIF driver for that, with the intent of delivering highest gigabits from >>> your hardware. >>> >>> On Mon, 17 Jun 2019 at 16:49, Gerhard Lazu wrote: >>> >>>> Hi, >>>> >>>> We are trying to understand what prevents the Erlang distribution link >>>> from saturating the network. Even though there is plenty of CPU, memory & >>>> network bandwidth, the Erlang distribution doesn't fully utilise available >>>> resources. Can you help us figure out why? >>>> >>>> We have a 3-node Erlang 22.0.2 cluster running on Ubuntu 16.04 x86 >>>> 64bit. >>>> >>>> This is the maximum network throughput between node-a & node-b, as >>>> measured by iperf: >>>> >>>> iperf -t 30 -c node-b >>>> ------------------------------------------------------------ >>>> Client connecting to 10.0.1.37, TCP port 5001 >>>> TCP window size: 45.0 KByte (default) >>>> ------------------------------------------------------------ >>>> [ 3] local 10.0.1.36 port 43576 connected with 10.0.1.37 port 5001 >>>> [ ID] Interval Transfer Bandwidth >>>> [ 3] 0.0-30.0 sec 78.8 GBytes 22.6 Gbits/sec >>>> >>>> We ran this multiple times, in different directions & with different >>>> degree of parallelism, the maximum network throughput is roughly 22 Gbit/s. >>>> >>>> We run the following command on node-a: >>>> >>>> B = fun F() -> rpc:cast('foo@REDACTED', erlang, is_binary, [<<0:10000000/unit:8>>]), F() end. >>>> [spawn(fun() -> B() end) || _ <- lists:seq(1, 100)]. >>>> >>>> This is what the network reports on node-a: >>>> >>>> dstat -n 1 10 >>>> -net/total- >>>> recv send >>>> 0 0 >>>> 676k 756M >>>> 643k 767M >>>> 584k 679M >>>> 693k 777M >>>> 648k 745M >>>> 660k 745M >>>> 667k 772M >>>> 651k 709M >>>> 675k 782M >>>> 688k 819M >>>> >>>> That roughly translates to 6 Gbit/s. In other words, the Erlang >>>> distribution link between node-a & node-b is maxing out at around ~6 >>>> Gbit/s. Erlang distribution is limited to 27% of what we are measuring >>>> consistently and repeatedly outside of Erlang. In other words, iperf is >>>> 3.6x faster than an Erlang distribution link. It gets better. >>>> >>>> If we start another 100 processes pumping 10Mbyte messages from node-a >>>> to node-c, we see the network throughput double: >>>> >>>> dstat -n 1 10 >>>> -net/total- >>>> recv send >>>> 0 0 >>>> 1303k 1463M >>>> 1248k 1360M >>>> 1332k 1458M >>>> 1480k 1569M >>>> 1339k 1455M >>>> 1413k 1494M >>>> 1395k 1431M >>>> 1359k 1514M >>>> 1438k 1564M >>>> 1379k 1489M >>>> >>>> So 2 distribution links - each to a separate node - utilise 12Gbit/s >>>> out of the 22Gbit/s available on node-a. >>>> >>>> What is preventing the distribution links pushing more data through? >>>> There is plenty of CPU & memory available (all nodes have 16 CPUs & 104GB >>>> MEM - n1-highmem-16): >>>> >>>> dstat -cm 1 10 >>>> ----total-cpu-usage---- ------memory-usage----- >>>> usr sys idl wai hiq siq| used buff cach free >>>> 10 6 84 0 0 1|16.3G 118M 284M 85.6G >>>> 20 6 73 0 0 1|16.3G 118M 284M 85.6G >>>> 20 6 74 0 0 0|16.3G 118M 284M 85.6G >>>> 18 6 76 0 0 0|16.4G 118M 284M 85.5G >>>> 19 6 74 0 0 1|16.4G 118M 284M 85.4G >>>> 17 4 78 0 0 0|16.5G 118M 284M 85.4G >>>> 20 6 74 0 0 0|16.5G 118M 284M 85.4G >>>> 19 6 74 0 0 0|16.5G 118M 284M 85.4G >>>> 19 5 76 0 0 1|16.5G 118M 284M 85.4G >>>> 18 6 75 0 0 0|16.5G 118M 284M 85.4G >>>> 18 6 75 0 0 0|16.6G 118M 284M 85.3G >>>> >>>> The only smoking gun is the distribution output queue buffer: >>>> https://grafana.gcp.rabbitmq.com/dashboard/snapshot/H329EfN3SFhsveA20ei7jC7JMFHAm8Ru?orgId=1&fullscreen&panelId=62 >>>> >>>> Speaking of which, we look forward to erlang/otp#2270 being merged: >>>> https://github.com/erlang/otp/pull/2270 >>>> >>>> All distribution metrics are available here: >>>> https://grafana.gcp.rabbitmq.com/dashboard/snapshot/H329EfN3SFhsveA20ei7jC7JMFHAm8Ru?orgId=1 >>>> >>>> If you want to see the state of distribution links & dist process state >>>> (they are all green btw), check the point-in-time metrics (they will expire >>>> in 15 days from today): >>>> https://grafana.gcp.rabbitmq.com/d/d-SFCCmZz/erlang-distribution?from=1560775955127&to=1560779424482 >>>> >>>> How can we tell what is preventing the distribution link from using all >>>> available bandwidth? >>>> >>>> Are we missing a configuration flag? These are all the relevant >>>> beam.smp flags that we are using: >>>> https://github.com/erlang/otp/pull/2270#issuecomment-500953352 >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From okaprinarjaya@REDACTED Mon Jun 17 20:24:04 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Tue, 18 Jun 2019 01:24:04 +0700 Subject: [erlang-questions] Why this code not being infinite recursive? In-Reply-To: References: Message-ID: Hi Artie Gold, Thank you for the additional information about this. what i do is just for learning purpose. So many syntax variation that i haven't understand yet. Thank you Pada tanggal Sen, 17 Jun 2019 pukul 23.45 Artie Gold < the.artiegold@REDACTED> menulis: > But, wait! Infinite (well, unbounded) recursion is a bad thing, right? At > the very least it will blow up the stack, right? > > Well, yes. It would. Except for *tail recursion*. When the recursive step > is at the end of the function (and there's no lingering state to maintain), > the compiler can optimize the recursion into a simple loop. As a result you > get the cleanliness of recursive solution without having to worry about the > space needed by the calculation being unbounded. > > Cheers, > --ag > > On Mon, Jun 17, 2019 at 11:06 AM I Gusti Ngurah Oka Prinarjaya < > okaprinarjaya@REDACTED> wrote: > >> Hi, >> >> OMG. Finally..... I know and understand how this code flowing. It takes >> days for me to understand. >> It turns out this code below: >> >> {ok, AcceptorSocket} = gen_tcp:accept(ListenSocket), >> >> is locking / blocking the execution of these lines: >> >> 1. spawn(fun() -> acceptor(ListenSocket) end), >> 2. handle(AcceptorSocket). >> >> Then, after we doing a connection to the port via telnet, the locking / >> blocking become released / unlocked / unblocked, then executing lines: 1 >> and create new process, >> lines: 2 listen / waiting the incoming message. Then make a new locking >> at the new created process for the next telnet connection. Yes this is an >> infinity recursive. >> The base case is killing main process that started from start_server/1 . exit(Pid, >> killed) >> >> Thank you :) >> >> >> >> >> Pada tanggal Jum, 14 Jun 2019 pukul 11.45 I Gusti Ngurah Oka Prinarjaya < >> okaprinarjaya@REDACTED> menulis: >> >>> Hi, >>> >>> I learn gen_tcp from learnyousomeerlangg book. I try to understand flow >>> of code below. >>> And i got confused with acceptor/1 . Because acceptor/1 call itself but >>> have no base case and just execute once. It's not usual. Why? >>> From my understanding it should be an infinity recursive. >>> >>> -module(naive_tcp). >>> -compile(export_all). >>> >>> start_server(Port) -> >>> Pid = spawn_link( >>> fun() -> >>> io:format("Spawned at start_server()~n"), >>> {ok, ListenSocket} = gen_tcp:listen(Port, [binary, {active, >>> false}]), >>> spawn(fun() -> acceptor(ListenSocket) end), >>> timer:sleep(infinity) >>> end >>> ), >>> {ok, Pid}. >>> >>> acceptor(ListenSocket) -> >>> io:format("I am acceptor~n"), >>> {ok, AcceptorSocket} = gen_tcp:accept(ListenSocket), >>> spawn(fun() -> acceptor(ListenSocket) end), >>> handle(AcceptorSocket). >>> >>> handle(AcceptorSocket) -> >>> inet:setopts(AcceptorSocket, [{active, once}]), >>> receive >>> {tcp, AcceptorSocket, <<"quit", _/binary>>} -> >>> gen_tcp:close(AcceptorSocket); >>> {tcp, AcceptorSocket, Message} -> >>> gen_tcp:send(AcceptorSocket, Message), >>> handle(AcceptorSocket) >>> end. >>> >>> Please enlightenment >>> >>> Thank you >>> >>> >>> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > -- > Artie Gold, Austin, Texas > -- > http://makeitsimpler.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fritchie@REDACTED Tue Jun 18 00:01:00 2019 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Mon, 17 Jun 2019 17:01:00 -0500 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? In-Reply-To: References: Message-ID: I've not tried using Partisan for the kind of single sender -> single receiver large blob throughput that y'all are attempting, but Partisan might make better use of high speed local networks in your case. http://partisan.cloud -Scott (standing in for Chris Meiklejohn ^_^) -------------- next part -------------- An HTML attachment was scrubbed... URL: From otp@REDACTED Tue Jun 18 10:01:42 2019 From: otp@REDACTED (Erlang/OTP) Date: Tue, 18 Jun 2019 10:01:42 +0200 Subject: [erlang-questions] Patch Package OTP 21.3.8.4 Released Message-ID: <20190618080142.GA63931@erix.ericsson.se> Patch Package: OTP 21.3.8.4 Git Tag: OTP-21.3.8.4 Date: 2019-06-18 Trouble Report Id: OTP-15370, OTP-15747, OTP-15863, OTP-15865, OTP-15867, OTP-15869, OTP-15870, OTP-15875, OTP-15879 Seq num: ERIERL-294, ERIERL-350, ERIERL-353, ERIERL-370, ERIERL-373, ERL-952, ERL-964, ERL-967, ERL-968 System: OTP Release: 21 Application: common_test-1.17.2.1, erts-10.3.5.3, kernel-6.3.1.2, public_key-1.6.6.1, ssl-9.2.3.3, stdlib-3.8.2.2 Predecessor: OTP 21.3.8.3 Check out the git tag OTP-21.3.8.4, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- common_test-1.17.2.1 -------------------------------------------- --------------------------------------------------------------------- The common_test-1.17.2.1 application can be applied independently of other applications on a full OTP 21 installation. --- Fixed Bugs and Malfunctions --- OTP-15863 Application(s): common_test Related Id(s): ERIERL-370 If a ct hook is installed in the suite/0 function in a test suite, then the hook's terminate/1 function would be called several times without it's init/2 function being called first. This is now corrected. OTP-15869 Application(s): common_test Related Id(s): ERIERL-350 If init_per_testcase fails, the test itself is skipped. According to the documentation, it should be possible to change the result to failed in a hook function. The only available hook function in this case is post_init_per_testcase, but changing the return value there did not affect the test case result. This is now corrected. Full runtime dependencies of common_test-1.17.2.1: compiler-6.0, crypto-3.6, debugger-4.1, erts-7.0, ftp-1.0.0, inets-6.0, kernel-4.0, observer-2.1, runtime_tools-1.8.16, sasl-2.4.2, snmp-5.1.2, ssh-4.0, stdlib-3.5, syntax_tools-1.7, tools-2.8, xmerl-1.3.8 --------------------------------------------------------------------- --- erts-10.3.5.3 --------------------------------------------------- --------------------------------------------------------------------- Note! The erts-10.3.5.3 application *cannot* be applied independently of other applications on an arbitrary OTP 21 installation. On a full OTP 21 installation, also the following runtime dependencies have to be satisfied: -- kernel-6.1 (first satisfied in OTP 21.1) -- sasl-3.3 (first satisfied in OTP 21.2) --- Fixed Bugs and Malfunctions --- OTP-15370 Application(s): erts Related Id(s): ERIERL-353 If you set {linger,{true,0}} on a gen_tcp listen socket, accept a connection on that socket, and then close the accepted socket, now the linger zero setting is transferred to the accepted socket. Before this correction that information was lost and the close behaviour on the accepted socket incorrect. OTP-15865 Application(s): erts Related Id(s): ERL-964 Fixed process_info(Pid,reductions) to not categorically increase reduction count of the measured process Pid. Repeated reduction measure of an idle process will most often (but not guaranteed) return the same value, like it behaved before OTP 21.3.8. OTP-15867 Application(s): erts Related Id(s): ERIERL-373 The runtime system disconnected a connection if it received an exit/2 signal where the recipient was a process on an old incarnation of the current node. That is, the receiving node had the same node name, but another "creation" number. The signal will now just be dropped since the receiving process no longer exists. --- Improvements and New Features --- OTP-15747 Application(s): erts, kernel Related Id(s): ERIERL-294 The possibility to send ancillary data, in particular the TOS field, has been added to gen_udp:send/4,5. Full runtime dependencies of erts-10.3.5.3: kernel-6.1, sasl-3.3, stdlib-3.5 --------------------------------------------------------------------- --- kernel-6.3.1.2 -------------------------------------------------- --------------------------------------------------------------------- Note! The kernel-6.3.1.2 application *cannot* be applied independently of other applications on an arbitrary OTP 21 installation. On a full OTP 21 installation, also the following runtime dependency has to be satisfied: -- erts-10.2.5 (first satisfied in OTP 21.2.7) --- Improvements and New Features --- OTP-15747 Application(s): erts, kernel Related Id(s): ERIERL-294 The possibility to send ancillary data, in particular the TOS field, has been added to gen_udp:send/4,5. Full runtime dependencies of kernel-6.3.1.2: erts-10.2.5, sasl-3.0, stdlib-3.5 --------------------------------------------------------------------- --- public_key-1.6.6.1 ---------------------------------------------- --------------------------------------------------------------------- The public_key-1.6.6.1 application can be applied independently of other applications on a full OTP 21 installation. --- Fixed Bugs and Malfunctions --- OTP-15870 Application(s): public_key Related Id(s): ERL-952 Support Pasword based encryption with AES Full runtime dependencies of public_key-1.6.6.1: asn1-3.0, crypto-3.8, erts-6.0, kernel-3.0, stdlib-3.5 --------------------------------------------------------------------- --- ssl-9.2.3.3 ----------------------------------------------------- --------------------------------------------------------------------- The ssl-9.2.3.3 application can be applied independently of other applications on a full OTP 21 installation. --- Fixed Bugs and Malfunctions --- OTP-15879 Application(s): ssl Related Id(s): ERL-968 Correct handshake handling, might cause strange symptoms such as ASN.1 certificate decoding issues. Full runtime dependencies of ssl-9.2.3.3: crypto-4.2, erts-10.0, inets-5.10.7, kernel-6.0, public_key-1.5, stdlib-3.5 --------------------------------------------------------------------- --- stdlib-3.8.2.2 -------------------------------------------------- --------------------------------------------------------------------- The stdlib-3.8.2.2 application can be applied independently of other applications on a full OTP 21 installation. --- Fixed Bugs and Malfunctions --- OTP-15875 Application(s): stdlib Related Id(s): ERL-967 Fix a bug that could cause a loop when formatting terms using the control sequences p or P and limiting the output with the option chars_limit. Full runtime dependencies of stdlib-3.8.2.2: compiler-5.0, crypto-3.3, erts-10.0, kernel-6.0, sasl-3.0 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From sperber@REDACTED Tue Jun 18 09:44:35 2019 From: sperber@REDACTED (Michael Sperber) Date: Tue, 18 Jun 2019 09:44:35 +0200 Subject: [erlang-questions] Call for Participation: Summer BOB 2019 (August 21, Berlin) Message-ID: ================================================================================ Summer BOB 2019 Conference ?What happens if we simply use what?s best?? August 21, 2019, Berlin co-located with ICFP 2019 http://bobkonf.de/2019-summer/ Program: http://bobkonf.de/2019-summer/program.html Registration: http://bobkonf.de/2019-summer/registration.html ================================================================================ Are you interested in technologies beyond the mainstream, that are a pleasure to use, and effective at getting the job done? BOB is the forum for developers, architects and builders to explore and discover the best tools available today for building software. Our goal is for all participants to leave the conference with new ideas to improve development back at the ranch. Summer BOB is a one-time-only event, in the spirit of the spectacular Winter BOB. The International Conference on Functional Programming is coming to town, and Summer BOB will be right in the middle of it, on the last day of ICFP proper, prior to all the workshops. Summer BOB will feature two tracks: one from practitioners, and one from researchers, and foster communication and cross-pollination between these communities. BOB features two tracks of seven talk each: One research track with invited talks, and one track by practitioners, designed to cross-pollinate and inspire. http://bobkonf.de/2019-summer/program.html Topics include distributed programming, testing, linear algebra, functional design patterns, type systems, formal methods, and interactive development. We are committed to diversity: We aim at exploring a wide range of tools in a welcoming and friendly crowd of diverse people. To that end, a number of support options for participants from groups under-represented in tech are available. http://bobkonf.de/2019-summer/registration.html NOTE: The early-bird rates expire on July 18, 2019! From otp@REDACTED Tue Jun 18 10:20:10 2019 From: otp@REDACTED (Erlang/OTP) Date: Tue, 18 Jun 2019 10:20:10 +0200 Subject: [erlang-questions] Patch Package OTP 22.0.4 Released Message-ID: <20190618082010.GA2198@erix.ericsson.se> Patch Package: OTP 22.0.4 Git Tag: OTP-22.0.4 Date: 2019-06-18 Trouble Report Id: OTP-15805, OTP-15819, OTP-15867, OTP-15879, OTP-15887, OTP-15888 Seq num: ERIERL-373, ERL-944, ERL-968, ERL-973, ERL-975 System: OTP Release: 22 Application: erts-10.4.3, kernel-6.4.1, ssl-9.3.3 Predecessor: OTP 22.0.3 Check out the git tag OTP-22.0.4, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- erts-10.4.3 ----------------------------------------------------- --------------------------------------------------------------------- The erts-10.4.3 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-15819 Application(s): erts Related Id(s): ERL-944 Fixed a buffer overflow when binary_to_existing_atom/2 and list_to_existing_atom/2 was used with the latin1 encoding. OTP-15867 Application(s): erts Related Id(s): ERIERL-373 The runtime system disconnected a connection if it received an exit/2 signal where the recipient was a process on an old incarnation of the current node. That is, the receiving node had the same node name, but another "creation" number. The signal will now just be dropped since the receiving process no longer exists. Full runtime dependencies of erts-10.4.3: kernel-6.1, sasl-3.3, stdlib-3.5 --------------------------------------------------------------------- --- kernel-6.4.1 ---------------------------------------------------- --------------------------------------------------------------------- The kernel-6.4.1 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-15805 Application(s): kernel user/user_drv could respond to io requests before they had been processed, which could cause data to be dropped if the emulator was halted soon after a call to io:format/2, such as in an escript. Full runtime dependencies of kernel-6.4.1: erts-10.2.5, sasl-3.0, stdlib-3.5 --------------------------------------------------------------------- --- ssl-9.3.3 ------------------------------------------------------- --------------------------------------------------------------------- The ssl-9.3.3 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-15879 Application(s): ssl Related Id(s): ERL-968 Correct handshake handling, might cause strange symptoms such as ASN.1 certificate decoding issues. OTP-15887 Application(s): ssl Related Id(s): ERL-973 Fix handling of the signature_algorithms_cert extension in the ClientHello handshake message. OTP-15888 Application(s): ssl Related Id(s): ERL-975 Handle new ClientHello extensions when handshake is paused by the {handshake, hello} ssl option. Full runtime dependencies of ssl-9.3.3: crypto-4.2, erts-10.0, inets-5.10.7, kernel-6.0, public_key-1.5, stdlib-3.5 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From kjnilsson@REDACTED Tue Jun 18 10:42:43 2019 From: kjnilsson@REDACTED (Karl Nilsson) Date: Tue, 18 Jun 2019 09:42:43 +0100 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? In-Reply-To: References: Message-ID: I work with Gerhard (OP) and just thought I'd chime in. Thanks for the alternative suggestions of using sockets / partisan. At some point we will look at this but for now we are working on making the best use possible of vanilla distributed erlang links. We do understand there is overhead and inherent inefficiencies with using distributed erlang for data transfer but isn't it surprising / concerning anyone else that it is only able to make use of 27% of total network potential? Would you not prefer it to be, say, 50% or even 75% if this is theoretically possible? So what we'd like to find out is whether there are genuine unsurmountable reasons for this or something that can be improved, either by tweaking config or by optimisations in the distribution layer. We will run more tests to get more data to reason about. If anyone has suggestions for good tests to run / metrics to inspect that would be very welcome. Cheers Karl On Mon, 17 Jun 2019 at 23:01, Scott Lystig Fritchie wrote: > I've not tried using Partisan for the kind of single sender -> single > receiver large blob throughput that y'all are attempting, but Partisan > might make better use of high speed local networks in your case. > > http://partisan.cloud > > -Scott (standing in for Chris Meiklejohn ^_^) > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- *Karl Nilsson* -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmytro.lytovchenko@REDACTED Tue Jun 18 10:49:04 2019 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Tue, 18 Jun 2019 10:49:04 +0200 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? In-Reply-To: References: Message-ID: Have a look at your - Linux kernel TCP settings - Memory throughput Gbit/s (the data is copied several times for sending and receiving) - Head of the line problem (if receiver doesn't read fast enough, the memory will be used to store the unconsumed data) - Message size might make some difference, OTP 22 splits packets over 64k into segments, so they also have to be stored before assembling in the destination - Kernel CPU time vs User CPU time (network driver and kernel overhead) On Tue, 18 Jun 2019 at 10:43, Karl Nilsson wrote: > I work with Gerhard (OP) and just thought I'd chime in. > > Thanks for the alternative suggestions of using sockets / partisan. At > some point we will look at this but for now we are working on making the > best use possible of vanilla distributed erlang links. We do understand > there is overhead and inherent inefficiencies with using distributed erlang > for data transfer but isn't it surprising / concerning anyone else that it > is only able to make use of 27% of total network potential? Would you not > prefer it to be, say, 50% or even 75% if this is theoretically possible? > > So what we'd like to find out is whether there are genuine unsurmountable > reasons for this or something that can be improved, either by tweaking > config or by optimisations in the distribution layer. > > We will run more tests to get more data to reason about. If anyone has > suggestions for good tests to run / metrics to inspect that would be very > welcome. > > Cheers > Karl > > On Mon, 17 Jun 2019 at 23:01, Scott Lystig Fritchie > wrote: > >> I've not tried using Partisan for the kind of single sender -> single >> receiver large blob throughput that y'all are attempting, but Partisan >> might make better use of high speed local networks in your case. >> >> http://partisan.cloud >> >> -Scott (standing in for Chris Meiklejohn ^_^) >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > -- > *Karl Nilsson* > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Tue Jun 18 12:07:02 2019 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 18 Jun 2019 12:07:02 +0200 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? In-Reply-To: References: Message-ID: On Mon, Jun 17, 2019 at 4:49 PM Gerhard Lazu wrote: > > B = fun F() -> rpc:cast('foo@REDACTED', erlang, is_binary, [<<0:10000000/unit:8>>]), F() end. > [spawn(fun() -> B() end) || _ <- lists:seq(1, 100)]. > > I wrote this code that is better able to saturate the network. Using rpc will create bottlenecks on the receiving side as it is the receiving process that does the decode. I get about 20% more traffic through when I do that, which is not as much as I was expecting. -module(dist_perf). -export([go/0,go/1]). go() -> go(100). go(N) -> spawn_link(fun stats/0), RemoteNode = 'bar@REDACTED', Pids = [spawn(RemoteNode, fun F() -> receive {From, Msg} -> From ! is_binary(Msg), F() end end) || _<- lists:seq(1,N)], Payload = <<0:10000000/unit:8>>, B = fun F(Pid) -> Pid ! {self(),Payload}, receive true -> F(Pid) end end, [spawn(fun() -> B(Pid) end) || Pid <- Pids ]. stats() -> {{_input,T0Inp},{_output,T0Out}} = erlang:statistics(io), timer:sleep(5000), {{_input,T1Inp},{_output,T1Out}} = erlang:statistics(io), io:format("Sent ~pMB/s, Recv ~pMB/s~n", [(T1Out - T0Out) / 1024 / 1024 / 5, (T1Inp - T0Inp) / 1024 / 1024 / 5]), stats(). I fired up linux perf to have a look at what was taking time and it is (as Dmytro said) the copying of data that takes time. There is one specific places that you hit with your test very badly: https://github.com/erlang/otp/blob/master/erts/emulator/beam/dist.c#L1565 This is the re-assembly of fragmented distribution messages. This happens because the messages sent are > 64kb. I change that limit to be 64MB on my machine that that doubled the throughput ( https://github.com/erlang/otp/blob/master/erts/emulator/beam/dist.h#L150). The reason why the re-assembly has to be done is that the emulator does not have a variant of erlang:binary_to_term/1 that takes an iovector as input. This could definitely be fixed and is something that we are planning on fixing, though it is not something we are working on right now. However, it will only change things in production if you send > 64KB messages. Also, just to be clear, the copying that is done by the distribution layer is: 1) use term_to_binary to create a buffer 2) use writev in inet_driver to copy buffer to kernel 3) use recv to copy data from kernel into a refc binary 4) if needed, use memcpy re-assemble fragments 5) use binary_to_term to decode term (there is no copy to the destination process as Dmyto said) Before OTP-22 there was an additional copy at the receiving side, but that has been removed. So imo there is only one copy that could potentially be removed. If you want to decrease the overhead of saturating the network with many small messages I would implement something like read_packets for tcp in the inet_driver. However, the real bottleneck quickly becomes memory allocations as each message needs two or three allocations, and when you receive a lot of messages those allocations add up. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Jun 18 12:09:05 2019 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 18 Jun 2019 12:09:05 +0200 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? In-Reply-To: References: Message-ID: On Mon, Jun 17, 2019 at 4:49 PM Gerhard Lazu wrote: > Hi, > > We are trying to understand what prevents the Erlang distribution link > from saturating the network. Even though there is plenty of CPU, memory & > network bandwidth, the Erlang distribution > You need to instrument. Try running something like perf(1) on the emulator and look where it is spending its time. This will be able to either confirm or reject Dmytro's viewpoint that you are looking at excessive copying. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gerhard@REDACTED Tue Jun 18 12:30:00 2019 From: gerhard@REDACTED (Gerhard Lazu) Date: Tue, 18 Jun 2019 11:30:00 +0100 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? In-Reply-To: References: Message-ID: Some great replies, thank you all. I'm putting it all together now and re-running on my end. Will update as I capture various observations. On Tue, Jun 18, 2019 at 11:07 AM Lukas Larsson wrote: > > > On Mon, Jun 17, 2019 at 4:49 PM Gerhard Lazu wrote: > >> >> B = fun F() -> rpc:cast('foo@REDACTED', erlang, is_binary, [<<0:10000000/unit:8>>]), F() end. >> [spawn(fun() -> B() end) || _ <- lists:seq(1, 100)]. >> >> I wrote this code that is better able to saturate the network. Using rpc > will create bottlenecks on the receiving side as it is the receiving > process that does the decode. I get about 20% more traffic through when I > do that, which is not as much as I was expecting. > > -module(dist_perf). > > -export([go/0,go/1]). > > go() -> > go(100). > go(N) -> > spawn_link(fun stats/0), > RemoteNode = 'bar@REDACTED', > Pids = [spawn(RemoteNode, fun F() -> receive {From, Msg} -> From ! > is_binary(Msg), F() end end) > || _<- lists:seq(1,N)], > Payload = <<0:10000000/unit:8>>, > B = fun F(Pid) -> Pid ! {self(),Payload}, receive true -> F(Pid) end > end, > [spawn(fun() -> B(Pid) end) || Pid <- Pids ]. > > > stats() -> > {{_input,T0Inp},{_output,T0Out}} = erlang:statistics(io), > timer:sleep(5000), > {{_input,T1Inp},{_output,T1Out}} = erlang:statistics(io), > io:format("Sent ~pMB/s, Recv ~pMB/s~n", > [(T1Out - T0Out) / 1024 / 1024 / 5, > (T1Inp - T0Inp) / 1024 / 1024 / 5]), > stats(). > > I fired up linux perf to have a look at what was taking time and it is (as > Dmytro said) the copying of data that takes time. There is one specific > places that you hit with your test very badly: > https://github.com/erlang/otp/blob/master/erts/emulator/beam/dist.c#L1565 > > This is the re-assembly of fragmented distribution messages. This happens > because the messages sent are > 64kb. I change that limit to be 64MB on my > machine that that doubled the throughput ( > https://github.com/erlang/otp/blob/master/erts/emulator/beam/dist.h#L150). > > The reason why the re-assembly has to be done is that the emulator does > not have a variant of erlang:binary_to_term/1 that takes an iovector as > input. This could definitely be fixed and is something that we are planning > on fixing, though it is not something we are working on right now. However, > it will only change things in production if you send > 64KB messages. > > Also, just to be clear, the copying that is done by the distribution layer > is: > > 1) use term_to_binary to create a buffer > 2) use writev in inet_driver to copy buffer to kernel > 3) use recv to copy data from kernel into a refc binary > 4) if needed, use memcpy re-assemble fragments > 5) use binary_to_term to decode term > > (there is no copy to the destination process as Dmyto said) > > Before OTP-22 there was an additional copy at the receiving side, but that > has been removed. So imo there is only one copy that could potentially be > removed. > > If you want to decrease the overhead of saturating the network with many > small messages I would implement something like read_packets for tcp in the > inet_driver. However, the real bottleneck quickly becomes memory > allocations as each message needs two or three allocations, and when you > receive a lot of messages those allocations add up. > > Lukas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gerhard@REDACTED Tue Jun 18 14:04:54 2019 From: gerhard@REDACTED (Gerhard Lazu) Date: Tue, 18 Jun 2019 13:04:54 +0100 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? In-Reply-To: References: Message-ID: *TL;DR Single distribution link was measured to peak at 14.4 Gbit/s, which is ~50% from network maximum, 28.5 Gbit/s.* I'm confirming Lukas' observations: using smaller payload sizes can result in up to 14.4 Gbit/s throughput per distribution link. Payloads of both 60KB & 120KB yield the same max throughput of 14.4 Gbit/s. When message payloads go from 1MB to 4MB, throughput drops from 12.8 Gbit/s to 9.6 Gbit/s. With 10MB payloads, distribution link maxes out at 7.6 Gbit/s. This is interesting because yesterday I couldn't get it above 6 Gbit/s. Since this is GCP, on paper each CPU gets 2 Gbit/s. At 16 CPUs we should be maxing out at 32 Gbit/s. I have just benchmarked with iperf, and today these fresh VMs are peaking at 28.5 Gbit/s (yesterday they were 22.5 Gbit/s). At 100MB payloads, distribution link makes out at 4.5 Gbit/s. All distribution-related metrics, including annotations for various message payloads can be found here: https://grafana.gcp.rabbitmq.com/dashboard/snapshot/SDr2EWgaD2KOX5i0H154UvzoHRS68FV0?orgId=1 On Tue, Jun 18, 2019 at 11:07 AM Lukas Larsson wrote: > > > On Mon, Jun 17, 2019 at 4:49 PM Gerhard Lazu wrote: > >> >> B = fun F() -> rpc:cast('foo@REDACTED', erlang, is_binary, [<<0:10000000/unit:8>>]), F() end. >> [spawn(fun() -> B() end) || _ <- lists:seq(1, 100)]. >> >> I wrote this code that is better able to saturate the network. Using rpc > will create bottlenecks on the receiving side as it is the receiving > process that does the decode. I get about 20% more traffic through when I > do that, which is not as much as I was expecting. > > -module(dist_perf). > > -export([go/0,go/1]). > > go() -> > go(100). > go(N) -> > spawn_link(fun stats/0), > RemoteNode = 'bar@REDACTED', > Pids = [spawn(RemoteNode, fun F() -> receive {From, Msg} -> From ! > is_binary(Msg), F() end end) > || _<- lists:seq(1,N)], > Payload = <<0:10000000/unit:8>>, > B = fun F(Pid) -> Pid ! {self(),Payload}, receive true -> F(Pid) end > end, > [spawn(fun() -> B(Pid) end) || Pid <- Pids ]. > > > stats() -> > {{_input,T0Inp},{_output,T0Out}} = erlang:statistics(io), > timer:sleep(5000), > {{_input,T1Inp},{_output,T1Out}} = erlang:statistics(io), > io:format("Sent ~pMB/s, Recv ~pMB/s~n", > [(T1Out - T0Out) / 1024 / 1024 / 5, > (T1Inp - T0Inp) / 1024 / 1024 / 5]), > stats(). > > I fired up linux perf to have a look at what was taking time and it is (as > Dmytro said) the copying of data that takes time. There is one specific > places that you hit with your test very badly: > https://github.com/erlang/otp/blob/master/erts/emulator/beam/dist.c#L1565 > > This is the re-assembly of fragmented distribution messages. This happens > because the messages sent are > 64kb. I change that limit to be 64MB on my > machine that that doubled the throughput ( > https://github.com/erlang/otp/blob/master/erts/emulator/beam/dist.h#L150). > > The reason why the re-assembly has to be done is that the emulator does > not have a variant of erlang:binary_to_term/1 that takes an iovector as > input. This could definitely be fixed and is something that we are planning > on fixing, though it is not something we are working on right now. However, > it will only change things in production if you send > 64KB messages. > > Also, just to be clear, the copying that is done by the distribution layer > is: > > 1) use term_to_binary to create a buffer > 2) use writev in inet_driver to copy buffer to kernel > 3) use recv to copy data from kernel into a refc binary > 4) if needed, use memcpy re-assemble fragments > 5) use binary_to_term to decode term > > (there is no copy to the destination process as Dmyto said) > > Before OTP-22 there was an additional copy at the receiving side, but that > has been removed. So imo there is only one copy that could potentially be > removed. > > If you want to decrease the overhead of saturating the network with many > small messages I would implement something like read_packets for tcp in the > inet_driver. However, the real bottleneck quickly becomes memory > allocations as each message needs two or three allocations, and when you > receive a lot of messages those allocations add up. > > Lukas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Tue Jun 18 15:27:57 2019 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 18 Jun 2019 15:27:57 +0200 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? In-Reply-To: References: Message-ID: You made me curious so I did some additional optimizations: https://github.com/erlang/otp/pull/2291 They added an additional 20-25% for me. On Tue, Jun 18, 2019 at 2:05 PM Gerhard Lazu wrote: > *TL;DR Single distribution link was measured to peak at 14.4 Gbit/s, which > is ~50% from network maximum, 28.5 Gbit/s.* > > I'm confirming Lukas' observations: using smaller payload sizes can result > in up to 14.4 Gbit/s throughput per distribution link. > > Payloads of both 60KB & 120KB yield the same max throughput of 14.4 Gbit/s. > > When message payloads go from 1MB to 4MB, throughput drops from 12.8 > Gbit/s to 9.6 Gbit/s. > > With 10MB payloads, distribution link maxes out at 7.6 Gbit/s. This is > interesting because yesterday I couldn't get it above 6 Gbit/s. > Since this is GCP, on paper each CPU gets 2 Gbit/s. At 16 CPUs we should > be maxing out at 32 Gbit/s. > I have just benchmarked with iperf, and today these fresh VMs are peaking > at 28.5 Gbit/s (yesterday they were 22.5 Gbit/s). > > At 100MB payloads, distribution link makes out at 4.5 Gbit/s. > > All distribution-related metrics, including annotations for various > message payloads can be found here: > https://grafana.gcp.rabbitmq.com/dashboard/snapshot/SDr2EWgaD2KOX5i0H154UvzoHRS68FV0?orgId=1 > > On Tue, Jun 18, 2019 at 11:07 AM Lukas Larsson wrote: > >> >> >> On Mon, Jun 17, 2019 at 4:49 PM Gerhard Lazu wrote: >> >>> >>> B = fun F() -> rpc:cast('foo@REDACTED', erlang, is_binary, [<<0:10000000/unit:8>>]), F() end. >>> [spawn(fun() -> B() end) || _ <- lists:seq(1, 100)]. >>> >>> I wrote this code that is better able to saturate the network. Using rpc >> will create bottlenecks on the receiving side as it is the receiving >> process that does the decode. I get about 20% more traffic through when I >> do that, which is not as much as I was expecting. >> >> -module(dist_perf). >> >> -export([go/0,go/1]). >> >> go() -> >> go(100). >> go(N) -> >> spawn_link(fun stats/0), >> RemoteNode = 'bar@REDACTED', >> Pids = [spawn(RemoteNode, fun F() -> receive {From, Msg} -> From ! >> is_binary(Msg), F() end end) >> || _<- lists:seq(1,N)], >> Payload = <<0:10000000/unit:8>>, >> B = fun F(Pid) -> Pid ! {self(),Payload}, receive true -> F(Pid) end >> end, >> [spawn(fun() -> B(Pid) end) || Pid <- Pids ]. >> >> >> stats() -> >> {{_input,T0Inp},{_output,T0Out}} = erlang:statistics(io), >> timer:sleep(5000), >> {{_input,T1Inp},{_output,T1Out}} = erlang:statistics(io), >> io:format("Sent ~pMB/s, Recv ~pMB/s~n", >> [(T1Out - T0Out) / 1024 / 1024 / 5, >> (T1Inp - T0Inp) / 1024 / 1024 / 5]), >> stats(). >> >> I fired up linux perf to have a look at what was taking time and it is >> (as Dmytro said) the copying of data that takes time. There is one specific >> places that you hit with your test very badly: >> https://github.com/erlang/otp/blob/master/erts/emulator/beam/dist.c#L1565 >> >> This is the re-assembly of fragmented distribution messages. This happens >> because the messages sent are > 64kb. I change that limit to be 64MB on my >> machine that that doubled the throughput ( >> https://github.com/erlang/otp/blob/master/erts/emulator/beam/dist.h#L150 >> ). >> >> The reason why the re-assembly has to be done is that the emulator does >> not have a variant of erlang:binary_to_term/1 that takes an iovector as >> input. This could definitely be fixed and is something that we are planning >> on fixing, though it is not something we are working on right now. However, >> it will only change things in production if you send > 64KB messages. >> >> Also, just to be clear, the copying that is done by the distribution >> layer is: >> >> 1) use term_to_binary to create a buffer >> 2) use writev in inet_driver to copy buffer to kernel >> 3) use recv to copy data from kernel into a refc binary >> 4) if needed, use memcpy re-assemble fragments >> 5) use binary_to_term to decode term >> >> (there is no copy to the destination process as Dmyto said) >> >> Before OTP-22 there was an additional copy at the receiving side, but >> that has been removed. So imo there is only one copy that could potentially >> be removed. >> >> If you want to decrease the overhead of saturating the network with many >> small messages I would implement something like read_packets for tcp in the >> inet_driver. However, the real bottleneck quickly becomes memory >> allocations as each message needs two or three allocations, and when you >> receive a lot of messages those allocations add up. >> >> Lukas >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gerhard@REDACTED Tue Jun 18 15:34:53 2019 From: gerhard@REDACTED (Gerhard Lazu) Date: Tue, 18 Jun 2019 14:34:53 +0100 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? In-Reply-To: References: Message-ID: Someone's on a roll today! Pulling the patch now and giving it a run for its money. Would it be worth benchmarking OTP v21.3.8.4? -------------- next part -------------- An HTML attachment was scrubbed... URL: From gerhard@REDACTED Tue Jun 18 23:14:14 2019 From: gerhard@REDACTED (Gerhard Lazu) Date: Tue, 18 Jun 2019 22:14:14 +0100 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? In-Reply-To: References: Message-ID: https://github.com/erlang/otp/pull/2291 is an improvement for larger payloads, not so much for smaller ones. 60 KB - 13.6 Gbit/s (down from 14.4 Gbit/s) 120 KB - 12.8 Gbit/s (down from 14.4 Gbit/s) 10MB - 8.8 Gbit/s (up from 7.6 Gbit/s) 100MB - 5.8 Gbit/s (up from 4.3 Gbit/s) This is OTP 22.0.2 https://grafana.gcp.rabbitmq.com/dashboard/snapshot/SDr2EWgaD2KOX5i0H154UvzoHRS68FV0 This is OTP 22.0.3 + PR 2291 https://grafana.gcp.rabbitmq.com/dashboard/snapshot/8yxY61o140gvAKXFthgIVCes2oW2Owa7 Will try https://github.com/erlang/otp/pull/2293 tomorrow On Tue, Jun 18, 2019 at 2:34 PM Gerhard Lazu wrote: > Someone's on a roll today! > > Pulling the patch now and giving it a run for its money. > > Would it be worth benchmarking OTP v21.3.8.4? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Tue Jun 18 23:40:48 2019 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 18 Jun 2019 14:40:48 -0700 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? In-Reply-To: References: Message-ID: <377122da-d87a-f832-4158-74e1bade524e@gmail.com> On 6/18/19 2:14 PM, Gerhard Lazu wrote: > https://github.com/erlang/otp/pull/2291 is an improvement for larger > payloads, not so much for smaller ones. > > 60 KB - 13.6 Gbit/s (down from 14.4 Gbit/s) > 120 KB - 12.8 Gbit/s (down from 14.4 Gbit/s) > 10MB - 8.8 Gbit/s (up from 7.6 Gbit/s) > 100MB - 5.8 Gbit/s (up from 4.3 Gbit/s) > > This is OTP 22.0.2 > https://grafana.gcp.rabbitmq.com/dashboard/snapshot/SDr2EWgaD2KOX5i0H154UvzoHRS68FV0 > > This is OTP 22.0.3 + PR 2291 > https://grafana.gcp.rabbitmq.com/dashboard/snapshot/8yxY61o140gvAKXFthgIVCes2oW2Owa7 > > > Will try https://github.com/erlang/otp/pull/2293?tomorrow > You should also try using the erl command line option +zdbbl when testing larger payloads (it defaults to 1024 KB with valid values from 1 - 2097151 KB). Best Regards, Michael From gerhard@REDACTED Tue Jun 18 23:47:25 2019 From: gerhard@REDACTED (Gerhard Lazu) Date: Tue, 18 Jun 2019 22:47:25 +0100 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? In-Reply-To: <377122da-d87a-f832-4158-74e1bade524e@gmail.com> References: <377122da-d87a-f832-4158-74e1bade524e@gmail.com> Message-ID: Our zdbbl defaults to 128000. This PR exposes how much of the zdbbl is being used: https://github.com/erlang/otp/pull/2270 We've tried larger values - 1280000 - which increase memory usage and make throughput marginally better (~1%), but spikier. On Tue, Jun 18, 2019 at 10:40 PM Michael Truog wrote: > On 6/18/19 2:14 PM, Gerhard Lazu wrote: > > https://github.com/erlang/otp/pull/2291 is an improvement for larger > > payloads, not so much for smaller ones. > > > > 60 KB - 13.6 Gbit/s (down from 14.4 Gbit/s) > > 120 KB - 12.8 Gbit/s (down from 14.4 Gbit/s) > > 10MB - 8.8 Gbit/s (up from 7.6 Gbit/s) > > 100MB - 5.8 Gbit/s (up from 4.3 Gbit/s) > > > > This is OTP 22.0.2 > > > https://grafana.gcp.rabbitmq.com/dashboard/snapshot/SDr2EWgaD2KOX5i0H154UvzoHRS68FV0 > > > > This is OTP 22.0.3 + PR 2291 > > > https://grafana.gcp.rabbitmq.com/dashboard/snapshot/8yxY61o140gvAKXFthgIVCes2oW2Owa7 > > > > > > Will try https://github.com/erlang/otp/pull/2293 tomorrow > > > You should also try using the erl command line option +zdbbl when > testing larger payloads (it defaults to 1024 KB with valid values from 1 > - 2097151 KB). > > Best Regards, > Michael > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Wed Jun 19 09:27:10 2019 From: lukas@REDACTED (Lukas Larsson) Date: Wed, 19 Jun 2019 09:27:10 +0200 Subject: [erlang-questions] Erlang distribution links don't fully utilise available resources - OTP 22.0.2 - Why? In-Reply-To: References: Message-ID: On Tue, Jun 18, 2019 at 11:15 PM Gerhard Lazu wrote: > https://github.com/erlang/otp/pull/2291 is an improvement for larger > payloads, not so much for smaller ones. > > 60 KB - 13.6 Gbit/s (down from 14.4 Gbit/s) > 120 KB - 12.8 Gbit/s (down from 14.4 Gbit/s) > 10MB - 8.8 Gbit/s (up from 7.6 Gbit/s) > 100MB - 5.8 Gbit/s (up from 4.3 Gbit/s) > > This is OTP 22.0.2 > https://grafana.gcp.rabbitmq.com/dashboard/snapshot/SDr2EWgaD2KOX5i0H154UvzoHRS68FV0 > > This is OTP 22.0.3 + PR 2291 > https://grafana.gcp.rabbitmq.com/dashboard/snapshot/8yxY61o140gvAKXFthgIVCes2oW2Owa7 > > Will try https://github.com/erlang/otp/pull/2293 tomorrow > Just keep in mind that these optimizations work great for when you are sending large binary terms. If you were to send the same data but as a list, you are back where you started again. So as always when you do benchmarks, make sure that you are testing with data that is close to what you expect your application to actually send. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From eckard.brauer@REDACTED Wed Jun 19 16:24:31 2019 From: eckard.brauer@REDACTED (Eckard Brauer) Date: Wed, 19 Jun 2019 16:24:31 +0200 Subject: [erlang-questions] How to debug erlang Message-ID: <20190619162431.04ca854a@gmx.de> Hello all, another beginner question: Is there any good description of how to debug Erlang applications? My current problem is cowboy on a Raspberry Pi - even the hello_erlang from the guide doesn't process requests, but I'd rather like to learn how debugging is possible at a more general level. Thanks in advance, Eckard -- Wir haften nicht f?r die korrekte Funktion der in dieser eMail enthaltenen Viren. We are not liable for correct function of the viruses in this email! :) From Oliver.Korpilla@REDACTED Wed Jun 19 16:42:39 2019 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Wed, 19 Jun 2019 16:42:39 +0200 Subject: [erlang-questions] How to debug erlang In-Reply-To: <20190619162431.04ca854a@gmx.de> References: <20190619162431.04ca854a@gmx.de> Message-ID: Are you aware of?https://www.erlang-in-anger.com/?? Might be worth a look. Oliver Gesendet:?Mittwoch, 19. Juni 2019 um 16:24 Uhr Von:?"Eckard Brauer" An:?erlang-questions@REDACTED Betreff:?[erlang-questions] How to debug erlang Hello all, another beginner question: Is there any good description of how to debug Erlang applications? My current problem is cowboy on a Raspberry Pi - even the hello_erlang from the guide doesn't process requests, but I'd rather like to learn how debugging is possible at a more general level. Thanks in advance, Eckard -- Wir haften nicht f?r die korrekte Funktion der in dieser eMail enthaltenen Viren. We are not liable for correct function of the viruses in this email! :) _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] From g@REDACTED Wed Jun 19 16:57:19 2019 From: g@REDACTED (Garrett Smith) Date: Wed, 19 Jun 2019 09:57:19 -0500 Subject: [erlang-questions] How to debug erlang In-Reply-To: <20190619162431.04ca854a@gmx.de> References: <20190619162431.04ca854a@gmx.de> Message-ID: I suggest getting into the habit of using tracing. It's one of Erlang's game changing features. In a pinch, print to stdout. But once you become proficient at tracing, it's very fast. I've been doing a lot Python programming lately (machine learning) and miss the ability to observe a program without a) modifying it or b) tediously stepping through code. I haven't use this but based on the author I already know it's excellent :) https://ferd.github.io/recon/recon_trace.html Perhaps there are others folks can recommend. I personally found the built-in dbg interface quite complex and wrote something for e2. Garrett On Wed, Jun 19, 2019 at 9:25 AM Eckard Brauer wrote: > > Hello all, > > another beginner question: Is there any good description of how to > debug Erlang applications? My current problem is cowboy on a Raspberry > Pi - even the hello_erlang from the guide doesn't process requests, but > I'd rather like to learn how debugging is possible at a more general > level. > > Thanks in advance, > Eckard > > -- > Wir haften nicht f?r die korrekte Funktion der in dieser eMail > enthaltenen Viren. We are not liable for correct function of the > viruses in this email! :) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From eckard.brauer@REDACTED Wed Jun 19 19:36:58 2019 From: eckard.brauer@REDACTED (Eckard Brauer) Date: Wed, 19 Jun 2019 19:36:58 +0200 Subject: [erlang-questions] How to debug erlang In-Reply-To: References: <20190619162431.04ca854a@gmx.de> Message-ID: <20190619193658.0b7225d4@gmx.de> Hello Oliver and Garret, thank you for the helpful hints - indeed, I already found "Erlang in anger", but didn't read it at the time - so I simply didn't remember. The recon_trace I will definitely have a look on. One more question: I believe to remember that I read at any occasion of a possibility to create a backtrace from within a code snippet, but didn't try it aut extensively at that moment, so I forgot about. Do you have one more hint for that - or did I mix up things with any other language? Eckard Am Wed, 19 Jun 2019 09:57:19 -0500 schrieb Garrett Smith : > I suggest getting into the habit of using tracing. It's one of > Erlang's game changing features. > > In a pinch, print to stdout. But once you become proficient at > tracing, it's very fast. > > I've been doing a lot Python programming lately (machine learning) and > miss the ability to observe a program without a) modifying it or b) > tediously stepping through code. > > I haven't use this but based on the author I already know it's > excellent :) > > https://ferd.github.io/recon/recon_trace.html > > Perhaps there are others folks can recommend. I personally found the > built-in dbg interface quite complex and wrote something for e2. > > Garrett > > > On Wed, Jun 19, 2019 at 9:25 AM Eckard Brauer > wrote: [...] -- From fchschneider@REDACTED Wed Jun 19 19:41:48 2019 From: fchschneider@REDACTED (Schneider) Date: Wed, 19 Jun 2019 19:41:48 +0200 Subject: [erlang-questions] How to debug erlang In-Reply-To: References: <20190619162431.04ca854a@gmx.de> Message-ID: <0B0272F5-B65B-407D-BA68-E67F5F14021B@gmail.com> Chapter 17 in Erlang Programming by Francesco Cesarini and Simon Thompson gives a good introduction. However, Erlang debugging and tracing does require quite some understanding of the system before you can use it effectively. It is hugely effective if you know what you?re doing, so read the book first, which dates back some ten years but is still well worth reading. Another very useful tool is the observer. Start with observer:start() from the Erlang shell. The other tool to use as a beginner is the GUI debugger. Start from the Erlang shell with debugger:start(). See the Erlang docs for it?s use. This is probably more inline with the tools you?re used to. Frans > Op 19 jun. 2019 om 16:57 heeft Garrett Smith het volgende geschreven: > > I suggest getting into the habit of using tracing. It's one of > Erlang's game changing features. > > In a pinch, print to stdout. But once you become proficient at > tracing, it's very fast. > > I've been doing a lot Python programming lately (machine learning) and > miss the ability to observe a program without a) modifying it or b) > tediously stepping through code. > > I haven't use this but based on the author I already know it's excellent :) > > https://ferd.github.io/recon/recon_trace.html > > Perhaps there are others folks can recommend. I personally found the > built-in dbg interface quite complex and wrote something for e2. > > Garrett > > >> On Wed, Jun 19, 2019 at 9:25 AM Eckard Brauer wrote: >> >> Hello all, >> >> another beginner question: Is there any good description of how to >> debug Erlang applications? My current problem is cowboy on a Raspberry >> Pi - even the hello_erlang from the guide doesn't process requests, but >> I'd rather like to learn how debugging is possible at a more general >> level. >> >> Thanks in advance, >> Eckard >> >> -- >> Wir haften nicht f?r die korrekte Funktion der in dieser eMail >> enthaltenen Viren. We are not liable for correct function of the >> viruses in this email! :) >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Wed Jun 19 21:13:28 2019 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 19 Jun 2019 22:13:28 +0300 Subject: [erlang-questions] erlang and sysfs_notify limitations Message-ID: Linux kernel has sysfs_notify mechanism. This mechanism allow to switch from periodical reading (polling) /sys files to blocking epoll call. userlevel application can open required /sys file and start waiting for events with blocking "poll" call. Kernel driver will call sysfs_notify when something changes and user application will be able to read new data. As far as I have understood, erlang right now cannot use this approach because this non-standard approach requires polling on regular files. I have made an illustration for this problem: https://github.com/maxlapshin/erl_notify Without adding it we cannot read gpio, temperature or mdraid events with new way. Is it possible to add this way of polling local files (non-sockets) to erlang? -------------- next part -------------- An HTML attachment was scrubbed... URL: From petergi@REDACTED Thu Jun 20 07:34:24 2019 From: petergi@REDACTED (Peter J Etheridge) Date: Thu, 20 Jun 2019 15:34:24 +1000 Subject: [erlang-questions] Dialyzer error message Message-ID: Dear Friends,on erlang v.22 & ubuntu 18 When i; dialyzer --gui i get this error; {"init terminating in do_boot",{{case_clause,{{nocatch,{dialyzer_error,[78,111,116,32,97,32,114,101,103,117,108,97,114,32,102,105,108,101,58,32,"/usr/lib/erlang/lib/erts-10.3.4/ebin/atomics.beam",10]}},[{dialyzer_plt,compute_md5_from_file,1,[{file,"dialyzer_plt.erl"},{line,543}]},{dialyzer_plt,compute_new_md5_1,3,[{file,"dialyzer_plt.erl"},{line,509}]},{dialyzer_plt,check_plt1,3,[{file,"dialyzer_plt.erl"},{line,485}]},{dialyzer_plt,'-subproc/1-fun-0-',1,[{file,"dialyzer_plt.erl"},{line,603}]}]}},[{dialyzer_cl,check_plt,3,[{file,"dialyzer_cl.erl"},{line,250}]},{dialyzer_cl,plt_common,3,[{file,"dialyzer_cl.erl"},{line,183}]},{dialyzer,'-cl_check_init/1-fun-0-',1,[{file,"dialyzer.erl"},{line,94}]},{dialyzer,doit,1,[{file,"dialyzer.erl"},{line,236}]},{dialyzer,plain_cl,0,[{file,"dialyzer.erl"},{line,65}]},{init,start_em,1,[]},{init,do_boot,3,[]}]}} init terminating in do_boot ({{case_clause,{{nocatch,{_}},[{_},{_},{_},{_}]}},[{dialyzer_cl,check_plt,3,[{_},{_}]},{dialyzer_cl,plt_common,3,[{_},{_}]},{dialyzer,-cl_check_init/1-fun-0-,1,[{_},{_}]}, how should i repair dialyzer? Thank you in advance,Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Thu Jun 20 08:54:34 2019 From: lukas@REDACTED (Lukas Larsson) Date: Thu, 20 Jun 2019 08:54:34 +0200 Subject: [erlang-questions] Dialyzer error message In-Reply-To: References: Message-ID: Hello, On Thu, Jun 20, 2019 at 7:34 AM Peter J Etheridge wrote: > Dear Friends, > on erlang v.22 & ubuntu 18 > When i; > > dialyzer --gui > > i get this error; > > {"init terminating in > do_boot",{{case_clause,{{nocatch,{dialyzer_error,[78,111,116,32,97,32,114,101,103,117,108,97,114,32,102,105,108,101,58,32,"/usr/lib/erlang/lib/erts-10.3.4/ebin/atomics.beam",10]}},[{dialyzer_plt,compute_md5_from_file,1,[{file,"dialyzer_plt.erl"},{line,543}]},{dialyzer_plt,compute_new_md5_1,3,[{file,"dialyzer_plt.erl"},{line,509}]},{dialyzer_plt,check_plt1,3,[{file,"dialyzer_plt.erl"},{line,485}]},{dialyzer_plt,'-subproc/1-fun-0-',1,[{file,"dialyzer_plt.erl"},{line,603}]}]}},[{dialyzer_cl,check_plt,3,[{file,"dialyzer_cl.erl"},{line,250}]},{dialyzer_cl,plt_common,3,[{file,"dialyzer_cl.erl"},{line,183}]},{dialyzer,'-cl_check_init/1-fun-0-',1,[{file,"dialyzer.erl"},{line,94}]},{dialyzer,doit,1,[{file,"dialyzer.erl"},{line,236}]},{dialyzer,plain_cl,0,[{file,"dialyzer.erl"},{line,65}]},{init,start_em,1,[]},{init,do_boot,3,[]}]}} > init terminating in do_boot > ({{case_clause,{{nocatch,{_}},[{_},{_},{_},{_}]}},[{dialyzer_cl,check_plt,3,[{_},{_}]},{dialyzer_cl,plt_common,3,[{_},{_}]},{dialyzer,-cl_check_init/1-fun-0-,1,[{_},{_}]}, > If you do: 1> io:format("~s",[[78,111,116,32,97,32,114,101,103,117,108,97,114,32,102,105,108,101,58,32,"/usr/lib/erlang/lib/erts-10.3.4/ebin/atomics.beam",10]]). Not a regular file: /usr/lib/erlang/lib/erts-10.3.4/ebin/atomics.beam You can see that dialyzer is trying to tell you that atomics.beam is not a regular file. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From toddg@REDACTED Wed Jun 19 18:06:06 2019 From: toddg@REDACTED (Todd) Date: Wed, 19 Jun 2019 16:06:06 +0000 Subject: [erlang-questions] Simple perf testing using rpc:pmap Message-ID: Hi everyone, Longtime lurker here. I decided to take the summer off and do something I've wanted to do for years: learn Erlang. Newbie questions to follow... Similar to the thread on 'distribution links utilizing resources'... I've been looking at Erlang in a couple of different ways, and here are some graphs (and code) to look at... I started by using Erlang as an execution framework for comparing the performance of some common sorting algorithms: * source : https://github.com/ToddG/experimental/blob/master/interview-questions/glassdoor/sorting/erlang/sortz/src/sortz.erl * results : https://github.com/ToddG/experimental/blob/master/interview-questions/glassdoor/sorting/erlang/sortz/docs/sorting_algorithms_analysis.ipynb Then I took one of the algorithms and looked at the performance of this 'embarrassingly parallel' problem as it was spread across the available processors on my laptop via 'rpc:pmap': * source : https://github.com/ToddG/experimental/blob/master/interview-questions/glassdoor/sorting/erlang/psortz/src/psortz.erl * resoults : https://github.com/ToddG/experimental/blob/master/interview-questions/glassdoor/sorting/erlang/psortz/docs/parallel_sorting_analysis.ipynb I was surprised a couple of things: * It is ridiculously easy to use `rpc:pmap` * However, the gains are mostly in adding a second or third processor, adding more processors yielded marginally better results, but really not worth the effort (IMHO) I'd like to better understand the forces at work here: Q: How to apply Amdahls Law here? I have enough data, I should be able to calculate the constant time factor... Q: Why is the constant time factor so large? I expected much better overall performance when adding >3 processors... Q: How does 'rpc:pmap' differ from the skel framework that Joe mentioned in this thread: https://groups.google.com/forum/#!topic/erlang-programming/_smlcd6S9v0 https://skel.weebly.com/about-skel.html Q: Does anyone have a write-up on how to measure performance of distributed Erlang/BEAM apps on linux? What tools are you using, what stats are the most relevant, what things are missing that you metric yourself, etc? Obviously there's a _mountain_ of tools and things to look at here... I've been looking into [sysstat (https://github.com/sysstat/sysstat](https://github.com/sysstat/sysstat)). One of the first things I did when analyzing the algorithms in Part 1 was to start writing metrics to ETS...but then I discovered that Erlang already has tools like cover, cprof, fprof... In fact, it seems that whenever you are looking for something you need, it's actually there, somewhere. -Todd Greenwood toddg@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From yusuke.masuda@REDACTED Wed Jun 19 23:31:00 2019 From: yusuke.masuda@REDACTED (Yusuke Masuda) Date: Thu, 20 Jun 2019 06:31:00 +0900 Subject: [erlang-questions] Uninstall manually on Windows Message-ID: Hi. I would like to ask about how to uninstall erlang on Windows manually. We use the erlang on Windows with embeding our installer. When we cancel our installation, our installer rolled back the installation. But some erlang files (maybe registry also) were remained, and we couldn't uninstall the erlang completely. I'm not sure the erlang installer supports cancellation and rollback though, I need to uninstall it anyway. So I need to get how to uninstall manually. Regards, Yusuke. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Thu Jun 20 09:52:16 2019 From: lukas@REDACTED (Lukas Larsson) Date: Thu, 20 Jun 2019 09:52:16 +0200 Subject: [erlang-questions] Uninstall manually on Windows In-Reply-To: References: Message-ID: Hello, On Thu, Jun 20, 2019 at 9:16 AM Yusuke Masuda wrote: > Hi. > I would like to ask about how to uninstall erlang on Windows manually. > > We use the erlang on Windows with embeding our installer. > When we cancel our installation, our installer rolled back the > installation. > But some erlang files (maybe registry also) were remained, and we couldn't > uninstall the erlang completely. > I'm not sure the erlang installer supports cancellation and rollback > though, I need to uninstall it anyway. > So I need to get how to uninstall manually. > The scripts used to install Erlang on windows are available here: https://github.com/erlang/otp/tree/master/erts/etc/win32/nsis You should be able to see from there what things are changed when it is installed and revert them. I don't know how well the installer handles roll-backs, but I would imagine that it could be problematic. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From petergi@REDACTED Thu Jun 20 10:14:18 2019 From: petergi@REDACTED (Peter J Etheridge) Date: Thu, 20 Jun 2019 18:14:18 +1000 Subject: [erlang-questions] Dialyzer error message In-Reply-To: Message-ID: <1c4392816cd27e3a618c7616078d34b273a3d360@webmail.iinet.net.au> Dear Lukas, Thank you for your prompt response. Is there anything i should do to stop this file from bothering dialyzer?Many thanks,Peter ----- Original Message ----- From: "Lukas Larsson" To:"Peter J Etheridge" Cc:"Erlang Questions" Sent:Thu, 20 Jun 2019 08:54:34 +0200 Subject:Re: [erlang-questions] Dialyzer error message Hello, On Thu, Jun 20, 2019 at 7:34 AM Peter J Etheridge wrote: Dear Friends,on erlang v.22 & ubuntu 18 When i; dialyzer --gui i get this error; {"init terminating in do_boot",{{case_clause,{{nocatch,{dialyzer_error,[78,111,116,32,97,32,114,101,103,117,108,97,114,32,102,105,108,101,58,32,"/usr/lib/erlang/lib/erts-10.3.4/ebin/atomics.beam",10]}},[{dialyzer_plt,compute_md5_from_file,1,[{file,"dialyzer_plt.erl"},{line,543}]},{dialyzer_plt,compute_new_md5_1,3,[{file,"dialyzer_plt.erl"},{line,509}]},{dialyzer_plt,check_plt1,3,[{file,"dialyzer_plt.erl"},{line,485}]},{dialyzer_plt,'-subproc/1-fun-0-',1,[{file,"dialyzer_plt.erl"},{line,603}]}]}},[{dialyzer_cl,check_plt,3,[{file,"dialyzer_cl.erl"},{line,250}]},{dialyzer_cl,plt_common,3,[{file,"dialyzer_cl.erl"},{line,183}]},{dialyzer,'-cl_check_init/1-fun-0-',1,[{file,"dialyzer.erl"},{line,94}]},{dialyzer,doit,1,[{file,"dialyzer.erl"},{line,236}]},{dialyzer,plain_cl,0,[{file,"dialyzer.erl"},{line,65}]},{init,start_em,1,[]},{init,do_boot,3,[]}]}} init terminating in do_boot ({{case_clause,{{nocatch,{_}},[{_},{_},{_},{_}]}},[{dialyzer_cl,check_plt,3,[{_},{_}]},{dialyzer_cl,plt_common,3,[{_},{_}]},{dialyzer,-cl_check_init/1-fun-0-,1,[{_},{_}]}, If you do: 1> io:format("~s",[[78,111,116,32,97,32,114,101,103,117,108,97,114,32,102,105,108,101,58,32,"/usr/lib/erlang/lib/erts-10.3.4/ebin/atomics.beam",10]]). Not a regular file: /usr/lib/erlang/lib/erts-10.34/ebin/atomics.beam You can see that dialyzer is trying to tell you that atomics.beam is not a regular file. Lukas Links: ------ [1] mailto:petergi@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From yusuke.masuda@REDACTED Thu Jun 20 11:17:32 2019 From: yusuke.masuda@REDACTED (Yusuke Masuda) Date: Thu, 20 Jun 2019 18:17:32 +0900 Subject: [erlang-questions] Uninstall manually on Windows In-Reply-To: References: Message-ID: Thanks, I'll check it out. 2019?6?20?(?) ??4:52 Lukas Larsson : > Hello, > > On Thu, Jun 20, 2019 at 9:16 AM Yusuke Masuda > wrote: > >> Hi. >> I would like to ask about how to uninstall erlang on Windows manually. >> >> We use the erlang on Windows with embeding our installer. >> When we cancel our installation, our installer rolled back the >> installation. >> But some erlang files (maybe registry also) were remained, and we >> couldn't uninstall the erlang completely. >> I'm not sure the erlang installer supports cancellation and rollback >> though, I need to uninstall it anyway. >> So I need to get how to uninstall manually. >> > > The scripts used to install Erlang on windows are available here: > https://github.com/erlang/otp/tree/master/erts/etc/win32/nsis > > You should be able to see from there what things are changed when it is > installed and revert them. > > I don't know how well the installer handles roll-backs, but I would > imagine that it could be problematic. > > Lukas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From icfp.publicity@REDACTED Mon Jun 24 23:13:08 2019 From: icfp.publicity@REDACTED (Sam Tobin-Hochstadt) Date: Mon, 24 Jun 2019 17:13:08 -0400 Subject: [erlang-questions] Call for Participation: ICFP 2019 Message-ID: <5d113ce473a70_25202adb8252c5b42057@homer.mail> ===================================================================== Call for Participation ICFP 2019 24th ACM SIGPLAN International Conference on Functional Programming and affiliated events August 18 - August 23, 2019 Berlin, Germany http://icfp19.sigplan.org/ Early Registration until July 18! ===================================================================== ICFP provides a forum for researchers and developers to hear about the latest work on the design, implementations, principles, and uses of functional programming. The conference covers the entire spectrum of work, from practice to theory, including its peripheries. This year, ICFP is co-located with BOBKonf! * Overview and affiliated events: http://icfp19.sigplan.org/home * Program: http://icfp19.sigplan.org/program/program-icfp-2019 * Accepted papers: http://icfp19.sigplan.org/track/icfp-2019-papers * Registration is available via: https://regmaster4.com/2019conf/ICFP19/register.php Early registration ends 18 July, 2019. * Programming contest: https://icfpcontest2019.github.io/ * Student Research Competition: https://icfp19.sigplan.org/track/icfp-2019-Student-Research-Competition * Follow us on Twitter for the latest news: http://twitter.com/icfp_conference In addition to BOBKonf (8/21), there are several events co-located with ICFP: * Erlang Workshop (8/18) * Functional Art, Music, Modeling and Design (8/23) * Functional High-Performance and Numerical Computing (8/18) * Haskell Implementors' Workshop (8/23) * Haskell Symposium (8/22-8/23) * miniKanren Workshop (8/22) * ML Family Workshop (8/22) * OCaml Workshop (8/23) * Programming Languages Mentoring Workshop (8/18) * Scheme Workshop (8/18) * Type-Driven Development (8/18) ### ICFP Organizers General Chair: Derek Dreyer (MPI-SWS, Germany) Artifact Evaluation Co-Chairs: Simon Marlow (Facebook, UK) Industrial Relations Chair: Alan Jeffrey (Mozilla Research, USA) Programming Contest Organiser: Ilya Sergey (Yale-NUS College, Singapore) Publicity and Web Chair: Sam Tobin-Hochstadt (Indiana University, USA) Student Research Competition Chair: William J. Bowman (University of British Columbia, Canada) Workshops Co-Chair: Christophe Scholliers (Universiteit Gent, Belgium) Jennifer Hackett (University of Nottingham, UK) Conference Manager: Annabel Satin (P.C.K.) ### PACMPL Volume 3, Issue ICFP 2019 Principal Editor: Fran?ois Pottier (Inria, France) Review Committee: Lennart Beringer (Princeton University, United States) Joachim Breitner (DFINITY Foundation, Germany) Laura M. Castro (University of A Coru?a, Spain) Ezgi ?i?ek (Facebook London, United Kingdom) Pierre-Evariste Dagand (LIP6/CNRS, France) Christos Dimoulas (Northwestern University, United States) Jacques-Henri Jourdan (CNRS, LRI, Universit? Paris-Sud, France) Andrew Kennedy (Facebook London, United Kingdom) Daan Leijen (Microsoft Research, United States) Kazutaka Matsuda (Tohoku University, Japan) Bruno C. d. S. Oliveira (University of Hong Kong, China) Klaus Ostermann (University of T?bingen, Germany) Jennifer Paykin (Galois, United States) Frank Pfenning (Carnegie Mellon University, USA) Mike Rainey (Indiana University, USA) Chung-chieh Shan (Indiana University, USA) Sam Staton (University of Oxford, UK) Pierre-Yves Strub (Ecole Polytechnique, France) German Vidal (Universitat Politecnica de Valencia, Spain) External Review Committee: Michael D. Adams (University of Utah, USA) Robert Atkey (University of Strathclyde, IK) Sheng Chen (University of Louisiana at Lafayette, USA) James Cheney (University of Edinburgh, UK) Adam Chlipala (Massachusetts Institute of Technology, USA) Evelyne Contejean (LRI, Universit? Paris-Sud, France) Germ?n Andr?s Delbianco (IRIF, Universit? Paris Diderot, France) Dominique Devriese (Vrije Universiteit Brussel, Belgium) Richard A. Eisenberg (Bryn Mawr College, USA) Conal Elliott (Target, USA) Sebastian Erdweg (Delft University of Technology, Netherlands) Michael Greenberg (Pomona College, USA) Adrien Guatto (IRIF, Universit? Paris Diderot, France) Jennifer Hackett (University of Nottingham, UK) Troels Henriksen (University of Copenhagen, Denmark) Chung-Kil Hur (Seoul National University, Republic of Korea) Roberto Ierusalimschy (PUC-Rio, Brazil) Ranjit Jhala (University of California, San Diego, USA) Ralf Jung (MPI-SWS, Germany) Ohad Kammar (University of Oxford, UK) Oleg Kiselyov (Tohoku University, Japan) Hsiang-Shang ?Josh? Ko (National Institute of Informatics, Japan) Ond?ej Lhot?k (University of Waterloo, Canada) Dan Licata (Wesleyan University, USA) Geoffrey Mainland (Drexel University, USA) Simon Marlow (Facebook, UK) Akimasa Morihata (University of Tokyo, Japan) Shin-Cheng Mu (Academia Sinica, Taiwan) Guillaume Munch-Maccagnoni (Inria, France) Kim Nguy?n (University of Paris-Sud, France) Ulf Norell (Gothenburg University, Sweden) Atsushi Ohori (Tohoku University, Japan) Rex Page (University of Oklahoma, USA) Zoe Paraskevopoulou (Princeton University, USA) Nadia Polikarpova (University of California, San Diego, USA) Jonathan Protzenko (Microsoft Research, USA) Tiark Rompf (Purdue University, USA) Andreas Rossberg (Dfinity, Germany) KC Sivaramakrishnan (University of Cambridge, UI) Nicholas Smallbone (Chalmers University of Technology, Sweden) Matthieu Sozeau (Inria, France) Sandro Stucki (Chalmers | University of Gothenburg, Sweden) Don Syme (Microsoft, UK) Zachary Tatlock (University of Washington, USA) Sam Tobin-Hochstadt (Indiana University, USA) Takeshi Tsukada (University of Tokyo, Japan) Tarmo Uustalu (Reykjavik University, Iceland) Benoit Valiron (LRI, CentraleSupelec, Univ. Paris Saclay, France) Daniel Winograd-Cort (University of Pennsylvania, USA) Nicolas Wu (University of Bristol, UK) From roger@REDACTED Tue Jun 25 09:15:44 2019 From: roger@REDACTED (Roger Lipscombe) Date: Tue, 25 Jun 2019 08:15:44 +0100 Subject: [erlang-questions] cover:analyze ambiguity Message-ID: Any chance of a better API for cover:analyze that doesn't have all of the ambiguity? cover:analyse(M, module) gets tripped up if I have a module named 'coverage' or 'calls'. Similarly, there's a comment in there that *hopes* that we don't have a module named 'html'. Seems like an improved interface (keep the old one as well, obviously) would be a good thing. From minhduc031@REDACTED Tue Jun 25 09:55:55 2019 From: minhduc031@REDACTED (=?UTF-8?B?xJDhu6ljIMSQ4buX?=) Date: Tue, 25 Jun 2019 14:55:55 +0700 Subject: [erlang-questions] How to reset epmd properly??? In-Reply-To: References: Message-ID: Dears, I am having a issue relates to epmd reset. I made a cluster with some nodes. One day, there is a node got damage that a new node comming can't communicate with this node. But other nodes in the cluster still work well with it. I killed epmd on the damage node and registered node again with a same listen port as before. But as I captured, there is no SEND_CHALLENGE and SEND_CHALLENGE_REPLY messages while the damage node and a new node (same cookie) pinging together. So could you help figure out what is problem? Firewall maybe not because the cluster is working well. Just a new node can't join. Thanks so much!!! -------------- next part -------------- An HTML attachment was scrubbed... URL: From serge@REDACTED Tue Jun 25 23:17:23 2019 From: serge@REDACTED (Serge Aleynikov) Date: Tue, 25 Jun 2019 17:17:23 -0400 Subject: [erlang-questions] port driver naming Message-ID: When writing a port driver the "driver_name" field in the ErlDrvEntry structure is defined to match the name of the driver file. Is it possible to name the driver file differently from what's specified in the ErlDrvEntry? I have two architectures x86 and x64, and would like to distribute a prebuilt library with my application in the 32 and 64 bit flavors, and have the application choose the proper driver filename depending on the target architecture of the host machine. For this reason I am wondering if it's possible to name a driver file as "myapp-x86" or "myapp-x64" with a driver name field containing "myapp". Is that doable? Regards, Serge -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Jun 26 02:58:05 2019 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 26 Jun 2019 03:58:05 +0300 Subject: [erlang-questions] port driver naming In-Reply-To: References: Message-ID: It would be a great idea! Have one deb package for all architectures at once. On Wed, Jun 26, 2019 at 12:17 AM Serge Aleynikov wrote: > When writing a port driver the "driver_name" field in the ErlDrvEntry > structure is defined to match the name of the driver file. > > Is it possible to name the driver file differently from what's specified > in the ErlDrvEntry? > > I have two architectures x86 and x64, and would like to distribute a > prebuilt library with my application in the 32 and 64 bit flavors, and have > the application choose the proper driver filename depending on the target > architecture of the host machine. For this reason I am wondering if it's > possible to name a driver file as "myapp-x86" or "myapp-x64" with a driver > name field containing "myapp". Is that doable? > > Regards, > > Serge > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Wed Jun 26 03:30:32 2019 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 25 Jun 2019 18:30:32 -0700 Subject: [erlang-questions] port driver naming In-Reply-To: References: Message-ID: <60993b66-b14d-b95b-7933-16636140b6ae@gmail.com> On 6/25/19 2:17 PM, Serge Aleynikov wrote: > When writing a port driver the "driver_name" field in the ErlDrvEntry > structure is defined to match the name of the driver file. > > Is it possible to name the driver file differently from what's > specified in the ErlDrvEntry? > > I have two architectures x86 and x64, and would like to distribute a > prebuilt library with my application in the 32 and 64 bit flavors, and > have the application choose the proper driver filename depending on > the target architecture of the host machine.? For this reason I am > wondering if it's possible to name a driver file as "myapp-x86" or > "myapp-x64" with a driver name field containing "myapp".? Is that doable? > > Regards, > > Serge Hi Serge, With the current Erlang/OTP source code, you can get this to work by having driver_name be different for each file by using the preprocessor to create the unique filename.? You would effectively be using the string "myapp-" #arch, though it would require a few macros to expand properly (in the C/C++). Best Regards, Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From serge@REDACTED Wed Jun 26 06:06:06 2019 From: serge@REDACTED (Serge Aleynikov) Date: Wed, 26 Jun 2019 00:06:06 -0400 Subject: [erlang-questions] port driver naming In-Reply-To: <60993b66-b14d-b95b-7933-16636140b6ae@gmail.com> References: <60993b66-b14d-b95b-7933-16636140b6ae@gmail.com> Message-ID: Yes, I've worked around the issue with the compile-time defines that are different for the 32 and 64 bit architectures. However, it's not entirely clear why there's a requirement for the driver name to match the file name. It seems that it would be more flexible to allow them to be different. On Tue, Jun 25, 2019 at 9:30 PM Michael Truog wrote: > On 6/25/19 2:17 PM, Serge Aleynikov wrote: > > When writing a port driver the "driver_name" field in the ErlDrvEntry > structure is defined to match the name of the driver file. > > Is it possible to name the driver file differently from what's specified > in the ErlDrvEntry? > > I have two architectures x86 and x64, and would like to distribute a > prebuilt library with my application in the 32 and 64 bit flavors, and have > the application choose the proper driver filename depending on the target > architecture of the host machine. For this reason I am wondering if it's > possible to name a driver file as "myapp-x86" or "myapp-x64" with a driver > name field containing "myapp". Is that doable? > > Regards, > > Serge > > Hi Serge, > > With the current Erlang/OTP source code, you can get this to work by > having driver_name be different for each file by using the preprocessor to > create the unique filename. You would effectively be using the string > "myapp-" #arch, though it would require a few macros to expand properly (in > the C/C++). > > Best Regards, > Michael > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Thu Jun 27 00:24:35 2019 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 26 Jun 2019 15:24:35 -0700 Subject: [erlang-questions] port driver naming In-Reply-To: References: <60993b66-b14d-b95b-7933-16636140b6ae@gmail.com> Message-ID: On 6/25/19 9:06 PM, Serge Aleynikov wrote: > Yes, I've worked around the issue with the compile-time defines that > are different for the 32 and 64 bit architectures.? However, it's not > entirely clear why there's a requirement for the driver name to match > the file name.? It seems that it would be more flexible to allow them > to be different. My impression is that it is beneficial to have the extra validation that the build artifact matches the appropriate content.? That could be more important when you have more variations that provide the same source code in different dynamic library files (even for the same architecture, multiple files).? However, the same constraint doesn't exist for NIFs and due to that inconsistency it may be hard to justify. Best Regards, Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From albin.stigo@REDACTED Thu Jun 27 11:09:29 2019 From: albin.stigo@REDACTED (=?UTF-8?B?QWxiaW4gU3RpZ8O2?=) Date: Thu, 27 Jun 2019 11:09:29 +0200 Subject: [erlang-questions] Are the wxWidgets bindings actively developed Message-ID: Hi, Are the wxWidgets bindings under active development and recommended for new applications? I thinking about starting a new application. Are there any major known limitations in the bindings? Looking at GitHub I see recent commits but just want to make sure before I invest significant time. --Albin From dangud@REDACTED Thu Jun 27 11:33:28 2019 From: dangud@REDACTED (Dan Gudmundsson) Date: Thu, 27 Jun 2019 11:33:28 +0200 Subject: [erlang-questions] Are the wxWidgets bindings actively developed In-Reply-To: References: Message-ID: Mostly bug-fixes nowadays, some new features are sneaked in here and there. We are still basing it on wxWidgets-2.8 API, but you should be able to use anything from wxWidgets 2.8.4 - 3.1.X (*) as a backend. (*) If you enable 2.8 compatibility when compiling wxWidgets 3.1.X But pull-requests are appreciated :-) On Thu, Jun 27, 2019 at 11:09 AM Albin Stig? wrote: > Hi, > > Are the wxWidgets bindings under active development and recommended > for new applications? > > I thinking about starting a new application. Are there any major known > limitations in the bindings? Looking at GitHub I see recent commits > but just want to make sure before I invest significant time. > > > --Albin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From albin.stigo@REDACTED Thu Jun 27 11:55:41 2019 From: albin.stigo@REDACTED (=?UTF-8?B?QWxiaW4gU3RpZ8O2?=) Date: Thu, 27 Jun 2019 11:55:41 +0200 Subject: [erlang-questions] Are the wxWidgets bindings actively developed In-Reply-To: References: Message-ID: Thanks Dan, I will think about it! It's so damn hard to choose a GUI library these days. --Albin On Thu, Jun 27, 2019 at 11:33 AM Dan Gudmundsson wrote: > > Mostly bug-fixes nowadays, some new features are sneaked in here and there. > > We are still basing it on wxWidgets-2.8 API, but you should be able to use anything from > wxWidgets 2.8.4 - 3.1.X (*) as a backend. > > (*) If you enable 2.8 compatibility when compiling wxWidgets 3.1.X > > But pull-requests are appreciated :-) > > > On Thu, Jun 27, 2019 at 11:09 AM Albin Stig? wrote: >> >> Hi, >> >> Are the wxWidgets bindings under active development and recommended >> for new applications? >> >> I thinking about starting a new application. Are there any major known >> limitations in the bindings? Looking at GitHub I see recent commits >> but just want to make sure before I invest significant time. >> >> >> --Albin >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From albin.stigo@REDACTED Thu Jun 27 14:07:49 2019 From: albin.stigo@REDACTED (=?UTF-8?B?QWxiaW4gU3RpZ8O2?=) Date: Thu, 27 Jun 2019 14:07:49 +0200 Subject: [erlang-questions] Are the wxWidgets bindings actively developed In-Reply-To: References: Message-ID: Sounds good Dan, does the version of wxwidgets you are targeting support modern open gl >= 3 with shaders and vertex array objects, and does the Erlang bindings cover this? I suspect OpenGL support is pretty good since you are using it for wings3d? I'm thinking about writing a Software Defined Radio application with frontend in erlang/wxwidgets and a c++ node backend for DSP. --Albin On Thu, Jun 27, 2019, 12:22 Dan Gudmundsson wrote: > Since I use it privately for wings3d it will be supported as long as I > work here :-D > > On Thu, Jun 27, 2019 at 11:55 AM Albin Stig? > wrote: > >> Thanks Dan, I will think about it! It's so damn hard to choose a GUI >> library these days. >> >> >> --Albin >> >> On Thu, Jun 27, 2019 at 11:33 AM Dan Gudmundsson >> wrote: >> > >> > Mostly bug-fixes nowadays, some new features are sneaked in here and >> there. >> > >> > We are still basing it on wxWidgets-2.8 API, but you should be able to >> use anything from >> > wxWidgets 2.8.4 - 3.1.X (*) as a backend. >> > >> > (*) If you enable 2.8 compatibility when compiling wxWidgets 3.1.X >> > >> > But pull-requests are appreciated :-) >> > >> > >> > On Thu, Jun 27, 2019 at 11:09 AM Albin Stig? >> wrote: >> >> >> >> Hi, >> >> >> >> Are the wxWidgets bindings under active development and recommended >> >> for new applications? >> >> >> >> I thinking about starting a new application. Are there any major known >> >> limitations in the bindings? Looking at GitHub I see recent commits >> >> but just want to make sure before I invest significant time. >> >> >> >> >> >> --Albin >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From askjuise@REDACTED Thu Jun 27 17:23:19 2019 From: askjuise@REDACTED (Alexander Petrovsky) Date: Thu, 27 Jun 2019 18:23:19 +0300 Subject: [erlang-questions] erlang 21.3 and erts-10.3 scheduler_4 got stuck Message-ID: Hello! Dear maintainers, during perf test I stumble upon that application freeze, don't respond to erlang pings, don't accept any new connections and don't write any logs. Seems like one of the Erlang schedulers got stuck in an endless loop. # top -H PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 15977 hbc 20 0 5103668 1.6g 5764 R 99.9 21.5 10:23.40 4_scheduler The trace from gdb attached. Is any additional information required? -- ?????????? ????????? / Alexander Petrovsky, Skype: askjuise Phone: +7 931 9877991 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- Threads: (gdb) info thread Id Target Id Frame 34 Thread 0x7f115b13f700 (LWP 15961) "sys_sig_dispatc" 0x00007f119c5d86fd in read () from /lib64/libpthread.so.0 33 Thread 0x7f1153bbf700 (LWP 15962) "sys_msg_dispatc" 0x00007f119c5d5965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 32 Thread 0x7f115b1bc700 (LWP 15963) "async_1" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 31 Thread 0x7f11533be700 (LWP 15964) "async_2" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 30 Thread 0x7f115339b700 (LWP 15965) "async_3" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 29 Thread 0x7f1153378700 (LWP 15966) "async_4" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 28 Thread 0x7f1153355700 (LWP 15967) "async_5" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 27 Thread 0x7f1153332700 (LWP 15968) "async_6" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 26 Thread 0x7f115330f700 (LWP 15969) "async_7" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 25 Thread 0x7f11532ec700 (LWP 15970) "async_8" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 24 Thread 0x7f11532c9700 (LWP 15971) "async_9" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 23 Thread 0x7f11532a6700 (LWP 15972) "async_10" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 22 Thread 0x7f115313f700 (LWP 15974) "1_scheduler" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 21 Thread 0x7f115303c700 (LWP 15975) "2_scheduler" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 20 Thread 0x7f1152d39700 (LWP 15976) "3_scheduler" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 * 19 Thread 0x7f1152f39700 (LWP 15977) "4_scheduler" erts_try_change_runq_proc (rq=, p=0x7f10ac9e1c38) at beam/erl_process.h:2347 18 Thread 0x7f1152e36700 (LWP 15978) "1_dirty_cpu_sch" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 17 Thread 0x7f1152de3700 (LWP 15979) "2_dirty_cpu_sch" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 16 Thread 0x7f1152d90700 (LWP 15980) "3_dirty_cpu_sch" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 15 Thread 0x7f11529ff700 (LWP 15981) "4_dirty_cpu_sch" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 14 Thread 0x7f11529ac700 (LWP 15982) "1_dirty_io_sche" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 13 Thread 0x7f1152959700 (LWP 15983) "2_dirty_io_sche" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 12 Thread 0x7f1152906700 (LWP 15984) "3_dirty_io_sche" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 11 Thread 0x7f11528b3700 (LWP 15985) "4_dirty_io_sche" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 10 Thread 0x7f1152860700 (LWP 15986) "5_dirty_io_sche" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 9 Thread 0x7f115280d700 (LWP 15987) "6_dirty_io_sche" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 8 Thread 0x7f11527ba700 (LWP 15988) "7_dirty_io_sche" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 7 Thread 0x7f1152767700 (LWP 15989) "8_dirty_io_sche" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 6 Thread 0x7f1152714700 (LWP 15990) "9_dirty_io_sche" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 5 Thread 0x7f11526c1700 (LWP 15991) "10_dirty_io_sch" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 4 Thread 0x7f115266e700 (LWP 15992) "aux" 0x00007f119c0e82cf in ppoll () from /lib64/libc.so.6 3 Thread 0x7f115261b700 (LWP 15993) "0_poller" 0x00007f119c0ed1c9 in syscall () from /lib64/libc.so.6 2 Thread 0x7f10d3fff700 (LWP 15998) "2_scheduler" 0x00007f119c0b9e2d in nanosleep () from /lib64/libc.so.6 1 Thread 0x7f119d541740 (LWP 15858) "beam.smp" 0x00007f119c0e9f73 in select () from /lib64/libc.so.6 -------------------------------------------------------------------------------- Full backtrace of the thread: (gdb) backtrace #0 erts_try_change_runq_proc (rq=, p=0x7f10ac9e1c38) at beam/erl_process.h:2347 #1 select_enqueue_run_queue (state=, p=0x7f10ac9e1c38, enq_prio=2, enqueue=-1) at beam/erl_process.c:6442 #2 add2runq (enqueue=-1, prio=2, proc=0x7f10ac9e1c38, state=, proxy=0x0) at beam/erl_process.c:6618 #3 0x000000000045d1dc in active_sys_enqueue (p=0x7f10ac9e1c38, sys_task=sys_task@REDACTED=0x7f115bec01a8, task_prio=2, enable_flags=1310720, enable_flags@REDACTED=262144, state=, fail_state_p=fail_state_p@REDACTED=0x7f1152f38a00) at beam/erl_process.c:6899 #4 0x00000000004678ab in schedule_process_sys_task (p=p@REDACTED=0x7f10ac9e1c38, prio=, st=st@REDACTED=0x7f115bec01a8, fail_state_p=fail_state_p@REDACTED=0x7f1152f38a00) at beam/erl_process.c:6946 #5 0x00000000004679dc in schedule_generic_sys_task (pid=pid@REDACTED=60065118711107, type=type@REDACTED=ERTS_PSTT_PRIO_SIG, prio=prio@REDACTED=2, arg0=arg0@REDACTED=11, arg1=18446744073709551611) at beam/erl_process.c:10845 #6 0x0000000000448717 in erts_sig_prio (pid=pid@REDACTED=60065118711107, prio=prio@REDACTED=2) at beam/erl_process.c:10869 #7 0x0000000000512f6c in maybe_elevate_sig_handling_prio (c_p=c_p@REDACTED=0x7f107088dda8, other=other@REDACTED=60065118711107) at beam/erl_proc_sig_queue.c:756 #8 0x0000000000516a4b in erts_proc_sig_send_process_info_request (c_p=c_p@REDACTED=0x7f107088dda8, to=to@REDACTED=60065118711107, item_ix=item_ix@REDACTED=0x7f1152f38b90, len=len@REDACTED=6, need_msgq_len=need_msgq_len@REDACTED=8, flags=flags@REDACTED=58, reserve_size=38, ref=ref@REDACTED=139710117906410) at beam/erl_proc_sig_queue.c:1622 #9 0x00000000004b4877 in process_info_bif (c_p=, pid=, opt=, always_wrap=, pi2=) at beam/erl_bif_info.c:1262 #10 0x0000000000451246 in process_main (x_reg_array=0x7f115b90c300, f_reg_array=0x19ac55e42c2e0) at x86_64-unknown-linux-gnu/opt/smp/beam_cold.h:59 #11 0x000000000046570b in sched_thread_func (vesdp=0x7f1154221a80) at beam/erl_process.c:8469 #12 0x000000000067da6f in thr_wrapper (vtwd=) at pthread/ethread.c:118 #13 0x00007f119c5d1dd5 in start_thread () from /lib64/libpthread.so.0 #14 0x00007f119c0f2ead in clone () from /lib64/libc.so.6 (gdb) bt full #0 ethr_native_atomic64_cmpxchg_mb (old=139712402563008, new=139712402562112, var=0x7f10ac9e1f68) at ../include/internal/x86_64/../i386/atomic.h:97 No locals. #1 ethr_atomic_cmpxchg (old_val=139712402563008, val=139712402562112, var=0x7f10ac9e1f68) at ../include/internal/ethr_atomics.h:3495 No locals. #2 erts_try_change_runq_proc (rq=, p=0x7f10ac9e1c38) at beam/erl_process.h:2344 new_rqint = 139712402562112 #3 select_enqueue_run_queue (state=, p=0x7f10ac9e1c38, enq_prio=2, enqueue=-1) at beam/erl_process.c:6442 bound = #4 add2runq (enqueue=-1, prio=2, proc=0x7f10ac9e1c38, state=, proxy=0x0) at beam/erl_process.c:6618 No locals. #5 0x000000000045d1dc in active_sys_enqueue (p=0x7f10ac9e1c38, sys_task=sys_task@REDACTED=0x7f115bec01a8, task_prio=2, enable_flags=1310720, enable_flags@REDACTED=262144, state=, fail_state_p=fail_state_p@REDACTED=0x7f1152f38a00) at beam/erl_process.c:6899 runnable_procs = n = 1852222 a = enq_prio = fail_state = status_locked = 0 enqueue = #6 0x00000000004678ab in schedule_process_sys_task (p=p@REDACTED=0x7f10ac9e1c38, prio=, st=st@REDACTED=0x7f115bec01a8, fail_state_p=fail_state_p@REDACTED=0x7f1152f38a00) at beam/erl_process.c:6946 fail_state = 1024 state = #7 0x00000000004679dc in schedule_generic_sys_task (pid=pid@REDACTED=60065118711107, type=type@REDACTED=ERTS_PSTT_PRIO_SIG, prio=prio@REDACTED=2, arg0=arg0@REDACTED=11, arg1=18446744073709551611) at beam/erl_process.c:10845 st = 0x7f115bec01a8 st_prio = fail_state = 1024 res = 0 #8 0x0000000000448717 in erts_sig_prio (pid=pid@REDACTED=60065118711107, prio=prio@REDACTED=2) at beam/erl_process.c:10869 No locals. #9 0x0000000000512f6c in maybe_elevate_sig_handling_prio (c_p=c_p@REDACTED=0x7f107088dda8, other=other@REDACTED=60065118711107) at beam/erl_proc_sig_queue.c:756 res = -1 my_prio = 2 #10 0x0000000000516a4b in erts_proc_sig_send_process_info_request (c_p=c_p@REDACTED=0x7f107088dda8, to=to@REDACTED=60065118711107, item_ix=item_ix@REDACTED=0x7f1152f38b90, len=len@REDACTED=6, need_msgq_len=need_msgq_len@REDACTED=8, flags=flags@REDACTED=58, reserve_size=38, ref=ref@REDACTED=139710117906410) at beam/erl_proc_sig_queue.c:1622 size = 164 res = 1 #11 0x00000000004b4877 in process_info_bif (c_p=, pid=, opt=, always_wrap=, pi2=) at beam/erl_bif_info.c:1262 ref = 139710117906410 enqueued = need_msgq_len = 8 hfact = {mode = FACTORY_HALLOC, p = 0x7f1154201fc0, hp_start = 0xa300004, hp = 0x45ea72 , hp_end = 0x1, message = 0x7f1154201fc8, heap_frags = 0x0, heap_frags_saved = 0x0, heap_frags_saved_used = 17179869184001, off_heap = 0xa300004, off_heap_saved = {first = 0x7f1100008000, overhead = 4292870143}, alloc_type = 1391692720} def_arr = {0, 2, 14, 17, 1, 5, 6024601, 0, 59910469, 3840, 1411521432, 32529, 41002, 32528, -2075412328, 32528, 0, 32529, -2075412328, 32528, 2, 0, 5353214, 0, 1604397112, 32528, 3891, 0, 3145791, 0, 6072834, 0, 42, 0, 0, 0} item_ix = 0x7f1152f38b90 rp = 0x0 state = 1539880576 ret = reds = 0 locks = 1 flags = 58 reserve_size = len = 6 res = #12 0x0000000000451246 in process_main (x_reg_array=0x7f115b90c300, f_reg_array=0x19ac55e42c2e0) at x86_64-unknown-linux-gnu/opt/smp/beam_cold.h:59 bif_nif_arity = 2 live_hf_end = 0x0 nif_bif_result = 139712403967904 init_done = 1 c_p = 0x7f107088dda8 reds_used = 4526578 reg = 0x7f115b90c300 E = 0x2 I = 0x7f1154358fa0 FCALLS = 0 opcodes = {0x44ef8b , 0x45039e , 0x4537de , 0x45075c , 0x44f826 , 0x450d8f , 0x4511f2 , 0x45373c , 0x456e99 , 0x45995a , 0x452827 , 0x4513ce , 0x452a74 , 0x452b4b , 0x45336d , 0x4533a6 , 0x457e4b , 0x4583da , 0x45a325 , 0x4533cc , 0x4533f4 , 0x4528a7 , 0x44f994 , 0x453d31 , 0x453c9f , 0x459978 , 0x450ae1 , 0x450d61 , 0x45934e , 0x4571dd , 0x44edd2 , 0x4571fc , 0x453d75 , 0x45159a , 0x45020e , 0x459734 , 0x451672 , 0x44ff2e , 0x453495 , 0x452e3c , 0x45a27a , 0x44f93c , 0x44f0b4 , 0x452f31 , 0x452143 , 0x44f0d9 , 0x4524be , 0x452028 , 0x45138e , 0x452174 , 0x4507bf , 0x452f60 , 0x452fa0 , 0x45300c , 0x45a12c , 0x450156 , 0x452fe2 , 0x45304e , 0x45a170 , 0x457243 , 0x45a1dd , 0x45a22a , 0x456b1b , 0x456bc3 , 0x459d3e , 0x451f3c , 0x45a422 , 0x45a480 , 0x4530b9 , 0x453110 , 0x45a395 , 0x451e3a , 0x453077 , 0x453af5 , 0x451e8c , 0x451b23 , 0x453f22 , 0x453fa8 , 0x4543d4 , 0x4532c3 , 0x458cb3 , 0x458438 , 0x4585e7 , 0x458487 , 0x453175 , 0x45459a , 0x45446a , 0x454605 , 0x458acd , 0x459921 , 0x453b9f , 0x45881b , 0x45a71e , 0x45a72c , 0x4589cb , 0x45965b , 0x454615 , 0x45880c , 0x4594e0 , 0x454553 , 0x4534be , 0x453549 , 0x458ecc , 0x4535b6 , 0x4535df , 0x454249 , 0x4599fb , 0x458d36 , 0x45455f , 0x454527 , 0x45458f , 0x458dfe , 0x458e4f , 0x456d95 , 0x45a5e0 , 0x44fc76 , 0x44ff56 , 0x458b54 , 0x44eda3 , 0x44fb09 , 0x451a18 , 0x44fdf0 , 0x4512be , 0x451ab4 , 0x44f6d1 , 0x44ee4e , 0x457251 , 0x44fb78 , 0x4540f3 , 0x45033d , 0x451a55 , 0x4540e6 , 0x450edf , 0x4510aa , 0x45adbb , 0x4516bc , 0x4531ff , 0x45042f , 0x4577f2 , 0x459099 , 0x456a32 , 0x452e66 , 0x452ee7 , 0x4520c2 , 0x459055 , 0x458fa7 , 0x458fe8 , 0x453ab0 , 0x459389 , 0x454665 , 0x4593fd , 0x4541b8 , 0x459554 , 0x4546ea , 0x459518 , 0x452bfd , 0x44f0ff , 0x450310 , 0x451d5c , 0x453c71 , 0x450648 , 0x4515e6 , 0x4503ed , 0x456a95 , 0x44ee92 , 0x4500dc , 0x44f496 , 0x452b69 , 0x44ffd0 , 0x44f08f , 0x44f704 , 0x452e1b , 0x451c3d , 0x4540a4 , 0x44feb1 , 0x4542cb , 0x451003 , 0x45a2bb , 0x451029 , 0x45a414 , 0x450985 , 0x4596a9 , 0x44ef26 , 0x45a793 , 0x45162f , 0x4505ba , 0x450906 , 0x450d23 , 0x453d7a , 0x453db7 , 0x45a9ba , 0x4541fc , 0x45a8d0 , 0x45aaff , 0x45a943 , 0x45920b , 0x453609 , 0x4529d4 , 0x456fc6 , 0x44fa7e ...} #13 0x000000000046570b in sched_thread_func (vesdp=0x7f1154221a80) at beam/erl_process.c:8469 callbacks = {arg = 0x7f11542028c0, wakeup = 0x466450 , prepare_wait = 0x465000 , wait = 0x4657d0 , finalize_wait = 0x464fe0 } esdp = 0x7f1154221a80 no = 4 #14 0x000000000067da6f in thr_wrapper (vtwd=) at pthread/ethread.c:118 result = 0 c = 0 '\000' res = twd = thr_func = 0x4655b0 arg = 0x7f1154221a80 tsep = 0x7f115b140100 #15 0x00007f119c5d1dd5 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #16 0x00007f119c0f2ead in clone () from /lib64/libc.so.6 No symbol table info available. -------------------------------------------------------------------------------- Loop steps: (gdb) ni 0x000000000045cfb7 97 in ../include/internal/x86_64/../i386/atomic.h (gdb) ni erts_try_change_runq_proc (rq=, p=0x7f10ac9e1c38) at beam/erl_process.h:2347 2347 beam/erl_process.h: No such file or directory. (gdb) 0x000000000045cfbf 2347 in beam/erl_process.h (gdb) ethr_native_atomic64_cmpxchg_mb (old=139712402563008, new=139712402562112, var=0x7f10ac9e1f68) at ../include/internal/x86_64/../i386/atomic.h:97 97 ../include/internal/x86_64/../i386/atomic.h: No such file or directory. (gdb) 0x000000000045cfb7 97 in ../include/internal/x86_64/../i386/atomic.h (gdb) erts_try_change_runq_proc (rq=, p=0x7f10ac9e1c38) at beam/erl_process.h:2347 2347 beam/erl_process.h: No such file or directory. -------------------------------------------------------------------------------- Variable states: (gdb) ethr_native_atomic64_cmpxchg_mb (old=139712402563008, new=139712402562112, var=0x7f10ac9e1f68) at ../include/internal/x86_64/../i386/atomic.h:97 97 ../include/internal/x86_64/../i386/atomic.h: No such file or directory. (gdb) p *var $1 = {counter = 139712402562112} Process state: (gdb) p p $3 = (Process *) 0x7f10ac9e1c38 (gdb) p *p $4 = {common = {id = 60065118711107, refc = {atmc = {counter = 2}, sint = 2}, tracer = 18446744073709551611, trace_flags = 0, timer = {counter = 0}, u = {alive = {started_interval = 28353, reg = 0x0, links = 0x7f10ac989a20, monitors = 0x7f10ac7d55d8, lt_monitors = 0x7f10aca17880}, release = {later = 28353, func = 0x0, data = 0x7f10ac989a20, next = 0x7f10ac7d55d8}}}, htop = 0x7f1075d6c3d8, stop = 0x7f1075d6c3d8, heap = 0x7f1075d6a1b8, hend = 0x7f1075d6c3d8, abandoned_heap = 0x0, heap_sz = 1092, min_heap_size = 233, min_vheap_size = 46422, max_heap_size = 3, fp_exception = 0, hipe = { nsp = 0x0, nstack = 0x0, nstend = 0x0, u = {ncallee = 0x0, closure = 0, callee_exp = 0x0}, nstgraylim = 0x0, nstblacklim = 0x0, ngra = 0x0, ncsp = 0x0, narity = 0, float_result = 0}, arity = 3, arg_reg = 0x7f10ac9e1d40, max_arg_reg = 6, def_arg_reg = {145163, 305995, 139708673204713, 0, 0, 4000}, cp = 0x9773d8 , i = 0x9773d0 , catches = 0, fcalls = 3879, rcount = 0, schedule_count = 7, reds = 135085, group_leader = 49409303796195, flags = 33562624, fvalue = 18446744073709551611, freason = 10, ftrace = 139708673204729, next = 0x0, sig_qs = {first = 0x0, last = 0x7f10ac9e1dd0, save = 0x7f10ac9e1dd0, cont = 0x0, cont_last = 0x7f10ac9e1de8, nmsigs = {next = 0x0, last = 0x0}, saved_last = 0x0, len = 0}, bif_timers = 0x7f10ac256fe8, dictionary = 0x7f10ba141e38, seq_trace_clock = 0, seq_trace_lastcnt = 0, seq_trace_token = 18446744073709551611, u = {terminate = 0x2370b, initial = {module = 145163, function = 305739, arity = 5}}, current = 0x7f1154364338, parent = 51642686791155, static_flags = 0, high_water = 0x7f1075d6c3d8, old_hend = 0x0, old_htop = 0x0, old_heap = 0x0, gen_gcs = 9, max_gen_gcs = 65535, off_heap = { first = 0x7f1075d6b838, overhead = 36}, mbuf = 0x0, live_hf_end = 0xfffffffffffffff8, msg_frag = 0x0, mbuf_sz = 0, psd = {counter = 0}, bin_vheap_sz = 46422, bin_old_vheap_sz = 46422, bin_old_vheap = 0, sys_task_qs = 0x0, dirty_sys_tasks = 0x0, state = {counter = 319}, dirty_state = {counter = 0}, sig_inq = {first = 0x0, last = 0x7f10ac9e1f00, len = 0, nmsigs = {next = 0x0, last = 0x0}}, trace_msg_q = 0x0, lock = {flags = {counter = 0}, queue = {0x0, 0x0, 0x0, 0x0, 0x0}}, scheduler_data = 0x0, run_queue = {counter = 139712402562112}} From askjuise@REDACTED Thu Jun 27 18:02:45 2019 From: askjuise@REDACTED (Alexander Petrovsky) Date: Thu, 27 Jun 2019 19:02:45 +0300 Subject: [erlang-questions] erlang 21.3 and erts-10.3 scheduler_4 got stuck In-Reply-To: References: Message-ID: Additional, steps to reproduce from my colleague: # cat ./loop-rep.gdb br erl_process.h:2341 cont set p->run_queue.counter=p->run_queue.counter+1 quit # gdb --pid 32025 -x ./loop-rep.gdb where pid 32025 is scheduler thread pid ??, 27 ???. 2019 ?. ? 18:23, Alexander Petrovsky : > Hello! > > Dear maintainers, during perf test I stumble upon that application freeze, > don't respond to erlang pings, don't accept any new connections and don't > write any logs. Seems like one of the Erlang schedulers got stuck in an > endless loop. > > # top -H > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 15977 hbc 20 0 5103668 1.6g 5764 R 99.9 21.5 10:23.40 4_scheduler > > The trace from gdb attached. Is any additional information required? > > -- > ?????????? ????????? / Alexander Petrovsky, > > Skype: askjuise > Phone: +7 931 9877991 > > -- ?????????? ????????? / Alexander Petrovsky, Skype: askjuise Phone: +7 931 9877991 -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Thu Jun 27 20:30:09 2019 From: eric.pailleau@REDACTED (Eric PAILLEAU) Date: Thu, 27 Jun 2019 20:30:09 +0200 Subject: [erlang-questions] Are the wxWidgets bindings actively developed In-Reply-To: Message-ID: An HTML attachment was scrubbed... URL: From zxq9@REDACTED Fri Jun 28 09:00:05 2019 From: zxq9@REDACTED (zxq9@REDACTED) Date: Fri, 28 Jun 2019 16:00:05 +0900 Subject: [erlang-questions] Are the wxWidgets bindings actively developed In-Reply-To: References: Message-ID: <3533024.51eS4ni4ot@takoyaki> On 2019?6?27???? 11?09?29? JST Albin Stig? wrote: > Hi, > > Are the wxWidgets bindings under active development and recommended > for new applications? > > I thinking about starting a new application. Are there any major known > limitations in the bindings? Looking at GitHub I see recent commits > but just want to make sure before I invest significant time. I use them all the time! Unfortunately tutorial material is a bit lacking, so if you aren't already familiar with Wx from previous projects (Python, C++, etc.) then I recommend running the wx:demo() and reading the source to it. -Craig From zxq9@REDACTED Fri Jun 28 09:04:46 2019 From: zxq9@REDACTED (zxq9@REDACTED) Date: Fri, 28 Jun 2019 16:04:46 +0900 Subject: [erlang-questions] Are the wxWidgets bindings actively developed In-Reply-To: References: Message-ID: <5813327.2dgxCJhaQs@takoyaki> On 2019?6?27???? 11?33?28? JST Dan Gudmundsson wrote: > Mostly bug-fixes nowadays, some new features are sneaked in here and there. > > We are still basing it on wxWidgets-2.8 API, but you should be able to use > anything from > wxWidgets 2.8.4 - 3.1.X (*) as a backend. > > (*) If you enable 2.8 compatibility when compiling wxWidgets 3.1.X > > But pull-requests are appreciated :-) Is 3.1 not as universally available as 2.8, or are we in "ain't broke, don't fix" mode in terms of forward migration? Just curious. I use wx all the time and am grateful to have a reliable GUI binding at all -- not all languages have something comparable built in to the distribution that actually works cross-platform. -Craig From dangud@REDACTED Fri Jun 28 09:54:21 2019 From: dangud@REDACTED (Dan Gudmundsson) Date: Fri, 28 Jun 2019 09:54:21 +0200 Subject: [erlang-questions] Are the wxWidgets bindings actively developed In-Reply-To: <5813327.2dgxCJhaQs@takoyaki> References: <5813327.2dgxCJhaQs@takoyaki> Message-ID: On Fri, Jun 28, 2019 at 9:05 AM wrote: > On 2019?6?27???? 11?33?28? JST Dan Gudmundsson wrote: > > Mostly bug-fixes nowadays, some new features are sneaked in here and > there. > > > > We are still basing it on wxWidgets-2.8 API, but you should be able to > use > > anything from > > wxWidgets 2.8.4 - 3.1.X (*) as a backend. > > > > (*) If you enable 2.8 compatibility when compiling wxWidgets 3.1.X > > > > But pull-requests are appreciated :-) > > Is 3.1 not as universally available as 2.8, or are we in "ain't > broke, don't fix" mode in terms of forward migration? > wxWidgets use even numbers as stable releases. So 3.1 is not considered as a stable api, thus not available as pre-built packages on many linux platforms. The problem is that many bug-fixes are not backported to 3.0, (and there have not been one released for a while) which causes some problems mostly on Mojave for instance. The wx binding should be able to compile against either version, though I have not added all new functionality in 3.0 and 3.1 releases. Also, I use 2.8 api when generating the code, so it's hacky to add 3.0 and 3.1 functionality, and I don't know how much work it will be to upgrade the code generator to be based on the new releases. And we still have some old machines around which need wxWidgets 2.8 to be able to compile, sigh. /dgud > Just curious. > I use wx all the time and am grateful to have a reliable GUI > binding at all -- not all languages have something comparable > built in to the distribution that actually works cross-platform. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Jun 28 10:46:37 2019 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 28 Jun 2019 11:46:37 +0300 Subject: [erlang-questions] Interconnect question: how to work with names Message-ID: I have server with hostname myserver.l This hostname is in /etc/hosts and it is pingable I have master node running on it with: erl -name master@REDACTED When I write software, I don't know what will be the hostname, so I write shell script for connecting to shell: erl -name debug -remsh master@REDACTED It fails: Erlang/OTP 21 [erts-10.3.5.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe] *** ERROR: Shell process terminated! (^G to start new job) *** When I provide hostname, it works: erl -name debug -remsh master@REDACTED Erlang/OTP 21 [erts-10.3.5.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe] Eshell V10.3.5.2 (abort with ^G) (master@REDACTED)1> What is the proper way to deal with this situation? I can see the master node in epmd names, I can connect to it, I can pass interconnect protocol. But I do not understand, why remsh doesn't connect to it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gerhard@REDACTED Fri Jun 28 10:52:36 2019 From: gerhard@REDACTED (Gerhard Lazu) Date: Fri, 28 Jun 2019 09:52:36 +0100 Subject: [erlang-questions] Interconnect question: how to work with names In-Reply-To: References: Message-ID: My understanding is that you are opening a remote shell to a node with a specific name. There is a node with name *master@REDACTED* on *myserver.l* There is no node with name *master@REDACTED *on host *127.0.0.1* On Fri, Jun 28, 2019 at 9:47 AM Max Lapshin wrote: > > I have server with hostname myserver.l > This hostname is in /etc/hosts and it is pingable > > I have master node running on it with: erl -name master@REDACTED > > When I write software, I don't know what will be the hostname, so I write > shell script for connecting to shell: > > erl -name debug -remsh master@REDACTED > > It fails: > > Erlang/OTP 21 [erts-10.3.5.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] > [async-threads:1] [hipe] > > *** ERROR: Shell process terminated! (^G to start new job) *** > > When I provide hostname, it works: > > erl -name debug -remsh master@REDACTED > Erlang/OTP 21 [erts-10.3.5.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] > [async-threads:1] [hipe] > > Eshell V10.3.5.2 (abort with ^G) > (master@REDACTED)1> > > > > What is the proper way to deal with this situation? > I can see the master node in epmd names, I can connect to it, I can pass > interconnect protocol. > But I do not understand, why remsh doesn't connect to it. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Jun 28 11:06:27 2019 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 28 Jun 2019 12:06:27 +0300 Subject: [erlang-questions] Interconnect question: how to work with names In-Reply-To: References: Message-ID: myserver.l and 127.0.0.1 is the same server. I run it from this server. There is node called "master" in epmd and I do not understand why I can connect to it via hostname: master@REDACTED and cannot connect to it via 127.0.0.1 On Fri, Jun 28, 2019 at 11:53 AM Gerhard Lazu wrote: > My understanding is that you are opening a remote shell to a node with a > specific name. > > There is a node with name *master@REDACTED* on *myserver.l* > > There is no node with name *master@REDACTED *on host > *127.0.0.1* > > On Fri, Jun 28, 2019 at 9:47 AM Max Lapshin wrote: > >> >> I have server with hostname myserver.l >> This hostname is in /etc/hosts and it is pingable >> >> I have master node running on it with: erl -name master@REDACTED >> >> When I write software, I don't know what will be the hostname, so I write >> shell script for connecting to shell: >> >> erl -name debug -remsh master@REDACTED >> >> It fails: >> >> Erlang/OTP 21 [erts-10.3.5.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] >> [async-threads:1] [hipe] >> >> *** ERROR: Shell process terminated! (^G to start new job) *** >> >> When I provide hostname, it works: >> >> erl -name debug -remsh master@REDACTED >> Erlang/OTP 21 [erts-10.3.5.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] >> [async-threads:1] [hipe] >> >> Eshell V10.3.5.2 (abort with ^G) >> (master@REDACTED)1> >> >> >> >> What is the proper way to deal with this situation? >> I can see the master node in epmd names, I can connect to it, I can pass >> interconnect protocol. >> But I do not understand, why remsh doesn't connect to it. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dszoboszlay@REDACTED Fri Jun 28 11:07:17 2019 From: dszoboszlay@REDACTED (=?UTF-8?Q?D=C3=A1niel_Szoboszlay?=) Date: Fri, 28 Jun 2019 11:07:17 +0200 Subject: [erlang-questions] Interconnect question: how to work with names In-Reply-To: References: Message-ID: The host part of the node name has to match, otherwise the master node would believe the debug node is trying to connect to someone else and would refuse the connection. Try this command instead: erl -name debug -remsh master@$(hostname) On Fri, 28 Jun 2019 at 10:53, Gerhard Lazu wrote: > My understanding is that you are opening a remote shell to a node with a > specific name. > > There is a node with name *master@REDACTED* on *myserver.l* > > There is no node with name *master@REDACTED *on host > *127.0.0.1* > > On Fri, Jun 28, 2019 at 9:47 AM Max Lapshin wrote: > >> >> I have server with hostname myserver.l >> This hostname is in /etc/hosts and it is pingable >> >> I have master node running on it with: erl -name master@REDACTED >> >> When I write software, I don't know what will be the hostname, so I write >> shell script for connecting to shell: >> >> erl -name debug -remsh master@REDACTED >> >> It fails: >> >> Erlang/OTP 21 [erts-10.3.5.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] >> [async-threads:1] [hipe] >> >> *** ERROR: Shell process terminated! (^G to start new job) *** >> >> When I provide hostname, it works: >> >> erl -name debug -remsh master@REDACTED >> Erlang/OTP 21 [erts-10.3.5.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] >> [async-threads:1] [hipe] >> >> Eshell V10.3.5.2 (abort with ^G) >> (master@REDACTED)1> >> >> >> >> What is the proper way to deal with this situation? >> I can see the master node in epmd names, I can connect to it, I can pass >> interconnect protocol. >> But I do not understand, why remsh doesn't connect to it. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dszoboszlay@REDACTED Fri Jun 28 11:17:13 2019 From: dszoboszlay@REDACTED (=?UTF-8?Q?D=C3=A1niel_Szoboszlay?=) Date: Fri, 28 Jun 2019 11:17:13 +0200 Subject: [erlang-questions] Interconnect question: how to work with names In-Reply-To: References: Message-ID: The name of the Erlang node is the full thing, both the part before and after the @ sign. master@REDACTED and master@REDACTED and master@REDACTED are all different nodes for Erlang. It is a bit confusing that epmd only stores the part of the name before the @. It is assumed that if you're talking to epmd on 127.0.0.1 or myserver.l than all the nodes you find there will have 127.0.0.1 or myserverl. in their name after the @ respectively. But this is of course not necessarily holds, since there can be multiple names/IP addresses that resolve to the same host. So for example Erlang would believe master@REDACTED and master@REDACTED are different nodes, in practice you cannot have both nodes running at the same time, because they would both try to register as master in the same epmd instance. (Using the loopback address in the node name can lead to further complications when you have remote nodes in the cluster. E.g. you can have debug@REDACTED connected to master@REDACTED when they're on the same host, but if master@REDACTED would also connect to master@REDACTED it would try to connect to debug@REDACTED on myotherserver.l.) On Fri, 28 Jun 2019 at 11:06, Max Lapshin wrote: > myserver.l and 127.0.0.1 is the same server. I run it from this server. > > There is node called "master" in epmd and I do not understand why I can > connect to it via hostname: master@REDACTED and > cannot connect to it via 127.0.0.1 > > > On Fri, Jun 28, 2019 at 11:53 AM Gerhard Lazu wrote: > >> My understanding is that you are opening a remote shell to a node with a >> specific name. >> >> There is a node with name *master@REDACTED* on *myserver.l* >> >> There is no node with name *master@REDACTED *on host >> *127.0.0.1* >> >> On Fri, Jun 28, 2019 at 9:47 AM Max Lapshin >> wrote: >> >>> >>> I have server with hostname myserver.l >>> This hostname is in /etc/hosts and it is pingable >>> >>> I have master node running on it with: erl -name master@REDACTED >>> >>> When I write software, I don't know what will be the hostname, so I write >>> shell script for connecting to shell: >>> >>> erl -name debug -remsh master@REDACTED >>> >>> It fails: >>> >>> Erlang/OTP 21 [erts-10.3.5.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] >>> [async-threads:1] [hipe] >>> >>> *** ERROR: Shell process terminated! (^G to start new job) *** >>> >>> When I provide hostname, it works: >>> >>> erl -name debug -remsh master@REDACTED >>> Erlang/OTP 21 [erts-10.3.5.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] >>> [async-threads:1] [hipe] >>> >>> Eshell V10.3.5.2 (abort with ^G) >>> (master@REDACTED)1> >>> >>> >>> >>> What is the proper way to deal with this situation? >>> I can see the master node in epmd names, I can connect to it, I can pass >>> interconnect protocol. >>> But I do not understand, why remsh doesn't connect to it. >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Jun 28 20:15:15 2019 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 28 Jun 2019 21:15:15 +0300 Subject: [erlang-questions] Interconnect question: how to work with names In-Reply-To: References: Message-ID: > Validates the node name and the version number: https://github.com/erlang/otp/blob/master/lib/kernel/src/dist_util.erl#L772-L773 Yes, this seems to be the answer. Ok, what have I done: 1) I launch my server with hostname server.l: master@REDACTED 2) I add this hostname to control tool: inet_db:set_lookup([file, dns]), inet_db:add_host({127,0,0,1}, ["server.l"]), Now I can connect to master@REDACTED Server admin can put any hostname that he wants, I do not depend on it anymore. On Fri, Jun 28, 2019 at 4:16 PM D?niel Szoboszlay wrote: > I'm not claiming the response to the challenge itself would be dependent > on the node names. It is not. > > What I'm trying to say, is that together with the challenge, in the same > SEND_CHALLENGE message there are two more pieces of information: the > protocol version and the node name. > > Upon receiving the challenge ( > https://github.com/erlang/otp/blob/master/lib/kernel/src/dist_util.erl#L770) the > node that's trying to connect: > > - Validates the node name and the version number: > https://github.com/erlang/otp/blob/master/lib/kernel/src/dist_util.erl#L772-L773 > - Only after this step will look at the challenge sent in the message: > https://github.com/erlang/otp/blob/master/lib/kernel/src/dist_util.erl#L777 > - And calculate a challenge response at the place you identified: > https://github.com/erlang/otp/blob/master/lib/kernel/src/dist_util.erl#L447 > > So the recv_challenge/1 function is where the connection attempt fails if > there's a mismatch in the node names. And that's what you originally asked: > where does this check happen, what prevents us from connecting to a node > with the wrong name (in the post-@ part)? > > If you rewrite the handshake on your own, and omit this check, than fine, > you can of course connect. But this is probably not a good thing to do, > because you've just introduced some inconsistency into the cluster: the > nodes will disagree on what their name is. That can cause a lot of > problems, not in the distribution connection layer probably, but higher up > in the application layer (e.g. you send the value of node() across to your > peer and it will believe you're not talking about yourself, but a third, > unknown node). > > And as a side note, the is_allowed check has nothing to do with this > problem. That's a rarely used feature where you can blacklist/whitelist > nodes that are allowed to connect to you. > > On Fri, 28 Jun 2019 at 14:07, Max Lapshin wrote: > >> I do not understand, what for to use words like "believe" >> >> I've rewritten the handshake from scratch and I see that nodes send their >> names, but encryption digest does not use node name. >> >> >> https://github.com/erlang/otp/blob/master/lib/kernel/src/dist_util.erl#L447 >> >> Perhaps problem is here: >> >> >> https://github.com/erlang/otp/blob/master/lib/kernel/src/dist_util.erl#L715 >> >> When we change node name, we can pass interconnect handshake, but do not >> allow to connect. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Sat Jun 29 22:41:32 2019 From: lloyd@REDACTED (lloyd@REDACTED) Date: Sat, 29 Jun 2019 16:41:32 -0400 (EDT) Subject: [erlang-questions] Erlang records to XML Message-ID: <1561840892.18146862@apps.rackspace.com> Hello, Can anyone suggest a way to convert Erlang records to XML? I can find many ways to go from XML to Erlang, but not the reverse. Many thanks, LRP From hugo@REDACTED Sat Jun 29 23:33:09 2019 From: hugo@REDACTED (Hugo Mills) Date: Sat, 29 Jun 2019 21:33:09 +0000 Subject: [erlang-questions] Erlang records to XML In-Reply-To: <1561840892.18146862@apps.rackspace.com> References: <1561840892.18146862@apps.rackspace.com> Message-ID: <20190629213309.GM32479@carfax.org.uk> On Sat, Jun 29, 2019 at 04:41:32PM -0400, lloyd@REDACTED wrote: > Can anyone suggest a way to convert Erlang records to XML? Off the shelf, nothing I'm aware of. > I can find many ways to go from XML to Erlang, but not the reverse. Well there's a bunch of different XML libraries for XML. xmerl, for example, is a part of the standard erlang distribution, and is able to serialise to XML. There's an example [1] in the xmerl docs. (The others should also be able to do it, but I only checked xmerl). If you want a function that takes an erlang record and outputs some kind of XML structure, then you're probably going to be disappointed. Remember that records are mostly a funny compile-time syntax for tuples, and that the names are (mostly) lost at runtime. You're going to have to build the internal data structures that your chosen XML library uses, then serialise from those. Alternatively, if you don't want to be able to read the XML you're writing, and it's not a massively complicated structure, it's usually fairly simple to just build the XML directly. Hugo. [1] http://erlang.org/doc/apps/xmerl/xmerl_ug.html#example--create-xml-out-of-arbitrary-data -- Hugo Mills | We demand rigidly defined areas of doubt and hugo@REDACTED carfax.org.uk | uncertainty! http://carfax.org.uk/ | Vroomfondel PGP: E2AB1DE4 | The Hitch-Hikers Guide to the Galaxy -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From lloyd@REDACTED Sun Jun 30 00:02:05 2019 From: lloyd@REDACTED (lloyd@REDACTED) Date: Sat, 29 Jun 2019 18:02:05 -0400 (EDT) Subject: [erlang-questions] Erlang records to XML In-Reply-To: <7F8EF4B9-4FF4-4ACA-A47D-1F90B174C9D1@ninefx.com> References: <1561840892.18146862@apps.rackspace.com> <7F8EF4B9-4FF4-4ACA-A47D-1F90B174C9D1@ninefx.com> Message-ID: <1561845725.492815332@apps.rackspace.com> Thanks, guys! You're the best. Lloyd -----Original Message----- From: drew.varner@REDACTED Sent: Saturday, June 29, 2019 5:58pm To: lloyd@REDACTED Cc: "Erlang-Questions Questions" Subject: Re: [erlang-questions] Erlang records to XML Lloyd, Look at the spec (not related to Dialyzer) system used by fxml_gen in ProcessOne?s fxml parser. They include an example in the main repo for XML-RPC here: https://github.com/processone/fast_xml/blob/master/spec/README.md They also use it for their XMPP library, if you?d like more examples. I believe it generates decoders/encoders between records and XML. - Drew > On Jun 29, 2019, at 4:41 PM, lloyd@REDACTED wrote: > > Hello, > > Can anyone suggest a way to convert Erlang records to XML? > > I can find many ways to go from XML to Erlang, but not the reverse. > > Many thanks, > > LRP > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From eckard.brauer@REDACTED Sun Jun 30 12:21:36 2019 From: eckard.brauer@REDACTED (Eckard Brauer) Date: Sun, 30 Jun 2019 12:21:36 +0200 Subject: [erlang-questions] Interconnect question: how to work with names In-Reply-To: References: Message-ID: <20190630122136.055d4359@gmx.de> So IMHO it could be wise to (more) explicitely point out that name resolution in Erlang is at least partially different from the system one (nsswitch.conf), and to include a compact description on that in the manual (didn't check if it already is there). Seems the question arises from time to time... Best regards, Eckard Am Fri, 28 Jun 2019 11:17:13 +0200 schrieb D?niel Szoboszlay : > The name of the Erlang node is the full thing, both the part before > and after the @ sign. > master@REDACTED and master@REDACTED and master@REDACTED are all > different nodes for Erlang. > > It is a bit confusing that epmd only stores the part of the name > before the @. It is assumed that if you're talking to epmd on > 127.0.0.1 or myserver.l than all the nodes you find there will have > 127.0.0.1 or myserverl. in their name after the @ respectively. But > this is of course not necessarily holds, since there can be multiple > names/IP addresses that resolve to the same host. > > So for example Erlang would believe master@REDACTED and > master@REDACTED are different nodes, in practice you cannot have > both nodes running at the same time, because they would both try to > register as master in the same epmd instance. (Using the loopback > address in the node name can lead to further complications when you > have remote nodes in the cluster. E.g. you can have debug@REDACTED > connected to master@REDACTED when they're on the same host, but if > master@REDACTED would also connect to master@REDACTED it > would try to connect to debug@REDACTED on myotherserver.l.) > > On Fri, 28 Jun 2019 at 11:06, Max Lapshin > wrote: > > > myserver.l and 127.0.0.1 is the same server. I run it from this > > server. > > > > There is node called "master" in epmd and I do not understand why > > I can connect to it via hostname: master@REDACTED and > > cannot connect to it via 127.0.0.1 > > > > > > On Fri, Jun 28, 2019 at 11:53 AM Gerhard Lazu > > wrote: > > > >> My understanding is that you are opening a remote shell to a node > >> with a specific name. > >> > >> There is a node with name *master@REDACTED* on *myserver.l* > >> > >> There is no node with name *master@REDACTED > >> *on host *127.0.0.1* > >> > >> On Fri, Jun 28, 2019 at 9:47 AM Max Lapshin > >> wrote: > >> > >>> > >>> I have server with hostname myserver.l > >>> This hostname is in /etc/hosts and it is pingable > >>> > >>> I have master node running on it with: erl -name > >>> master@REDACTED > >>> > >>> When I write software, I don't know what will be the hostname, so > >>> I write shell script for connecting to shell: > >>> > >>> erl -name debug -remsh master@REDACTED > >>> > >>> It fails: > >>> > >>> Erlang/OTP 21 [erts-10.3.5.2] [source] [64-bit] [smp:8:8] > >>> [ds:8:8:10] [async-threads:1] [hipe] > >>> > >>> *** ERROR: Shell process terminated! (^G to start new job) *** > >>> > >>> When I provide hostname, it works: > >>> > >>> erl -name debug -remsh master@REDACTED > >>> Erlang/OTP 21 [erts-10.3.5.2] [source] [64-bit] [smp:8:8] > >>> [ds:8:8:10] [async-threads:1] [hipe] > >>> > >>> Eshell V10.3.5.2 (abort with ^G) > >>> (master@REDACTED)1> > >>> > >>> > >>> > >>> What is the proper way to deal with this situation? > >>> I can see the master node in epmd names, I can connect to it, I > >>> can pass interconnect protocol. > >>> But I do not understand, why remsh doesn't connect to it. > >>> > >>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >>> > >> _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -- Wir haften nicht f?r die korrekte Funktion der in dieser eMail enthaltenen Viren. We are not liable for correct function of the viruses in this email! :)