From mikpelinux@REDACTED Sun Feb 1 12:12:40 2015 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Sun, 1 Feb 2015 12:12:40 +0100 Subject: [erlang-bugs] =?utf-8?q?Conflicting_types_for_=E2=80=98ethr=5Fnat?= =?utf-8?q?ive=5Fdw=5Fatomic=5Ft=E2=80=99_on_armv7a-hardfloat-linux?= =?utf-8?q?-gnueabi?= In-Reply-To: References: Message-ID: <21710.2600.113459.280816@gargle.gargle.HOWL> Dirkjan Ochtman writes: > Hi there, > > I'm the package maintainer for Erlang on Gentoo. Recently, I got this > bug report: > > https://bugs.gentoo.org/show_bug.cgi?id=537570 > > Apparently, with gcc-4.8.4, there's this problem: > > In file included from ../include/internal/gcc/ethread.h:48:0, > from ../include/internal/ethread.h:368, > from beam/erl_threads.h:264, > from beam/erl_smp.h:27, > from beam/sys.h:434, > from beam/export.h:24, > from armv7a-hardfloat-linux-gnueabi/erl_pbifs.c:9: > ../include/internal/gcc/ethr_dw_atomic.h:81:3: error: conflicting > types for ?ethr_native_dw_atomic_t? > } ethr_native_dw_atomic_t; > > I looked around the source tree a bit, but since the erl_pbifs.c seems > to be generated by a Perl script and my Perl skills aren't so strong, > I couldn't really figure out the problem. > > Cheers, > > Dirkjan > > P.S. I'm not subscribed to erlang-bugs, please CC me on any replies. I believe there may be a conflict between the code paths for libatomic_ops and gcc's sync primitives. Try configuring with --with-libatomic_ops=no. /Mikael From sergej.jurecko@REDACTED Sun Feb 1 09:47:24 2015 From: sergej.jurecko@REDACTED (Sergej Jurecko) Date: Sun, 1 Feb 2015 09:47:24 +0100 Subject: [erlang-bugs] maps inside records Message-ID: <219FB3AA-2888-40B2-9FEC-88D7A287D8C1@gmail.com> Tested in 17.3 and 17.4 Incorrect behavior 1: -record(ev,{info}). test() -> test(#ev{info = #{type => dir, name => "My folder"}}). test(#ev{info = #{type := dir} = I} = E) -> io:format("E = ~p~n",[E]), io:format("E#event.info = ~p~n",[E#ev.info]), io:format("I = ~p~n",[I]). Calling test/0 will print: E = {ev,#{name => "My folder",type => dir}} E#event.info = #{type => dir} I = #{name => "My folder",type => dir} Why is E#event.info without name field? Incorrect behavior 2: test1() -> Z = #{type => dir,name => "My folder"}, io:format("Z = ~p ~n",[Z]), P = #ev{info = Z}, io:format("P = ~p ~n",[P]), test1(P). test1(#ev{info = #{type := dir, name := Name}} = E) -> io:format("E = ~p ~n",[E]), E#ev.info. Calling test1/0 will print: ecds:test1(). Z = #{name => "My folder",type => dir} P = {ev,#{name => "My folder",type => dir}} E = {ev,#{name => "My folder",type => dir}} ** exception error: bad argument in function ecds:test1/1 Exception happened in second header of test1, yet E was printed. Sergej From atill@REDACTED Sun Feb 1 12:19:07 2015 From: atill@REDACTED (Andy Till) Date: Sun, 01 Feb 2015 11:19:07 +0000 Subject: [erlang-bugs] eprof does not print results on short lived procs Message-ID: <54CE0BAB.7040702@mail.com> Hi When I spawn a process to run eprof, no results get printed. For example if this is run from the shell: Self = self(). eprof:start(). Eprof_fn = fun() -> eprof:start_profiling([Self]), timer:sleep(2000), eprof:stop_profiling(), eprof:analyze(total) end. Eprof_fn(). spawn(Eprof_fn). I would expect this to print out the results twice but only the profile run from the shell process prints out. Running this in a different orders does not help. This also affects RPC. I had a module containing a function similar to that above which would not print results to a file when called from a remote node. Cheers Andy From bjorn@REDACTED Mon Feb 2 08:04:47 2015 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 2 Feb 2015 08:04:47 +0100 Subject: [erlang-bugs] maps inside records In-Reply-To: <219FB3AA-2888-40B2-9FEC-88D7A287D8C1@gmail.com> References: <219FB3AA-2888-40B2-9FEC-88D7A287D8C1@gmail.com> Message-ID: On Sun, Feb 1, 2015 at 9:47 AM, Sergej Jurecko wrote: > Tested in 17.3 and 17.4 > > Incorrect behavior 1: > > -record(ev,{info}). > test() -> > test(#ev{info = #{type => dir, name => "My folder"}}). > test(#ev{info = #{type := dir} = I} = E) -> > io:format("E = ~p~n",[E]), > io:format("E#event.info = ~p~n",[E#ev.info]), > io:format("I = ~p~n",[I]). > > Calling test/0 will print: > E = {ev,#{name => "My folder",type => dir}} > E#event.info = #{type => dir} > I = #{name => "My folder",type => dir} > > Why is E#event.info without name field? > The behaviour is the expected. Records are compile-time construct that are translated to tuples. So at run-time the record is a tuple and will be printed as a tuple. > > Incorrect behavior 2: > test1() -> > Z = #{type => dir,name => "My folder"}, > io:format("Z = ~p ~n",[Z]), > P = #ev{info = Z}, > io:format("P = ~p ~n",[P]), > test1(P). > test1(#ev{info = #{type := dir, name := Name}} = E) -> > io:format("E = ~p ~n",[E]), > E#ev.info. > > Calling test1/0 will print: > > ecds:test1(). > Z = #{name => "My folder",type => dir} > P = {ev,#{name => "My folder",type => dir}} > E = {ev,#{name => "My folder",type => dir}} > ** exception error: bad argument > in function ecds:test1/1 > > Exception happened in second header of test1, yet E was printed. > An unsafe optimisation causes that strange behavior. The correction has already been merged to the 'maint' branch and will be included in the next maintenance release. /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From sergej.jurecko@REDACTED Mon Feb 2 08:23:44 2015 From: sergej.jurecko@REDACTED (=?UTF-8?Q?Sergej_Jure=C4=8Dko?=) Date: Mon, 2 Feb 2015 08:23:44 +0100 Subject: [erlang-bugs] maps inside records In-Reply-To: References: <219FB3AA-2888-40B2-9FEC-88D7A287D8C1@gmail.com> Message-ID: On Mon, Feb 2, 2015 at 8:04 AM, Bj?rn Gustavsson wrote: > On Sun, Feb 1, 2015 at 9:47 AM, Sergej Jurecko > wrote: > > Tested in 17.3 and 17.4 > > > > Incorrect behavior 1: > > > > -record(ev,{info}). > > test() -> > > test(#ev{info = #{type => dir, name => "My folder"}}). > > test(#ev{info = #{type := dir} = I} = E) -> > > io:format("E = ~p~n",[E]), > > io:format("E#event.info = ~p~n",[E#ev.info]), > > io:format("I = ~p~n",[I]). > > > > Calling test/0 will print: > > E = {ev,#{name => "My folder",type => dir}} > > E#event.info = #{type => dir} > > I = #{name => "My folder",type => dir} > > > > Why is E#event.info without name field? > > > > The behaviour is the expected. > > Records are compile-time construct that are translated > to tuples. So at run-time the record is a tuple and > will be printed as a tuple. > Yes but E#event.info value is not a record. It is a map which should contain #{type => dir, name => "My folder"}, yet when trying to extract that record we only get #{type => dir}. E#event.info should be the same as I, yet it is not. Sergej -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Mon Feb 2 09:06:33 2015 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 2 Feb 2015 09:06:33 +0100 Subject: [erlang-bugs] maps inside records In-Reply-To: References: <219FB3AA-2888-40B2-9FEC-88D7A287D8C1@gmail.com> Message-ID: Hi Sergej, Is the code you pasted in the mails correct? You have references to both #event and #ev -- maybe that's the cause? regards, Vlad On Mon, Feb 2, 2015 at 8:23 AM, Sergej Jure?ko wrote: > > > On Mon, Feb 2, 2015 at 8:04 AM, Bj?rn Gustavsson wrote: > >> On Sun, Feb 1, 2015 at 9:47 AM, Sergej Jurecko >> wrote: >> > Tested in 17.3 and 17.4 >> > >> > Incorrect behavior 1: >> > >> > -record(ev,{info}). >> > test() -> >> > test(#ev{info = #{type => dir, name => "My folder"}}). >> > test(#ev{info = #{type := dir} = I} = E) -> >> > io:format("E = ~p~n",[E]), >> > io:format("E#event.info = ~p~n",[E#ev.info]), >> > io:format("I = ~p~n",[I]). >> > >> > Calling test/0 will print: >> > E = {ev,#{name => "My folder",type => dir}} >> > E#event.info = #{type => dir} >> > I = #{name => "My folder",type => dir} >> > >> > Why is E#event.info without name field? >> > >> >> The behaviour is the expected. >> >> Records are compile-time construct that are translated >> to tuples. So at run-time the record is a tuple and >> will be printed as a tuple. >> > > > Yes but E#event.info value is not a record. It is a map which should > contain #{type => dir, name => "My folder"}, yet when trying to extract > that record we only get #{type => dir}. > > E#event.info should be the same as I, yet it is not. > > > Sergej > > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Mon Feb 2 09:14:29 2015 From: sergej.jurecko@REDACTED (=?UTF-8?Q?Sergej_Jure=C4=8Dko?=) Date: Mon, 2 Feb 2015 09:14:29 +0100 Subject: [erlang-bugs] maps inside records In-Reply-To: References: <219FB3AA-2888-40B2-9FEC-88D7A287D8C1@gmail.com> Message-ID: Oh right. The #event is just in the print string, not the actual code. The code compiles and runs. Sergej On Mon, Feb 2, 2015 at 9:06 AM, Vlad Dumitrescu wrote: > Hi Sergej, > > Is the code you pasted in the mails correct? You have references to both > #event and #ev -- maybe that's the cause? > > regards, > Vlad > > > On Mon, Feb 2, 2015 at 8:23 AM, Sergej Jure?ko > wrote: > >> >> >> On Mon, Feb 2, 2015 at 8:04 AM, Bj?rn Gustavsson >> wrote: >> >>> On Sun, Feb 1, 2015 at 9:47 AM, Sergej Jurecko >>> wrote: >>> > Tested in 17.3 and 17.4 >>> > >>> > Incorrect behavior 1: >>> > >>> > -record(ev,{info}). >>> > test() -> >>> > test(#ev{info = #{type => dir, name => "My folder"}}). >>> > test(#ev{info = #{type := dir} = I} = E) -> >>> > io:format("E = ~p~n",[E]), >>> > io:format("E#event.info = ~p~n",[E#ev.info]), >>> > io:format("I = ~p~n",[I]). >>> > >>> > Calling test/0 will print: >>> > E = {ev,#{name => "My folder",type => dir}} >>> > E#event.info = #{type => dir} >>> > I = #{name => "My folder",type => dir} >>> > >>> > Why is E#event.info without name field? >>> > >>> >>> The behaviour is the expected. >>> >>> Records are compile-time construct that are translated >>> to tuples. So at run-time the record is a tuple and >>> will be printed as a tuple. >>> >> >> >> Yes but E#event.info value is not a record. It is a map which should >> contain #{type => dir, name => "My folder"}, yet when trying to extract >> that record we only get #{type => dir}. >> >> E#event.info should be the same as I, yet it is not. >> >> >> Sergej >> >> _______________________________________________ >> erlang-bugs mailing list >> erlang-bugs@REDACTED >> http://erlang.org/mailman/listinfo/erlang-bugs >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Mon Feb 2 10:10:49 2015 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 2 Feb 2015 10:10:49 +0100 Subject: [erlang-bugs] maps inside records In-Reply-To: References: <219FB3AA-2888-40B2-9FEC-88D7A287D8C1@gmail.com> Message-ID: Sorry, you're right, I just skimmed your code... This is certainly a bug, as accessing the info field (in any way) returns a truncated map. regards, Vlad On Mon, Feb 2, 2015 at 9:14 AM, Sergej Jure?ko wrote: > Oh right. The #event is just in the print string, not the actual code. The > code compiles and runs. > > > Sergej > > On Mon, Feb 2, 2015 at 9:06 AM, Vlad Dumitrescu > wrote: > >> Hi Sergej, >> >> Is the code you pasted in the mails correct? You have references to both >> #event and #ev -- maybe that's the cause? >> >> regards, >> Vlad >> >> >> On Mon, Feb 2, 2015 at 8:23 AM, Sergej Jure?ko >> wrote: >> >>> >>> >>> On Mon, Feb 2, 2015 at 8:04 AM, Bj?rn Gustavsson >>> wrote: >>> >>>> On Sun, Feb 1, 2015 at 9:47 AM, Sergej Jurecko < >>>> sergej.jurecko@REDACTED> wrote: >>>> > Tested in 17.3 and 17.4 >>>> > >>>> > Incorrect behavior 1: >>>> > >>>> > -record(ev,{info}). >>>> > test() -> >>>> > test(#ev{info = #{type => dir, name => "My folder"}}). >>>> > test(#ev{info = #{type := dir} = I} = E) -> >>>> > io:format("E = ~p~n",[E]), >>>> > io:format("E#event.info = ~p~n",[E#ev.info]), >>>> > io:format("I = ~p~n",[I]). >>>> > >>>> > Calling test/0 will print: >>>> > E = {ev,#{name => "My folder",type => dir}} >>>> > E#event.info = #{type => dir} >>>> > I = #{name => "My folder",type => dir} >>>> > >>>> > Why is E#event.info without name field? >>>> > >>>> >>>> The behaviour is the expected. >>>> >>>> Records are compile-time construct that are translated >>>> to tuples. So at run-time the record is a tuple and >>>> will be printed as a tuple. >>>> >>> >>> >>> Yes but E#event.info value is not a record. It is a map which should >>> contain #{type => dir, name => "My folder"}, yet when trying to extract >>> that record we only get #{type => dir}. >>> >>> E#event.info should be the same as I, yet it is not. >>> >>> >>> Sergej >>> >>> _______________________________________________ >>> erlang-bugs mailing list >>> erlang-bugs@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-bugs >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn@REDACTED Mon Feb 2 14:34:45 2015 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 2 Feb 2015 14:34:45 +0100 Subject: [erlang-bugs] maps inside records In-Reply-To: References: <219FB3AA-2888-40B2-9FEC-88D7A287D8C1@gmail.com> Message-ID: On Mon, Feb 2, 2015 at 8:23 AM, Sergej Jure?ko wrote: > [...] > Yes but E#event.info value is not a record. It is a map which should contain > #{type => dir, name => "My folder"}, yet when trying to extract that record > we only get #{type => dir}. > > E#event.info should be the same as I, yet it is not. > Thanks for the clarification. The bug is fixed in the 'maint' branch and the correction will be included in the next maintenance release. /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From Aleksander.Nycz@REDACTED Tue Feb 3 10:16:50 2015 From: Aleksander.Nycz@REDACTED (Aleksander Nycz) Date: Tue, 03 Feb 2015 10:16:50 +0100 Subject: [erlang-bugs] xmerl - how to clean xsd_state rec Message-ID: <54D09202.8060104@comarch.pl> Hello, Function xmerl_xsd:process_schema returns {ok, State} where State is the global state of the validator. It is representated by the #xsd_state{} record. Record xsd_state contains field 'table' that is ets tid. I must change Schema and call function xmerl_xsd:process_schema once again to obtain new global state. And I'm wondering how to clean old state (eg. delete old ets) ... Regards Aleksander Nycz -- Aleksander Nycz Telco_021 BSS R&D Comarch SA Phone: +48 17 785 59 09 Mobile: +48 691 464 275 website: www.comarch.pl -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4916 bytes Desc: Kryptograficzna sygnatura S/MIME URL: From tuncer.ayaz@REDACTED Tue Feb 3 23:24:06 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 3 Feb 2015 23:24:06 +0100 Subject: [erlang-bugs] (unconditional) rebuild caused by gen_git_version In-Reply-To: References: <53048D97.6000301@erlang.org> Message-ID: On Wed, Feb 26, 2014 at 7:31 PM, Tuncer Ayaz wrote: > On Wed, Feb 19, 2014 at 11:55 AM, Lukas Larsson wrote: > > Hello, > > > > When implementing this I noticed it as well and tried to do > > something about it. Alas it proved quite difficult and I had to > > settle on a compromise where it does a rebuild once. i.e. > > > > commit change > > make > > LD > > make > > LD > > make > > make > > make > > make > > With GNU Make 4.0, gcc 4.8.2, and binutils 2.24: > # unmodified tree since last build > make > LD > make > LD > make > LD > make > LD > make > LD > make > LD > > > At least that is the way it works for me on Ubuntu Linux. > > > > If someone else feels like having a go at making it work they are > > most welcome. Lukas, your fix from November (9adde956) has resolved this for me. Thanks! From tuncer.ayaz@REDACTED Tue Feb 3 23:25:19 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 3 Feb 2015 23:25:19 +0100 Subject: [erlang-bugs] UBsan int overflows In-Reply-To: <5475E034.3030100@erlang.org> References: <541FF3B5.8000304@erlang.org> <5475E034.3030100@erlang.org> Message-ID: On Wed, Nov 26, 2014 at 3:14 PM, Lukas Larsson wrote: > > On 26/11/14 12:29, Tuncer Ayaz wrote: >> >> On Tue, Sep 23, 2014 at 11:05 AM, Tuncer Ayaz wrote: >>> >>> On Mon, Sep 22, 2014 at 12:02 PM, Lukas Larsson wrote: >>>> >>>> On 17/09/14 19:23, Tuncer Ayaz wrote: >>>>> >>>>> On Wed, Sep 17, 2014 at 2:33 PM, Tuncer Ayaz wrote: >>>>>> >>>>>> Building OTP-17.3 with --enable-sanitizers=undefined, you can see >>>>>> the following during the build of lib/wx/examples/demo: >>>>>> ERLC ex_aui.beam >>>>>> beam/bif.c:2828:5: runtime error: signed integer overflow: >>>>>> 426141286 * 10 cannot be represented in type 'int' >>>>>> >>>>>> I thought this was also fixed. >>>> >>>> As did I. I wonder where that fix went... I've created a new fix >>>> for it. >>>> >>>>> More during dialyzer --build_plt: >>>>> beam/erl_process.c:9024:2: runtime error: signed integer overflow: >>>>> 2147482509 + 2001 cannot be represented in type 'int' >>>>> beam/erl_process.c:4908:35: runtime error: signed integer overflow: >>>>> 3984 * 625175 cannot be represented in type 'int' >>>>> >>>> I've noticed these as well and a couple of other ones in >>>> erl_process.c. I've added them to our backlog to fix. >>> >>> Thanks, and please keep us posted. >> >> What's the status? > > > bif.c has been fixed, erl_process.c has not been fixed yet. Any status update? From tony@REDACTED Sun Feb 8 14:32:58 2015 From: tony@REDACTED (Tony Rogvall) Date: Sun, 8 Feb 2015 14:32:58 +0100 Subject: [erlang-bugs] ssh known_hosts Message-ID: <2AE4E1A1-3D1A-4D9B-A32E-AB5EB8820835@rogvall.se> Hi! I just noticed that Erlang ssh does not recognize the (newish) format: [
]: nor the hashed format: |1|| The first is convenient to understand when you have Erlang clients that connect to ssh servers on ports other than 22, which is kind of normal for Erlang ssh daemons. The hashed format just up the security a bit. Accept a PL ? BR /Tony -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From Ingela.Anderton.Andin@REDACTED Wed Feb 11 09:44:48 2015 From: Ingela.Anderton.Andin@REDACTED (Ingela Anderton Andin) Date: Wed, 11 Feb 2015 09:44:48 +0100 Subject: [erlang-bugs] Fwd: Re: ssh known_hosts In-Reply-To: <54D86A46.1020207@ericsson.com> References: <54D86A46.1020207@ericsson.com> Message-ID: <54DB1680.9050907@ericsson.com> -------- Forwarded Message -------- Subject: Re: [erlang-bugs] ssh known_hosts Date: Mon, 9 Feb 2015 09:05:26 +0100 From: Ingela Anderton Andin To: Tony Rogvall Hi! On 02/08/2015 02:32 PM, Tony Rogvall wrote: > Hi! > > I just noticed that Erlang ssh does not recognize the (newish) format: > > [
]: > > nor the hashed format: > > |1|| > > The first is convenient to understand when you have Erlang clients > that connect to ssh servers on ports other than 22, > which is kind of normal for Erlang ssh daemons. > > The hashed format just up the security a bit. > > Accept a PL ? > Yes pullrequest please :) Note that it might be in the module pubkey_ssh in public_key that would be the place to add it. Regards Ingela Erlang/OTP team - Ericsson AB From james@REDACTED Sun Feb 15 17:36:21 2015 From: james@REDACTED (James Fish) Date: Sun, 15 Feb 2015 16:36:21 +0000 Subject: [erlang-bugs] Dialyzer crashes when checking a PLT (via Erlang API) returns warnings Message-ID: Hi, Dialyzer crashes when checking a plt (before running the main analysis type) if a warning is found when using dialyzer:run/1. Please find an archive attached with a minimal example and run the escript plt_check (with OTP >= 17.0 as no_unknown is used to hide unrelated output). In OTP 17.4 it will crash with the following output: Error: {case_clause,{ok,2}} Stacktrace: [{dialyzer,run,1,[{file,"dialyzer.erl"},{line,167}]}, {plt_check__escript__1424__17011__604940,run,1, [{file,"./plt_check"},{line,21}]}, {escript,run,2,[{file,"escript.erl"},{line,752}]}, {escript,start,1,[{file,"escript.erl"},{line,276}]}, {init,start_it,1,[]}, {init,start_em,1,[]}] The expected output is: plt_check.erl:6: Call to missing or unexported function erlang:'not_exported!'/0 On a related note, it would appear that using plt_check checks the plt twice. Hopefully this can be resolved at the same time. Regards, James -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: plt_check.tar.gz Type: application/x-gzip Size: 726 bytes Desc: not available URL: From klas.johansson@REDACTED Tue Feb 17 20:23:20 2015 From: klas.johansson@REDACTED (Klas Johansson) Date: Tue, 17 Feb 2015 20:23:20 +0100 Subject: [erlang-bugs] `undefined parse transform' parse transforms which call non-existing modules Message-ID: Hi, I noticed that a warning has been added to the compiler which says `undefined parse transform' instead of throwing a (potentially large) parse tree in the face of the user. Nice! This is the commit: dba5396 compile: Give a friendler error message if a parse transform cannot be found I discovered that is has an unfortunate side-effect: If an existing parse transform calls a non-existing module, the check will misinterpret this and print the `undefined parse transform' error. Tested with compiler-5.0.3 on Erlang/OTP 17.4. Example parse transform: -module(x). -compile(export_all). parse_transform(_Forms, _Opts) -> non:existing(). Cheers, Klas -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.norell@REDACTED Thu Feb 19 10:35:27 2015 From: ulf.norell@REDACTED (Ulf Norell) Date: Thu, 19 Feb 2015 10:35:27 +0100 Subject: [erlang-bugs] Internal consitency check failure Message-ID: Messing around with my random program generator again I got the following message: 1> c(bug). bug: function f/0+18: Internal consistency check failed - please report this bug. Instruction: {gc_bif,'+',{f,0},1,[{x,0},{atom,hockey}],{x,0}} Error: unknown_catch_try_state: Here's the (somewhat contrived) program -module(bug). -compile(export_all). f() -> receive _ -> (b = fun() -> ok end) + hockey, +x after 0 -> ok end, try (a = fun() -> ok end) + hockey, + y catch _ -> ok end. There are a number of intriguing things here - Breaking the 'hockey' line anywhere gets rid of the bug(!!). - The two 'hockey' atoms need to be the same (but not necessarily sports related). - I tried reasonably hard to make the program smaller, but failed. I get the same error on all distributions I tried (R14 up to R17.4.1). Master didn't build for me. / Ulf -------------- next part -------------- An HTML attachment was scrubbed... URL: From rich.neswold@REDACTED Wed Feb 18 21:55:53 2015 From: rich.neswold@REDACTED (Rich Neswold) Date: Wed, 18 Feb 2015 14:55:53 -0600 Subject: [erlang-bugs] Crash when using -on_load() directive. Message-ID: Hello, I'm trying to use the -on_load() directive on NetBSD/amd64, but I'm having problems. Even this simple test file (test.erl) fails: -module(test). -on_load(load_my_nifs/0). load_my_nifs() -> io:format("Loaded module.~n"), ok. Try to compile as byte-code and then native: $ erl Erlang/OTP 17 (erts-6.3) [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] Eshell V6.3 (abort with ^G) 1> c(test). Loaded module. {ok,test} 2> c(test, [native]). EXITED with reason {'trans_fun/2',on_load} @hipe_beam_to_icode:1174 =ERROR REPORT==== 18-Feb-2015::13:42:46 === Error in process <0.37.0> with exit value: {{badmatch,{'EXIT',{{hipe_beam_to_icode,1174,{'trans_fun/2',on_load}},[{hipe_beam_to_icode,trans_fun,2,[]},{hipe_beam_to_icode,trans_fun,2,[]},{hipe_beam_to_icode,trans_mfa_code,5,[]},{hipe_beam_to_icode,trans_beam_function_chunk... test.erl: internal error in native_compile; crash reason: {badmatch,{'EXIT',{{hipe_beam_to_icode,1174,{'trans_fun/2',on_load}}, [{hipe_beam_to_icode,trans_fun,2,[]}, {hipe_beam_to_icode,trans_fun,2,[]}, {hipe_beam_to_icode,trans_mfa_code,5,[]}, {hipe_beam_to_icode,trans_beam_function_chunk,2,[]}, {hipe_beam_to_icode,'-module/2-lc$^1/1-1-',2,[]}, {hipe,get_beam_icode,4,[]}, {hipe,'-run_compiler_1/3-fun-0-',4,[]}, {hipe_icode_callgraph,get_called_modules,1,[]}]}}} in function hipe:get_beam_icode/4 in call from hipe:'-run_compiler_1/3-fun-0-'/4 in call from hipe_icode_callgraph:get_called_modules/1 error 3> This also happens on an R16B03-1 installation. The documentation doesn't mention the directive is limited to byte-code compiles. But even if it was, the native compiler should print an error instead of crashing. Are other platforms having this problem? -- Rich -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Thu Feb 19 13:01:05 2015 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Thu, 19 Feb 2015 13:01:05 +0100 Subject: [erlang-bugs] Crash when using -on_load() directive. In-Reply-To: References: Message-ID: <21733.53377.513525.543098@gargle.gargle.HOWL> Rich Neswold writes: > Hello, > > I'm trying to use the -on_load() directive on NetBSD/amd64, but I'm having > problems. Even this simple test file (test.erl) fails: > > -module(test). > -on_load(load_my_nifs/0). > > load_my_nifs() -> > io:format("Loaded module.~n"), > ok. > > Try to compile as byte-code and then native: > > $ erl > Erlang/OTP 17 (erts-6.3) [source] [64-bit] [smp:8:8] [async-threads:10] > [hipe] [kernel-poll:false] > > Eshell V6.3 (abort with ^G) > 1> c(test). > Loaded module. > {ok,test} > 2> c(test, [native]). > EXITED with reason {'trans_fun/2',on_load} > @hipe_beam_to_icode:1174 > > =ERROR REPORT==== 18-Feb-2015::13:42:46 === > Error in process <0.37.0> with exit value: > {{badmatch,{'EXIT',{{hipe_beam_to_icode,1174,{'trans_fun/2',on_load}},[{hipe_beam_to_icode,trans_fun,2,[]},{hipe_beam_to_icode,trans_fun,2,[]},{hipe_beam_to_icode,trans_mfa_code,5,[]},{hipe_beam_to_icode,trans_beam_function_chunk... > > test.erl: internal error in native_compile; > crash > reason: {badmatch,{'EXIT',{{hipe_beam_to_icode,1174,{'trans_fun/2',on_load}}, > [{hipe_beam_to_icode,trans_fun,2,[]}, > {hipe_beam_to_icode,trans_fun,2,[]}, > {hipe_beam_to_icode,trans_mfa_code,5,[]}, > {hipe_beam_to_icode,trans_beam_function_chunk,2,[]}, > {hipe_beam_to_icode,'-module/2-lc$^1/1-1-',2,[]}, > {hipe,get_beam_icode,4,[]}, > {hipe,'-run_compiler_1/3-fun-0-',4,[]}, > {hipe_icode_callgraph,get_called_modules,1,[]}]}}} > > in function hipe:get_beam_icode/4 > in call from hipe:'-run_compiler_1/3-fun-0-'/4 > in call from hipe_icode_callgraph:get_called_modules/1 > error > 3> > > > This also happens on an R16B03-1 installation. The documentation doesn't > mention the directive is limited to byte-code compiles. But even if it was, > the native compiler should print an error instead of crashing. > > Are other platforms having this problem? It's a generic problem affecting all platforms. Code loading is complex, adding native code to the mix makes it more so, and then on_load adds yet another dimension of complex interactions. Yes, the compiler should reject the code gracefully instead. From bjorn@REDACTED Thu Feb 19 14:35:17 2015 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 19 Feb 2015 14:35:17 +0100 Subject: [erlang-bugs] Internal consitency check failure In-Reply-To: References: Message-ID: On Thu, Feb 19, 2015 at 10:35 AM, Ulf Norell wrote: > Messing around with my random program generator again I got the following > message: > > 1> c(bug). > > bug: function f/0+18: > > Internal consistency check failed - please report this bug. > > Instruction: {gc_bif,'+',{f,0},1,[{x,0},{atom,hockey}],{x,0}} > > Error: unknown_catch_try_state: > Here is a correction: https://github.com/bjorng/otp/commit/d4fbd1f61a1d346415e45a46dbe5e3cdb4c67d49 > > Here's the (somewhat contrived) program > > -module(bug). > -compile(export_all). > f() -> > receive _ -> (b = fun() -> ok end) > + hockey, +x after 0 -> ok end, try (a = fun() -> ok end) + hockey, + > y catch _ -> ok end. > > There are a number of intriguing things here > > - Breaking the 'hockey' line anywhere gets rid of the bug(!!). The problematic optimimization was not done when line numbers didn't match. /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From n.oxyde@REDACTED Sat Feb 21 01:44:39 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Sat, 21 Feb 2015 01:44:39 +0100 Subject: [erlang-bugs] Internal consitency check failure In-Reply-To: References: Message-ID: Le 19 f?vr. 2015 ? 14:35, Bj?rn Gustavsson a ?crit : > On Thu, Feb 19, 2015 at 10:35 AM, Ulf Norell wrote: >> Messing around with my random program generator again I got the following >> message: >> >> 1> c(bug). >> >> bug: function f/0+18: >> >> Internal consistency check failed - please report this bug. >> >> Instruction: {gc_bif,'+',{f,0},1,[{x,0},{atom,hockey}],{x,0}} >> >> Error: unknown_catch_try_state: >> > > Here is a correction: > > https://github.com/bjorng/otp/commit/d4fbd1f61a1d346415e45a46dbe5e3cdb4c67d49 > >> >> Here's the (somewhat contrived) program >> >> -module(bug). >> -compile(export_all). >> f() -> >> receive _ -> (b = fun() -> ok end) >> + hockey, +x after 0 -> ok end, try (a = fun() -> ok end) + hockey, + >> y catch _ -> ok end. >> >> There are a number of intriguing things here >> >> - Breaking the 'hockey' line anywhere gets rid of the bug(!!). > > The problematic optimimization was not done when line > numbers didn't match. Maybe some compiler tests should also be run with +no_line_info or something? From essen@REDACTED Sat Feb 21 14:42:00 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sat, 21 Feb 2015 14:42:00 +0100 Subject: [erlang-bugs] SSL suite test failure Message-ID: <54E88B28.5020203@ninenines.eu> Had the following failure while running the SSL test suite: - - - - - - - - - - - - - - - - - - - - - - - - - - ssl_to_openssl_SUITE:basic_erlang_server_openssl_client failed on line 258 Reason: {test_case_failed,{{expected,{<0.14692.0>,ok}}, {got, {<0.14692.0>, {badrpc, {'EXIT', {function_clause, [{ssl,connection_info, [{error,{tls_alert,"handshake failure"}}], [{file,"ssl.erl"},{line,293}]}, {ssl_to_openssl_SUITE,erlang_ssl_receive,2, [{file,"ssl_to_openssl_SUITE.erl"},{line,1247}]}, {rpc,local_call,3,[{file,"rpc.erl"},{line,329}]}, {ssl_test_lib,do_run_server,3, [{file,"ssl_test_lib.erl"},{line,92}]}]}}}}}}} - - - - - - - - - - - - - - - - - - - - - - - - - - Only had the failure the first time, it worked on subsequent tries, so I am not sure what triggered it. -- Lo?c Hoguin http://ninenines.eu From bjorn@REDACTED Mon Feb 23 07:48:07 2015 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 23 Feb 2015 07:48:07 +0100 Subject: [erlang-bugs] Internal consitency check failure In-Reply-To: References: Message-ID: On Sat, Feb 21, 2015 at 1:44 AM, Anthony Ramine wrote: > Le 19 f?vr. 2015 ? 14:35, Bj?rn Gustavsson a ?crit : > >> The problematic optimimization was not done when line >> numbers didn't match. > > Maybe some compiler tests should also be run with +no_line_info or something? > no_line_info removes the line numbers in the beam_clean pass, which is run after most of the optimisation passes. This is done on purpose so that the optimisation passes can assume that line instructions are always there. /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From nico.kruber@REDACTED Mon Feb 23 10:17:29 2015 From: nico.kruber@REDACTED (Nico Kruber) Date: Mon, 23 Feb 2015 10:17:29 +0100 Subject: [erlang-bugs] Common Test problems with includes since 17.4 In-Reply-To: <15459112.HWeEvvLCd6@nico-pc.lan> References: <5534528.FObQ9azk4v@csr-pc40.zib.de> <54C02E38.9070102@erlang.org> <15459112.HWeEvvLCd6@nico-pc.lan> Message-ID: <11220467.UuGYjJe8pI@csr-pc40.zib.de> Hi Peter, is there a fix yet and will it make it into the next release? Thanks Nico On Friday 23 Jan 2015 12:50:09 you wrote: > Thank you Peter, > please tell my when you commit the patch and I'll integrate it into the > openSUSE packages so I can run tests again > (at the moment I'm symlinking all hrl files from my external include > directory into the test directory as a workaround) > > > Nico > > On Wednesday 21 Jan 2015 23:54:48 Peter Andersson wrote: > > Hi Nico, > > > > I know what's causing the error now. There's a problem with some > > recently introduced code in the erl->html converter in Common Test. > > We'll have this fixed shortly! > > > > Best, > > Peter > > > > Ericsson AB, Erlang/OTP > > > > On 2015-01-20 14:24, Nico Kruber wrote: > > > Since Erlang 17.4 Common Test seems to have problems with included .hrl > > > files outside of the folder the unit test resides in (even when passing > > > the appropriate "include" option). The make process succeeds but when > > > the > > > test is actually run, a badarg is thrown: > > > > > > EXIT, reason {{badarg,{tree,macro,{attr,0,[],none},{macro, > > > {var,0,'RT'},none}}}, > > > [{erl_syntax,concrete,1,[{file,"erl_syntax.erl"},{line,6188}]}, > > > > > > {erl_syntax,concrete_list,1, [{file,"erl_syntax.erl"},{line,6192}]}, > > > {erl_syntax,concrete_list,1, [{file,"erl_syntax.erl"},{line,6192}]}, > > > {erl_syntax,concrete,1,[{file,"erl_syntax.erl"},{line,6162}]}, > > > {erl_syntax,concrete,1,[{file,"erl_syntax.erl"},{line,6159}]}, > > > {erl_syntax,concrete_list,1, [{file,"erl_syntax.erl"},{line,6192}]}, > > > {erl_syntax,concrete_list,1, [{file,"erl_syntax.erl"},{line,6192}]}, > > > {erl_syntax,concrete,1,[{file,"erl_syntax.erl"},{line,6162}]}]} > > > > > > == Steps to reproduce === > > > > > > * copy the attached test_SUITE.erl into a test folder and put > > > test_SUITE.hrl into a sub-folder named "include" > > > * run any of these two commands: > > > ct_run -include "include" -suite test_SUITE > > > CT_INCLUDE_PATH=include ct_run -suite test_SUITE > > > > > > => if the .hrl file is put into the test folder, everything is fine > > > though > > > > > > > > > > > > Nico -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: This is a digitally signed message part. URL: From Ingela.Anderton.Andin@REDACTED Mon Feb 23 11:38:09 2015 From: Ingela.Anderton.Andin@REDACTED (Ingela Anderton Andin) Date: Mon, 23 Feb 2015 11:38:09 +0100 Subject: [erlang-bugs] SSL suite test failure In-Reply-To: <54E88B28.5020203@ninenines.eu> References: <54E88B28.5020203@ninenines.eu> Message-ID: <54EB0311.1070707@ericsson.com> Hi! It might be a timing issue in the test suite. As the test suite always spawns the openssl server and waits for it to come up, but we have not found a perfect way of telling that it is up. If the server is not listning yet when we try to connect the handshake will of course fail making the test case fail. However I have not seen this in a very long time. Regards Ingela On 02/21/2015 02:42 PM, Lo?c Hoguin wrote: > Had the following failure while running the SSL test suite: > > - - - - - - - - - - - - - - - - - - - - - - - - - - > ssl_to_openssl_SUITE:basic_erlang_server_openssl_client failed on line 258 > Reason: {test_case_failed,{{expected,{<0.14692.0>,ok}}, > {got, > {<0.14692.0>, > {badrpc, > {'EXIT', > {function_clause, > [{ssl,connection_info, > [{error,{tls_alert,"handshake failure"}}], > [{file,"ssl.erl"},{line,293}]}, > {ssl_to_openssl_SUITE,erlang_ssl_receive,2, > > [{file,"ssl_to_openssl_SUITE.erl"},{line,1247}]}, > {rpc,local_call,3,[{file,"rpc.erl"},{line,329}]}, > {ssl_test_lib,do_run_server,3, > [{file,"ssl_test_lib.erl"},{line,92}]}]}}}}}}} > - - - - - - - - - - - - - - - - - - - - - - - - - - > > Only had the failure the first time, it worked on subsequent tries, so I > am not sure what triggered it. > From jonas.boberg@REDACTED Mon Feb 23 11:52:36 2015 From: jonas.boberg@REDACTED (Jonas Boberg) Date: Mon, 23 Feb 2015 19:52:36 +0900 Subject: [erlang-bugs] SSL suite test failure In-Reply-To: <54EB0311.1070707@ericsson.com> References: <54E88B28.5020203@ninenines.eu> <54EB0311.1070707@ericsson.com> Message-ID: <15B1E503-8F7E-432F-85B6-B3623A3B97DC@nomasystems.com> Hi, To wait until the server is up, I suggest that the test case setup repeatedly tries to initiate a TCP connection (with a low connection timeout). When a TCP connection attempt has succeeded, the test case can progress. Regards Jonas > On 23 Feb 2015, at 19:38, Ingela Anderton Andin wrote: > > It might be a timing issue in the test suite. As the test suite always spawns the openssl server and waits for it to come up, but we have not found a perfect way of telling that it is up. From jonas.boberg@REDACTED Mon Feb 23 11:52:36 2015 From: jonas.boberg@REDACTED (Jonas Boberg) Date: Mon, 23 Feb 2015 19:52:36 +0900 Subject: [erlang-bugs] SSL suite test failure In-Reply-To: <54EB0311.1070707@ericsson.com> References: <54E88B28.5020203@ninenines.eu> <54EB0311.1070707@ericsson.com> Message-ID: Hi, To wait until the server is up, I suggest that the test case setup repeatedly tries to initiate a TCP connection (with a low connection timeout). When a TCP connection attempt has succeeded, the test case can progress. Regards Jonas > On 23 Feb 2015, at 19:38, Ingela Anderton Andin wrote: > > It might be a timing issue in the test suite. As the test suite always spawns the openssl server and waits for it to come up, but we have not found a perfect way of telling that it is up. From Ingela.Anderton.Andin@REDACTED Mon Feb 23 11:56:39 2015 From: Ingela.Anderton.Andin@REDACTED (Ingela Anderton Andin) Date: Mon, 23 Feb 2015 11:56:39 +0100 Subject: [erlang-bugs] SSL suite test failure In-Reply-To: References: <54E88B28.5020203@ninenines.eu> <54EB0311.1070707@ericsson.com> Message-ID: <54EB0767.3030807@ericsson.com> Hi! On 02/23/2015 11:52 AM, Jonas Boberg wrote: > Hi, > > To wait until the server is up, I suggest that the test case setup repeatedly tries to initiate a TCP connection (with a low connection timeout). > When a TCP connection attempt has succeeded, the test case can progress. That sounds like a good idea, I think it could be an improvement over the current approach. Regards Ingela > Regards > Jonas > >> On 23 Feb 2015, at 19:38, Ingela Anderton Andin wrote: >> >> It might be a timing issue in the test suite. As the test suite always spawns the openssl server and waits for it to come up, but we have not found a perfect way of telling that it is up. > From mikpelinux@REDACTED Mon Feb 23 12:30:26 2015 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Mon, 23 Feb 2015 12:30:26 +0100 Subject: [erlang-bugs] hipe_mfait_lock In-Reply-To: References: <54B698EA.9090701@erix.ericsson.se> Message-ID: <21739.3922.436853.982577@gargle.gargle.HOWL> Louis-Philippe Gauthier writes: > Hi Sverker, > That's not a bad idea, I'll try that approach! > > Thanks, > LP > > On Thu, Jan 15, 2015 at 12:27 AM, Sverker Eriksson < > sverker.eriksson@REDACTED> wrote: > > > One quite simple improvement is to change the hipe_mfait_lock to be a > > read-write mutex (erts_smp_rwmtx_t). > > And then do read-lock at lookup and exclusive write-lock only when > > inserting a new entry in the table. > > > > I don't think that requires much knowledge about the VM to pull off ;-) . > > > > /Sverker, Erlang/OTP > > > > > > > > > > On 01/14/2015 12:34 AM, Louis-Philippe Gauthier wrote: > > > > Hi erlang-bugs, > > I couple of months ago I tried running our full application using HiPE. I > > ran into several issues, some manageable, some not... The most problematic > > issue was due to locking, specifically, the hipe_mfait_lock (you can see > > the lock counter output in the gist bellow). > > https://gist.github.com/lpgauth/2b3220f4bceeed6f62d0 > > > > Looking at the code it's obvious that this is a known problem... the > > following comment was added when the lock was added in 2010. > > > > "XXX: Redesign apply et al to avoid those updates."https://github.com/erlang/otp/blob/maint/erts/emulator/hipe/hipe_bif0.c#L1218 > > > > Unfortunately, I'm don't know the runtime enough to start patching it... so > > instead I'm reporting it. Please try the attached patch (against 17.4, should apply to older versions too by I haven't checked), and let me know if it makes a noticeable improvement to your use case. Run-time apply:s now do their lookups using a shared read-lock, and only if their lookups fail do they re-lock with an exclusive write-lock in order to insert new data. Module loading functions always take exclusive locks, which matches the current behaviour. /Mikael -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: otp17-hipe-mfait-rwlock.patch URL: From louis-philippe.gauthier@REDACTED Mon Feb 23 15:28:22 2015 From: louis-philippe.gauthier@REDACTED (Louis-Philippe Gauthier) Date: Mon, 23 Feb 2015 09:28:22 -0500 Subject: [erlang-bugs] hipe_mfait_lock In-Reply-To: <21739.3922.436853.982577@gargle.gargle.HOWL> References: <54B698EA.9090701@erix.ericsson.se> <21739.3922.436853.982577@gargle.gargle.HOWL> Message-ID: Hi Mikael, Awesome work, I'll give the patch a try this week. Thanks, LP On Mon, Feb 23, 2015 at 6:30 AM, Mikael Pettersson wrote: > Louis-Philippe Gauthier writes: > > Hi Sverker, > > That's not a bad idea, I'll try that approach! > > > > Thanks, > > LP > > > > On Thu, Jan 15, 2015 at 12:27 AM, Sverker Eriksson < > > sverker.eriksson@REDACTED> wrote: > > > > > One quite simple improvement is to change the hipe_mfait_lock to be a > > > read-write mutex (erts_smp_rwmtx_t). > > > And then do read-lock at lookup and exclusive write-lock only when > > > inserting a new entry in the table. > > > > > > I don't think that requires much knowledge about the VM to pull off > ;-) . > > > > > > /Sverker, Erlang/OTP > > > > > > > > > > > > > > > On 01/14/2015 12:34 AM, Louis-Philippe Gauthier wrote: > > > > > > Hi erlang-bugs, > > > I couple of months ago I tried running our full application using > HiPE. I > > > ran into several issues, some manageable, some not... The most > problematic > > > issue was due to locking, specifically, the hipe_mfait_lock (you can > see > > > the lock counter output in the gist bellow). > > > https://gist.github.com/lpgauth/2b3220f4bceeed6f62d0 > > > > > > Looking at the code it's obvious that this is a known problem... the > > > following comment was added when the lock was added in 2010. > > > > > > "XXX: Redesign apply et al to avoid those updates." > https://github.com/erlang/otp/blob/maint/erts/emulator/hipe/hipe_bif0.c#L1218 > > > > > > Unfortunately, I'm don't know the runtime enough to start patching > it... so > > > instead I'm reporting it. > > Please try the attached patch (against 17.4, should apply to older > versions too by I haven't checked), and let me know if it makes a > noticeable improvement to your use case. > > Run-time apply:s now do their lookups using a shared read-lock, and > only if their lookups fail do they re-lock with an exclusive write-lock > in order to insert new data. Module loading functions always take > exclusive locks, which matches the current behaviour. > > /Mikael > > > --- otp_src_17.4/erts/emulator/hipe/hipe_bif0.c.~1~ 2014-12-09 > 21:11:07.000000000 +0100 > +++ otp_src_17.4/erts/emulator/hipe/hipe_bif0.c 2015-02-22 > 21:32:55.361884741 +0100 > @@ -1217,22 +1217,32 @@ static struct { > * they create a new stub for the mfa, which forces locking. > * XXX: Redesign apply et al to avoid those updates. > */ > - erts_smp_mtx_t lock; > + erts_smp_rwmtx_t lock; > } hipe_mfa_info_table; > > static inline void hipe_mfa_info_table_init_lock(void) > { > - erts_smp_mtx_init(&hipe_mfa_info_table.lock, "hipe_mfait_lock"); > + erts_smp_rwmtx_init(&hipe_mfa_info_table.lock, "hipe_mfait_lock"); > } > > -static inline void hipe_mfa_info_table_lock(void) > +static inline void hipe_mfa_info_table_rlock(void) > { > - erts_smp_mtx_lock(&hipe_mfa_info_table.lock); > + erts_smp_rwmtx_rlock(&hipe_mfa_info_table.lock); > } > > -static inline void hipe_mfa_info_table_unlock(void) > +static inline void hipe_mfa_info_table_runlock(void) > { > - erts_smp_mtx_unlock(&hipe_mfa_info_table.lock); > + erts_smp_rwmtx_runlock(&hipe_mfa_info_table.lock); > +} > + > +static inline void hipe_mfa_info_table_rwlock(void) > +{ > + erts_smp_rwmtx_rwlock(&hipe_mfa_info_table.lock); > +} > + > +static inline void hipe_mfa_info_table_rwunlock(void) > +{ > + erts_smp_rwmtx_rwunlock(&hipe_mfa_info_table.lock); > } > > #define HIPE_MFA_HASH(M,F,A) ((M) * (F) + (A)) > @@ -1333,6 +1343,7 @@ void *hipe_mfa_find_na(Eterm m, Eterm f, > } > #endif > > +/* PRE: called with write lock held */ > static struct hipe_mfa_info *hipe_mfa_info_table_put_locked(Eterm m, > Eterm f, unsigned int arity) > { > unsigned long h; > @@ -1362,7 +1373,7 @@ static void hipe_mfa_set_na(Eterm m, Ete > { > struct hipe_mfa_info *p; > > - hipe_mfa_info_table_lock(); > + hipe_mfa_info_table_rwlock(); > p = hipe_mfa_info_table_put_locked(m, f, arity); > #ifdef DEBUG_LINKER > printf("%s: ", __FUNCTION__); > @@ -1372,7 +1383,7 @@ static void hipe_mfa_set_na(Eterm m, Ete > p->local_address = address; > if (is_exported) > p->remote_address = address; > - hipe_mfa_info_table_unlock(); > + hipe_mfa_info_table_rwunlock(); > } > > #if defined(__powerpc__) || defined(__ppc__) || defined(__powerpc64__) || > defined(__arm__) > @@ -1381,10 +1392,10 @@ void *hipe_mfa_get_trampoline(Eterm m, E > struct hipe_mfa_info *p; > void *trampoline; > > - hipe_mfa_info_table_lock(); > - p = hipe_mfa_info_table_put_locked(m, f, arity); > - trampoline = p->trampoline; > - hipe_mfa_info_table_unlock(); > + hipe_mfa_info_table_rlock(); > + p = hipe_mfa_info_table_get_locked(m, f, arity); > + trampoline = p ? p->trampoline : NULL; > + hipe_mfa_info_table_runlock(); > return trampoline; > } > > @@ -1392,10 +1403,10 @@ void hipe_mfa_set_trampoline(Eterm m, Et > { > struct hipe_mfa_info *p; > > - hipe_mfa_info_table_lock(); > + hipe_mfa_info_table_rwlock(); > p = hipe_mfa_info_table_put_locked(m, f, arity); > p->trampoline = trampoline; > - hipe_mfa_info_table_unlock(); > + hipe_mfa_info_table_rwunlock(); > } > #endif > > @@ -1426,7 +1437,7 @@ BIF_RETTYPE hipe_bifs_invalidate_funinfo > struct mfa mfa; > struct hipe_mfa_info *p; > > - hipe_mfa_info_table_lock(); > + hipe_mfa_info_table_rwlock(); > lst = BIF_ARG_1; > while (is_list(lst)) { > if (!term_to_mfa(CAR(list_val(lst)), &mfa)) > @@ -1455,7 +1466,7 @@ BIF_RETTYPE hipe_bifs_invalidate_funinfo > } > } > } > - hipe_mfa_info_table_unlock(); > + hipe_mfa_info_table_rwunlock(); > if (is_not_nil(lst)) > BIF_ERROR(BIF_P, BADARG); > BIF_RET(NIL); > @@ -1469,7 +1480,7 @@ void hipe_mfa_save_orig_beam_op(Eterm mo > orig_beam_op = pc[0]; > if (orig_beam_op != BeamOpCode(op_hipe_trap_call_closure) && > orig_beam_op != BeamOpCode(op_hipe_trap_call)) { > - hipe_mfa_info_table_lock(); > + hipe_mfa_info_table_rwlock(); > p = hipe_mfa_info_table_put_locked(mod, fun, ari); > #ifdef DEBUG_LINKER > printf("%s: ", __FUNCTION__); > @@ -1478,7 +1489,7 @@ void hipe_mfa_save_orig_beam_op(Eterm mo > #endif > p->beam_code = pc; > p->orig_beam_op = orig_beam_op; > - hipe_mfa_info_table_unlock(); > + hipe_mfa_info_table_rwunlock(); > } else { > #ifdef DEBUG_LINKER > printf("%s: ", __FUNCTION__); > @@ -1505,7 +1516,8 @@ static void *hipe_make_stub(Eterm m, Ete > return StubAddress; > } > > -static void *hipe_get_na_nofail_locked(Eterm m, Eterm f, unsigned int a, > int is_remote) > +/* PRE: called with read or write lock held */ > +static void *hipe_get_na_try_locked(Eterm m, Eterm f, unsigned int a, int > is_remote, struct hipe_mfa_info **pp) > { > struct hipe_mfa_info *p; > void *address; > @@ -1523,7 +1535,20 @@ static void *hipe_get_na_nofail_locked(E > address = p->remote_address; > if (address) > return address; > - } else > + } > + /* Caller must take the slow path with the write lock held, but allow > + it to avoid some work if it already holds the write lock. */ > + if (pp) > + *pp = p; > + return NULL; > +} > + > +/* PRE: called with write lock held */ > +static void *hipe_get_na_slow_locked(Eterm m, Eterm f, unsigned int a, > int is_remote, struct hipe_mfa_info *p) > +{ > + void *address; > + > + if (!p) > p = hipe_mfa_info_table_put_locked(m, f, a); > address = hipe_make_stub(m, f, a, is_remote); > /* XXX: how to tell if a BEAM MFA is exported or not? */ > @@ -1531,14 +1556,34 @@ static void *hipe_get_na_nofail_locked(E > return address; > } > > +/* PRE: called with write lock held */ > +static void *hipe_get_na_nofail_locked(Eterm m, Eterm f, unsigned int a, > int is_remote) > +{ > + struct hipe_mfa_info *p /*= NULL*/; > + void *address; > + > + address = hipe_get_na_try_locked(m, f, a, is_remote, &p); > + if (address) > + return address; > + > + address = hipe_get_na_slow_locked(m, f, a, is_remote, p); > + return address; > +} > + > static void *hipe_get_na_nofail(Eterm m, Eterm f, unsigned int a, int > is_remote) > { > - void *p; > + void *address; > > - hipe_mfa_info_table_lock(); > - p = hipe_get_na_nofail_locked(m, f, a, is_remote); > - hipe_mfa_info_table_unlock(); > - return p; > + hipe_mfa_info_table_rlock(); > + address = hipe_get_na_try_locked(m, f, a, is_remote, NULL); > + hipe_mfa_info_table_runlock(); > + if (address) > + return address; > + > + hipe_mfa_info_table_rwlock(); > + address = hipe_get_na_slow_locked(m, f, a, is_remote, NULL); > + hipe_mfa_info_table_rwunlock(); > + return address; > } > > /* used for apply/3 in hipe_mode_switch */ > @@ -1617,7 +1662,7 @@ int hipe_find_mfa_from_ra(const void *ra > /* Note about locking: the table is only updated from the > loader, which runs with the rest of the system suspended. */ > /* XXX: alas not true; see comment at hipe_mfa_info_table.lock */ > - hipe_mfa_info_table_lock(); > + hipe_mfa_info_table_rwlock(); > bucket = hipe_mfa_info_table.bucket; > nrbuckets = 1 << hipe_mfa_info_table.log2size; > mfa = NULL; > @@ -1638,7 +1683,7 @@ int hipe_find_mfa_from_ra(const void *ra > *f = mfa->f; > *a = mfa->a; > } > - hipe_mfa_info_table_unlock(); > + hipe_mfa_info_table_rwunlock(); > return mfa ? 1 : 0; > } > > @@ -1715,7 +1760,7 @@ BIF_RETTYPE hipe_bifs_add_ref_2(BIF_ALIS > default: > goto badarg; > } > - hipe_mfa_info_table_lock(); > + hipe_mfa_info_table_rwlock(); > callee_mfa = hipe_mfa_info_table_put_locked(callee.mod, callee.fun, > callee.ari); > caller_mfa = hipe_mfa_info_table_put_locked(caller.mod, caller.fun, > caller.ari); > > @@ -1731,7 +1776,7 @@ BIF_RETTYPE hipe_bifs_add_ref_2(BIF_ALIS > ref->flags = flags; > ref->next = callee_mfa->referred_from; > callee_mfa->referred_from = ref; > - hipe_mfa_info_table_unlock(); > + hipe_mfa_info_table_rwunlock(); > > BIF_RET(NIL); > > @@ -1751,12 +1796,12 @@ BIF_RETTYPE hipe_bifs_mark_referred_from > > if (!term_to_mfa(BIF_ARG_1, &mfa)) > BIF_ERROR(BIF_P, BADARG); > - hipe_mfa_info_table_lock(); > + hipe_mfa_info_table_rwlock(); > p = hipe_mfa_info_table_get_locked(mfa.mod, mfa.fun, mfa.ari); > if (p) > for (ref = p->referred_from; ref != NULL; ref = ref->next) > ref->flags |= REF_FLAG_PENDING_REDIRECT; > - hipe_mfa_info_table_unlock(); > + hipe_mfa_info_table_rwunlock(); > BIF_RET(NIL); > } > > @@ -1770,7 +1815,7 @@ static void hipe_purge_all_refs(void) > struct hipe_mfa_info **bucket; > unsigned int i, nrbuckets; > > - hipe_mfa_info_table_lock(); > + hipe_mfa_info_table_rwlock(); > > bucket = hipe_mfa_info_table.bucket; > nrbuckets = 1 << hipe_mfa_info_table.log2size; > @@ -1792,7 +1837,7 @@ static void hipe_purge_all_refs(void) > erts_free(ERTS_ALC_T_HIPE, mfa); > } > } > - hipe_mfa_info_table_unlock(); > + hipe_mfa_info_table_rwunlock(); > } > > BIF_RETTYPE hipe_bifs_remove_refs_from_1(BIF_ALIST_1) > @@ -1809,7 +1854,7 @@ BIF_RETTYPE hipe_bifs_remove_refs_from_1 > > if (!term_to_mfa(BIF_ARG_1, &mfa)) > BIF_ERROR(BIF_P, BADARG); > - hipe_mfa_info_table_lock(); > + hipe_mfa_info_table_rwlock(); > caller_mfa = hipe_mfa_info_table_get_locked(mfa.mod, mfa.fun, > mfa.ari); > if (caller_mfa) { > refers_to = caller_mfa->refers_to; > @@ -1840,7 +1885,7 @@ BIF_RETTYPE hipe_bifs_remove_refs_from_1 > } > caller_mfa->refers_to = NULL; > } > - hipe_mfa_info_table_unlock(); > + hipe_mfa_info_table_rwunlock(); > BIF_RET(am_ok); > } > > @@ -1859,7 +1904,7 @@ BIF_RETTYPE hipe_bifs_redirect_referred_ > > if (!term_to_mfa(BIF_ARG_1, &mfa)) > BIF_ERROR(BIF_P, BADARG); > - hipe_mfa_info_table_lock(); > + hipe_mfa_info_table_rwlock(); > p = hipe_mfa_info_table_get_locked(mfa.mod, mfa.fun, mfa.ari); > if (p) { > prev = &p->referred_from; > @@ -1890,7 +1935,7 @@ BIF_RETTYPE hipe_bifs_redirect_referred_ > } > } > } > - hipe_mfa_info_table_unlock(); > + hipe_mfa_info_table_rwunlock(); > BIF_RET(NIL); > } > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From louis-philippe.gauthier@REDACTED Mon Feb 23 17:58:13 2015 From: louis-philippe.gauthier@REDACTED (Louis-Philippe Gauthier) Date: Mon, 23 Feb 2015 11:58:13 -0500 Subject: [erlang-bugs] hipe_mfait_lock In-Reply-To: References: <54B698EA.9090701@erix.ericsson.se> <21739.3922.436853.982577@gargle.gargle.HOWL> Message-ID: Hi Mikael, I got a bit excited and tried it right away. Seems to fix my contention issue on hipe_mfait_lock. Here's the new lock count output (hipe_mfait_lock is not there!): https://gist.github.com/lpgauth/e85efa59b96726af235d Thanks, LP On Mon, Feb 23, 2015 at 9:28 AM, Louis-Philippe Gauthier < louis-philippe.gauthier@REDACTED> wrote: > Hi Mikael, > Awesome work, I'll give the patch a try this week. > > Thanks, > LP > > On Mon, Feb 23, 2015 at 6:30 AM, Mikael Pettersson > wrote: > >> Louis-Philippe Gauthier writes: >> > Hi Sverker, >> > That's not a bad idea, I'll try that approach! >> > >> > Thanks, >> > LP >> > >> > On Thu, Jan 15, 2015 at 12:27 AM, Sverker Eriksson < >> > sverker.eriksson@REDACTED> wrote: >> > >> > > One quite simple improvement is to change the hipe_mfait_lock to be >> a >> > > read-write mutex (erts_smp_rwmtx_t). >> > > And then do read-lock at lookup and exclusive write-lock only when >> > > inserting a new entry in the table. >> > > >> > > I don't think that requires much knowledge about the VM to pull off >> ;-) . >> > > >> > > /Sverker, Erlang/OTP >> > > >> > > >> > > >> > > >> > > On 01/14/2015 12:34 AM, Louis-Philippe Gauthier wrote: >> > > >> > > Hi erlang-bugs, >> > > I couple of months ago I tried running our full application using >> HiPE. I >> > > ran into several issues, some manageable, some not... The most >> problematic >> > > issue was due to locking, specifically, the hipe_mfait_lock (you can >> see >> > > the lock counter output in the gist bellow). >> > > https://gist.github.com/lpgauth/2b3220f4bceeed6f62d0 >> > > >> > > Looking at the code it's obvious that this is a known problem... the >> > > following comment was added when the lock was added in 2010. >> > > >> > > "XXX: Redesign apply et al to avoid those updates." >> https://github.com/erlang/otp/blob/maint/erts/emulator/hipe/hipe_bif0.c#L1218 >> > > >> > > Unfortunately, I'm don't know the runtime enough to start patching >> it... so >> > > instead I'm reporting it. >> >> Please try the attached patch (against 17.4, should apply to older >> versions too by I haven't checked), and let me know if it makes a >> noticeable improvement to your use case. >> >> Run-time apply:s now do their lookups using a shared read-lock, and >> only if their lookups fail do they re-lock with an exclusive write-lock >> in order to insert new data. Module loading functions always take >> exclusive locks, which matches the current behaviour. >> >> /Mikael >> >> >> --- otp_src_17.4/erts/emulator/hipe/hipe_bif0.c.~1~ 2014-12-09 >> 21:11:07.000000000 +0100 >> +++ otp_src_17.4/erts/emulator/hipe/hipe_bif0.c 2015-02-22 >> 21:32:55.361884741 +0100 >> @@ -1217,22 +1217,32 @@ static struct { >> * they create a new stub for the mfa, which forces locking. >> * XXX: Redesign apply et al to avoid those updates. >> */ >> - erts_smp_mtx_t lock; >> + erts_smp_rwmtx_t lock; >> } hipe_mfa_info_table; >> >> static inline void hipe_mfa_info_table_init_lock(void) >> { >> - erts_smp_mtx_init(&hipe_mfa_info_table.lock, "hipe_mfait_lock"); >> + erts_smp_rwmtx_init(&hipe_mfa_info_table.lock, "hipe_mfait_lock"); >> } >> >> -static inline void hipe_mfa_info_table_lock(void) >> +static inline void hipe_mfa_info_table_rlock(void) >> { >> - erts_smp_mtx_lock(&hipe_mfa_info_table.lock); >> + erts_smp_rwmtx_rlock(&hipe_mfa_info_table.lock); >> } >> >> -static inline void hipe_mfa_info_table_unlock(void) >> +static inline void hipe_mfa_info_table_runlock(void) >> { >> - erts_smp_mtx_unlock(&hipe_mfa_info_table.lock); >> + erts_smp_rwmtx_runlock(&hipe_mfa_info_table.lock); >> +} >> + >> +static inline void hipe_mfa_info_table_rwlock(void) >> +{ >> + erts_smp_rwmtx_rwlock(&hipe_mfa_info_table.lock); >> +} >> + >> +static inline void hipe_mfa_info_table_rwunlock(void) >> +{ >> + erts_smp_rwmtx_rwunlock(&hipe_mfa_info_table.lock); >> } >> >> #define HIPE_MFA_HASH(M,F,A) ((M) * (F) + (A)) >> @@ -1333,6 +1343,7 @@ void *hipe_mfa_find_na(Eterm m, Eterm f, >> } >> #endif >> >> +/* PRE: called with write lock held */ >> static struct hipe_mfa_info *hipe_mfa_info_table_put_locked(Eterm m, >> Eterm f, unsigned int arity) >> { >> unsigned long h; >> @@ -1362,7 +1373,7 @@ static void hipe_mfa_set_na(Eterm m, Ete >> { >> struct hipe_mfa_info *p; >> >> - hipe_mfa_info_table_lock(); >> + hipe_mfa_info_table_rwlock(); >> p = hipe_mfa_info_table_put_locked(m, f, arity); >> #ifdef DEBUG_LINKER >> printf("%s: ", __FUNCTION__); >> @@ -1372,7 +1383,7 @@ static void hipe_mfa_set_na(Eterm m, Ete >> p->local_address = address; >> if (is_exported) >> p->remote_address = address; >> - hipe_mfa_info_table_unlock(); >> + hipe_mfa_info_table_rwunlock(); >> } >> >> #if defined(__powerpc__) || defined(__ppc__) || defined(__powerpc64__) >> || defined(__arm__) >> @@ -1381,10 +1392,10 @@ void *hipe_mfa_get_trampoline(Eterm m, E >> struct hipe_mfa_info *p; >> void *trampoline; >> >> - hipe_mfa_info_table_lock(); >> - p = hipe_mfa_info_table_put_locked(m, f, arity); >> - trampoline = p->trampoline; >> - hipe_mfa_info_table_unlock(); >> + hipe_mfa_info_table_rlock(); >> + p = hipe_mfa_info_table_get_locked(m, f, arity); >> + trampoline = p ? p->trampoline : NULL; >> + hipe_mfa_info_table_runlock(); >> return trampoline; >> } >> >> @@ -1392,10 +1403,10 @@ void hipe_mfa_set_trampoline(Eterm m, Et >> { >> struct hipe_mfa_info *p; >> >> - hipe_mfa_info_table_lock(); >> + hipe_mfa_info_table_rwlock(); >> p = hipe_mfa_info_table_put_locked(m, f, arity); >> p->trampoline = trampoline; >> - hipe_mfa_info_table_unlock(); >> + hipe_mfa_info_table_rwunlock(); >> } >> #endif >> >> @@ -1426,7 +1437,7 @@ BIF_RETTYPE hipe_bifs_invalidate_funinfo >> struct mfa mfa; >> struct hipe_mfa_info *p; >> >> - hipe_mfa_info_table_lock(); >> + hipe_mfa_info_table_rwlock(); >> lst = BIF_ARG_1; >> while (is_list(lst)) { >> if (!term_to_mfa(CAR(list_val(lst)), &mfa)) >> @@ -1455,7 +1466,7 @@ BIF_RETTYPE hipe_bifs_invalidate_funinfo >> } >> } >> } >> - hipe_mfa_info_table_unlock(); >> + hipe_mfa_info_table_rwunlock(); >> if (is_not_nil(lst)) >> BIF_ERROR(BIF_P, BADARG); >> BIF_RET(NIL); >> @@ -1469,7 +1480,7 @@ void hipe_mfa_save_orig_beam_op(Eterm mo >> orig_beam_op = pc[0]; >> if (orig_beam_op != BeamOpCode(op_hipe_trap_call_closure) && >> orig_beam_op != BeamOpCode(op_hipe_trap_call)) { >> - hipe_mfa_info_table_lock(); >> + hipe_mfa_info_table_rwlock(); >> p = hipe_mfa_info_table_put_locked(mod, fun, ari); >> #ifdef DEBUG_LINKER >> printf("%s: ", __FUNCTION__); >> @@ -1478,7 +1489,7 @@ void hipe_mfa_save_orig_beam_op(Eterm mo >> #endif >> p->beam_code = pc; >> p->orig_beam_op = orig_beam_op; >> - hipe_mfa_info_table_unlock(); >> + hipe_mfa_info_table_rwunlock(); >> } else { >> #ifdef DEBUG_LINKER >> printf("%s: ", __FUNCTION__); >> @@ -1505,7 +1516,8 @@ static void *hipe_make_stub(Eterm m, Ete >> return StubAddress; >> } >> >> -static void *hipe_get_na_nofail_locked(Eterm m, Eterm f, unsigned int a, >> int is_remote) >> +/* PRE: called with read or write lock held */ >> +static void *hipe_get_na_try_locked(Eterm m, Eterm f, unsigned int a, >> int is_remote, struct hipe_mfa_info **pp) >> { >> struct hipe_mfa_info *p; >> void *address; >> @@ -1523,7 +1535,20 @@ static void *hipe_get_na_nofail_locked(E >> address = p->remote_address; >> if (address) >> return address; >> - } else >> + } >> + /* Caller must take the slow path with the write lock held, but allow >> + it to avoid some work if it already holds the write lock. */ >> + if (pp) >> + *pp = p; >> + return NULL; >> +} >> + >> +/* PRE: called with write lock held */ >> +static void *hipe_get_na_slow_locked(Eterm m, Eterm f, unsigned int a, >> int is_remote, struct hipe_mfa_info *p) >> +{ >> + void *address; >> + >> + if (!p) >> p = hipe_mfa_info_table_put_locked(m, f, a); >> address = hipe_make_stub(m, f, a, is_remote); >> /* XXX: how to tell if a BEAM MFA is exported or not? */ >> @@ -1531,14 +1556,34 @@ static void *hipe_get_na_nofail_locked(E >> return address; >> } >> >> +/* PRE: called with write lock held */ >> +static void *hipe_get_na_nofail_locked(Eterm m, Eterm f, unsigned int a, >> int is_remote) >> +{ >> + struct hipe_mfa_info *p /*= NULL*/; >> + void *address; >> + >> + address = hipe_get_na_try_locked(m, f, a, is_remote, &p); >> + if (address) >> + return address; >> + >> + address = hipe_get_na_slow_locked(m, f, a, is_remote, p); >> + return address; >> +} >> + >> static void *hipe_get_na_nofail(Eterm m, Eterm f, unsigned int a, int >> is_remote) >> { >> - void *p; >> + void *address; >> >> - hipe_mfa_info_table_lock(); >> - p = hipe_get_na_nofail_locked(m, f, a, is_remote); >> - hipe_mfa_info_table_unlock(); >> - return p; >> + hipe_mfa_info_table_rlock(); >> + address = hipe_get_na_try_locked(m, f, a, is_remote, NULL); >> + hipe_mfa_info_table_runlock(); >> + if (address) >> + return address; >> + >> + hipe_mfa_info_table_rwlock(); >> + address = hipe_get_na_slow_locked(m, f, a, is_remote, NULL); >> + hipe_mfa_info_table_rwunlock(); >> + return address; >> } >> >> /* used for apply/3 in hipe_mode_switch */ >> @@ -1617,7 +1662,7 @@ int hipe_find_mfa_from_ra(const void *ra >> /* Note about locking: the table is only updated from the >> loader, which runs with the rest of the system suspended. */ >> /* XXX: alas not true; see comment at hipe_mfa_info_table.lock */ >> - hipe_mfa_info_table_lock(); >> + hipe_mfa_info_table_rwlock(); >> bucket = hipe_mfa_info_table.bucket; >> nrbuckets = 1 << hipe_mfa_info_table.log2size; >> mfa = NULL; >> @@ -1638,7 +1683,7 @@ int hipe_find_mfa_from_ra(const void *ra >> *f = mfa->f; >> *a = mfa->a; >> } >> - hipe_mfa_info_table_unlock(); >> + hipe_mfa_info_table_rwunlock(); >> return mfa ? 1 : 0; >> } >> >> @@ -1715,7 +1760,7 @@ BIF_RETTYPE hipe_bifs_add_ref_2(BIF_ALIS >> default: >> goto badarg; >> } >> - hipe_mfa_info_table_lock(); >> + hipe_mfa_info_table_rwlock(); >> callee_mfa = hipe_mfa_info_table_put_locked(callee.mod, callee.fun, >> callee.ari); >> caller_mfa = hipe_mfa_info_table_put_locked(caller.mod, caller.fun, >> caller.ari); >> >> @@ -1731,7 +1776,7 @@ BIF_RETTYPE hipe_bifs_add_ref_2(BIF_ALIS >> ref->flags = flags; >> ref->next = callee_mfa->referred_from; >> callee_mfa->referred_from = ref; >> - hipe_mfa_info_table_unlock(); >> + hipe_mfa_info_table_rwunlock(); >> >> BIF_RET(NIL); >> >> @@ -1751,12 +1796,12 @@ BIF_RETTYPE hipe_bifs_mark_referred_from >> >> if (!term_to_mfa(BIF_ARG_1, &mfa)) >> BIF_ERROR(BIF_P, BADARG); >> - hipe_mfa_info_table_lock(); >> + hipe_mfa_info_table_rwlock(); >> p = hipe_mfa_info_table_get_locked(mfa.mod, mfa.fun, mfa.ari); >> if (p) >> for (ref = p->referred_from; ref != NULL; ref = ref->next) >> ref->flags |= REF_FLAG_PENDING_REDIRECT; >> - hipe_mfa_info_table_unlock(); >> + hipe_mfa_info_table_rwunlock(); >> BIF_RET(NIL); >> } >> >> @@ -1770,7 +1815,7 @@ static void hipe_purge_all_refs(void) >> struct hipe_mfa_info **bucket; >> unsigned int i, nrbuckets; >> >> - hipe_mfa_info_table_lock(); >> + hipe_mfa_info_table_rwlock(); >> >> bucket = hipe_mfa_info_table.bucket; >> nrbuckets = 1 << hipe_mfa_info_table.log2size; >> @@ -1792,7 +1837,7 @@ static void hipe_purge_all_refs(void) >> erts_free(ERTS_ALC_T_HIPE, mfa); >> } >> } >> - hipe_mfa_info_table_unlock(); >> + hipe_mfa_info_table_rwunlock(); >> } >> >> BIF_RETTYPE hipe_bifs_remove_refs_from_1(BIF_ALIST_1) >> @@ -1809,7 +1854,7 @@ BIF_RETTYPE hipe_bifs_remove_refs_from_1 >> >> if (!term_to_mfa(BIF_ARG_1, &mfa)) >> BIF_ERROR(BIF_P, BADARG); >> - hipe_mfa_info_table_lock(); >> + hipe_mfa_info_table_rwlock(); >> caller_mfa = hipe_mfa_info_table_get_locked(mfa.mod, mfa.fun, >> mfa.ari); >> if (caller_mfa) { >> refers_to = caller_mfa->refers_to; >> @@ -1840,7 +1885,7 @@ BIF_RETTYPE hipe_bifs_remove_refs_from_1 >> } >> caller_mfa->refers_to = NULL; >> } >> - hipe_mfa_info_table_unlock(); >> + hipe_mfa_info_table_rwunlock(); >> BIF_RET(am_ok); >> } >> >> @@ -1859,7 +1904,7 @@ BIF_RETTYPE hipe_bifs_redirect_referred_ >> >> if (!term_to_mfa(BIF_ARG_1, &mfa)) >> BIF_ERROR(BIF_P, BADARG); >> - hipe_mfa_info_table_lock(); >> + hipe_mfa_info_table_rwlock(); >> p = hipe_mfa_info_table_get_locked(mfa.mod, mfa.fun, mfa.ari); >> if (p) { >> prev = &p->referred_from; >> @@ -1890,7 +1935,7 @@ BIF_RETTYPE hipe_bifs_redirect_referred_ >> } >> } >> } >> - hipe_mfa_info_table_unlock(); >> + hipe_mfa_info_table_rwunlock(); >> BIF_RET(NIL); >> } >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Mon Feb 23 19:34:20 2015 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Mon, 23 Feb 2015 19:34:20 +0100 Subject: [erlang-bugs] hipe_mfait_lock In-Reply-To: References: <54B698EA.9090701@erix.ericsson.se> <21739.3922.436853.982577@gargle.gargle.HOWL> Message-ID: <21739.29356.817491.557049@gargle.gargle.HOWL> Louis-Philippe Gauthier writes: > Hi Mikael, > I got a bit excited and tried it right away. Seems to fix my contention > issue on hipe_mfait_lock. > > Here's the new lock count output (hipe_mfait_lock is not there!): > > https://gist.github.com/lpgauth/e85efa59b96726af235d > > Thanks, > LP Thanks for confirming that it had the intended effect. I've sent OTP a pull request for it. /Mikael > On Mon, Feb 23, 2015 at 9:28 AM, Louis-Philippe Gauthier < > louis-philippe.gauthier@REDACTED> wrote: > > > Hi Mikael, > > Awesome work, I'll give the patch a try this week. > > > > Thanks, > > LP > > > > On Mon, Feb 23, 2015 at 6:30 AM, Mikael Pettersson > > wrote: > > > >> Louis-Philippe Gauthier writes: > >> > Hi Sverker, > >> > That's not a bad idea, I'll try that approach! > >> > > >> > Thanks, > >> > LP > >> > > >> > On Thu, Jan 15, 2015 at 12:27 AM, Sverker Eriksson < > >> > sverker.eriksson@REDACTED> wrote: > >> > > >> > > One quite simple improvement is to change the hipe_mfait_lock to be > >> a > >> > > read-write mutex (erts_smp_rwmtx_t). > >> > > And then do read-lock at lookup and exclusive write-lock only when > >> > > inserting a new entry in the table. > >> > > > >> > > I don't think that requires much knowledge about the VM to pull off > >> ;-) . > >> > > > >> > > /Sverker, Erlang/OTP > >> > > > >> > > > >> > > > >> > > > >> > > On 01/14/2015 12:34 AM, Louis-Philippe Gauthier wrote: > >> > > > >> > > Hi erlang-bugs, > >> > > I couple of months ago I tried running our full application using > >> HiPE. I > >> > > ran into several issues, some manageable, some not... The most > >> problematic > >> > > issue was due to locking, specifically, the hipe_mfait_lock (you can > >> see > >> > > the lock counter output in the gist bellow). > >> > > https://gist.github.com/lpgauth/2b3220f4bceeed6f62d0 > >> > > > >> > > Looking at the code it's obvious that this is a known problem... the > >> > > following comment was added when the lock was added in 2010. > >> > > > >> > > "XXX: Redesign apply et al to avoid those updates." > >> https://github.com/erlang/otp/blob/maint/erts/emulator/hipe/hipe_bif0.c#L1218 > >> > > > >> > > Unfortunately, I'm don't know the runtime enough to start patching > >> it... so > >> > > instead I'm reporting it. > >> > >> Please try the attached patch (against 17.4, should apply to older > >> versions too by I haven't checked), and let me know if it makes a > >> noticeable improvement to your use case. > >> > >> Run-time apply:s now do their lookups using a shared read-lock, and > >> only if their lookups fail do they re-lock with an exclusive write-lock > >> in order to insert new data. Module loading functions always take > >> exclusive locks, which matches the current behaviour. > >> > >> /Mikael > >> > >> > >> --- otp_src_17.4/erts/emulator/hipe/hipe_bif0.c.~1~ 2014-12-09 > >> 21:11:07.000000000 +0100 > >> +++ otp_src_17.4/erts/emulator/hipe/hipe_bif0.c 2015-02-22 > >> 21:32:55.361884741 +0100 > >> @@ -1217,22 +1217,32 @@ static struct { > >> * they create a new stub for the mfa, which forces locking. > >> * XXX: Redesign apply et al to avoid those updates. > >> */ > >> - erts_smp_mtx_t lock; > >> + erts_smp_rwmtx_t lock; > >> } hipe_mfa_info_table; > >> > >> static inline void hipe_mfa_info_table_init_lock(void) > >> { > >> - erts_smp_mtx_init(&hipe_mfa_info_table.lock, "hipe_mfait_lock"); > >> + erts_smp_rwmtx_init(&hipe_mfa_info_table.lock, "hipe_mfait_lock"); > >> } > >> > >> -static inline void hipe_mfa_info_table_lock(void) > >> +static inline void hipe_mfa_info_table_rlock(void) > >> { > >> - erts_smp_mtx_lock(&hipe_mfa_info_table.lock); > >> + erts_smp_rwmtx_rlock(&hipe_mfa_info_table.lock); > >> } > >> > >> -static inline void hipe_mfa_info_table_unlock(void) > >> +static inline void hipe_mfa_info_table_runlock(void) > >> { > >> - erts_smp_mtx_unlock(&hipe_mfa_info_table.lock); > >> + erts_smp_rwmtx_runlock(&hipe_mfa_info_table.lock); > >> +} > >> + > >> +static inline void hipe_mfa_info_table_rwlock(void) > >> +{ > >> + erts_smp_rwmtx_rwlock(&hipe_mfa_info_table.lock); > >> +} > >> + > >> +static inline void hipe_mfa_info_table_rwunlock(void) > >> +{ > >> + erts_smp_rwmtx_rwunlock(&hipe_mfa_info_table.lock); > >> } > >> > >> #define HIPE_MFA_HASH(M,F,A) ((M) * (F) + (A)) > >> @@ -1333,6 +1343,7 @@ void *hipe_mfa_find_na(Eterm m, Eterm f, > >> } > >> #endif > >> > >> +/* PRE: called with write lock held */ > >> static struct hipe_mfa_info *hipe_mfa_info_table_put_locked(Eterm m, > >> Eterm f, unsigned int arity) > >> { > >> unsigned long h; > >> @@ -1362,7 +1373,7 @@ static void hipe_mfa_set_na(Eterm m, Ete > >> { > >> struct hipe_mfa_info *p; > >> > >> - hipe_mfa_info_table_lock(); > >> + hipe_mfa_info_table_rwlock(); > >> p = hipe_mfa_info_table_put_locked(m, f, arity); > >> #ifdef DEBUG_LINKER > >> printf("%s: ", __FUNCTION__); > >> @@ -1372,7 +1383,7 @@ static void hipe_mfa_set_na(Eterm m, Ete > >> p->local_address = address; > >> if (is_exported) > >> p->remote_address = address; > >> - hipe_mfa_info_table_unlock(); > >> + hipe_mfa_info_table_rwunlock(); > >> } > >> > >> #if defined(__powerpc__) || defined(__ppc__) || defined(__powerpc64__) > >> || defined(__arm__) > >> @@ -1381,10 +1392,10 @@ void *hipe_mfa_get_trampoline(Eterm m, E > >> struct hipe_mfa_info *p; > >> void *trampoline; > >> > >> - hipe_mfa_info_table_lock(); > >> - p = hipe_mfa_info_table_put_locked(m, f, arity); > >> - trampoline = p->trampoline; > >> - hipe_mfa_info_table_unlock(); > >> + hipe_mfa_info_table_rlock(); > >> + p = hipe_mfa_info_table_get_locked(m, f, arity); > >> + trampoline = p ? p->trampoline : NULL; > >> + hipe_mfa_info_table_runlock(); > >> return trampoline; > >> } > >> > >> @@ -1392,10 +1403,10 @@ void hipe_mfa_set_trampoline(Eterm m, Et > >> { > >> struct hipe_mfa_info *p; > >> > >> - hipe_mfa_info_table_lock(); > >> + hipe_mfa_info_table_rwlock(); > >> p = hipe_mfa_info_table_put_locked(m, f, arity); > >> p->trampoline = trampoline; > >> - hipe_mfa_info_table_unlock(); > >> + hipe_mfa_info_table_rwunlock(); > >> } > >> #endif > >> > >> @@ -1426,7 +1437,7 @@ BIF_RETTYPE hipe_bifs_invalidate_funinfo > >> struct mfa mfa; > >> struct hipe_mfa_info *p; > >> > >> - hipe_mfa_info_table_lock(); > >> + hipe_mfa_info_table_rwlock(); > >> lst = BIF_ARG_1; > >> while (is_list(lst)) { > >> if (!term_to_mfa(CAR(list_val(lst)), &mfa)) > >> @@ -1455,7 +1466,7 @@ BIF_RETTYPE hipe_bifs_invalidate_funinfo > >> } > >> } > >> } > >> - hipe_mfa_info_table_unlock(); > >> + hipe_mfa_info_table_rwunlock(); > >> if (is_not_nil(lst)) > >> BIF_ERROR(BIF_P, BADARG); > >> BIF_RET(NIL); > >> @@ -1469,7 +1480,7 @@ void hipe_mfa_save_orig_beam_op(Eterm mo > >> orig_beam_op = pc[0]; > >> if (orig_beam_op != BeamOpCode(op_hipe_trap_call_closure) && > >> orig_beam_op != BeamOpCode(op_hipe_trap_call)) { > >> - hipe_mfa_info_table_lock(); > >> + hipe_mfa_info_table_rwlock(); > >> p = hipe_mfa_info_table_put_locked(mod, fun, ari); > >> #ifdef DEBUG_LINKER > >> printf("%s: ", __FUNCTION__); > >> @@ -1478,7 +1489,7 @@ void hipe_mfa_save_orig_beam_op(Eterm mo > >> #endif > >> p->beam_code = pc; > >> p->orig_beam_op = orig_beam_op; > >> - hipe_mfa_info_table_unlock(); > >> + hipe_mfa_info_table_rwunlock(); > >> } else { > >> #ifdef DEBUG_LINKER > >> printf("%s: ", __FUNCTION__); > >> @@ -1505,7 +1516,8 @@ static void *hipe_make_stub(Eterm m, Ete > >> return StubAddress; > >> } > >> > >> -static void *hipe_get_na_nofail_locked(Eterm m, Eterm f, unsigned int a, > >> int is_remote) > >> +/* PRE: called with read or write lock held */ > >> +static void *hipe_get_na_try_locked(Eterm m, Eterm f, unsigned int a, > >> int is_remote, struct hipe_mfa_info **pp) > >> { > >> struct hipe_mfa_info *p; > >> void *address; > >> @@ -1523,7 +1535,20 @@ static void *hipe_get_na_nofail_locked(E > >> address = p->remote_address; > >> if (address) > >> return address; > >> - } else > >> + } > >> + /* Caller must take the slow path with the write lock held, but allow > >> + it to avoid some work if it already holds the write lock. */ > >> + if (pp) > >> + *pp = p; > >> + return NULL; > >> +} > >> + > >> +/* PRE: called with write lock held */ > >> +static void *hipe_get_na_slow_locked(Eterm m, Eterm f, unsigned int a, > >> int is_remote, struct hipe_mfa_info *p) > >> +{ > >> + void *address; > >> + > >> + if (!p) > >> p = hipe_mfa_info_table_put_locked(m, f, a); > >> address = hipe_make_stub(m, f, a, is_remote); > >> /* XXX: how to tell if a BEAM MFA is exported or not? */ > >> @@ -1531,14 +1556,34 @@ static void *hipe_get_na_nofail_locked(E > >> return address; > >> } > >> > >> +/* PRE: called with write lock held */ > >> +static void *hipe_get_na_nofail_locked(Eterm m, Eterm f, unsigned int a, > >> int is_remote) > >> +{ > >> + struct hipe_mfa_info *p /*= NULL*/; > >> + void *address; > >> + > >> + address = hipe_get_na_try_locked(m, f, a, is_remote, &p); > >> + if (address) > >> + return address; > >> + > >> + address = hipe_get_na_slow_locked(m, f, a, is_remote, p); > >> + return address; > >> +} > >> + > >> static void *hipe_get_na_nofail(Eterm m, Eterm f, unsigned int a, int > >> is_remote) > >> { > >> - void *p; > >> + void *address; > >> > >> - hipe_mfa_info_table_lock(); > >> - p = hipe_get_na_nofail_locked(m, f, a, is_remote); > >> - hipe_mfa_info_table_unlock(); > >> - return p; > >> + hipe_mfa_info_table_rlock(); > >> + address = hipe_get_na_try_locked(m, f, a, is_remote, NULL); > >> + hipe_mfa_info_table_runlock(); > >> + if (address) > >> + return address; > >> + > >> + hipe_mfa_info_table_rwlock(); > >> + address = hipe_get_na_slow_locked(m, f, a, is_remote, NULL); > >> + hipe_mfa_info_table_rwunlock(); > >> + return address; > >> } > >> > >> /* used for apply/3 in hipe_mode_switch */ > >> @@ -1617,7 +1662,7 @@ int hipe_find_mfa_from_ra(const void *ra > >> /* Note about locking: the table is only updated from the > >> loader, which runs with the rest of the system suspended. */ > >> /* XXX: alas not true; see comment at hipe_mfa_info_table.lock */ > >> - hipe_mfa_info_table_lock(); > >> + hipe_mfa_info_table_rwlock(); > >> bucket = hipe_mfa_info_table.bucket; > >> nrbuckets = 1 << hipe_mfa_info_table.log2size; > >> mfa = NULL; > >> @@ -1638,7 +1683,7 @@ int hipe_find_mfa_from_ra(const void *ra > >> *f = mfa->f; > >> *a = mfa->a; > >> } > >> - hipe_mfa_info_table_unlock(); > >> + hipe_mfa_info_table_rwunlock(); > >> return mfa ? 1 : 0; > >> } > >> > >> @@ -1715,7 +1760,7 @@ BIF_RETTYPE hipe_bifs_add_ref_2(BIF_ALIS > >> default: > >> goto badarg; > >> } > >> - hipe_mfa_info_table_lock(); > >> + hipe_mfa_info_table_rwlock(); > >> callee_mfa = hipe_mfa_info_table_put_locked(callee.mod, callee.fun, > >> callee.ari); > >> caller_mfa = hipe_mfa_info_table_put_locked(caller.mod, caller.fun, > >> caller.ari); > >> > >> @@ -1731,7 +1776,7 @@ BIF_RETTYPE hipe_bifs_add_ref_2(BIF_ALIS > >> ref->flags = flags; > >> ref->next = callee_mfa->referred_from; > >> callee_mfa->referred_from = ref; > >> - hipe_mfa_info_table_unlock(); > >> + hipe_mfa_info_table_rwunlock(); > >> > >> BIF_RET(NIL); > >> > >> @@ -1751,12 +1796,12 @@ BIF_RETTYPE hipe_bifs_mark_referred_from > >> > >> if (!term_to_mfa(BIF_ARG_1, &mfa)) > >> BIF_ERROR(BIF_P, BADARG); > >> - hipe_mfa_info_table_lock(); > >> + hipe_mfa_info_table_rwlock(); > >> p = hipe_mfa_info_table_get_locked(mfa.mod, mfa.fun, mfa.ari); > >> if (p) > >> for (ref = p->referred_from; ref != NULL; ref = ref->next) > >> ref->flags |= REF_FLAG_PENDING_REDIRECT; > >> - hipe_mfa_info_table_unlock(); > >> + hipe_mfa_info_table_rwunlock(); > >> BIF_RET(NIL); > >> } > >> > >> @@ -1770,7 +1815,7 @@ static void hipe_purge_all_refs(void) > >> struct hipe_mfa_info **bucket; > >> unsigned int i, nrbuckets; > >> > >> - hipe_mfa_info_table_lock(); > >> + hipe_mfa_info_table_rwlock(); > >> > >> bucket = hipe_mfa_info_table.bucket; > >> nrbuckets = 1 << hipe_mfa_info_table.log2size; > >> @@ -1792,7 +1837,7 @@ static void hipe_purge_all_refs(void) > >> erts_free(ERTS_ALC_T_HIPE, mfa); > >> } > >> } > >> - hipe_mfa_info_table_unlock(); > >> + hipe_mfa_info_table_rwunlock(); > >> } > >> > >> BIF_RETTYPE hipe_bifs_remove_refs_from_1(BIF_ALIST_1) > >> @@ -1809,7 +1854,7 @@ BIF_RETTYPE hipe_bifs_remove_refs_from_1 > >> > >> if (!term_to_mfa(BIF_ARG_1, &mfa)) > >> BIF_ERROR(BIF_P, BADARG); > >> - hipe_mfa_info_table_lock(); > >> + hipe_mfa_info_table_rwlock(); > >> caller_mfa = hipe_mfa_info_table_get_locked(mfa.mod, mfa.fun, > >> mfa.ari); > >> if (caller_mfa) { > >> refers_to = caller_mfa->refers_to; > >> @@ -1840,7 +1885,7 @@ BIF_RETTYPE hipe_bifs_remove_refs_from_1 > >> } > >> caller_mfa->refers_to = NULL; > >> } > >> - hipe_mfa_info_table_unlock(); > >> + hipe_mfa_info_table_rwunlock(); > >> BIF_RET(am_ok); > >> } > >> > >> @@ -1859,7 +1904,7 @@ BIF_RETTYPE hipe_bifs_redirect_referred_ > >> > >> if (!term_to_mfa(BIF_ARG_1, &mfa)) > >> BIF_ERROR(BIF_P, BADARG); > >> - hipe_mfa_info_table_lock(); > >> + hipe_mfa_info_table_rwlock(); > >> p = hipe_mfa_info_table_get_locked(mfa.mod, mfa.fun, mfa.ari); > >> if (p) { > >> prev = &p->referred_from; > >> @@ -1890,7 +1935,7 @@ BIF_RETTYPE hipe_bifs_redirect_referred_ > >> } > >> } > >> } > >> - hipe_mfa_info_table_unlock(); > >> + hipe_mfa_info_table_rwunlock(); > >> BIF_RET(NIL); > >> } > >> > >> > >> > > -- From essen@REDACTED Tue Feb 24 17:40:39 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 24 Feb 2015 17:40:39 +0100 Subject: [erlang-bugs] SSL suite test failure In-Reply-To: <54EB0311.1070707@ericsson.com> References: <54E88B28.5020203@ninenines.eu> <54EB0311.1070707@ericsson.com> Message-ID: <54ECA987.2000703@ninenines.eu> Hello, On 02/23/2015 11:38 AM, Ingela Anderton Andin wrote: > Hi! > > It might be a timing issue in the test suite. As the test suite always > spawns the openssl server and waits for it to come up, but we have not > found a perfect way of telling that it is up. Are you sure about that? This is the "basic_erlang_server_openssl_client" test, sounds like the server is Erlang and the client OpenSSL and not the other way around. Cheers, -- Lo?c Hoguin http://ninenines.eu From essen@REDACTED Thu Feb 26 18:31:27 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Thu, 26 Feb 2015 18:31:27 +0100 Subject: [erlang-bugs] ssl_basic_SUITE:defaults intermittent test failure Message-ID: <54EF586F.3020005@ninenines.eu> Hello, I had the following happen today with ssl_basic_SUITE:defaults test. This is completely unrelated to the changes I am working on AFAIK. *** User 2015-02-26 17:56:34.477 *** TLS/SSL version [{3,0}] *** CT Error Notification 2015-02-26 17:56:34.478 *** ssl_basic_SUITE:defaults failed on line 2542 Reason: {badmatch,true} Full error description and stacktrace === Ended at 2015-02-26 17:56:34 === location [{ssl_basic_SUITE,defaults,2542}, {test_server,ts_tc,1415}, {test_server,run_test_case_eval1,1028}, {test_server,run_test_case_eval,976}] === reason = no match of right hand side value true in function ssl_basic_SUITE:defaults/1 (ssl_basic_SUITE.erl, line 2542) in call from test_server:ts_tc/3 (test_server.erl, line 1415) in call from test_server:run_test_case_eval1/6 (test_server.erl, line 1028) in call from test_server:run_test_case_eval/9 (test_server.erl, line 976) -- Lo?c Hoguin http://ninenines.eu