From efcasado@REDACTED Thu May 1 09:19:42 2014 From: efcasado@REDACTED (=?UTF-8?Q?Enrique_Fern=C3=A1ndez_Casado?=) Date: Thu, 1 May 2014 09:19:42 +0200 Subject: [erlang-patches] Put compile options from code into parse_transform/2 options variable Message-ID: Put compile options from code (i.e. -compile(myoption1)) into the options variable used as second argument in parse_transform/2. This makes it possible to access all compile options (i.e. the ones specified in the command-line and the ones specified in code) in a consistent way. Here is a simple example: (dummy_module.erl) -module(dummy_module). -compile({parse_transform, dummy_transform}). -compile(dummy_transform_option1). (dummy_transform.erl) -module(dummy_transform). -export([parse_transform/2]). parse_transform(Forms, Opts) -> true = lists:member(dummy_transform_option1, Opts), Forms. Links: git fetch git://github.com/efcasado/otp.git compile-options https://github.com/efcasado/otp/compare/erlang:maint...compile-options https://github.com/efcasado/otp/compare/erlang:maint...compile-options.patch https://github.com/erlang/otp/pull/350 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Thu May 1 14:51:04 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 1 May 2014 08:51:04 -0400 Subject: [erlang-patches] Put compile options from code into parse_transform/2 options variable In-Reply-To: References: Message-ID: <20140501125102.GF84891@ferdair.local> What happens if I have a module defined as: -module(dummy_module). -compile({parse_transform, dummy_transform1}). -compile({parse_transform, dummy_transform2}). -compile(dummy_transform_option). -compile(other_dummy_transform_option). How does that end up working? Regards, Fred. On 05/01, Enrique Fern?ndez Casado wrote: > Put compile options from code (i.e. -compile(myoption1)) into the options > variable used as second argument in parse_transform/2. This makes it > possible to access all compile options (i.e. the ones specified in the > command-line and the ones specified in code) in a consistent way. > > Here is a simple example: > > (dummy_module.erl) > > -module(dummy_module). > > -compile({parse_transform, dummy_transform}). > -compile(dummy_transform_option1). > > > > (dummy_transform.erl) > > -module(dummy_transform). > > -export([parse_transform/2]). > > parse_transform(Forms, Opts) -> > true = lists:member(dummy_transform_option1, Opts), > Forms. > > > Links: > > git fetch git://github.com/efcasado/otp.git compile-options > > https://github.com/efcasado/otp/compare/erlang:maint...compile-options > https://github.com/efcasado/otp/compare/erlang:maint...compile-options.patch > > https://github.com/erlang/otp/pull/350 > _______________________________________________ > erlang-patches mailing list > erlang-patches@REDACTED > http://erlang.org/mailman/listinfo/erlang-patches From efcasado@REDACTED Thu May 1 19:47:54 2014 From: efcasado@REDACTED (=?UTF-8?Q?Enrique_Fern=C3=A1ndez_Casado?=) Date: Thu, 1 May 2014 19:47:54 +0200 Subject: [erlang-patches] Put compile options from code into parse_transform/2 options variable In-Reply-To: <20140501125102.GF84891@ferdair.local> References: <20140501125102.GF84891@ferdair.local> Message-ID: AFAIK, there is no way to specify per-transform options. So all four compile options will end up in the "options" variable of the two specified parse transforms. Exactly as if you had compiled "dummy_module" as: erlc "+{parse_transform, dummy_transform1}" "+{parse_transform, dummy_transform2}" \ +dummy_transform_option +other_dummy_transform_option dummy_module.erl If someone were to implement support for per-transform options, it may be worth considering to add an (optional) third element to the "parse_transform" compile option (i.e. {parse_transform, Module, Opts}). Regards, Enrique 2014-05-01 14:51 GMT+02:00 Fred Hebert : > What happens if I have a module defined as: > > -module(dummy_module). > > -compile({parse_transform, dummy_transform1}). > -compile({parse_transform, dummy_transform2}). > -compile(dummy_transform_option). > -compile(other_dummy_transform_option). > > How does that end up working? > > Regards, > Fred. > > On 05/01, Enrique Fern?ndez Casado wrote: > > Put compile options from code (i.e. -compile(myoption1)) into the options > > variable used as second argument in parse_transform/2. This makes it > > possible to access all compile options (i.e. the ones specified in the > > command-line and the ones specified in code) in a consistent way. > > > > Here is a simple example: > > > > (dummy_module.erl) > > > > -module(dummy_module). > > > > -compile({parse_transform, dummy_transform}). > > -compile(dummy_transform_option1). > > > > > > > > (dummy_transform.erl) > > > > -module(dummy_transform). > > > > -export([parse_transform/2]). > > > > parse_transform(Forms, Opts) -> > > true = lists:member(dummy_transform_option1, Opts), > > Forms. > > > > > > Links: > > > > git fetch git://github.com/efcasado/otp.git compile-options > > > > https://github.com/efcasado/otp/compare/erlang:maint...compile-options > > > https://github.com/efcasado/otp/compare/erlang:maint...compile-options.patch > > > > https://github.com/erlang/otp/pull/350 > > > _______________________________________________ > > erlang-patches mailing list > > erlang-patches@REDACTED > > http://erlang.org/mailman/listinfo/erlang-patches > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Thu May 1 21:40:42 2014 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 1 May 2014 21:40:42 +0200 Subject: [erlang-patches] Put compile options from code into parse_transform/2 options variable In-Reply-To: References: <20140501125102.GF84891@ferdair.local> Message-ID: <1425B31E-DFC1-4EB7-BC0B-BD417C086CE3@feuerlabs.com> I have done a few things in this area: When using the parse_trans transform functions, you can provide an option, either as a compile attribute in the code or added as an option to compile, to save a pretty-printed result or simply write the resulting forms to a file: https://github.com/uwiger/parse_trans/blob/master/src/parse_trans.erl#L397 I haven?t exported parse_trans:option_value/3, but it?s fairly simple: https://github.com/uwiger/parse_trans/blob/master/src/parse_trans.erl#L457 In another project, I played around with Igor for module inlining. Igor takes some options, which makes it awkward to use directly as a parse transform, so I wrote a wrapper: https://github.com/Feuerlabs/exometer/blob/master/src/exometer_igor.erl BR, Ulf W On 01 May 2014, at 19:47, Enrique Fern?ndez Casado wrote: > AFAIK, there is no way to specify per-transform options. So > all four compile options will end up in the "options" variable > of the two specified parse transforms. Exactly as if you had > compiled "dummy_module" as: > > erlc "+{parse_transform, dummy_transform1}" "+{parse_transform, dummy_transform2}" \ > +dummy_transform_option +other_dummy_transform_option dummy_module.erl > > If someone were to implement support for per-transform options, > it may be worth considering to add an (optional) third element to > the "parse_transform" compile option (i.e. {parse_transform, Module, Opts}). > > Regards, > Enrique > > > 2014-05-01 14:51 GMT+02:00 Fred Hebert : > What happens if I have a module defined as: > > -module(dummy_module). > > -compile({parse_transform, dummy_transform1}). > -compile({parse_transform, dummy_transform2}). > -compile(dummy_transform_option). > -compile(other_dummy_transform_option). > > How does that end up working? > > Regards, > Fred. > > On 05/01, Enrique Fern?ndez Casado wrote: > > Put compile options from code (i.e. -compile(myoption1)) into the options > > variable used as second argument in parse_transform/2. This makes it > > possible to access all compile options (i.e. the ones specified in the > > command-line and the ones specified in code) in a consistent way. > > > > Here is a simple example: > > > > (dummy_module.erl) > > > > -module(dummy_module). > > > > -compile({parse_transform, dummy_transform}). > > -compile(dummy_transform_option1). > > > > > > > > (dummy_transform.erl) > > > > -module(dummy_transform). > > > > -export([parse_transform/2]). > > > > parse_transform(Forms, Opts) -> > > true = lists:member(dummy_transform_option1, Opts), > > Forms. > > > > > > Links: > > > > git fetch git://github.com/efcasado/otp.git compile-options > > > > https://github.com/efcasado/otp/compare/erlang:maint...compile-options > > https://github.com/efcasado/otp/compare/erlang:maint...compile-options.patch > > > > https://github.com/erlang/otp/pull/350 > > > _______________________________________________ > > erlang-patches mailing list > > erlang-patches@REDACTED > > http://erlang.org/mailman/listinfo/erlang-patches > > > _______________________________________________ > erlang-patches mailing list > erlang-patches@REDACTED > http://erlang.org/mailman/listinfo/erlang-patches Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From efcasado@REDACTED Thu May 1 22:36:46 2014 From: efcasado@REDACTED (=?UTF-8?Q?Enrique_Fern=C3=A1ndez_Casado?=) Date: Thu, 1 May 2014 22:36:46 +0200 Subject: [erlang-patches] Put compile options from code into parse_transform/2 options variable In-Reply-To: <1425B31E-DFC1-4EB7-BC0B-BD417C086CE3@feuerlabs.com> References: <20140501125102.GF84891@ferdair.local> <1425B31E-DFC1-4EB7-BC0B-BD417C086CE3@feuerlabs.com> Message-ID: Thanks for sharing, Ulf. The solution you propose in parse_trans:option_value/3 is exactly what I ended up implementing in my project. However, IMHO, it does not feel right that one has to scan the forms of a module in order to get hold of compile options specified in the module being parse transformed. Specially when the parse_transform/2 function has an "options" variable that already contains a list of **some** compile options (including any {parse_transform, Module} option that may have been defined in that module. This sounds a little bit counterintuitive to me. IMHO, the second argument of the parse_transform/2 function should contain a list of all compile options targeting the module being parse transformed, unless there is a good reason for not doing so. 2014-05-01 21:40 GMT+02:00 Ulf Wiger : > > I have done a few things in this area: > > When using the parse_trans transform functions, you can provide an option, > either as a compile attribute in the code or added as an option to compile, > to save a pretty-printed result or simply write the resulting forms to a > file: > > https://github.com/uwiger/parse_trans/blob/master/src/parse_trans.erl#L397 > > I haven?t exported parse_trans:option_value/3, but it?s fairly simple: > https://github.com/uwiger/parse_trans/blob/master/src/parse_trans.erl#L457 > > In another project, I played around with Igor for module inlining. Igor > takes some options, which makes it awkward to use directly as a parse > transform, so I wrote a wrapper: > > https://github.com/Feuerlabs/exometer/blob/master/src/exometer_igor.erl > > BR, > Ulf W > > On 01 May 2014, at 19:47, Enrique Fern?ndez Casado > wrote: > > AFAIK, there is no way to specify per-transform options. So > all four compile options will end up in the "options" variable > of the two specified parse transforms. Exactly as if you had > compiled "dummy_module" as: > > erlc "+{parse_transform, dummy_transform1}" "+{parse_transform, > dummy_transform2}" \ > +dummy_transform_option +other_dummy_transform_option dummy_module.erl > > If someone were to implement support for per-transform options, > it may be worth considering to add an (optional) third element to > the "parse_transform" compile option (i.e. {parse_transform, Module, > Opts}). > > Regards, > Enrique > > > 2014-05-01 14:51 GMT+02:00 Fred Hebert : > >> What happens if I have a module defined as: >> >> -module(dummy_module). >> >> -compile({parse_transform, dummy_transform1}). >> -compile({parse_transform, dummy_transform2}). >> -compile(dummy_transform_option). >> -compile(other_dummy_transform_option). >> >> How does that end up working? >> >> Regards, >> Fred. >> >> On 05/01, Enrique Fern?ndez Casado wrote: >> > Put compile options from code (i.e. -compile(myoption1)) into the >> options >> > variable used as second argument in parse_transform/2. This makes it >> > possible to access all compile options (i.e. the ones specified in the >> > command-line and the ones specified in code) in a consistent way. >> > >> > Here is a simple example: >> > >> > (dummy_module.erl) >> > >> > -module(dummy_module). >> > >> > -compile({parse_transform, dummy_transform}). >> > -compile(dummy_transform_option1). >> > >> > >> > >> > (dummy_transform.erl) >> > >> > -module(dummy_transform). >> > >> > -export([parse_transform/2]). >> > >> > parse_transform(Forms, Opts) -> >> > true = lists:member(dummy_transform_option1, Opts), >> > Forms. >> > >> > >> > Links: >> > >> > git fetch git://github.com/efcasado/otp.git compile-options >> > >> > https://github.com/efcasado/otp/compare/erlang:maint...compile-options >> > >> https://github.com/efcasado/otp/compare/erlang:maint...compile-options.patch >> > >> > https://github.com/erlang/otp/pull/350 >> >> > _______________________________________________ >> > erlang-patches mailing list >> > erlang-patches@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-patches >> >> > _______________________________________________ > erlang-patches mailing list > erlang-patches@REDACTED > http://erlang.org/mailman/listinfo/erlang-patches > > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Mon May 12 14:17:13 2014 From: erlangsiri@REDACTED (Siri Hansen) Date: Mon, 12 May 2014 14:17:13 +0200 Subject: [erlang-patches] Supervisor shutdown reason when reaching max restarts In-Reply-To: <12F2115FD1CCEE4294943B2608A18FA3D0E42BCC@MAIL01.win.lbaum.eu> References: <12F2115FD1CCEE4294943B2608A18FA361B08906@MAIL01.win.lbaum.eu> <12F2115FD1CCEE4294943B2608A18FA361B0B930@MAIL01.win.lbaum.eu> <12F2115FD1CCEE4294943B2608A18FA3D0E42BCC@MAIL01.win.lbaum.eu> Message-ID: Hi Tobias! It seems this feature would be "good to have" but is not totally necessary. Since there has been very little feeback on this thread, and given the backwards incompatibility, we have thus decided to reject it. Thanks for you effort! Best Regards /siri 2014-04-16 15:49 GMT+02:00 Tobias Schlager : > Hi Siri, > > I already solved the problem for my use case. In my case the supervisors > of type C never exit gracefully except for the case the application gets > stopped, so I ended up 'disarming' the monitors of process B in an > 'application:prep_stop/1' callback. > > Of course, this is a debatable patch because it introduces backwards > incompatible behavior and I will not argue if it gets declined. I just > thought that having differentiated exit reason of supervisors would be a > good thing to have in general (and that according to the documentation > {shutdown, Reason} exits must be supported by callback implementations > anyways). > > Regards > Tobias > > -- > > Lindenbaum GmbH > Dipl.-Inform. Tobias Schlager, Software Engineer > Erbprinzenstr. 4-12, 76133 Karlsruhe, Deutschland > Tel: +49 721 48 08 48 - 000, Fax: +49 721 48 08 48 - 801, > E-Mail: tobias.schlager@REDACTED > > Firmensitz: Erbprinzenstr. 4-12, Eingang A, 76133 Karlsruhe > Registergericht: Amtsgericht Mannheim, HRB 706184 > Gesch?ftsf?hrer: Dr. Ralf Nikolai > Steuernummer: 35007/02060, USt. ID: DE 263797265 > > http://www.lindenbaum.eu > > > ------------------------------ > *Von:* Siri Hansen [erlangsiri@REDACTED] > *Gesendet:* Mittwoch, 16. April 2014 14:54 > > *An:* Tobias Schlager > *Cc:* erlang-patches@REDACTED > *Betreff:* Re: [erlang-patches] Supervisor shutdown reason when reaching > max restarts > > Hello Tobias and the list! > > This review has finally made its way up the backlog. Thanks for your > patience! > > Is this use case still relevant? > > I've started looking at the compatibility issue. As far as I can see > there shouldn't be a big problem upwards in a normal supervision tree, > since this is internal to OTP. Downwards, however, there might be problems > since exit trapping children will get {shutdown,Reason} instead of shutdown > when a supervisor terminates due to a failed restart attempt. In the case > of a gen_server or gen_fsm, this exit reason will be propagated to the > callback module via Mod:terminate/2. The value 'shutdown', used when the > supervisor orders termination, is documented for this callback function. > For other proc_lib processes the receive statement is implemented > individually, so even here the change will be exposed to the user (if > matching the exit reason). > > Links and monitors across the supervision tree might also be a problem, > of course. And the case when something other than a supervisor or the > application_master is the parent of a supervisor - I don't know if this is > very common, though, and I find the incompatibility downwards in the > supervision tree most alarming. > > As you know, we need to be very careful with backwards incompatible > changes in core functionality, so for the time being we would like to > investigate the general need for this functionality and if there are other > possible ways to go. > > Some questions for the list: > > 1. Is the use case described in the Tobias' previous mail a common one? > If yes, how have you solved it? > > 2. Do you normally care about the exit reason in your Mod:terminate/2 > function or in your proc_lib receive statement when getting an EXIT from > the parent? > > 3. Any other comments on this? > > Thanks > /siri > > > > 2013-08-23 14:17 GMT+02:00 Siri Hansen : > >> Tobias, thanks for a good explanation. I will post this into the ticket >> and we'll take it into account when finishing the review. >> Thanks again for your contribution! >> /siri >> >> >> 2013/8/23 Tobias Schlager >> >>> Hi Siri, >>> >>> glad to hear from you, I'll try to do my best to explain the use case I >>> have in mind. >>> >>> Consider you have a one_for_one (or simple_one_for_one) supervisor A >>> with a worker child B that dynamically adds children to A (using >>> supervisor:start_child/2). Now consider these children also are supervisors >>> of type C with various statically configured workers. I now would like to >>> monitor supervisors of type C from worker B to be able to take some action >>> when *something goes wrong* at one of the C supervisors (e.g. C crashed >>> because of one of its subworkers). However, I can't differentiate between >>> 'something went wrong' or a supervisor C just exited gracefully (e.g. the >>> application was stopped) because supervisors only exit with reason normal >>> or shutdown. It is arguable whether to use another restart type for C >>> supervisors in order to propagate the exit. However, I don't want to crash >>> the whole supervision just to be able to tell that something failed to >>> restart somewhere down the supervision path. >>> >>> In general, the new exit reasons are visible to all processes linked >>> with supervisors or monitoring them (so parent supervisors as well as the >>> application master will see these reasons). This is why I chose the >>> '{shutdown, Reason}' format, which must be supported (according to the >>> documentation this is considered a normal exit reason). Thus, changing the >>> exit reasons will not affect the behaviour of supervision hierarchies >>> (verified by the test suite) or the application master (as far as I can >>> tell). The backward incompatibilty is located in processes depending on the >>> undocumented behaviour of supervisors always exiting with normal or >>> shutdown and not with '{shutdown, Reason}'. >>> >>> I hope, that my explanation makes things a bit clearer (and not worse). >>> >>> Regards >>> Tobias >>> >>> ------------------------------ >>> *Von:* Siri Hansen [erlangsiri@REDACTED] >>> *Gesendet:* Freitag, 23. August 2013 09:50 >>> *An:* Tobias Schlager >>> *Cc:* erlang-patches@REDACTED >>> *Betreff:* Re: [erlang-patches] Supervisor shutdown reason when >>> reaching max restarts >>> >>> Hi Tobias! >>> Thank you for the patch. We have discussed this on OTP Technical Board, >>> and have come to the conclusion that some more investigation is needed of >>> the potential backwards incompatibility. I have written a ticket and the >>> job will be prioritized into our backlog. Unfortunately we won't make it >>> before the next release (R16B02). >>> >>> In order to help us a bit on the way, could you please provide some >>> more information about your use case? You say that you are monitoring the >>> supervisor from another process, do you mean other process than the >>> supervisor's supervisor? If so, could you explain this architecture a bit >>> more? >>> >>> Who else will see this exit reason? - application_master? - the parent >>> supervisors? - other? >>> >>> Thanks again! >>> Regards >>> /siri >>> >>> >>> >>> 2013/7/4 Tobias Schlager >>> >>>> Hi, >>>> >>>> this patch changes the behaviour of supervisors to exit with a more >>>> specific reason when exiting due to a maximum restart limit hit. This is >>>> especially useful (or even necessary) to distinguish between normal and >>>> erroneous process terminations when monitoring a supervisor from another >>>> process. >>>> >>>> In the above case a supervisor would now exit with {shutdown, >>>> {reached_max_restart_intensity, Child}} where Child is whatever is >>>> available to describe the child, either a child id or in case of a >>>> simple_one_for_one supervisor the offending child's process id. The patch >>>> should not affect the OTP restart behaviour (also for cascaded supervisors) >>>> since a subclass of 'normal' exit reasons is used. >>>> >>>> I'm aware that there is some potential backward incompatibility for >>>> people that do not expect {shutdown, Reason} when monitoring a supervisor. >>>> However, the feature of exiting normally with {shutdown, Reason} has been >>>> around for quite a while now and I think this could be a sensible place to >>>> use it. Let me know what you think. >>>> >>>> The patch does include tests and updated documentation. >>>> >>>> git fetch https://github.com/schlagert/otp.gitsupervisor_shutdown_reason >>>> >>>> >>>> https://github.com/schlagert/otp/compare/erlang:master...supervisor_shutdown_reason >>>> >>>> https://github.com/schlagert/otp/compare/erlang:master...supervisor_shutdown_reason.patch >>>> >>>> Regards >>>> Tobias >>>> _______________________________________________ >>>> erlang-patches mailing list >>>> erlang-patches@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-patches >>>> >>> >>> >> > > _______________________________________________ > erlang-patches mailing list > erlang-patches@REDACTED > http://erlang.org/mailman/listinfo/erlang-patches > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Wed May 14 17:08:02 2014 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Wed, 14 May 2014 17:08:02 +0200 Subject: [erlang-patches] Fix efile_openfile() to handle stat() failure Message-ID: <21363.34514.633141.62348@gargle.gargle.HOWL> If the initial stat() fails then efile_openfile() will still proceed to open() the file. If that succeeds and the caller passed a non-NULL pSize, then it will copy bogus data from the statbuf into *pSize. This has been observed to cause file:read_file/1 to return truncated file data with no error indication. The use case involved a large file system mounted via NFS, with some directories containing large number of files, and NFS mount options that allow the NFS client to return EIO if the NFS server does not respond quickly enough. Depending on the caching state of the client and server machines, a few stat() calls (fewer than 1 per 10 million) would take long enough to trigger EIO errors, but subsequent open() calls would succeed, and read_file/1 would return truncated data. This sequence of events has been observed via "strace" on beam.smp. Signed-off-by: Mikael Pettersson Links: git fetch git://github.com/mikpe/otp.git openfile-dont-use-undefined-statbuf https://github.com/mikpe/otp/compare/erlang:maint...openfile-dont-use-undefined-statbuf https://github.com/mikpe/otp/compare/erlang:maint...openfile-dont-use-undefined-statbuf.patch https://github.com/erlang/otp/pull/368 From tobias.lindahl@REDACTED Mon May 26 15:26:02 2014 From: tobias.lindahl@REDACTED (Tobias Lindahl) Date: Mon, 26 May 2014 15:26:02 +0200 Subject: [erlang-patches] Improve mnesia_locker for large amounts of locks in the same transaction Message-ID: I've found that taking a large amount (from 1000 and upwards) of mnesia locks in the same transaction scales poorly. It seems to boil down to mnesia_locker keeping a bag ets table to store the reference between a transaction id (TID) and all locks held by the transaction. Changing the ets table to an ordered set table, and tweaking the code to preserve the same semantics seems to dramatically improve the scaling. If you need a test program to show the effect, just tell me. Links: https://github.com/erlang/otp/pull/382 git fetch git@REDACTED:gorillainduction/otp.git improve_mnesia_locker_complexity https://github.com/gorillainduction/otp/compare/erlang:maint...improve_mnesia_locker_complexity https://github.com/gorillainduction/otp/compare/erlang:maint...improve_mnesia_locker_complexity.patch -------------- next part -------------- An HTML attachment was scrubbed... URL: