From vladdu55@REDACTED Thu Jan 1 15:12:08 2015 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 1 Jan 2015 15:12:08 +0100 Subject: [erlang-patches] [jinterface] add erlang term parse/match/bind features In-Reply-To: <54A424CD.9010306@gmail.com> References: <54A31182.5070606@gmail.com> <54A424CD.9010306@gmail.com> Message-ID: Hi Dmitriy, I'm still going through your code, but I wonder if you could explain why equals should be implemented in terms of match? I think these are two different operations and can be relevant only if checking equals between a regular term and a pattern, but I can't see any reason to want to do that because these should never be equal. Or maybe I am missing a fine point?... I am also wondering if OEObject itself could implement OEMatcher and just call match/bind recursively instead of checking "instanceof OEMatcher" in OEList, OEMap and OETuple. It feels easier to understand (less conditionals in the code). In my implementation, I did it in a slightly different way, mostly in order to modify existing classes as little as possible, by using an utility class to traverse terms and do matching and binding. I also have more advanced conversion routines between Java and Erlang types (most useful are for Strings, List<->list and Array<->tuple/binary). Maybe you would like to check at https://github.com/vladdu/otp/compare/erlang:master...jinterface_new_api? Maybe we should try to unify our code first and then submit it to OTP? Happy New Year and I guess that for you it's even Merry Christmas in a week, right? Vlad On Wed, Dec 31, 2014 at 5:31 PM, Dmitriy Kargapolov < dmitriy.kargapolov@REDACTED> wrote: > Hi Vlad, > Thank you much for your notes. > > I agree some names were not good enough, I changed these per your > suggestion. > Old comment removed as well. > > As to OtpErangMap class - I was not going to re-implement it initially. > But in order to add match/bind I had to do this. I tried to not add much > new methods, keeping implementation rather "just enough" to work with maps. > You are very welcome to add/change whatever you see reasonable in separate > PR though. > > Best Regards and Happy New Year! > - Dmitriy. > > git fetch https://github.com/x0id/otp.git jinterface_pattern_matching > > > https://github.com/x0id/otp/compare/erlang:master...jinterface_pattern_matching > > https://github.com/x0id/otp/compare/erlang:master...jinterface_pattern_matching.patch > > > On 12/31/2014 10:27 AM, Vlad Dumitrescu wrote: > > Hi Dmitriy, > > Nice implementation! I have something similar that I wanted to submit, > but I like some of your details better. > > I have a few comments, after browsing the code just briefly: > > - I would prefer to have the OEMap changes as a separate PR, as it's a > separate issue and there are more things to address there, like for > example, OEMap could also implement Map, like OEList implements Iterable > (and might implement List). > - I don't like the name OEVarrier, it doesn't mean anything. Maybe > OEMatcher would be better? > - OEBind might be clearer as OEBinding, as it contains a set of bindings? > - I think there are still some merge issues, like for example OEList:299 > where the comment is from the old equals method > > best regards, > Vlad > > > On Tue, Dec 30, 2014 at 9:56 PM, Dmitriy Kargapolov < > dmitriy.kargapolov@REDACTED> wrote: > >> This implements functionality similar to following C functions, which are >> part of erl_interface application: >> - ETERM *erl_format(FormatStr, ...); >> - int erl_match(ETERM *Pattern, ETERM *Term); >> >> To acheve this new classes introduced: >> * OtpErlangVar - variable placeholder; >> * OtpErlangBind - variable values collection; >> * OtpErlangParser - "erl_format" parser implementation; >> * OtpErlangPattern - pattern abstraction with match/bind functions; >> >> Classes representing composite objects OtpErlangList, OtpErlangTuple, >> OtpErlangMap and new OtpErlangVar implement interface OtpErlangVarrier >> defining match and bind functions for these objects. >> >> Class OtpErlangMap reworked to be based on HashMap instead of two >> separate lists keeping keys and values. This is close to native semantics >> of maps and makes easier implementing basic map manipulations. >> >> It addition to OtpErlangBind custom user's class may be used as receiver >> of matched variables values. Java reflection is used to prepare variable >> value setters during the parse stage. Java doc has more details and >> examples. Test cases implemented. >> >> git fetch https://github.com/x0id/otp.git jinterface_pattern_matching >> >> Thanks. >> >> >> >> >> >> _______________________________________________ >> erlang-patches mailing list >> erlang-patches@REDACTED >> http://erlang.org/mailman/listinfo/erlang-patches >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitriy.kargapolov@REDACTED Thu Jan 1 20:26:21 2015 From: dmitriy.kargapolov@REDACTED (Dmitriy Kargapolov) Date: Thu, 01 Jan 2015 14:26:21 -0500 Subject: [erlang-patches] [jinterface] add erlang term parse/match/bind features In-Reply-To: References: <54A31182.5070606@gmail.com> <54A424CD.9010306@gmail.com> Message-ID: <54A59F5D.5050503@gmail.com> Hi Vlad, While continuing study your code I tried to answer your questions. Reading this please keep in mind the difference in our approaches. In my approach existing objects structure is re-used to keep pattern elements. This was done for the sake of simplicity, to keep code less polluted with new classes and interfaces. JInterface initially announced as low-level interface to the Erlang, and I tried to follow this way. On 01/01/2015 09:12 AM, Vlad Dumitrescu wrote: > Hi Dmitriy, > > I'm still going through your code, but I wonder if you could explain > why equals should be implemented in terms of match? I think these are > two different operations and can be relevant only if checking equals > between a regular term and a pattern, but I can't see any reason to > want to do that because these should never be equal. Or maybe I am > missing a fine point?... This was done for simplicity, to avoid duplicating the code, especially when operating on complex data like lists, tuples, maps. Less code - less errors. Term considered to be pattern in opposite to regular object if it contains variable placeholders. In such case it should not be used in any way but for matching against other regular terms. (Regular term can be considered as an edge case of pattern with no variables and therefore can also be used as left side of the match operation). In the light of above, calling equals method for the pattern(s) does not make any sense and should not bother anyone. For regular term(s) match and equals methods carry same semantics. I know that equals method now has some runtime overheads. Imho this is reasonable sacrificing. The equals() method is not much important. Java code must preferably work with Java objects, not Erlang terms. What is really important is to have effective way of importing Erlang terms translating them to what is suitable for Java code. So much() implementation is a priority over equals(). > > I am also wondering if OEObject itself could implement OEMatcher and > just call match/bind recursively instead of checking "instanceof > OEMatcher" in OEList, OEMap and OETuple. It feels easier to understand > (less conditionals in the code). In such case match/bin would be methods of all objects, including atoms, integers, strings etc. This does not make much sense imho. I don?t think it is reasonable to replace one imperfection by another one. > > In my implementation, I did it in a slightly different way, mostly in > order to modify existing classes as little as possible, by using an > utility class to traverse terms and do matching and binding. Our approaches are fundamentally different. I also tried to keep as much code untouched as possible, but was reasonable lazy to re-create complex Erlang terms structure in separate pattern object. My approach is a compromise. It does require touching the core code. Your approach is smart way to handle pattern matching without touching base code. It may be (I guess) made as an extension to core JInterface in separate jar having JInterface as a dependency. Frankly I didn?t have enough time to follow your way, my code contains less modifications and additions to the JInterface and probably took less efforts to implement. > I also have more advanced conversion routines between Java and Erlang > types (most useful are for Strings, List<->list and > Array<->tuple/binary). Maybe you would like to check > at https://github.com/vladdu/otp/compare/erlang:master...jinterface_new_api? I made for my project additional abstractions for type conversion including lists of objects and maps. I?m not ready to share this, since it is not perfect at all. And I don?t see any comprehensive generic way of mapping Erlang data to/from Java data. Anyway, I think that such stuff may be a separate library dedicated to type conversions and other aspects of encode/decode of complex user?s data. JInterface as a low-level library looks better when free of such additions. It is enough to automate Erlang term structure matching moving type verification and conversion to the next layer of abstraction or to user?s code (like custom binders in my case). > > Maybe we should try to unify our code first and then submit it to OTP? First of all, we (and community) should vote for the core approach, answering the question: should we intervene into the base code or make parallel patterns object structure? My opinion is that making parallel structures is more error-prone and makes maintenance harder due to more similar code to repeat in different places. Second (less important though) question would be: how reasonable is to add fancy data conversion helpers to the base JInterface library. In the code I submitted there is may be space for improvements in the OEParser class. If we'd like to bring basic data type verification/conversions to the code. I would prefer to avoid this though, at least for the initial version. > > Happy New Year and I guess that for you it's even Merry Christmas in a > week, right? Yes, thank you! > Vlad > > > On Wed, Dec 31, 2014 at 5:31 PM, Dmitriy Kargapolov > > > wrote: > > Hi Vlad, > Thank you much for your notes. > > I agree some names were not good enough, I changed these per your > suggestion. > Old comment removed as well. > > As to OtpErangMap class - I was not going to re-implement it > initially. But in order to add match/bind I had to do this. I > tried to not add much new methods, keeping implementation rather > "just enough" to work with maps. You are very welcome to > add/change whatever you see reasonable in separate PR though. > > Best Regards and Happy New Year! > - Dmitriy. > > git fetch https://github.com/x0id/otp.git jinterface_pattern_matching > > https://github.com/x0id/otp/compare/erlang:master...jinterface_pattern_matching > https://github.com/x0id/otp/compare/erlang:master...jinterface_pattern_matching.patch > > > > On 12/31/2014 10:27 AM, Vlad Dumitrescu wrote: >> Hi Dmitriy, >> >> Nice implementation! I have something similar that I wanted to >> submit, but I like some of your details better. >> >> I have a few comments, after browsing the code just briefly: >> >> - I would prefer to have the OEMap changes as a separate PR, as >> it's a separate issue and there are more things to address there, >> like for example, OEMap could also implement Map, like OEList >> implements Iterable (and might implement List). >> - I don't like the name OEVarrier, it doesn't mean anything. >> Maybe OEMatcher would be better? >> - OEBind might be clearer as OEBinding, as it contains a set of >> bindings? >> - I think there are still some merge issues, like for example >> OEList:299 where the comment is from the old equals method >> >> best regards, >> Vlad >> >> >> On Tue, Dec 30, 2014 at 9:56 PM, Dmitriy Kargapolov >> > > wrote: >> >> This implements functionality similar to following C >> functions, which are part of erl_interface application: >> - ETERM *erl_format(FormatStr, ...); >> - int erl_match(ETERM *Pattern, ETERM *Term); >> >> To acheve this new classes introduced: >> * OtpErlangVar - variable placeholder; >> * OtpErlangBind - variable values collection; >> * OtpErlangParser - "erl_format" parser implementation; >> * OtpErlangPattern - pattern abstraction with match/bind >> functions; >> >> Classes representing composite objects OtpErlangList, >> OtpErlangTuple, OtpErlangMap and new OtpErlangVar implement >> interface OtpErlangVarrier defining match and bind functions >> for these objects. >> >> Class OtpErlangMap reworked to be based on HashMap instead of >> two separate lists keeping keys and values. This is close to >> native semantics of maps and makes easier implementing basic >> map manipulations. >> >> It addition to OtpErlangBind custom user's class may be used >> as receiver of matched variables values. Java reflection is >> used to prepare variable value setters during the parse >> stage. Java doc has more details and examples. Test cases >> implemented. >> >> git fetch https://github.com/x0id/otp.git >> jinterface_pattern_matching >> >> Thanks. >> >> >> >> >> >> _______________________________________________ >> erlang-patches mailing list >> erlang-patches@REDACTED >> http://erlang.org/mailman/listinfo/erlang-patches >> >> > > From dmitriy.kargapolov@REDACTED Sun Jan 4 19:02:06 2015 From: dmitriy.kargapolov@REDACTED (Dmitriy Kargapolov) Date: Sun, 04 Jan 2015 13:02:06 -0500 Subject: [erlang-patches] [jinterface] transport factory implementation Message-ID: <54A9801E.5080109@gmail.com> Transport factory basic implementation added. This makes possible creating connections between nodes using ssh channels or ssl. Default transport factory based on standart Socket/ServerSocket classes is provided. Modifications are backward compatible. Custom transport factory may be specified using 2 ways - via system property or defining special tuner class. Documentation updated. Test cases added. git fetch https://github.com/x0id/otp.git jinterface_transport_factory https://github.com/x0id/otp/compare/erlang:master...jinterface_transport_factory https://github.com/x0id/otp/compare/erlang:master...jinterface_transport_factory.patch Thanks. From vladdu55@REDACTED Tue Jan 6 17:07:12 2015 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 6 Jan 2015 17:07:12 +0100 Subject: [erlang-patches] [jinterface] transport factory implementation In-Reply-To: <54A9801E.5080109@gmail.com> References: <54A9801E.5080109@gmail.com> Message-ID: Hi! Thanks for this. I will look closer at the code in the next days. My first comment is that I think there should be more examples on how to set up a different transport and maybe on how to implement other transports. I suppose that a prerequisite for this to work is that the Erlang nodes and epmd support and use that transport method too? best regards, Vlad On Sun, Jan 4, 2015 at 7:02 PM, Dmitriy Kargapolov < dmitriy.kargapolov@REDACTED> wrote: > Transport factory basic implementation added. This makes possible > creating connections between nodes using ssh channels or ssl. > Default transport factory based on standart Socket/ServerSocket > classes is provided. Modifications are backward compatible. > Custom transport factory may be specified using 2 ways - via > system property or defining special tuner class. > Documentation updated. Test cases added. > > git fetch https://github.com/x0id/otp.git jinterface_transport_factory > > > https://github.com/x0id/otp/compare/erlang:master...jinterface_transport_factory > > https://github.com/x0id/otp/compare/erlang:master...jinterface_transport_factory.patch > > Thanks. > > > > > > > _______________________________________________ > erlang-patches mailing list > erlang-patches@REDACTED > http://erlang.org/mailman/listinfo/erlang-patches > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitriy.kargapolov@REDACTED Wed Jan 7 21:35:16 2015 From: dmitriy.kargapolov@REDACTED (Dmitriy Kargapolov) Date: Wed, 07 Jan 2015 15:35:16 -0500 Subject: [erlang-patches] [jinterface] transport factory implementation In-Reply-To: References: <54A9801E.5080109@gmail.com> Message-ID: <54AD9884.6010407@gmail.com> Two test cases show how to setup a custom transport in different ways, may be considered as examples. In general there are no prerequisites from other components. One use case is implementing ssh-tunnel for Java client while connecting to regular Erlang node. This is useful for making secure connection between Java client and Erlang server node(s). Such a transport could use, for example JSch for client, on the server side there will be regular TCP connection established from sshd to beam (or to epmd process, when client queries EPMD). Other use may, of course, require some development on Erlang side as well. This is not covered by this work. On 01/06/2015 11:07 AM, Vlad Dumitrescu wrote: > Hi! > > Thanks for this. I will look closer at the code in the next days. My > first comment is that I think there should be more examples on how to > set up a different transport and maybe on how to implement other > transports. I suppose that a prerequisite for this to work is that the > Erlang nodes and epmd support and use that transport method too? > > best regards, > Vlad > > > On Sun, Jan 4, 2015 at 7:02 PM, Dmitriy Kargapolov > > > wrote: > > Transport factory basic implementation added. This makes possible > creating connections between nodes using ssh channels or ssl. > Default transport factory based on standart Socket/ServerSocket > classes is provided. Modifications are backward compatible. > Custom transport factory may be specified using 2 ways - via > system property or defining special tuner class. > Documentation updated. Test cases added. > > git fetch https://github.com/x0id/otp.git jinterface_transport_factory > > https://github.com/x0id/otp/compare/erlang:master...jinterface_transport_factory > https://github.com/x0id/otp/compare/erlang:master...jinterface_transport_factory.patch > > Thanks. > > > > > > > _______________________________________________ > erlang-patches mailing list > erlang-patches@REDACTED > http://erlang.org/mailman/listinfo/erlang-patches > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Thu Jan 8 20:28:55 2015 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Thu, 8 Jan 2015 20:28:55 +0100 Subject: [erlang-patches] [HiPE] fix fconv translation and fmove generation Message-ID: <21678.55927.474222.913952@gargle.gargle.HOWL> The following test case crashes the HiPE compiler on all targets except ARM: ==snip== -module(trivial_13). -export([test/0]). test() -> return_it(42). return_it(X) -> R = (2 * X) / 2, round(R). ==snip== This actually triggers two independent bugs: RTL produces an #fconv{} instruction with an immediate operand, but the backends unconditionally access the operand as a temporary. This results in broken representation in the backends and eventually they crash. In this case, the bug is in the backends. The backends must check for immediate operands in #fconv{} and translate them accordingly for their respective targets. With the above fixed the backends still crash, generally with complaints about inconsistent and/or incomplete register allocation. This happens because hipe_rtl:phi_remove_pred/2 produces a #move{} instruction with floating-point temporaries as operands, even though such moves MUST be #fmove{} instructions. This is fixed by updating phi_remove_pred/2 to check for and handle floating-point moves. I also added type checks to the #move{} and #fmove{} constructor and setter functions to ensure that similar mishaps cannot happen again. Tested with the HiPE test suite on x86_64, sparc64 (32-bit), and powerpc64 (32-bit): fixes the test case with no new failures. The ARM backend is still "soft-float" and not affected by these problems. (This test case has actually been in the HiPE test suite for a long time, but it didn't fail because it was compiled in single-MFA mode (only return_it/1 was compiled), limiting the effectiveness of interprocedural optimizations. That mode was removed some time ago (R15?), and compiling the test case in whole-module mode exposed the problems.) Signed-off-by: Mikael Pettersson Links: git fetch git://github.com/mikpe/otp.git hipe-fconv-fmove-fixes https://github.com/mikpe/otp/compare/erlang:maint...hipe-fconv-fmove-fixes https://github.com/mikpe/otp/compare/erlang:maint...hipe-fconv-fmove-fixes.patch https://github.com/erlang/otp/pull/580 From mikpelinux@REDACTED Sun Jan 11 14:35:58 2015 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Sun, 11 Jan 2015 14:35:58 +0100 Subject: [erlang-patches] hipe: fix ARM/Thumb interworking Message-ID: <21682.31806.506207.353586@gargle.gargle.HOWL> HiPE on ARM is currently severely broken if the rest of the VM is compiled to run in Thumb mode -- calling native code quickly ends up executing code in the wrong mode and crashing the VM. This is a problem on e.g. Ubuntu which configures its system GCC to generate Thumb by default. It can also be triggered by overriding CC or CFLAGS when compiling the VM. There were three issues that caused the breakage: 1. Assembly-coded functions in hipe_arm_glue.S weren't explicitly tagged as functions, preventing the linker from generating the correct mode-switching call instructions for calls from C to these functions. Fixed by tagging those symbols as functions. 2. A few BIF wrappers were so simple that they performed tailcalls to the C BIFs. This fails to switch mode when C is in Thumb. Fixed by performing ordinary recursive calls when C is in Thumb. 3. The assembly-coded source files weren't explicitly tagged as ARM. Tested with the HiPE testsuite on ARMv7, with the VM built as ARM and as Thumb. Also manually inspected the object code for the beam executable and checked that call sites from C to HiPE's ARM runtime code and vice versa used the correct mode-switching instructions. Signed-off-by: Mikael Pettersson Links: git fetch git://github.com/mikpe/otp.git hipe-arm-interworking https://github.com/mikpe/otp/compare/erlang:maint...hipe-arm-interworking https://github.com/mikpe/otp/compare/erlang:maint...hipe-arm-interworking.patch https://github.com/erlang/otp/pull/583 From mikpelinux@REDACTED Sun Jan 11 17:43:14 2015 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Sun, 11 Jan 2015 17:43:14 +0100 Subject: [erlang-patches] hipe: code allocation fixes Message-ID: <21682.43042.991749.913799@gargle.gargle.HOWL> This branch contains cleanups and fixes related to memory allocation for code in the HiPE runtime. - it changes generic code to raise Erlang-level exceptions on allocation failures - it changes platform-specific code to signal allocation failures rather than crashing the VM (with abort() or SEGV from NULL pointer dereferences) - it removes obsolete and disabled BIFs related to code management - it eliminates a useless internal platform-specific macro Tested with the HiPE testsuite on x86_64 (-m64 and -m32), sparc64 -m32, armv7, and ppc64 -m32. Signed-off-by: Mikael Pettersson Links: git fetch git://github.com/mikpe/otp.git hipe-code-alloc-fixes https://github.com/mikpe/otp/compare/erlang:maint...hipe-code-alloc-fixes https://github.com/mikpe/otp/compare/erlang:maint...hipe-code-alloc-fixes.patch https://github.com/erlang/otp/pull/584 From erlangsiri@REDACTED Thu Jan 15 16:00:34 2015 From: erlangsiri@REDACTED (Siri Hansen) Date: Thu, 15 Jan 2015 16:00:34 +0100 Subject: [erlang-patches] Add supervisor:start_child/3 to limit the number of children In-Reply-To: References: <20130403121637.GB338@aluminium.local> <20130403122904.GA82915@ferdair.local> <20130403125325.GD338@aluminium.local> <20130410120812.GB99642@aluminium.local> <20130410125109.GA24064@ferdair.local> <20130411054741.GC99642@aluminium.local> Message-ID: Hi Vance, I'm happy to announce that in OTP-18, the supervisor API will be improved to allow supervisor flags and childspecs declared as maps (similar to http://erlang.org/pipermail/erlang-patches/2012-January/002574.html, except maps are used instead of proplists). This will allow for additions to these data structures. We now wonder if the functionality handled in the current thread is still wanted, and if so if the patch should be rewritten to use the new API? Regards /siri 2013-04-12 18:53 GMT+02:00 Siri Hansen : > Hi Vance, > > sorry for the delayed answer. We have had some discussions within our team > and we do find your idea interesting. We do, however, not really like the > idea of setting the limit in the call to supervisor:start_child, but rather > think it should be possible to set such a property on the supervisor or in > the child spec. The drawback of this, as you say in an earlier mail, is > that it would require changing the supervisor spec - and that is a much > bigger change. We do, however, plan to make the supervisor API a bit more > flexible and by that adding the possibility of introducing new properties. > > The timeframe for this is not yet set, and a contribution would of course > help speeding things up :) We already got a patch a good year ago ( > http://erlang.org/pipermail/erlang-patches/2012-January/002574.html) but > it was never completed... > > /siri > > > 2013/4/11 Vance Shipley > >> On Wed, Apr 10, 2013 at 08:51:11AM -0400, Fred Hebert wrote: >> } Have you considered using ETS counters, and possibly a monitor process? >> ... >> } If you're in the kind of position where you need to limit the number of >> } transactions to avoid falling over, it will *not* reduce the number of >> } messages sent to the supervisor, and if you start going over the top, >> } you'll time out no matter what, just because the supervisor won't be >> } able to keep up with the demand. >> >> Fred, >> >> Your approach is quite valid however it addresses an issue I am not as >> yet considering. I am concerned not about overload protection but in >> policy enforcement. The supervisor should have no more than N workers. >> The correct place to address that issue is in the supervisor. True, I >> could address it otherwise but I propose a small change to support this >> in the OTP implementation. >> >> The alternative solution which my coworkers have historically used is >> long lived pools of processes. I believe that the Erlang way is to have >> a process with a life cycle matching the transaction's. It makes me much >> happier to eliminate the pools. >> >> -- >> -Vance >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Fri Jan 16 12:17:50 2015 From: vances@REDACTED (Vance Shipley) Date: Fri, 16 Jan 2015 16:47:50 +0530 Subject: [erlang-patches] Add supervisor:start_child/3 to limit the number of children In-Reply-To: References: <20130403121637.GB338@aluminium.local> <20130403122904.GA82915@ferdair.local> <20130403125325.GD338@aluminium.local> <20130410120812.GB99642@aluminium.local> <20130410125109.GA24064@ferdair.local> <20130411054741.GC99642@aluminium.local> Message-ID: On Thu, Jan 15, 2015 at 8:30 PM, Siri Hansen wrote: > I'm happy to announce that in OTP-18, the supervisor API will be improved to allow supervisor flags and childspecs declared as maps (similar to http://erlang.org/pipermail/erlang-patches/2012-January/002574.html, except maps are used instead of proplists). This will allow for additions to these data structures. We now wonder if the functionality handled in the current thread is still wanted, and if so if the patch should be rewritten to use the new API? Yes, indeed it is. I will submit a new patch. -- -Vance From tuncer.ayaz@REDACTED Fri Jan 16 13:06:32 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 16 Jan 2015 13:06:32 +0100 Subject: [erlang-patches] Add supervisor:start_child/3 to limit the number of children In-Reply-To: References: <20130403121637.GB338@aluminium.local> <20130403122904.GA82915@ferdair.local> <20130403125325.GD338@aluminium.local> <20130410120812.GB99642@aluminium.local> <20130410125109.GA24064@ferdair.local> <20130411054741.GC99642@aluminium.local> Message-ID: > On Thu, Jan 15, 2015 at 8:30 PM, Siri Hansen wrote: > I'm happy to announce that in OTP-18, the supervisor API will be > improved to allow supervisor flags and childspecs declared as maps > (similar to > http://erlang.org/pipermail/erlang-patches/2012-January/002574.html, > except maps are used instead of proplists). This will allow for > additions to these data structures. To be totally clear, it will be optional and require 18.x, but existing code will still work, right? From erlangsiri@REDACTED Fri Jan 16 14:15:09 2015 From: erlangsiri@REDACTED (Siri Hansen) Date: Fri, 16 Jan 2015 14:15:09 +0100 Subject: [erlang-patches] Add supervisor:start_child/3 to limit the number of children In-Reply-To: References: <20130403121637.GB338@aluminium.local> <20130403122904.GA82915@ferdair.local> <20130403125325.GD338@aluminium.local> <20130410120812.GB99642@aluminium.local> <20130410125109.GA24064@ferdair.local> <20130411054741.GC99642@aluminium.local> Message-ID: 2015-01-16 12:17 GMT+01:00 Vance Shipley : > On Thu, Jan 15, 2015 at 8:30 PM, Siri Hansen wrote: > > I'm happy to announce that in OTP-18, the supervisor API will be > improved to allow supervisor flags and childspecs declared as maps (similar > to http://erlang.org/pipermail/erlang-patches/2012-January/002574.html, > except maps are used instead of proplists). This will allow for additions > to these data structures. We now wonder if the functionality handled in the > current thread is still wanted, and if so if the patch should be rewritten > to use the new API? > > Yes, indeed it is. I will submit a new patch. > Great! As you know we need to be extra careful when including new functionality in the OTP base, so we might need to go another round with OTP Technical Board. So to avoid wasting time it might be a good idea to give us a short description of the design (including API) before implementing the details and we will give you feedback as soon as possible. /siri -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Fri Jan 16 16:16:08 2015 From: erlangsiri@REDACTED (Siri Hansen) Date: Fri, 16 Jan 2015 16:16:08 +0100 Subject: [erlang-patches] Fwd: Add supervisor:start_child/3 to limit the number of children In-Reply-To: References: <20130403121637.GB338@aluminium.local> <20130403122904.GA82915@ferdair.local> <20130403125325.GD338@aluminium.local> <20130410120812.GB99642@aluminium.local> <20130410125109.GA24064@ferdair.local> <20130411054741.GC99642@aluminium.local> Message-ID: sorry, forgot to copy the list... ---------- Forwarded message ---------- From: Siri Hansen Date: 2015-01-16 14:07 GMT+01:00 Subject: Re: [erlang-patches] Add supervisor:start_child/3 to limit the number of children To: Tuncer Ayaz 2015-01-16 13:06 GMT+01:00 Tuncer Ayaz : > > On Thu, Jan 15, 2015 at 8:30 PM, Siri Hansen wrote: > > I'm happy to announce that in OTP-18, the supervisor API will be > > improved to allow supervisor flags and childspecs declared as maps > > (similar to > > http://erlang.org/pipermail/erlang-patches/2012-January/002574.html, > > except maps are used instead of proplists). This will allow for > > additions to these data structures. > > To be totally clear, it will be optional and require 18.x, but > existing code will still work, right? > That is correct! It was merged to master on November 6th - this is the merge commit: https://github.com/erlang/otp/commit/eeba41c3f9a56948d62de66065851f509ae02b43 - if you want to look at the details... /siri -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel@REDACTED Sat Jan 17 09:51:24 2015 From: daniel@REDACTED (Daniel White) Date: Sat, 17 Jan 2015 19:51:24 +1100 Subject: [erlang-patches] Prevent zip:zip_open/1,2 from leaking ports Message-ID: The case was discovered where a parent process would exit before closing the zip file. The result was that a port would be left open indefinitely, as the small zip server would not detect this condition. By comparison, the file module will close the associated port when the parent exits for any reason. This change would make the zip module more consistent with the semantics of similar modules. This change is breaking for any callers expecting to pass the handle to another process for processing (assuming it exits). With this in mind, it may need a bit more thought with regards to: 1. Is this the best way to cope with a parent process exiting? 2. Should an equivalent of gen_tcp:controlling_process/2 to deal with migrating an open zip to another process? 3. If backwards compatibility is necessary, then would it be worth introducing new functions without the "zip_" prefix? git fetch git://github.com/danielwhite/otp.git zip-port-leak-fix https://github.com/danielwhite/otp/compare/erlang:master...zip-port-leak-fix https://github.com/danielwhite/otp/compare/erlang:master...zip-port-leak-fix.patch https://github.com/erlang/otp/pull/587 -- Daniel White From dmitriy.kargapolov@REDACTED Sun Jan 18 18:18:47 2015 From: dmitriy.kargapolov@REDACTED (Dmitriy Kargapolov) Date: Sun, 18 Jan 2015 12:18:47 -0500 Subject: [erlang-patches] [jinterface] add erlang term parse/match/bind features In-Reply-To: <54A59F5D.5050503@gmail.com> References: <54A31182.5070606@gmail.com> <54A424CD.9010306@gmail.com> <54A59F5D.5050503@gmail.com> Message-ID: Hi All, We couldn't find easy way to merge our implementations at this time. Instead I submitted PR #589: https://github.com/erlang/otp/pull/589, which comprises of very small addition to the JInterface library (match() and bind() generic methods). On top of this base change both approaches can be implemented (even with no more modifications to the core JInterface library). Thanks. On Thu, Jan 1, 2015 at 2:26 PM, Dmitriy Kargapolov < dmitriy.kargapolov@REDACTED> wrote: > Hi Vlad, > While continuing study your code I tried to answer your questions. > Reading this please keep in mind the difference in our approaches. In my > approach existing objects structure is re-used to keep pattern elements. > This was done for the sake of simplicity, to keep code less polluted > with new classes and interfaces. JInterface initially announced as > low-level interface to the Erlang, and I tried to follow this way. > > On 01/01/2015 09:12 AM, Vlad Dumitrescu wrote: > > Hi Dmitriy, > > > > I'm still going through your code, but I wonder if you could explain > > why equals should be implemented in terms of match? I think these are > > two different operations and can be relevant only if checking equals > > between a regular term and a pattern, but I can't see any reason to > > want to do that because these should never be equal. Or maybe I am > > missing a fine point?... > This was done for simplicity, to avoid duplicating the code, especially > when operating on complex data like lists, tuples, maps. Less code - > less errors. > Term considered to be pattern in opposite to regular object if it > contains variable placeholders. In such case it should not be used in > any way but for matching against other regular terms. (Regular term can > be considered as an edge case of pattern with no variables and therefore > can also be used as left side of the match operation). > In the light of above, calling equals method for the pattern(s) does not > make any sense and should not bother anyone. For regular term(s) match > and equals methods carry same semantics. > I know that equals method now has some runtime overheads. Imho this is > reasonable sacrificing. The equals() method is not much important. Java > code must preferably work with Java objects, not Erlang terms. What is > really important is to have effective way of importing Erlang terms > translating them to what is suitable for Java code. So much() > implementation is a priority over equals(). > > > > I am also wondering if OEObject itself could implement OEMatcher and > > just call match/bind recursively instead of checking "instanceof > > OEMatcher" in OEList, OEMap and OETuple. It feels easier to understand > > (less conditionals in the code). > In such case match/bin would be methods of all objects, including atoms, > integers, strings etc. This does not make much sense imho. I don?t think > it is reasonable to replace one imperfection by another one. > > > > In my implementation, I did it in a slightly different way, mostly in > > order to modify existing classes as little as possible, by using an > > utility class to traverse terms and do matching and binding. > Our approaches are fundamentally different. I also tried to keep as much > code untouched as possible, but was reasonable lazy to re-create complex > Erlang terms structure in separate pattern object. My approach is a > compromise. It does require touching the core code. Your approach is > smart way to handle pattern matching without touching base code. It may > be (I guess) made as an extension to core JInterface in separate jar > having JInterface as a dependency. Frankly I didn?t have enough time to > follow your way, my code contains less modifications and additions to > the JInterface and probably took less efforts to implement. > > > I also have more advanced conversion routines between Java and Erlang > > types (most useful are for Strings, List<->list and > > Array<->tuple/binary). Maybe you would like to check > > at > https://github.com/vladdu/otp/compare/erlang:master...jinterface_new_api? > I made for my project additional abstractions for type conversion > including lists of objects and maps. I?m not ready to share this, since > it is not perfect at all. And I don?t see any comprehensive generic way > of mapping Erlang data to/from Java data. Anyway, I think that such > stuff may be a separate library dedicated to type conversions and other > aspects of encode/decode of complex user?s data. JInterface as a > low-level library looks better when free of such additions. It is enough > to automate Erlang term structure matching moving type verification and > conversion to the next layer of abstraction or to user?s code (like > custom binders in my case). > > > > > Maybe we should try to unify our code first and then submit it to OTP? > First of all, we (and community) should vote for the core approach, > answering the question: should we intervene into the base code or make > parallel patterns object structure? My opinion is that making parallel > structures is more error-prone and makes maintenance harder due to more > similar code to repeat in different places. > Second (less important though) question would be: how reasonable is to > add fancy data conversion helpers to the base JInterface library. > In the code I submitted there is may be space for improvements in the > OEParser class. If we'd like to bring basic data type > verification/conversions to the code. I would prefer to avoid this > though, at least for the initial version. > > > > Happy New Year and I guess that for you it's even Merry Christmas in a > > week, right? > Yes, thank you! > > > Vlad > > > > > > On Wed, Dec 31, 2014 at 5:31 PM, Dmitriy Kargapolov > > > > > wrote: > > > > Hi Vlad, > > Thank you much for your notes. > > > > I agree some names were not good enough, I changed these per your > > suggestion. > > Old comment removed as well. > > > > As to OtpErangMap class - I was not going to re-implement it > > initially. But in order to add match/bind I had to do this. I > > tried to not add much new methods, keeping implementation rather > > "just enough" to work with maps. You are very welcome to > > add/change whatever you see reasonable in separate PR though. > > > > Best Regards and Happy New Year! > > - Dmitriy. > > > > git fetch https://github.com/x0id/otp.git > jinterface_pattern_matching > > > > > https://github.com/x0id/otp/compare/erlang:master...jinterface_pattern_matching > > > https://github.com/x0id/otp/compare/erlang:master...jinterface_pattern_matching.patch > > > > > > > > On 12/31/2014 10:27 AM, Vlad Dumitrescu wrote: > >> Hi Dmitriy, > >> > >> Nice implementation! I have something similar that I wanted to > >> submit, but I like some of your details better. > >> > >> I have a few comments, after browsing the code just briefly: > >> > >> - I would prefer to have the OEMap changes as a separate PR, as > >> it's a separate issue and there are more things to address there, > >> like for example, OEMap could also implement Map, like OEList > >> implements Iterable (and might implement List). > >> - I don't like the name OEVarrier, it doesn't mean anything. > >> Maybe OEMatcher would be better? > >> - OEBind might be clearer as OEBinding, as it contains a set of > >> bindings? > >> - I think there are still some merge issues, like for example > >> OEList:299 where the comment is from the old equals method > >> > >> best regards, > >> Vlad > >> > >> > >> On Tue, Dec 30, 2014 at 9:56 PM, Dmitriy Kargapolov > >> >> > wrote: > >> > >> This implements functionality similar to following C > >> functions, which are part of erl_interface application: > >> - ETERM *erl_format(FormatStr, ...); > >> - int erl_match(ETERM *Pattern, ETERM *Term); > >> > >> To acheve this new classes introduced: > >> * OtpErlangVar - variable placeholder; > >> * OtpErlangBind - variable values collection; > >> * OtpErlangParser - "erl_format" parser implementation; > >> * OtpErlangPattern - pattern abstraction with match/bind > >> functions; > >> > >> Classes representing composite objects OtpErlangList, > >> OtpErlangTuple, OtpErlangMap and new OtpErlangVar implement > >> interface OtpErlangVarrier defining match and bind functions > >> for these objects. > >> > >> Class OtpErlangMap reworked to be based on HashMap instead of > >> two separate lists keeping keys and values. This is close to > >> native semantics of maps and makes easier implementing basic > >> map manipulations. > >> > >> It addition to OtpErlangBind custom user's class may be used > >> as receiver of matched variables values. Java reflection is > >> used to prepare variable value setters during the parse > >> stage. Java doc has more details and examples. Test cases > >> implemented. > >> > >> git fetch https://github.com/x0id/otp.git > >> jinterface_pattern_matching > >> > >> Thanks. > >> > >> > >> > >> > >> > >> _______________________________________________ > >> erlang-patches mailing list > >> erlang-patches@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-patches > >> > >> > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cian@REDACTED Sun Jan 18 23:59:40 2015 From: cian@REDACTED (Cian Synnott) Date: Sun, 18 Jan 2015 22:59:40 +0000 Subject: [erlang-patches] Fix index for #person.address in create_table/2 Message-ID: A tiny documentation nit: {index, [2]} refers to #person.name rather than #person.address, which caused a little confusion in #erlang today. {index, [4]} is the correct "hard coded" field for #person.address. git fetch git://github.com/emauton/otp.git mnesia_create_table_docfix https://github.com/emauton/otp/compare/erlang:maint...mnesia_create_table_docfix https://github.com/emauton/otp/compare/erlang:maint...mnesia_create_table_docfix.patch https://github.com/erlang/otp/pull/591 Cian From serge@REDACTED Fri Jan 30 03:03:01 2015 From: serge@REDACTED (Serge Aleynikov) Date: Thu, 29 Jan 2015 21:03:01 -0500 Subject: [erlang-patches] inet:fdopen/2 fix for supporting externally open fd's of AF_LOCAL address family Message-ID: When a AF_LOCAL (a.k.a. AF_UNIX) file descriptor is created externally (e.g. Unix Domain Socket) and passed to `gen_tcp:listen(0, [{fd, FD}])`, the present implementation incorrectly assigns the address family to be equal to `inet`, which in the inet_drv driver translats to AF_INET instead of AF_LOCAL (or AF_UNIX), and an `einval` error code is returned. This patch fixes this problem such that the file descriptors of the `local` address family are properly supported when such a file descriptor is passed to the inet:fdopen/5, gen_tcp:connect/3, gen_tcp:listen/2, gen_udp:open/2 calls via {fd, FD::integer()} option. In order to connect a socket to a Unix Domain file descriptor use the following options: 1> FD = ... % Open the AF_LOCAL *server* file descriptor 2> {ok, Sock} = gen_tcp:listen(0, [local, {fd,FD} | OtherOptions]). % Now use the socket using gen_tcp module: 3> gen_tcp:send(Sock, <<"abc">>). FD = ... % Open the AF_LOCAL *client* file descriptor 2> {ok, Sock} = gen_tcp:connect(0, [local, {fd,FD} | OtherOptions]). % Now use the socket using gen_tcp module: 3> inet:setopts(S, [{active, once}]), 4> receive Msg -> Msg end. {tcp,#Port<0.1195>,"abc"} 5> inet:setopts(S, [{active, false}]). 6> gen_tcp:recv(S,0,1000). {ok,"efg"} Note that in case of UDP client in order to preserve the active socket signature {udp, ErlPort, Addr, Port, Data} and passive socket's gen_udp:recv3 return signature ({ok, {Address, Port, Packet}}), the Address on local socket family is a string containing the underlying socket's filename, and Port=0. E.g.: {ok, {"/tmp/test.sock", 0, <<"some data">>}} = gen_udp:recv(S, 0, 1000). git fetch https://github.com/saleyn/otp uds https://github.com/saleyn/otp/compare/erlang:maint...uds https://github.com/saleyn/otp/compare/erlang:maint...uds.patch A sample project using this patch can be found here: https://github.com/saleyn/euds. It uses a NIF library to create and FD of AF_LOCAL family, and passes it to either gen_tcp or gen_udp for further handling. Regards, Serge -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Fri Jan 30 09:52:15 2015 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 30 Jan 2015 09:52:15 +0100 Subject: [erlang-patches] (no subject) Message-ID: OTP-11660: make eunit unicode safe The most critical part is eunit_surefire, because the generated XML must be correct UTF-8, otherwise tools will refuse to work with it. The tricky part is that the input can contain mixed encodings and we can't know which encoding applies to each text chunk. This is because input can be from io:format in the tests. Also, different test modules might be encoded differently and that can't be detected at runtime. We have to avoid outputting both bad utf-8 and doubly encoded utf-8. The simplest solution is to let the encoder skip characters it chokes on, replacing them with '?'. > git fetch git://github.com/vladdu/otp.git eunit_unicode_OTP11660 > > https://github.com/vladdu/otp/compare/erlang > :master...eunit_unicode_OTP11660 > https://github.com/vladdu/otp/compare/erlang > :master...eunit_unicode_OTP11660.patch regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: