From magnus.ottenklinger@REDACTED Fri Jul 1 08:13:01 2016 From: magnus.ottenklinger@REDACTED (Magnus Ottenklinger) Date: Fri, 1 Jul 2016 06:13:01 +0000 Subject: [erlang-questions] A question on EINVAL in active-mode TCP In-Reply-To: References: , Message-ID: > Knowing what operating system it is could help. Low level TCP isn't entirely the same thing in its response codes, even though POSIX exists. There are some things down here which are up to "interpretation" for some levels of interpretation. The operating system is Linux 4.6.3. I prematurely omitted that information from my original email because I already believed to have tracked down the issue to be within OTP. From magnus.ottenklinger@REDACTED Fri Jul 1 09:01:04 2016 From: magnus.ottenklinger@REDACTED (Magnus Ottenklinger) Date: Fri, 1 Jul 2016 07:01:04 +0000 Subject: [erlang-questions] A question on EINVAL in active-mode TCP In-Reply-To: References: , Message-ID: > Von: James Fish > Gesendet: Donnerstag, 30. Juni 2016 16:58 > An: Jesper Louis Andersen > Cc: Magnus Ottenklinger; erlang-questions@REDACTED > Betreff: Re: [erlang-questions] A question on EINVAL in active-mode TCP > > I believe the race condition occurring is that the gen_tcp port will exit after it sends a {tcp_closed, Socket} message when in active mode. Yes, I think that is exactly what happens. Thanks for pointing that out. It seems that I am able to `receive {tcp_closed,Sock}` after `EINVAL`, so it should be possible to recover from the error by selectively receiving `tcp_closed`. From jesper.louis.andersen@REDACTED Fri Jul 1 10:00:42 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 1 Jul 2016 10:00:42 +0200 Subject: [erlang-questions] A question on EINVAL in active-mode TCP In-Reply-To: References: Message-ID: On Fri, Jul 1, 2016 at 8:13 AM, Magnus Ottenklinger < magnus.ottenklinger@REDACTED> wrote: > The operating system is Linux 4.6.3. I prematurely omitted that > information from my original email because I already believed to have > tracked down the issue to be within OTP. Definitely sounds like James Fish has the most plausible reason. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Fri Jul 1 17:18:37 2016 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Fri, 1 Jul 2016 17:18:37 +0200 Subject: [erlang-questions] Static callback in NIF In-Reply-To: <508348d9-30ee-c5ce-cc80-a4ce5d530759@gmail.com> References: <7549f4ba-467c-a16a-ba48-5455e54d1015@gmail.com> <508348d9-30ee-c5ce-cc80-a4ce5d530759@gmail.com> Message-ID: <577689CD.6090201@ericsson.com> The only legit way to create an ErlNifEnv is to call enif_alloc_env(). I bet the compiler warned about your static ErlNifEnv just before you went hunting for erts internal header files :-) In OTP-19.0 the 'msg_env' argument to enif_send() can be NULL in which case the message will be copied and you can reuse the environment for the next message with enif_clear_env(). This is effective if the message is small. /Sverker, Erlang/OTP On 06/30/2016 08:31 PM, Igor Clark wrote: > Thanks Daniel, good to hear. > > Thanks also Roger & Sergej for your replies. I'll try out > enif_alloc_env()'ing a new ErlNifEnv each time the callback uses > enif_send(), rather than just leaving it static. > > Cheers! > > Igor > > On 30/06/2016 07:45, Daniel Goertzen wrote: >> Static vs priv_data are functionally the same here so it doesn't >> matter which way you go. I can empathize with your sense of dread; >> there are a lot of rules to keep track of for keeping Undefined >> Behavior at bay. But you seem to have a good handle on things. >> >> On Thu, Jun 30, 2016 at 6:02 AM Igor Clark > > wrote: >> >> Hi folks, >> >> I've got a NIF that uses some library code to talk to specific >> hardware. >> It's a hobby project with only one user (me) and no real performance >> concerns, so what I've got works well at the moment, but I think I'm >> doing some sneaky/dirty stuff and would like to know the best way >> to do >> what I need. >> >> Sending messages outwards from erlang->C->HW is easy, very quick, >> and >> works fine. I return a pointer to the HW reference back to erlang >> using >> the enif_*_resource functions, and manage keeping track of >> everything on >> the erlang side, which feels pretty natural. >> >> Coming the other way works fine too, but relies on a C function >> callback >> which gets called when the hardware has a message for me. Right >> now I >> just have a static function in the NIF C code which I pass to the >> library. I create a static ErlNifEnv on NIF load() which I keep >> around >> and use in the callback to send messages to a specified erlang Pid, >> passed in via enif_get_local_pid() in another NIF function and also >> stored statically. This works a treat, but I'm feeling some pretty >> strong dread that it's very much the wrong way to do things, and >> asking >> for scheduler headaches/explosions. >> >> I'm planning to try storing the various resources in priv_data at >> load() >> time, on the theory that that way the memory would at least be >> managed >> by the NIF system rather than just as enif_alloc()'ing static >> pointers, >> but I'm not sure if that would make any diffrence if code >> external to >> the scheduler calls back into it. >> >> I've looked into running this part as a C node or a port that sends >> messages with the HW data in a callback in its own process, and the >> communication seems straightforward enough, but it also seems like I >> immediately need to start designing mechanisms to deal with >> working out >> where to send received messages, almost a protocol in itself. >> Whereas >> with the NIF+callback method I have a lot of the work done for me - >> except, of course, for the synchronisation and memory management, >> which >> is the bit I'm worried about. >> >> FWIW the callback code doesn't modify any of the static data >> structures >> directly, it just calls library code which uses the stored >> references to >> work out which hardware device & channel to send the message to. >> >> What's the best practice here? Is a callback in a NIF OK if it's >> stored >> in priv_data, or is it never OK? What's the best way to do this if >> not? >> >> Would appreciate any tips! >> >> Cheers, >> Igor >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.clark@REDACTED Fri Jul 1 19:38:03 2016 From: igor.clark@REDACTED (Igor Clark) Date: Fri, 1 Jul 2016 10:38:03 -0700 Subject: [erlang-questions] Static callback in NIF In-Reply-To: <577689CD.6090201@ericsson.com> References: <7549f4ba-467c-a16a-ba48-5455e54d1015@gmail.com> <508348d9-30ee-c5ce-cc80-a4ce5d530759@gmail.com> <577689CD.6090201@ericsson.com> Message-ID: Hi Sverker, thanks. I'm only using enif_alloc_env() to create the static ErlNifEnv, in the NIF load(). I haven't had compiler warnings about it - certainly haven't noticed any, and just did a clean+build to check. (I'm using the rebar3 port compiler on erlang 18.3, erts-7.3 on OSX 10.11.5. It doesn't seem to show warnings without errors, and I can't find a verbose mode - so I used 'rebar comp --verbose' to inspect 'cc' lines and warnings, and there's nothing there about the static ErlNifEnv, either.) I haven't been doing any digging in erts headers, as I'd be pretty nervous about doing anything undocumented :-) Am I missing something? Cheers, Igor On 01/07/2016 08:18, Sverker Eriksson wrote: > The only legit way to create an ErlNifEnv is to call enif_alloc_env(). > > I bet the compiler warned about your static ErlNifEnv > just before you went hunting for erts internal header files :-) > > In OTP-19.0 the 'msg_env' argument to enif_send() can be NULL > in which case the message will be copied and you can reuse > the environment for the next message with enif_clear_env(). > This is effective if the message is small. > > /Sverker, Erlang/OTP > > > On 06/30/2016 08:31 PM, Igor Clark wrote: >> Thanks Daniel, good to hear. >> >> Thanks also Roger & Sergej for your replies. I'll try out >> enif_alloc_env()'ing a new ErlNifEnv each time the callback uses >> enif_send(), rather than just leaving it static. >> >> Cheers! >> >> Igor >> >> On 30/06/2016 07:45, Daniel Goertzen wrote: >>> Static vs priv_data are functionally the same here so it doesn't >>> matter which way you go. I can empathize with your sense of dread; >>> there are a lot of rules to keep track of for keeping Undefined >>> Behavior at bay. But you seem to have a good handle on things. >>> >>> On Thu, Jun 30, 2016 at 6:02 AM Igor Clark >> > wrote: >>> >>> Hi folks, >>> >>> I've got a NIF that uses some library code to talk to specific >>> hardware. >>> It's a hobby project with only one user (me) and no real >>> performance >>> concerns, so what I've got works well at the moment, but I think >>> I'm >>> doing some sneaky/dirty stuff and would like to know the best way >>> to do >>> what I need. >>> >>> Sending messages outwards from erlang->C->HW is easy, very >>> quick, and >>> works fine. I return a pointer to the HW reference back to erlang >>> using >>> the enif_*_resource functions, and manage keeping track of >>> everything on >>> the erlang side, which feels pretty natural. >>> >>> Coming the other way works fine too, but relies on a C function >>> callback >>> which gets called when the hardware has a message for me. Right >>> now I >>> just have a static function in the NIF C code which I pass to the >>> library. I create a static ErlNifEnv on NIF load() which I keep >>> around >>> and use in the callback to send messages to a specified erlang Pid, >>> passed in via enif_get_local_pid() in another NIF function and also >>> stored statically. This works a treat, but I'm feeling some pretty >>> strong dread that it's very much the wrong way to do things, and >>> asking >>> for scheduler headaches/explosions. >>> >>> I'm planning to try storing the various resources in priv_data at >>> load() >>> time, on the theory that that way the memory would at least be >>> managed >>> by the NIF system rather than just as enif_alloc()'ing static >>> pointers, >>> but I'm not sure if that would make any diffrence if code >>> external to >>> the scheduler calls back into it. >>> >>> I've looked into running this part as a C node or a port that sends >>> messages with the HW data in a callback in its own process, and the >>> communication seems straightforward enough, but it also seems >>> like I >>> immediately need to start designing mechanisms to deal with >>> working out >>> where to send received messages, almost a protocol in itself. >>> Whereas >>> with the NIF+callback method I have a lot of the work done for me - >>> except, of course, for the synchronisation and memory management, >>> which >>> is the bit I'm worried about. >>> >>> FWIW the callback code doesn't modify any of the static data >>> structures >>> directly, it just calls library code which uses the stored >>> references to >>> work out which hardware device & channel to send the message to. >>> >>> What's the best practice here? Is a callback in a NIF OK if it's >>> stored >>> in priv_data, or is it never OK? What's the best way to do this if >>> not? >>> >>> Would appreciate any tips! >>> >>> Cheers, >>> Igor >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Fri Jul 1 20:08:14 2016 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Fri, 1 Jul 2016 20:08:14 +0200 Subject: [erlang-questions] Static callback in NIF In-Reply-To: References: <7549f4ba-467c-a16a-ba48-5455e54d1015@gmail.com> <508348d9-30ee-c5ce-cc80-a4ce5d530759@gmail.com> <577689CD.6090201@ericsson.com> Message-ID: <5776B18E.1010402@ericsson.com> My mistake. I read "I create a static ErlNifEnv ..." and thought you did static ErlNifEnv my_env; which is not allowed and can only compile if you include internal headers. As long as you only have one callback thread, then it's totally fine to allocate one "static" ErlNifEnv, use that in your callback to create the message, call enif_send() and then call enif_clear_env() to prepare for next callback. /Sverker On 07/01/2016 07:38 PM, Igor Clark wrote: > Hi Sverker, thanks. > > I'm only using enif_alloc_env() to create the static ErlNifEnv, in the > NIF load(). I haven't had compiler warnings about it - certainly > haven't noticed any, and just did a clean+build to check. > > (I'm using the rebar3 port compiler on erlang 18.3, erts-7.3 on OSX > 10.11.5. It doesn't seem to show warnings without errors, and I can't > find a verbose mode - so I used 'rebar comp --verbose' to inspect 'cc' > lines and warnings, and there's nothing there about the static > ErlNifEnv, either.) > > I haven't been doing any digging in erts headers, as I'd be pretty > nervous about doing anything undocumented :-) > > Am I missing something? > > Cheers, > > Igor > > > On 01/07/2016 08:18, Sverker Eriksson wrote: >> The only legit way to create an ErlNifEnv is to call enif_alloc_env(). >> >> I bet the compiler warned about your static ErlNifEnv >> just before you went hunting for erts internal header files :-) >> >> In OTP-19.0 the 'msg_env' argument to enif_send() can be NULL >> in which case the message will be copied and you can reuse >> the environment for the next message with enif_clear_env(). >> This is effective if the message is small. >> >> /Sverker, Erlang/OTP >> >> >> On 06/30/2016 08:31 PM, Igor Clark wrote: >>> Thanks Daniel, good to hear. >>> >>> Thanks also Roger & Sergej for your replies. I'll try out >>> enif_alloc_env()'ing a new ErlNifEnv each time the callback uses >>> enif_send(), rather than just leaving it static. >>> >>> Cheers! >>> >>> Igor >>> >>> On 30/06/2016 07:45, Daniel Goertzen wrote: >>>> Static vs priv_data are functionally the same here so it doesn't >>>> matter which way you go. I can empathize with your sense of dread; >>>> there are a lot of rules to keep track of for keeping Undefined >>>> Behavior at bay. But you seem to have a good handle on things. >>>> >>>> On Thu, Jun 30, 2016 at 6:02 AM Igor Clark >>> > wrote: >>>> >>>> Hi folks, >>>> >>>> I've got a NIF that uses some library code to talk to specific >>>> hardware. >>>> It's a hobby project with only one user (me) and no real >>>> performance >>>> concerns, so what I've got works well at the moment, but I >>>> think I'm >>>> doing some sneaky/dirty stuff and would like to know the best way >>>> to do >>>> what I need. >>>> >>>> Sending messages outwards from erlang->C->HW is easy, very >>>> quick, and >>>> works fine. I return a pointer to the HW reference back to erlang >>>> using >>>> the enif_*_resource functions, and manage keeping track of >>>> everything on >>>> the erlang side, which feels pretty natural. >>>> >>>> Coming the other way works fine too, but relies on a C function >>>> callback >>>> which gets called when the hardware has a message for me. Right >>>> now I >>>> just have a static function in the NIF C code which I pass to the >>>> library. I create a static ErlNifEnv on NIF load() which I keep >>>> around >>>> and use in the callback to send messages to a specified erlang >>>> Pid, >>>> passed in via enif_get_local_pid() in another NIF function and >>>> also >>>> stored statically. This works a treat, but I'm feeling some pretty >>>> strong dread that it's very much the wrong way to do things, and >>>> asking >>>> for scheduler headaches/explosions. >>>> >>>> I'm planning to try storing the various resources in priv_data at >>>> load() >>>> time, on the theory that that way the memory would at least be >>>> managed >>>> by the NIF system rather than just as enif_alloc()'ing static >>>> pointers, >>>> but I'm not sure if that would make any diffrence if code >>>> external to >>>> the scheduler calls back into it. >>>> >>>> I've looked into running this part as a C node or a port that >>>> sends >>>> messages with the HW data in a callback in its own process, and >>>> the >>>> communication seems straightforward enough, but it also seems >>>> like I >>>> immediately need to start designing mechanisms to deal with >>>> working out >>>> where to send received messages, almost a protocol in itself. >>>> Whereas >>>> with the NIF+callback method I have a lot of the work done for >>>> me - >>>> except, of course, for the synchronisation and memory management, >>>> which >>>> is the bit I'm worried about. >>>> >>>> FWIW the callback code doesn't modify any of the static data >>>> structures >>>> directly, it just calls library code which uses the stored >>>> references to >>>> work out which hardware device & channel to send the message to. >>>> >>>> What's the best practice here? Is a callback in a NIF OK if it's >>>> stored >>>> in priv_data, or is it never OK? What's the best way to do this if >>>> not? >>>> >>>> Would appreciate any tips! >>>> >>>> Cheers, >>>> Igor >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > > From igor.clark@REDACTED Sat Jul 2 00:41:38 2016 From: igor.clark@REDACTED (Igor Clark) Date: Fri, 1 Jul 2016 15:41:38 -0700 Subject: [erlang-questions] Static callback in NIF In-Reply-To: <5776B18E.1010402@ericsson.com> References: <7549f4ba-467c-a16a-ba48-5455e54d1015@gmail.com> <508348d9-30ee-c5ce-cc80-a4ce5d530759@gmail.com> <577689CD.6090201@ericsson.com> <5776B18E.1010402@ericsson.com> Message-ID: <375095AA-B31B-4109-9178-577BD6460AA0@gmail.com> Ah, got you. I'm not explicitly creating any threads at all; I just pass the callback to the library code in load() and return, so it sounds like that's OK. Great news for me! Thanks for your help Sverker. Best, Igor > On Jul 1, 2016, at 11:08 AM, Sverker Eriksson wrote: > > My mistake. I read "I create a static ErlNifEnv ..." and thought you did > > static ErlNifEnv my_env; > > which is not allowed and can only compile if you include internal headers. > > As long as you only have one callback thread, then it's totally fine to allocate > one "static" ErlNifEnv, use that in your callback to create the message, > call enif_send() and then call enif_clear_env() to prepare for next callback. > > /Sverker > > >> On 07/01/2016 07:38 PM, Igor Clark wrote: >> Hi Sverker, thanks. >> >> I'm only using enif_alloc_env() to create the static ErlNifEnv, in the NIF load(). I haven't had compiler warnings about it - certainly haven't noticed any, and just did a clean+build to check. >> >> (I'm using the rebar3 port compiler on erlang 18.3, erts-7.3 on OSX 10.11.5. It doesn't seem to show warnings without errors, and I can't find a verbose mode - so I used 'rebar comp --verbose' to inspect 'cc' lines and warnings, and there's nothing there about the static ErlNifEnv, either.) >> >> I haven't been doing any digging in erts headers, as I'd be pretty nervous about doing anything undocumented :-) >> >> Am I missing something? >> >> Cheers, >> >> Igor >> >> >>> On 01/07/2016 08:18, Sverker Eriksson wrote: >>> The only legit way to create an ErlNifEnv is to call enif_alloc_env(). >>> >>> I bet the compiler warned about your static ErlNifEnv >>> just before you went hunting for erts internal header files :-) >>> >>> In OTP-19.0 the 'msg_env' argument to enif_send() can be NULL >>> in which case the message will be copied and you can reuse >>> the environment for the next message with enif_clear_env(). >>> This is effective if the message is small. >>> >>> /Sverker, Erlang/OTP >>> >>> >>>> On 06/30/2016 08:31 PM, Igor Clark wrote: >>>> Thanks Daniel, good to hear. >>>> >>>> Thanks also Roger & Sergej for your replies. I'll try out enif_alloc_env()'ing a new ErlNifEnv each time the callback uses enif_send(), rather than just leaving it static. >>>> >>>> Cheers! >>>> >>>> Igor >>>> >>>>> On 30/06/2016 07:45, Daniel Goertzen wrote: >>>>> Static vs priv_data are functionally the same here so it doesn't matter which way you go. I can empathize with your sense of dread; there are a lot of rules to keep track of for keeping Undefined Behavior at bay. But you seem to have a good handle on things. >>>>> >>>>> On Thu, Jun 30, 2016 at 6:02 AM Igor Clark > wrote: >>>>> >>>>> Hi folks, >>>>> >>>>> I've got a NIF that uses some library code to talk to specific >>>>> hardware. >>>>> It's a hobby project with only one user (me) and no real performance >>>>> concerns, so what I've got works well at the moment, but I think I'm >>>>> doing some sneaky/dirty stuff and would like to know the best way >>>>> to do >>>>> what I need. >>>>> >>>>> Sending messages outwards from erlang->C->HW is easy, very quick, and >>>>> works fine. I return a pointer to the HW reference back to erlang >>>>> using >>>>> the enif_*_resource functions, and manage keeping track of >>>>> everything on >>>>> the erlang side, which feels pretty natural. >>>>> >>>>> Coming the other way works fine too, but relies on a C function >>>>> callback >>>>> which gets called when the hardware has a message for me. Right now I >>>>> just have a static function in the NIF C code which I pass to the >>>>> library. I create a static ErlNifEnv on NIF load() which I keep around >>>>> and use in the callback to send messages to a specified erlang Pid, >>>>> passed in via enif_get_local_pid() in another NIF function and also >>>>> stored statically. This works a treat, but I'm feeling some pretty >>>>> strong dread that it's very much the wrong way to do things, and >>>>> asking >>>>> for scheduler headaches/explosions. >>>>> >>>>> I'm planning to try storing the various resources in priv_data at >>>>> load() >>>>> time, on the theory that that way the memory would at least be managed >>>>> by the NIF system rather than just as enif_alloc()'ing static >>>>> pointers, >>>>> but I'm not sure if that would make any diffrence if code external to >>>>> the scheduler calls back into it. >>>>> >>>>> I've looked into running this part as a C node or a port that sends >>>>> messages with the HW data in a callback in its own process, and the >>>>> communication seems straightforward enough, but it also seems like I >>>>> immediately need to start designing mechanisms to deal with >>>>> working out >>>>> where to send received messages, almost a protocol in itself. Whereas >>>>> with the NIF+callback method I have a lot of the work done for me - >>>>> except, of course, for the synchronisation and memory management, >>>>> which >>>>> is the bit I'm worried about. >>>>> >>>>> FWIW the callback code doesn't modify any of the static data >>>>> structures >>>>> directly, it just calls library code which uses the stored >>>>> references to >>>>> work out which hardware device & channel to send the message to. >>>>> >>>>> What's the best practice here? Is a callback in a NIF OK if it's >>>>> stored >>>>> in priv_data, or is it never OK? What's the best way to do this if >>>>> not? >>>>> >>>>> Would appreciate any tips! >>>>> >>>>> Cheers, >>>>> Igor >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions > From felixgallo@REDACTED Sun Jul 3 07:44:30 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Sat, 2 Jul 2016 22:44:30 -0700 Subject: [erlang-questions] 19.0 vs wx on a mac Message-ID: With a bog standard wx 3.0 installation that had worked for the 17 and 18 series on a mac and hadn't changed for about a year, I was able to compile a wx-aware 19.0, but running observer caused obscure wx crashes when I clicked on any buttons. I figured maybe, since the release notes mention wx 3.1, that the HOWTO/INSTALL.md doc hadn't been updated, and so I tried to install wx 3.1.0 and compile with that. However, the wx application refuses to compile, because in its config.log we see: conftest.cc:54:33: error: no matching constructor for initialization of 'wxGLCanvas' wxGLCanvas * bar = new wxGLCanvas((wxWindow *) NULL, -1, ^ ~~~~~~~~~~~~~~~~~~~~~~ /usr/local/include/wx-3.1/wx/osx/glcanvas.h:69:5: note: candidate constructor not viable: no known conversion from 'const wxPoint' to 'const int *' for 3rd argument wxGLCanvas(wxWindow *parent, ^ /usr/local/include/wx-3.1/wx/osx/glcanvas.h:60:5: note: candidate constructor not viable: no known conversion from 'int' to 'const wxGLAttributes' for 2nd argument ... which results in a failure to link, and so wx is skipped. That looks like the configure script is not wx-3.1 aware or something of that nature, so I'm now backpedalling to 3.0.0. Is anyone aware of the proper combination of sigils, incantations, and cargo cult mumbo jumbo necessary to properly compile a wx-aware 19.0 on a mac in such a way that the resulting wx doesn't crash? F. -------------- next part -------------- An HTML attachment was scrubbed... URL: From klacke@REDACTED Sun Jul 3 12:08:03 2016 From: klacke@REDACTED (Claes Wikstrom) Date: Sun, 3 Jul 2016 12:08:03 +0200 Subject: [erlang-questions] Yaws 2.0.3 Message-ID: <5778E403.6010502@hyber.org> New release of Yaws available now. 2.0.3 http://yaws.hyber.org Enjoy /klacke From daniel.goertzen@REDACTED Mon Jul 4 03:51:44 2016 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Mon, 04 Jul 2016 01:51:44 +0000 Subject: [erlang-questions] Static callback in NIF In-Reply-To: <577689CD.6090201@ericsson.com> References: <7549f4ba-467c-a16a-ba48-5455e54d1015@gmail.com> <508348d9-30ee-c5ce-cc80-a4ce5d530759@gmail.com> <577689CD.6090201@ericsson.com> Message-ID: Off topic: the docs say to set env to null (not msg_env) when calling enif_send() from a thread. What is the consequence of setting it null in a NIF call? I'm writing Rust wrappers, and I can't think of a compile time or runtime check for when env needs to be null. I've been able to protect basically everything else in the NIF API, but not that. On Fri, Jul 1, 2016 at 10:18 AM Sverker Eriksson < sverker.eriksson@REDACTED> wrote: > The only legit way to create an ErlNifEnv is to call enif_alloc_env(). > > I bet the compiler warned about your static ErlNifEnv > just before you went hunting for erts internal header files :-) > > In OTP-19.0 the 'msg_env' argument to enif_send() can be NULL > in which case the message will be copied and you can reuse > the environment for the next message with enif_clear_env(). > This is effective if the message is small. > > /Sverker, Erlang/OTP > > > > On 06/30/2016 08:31 PM, Igor Clark wrote: > > Thanks Daniel, good to hear. > > Thanks also Roger & Sergej for your replies. I'll try out > enif_alloc_env()'ing a new ErlNifEnv each time the callback uses > enif_send(), rather than just leaving it static. > > Cheers! > > Igor > > On 30/06/2016 07:45, Daniel Goertzen wrote: > > Static vs priv_data are functionally the same here so it doesn't matter > which way you go. I can empathize with your sense of dread; there are a > lot of rules to keep track of for keeping Undefined Behavior at bay. But > you seem to have a good handle on things. > > On Thu, Jun 30, 2016 at 6:02 AM Igor Clark > wrote: > > Hi folks, > > I've got a NIF that uses some library code to talk to specific > hardware. > It's a hobby project with only one user (me) and no real performance > concerns, so what I've got works well at the moment, but I think I'm > doing some sneaky/dirty stuff and would like to know the best way > to do > what I need. > > Sending messages outwards from erlang->C->HW is easy, very quick, and > works fine. I return a pointer to the HW reference back to erlang > using > the enif_*_resource functions, and manage keeping track of > everything on > the erlang side, which feels pretty natural. > > Coming the other way works fine too, but relies on a C function > callback > which gets called when the hardware has a message for me. Right now I > just have a static function in the NIF C code which I pass to the > library. I create a static ErlNifEnv on NIF load() which I keep around > and use in the callback to send messages to a specified erlang Pid, > passed in via enif_get_local_pid() in another NIF function and also > stored statically. This works a treat, but I'm feeling some pretty > strong dread that it's very much the wrong way to do things, and > asking > for scheduler headaches/explosions. > > I'm planning to try storing the various resources in priv_data at > load() > time, on the theory that that way the memory would at least be managed > by the NIF system rather than just as enif_alloc()'ing static > pointers, > but I'm not sure if that would make any diffrence if code external to > the scheduler calls back into it. > > I've looked into running this part as a C node or a port that sends > messages with the HW data in a callback in its own process, and the > communication seems straightforward enough, but it also seems like I > immediately need to start designing mechanisms to deal with > working out > where to send received messages, almost a protocol in itself. Whereas > with the NIF+callback method I have a lot of the work done for me - > except, of course, for the synchronisation and memory management, > which > is the bit I'm worried about. > > FWIW the callback code doesn't modify any of the static data > structures > directly, it just calls library code which uses the stored > references to > work out which hardware device & channel to send the message to. > > What's the best practice here? Is a callback in a NIF OK if it's > stored > in priv_data, or is it never OK? What's the best way to do this if > not? > > Would appreciate any tips! > > Cheers, > Igor > _______________________________________________ > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Mon Jul 4 10:18:07 2016 From: dangud@REDACTED (Dan Gudmundsson) Date: Mon, 04 Jul 2016 08:18:07 +0000 Subject: [erlang-questions] 19.0 vs wx on a mac In-Reply-To: References: Message-ID: wxWidgets-3.0.2 should work out of the box. If you use wxWidgets-3.1 you need to compile wxWidgets with --enable-compat28 On Sun, Jul 3, 2016 at 7:44 AM Felix Gallo wrote: > With a bog standard wx 3.0 installation that had worked for the 17 and 18 > series on a mac and hadn't changed for about a year, I was able to compile > a wx-aware 19.0, but running observer caused obscure wx crashes when I > clicked on any buttons. > > I figured maybe, since the release notes mention wx 3.1, that the > HOWTO/INSTALL.md doc hadn't been updated, and so I tried to install wx > 3.1.0 and compile with that. However, the wx application refuses to > compile, because in its config.log we see: > > conftest.cc:54:33: error: no matching constructor for initialization of > 'wxGLCanvas' > wxGLCanvas * bar = new wxGLCanvas((wxWindow *) > NULL, -1, > ^ > ~~~~~~~~~~~~~~~~~~~~~~ > /usr/local/include/wx-3.1/wx/osx/glcanvas.h:69:5: note: candidate > constructor not viable: no known conversion from 'const wxPoint' to 'const > int *' for 3rd argument > wxGLCanvas(wxWindow *parent, > ^ > /usr/local/include/wx-3.1/wx/osx/glcanvas.h:60:5: note: candidate > constructor not viable: no known conversion from 'int' to 'const > wxGLAttributes' for 2nd argument > ... > > which results in a failure to link, and so wx is skipped. > > That looks like the configure script is not wx-3.1 aware or something of > that nature, so I'm now backpedalling to 3.0.0. > > Is anyone aware of the proper combination of sigils, incantations, and > cargo cult mumbo jumbo necessary to properly compile a wx-aware 19.0 on a > mac in such a way that the resulting wx doesn't crash? > > F. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Mon Jul 4 17:35:03 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Mon, 4 Jul 2016 17:35:03 +0200 Subject: [erlang-questions] what means busy_wait in msacc erl 19.0? In-Reply-To: References: Message-ID: In case you are interested Max, I have some new functionality for msacc that I will put in 19.1 or 20 or something which allows you to to see which BIF call is consuming the most time. You can get the changes here: https://github.com/garazdawi/otp/commits/lukas/erts/dirty_msacc_cleanup At the moment you have to enable it through changing a define in erl_msacc.h ( https://github.com/garazdawi/otp/blob/lukas/erts/dirty_msacc_cleanup/erts/emulator/beam/erl_msacc.h#L40 ) Now I just have to add histogram buckets for each slot like lcnt does and it may start to be useful to hunt down nifs/bifs that take too long :) Lukas On Mon, Jun 27, 2016 at 8:12 PM, Max Lapshin wrote: > Only on real hardware. We don't even debug such issues on a virtual > machines. > > It is not possible to take a crash dump, but it is possible to make > recon_alloc. I will make it. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Jul 4 18:18:56 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 4 Jul 2016 19:18:56 +0300 Subject: [erlang-questions] what means busy_wait in msacc erl 19.0? In-Reply-To: References: Message-ID: Of course I'm interested! We've got into performance troubles at the moment that cannot be tracked by simple fprof or etop, so we will be glad to each tool. On Mon, Jul 4, 2016 at 6:35 PM, Lukas Larsson wrote: > In case you are interested Max, I have some new functionality for msacc > that I will put in 19.1 or 20 or something which allows you to to see which > BIF call is consuming the most time. You can get the changes here: > > https://github.com/garazdawi/otp/commits/lukas/erts/dirty_msacc_cleanup > > At the moment you have to enable it through changing a define in > erl_msacc.h ( > https://github.com/garazdawi/otp/blob/lukas/erts/dirty_msacc_cleanup/erts/emulator/beam/erl_msacc.h#L40 > ) > > Now I just have to add histogram buckets for each slot like lcnt does and > it may start to be useful to hunt down nifs/bifs that take too long :) > > Lukas > > On Mon, Jun 27, 2016 at 8:12 PM, Max Lapshin > wrote: > >> Only on real hardware. We don't even debug such issues on a virtual >> machines. >> >> It is not possible to take a crash dump, but it is possible to make >> recon_alloc. I will make it. >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From liuzhongzheng2012@REDACTED Tue Jul 5 04:31:34 2016 From: liuzhongzheng2012@REDACTED (Zhongzheng Liu) Date: Tue, 5 Jul 2016 10:31:34 +0800 Subject: [erlang-questions] fprof overestimate the cost of list comprehension ? Message-ID: Hi mail list: I profiled my program and found the performance of tail recursion and reverse is always better than list comprehension in fprof. I feel strange after reading the discuss in mail list < http://erlang.org/pipermail/erlang-questions/2016-June/089479.html> and wrote a small test myself. The result showed that list comprehension run faster in timer:tc/3, but slower in fprof. I think the result of timer:tc/3 is more believable and fprof is overestimate the cost of list comprehension. This will be misleading when I do performance optimization guided by fprof. Why it happens and how to correct the result of fprof? Best Liu Zhongzheng ======================================= 6> timer:tc(tst, repeat_tail_recu, [10000000]). {4165000,ok} 7> timer:tc(tst, repeat_list_comp, [10000000]). {3713000,ok} ======================================= 1> fprof:apply(tst, compare, [10000]). ok 2> fprof:profile(). Reading trace data... End of trace! ok 3> fprof:analyse(). % CNT ACC OWN [{ totals, 470633, 875.885, 860.659}]. %%% {[{{error_handler,undefined_function,3}, 1, 860.879, 0.003}, {{tst,compare,1}, 10000, 0.000, 35.380}], { {tst,compare,1}, 10001, 860.879, 35.383}, % [{{tst,list_comp,1}, 10000, 457.673, 10.006}, {{tst,tail_recu,1}, 10000, 367.803, 10.005}, {suspend, 20, 0.020, 0.000}, {{tst,compare,1}, 10000, 0.000, 35.380}]}. ======================================= Here is my test code. %% tst.erl -module(tst). -compile([export_all]). list_comp(L) -> [sq(I) || I <- L]. tail_recu(L) -> tail_recu(L, []). tail_recu([H|T], Acc) -> tail_recu(T, [sq(H)|Acc]); tail_recu([], Acc) -> lists:reverse(Acc). sq(X) -> X*X. compare(0) -> ok; compare(N) -> L = [1,2,3,4,5,6,7,8,9,10], list_comp(L), tail_recu(L), compare(N-1). repeat_list_comp(0) -> ok; repeat_list_comp(N) -> L = [1,2,3,4,5,6,7,8,9,10], list_comp(L), repeat_list_comp(N-1). repeat_tail_recu(0) -> ok; repeat_tail_recu(N) -> L = [1,2,3,4,5,6,7,8,9,10], tail_recu(L), repeat_tail_recu(N-1). % eof -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Tue Jul 5 05:02:03 2016 From: zxq9@REDACTED (zxq9) Date: Tue, 05 Jul 2016 12:02:03 +0900 Subject: [erlang-questions] fprof overestimate the cost of list comprehension ? In-Reply-To: References: Message-ID: <8294963.qPJ3MY5KuR@changa> On 2016?7?5? ??? 10:31:34 Zhongzheng Liu wrote: > Hi mail list: > > I profiled my program and found the performance of tail recursion and > reverse is always better than list comprehension in fprof. I feel strange > after reading the discuss in mail list < > http://erlang.org/pipermail/erlang-questions/2016-June/089479.html> and > wrote a small test myself. > > The result showed that list comprehension run faster in timer:tc/3, but > slower in fprof. > > I think the result of timer:tc/3 is more believable and fprof is > overestimate the cost of list comprehension. This will be misleading when I > do performance optimization guided by fprof. > > Why it happens and how to correct the result of fprof? Be careful how much faith you put in microbenchmarks like this outside any actual operational context. Expressiveness is dramatically more important than whether the iteration mechanism X is slightly faster than occasionally-equivalent mechanism Y. Iteration itself is nearly always dwarfed by the actual computation that is taking place, especially if there is a side-effect involved. As far as timer:tc VS fprof... they are two different tools designed to do two very different things. One gives you a very rough (often uselessly rough) way to turn almost any call into a microbenchmark (slick, but not really where the useful part of testing resides as far as your users are ever concerned), the other gives you a system-sized profile of where your program is spending its time. Its not even apples to oranges, more like comparing oats to quarterhorses. As for *why* the discrepancy... It probably has to do with fprof using trace, and timer:tc just executing something and telling you the time diff. A lot more steps are involved in tracing to get an execution profile. One of the interesting differences is that when profiling you *don't care* how *long* something took as measured by the wall clock. Be wary of microbenchmarks. -Craig From publicityifl@REDACTED Tue Jul 5 12:17:09 2016 From: publicityifl@REDACTED (publicityifl@REDACTED) Date: Tue, 05 Jul 2016 10:17:09 +0000 Subject: [erlang-questions] 2nd CfP: IFL 2016 (28th Symposium on Implementation and Application of Functional Languages) Message-ID: <94eb2c13d44c69d8e10536e0c17e@google.com> Hello, Please, find below the second call for papers for IFL 2016. Please forward these to anyone you think may be interested. Apologies for any duplicates you may receive. best regards, Jurriaan Hage Publicity Chair of IFL --- IFL 2016 - 2nd Call for papers 28th SYMPOSIUM ON IMPLEMENTATION AND APPLICATION OF FUNCTIONAL LANGUAGES - IFL 2016 KU Leuven, Belgium In cooperation with ACM SIGPLAN August 31 - September 2, 2016 https://dtai.cs.kuleuven.be/events/ifl2016/ Scope The goal of the IFL symposia is to bring together researchers actively engaged in the implementation and application of functional and function-based programming languages. IFL 2016 will be a venue for researchers to present and discuss new ideas and concepts, work in progress, and publication-ripe results related to the implementation and application of functional languages and function-based programming. Peer-review Following the IFL tradition, IFL 2016 will use a post-symposium review process to produce the formal proceedings. All participants of IFL 2016 are invited to submit either a draft paper or an extended abstract describing work to be presented at the symposium. At no time may work submitted to IFL be simultaneously submitted to other venues; submissions must adhere to ACM SIGPLAN's republication policy: http://www.sigplan.org/Resources/Policies/Republication The submissions will be screened by the program committee chair to make sure they are within the scope of IFL, and will appear in the draft proceedings distributed at the symposium. Submissions appearing in the draft proceedings are not peer-reviewed publications. Hence, publications that appear only in the draft proceedings are not subject to the ACM SIGPLAN republication policy. After the symposium, authors will be given the opportunity to incorporate the feedback from discussions at the symposium and will be invited to submit a revised full article for the formal review process. From the revised submissions, the program committee will select papers for the formal proceedings considering their correctness, novelty, originality, relevance, significance, and clarity. The formal proceedings will appear in the International Conference Proceedings Series of the ACM Digital Library. Important dates August 1: Submission deadline draft papers August 3: Notification of acceptance for presentation August 5: Early registration deadline August 12: Late registration deadline August 22: Submission deadline for pre-symposium proceedings August 31 - September 2: IFL Symposium December 1: Submission deadline for post-symposium proceedings January 31, 2017: Notification of acceptance for post-symposium proceedings March 15, 2017: Camera-ready version for post-symposium proceedings Submission details Prospective authors are encouraged to submit papers or extended abstracts to be published in the draft proceedings and to present them at the symposium. All contributions must be written in English. Papers must use the new ACM two columns conference format, which can be found at: http://www.acm.org/publications/proceedings-template For the pre-symposium proceedings we adopt a 'weak' page limit of 12 pages. For the post-symposium proceedings the page limit of 12 pages is firm. Authors submit through EasyChair: https://easychair.org/conferences/?conf=ifl2016 Topics IFL welcomes submissions describing practical and theoretical work as well as submissions describing applications and tools in the context of functional programming. If you are not sure whether your work is appropriate for IFL 2016, please contact the PC chair at tom.schrijvers@REDACTED Topics of interest include, but are not limited to: - language concepts - type systems, type checking, type inferencing - compilation techniques - staged compilation - run-time function specialization - run-time code generation - partial evaluation - (abstract) interpretation - metaprogramming - generic programming - automatic program generation - array processing - concurrent/parallel programming - concurrent/parallel program execution - embedded systems - web applications - (embedded) domain specific languages - security - novel memory management techniques - run-time profiling performance measurements - debugging and tracing - virtual/abstract machine architectures - validation, verification of functional programs - tools and programming techniques - (industrial) applications Peter Landin Prize The Peter Landin Prize is awarded to the best paper presented at the symposium every year. The honored article is selected by the program committee based on the submissions received for the formal review process. The prize carries a cash award equivalent to 150 Euros. Programme committee Chair: Tom Schrijvers, KU Leuven, Belgium - Sandrine Blazy, University of Rennes 1, France - Laura Castro, University of A Coru?a, Spain - Jacques, Garrigue, Nagoya University, Japan - Clemens Grelck, University of Amsterdam, The Netherlands - Zoltan Horvath, Eotvos Lorand University, Hungary - Jan Martin Jansen, Netherlands Defence Academy, The Netherlands - Mauro Jaskelioff, CIFASIS/Universidad Nacional de Rosario, Argentina - Patricia Johann, Appalachian State University, USA - Wolfram Kahl, McMaster University, Canada - Pieter Koopman, Radboud University Nijmegen, The Netherlands - Shin-Cheng Mu, Academia Sinica, Taiwan - Henrik Nilsson, University of Nottingham, UK - Nikolaos Papaspyrou, National Technical University of Athens, Greece - Atze van der Ploeg, Chalmers University of Technology, Sweden - Matija Pretnar, University of Ljubljana, Slovenia - Tillmann Rendel, University of T?bingen, Germany - Christophe Scholliers, Universiteit Gent, Belgium - Sven-Bodo Scholz, Heriot-Watt University, UK - Melinda Toth, Eotvos Lorand University, Hungary - Meng Wang, University of Kent, UK - Jeremy Yallop, University of Cambridge, UK Venue The 28th IFL will be held in association with the Faculty of Computer Science, KU Leuven, Belgium. Leuven is centrally located in Belgium and can be easily reached from Brussels Airport by train (~15 minutes). The venue in the Arenberg Castle park can be reached by foot, bus or taxi from the city center. See the website for more information on the venue. -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard.jonas@REDACTED Wed Jul 6 09:45:44 2016 From: richard.jonas@REDACTED (Richard Jonas) Date: Wed, 6 Jul 2016 09:45:44 +0200 Subject: [erlang-questions] Scaling bcrypt Message-ID: Hey guys, last time I wrote about some problems how bcrypt (at least current erlang bcrypt implementations) cannot be scale. The original problem what that bcrypt driver (github chef/erlang-bcrypt) is using port and nif implementation. The nif implementation behind the scenes it uses a gen_server which serializes all the requests are coming. The port implementation uses a port pool which can be configured. Our problem comes from the fact that a good hashing algorithm should be slow by design. bcrypt uses a round parameter (3-16) with which one can say that bcrypt should use 2^(3-16) cycles during computing. According to security guidelines a good hashing algorithm should take 0.1-0.5 seconds to be computed, so now bcrypt should run with round 10-12 depending on the hardware. It needs to be slow in order that one cannot find a hash-collision easily. Now you can imagine the problem. With load tests we easily made message queues and the problem is that the computing has sometimes very very big latencies (50-60 seconds depending on the number of concurrent users). The nif implementation spoils one scheduler thread but if I have 32 cores I cannot scale the computing. The problem I wanted to solve is to split the bcrypt_hashpw nif into 3 parts: initialization, computing some predefined number of rounds, encryption. I used enif_schedule_nif to schedule the computing of the next task. https://github.com/jonasrichard/erlang-bcrypt/blob/master/c_src/bcrypt_nif.c The implementation is based on chef/erlang-bcrypt and vinoski/bitwise. The first loadtests are promising since latencies are much more even. With 10 concurrent users (basho bench + bcrypt driver) the latency was (500ms 1100ms / mean median), with 50 users (2800ms ~6000ms) which 5x increase, let us say. My questions are: - how can I measure if my nif "slices" took no longer than 1ms - how can I implement some kind of adaptive computing? My bcrypt_compute computes 64 rounds which sometimes below 1ms, sometimes above 1ms. https://github.com/jonasrichard/erlang-bcrypt/blob/bb96149332c8c6b6135698dfbc21e19e554bc066/c_src/bcrypt_nif.c#L133 If I use some kernel gettime function, can it cause blocking? - how can I be sure that no garbage left in memory. I try to watch out putting release functions in correct places but it would be good to measure. Thanks, Richard -- Richard Jonas Erlang Solutions Hungary Kft Address: Riverpark Office K.32 K?zrakt?r street 32. 3/1. 1093 Budapest Hungary Phone/fax: +36-1-7000-654 -------------- next part -------------- An HTML attachment was scrubbed... URL: From brandjoe@REDACTED Wed Jul 6 17:16:48 2016 From: brandjoe@REDACTED (=?UTF-8?Q?J=c3=b6rgen_Brandt?=) Date: Wed, 6 Jul 2016 17:16:48 +0200 Subject: [erlang-questions] Petri nets Message-ID: <577D20E0.2020602@hu-berlin.de> Dear all, recently I was curious about programming with Petri nets. I have the impression that concurrent nature of Petri nets would be extremely useful for many types of applications. Also, with the gen_fsm and the gen_statem being part of Erlang's standard library the idea of implementing application models in terms of behaviours filling in the missing callback functions while abstracting away the brittle parts concurrency and communication would have a strong appeal. Imho, Petri nets can be defined in exactly the same way. This idea has also been discussed in a mailing list entry by Claus Reinke [1] who happens to have written a paper about implementing Petri nets in Haskell [2] in which he observes that functional languages are well suited for implementing Petri nets and points out Erlang as a potential candidate also with regard to concurrency aspects. In addition, I stumbled over a compelling blog post which motivates the use of Petri nets for application design in the context of telecommunication applications [3]. The author goes on to presents his Petri net programming library in C# and even has proposed a gen_pn behaviour for Erlang [4]. However, the aforementioned gen_pn library has not been touched for six years and the email from Claus Reinke has been sitting in the Erlang mailing list archives for 13 years (unanswered). My question is, in your opinion, is the idea of using Petri nets as a programming model in Erlang just a bad idea? Or has the right library just not been conceived to date? Do you use Petri nets to sketch out an application's behavior but then translate it to a different model prior to implementation? How can a telecommunication-focused programming ecosystem have avoided Petri nets for such a long time while readily absorbing ever more flavors of state machines? That said, I would be very glad to receive your feedback on a Petri net library I recently came up with. So you have something to base your criticism on (; [1] http://erlang.org/pipermail/erlang-questions/2003-November/010704.html [2] http://community.haskell.org/~claus/publications/HCPN.html [3] https://aabs.wordpress.com/2010/03/10/programming-with-petri-nets/ [4] https://github.com/aabs/gen_pn From brandjoe@REDACTED Wed Jul 6 17:17:51 2016 From: brandjoe@REDACTED (=?UTF-8?Q?J=c3=b6rgen_Brandt?=) Date: Wed, 6 Jul 2016 17:17:51 +0200 Subject: [erlang-questions] Petri nets In-Reply-To: <577D20E0.2020602@hu-berlin.de> References: <577D20E0.2020602@hu-berlin.de> Message-ID: <577D211F.4010706@hu-berlin.de> And here the link to the library ... https://github.com/joergen7/gen_pnet On 06.07.2016 17:16, J?rgen Brandt wrote: > Dear all, > > recently I was curious about programming with Petri nets. I have > the impression that concurrent nature of Petri nets would be > extremely useful for many types of applications. > > Also, with the gen_fsm and the gen_statem being part of Erlang's > standard library the idea of implementing application models in > terms of behaviours filling in the missing callback functions > while abstracting away the brittle parts concurrency and > communication would have a strong appeal. Imho, Petri nets can be > defined in exactly the same way. > > This idea has also been discussed in a mailing list entry by Claus > Reinke [1] who happens to have written a paper about implementing > Petri nets in Haskell [2] in which he observes that functional > languages are well suited for implementing Petri nets and points > out Erlang as a potential candidate also with regard to concurrency > aspects. > > In addition, I stumbled over a compelling blog post which > motivates the use of Petri nets for application design in the > context of telecommunication applications [3]. The author goes on > to presents his Petri net programming library in C# and even has > proposed a gen_pn behaviour for Erlang [4]. > > However, the aforementioned gen_pn library has not been touched > for six years and the email from Claus Reinke has been sitting in > the Erlang mailing list archives for 13 years (unanswered). > > My question is, in your opinion, is the idea of using Petri nets as > a programming model in Erlang just a bad idea? Or has the right > library just not been conceived to date? Do you use Petri nets to > sketch out an application's behavior but then translate it to a > different model prior to implementation? How can a > telecommunication-focused programming ecosystem have avoided Petri > nets for such a long time while readily absorbing ever more flavors > of state machines? > > That said, I would be very glad to receive your feedback on a > Petri net library I recently came up with. So you have something to > base your criticism on (; > > [1] > http://erlang.org/pipermail/erlang-questions/2003-November/010704.html > > [2] http://community.haskell.org/~claus/publications/HCPN.html > [3] > https://aabs.wordpress.com/2010/03/10/programming-with-petri-nets/ > [4] https://github.com/aabs/gen_pn > _______________________________________________ erlang-questions > mailing list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From list1@REDACTED Wed Jul 6 19:24:58 2016 From: list1@REDACTED (Grzegorz Junka) Date: Wed, 6 Jul 2016 17:24:58 +0000 Subject: [erlang-questions] Petri nets In-Reply-To: <577D211F.4010706@hu-berlin.de> References: <577D20E0.2020602@hu-berlin.de> <577D211F.4010706@hu-berlin.de> Message-ID: <4602d708-a077-a59f-dc28-fde0482a0437@gjunka.com> In my view Petri nets somehow share the fate of expert systems. They are just theoretical tools that's great to know about but not very practical to solve real-life problems. This is because any problem one need to solve would need to fit in the domain in which that tool operates. In case of expert systems it is the inference based on facts, but how do you add new facts, validate them, remove obsolete, etc? In real life scenarios the knowledge is already somewhere and one only need to write the inference on top of it. Trying to squeeze the whole application into the tool that solves the specific problem is someone counterintuitive (e.g. implementing all IO and adding, validating and removing facts in Prolog only because Prolog is great as inference machine). In my opinion the same can be said about Petri nets. When I have to write an application solving a specific problem, trying to solve some of its parts using Petri nets may not give me the required benefit because of the required shift between different domains. Having said that, it maybe just a matter of integrating Petri nets into the language at the correct level, similarly to how Erlang integrated message passing and green processes into the language. But can you think about a problem that could be better solved by having multiple gen_pn representing nodes of concurrent computations in a Petri net rather than green Erlang threads communicating with each other using message passing? Grzegorz On 06/07/2016 15:17, J?rgen Brandt wrote: > And here the link to the library ... > > https://github.com/joergen7/gen_pnet > > On 06.07.2016 17:16, J?rgen Brandt wrote: >> Dear all, >> >> recently I was curious about programming with Petri nets. I have >> the impression that concurrent nature of Petri nets would be >> extremely useful for many types of applications. >> >> Also, with the gen_fsm and the gen_statem being part of Erlang's >> standard library the idea of implementing application models in >> terms of behaviours filling in the missing callback functions >> while abstracting away the brittle parts concurrency and >> communication would have a strong appeal. Imho, Petri nets can be >> defined in exactly the same way. >> >> This idea has also been discussed in a mailing list entry by Claus >> Reinke [1] who happens to have written a paper about implementing >> Petri nets in Haskell [2] in which he observes that functional >> languages are well suited for implementing Petri nets and points >> out Erlang as a potential candidate also with regard to concurrency >> aspects. >> >> In addition, I stumbled over a compelling blog post which >> motivates the use of Petri nets for application design in the >> context of telecommunication applications [3]. The author goes on >> to presents his Petri net programming library in C# and even has >> proposed a gen_pn behaviour for Erlang [4]. >> >> However, the aforementioned gen_pn library has not been touched >> for six years and the email from Claus Reinke has been sitting in >> the Erlang mailing list archives for 13 years (unanswered). >> >> My question is, in your opinion, is the idea of using Petri nets as >> a programming model in Erlang just a bad idea? Or has the right >> library just not been conceived to date? Do you use Petri nets to >> sketch out an application's behavior but then translate it to a >> different model prior to implementation? How can a >> telecommunication-focused programming ecosystem have avoided Petri >> nets for such a long time while readily absorbing ever more flavors >> of state machines? >> >> That said, I would be very glad to receive your feedback on a >> Petri net library I recently came up with. So you have something to >> base your criticism on (; >> >> [1] >> http://erlang.org/pipermail/erlang-questions/2003-November/010704.html >> >> > [2] http://community.haskell.org/~claus/publications/HCPN.html >> [3] >> https://aabs.wordpress.com/2010/03/10/programming-with-petri-nets/ >> [4] https://github.com/aabs/gen_pn >> _______________________________________________ erlang-questions >> mailing list erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From walter.weinmann@REDACTED Wed Jul 6 19:40:40 2016 From: walter.weinmann@REDACTED (Walter Weinmann) Date: Wed, 6 Jul 2016 19:40:40 +0200 Subject: [erlang-questions] ocparse - the openCypher parser written in Erlang Message-ID: Version 1.0.0 of *ocparse *is now available at Github. *ocparse *is a production-ready *openCypher *parser written in pure Erlang. *ocparse *is closely aligned to the *openCypher *project and in future will be adapted on a regular basis as the *openCypher *project evolves. The *openCypher *project aims to deliver a full and open specification of the industry?s most widely adopted graph database query language: *Cypher*. And, with the EBNF file the project provides the basis for the definition of the LALR grammar. Links: - *ocparse*: https://github.com/walter-weinmann/ocparse - *openCypher*: https://github.com/opencypher/openCypher - *Cypher*: https://neo4j.com/docs/developer-manual/current/#cypher-query-lang Comments and suggestions are welcome. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nyarly@REDACTED Wed Jul 6 22:18:30 2016 From: nyarly@REDACTED (Judson Lester) Date: Wed, 06 Jul 2016 20:18:30 +0000 Subject: [erlang-questions] Petri nets In-Reply-To: <4602d708-a077-a59f-dc28-fde0482a0437@gjunka.com> References: <577D20E0.2020602@hu-berlin.de> <577D211F.4010706@hu-berlin.de> <4602d708-a077-a59f-dc28-fde0482a0437@gjunka.com> Message-ID: I've been casually working on a project that's something like a Petri net applied to the software build problem. There's ways in which it's not, strictly speaking, a Petri net, but I've thought for a while that the underlying graph processing model might be useful elsewhere: https://git.lrdesign.com/judson/ergo On Wed, Jul 6, 2016 at 10:25 AM Grzegorz Junka wrote: > In my view Petri nets somehow share the fate of expert systems. They are > just theoretical tools that's great to know about but not very practical > to solve real-life problems. This is because any problem one need to > solve would need to fit in the domain in which that tool operates. > > In case of expert systems it is the inference based on facts, but how do > you add new facts, validate them, remove obsolete, etc? In real life > scenarios the knowledge is already somewhere and one only need to write > the inference on top of it. Trying to squeeze the whole application into > the tool that solves the specific problem is someone counterintuitive > (e.g. implementing all IO and adding, validating and removing facts in > Prolog only because Prolog is great as inference machine). > > In my opinion the same can be said about Petri nets. When I have to > write an application solving a specific problem, trying to solve some of > its parts using Petri nets may not give me the required benefit because > of the required shift between different domains. > > Having said that, it maybe just a matter of integrating Petri nets into > the language at the correct level, similarly to how Erlang integrated > message passing and green processes into the language. > > But can you think about a problem that could be better solved by having > multiple gen_pn representing nodes of concurrent computations in a Petri > net rather than green Erlang threads communicating with each other using > message passing? > > Grzegorz > > > On 06/07/2016 15:17, J?rgen Brandt wrote: > > And here the link to the library ... > > > > https://github.com/joergen7/gen_pnet > > > > On 06.07.2016 17:16, J?rgen Brandt wrote: > >> Dear all, > >> > >> recently I was curious about programming with Petri nets. I have > >> the impression that concurrent nature of Petri nets would be > >> extremely useful for many types of applications. > >> > >> Also, with the gen_fsm and the gen_statem being part of Erlang's > >> standard library the idea of implementing application models in > >> terms of behaviours filling in the missing callback functions > >> while abstracting away the brittle parts concurrency and > >> communication would have a strong appeal. Imho, Petri nets can be > >> defined in exactly the same way. > >> > >> This idea has also been discussed in a mailing list entry by Claus > >> Reinke [1] who happens to have written a paper about implementing > >> Petri nets in Haskell [2] in which he observes that functional > >> languages are well suited for implementing Petri nets and points > >> out Erlang as a potential candidate also with regard to concurrency > >> aspects. > >> > >> In addition, I stumbled over a compelling blog post which > >> motivates the use of Petri nets for application design in the > >> context of telecommunication applications [3]. The author goes on > >> to presents his Petri net programming library in C# and even has > >> proposed a gen_pn behaviour for Erlang [4]. > >> > >> However, the aforementioned gen_pn library has not been touched > >> for six years and the email from Claus Reinke has been sitting in > >> the Erlang mailing list archives for 13 years (unanswered). > >> > >> My question is, in your opinion, is the idea of using Petri nets as > >> a programming model in Erlang just a bad idea? Or has the right > >> library just not been conceived to date? Do you use Petri nets to > >> sketch out an application's behavior but then translate it to a > >> different model prior to implementation? How can a > >> telecommunication-focused programming ecosystem have avoided Petri > >> nets for such a long time while readily absorbing ever more flavors > >> of state machines? > >> > >> That said, I would be very glad to receive your feedback on a > >> Petri net library I recently came up with. So you have something to > >> base your criticism on (; > >> > >> [1] > >> http://erlang.org/pipermail/erlang-questions/2003-November/010704.html > >> > >> > > [2] http://community.haskell.org/~claus/publications/HCPN.html > >> [3] > >> https://aabs.wordpress.com/2010/03/10/programming-with-petri-nets/ > >> [4] https://github.com/aabs/gen_pn > >> _______________________________________________ erlang-questions > >> mailing list erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gomoripeti@REDACTED Wed Jul 6 22:50:34 2016 From: gomoripeti@REDACTED (=?UTF-8?B?UGV0aSBHw7Ztw7ZyaQ==?=) Date: Wed, 6 Jul 2016 22:50:34 +0200 Subject: [erlang-questions] infinite loop in merl_tranform Message-ID: Hello, While being knee deep in metaprogramming with merl (OTP 18.3) I made a small typo which led the compilation of my module into an infinite loop. I managed to shrink it down to a module like this -module(infinity). -export([f/1]). -include_lib("syntax_tools/include/merl.hrl"). f(V) -> merl:qquote("@var", [{var, V}]). ( or this oneliner call Form = merl:quote("f(V) -> merl:qquote(\"@var\", [{var, V}])."), merl_transform:parse_transform([Form], []). ) Seems to me that the optimisation to create templates at compile time in merl_transform:expand_qquote is tricked. Can this be considered a bug? I would expect either a compilation error or just silently creating a module which will throw at runtime. thanks Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Jul 7 07:38:19 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 7 Jul 2016 17:38:19 +1200 Subject: [erlang-questions] Petri nets In-Reply-To: <4602d708-a077-a59f-dc28-fde0482a0437@gjunka.com> References: <577D20E0.2020602@hu-berlin.de> <577D211F.4010706@hu-berlin.de> <4602d708-a077-a59f-dc28-fde0482a0437@gjunka.com> Message-ID: Part of the issue is that "Petri nets" sensu strictu are one member of a large family of Place/Transition formalisms. There is a large literature about them. The information science department here used to be very keen on Coloured Nets, see https://ourarchive.otago.ac.nz/bitstream/handle/10523/912/dp1996-04-online.pdf for example. The big question is what using P/T nets buys you. If you use a P/T formalism that (a) has tools that give you useful static analyses (b) lets you plug in code and data structures (c) and the analyses remain valid when you have done so then that could be very useful. (For what it's worth, Expert Systems are still being used in practical systems, they just aren't *called* Expert Systems any more. They're called Rule Engines, or Semantic Web, or other things.) From carlsson.richard@REDACTED Fri Jul 8 10:48:19 2016 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 8 Jul 2016 10:48:19 +0200 Subject: [erlang-questions] infinite loop in merl_tranform In-Reply-To: References: Message-ID: Thanks for the report. Definitely sounds like a bug. I'm on vacation right now, but will look at it as soon as I can. Den 6 jul 2016 22:50 skrev "Peti G?m?ri" : Hello, While being knee deep in metaprogramming with merl (OTP 18.3) I made a small typo which led the compilation of my module into an infinite loop. I managed to shrink it down to a module like this -module(infinity). -export([f/1]). -include_lib("syntax_tools/include/merl.hrl"). f(V) -> merl:qquote("@var", [{var, V}]). ( or this oneliner call Form = merl:quote("f(V) -> merl:qquote(\"@var\", [{var, V}])."), merl_transform:parse_transform([Form], []). ) Seems to me that the optimisation to create templates at compile time in merl_transform:expand_qquote is tricked. Can this be considered a bug? I would expect either a compilation error or just silently creating a module which will throw at runtime. thanks Peter _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From community-manager@REDACTED Fri Jul 8 16:50:36 2016 From: community-manager@REDACTED (Bruce Yinhe) Date: Fri, 8 Jul 2016 16:50:36 +0200 Subject: [erlang-questions] New Erlang job openings Message-ID: Hi There are 19 Erlang-related job openings on https://erlangcentral.org/jobs. Subscribe to weekly Erlang job updates: https://erlangcentral.org/login/?action=register Publish an Erlang job opening: https://erlangcentral.org/jobs/add Erlang Mid-Senior Level Developer - GameAnalytics - London, UK http://erlangcentral.org/erlang-mid-senior-level-developer-gameanalytics/ Apply before: 2016-07-14 Erlang & Java Developer - Darwin Recruitment - London, UK http://erlangcentral.org/erlang-java-developer-darwin-recruitment/ Apply before: 2016-07-28 Backend software engineer - Electic Imp - Any (remote); London, UK preferred, UK http://erlangcentral.org/backend-software-engineer-electric-imp/ Apply before: 2016-07-30 Platform and Data Engineer - Handy's culture deck - New York City, USA http://erlangcentral.org/platform-and-data-engineer-handy/ Apply before: 2016-07-30 Senior Erlang Developer - Darwin Recruitment - London, UK http://erlangcentral.org/senior-erlang-developer-darwin-recruitment/ Apply before: 2016-07-30 Backend Engineer - NPL - San Francisco, CA, USA http://erlangcentral.org/backend-engineer-netpowerandlight/ Apply before: 2016-08-30 Director Of Engineering/Lead Developer ? Elixir - Curadora - Philadelphia, PA, USA http://erlangcentral.org/director-of-engineeringlead-developer-elixir-curadora/ Apply before: 2016-08-30 Erlang Software Engineer - Cisco - Stockholm, Swedem http://erlangcentral.org/erlang-software-engineer-cisco/ Apply before: 2016-08-30 Online Services Programmer - Creative Assembly - , UK http://erlangcentral.org/online-services-programmer-creative-assembly/ Apply before: 2016-08-30 Senior Elixir Engineer - HelloSign - San Francisco, CA, USA http://erlangcentral.org/senior-elixir-engineer-hellosign/ Apply before: 2016-08-30 Erlang Developer - Darwin Recruitment - London, UK http://erlangcentral.org/erlang-developer-darwin/ Apply before: 2016-10-30 Erlang Developer in Test - Darwin Recruitment - BIllericay, Essex, UK http://erlangcentral.org/erlang-developer-in-test-darwin/ Apply before: 2016-10-30 Erlang Developer - Krubera Group - Hertfordshire, UK http://erlangcentral.org/erlang-developer-krubera-group/ Apply before: 2016-11-04 Software Developer ? Erlang - Darwin Recruitment - Cardiff, Wales, UK http://erlangcentral.org/software-developer-erlang-darwin/ Apply before: 2016-11-29 Senior Erlang Developer - Dasudian - Shenzhen, China http://erlangcentral.org/senior-erlang-developer-dasudian-company/ Apply before: 2016-12-11 Senior Erlang Developer - Dasudian - Shenzhen, China http://erlangcentral.org/senior-erlang-developer-dasudian/ Apply before: 2016-12-11 Erlang Software Engineer - F Technologies - Dubai, UAE http://erlangcentral.org/erlang-software-engineer-f-technologies/ Apply before: 2016-12-30 Senior Erlang Developer - Dasudian - Shenzhen, China http://erlangcentral.org/senior-erlang-developer-dasudian-2/ Apply before: 2016-12-30 Back-end Engineer - 2600hz - San Francisco, CA, USA http://erlangcentral.org/back-end-engineer-2600hz/ Apply before: 2017-08-30 *Bruce Yinhe* Community Manager Industrial Erlang User Group +46 72 311 43 89 community-manager@REDACTED -- Visit our Erlang community site ErlangCentral.org | @ErlangCentral | Industrial Erlang User Group -------------- next part -------------- An HTML attachment was scrubbed... URL: From pjotr.public12@REDACTED Sat Jul 9 13:46:17 2016 From: pjotr.public12@REDACTED (Pjotr Prins) Date: Sat, 9 Jul 2016 13:46:17 +0200 Subject: [erlang-questions] Reproducible Erlang builds - accepted in GNU Guix! In-Reply-To: <20160614075005.GA29200@thebird.nl> References: <20160609173127.GA30990@thebird.nl> <20160612083500.GA13121@thebird.nl> <20160614075005.GA29200@thebird.nl> Message-ID: <20160709114617.GA31729@thebird.nl> Good news. Erlang has been accepted as a GNU Guix package: https://www.gnu.org/software/guix/packages/ The package definition is here: http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/erlang.scm you can see, thanks to Leo's work, we still patch out the time stamp from a few files, but now we have a byte-reproducible build! This means that Guix is probably the easiest way to install Erlang reproducibly with *all* its dependencies on *any* Linux distribution. Note we included the debugger too and it works :) I have been testing this setup developing against Elixir. I'll add an Elixir package too, one of these days. We use it in a production environment. Pj. From gomoripeti@REDACTED Sat Jul 9 17:57:50 2016 From: gomoripeti@REDACTED (=?UTF-8?B?UGV0aSBHw7Ztw7ZyaQ==?=) Date: Sat, 9 Jul 2016 17:57:50 +0200 Subject: [erlang-questions] infinite loop in merl_tranform In-Reply-To: References: Message-ID: Hi Richard, I tried to address the issue in this PR https://github.com/erlang/otp/pull/1123 As a future improvement idea the parse transform could emit compilation errors. I'm not sure if it is easy to differentiate between cases which would fail at runtime too and cases which only fail during compilation (if there is any) On Fri, Jul 8, 2016 at 10:48 AM, Richard Carlsson < carlsson.richard@REDACTED> wrote: > Thanks for the report. Definitely sounds like a bug. I'm on vacation right > now, but will look at it as soon as I can. > Den 6 jul 2016 22:50 skrev "Peti G?m?ri" : > > Hello, > > While being knee deep in metaprogramming with merl (OTP 18.3) I made a > small typo which led the compilation of my module into an infinite loop. > > I managed to shrink it down to a module like this > > -module(infinity). > -export([f/1]). > -include_lib("syntax_tools/include/merl.hrl"). > f(V) -> merl:qquote("@var", [{var, V}]). > > ( or this oneliner call > Form = merl:quote("f(V) -> merl:qquote(\"@var\", [{var, V}])."), > merl_transform:parse_transform([Form], []). > ) > > Seems to me that the optimisation to create templates at compile time in > merl_transform:expand_qquote is tricked. > > Can this be considered a bug? I would expect either a compilation error or > just silently creating a module which will throw at runtime. > > > thanks > Peter > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxnotdead@REDACTED Sat Jul 9 19:54:01 2016 From: zxnotdead@REDACTED (Constantin Kulikov) Date: Sat, 9 Jul 2016 20:54:01 +0300 Subject: [erlang-questions] Add a new data structure to erlang VM. Message-ID: If I want to add a new datastructure to erlang(my local fork of the erlang-otp from github), what should I do? Yes, I mean a C-level module realization. Let's say I want to add a double linked list(just for simplicity) module with an interface like that: dlist:new() -> ?some king of reference to a list? dlist:push(item) -> ?reference to a list? dlist:pop() -> {item, ?reference to a list?} etc... Is it possible at all? Has anyone ever tried to do something like this? What definitions to what files should I add? How must I allocate my data structure? How must I reference items in it? Can someone point me to source locations where erlang's list/ets table/tuple operations(creating a new, storing a reference to objects) defined? -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Sat Jul 9 20:59:00 2016 From: vinoski@REDACTED (Steve Vinoski) Date: Sat, 9 Jul 2016 14:59:00 -0400 Subject: [erlang-questions] Add a new data structure to erlang VM. In-Reply-To: References: Message-ID: On Sat, Jul 9, 2016 at 1:54 PM, Constantin Kulikov wrote: > If I want to add a new datastructure to erlang(my local fork of the > erlang-otp from github), what should I do? Yes, I mean a C-level module > realization. > > Let's say I want to add a double linked list(just for simplicity) module > with an interface like that: > dlist:new() -> ?some king of reference to a list? > dlist:push(item) -> ?reference to a list? > dlist:pop() -> {item, ?reference to a list?} > etc... > > Is it possible at all? Has anyone ever tried to do something like this? > What definitions to what files should I add? How must I allocate my data > structure? How must I reference items in it? > Can someone point me to source locations where erlang's list/ets > table/tuple operations(creating a new, storing a reference to objects) > defined? > See http://erlang.org/doc/tutorial/nif.html http://erlang.org/doc/man/erl_nif.html --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Sun Jul 10 05:55:52 2016 From: sdl.web@REDACTED (Leo Liu) Date: Sun, 10 Jul 2016 11:55:52 +0800 Subject: [erlang-questions] bad certificate for https://bugs.erlang.org Message-ID: The Erlang issue tracker is broken for a few days. Leo From eric.pailleau@REDACTED Sun Jul 10 10:25:55 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Sun, 10 Jul 2016 10:25:55 +0200 Subject: [erlang-questions] bad certificate for https://bugs.erlang.org In-Reply-To: References: Message-ID: <7lid51pc56bn8aplqcupqksg.1468139155213@email.android.com> Hello, Yes and https send a self signed certificate. Browser ask to add a security exception but cert is C=Au while servers in Poland. There is a serious MITM risk. Do not add security exception and do not give your credentials. Am I paranoid ? "Envoy? depuis mon mobile " Eric ---- Leo Liu a ?crit ---- >The Erlang issue tracker is broken for a few days. > >Leo > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Sun Jul 10 13:16:29 2016 From: ok@REDACTED (ok@REDACTED) Date: Sun, 10 Jul 2016 23:16:29 +1200 Subject: [erlang-questions] Add a new data structure to erlang VM. In-Reply-To: References: Message-ID: <061a30059b36fbe495d691d02dc010e5.squirrel@chasm.otago.ac.nz> > On Sat, Jul 9, 2016 at 1:54 PM, Constantin Kulikov > wrote: > >> If I want to add a new datastructure to erlang(my local fork of the >> erlang-otp from github), The example of a doubly linked list is a poor one because Erlang data structures are supposed to be immutable and inconsequence acyclic. (Suppose A points to B. Then B must have been created before A.) The garbage collector is entitled to rely on this. The best you can do is to have some sort of C object kept *out* of the Erlang per-process heaps (perhaps along the lines of large binaries, perhaps even hidden inside large binaries) and only provide some sort of possibly encoded *reference* to these things to Erlang. No, I tell a lie. The *best* thing is to NOT add a new low-level data structure to Erlang. Mixing data structures between languages with different memory management strategies is for people who like pain too much. Perhaps you could open up a little about what problem you are hoping to solve by doing this. From jesper.louis.andersen@REDACTED Sun Jul 10 14:25:15 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 10 Jul 2016 14:25:15 +0200 Subject: [erlang-questions] Add a new data structure to erlang VM. In-Reply-To: <061a30059b36fbe495d691d02dc010e5.squirrel@chasm.otago.ac.nz> References: <061a30059b36fbe495d691d02dc010e5.squirrel@chasm.otago.ac.nz> Message-ID: On Sun, Jul 10, 2016 at 1:16 PM, wrote: > No, I tell a lie. The *best* thing is to NOT add a new low-level > data structure to Erlang. Mixing data structures between languages > with different memory management strategies is for people who like > pain too much. > The only time where this is somewhat acceptable is the case where the underlying data structure can be embedded in the host language. In Erlang's case, this demands that the underlying data structure be persistent rather than ephemeral[0]. Some times, you can write a data structure which has (locally) benign effects, but where the interface to that data structure is safe from the outside. In that case, you may rewrite the data structure due to demands of efficiency in your program. In general though, it tend to be a better long-term stategy to implement such a data structure in Erlang, *before* dropping to C. That way, you have a reference implementation you can use as a test case for verifying the correctness of the low-level embedded efficient datastructure. Also, as ok@ writes, the GC interface between different languages is often treacherous waters[1]. [0] Roughly: that you can refer to older versions of the data structure in the code. This is true for any Erlang data structure, except the process dictionary and ETS tables. In particular, any destructive update is forbidden. [1] OCaml recently added "ephemerons" to their language for this very reason. They allow the usual "weak" references with an elegant extra set of rules, which allows more elegant implementations of e.g., LRU caches. Having those in the language often allows the GC to interplay with foreign data in a nicer way. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxnotdead@REDACTED Sun Jul 10 12:10:44 2016 From: zxnotdead@REDACTED (Constantin Kulikov) Date: Sun, 10 Jul 2016 13:10:44 +0300 Subject: [erlang-questions] Add a new data structure to erlang VM. In-Reply-To: References: Message-ID: This seem to work https://gist.github.com/Bad-ptr/3ea0d6d65ae18b5b68c6cb15a96fef8e . However the documentation says that: > All terms of type ERL_NIF_TERM belong to an environment of type ErlNifEnv. The lifetime of a term is controlled by the lifetime of its environment object. > Variables of type ERL_NIF_TERM can refer to any Erlang term. This is an opaque type and values of it can only by used either as arguments to API functions or as return values from NIFs. All ERL_NIF_TERM's belong to an environment (ErlNifEnv). A term can not be destructed individually, it is valid until its environment is destructed. > The environment is only valid in the thread where it was supplied as argument until the NIF returns. It is thus useless and dangerous to store pointers to process bound environments between NIF calls. So, it is not clear to me is it safe to store an ERL_NIF_TERM(in a form as it passed to a NIF function) in a datastructure created by a NIF function? And does the GC aware that this terms must not be collected ? Etc. On 9 July 2016 at 21:59, Steve Vinoski wrote: > > > On Sat, Jul 9, 2016 at 1:54 PM, Constantin Kulikov > wrote: > >> If I want to add a new datastructure to erlang(my local fork of the >> erlang-otp from github), what should I do? Yes, I mean a C-level module >> realization. >> >> Let's say I want to add a double linked list(just for simplicity) module >> with an interface like that: >> dlist:new() -> ?some king of reference to a list? >> dlist:push(item) -> ?reference to a list? >> dlist:pop() -> {item, ?reference to a list?} >> etc... >> >> Is it possible at all? Has anyone ever tried to do something like this? >> What definitions to what files should I add? How must I allocate my data >> structure? How must I reference items in it? >> Can someone point me to source locations where erlang's list/ets >> table/tuple operations(creating a new, storing a reference to objects) >> defined? >> > > See > > http://erlang.org/doc/tutorial/nif.html > http://erlang.org/doc/man/erl_nif.html > > --steve > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Mon Jul 11 03:21:00 2016 From: rvirding@REDACTED (Robert Virding) Date: Mon, 11 Jul 2016 03:21:00 +0200 Subject: [erlang-questions] Add a new data structure to erlang VM. In-Reply-To: References: Message-ID: It feels a bit off adding a mutable data structure to erlang where everything else by design is immutable. Robert On 9 July 2016 at 19:54, Constantin Kulikov wrote: > If I want to add a new datastructure to erlang(my local fork of the > erlang-otp from github), what should I do? Yes, I mean a C-level module > realization. > > Let's say I want to add a double linked list(just for simplicity) module > with an interface like that: > dlist:new() -> ?some king of reference to a list? > dlist:push(item) -> ?reference to a list? > dlist:pop() -> {item, ?reference to a list?} > etc... > > Is it possible at all? Has anyone ever tried to do something like this? > What definitions to what files should I add? How must I allocate my data > structure? How must I reference items in it? > Can someone point me to source locations where erlang's list/ets > table/tuple operations(creating a new, storing a reference to objects) > defined? > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sargun@REDACTED Mon Jul 11 05:07:07 2016 From: sargun@REDACTED (Sargun Dhillon) Date: Sun, 10 Jul 2016 20:07:07 -0700 Subject: [erlang-questions] (D)TLS with SCTP? Message-ID: Has anyone done (D)TLS over SCTP? I know DTLS in Erlang isn't fully baked yet, but it seems better suited for SCTP. I tried to pass a one-to-one peeled off SCTP socket to SSL, and it seems not to work. The next option seems to be to implement a custom transport / callback module to wrap SCTP and present gen_tcp semantics. My only worry there is then I'll lose the datagram semantics of SCTP, and I'll have to do my own work to encrypt other streams. From zxnotdead@REDACTED Mon Jul 11 07:17:07 2016 From: zxnotdead@REDACTED (Constantin Kulikov) Date: Mon, 11 Jul 2016 08:17:07 +0300 Subject: [erlang-questions] Add a new data structure to erlang VM. In-Reply-To: References: Message-ID: > everything else by design is immutable Tell this to ets tables and process dicts.) On 11 July 2016 at 04:21, Robert Virding wrote: > It feels a bit off adding a mutable data structure to erlang where > everything else by design is immutable. > > Robert > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon Jul 11 07:31:24 2016 From: ok@REDACTED (ok@REDACTED) Date: Mon, 11 Jul 2016 17:31:24 +1200 Subject: [erlang-questions] Add a new data structure to erlang VM. In-Reply-To: References: Message-ID: <854196bb787acb42fe40c5734e60f251.squirrel@chasm.otago.ac.nz> >> everything else by design is immutable > > Tell this to ets tables and process dicts.) > ETS tables are not data structures in an Erlang process. They are outside all processes and act in some ways. Each process has its own dictionary, and if you search the archives, you will discover that process dictionaries can in principle be eliminated by a source to source transform that simply passes the current value around as an extra parameter. The key point is that a process's dictionary in and of itself is *not* represented as a data structure inside the process. If a process dictionary D contains an association K -> V, then V is not mutable, K is not mutable, the association K->V is not a data object visible to the process, and D is not a data object visible to the process. Another way to model process dictionaries would be to model a dictionary as a sibling process, with operations on the dictionary done by RPC calls. From kennethlakin@REDACTED Mon Jul 11 07:56:34 2016 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Sun, 10 Jul 2016 22:56:34 -0700 Subject: [erlang-questions] (D)TLS with SCTP? In-Reply-To: References: Message-ID: <57833512.7050101@gmail.com> On 07/10/2016 08:07 PM, Sargun Dhillon wrote: > The next option seems to be to implement a custom transport / callback > module to wrap SCTP and present gen_tcp semantics. My only worry there > is then I'll lose the datagram semantics of SCTP, and I'll have to do > my own work to encrypt other streams. I'm using the ssl module to do PEAP/RADIUS (which is TLS-in-EAP-in-RADIUS-over-UDP) which seems to work just fine, and wasn't too complicated. I had to implement get/setopts, controlling_process, listen, close, peername, port, and send, along with some bookkeeping to fake some of that data. (TLS data that I receive is sent as messages to the process (created and managed by ssl module code) that's managing the TLS connection. This seems to work just fine.) My wrapper also creates (and manages) fake sockets that are handed off to ssl, as the ssl module *really* wants to have a socket-shaped piece of data to play with while it does its thing. Maybe I'm misunderstanding, but it looks like SCTP has a notion of multiplexed data streams within a single connection? So, I think you'll need demultiplex those data streams and figure out a way to present them to the ssl module as if each stream was an individual TCP connection. Additionally, I expect that the ssl module will get cranky if it gets out-of-order TLS data. I *know* that it gets cranky if a TLS record goes missing. So, you might have to do additional bookkeeping if SCTP gives you out-of-order packets. Hope this helps and wasn't a waste of your time! -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From zxnotdead@REDACTED Mon Jul 11 11:07:03 2016 From: zxnotdead@REDACTED (Constantin Kulikov) Date: Mon, 11 Jul 2016 12:07:03 +0300 Subject: [erlang-questions] Add a new data structure to erlang VM. In-Reply-To: <854196bb787acb42fe40c5734e60f251.squirrel@chasm.otago.ac.nz> References: <854196bb787acb42fe40c5734e60f251.squirrel@chasm.otago.ac.nz> Message-ID: > If a process dictionary > D contains an association K -> V, then V is not mutable, > K is not mutable, the association K->V is not a data object > visible to the process The same rules could be applied to a doubly linked list or a 'queue' that internally implemented as a doubly linked list. > and D is not a data object visible to the process > ETS tables are not data structures in an Erlang process. Anyway, they are mutable(right?) and I didn't ask that a new data structure must necessarily reside in an Erlang process. > Another way to model process dictionaries would be to model a dictionary as a sibling process Doubly linked lists are also could be modeled as a chain of processes. Looks like anything could be modeled in a terms of communicating processes(actors), but it will not be efficient otherwise the erlang's linked lists, maps, ets tables, tuples would have been made like that. On 11 July 2016 at 08:31, wrote: > >> everything else by design is immutable > > > > Tell this to ets tables and process dicts.) > > > > ETS tables are not data structures in an Erlang process. > They are outside all processes and act in some ways. > > Each process has its own dictionary, and if you search > the archives, you will discover that process dictionaries > can in principle be eliminated by a source to source > transform that simply passes the current value around as > an extra parameter. The key point is that a process's > dictionary in and of itself is *not* represented as a > data structure inside the process. If a process dictionary > D contains an association K -> V, then V is not mutable, > K is not mutable, the association K->V is not a data object > visible to the process, and D is not a data object visible > to the process. > > Another way to model process dictionaries would be to model > a dictionary as a sibling process, with operations on the > dictionary done by RPC calls. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Mon Jul 11 14:36:15 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 11 Jul 2016 14:36:15 +0200 Subject: [erlang-questions] Inconsistent hostnames In-Reply-To: References: Message-ID: <20160711123615.GA37988@erix.ericsson.se> On Tue, Jun 21, 2016 at 07:11:54PM +0000, Ryan Stewart wrote: > In deploying my erlang app to some new CentOS servers with erlang 18.3, I > came across an oddity in networking. The server's networking config, which > is admittedly messed up, looks something like: > short hostname: bob > long hostname: bob.prod.example.net > domain: example.com > > Now when I start erlang with "erl -sname x", net_adm:localhost() returns " > bob.prod.example.net". > > However, if I start erlang with "erl -name x", net_adm:localhost() returns " > bob.example.com". > > I believe this comes from net_kernel:create_hostpart/2, where it follows > different paths on short vs long node names. On a long node name, it > appears to use short hostname + domain, where on a short node name, it > simply uses the long hostname. That decision appears to trickle through the > rest of the networking libraries. > > I understand that we need to fix the config on the server. I just wondered > if this might be considered a bug in erlang as well. This behaviour is old, and changing it would either surprise either some important "short names" using application or ditto "long names" one. For -name the Erlang networking code tries to parse the OS configuration files e.g /etc/resolv.conf to figure out the domain name. net_adm:localhost() then puts them together. For -sname the Erlang networking code tries to look up the short hostname using the native resolver to find a fully qualified alias, and takes the domain from that. We might call this an annoying irregularity of not very big practical importance, probably not worth changing... I actually do not understand why net_adm:localhost() returns the long hostname for -sname in the first place. I think the documentation leaves room for returning the short hostname, but changing that could probably affect which IP address is used to contact EPMD which would surprise some important "short names" using application. > > Ryan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-questions@REDACTED Mon Jul 11 14:49:15 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 11 Jul 2016 14:49:15 +0200 Subject: [erlang-questions] bad certificate for https://bugs.erlang.org In-Reply-To: <7lid51pc56bn8aplqcupqksg.1468139155213@email.android.com> References: <7lid51pc56bn8aplqcupqksg.1468139155213@email.android.com> Message-ID: <20160711124915.GB37988@erix.ericsson.se> On Sun, Jul 10, 2016 at 10:25:55AM +0200, ?ric Pailleau wrote: > Hello, > Yes and https send a self signed certificate. > Browser ask to add a security exception but cert is C=Au while servers in Poland. > There is a serious MITM risk. > Do not add security exception and do not give your credentials. Am I paranoid ? It seems that the maintenance team at Erlang Solutions has misplaced the correct certificate during an upgrade and I hope that they will soon fix that... -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From Oliver.Korpilla@REDACTED Mon Jul 11 15:08:17 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Mon, 11 Jul 2016 15:08:17 +0200 Subject: [erlang-questions] Inconsistent hostnames In-Reply-To: <20160711123615.GA37988@erix.ericsson.se> References: , <20160711123615.GA37988@erix.ericsson.se> Message-ID: Hello, Raimo. Agreed, it could cause surprises, but only on upgrading the system to a newer OTP. Wouldn't it be a bit naive to expect no changes whatsoever when doing that, anyway? Regards, Oliver ? Gesendet:?Montag, 11. Juli 2016 um 14:36 Uhr Von:?"Raimo Niskanen" An:?erlang-questions@REDACTED Betreff:?Re: [erlang-questions] Inconsistent hostnames On Tue, Jun 21, 2016 at 07:11:54PM +0000, Ryan Stewart wrote: > In deploying my erlang app to some new CentOS servers with erlang 18.3, I > came across an oddity in networking. The server's networking config, which > is admittedly messed up, looks something like: > short hostname: bob > long hostname: bob.prod.example.net > domain: example.com > > Now when I start erlang with "erl -sname x", net_adm:localhost() returns " > bob.prod.example.net". > > However, if I start erlang with "erl -name x", net_adm:localhost() returns " > bob.example.com". > > I believe this comes from net_kernel:create_hostpart/2, where it follows > different paths on short vs long node names. On a long node name, it > appears to use short hostname + domain, where on a short node name, it > simply uses the long hostname. That decision appears to trickle through the > rest of the networking libraries. > > I understand that we need to fix the config on the server. I just wondered > if this might be considered a bug in erlang as well. This behaviour is old, and changing it would either surprise either some important "short names" using application or ditto "long names" one. For -name the Erlang networking code tries to parse the OS configuration files e.g /etc/resolv.conf to figure out the domain name. net_adm:localhost() then puts them together. For -sname the Erlang networking code tries to look up the short hostname using the native resolver to find a fully qualified alias, and takes the domain from that. We might call this an annoying irregularity of not very big practical importance, probably not worth changing... I actually do not understand why net_adm:localhost() returns the long hostname for -sname in the first place. I think the documentation leaves room for returning the short hostname, but changing that could probably affect which IP address is used to contact EPMD which would surprise some important "short names" using application. > > Ryan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] From raimo+erlang-questions@REDACTED Mon Jul 11 15:28:19 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 11 Jul 2016 15:28:19 +0200 Subject: [erlang-questions] Inconsistent hostnames In-Reply-To: References: <20160711123615.GA37988@erix.ericsson.se> Message-ID: <20160711132819.GC37988@erix.ericsson.se> On Mon, Jul 11, 2016 at 03:08:17PM +0200, Oliver Korpilla wrote: > Hello, Raimo. > > Agreed, it could cause surprises, but only on upgrading the system to a newer OTP. Wouldn't it be a bit naive to expect no changes whatsoever when doing that, anyway? "Naive" might be a harsh word. We have historically had internal customers that pay our salary refusing to upgrade to a newer OTP for lesser reasons thus forcing us to maintain an old major release forever (at least for very long), and we do not have those resources. But times have changed for the better so this might be called an inconsistency that should be fixed. It is not my call, though. And I predict that it will be cumbersome to figure out what the "correct" behaviour would be for both cases on all supported systems and (mis)configurations? / Raimo > > Regards, > Oliver > ? > > Gesendet:?Montag, 11. Juli 2016 um 14:36 Uhr > Von:?"Raimo Niskanen" > An:?erlang-questions@REDACTED > Betreff:?Re: [erlang-questions] Inconsistent hostnames > On Tue, Jun 21, 2016 at 07:11:54PM +0000, Ryan Stewart wrote: > > In deploying my erlang app to some new CentOS servers with erlang 18.3, I > > came across an oddity in networking. The server's networking config, which > > is admittedly messed up, looks something like: > > short hostname: bob > > long hostname: bob.prod.example.net > > domain: example.com > > > > Now when I start erlang with "erl -sname x", net_adm:localhost() returns " > > bob.prod.example.net". > > > > However, if I start erlang with "erl -name x", net_adm:localhost() returns " > > bob.example.com". > > > > I believe this comes from net_kernel:create_hostpart/2, where it follows > > different paths on short vs long node names. On a long node name, it > > appears to use short hostname + domain, where on a short node name, it > > simply uses the long hostname. That decision appears to trickle through the > > rest of the networking libraries. > > > > I understand that we need to fix the config on the server. I just wondered > > if this might be considered a bug in erlang as well. > > This behaviour is old, and changing it would either surprise either some > important "short names" using application or ditto "long names" one. > > For -name the Erlang networking code tries to parse the OS configuration > files e.g /etc/resolv.conf to figure out the domain name. > net_adm:localhost() then puts them together. > > For -sname the Erlang networking code tries to look up the short hostname > using the native resolver to find a fully qualified alias, and takes the > domain from that. > > We might call this an annoying irregularity of not very big practical > importance, probably not worth changing... > > I actually do not understand why net_adm:localhost() returns the long > hostname for -sname in the first place. I think the documentation leaves > room for returning the short hostname, but changing that could probably > affect which IP address is used to contact EPMD which would surprise > some important "short names" using application. > > > > > Ryan > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From ok@REDACTED Tue Jul 12 00:44:48 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 12 Jul 2016 10:44:48 +1200 Subject: [erlang-questions] Add a new data structure to erlang VM. In-Reply-To: References: <854196bb787acb42fe40c5734e60f251.squirrel@chasm.otago.ac.nz> Message-ID: <93a418ed-f006-0951-261f-7d95cee5ba38@cs.otago.ac.nz> On 11/07/16 9:07 PM, Constantin Kulikov wrote: >> ETS tables are not data structures in an Erlang process. >Anyway, they are mutable(right?) Yes, ETS tables are mutable. Which is why they are not *Erlang* data structures. > and I didn't ask that a new data structure must necessarily reside in > an Erlang process. If we are talking about something that Erlang code has a *name* for (represented as say a reference) but cannot work with directly, then adding a data structure to Erlang is pretty much what NIFs are for. Objects that are large, long-lived, few, and have explicitly managed lifetimes are a good fit for that. Data base connections, special access to devices, that sort of thing. Objects that are small, multitudinous, and evanescent are not. The thing is that we are all floundering in the dark because you have yet to tell us what it is you actually want to add and what problem you expect to solve by adding it, and for that matter why it needs to be in the Erlang VM at all instead of being a C node. Background: I worked on Xerox Quintus Prolog, which put Quintus Prolog and Interlisp-D in the same box. The design we ended up with was the least painful and most efficient we could devise, and basically Prolog was given a chunk of the address space and Interlisp-D was given the rest and cross-language procedure calls involved copying. We worked hard to make sure each language's GC had to know as little as possible about the other. Around the same time, the Poplog system -- which is open source these days -- was in flower. That put Pop-11, Prolog, Common Lisp, SML, and I think Scheme in the one address space. It was and is an impressive piece of work but meant that Prolog and SML performance suffered. In contrast, the Prolog component of XQP did not suffer, as we didn't have to make compromises. From zzantozz@REDACTED Tue Jul 12 03:12:57 2016 From: zzantozz@REDACTED (Ryan Stewart) Date: Tue, 12 Jul 2016 01:12:57 +0000 Subject: [erlang-questions] Inconsistent hostnames In-Reply-To: <20160711132819.GC37988@erix.ericsson.se> References: <20160711123615.GA37988@erix.ericsson.se> <20160711132819.GC37988@erix.ericsson.se> Message-ID: > And I predict that it will be cumbersome to figure out what the "correct" > behaviour would be for both cases on all supported systems > and (mis)configurations? Well, I'd think that at least returning non-conflicting values in the various cases would be an improvement. Some sort of documentation of or warning about the behavior would be handy, too. I ran across it while building out some config management in Ansible, and it took some time to figure out why Erlang was complaining about its node names. On Mon, Jul 11, 2016 at 8:28 AM Raimo Niskanen < raimo+erlang-questions@REDACTED> wrote: > On Mon, Jul 11, 2016 at 03:08:17PM +0200, Oliver Korpilla wrote: > > Hello, Raimo. > > > > Agreed, it could cause surprises, but only on upgrading the system to a > newer OTP. Wouldn't it be a bit naive to expect no changes whatsoever when > doing that, anyway? > > "Naive" might be a harsh word. We have historically had internal customers > that pay our salary refusing to upgrade to a newer OTP for lesser reasons > thus forcing us to maintain an old major release forever > (at least for very long), and we do not have those resources. > > But times have changed for the better so this might be called an > inconsistency that should be fixed. It is not my call, though. > > And I predict that it will be cumbersome to figure out what the "correct" > behaviour would be for both cases on all supported systems > and (mis)configurations? > > / Raimo > > > > > > Regards, > > Oliver > > > > > > Gesendet: Montag, 11. Juli 2016 um 14:36 Uhr > > Von: "Raimo Niskanen" > > An: erlang-questions@REDACTED > > Betreff: Re: [erlang-questions] Inconsistent hostnames > > On Tue, Jun 21, 2016 at 07:11:54PM +0000, Ryan Stewart wrote: > > > In deploying my erlang app to some new CentOS servers with erlang > 18.3, I > > > came across an oddity in networking. The server's networking config, > which > > > is admittedly messed up, looks something like: > > > short hostname: bob > > > long hostname: bob.prod.example.net > > > domain: example.com > > > > > > Now when I start erlang with "erl -sname x", net_adm:localhost() > returns " > > > bob.prod.example.net". > > > > > > However, if I start erlang with "erl -name x", net_adm:localhost() > returns " > > > bob.example.com". > > > > > > I believe this comes from net_kernel:create_hostpart/2, where it > follows > > > different paths on short vs long node names. On a long node name, it > > > appears to use short hostname + domain, where on a short node name, it > > > simply uses the long hostname. That decision appears to trickle > through the > > > rest of the networking libraries. > > > > > > I understand that we need to fix the config on the server. I just > wondered > > > if this might be considered a bug in erlang as well. > > > > This behaviour is old, and changing it would either surprise either some > > important "short names" using application or ditto "long names" one. > > > > For -name the Erlang networking code tries to parse the OS configuration > > files e.g /etc/resolv.conf to figure out the domain name. > > net_adm:localhost() then puts them together. > > > > For -sname the Erlang networking code tries to look up the short hostname > > using the native resolver to find a fully qualified alias, and takes the > > domain from that. > > > > We might call this an annoying irregularity of not very big practical > > importance, probably not worth changing... > > > > I actually do not understand why net_adm:localhost() returns the long > > hostname for -sname in the first place. I think the documentation leaves > > room for returning the short hostname, but changing that could probably > > affect which IP address is used to contact EPMD which would surprise > > some important "short names" using application. > > > > > > > > Ryan > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Tue Jul 12 09:32:54 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 12 Jul 2016 09:32:54 +0200 Subject: [erlang-questions] Inconsistent hostnames In-Reply-To: References: <20160711123615.GA37988@erix.ericsson.se> <20160711132819.GC37988@erix.ericsson.se> Message-ID: Hi, I ran into the same kind of problem when interfacing with Java nodes, but I guess it happens with every external system where both parties need to come up with the same host name. If the name resolution is done differently, there are good chances that the names chosen on each side are different. One way to improve that is to resolve the hostname before starting the VM and use "-name foo@REDACTED" or "-sname foo@REDACTED", instead of relying on Erlang's resolution algorithm. It helps in many cases, but not all (for example if the short name has a dot, like 'myhost.local', Erlang doesn't like that). best regards, Vlad On Tue, Jul 12, 2016 at 3:12 AM, Ryan Stewart wrote: > > And I predict that it will be cumbersome to figure out what the > "correct" > > behaviour would be for both cases on all supported systems > > and (mis)configurations? > > Well, I'd think that at least returning non-conflicting values in the > various cases would be an improvement. Some sort of documentation of or > warning about the behavior would be handy, too. I ran across it while > building out some config management in Ansible, and it took some time to > figure out why Erlang was complaining about its node names. > > On Mon, Jul 11, 2016 at 8:28 AM Raimo Niskanen < > raimo+erlang-questions@REDACTED> wrote: > >> On Mon, Jul 11, 2016 at 03:08:17PM +0200, Oliver Korpilla wrote: >> > Hello, Raimo. >> > >> > Agreed, it could cause surprises, but only on upgrading the system to a >> newer OTP. Wouldn't it be a bit naive to expect no changes whatsoever when >> doing that, anyway? >> >> "Naive" might be a harsh word. We have historically had internal >> customers >> that pay our salary refusing to upgrade to a newer OTP for lesser reasons >> thus forcing us to maintain an old major release forever >> (at least for very long), and we do not have those resources. >> >> But times have changed for the better so this might be called an >> inconsistency that should be fixed. It is not my call, though. >> >> And I predict that it will be cumbersome to figure out what the "correct" >> behaviour would be for both cases on all supported systems >> and (mis)configurations? >> >> / Raimo >> >> >> > >> > Regards, >> > Oliver >> > >> > >> > Gesendet: Montag, 11. Juli 2016 um 14:36 Uhr >> > Von: "Raimo Niskanen" >> > An: erlang-questions@REDACTED >> > Betreff: Re: [erlang-questions] Inconsistent hostnames >> > On Tue, Jun 21, 2016 at 07:11:54PM +0000, Ryan Stewart wrote: >> > > In deploying my erlang app to some new CentOS servers with erlang >> 18.3, I >> > > came across an oddity in networking. The server's networking config, >> which >> > > is admittedly messed up, looks something like: >> > > short hostname: bob >> > > long hostname: bob.prod.example.net >> > > domain: example.com >> > > >> > > Now when I start erlang with "erl -sname x", net_adm:localhost() >> returns " >> > > bob.prod.example.net". >> > > >> > > However, if I start erlang with "erl -name x", net_adm:localhost() >> returns " >> > > bob.example.com". >> > > >> > > I believe this comes from net_kernel:create_hostpart/2, where it >> follows >> > > different paths on short vs long node names. On a long node name, it >> > > appears to use short hostname + domain, where on a short node name, it >> > > simply uses the long hostname. That decision appears to trickle >> through the >> > > rest of the networking libraries. >> > > >> > > I understand that we need to fix the config on the server. I just >> wondered >> > > if this might be considered a bug in erlang as well. >> > >> > This behaviour is old, and changing it would either surprise either some >> > important "short names" using application or ditto "long names" one. >> > >> > For -name the Erlang networking code tries to parse the OS configuration >> > files e.g /etc/resolv.conf to figure out the domain name. >> > net_adm:localhost() then puts them together. >> > >> > For -sname the Erlang networking code tries to look up the short >> hostname >> > using the native resolver to find a fully qualified alias, and takes the >> > domain from that. >> > >> > We might call this an annoying irregularity of not very big practical >> > importance, probably not worth changing... >> > >> > I actually do not understand why net_adm:localhost() returns the long >> > hostname for -sname in the first place. I think the documentation leaves >> > room for returning the short hostname, but changing that could probably >> > affect which IP address is used to contact EPMD which would surprise >> > some important "short names" using application. >> > >> > > >> > > Ryan >> > >> > > _______________________________________________ >> > > erlang-questions mailing list >> > > erlang-questions@REDACTED >> > > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> > -- >> > >> > / Raimo Niskanen, Erlang/OTP, Ericsson AB >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > >> http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] >> >> -- >> >> / Raimo Niskanen, Erlang/OTP, Ericsson AB >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Tue Jul 12 14:53:14 2016 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Tue, 12 Jul 2016 12:53:14 +0000 Subject: [erlang-questions] Add a new data structure to erlang VM. In-Reply-To: References: Message-ID: This seems to work because you are lucky. :) You can think of ErlNifEnv as an array, and ERL_NIF_TERM as an index into that array. The indexes can be copied however you like, but they are only valid on the original array. Your code is storing an index, but is not bringing along the original array. In some cases this will still work (atom terms don't need their environment), or seem to work, but you've planted the seeds of a future mysterious segfault. Now you can't keep the ErlNifEnv from your NIF function; you need to use enif_alloc_env() and then copy terms into that ErlNifEnv with enif_make_copy(). On Sun, Jul 10, 2016 at 12:17 PM Constantin Kulikov wrote: > This seem to work > https://gist.github.com/Bad-ptr/3ea0d6d65ae18b5b68c6cb15a96fef8e . > > However the documentation says that: > > All terms of type ERL_NIF_TERM belong to an environment of type > ErlNifEnv. The lifetime of a term is controlled by the lifetime of its > environment object. > > > Variables of type ERL_NIF_TERM can refer to any Erlang term. This is an > opaque type and values of it can only by used either as arguments to API > functions or as return values from NIFs. All ERL_NIF_TERM's belong to an > environment (ErlNifEnv). A term can not be destructed individually, it is > valid until its environment is destructed. > > > The environment is only valid in the thread where it was supplied as > argument until the NIF returns. It is thus useless and dangerous to store > pointers to process bound environments between NIF calls. > > So, it is not clear to me is it safe to store an ERL_NIF_TERM(in a form as > it passed to a NIF function) in a datastructure created by a NIF function? > And does the GC aware that this terms must not be collected ? Etc. > > > On 9 July 2016 at 21:59, Steve Vinoski wrote: > >> >> >> On Sat, Jul 9, 2016 at 1:54 PM, Constantin Kulikov >> wrote: >> >>> If I want to add a new datastructure to erlang(my local fork of the >>> erlang-otp from github), what should I do? Yes, I mean a C-level module >>> realization. >>> >>> Let's say I want to add a double linked list(just for simplicity) module >>> with an interface like that: >>> dlist:new() -> ?some king of reference to a list? >>> dlist:push(item) -> ?reference to a list? >>> dlist:pop() -> {item, ?reference to a list?} >>> etc... >>> >>> Is it possible at all? Has anyone ever tried to do something like this? >>> What definitions to what files should I add? How must I allocate my data >>> structure? How must I reference items in it? >>> Can someone point me to source locations where erlang's list/ets >>> table/tuple operations(creating a new, storing a reference to objects) >>> defined? >>> >> >> See >> >> http://erlang.org/doc/tutorial/nif.html >> http://erlang.org/doc/man/erl_nif.html >> >> --steve >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Tue Jul 12 16:25:20 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 12 Jul 2016 16:25:20 +0200 Subject: [erlang-questions] bad certificate for https://bugs.erlang.org In-Reply-To: <20160711124915.GB37988@erix.ericsson.se> References: <7lid51pc56bn8aplqcupqksg.1468139155213@email.android.com> <20160711124915.GB37988@erix.ericsson.se> Message-ID: <20160712142520.GA91263@erix.ericsson.se> On Mon, Jul 11, 2016 at 02:49:15PM +0200, Raimo Niskanen wrote: > On Sun, Jul 10, 2016 at 10:25:55AM +0200, ?ric Pailleau wrote: > > Hello, > > Yes and https send a self signed certificate. > > Browser ask to add a security exception but cert is C=Au while servers in Poland. > > There is a serious MITM risk. > > Do not add security exception and do not give your credentials. Am I paranoid ? > > It seems that the maintenance team at Erlang Solutions has misplaced the correct > certificate during an upgrade and I hope that they will soon fix that... The problem should be fixed now! -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From garry@REDACTED Wed Jul 13 02:00:34 2016 From: garry@REDACTED (Garry Hodgson) Date: Tue, 12 Jul 2016 20:00:34 -0400 Subject: [erlang-questions] why does rpc:call do this? Message-ID: <578584A2.60407@research.att.com> I noticed in lib/kernel/src/rpc.erl that got me curious, as I'm implementing something related. It seems like there's more work going on than needed, so I suspect I'm missing some subtlety. So, rpc:call() is a gen_server call to: handle_call({call, Mod, Fun, Args, Gleader}, To, S) -> handle_call_call(Mod, Fun, Args, Gleader, To, S); which just invokes: handle_call_call(Mod, Fun, Args, Gleader, To, S) -> RpcServer = self(), %% Spawn not to block the rpc server. {Caller,_} = erlang:spawn_monitor( fun () -> set_group_leader(Gleader), Reply = %% in case some sucker rex'es %% something that throws case catch apply(Mod, Fun, Args) of {'EXIT', _} = Exit -> {badrpc, Exit}; Result -> Result end, RpcServer ! {self(), {reply, Reply}} end), {noreply, gb_trees:insert(Caller, To, S)}. which spawns a process to run the function, stash its results in S, and send results as an erlang message to rpc process. That will then get handled as info message to rpc gen_server, handled here: handle_info({Caller, {reply, Reply}}, S) -> case gb_trees:lookup(Caller, S) of {value, To} -> receive {'DOWN', _, process, Caller, _} -> gen_server:reply(To, Reply), {noreply, gb_trees:delete(Caller, S)} end; none -> {noreply, S} end; which calls gen_server:reply() to send result to original caller. So the question is, why do the extra overhead of stashing result in gb_tree S and sending it as erlang message, vs. just having the fun in handle_call_call just use gen_server:reply() directly? is it to clean up the DOWN message from the spawn_monitor? if so, why is the monitor part of that needed? What benefit does it provide? Any insights appreciated. Thanks -- Garry Hodgson Lead Member of Technical Staff AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." From cian@REDACTED Wed Jul 13 02:47:57 2016 From: cian@REDACTED (Cian Synnott) Date: Wed, 13 Jul 2016 01:47:57 +0100 Subject: [erlang-questions] why does rpc:call do this? In-Reply-To: <578584A2.60407@research.att.com> References: <578584A2.60407@research.att.com> Message-ID: Hi Garry, On Wed, Jul 13, 2016 at 1:00 AM, Garry Hodgson wrote: > which spawns a process to run the function, stash its results in S, and send > results as an erlang message to rpc process. > The trick is that it is not the *result* that's stashed in `S` (the RPC gen_server's state), but a mapping of `Caller` : `To`. `Caller` here is the pid of the spawned process which does the actual apply/3 call. `To` is the RPC client that initiated all this. As the comment notes, the apply/3 is done in a separate process so that the RPC gen_server itself can get on with handling its next message while some set of potentially long-running apply's happen in the background. When the apply/3 is done, `Caller` sends back a message that's caught by handle_info as you note. The gen_server looks up the `Caller` : `To` mapping from earlier, to discover what client should receive the results sent back by the pid `Caller`. The monitor is setup so that the RPC gen_server can clean up its state if an apply/3 subprocess dies without sending back a message; presumably another clause of that same handle_info deals with that case. Cian From cian@REDACTED Wed Jul 13 03:09:50 2016 From: cian@REDACTED (Cian Synnott) Date: Wed, 13 Jul 2016 02:09:50 +0100 Subject: [erlang-questions] why does rpc:call do this? In-Reply-To: References: <578584A2.60407@research.att.com> Message-ID: Ah, I think I may have misunderstood your question. IIUC now, you're asking why the RPC gen_server remains in the loop once the work has been delegated to the spawned subprocess. I *think* that's just to properly handle reporting back to the client when the apply/3 subprocess blows up for some reason. Without the RPC gen_server monitoring for that, the client would fire (potentially bad) requests into the void without any acknowledgement or error messages - which of course may happen anyway, but I guess OTP is trying to be polite and provide as much information as possible. Cian From cian@REDACTED Wed Jul 13 03:14:22 2016 From: cian@REDACTED (Cian Synnott) Date: Wed, 13 Jul 2016 02:14:22 +0100 Subject: [erlang-questions] why does rpc:call do this? In-Reply-To: References: <578584A2.60407@research.att.com> Message-ID: On Wed, Jul 13, 2016 at 2:09 AM, Cian Synnott wrote: > I *think* that's just to properly handle reporting back to the client > when the apply/3 subprocess blows up for some reason. > FWIW, it looks like the current RPC server behaves more like you expected, cutting out some messages: https://github.com/erlang/otp/blob/maint/lib/kernel/src/rpc.erl#L178 and continues to do the cleanup & error reporting parts in handle_info: https://github.com/erlang/otp/blob/maint/lib/kernel/src/rpc.erl#L133 It uses maps rather than a gb_trees, too. :o) Cian From andre@REDACTED Tue Jul 12 19:51:50 2016 From: andre@REDACTED (=?utf-8?Q?Andr=C3=A9_Cruz?=) Date: Tue, 12 Jul 2016 18:51:50 +0100 Subject: [erlang-questions] Different SSL behaviours, how to pick ciphers? Message-ID: <77B4262B-22F1-4AB1-9B28-A6BF2AF07064@cabine.org> Hello. I'm observing different behaviours when running my Erlang code on two Erlang 18 beam instances. One is running on my macos machine provided by home-brew, and the other comes from the erlang:18 container: macos: Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] [dtrace] Eshell V7.3 (abort with ^G) 1> ssl:versions(). [{ssl_app,"7.3"}, {supported,['tlsv1.2','tlsv1.1',tlsv1]}, {available,['tlsv1.2','tlsv1.1',tlsv1,sslv3]}] 2> ssl:connect("test.search.windows.net", 443, [], infinity). {ok,{sslsocket,{gen_tcp,#Port<0.29349>,tls_connection, undefined}, <0.109.0>}} linux container: Erlang/OTP 18 [erts-7.3.1] [source] [64-bit] [smp:2:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V7.3.1 (abort with ^G) 1> ssl:versions(). [{ssl_app,"7.3.3"}, {supported,['tlsv1.2','tlsv1.1',tlsv1]}, {available,['tlsv1.2','tlsv1.1',tlsv1,sslv3]}] 2> ssl:connect("test.search.windows.net", 443, [], infinity). {error,closed} As can be seen I cannot establish a connection using the container version of Erlang. Looking at the traffic I can see that the ClientHello message specifies SSLv3 ciphers, while the version that works uses TLS1.2. How can I influence this choice of ciphers? Is it a problem with the openssl lib in the container image? Thank you and best regards, Andr? Cruz From zxnotdead@REDACTED Wed Jul 13 10:31:28 2016 From: zxnotdead@REDACTED (Constantin Kulikov) Date: Wed, 13 Jul 2016 11:31:28 +0300 Subject: [erlang-questions] Add a new data structure to erlang VM. In-Reply-To: References: Message-ID: > This seems to work because you are lucky. :) Yes, it's crashing randomly after a number of pushes/pops. > You can think of ErlNifEnv as an array, and ERL_NIF_TERM as an index into that array. The indexes can be copied however you like, but they are only valid on the original array. Your code is storing an index, but is not bringing along the original array. In some cases this will still work (atom terms don't need their environment), or seem to work, but you've planted the seeds of a future mysterious segfault. > Now you can't keep the ErlNifEnv from your NIF function; you need to use enif_alloc_env() and then copy terms into that ErlNifEnv with enif_make_copy(). Thanks for the clarification. On 12 July 2016 at 15:53, Daniel Goertzen wrote: > This seems to work because you are lucky. :) > > You can think of ErlNifEnv as an array, and ERL_NIF_TERM as an index into > that array. The indexes can be copied however you like, but they are only > valid on the original array. Your code is storing an index, but is not > bringing along the original array. In some cases this will still work > (atom terms don't need their environment), or seem to work, but you've > planted the seeds of a future mysterious segfault. > > Now you can't keep the ErlNifEnv from your NIF function; you need to use > enif_alloc_env() and then copy terms into that ErlNifEnv with > enif_make_copy(). > > > > On Sun, Jul 10, 2016 at 12:17 PM Constantin Kulikov > wrote: > >> This seem to work >> https://gist.github.com/Bad-ptr/3ea0d6d65ae18b5b68c6cb15a96fef8e . >> >> However the documentation says that: >> > All terms of type ERL_NIF_TERM belong to an environment of type >> ErlNifEnv. The lifetime of a term is controlled by the lifetime of its >> environment object. >> >> > Variables of type ERL_NIF_TERM can refer to any Erlang term. This is an >> opaque type and values of it can only by used either as arguments to API >> functions or as return values from NIFs. All ERL_NIF_TERM's belong to an >> environment (ErlNifEnv). A term can not be destructed individually, it is >> valid until its environment is destructed. >> >> > The environment is only valid in the thread where it was supplied as >> argument until the NIF returns. It is thus useless and dangerous to store >> pointers to process bound environments between NIF calls. >> >> So, it is not clear to me is it safe to store an ERL_NIF_TERM(in a form >> as it passed to a NIF function) in a datastructure created by a NIF >> function? And does the GC aware that this terms must not be collected ? Etc. >> >> >> On 9 July 2016 at 21:59, Steve Vinoski wrote: >> >>> >>> >>> On Sat, Jul 9, 2016 at 1:54 PM, Constantin Kulikov >>> wrote: >>> >>>> If I want to add a new datastructure to erlang(my local fork of the >>>> erlang-otp from github), what should I do? Yes, I mean a C-level module >>>> realization. >>>> >>>> Let's say I want to add a double linked list(just for simplicity) >>>> module with an interface like that: >>>> dlist:new() -> ?some king of reference to a list? >>>> dlist:push(item) -> ?reference to a list? >>>> dlist:pop() -> {item, ?reference to a list?} >>>> etc... >>>> >>>> Is it possible at all? Has anyone ever tried to do something like this? >>>> What definitions to what files should I add? How must I allocate my >>>> data structure? How must I reference items in it? >>>> Can someone point me to source locations where erlang's list/ets >>>> table/tuple operations(creating a new, storing a reference to objects) >>>> defined? >>>> >>> >>> See >>> >>> http://erlang.org/doc/tutorial/nif.html >>> http://erlang.org/doc/man/erl_nif.html >>> >>> --steve >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxnotdead@REDACTED Wed Jul 13 11:31:30 2016 From: zxnotdead@REDACTED (Constantin Kulikov) Date: Wed, 13 Jul 2016 12:31:30 +0300 Subject: [erlang-questions] Add a new data structure to erlang VM. In-Reply-To: <93a418ed-f006-0951-261f-7d95cee5ba38@cs.otago.ac.nz> References: <854196bb787acb42fe40c5734e60f251.squirrel@chasm.otago.ac.nz> <93a418ed-f006-0951-261f-7d95cee5ba38@cs.otago.ac.nz> Message-ID: > ETS tables are mutable. Which is why they are not *Erlang* data structures. Just to clarify, are objects copied when we put/get them from an ets table? Looks like I mixed and confused some of my statements. Looks like I want my data structure to reside in the Erlang process' memory, because I don't want to always copy data when I put/get something in/from that data structure. However it doesn't necessarily have to be as transparent to the programmer as the Erlang's lists. With NIFs I always must copy data between environments. And it looks like enif_make_copy is doing a full copy of an object: https://github.com/erlang/otp/blob/3b7a6ffddc819bf305353a593904cea9e932e7dc/erts/emulator/beam/erl_nif.c#L795 https://github.com/erlang/otp/blob/3b7a6ffddc819bf305353a593904cea9e932e7dc/erts/emulator/beam/copy.c#L73 https://github.com/erlang/otp/blob/3b7a6ffddc819bf305353a593904cea9e932e7dc/erts/emulator/beam/global.h#L1127 https://github.com/erlang/otp/blob/3b7a6ffddc819bf305353a593904cea9e932e7dc/erts/emulator/beam/copy.c#L602 And I can only add a term to the environment, but can not free a term other than freeing the environment entirely with all it's terms. I would do enif_make_env for each single object that I want to store, but it looks like a bad idea: http://erlang.2086793.n4.nabble.com/erl-nif-questions-regarding-resource-handling-td4673810.html#a4673832 > The current implementation of environments from enif_make_env() has a quite large constant memory overhead. > A lot of environments with small/few terms in them will not be memory efficient. The next step is to make a "custom GC" based on managing these environments like here: https://github.com/fauxsoup/neural (hmm, maybe Erlang's GC works like this) Well, the more I dive into that, the more I see that adding a data structure to the Erlang VM level is not so easy and doesn't worth it.) Thanks for your help. On 12 July 2016 at 01:44, Richard A. O'Keefe wrote: > > > On 11/07/16 9:07 PM, Constantin Kulikov wrote: > >> ETS tables are not data structures in an Erlang process. > > >Anyway, they are mutable(right?) > Yes, ETS tables are mutable. Which is why they are not *Erlang* data > structures. > >> and I didn't ask that a new data structure must necessarily reside in an >> Erlang process. >> > If we are talking about something that Erlang code has a *name* for > (represented as say a reference) but cannot work with directly, then > adding a data structure to Erlang is pretty much what NIFs are for. > > Objects that are large, long-lived, few, and have explicitly managed > lifetimes are a good fit for that. Data base connections, special > access to devices, that sort of thing. > > Objects that are small, multitudinous, and evanescent are not. > > The thing is that we are all floundering in the dark because you have > yet to tell us what it is you actually want to add and what problem > you expect to solve by adding it, and for that matter why it needs to > be in the Erlang VM at all instead of being a C node. > > Background: I worked on Xerox Quintus Prolog, which put Quintus Prolog > and Interlisp-D in the same box. The design we ended up with was the > least painful and most efficient we could devise, and basically Prolog was > given a chunk of the address space and Interlisp-D was given the rest > and cross-language procedure calls involved copying. We worked hard > to make sure each language's GC had to know as little as possible about > the other. Around the same time, the Poplog system -- which is open > source these days -- was in flower. That put Pop-11, Prolog, Common > Lisp, SML, and I think Scheme in the one address space. It was and is > an impressive piece of work but meant that Prolog and SML performance > suffered. In contrast, the Prolog component of XQP did not suffer, as > we didn't have to make compromises. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andre@REDACTED Wed Jul 13 11:47:46 2016 From: andre@REDACTED (=?utf-8?Q?Andr=C3=A9_Cruz?=) Date: Wed, 13 Jul 2016 10:47:46 +0100 Subject: [erlang-questions] Different SSL behaviours, how to pick ciphers? In-Reply-To: <77B4262B-22F1-4AB1-9B28-A6BF2AF07064@cabine.org> References: <77B4262B-22F1-4AB1-9B28-A6BF2AF07064@cabine.org> Message-ID: On 12 Jul 2016, at 18:51, Andr? Cruz wrote: > > Hello. > > I'm observing different behaviours when running my Erlang code on two Erlang 18 beam instances. One is running on my macos machine provided by home-brew, and the other comes from the erlang:18 container: Upon further investigation I've noticed that this behaviour started happening with the erlang 18.3.4 deb package provided by Erlang Solutions. Version 18.3 does not exhibit this problem. Andr? From jesper.louis.andersen@REDACTED Wed Jul 13 12:33:10 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 13 Jul 2016 12:33:10 +0200 Subject: [erlang-questions] Add a new data structure to erlang VM. In-Reply-To: References: <854196bb787acb42fe40c5734e60f251.squirrel@chasm.otago.ac.nz> <93a418ed-f006-0951-261f-7d95cee5ba38@cs.otago.ac.nz> Message-ID: On Wed, Jul 13, 2016 at 11:31 AM, Constantin Kulikov wrote: > Just to clarify, are objects copied when we put/get them from an ets table? Yes. Consider they were not. We have an object X in the table. Process P1 gets X Process P2 updates X to X' in the ETS table. Now, according to immutability, P1's X should still be the same, but a reference to the object is X' in the table now, and thus P1 reads X' instead, breaking immutability of process P1. The next case is even funnier: P1 gets X P1 sends X to P3 P2 updates X to X' Note that P3 is one a different node in the cluster. This has to work seamlessly in Erlang, or you are in deep trouble. In other words, we are looking at a situation where you have to implement some copy-on-write scheme underneath in order to make things look like they do today. It may be possible, but one has to evaluate the implementation complexity over the efficiency gain. Copying will actually be faster in many cases on modern CPUs because it implicitly breaks cache conflicts. In many cases, the trick with ETS is: * Not storing large objects. Break them apart, so you can request subparts * Fetch objects once, and subscribe to changes on objects * Limit copying with ets:lookup_element/3 so you only copy over relevant parts of a tuple. "copying" an integer is mostly just a register transfer for instance. Another thing you may want to care about is latency. If you equip your own tuple space with a GC, as in NEURAL, you have to account for the latency when GC occurs. ETS is specifically built for the case where such a GC latency is unacceptable. This means it is fair to make a trade-off where lookup latency is worse but predictable, but there will be *no* garbage collections at all. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From andre@REDACTED Wed Jul 13 13:34:22 2016 From: andre@REDACTED (=?utf-8?Q?Andr=C3=A9_Cruz?=) Date: Wed, 13 Jul 2016 12:34:22 +0100 Subject: [erlang-questions] Different SSL behaviours, how to pick ciphers? In-Reply-To: References: <77B4262B-22F1-4AB1-9B28-A6BF2AF07064@cabine.org> Message-ID: <32B3A8C9-51DA-48D3-8D54-2C26D83AC1EB@cabine.org> On 13 Jul 2016, at 10:47, Andr? Cruz wrote: > > On 12 Jul 2016, at 18:51, Andr? Cruz wrote: >> >> Hello. >> >> I'm observing different behaviours when running my Erlang code on two Erlang 18 beam instances. One is running on my macos machine provided by home-brew, and the other comes from the erlang:18 container: > > Upon further investigation I've noticed that this behaviour started happening with the erlang 18.3.4 deb package provided by Erlang Solutions. Version 18.3 does not exhibit this problem. So, after trying all intermediate versions it seems this behaviour started happening on the transition 18.3.1 -> 18.3.2. Andr? From mononcqc@REDACTED Wed Jul 13 15:41:40 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 13 Jul 2016 09:41:40 -0400 Subject: [erlang-questions] Different SSL behaviours, how to pick ciphers? In-Reply-To: <77B4262B-22F1-4AB1-9B28-A6BF2AF07064@cabine.org> References: <77B4262B-22F1-4AB1-9B28-A6BF2AF07064@cabine.org> Message-ID: <20160713134134.GA51637@fhebert-ltm2.internal.salesforce.com> On 07/12, Andr? Cruz wrote: >As can be seen I cannot establish a connection using the container >version of Erlang. Looking at the traffic I can see that the >ClientHello message specifies SSLv3 ciphers, while the version that >works uses TLS1.2. How can I influence this choice of ciphers? Is it a >problem with the openssl lib in the container image? > You should at the very least have some basic configuration of SSL in Erlang -- the one that ships stock isn't particularly great. Say for example: [ {ciphers, Ciphers}, {honor_cipher_order, true}, {secure_renegotiate, true}, {client_renegotiation, false}, {versions, ['tlsv1.2', 'tlsv1.1', 'tlsv1']} ] Where `Ciphers' is a list based on either the tools at https://wiki.mozilla.org/Security/Server_Side_TLS or http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/elb-security-policy-table.html so you avoid ciphersuites deemed unsafe or which could give you compatibility issues (with say HTTP/2, which has its own blacklist of certificates) As a client -- and possibly as a server -- you'll also want to validate SSL certificates. To do so, you can use `certifi' to provide a decent set of root CAs (https://github.com/certifi/erlang-certifi) and `ssl_verify_fun' (https://hex.pm/packages/ssl_verify_fun) to do the actual validation: [ {verify, verify_peer}, {depth, 2}, {cacerts, certifi:cacerts()}, {server_name_indication, Hostname}, {verify_fun, {fun ssl_verify_hostname:verify_fun/3, [{check_hostname, Hostname}]} ] With these options, certificate validation can take place. In case you want to do certificate pinning (validating that the request comes or goes to a known certificate -- more useful if you're working with self-signed stuff), I've so far used `Tak' (https://github.com/heroku/tak) to generate the SSL configuration required. Regards, Fred. From andre@REDACTED Wed Jul 13 17:27:22 2016 From: andre@REDACTED (=?iso-8859-1?Q?Andr=E9_Cruz?=) Date: Wed, 13 Jul 2016 16:27:22 +0100 Subject: [erlang-questions] Different SSL behaviours, how to pick ciphers? In-Reply-To: <20160713134134.GA51637@fhebert-ltm2.internal.salesforce.com> References: <77B4262B-22F1-4AB1-9B28-A6BF2AF07064@cabine.org> <20160713134134.GA51637@fhebert-ltm2.internal.salesforce.com> Message-ID: <8E730584-E028-4AD3-942F-B38846F3F555@cabine.org> Hello Fred. > On 13 Jul 2016, at 14:41, Fred Hebert wrote: > > On 07/12, Andr? Cruz wrote: >> As can be seen I cannot establish a connection using the container version of Erlang. Looking at the traffic I can see that the ClientHello message specifies SSLv3 ciphers, while the version that works uses TLS1.2. How can I influence this choice of ciphers? Is it a problem with the openssl lib in the container image? >> > > You should at the very least have some basic configuration of SSL in Erlang -- the one that ships stock isn't particularly great. I've found the difference in the default SSL configuration between 18.3.1 and 18.3.2. 18.3.1 uses TLS1.2 records: TLSv1.2 Record Layer: Handshake Protocol: Client Hello Content Type: Handshake (22) Version: TLS 1.0 (0x0301) Length: 279 18.3.2 uses SSL records: SSL Record Layer: Handshake Protocol: Client Hello Content Type: Handshake (22) Version: TLS 1.0 (0x0301) Length: 249 It's strange to change this default in a minor version upgrade. Is this something that can be configured? I've found that some SSL servers drop the connection immediately when SSL records are used. Thanks, Andr? From andre@REDACTED Wed Jul 13 19:08:12 2016 From: andre@REDACTED (=?iso-8859-1?Q?Andr=E9_Cruz?=) Date: Wed, 13 Jul 2016 18:08:12 +0100 Subject: [erlang-questions] Different SSL behaviours, how to pick ciphers? In-Reply-To: <20160713160246.GB51637@fhebert-ltm2.internal.salesforce.com> References: <77B4262B-22F1-4AB1-9B28-A6BF2AF07064@cabine.org> <20160713134134.GA51637@fhebert-ltm2.internal.salesforce.com> <8E730584-E028-4AD3-942F-B38846F3F555@cabine.org> <20160713160246.GB51637@fhebert-ltm2.internal.salesforce.com> Message-ID: On 13 Jul 2016, at 17:02, Fred Hebert wrote: > > On 07/13, Andr? Cruz wrote: >> >> It's strange to change this default in a minor version upgrade. Is this something that can be configured? I've found that some SSL servers drop the connection immediately when SSL records are used. >> > > The `versions' tuple should specify the versions in order if you have them configured for your socket. The `honor_cipher_order` thing lets the server force the order you gave as the order to honor for ciphers. The problem was that since Erlang 18.3.2 the "signature_algorithms" extension is only sent if the supported ssl versions is just 'tlsv1.2'. Since the default is tls 1 though 1.2, this extension is not sent and some servers require it. It's strange that I'm the only one with this problem... Anyway, the solution in this case was to specify {versions, ['tlsv1.2']}, but if you need to support other tls versions you may need to manually fill the "signature_algs" option. Best regards, Andr? Cruz From baswegh@REDACTED Thu Jul 14 08:04:49 2016 From: baswegh@REDACTED (Bas Wegh) Date: Thu, 14 Jul 2016 07:04:49 +0100 Subject: [erlang-questions] [ANN] OpenId Connect client library oidcc Message-ID: <20160714060449.GA4723@lenovo6298.kit.edu> Hi all, I have implemented a client library for OpenId Connect which can be found at github: https://github.com/indigo-dc/oidcc A bit more background for those interested: The library is part of implementing a product, the Token Translation Service[1], for the INDIGO DataCloud[2] project. Please have a look and let me/us know what you think. Feedback is very welcome. [1] https://github.com/indigo-dc/tts [2] https://www.indigo-datacloud.eu/ -- Bas Wegh From felixgallo@REDACTED Thu Jul 14 14:37:10 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Thu, 14 Jul 2016 05:37:10 -0700 Subject: [erlang-questions] Elixir's GenStage In-Reply-To: References: Message-ID: Quite possibly of interest: http://elixir-lang.org/blog/2016/07/14/announcing-genstage/ The inimitable Mr. Valim and company are doing some really interesting thinking around composable data processing in BEAM. F. -------------- next part -------------- An HTML attachment was scrubbed... URL: From abeniaminov@REDACTED Thu Jul 14 20:06:19 2016 From: abeniaminov@REDACTED (Alexandre Beniaminov) Date: Thu, 14 Jul 2016 21:06:19 +0300 Subject: [erlang-questions] atomic ets:take_first, ets:take_last Message-ID: Dear Developers, is case ets:first(T) of '$end_of_table' -> '$end_of_table'; Key -> ets:take(T, Key) end atomic for ordered_set ets ? If not it would be great to have atomic function like ets:take_first and ets:take_last -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Jul 14 20:19:41 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 14 Jul 2016 20:19:41 +0200 Subject: [erlang-questions] atomic ets:take_first, ets:take_last In-Reply-To: References: Message-ID: On Thu, Jul 14, 2016 at 8:06 PM, Alexandre Beniaminov wrote: > atomic You have to say what you expect to be atomic/linearizable. In your above code, suppose you are being scheduled out between the call to ets:first/1 and ets:take/2. Now, anything can happen in between, so in particular, ets:take/2 may return [] if the key was deleted or taken out by another process between the two calls. But taken individually, the set of ets:first/1, ets:take/2 and ets:lookup/2 ought to be linearizable. I.e., there is some point in time (the atomic commit-point) at which ets:take/2 removes the objects. After this commit-point first/1 and lookup/2 will not see the Key present in the table anymore. So the question of if it is atomic or not depends on the precise definition of what you ask. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Fri Jul 15 01:12:33 2016 From: rvirding@REDACTED (Robert Virding) Date: Fri, 15 Jul 2016 01:12:33 +0200 Subject: [erlang-questions] Add a new data structure to erlang VM. In-Reply-To: References: Message-ID: The data in an ets table and a process dictionary is not mutable! As ROK points out the ets tables are outside all processes. In fact an ets tables behaves in many ways as if it was a process and the table elements were local to that process even if they are not implemented as a process. I have in fact implemented ets tables using processes and got the same behaviour. Implementing a doubly linked list will makes the actual data structure mutable. Robert On 11 July 2016 at 07:17, Constantin Kulikov wrote: > > everything else by design is immutable > > Tell this to ets tables and process dicts.) > > > On 11 July 2016 at 04:21, Robert Virding wrote: > >> It feels a bit off adding a mutable data structure to erlang where >> everything else by design is immutable. >> >> Robert >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Catenacci@REDACTED Fri Jul 15 02:19:20 2016 From: Catenacci@REDACTED (Onorio Catenacci) Date: Thu, 14 Jul 2016 20:19:20 -0400 Subject: [erlang-questions] Reactors, Channels and Event Streams for Composable Distributed Programming Message-ID: Via FourShortLinks: https://www.researchgate.net/publication/290195429_Reactors_Channels_and_Event_Streams_for_Composable_Distributed_Programming It's a free PDF but you have to register on the site. Given the audience here I thought it might present some interest to some of you. -- Onorio Catenacci http://onor.io http://www.google.com/+OnorioCatenacci -------------- next part -------------- An HTML attachment was scrubbed... URL: From abeniaminov@REDACTED Fri Jul 15 00:48:49 2016 From: abeniaminov@REDACTED (Alexandre Beniaminov) Date: Fri, 15 Jul 2016 01:48:49 +0300 Subject: [erlang-questions] atomic ets:take_first, ets:take_last In-Reply-To: References: Message-ID: Thank you. You're right, and you've answered on part of my question: << Now, anything can happen in between, so in particular, ets:take/2 may return [] if the key was deleted or taken out by another process between the two calls.>> I would like to ask the community what do you think about to add to ets module two atomic functions: ets:take_first/1 and ets:take_last/1 ? 2016-07-15 1:39 GMT+03:00 Alexandre Beniaminov : > Thank you. > You're right, and you've answered on part of my question: > > << Now, anything can happen in between, so in particular, ets:take/2 may > return [] if the key was deleted or taken out by another process between > the two calls.>> > > I would like to ask the community what do you think about to add to ets > module two atomic functions: ets:take_first/1 and ets:take_last/1 ? > > 2016-07-14 21:19 GMT+03:00 Jesper Louis Andersen < > jesper.louis.andersen@REDACTED>: > >> >> On Thu, Jul 14, 2016 at 8:06 PM, Alexandre Beniaminov < >> abeniaminov@REDACTED> wrote: >> >>> atomic >> >> >> You have to say what you expect to be atomic/linearizable. In your above >> code, suppose you are being scheduled out between the call to ets:first/1 >> and ets:take/2. Now, anything can happen in between, so in particular, >> ets:take/2 may return [] if the key was deleted or taken out by another >> process between the two calls. >> >> But taken individually, the set of ets:first/1, ets:take/2 and >> ets:lookup/2 ought to be linearizable. I.e., there is some point in time >> (the atomic commit-point) at which ets:take/2 removes the objects. After >> this commit-point first/1 and lookup/2 will not see the Key present in the >> table anymore. >> >> So the question of if it is atomic or not depends on the precise >> definition of what you ask. >> >> >> -- >> J. >> > > > > -- > ? ?????????, > ????????? ?????????? > -- ? ?????????, ????????? ?????????? -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Jul 15 10:14:07 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 15 Jul 2016 11:14:07 +0300 Subject: [erlang-questions] Add a new data structure to erlang VM. In-Reply-To: References: Message-ID: Well, I don't see any crime in creating create-once structure like: double_linked_list:from_list(List) that will have some behaviour allowing to go forward and backward. But this structure must be create once, not extendable and updatable. And still it looks like a mind training, not a real task. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cchalasani@REDACTED Fri Jul 15 12:47:55 2016 From: cchalasani@REDACTED (Chaitanya Chalasani) Date: Fri, 15 Jul 2016 16:17:55 +0530 Subject: [erlang-questions] Better way to check if a set of keys exists in a mnesia table? Message-ID: <989F10C2-C30F-4462-A25E-CEC617A11950@me.com> I have a table with an UUID as the primary key / first element of the record. What is the efficient way to check if a given set of UUIDs are valid primary key for that table. I can think for three different solutions - use mnesia:all_leys(TableName) and perform lists subset check. However, if the table contains over a million records, fetching all the keys for every check isn?t a nice solution. use mnesia:read(TableName, Key) and check on the response. However, if the row is a big enough, trying to get the whole row for a simple key check isn?t that good either. use ets:member(TableName, Key). A better solution than the above but doesn?t work on remote tables. Which one of the above is the least bad solution or is there a better one hidden under the documents. /Chaitanya -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Fri Jul 15 13:40:01 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Fri, 15 Jul 2016 13:40:01 +0200 Subject: [erlang-questions] Better way to check if a set of keys exists in a mnesia table? In-Reply-To: <989F10C2-C30F-4462-A25E-CEC617A11950@me.com> References: <989F10C2-C30F-4462-A25E-CEC617A11950@me.com> Message-ID: <2540fphnvvmjhf4hed92u95g.1468582801687@email.android.com> Hi, I would probably add a record entry with second element of the UUID: xxxxxxxx-4c31-... 43c1 here and add an index on it. It may increase speed a lot on big tables. This have to be tested however. "Envoy? depuis mon mobile " Eric ---- Chaitanya Chalasani a ?crit ---- >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.koorn@REDACTED Fri Jul 15 14:16:15 2016 From: eric.koorn@REDACTED (eric koorn) Date: Fri, 15 Jul 2016 14:16:15 +0200 Subject: [erlang-questions] Reactors, Channels and Event Streams for Composable Distributed Programming In-Reply-To: References: Message-ID: Also free and without registration: http://sci-hub.cc/10.1145/2814228.2814245 On Fri, Jul 15, 2016 at 2:19 AM, Onorio Catenacci wrote: > Via FourShortLinks: > > https://www.researchgate.net/publication/290195429_Reactors_Channels_and_Event_Streams_for_Composable_Distributed_Programming > > It's a free PDF but you have to register on the site. Given the audience > here I thought it might present some interest to some of you. > > -- > Onorio Catenacci > > http://onor.io > http://www.google.com/+OnorioCatenacci > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Ciao, Eric Koorn From bjorn-egil.xb.dahlberg@REDACTED Fri Jul 15 14:54:54 2016 From: bjorn-egil.xb.dahlberg@REDACTED (=?UTF-8?Q?Bj=c3=b6rn-Egil_Dahlberg_XB?=) Date: Fri, 15 Jul 2016 14:54:54 +0200 Subject: [erlang-questions] Patch package OTP 19.0.2 released Message-ID: <5788DD1E.3090801@ericsson.com> Patch Package: OTP 19.0.2 Git Tag: OTP-19.0.2 Date: 2016-07-15 Trouble Report Id: OTP-13719, OTP-13731, OTP-13732, OTP-13738 Seq num: seq13142 System: OTP Release: 19 Application: compiler-7.0.1, erts-8.0.2, stdlib-3.0.1 Predecessor: OTP 19.0.1 Check out the git tag OTP-19.0.2, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- compiler-7.0.1 -------------------------------------------------- --------------------------------------------------------------------- The compiler-7.0.1 application can be applied independently of other applications on a full OTP 19 installation. --- Fixed Bugs and Malfunctions --- OTP-13738 Application(s): compiler Related Id(s): ERL-190 A literal binary matching regression was introduced in 19.0 where a match could fail to resolve to the right clause. This has now been fixed. Full runtime dependencies of compiler-7.0.1: crypto-3.6, erts-7.0, hipe-3.12, kernel-4.0, stdlib-2.5 --------------------------------------------------------------------- --- erts-8.0.2 ------------------------------------------------------ --------------------------------------------------------------------- The erts-8.0.2 application can be applied independently of other applications on a full OTP 19 installation. --- Fixed Bugs and Malfunctions --- OTP-13731 Application(s): erts Related Id(s): ERL-188 Fix scheduler deadlock bug in ets:update_counter/4 when key is not found and inserting the default object causes the table to grow. OTP-13732 Application(s): erts Related Id(s): seq13142 Fix VM abort "Overrun stack and heap" in garbage collection triggered by a bsl operation and some very specific heap conditions. Full runtime dependencies of erts-8.0.2: kernel-5.0, sasl-3.0, stdlib-3.0 --------------------------------------------------------------------- --- stdlib-3.0.1 ---------------------------------------------------- --------------------------------------------------------------------- The stdlib-3.0.1 application can be applied independently of other applications on a full OTP 19 installation. --- Fixed Bugs and Malfunctions --- OTP-13719 Application(s): stdlib Related Id(s): ERL-182 Correct a bug regarding typed records in the Erlang shell. The bug was introduced in OTP-19.0. Full runtime dependencies of stdlib-3.0.1: compiler-5.0, crypto-3.3, erts-8.0, kernel-5.0, sasl-3.0 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- This patch is only available through GitHub, https://github.com/erlang/otp Regards, Bj?rn-Egil From cchalasani@REDACTED Fri Jul 15 15:34:34 2016 From: cchalasani@REDACTED (Chaitanya Chalasani) Date: Fri, 15 Jul 2016 19:04:34 +0530 Subject: [erlang-questions] Better way to check if a set of keys exists in a mnesia table? In-Reply-To: <2540fphnvvmjhf4hed92u95g.1468582801687@email.android.com> References: <989F10C2-C30F-4462-A25E-CEC617A11950@me.com> <2540fphnvvmjhf4hed92u95g.1468582801687@email.android.com> Message-ID: Thank you Eric. I would ponder on that. > On 15-Jul-2016, at 17:10, ?ric Pailleau wrote: > > Hi, > I would probably add a record entry with second element of the UUID: xxxxxxxx-4c31-... > 43c1 here and add an index on it. > It may increase speed a lot on big tables. > This have to be tested however. > > "Envoy? depuis mon mobile " Eric > > > > ---- Chaitanya Chalasani a ?crit ---- > > I have a table with an UUID as the primary key / first element of the record. > > What is the efficient way to check if a given set of UUIDs are valid primary key for that table. > > I can think for three different solutions - > use mnesia:all_leys(TableName) and perform lists subset check. However, if the table contains over a million records, fetching all the keys for every check isn?t a nice solution. > use mnesia:read(TableName, Key) and check on the response. However, if the row is a big enough, trying to get the whole row for a simple key check isn?t that good either. > use ets:member(TableName, Key). A better solution than the above but doesn?t work on remote tables. > > Which one of the above is the least bad solution or is there a better one hidden under the documents. > > /Chaitanya -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Fri Jul 15 15:57:18 2016 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 15 Jul 2016 14:57:18 +0100 Subject: [erlang-questions] Deterministic destruction of NIF resources Message-ID: Who owns NIF resources? When are they garbage collected? They don't seem to be owned by the erlang process that called into the NIF to allocate the resource, because they're not garbage collected when that process dies. Or, to look at it another way: how do I return a resource from a NIF, and have it destructed deterministically? From sergej.jurecko@REDACTED Fri Jul 15 16:10:57 2016 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Fri, 15 Jul 2016 16:10:57 +0200 Subject: [erlang-questions] Deterministic destruction of NIF resources In-Reply-To: References: Message-ID: <0075DF79-88A3-4588-BBEE-81852B50A7CB@gmail.com> > On 15 Jul 2016, at 15:57, Roger Lipscombe wrote: > > because they're not garbage collected when that > process dies. Then there is an outstanding reference to the resource. Has it been sent to another process or stored in an ETS? Sergej -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Fri Jul 15 16:39:18 2016 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 15 Jul 2016 15:39:18 +0100 Subject: [erlang-questions] Deterministic destruction of NIF resources In-Reply-To: <0075DF79-88A3-4588-BBEE-81852B50A7CB@gmail.com> References: <0075DF79-88A3-4588-BBEE-81852B50A7CB@gmail.com> Message-ID: Sorry, let me rephrase: they're not garbage collected *when* that process dies. The destructor is sometimes called immediately, and sometimes at some point in the future. Can I make this deterministic? And, no, the resource has not been sent to another process or stored in ETS. On 15 July 2016 at 15:10, Sergej Jure?ko wrote: > > On 15 Jul 2016, at 15:57, Roger Lipscombe wrote: > > because they're not garbage collected when that > process dies. > > > Then there is an outstanding reference to the resource. Has it been sent to > another process or stored in an ETS? > > > Sergej From garazdawi@REDACTED Fri Jul 15 16:53:40 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Fri, 15 Jul 2016 16:53:40 +0200 Subject: [erlang-questions] Deterministic destruction of NIF resources In-Reply-To: References: <0075DF79-88A3-4588-BBEE-81852B50A7CB@gmail.com> Message-ID: How do you know when the process dies? If you aren't already you should use the trace flag 'exiting' ( http://erlang.org/doc/man/erlang.html#trace-3) to figure out when a process actually has died. ERTS will delay process death after you send it a termination signal for various reasons. On Fri, Jul 15, 2016 at 4:39 PM, Roger Lipscombe wrote: > Sorry, let me rephrase: they're not garbage collected *when* that > process dies. The destructor is sometimes called immediately, and > sometimes at some point in the future. Can I make this deterministic? > > And, no, the resource has not been sent to another process or stored in > ETS. > > On 15 July 2016 at 15:10, Sergej Jure?ko wrote: > > > > On 15 Jul 2016, at 15:57, Roger Lipscombe > wrote: > > > > because they're not garbage collected when that > > process dies. > > > > > > Then there is an outstanding reference to the resource. Has it been sent > to > > another process or stored in an ETS? > > > > > > Sergej > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Fri Jul 15 17:05:11 2016 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Fri, 15 Jul 2016 15:05:11 +0000 Subject: [erlang-questions] Deterministic destruction of NIF resources In-Reply-To: References: Message-ID: Here's a sequence I used to get deterministic resource destruction https://github.com/goertzenator/nifpp/blob/master/examples/nifpptest.erl#L215-L224 It's been a few years since I've run that however; there's a chance newer Erlangs might break it. On Fri, Jul 15, 2016 at 8:57 AM Roger Lipscombe wrote: > Who owns NIF resources? When are they garbage collected? They don't > seem to be owned by the erlang process that called into the NIF to > allocate the resource, because they're not garbage collected when that > process dies. > > Or, to look at it another way: how do I return a resource from a NIF, > and have it destructed deterministically? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Fri Jul 15 17:54:25 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 15 Jul 2016 17:54:25 +0200 Subject: [erlang-questions] Reactors, Channels and Event Streams for Composable Distributed Programming In-Reply-To: References: Message-ID: On Fri, Jul 15, 2016 at 2:19 AM, Onorio Catenacci wrote: > It's a free PDF but you have to register on the site. Given the audience > here I thought it might present some interest to some of you. It is an interesting paper, which touches of many of the possible generalizations that are possible in Erlang's messaging model. * In Erlang, a receive clause is static and receive patterns are not first class variables. In other words, you cannot *combine* different receive clauses which, as the paper mentions, hurt modularity at this level. On the other hand, it is easier to optimize receive clauses if they are static since they will always be module-local. * The paper curiously omits Concurrent ML as related work. CML subsumes most of the paper and was written back in 1993. In CML there are two abstractions: one, you can combine "receives" by combining the selectors of those receives. Second, the result of a receive are events, which can *also* be combined. It adds another abstraction layer. But the cost of this abstraction is that it will not work in a distributed setting. It is local-only. * The paper makes no attempt at handling multi-party communication with cycles. Assume a client C, a proxy P and a worker W: C messages P P forwards the message to W W responds to C This is obviously faster than factoring every message back through P. And it is a quite common pattern in Erlang systems. These are also protocol patterns which needs to be handled well by protocol typing. * The paper has no mention of error handling in the situation of a combined reactor. We better have a way to identify what part of the reactor failed, and is it possible to let other parts of the combination continue even if other parts failed? That said, it is *always* nice to have new ways of thinking about concurrency. And the authors definitely has a point with the need for a way to "combine" actors algebraically. It would remove lots of boiler-plate code since the systems can often be "layered". I think the reason we don't have the need is that *most* processes can be written as gen-servers, and their receive patterns are pretty straightforward. They want to read every kind of message and forward it to the callback module, unless the process is part of a call. But this is fairly easy to handle with static receive clauses. I'm not saying the idea is bad, but if you can demonstrate a problem in Erlang which would be helped a lot by the construction, then you have a far better case. If, OTOH, every construction has a direct nice alternative in Erlang, then you may want to conclude the abstraction wasn't so useful after all. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Fri Jul 15 18:09:47 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Fri, 15 Jul 2016 09:09:47 -0700 Subject: [erlang-questions] hipe - erllvm: finished, quiet, or other? Message-ID: I've been playing around with Pony (http://www.ponylang.org/) recently, and it stirred me to wonder about the current state of numerical performance directly in erlang. Although there's a convenient safety net in clean and dirty NIFs, it does feel pretty pleasurable to not have to orchestrate the dance of multiple languages and layers in order to go fast. Both HiPE and erllvm seem pretty quiet these days; there's commentary and work in mainline otp suggesting that erllvm is still under some kind of development, but it's been a few years since the main erllvm page has been updated, and I'm having difficulty getting a recent llvm to work (which is not unusual, given that each llvm release involves breaking changes, and they move pretty rapidly). Where's the bleeding edge on this? F. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Fri Jul 15 21:05:42 2016 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 15 Jul 2016 20:05:42 +0100 Subject: [erlang-questions] Deterministic destruction of NIF resources In-Reply-To: References: <0075DF79-88A3-4588-BBEE-81852B50A7CB@gmail.com> Message-ID: Assuming I'm tracing correctly (see below), that's not the cause. I see the 'exit' trace events fairly soon after the process dies, but I don't see the resource destructor being called until later, and usually in a clump: Alloc wrapper resource 0x2b038eb01620 sup <0.2167.0> spawned <0.3013.0> Alloc wrapper resource 0x2b038ec80328 <0.3013.0> exit {timeout,moo} sup <0.2167.0> spawned <0.3049.0> Alloc wrapper resource 0x2b038ec801e0 Free wrapper resource 0x2b038ec80328 Free wrapper resource 0x2b038eb01620 <0.3049.0> exit {timeout,moo} sup <0.2167.0> spawned <0.15256.0> Alloc wrapper resource 0x2b038ea80c70 <0.15256.0> exit {timeout,moo} sup <0.2167.0> spawned <0.17734.0> Alloc wrapper resource 0x2b038eac7de0 <0.17734.0> exit {timeout,moo} sup <0.2167.0> spawned <0.18825.0> Alloc wrapper resource 0x2b038eb48098 <0.18825.0> exit {timeout,moo} sup <0.2167.0> spawned <0.19267.0> Alloc wrapper resource 0x2b038eb00260 <0.19267.0> exit {timeout,moo} sup <0.2167.0> spawned <0.19616.0> Alloc wrapper resource 0x2b038ea80b88 <0.19616.0> exit {timeout,moo} sup <0.2167.0> spawned <0.20068.0> Alloc wrapper resource 0x2b038eac2930 <0.20068.0> exit {timeout,moo} sup <0.2167.0> spawned <0.20424.0> Alloc wrapper resource 0x2b038eac1328 <0.20424.0> exit {timeout,moo} sup <0.2167.0> spawned <0.20649.0> Alloc wrapper resource 0x2b038eac2aa0 <0.20649.0> exit {timeout,moo} sup <0.2167.0> spawned <0.20663.0> Alloc wrapper resource 0x2b038ea84118 Free wrapper resource 0x2b038eac2aa0 Free wrapper resource 0x2b038eac1328 Free wrapper resource 0x2b038eac2930 Free wrapper resource 0x2b038ea80b88 Free wrapper resource 0x2b038eb00260 Free wrapper resource 0x2b038eb48098 Free wrapper resource 0x2b038eac7de0 Free wrapper resource 0x2b038ea80c70 Free wrapper resource 0x2b038ec801e0 <0.20663.0> exit {timeout,moo} sup <0.2167.0> spawned <0.20679.0> Alloc wrapper resource 0x2b038ea80b88 The "... wrapper resource ..." lines are from the NIF; the other lines are from tracing: SupPid = whereis(agents_sup). TFun = fun TFun(State) -> receive {trace, ByPid, spawn, Pid, _} when ByPid =:= SupPid -> io:format("sup ~p spawned ~p~n", [ByPid, Pid]), State2 = [Pid | State], TFun(State2); {trace, Pid, exit, Reason} = M -> case lists:member(Pid, State) of true -> io:format("~p exit ~p~n", [Pid, Reason]); _ -> ok end, TFun(State); Other -> TFun(State) end end. TPid = spawn(fun() -> TFun([]) end). erlang:trace(all, true, [procs, {tracer, TPid}]). On 15 July 2016 at 15:53, Lukas Larsson wrote: > How do you know when the process dies? > > If you aren't already you should use the trace flag 'exiting' > (http://erlang.org/doc/man/erlang.html#trace-3) to figure out when a process > actually has died. ERTS will delay process death after you send it a > termination signal for various reasons. > > On Fri, Jul 15, 2016 at 4:39 PM, Roger Lipscombe > wrote: >> >> Sorry, let me rephrase: they're not garbage collected *when* that >> process dies. The destructor is sometimes called immediately, and >> sometimes at some point in the future. Can I make this deterministic? >> >> And, no, the resource has not been sent to another process or stored in >> ETS. >> >> On 15 July 2016 at 15:10, Sergej Jure?ko wrote: >> > >> > On 15 Jul 2016, at 15:57, Roger Lipscombe >> > wrote: >> > >> > because they're not garbage collected when that >> > process dies. >> > >> > >> > Then there is an outstanding reference to the resource. Has it been sent >> > to >> > another process or stored in an ETS? >> > >> > >> > Sergej >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From Catenacci@REDACTED Fri Jul 15 22:51:18 2016 From: Catenacci@REDACTED (Onorio Catenacci) Date: Fri, 15 Jul 2016 16:51:18 -0400 Subject: [erlang-questions] Reactors, Channels and Event Streams for Composable Distributed Programming In-Reply-To: References: Message-ID: Thanks for taking the time to comment Jesper. Your comments are certainly illuminating for someone like myself who doesn't know the various details of Erlang's messaging model as deeply as you do. On Fri, Jul 15, 2016 at 11:54 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Fri, Jul 15, 2016 at 2:19 AM, Onorio Catenacci > wrote: > >> It's a free PDF but you have to register on the site. Given the audience >> here I thought it might present some interest to some of you. > > > It is an interesting paper, which touches of many of the possible > generalizations that are possible in Erlang's messaging model. > > * In Erlang, a receive clause is static and receive patterns are not first > class variables. In other words, you cannot *combine* different receive > clauses which, as the paper mentions, hurt modularity at this level. On the > other hand, it is easier to optimize receive clauses if they are static > since they will always be module-local. > > * The paper curiously omits Concurrent ML as related work. CML subsumes > most of the paper and was written back in 1993. In CML there are two > abstractions: one, you can combine "receives" by combining the selectors of > those receives. Second, the result of a receive are events, which can > *also* be combined. It adds another abstraction layer. But the cost of this > abstraction is that it will not work in a distributed setting. It is > local-only. > > * The paper makes no attempt at handling multi-party communication with > cycles. Assume a client C, a proxy P and a worker W: > > C messages P > P forwards the message to W > W responds to C > > This is obviously faster than factoring every message back through P. And > it is a quite common pattern in Erlang systems. These are also protocol > patterns which needs to be handled well by protocol typing. > > * The paper has no mention of error handling in the situation of a > combined reactor. We better have a way to identify what part of the reactor > failed, and is it possible to let other parts of the combination continue > even if other parts failed? > > That said, it is *always* nice to have new ways of thinking about > concurrency. And the authors definitely has a point with the need for a way > to "combine" actors algebraically. It would remove lots of boiler-plate > code since the systems can often be "layered". I think the reason we don't > have the need is that *most* processes can be written as gen-servers, and > their receive patterns are pretty straightforward. They want to read every > kind of message and forward it to the callback module, unless the process > is part of a call. But this is fairly easy to handle with static receive > clauses. I'm not saying the idea is bad, but if you can demonstrate a > problem in Erlang which would be helped a lot by the construction, then you > have a far better case. If, OTOH, every construction has a direct nice > alternative in Erlang, then you may want to conclude the abstraction wasn't > so useful after all. > > > > -- > J. > -- Onorio Catenacci http://onor.io http://www.google.com/+OnorioCatenacci -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Fri Jul 15 23:29:09 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Fri, 15 Jul 2016 23:29:09 +0200 Subject: [erlang-questions] Deterministic destruction of NIF resources In-Reply-To: References: <0075DF79-88A3-4588-BBEE-81852B50A7CB@gmail.com> Message-ID: Apparently I wasn't clear enough. You want to look at the 'exiting' trace flag, not the 'procs' flags. i.e. see this code: go() -> > TFun = fun TFun() -> > receive > Other -> > io:format("~p~n",[Other]), > TFun() > end > end, > TPid = spawn(TFun), > erlang:trace(self(), true, [set_on_spawn, procs, exiting, {tracer, > TPid}]), > Pid = spawn(fun loop/0), > timer:sleep(100), > exit(Pid, die). > loop() -> > [ets:new(hej,[]) || _ <- lists:seq(1,100)], > receive after infinity -> ok end. when you run Mod:go() you will get a bunch of printouts: 1> trace:go(). > {trace,<0.33.0>,spawn,<0.36.0>,{erlang,apply,[#Fun,[]]}} > {trace,<0.36.0>,in_exiting,0} > {trace,<0.36.0>,exit,die} > true > {trace,<0.36.0>,out_exiting,0} > {trace,<0.36.0>,in_exiting,0} > {trace,<0.36.0>,out_exiting,0} > {trace,<0.36.0>,in_exiting,0} > {trace,<0.36.0>,out_exiting,0} > {trace,<0.36.0>,in_exiting,0} > {trace,<0.36.0>,out_exiting,0} > {trace,<0.36.0>,in_exiting,0} > {trace,<0.36.0>,out_exiting,0} > {trace,<0.36.0>,in_exiting,0} > {trace,<0.36.0>,out_exited,0} as you can see, the 'exit' trace message is sent relatively early, but the heap of the process is not cleaned up until when the 'out_exited' trace message is sent. In my example the process gets all of these extra schedules after it has decided to die in order to clean up the ets tables that is responsible for. There are a bunch of reasons why this may happen, not just ets table cleanup. What the reason is cannot be observed without instrumenting the vm. On Fri, Jul 15, 2016 at 9:05 PM, Roger Lipscombe wrote: > Assuming I'm tracing correctly (see below), that's not the cause. I > see the 'exit' trace events fairly soon after the process dies, but I > don't see the resource destructor being called until later, and > usually in a clump: > > Alloc wrapper resource 0x2b038eb01620 > sup <0.2167.0> spawned <0.3013.0> > Alloc wrapper resource 0x2b038ec80328 > <0.3013.0> exit {timeout,moo} > sup <0.2167.0> spawned <0.3049.0> > Alloc wrapper resource 0x2b038ec801e0 > Free wrapper resource 0x2b038ec80328 > Free wrapper resource 0x2b038eb01620 > <0.3049.0> exit {timeout,moo} > sup <0.2167.0> spawned <0.15256.0> > Alloc wrapper resource 0x2b038ea80c70 > <0.15256.0> exit {timeout,moo} > sup <0.2167.0> spawned <0.17734.0> > Alloc wrapper resource 0x2b038eac7de0 > <0.17734.0> exit {timeout,moo} > sup <0.2167.0> spawned <0.18825.0> > Alloc wrapper resource 0x2b038eb48098 > <0.18825.0> exit {timeout,moo} > sup <0.2167.0> spawned <0.19267.0> > Alloc wrapper resource 0x2b038eb00260 > <0.19267.0> exit {timeout,moo} > sup <0.2167.0> spawned <0.19616.0> > Alloc wrapper resource 0x2b038ea80b88 > <0.19616.0> exit {timeout,moo} > sup <0.2167.0> spawned <0.20068.0> > Alloc wrapper resource 0x2b038eac2930 > <0.20068.0> exit {timeout,moo} > sup <0.2167.0> spawned <0.20424.0> > Alloc wrapper resource 0x2b038eac1328 > <0.20424.0> exit {timeout,moo} > sup <0.2167.0> spawned <0.20649.0> > Alloc wrapper resource 0x2b038eac2aa0 > <0.20649.0> exit {timeout,moo} > sup <0.2167.0> spawned <0.20663.0> > Alloc wrapper resource 0x2b038ea84118 > Free wrapper resource 0x2b038eac2aa0 > Free wrapper resource 0x2b038eac1328 > Free wrapper resource 0x2b038eac2930 > Free wrapper resource 0x2b038ea80b88 > Free wrapper resource 0x2b038eb00260 > Free wrapper resource 0x2b038eb48098 > Free wrapper resource 0x2b038eac7de0 > Free wrapper resource 0x2b038ea80c70 > Free wrapper resource 0x2b038ec801e0 > <0.20663.0> exit {timeout,moo} > sup <0.2167.0> spawned <0.20679.0> > Alloc wrapper resource 0x2b038ea80b88 > > The "... wrapper resource ..." lines are from the NIF; the other lines > are from tracing: > > SupPid = whereis(agents_sup). > > TFun = fun TFun(State) -> > receive > {trace, ByPid, spawn, Pid, _} when ByPid =:= SupPid -> > io:format("sup ~p spawned ~p~n", [ByPid, Pid]), > State2 = [Pid | State], > TFun(State2); > {trace, Pid, exit, Reason} = M -> > case lists:member(Pid, State) of > true -> > io:format("~p exit ~p~n", [Pid, Reason]); > _ -> > ok > end, > TFun(State); > Other -> > TFun(State) > end > end. > TPid = spawn(fun() -> TFun([]) end). > erlang:trace(all, true, [procs, {tracer, TPid}]). > > > On 15 July 2016 at 15:53, Lukas Larsson wrote: > > How do you know when the process dies? > > > > If you aren't already you should use the trace flag 'exiting' > > (http://erlang.org/doc/man/erlang.html#trace-3) to figure out when a > process > > actually has died. ERTS will delay process death after you send it a > > termination signal for various reasons. > > > > On Fri, Jul 15, 2016 at 4:39 PM, Roger Lipscombe > > > wrote: > >> > >> Sorry, let me rephrase: they're not garbage collected *when* that > >> process dies. The destructor is sometimes called immediately, and > >> sometimes at some point in the future. Can I make this deterministic? > >> > >> And, no, the resource has not been sent to another process or stored in > >> ETS. > >> > >> On 15 July 2016 at 15:10, Sergej Jure?ko > wrote: > >> > > >> > On 15 Jul 2016, at 15:57, Roger Lipscombe > >> > wrote: > >> > > >> > because they're not garbage collected when that > >> > process dies. > >> > > >> > > >> > Then there is an outstanding reference to the resource. Has it been > sent > >> > to > >> > another process or stored in an ETS? > >> > > >> > > >> > Sergej > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Sat Jul 16 01:20:14 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 15 Jul 2016 23:20:14 +0000 Subject: [erlang-questions] dirty nifs ETA? Message-ID: I'm wondering if there is any ETA for having the dirty nifs considered as stable and compiled by default with an Erlang release. What's still on the TODO? Having them as stable would help to ship a release with them. - beno?t -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Sat Jul 16 10:06:02 2016 From: roger@REDACTED (Roger Lipscombe) Date: Sat, 16 Jul 2016 09:06:02 +0100 Subject: [erlang-questions] Deterministic destruction of NIF resources In-Reply-To: References: <0075DF79-88A3-4588-BBEE-81852B50A7CB@gmail.com> Message-ID: My apologies, I've not used trace flags before. Anyway, by tweaking the trace to [exiting, procs, {tracer, TPid}], I *don't* see any 'in_exiting' or 'out_exiting' events; I *do* see 'out_exited' events, but they happen (more-or-less) immediately after the 'exit' events, but the resource destructor is still delayed w.r.t. the 'out_exited' events. Note that I'm still using my slightly dodgy mechanism for filtering the trace to only the processes I'm interested in. I tried to use the 'set_on_spawn' flag, and specify the process to be traced as whereis(my_sup), but I couldn't get that to work. In my testing, the destructor is delayed by a few seconds (when repeatedly -- manually, so not particularly quickly -- restarting one test process). In production, we're seeing the destructor being delayed an arbitrarily long period, which is kinda important, because the destructor is responsible for killing a background thread, and we'd prefer it not to continue after the associated Erlang process has died. As far as I can tell from reading the OTP source, it looks like resources are allocated from the binary heap, rather than any particular process heap, which suggests to me that they'll be collected when the binary heap undergoes a GC. What triggers that? On 15 July 2016 at 22:29, Lukas Larsson wrote: > Apparently I wasn't clear enough. You want to look at the 'exiting' trace > flag, not the 'procs' flags. i.e. see this code: > > >> go() -> >> TFun = fun TFun() -> >> receive >> Other -> >> io:format("~p~n",[Other]), >> TFun() >> end >> end, >> TPid = spawn(TFun), >> erlang:trace(self(), true, [set_on_spawn, procs, exiting, {tracer, >> TPid}]), >> Pid = spawn(fun loop/0), >> timer:sleep(100), >> exit(Pid, die). >> loop() -> >> [ets:new(hej,[]) || _ <- lists:seq(1,100)], >> receive after infinity -> ok end. > > > when you run Mod:go() you will get a bunch of printouts: > >> 1> trace:go(). >> {trace,<0.33.0>,spawn,<0.36.0>,{erlang,apply,[#Fun,[]]}} >> {trace,<0.36.0>,in_exiting,0} >> {trace,<0.36.0>,exit,die} >> true >> {trace,<0.36.0>,out_exiting,0} >> {trace,<0.36.0>,in_exiting,0} >> {trace,<0.36.0>,out_exiting,0} >> {trace,<0.36.0>,in_exiting,0} >> {trace,<0.36.0>,out_exiting,0} >> {trace,<0.36.0>,in_exiting,0} >> {trace,<0.36.0>,out_exiting,0} >> {trace,<0.36.0>,in_exiting,0} >> {trace,<0.36.0>,out_exiting,0} >> {trace,<0.36.0>,in_exiting,0} >> {trace,<0.36.0>,out_exited,0} > > > as you can see, the 'exit' trace message is sent relatively early, but the > heap of the process is not cleaned up until when the 'out_exited' trace > message is sent. In my example the process gets all of these extra schedules > after it has decided to die in order to clean up the ets tables that is > responsible for. There are a bunch of reasons why this may happen, not just > ets table cleanup. What the reason is cannot be observed without > instrumenting the vm. > > On Fri, Jul 15, 2016 at 9:05 PM, Roger Lipscombe > wrote: >> >> Assuming I'm tracing correctly (see below), that's not the cause. I >> see the 'exit' trace events fairly soon after the process dies, but I >> don't see the resource destructor being called until later, and >> usually in a clump: >> >> Alloc wrapper resource 0x2b038eb01620 >> sup <0.2167.0> spawned <0.3013.0> >> Alloc wrapper resource 0x2b038ec80328 >> <0.3013.0> exit {timeout,moo} >> sup <0.2167.0> spawned <0.3049.0> >> Alloc wrapper resource 0x2b038ec801e0 >> Free wrapper resource 0x2b038ec80328 >> Free wrapper resource 0x2b038eb01620 >> <0.3049.0> exit {timeout,moo} >> sup <0.2167.0> spawned <0.15256.0> >> Alloc wrapper resource 0x2b038ea80c70 >> <0.15256.0> exit {timeout,moo} >> sup <0.2167.0> spawned <0.17734.0> >> Alloc wrapper resource 0x2b038eac7de0 >> <0.17734.0> exit {timeout,moo} >> sup <0.2167.0> spawned <0.18825.0> >> Alloc wrapper resource 0x2b038eb48098 >> <0.18825.0> exit {timeout,moo} >> sup <0.2167.0> spawned <0.19267.0> >> Alloc wrapper resource 0x2b038eb00260 >> <0.19267.0> exit {timeout,moo} >> sup <0.2167.0> spawned <0.19616.0> >> Alloc wrapper resource 0x2b038ea80b88 >> <0.19616.0> exit {timeout,moo} >> sup <0.2167.0> spawned <0.20068.0> >> Alloc wrapper resource 0x2b038eac2930 >> <0.20068.0> exit {timeout,moo} >> sup <0.2167.0> spawned <0.20424.0> >> Alloc wrapper resource 0x2b038eac1328 >> <0.20424.0> exit {timeout,moo} >> sup <0.2167.0> spawned <0.20649.0> >> Alloc wrapper resource 0x2b038eac2aa0 >> <0.20649.0> exit {timeout,moo} >> sup <0.2167.0> spawned <0.20663.0> >> Alloc wrapper resource 0x2b038ea84118 >> Free wrapper resource 0x2b038eac2aa0 >> Free wrapper resource 0x2b038eac1328 >> Free wrapper resource 0x2b038eac2930 >> Free wrapper resource 0x2b038ea80b88 >> Free wrapper resource 0x2b038eb00260 >> Free wrapper resource 0x2b038eb48098 >> Free wrapper resource 0x2b038eac7de0 >> Free wrapper resource 0x2b038ea80c70 >> Free wrapper resource 0x2b038ec801e0 >> <0.20663.0> exit {timeout,moo} >> sup <0.2167.0> spawned <0.20679.0> >> Alloc wrapper resource 0x2b038ea80b88 >> >> The "... wrapper resource ..." lines are from the NIF; the other lines >> are from tracing: >> >> SupPid = whereis(agents_sup). >> >> TFun = fun TFun(State) -> >> receive >> {trace, ByPid, spawn, Pid, _} when ByPid =:= SupPid -> >> io:format("sup ~p spawned ~p~n", [ByPid, Pid]), >> State2 = [Pid | State], >> TFun(State2); >> {trace, Pid, exit, Reason} = M -> >> case lists:member(Pid, State) of >> true -> >> io:format("~p exit ~p~n", [Pid, Reason]); >> _ -> >> ok >> end, >> TFun(State); >> Other -> >> TFun(State) >> end >> end. >> TPid = spawn(fun() -> TFun([]) end). >> erlang:trace(all, true, [procs, {tracer, TPid}]). >> >> >> On 15 July 2016 at 15:53, Lukas Larsson wrote: >> > How do you know when the process dies? >> > >> > If you aren't already you should use the trace flag 'exiting' >> > (http://erlang.org/doc/man/erlang.html#trace-3) to figure out when a >> > process >> > actually has died. ERTS will delay process death after you send it a >> > termination signal for various reasons. >> > >> > On Fri, Jul 15, 2016 at 4:39 PM, Roger Lipscombe >> > >> > wrote: >> >> >> >> Sorry, let me rephrase: they're not garbage collected *when* that >> >> process dies. The destructor is sometimes called immediately, and >> >> sometimes at some point in the future. Can I make this deterministic? >> >> >> >> And, no, the resource has not been sent to another process or stored in >> >> ETS. >> >> >> >> On 15 July 2016 at 15:10, Sergej Jure?ko >> >> wrote: >> >> > >> >> > On 15 Jul 2016, at 15:57, Roger Lipscombe >> >> > wrote: >> >> > >> >> > because they're not garbage collected when that >> >> > process dies. >> >> > >> >> > >> >> > Then there is an outstanding reference to the resource. Has it been >> >> > sent >> >> > to >> >> > another process or stored in an ETS? >> >> > >> >> > >> >> > Sergej >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> > > > From mikpelinux@REDACTED Sat Jul 16 16:26:44 2016 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Sat, 16 Jul 2016 16:26:44 +0200 Subject: [erlang-questions] Better way to check if a set of keys exists in a mnesia table? In-Reply-To: <989F10C2-C30F-4462-A25E-CEC617A11950@me.com> References: <989F10C2-C30F-4462-A25E-CEC617A11950@me.com> Message-ID: <22410.17444.386377.755520@gargle.gargle.HOWL> Chaitanya Chalasani writes: > I have a table with an UUID as the primary key / first element of the record. > > What is the efficient way to check if a given set of UUIDs are valid primary key for that table. > > I can think for three different solutions - > use mnesia:all_leys(TableName) and perform lists subset check. However, if the table contains over a million records, fetching all the keys for every check isn?t a nice solution. > use mnesia:read(TableName, Key) and check on the response. However, if the row is a big enough, trying to get the whole row for a simple key check isn?t that good either. > use ets:member(TableName, Key). A better solution than the above but doesn?t work on remote tables. > > Which one of the above is the least bad solution or is there a better one hidden under the documents. all_keys can be horribly expensive and should be avoided if possible, but for small tables it may be acceptable. I'd do one of the following: 1. mnesia:dirty_read(T, K) and check result for [] vs [_|_] Pro: easy, works Con: the data copy may be expensive for large records 2. Make the table an ordered_set; mnesia:dirty_prev(T, mnesia:dirty_next(T, K)) and check if K is returned Pro: avoids the data copy Con: requires an ordered_set, requires code to handle boundary conditions wrt '$end_of_table' 3. Store the keys w/o data in a separate table, then do a dirty_read in that Pro: reduces copying Con: requires more storage, the lookup in the side table won't provide cache hints to help your access in the main table (but that may be Ok if the side table is hit orders of magnitude more often) One could implement some sort of sparse bitmap or range tree and use that to record key presence, but I'm not sure it would be worthwhile in Erlang. From mikpelinux@REDACTED Sat Jul 16 16:36:42 2016 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Sat, 16 Jul 2016 16:36:42 +0200 Subject: [erlang-questions] atomic ets:take_first, ets:take_last In-Reply-To: References: Message-ID: <22410.18042.317054.65935@gargle.gargle.HOWL> Alexandre Beniaminov writes: > Thank you. > You're right, and you've answered on part of my question: > > << Now, anything can happen in between, so in particular, ets:take/2 may > return [] if the key was deleted or taken out by another process between > the two calls.>> > > I would like to ask the community what do you think about to add to ets > module two atomic functions: ets:take_first/1 and ets:take_last/1 ? I would be more interested in a generalized atomic compare_and_ function, where could be to delete the record or to update one of its fields. take_first could then be synthesized on top of first + compare_and_delete. My interest here is in building synchronisation objects without having to call a helper process (i.e. a gen_server or similar) to maintain the state (which is in an ETS table). > > 2016-07-15 1:39 GMT+03:00 Alexandre Beniaminov : > > > Thank you. > > You're right, and you've answered on part of my question: > > > > << Now, anything can happen in between, so in particular, ets:take/2 may > > return [] if the key was deleted or taken out by another process between > > the two calls.>> > > > > I would like to ask the community what do you think about to add to ets > > module two atomic functions: ets:take_first/1 and ets:take_last/1 ? > > > > 2016-07-14 21:19 GMT+03:00 Jesper Louis Andersen < > > jesper.louis.andersen@REDACTED>: > > > >> > >> On Thu, Jul 14, 2016 at 8:06 PM, Alexandre Beniaminov < > >> abeniaminov@REDACTED> wrote: > >> > >>> atomic > >> > >> > >> You have to say what you expect to be atomic/linearizable. In your above > >> code, suppose you are being scheduled out between the call to ets:first/1 > >> and ets:take/2. Now, anything can happen in between, so in particular, > >> ets:take/2 may return [] if the key was deleted or taken out by another > >> process between the two calls. > >> > >> But taken individually, the set of ets:first/1, ets:take/2 and > >> ets:lookup/2 ought to be linearizable. I.e., there is some point in time > >> (the atomic commit-point) at which ets:take/2 removes the objects. After > >> this commit-point first/1 and lookup/2 will not see the Key present in the > >> table anymore. > >> > >> So the question of if it is atomic or not depends on the precise > >> definition of what you ask. > >> > >> > >> -- > >> J. > >> > > > > > > > > -- > > ? ?????????, > > ????????? ?????????? > > > > > > -- > ? ?????????, > ????????? ?????????? > > ---------------------------------------------------------------------- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- From max.lapshin@REDACTED Sat Jul 16 17:55:45 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 16 Jul 2016 18:55:45 +0300 Subject: [erlang-questions] Deterministic destruction of NIF resources In-Reply-To: References: <0075DF79-88A3-4588-BBEE-81852B50A7CB@gmail.com> Message-ID: It is an old question about finalizators in GC languages. If you want determined time of deallocation of resources, then you must do it explicitly and handle deallocated state of your NIF resource later -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sat Jul 16 21:10:44 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 16 Jul 2016 21:10:44 +0200 Subject: [erlang-questions] Reactors, Channels and Event Streams for Composable Distributed Programming In-Reply-To: References: Message-ID: On Fri, Jul 15, 2016 at 10:51 PM, Onorio Catenacci wrote: > Your comments are certainly illuminating for someone like myself who > doesn't know the various details of Erlang's messaging model as deeply as > you do. Another quite interesting concurrency model is STM from Haskell. It also supports composition of atomic operations. That model, at a lower expression level can be found in the "reagents" idea[0], which is probably becoming the basic concurrency model of multi-core OCaml. It is an exiting time to be interested in concurrency for sure. In some way, the Erlang model is the Ur-model of everything. It is the simplest model you can build over an unreliable network at the base. Everything else has to "factor through" the Erlang model, but may provide for a nicer system on top. In other ways it is not the Ur-model. I regard the fact you can't compose receive-expressions as a mistake for instance. It would require us to have receive patterns as first-class. [0] http://kcsrk.info/ocaml/multicore/2016/06/11/lock-free/ -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sat Jul 16 21:18:09 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 16 Jul 2016 21:18:09 +0200 Subject: [erlang-questions] Add a new data structure to erlang VM. In-Reply-To: References: Message-ID: On Fri, Jul 15, 2016 at 1:12 AM, Robert Virding wrote: > The data in an ets table and a process dictionary is not mutable! The memory in C is not mutable! We can construct a C system by having each memory slot be a process supporting two operations get : Loc -> Value set : Loc -> Value -> unit Hence, we can implement the C language without ever mutating things. Rather, we send messages to each memory location. I note that the asynchronicity of the "set" operation is truer to reality than the usual assumption: the cache is not immediately updated and requires coordination / memory-barriers to be safely updated. In other words, the modeling would be truer to a real system than the usual implementation where store updates are assumed to happen at once. Heck, the memory is not even linearizable in the usual sense. The other view is that this is a silly idea and of course ETS is a mutable store. But I don't think so! In reality, what is going on here is that mutability is all about process communication. And the communication is effectful, not the storage contents! -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From David.Dillard@REDACTED Sat Jul 16 16:59:44 2016 From: David.Dillard@REDACTED (David Dillard) Date: Sat, 16 Jul 2016 14:59:44 +0000 Subject: [erlang-questions] Supported Versions/EOL/Lifecycle Message-ID: <7eecd4188c374dde88808ed3dded4f68@TUS3XCHCLUPIN14.community.veritas.com> Hi, I'm trying to determine what versions of Erlang are supported by the community. I've done some searching and I found a few references to "supported versions", but nothing stating what they are (or were) or any kind of policy. For example, a number of open source projects will support the latest branch for both bugs and security issues and the previous branch only for security issues. Can someone point me to this information for Erlang? Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathanfiedler@REDACTED Sat Jul 16 19:13:56 2016 From: nathanfiedler@REDACTED (Nathan Fiedler) Date: Sat, 16 Jul 2016 10:13:56 -0700 Subject: [erlang-questions] Ports and piping output between them Message-ID: My ultimate goal is to run two external processes, with the output of one feeding into the input of the other, a la shell piping. So something like `zfs send | zfs recv` or `tar | split`, but from within my Erlang application. Below is my simple example that fairly quickly balloons memory out of control and eventually explodes (i.e. the OS kills it). #!/usr/bin/env escript %%! main(_Args) -> SendCmd = "cat /dev/random", % % not using 'binary' with open_port makes memory size grow really fast, % but that's fine, I want that option anyway, just pointing it out % SendPort = erlang:open_port({spawn, SendCmd}, [exit_status, binary]), RecvCmd = "strings", RecvPort = erlang:open_port({spawn, RecvCmd}, [exit_status, binary]), {ok, 0} = pipe_until_exit(SendPort, RecvPort, 0), ensure_port_closed(SendPort), ensure_port_closed(RecvPort), ok. pipe_until_exit(SendPort, RecvPort, N) -> % % invoking garbage_collect/0 helps a little bit, but memory still grows % io:format("iteration ~w~n", [N]), receive {SendPort, {exit_status, Status}} -> {ok, Status}; {RecvPort, {exit_status, Status}} -> io:format("error: receive port exited ~w~n", [Status]); {SendPort, {data, Data}} -> true = erlang:port_command(RecvPort, Data), pipe_until_exit(SendPort, RecvPort, N + 1); {RecvPort, {data, _Data}} -> pipe_until_exit(SendPort, RecvPort, N + 1); Msg -> io:format("some other message: ~w", [Msg]) end. ensure_port_closed(Port) -> case erlang:port_info(Port) of undefined -> ok; _ -> erlang:port_close(Port) end. My assumption is that I am doing something wrong, but I can't see it. I'll explore other solutions, such as simply generating a shell script and invoking it. In the mean time, if you can see something obvious, please let me know. Yes, I am aware of os:cmd/1, but that does not return the exit code, which I really want to have so I know that something at least appeared to work (zfs send|zfs recv and tar|split do not generate any success indication other than exit code). Thanks n -------------- next part -------------- An HTML attachment was scrubbed... URL: From cchalasani@REDACTED Sun Jul 17 06:47:02 2016 From: cchalasani@REDACTED (Chaitanya Chalasani) Date: Sun, 17 Jul 2016 10:17:02 +0530 Subject: [erlang-questions] Better way to check if a set of keys exists in a mnesia table? In-Reply-To: <22410.17444.386377.755520@gargle.gargle.HOWL> References: <989F10C2-C30F-4462-A25E-CEC617A11950@me.com> <22410.17444.386377.755520@gargle.gargle.HOWL> Message-ID: <5B56B584-AC16-493D-B6B6-077CBE4337CF@me.com> On 16-Jul-2016, at 19:56, Mikael Pettersson wrote: > all_keys can be horribly expensive and should be avoided if possible, but for small tables it may be acceptable. > > I'd do one of the following: > > 1. mnesia:dirty_read(T, K) and check result for [] vs [_|_] > Pro: easy, works > Con: the data copy may be expensive for large records Yes Indeed. > > 2. Make the table an ordered_set; mnesia:dirty_prev(T, mnesia:dirty_next(T, K)) and check if K is returned > Pro: avoids the data copy > Con: requires an ordered_set, requires code to handle boundary conditions wrt '$end_of_table? Using UUID as primary key, the ordered_set might eventually slow down my writes. > > 3. Store the keys w/o data in a separate table, then do a dirty_read in that > Pro: reduces copying > Con: requires more storage, the lookup in the side table won't provide cache hints to help your access > in the main table (but that may be Ok if the side table is hit orders of magnitude more often) > > One could implement some sort of sparse bitmap or range tree and use that to record key presence, but I'm > not sure it would be worthwhile in Erlang. Yes, I am looking into this possibility as Eric also has suggested the same approach. I can think of using bitmap if it doesn?t complicate the solution beyond the performance again. Also, when I was going through my use case, I figured out the chance of a table being remote is rare enough to make peace with ets:member and tried to implement as shown below - is_key(Tname, Key) -> case catch ets:member(Tname, Key) of {'EXIT', _Reason} -> is_remote_key(Tname, Key); Boolean -> Boolean end. is_remote_key(Tname, Key) -> case mnesia:dirty_read(Tname, Key) of [] -> false; _ -> true end. are_all_keys(Tname, Keys) -> Fun = case mnesia:table_info(Tname, storage_type) of unknown -> fun is_remote_key/2; _ -> fun is_key/2 end, are_all_keys(Tname, Keys, Fun). are_all_keys(_Tname, [], _Fun) -> true; are_all_keys(Tname, [Key|Keys], Fun) -> case Fun(Tname, Key) of false -> false; true -> are_all_keys(Tname, Keys, Fun) end. Below are the latencies when checked with timer:tc. *** Table has a local copy *** 13> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3]]). {11,true} 14> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3,2,1,1,2,3]]). {14,true} 15> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3,2,3,1,2,3,1]]). {14,true} 16> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3,2,3,1,2,3,1]]). {13,true} 17> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3,2,3,1,2,3,1]]). {13,true} 18> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3,2,3,1,2,3,1]]). {14,true} *** Table is remote *** 9> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3]]). {975,true} 10> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3,2,3,1,2,3,1]]). {2151,true} 11> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3,2,3,1,2,3,1]]). {2003,true} 12> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3,2,3,1,2,3,1]]). {1898,true} 13> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3,2,3,1,2,3,1]]). {2027,true} Though I didn?t use UUIDs in my example I think this is optimized enough. Please suggest otherwise. /Chaitanya From garazdawi@REDACTED Sun Jul 17 09:37:50 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Sun, 17 Jul 2016 09:37:50 +0200 Subject: [erlang-questions] Deterministic destruction of NIF resources In-Reply-To: References: <0075DF79-88A3-4588-BBEE-81852B50A7CB@gmail.com> Message-ID: On Sat, Jul 16, 2016 at 10:06 AM, Roger Lipscombe wrote: > > In my testing, the destructor is delayed by a few seconds (when > repeatedly -- manually, so not particularly quickly -- restarting one > test process). In production, we're seeing the destructor being > delayed an arbitrarily long period, which is kinda important, because > the destructor is responsible for killing a background thread, and > we'd prefer it not to continue after the associated Erlang process has > died. > > As far as I can tell from reading the OTP source, it looks like > resources are allocated from the binary heap, rather than any > particular process heap, which suggests to me that they'll be > collected when the binary heap undergoes a GC. What triggers that? > The binary heap never undergoes a gc, there is just the term heap gc. The binaries (which as you say include nif resources) are reference counted and de-allocated when all references are gone. I can't say for sure why your references are being held on for longer, if you want to dig deeper: * the sweep of the binaries is here: https://github.com/erlang/otp/blob/maint/erts/emulator/beam/erl_process.c#L11674 * the deletion of the process is here: https://github.com/erlang/otp/blob/maint/erts/emulator/beam/erl_process.c#L12857 * the calling of the destructor is done by erts_bin_free here: https://github.com/erlang/otp/blob/maint/erts/emulator/beam/erl_message.c#L171 One of the great mantras of ERTS is "Why do it now when it would be cheaper/easier to do it later". So it may well be that something somewhere along the path delays and batches all the operations up. As Max says, in order to be sure when the release will happen. Release all references in the code and manually call the GC. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Sun Jul 17 09:44:21 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Sun, 17 Jul 2016 09:44:21 +0200 Subject: [erlang-questions] Ports and piping output between them In-Reply-To: References: Message-ID: Hello, There is no back-pressure mechanism for open_port, so when you call it on "cat /dev/random" you will get an endless stream of data into the process' message queue which Erlang most likely cannot keep up with and that makes the memory grow until it explodes. In order to get back-pressure you have to put something in between "cat /dev/random" and the erlang port which you processes can talk to and request more bytes instead of it just sending as much as it can. A small bash script which sends X bytes for every Y bytes it receives could do the trick. Lukas On Sat, Jul 16, 2016 at 7:13 PM, Nathan Fiedler wrote: > My ultimate goal is to run two external processes, with the output of one > feeding into the input of the other, a la shell piping. So something like > `zfs send | zfs recv` or `tar | split`, but from within my Erlang > application. Below is my simple example that fairly quickly balloons memory > out of control and eventually explodes (i.e. the OS kills it). > > #!/usr/bin/env escript > %%! > > main(_Args) -> > SendCmd = "cat /dev/random", > % > % not using 'binary' with open_port makes memory size grow really fast, > % but that's fine, I want that option anyway, just pointing it out > % > SendPort = erlang:open_port({spawn, SendCmd}, [exit_status, binary]), > RecvCmd = "strings", > RecvPort = erlang:open_port({spawn, RecvCmd}, [exit_status, binary]), > {ok, 0} = pipe_until_exit(SendPort, RecvPort, 0), > ensure_port_closed(SendPort), > ensure_port_closed(RecvPort), > ok. > > pipe_until_exit(SendPort, RecvPort, N) -> > % > % invoking garbage_collect/0 helps a little bit, but memory still grows > % > io:format("iteration ~w~n", [N]), > receive > {SendPort, {exit_status, Status}} -> > {ok, Status}; > {RecvPort, {exit_status, Status}} -> > io:format("error: receive port exited ~w~n", [Status]); > {SendPort, {data, Data}} -> > true = erlang:port_command(RecvPort, Data), > pipe_until_exit(SendPort, RecvPort, N + 1); > {RecvPort, {data, _Data}} -> > pipe_until_exit(SendPort, RecvPort, N + 1); > Msg -> > io:format("some other message: ~w", [Msg]) > end. > > ensure_port_closed(Port) -> > case erlang:port_info(Port) of > undefined -> ok; > _ -> erlang:port_close(Port) > end. > > My assumption is that I am doing something wrong, but I can't see it. I'll > explore other solutions, such as simply generating a shell script and > invoking it. In the mean time, if you can see something obvious, please let > me know. > > Yes, I am aware of os:cmd/1, but that does not return the exit code, which > I really want to have so I know that something at least appeared to work > (zfs send|zfs recv and tar|split do not generate any success indication > other than exit code). > > Thanks > > n > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sun Jul 17 12:01:40 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 17 Jul 2016 13:01:40 +0300 Subject: [erlang-questions] Deterministic destruction of NIF resources In-Reply-To: References: <0075DF79-88A3-4588-BBEE-81852B50A7CB@gmail.com> Message-ID: I meant not triggering GC, but calling some explicit function like: my_resource:release(NifMagicBinary) that will release everything in C level and convert NifMagicBinary to something with state "deallocated and not ready for using anymore" -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierrefenoll@REDACTED Sun Jul 17 12:15:10 2016 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Sun, 17 Jul 2016 12:15:10 +0200 Subject: [erlang-questions] Ports and piping output between them In-Reply-To: References: Message-ID: <5A05A5D0-2CA4-4200-9D52-CD4317460B37@gmail.com> If your issue has more to do with getting the output error code then why not echo PIPESTATUS? http://stackoverflow.com/a/9168523 > On 16 Jul 2016, at 19:13, Nathan Fiedler wrote: > > My ultimate goal is to run two external processes, with the output of one feeding into the input of the other, a la shell piping. So something like `zfs send | zfs recv` or `tar | split`, but from within my Erlang application. Below is my simple example that fairly quickly balloons memory out of control and eventually explodes (i.e. the OS kills it). > > #!/usr/bin/env escript > %%! > > main(_Args) -> > SendCmd = "cat /dev/random", > % > % not using 'binary' with open_port makes memory size grow really fast, > % but that's fine, I want that option anyway, just pointing it out > % > SendPort = erlang:open_port({spawn, SendCmd}, [exit_status, binary]), > RecvCmd = "strings", > RecvPort = erlang:open_port({spawn, RecvCmd}, [exit_status, binary]), > {ok, 0} = pipe_until_exit(SendPort, RecvPort, 0), > ensure_port_closed(SendPort), > ensure_port_closed(RecvPort), > ok. > > pipe_until_exit(SendPort, RecvPort, N) -> > % > % invoking garbage_collect/0 helps a little bit, but memory still grows > % > io:format("iteration ~w~n", [N]), > receive > {SendPort, {exit_status, Status}} -> > {ok, Status}; > {RecvPort, {exit_status, Status}} -> > io:format("error: receive port exited ~w~n", [Status]); > {SendPort, {data, Data}} -> > true = erlang:port_command(RecvPort, Data), > pipe_until_exit(SendPort, RecvPort, N + 1); > {RecvPort, {data, _Data}} -> > pipe_until_exit(SendPort, RecvPort, N + 1); > Msg -> > io:format("some other message: ~w", [Msg]) > end. > > ensure_port_closed(Port) -> > case erlang:port_info(Port) of > undefined -> ok; > _ -> erlang:port_close(Port) > end. > > My assumption is that I am doing something wrong, but I can't see it. I'll explore other solutions, such as simply generating a shell script and invoking it. In the mean time, if you can see something obvious, please let me know. > > Yes, I am aware of os:cmd/1, but that does not return the exit code, which I really want to have so I know that something at least appeared to work (zfs send|zfs recv and tar|split do not generate any success indication other than exit code). > > Thanks > > n > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From marco.molteni@REDACTED Sun Jul 17 16:43:52 2016 From: marco.molteni@REDACTED (Marco Molteni) Date: Sun, 17 Jul 2016 16:43:52 +0200 Subject: [erlang-questions] is it possible to reuse a "-type" specification for a function in a "-spec" ? Message-ID: <86F62500-C062-42FF-9EC3-EA05616A2389@laposte.net> Hello, say I have a function that takes a function as an argument and I want to write type specifications for them: -type decider() :: fun((pos_integer(), #node{}) -> reject | accept | continue). -spec foo(Decider :: decider()) -> boolean(). foo(Decider) -> ... When implementing the callback, I would like to reuse the type decider(): Instead of duplicating the same type information (error prone, can go out of sync): -spec my_cb(pos_integer(), #node{}) -> reject | accept | continue. my_cb(X, Tree) -> ... I would like to write something like: -spec my_cb :: decider(). my_cb(X, Tree) -> ... Is there a way? thanks! marco From bernard@REDACTED Mon Jul 18 01:56:44 2016 From: bernard@REDACTED (Bernard Duggan) Date: Mon, 18 Jul 2016 09:56:44 +1000 Subject: [erlang-questions] Better way to check if a set of keys exists in a mnesia table? In-Reply-To: <5B56B584-AC16-493D-B6B6-077CBE4337CF@me.com> References: <989F10C2-C30F-4462-A25E-CEC617A11950@me.com> <22410.17444.386377.755520@gargle.gargle.HOWL> <5B56B584-AC16-493D-B6B6-077CBE4337CF@me.com> Message-ID: [Sorry for the duplicate, Chaitanya - I meant to reply to the list] I may be missing something, but what's wrong with mneisa:[dirty_]select/2? mnesia:dirty_select(my_table, [{#my_record{uuid=UUID, _='_'}, [], [true]}]) Pro: Works on any table type, no large row copying, no extra storage, automatically uses index if available. Cons: Only about 3 people on the planet can write matchspecs off the top of their head. will return [true] where the UUID is present, and [] otherwise. No issue with big rows, nor with lots of entries. If you're feeling really adventurous, you can even do all the UUIDs in one call: mnesia:dirty_select(my_table, [{#my_record{uuid=UUID, _='_'}, [], [true]} || UUID <- MyListOfUUIDs]) If the reuslt is [], none were present. If it's equal in length to MyListOfUUIDs then they were all present. B On Sun, Jul 17, 2016 at 2:47 PM, Chaitanya Chalasani wrote: > On 16-Jul-2016, at 19:56, Mikael Pettersson wrote: > > all_keys can be horribly expensive and should be avoided if possible, > but for small tables it may be acceptable. > > > > I'd do one of the following: > > > > 1. mnesia:dirty_read(T, K) and check result for [] vs [_|_] > > Pro: easy, works > > Con: the data copy may be expensive for large records > > Yes Indeed. > > > > > 2. Make the table an ordered_set; mnesia:dirty_prev(T, > mnesia:dirty_next(T, K)) and check if K is returned > > Pro: avoids the data copy > > Con: requires an ordered_set, requires code to handle boundary > conditions wrt '$end_of_table? > > Using UUID as primary key, the ordered_set might eventually slow down my > writes. > > > > > 3. Store the keys w/o data in a separate table, then do a dirty_read in > that > > Pro: reduces copying > > Con: requires more storage, the lookup in the side table won't provide > cache hints to help your access > > in the main table (but that may be Ok if the side table is hit > orders of magnitude more often) > > > > One could implement some sort of sparse bitmap or range tree and use > that to record key presence, but I'm > > not sure it would be worthwhile in Erlang. > > Yes, I am looking into this possibility as Eric also has suggested the > same approach. I can think of using bitmap if it doesn?t complicate the > solution beyond the performance again. > > Also, when I was going through my use case, I figured out the chance of a > table being remote is rare enough to make peace with ets:member and tried > to implement as shown below - > > is_key(Tname, Key) -> > case catch ets:member(Tname, Key) of > {'EXIT', _Reason} -> > is_remote_key(Tname, Key); > Boolean -> Boolean > end. > > is_remote_key(Tname, Key) -> > case mnesia:dirty_read(Tname, Key) of > [] -> false; > _ -> true > end. > > are_all_keys(Tname, Keys) -> > Fun = case mnesia:table_info(Tname, storage_type) of > unknown -> fun is_remote_key/2; > _ -> fun is_key/2 > end, > are_all_keys(Tname, Keys, Fun). > > are_all_keys(_Tname, [], _Fun) -> true; > are_all_keys(Tname, [Key|Keys], Fun) -> > case Fun(Tname, Key) of > false -> false; > true -> are_all_keys(Tname, Keys, Fun) > end. > > Below are the latencies when checked with timer:tc. > > *** Table has a local copy *** > 13> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3]]). > {11,true} > 14> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3,2,1,1,2,3]]). > {14,true} > 15> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3,2,3,1,2,3,1]]). > {14,true} > 16> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3,2,3,1,2,3,1]]). > {13,true} > 17> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3,2,3,1,2,3,1]]). > {13,true} > 18> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3,2,3,1,2,3,1]]). > {14,true} > > *** Table is remote *** > 9> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3]]). > {975,true} > 10> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3,2,3,1,2,3,1]]). > {2151,true} > 11> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3,2,3,1,2,3,1]]). > {2003,true} > 12> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3,2,3,1,2,3,1]]). > {1898,true} > 13> timer:tc(mnesiaKeys, are_all_keys, [test, [1,2,3,2,3,1,2,3,1]]). > {2027,true} > > Though I didn?t use UUIDs in my example I think this is optimized enough. > Please suggest otherwise. > > > /Chaitanya > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathanfiedler@REDACTED Mon Jul 18 01:59:43 2016 From: nathanfiedler@REDACTED (Nathan Fiedler) Date: Sun, 17 Jul 2016 16:59:43 -0700 Subject: [erlang-questions] Ports and piping output between them In-Reply-To: References: Message-ID: Thanks for that explanation, Lukas, that definitely makes sense. I've gone with generating a simple shell script that invokes tar and pipes it to split. n On Sun, Jul 17, 2016 at 12:44 AM, Lukas Larsson wrote: > Hello, > > There is no back-pressure mechanism for open_port, so when you call it on > "cat /dev/random" you will get an endless stream of data into the process' > message queue which Erlang most likely cannot keep up with and that makes > the memory grow until it explodes. In order to get back-pressure you have > to put something in between "cat /dev/random" and the erlang port which you > processes can talk to and request more bytes instead of it just sending as > much as it can. A small bash script which sends X bytes for every Y bytes > it receives could do the trick. > > Lukas > > On Sat, Jul 16, 2016 at 7:13 PM, Nathan Fiedler > wrote: > >> My ultimate goal is to run two external processes, with the output of one >> feeding into the input of the other, a la shell piping. So something like >> `zfs send | zfs recv` or `tar | split`, but from within my Erlang >> application. Below is my simple example that fairly quickly balloons memory >> out of control and eventually explodes (i.e. the OS kills it). >> >> #!/usr/bin/env escript >> %%! >> >> main(_Args) -> >> SendCmd = "cat /dev/random", >> % >> % not using 'binary' with open_port makes memory size grow really >> fast, >> % but that's fine, I want that option anyway, just pointing it out >> % >> SendPort = erlang:open_port({spawn, SendCmd}, [exit_status, binary]), >> RecvCmd = "strings", >> RecvPort = erlang:open_port({spawn, RecvCmd}, [exit_status, binary]), >> {ok, 0} = pipe_until_exit(SendPort, RecvPort, 0), >> ensure_port_closed(SendPort), >> ensure_port_closed(RecvPort), >> ok. >> >> pipe_until_exit(SendPort, RecvPort, N) -> >> % >> % invoking garbage_collect/0 helps a little bit, but memory still >> grows >> % >> io:format("iteration ~w~n", [N]), >> receive >> {SendPort, {exit_status, Status}} -> >> {ok, Status}; >> {RecvPort, {exit_status, Status}} -> >> io:format("error: receive port exited ~w~n", [Status]); >> {SendPort, {data, Data}} -> >> true = erlang:port_command(RecvPort, Data), >> pipe_until_exit(SendPort, RecvPort, N + 1); >> {RecvPort, {data, _Data}} -> >> pipe_until_exit(SendPort, RecvPort, N + 1); >> Msg -> >> io:format("some other message: ~w", [Msg]) >> end. >> >> ensure_port_closed(Port) -> >> case erlang:port_info(Port) of >> undefined -> ok; >> _ -> erlang:port_close(Port) >> end. >> >> My assumption is that I am doing something wrong, but I can't see it. >> I'll explore other solutions, such as simply generating a shell script and >> invoking it. In the mean time, if you can see something obvious, please let >> me know. >> >> Yes, I am aware of os:cmd/1, but that does not return the exit code, >> which I really want to have so I know that something at least appeared to >> work (zfs send|zfs recv and tar|split do not generate any success >> indication other than exit code). >> >> Thanks >> >> n >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathanfiedler@REDACTED Mon Jul 18 02:02:02 2016 From: nathanfiedler@REDACTED (Nathan Fiedler) Date: Sun, 17 Jul 2016 17:02:02 -0700 Subject: [erlang-questions] Ports and piping output between them In-Reply-To: <5A05A5D0-2CA4-4200-9D52-CD4317460B37@gmail.com> References: <5A05A5D0-2CA4-4200-9D52-CD4317460B37@gmail.com> Message-ID: That's awesome, I love it. I've never heard of this, but it's very handy to know. Thanks for pointing it out. I'll use that in the shell script that I generate so I can return that exit code specifically. n On Sun, Jul 17, 2016 at 3:15 AM, Pierre Fenoll wrote: > If your issue has more to do with getting the output error code then why > not echo PIPESTATUS? > > http://stackoverflow.com/a/9168523 > > On 16 Jul 2016, at 19:13, Nathan Fiedler wrote: > > My ultimate goal is to run two external processes, with the output of one > feeding into the input of the other, a la shell piping. So something like > `zfs send | zfs recv` or `tar | split`, but from within my Erlang > application. Below is my simple example that fairly quickly balloons memory > out of control and eventually explodes (i.e. the OS kills it). > > #!/usr/bin/env escript > %%! > > main(_Args) -> > SendCmd = "cat /dev/random", > % > % not using 'binary' with open_port makes memory size grow really fast, > % but that's fine, I want that option anyway, just pointing it out > % > SendPort = erlang:open_port({spawn, SendCmd}, [exit_status, binary]), > RecvCmd = "strings", > RecvPort = erlang:open_port({spawn, RecvCmd}, [exit_status, binary]), > {ok, 0} = pipe_until_exit(SendPort, RecvPort, 0), > ensure_port_closed(SendPort), > ensure_port_closed(RecvPort), > ok. > > pipe_until_exit(SendPort, RecvPort, N) -> > % > % invoking garbage_collect/0 helps a little bit, but memory still grows > % > io:format("iteration ~w~n", [N]), > receive > {SendPort, {exit_status, Status}} -> > {ok, Status}; > {RecvPort, {exit_status, Status}} -> > io:format("error: receive port exited ~w~n", [Status]); > {SendPort, {data, Data}} -> > true = erlang:port_command(RecvPort, Data), > pipe_until_exit(SendPort, RecvPort, N + 1); > {RecvPort, {data, _Data}} -> > pipe_until_exit(SendPort, RecvPort, N + 1); > Msg -> > io:format("some other message: ~w", [Msg]) > end. > > ensure_port_closed(Port) -> > case erlang:port_info(Port) of > undefined -> ok; > _ -> erlang:port_close(Port) > end. > > My assumption is that I am doing something wrong, but I can't see it. I'll > explore other solutions, such as simply generating a shell script and > invoking it. In the mean time, if you can see something obvious, please let > me know. > > Yes, I am aware of os:cmd/1, but that does not return the exit code, which > I really want to have so I know that something at least appeared to work > (zfs send|zfs recv and tar|split do not generate any success indication > other than exit code). > > Thanks > > n > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From grimmer.matthias@REDACTED Mon Jul 18 09:43:12 2016 From: grimmer.matthias@REDACTED (Matthias Grimmer) Date: Mon, 18 Jul 2016 09:43:12 +0200 Subject: [erlang-questions] Call for Papers: 1st Workshop on ReUsable and Modular Programming Language Ecosystems (RUMPLE'16) Message-ID: ============================================================================ Call for Papers: RUMPLE?16 1st Workshop on ReUsable and Modular Programming Language Ecosystems Co-located with SPLASH Oct/Nov, 2016, Amsterdam, Netherlands http://2016.splashcon.org/track/rumple2016 ============================================================================ The RUMPLE workshop is a venue for discussing modular approaches to programming language implementations, extensible virtual machine architectures, as well as reusable runtime components such as dynamic compilers, interpreters, or garbage collectors. The main goal of the workshop is to bring together researchers and practitioners, and facilitate the sharing of experiences and ideas. Relevant topics include, but are definitely not limited to, the following: - Extensible VM design (compiler- or interpreter-based VMs) - Reusable implementation of runtime components (e.g. interpreters, garbage collectors, intermediate representations) - Static and dynamic compiler techniques for different languages - Multi-language runtimes and mechanisms for cross-language interoperability between different languages - Tooling support for different languages (e.g. debugging, profiling, etc.) - Modular language implementations that use existing frameworks and systems - Case studies of existing language implementations, virtual machines, and runtime components (e.g. design choices, tradeoffs, etc.) - New research ideas on how we want to build languages in the future. Workshop Format and Submissions This workshop welcomes the presentation and discussion of new ideas and emerging problems that give a chance for interaction and exchange. We accept presentation proposals in the form of extended abstracts (1-4 pages). Accepted abstracts will be published on the workshop's website before the workshop date. Submissions should use the ACM SIGPLAN Conference Format, 10 point font, using the font family Times New Roman and numeric citation style. All submissions should be in PDF format. Please submit abstracts through http://ssw.jku.at/rumple/ Important Dates - Exended abstract submission: 1 Aug 2016 - Author notification: 5 Sep 2016 All deadlines are Anywhere on Earth (AoE), i.e. GMT/UTC?12:00 hour - Workshop: 31 Oct 2016 Program Committee Walter Binder, University of Lugano Carl Friedrich Bolz, King's College London Richard Jones, University of Kent Stephen Kell, University of Cambridge Jan Vitek, Northeastern University Christian Wimmer, Oracle Labs Workshop Organizers Matthias Grimmer, Johannes Kepler University Linz, Austria Laurence Tratt, King's College London, United Kingdom Adam Welc, Oracle Labs, United States For questions or concerns, please mail to matthias.grimmer at jku.at From donpedrothird@REDACTED Mon Jul 18 10:30:20 2016 From: donpedrothird@REDACTED (John Doe) Date: Mon, 18 Jul 2016 11:30:20 +0300 Subject: [erlang-questions] Deterministic destruction of NIF resources In-Reply-To: References: <0075DF79-88A3-4588-BBEE-81852B50A7CB@gmail.com> Message-ID: They are often held by the repl. If one prints an allocated resource in Erlang console it won't be released. 2016-07-17 10:37 GMT+03:00 Lukas Larsson : > > > On Sat, Jul 16, 2016 at 10:06 AM, Roger Lipscombe > wrote: > >> >> In my testing, the destructor is delayed by a few seconds (when >> repeatedly -- manually, so not particularly quickly -- restarting one >> test process). In production, we're seeing the destructor being >> delayed an arbitrarily long period, which is kinda important, because >> the destructor is responsible for killing a background thread, and >> we'd prefer it not to continue after the associated Erlang process has >> died. >> >> As far as I can tell from reading the OTP source, it looks like >> resources are allocated from the binary heap, rather than any >> particular process heap, which suggests to me that they'll be >> collected when the binary heap undergoes a GC. What triggers that? >> > > The binary heap never undergoes a gc, there is just the term heap gc. The > binaries (which as you say include nif resources) are reference counted and > de-allocated when all references are gone. I can't say for sure why your > references are being held on for longer, if you want to dig deeper: > > * the sweep of the binaries is here: > https://github.com/erlang/otp/blob/maint/erts/emulator/beam/erl_process.c#L11674 > * the deletion of the process is here: > https://github.com/erlang/otp/blob/maint/erts/emulator/beam/erl_process.c#L12857 > * the calling of the destructor is done by erts_bin_free here: > https://github.com/erlang/otp/blob/maint/erts/emulator/beam/erl_message.c#L171 > > One of the great mantras of ERTS is "Why do it now when it would be > cheaper/easier to do it later". So it may well be that something somewhere > along the path delays and batches all the operations up. > > As Max says, in order to be sure when the release will happen. Release all > references in the code and manually call the GC. > > Lukas > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Jul 18 11:38:31 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Mon, 18 Jul 2016 11:38:31 +0200 Subject: [erlang-questions] Deterministic destruction of NIF resources In-Reply-To: References: <0075DF79-88A3-4588-BBEE-81852B50A7CB@gmail.com> Message-ID: <2bb5e581-2fab-9657-c011-229d0767f952@ninenines.eu> Wow, nice tip! Wonder if anything can be done to fix it? Perhaps the repl can check if a variable is/contains a resource and replace it with <<>> for further usage? Can that be done? On 07/18/2016 10:30 AM, John Doe wrote: > They are often held by the repl. If one prints an allocated resource in > Erlang console it won't be released. > > 2016-07-17 10:37 GMT+03:00 Lukas Larsson >: > > > > On Sat, Jul 16, 2016 at 10:06 AM, Roger Lipscombe > > wrote: > > > In my testing, the destructor is delayed by a few seconds (when > repeatedly -- manually, so not particularly quickly -- > restarting one > test process). In production, we're seeing the destructor being > delayed an arbitrarily long period, which is kinda important, > because > the destructor is responsible for killing a background thread, and > we'd prefer it not to continue after the associated Erlang > process has > died. > > As far as I can tell from reading the OTP source, it looks like > resources are allocated from the binary heap, rather than any > particular process heap, which suggests to me that they'll be > collected when the binary heap undergoes a GC. What triggers that? > > > The binary heap never undergoes a gc, there is just the term heap > gc. The binaries (which as you say include nif resources) are > reference counted and de-allocated when all references are gone. I > can't say for sure why your references are being held on for longer, > if you want to dig deeper: > > * the sweep of the binaries is > here: https://github.com/erlang/otp/blob/maint/erts/emulator/beam/erl_process.c#L11674 > * the deletion of the process is > here: https://github.com/erlang/otp/blob/maint/erts/emulator/beam/erl_process.c#L12857 > * the calling of the destructor is done by erts_bin_free here: > https://github.com/erlang/otp/blob/maint/erts/emulator/beam/erl_message.c#L171 > > One of the great mantras of ERTS is "Why do it now when it would be > cheaper/easier to do it later". So it may well be that something > somewhere along the path delays and batches all the operations up. > > As Max says, in order to be sure when the release will happen. > Release all references in the code and manually call the GC. > > Lukas > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From silviu.cpp@REDACTED Mon Jul 18 13:14:45 2016 From: silviu.cpp@REDACTED (Caragea Silviu) Date: Mon, 18 Jul 2016 14:14:45 +0300 Subject: [erlang-questions] ErlNifPid question Message-ID: Hello having two ErlNifPid how I can compare them to know if they refer to the same PID or not ? Other than avoiding calling enif_make_pid like: enif_is_identical(enif_make_pid(env, &pid1), enif_make_pid(env, &pid2)) Silviu -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Mon Jul 18 14:36:58 2016 From: vinoski@REDACTED (Steve Vinoski) Date: Mon, 18 Jul 2016 08:36:58 -0400 Subject: [erlang-questions] ErlNifPid question In-Reply-To: References: Message-ID: On Mon, Jul 18, 2016 at 7:14 AM, Caragea Silviu wrote: > Hello having two ErlNifPid how I can compare them to know if they refer to > the same PID or not ? > > Other than avoiding calling enif_make_pid like: > > enif_is_identical(enif_make_pid(env, &pid1), enif_make_pid(env, &pid2)) > This is the right way to do it. ErlNifPid is a struct with a term member but is documented to be an opaque type, so you're not supposed to access its term directly. But note that enif_make_pid is a C macro that does exactly that, so there's no function call overhead from using it. --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From rickard@REDACTED Mon Jul 18 16:47:56 2016 From: rickard@REDACTED (Rickard Green) Date: Mon, 18 Jul 2016 16:47:56 +0200 Subject: [erlang-questions] dirty nifs ETA? In-Reply-To: References: Message-ID: On Sat, Jul 16, 2016 at 1:20 AM, Benoit Chesneau wrote: > I'm wondering if there is any ETA for having the dirty nifs considered as > stable and compiled by default with an Erlang release. What's still on the > TODO? > > > Having them as stable would help to ship a release with them. > > > - beno?t > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > The plan is currently OTP 20. Cut from : "Currently known issues that are planned to be fixed: * Since purging of a module currently might need to garbage collect a process in order to determine if it has references to the module, a process executing a dirty NIF might delay purging for a very long time. Delaying a purge operation implies delaying all code loading operations which might cause severe problems for the system as a whole." Besides the above, writing more test-cases and test the implementation more extensively is also on the todo-list. Regards, Rickard -- Rickard Green, Erlang/OTP, Ericsson AB From icfp.publicity@REDACTED Tue Jul 19 07:44:03 2016 From: icfp.publicity@REDACTED (Lindsey Kuper) Date: Mon, 18 Jul 2016 22:44:03 -0700 Subject: [erlang-questions] Call for Participation: ICFP 2016 Message-ID: <578dbe234d804_17b03fddc1c65be059813@landin.local.mail> [ Early registration ends 17 August. ] ===================================================================== Call for Participation ICFP 2016 21st ACM SIGPLAN International Conference on Functional Programming and affiliated events September 18 - September 24, 2016 Nara, Japan http://conf.researchr.org/home/icfp-2016 ===================================================================== ICFP provides a forum for researchers and developers to hear about the latest work on the design, implementations, principles, and uses of functional programming. The conference covers the entire spectrum of work, from practice to theory, including its peripheries. A full week dedicated to functional programming: 1 conference, 1 symposium, 10 workshops, tutorials, programming contest results, student research competition, and mentoring workshop * Overview and affiliated events: http://conf.researchr.org/home/icfp-2016 * Program: http://conf.researchr.org/program/icfp-2016/program-icfp-2016 * Accepted Papers: http://conf.researchr.org/track/icfp-2016/icfp-2016-papers#event-overview * Registration is available via: https://regmaster4.com/2016conf/ICFP16/register.php Early registration is due 17 August, 2016. * Programming contest, 5-8 August, 2016: http://2016.icfpcontest.org/ * Student Research Competition (deadline: 3 August, 2016): http://conf.researchr.org/info/icfp-2016/student-research-competition * Follow @icfp_conference on twitter for the latest news: http://twitter.com/icfp_conference There are several events affiliated with ICFP: Sunday, September 18 Workshop on Higher-order Programming with Effects Workshop on Type-Driven Development Scheme and Functional Programming Workshop Programming Languages Mentoring Workshop Monday, September 19 ? Wednesday, September 21 ICFP Thursday, September 22 Haskell Symposium ? Day 1 ML Family Workshop Workshop on Functional High-Performance Computing Commercial Users of Functional Programming ? Day 1 Friday, September 23 Haskell Symposium ? Day 2 OCaml Workshop Erlang Workshop Commercial Users of Functional Programming ? Day 2 Saturday, September 5 Commercial Users of Functional Programming ? Day 3 Haskell Implementors Workshop Functional Art, Music, Modeling and Design Conference Organizers General Co-Chairs: Jacques Garrigue, Nagoya University Gabriele Keller, University of New South Wales Program Chair: Eijiro Sumii, Tohoku University Local Arrangements Co-Chairs: Shinya Katsumata, Kyoto University Susumu Nishimura, Kyoto University Industrial Relations Chair: Rian Trinkle, Obsidian Systems LLC Workshop Co-Chairs: Nicolas Wu, University of Bristol Andres Loeh, Well-Typed LLP Programming Contest Chair: Keisuke Nakano, The University of Electro-Communications Student Research Competition Chair: David Van Horn, University of Maryland, College Park Mentoring Workshop Co-Chairs: Amal Ahmed, Northeastern University Robby Findler, Northwestern University Atsushi Igarashi, Kyoto Universty Publicity Chair: Lindsey Kuper, Intel Labs Video Chair: Iavor Diatchki, Galois Jose Calderon, Galois Student Volunteer Co-Chairs: Yosuke Fukuda, Kyoto University Yuki Nishida, Kyoto University Gabriel Scherer, INRIA Industrial partners: Platinum partners Jane Street Capital Ahrefs Gold partners Mozilla Research Silver partners Ambiata Tsuru Capital Bronze partners Awake Networks Microsoft Research ===================================================================== From mkbucc@REDACTED Wed Jul 20 03:16:47 2016 From: mkbucc@REDACTED (Mark Bucciarelli) Date: Tue, 19 Jul 2016 21:16:47 -0400 Subject: [erlang-questions] erlang/otp: running tests Message-ID: Hi, While fixing a small bug in erl_tidy, I followed the TESTING.md instructions to use the ts module. However, when I followed the instructions, ts:run(syntax_tools, syntax_tools_SUITE, [batch]) I found it did not notice (or recompile) changes to the test file or erl_tidy.erl. Initially, I stuck with the ts:run() approach, and hacked a makefile specific to my one test. Eventually, I ended up using ct_run in lib/syntax_tools, as that picks up changes when it runs the tests. How do the erlang developers work when writing tests and working on small pieces of the code base and flipping between writing tests and writing code? Thanks, Mark -- Blogging at markbucciarelli.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger@REDACTED Wed Jul 20 16:07:25 2016 From: holger@REDACTED (Holger =?iso-8859-1?Q?Wei=DF?=) Date: Wed, 20 Jul 2016 16:07:25 +0200 Subject: [erlang-questions] mnesia_ext +leveldb In-Reply-To: References: Message-ID: <20160720140725.GA609037@zedat.fu-berlin.de> * Matthias Rieber [2016-06-23 08:20]: > On Tue, 23 Feb 2016, Richard Carlsson wrote: > > Gah. Sorry, everybody, it's just been busy times here. Will try to start > > pulling that string again. Thanks for the reminder. > > since Erlang 19 has been released with mnesia_ext I'd like to ask if there > are any news on this topic. Me too! :-) > > 2016-02-23 13:38 GMT+01:00 Matthias Rieber : > > > is leveldb for mnesia_ext already open-sourced? I haven't found it on > > > https://github.com/klarna From marco.molteni@REDACTED Wed Jul 20 21:21:38 2016 From: marco.molteni@REDACTED (Marco Molteni) Date: Wed, 20 Jul 2016 21:21:38 +0200 Subject: [erlang-questions] is it possible to reuse a "-type" specification for a function in a "-spec" ? Message-ID: <1A2D1401-1310-484A-8329-E6294000834C@laposte.net> Hello, say I have a function that takes a function as an argument and I want to write type specifications for them: -type decider() :: fun((pos_integer(), #node{}) -> reject | accept | continue). -spec foo(Decider :: decider()) -> boolean(). foo(Decider) -> ... When implementing the callback, I would like to reuse the type decider(): Instead of duplicating the same type information (error prone, can go out of sync): -spec my_cb(pos_integer(), #node{}) -> reject | accept | continue. my_cb(X, Tree) -> ... I would like to write something like: -spec my_cb :: decider(). my_cb(X, Tree) -> ... Is there a way? thanks! marco From semmitmondo@REDACTED Wed Jul 20 22:08:09 2016 From: semmitmondo@REDACTED (semmit mondo) Date: Wed, 20 Jul 2016 22:08:09 +0200 (CEST) Subject: [erlang-questions] application state Message-ID: Hi, What's the benefit of having application state returned in start(Type, Args) that is passed down to stop(State) in an OTP application module? How is this useful? From jesper.louis.andersen@REDACTED Thu Jul 21 00:11:35 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 21 Jul 2016 00:11:35 +0200 Subject: [erlang-questions] application state In-Reply-To: References: Message-ID: On Wed, Jul 20, 2016 at 10:08 PM, semmit mondo wrote: > What's the benefit of having application state returned in start(Type, > Args) that is passed down to stop(State) in an OTP application module? How > is this useful? Suppose you have some state which is dynamically generated when you start the application, a Pid or Ref say. Since the data is dynamic, there is no way you can "guess" it in the stop/1 function, so it must be passed along. It is not that often I use this feature, but when I do, it is really nice to have. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Thu Jul 21 18:10:24 2016 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Thu, 21 Jul 2016 18:10:24 +0200 Subject: [erlang-questions] mnesia_ext +leveldb In-Reply-To: <20160720140725.GA609037@zedat.fu-berlin.de> References: <20160720140725.GA609037@zedat.fu-berlin.de> Message-ID: <22416.62448.440973.468555@gargle.gargle.HOWL> Holger Wei? writes: > * Matthias Rieber [2016-06-23 08:20]: > > On Tue, 23 Feb 2016, Richard Carlsson wrote: > > > Gah. Sorry, everybody, it's just been busy times here. Will try to start > > > pulling that string again. Thanks for the reminder. > > > > since Erlang 19 has been released with mnesia_ext I'd like to ask if there > > are any news on this topic. > > Me too! :-) I hope to have it out by mid or late August. > > > 2016-02-23 13:38 GMT+01:00 Matthias Rieber : > > > > is leveldb for mnesia_ext already open-sourced? I haven't found it on > > > > https://github.com/klarna > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- From sashang@REDACTED Fri Jul 22 03:42:51 2016 From: sashang@REDACTED (Sashan Govender) Date: Fri, 22 Jul 2016 01:42:51 +0000 Subject: [erlang-questions] comparison with opensaf Message-ID: Hi I'm interested if anyone has done a comparison the Erlang OTP and OpenSAF. It seems to me there is a significant amount of overlap in the but I don't have the expertise in both systems to form a comprehensive picture. As far as I can tell mnesia is like OpenSAF IMM. They can both be used as configuration databases for a cluster. They both are strongly consistent. I know IMM certainly favours consistency over availabilty. OTP is obviously in Erlang while OpenSAF is in C/C++. Any comparisons out there about these two systems? Or am I wildly off the mark thinking that they are similar? -------------- next part -------------- An HTML attachment was scrubbed... URL: From Eric.desCourtis@REDACTED Fri Jul 22 16:58:30 2016 From: Eric.desCourtis@REDACTED (Eric des Courtis) Date: Fri, 22 Jul 2016 10:58:30 -0400 Subject: [erlang-questions] comparison with opensaf In-Reply-To: References: Message-ID: This is a very hard question to answer because I would say OpenSAF is rather obscure. But I can tell you that while there are some overlaps between the two technologies particularly when it comes design patterns. The fact that Erlang is done entirely from the ground up (language, runtime, otp patterns etc...) specifically for high availability means that you should in theory experience much less friction when designing this sort of system in Erlang. My feeling from looking at the documentation is that this isn't well documented. In short I wouldn't consider is competition to Erlang in it's current state for any project. On Thu, Jul 21, 2016 at 9:42 PM, Sashan Govender wrote: > Hi > > I'm interested if anyone has done a comparison the Erlang OTP and OpenSAF. > It seems to me there is a significant amount of overlap in the but I don't > have the expertise in both systems to form a comprehensive picture. As far > as I can tell mnesia is like OpenSAF IMM. They can both be used as > configuration databases for a cluster. They both are strongly consistent. I > know IMM certainly favours consistency over availabilty. OTP is obviously > in Erlang while OpenSAF is in C/C++. Any comparisons out there about these > two systems? Or am I wildly off the mark thinking that they are similar? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jflatow@REDACTED Fri Jul 22 21:37:14 2016 From: jflatow@REDACTED (Jared Flatow) Date: Fri, 22 Jul 2016 12:37:14 -0700 Subject: [erlang-questions] JFDB / Erlang Database Message-ID: Hi all, A few months ago, I was once again thinking about some of the problems I hit with every new Erlang project. Here?s how it usually goes: - I want to store data from Erlang, persisted on disk - I want to be able to search the data using arbitrary indices - DETS can work, but: - there?s the size limit - you have to do a lot of work to cache in memory, and even then it?s all or nothing - the keys can often take up a lot of (redundant) space, especially for ?nested? data - it doesn?t support any kind of range searches - writing the data and then the indices is not atomic, so you have to do a lot of work to make sure it?s always consistent - KV stores have the same problem about managing updating of data / indices - Relational databases do not let you store / index complex Erlang terms easily - Sophisticated searching (i.e. logical combinations of keys) is almost non-existent - Nothing *just works*, persisting and querying data from Erlang requires a ton of thought I decided (finally) to do something about it - my solution is called JFDB (https://github.com/jflatow/jfdb ). I?ve been using it for a few months with great success. I thought others here might find it useful, especially with all the recent discussion about new mnesia backends and other Erlang data structures. The implementation is pretty wild - I don?t really know of anything similar - I think the best way to describe it is as a log-structured merge trie. If you try it, I apologize in advance for a lack of documentation, tests, and the inevitable bugs you will hit. I think it?s in pretty good shape now, there are no known bugs, but it of course comes with a big disclaimer about being new and experimental. I?m actively working to improve all these things. If you have any feedback, I would love to hear it! Best, jared -------------- next part -------------- An HTML attachment was scrubbed... URL: From lheadley@REDACTED Fri Jul 22 23:59:14 2016 From: lheadley@REDACTED (Lyn Headley) Date: Fri, 22 Jul 2016 14:59:14 -0700 Subject: [erlang-questions] JFDB / Erlang Database In-Reply-To: References: Message-ID: This looks very nice to me, and I'm excited to see where it goes. I've also had to think a lot lately about the persistence solutions available to erlang programmers, and still feel like I haven't fallen in love yet. Your list of issues rings quite true to me. My current project rolls its own (non-atomic) thing that I'm hoping will be good enough, but deep down I can see the problems with it. Looking forward to seeing this project grow! On Fri, Jul 22, 2016 at 12:37 PM, Jared Flatow wrote: > Hi all, > > A few months ago, I was once again thinking about some of the problems I hit > with every new Erlang project. Here?s how it usually goes: > > - I want to store data from Erlang, persisted on disk > - I want to be able to search the data using arbitrary indices > - DETS can work, but: > - there?s the size limit > - you have to do a lot of work to cache in memory, and even then it?s all > or nothing > - the keys can often take up a lot of (redundant) space, especially for > ?nested? data > - it doesn?t support any kind of range searches > - writing the data and then the indices is not atomic, so you have to do > a lot of work to make sure it?s always consistent > - KV stores have the same problem about managing updating of data / indices > - Relational databases do not let you store / index complex Erlang terms > easily > - Sophisticated searching (i.e. logical combinations of keys) is almost > non-existent > - Nothing *just works*, persisting and querying data from Erlang requires a > ton of thought > > I decided (finally) to do something about it - my solution is called JFDB > (https://github.com/jflatow/jfdb). I?ve been using it for a few months with > great success. I thought others here might find it useful, especially with > all the recent discussion about new mnesia backends and other Erlang data > structures. The implementation is pretty wild - I don?t really know of > anything similar - I think the best way to describe it is as a > log-structured merge trie. > > If you try it, I apologize in advance for a lack of documentation, tests, > and the inevitable bugs you will hit. I think it?s in pretty good shape now, > there are no known bugs, but it of course comes with a big disclaimer about > being new and experimental. I?m actively working to improve all these > things. If you have any feedback, I would love to hear it! > > Best, > jared > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From lheadley@REDACTED Sat Jul 23 01:41:16 2016 From: lheadley@REDACTED (Lyn Headley) Date: Fri, 22 Jul 2016 16:41:16 -0700 Subject: [erlang-questions] JFDB / Erlang Database In-Reply-To: References: Message-ID: After reading this a little more: Can you give an example of a range query? Can you describe the fault-tolerance characteristics? On Fri, Jul 22, 2016 at 2:59 PM, Lyn Headley wrote: > This looks very nice to me, and I'm excited to see where it goes. I've > also had to think a lot lately about the persistence solutions > available to erlang programmers, and still feel like I haven't fallen > in love yet. Your list of issues rings quite true to me. My current > project rolls its own (non-atomic) thing that I'm hoping will be good > enough, but deep down I can see the problems with it. > > Looking forward to seeing this project grow! > > > On Fri, Jul 22, 2016 at 12:37 PM, Jared Flatow wrote: >> Hi all, >> >> A few months ago, I was once again thinking about some of the problems I hit >> with every new Erlang project. Here?s how it usually goes: >> >> - I want to store data from Erlang, persisted on disk >> - I want to be able to search the data using arbitrary indices >> - DETS can work, but: >> - there?s the size limit >> - you have to do a lot of work to cache in memory, and even then it?s all >> or nothing >> - the keys can often take up a lot of (redundant) space, especially for >> ?nested? data >> - it doesn?t support any kind of range searches >> - writing the data and then the indices is not atomic, so you have to do >> a lot of work to make sure it?s always consistent >> - KV stores have the same problem about managing updating of data / indices >> - Relational databases do not let you store / index complex Erlang terms >> easily >> - Sophisticated searching (i.e. logical combinations of keys) is almost >> non-existent >> - Nothing *just works*, persisting and querying data from Erlang requires a >> ton of thought >> >> I decided (finally) to do something about it - my solution is called JFDB >> (https://github.com/jflatow/jfdb). I?ve been using it for a few months with >> great success. I thought others here might find it useful, especially with >> all the recent discussion about new mnesia backends and other Erlang data >> structures. The implementation is pretty wild - I don?t really know of >> anything similar - I think the best way to describe it is as a >> log-structured merge trie. >> >> If you try it, I apologize in advance for a lack of documentation, tests, >> and the inevitable bugs you will hit. I think it?s in pretty good shape now, >> there are no known bugs, but it of course comes with a big disclaimer about >> being new and experimental. I?m actively working to improve all these >> things. If you have any feedback, I would love to hear it! >> >> Best, >> jared >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From lheadley@REDACTED Sat Jul 23 01:59:56 2016 From: lheadley@REDACTED (Lyn Headley) Date: Fri, 22 Jul 2016 16:59:56 -0700 Subject: [erlang-questions] JFDB / Erlang Database In-Reply-To: <99BDE1E8-D37F-4BBB-B8B0-79265A0C3C00@gmail.com> References: <99BDE1E8-D37F-4BBB-B8B0-79265A0C3C00@gmail.com> Message-ID: Can you describe how you would approach backup/restore? Also, concurrency. Can you support concurrent reads? Concurrent writes? On Fri, Jul 22, 2016 at 4:51 PM, Jared Flatow wrote: > Thanks for your enthusiasm :) > > As far as range queries - these are currently only exposed through the Erlang API via the prefix queries. However, an arbitrary {StartKey, StopKey} could easily be added to the jfdb:fold/3 function if needed. I just haven?t really needed anything beyond prefixes yet. > > As far as fault tolerance, the DB is designed to be incorruptible no matter if/when it crashes. Changes are appended to the keys file, until compaction, at which point a new keys file with a single level is created, and then atomically replaces the old one. In general if you don?t flush, it?s possible changes have not hit the disk yet. Whenever you want to be sure that something hits disk, you can flush. Space in the vals file is never reused until one commit (to disk) after the deallocation occurs in the keys file. > >> On Jul 22, 2016, at 4:41 PM, Lyn Headley wrote: >> >> After reading this a little more: >> >> Can you give an example of a range query? >> Can you describe the fault-tolerance characteristics? >> >> On Fri, Jul 22, 2016 at 2:59 PM, Lyn Headley wrote: >>> This looks very nice to me, and I'm excited to see where it goes. I've >>> also had to think a lot lately about the persistence solutions >>> available to erlang programmers, and still feel like I haven't fallen >>> in love yet. Your list of issues rings quite true to me. My current >>> project rolls its own (non-atomic) thing that I'm hoping will be good >>> enough, but deep down I can see the problems with it. >>> >>> Looking forward to seeing this project grow! >>> >>> >>> On Fri, Jul 22, 2016 at 12:37 PM, Jared Flatow wrote: >>>> Hi all, >>>> >>>> A few months ago, I was once again thinking about some of the problems I hit >>>> with every new Erlang project. Here?s how it usually goes: >>>> >>>> - I want to store data from Erlang, persisted on disk >>>> - I want to be able to search the data using arbitrary indices >>>> - DETS can work, but: >>>> - there?s the size limit >>>> - you have to do a lot of work to cache in memory, and even then it?s all >>>> or nothing >>>> - the keys can often take up a lot of (redundant) space, especially for >>>> ?nested? data >>>> - it doesn?t support any kind of range searches >>>> - writing the data and then the indices is not atomic, so you have to do >>>> a lot of work to make sure it?s always consistent >>>> - KV stores have the same problem about managing updating of data / indices >>>> - Relational databases do not let you store / index complex Erlang terms >>>> easily >>>> - Sophisticated searching (i.e. logical combinations of keys) is almost >>>> non-existent >>>> - Nothing *just works*, persisting and querying data from Erlang requires a >>>> ton of thought >>>> >>>> I decided (finally) to do something about it - my solution is called JFDB >>>> (https://github.com/jflatow/jfdb). I?ve been using it for a few months with >>>> great success. I thought others here might find it useful, especially with >>>> all the recent discussion about new mnesia backends and other Erlang data >>>> structures. The implementation is pretty wild - I don?t really know of >>>> anything similar - I think the best way to describe it is as a >>>> log-structured merge trie. >>>> >>>> If you try it, I apologize in advance for a lack of documentation, tests, >>>> and the inevitable bugs you will hit. I think it?s in pretty good shape now, >>>> there are no known bugs, but it of course comes with a big disclaimer about >>>> being new and experimental. I?m actively working to improve all these >>>> things. If you have any feedback, I would love to hear it! >>>> >>>> Best, >>>> jared >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> > From jflatow@REDACTED Sat Jul 23 01:51:18 2016 From: jflatow@REDACTED (Jared Flatow) Date: Fri, 22 Jul 2016 16:51:18 -0700 Subject: [erlang-questions] JFDB / Erlang Database In-Reply-To: References: Message-ID: <99BDE1E8-D37F-4BBB-B8B0-79265A0C3C00@gmail.com> Thanks for your enthusiasm :) As far as range queries - these are currently only exposed through the Erlang API via the prefix queries. However, an arbitrary {StartKey, StopKey} could easily be added to the jfdb:fold/3 function if needed. I just haven?t really needed anything beyond prefixes yet. As far as fault tolerance, the DB is designed to be incorruptible no matter if/when it crashes. Changes are appended to the keys file, until compaction, at which point a new keys file with a single level is created, and then atomically replaces the old one. In general if you don?t flush, it?s possible changes have not hit the disk yet. Whenever you want to be sure that something hits disk, you can flush. Space in the vals file is never reused until one commit (to disk) after the deallocation occurs in the keys file. > On Jul 22, 2016, at 4:41 PM, Lyn Headley wrote: > > After reading this a little more: > > Can you give an example of a range query? > Can you describe the fault-tolerance characteristics? > > On Fri, Jul 22, 2016 at 2:59 PM, Lyn Headley wrote: >> This looks very nice to me, and I'm excited to see where it goes. I've >> also had to think a lot lately about the persistence solutions >> available to erlang programmers, and still feel like I haven't fallen >> in love yet. Your list of issues rings quite true to me. My current >> project rolls its own (non-atomic) thing that I'm hoping will be good >> enough, but deep down I can see the problems with it. >> >> Looking forward to seeing this project grow! >> >> >> On Fri, Jul 22, 2016 at 12:37 PM, Jared Flatow wrote: >>> Hi all, >>> >>> A few months ago, I was once again thinking about some of the problems I hit >>> with every new Erlang project. Here?s how it usually goes: >>> >>> - I want to store data from Erlang, persisted on disk >>> - I want to be able to search the data using arbitrary indices >>> - DETS can work, but: >>> - there?s the size limit >>> - you have to do a lot of work to cache in memory, and even then it?s all >>> or nothing >>> - the keys can often take up a lot of (redundant) space, especially for >>> ?nested? data >>> - it doesn?t support any kind of range searches >>> - writing the data and then the indices is not atomic, so you have to do >>> a lot of work to make sure it?s always consistent >>> - KV stores have the same problem about managing updating of data / indices >>> - Relational databases do not let you store / index complex Erlang terms >>> easily >>> - Sophisticated searching (i.e. logical combinations of keys) is almost >>> non-existent >>> - Nothing *just works*, persisting and querying data from Erlang requires a >>> ton of thought >>> >>> I decided (finally) to do something about it - my solution is called JFDB >>> (https://github.com/jflatow/jfdb). I?ve been using it for a few months with >>> great success. I thought others here might find it useful, especially with >>> all the recent discussion about new mnesia backends and other Erlang data >>> structures. The implementation is pretty wild - I don?t really know of >>> anything similar - I think the best way to describe it is as a >>> log-structured merge trie. >>> >>> If you try it, I apologize in advance for a lack of documentation, tests, >>> and the inevitable bugs you will hit. I think it?s in pretty good shape now, >>> there are no known bugs, but it of course comes with a big disclaimer about >>> being new and experimental. I?m actively working to improve all these >>> things. If you have any feedback, I would love to hear it! >>> >>> Best, >>> jared >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> From jflatow@REDACTED Sat Jul 23 02:09:34 2016 From: jflatow@REDACTED (Jared Flatow) Date: Fri, 22 Jul 2016 17:09:34 -0700 Subject: [erlang-questions] JFDB / Erlang Database In-Reply-To: References: <99BDE1E8-D37F-4BBB-B8B0-79265A0C3C00@gmail.com> Message-ID: <54CD0468-6101-4FF1-817B-5EF2BAAF642A@gmail.com> > On Jul 22, 2016, at 4:59 PM, Lyn Headley wrote: > > Can you describe how you would approach backup/restore? You can simply backup / restore the keys / vals files. They cannot really exist in a corrupted state (theoretically, of course that needs to be verified). > Also, concurrency. Can you support concurrent reads? Concurrent writes? No a single db cannot be used concurrently, although in Erlang the resource has its own queue/thread so everything is serialized to it anyway. They are pretty lightweight, so I would just use multiple dbs. If you want to modify values, as in jfdb:modify/3 function which does a lookup then an assign, you should have a single Erlang process owning the resource. This can be improved if NIFs are ever able to call Erlang functions. > On Fri, Jul 22, 2016 at 4:51 PM, Jared Flatow wrote: >> Thanks for your enthusiasm :) >> >> As far as range queries - these are currently only exposed through the Erlang API via the prefix queries. However, an arbitrary {StartKey, StopKey} could easily be added to the jfdb:fold/3 function if needed. I just haven?t really needed anything beyond prefixes yet. >> >> As far as fault tolerance, the DB is designed to be incorruptible no matter if/when it crashes. Changes are appended to the keys file, until compaction, at which point a new keys file with a single level is created, and then atomically replaces the old one. In general if you don?t flush, it?s possible changes have not hit the disk yet. Whenever you want to be sure that something hits disk, you can flush. Space in the vals file is never reused until one commit (to disk) after the deallocation occurs in the keys file. >> >>> On Jul 22, 2016, at 4:41 PM, Lyn Headley wrote: >>> >>> After reading this a little more: >>> >>> Can you give an example of a range query? >>> Can you describe the fault-tolerance characteristics? >>> >>> On Fri, Jul 22, 2016 at 2:59 PM, Lyn Headley wrote: >>>> This looks very nice to me, and I'm excited to see where it goes. I've >>>> also had to think a lot lately about the persistence solutions >>>> available to erlang programmers, and still feel like I haven't fallen >>>> in love yet. Your list of issues rings quite true to me. My current >>>> project rolls its own (non-atomic) thing that I'm hoping will be good >>>> enough, but deep down I can see the problems with it. >>>> >>>> Looking forward to seeing this project grow! >>>> >>>> >>>> On Fri, Jul 22, 2016 at 12:37 PM, Jared Flatow wrote: >>>>> Hi all, >>>>> >>>>> A few months ago, I was once again thinking about some of the problems I hit >>>>> with every new Erlang project. Here?s how it usually goes: >>>>> >>>>> - I want to store data from Erlang, persisted on disk >>>>> - I want to be able to search the data using arbitrary indices >>>>> - DETS can work, but: >>>>> - there?s the size limit >>>>> - you have to do a lot of work to cache in memory, and even then it?s all >>>>> or nothing >>>>> - the keys can often take up a lot of (redundant) space, especially for >>>>> ?nested? data >>>>> - it doesn?t support any kind of range searches >>>>> - writing the data and then the indices is not atomic, so you have to do >>>>> a lot of work to make sure it?s always consistent >>>>> - KV stores have the same problem about managing updating of data / indices >>>>> - Relational databases do not let you store / index complex Erlang terms >>>>> easily >>>>> - Sophisticated searching (i.e. logical combinations of keys) is almost >>>>> non-existent >>>>> - Nothing *just works*, persisting and querying data from Erlang requires a >>>>> ton of thought >>>>> >>>>> I decided (finally) to do something about it - my solution is called JFDB >>>>> (https://github.com/jflatow/jfdb). I?ve been using it for a few months with >>>>> great success. I thought others here might find it useful, especially with >>>>> all the recent discussion about new mnesia backends and other Erlang data >>>>> structures. The implementation is pretty wild - I don?t really know of >>>>> anything similar - I think the best way to describe it is as a >>>>> log-structured merge trie. >>>>> >>>>> If you try it, I apologize in advance for a lack of documentation, tests, >>>>> and the inevitable bugs you will hit. I think it?s in pretty good shape now, >>>>> there are no known bugs, but it of course comes with a big disclaimer about >>>>> being new and experimental. I?m actively working to improve all these >>>>> things. If you have any feedback, I would love to hear it! >>>>> >>>>> Best, >>>>> jared >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >> From sashang@REDACTED Sat Jul 23 02:46:25 2016 From: sashang@REDACTED (Sashan Govender) Date: Sat, 23 Jul 2016 10:46:25 +1000 Subject: [erlang-questions] comparison with opensaf In-Reply-To: References: Message-ID: Not sure which documentation set you saw but the one over here looks quite detailed. The AMF documentation alone is 500 pages. http://devel.opensaf.org/SAI-AIS-AMF-B.04.01.AL.pdf AMF as far as I can tell is like the OTP supervisor process. In terms of industry applications I know it's used in Ericsson's DSC (Diameter Signalling Controller). On Sat, Jul 23, 2016 at 12:58 AM, Eric des Courtis wrote: > This is a very hard question to answer because I would say OpenSAF is rather > obscure. But I can tell you that while there are some overlaps between the > two technologies particularly when it comes design patterns. The fact that > Erlang is done entirely from the ground up (language, runtime, otp patterns > etc...) specifically for high availability means that you should in theory > experience much less friction when designing this sort of system in Erlang. > > My feeling from looking at the documentation is that this isn't well > documented. In short I wouldn't consider is competition to Erlang in it's > current state for any project. > > On Thu, Jul 21, 2016 at 9:42 PM, Sashan Govender wrote: >> >> Hi >> >> I'm interested if anyone has done a comparison the Erlang OTP and OpenSAF. >> It seems to me there is a significant amount of overlap in the but I don't >> have the expertise in both systems to form a comprehensive picture. As far >> as I can tell mnesia is like OpenSAF IMM. They can both be used as >> configuration databases for a cluster. They both are strongly consistent. I >> know IMM certainly favours consistency over availabilty. OTP is obviously in >> Erlang while OpenSAF is in C/C++. Any comparisons out there about these two >> systems? Or am I wildly off the mark thinking that they are similar? >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > From g.santomaggio@REDACTED Sat Jul 23 18:29:06 2016 From: g.santomaggio@REDACTED (Gabriele Santomaggio) Date: Sat, 23 Jul 2016 18:29:06 +0200 Subject: [erlang-questions] exception error: bad argument in function crypto:ec_key_generate/2 Message-ID: Hi, Executing this code: crypto:generate_key(ecdh, secp112r2). on centos 6/7 I get this error: 1> crypto:generate_key(ecdh, secp112r2). ** exception error: bad argument in function crypto:ec_key_generate/2 called as crypto:ec_key_generate({{prime_field, <<219,124,42,191,98,227,94,102,128,118,190,173,32,139>>}, {<<97,39,194,76,5,243,138,10,170,246,92,14,240,44>>, <<81,222,241,129,93,181,237,116,252,195,76,133,215,9>>, <<0,39,87,161,17,77,105,110,103,104,117,97,81,117,83,22, 192,94,11,212>>}, <<4,75,163,10,181,232,146,180,225,100,157,208,146,134,67, 173,205,70,245,136,46,55,71,222,243,...>>, <<54,223,10,175,216,184,215,89,124,161,5,32,208,75>>, <<4>>}, undefined) 2> The same code on Ubuntu works correctly: 4> crypto:generate_key(ecdh, secp112r2). {<<4,80,204,3,54,107,208,108,63,20,107,12,38,106,212,97, 102,31,150,82,196,11,163,51,255,158,213,19,...>>, <<44,243,165,184,148,43,124,19,234,144,220,206,39,234>>} 5> Notes: I tried with different Erlang VM from starting from 18.3. I tried also to rebuild the last OPENSSL version on centos without success. The code above is a semplication from my original problem that is the follow: Args={{prime_field,<<255,255,255,255,255,255,255,255,255,255, 255,255,255,255,255,255,0,0,0,0,0,0,0,0, 0,0,0,1>>}, {<<"????????????????????????????">>, <<180,5,10,133,12,4,179,171,245,65,50,86,80,68,176,183, 215,191,216,186,39,11,57,67,35,85,255,180>>, <<189,113,52,71,153,213,199,252,220,69,181,159,163,185, 171,143,106,148,139,197>>}, <<4,183,14,12,189,107,180,191,127,50,19,144,185,74,3,193, 211,86,194,17,34,52,50,128,214,17,92,29,33,189,55,99, 136,181,247,35,251,76,34,223,230,205,67,117,160,90,7, 71,100,68,213,129,153,133,0,126,52>>, <<255,255,255,255,255,255,255,255,255,255,255,255,255, 255,22,162,224,184,240,62,19,221,41,69,92,92,42,61>>, <<1>>}. crypto:generate_key(ecdh, Args). it works on ubuntu and not in Centos 6/7 Thank you! GaS -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Sun Jul 24 00:42:23 2016 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Sat, 23 Jul 2016 15:42:23 -0700 Subject: [erlang-questions] exception error: bad argument in function crypto:ec_key_generate/2 In-Reply-To: References: Message-ID: <5793F2CF.4060904@gmail.com> On 07/23/2016 09:29 AM, Gabriele Santomaggio wrote: > Executing this code: > > crypto:generate_key(ecdh, secp112r2). > > on centos 6/7 I get this error: This is a shot in the dark, but does the list returned by crypto:ec_curves() contain secp112r2 on your CentOS machine? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From Eric.desCourtis@REDACTED Sun Jul 24 04:10:57 2016 From: Eric.desCourtis@REDACTED (Eric des Courtis) Date: Sun, 24 Jul 2016 02:10:57 +0000 Subject: [erlang-questions] comparison with opensaf In-Reply-To: References: Message-ID: Personally I think it's complicated for nothing. The amount of documentation has nothing to do with how well it is documented. If you have to read 500 pages to understand what a supervisor does that seems like massive over engineering. On Fri, Jul 22, 2016, 8:46 PM Sashan Govender wrote: > Not sure which documentation set you saw but the one over here looks > quite detailed. The AMF documentation alone is 500 pages. > > http://devel.opensaf.org/SAI-AIS-AMF-B.04.01.AL.pdf > > AMF as far as I can tell is like the OTP supervisor process. > > In terms of industry applications I know it's used in Ericsson's DSC > (Diameter Signalling Controller). > > On Sat, Jul 23, 2016 at 12:58 AM, Eric des Courtis > wrote: > > This is a very hard question to answer because I would say OpenSAF is > rather > > obscure. But I can tell you that while there are some overlaps between > the > > two technologies particularly when it comes design patterns. The fact > that > > Erlang is done entirely from the ground up (language, runtime, otp > patterns > > etc...) specifically for high availability means that you should in > theory > > experience much less friction when designing this sort of system in > Erlang. > > > > My feeling from looking at the documentation is that this isn't well > > documented. In short I wouldn't consider is competition to Erlang in it's > > current state for any project. > > > > On Thu, Jul 21, 2016 at 9:42 PM, Sashan Govender > wrote: > >> > >> Hi > >> > >> I'm interested if anyone has done a comparison the Erlang OTP and > OpenSAF. > >> It seems to me there is a significant amount of overlap in the but I > don't > >> have the expertise in both systems to form a comprehensive picture. As > far > >> as I can tell mnesia is like OpenSAF IMM. They can both be used as > >> configuration databases for a cluster. They both are strongly > consistent. I > >> know IMM certainly favours consistency over availabilty. OTP is > obviously in > >> Erlang while OpenSAF is in C/C++. Any comparisons out there about these > two > >> systems? Or am I wildly off the mark thinking that they are similar? > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashang@REDACTED Sun Jul 24 06:37:39 2016 From: sashang@REDACTED (Sashan Govender) Date: Sun, 24 Jul 2016 14:37:39 +1000 Subject: [erlang-questions] comparison with opensaf In-Reply-To: References: Message-ID: I couldn't agree more. The amount of documentation doesn't correlate with quality, and yes I think OpenSAF is over engineered, to say the least. On Sun, Jul 24, 2016 at 12:10 PM, Eric des Courtis wrote: > Personally I think it's complicated for nothing. The amount of > documentation has nothing to do with how well it is documented. If you have > to read 500 pages to understand what a supervisor does that seems like > massive over engineering. > > > On Fri, Jul 22, 2016, 8:46 PM Sashan Govender wrote: >> >> Not sure which documentation set you saw but the one over here looks >> quite detailed. The AMF documentation alone is 500 pages. >> >> http://devel.opensaf.org/SAI-AIS-AMF-B.04.01.AL.pdf >> >> AMF as far as I can tell is like the OTP supervisor process. >> >> In terms of industry applications I know it's used in Ericsson's DSC >> (Diameter Signalling Controller). >> >> On Sat, Jul 23, 2016 at 12:58 AM, Eric des Courtis >> wrote: >> > This is a very hard question to answer because I would say OpenSAF is >> > rather >> > obscure. But I can tell you that while there are some overlaps between >> > the >> > two technologies particularly when it comes design patterns. The fact >> > that >> > Erlang is done entirely from the ground up (language, runtime, otp >> > patterns >> > etc...) specifically for high availability means that you should in >> > theory >> > experience much less friction when designing this sort of system in >> > Erlang. >> > >> > My feeling from looking at the documentation is that this isn't well >> > documented. In short I wouldn't consider is competition to Erlang in >> > it's >> > current state for any project. >> > >> > On Thu, Jul 21, 2016 at 9:42 PM, Sashan Govender >> > wrote: >> >> >> >> Hi >> >> >> >> I'm interested if anyone has done a comparison the Erlang OTP and >> >> OpenSAF. >> >> It seems to me there is a significant amount of overlap in the but I >> >> don't >> >> have the expertise in both systems to form a comprehensive picture. As >> >> far >> >> as I can tell mnesia is like OpenSAF IMM. They can both be used as >> >> configuration databases for a cluster. They both are strongly >> >> consistent. I >> >> know IMM certainly favours consistency over availabilty. OTP is >> >> obviously in >> >> Erlang while OpenSAF is in C/C++. Any comparisons out there about these >> >> two >> >> systems? Or am I wildly off the mark thinking that they are similar? >> >> >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > From g.santomaggio@REDACTED Sun Jul 24 16:43:25 2016 From: g.santomaggio@REDACTED (Gabriele Santomaggio) Date: Sun, 24 Jul 2016 16:43:25 +0200 Subject: [erlang-questions] exception error: bad argument in function crypto:ec_key_generate/2 Message-ID: <8560FC6A-5C27-4C81-B0F3-488BF2536480@gmail.com> On 07/23/2016 09:29 AM, Gabriele Santomaggio wrote: >> Executing this code: >> >> crypto:generate_key(ecdh, secp112r2). >> >> on centos 6/7 I get this error: >This is a shot in the dark, but does the list returned by >crypto:ec_curves() contain secp112r2 on your CentOS machine? Hi, 6> crypto:ec_curves(). [secp112r1,secp112r2,secp128r1,secp128r2,secp160k1, secp160r1,secp160r2,secp192r1,secp192k1,secp224k1,secp224r1, secp256k1,secp256r1,secp384r1,secp521r1,prime192v1, prime192v2,prime192v3,prime239v1,prime239v2,prime239v3, The main problem btw is a ssl connection to an LDPAs if i execute openssl s_client -connect my server:636 in the same machine it works correctly. If I try using {ok, Socket} = ssl:connect("my server", 636, [], infinity). I get the exception exception error: bad argument After a few debug I found that this is the problem: Args={{prime_field,<<255,255,255,255,255,255,255,255,255,255, 255,255,255,255,255,255,0,0,0,0,0,0,0,0, 0,0,0,1>>}, {<<"????????????????????????????">>, <<180,5,10,133,12,4,179,171,245,65,50,86,80,68,176,183, 215,191,216,186,39,11,57,67,35,85,255,180>>, <<189,113,52,71,153,213,199,252,220,69,181,159,163,185, 171,143,106,148,139,197>>}, <<4,183,14,12,189,107,180,191,127,50,19,144,185,74,3,193, 211,86,194,17,34,52,50,128,214,17,92,29,33,189,55,99, 136,181,247,35,251,76,34,223,230,205,67,117,160,90,7, 71,100,68,213,129,153,133,0,126,52>>, <<255,255,255,255,255,255,255,255,255,255,255,255,255, 255,22,162,224,184,240,62,19,221,41,69,92,92,42,61>>, <<1>>}. crypto:generate_key(ecdh, Args). * exception error: bad argument in function crypto:ec_key_generate/2 called as crypto:ec_key_generate({{prime_field, <<219,124,42,191,98,227,94,102,128,118,190,173,32,139>>}, {<<97,39,194,76,5,243,138,10,170,246,92,14,240,44>>, <<81,222,241,129,93,181,237,116,252,195,76,133,215,9>>, <<0,39,87,161,17,77,105,110,103,104,117,97,81,117,83,22, 192,94,11,212>>}, <<4,75,163,10,181,232,146,180,225,100,157,208,146,134,67, 173,205,70,245,136,46,55,71,222,243,...>>, <<54,223,10,175,216,184,215,89,124,161,5,32,208,75>>, <<4>>}, undefined) I suppose that there is a general problem when I try to use crypto:generate_key(ecdh,?) Tried it on Centos 6 and 7. Same errors Thank you. GaS -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Sun Jul 24 22:19:36 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Sun, 24 Jul 2016 22:19:36 +0200 Subject: [erlang-questions] exception error: bad argument in function crypto:ec_key_generate/2 In-Reply-To: <8560FC6A-5C27-4C81-B0F3-488BF2536480@gmail.com> References: <8560FC6A-5C27-4C81-B0F3-488BF2536480@gmail.com> Message-ID: Hi, This is probably the solution and the cause... https://www.internetstaff.com/enable-elliptical-curve-diffie-hellman-ecdhe-linux/ Regars "Envoy? depuis mon mobile " Eric ---- Gabriele Santomaggio a ?crit ---- >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From sid5@REDACTED Mon Jul 25 00:09:36 2016 From: sid5@REDACTED (Sid Muller) Date: Mon, 25 Jul 2016 00:09:36 +0200 Subject: [erlang-questions] Sending fun/1 over the network and apply/2 failure Message-ID: Hi everyone, I am having an issue that I don't really understand why it's happening. In the shell I create a fun on one node: Fa = fun(A)->A end. #Fun Then call term_to_binary() which gives me the following: <<131,112,0,0,2,158,1,96,205,72,68,75,104,221,132,114,190,222,211,56,153,90,202,0,0,0,6,0,0,0,1,100,0,8,101,114,108,95,101,118,97,108,97,6,98,3,6,106,66,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,104,4,106,104,2,100,0,4,101,118,97,108,112,0,0,1,161,3,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,21,0,0,0,4,100,0,5,115,104,101,108,108,97,21,98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,104,2,100,0,5,118,97,108,117,101,112,0,0,0,96,2,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,5,0,0,0,1,100,0,5,115,104,101,108,108,97,5,98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,98,0,0,32,15,112,0,0,0,208,1,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,12,0,0,0,3,100,0,5,115,104,101,108,108,97,12,98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,104,2,100,0,5,118,97,108,117,101,112,0,0,0,96,2,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,5,0,0,0,1,100,0,5,115,104,101,108,108,97,5,98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,98,0,0,32,15,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,104,2,100,0,5,118,97,108,117,101,112,0,0,0,96,2,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,5,0,0,0,1,100,0,5,115,104,101,108,108,97,5,98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,108,0,0,0,1,104,5,100,0,6,99,108,97,117,115,101,97,1,108,0,0,0,1,104,3,100,0,3,118,97,114,97,1,100,0,1,65,106,106,108,0,0,0,1,104,3,100,0,3,118,97,114,97,1,100,0,1,65,106,106>> Now if I cut and paste this into another console like this: Fa = binary_to_term(<<131,112,0,0,2,158,1,96,205,72,68,75,104,221,132,114,190,222,211,56,153,90,202,0,0,0,6,0,0,0,1,100,0,8,101,114,108,95,101,118,97,108,97,6,98,3,6,106,66,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,104,4,106,104,2,100,0,4,101,118,97,108,112,0,0,1,161,3,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,21,0,0,0,4,100,0,5,115,104,101,108,108,97,21,98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,104,2,100,0,5,118,97,108,117,101,112,0,0,0,96,2,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,5,0,0,0,1,100,0,5,115,104,101,108,108,97,5,98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,98,0,0,32,15,112,0,0,0,208,1,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,12,0,0,0,3,100,0,5,115,104,101,108,108,97,12,98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,104,2,100,0,5,118,97,108,117,101,112,0,0,0,96,2,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,5,0,0,0,1,100,0,5,115,104,101,108,108,97,5,98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,98,0,0,32,15,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,104,2,100,0,5,118,97,108,117,101,112,0,0,0,96,2,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,5,0,0,0,1,100,0,5,115,104,101,108,108,97,5,98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,108,0,0,0,1,104,5,100,0,6,99,108,97,117,115,101,97,1,108,0,0,0,1,104,3,100,0,3,118,97,114,97,1,100,0,1,65,106,106,108,0,0,0,1,104,3,100,0,3,118,97,114,97,1,100,0,1,65,106,106>>). I get: #Fun And then I call: apply(Fa, ["Param"]). And the code works fine, as long as I do that on the console. But if I do that in erlang code and send the binary over the network, and I print it to make sure that it's the same as the binary above. Then when I call apply it fails. I get this: Error in process <0.5040.1> with exit value: {{badfun,#Fun}, .... I am completely lost why cut and paste works from the console but from erlang code it fails, even though everything is the same, both binary and the parameters I'm passing to apply/2. What am I missing, does anyone have any pointers? I find it very strange... From zxq9@REDACTED Mon Jul 25 05:33:15 2016 From: zxq9@REDACTED (zxq9) Date: Mon, 25 Jul 2016 12:33:15 +0900 Subject: [erlang-questions] Sending fun/1 over the network and apply/2 failure In-Reply-To: References: Message-ID: <113043122.kRR27Ah4Zx@changa> On 2016?7?25? ??? 00:09:36 Sid Muller wrote: > Hi everyone, > > I am having an issue that I don't really understand why it's happening. > > In the shell I create a fun on one node: > Fa = fun(A)->A end. > #Fun > > Then call term_to_binary() > > which gives me the following: ... snip ... > I get: > #Fun > > And then I call: > apply(Fa, ["Param"]). > > And the code works fine, as long as I do that on the console. > > But if I do that in erlang code and send the binary over the network, and I print it to make sure that it's the same as the binary above. Then when I call apply it fails. I get this: > > Error in process <0.5040.1> with exit value: > {{badfun,#Fun}, .... > > > I am completely lost why cut and paste works from the console but from erlang code it fails, even though everything is the same, both binary and the parameters I'm passing to apply/2. > > What am I missing, does anyone have any pointers? I find it very strange... Funs are not portable between nodes the way other values are. I'm fuzzy on the implementation details, but they basically work as a reference to a locally stored fun + whatever data is bound within that particular fun. (I'm mildly curious how this works in the shell... anyone care to explain it?) When you try to execute this fun reference somewhere else it cannot be dereferenced because the underlying fun was never defined on the other node. There is a proposal for "portable funs": http://www.erlang.org/erlang-enhancement-proposals/eep-0015.html But its still a low priority. An aside on funs in Erlang... It is rare to see long-lived funs (and in a system where you expect hot upgrades this would be unwise anyway -- because you would have this legacy fun floating around for a long time). While it is pretty common (and super convenient) to define protocols over TCP as term_to_binary/binary_to_term, it is *still* important to write the socket handlers in a way that they "implement a protocol" instead of execute raw values. In Erlang I find that lambdas are useful mostly as a way of closing over some ongoing state to curry out the parts of an operation that don't fit cleanly into a list operation -- or especially to write dispatch functions as funs so that each dispatched call can be to any function of any arity instead of writing a ton of wrappers. Its not like guile where I would feel comfortable writing a system init function that composes a system's start function and then kicks the whole thing off as one giant lambda. -Craig From raimo+erlang-questions@REDACTED Mon Jul 25 10:50:44 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 25 Jul 2016 10:50:44 +0200 Subject: [erlang-questions] Patch Package OTP 18.3.4.2 Released Message-ID: <20160725085044.GA97448@erix.ericsson.se> Patch Package: OTP 18.3.4.2 Git Tag: OTP-18.3.4.2 Date: 2016-07-25 Trouble Report Id: OTP-13730, OTP-13731, OTP-13753 Seq num: seq13135, seq13150 System: OTP Release: 18 Application: common_test-1.12.1.1, erts-7.3.1.1, ssl-7.3.3.1 Predecessor: OTP 18.3.4.1 Check out the git tag OTP-18.3.4.2, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- common_test-1.12.1.1 -------------------------------------------- --------------------------------------------------------------------- The common_test-1.12.1.1 application can be applied independently of other applications on a full OTP 18 installation. --- Fixed Bugs and Malfunctions --- OTP-13730 Application(s): common_test Related Id(s): seq13135 If the telnet server would pause during transmission of a line of text before terminating the line, the ct_telnet:expect/3 function would print the line twice in the test case HTML log. This problem has been fixed. Full runtime dependencies of common_test-1.12.1.1: compiler-6.0, crypto-3.6, debugger-4.1, erts-7.0, inets-6.0, kernel-4.0, observer-2.1, runtime_tools-1.8.16, sasl-2.4.2, snmp-5.1.2, ssh-4.0, stdlib-2.5, test_server-3.9, tools-2.8, xmerl-1.3.8 --------------------------------------------------------------------- --- erts-7.3.1.1 ---------------------------------------------------- --------------------------------------------------------------------- The erts-7.3.1.1 application can be applied independently of other applications on a full OTP 18 installation. --- Fixed Bugs and Malfunctions --- OTP-13731 Application(s): erts Related Id(s): ERL-188 Fix scheduler deadlock bug in ets:update_counter/4 when key is not found and inserting the default object causes the table to grow. Full runtime dependencies of erts-7.3.1.1: kernel-4.0, sasl-2.4, stdlib-2.5 --------------------------------------------------------------------- --- ssl-7.3.3.1 ----------------------------------------------------- --------------------------------------------------------------------- The ssl-7.3.3.1 application can be applied independently of other applications on a full OTP 18 installation. --- Fixed Bugs and Malfunctions --- OTP-13753 Application(s): ssl Related Id(s): seq13150 The TLS/SSL protocol version selection for the SSL server has been corrected to follow RFC 5246 Appendix E.1 especially in case where the list of supported versions has gaps. Now the server selects the highest protocol version it supports that is not higher than what the client supports. Full runtime dependencies of ssl-7.3.3.1: crypto-3.3, erts-7.0, inets-5.10.7, kernel-3.0, public_key-1.0, stdlib-2.0 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From mikpelinux@REDACTED Mon Jul 25 13:49:29 2016 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Mon, 25 Jul 2016 13:49:29 +0200 Subject: [erlang-questions] Sending fun/1 over the network and apply/2 failure In-Reply-To: References: Message-ID: <22421.64713.214940.123130@gargle.gargle.HOWL> Sid Muller writes: > Hi everyone, > > I am having an issue that I don't really understand why it's happening. > > In the shell I create a fun on one node: > Fa = fun(A)->A end. > #Fun > > Then call term_to_binary() > > which gives me the following: > > <<131,112,0,0,2,158,1,96,205,72,68,75,104,221,132,114,190,222,211,56,153,90,202,0,0,0,6,0,0,0,1,100,0,8,101,114,108,95,101,118,97,108,97,6,98,3,6,106,66,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,104,4,106,104,2,100,0,4,101,118,97,108,112,0,0,1,161,3,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,21,0,0,0,4,100,0,5,115,104,101,108,108,97,21,98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,104,2,100,0,5,118,97,108,117,101,112,0,0,0,96,2,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,5,0,0,0,1,100,0,5,115,104,101,108,108,97,5,98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,98,0,0,32,15,112,0,0,0,208,1,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,12,0,0,0,3,100,0,5,115,104,101,108,108,97,12,98,4,243,242,217,103 > ,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,104,2,100,0,5,118,97,108,117,101,112,0,0,0,96,2,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,5,0,0,0,1,100,0,5,115,104,101,108,108,97,5,98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,98,0,0,32,15,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,104,2,100,0,5,118,97,108,117,101,112,0,0,0,96,2,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,5,0,0,0,1,100,0,5,115,104,101,108,108,97,5,98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,108,0,0,0,1,104,5,100,0,6,99,108,97,117,115,101,97,1,108,0 > ,0,0,1,104,3,100,0,3,118,97,114,97,1,100,0,1,65,106,106,108,0,0,0,1,104,3,100,0,3,118,97,114,97,1,100,0,1,65,106,106>> > > Now if I cut and paste this into another console like this: > > Fa = binary_to_term(<<131,112,0,0,2,158,1,96,205,72,68,75,104,221,132,114,190,222,211,56,153,90,202,0,0,0,6,0,0,0,1,100,0,8,101,114,108,95,101,118,97,108,97,6,98,3,6,106,66,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,104,4,106,104,2,100,0,4,101,118,97,108,112,0,0,1,161,3,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,21,0,0,0,4,100,0,5,115,104,101,108,108,97,21,98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,104,2,100,0,5,118,97,108,117,101,112,0,0,0,96,2,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,5,0,0,0,1,100,0,5,115,104,101,108,108,97,5,98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,98,0,0,32,15,112,0,0,0,208,1,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,12,0,0,0,3,100,0,5,115,104,101,108,108,97,12, > 98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,104,2,100,0,5,118,97,108,117,101,112,0,0,0,96,2,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,5,0,0,0,1,100,0,5,115,104,101,108,108,97,5,98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,98,0,0,32,15,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,104,2,100,0,5,118,97,108,117,101,112,0,0,0,96,2,158,126,91,59,128,49,98,27,137,222,82,19,201,52,153,138,0,0,0,5,0,0,0,1,100,0,5,115,104,101,108,108,97,5,98,4,243,242,217,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,33,0,0,0,0,0,103,100,0,13,110,111,110,111,100,101,64,110,111,104,111,115,116,0,0,0,27,0,0,0,0,0,108,0,0,0,1,104,5,100,0,6,99,108,97,11 > 7,115,101,97,1,108,0,0,0,1,104,3,100,0,3,118,97,114,97,1,100,0,1,65,106,106,108,0,0,0,1,104,3,100,0,3,118,97,114,97,1,100,0,1,65,106,106>>). > > I get: > #Fun > > And then I call: > apply(Fa, ["Param"]). > > And the code works fine, as long as I do that on the console. > > But if I do that in erlang code and send the binary over the network, and I print it to make sure that it's the same as the binary above. Then when I call apply it fails. I get this: > > Error in process <0.5040.1> with exit value: > {{badfun,#Fun}, .... > > > I am completely lost why cut and paste works from the console but from erlang code it fails, even though everything is the same, both binary and the parameters I'm passing to apply/2. > > What am I missing, does anyone have any pointers? I find it very strange... A closure only contains a reference to its code, not a copy of the code itself. Taking a closure created in module M, passing it to another Erlang node and applying it there, requires that the module M is present there as well. If it isn't, or if the modules differ too much, you get a badfun exception. The code part of a closure created in the shell refers to a module in the Erlang interpreter (erl_eval or something like that) with the actual expression being a plain term in the closed over free variables (which the module can interpret). That's why the shell's closures work in some cases where closures from beam files don't. From yangzhenguo@REDACTED Mon Jul 25 13:38:44 2016 From: yangzhenguo@REDACTED (Yang Zhenguo) Date: Mon, 25 Jul 2016 19:38:44 +0800 Subject: [erlang-questions] The way to backup mnesia data daily In-Reply-To: References: Message-ID: Hi Team, Any suggestion on it? Thanks, Zhenguo 2016-07-24 22:14 GMT+08:00 Yang Zhenguo : > Hi Folks, > > Having a question about mnesia backup. > > We have an application running based on Erlang Cowboy and Mnesia. I am > wondering to know what is a good way to backup mnesia data daily, since we > only have one node. > > I followed the steps here > > http://stackoverflow.com/questions/463400/how-to-rename-the-node-running-a-mnesia-database > > But got the errors below: > {aborted,{'EXIT',{aborted,{bad_commit,{missing_lock,'web_server@REDACTED > '}}}}??} > > The reason I want to change node name is in case I deploy it in another > machine with different ip address. > > May I have your suggestion? Many thanks. > > -- > Zhenguo Yang > www.prinbit.com > -- Zhenguo Yang www.prinbit.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From sid5@REDACTED Mon Jul 25 14:17:00 2016 From: sid5@REDACTED (Sid Muller) Date: Mon, 25 Jul 2016 14:17:00 +0200 Subject: [erlang-questions] Sending fun/1 over the network and apply/2 failure In-Reply-To: <22421.64713.214940.123130@gargle.gargle.HOWL> References: , <22421.64713.214940.123130@gargle.gargle.HOWL> Message-ID: > Sent: Monday, July 25, 2016 at 6:49 AM > From: "Mikael Pettersson" > To: "Sid Muller" > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Sending fun/1 over the network and apply/2 failure > A closure only contains a reference to its code, not a copy of the code > itself. Taking a closure created in module M, passing it to another > Erlang node and applying it there, requires that the module M is present > there as well. If it isn't, or if the modules differ too much, you get > a badfun exception. > > The code part of a closure created in the shell refers to a module in > the Erlang interpreter (erl_eval or something like that) with the actual > expression being a plain term in the closed over free variables (which > the module can interpret). That's why the shell's closures work in some > cases where closures from beam files don't. Ahhh, OK that makes sense. Thank you From erlang@REDACTED Mon Jul 25 15:14:45 2016 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 25 Jul 2016 15:14:45 +0200 Subject: [erlang-questions] Sending fun/1 over the network and apply/2 failure In-Reply-To: References: <22421.64713.214940.123130@gargle.gargle.HOWL> Message-ID: Erlang funs are not portable over network nodes. The reason being that the underlying beam code that implements the funs is not guaranteed to be the same on all participating nodes. If you can ensure prior to evaluating the fun that the compiled module code is available on all machines then funs are portable (note you also must have the same version of Erlang running on all nodes). The reason is largely historical. The first Erlang clusters ran on nodes with a common NFS mounted file store, so although the nodes ran on different physical machines they accessed the same file store. In the usual mode of operation files would be updated on one node then the entire application restarted over several nodes and thus all nodes saw the same version of the compiled code. If the odd process crashed due to code incompatibilities it would be restarted (think supervision trees) and on restart pick up the latest version of the code (so in practise this worked, though it's not 100% theoretically sound - there are edges cases where this would not work). If you want to make this work you just have to make sure that all nodes have the same version of the module(s) concerned. I guess just should be in quotes (ie "just") since this is very difficult (having 2 different versions of code on one node is just about understandable) but doing this over multiple nodes is far more difficult. The "send the same code to all machines - then restart" strategy is just about the easiest way to make this work. Cheers /Joe On Mon, Jul 25, 2016 at 2:17 PM, Sid Muller wrote: >> Sent: Monday, July 25, 2016 at 6:49 AM >> From: "Mikael Pettersson" >> To: "Sid Muller" >> Cc: erlang-questions@REDACTED >> Subject: Re: [erlang-questions] Sending fun/1 over the network and apply/2 failure > >> A closure only contains a reference to its code, not a copy of the code >> itself. Taking a closure created in module M, passing it to another >> Erlang node and applying it there, requires that the module M is present >> there as well. If it isn't, or if the modules differ too much, you get >> a badfun exception. >> >> The code part of a closure created in the shell refers to a module in >> the Erlang interpreter (erl_eval or something like that) with the actual >> expression being a plain term in the closed over free variables (which >> the module can interpret). That's why the shell's closures work in some >> cases where closures from beam files don't. > > Ahhh, OK that makes sense. > > Thank you > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From sid5@REDACTED Mon Jul 25 15:20:06 2016 From: sid5@REDACTED (Sid Muller) Date: Mon, 25 Jul 2016 15:20:06 +0200 Subject: [erlang-questions] Sending fun/1 over the network and apply/2 failure In-Reply-To: References: <22421.64713.214940.123130@gargle.gargle.HOWL> , Message-ID: > Sent: Monday, July 25, 2016 at 8:14 AM > From: "Joe Armstrong" > To: "Sid Muller" > Cc: "Mikael Pettersson" , Erlang > Subject: Re: [erlang-questions] Sending fun/1 over the network and apply/2 failure > [SNIP] > If you want to make this work you just have to make sure that all > nodes have the same > version of the module(s) concerned. I guess just should be in quotes > (ie "just") since > this is very difficult (having 2 different versions of code on one > node is just about > understandable) but doing this over multiple nodes is far more difficult. > > The "send the same code to all machines - then restart" strategy is > just about the > easiest way to make this work. Hi, Joe- yes that's what I ended up doing. I put the code into the module, send it over, upgrade, and then run the fun.... And it works, I was assuming previously that since I was sending an anon function it would work but I guess due to reasons pointed out by Mikael this didn't work... From ola.a.andersson@REDACTED Mon Jul 25 15:42:45 2016 From: ola.a.andersson@REDACTED (Ola Andersson A) Date: Mon, 25 Jul 2016 13:42:45 +0000 Subject: [erlang-questions] comparison with opensaf In-Reply-To: References: Message-ID: I have implemented parts of the SAF specifications in Erlang. IMM is definitely one of the most ridiculously overcomplicated specifications I have seen. My hat off to the developers of OpenSAF for successfully creating a working implementation in C. It's hard enough to do it in Erlang but it must have been a nightmare developing in C. I haven't checked how many LOC it is but I'm guessing it's a lot. > -----Original Message----- > From: erlang-questions-bounces@REDACTED [mailto:erlang-questions- > bounces@REDACTED] On Behalf Of Sashan Govender > Sent: den 24 juli 2016 06:38 > To: Eric des Courtis > Cc: Erlang > Subject: Re: [erlang-questions] comparison with opensaf > > I couldn't agree more. The amount of documentation doesn't correlate with > quality, and yes I think OpenSAF is over engineered, to say the least. > > On Sun, Jul 24, 2016 at 12:10 PM, Eric des Courtis > wrote: > > Personally I think it's complicated for nothing. The amount of > > documentation has nothing to do with how well it is documented. If you > > have to read 500 pages to understand what a supervisor does that seems > > like massive over engineering. > > > > > > On Fri, Jul 22, 2016, 8:46 PM Sashan Govender > wrote: > >> > >> Not sure which documentation set you saw but the one over here looks > >> quite detailed. The AMF documentation alone is 500 pages. > >> > >> http://devel.opensaf.org/SAI-AIS-AMF-B.04.01.AL.pdf > >> > >> AMF as far as I can tell is like the OTP supervisor process. > >> > >> In terms of industry applications I know it's used in Ericsson's DSC > >> (Diameter Signalling Controller). > >> > >> On Sat, Jul 23, 2016 at 12:58 AM, Eric des Courtis > >> wrote: > >> > This is a very hard question to answer because I would say OpenSAF > >> > is rather obscure. But I can tell you that while there are some > >> > overlaps between the two technologies particularly when it comes > >> > design patterns. The fact that Erlang is done entirely from the > >> > ground up (language, runtime, otp patterns > >> > etc...) specifically for high availability means that you should in > >> > theory experience much less friction when designing this sort of > >> > system in Erlang. > >> > > >> > My feeling from looking at the documentation is that this isn't > >> > well documented. In short I wouldn't consider is competition to > >> > Erlang in it's current state for any project. > >> > > >> > On Thu, Jul 21, 2016 at 9:42 PM, Sashan Govender > >> > > >> > wrote: > >> >> > >> >> Hi > >> >> > >> >> I'm interested if anyone has done a comparison the Erlang OTP and > >> >> OpenSAF. > >> >> It seems to me there is a significant amount of overlap in the but > >> >> I don't have the expertise in both systems to form a comprehensive > >> >> picture. As far as I can tell mnesia is like OpenSAF IMM. They can > >> >> both be used as configuration databases for a cluster. They both > >> >> are strongly consistent. I know IMM certainly favours consistency > >> >> over availabilty. OTP is obviously in Erlang while OpenSAF is in > >> >> C/C++. Any comparisons out there about these two systems? Or am I > >> >> wildly off the mark thinking that they are similar? > >> >> > >> >> _______________________________________________ > >> >> erlang-questions mailing list > >> >> erlang-questions@REDACTED > >> >> http://erlang.org/mailman/listinfo/erlang-questions > >> >> > >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From tony@REDACTED Mon Jul 25 15:47:23 2016 From: tony@REDACTED (Tony Rogvall) Date: Mon, 25 Jul 2016 15:47:23 +0200 Subject: [erlang-questions] Sending fun/1 over the network and apply/2 failure In-Reply-To: References: <22421.64713.214940.123130@gargle.gargle.HOWL> Message-ID: <87254682-E87B-4E8C-98B8-B26B3FB418BA@rogvall.se> I did some experiments for ?fun? a couple of years ago. This can surely be used for some non critical applications. You can modify the error_handler.erl ( global or per use ) Change undefined_lambda/3 to: undefined_lambda(Module, Fun, Args) -> case ensure_loaded(Module) of {module, Module} -> %% There is no need (and no way) to test if the fun is present. %% apply/2 will not call us again if the fun is missing. apply(Fun, Args); {module, _} -> crash(Fun, Args); _Other -> {pid,Pid} = erlang:fun_info(Fun,pid), case try_load_remote_code(Pid,Module) of ok -> case ensure_loaded(Module) of {module, Module} -> apply(Fun, Args); _ -> crash(Fun, Args) end; _ -> crash(Fun, Args) end end. and add the function to load remote code. %% %% Try load code from remote node %% try_load_remote_code(Pid,Module) -> if node() =:= node(Pid) -> ok; %% no need to load true -> case rpc:call(node(Pid),code,get_object_code,[Module]) of {Module, Binary, Filename} -> case code:load_binary(Module,Filename,Binary) of {module,Module} -> ok; _ -> error end; error -> error end end. Home assignment: Wrap Funs with a resource handle ( https://github.com/tonyrog/resource ) the will enable garbage collection of the code. :-) /Tony > On 25 jul 2016, at 15:20, Sid Muller wrote: > >> Sent: Monday, July 25, 2016 at 8:14 AM >> From: "Joe Armstrong" >> To: "Sid Muller" >> Cc: "Mikael Pettersson" , Erlang >> Subject: Re: [erlang-questions] Sending fun/1 over the network and apply/2 failure >> > [SNIP] >> If you want to make this work you just have to make sure that all >> nodes have the same >> version of the module(s) concerned. I guess just should be in quotes >> (ie "just") since >> this is very difficult (having 2 different versions of code on one >> node is just about >> understandable) but doing this over multiple nodes is far more difficult. >> >> The "send the same code to all machines - then restart" strategy is >> just about the >> easiest way to make this work. > > Hi, Joe- > > yes that's what I ended up doing. I put the code into the module, send it over, upgrade, and then run the fun.... > > And it works, I was assuming previously that since I was sending an anon function it would work but I guess due to reasons pointed out by Mikael this didn't work... > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From buday.gergely@REDACTED Mon Jul 25 14:42:04 2016 From: buday.gergely@REDACTED (Gergely Buday) Date: Mon, 25 Jul 2016 14:42:04 +0200 Subject: [erlang-questions] difficult concurrent programs in practice Message-ID: <9fdbbe0969abb4cdc25fa2e2e765785d@uni-eszterhazy.hu> Hi, I am interested in the verification of concurrent programs. Are there papers/blog posts discussing concurrent Erlang programs that occur in practice and are difficult to get right? I know of ConcuError but feel free to point to any literature, I might not know it. - Gergely -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Jul 26 04:11:10 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 26 Jul 2016 14:11:10 +1200 Subject: [erlang-questions] Sending fun/1 over the network and apply/2 failure In-Reply-To: <22421.64713.214940.123130@gargle.gargle.HOWL> References: <22421.64713.214940.123130@gargle.gargle.HOWL> Message-ID: On 25/07/16 11:49 PM, Mikael Pettersson wrote: > The code part of a closure created in the shell refers to a module in > the Erlang interpreter (erl_eval or something like that) with the actual > expression being a plain term in the closed over free variables (which > the module can interpret). That's why the shell's closures work in some > cases where closures from beam files don't. Given that sending a fun from one node to another can be a sensible thing to do, this suggests to me that a simple hack might be to allow someone to specify that a particular fun should be processed as if it had appeared in the shell, something like shell:fun (...) -> ... end This is something that could probably be whipped up using a parse transform. As for me, I internalised the "funs-are-node-specific" idea a long time ago, so it doesn't matter enough to me for me to do it. From sashang@REDACTED Tue Jul 26 05:10:49 2016 From: sashang@REDACTED (Sashan Govender) Date: Tue, 26 Jul 2016 03:10:49 +0000 Subject: [erlang-questions] comparison with opensaf In-Reply-To: References: Message-ID: Regarding IMM would it be correct to say it's roughly equivalent to mnesia? I'm just trying to draw up rough equivalences between the OpenSAF services and those in the Erlang/OTP. From what I have read about mnesia it provides a persistant data store that tries to be highly consistent. From http://learnyousomeerlang.com/mnesia it says 'Mnesia sits on the CP side'. IMM has the same property. In fact it tries very hard to be consistent and sacrifices availabilty by not allowing writes for several minutes in some cases while it sorts itself out. It's typical usage scenario is in storing the configuration of a cluster, which I'm guessing mnesia can do as well. On Mon, Jul 25, 2016 at 11:42 PM Ola Andersson A < ola.a.andersson@REDACTED> wrote: > I have implemented parts of the SAF specifications in Erlang. IMM is > definitely one of the most ridiculously overcomplicated specifications I > have seen. > My hat off to the developers of OpenSAF for successfully creating a > working implementation in C. It's hard enough to do it in Erlang but it > must have been a nightmare developing in C. I haven't checked how many LOC > it is but I'm guessing it's a lot. > > > > -----Original Message----- > > From: erlang-questions-bounces@REDACTED [mailto:erlang-questions- > > bounces@REDACTED] On Behalf Of Sashan Govender > > Sent: den 24 juli 2016 06:38 > > To: Eric des Courtis > > Cc: Erlang > > Subject: Re: [erlang-questions] comparison with opensaf > > > > I couldn't agree more. The amount of documentation doesn't correlate with > > quality, and yes I think OpenSAF is over engineered, to say the least. > > > > On Sun, Jul 24, 2016 at 12:10 PM, Eric des Courtis > > wrote: > > > Personally I think it's complicated for nothing. The amount of > > > documentation has nothing to do with how well it is documented. If you > > > have to read 500 pages to understand what a supervisor does that seems > > > like massive over engineering. > > > > > > > > > On Fri, Jul 22, 2016, 8:46 PM Sashan Govender > > wrote: > > >> > > >> Not sure which documentation set you saw but the one over here looks > > >> quite detailed. The AMF documentation alone is 500 pages. > > >> > > >> http://devel.opensaf.org/SAI-AIS-AMF-B.04.01.AL.pdf > > >> > > >> AMF as far as I can tell is like the OTP supervisor process. > > >> > > >> In terms of industry applications I know it's used in Ericsson's DSC > > >> (Diameter Signalling Controller). > > >> > > >> On Sat, Jul 23, 2016 at 12:58 AM, Eric des Courtis > > >> wrote: > > >> > This is a very hard question to answer because I would say OpenSAF > > >> > is rather obscure. But I can tell you that while there are some > > >> > overlaps between the two technologies particularly when it comes > > >> > design patterns. The fact that Erlang is done entirely from the > > >> > ground up (language, runtime, otp patterns > > >> > etc...) specifically for high availability means that you should in > > >> > theory experience much less friction when designing this sort of > > >> > system in Erlang. > > >> > > > >> > My feeling from looking at the documentation is that this isn't > > >> > well documented. In short I wouldn't consider is competition to > > >> > Erlang in it's current state for any project. > > >> > > > >> > On Thu, Jul 21, 2016 at 9:42 PM, Sashan Govender > > >> > > > >> > wrote: > > >> >> > > >> >> Hi > > >> >> > > >> >> I'm interested if anyone has done a comparison the Erlang OTP and > > >> >> OpenSAF. > > >> >> It seems to me there is a significant amount of overlap in the but > > >> >> I don't have the expertise in both systems to form a comprehensive > > >> >> picture. As far as I can tell mnesia is like OpenSAF IMM. They can > > >> >> both be used as configuration databases for a cluster. They both > > >> >> are strongly consistent. I know IMM certainly favours consistency > > >> >> over availabilty. OTP is obviously in Erlang while OpenSAF is in > > >> >> C/C++. Any comparisons out there about these two systems? Or am I > > >> >> wildly off the mark thinking that they are similar? > > >> >> > > >> >> _______________________________________________ > > >> >> erlang-questions mailing list > > >> >> erlang-questions@REDACTED > > >> >> http://erlang.org/mailman/listinfo/erlang-questions > > >> >> > > >> > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Tue Jul 26 10:00:24 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 26 Jul 2016 10:00:24 +0200 Subject: [erlang-questions] gen_statem Auto Callback Mode Message-ID: <20160726080024.GA36806@erix.ericsson.se> Hello list! I have created a pull request for changing the new experimental behaviour gen_statem to automatically select callback mode or as I now call it callback flavour: https://github.com/erlang/otp/pull/1133 -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From ola.a.andersson@REDACTED Tue Jul 26 10:04:00 2016 From: ola.a.andersson@REDACTED (Ola Andersson A) Date: Tue, 26 Jul 2016 08:04:00 +0000 Subject: [erlang-questions] comparison with opensaf In-Reply-To: References: Message-ID: No, not really equivalent. IMM provides much more functionality than mnesia. It?s not just a data store. IMM consist of two parts; OM and OI. OM is the Object Management API. The OI part is a set of callbacks that lets the user provide an ObjectImplementer for an object or class of objects. IMM has both persistent configuration objects and dynamic runtime objects. Runtime object may also be cached. Configuration objects may contain runtime attributes and the other way around. Runtime objects and attributes are owned by the ObjectImplementer. For configuration objects an OI provides validation functionality with the possibility to reject a so called CCB (Configuration Change Bundle). This is just the basics of IMM. There is a lot more available. OpenSAF has also extended the SAF IMM specification with additional functionality. From: Sashan Govender [mailto:sashang@REDACTED] Sent: den 26 juli 2016 05:11 To: Ola Andersson A ; Eric des Courtis Cc: Erlang Subject: Re: [erlang-questions] comparison with opensaf Regarding IMM would it be correct to say it's roughly equivalent to mnesia? I'm just trying to draw up rough equivalences between the OpenSAF services and those in the Erlang/OTP. From what I have read about mnesia it provides a persistant data store that tries to be highly consistent. From http://learnyousomeerlang.com/mnesia it says 'Mnesia sits on the CP side'. IMM has the same property. In fact it tries very hard to be consistent and sacrifices availabilty by not allowing writes for several minutes in some cases while it sorts itself out. It's typical usage scenario is in storing the configuration of a cluster, which I'm guessing mnesia can do as well. On Mon, Jul 25, 2016 at 11:42 PM Ola Andersson A > wrote: I have implemented parts of the SAF specifications in Erlang. IMM is definitely one of the most ridiculously overcomplicated specifications I have seen. My hat off to the developers of OpenSAF for successfully creating a working implementation in C. It's hard enough to do it in Erlang but it must have been a nightmare developing in C. I haven't checked how many LOC it is but I'm guessing it's a lot. > -----Original Message----- > From: erlang-questions-bounces@REDACTED [mailto:erlang-questions- > bounces@REDACTED] On Behalf Of Sashan Govender > Sent: den 24 juli 2016 06:38 > To: Eric des Courtis > > Cc: Erlang > > Subject: Re: [erlang-questions] comparison with opensaf > > I couldn't agree more. The amount of documentation doesn't correlate with > quality, and yes I think OpenSAF is over engineered, to say the least. > > On Sun, Jul 24, 2016 at 12:10 PM, Eric des Courtis > > wrote: > > Personally I think it's complicated for nothing. The amount of > > documentation has nothing to do with how well it is documented. If you > > have to read 500 pages to understand what a supervisor does that seems > > like massive over engineering. > > > > > > On Fri, Jul 22, 2016, 8:46 PM Sashan Govender > > wrote: > >> > >> Not sure which documentation set you saw but the one over here looks > >> quite detailed. The AMF documentation alone is 500 pages. > >> > >> http://devel.opensaf.org/SAI-AIS-AMF-B.04.01.AL.pdf > >> > >> AMF as far as I can tell is like the OTP supervisor process. > >> > >> In terms of industry applications I know it's used in Ericsson's DSC > >> (Diameter Signalling Controller). > >> > >> On Sat, Jul 23, 2016 at 12:58 AM, Eric des Courtis > >> > wrote: > >> > This is a very hard question to answer because I would say OpenSAF > >> > is rather obscure. But I can tell you that while there are some > >> > overlaps between the two technologies particularly when it comes > >> > design patterns. The fact that Erlang is done entirely from the > >> > ground up (language, runtime, otp patterns > >> > etc...) specifically for high availability means that you should in > >> > theory experience much less friction when designing this sort of > >> > system in Erlang. > >> > > >> > My feeling from looking at the documentation is that this isn't > >> > well documented. In short I wouldn't consider is competition to > >> > Erlang in it's current state for any project. > >> > > >> > On Thu, Jul 21, 2016 at 9:42 PM, Sashan Govender > >> > > > >> > wrote: > >> >> > >> >> Hi > >> >> > >> >> I'm interested if anyone has done a comparison the Erlang OTP and > >> >> OpenSAF. > >> >> It seems to me there is a significant amount of overlap in the but > >> >> I don't have the expertise in both systems to form a comprehensive > >> >> picture. As far as I can tell mnesia is like OpenSAF IMM. They can > >> >> both be used as configuration databases for a cluster. They both > >> >> are strongly consistent. I know IMM certainly favours consistency > >> >> over availabilty. OTP is obviously in Erlang while OpenSAF is in > >> >> C/C++. Any comparisons out there about these two systems? Or am I > >> >> wildly off the mark thinking that they are similar? > >> >> > >> >> _______________________________________________ > >> >> erlang-questions mailing list > >> >> erlang-questions@REDACTED > >> >> http://erlang.org/mailman/listinfo/erlang-questions > >> >> > >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From yangzhenguo@REDACTED Mon Jul 25 16:20:46 2016 From: yangzhenguo@REDACTED (Yang Zhenguo) Date: Mon, 25 Jul 2016 22:20:46 +0800 Subject: [erlang-questions] The way to backup mnesia data daily In-Reply-To: References: Message-ID: I tried to add checkpoint, but got the following error: mnesia:activate_checkpoint([]). {error,{combine_error,{{1469,456316,975708}, 'web_server@REDACTED'}, [{min,[]},{max,[]}]}} 2016-07-25 19:38 GMT+08:00 Yang Zhenguo : > Hi Team, > > Any suggestion on it? > > Thanks, > Zhenguo > > 2016-07-24 22:14 GMT+08:00 Yang Zhenguo : > >> Hi Folks, >> >> Having a question about mnesia backup. >> >> We have an application running based on Erlang Cowboy and Mnesia. I am >> wondering to know what is a good way to backup mnesia data daily, since we >> only have one node. >> >> I followed the steps here >> >> http://stackoverflow.com/questions/463400/how-to-rename-the-node-running-a-mnesia-database >> >> But got the errors below: >> >> {aborted,{'EXIT',{aborted,{bad_commit,{missing_lock,'web_server@REDACTED >> '}}}}??} >> >> The reason I want to change node name is in case I deploy it in another >> machine with different ip address. >> >> May I have your suggestion? Many thanks. >> >> -- >> Zhenguo Yang >> www.prinbit.com >> > > > > -- > Zhenguo Yang > www.prinbit.com > -- Zhenguo Yang www.prinbit.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From z@REDACTED Tue Jul 26 12:11:28 2016 From: z@REDACTED (Danil Zagoskin) Date: Tue, 26 Jul 2016 13:11:28 +0300 Subject: [erlang-questions] LCNT: understanding proc_* and db_hash_slot collisions Message-ID: Hello! I am inspecting performance issues in an Erlang/OTP system. I've run LCNT and I see results I cannot interpret. After lcnt:swap_pid_keys(), I see these conflicts (only top of the table): lock id #tries #collisions collisions [%] time [us] duration [%] ----- --- ------- ------------ --------------- ---------- ------------- db_hash_slot 896 192708 28684 14.8847 8164589 81.4562 flu_pulsedb 6 21477 8561 39.8612 2118861 21.1394 run_queue 42 6427021 169979 2.6448 339177 3.3839 pix_lock 1024 4384 229 5.2235 209468 2.0898 pollset 1 303839 16477 5.4229 134808 1.3449 db_tab 146 261929 926 0.3535 107955 1.0770 Here I see two problems: lots of ETS locking (db_hash_slot) and something with a particular process (flu_pulsedb). Inspecting the process gives me lock id #tries #collisions collisions [%] time [us] duration [%] histogram [log2(us)] ----- --- ------- ------------ --------------- ---------- ------------- --------------------- flu_pulsedb proc_main 6989 4242 60.6954 1161759 11.5906 | ...........XX... | flu_pulsedb proc_msgq 7934 3754 47.3154 669383 6.6783 | .xXXxxXxx..xXX... | flu_pulsedb proc_status 5489 521 9.4917 287153 2.8649 | ..xxxxxxxxxxxXX.. | flu_pulsedb proc_link 864 44 5.0926 566 0.0056 | ...XxX..... | flu_pulsedb proc_btm 0 0 0.0000 0 0.0000 | | flu_pulsedb proc_trace 201 0 0.0000 0 0.0000 | | Context: this process is a data receiver. Each sender first checks its message queue length and then sends a message if queue is not very long. This happens about 220 times a second. Then this process accumulates some data and writes it to disk periodically. What do proc_main, proc_msgq and proc_status locks mean? Why at all are collisions possible here? What should I see next to optimise this bottleneck? Next, inspecting db_hash_slot gives me 20 rows all alike (only top few shown): lock id #tries #collisions collisions [%] time [us] duration [%] histogram [log2(us)] ----- --- ------- ------------ --------------- ---------- ------------- --------------------- db_hash_slot 0 492 299 60.7724 107552 1.0730 | ...XX. . | db_hash_slot 1 492 287 58.3333 101951 1.0171 | . ..XX. . | db_hash_slot 48 480 248 51.6667 99486 0.9925 | ...xXx. | db_hash_slot 47 480 248 51.6667 96443 0.9622 | ...XXx | db_hash_slot 2 574 304 52.9617 92952 0.9274 | . ....XX. . | How do I see what ETS tables are causing this high collision rate? Is there any way to map lock id (here: 0, 1, 48, 47, 2) to a table id? Or maybe there is a better tool for ETS profiling? -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenji@REDACTED Tue Jul 26 13:01:01 2016 From: kenji@REDACTED (Kenji Rikitake) Date: Tue, 26 Jul 2016 20:01:01 +0900 Subject: [erlang-questions] file:sendfile/2 benchmarks? Message-ID: I am experimenting new sendfile(2) options for FreeBSD 11 on OTP 19.0.2. While the new options do not seem to break the function, I cannot measure the possible performance gain yet. Is there any good benchmark tool for measuring file:sendfile/2 performance? (I think Yaws uses Erlang VM file:sendfile/2 already) Note that Tuncer Ayaz gives the original idea on an earlier implementation of OTP as: https://github.com/tuncer/sendfile Nginx's summary for FreeBSD 11 sendfile(2) features (with Netflix): https://www.nginx.com/blog/nginx-and-netflix-contribute-new-sendfile2-to-freebsd/ And for those who want to test my patch, it's there at: https://github.com/erlang/otp/compare/maint...jj1bdx:jj1bdx-19.0.2-freebsd-sendfile Regards, Kenji Rikitake -------------- next part -------------- An HTML attachment was scrubbed... URL: From Eric.desCourtis@REDACTED Tue Jul 26 18:11:45 2016 From: Eric.desCourtis@REDACTED (Eric des Courtis) Date: Tue, 26 Jul 2016 12:11:45 -0400 Subject: [erlang-questions] comparison with opensaf In-Reply-To: References: Message-ID: ?There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult.? ? C.A.R. Hoare On Tue, Jul 26, 2016 at 4:04 AM, Ola Andersson A < ola.a.andersson@REDACTED> wrote: > No, not really equivalent. IMM provides much more functionality than > mnesia. It?s not just a data store. > > IMM consist of two parts; OM and OI. OM is the Object Management API. The > OI part is a set of callbacks that lets the user provide an > ObjectImplementer for an object or class of objects. > > IMM has both persistent configuration objects and dynamic runtime objects. > Runtime object may also be cached. > > Configuration objects may contain runtime attributes and the other way > around. Runtime objects and attributes are owned by the ObjectImplementer. > > For configuration objects an OI provides validation functionality with the > possibility to reject a so called CCB (Configuration Change Bundle). > > > > This is just the basics of IMM. There is a lot more available. OpenSAF has > also extended the SAF IMM specification with additional functionality. > > > > *From:* Sashan Govender [mailto:sashang@REDACTED] > *Sent:* den 26 juli 2016 05:11 > *To:* Ola Andersson A ; Eric des Courtis < > Eric.desCourtis@REDACTED> > > *Cc:* Erlang > *Subject:* Re: [erlang-questions] comparison with opensaf > > > > Regarding IMM would it be correct to say it's roughly equivalent to > mnesia? I'm just trying to draw up rough equivalences between the OpenSAF > services and those in the Erlang/OTP. From what I have read about mnesia it > provides a persistant data store that tries to be highly consistent. From > http://learnyousomeerlang.com/mnesia it says 'Mnesia sits on the CP > side'. IMM has the same property. In fact it tries very hard to be > consistent and sacrifices availabilty by not allowing writes for several > minutes in some cases while it sorts itself out. It's typical usage > scenario is in storing the configuration of a cluster, which I'm guessing > mnesia can do as well. > > > > On Mon, Jul 25, 2016 at 11:42 PM Ola Andersson A < > ola.a.andersson@REDACTED> wrote: > > I have implemented parts of the SAF specifications in Erlang. IMM is > definitely one of the most ridiculously overcomplicated specifications I > have seen. > My hat off to the developers of OpenSAF for successfully creating a > working implementation in C. It's hard enough to do it in Erlang but it > must have been a nightmare developing in C. I haven't checked how many LOC > it is but I'm guessing it's a lot. > > > > -----Original Message----- > > From: erlang-questions-bounces@REDACTED [mailto:erlang-questions- > > bounces@REDACTED] On Behalf Of Sashan Govender > > Sent: den 24 juli 2016 06:38 > > To: Eric des Courtis > > Cc: Erlang > > Subject: Re: [erlang-questions] comparison with opensaf > > > > I couldn't agree more. The amount of documentation doesn't correlate with > > quality, and yes I think OpenSAF is over engineered, to say the least. > > > > On Sun, Jul 24, 2016 at 12:10 PM, Eric des Courtis > > wrote: > > > Personally I think it's complicated for nothing. The amount of > > > documentation has nothing to do with how well it is documented. If you > > > have to read 500 pages to understand what a supervisor does that seems > > > like massive over engineering. > > > > > > > > > On Fri, Jul 22, 2016, 8:46 PM Sashan Govender > > wrote: > > >> > > >> Not sure which documentation set you saw but the one over here looks > > >> quite detailed. The AMF documentation alone is 500 pages. > > >> > > >> http://devel.opensaf.org/SAI-AIS-AMF-B.04.01.AL.pdf > > >> > > >> AMF as far as I can tell is like the OTP supervisor process. > > >> > > >> In terms of industry applications I know it's used in Ericsson's DSC > > >> (Diameter Signalling Controller). > > >> > > >> On Sat, Jul 23, 2016 at 12:58 AM, Eric des Courtis > > >> wrote: > > >> > This is a very hard question to answer because I would say OpenSAF > > >> > is rather obscure. But I can tell you that while there are some > > >> > overlaps between the two technologies particularly when it comes > > >> > design patterns. The fact that Erlang is done entirely from the > > >> > ground up (language, runtime, otp patterns > > >> > etc...) specifically for high availability means that you should in > > >> > theory experience much less friction when designing this sort of > > >> > system in Erlang. > > >> > > > >> > My feeling from looking at the documentation is that this isn't > > >> > well documented. In short I wouldn't consider is competition to > > >> > Erlang in it's current state for any project. > > >> > > > >> > On Thu, Jul 21, 2016 at 9:42 PM, Sashan Govender > > >> > > > >> > wrote: > > >> >> > > >> >> Hi > > >> >> > > >> >> I'm interested if anyone has done a comparison the Erlang OTP and > > >> >> OpenSAF. > > >> >> It seems to me there is a significant amount of overlap in the but > > >> >> I don't have the expertise in both systems to form a comprehensive > > >> >> picture. As far as I can tell mnesia is like OpenSAF IMM. They can > > >> >> both be used as configuration databases for a cluster. They both > > >> >> are strongly consistent. I know IMM certainly favours consistency > > >> >> over availabilty. OTP is obviously in Erlang while OpenSAF is in > > >> >> C/C++. Any comparisons out there about these two systems? Or am I > > >> >> wildly off the mark thinking that they are similar? > > >> >> > > >> >> _______________________________________________ > > >> >> erlang-questions mailing list > > >> >> erlang-questions@REDACTED > > >> >> http://erlang.org/mailman/listinfo/erlang-questions > > >> >> > > >> > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From arshadansari27@REDACTED Wed Jul 27 10:00:42 2016 From: arshadansari27@REDACTED (Arshad Ansari) Date: Wed, 27 Jul 2016 08:00:42 +0000 Subject: [erlang-questions] Erlang-diameter: Issue when trying to add multiple servers to a relay Message-ID: Hello there, Firstly, I've just picked up erlang this week and I'm suppose to evaluate Erlang-Diameter for my company (I'm even new to telecom domain and hence to diameter protocol). So, I'm a complete noob in this area. I'm using the example provided by OTP in lib/diameter/examples/code folder. I'm using the relay (listening on 3868) code given there to connect to the two instance of server that I have seperately started on 3871 and 3872. When I connect to these two servers, the connection is established with both but the Peer Up in relay_cb.erl is called only for the first server I tried to connect to. Hence forth, all the requests from clients are relayed to this single Peer. I'm trying to load balance the requests and therefore sharing client connections across multiple server. I would like and appreciate some help in that direction. I haven't changed much from the example directory. I'm just listing the params I'm using for connection as shown below: Server uses the following config: Server 1: [{'Origin-Host', "3871.example.com"}, {'Origin-Realm', "example.com"}, {'Vendor-Id', 193}, {'Product-Name', "3871-Server-Erlang"}, {'Auth-Application-Id', [0]}, {restrict_connections, false}, {string_decode, false}, {application, [{alias, common}, {dictionary, diameter_gen_base_rfc6733}, {module, server_cb}]}]). Server 2: [{'Origin-Host', "3872.example.com"}, {'Origin-Realm', "example.com"}, {'Vendor-Id', 193}, {'Product-Name', "3872-Server-Erlang"}, {'Auth-Application-Id', [0]}, {restrict_connections, false}, {string_decode, false}, {application, [{alias, common}, {dictionary, diameter_gen_base_rfc6733}, {module, server_cb}]}]). Relay: [{'Origin-Host', "arshad.example.com"}, {'Origin-Realm', "example.com"}, {'Vendor-Id', 193}, {'Product-Name', "RelayAgent"}, {'Auth-Application-Id', [?DIAMETER_APP_ID_RELAY]}, {string_decode, false}, {application, [{alias, relay}, {dictionary, ?DIAMETER_DICT_RELAY}, {module, relay_cb}]}]). -------------- next part -------------- An HTML attachment was scrubbed... URL: From zkessin@REDACTED Wed Jul 27 13:40:30 2016 From: zkessin@REDACTED (Zachary Kessin) Date: Wed, 27 Jul 2016 14:40:30 +0300 Subject: [erlang-questions] Connecting to an exisitg release Message-ID: I am kind of confused I created a release with rebar3 and started it up with the script that it provided me. But how do I connect with it? If I start up a process with the scripts generated by a release appname/bin/appname start it runs in the background. If I then connect to it with appname/bin/appname attach it seems that when I kill that session it will also kill the main process. What is the correct way to do this? Can we put a best practice here into the rebar3 docs Zach -- Zach Kessin Twitter: @zkessin Skype: zachkessin ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From swvist@REDACTED Wed Jul 27 14:20:17 2016 From: swvist@REDACTED (Vipin Nair) Date: Wed, 27 Jul 2016 17:50:17 +0530 Subject: [erlang-questions] Connecting to an exisitg release In-Reply-To: References: Message-ID: > > I created a release with rebar3 and started it up with the script that it > provided me. But how do I connect with it? > > appname/bin/appname remote_console will give you a remote shell. -- Regards, Vipin -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.widgren@REDACTED Wed Jul 27 14:27:52 2016 From: daniel.widgren@REDACTED (daniel.widgren@REDACTED) Date: Wed, 27 Jul 2016 12:27:52 +0000 (UTC) Subject: [erlang-questions] Connecting to an exisitg release In-Reply-To: References: Message-ID: Or after doing a attach do a ctrl-d and you will logout from that shell. Instead of using ctrl-c that will kill the node. Get Outlook for Android On Wed, Jul 27, 2016 at 2:20 PM +0200, "Vipin Nair" wrote: I created a release with rebar3 and started it up with the script that it provided me. But how do I connect with it? appname/bin/appname remote_console will give you a remote shell. -- Regards, Vipin -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Wed Jul 27 15:45:44 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 27 Jul 2016 15:45:44 +0200 Subject: [erlang-questions] gen_statem Auto Callback Mode In-Reply-To: <20160726080024.GA36806@erix.ericsson.se> References: <20160726080024.GA36806@erix.ericsson.se> Message-ID: <20160727134544.GB36806@erix.ericsson.se> On Tue, Jul 26, 2016 at 10:00:24AM +0200, Raimo Niskanen wrote: > Hello list! > > I have created a pull request for changing the new experimental behaviour > gen_statem to automatically select callback mode or as I now call it > callback flavour: > > https://github.com/erlang/otp/pull/1133 And here is an alternative pull request using Module:callback_mode/0 to select callback mode: https://github.com/erlang/otp/pull/1135 -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From kostis@REDACTED Thu Jul 28 07:25:17 2016 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 28 Jul 2016 08:25:17 +0300 Subject: [erlang-questions] is it possible to reuse a "-type" specification for a function in a "-spec" ? In-Reply-To: <1A2D1401-1310-484A-8329-E6294000834C@laposte.net> References: <1A2D1401-1310-484A-8329-E6294000834C@laposte.net> Message-ID: <1277fc8c-676c-7ce8-58df-89b76cfb2301@cs.ntua.gr> On 07/20/2016 10:21 PM, Marco Molteni wrote: > Hello, > > say I have a function that takes a function as an argument and I want > to write type specifications for them: > > -type decider() :: fun((pos_integer(), #node{}) -> > reject | accept | continue). > > -spec foo(Decider :: decider()) -> boolean(). > foo(Decider) -> ... > > When implementing the callback, I would like to reuse the type decider(): > > Instead of duplicating the same type information (error prone, can go > out of sync): > > -spec my_cb(pos_integer(), #node{}) -> reject | accept | continue. > my_cb(X, Tree) -> ... > > I would like to write something like: > > -spec my_cb :: decider(). > my_cb(X, Tree) -> ... > > Is there a way? Yes, an ugly one, using macros. I would not recommend using that option though. Kostis PS. Some of the duplication in your example, can be avoided in a nice way by giving names to types. For example, you can define: -type node_rec() :: #node{}. -type decider_ret() :: reject | accept | continue. -type decider_fun() :: fun((pos_integer(),node_rec()) -> decider_ret()). From jj@REDACTED Thu Jul 28 07:47:37 2016 From: jj@REDACTED (Giovanni Giorgi) Date: Thu, 28 Jul 2016 07:47:37 +0200 Subject: [erlang-questions] Beginner gen_server and child monitoring Message-ID: Hi all, I am an erlang newbies. I am developing a small stock exchange monitoring application, after reading Francesco Cesarini Erlang book and wpriting some small libraries: https://github.com/daitangio/erprice I have a gen_server which take a monitor request, in the form {StockTicker, Market, droplimitvalue}. Then the gen_server spawn a child process which track the price, sleeping at some interval. The child process is monitored. The child process die when the stock goes below the price drop but before that sends a notification to its gen_server Now the question is: Is a correct approach? What is the best way of spawn-ing child which must loop and loop? I think the child cannot be a gen_server because its interface did not seem to be thinked for continuing monitoring process. There is some idiomatic behavior must I implement? I can spawn_monitot child by hand? It is a good strategy? -- Giovanni Giorgi jj@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Thu Jul 28 08:12:47 2016 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 28 Jul 2016 09:12:47 +0300 Subject: [erlang-questions] hipe - erllvm: finished, quiet, or other? In-Reply-To: References: Message-ID: <35bc0201-3f19-3d64-a18d-37e3b1e4d5b9@cs.ntua.gr> On 07/15/2016 07:09 PM, Felix Gallo wrote: > > Both HiPE and erllvm seem pretty quiet these days; there's commentary > and work in mainline otp suggesting that erllvm is still under some kind > of development, but it's been a few years since the main erllvm page has > been updated, and I'm having difficulty getting a recent llvm to work > (which is not unusual, given that each llvm release involves breaking > changes, and they move pretty rapidly). > > Where's the bleeding edge on this? First of all, just in case you did not know it, note that ErLLVM has been fully included in Erlang/OTP for a couple of years now, as an alternative backend to the native code compiler. You can use it simply by specifying the compiler option {hipe, [to_llvm]} to the list of compiler options of the BEAM compiler. However, the performance you can expect to get is not very much different than that of the vanilla HiPE compiler, and given that the compilation times are higher, there is probably not very much reason to prefer it over HiPE unless you really have some code that the LLVM compiler is good in optimizing. Both HiPE and ErLLVM are quite stable these days. There is some development still going on, for example there have been many pull requests that made it into 19 and some that are still cooking on github, but the level of activity is not so high these days. Since the inclusion of ErLLVM in Erlang/OTP, we have not felt the need to update the erllvm page. Also, unfortunately, we have not really been following the development of LLVM and consequently the erllvm compiler in Erlang/OTP releases in 18.x and prior do not work with recent LLVM versions. But there has been quite a lot of activity during the first half of 2016 and, as far as we know, on x86_84, the ErLLVM code in 19.0 works OK with all LLVM versions >= 3.5, including the most recent one(s). Please send a report if you have trouble using HiPE on some platform or the ErLLVM in 19 on x86_64. It will be news to us. (There are known issues with ErLLVM on x86, but these are due to bugs of LLVM.) Anyway, the bottomline is that we are still doing our best to support HiPE and ErLLVM by fixing bugs and doing performance improvements as we discover them, But I would classify HiPE and ErLLVM in 19.0 as usable. Kostis PS. Apologies for the slow response; it's vacation time... From arshadansari27@REDACTED Thu Jul 28 08:35:46 2016 From: arshadansari27@REDACTED (Arshad Ansari) Date: Thu, 28 Jul 2016 06:35:46 +0000 Subject: [erlang-questions] Erlang-diameter: Issue when trying to add multiple servers to a relay In-Reply-To: References: Message-ID: Hello Chandru, Thank you for the reply. Here is what I have done. I'm using the example code given in OTP library available here at https://github.com/erlang/otp/tree/master/lib/diameter/examples/code. The node.erl being common across client.erl, server.erl and relay.erl. However, I have a seperate version of node.erl for each of these systems. My entire code is here: https://github.com/arshadansari27/erlang-diameter-eval The code I'm using the run the client.erl and client_cb.erl is as follows: -module(runner4). -export([run/0, one_client/0, stop/0]). run() -> diameter:start(), client:start(), client:connect(tcp). stop() -> client:stop(), diameter:stop(). one_client() -> client:call(). I'm calling runner4:run() first then runner4:one_client() and finally runner4:stop() so I can see tcp dumps and all after each and every step. Client connects to Relay and Relay forwards the requests to Server instances and gets a response from there and returns to the client. So that part is working. I was even able to do a little hack in pick_peer as shown below, so I can balance the incoming requests across multiple server instances. In my case, there are only two instances so I'm just selecting in a round robin manner and returning that peer. pick_peer(Peers, _, _SvcName, _State, relayed) -> [{_, Peer_counter}] = ets:lookup(mytable_relay, peers), Peer_count = other_counter:inc(Peer_counter), Index = (Peer_count rem 2) + 1, % List = lists:map(fun(N) -> {Xs, _} = N, Xs end, Peers), Peer = lists:nth(Index, Peers), {ok, Peer}. With this I'm also able to load balance across different server instances from relay at exact 50/50 ration. So I was able to solve much of the original problem I had. The problem that remains is that what I have done is firstly a hack and secondly inexplicable. And what follows is the current problem. When I start relay and then start both the server instances, relay calls peer up for both the servers and the pick_peer's first argument (local nodes) has both the servers as peers. Whereas when I start the server instances first and then relay, then only one time pick_peer is called and with only one of the server instance. Even the tcp dump on wireshark does not show the CER exchange between relay and servers, whereas the CER exchange is there from client to relay. This is what is confusing me. Why would the order of starting the server/relay allow peers to be added and/or not as well as why no CER exchange between relay and server instances? They do have TCP packets exchanged between them, but not the Diameter protocol CER exchange that is present for Client-relay communication. Given below is the code that starts the relay and connects it to both the server instances that are running on the same machine. % runner.erl -module(runner). -export([run/0]). run() -> diameter:start(), relay:start(), relay:connect(tcp, 1, true), relay:connect(tcp, 2, true), relay:listen(tcp). % relay.erl -module(relay). -include_lib("diameter/include/diameter.hrl"). -include_lib("diameter/include/diameter_gen_base_rfc3588.hrl"). -include_lib("diameter/include/diameter_gen_relay.hrl"). -export([start/1, start/2, listen/2, connect/2, connect/3, stop/1, handle_info/2]). -export([start/0, listen/1, connect/1, stop/0]). -define(DEF_SVC_NAME, ?MODULE). %% The service configuration. -define(SERVICE(Name), [{'Origin-Host', "pcrf.relay.com"}, {'Origin-Realm', "relay.com"}, {'Vendor-Id', 193}, {'Product-Name', "RelayAgent"}, {'Auth-Application-Id', [?DIAMETER_APP_ID_RELAY]}, % 16#FFFFFFFF {string_decode, false}, {use_shared_peers, true}, {application, [{alias, relay}, {dictionary, ?DIAMETER_DICT_RELAY}, % diameter_gen_relay {module, relay_cb}]}, {share_peers, true} ]). %% SKIPPING the code that is same as the original one given on that link.. connect(T, SName, Multi) when Multi == true -> node:connect(?DEF_SVC_NAME, T, SName). % node.erl %% SKIPPING the code that is same as the original one given on that link.. -define(RELAY_PORT, 3868). -define(SERVER_PORT1, 3871). -define(SERVER_PORT2, 3872). %% connect/2 -spec connect(diameter:service_name(), client_opts()) -> {ok, diameter:transport_ref()} | {error, term()}. connect(Name, Opts) when is_list(Opts) -> io:format("Connection Options ~p\n", [{connect, Opts}]), diameter:add_transport(Name, {connect, Opts}); connect(Name, {T, Opts}) -> connect(Name, Opts ++ client_opts(T)); connect(Name, T) -> connect(Name, [{connect_timer, 5000} | client_opts(T)]). connect(Name, T, SName) -> connect(Name, [{connect_timer, 5000} | client_opts({T, SName ,true})]). client_opts({T, LA, RA, RP}) when T == all; %% backwards compatibility T == any -> [[S, {C,Os}], T] = [client_opts({P, LA, RA, RP}) || P <- [sctp,tcp]], [S, {C,Os,2000} | T]; client_opts({T, LA, RA, RP}) -> [{transport_module, tmod(T)}, {transport_config, [{raddr, addr(RA)}, {rport, RP}, {reuseaddr, true} | ip(LA)]}]; client_opts({T, S_Port, Multi}) when Multi == true -> Port = case S_Port of 1 -> ?SERVER_PORT1; 2 -> ?SERVER_PORT2 end, client_opts({T, loopback, remotelocal, Port}); client_opts({T, RA, RP}) -> client_opts({T, default, RA, RP}); addr(remotelocal) -> {172,16,101,146}; ip(remotelocal) -> {172,16,101,146}; REALM INFO ------------------ client: Origin-Host: first.client.com Origin-Realm: client.com Relay: Origin-Host: pcrf.realy.com Origin-Realm: realy.com Server 1: Origin-Host: 3671.server.com Origin-Realm: server.com Server 2: Origin-Host: 3672.server.com Origin-Realm: server.com I hope that covers everything. Any help or direction is very much appreciated. Regards, Arshad On 28-Jul-2016 2:56 AM, "Chandru" wrote: > Hi Arshad, > > First, some general advice. I've used the Erlang diameter stack quite > extensively to build telecom applications and it is pretty rock solid. I > wouldn't hesitate to recommend using it. There are several mission critical > services built using this stack running in production at EE (largest mobile > operator) in the UK. > > Re the specific problem you are facing, how are you generating the > requests on the client side? Some packet captures or the client code would > be useful to see. > > cheers, > Chandru > > On 27 July 2016 at 09:00, Arshad Ansari wrote: > >> Hello there, >> >> Firstly, I've just picked up erlang this week and I'm suppose to evaluate >> Erlang-Diameter for my company (I'm even new to telecom domain and hence to >> diameter protocol). So, I'm a complete noob in this area. I'm using the >> example provided by OTP in lib/diameter/examples/code folder. I'm using the >> relay (listening on 3868) code given there to connect to the two instance >> of server that I have seperately started on 3871 and 3872. When I connect >> to these two servers, the connection is established with both but the Peer >> Up in relay_cb.erl is called only for the first server I tried to connect >> to. Hence forth, all the requests from clients are relayed to this single >> Peer. I'm trying to load balance the requests and therefore sharing client >> connections across multiple server. I would like and appreciate some help >> in that direction. I haven't changed much from the example directory. I'm >> just listing the params I'm using for connection as shown below: >> >> Server uses the following config: >> Server 1: >> [{'Origin-Host', "3871.example.com"}, >> {'Origin-Realm', "example.com"}, >> {'Vendor-Id', 193}, >> {'Product-Name', "3871-Server-Erlang"}, >> {'Auth-Application-Id', [0]}, >> {restrict_connections, false}, >> {string_decode, false}, >> {application, [{alias, common}, >> {dictionary, >> diameter_gen_base_rfc6733}, >> {module, server_cb}]}]). >> >> Server 2: >> [{'Origin-Host', "3872.example.com"}, >> {'Origin-Realm', "example.com"}, >> {'Vendor-Id', 193}, >> {'Product-Name', "3872-Server-Erlang"}, >> {'Auth-Application-Id', [0]}, >> {restrict_connections, false}, >> {string_decode, false}, >> {application, [{alias, common}, >> {dictionary, >> diameter_gen_base_rfc6733}, >> {module, server_cb}]}]). >> >> Relay: >> [{'Origin-Host', "arshad.example.com"}, >> {'Origin-Realm', "example.com"}, >> {'Vendor-Id', 193}, >> {'Product-Name', "RelayAgent"}, >> {'Auth-Application-Id', >> [?DIAMETER_APP_ID_RELAY]}, >> {string_decode, false}, >> {application, [{alias, relay}, >> {dictionary, >> ?DIAMETER_DICT_RELAY}, >> {module, relay_cb}]}]). >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Thu Jul 28 17:20:36 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Thu, 28 Jul 2016 08:20:36 -0700 Subject: [erlang-questions] hipe - erllvm: finished, quiet, or other? In-Reply-To: <35bc0201-3f19-3d64-a18d-37e3b1e4d5b9@cs.ntua.gr> References: <35bc0201-3f19-3d64-a18d-37e3b1e4d5b9@cs.ntua.gr> Message-ID: Kostis, thanks for taking time from your vacation to answer. :) I'm glad to hear erllvm is still alive and kicking. I did have trouble building it on a few platforms with various llvm releases; I'll see if I can repeat and distill the issues when I get a chance. F. On Wed, Jul 27, 2016 at 11:12 PM, Kostis Sagonas wrote: > On 07/15/2016 07:09 PM, Felix Gallo wrote: > >> >> Both HiPE and erllvm seem pretty quiet these days; there's commentary >> and work in mainline otp suggesting that erllvm is still under some kind >> of development, but it's been a few years since the main erllvm page has >> been updated, and I'm having difficulty getting a recent llvm to work >> (which is not unusual, given that each llvm release involves breaking >> changes, and they move pretty rapidly). >> >> Where's the bleeding edge on this? >> > > First of all, just in case you did not know it, note that ErLLVM has been > fully included in Erlang/OTP for a couple of years now, as an alternative > backend to the native code compiler. You can use it simply by specifying > the compiler option {hipe, [to_llvm]} to the list of compiler options of > the BEAM compiler. However, the performance you can expect to get is not > very much different than that of the vanilla HiPE compiler, and given that > the compilation times are higher, there is probably not very much reason to > prefer it over HiPE unless you really have some code that the LLVM compiler > is good in optimizing. > > Both HiPE and ErLLVM are quite stable these days. There is some > development still going on, for example there have been many pull requests > that made it into 19 and some that are still cooking on github, but the > level of activity is not so high these days. > > Since the inclusion of ErLLVM in Erlang/OTP, we have not felt the need to > update the erllvm page. Also, unfortunately, we have not really been > following the development of LLVM and consequently the erllvm compiler in > Erlang/OTP releases in 18.x and prior do not work with recent LLVM > versions. But there has been quite a lot of activity during the first half > of 2016 and, as far as we know, on x86_84, the ErLLVM code in 19.0 works OK > with all LLVM versions >= 3.5, including the most recent one(s). Please > send a report if you have trouble using HiPE on some platform or the ErLLVM > in 19 on x86_64. It will be news to us. (There are known issues with > ErLLVM on x86, but these are due to bugs of LLVM.) > > Anyway, the bottomline is that we are still doing our best to support HiPE > and ErLLVM by fixing bugs and doing performance improvements as we discover > them, But I would classify HiPE and ErLLVM in 19.0 as usable. > > Kostis > > PS. Apologies for the slow response; it's vacation time... > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marco.molteni@REDACTED Thu Jul 28 21:35:21 2016 From: marco.molteni@REDACTED (Marco Molteni) Date: Thu, 28 Jul 2016 21:35:21 +0200 Subject: [erlang-questions] is it possible to reuse a "-type" specification for a function in a "-spec" ? In-Reply-To: <1277fc8c-676c-7ce8-58df-89b76cfb2301@cs.ntua.gr> References: <1A2D1401-1310-484A-8329-E6294000834C@laposte.net> <1277fc8c-676c-7ce8-58df-89b76cfb2301@cs.ntua.gr> Message-ID: <063BFD5C-8C91-4905-9174-3414FDBAF7B8@laposte.net> > On 28 Jul 2016, at 07:25, Kostis Sagonas wrote: > > On 07/20/2016 10:21 PM, Marco Molteni wrote: [..] >> I would like to write something like: >> >> -spec my_cb :: decider(). >> my_cb(X, Tree) -> ... >> >> Is there a way? > > Yes, an ugly one, using macros. > > I would not recommend using that option though. Ah, I see, and I see why you don't recommend it... > Kostis > > PS. Some of the duplication in your example, can be avoided in a nice way by giving names to types. For example, you can define: > > -type node_rec() :: #node{}. > -type decider_ret() :: reject | accept | continue. > -type decider_fun() :: fun((pos_integer(),node_rec()) -> decider_ret()). I though of something along these lines. So this is the best that can be done, OK. thanks for your answer! marco.m From community-manager@REDACTED Fri Jul 29 19:01:39 2016 From: community-manager@REDACTED (Bruce Yinhe) Date: Fri, 29 Jul 2016 19:01:39 +0200 Subject: [erlang-questions] Erlang job openings on erlangcentral.org/jobs Message-ID: Subscribe to weekly Erlang job updates: https://erlangcentral.org/login/?action=register Publish an Erlang job opening: https://erlangcentral.org/jobs/add Backend software engineer - Electic Imp - Any (remote); London, UK preferred, UK http://erlangcentral.org/backend-software-engineer-electric-imp/ Apply before: 2016-07-30 Platform and Data Engineer - Handy's culture deck - New York City, USA http://erlangcentral.org/platform-and-data-engineer-handy/ Apply before: 2016-07-30 Senior Erlang Developer - Darwin Recruitment - London, UK http://erlangcentral.org/senior-erlang-developer-darwin-recruitment/ Apply before: 2016-07-30 Backend Engineer - NPL - San Francisco, CA, USA http://erlangcentral.org/backend-engineer-netpowerandlight/ Apply before: 2016-08-30 Director Of Engineering/Lead Developer ? Elixir - Curadora - Philadelphia, PA, USA http://erlangcentral.org/director-of-engineeringlead-developer-elixir-curadora/ Apply before: 2016-08-30 Erlang Software Engineer - Cisco - Stockholm, Swedem http://erlangcentral.org/erlang-software-engineer-cisco/ Apply before: 2016-08-30 Online Services Programmer - Creative Assembly - , UK http://erlangcentral.org/online-services-programmer-creative-assembly/ Apply before: 2016-08-30 Senior Elixir Engineer - HelloSign - San Francisco, CA, USA http://erlangcentral.org/senior-elixir-engineer-hellosign/ Apply before: 2016-08-30 Erlang Developer - Darwin Recruitment - London, UK http://erlangcentral.org/erlang-developer-darwin/ Apply before: 2016-10-30 Erlang Developer in Test - Darwin Recruitment - BIllericay, Essex, UK http://erlangcentral.org/erlang-developer-in-test-darwin/ Apply before: 2016-10-30 Erlang Developer - Krubera Group - Hertfordshire, UK http://erlangcentral.org/erlang-developer-krubera-group/ Apply before: 2016-11-04 Software Developer ? Erlang - Darwin Recruitment - Cardiff, Wales, UK http://erlangcentral.org/software-developer-erlang-darwin/ Apply before: 2016-11-29 Senior Erlang Developer - Dasudian - Shenzhen, China http://erlangcentral.org/senior-erlang-developer-dasudian-company/ Apply before: 2016-12-11 Senior Erlang Developer - Dasudian - Shenzhen, China http://erlangcentral.org/senior-erlang-developer-dasudian/ Apply before: 2016-12-11 Erlang Software Engineer - F Technologies - Dubai, UAE http://erlangcentral.org/erlang-software-engineer-f-technologies/ Apply before: 2016-12-30 Senior Erlang Developer - Dasudian - Shenzhen, China http://erlangcentral.org/senior-erlang-developer-dasudian-2/ Apply before: 2016-12-30 Back-end Engineer - 2600hz - San Francisco, CA, USA http://erlangcentral.org/back-end-engineer-2600hz/ Apply before: 2017-08-30 -------------- next part -------------- An HTML attachment was scrubbed... URL: From 06jameskingsbery@REDACTED Fri Jul 29 21:15:15 2016 From: 06jameskingsbery@REDACTED (James Kingsbery) Date: Fri, 29 Jul 2016 15:15:15 -0400 Subject: [erlang-questions] The way to backup mnesia data daily In-Reply-To: References: Message-ID: I've followed those instructions before with success. I think you need to give more to go on - is 'web_server@REDACTED' the new node name? When you look at the node's name (eg, as shown in mnesia:info/0) what do you see? I would add that you should look at what your box is doing when you run backups. Depending on what sort of data you have stored, it could use a lot of resources and make the box sluggish or even cause a kernel panic. On Mon, Jul 25, 2016 at 7:38 AM, Yang Zhenguo wrote: > Hi Team, > > Any suggestion on it? > > Thanks, > Zhenguo > > 2016-07-24 22:14 GMT+08:00 Yang Zhenguo : > >> Hi Folks, >> >> Having a question about mnesia backup. >> >> We have an application running based on Erlang Cowboy and Mnesia. I am >> wondering to know what is a good way to backup mnesia data daily, since we >> only have one node. >> >> I followed the steps here >> >> http://stackoverflow.com/questions/463400/how-to-rename-the-node-running-a-mnesia-database >> >> But got the errors below: >> >> {aborted,{'EXIT',{aborted,{bad_commit,{missing_lock,'web_server@REDACTED >> '}}}}??} >> >> The reason I want to change node name is in case I deploy it in another >> machine with different ip address. >> >> May I have your suggestion? Many thanks. >> >> -- >> Zhenguo Yang >> www.prinbit.com >> > > > > -- > Zhenguo Yang > www.prinbit.com > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Sat Jul 30 09:30:01 2016 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 30 Jul 2016 09:30:01 +0200 Subject: [erlang-questions] [ANN] erlang-rand-compat Message-ID: erlang-rand-compat, available at https://github.com/tuncer/erlang-rand-compat The library allows you to generate a compatibility module, exposing the common subset API of `rand` and `random`. It is meant to be used as a migration helper in projects that want to run on Erlang/OTP installations that may or may not have the new `rand` module. It does this by using `rand` if available or falling back to `random` if not. Originally written for Triq and rebar, and now released separately after Kenji Rikitake suggested that other projects might want to reuse it. From arunp@REDACTED Sat Jul 30 16:48:02 2016 From: arunp@REDACTED (ARUN P) Date: Sat, 30 Jul 2016 20:18:02 +0530 Subject: [erlang-questions] rpc call from erlang node to cnode Message-ID: <579CBE22.7080206@utl.in> Hi , can anyone please suggest me is there any possible way to do rpc call from erlang node to cnode. I understand that there is no direct mechanism available to make rpc to cnode, but am curious because of the fact that, am able to make rpc call from cnode to erlang node, then why vice-vers is not possible. Somebody kindly assist me. Thanks in advance. Arun From vladdu55@REDACTED Sun Jul 31 10:16:33 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Sun, 31 Jul 2016 10:16:33 +0200 Subject: [erlang-questions] rpc call from erlang node to cnode In-Reply-To: <579CBE22.7080206@utl.in> References: <579CBE22.7080206@utl.in> Message-ID: Hi Arun, Well, it is possible to do RPC calls in all directions, but what is missing is a framework on the cnode that will handle it. An RPC call is simply a mesage from a node to another. Cnodes are not beam VMs and you would have to implement code that receives the message, interpret it to find out what to call (because cnodes know nothing about Erlang code and libraries), execute the call and return the result as another message. My guess is that cnodes were thought in the beginning to serve as "inputs" in the system, interfacing with the world. This has partially been superseded by NIFs. Why do you need to make RPC calls to a cnode? best regards, Vlad On Sat, Jul 30, 2016 at 4:48 PM, ARUN P wrote: > Hi , > > can anyone please suggest me is there any possible way to do rpc call > from erlang node to cnode. I understand that there is no direct mechanism > available to make rpc to cnode, but am curious because of the fact that, am > able to make rpc call from cnode to erlang node, then why vice-vers is not > possible. Somebody kindly assist me. > > Thanks in advance. > Arun > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Sun Jul 31 11:45:42 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Sun, 31 Jul 2016 11:45:42 +0200 Subject: [erlang-questions] LCNT: understanding proc_* and db_hash_slot collisions In-Reply-To: References: Message-ID: On Tue, Jul 26, 2016 at 12:11 PM, Danil Zagoskin wrote: > Hello! > Hello! > Inspecting the process gives me > lock id #tries #collisions collisions [%] time [us] > duration [%] histogram [log2(us)] > ----- --- ------- ------------ --------------- ---------- > ------------- --------------------- > flu_pulsedb proc_main 6989 4242 60.6954 1161759 > 11.5906 | ...........XX... | > flu_pulsedb proc_msgq 7934 3754 47.3154 669383 > 6.6783 | .xXXxxXxx..xXX... | > flu_pulsedb proc_status 5489 521 9.4917 287153 > 2.8649 | ..xxxxxxxxxxxXX.. | > flu_pulsedb proc_link 864 44 5.0926 566 > 0.0056 | ...XxX..... | > flu_pulsedb proc_btm 0 0 0.0000 0 > 0.0000 | | > flu_pulsedb proc_trace 201 0 0.0000 0 > 0.0000 | | > > Context: this process is a data receiver. Each sender first checks its > message queue length and then sends a message > if queue is not very long. This happens about 220 times a second. Then > this process accumulates some data and > writes it to disk periodically. > What do proc_main, proc_msgq and proc_status locks mean? > proc_main is the main execution lock, proc_msgq is the lock protecting the external message queue, and proc_status is the lock protecting the process status ans various other things. > Why at all are collisions possible here? > When doing various operations, different locks are needed in order to guarantee the order of events. For instance when sending a message the proc_msgq lock is needed. However when checking the size of the message queue both the proc_main, proc_msgq are needed. So if many processes continually check the message queue size of a single other process you will get a lot of conflict on both the main and msgq lock. > What should I see next to optimise this bottleneck? > Don't check the message queue length of another process unless you really have to, and if you do have to, do it as seldom as you can. Checking the length of a message queue is a deceptively expensive operation. > > Next, inspecting db_hash_slot gives me 20 rows all alike (only top few > shown): > lock id #tries #collisions collisions [%] time [us] > duration [%] histogram [log2(us)] > ----- --- ------- ------------ --------------- ---------- > ------------- --------------------- > db_hash_slot 0 492 299 60.7724 107552 > 1.0730 | ...XX. . | > db_hash_slot 1 492 287 58.3333 101951 > 1.0171 | . ..XX. . | > db_hash_slot 48 480 248 51.6667 99486 > 0.9925 | ...xXx. | > db_hash_slot 47 480 248 51.6667 96443 > 0.9622 | ...XXx | > db_hash_slot 2 574 304 52.9617 92952 > 0.9274 | . ....XX. . | > > How do I see what ETS tables are causing this high collision rate? > Is there any way to map lock id (here: 0, 1, 48, 47, 2) to a table id? > iirc the id used in the lock checker should be the same as the table id. > Or maybe there is a better tool for ETS profiling? > for detection of ETS conflicts there is no better tool. For general ETS performance, I would use the same tools as for all erlang code, i.e. cprof, eprof, fprof and friends. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From zkessin@REDACTED Sun Jul 31 14:26:38 2016 From: zkessin@REDACTED (Zachary Kessin) Date: Sun, 31 Jul 2016 15:26:38 +0300 Subject: [erlang-questions] Mnesia Problem Message-ID: Hi All I think I am missing something here. I am setting up a system with 2 nodes app@REDACTED and app@REDACTED I want to have a bunch of mnesia tables that exist on both nodes. I created the tables with the correct nodelist on prod-1, and when I do a table info there it shows a ram copy on both nodes, but when I go to prod-2 the table does not show up. what am I missing here? Zach -- Zach Kessin SquareTarget Twitter: @zkessin Skype: zachkessin ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sun Jul 31 14:37:23 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 31 Jul 2016 15:37:23 +0300 Subject: [erlang-questions] LCNT: understanding proc_* and db_hash_slot collisions In-Reply-To: References: Message-ID: Very nice, so it seems that our measuring process_info(Pid,message_queue_len) is really expensive? -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sun Jul 31 14:38:35 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 31 Jul 2016 15:38:35 +0300 Subject: [erlang-questions] LCNT: understanding proc_* and db_hash_slot collisions In-Reply-To: References: Message-ID: We can do it (measure message_queue_len) about 200 000 times per second. It is bad practice? -------------- next part -------------- An HTML attachment was scrubbed... URL: From matwey.kornilov@REDACTED Sun Jul 31 20:55:49 2016 From: matwey.kornilov@REDACTED (Matwey V. Kornilov) Date: Sun, 31 Jul 2016 21:55:49 +0300 Subject: [erlang-questions] httpc: connection per request Message-ID: Hello, I use httpc in streaming mode to obtain the data from the remote hosts. This data are processed in the special manner. The processing of some stream pairs are depend on each other, i.e. the processing process simultaneously reads both HTTP responses (there is huge amount of data in each response, so I use streaming mode). When the pair is located at the same server I would like to have one connection per request, because I will easily run into dead lock otherwise as the following. The processor read some data from the first response stream and want to read data from the second in order to make some decision and process further. But the second request is awaiting until first response finished inside httpc. But the first response will never be finished because it is stop to read by processor which want second request running in parallel. I would like to have one TCP connection per HTTP request in order not to deadlock. Is {max_keep_alive_length, 1} sufficient to achieve this? From vasdeveloper@REDACTED Sun Jul 31 23:10:02 2016 From: vasdeveloper@REDACTED (Theepan) Date: Mon, 1 Aug 2016 02:40:02 +0530 Subject: [erlang-questions] JSX 2.8.0 Issue Message-ID: Hi There, I encounter an issue with JSX where -- jsx:encode(maps:to_list(#{<<"library">> => <<"jsx">>, <<"awesome">> => true})). -- works both on both OSX and CentOS. But -- jsx:encode(#{<<"library">> => <<"jsx">>, <<"awesome">> => true}). -- works on OSX but not on CentOS. I am using the same JSX version on both the machines, and I am using the example from the JSX GitHub page. Please see the logs below: ON CENTOS --------- (eventcx_server@REDACTED)18> (eventcx_server@REDACTED)18> (eventcx_server@REDACTED)18> (eventcx_server@REDACTED)18> jsx:encode(#{<<"library">> => <<"jsx">>, <<"awesome">> => true}). ** exception error: bad argument in function jsx_parser:value/4 (../src/jsx_parser.erl, line 163) (eventcx_server@REDACTED)19> (eventcx_server@REDACTED)19> (eventcx_server@REDACTED)19> (eventcx_server@REDACTED)19> jsx:encode(maps:to_list(#{<<"library">> => <<"jsx">>, <<"awesome">> => true})). <<"{\"awesome\":true,\"library\":\"jsx\"}">> (eventcx_server@REDACTED)20> (eventcx_server@REDACTED)20> (eventcx_server@REDACTED)20> [Quit] [root@REDACTED eventcx_server]# [root@REDACTED eventcx_server]# uname -a Linux echowave 3.10.0-327.el7.x86_64 #1 SMP Thu Nov 19 22:10:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux [root@REDACTED eventcx_server]# ON OSX ------- 1> 1> 1> jsx:encode(#{<<"library">> => <<"jsx">>, <<"awesome">> => true}). <<"{\"awesome\":true,\"library\":\"jsx\"}">> 2> 2> 2> 2> jsx:encode(maps:to_list(#{<<"library">> => <<"jsx">>, <<"awesome">> => true})). <<"{\"awesome\":true,\"library\":\"jsx\"}">> 3> 3> 3> q(). ok 4> Piriyatheepans-MacBook-Pro:~ piriyatheepan$ Piriyatheepans-MacBook-Pro:~ piriyatheepan$ uname -a Darwin Piriyatheepans-MacBook-Pro.local 15.4.0 Darwin Kernel Version 15.4.0: Fri Feb 26 22:08:05 PST 2016; root:xnu-3248.40.184~3/RELEASE_X86_64 x86_64 Piriyatheepans-MacBook-Pro:~ piriyatheepan$ Piriyatheepans-MacBook-Pro:~ piriyatheepan$ -------------- next part -------------- An HTML attachment was scrubbed... URL: From tty.erlang@REDACTED Sun Jul 31 23:40:02 2016 From: tty.erlang@REDACTED (T Ty) Date: Sun, 31 Jul 2016 22:40:02 +0100 Subject: [erlang-questions] Mnesia Problem In-Reply-To: References: Message-ID: What are the steps you used to setup the tables ? Were both nodes connected before you created the table ? On Sun, Jul 31, 2016 at 1:26 PM, Zachary Kessin wrote: > Hi All > > I think I am missing something here. I am setting up a system with 2 nodes > app@REDACTED and app@REDACTED I want > to have a bunch of mnesia tables that exist on both nodes. > > I created the tables with the correct nodelist on prod-1, and when I do a > table info there it shows a ram copy on both nodes, but when I go to prod-2 > the table does not show up. > > what am I missing here? > > Zach > -- > Zach Kessin > SquareTarget > Twitter: @zkessin > Skype: zachkessin > ? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: