From vinoski@REDACTED Wed Jul 1 03:48:32 2015 From: vinoski@REDACTED (Steve Vinoski) Date: Tue, 30 Jun 2015 21:48:32 -0400 Subject: [erlang-questions] configure http methods in yaws In-Reply-To: References: Message-ID: On Tue, Jun 30, 2015 at 11:09 AM, Bogdan Andu wrote: > > It is intended this behavior and only the headers are read > and only a part of them because neither query params of a GET > are not read fro socket. > For a GET request, query params are part of the requested URL. This means that for a dispatchmod, they're available in the req field of the arg, specifically as part of the request path in the http_request record occupying the arg.req field. Assuming the http_request.path field is an {abs_path, RawPath} tuple, your dispatchmod can get the query params by calling yaws_api:url_decode_q_split(RawPath). which will give you a {Path, QueryData} tuple. If you then set the arg.querydata field to QueryData, you should be able to call yaws_api:parse_query(Arg) to get a list of key/value query param pairs. Keep in mind that a dispatchmod is suitable only for systems that already are mostly capable of handling web requests, such as the Webmachine-Yaws integration I mentioned in my last email. It sounds like dispatchmod is not suitable for your application, given that you keep asking for parsing of query params, handling POST bodies, and other capabilities that Yaws already provides for you in its normal request handling path. The dispatchmod is specifically designed to bypass all that because it's assumed you have other code to handle it for you in that case. > So, basically the body is not read in a dispatchmod, right? > Only a part of the headers. Or all? > For a dispatchmod, the body is not read. All headers are read and stored in Arg. > And then in dispatchmod how do I know there are unparsed headers > or not to know exactly when to parse the body of the request ? > All headers are parsed. If the request has a body, it's ready to be read off the socket by the dispatchmod. I've copied the erlyaws mailing list on this reply, because I really REALLY urge you once again to take this conversation to that list. I don't think erlang-questions is the right place for detailed Yaws questions. --steve > > Thanks, > Bogdan > > > > > On Tue, Jun 30, 2015 at 3:36 PM, Steve Vinoski wrote: > >> >> >> On Tue, Jun 30, 2015 at 6:38 AM, Bogdan Andu wrote: >> >>> I know intercept module does not have clidata populated. >>> >>> I was saying that in dispatch module I want POST data. >>> >>> In a configuration like this: >>> >>> .... >>> >>> port = 8088 >>> listen = 127.0.0.1 >>> listen_backlog = 100 >>> dispatchmod = dispatch_rewrite >>> docroot = /tmp >>> revproxy = / http://127.0.0.1:8080/ intercept_mod intercept_cgi >>> >>> >>> I thought dispatch_rewrite to give me clidata, but clidata for a POST >>> to a cgi script remains undefined. >>> >> >> As its name implies, a dispatch module is useful when you're taking over >> dispatching from Yaws. For example, I've used it for a video delivery >> application and for integrating webmachine and Yaws, the latter because >> webmachine already performs all its own request processing and reply >> delivery. For the dispatchmod case Yaws reads just enough of the request to >> build a minimal #arg{} and assumes the dispatch module will handle the >> rest, including reading additional data from the socket when warranted, so >> if your dispatchmod is expecting POSTs, it will have to handle them itself. >> >> --steve >> >> >> >>> >>> Thanks, >>> Bogdan >>> >>> >>> On Mon, Jun 29, 2015 at 8:48 PM, Steve Vinoski wrote: >>> >>>> >>>> >>>> On Mon, Jun 29, 2015 at 11:10 AM, Bogdan Andu wrote: >>>> >>>>> Thanks for reply. >>>>> >>>>> I tested myself yaws in revproxy and >>>>> I like it. >>>>> >>>>> Although I don't know how to capture POST data >>>>> which should be present in Arg#arg.clidata field >>>>> which is also undefined. >>>>> >>>>> I searched the web and docs and found nothing. >>>>> >>>>> I wrongly assumed that field Arg#arg.querydata hold POST data >>>>> >>>>> I want POST data to apply some regular expression checks >>>>> (like mod_rewrite in Apache) on them >>>>> >>>> >>>> Pretty sure an intercept_mod has access only to information about the >>>> request, and to the headers. The revproxy code uses an internal state >>>> record to track details about POST data and such, but an intercept mod >>>> doesn't have access to that state. You might consider posting an issue to >>>> the yaws github project (https://github.com/klacke/yaws) to see if >>>> this functionality can be added. >>>> >>>> --steve >>>> >>>> >>>> >>>>> Thanks, >>>>> Bogdan >>>>> >>>>> >>>>> On Mon, Jun 29, 2015 at 5:49 PM, Steve Vinoski >>>>> wrote: >>>>> >>>>>> >>>>>> >>>>>> On Sun, Jun 28, 2015 at 11:20 AM, Bogdan Andu >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I know the docs >>>>>>> and I run internally Yaws in reverse proxy mode >>>>>>> and I want this in Internet facing setup also >>>>>>> >>>>>>> I searched the net 'yaws reverse proxy' and I found: >>>>>>> >>>>>>> 1) >>>>>>> http://stackoverflow.com/questions/917546/has-anybody-used-yaws-server-as-an-http-proxy >>>>>>> although 6 years old, is Yaws in reverse proxy mode comparable >>>>>>> or even >>>>>>> better than varnis, haproxy, nginx? >>>>>>> >>>>>> >>>>>> That question/answer is out of date, since revproxy code was largely >>>>>> refactored in the 2012-2014 timeframe. >>>>>> >>>>>> >>>>>>> 2) http://osdir.com/ml/web.server.yaws.general/2007-12/msg00000.html >>>>>>> this one is from klacke and he speaks about difficulties he >>>>>>> encountered writing revproxy engine.as well as an OTP limitation >>>>>>> being the main obstacle in overcoming these. >>>>>>> Are these still apply today? >>>>>>> >>>>>> >>>>>> Probably not, given the rewrite. >>>>>> >>>>>> >>>>>>> 3) also found this: >>>>>>> http://www.erlang-factory.com/upload/presentations/752/reed-efsf2013-whatsapp.pdf >>>>>>> it seems whatsapp useses or used yaws in revproxy mode with some >>>>>>> tweaks >>>>>>> >>>>>>> The setup I want is simple: >>>>>>> >>>>>>> Because I have the applications written in other language >>>>>>> and for rewriting them in Erlang I dont have the time I must >>>>>>> use this setup: >>>>>>> >>>>>>> - Yaws in front-end in revproxy mode with interception module with >>>>>>> plenty of check >>>>>>> on headers, cookies, etc >>>>>>> - Twiggy/Starlet/Starman as back-end or psgi server >>>>>>> more info here: >>>>>>> http://www.slideshare.net/kazeburo/yapc2013psgi-plack >>>>>>> >>>>>>> I want to keepalive connections between yaws and psgi servers to >>>>>>> avoid >>>>>>> 3-way handshake overhead. Although with outside world I want >>>>>>> disable keepalive >>>>>>> and set 'connection: close' and all of thiese can be done by >>>>>>> altering the headers >>>>>>> tru interception module. >>>>>>> >>>>>>> >>>>>>> Is it safe to keepalive connections between Yaws and Twiggy for >>>>>>> example which is an Libevent implementation web server? >>>>>>> Or Starlet/Starman like web server which has a parallel pre-fork >>>>>>> model? >>>>>>> >>>>>> >>>>>> Seems like it should be OK but I've never used the backend servers >>>>>> you're talking about, so I can't say for sure whether it's safe or not. You >>>>>> might try asking on the erlyaws mailing list (see >>>>>> https://lists.sourceforge.net/lists/listinfo/erlyaws-list). >>>>>> >>>>>> --steve >>>>>> >>>>>> >>>>>> >>>>>>> Thanks and sorry for long post, >>>>>>> >>>>>>> Bogdan >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Sun, Jun 28, 2015 at 4:51 AM, Steve Vinoski >>>>>>> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Sat, Jun 27, 2015 at 4:58 AM, Bogdan Andu >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> About yaws as reverse proxy.. >>>>>>>>> >>>>>>>>> I want to use yaws as a reverse proxy in a >>>>>>>>> http -> http setup. no ssl involved whatsoever. >>>>>>>>> >>>>>>>>> I am interested in interception module where I want >>>>>>>>> to apply various checks on headers, query string, etc >>>>>>>>> making this some kind of www firewall . >>>>>>>>> >>>>>>>>> Is this feature of yaws production ready ? >>>>>>>>> >>>>>>>> >>>>>>>> Yes. You can find details about it in chapter 13 of >>>>>>>> http://yaws.hyber.org/yaws.pdf, or under the revproxy section of >>>>>>>> http://yaws.hyber.org/yman.yaws?page=yaws.conf . >>>>>>>> >>>>>>>> --steve >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Bogdan >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Feb 13, 2015 at 6:14 PM, Steve Vinoski >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> I don't recall seeing Yaws users asking for this config feature >>>>>>>>>> in the past, so it's unlikely we'll add it. But what you're asking for -- a >>>>>>>>>> configuration point for methods -- would be implemented much as I've shown >>>>>>>>>> in my previous emails, much like a dispatchmod. The dispatchmod is as early >>>>>>>>>> in the request handling process as you can get after the formation of the >>>>>>>>>> #arg{}. The dispatchmod code I provided requires less configuration than >>>>>>>>>> what you're showing, even for the default case, plus if having to have a >>>>>>>>>> new module concerns you, the dispatch/1 function can be added to some other >>>>>>>>>> existing module you already have instead. >>>>>>>>>> >>>>>>>>>> --steve >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, Feb 13, 2015 at 4:30 AM, Bogdan Andu >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Yes but the point is to have a default configuration that can be >>>>>>>>>>> overridden by such a mechanism >>>>>>>>>>> if one is configured. >>>>>>>>>>> The 99 percent of cases only need a default behaviour. >>>>>>>>>>> >>>>>>>>>>> The way I see this is to have something like that (all in one): >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> mod_405=my_405_handle_module >>>>>>>>>>> .... >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> in this way we can also customize the response if a method other >>>>>>>>>>> than GET or POST is sent to the server >>>>>>>>>>> >>>>>>>>>>> On Fri, Feb 13, 2015 at 10:17 AM, Imants Cekusins < >>>>>>>>>>> imantc@REDACTED> wrote: >>>>>>>>>>> >>>>>>>>>>>> > Traffic with methods not allowed should be discarded with 405 >>>>>>>>>>>> >>>>>>>>>>>> you see, someone else might prefer another action depending on >>>>>>>>>>>> method >>>>>>>>>>>> not allowed. >>>>>>>>>>>> >>>>>>>>>>>> a dedicated attribute may be convenient but then someone would >>>>>>>>>>>> ask: >>>>>>>>>>>> "how do I change the response code? how do I redirect?". Current >>>>>>>>>>>> approach gives you choice. >>>>>>>>>>>> >>>>>>>>>>>> one of those cases when there is more than one approach, a >>>>>>>>>>>> prefers A, >>>>>>>>>>>> b prefers B. Both have a valid point. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bog495@REDACTED Wed Jul 1 09:09:19 2015 From: bog495@REDACTED (Bogdan Andu) Date: Wed, 1 Jul 2015 10:09:19 +0300 Subject: [erlang-questions] configure http methods in yaws In-Reply-To: References: Message-ID: Thanks for detailed explanations and sorry for 'abusing' this list with yaws stuff. Bogdan On Wed, Jul 1, 2015 at 4:48 AM, Steve Vinoski wrote: > > > On Tue, Jun 30, 2015 at 11:09 AM, Bogdan Andu wrote: > >> >> It is intended this behavior and only the headers are read >> and only a part of them because neither query params of a GET >> are not read fro socket. >> > > For a GET request, query params are part of the requested URL. This means > that for a dispatchmod, they're available in the req field of the arg, > specifically as part of the request path in the http_request record > occupying the arg.req field. Assuming the http_request.path field is an > {abs_path, RawPath} tuple, your dispatchmod can get the query params by > calling yaws_api:url_decode_q_split(RawPath). which will give you a {Path, > QueryData} tuple. If you then set the arg.querydata field to QueryData, you > should be able to call yaws_api:parse_query(Arg) to get a list of key/value > query param pairs. > > Keep in mind that a dispatchmod is suitable only for systems that already > are mostly capable of handling web requests, such as the Webmachine-Yaws > integration I mentioned in my last email. It sounds like dispatchmod is not > suitable for your application, given that you keep asking for parsing of > query params, handling POST bodies, and other capabilities that Yaws > already provides for you in its normal request handling path. The > dispatchmod is specifically designed to bypass all that because it's > assumed you have other code to handle it for you in that case. > > >> So, basically the body is not read in a dispatchmod, right? >> Only a part of the headers. Or all? >> > > For a dispatchmod, the body is not read. All headers are read and stored > in Arg. > > >> And then in dispatchmod how do I know there are unparsed headers >> or not to know exactly when to parse the body of the request ? >> > > All headers are parsed. If the request has a body, it's ready to be read > off the socket by the dispatchmod. > > I've copied the erlyaws mailing list on this reply, because I really > REALLY urge you once again to take this conversation to that list. I don't > think erlang-questions is the right place for detailed Yaws questions. > > --steve > > > > > >> >> Thanks, >> Bogdan >> >> >> >> >> On Tue, Jun 30, 2015 at 3:36 PM, Steve Vinoski wrote: >> >>> >>> >>> On Tue, Jun 30, 2015 at 6:38 AM, Bogdan Andu wrote: >>> >>>> I know intercept module does not have clidata populated. >>>> >>>> I was saying that in dispatch module I want POST data. >>>> >>>> In a configuration like this: >>>> >>>> .... >>>> >>>> port = 8088 >>>> listen = 127.0.0.1 >>>> listen_backlog = 100 >>>> dispatchmod = dispatch_rewrite >>>> docroot = /tmp >>>> revproxy = / http://127.0.0.1:8080/ intercept_mod intercept_cgi >>>> >>>> >>>> I thought dispatch_rewrite to give me clidata, but clidata for a POST >>>> to a cgi script remains undefined. >>>> >>> >>> As its name implies, a dispatch module is useful when you're taking over >>> dispatching from Yaws. For example, I've used it for a video delivery >>> application and for integrating webmachine and Yaws, the latter because >>> webmachine already performs all its own request processing and reply >>> delivery. For the dispatchmod case Yaws reads just enough of the request to >>> build a minimal #arg{} and assumes the dispatch module will handle the >>> rest, including reading additional data from the socket when warranted, so >>> if your dispatchmod is expecting POSTs, it will have to handle them itself. >>> >>> --steve >>> >>> >>> >>>> >>>> Thanks, >>>> Bogdan >>>> >>>> >>>> On Mon, Jun 29, 2015 at 8:48 PM, Steve Vinoski >>>> wrote: >>>> >>>>> >>>>> >>>>> On Mon, Jun 29, 2015 at 11:10 AM, Bogdan Andu >>>>> wrote: >>>>> >>>>>> Thanks for reply. >>>>>> >>>>>> I tested myself yaws in revproxy and >>>>>> I like it. >>>>>> >>>>>> Although I don't know how to capture POST data >>>>>> which should be present in Arg#arg.clidata field >>>>>> which is also undefined. >>>>>> >>>>>> I searched the web and docs and found nothing. >>>>>> >>>>>> I wrongly assumed that field Arg#arg.querydata hold POST data >>>>>> >>>>>> I want POST data to apply some regular expression checks >>>>>> (like mod_rewrite in Apache) on them >>>>>> >>>>> >>>>> Pretty sure an intercept_mod has access only to information about the >>>>> request, and to the headers. The revproxy code uses an internal state >>>>> record to track details about POST data and such, but an intercept mod >>>>> doesn't have access to that state. You might consider posting an issue to >>>>> the yaws github project (https://github.com/klacke/yaws) to see if >>>>> this functionality can be added. >>>>> >>>>> --steve >>>>> >>>>> >>>>> >>>>>> Thanks, >>>>>> Bogdan >>>>>> >>>>>> >>>>>> On Mon, Jun 29, 2015 at 5:49 PM, Steve Vinoski >>>>>> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On Sun, Jun 28, 2015 at 11:20 AM, Bogdan Andu >>>>>>> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I know the docs >>>>>>>> and I run internally Yaws in reverse proxy mode >>>>>>>> and I want this in Internet facing setup also >>>>>>>> >>>>>>>> I searched the net 'yaws reverse proxy' and I found: >>>>>>>> >>>>>>>> 1) >>>>>>>> http://stackoverflow.com/questions/917546/has-anybody-used-yaws-server-as-an-http-proxy >>>>>>>> although 6 years old, is Yaws in reverse proxy mode comparable >>>>>>>> or even >>>>>>>> better than varnis, haproxy, nginx? >>>>>>>> >>>>>>> >>>>>>> That question/answer is out of date, since revproxy code was largely >>>>>>> refactored in the 2012-2014 timeframe. >>>>>>> >>>>>>> >>>>>>>> 2) >>>>>>>> http://osdir.com/ml/web.server.yaws.general/2007-12/msg00000.html >>>>>>>> this one is from klacke and he speaks about difficulties he >>>>>>>> encountered writing revproxy engine.as well as an OTP >>>>>>>> limitation >>>>>>>> being the main obstacle in overcoming these. >>>>>>>> Are these still apply today? >>>>>>>> >>>>>>> >>>>>>> Probably not, given the rewrite. >>>>>>> >>>>>>> >>>>>>>> 3) also found this: >>>>>>>> http://www.erlang-factory.com/upload/presentations/752/reed-efsf2013-whatsapp.pdf >>>>>>>> it seems whatsapp useses or used yaws in revproxy mode with >>>>>>>> some tweaks >>>>>>>> >>>>>>>> The setup I want is simple: >>>>>>>> >>>>>>>> Because I have the applications written in other language >>>>>>>> and for rewriting them in Erlang I dont have the time I must >>>>>>>> use this setup: >>>>>>>> >>>>>>>> - Yaws in front-end in revproxy mode with interception module with >>>>>>>> plenty of check >>>>>>>> on headers, cookies, etc >>>>>>>> - Twiggy/Starlet/Starman as back-end or psgi server >>>>>>>> more info here: >>>>>>>> http://www.slideshare.net/kazeburo/yapc2013psgi-plack >>>>>>>> >>>>>>>> I want to keepalive connections between yaws and psgi servers to >>>>>>>> avoid >>>>>>>> 3-way handshake overhead. Although with outside world I want >>>>>>>> disable keepalive >>>>>>>> and set 'connection: close' and all of thiese can be done by >>>>>>>> altering the headers >>>>>>>> tru interception module. >>>>>>>> >>>>>>>> >>>>>>>> Is it safe to keepalive connections between Yaws and Twiggy for >>>>>>>> example which is an Libevent implementation web server? >>>>>>>> Or Starlet/Starman like web server which has a parallel pre-fork >>>>>>>> model? >>>>>>>> >>>>>>> >>>>>>> Seems like it should be OK but I've never used the backend servers >>>>>>> you're talking about, so I can't say for sure whether it's safe or not. You >>>>>>> might try asking on the erlyaws mailing list (see >>>>>>> https://lists.sourceforge.net/lists/listinfo/erlyaws-list). >>>>>>> >>>>>>> --steve >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Thanks and sorry for long post, >>>>>>>> >>>>>>>> Bogdan >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Sun, Jun 28, 2015 at 4:51 AM, Steve Vinoski >>>>>>>> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sat, Jun 27, 2015 at 4:58 AM, Bogdan Andu >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> About yaws as reverse proxy.. >>>>>>>>>> >>>>>>>>>> I want to use yaws as a reverse proxy in a >>>>>>>>>> http -> http setup. no ssl involved whatsoever. >>>>>>>>>> >>>>>>>>>> I am interested in interception module where I want >>>>>>>>>> to apply various checks on headers, query string, etc >>>>>>>>>> making this some kind of www firewall . >>>>>>>>>> >>>>>>>>>> Is this feature of yaws production ready ? >>>>>>>>>> >>>>>>>>> >>>>>>>>> Yes. You can find details about it in chapter 13 of >>>>>>>>> http://yaws.hyber.org/yaws.pdf, or under the revproxy section of >>>>>>>>> http://yaws.hyber.org/yman.yaws?page=yaws.conf . >>>>>>>>> >>>>>>>>> --steve >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Bogdan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, Feb 13, 2015 at 6:14 PM, Steve Vinoski >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> I don't recall seeing Yaws users asking for this config feature >>>>>>>>>>> in the past, so it's unlikely we'll add it. But what you're asking for -- a >>>>>>>>>>> configuration point for methods -- would be implemented much as I've shown >>>>>>>>>>> in my previous emails, much like a dispatchmod. The dispatchmod is as early >>>>>>>>>>> in the request handling process as you can get after the formation of the >>>>>>>>>>> #arg{}. The dispatchmod code I provided requires less configuration than >>>>>>>>>>> what you're showing, even for the default case, plus if having to have a >>>>>>>>>>> new module concerns you, the dispatch/1 function can be added to some other >>>>>>>>>>> existing module you already have instead. >>>>>>>>>>> >>>>>>>>>>> --steve >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Fri, Feb 13, 2015 at 4:30 AM, Bogdan Andu >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Yes but the point is to have a default configuration that can >>>>>>>>>>>> be overridden by such a mechanism >>>>>>>>>>>> if one is configured. >>>>>>>>>>>> The 99 percent of cases only need a default behaviour. >>>>>>>>>>>> >>>>>>>>>>>> The way I see this is to have something like that (all in one): >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> mod_405=my_405_handle_module >>>>>>>>>>>> .... >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> in this way we can also customize the response if a method >>>>>>>>>>>> other than GET or POST is sent to the server >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Feb 13, 2015 at 10:17 AM, Imants Cekusins < >>>>>>>>>>>> imantc@REDACTED> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> > Traffic with methods not allowed should be discarded with 405 >>>>>>>>>>>>> >>>>>>>>>>>>> you see, someone else might prefer another action depending on >>>>>>>>>>>>> method >>>>>>>>>>>>> not allowed. >>>>>>>>>>>>> >>>>>>>>>>>>> a dedicated attribute may be convenient but then someone would >>>>>>>>>>>>> ask: >>>>>>>>>>>>> "how do I change the response code? how do I redirect?". >>>>>>>>>>>>> Current >>>>>>>>>>>>> approach gives you choice. >>>>>>>>>>>>> >>>>>>>>>>>>> one of those cases when there is more than one approach, a >>>>>>>>>>>>> prefers A, >>>>>>>>>>>>> b prefers B. Both have a valid point. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From s.j.thompson@REDACTED Wed Jul 1 15:20:30 2015 From: s.j.thompson@REDACTED (Simon Thompson) Date: Wed, 1 Jul 2015 14:20:30 +0100 Subject: [erlang-questions] Master Classes in Erlang from Joe Armstrong, Francesco Cesarini and Simon Thompson Message-ID: <18F697B0-5CCD-48AF-85BA-8800A1B7609D@kent.ac.uk> I?m delighted to announce that Kent has just released Master Classes in Erlang from Joe Armstrong, Francesco Cesarini and Simon Thompson ? they are freely available from YouTube Link: http://www.cs.kent.ac.uk/ErlangMasterClasses/ News item: http://www.kent.ac.uk/news/kentlife/6083/launch-of-free-online-masterclasses-in-programming We hope that you find them useful. Simon Simon Thompson | Professor of Logic and Computation School of Computing | University of Kent | Canterbury, CT2 7NF, UK s.j.thompson@REDACTED | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt From nmarino@REDACTED Wed Jul 1 16:58:20 2015 From: nmarino@REDACTED (Nick Marino) Date: Wed, 1 Jul 2015 10:58:20 -0400 Subject: [erlang-questions] is there rr() finctionality outside of shell? In-Reply-To: References: Message-ID: The shell uses the io_lib_pretty module to format records, so you could try tapping into that. It's undocumented, so YMMV, stuff could break in future versions, etc. But if you're careful with upgrades and such then you'll probably be okay. More info on how to use io_lib_pretty at these links: http://amiest-devblog.blogspot.com/2008/05/iolibpretty-nice-secret-module.html http://erlang.2086793.n4.nabble.com/Pretty-printing-records-td2110531.html Nick On Sat, Jun 27, 2015 at 5:31 AM, Alex Shneyderman wrote: > Hi, all! > > I would like to nicely format dbg output. I have a formatter function > but io:format() out of there produces plain tuples. Even if I am > running this dbg:tc() code within the shell that saw rr() call > beforehand. > > Is there a lib some place that helps with formatting records nicely, > besides trying to dig it out the shell code? > > Cheers, > Alex. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nico.kruber@REDACTED Thu Jul 2 00:29:39 2015 From: nico.kruber@REDACTED (Nico Kruber) Date: Thu, 02 Jul 2015 00:29:39 +0200 Subject: [erlang-questions] Erlang 18.0, hanging erlang:memory(total)? Message-ID: <1680304.mhPGdp6mty@nico-pc> Hi folks, is it possible that erlang:memory/1 can hang? Today I had a unit test failing in the middle of its execution (it runs the same test over and over again and it was like the 20th time it was run). There was a timeout (>50s) in one process not being able to finish its message handling and erlang:process_info(Pid, current_stacktrace) is repeatedly showing the following stacktrace during these 50s: {current_stacktrace, [{erlang,receive_emd,3,[]}, {erlang,get_mem_data,3,[]}, {erlang,memory,1,[]}, {dht_node_state,details,1, [{file,"src/dht_node_state.erl"},{line,510}]}, {dht_node,on,2,[{file,"src/dht_node.erl"},{line,384}]}, {mockup_dht_node,on,2, [{file,"test/mockup_dht_node.erl"},{line,67}]}, {gen_component,on,3, [{file,"src/gen_component.erl"},{line,614}]}]} The line in the dht_node_state module literally only calls erlang:memory(total) and creates a record with this info. Apparently, it never gets out of erlang:memory/1 :( Thanks for your help Nico -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: This is a digitally signed message part. URL: From ulf@REDACTED Thu Jul 2 11:05:57 2015 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 2 Jul 2015 11:05:57 +0200 Subject: [erlang-questions] is there rr() finctionality outside of shell? In-Reply-To: References: Message-ID: I have made some wrappers around ttb and io_lib_pretty in the locks application. The main purpose is to capture trace from complex multi-node tests, such as 5-node netsplit: https://github.com/uwiger/locks/blob/master/test/locks_leader_tests.erl#L75 If the test goes well, trace is discarded: https://github.com/uwiger/locks/blob/master/test/locks_leader_tests.erl#L115 If it fails, the trace logs are fetched: https://github.com/uwiger/locks/blob/master/test/locks_leader_tests.erl#L109 The trace wrapper itself is in locks_ttb.erl: https://github.com/uwiger/locks/blob/master/src/locks_ttb.erl#L14 TTB uses pretty much the same API as dbg, but adds the ability to merge trace logs from different nodes. To view a pretty-printed merged log, use locks_ttb:format/2 https://github.com/uwiger/locks/blob/master/src/locks_ttb.erl#L38 The pretty-printed output starts with an emacs option header for erlang-mode. To get records nicely formatted in the trace output, I export callback functions from relevant modules, e.g. https://github.com/uwiger/locks/blob/master/src/locks_agent.erl#L122 This is used by locks_ttb:record_print_fun/1 https://github.com/uwiger/locks/blob/master/src/locks_ttb.erl#L101 Wading through reams of trace output is still boring, but at least much more accessible this way. BR, Ulf W > On 01 Jul 2015, at 16:58, Nick Marino wrote: > > The shell uses the io_lib_pretty module to format records, so you could try tapping into that. It's undocumented, so YMMV, stuff could break in future versions, etc. But if you're careful with upgrades and such then you'll probably be okay. > > More info on how to use io_lib_pretty at these links: > http://amiest-devblog.blogspot.com/2008/05/iolibpretty-nice-secret-module.html > http://erlang.2086793.n4.nabble.com/Pretty-printing-records-td2110531.html > > Nick > > On Sat, Jun 27, 2015 at 5:31 AM, Alex Shneyderman > wrote: > Hi, all! > > I would like to nicely format dbg output. I have a formatter function > but io:format() out of there produces plain tuples. Even if I am > running this dbg:tc() code within the shell that saw rr() call > beforehand. > > Is there a lib some place that helps with formatting records nicely, > besides trying to dig it out the shell code? > > Cheers, > Alex. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Thu Jul 2 11:25:31 2015 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Thu, 2 Jul 2015 11:25:31 +0200 Subject: [erlang-questions] is there rr() finctionality outside of shell? In-Reply-To: References: Message-ID: On Thu, Jul 2, 2015 at 11:05 AM, Ulf Wiger wrote: > I have made some wrappers around ttb and io_lib_pretty in the locks > application. > > ... snip ... > > Wading through reams of trace output is still boring, but at least much more > accessible this way. Do you have an example 5-node netsplit trace from a failed test we could take a look at? Motiejus From ulf@REDACTED Thu Jul 2 12:14:00 2015 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 2 Jul 2015 12:14:00 +0200 Subject: [erlang-questions] is there rr() finctionality outside of shell? In-Reply-To: References: Message-ID: Here?s a partial from a failed trace (gist thought the full file was too big): https://gist.github.com/uwiger/48673e05af3085cbc863 The lines that look like: Nodes = [{"7096",locks_1@REDACTED}] map pid ?prefixes? (as in <7096.79.0>) to nodes in order to facilitate reading. The leading numbers (all zeroes in this extract) correspond to milliseconds after trace start. BR, Ulf W > On 02 Jul 2015, at 11:25, Motiejus Jak?tys wrote: > > On Thu, Jul 2, 2015 at 11:05 AM, Ulf Wiger wrote: >> I have made some wrappers around ttb and io_lib_pretty in the locks >> application. >> >> ... snip ... >> >> Wading through reams of trace output is still boring, but at least much more >> accessible this way. > > Do you have an example 5-node netsplit trace from a failed test we > could take a look at? > > Motiejus Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From rickard@REDACTED Thu Jul 2 14:23:01 2015 From: rickard@REDACTED (Rickard Green) Date: Thu, 2 Jul 2015 14:23:01 +0200 Subject: [erlang-questions] Erlang 18.0, hanging erlang:memory(total)? In-Reply-To: <1680304.mhPGdp6mty@nico-pc> References: <1680304.mhPGdp6mty@nico-pc> Message-ID: On Thu, Jul 2, 2015 at 12:29 AM, Nico Kruber wrote: > Hi folks, > is it possible that erlang:memory/1 can hang? > > Today I had a unit test failing in the middle of its execution (it runs the > same test over and over again and it was like the 20th time it was run). There > was a timeout (>50s) in one process not being able to finish its message > handling and erlang:process_info(Pid, current_stacktrace) is repeatedly > showing the following stacktrace during these 50s: > > {current_stacktrace, > [{erlang,receive_emd,3,[]}, > {erlang,get_mem_data,3,[]}, > {erlang,memory,1,[]}, > {dht_node_state,details,1, > [{file,"src/dht_node_state.erl"},{line,510}]}, > {dht_node,on,2,[{file,"src/dht_node.erl"},{line,384}]}, > {mockup_dht_node,on,2, > [{file,"test/mockup_dht_node.erl"},{line,67}]}, > {gen_component,on,3, > [{file,"src/gen_component.erl"},{line,614}]}]} > > The line in the dht_node_state module literally only calls > erlang:memory(total) and creates a record with this info. Apparently, it never > gets out of erlang:memory/1 :( > > > Thanks for your help > Nico > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > The bug fixed in OTP-18.0.1 (and OTP-17.5.6.1) can cause such a hang. Regards, Rickard -- Rickard Green, Erlang/OTP, Ericsson AB From carlsson.richard@REDACTED Thu Jul 2 16:06:04 2015 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 2 Jul 2015 16:06:04 +0200 Subject: [erlang-questions] [ANN] mnesia_ext patch now available Message-ID: The mnesia_ext patch adds functionality to mnesia for registering new table types using arbitrary plug-in backends. This is a much cleaned up and improved version of the earlier "mnesiaex" patch. Most of the work has been done by Ulf Wiger, and we release it with his consent. Pull request here: https://github.com/erlang/otp/pull/783/ The patch is available in all flavours from R15 to 18.0: https://github.com/klarna/otp/tree/OTP-18.0-mnesia_ext https://github.com/klarna/otp/tree/OTP-17.5.6-mnesia_ext https://github.com/klarna/otp/tree/OTP_R16B03-1-mnesia_ext https://github.com/klarna/otp/tree/OTP_R15B03-1-mnesia_ext We have been running the R15 version in production for many months now, without any known issues. /Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Jul 2 22:05:13 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Thu, 02 Jul 2015 22:05:13 +0200 Subject: [erlang-questions] [ANN] Cowboy 1.0.2 Message-ID: <55959979.3030500@ninenines.eu> Just a heads up: Cowboy has been updated to 1.0.2. This release fixes the building of the project for some users (I was fool enough to rely on files from the Internet to build; won't happen again). The following changes from the 1.0 branch are included: https://github.com/ninenines/cowboy/compare/1.0.1...1.0.2 As my time on the computer is limited for a couple weeks, I did not update the changelog yet. I will do so later on. Enjoy! -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From nico.kruber@REDACTED Thu Jul 2 22:42:40 2015 From: nico.kruber@REDACTED (Nico Kruber) Date: Thu, 02 Jul 2015 22:42:40 +0200 Subject: [erlang-questions] Erlang 18.0, hanging erlang:memory(total)? In-Reply-To: References: <1680304.mhPGdp6mty@nico-pc> Message-ID: <2045428.YKYjWFu1Gn@nico-pc> On Thursday 02 Jul 2015 14:23:01 Rickard Green wrote: > On Thu, Jul 2, 2015 at 12:29 AM, Nico Kruber wrote: > > Hi folks, > > is it possible that erlang:memory/1 can hang? > > > > Today I had a unit test failing in the middle of its execution (it runs > > the > > same test over and over again and it was like the 20th time it was run). > > There was a timeout (>50s) in one process not being able to finish its > > message handling and erlang:process_info(Pid, current_stacktrace) is > > repeatedly showing the following stacktrace during these 50s: > > > > {current_stacktrace, > > > > [{erlang,receive_emd,3,[]}, > > > > {erlang,get_mem_data,3,[]}, > > {erlang,memory,1,[]}, > > {dht_node_state,details,1, > > > > [{file,"src/dht_node_state.erl"},{line,510}]}, > > > > {dht_node,on,2,[{file,"src/dht_node.erl"},{line,384}]}, > > {mockup_dht_node,on,2, > > > > [{file,"test/mockup_dht_node.erl"},{line,67}]}, > > > > {gen_component,on,3, > > > > [{file,"src/gen_component.erl"},{line,614}]}]} > > > > The line in the dht_node_state module literally only calls > > erlang:memory(total) and creates a record with this info. Apparently, it > > never gets out of erlang:memory/1 :( > > > > > > Thanks for your help > > Nico > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > The bug fixed in OTP-18.0.1 (and OTP-17.5.6.1) can cause such a hang. > > Regards, > Rickard ok, I'll keep an eye on 18.0.1 on our test systems -> note however, that I encountered the bug not "just after emulator start" as stated in the release log. Also since I ran several tests before the failing one, several calls to erlang:memory/1 succeeded before the hang! Nico -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: This is a digitally signed message part. URL: From sverker.eriksson@REDACTED Fri Jul 3 15:32:52 2015 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Fri, 3 Jul 2015 15:32:52 +0200 Subject: [erlang-questions] Erlang 18.0, hanging erlang:memory(total)? In-Reply-To: <2045428.YKYjWFu1Gn@nico-pc> References: <1680304.mhPGdp6mty@nico-pc> <2045428.YKYjWFu1Gn@nico-pc> Message-ID: <55968F04.70806@ericsson.com> On 07/02/2015 10:42 PM, Nico Kruber wrote: > On Thursday 02 Jul 2015 14:23:01 Rickard Green wrote: >> On Thu, Jul 2, 2015 at 12:29 AM, Nico Kruber wrote: >>> Hi folks, >>> is it possible that erlang:memory/1 can hang? >>> >>> Today I had a unit test failing in the middle of its execution (it runs >>> the >>> same test over and over again and it was like the 20th time it was run). >>> There was a timeout (>50s) in one process not being able to finish its >>> message handling and erlang:process_info(Pid, current_stacktrace) is >>> repeatedly showing the following stacktrace during these 50s: >>> >>> {current_stacktrace, >>> >>> [{erlang,receive_emd,3,[]}, >>> >>> {erlang,get_mem_data,3,[]}, >>> {erlang,memory,1,[]}, >>> {dht_node_state,details,1, >>> >>> [{file,"src/dht_node_state.erl"},{line,510}]}, >>> >>> {dht_node,on,2,[{file,"src/dht_node.erl"},{line,384}]}, >>> {mockup_dht_node,on,2, >>> >>> [{file,"test/mockup_dht_node.erl"},{line,67}]}, >>> >>> {gen_component,on,3, >>> >>> [{file,"src/gen_component.erl"},{line,614}]}]} >>> >>> The line in the dht_node_state module literally only calls >>> erlang:memory(total) and creates a record with this info. Apparently, it >>> never gets out of erlang:memory/1 :( >>> >>> >>> Thanks for your help >>> Nico >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> The bug fixed in OTP-18.0.1 (and OTP-17.5.6.1) can cause such a hang. >> >> Regards, >> Rickard > ok, I'll keep an eye on 18.0.1 on our test systems > -> note however, that I encountered the bug not "just after emulator start" as > stated in the release log. Also since I ran several tests before the failing > one, several calls to erlang:memory/1 succeeded before the hang! > > > Nico > The bug fixed in 18.0.1 (OTP-12859) is not specific to emulator start, but that is only where we have seen it so far, and we think the VM might be extra vulnerable for this bug at start. The bug is in the general logic to wake up scheduler threads that is blocking for IO events, and can happen "anytime". It's a very specific race between threads however, so you have to be quite unlucky for it to happen. The symptom is actually never an eternal hanging as the polling thread will eventually be released by its next finite timeout, which can be anything from a couple of milliseconds to several hours. /Sverker, Erlang/OTP ... From marc@REDACTED Fri Jul 3 17:49:15 2015 From: marc@REDACTED (Marc Worrell) Date: Fri, 3 Jul 2015 17:49:15 +0200 Subject: [erlang-questions] [ANN] Zotonic release 0.13.0 Message-ID: <3D896276-D5FE-4E65-AC68-CCA8FCDB2664@worrell.nl> Zotonic is the Erlang web framework and content management system. Today we released version 0.13.0. Major changes in this release are: ? New ACL module: mod_acl_user_groups (see below) ? New module: mod_content_groups ? New model: m_hierarchy, for category, content_group and user_group hierarchies ? Category renumbering is now a lot faster and more incremental ? Addition of Material Design art work ? New module: mod_email_status ? New module: mod_component ? DNSBL checks for incoming emails ? More strict sanitization of content ? Social integration with Twitter, Facebook, LinkedIn, and Instagram ? is_dependent Resources ? Refactored automatic compile and load of changed files This release also incorporates all fixes in the 0.12.x branch. Complete release notes are at: http://zotonic.com/docs/latest/dev/releasenotes/rel_0.13.0.html There were more than 600 commits since release 0.12.0. The committers were: Alain O?Dea, Arjan Scherpenisse, Arthur Clemens, David de Boer, Jeff Bell, Maas-Maarten Zeeman, Marc Worrell, Marco Wessel, Paul Guyot, Paul Monson, Sergei, Witeman Zheng, imagency, and ??. Besides these commits many people contributed to this release. Big thanks to all of you! Kind Regards, Marc Worrell -------------- next part -------------- An HTML attachment was scrubbed... URL: From gumm@REDACTED Fri Jul 3 18:27:38 2015 From: gumm@REDACTED (Jesse Gumm) Date: Fri, 3 Jul 2015 11:27:38 -0500 Subject: [erlang-questions] [ANN] Zotonic release 0.13.0 In-Reply-To: <3D896276-D5FE-4E65-AC68-CCA8FCDB2664@worrell.nl> References: <3D896276-D5FE-4E65-AC68-CCA8FCDB2664@worrell.nl> Message-ID: Congrats! On Jul 3, 2015 10:49 AM, "Marc Worrell" wrote: > Zotonic is the Erlang web framework and content management system. > > Today we released version 0.13.0. > > Major changes in this release are: > > ? New ACL module: mod_acl_user_groups (see below) > ? New module: mod_content_groups > ? New model: m_hierarchy, for category, content_group and user_group > hierarchies > ? Category renumbering is now a lot faster and more incremental > ? Addition of Material Design art work > ? New module: mod_email_status > ? New module: mod_component > ? DNSBL checks for incoming emails > ? More strict sanitization of content > ? Social integration with Twitter, Facebook, LinkedIn, and Instagram > ? is_dependent Resources > ? Refactored automatic compile and load of changed files > > This release also incorporates all fixes in the 0.12.x branch. > > Complete release notes are at: > > http://zotonic.com/docs/latest/dev/releasenotes/rel_0.13.0.html > > > There were more than 600 commits since release 0.12.0. > > The committers were: Alain O?Dea, Arjan Scherpenisse, Arthur Clemens, > David de Boer, Jeff Bell, Maas-Maarten Zeeman, Marc Worrell, Marco Wessel, > Paul Guyot, Paul Monson, Sergei, Witeman Zheng, imagency, and ??. > > Besides these commits many people contributed to this release. > > Big thanks to all of you! > > > Kind Regards, > > Marc Worrell > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nico.kruber@REDACTED Fri Jul 3 23:07:59 2015 From: nico.kruber@REDACTED (Nico Kruber) Date: Fri, 03 Jul 2015 23:07:59 +0200 Subject: [erlang-questions] Erlang 18.0, hanging erlang:memory(total)? In-Reply-To: <55968F04.70806@ericsson.com> References: <1680304.mhPGdp6mty@nico-pc> <2045428.YKYjWFu1Gn@nico-pc> <55968F04.70806@ericsson.com> Message-ID: <2801934.DrNvHo5YAQ@nico-pc> On Friday 03 Jul 2015 15:32:52 Sverker Eriksson wrote: > On 07/02/2015 10:42 PM, Nico Kruber wrote: > > On Thursday 02 Jul 2015 14:23:01 Rickard Green wrote: > >> On Thu, Jul 2, 2015 at 12:29 AM, Nico Kruber wrote: > >>> Hi folks, > >>> is it possible that erlang:memory/1 can hang? > >>> > >>> Today I had a unit test failing in the middle of its execution (it runs > >>> the > >>> same test over and over again and it was like the 20th time it was run). > >>> There was a timeout (>50s) in one process not being able to finish its > >>> message handling and erlang:process_info(Pid, current_stacktrace) is > >>> repeatedly showing the following stacktrace during these 50s: > >>> > >>> {current_stacktrace, > >>> > >>> [{erlang,receive_emd,3,[]}, > >>> > >>> {erlang,get_mem_data,3,[]}, > >>> {erlang,memory,1,[]}, > >>> {dht_node_state,details,1, > >>> > >>> [{file,"src/dht_node_state.erl"},{line,510}]}, > >>> > >>> {dht_node,on,2,[{file,"src/dht_node.erl"},{line,384}]}, > >>> {mockup_dht_node,on,2, > >>> > >>> [{file,"test/mockup_dht_node.erl"},{line,67}]}, > >>> > >>> {gen_component,on,3, > >>> > >>> [{file,"src/gen_component.erl"},{line,614}]}]} > >>> > >>> The line in the dht_node_state module literally only calls > >>> erlang:memory(total) and creates a record with this info. Apparently, it > >>> never gets out of erlang:memory/1 :( > >>> > >>> > >>> Thanks for your help > >>> Nico > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> The bug fixed in OTP-18.0.1 (and OTP-17.5.6.1) can cause such a hang. > >> > >> Regards, > >> Rickard > > > > ok, I'll keep an eye on 18.0.1 on our test systems > > -> note however, that I encountered the bug not "just after emulator > > start" as stated in the release log. Also since I ran several tests > > before the failing one, several calls to erlang:memory/1 succeeded before > > the hang! > > > > > > Nico > > The bug fixed in 18.0.1 (OTP-12859) is not specific to emulator start, > but that is only where we have seen it so far, and we think the VM might be > extra vulnerable for this bug at start. > The bug is in the general logic to wake up scheduler threads > that is blocking for IO events, and can happen "anytime". It's a very > specific > race between threads however, so you have to be quite unlucky for it to > happen. > > The symptom is actually never an eternal hanging as the polling thread will > eventually be released by its next finite timeout, which can be anything > from > a couple of milliseconds to several hours. > > /Sverker, Erlang/OTP > > ... Thanks for your explanation, Sverker, I've actually only seen this error this one time with Erlang 18.0 and no other Erlang versions so far. We changed these tests to using 18.0.1 now and I'll see what happens. Our "unlucky" test machine has been quite a gift though since through that machine we found several bugs in our code as well ;) Regards Nico -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: This is a digitally signed message part. URL: From bchesneau@REDACTED Sat Jul 4 10:56:47 2015 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sat, 04 Jul 2015 08:56:47 +0000 Subject: [erlang-questions] How performant is ets:select ? Message-ID: Does anyone have links or knows how ets:select is performing? Not that I have any issue with it right now, but I am wondering if there are already some benchmarks or known limits about it I should be aware. Also how does it works internally, the doc let me think it is traversing all the keys but not sure about it. - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sat Jul 4 13:50:14 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 4 Jul 2015 14:50:14 +0300 Subject: [erlang-questions] How performant is ets:select ? In-Reply-To: References: Message-ID: It depends on the size of your table and access rate. We have a 'sessions' table in Flussonic, it is accessed several times on each request (up to several thousands per second). There is a strict rule: we NEVER make select on it, because it is bad. So we have to make secondary indexes on it. Other tables allow to make select on them. -------------- next part -------------- An HTML attachment was scrubbed... URL: From freza@REDACTED Sat Jul 4 14:42:03 2015 From: freza@REDACTED (Jachym Holecek) Date: Sat, 4 Jul 2015 08:42:03 -0400 Subject: [erlang-questions] How performant is ets:select ? In-Reply-To: References: Message-ID: <20150704124203.GA3489@circlewave.net> # Benoit Chesneau 2015-07-04: > Does anyone have links or knows how ets:select is performing? Not that I > have any issue with it right now, but I am wondering if there are already > some benchmarks or known limits about it I should be aware. > > Also how does it works internally, the doc let me think it is traversing > all the keys but not sure about it. The ordered_set variety should be doing a prefix scan whenever it can, rather than full table scan. Not sure how strong it is at figuring out more general range-scans. Some discussion is here: http://www.erlang.org/doc/efficiency_guide/tablesDatabases.html The ets:select/3 + ets:select/1 variants can help you place upper bounds on some of the costs, a bit indirectly perhaps. That's more on the safety engineering side of things I suppose. Breaking things into multiple tables of appropriate types and layouts as suggested by Max is sound advice -- once you have a solid data model and understand your access patterns well. HTH, -- Jachym From mails@REDACTED Sat Jul 4 16:29:04 2015 From: mails@REDACTED (Rodrigo Dominguez) Date: Sat, 4 Jul 2015 11:29:04 -0300 Subject: [erlang-questions] supervisor with priorities? Message-ID: <3F617183-7260-4FB4-A6AC-82C218FA5614@rorra.com.ar> I have four gen_servers, and I would like to do a special supervision. gen_servers A, B, and C, they all depends on gen_server D. If gen_server A goes down, I want to restart it without affecting the others. If gen_server B goes down, I want to restart it without affecting the others. If gen_server C goes down, I want to restart it without affecting the others. If gen_server D goes down, I want kill and restart all of the gen_servers. Any hint now how to structure the supervisor tree or how to make it work with OTP? I usually use supervisors with one_for_one and one_for_all but I?m clueless on how to archive the desired behavior. Rodrigo Dominguez mails@REDACTED From seancribbs@REDACTED Sat Jul 4 17:44:04 2015 From: seancribbs@REDACTED (Sean Cribbs) Date: Sat, 4 Jul 2015 10:44:04 -0500 Subject: [erlang-questions] supervisor with priorities? In-Reply-To: <3F617183-7260-4FB4-A6AC-82C218FA5614@rorra.com.ar> References: <3F617183-7260-4FB4-A6AC-82C218FA5614@rorra.com.ar> Message-ID: Rodrigo, It sounds like you need two levels of supervision: S1 supervises D and S2, with a "rest_for_one" strategy, with D first. When D dies, this kills S2 and all of its supervised processes. S2 supervises A, B, and C with a "one_for_one" strategy. When any one of them dies, they restart independently. However, since they are under S2, they ALL get restarted when S2 restarts as a consequence of D restarting. Hope that helps! On Sat, Jul 4, 2015 at 9:29 AM, Rodrigo Dominguez wrote: > I have four gen_servers, and I would like to do a special supervision. > > gen_servers A, B, and C, they all depends on gen_server D. > > If gen_server A goes down, I want to restart it without affecting the > others. > If gen_server B goes down, I want to restart it without affecting the > others. > If gen_server C goes down, I want to restart it without affecting the > others. > > If gen_server D goes down, I want kill and restart all of the gen_servers. > > Any hint now how to structure the supervisor tree or how to make it work > with OTP? I usually use supervisors with one_for_one and one_for_all but > I?m clueless on how to archive the desired behavior. > > Rodrigo Dominguez > mails@REDACTED > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kovyl2404@REDACTED Sun Jul 5 19:11:51 2015 From: kovyl2404@REDACTED (Viacheslav V. Kovalev) Date: Sun, 5 Jul 2015 20:11:51 +0300 Subject: [erlang-questions] erl_boot_server and loading code over network Message-ID: Hi folks! I'm playing with erl_boot_servers and have several questions about how code get loaded to node over the network. I've started with trivial release (stdlib, kernel and one my stub application which actually does nothing). Here is it: [vkovalev@REDACTED foobar]$ tree -L 2 . |-- bin | |-- foobar | |-- foobar-0.0.0+build.1.ref307ae38 | |-- install_upgrade.escript | |-- nodetool | `-- start_clean.boot |-- erts-6.1 | |-- bin | |-- doc | |-- include | |-- lib | |-- man | `-- src |-- lib | |-- foobar-0.1.0 | |-- kernel-3.0.1 | |-- sasl-2.4 | `-- stdlib-2.1 `-- releases |-- 0.0.0+build.1.ref307ae38 |-- RELEASES `-- start_erl.data Then I start boot node: [vkovalev@REDACTED foobar]$ erl -sname boot -pa lib/*/ebin -pa releases/0.0.0+build.1.ref307ae38/ -s erl_boot_server start localhost (boot@REDACTED)1> {ok, _, _} = erl_prim_loader:get_file("foobar.boot"). (boot@REDACTED)2> {ok, _, _} = erl_prim_loader:get_file("foobar_app.beam"). Alright. Seems, all necessary code available. Well, then start slave node. Note, I intendedly starts it in /tmp or somewhere away of my release. [vkovalev@REDACTED /tmp]$ erl -sname slave -loader inet -hosts 127.0.0.1 -boot foobar {"init terminating in do_boot",{'cannot get bootfile','foobar.boot'}} Crash dump was written to: erl_crash.dump init terminating in do_boot () I dug into erl_prim_loader, init and friends and got confused with all those paths manipulation that they performs. 1. First thing - https://github.com/erlang/otp/blob/maint/erts/preloaded/src/erl_prim_loader.erl#L669. erl_prim_loader (in inet mode) for some (unclear for me) reasons tries to cripple any requested module with local (clientside) paths. 2. It seems there is no way to force loader on slave node to keep its paths empty: https://github.com/erlang/otp/blob/maint/erts/preloaded/src/init.erl#L697 3. Paths in my bootscript looks like {path,["$ROOT/lib/kernel-4.0/ebin","$ROOT/lib/stdlib-2.5/ebin"]}, so it seems, if I'll get bootscript loaded, anyway, I won't be able boot system with it. What's going on? Is erlang network boot feature broken? Or just my brains? How could I get node successfully network-booted? From bchesneau@REDACTED Sun Jul 5 19:51:16 2015 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sun, 05 Jul 2015 17:51:16 +0000 Subject: [erlang-questions] How performant is ets:select ? In-Reply-To: <20150704124203.GA3489@circlewave.net> References: <20150704124203.GA3489@circlewave.net> Message-ID: thanks for the links an tips. It gives me some good direction. I will report soon my findings as well :) On Sat 4 Jul 2015 at 14:45 Jachym Holecek wrote: > # Benoit Chesneau 2015-07-04: > > Does anyone have links or knows how ets:select is performing? Not that I > > have any issue with it right now, but I am wondering if there are already > > some benchmarks or known limits about it I should be aware. > > > > Also how does it works internally, the doc let me think it is traversing > > all the keys but not sure about it. > > The ordered_set variety should be doing a prefix scan whenever it can, > rather than full table scan. Not sure how strong it is at figuring out > more general range-scans. > > Some discussion is here: > > http://www.erlang.org/doc/efficiency_guide/tablesDatabases.html > > The ets:select/3 + ets:select/1 variants can help you place upper bounds > on some of the costs, a bit indirectly perhaps. That's more on the safety > engineering side of things I suppose. > > Breaking things into multiple tables of appropriate types and layouts as > suggested by Max is sound advice -- once you have a solid data model and > understand your access patterns well. > > HTH, > -- Jachym > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sun Jul 5 20:11:03 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 5 Jul 2015 20:11:03 +0200 Subject: [erlang-questions] How performant is ets:select ? In-Reply-To: References: Message-ID: On Sat, Jul 4, 2015 at 10:56 AM, Benoit Chesneau wrote: > Also how does it works internally, the doc let me think it is traversing > all the keys but not sure about it. IIRC select makes the optimizations it can. If the key is present in the selection it can narrow the amount of work it has to do by a lookup on the key. If, on the other hand, the key is not present, then it has to scan the full table. Jachym notes the special case on ordered_set tables: If the key is a composite tuple, and you are searching a prefix of that tuple, then the system can use the ordering to narrow down the keys to search. For example, searching for a key {8, '_'} on an ordered_set only has to look at the range {8, X} for all X, and this is a O(lg n) operation to find the bound and then a linear scan over the ordered set. select that has to do a full table scan are best limited to small'ish tables (for some notion of small mostly defined by your memory bandwidth) or rarely executed work that can run in the background. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guninalexander@REDACTED Sun Jul 5 20:34:11 2015 From: guninalexander@REDACTED (Alexander Gunin) Date: Sun, 5 Jul 2015 21:34:11 +0300 Subject: [erlang-questions] How performant is ets:select ? In-Reply-To: References: Message-ID: <183561B6-3576-457F-908B-6594CF43BBCD@gmail.com> Is true that ets optimizes composite key scan(like {8,'_'}) for select? I can't find it in documentation. Only this from ets:match documentation "If the key is specified in the pattern, the match is very efficient. If the key is not specified, i.e. if it is a variable or an underscore, the entire table must be searched. ..." > On 05 Jul 2015, at 21:11, Jesper Louis Andersen wrote: > > >> On Sat, Jul 4, 2015 at 10:56 AM, Benoit Chesneau wrote: >> Also how does it works internally, the doc let me think it is traversing all the keys but not sure about it. > > IIRC select makes the optimizations it can. If the key is present in the selection it can narrow the amount of work it has to do by a lookup on the key. If, on the other hand, the key is not present, then it has to scan the full table. Jachym notes the special case on ordered_set tables: If the key is a composite tuple, and you are searching a prefix of that tuple, then the system can use the ordering to narrow down the keys to search. For example, searching for a key {8, '_'} on an ordered_set only has to look at the range {8, X} for all X, and this is a O(lg n) operation to find the bound and then a linear scan over the ordered set. > > select that has to do a full table scan are best limited to small'ish tables (for some notion of small mostly defined by your memory bandwidth) or rarely executed work that can run in the background. > > > -- > J. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From dch@REDACTED Sun Jul 5 21:48:10 2015 From: dch@REDACTED (Dave Cottlehuber) Date: Sun, 05 Jul 2015 21:48:10 +0200 Subject: [erlang-questions] epmd behind firewall prevents observer starting up quickly Message-ID: <1436125690.2647902.315690921.3F050137@webmail.messagingengine.com> I use observer quite a bit, but when I start it in distributed node it takes ~80 seconds to start up, on 1 computer only that has a valid DNS-resolvable hostname but is sensibly firewalled. > time erl -s observer -sname foo ^C 0.85s user 0.24s system 1% cpu 1:19.43 total turns out that inside observer_wx.erl it calls (finally) erl_epmd:names/1 which times out, even though epmd is running & available on the right port. > epmd -d -d -d & epmd: Sun Jul 5 21:34:20 2015: epmd running - daemon = 0 epmd: Sun Jul 5 21:34:20 2015: try to initiate listening port 4369 epmd: Sun Jul 5 21:34:20 2015: entering the main select() loop > erl -sname foo erl_epmd:names(net_adm:localhost()). %% hangs https://github.com/erlang/otp/blob/master/lib/kernel/src/erl_epmd.erl#L426-L432 The issue appears to be that net_adm:localhost() returns the hostname for this computer, which resolves to its NATted external IP, which is sensibly blocked from receiving external traffic for epmd's port range, and therefore the function times out as the socket fails to connect. > erl -name foo Erlang/OTP 18 [erts-7.0.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace] Eshell V7.0.1 (abort with ^G) (foo@REDACTED)1> nodes(). [] (foo@REDACTED)2> net_adm:ping('shell@REDACTED'). pong (foo@REDACTED)3> nodes(). ['shell@REDACTED'] (foo@REDACTED)4> erl_epmd:open({9,1,1,1}). %% changed to protect the innocent {error,etimedout} (foo@REDACTED)5> erl_epmd:open({127,0,0,1}). {ok,#Port<0.760>} (foo@REDACTED)6> observer eventually works correctly after the 60+ timeout, but I'd like to know if there's any way to work around this? I'm hesitant to call it a bug, but it does seem reasonable for observer & other tools to work on systems that have FQDN set and are behind firewalls. A+ Dave From rdm@REDACTED Sun Jul 5 23:17:41 2015 From: rdm@REDACTED (Rich Morin) Date: Sun, 5 Jul 2015 14:17:41 -0700 Subject: [erlang-questions] The Abstract Format redux Message-ID: <6679A9F2-E9C2-41E1-8D0F-13330A43ED98@cfcl.com> I've been working on an revised version of the document that describes the standard representation of Erlang parse trees: The Abstract Format http://erlang.org/doc/apps/erts/absform.html My (expanded, annotated) version is available as: The Abstract Format redux http://wiki.cfcl.com/Projects/Elixir/Erlex/Erlang_AF Most of the copy editing and reformatting is done, so I think it may be time to ask for feedback: comments, corrections, suggestions, etc. -r -- http://www.cfcl.com/rdm Rich Morin rdm@REDACTED http://www.cfcl.com/rdm/resume San Bruno, CA, USA +1 650-873-7841 Software system design, development, and documentation From technion@REDACTED Mon Jul 6 01:11:50 2015 From: technion@REDACTED (Technion) Date: Sun, 5 Jul 2015 23:11:50 +0000 Subject: [erlang-questions] Issues attaching to an Erlang process after hot code load Message-ID: Hi, I have found recently found out that Erlang questions in Stackoverflow earn a person a tumbleweed award [??] Throughout development, I have in general had no issues starting a release, and running "attach". I have however found after performing a hot code load, this functionality appears to break. I fully documented (https://github.com/technion/erlvulnscan) in the "Hot code upgrade" heading the process I used. This process appears to have worked correctly, however, I then lose the ability to attach, and more importantly, send "stop" commands when I later wish to perform maintenance on this process. Using the old release directory: $ ./erlvulnscan-1.02/bin/erlvulnscan attach escript: Failed to open file: /home/technion/erlvulnscan/rel/erlvulnscan-1.02/erts-6.4/bin/nodetool Node is not running! In the new one: $ ./erlvulnscan-1.03/bin/erlvulnscan attach pong Can't access pipe directory /tmp//home/technion/erlvulnscan/rel/erlvulnscan-1.03/: No such file or directory I did ultimately find I could connect using this command: $ ./erlvulnscan-1.03/erts-6.4/bin/to_erl /tmp/home/technion/erlvulnscan/rel/erlvulnscan-1.02/erlang.pipe.2 However this feels like an absurd workaround - like something is obviously broken to get into this state. I can kill the process (using the kill command) after which it can be started and stopped without issue, suggesting something was done wrong, or got broken, during the code load process. I'm not sure if it's an Erlangism I should get used to, but the location of these pipes in general feels very unusual. I don't see why an entire directory structure would be recreated under /tmp, I'd expect to either sit under the release' own directory, or one directory deep in /tmp. The fact it's a deterministic file and directory name could lead to security issues. Any assistance appreciated (any comments on code or general design also appreciated, this is my first Erlang project). -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: OutlookEmoji-??.png Type: image/png Size: 488 bytes Desc: OutlookEmoji-??.png URL: From bog495@REDACTED Mon Jul 6 10:33:00 2015 From: bog495@REDACTED (Bogdan Andu) Date: Mon, 6 Jul 2015 11:33:00 +0300 Subject: [erlang-questions] Heart behavior Message-ID: Hi, I was made some experiments with heart and I found something surpizing, athough it does the job. I start a erlang vm in daemon mode under user called _user0 with home in /var/app like this (from a shell script /var/app/appd run with sudo as a priv user): case $1 in start) su - _user0 -c "$ERL -boot start_sasl -config $LOG +K true +A 4 -sname $NODE -heart -detached -s app_ctl start $NODE" ;; restart) /usr/local/lib/erlang/lib/erl_interface-3.7.20/bin/erl_call -q -sname $NODE sleep 2 $ERL -boot start_sasl -config $LOG +K true +A 4 -sname $NODE\ -heart -detached -s app_ctl start $NODE ;; .... exit 0 environment vars are(under user _user0): HEART_COMMAND=/bin/sh /var/app/appd restart ERL_CRASH_DUMP_SECONDS=10 I have noticed 3 problems: 1) Starting the daemon (as a priv user) with sudo sh /var/app/appd start it starts the heart subsystem, but when I issue sudo kill -9 , the erlang vm is killed but heart never restarts it; Running as _user0 the command /bin/sh /var/app/appd restart manually heart restarts the system monitored after was killed; 2) Everytime I kill a vm monitored by heart with kill -9 the heart procces restarts it immediately, and after that the heart process dies itself,and if in restart is not mentioned -heart option, the heart process is not restarted for the newly restarted erlang vm. 3) It seems the default timeout of 60 seconds is not respected because the vm is restarted immediately -heart option is specified in restart script; So a heart process is tied up to an erlang vm that it monitors and it dies after it spawns another erlang vm? The docs are not clear about these. Having said these what are the best practices to use heart and why heart behaves like above? It seems heart works with kill -KILL|SIGV , but I am not sure what happens if the erlang vm crashes when runs out of memory of file descriptors. Is the vm restarted by heart in these conditions? System: OTP 17.5 64 bits Thanks, Bogdan -------------- next part -------------- An HTML attachment was scrubbed... URL: From bog495@REDACTED Mon Jul 6 10:48:19 2015 From: bog495@REDACTED (Bogdan Andu) Date: Mon, 6 Jul 2015 11:48:19 +0300 Subject: [erlang-questions] Heart behavior Message-ID: Hi, I was made some experiments with heart and I found something surpizing, athough it does the job. I start a erlang vm in daemon mode under user called _user0 with home in /var/app like this (from a shell script /var/app/appd run with sudo as a priv user): case $1 in start) su - _user0 -c "$ERL -boot start_sasl -config $LOG +K true +A 4 -sname $NODE -heart -detached -s app_ctl start $NODE" ;; restart) /usr/local/lib/erlang/lib/erl_interface-3.7.20/bin/erl_call -q -sname $NODE sleep 2 $ERL -boot start_sasl -config $LOG +K true +A 4 -sname $NODE\ -heart -detached -s app_ctl start $NODE ;; .... exit 0 environment vars are(under user _user0): HEART_COMMAND=/bin/sh /var/app/appd restart ERL_CRASH_DUMP_SECONDS=10 I have noticed 3 problems: 1) Starting the daemon (as a priv user) with sudo sh /var/app/appd start it starts the heart subsystem, but when I issue sudo kill -9 , the erlang vm is killed but heart never restarts it; Running as _user0 the command /bin/sh /var/app/appd restart manually heart restarts the system monitored after was killed; 2) Everytime I kill a vm monitored by heart with kill -9 the heart procces restarts it immediately, and after that the heart process dies itself,and if in restart is not mentioned -heart option, the heart process is not restarted for the newly restarted erlang vm. 3) It seems the default timeout of 60 seconds is not respected because the vm is restarted immediately -heart option is specified in restart script; So a heart process is tied up to an erlang vm that it monitors and it dies after it spawns another erlang vm? The docs are not clear about these. Having said these what are the best practices to use heart and why heart behaves like above? It seems heart works with kill -KILL|SIGV , but I am not sure what happens if the erlang vm crashes when runs out of memory of file descriptors. Is the vm restarted by heart in these conditions? System: OTP 17.5 64 bits Thanks, Bogdan -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Mon Jul 6 10:58:06 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 6 Jul 2015 10:58:06 +0200 Subject: [erlang-questions] [ANN] Syn: a global process registry Message-ID: All, Today I've open sourced Syn, a global process registry. Syn is a process registry that has the following features: - Global (i.e. a process is uniquely identified with a Key across all the nodes of a cluster). - Any term can be used as Key. - Fast writes. - Automatically handles conflict resolution (such as net splits). - Configurable callbacks. It can be found here: https://github.com/ostinelli/syn An article discussing various pre-existing global process registries (erlang native modules, gproc, and cpg) and why I've built Syn can be read here: http://www.ostinelli.net/an-evaluation-of-erlang-global-process-registries-meet-syn/ Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Mon Jul 6 11:36:07 2015 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 6 Jul 2015 11:36:07 +0200 Subject: [erlang-questions] Heart behavior In-Reply-To: References: Message-ID: Hello Bogdan, See some answers inline: On Mon, Jul 6, 2015 at 10:33 AM, Bogdan Andu wrote: > Hi, > > I was made some experiments with heart > and I found something surpizing, athough it > does the job. > > I start a erlang vm in daemon mode under user called > _user0 with home in /var/app like this > (from a shell script /var/app/appd run with sudo as a priv user): > > case $1 in > start) > > su - _user0 -c "$ERL -boot start_sasl -config $LOG +K true +A 4 -sname > $NODE -heart -detached -s app_ctl start $NODE" > > ;; > > restart) > /usr/local/lib/erlang/lib/erl_interface-3.7.20/bin/erl_call -q -sname > $NODE > sleep 2 > $ERL -boot start_sasl -config $LOG +K true +A 4 -sname $NODE\ > -heart -detached -s app_ctl start $NODE > ;; > > .... > > exit 0 > > environment vars are(under user _user0): > > HEART_COMMAND=/bin/sh /var/app/appd restart > ERL_CRASH_DUMP_SECONDS=10 > > I have noticed 3 problems: > 1) Starting the daemon (as a priv user) with sudo sh /var/app/appd start > it starts the heart subsystem, but when I issue sudo kill -9 > vm-monitored-byheart>, the erlang vm is killed but heart never > restarts it; > Running as _user0 the command /bin/sh /var/app/appd restart manually > heart restarts the system monitored after was killed; > It sounds as if heart for some reason cannot execute the HEART_COMMAND. Why that might be I don't know, maybe you could try to run it as a non-daemon, or at least redirect the stderr printouts to some file. heart might print things to stderr if it cannot execute HEART_COMMAND. > > 2) Everytime I kill a vm monitored by heart with kill -9 the > heart procces restarts it immediately, and after that the heart process > dies itself,and if in restart is not mentioned -heart option, the heart > process is not restarted for the newly restarted erlang vm. > When you supply -heart to the erlang vm command line you tell that VM to monitor itself using the heart mechanism. So if, as you say, you don't pass -heart on the HEART_COMMAND command, the new vm will not be restarted. This is by design. > > 3) It seems the default timeout of 60 seconds is not respected because > the vm is restarted immediately -heart option is specified in restart > script; > Which timeout is it that you are referring to here? HEART_BEAT_TIMEOUT or HEART_BEAT_BOOT_DELAY? HEART_BEAT_TIMEOUT is the maximum time it will take for heart to detect that something is wrong with the VM, if it can detect that something is wrong earlier then it will. > > So a heart process is tied up to an erlang vm that it monitors and it dies > after it spawns another erlang vm? > yes > The docs are not clear about these. > > Having said these what are the best practices to use heart and why > heart behaves like above? > > It seems heart works with kill -KILL|SIGV , but I am not sure > what happens if the erlang vm crashes when runs out of memory of file > descriptors. > Is the vm restarted by heart in these conditions? > It should be. The only reason for heart not to restart the VM (that I can think of right now) is if you call init:stop(), or if the command line that you gave to HEART_COMMAND does not work. > > System: OTP 17.5 64 bits > > Thanks, > Bogdan > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bog495@REDACTED Mon Jul 6 12:15:22 2015 From: bog495@REDACTED (Bogdan Andu) Date: Mon, 6 Jul 2015 13:15:22 +0300 Subject: [erlang-questions] Heart behavior In-Reply-To: References: Message-ID: Hi, Is HEART_BEAT_TIMEOUT I was refferd to. The doc says: "This modules contains the interface to the heart process. heart sends periodic heartbeats to an external port program, which is also named heart." So the external proccess, heart, is run with the same credentials as the vm user and is started also by vm, right? When the vm crashes, this external heart process kills itself after re-spawning another vm, right? and so on.. new vm with new heart external process >From the docs I understand that the external heart process never dies and after restarts a crashed vm it monitors the new vm with new pid. What is the rate at which heart sends within vm periodic hearbeats to external heart process ? It seems very high. Does this add some load on the monitored vm? Another issue I observe is that heart never logs the crash/restart events in the application's logs, configured like this (in app.config and started with -boot start_sasl -config /var/app/app ): [{sasl, [ {sasl_error_logger, false}, %% define the parameters of the rotating log %% the log file directory {error_logger_mf_dir,"/var/app/logs"}, %% # bytes per logfile {error_logger_mf_maxbytes,10485760}, % 10 MB %% maximum number of logfiles {error_logger_mf_maxfiles, 10} ]}] I was expecting to see some heart activity logged, but there is nothing. What must be done to log heart events in application's log or anywhere else? Because I want to monitor that heart log file and be notified by e-mail when such events occurs. Thanks, Bogdan On Mon, Jul 6, 2015 at 12:36 PM, Lukas Larsson wrote: > Hello Bogdan, > > See some answers inline: > > On Mon, Jul 6, 2015 at 10:33 AM, Bogdan Andu wrote: > >> Hi, >> >> I was made some experiments with heart >> and I found something surpizing, athough it >> does the job. >> >> I start a erlang vm in daemon mode under user called >> _user0 with home in /var/app like this >> (from a shell script /var/app/appd run with sudo as a priv user): >> >> case $1 in >> start) >> >> su - _user0 -c "$ERL -boot start_sasl -config $LOG +K true +A 4 -sname >> $NODE -heart -detached -s app_ctl start $NODE" >> >> ;; >> >> restart) >> /usr/local/lib/erlang/lib/erl_interface-3.7.20/bin/erl_call -q >> -sname $NODE >> sleep 2 >> $ERL -boot start_sasl -config $LOG +K true +A 4 -sname $NODE\ >> -heart -detached -s app_ctl start $NODE >> ;; >> >> .... >> >> exit 0 >> >> environment vars are(under user _user0): >> >> HEART_COMMAND=/bin/sh /var/app/appd restart >> ERL_CRASH_DUMP_SECONDS=10 >> >> I have noticed 3 problems: >> 1) Starting the daemon (as a priv user) with sudo sh /var/app/appd start >> it starts the heart subsystem, but when I issue sudo kill -9 >> > vm-monitored-byheart>, the erlang vm is killed but heart never >> restarts it; >> Running as _user0 the command /bin/sh /var/app/appd restart manually >> heart restarts the system monitored after was killed; >> > > It sounds as if heart for some reason cannot execute the HEART_COMMAND. > Why that might be I don't know, maybe you could try to run it as a > non-daemon, or at least redirect the stderr printouts to some file. heart > might print things to stderr if it cannot execute HEART_COMMAND. > > >> >> 2) Everytime I kill a vm monitored by heart with kill -9 the >> heart procces restarts it immediately, and after that the heart process >> dies itself,and if in restart is not mentioned -heart option, the heart >> process is not restarted for the newly restarted erlang vm. >> > > When you supply -heart to the erlang vm command line you tell that VM to > monitor itself using the heart mechanism. So if, as you say, you don't pass > -heart on the HEART_COMMAND command, the new vm will not be restarted. This > is by design. > > >> >> 3) It seems the default timeout of 60 seconds is not respected because >> the vm is restarted immediately -heart option is specified in restart >> script; >> > > Which timeout is it that you are referring to here? HEART_BEAT_TIMEOUT or > HEART_BEAT_BOOT_DELAY? HEART_BEAT_TIMEOUT is the maximum time it will take > for heart to detect that something is wrong with the VM, if it can detect > that something is wrong earlier then it will. > > >> >> So a heart process is tied up to an erlang vm that it monitors and it >> dies after it spawns another erlang vm? >> > > yes > > >> The docs are not clear about these. >> >> Having said these what are the best practices to use heart and why >> heart behaves like above? >> >> It seems heart works with kill -KILL|SIGV , but I am not sure >> what happens if the erlang vm crashes when runs out of memory of file >> descriptors. >> Is the vm restarted by heart in these conditions? >> > > It should be. The only reason for heart not to restart the VM (that I can > think of right now) is if you call init:stop(), or if the command line that > you gave to HEART_COMMAND does not work. > > >> >> System: OTP 17.5 64 bits >> >> Thanks, >> Bogdan >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Mon Jul 6 13:38:33 2015 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 6 Jul 2015 13:38:33 +0200 Subject: [erlang-questions] Heart behavior In-Reply-To: References: Message-ID: Hello, On Mon, Jul 6, 2015 at 12:15 PM, Bogdan Andu wrote: > Hi, > > Is HEART_BEAT_TIMEOUT I was refferd to. > > The doc says: > "This modules contains the interface to the heart process. heart sends > periodic heartbeats to an external port program, which is also named heart." > > So the external proccess, heart, is run with the same credentials as the > vm user > and is started also by vm, right? > yes > When the vm crashes, > this external heart process kills itself after re-spawning another vm, > right? > yes > > and so on.. new vm with new heart external process > > From the docs I understand that the external heart process never dies > and after restarts a crashed vm it monitors the new vm with new pid. > It would be great if you could help us get the documentation clearer. I've been working with this for many years now and I've lost all sense of what is obvious and what is not. So getting help in clarifying the documentation from people who are just getting familiar with the subject is great. If you feel like contributing, the relevant documentation file can be found here: https://github.com/erlang/otp/blob/master/lib/kernel/doc/src/heart.xml > > What is the rate at which heart sends within vm periodic hearbeats to > external heart process ? > 5 seconds, https://github.com/erlang/otp/blob/master/lib/kernel/src/heart.erl#L48 > It seems very high. Does this add some load on the monitored vm? > Some, but very little in the big scheme of things. > Another issue I observe is that heart never logs the crash/restart events > in the > application's logs, configured like this (in app.config and started with > -boot start_sasl -config /var/app/app ): > > [{sasl, [ > {sasl_error_logger, false}, > %% define the parameters of the rotating log > %% the log file directory > {error_logger_mf_dir,"/var/app/logs"}, > %% # bytes per logfile > {error_logger_mf_maxbytes,10485760}, % 10 MB > %% maximum number of logfiles > {error_logger_mf_maxfiles, 10} > ]}] > > I was expecting to see some heart activity logged, but > there is nothing. > I would add logging in the HEART_COMMAND script. You can write an as complex bash script as you want in the HEART_COMMAND, so just prefixing the erlang start with a call to logger would add a message in the syslog saying that erlang has been restarted. You can even call sendmail here if you want to :). When you crash you will get an erl_crash.dump which is what you need together with the application specific logs. So you may even want to write a small script that bundles all this together in a .tgz and sends it to you. When things go really bad, like if you segfault for some reason, then you'll probably want to setup your /proc/sys/kernel/core_pattern to do something for you. When things go really really bad, like if the oom killer comes and kills you, then I don't think there is much that you can do. I've seen the oom killer come and kill both the erlang vm and heart at the same time, which is really nasty. Hopefully you will get something in your syslog that tells you what might be wrong. > > What must be done to log heart events in application's log > or anywhere else? Because I want to monitor that heart log file and > be notified by e-mail when such events occurs. > The heart program prints it's logs to stderr. Since it is not started with stderr_to_stdout, all stderr it does will end up in the erlang vm's stderr. So you want to forward that to the file you want. This of course means that you cannot run as -detached. Maybe you want to use run_erl/to_erl to do the daemonization. run_erl will log stdout and stderr to a log file that you can use to debug stuff. > > Thanks, > Bogdan > > > > On Mon, Jul 6, 2015 at 12:36 PM, Lukas Larsson > wrote: > >> Hello Bogdan, >> >> See some answers inline: >> >> On Mon, Jul 6, 2015 at 10:33 AM, Bogdan Andu wrote: >> >>> Hi, >>> >>> I was made some experiments with heart >>> and I found something surpizing, athough it >>> does the job. >>> >>> I start a erlang vm in daemon mode under user called >>> _user0 with home in /var/app like this >>> (from a shell script /var/app/appd run with sudo as a priv user): >>> >>> case $1 in >>> start) >>> >>> su - _user0 -c "$ERL -boot start_sasl -config $LOG +K true +A 4 -sname >>> $NODE -heart -detached -s app_ctl start $NODE" >>> >>> ;; >>> >>> restart) >>> /usr/local/lib/erlang/lib/erl_interface-3.7.20/bin/erl_call -q >>> -sname $NODE >>> sleep 2 >>> $ERL -boot start_sasl -config $LOG +K true +A 4 -sname $NODE\ >>> -heart -detached -s app_ctl start $NODE >>> ;; >>> >>> .... >>> >>> exit 0 >>> >>> environment vars are(under user _user0): >>> >>> HEART_COMMAND=/bin/sh /var/app/appd restart >>> ERL_CRASH_DUMP_SECONDS=10 >>> >>> I have noticed 3 problems: >>> 1) Starting the daemon (as a priv user) with sudo sh /var/app/appd start >>> it starts the heart subsystem, but when I issue sudo kill -9 >>> >> vm-monitored-byheart>, the erlang vm is killed but heart never >>> restarts it; >>> Running as _user0 the command /bin/sh /var/app/appd restart manually >>> heart restarts the system monitored after was killed; >>> >> >> It sounds as if heart for some reason cannot execute the HEART_COMMAND. >> Why that might be I don't know, maybe you could try to run it as a >> non-daemon, or at least redirect the stderr printouts to some file. heart >> might print things to stderr if it cannot execute HEART_COMMAND. >> >> >>> >>> 2) Everytime I kill a vm monitored by heart with kill -9 the >>> heart procces restarts it immediately, and after that the heart process >>> dies itself,and if in restart is not mentioned -heart option, the heart >>> process is not restarted for the newly restarted erlang vm. >>> >> >> When you supply -heart to the erlang vm command line you tell that VM to >> monitor itself using the heart mechanism. So if, as you say, you don't pass >> -heart on the HEART_COMMAND command, the new vm will not be restarted. This >> is by design. >> >> >>> >>> 3) It seems the default timeout of 60 seconds is not respected because >>> the vm is restarted immediately -heart option is specified in restart >>> script; >>> >> >> Which timeout is it that you are referring to here? HEART_BEAT_TIMEOUT or >> HEART_BEAT_BOOT_DELAY? HEART_BEAT_TIMEOUT is the maximum time it will take >> for heart to detect that something is wrong with the VM, if it can detect >> that something is wrong earlier then it will. >> >> >>> >>> So a heart process is tied up to an erlang vm that it monitors and it >>> dies after it spawns another erlang vm? >>> >> >> yes >> >> >>> The docs are not clear about these. >>> >>> Having said these what are the best practices to use heart and why >>> heart behaves like above? >>> >>> It seems heart works with kill -KILL|SIGV , but I am not sure >>> what happens if the erlang vm crashes when runs out of memory of file >>> descriptors. >>> Is the vm restarted by heart in these conditions? >>> >> >> It should be. The only reason for heart not to restart the VM (that I can >> think of right now) is if you call init:stop(), or if the command line that >> you gave to HEART_COMMAND does not work. >> >> >>> >>> System: OTP 17.5 64 bits >>> >>> Thanks, >>> Bogdan >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Jul 6 15:01:55 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 6 Jul 2015 16:01:55 +0300 Subject: [erlang-questions] otp_src_18.0.1.tgz Message-ID: There was a news about patch release 18.0.1 but I don't see it on http://www.erlang.org/download.html Is it for git builders only? -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Mon Jul 6 17:08:53 2015 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Mon, 6 Jul 2015 17:08:53 +0200 Subject: [erlang-questions] otp_src_18.0.1.tgz In-Reply-To: References: Message-ID: <559A9A05.70900@ericsson.com> Patches like OTP-18.0.1 usually originate from requests from our paying customers and are only published with a tag on github and a mail on erlang-questions. /Sverker, Erlang/OTP Ericsson On 07/06/2015 03:01 PM, Max Lapshin wrote: > There was a news about patch release 18.0.1 but I don't see it on > http://www.erlang.org/download.html > > Is it for git builders only? > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean@REDACTED Mon Jul 6 18:00:02 2015 From: sean@REDACTED (Functional Jobs) Date: Mon, 6 Jul 2015 12:00:02 -0400 Subject: [erlang-questions] New Functional Programming Job Opportunities Message-ID: <559aa603de380@functionaljobs.com> Here are some functional programming job opportunities that were posted recently: Senior Software Engineer, Distributed Systems at Couchbase http://functionaljobs.com/jobs/8841-senior-software-engineer-distributed-systems-at-couchbase Cheers, Sean Murphy FunctionalJobs.com From bchesneau@REDACTED Mon Jul 6 18:25:53 2015 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 06 Jul 2015 16:25:53 +0000 Subject: [erlang-questions] How performant is ets:select ? In-Reply-To: <183561B6-3576-457F-908B-6594CF43BBCD@gmail.com> References: <183561B6-3576-457F-908B-6594CF43BBCD@gmail.com> Message-ID: i guess it depends where you match '_' . prefixed match in an ordset should be faster to find. Anyway for now i am sticking for an ordset with limited capabilities to match. I will report how it goes. - beno?t On Sun 5 Jul 2015 at 20:34 Alexander Gunin wrote: > Is true that ets optimizes composite key scan(like {8,'_'}) for select? I > can't find it in documentation. Only this from ets:match documentation "If > the key is specified in the pattern, the match is very efficient. If the > key is not specified, i.e. if it is a variable or an underscore, the entire > table must be searched. ..." > > > > On 05 Jul 2015, at 21:11, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > > > On Sat, Jul 4, 2015 at 10:56 AM, Benoit Chesneau > wrote: > >> Also how does it works internally, the doc let me think it is traversing >> all the keys but not sure about it. > > > IIRC select makes the optimizations it can. If the key is present in the > selection it can narrow the amount of work it has to do by a lookup on the > key. If, on the other hand, the key is not present, then it has to scan the > full table. Jachym notes the special case on ordered_set tables: If the key > is a composite tuple, and you are searching a prefix of that tuple, then > the system can use the ordering to narrow down the keys to search. For > example, searching for a key {8, '_'} on an ordered_set only has to look at > the range {8, X} for all X, and this is a O(lg n) operation to find the > bound and then a linear scan over the ordered set. > > select that has to do a full table scan are best limited to small'ish > tables (for some notion of small mostly defined by your memory bandwidth) > or rarely executed work that can run in the background. > > > -- > J. > > _______________________________________________ > > > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Mon Jul 6 19:15:58 2015 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Mon, 06 Jul 2015 19:15:58 +0200 Subject: [erlang-questions] Issues attaching to an Erlang process after hot code load In-Reply-To: Message-ID: Hi, There is no security issues in Erlang. As far there is no security at all... Once you know this, everything become simpler. Note a release upgrade may need a VM reboot in some case, when changing Erlang core libraries or Erlang release. So "hot code load" does not necessary imply no VM stop. On your problem, I agree, pipes locations is a problem. Regards Le?6 juil. 2015 01:11, Technion a ?crit?: > > Hi, > > > I have found recently found out that Erlang questions in Stackoverflow earn a person a tumbleweed award > > > Throughout development, I have in general had no issues starting a release, and running "attach". I have however found after performing a hot code load, this functionality appears to break. > > I fully documented (https://github.com/technion/erlvulnscan) in the "Hot code upgrade" heading the process I used. This process appears to have worked correctly, however, I then lose the ability to attach, and more importantly, send "stop" commands when I later wish to perform maintenance on this process. > > > Using the old release directory: > > $ ./erlvulnscan-1.02/bin/erlvulnscan attach > escript: Failed to open > file: /home/technion/erlvulnscan/rel/erlvulnscan-1.02/erts-6.4/bin/nodetool > Node is not running! > > In the new one: > > $ ./erlvulnscan-1.03/bin/erlvulnscan attach > pong > Can't access pipe > directory /tmp//home/technion/erlvulnscan/rel/erlvulnscan-1.03/: > No such file or directory > > I did ultimately find I could connect using this command: > > $ ./erlvulnscan-1.03/erts-6.4/bin/to_erl > /tmp/home/technion/erlvulnscan/rel/erlvulnscan-1.02/erlang.pipe.2 > > However this feels like an absurd workaround - like something is obviously broken to get into this state. > > I can kill the process (using the kill command) after which it can be started and stopped without issue, suggesting something was done wrong, or got broken, during the code load process. > > > I'm not sure if it's an Erlangism I should get used to, but the location of these pipes in general feels very unusual. I don't see why an entire directory structure would be recreated under /tmp, I'd expect to either sit under the release' own directory, or one directory deep in /tmp. The fact it's a deterministic file and directory name could lead to security issues. > > > ? > > Any assistance appreciated (any comments on code or general design also appreciated, this is my first Erlang project). From essen@REDACTED Mon Jul 6 23:36:50 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Mon, 06 Jul 2015 23:36:50 +0200 Subject: [erlang-questions] [ANN] Syn: a global process registry In-Reply-To: References: Message-ID: <559AF4F2.3020205@ninenines.eu> Hi, Other than the comments already on your post: "I?m not sure why Syn is performing better with 3 nodes than with 2 (and I?ve repeated this test more than once)." This is the good thing about these kind of benchmarks, they allow you to find oddities in the behaviour of your code. So figure this thing out. :-) I will add the package to the index in a few. Cheers, On 07/06/2015 10:58 AM, Roberto Ostinelli wrote: > All, > Today I've open sourced Syn, a global process registry. > > Syn is a process registry that has the following features: > > * Global (i.e. a process is uniquely identified with a Key across all > the nodes of a cluster). > * Any term can be used as Key. > * Fast writes. > * Automatically handles conflict resolution (such as net splits). > * Configurable callbacks. > > It can be found here: > https://github.com/ostinelli/syn > > An article discussing various pre-existing global process registries > (erlang native modules, gproc, and cpg) and why I've built Syn can be > read here: > http://www.ostinelli.net/an-evaluation-of-erlang-global-process-registries-meet-syn/ > > Best, > r. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From garret.smith@REDACTED Tue Jul 7 02:25:17 2015 From: garret.smith@REDACTED (Garret Smith) Date: Mon, 6 Jul 2015 17:25:17 -0700 Subject: [erlang-questions] Change in abstract format of user-specified -types in 18.0 (breaks PropEr) Message-ID: This code: -module(tt). -type a() :: atom(). -type b() :: a(). has a different abstract syntax in 18.0 vs 17.5 as returned by epp:parse_file("tt.erl", []) in 18.0: {ok,[{attribute,1,file,{"tt.erl",1}}, {attribute,1,module,tt}, {attribute,3,type,{a,{type,3,atom,[]},[]}}, {attribute,4,type,{b,{user_type,4,a,[]},[]}}, <-- user_type {eof,6}]} and in 17.0: {ok,[{attribute,1,file,{"tt.erl",1}}, {attribute,1,module,tt}, {attribute,3,type,{a,{type,3,atom,[]},[]}}, {attribute,4,type,{b,{type,4,a,[]},[]}}, <-- type {eof,6}]} When I use PropEr on a module containing module-defined types like 'b' above, it fails with: Error: The typeserver encountered an error: {unsupported_type,{user_type,45,point,[]}}. The module I'm checking defines -type point() and uses it in other types. Question: is this a bug, since 17.x included user_type in the abstract format, but apparently never actually used it? If this was an intended fix, I didn't read about it in the release notes (only about improving the documentation of the abstract format). Any other code, including parse transforms, that work with the abstract format could be affected as well. Thanks, Garret From technion@REDACTED Tue Jul 7 01:22:06 2015 From: technion@REDACTED (Technion) Date: Mon, 6 Jul 2015 23:22:06 +0000 Subject: [erlang-questions] Issues attaching to an Erlang process after hot code load In-Reply-To: References: , Message-ID: Hi Eric, Thank you for this advise. Simpler yes - and no. I'd rather fix or improve such things where I can. I can edit PIPE_DIR in my own releases easily enough, and potentially patch rebar to stop defaulting to /tmp in new release runners. I would send a PR, but this feels like a deliberate design so I don't know if it will go anywhere. I wouldn't start to tackle that however until I have this attach/stop issue under control. I do understand there will still be situations requiring a reboot, but given the feature exists I'd like to be able to use it correctly - given the way things break it doesn't seem "correct" right now. ________________________________________ From: ?ric Pailleau Sent: Tuesday, 7 July 2015 3:15 AM To: Technion Cc: Erlang Questions Subject: Re: [erlang-questions] Issues attaching to an Erlang process after hot code load Hi, There is no security issues in Erlang. As far there is no security at all... Once you know this, everything become simpler. Note a release upgrade may need a VM reboot in some case, when changing Erlang core libraries or Erlang release. So "hot code load" does not necessary imply no VM stop. On your problem, I agree, pipes locations is a problem. Regards Le 6 juil. 2015 01:11, Technion a ?crit : > > Hi, > > > I have found recently found out that Erlang questions in Stackoverflow earn a person a tumbleweed award > > > Throughout development, I have in general had no issues starting a release, and running "attach". I have however found after performing a hot code load, this functionality appears to break. > > I fully documented (https://github.com/technion/erlvulnscan) in the "Hot code upgrade" heading the process I used. This process appears to have worked correctly, however, I then lose the ability to attach, and more importantly, send "stop" commands when I later wish to perform maintenance on this process. > > > Using the old release directory: > > $ ./erlvulnscan-1.02/bin/erlvulnscan attach > escript: Failed to open > file: /home/technion/erlvulnscan/rel/erlvulnscan-1.02/erts-6.4/bin/nodetool > Node is not running! > > In the new one: > > $ ./erlvulnscan-1.03/bin/erlvulnscan attach > pong > Can't access pipe > directory /tmp//home/technion/erlvulnscan/rel/erlvulnscan-1.03/: > No such file or directory > > I did ultimately find I could connect using this command: > > $ ./erlvulnscan-1.03/erts-6.4/bin/to_erl > /tmp/home/technion/erlvulnscan/rel/erlvulnscan-1.02/erlang.pipe.2 > > However this feels like an absurd workaround - like something is obviously broken to get into this state. > > I can kill the process (using the kill command) after which it can be started and stopped without issue, suggesting something was done wrong, or got broken, during the code load process. > > > I'm not sure if it's an Erlangism I should get used to, but the location of these pipes in general feels very unusual. I don't see why an entire directory structure would be recreated under /tmp, I'd expect to either sit under the release' own directory, or one directory deep in /tmp. The fact it's a deterministic file and directory name could lead to security issues. > > > > > Any assistance appreciated (any comments on code or general design also appreciated, this is my first Erlang project). From dch@REDACTED Sun Jul 5 21:48:10 2015 From: dch@REDACTED (Dave Cottlehuber) Date: Sun, 05 Jul 2015 21:48:10 +0200 Subject: [erlang-questions] epmd behind firewall prevents observer starting up quickly Message-ID: <1436125690.2647902.315690921.3F050137@webmail.messagingengine.com> I use observer quite a bit, but when I start it in distributed node it takes ~80 seconds to start up, on 1 computer only that has a valid DNS-resolvable hostname but is sensibly firewalled. > time erl -s observer -sname foo ^C 0.85s user 0.24s system 1% cpu 1:19.43 total turns out that inside observer_wx.erl it calls (finally) erl_epmd:names/1 which times out, even though epmd is running & available on the right port. > epmd -d -d -d & epmd: Sun Jul 5 21:34:20 2015: epmd running - daemon = 0 epmd: Sun Jul 5 21:34:20 2015: try to initiate listening port 4369 epmd: Sun Jul 5 21:34:20 2015: entering the main select() loop > erl -sname foo erl_epmd:names(net_adm:localhost()). %% hangs https://github.com/erlang/otp/blob/master/lib/kernel/src/erl_epmd.erl#L426-L432 The issue appears to be that net_adm:localhost() returns the hostname for this computer, which resolves to its NATted external IP, which is sensibly blocked from receiving external traffic for epmd's port range, and therefore the function times out as the socket fails to connect. > erl -name foo Erlang/OTP 18 [erts-7.0.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace] Eshell V7.0.1 (abort with ^G) (foo@REDACTED)1> nodes(). [] (foo@REDACTED)2> net_adm:ping('shell@REDACTED'). pong (foo@REDACTED)3> nodes(). ['shell@REDACTED'] (foo@REDACTED)4> erl_epmd:open({9,1,1,1}). %% changed to protect the innocent {error,etimedout} (foo@REDACTED)5> erl_epmd:open({127,0,0,1}). {ok,#Port<0.760>} (foo@REDACTED)6> observer eventually works correctly after the 60+ timeout, but I'd like to know if there's any way to work around this? I'm hesitant to call it a bug, but it does seem reasonable for observer & other tools to work on systems that have FQDN set and are behind firewalls. A+ Dave From roberto@REDACTED Tue Jul 7 11:06:44 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 7 Jul 2015 11:06:44 +0200 Subject: [erlang-questions] [ANN] Syn: a global process registry In-Reply-To: <559AF4F2.3020205@ninenines.eu> References: <559AF4F2.3020205@ninenines.eu> Message-ID: > > "I?m not sure why Syn is performing better with 3 nodes than with 2 (and > I?ve repeated this test more than once)." > > This is the good thing about these kind of benchmarks, they allow you to > find oddities in the behaviour of your code. So figure this thing out. :-) > My first reaction was that in the case of three nodes, mnesia async works better because the load of syncing data is shared between nodes (similarly as what the torrent protocol does, for instance), *and* the cluster noise is less relevant. >From 4 nodes on, the noise becomes more relevant, hence things start to slow down again. For that matters, since I'm using mnesia dirty writes: why is there any loss of performance when using dirty writes on a distributed mnesia table if the number of nodes increases? I mean, being dirty writes it actually shouldn't matter that there are multiple nodes. Sure, it'll take longer to propagate, but why is the write operation slower? I would love to hear from some mnesia specialist. Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pavelmart@REDACTED Tue Jul 7 11:52:16 2015 From: pavelmart@REDACTED (Pavel Martynenko) Date: Tue, 7 Jul 2015 17:52:16 +0800 Subject: [erlang-questions] OTP-18 os:timestamp 10 times slower on OSX compared to OTP-17 Message-ID: Poor performance of os:timestamp and erlang:now on OSX compared to OTP-17. Here is the comparison of 1M cycles. On OSX Darwin Kernel Version 14.3.0 (hardware) Eshell V6.4 (abort with ^G) 1> erlang:now/0 232390 os:timestamp/0 116950 Eshell V7.0 (abort with ^G) 1> erlang:now/0 1416680 erlang:timestamp/0 1151160 erlang:monotonic_time/0 1129957 erlang:system_time/0 1137389 erlang:time_offset/0 50977 erlang:unique_integer/0 52861 t:-test/0-fun-0-/0 1214256 os:timestamp/0 1003910 os:system_time/0 1008520 And Linux 3.16.0-41-generic (virtualised on the same hardware as OSX) Eshell V6.4.1 (abort with ^G) 1> erlang:now/0 2400925 os:timestamp/0 2223445 Eshell V7.0 (abort with ^G) 1> erlang:now/0 2919313 erlang:timestamp/0 2654415 erlang:monotonic_time/0 2412814 erlang:system_time/0 2478186 erlang:time_offset/0 43202 erlang:unique_integer/0 34817 t:-test/0-fun-0-/0 2594927 os:timestamp/0 2065593 os:system_time/0 2007291 Where t:-test/0-fun-0-/0 is fun () -> {erlang:monotonic_time(), erlang:unique_integer([monotonic])} end. -- Cheers, Pavlo Martynenko -------------- next part -------------- An HTML attachment was scrubbed... URL: From zandra@REDACTED Tue Jul 7 15:32:28 2015 From: zandra@REDACTED (Zandra Hird) Date: Tue, 7 Jul 2015 15:32:28 +0200 Subject: [erlang-questions] Patch Package OTP 17.5.6.2 Released Message-ID: <559BD4EC.1050506@erlang.org> Patch Package: OTP 17.5.6.2 Git Tag: OTP-17.5.6.2 Check out the git tag OTP-17.5.6.2, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- erts-6.4.1.2 ---------------------------------------------------- --------------------------------------------------------------------- The erts-6.4.1.2 application can be applied independently of other applications on a full OTP 17 installation. --- Fixed Bugs and Malfunctions --- OTP-12889 Application(s): erts Related Id(s): seq12885 A process could end up in an inconsistent half exited state in the runtime system without SMP support. This could occur if the processes was traced by a port that it also was linked to, and the port terminated abnormally while handling a trace message for the process. This bug has always existed in the runtime system without SMP support, but never in the runtime system with SMP support. Full runtime dependencies of erts-6.4.1.2: kernel-3.0, sasl-2.4, stdlib-2.0 --------------------------------------------------------------------- --- runtime_tools-1.8.16.1 ------------------------------------------ --------------------------------------------------------------------- The runtime_tools-1.8.16.1 application can be applied independently of other applications on a full OTP 17 installation. --- Fixed Bugs and Malfunctions --- OTP-12890 Application(s): runtime_tools Related Id(s): seq12885 The trace_file_drv did not handle EINTR correct which caused it to fail when the runtime system received a signal. Full runtime dependencies of runtime_tools-1.8.16.1: erts-6.0, kernel-3.0, mnesia-4.12, stdlib-2.0 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From mononcqc@REDACTED Tue Jul 7 15:43:19 2015 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 7 Jul 2015 09:43:19 -0400 Subject: [erlang-questions] [ANN] Syn: a global process registry In-Reply-To: References: Message-ID: <20150707134318.GF72759@ferdair.local> On 07/06, Roberto Ostinelli wrote: >Syn is a process registry that has the following features: > > - Global (i.e. a process is uniquely identified with a Key across all > the nodes of a cluster). > - Any term can be used as Key. > - Fast writes. > - Automatically handles conflict resolution (such as net splits). > - Configurable callbacks. > One of the things mentioned in your article was that because you used mostly unique device names, you didn't have to worry much about conflicts in names, and could consequently relax the consistency properties to go for eventual consistency. There is however no details about how this takes place. Attributes that are fun to know are: - What's the conflict resolution mechanism - how long does it take to detect a conflict - how long does it take to resolve a conflict For example, I looked at the following code: https://github.com/ostinelli/syn/blob/master/src/syn_consistency.erl#L255-L262 case CallbackModule of undefined -> error_logger:warning_msg("Found a double process for ~s, killing it on local node ~p", [Key, node()]), exit(LocalProcessPid, kill); _ -> spawn(fun() -> error_logger:warning_msg("Found a double process for ~s, about to trigger callback on local node ~p", [Key, node()]), CallbackModule:CallbackFunction(Key, LocalProcessPid) end) end And this makes it look like it is possible for two nodes to find conflicting pids, and if they find it at the same time, both processes are killed at once. This can be worked-around by setting up a function that always picks the same pid no matter who executes it (exit(max(P1,P2), kill), for example), but killing the local pid always risks having all nodes involved making that same decision and then having nobody left as soon as there's a conflict. So what could be the impact of this on a cluster where the conflict rate is higher, say 80%? Would an app like Syn mostly kill my entire cluster if I don't configure it properly? Or maybe I misunderstood something from my very brief reading of the code. The speed boost is interesting, but without more details about the app's handling of conflict when the uniqueness of names isn't guaranteed, it's hard to make myself a solid idea of how it would go in the wild. Regards, Fred. From pavelmart@REDACTED Tue Jul 7 14:48:29 2015 From: pavelmart@REDACTED (Pavel Martynenko) Date: Tue, 7 Jul 2015 20:48:29 +0800 Subject: [erlang-questions] OTP-18 os:timestamp 10 times slower on OSX compared to OTP-17 In-Reply-To: References: Message-ID: Linux 2.6.26-2-686 (hardware) Eshell V6.4 (abort with ^G) 1> erlang:now/0 1012080 os:timestamp/0 779001 Eshell V7.0 (abort with ^G) 1> erlang:now/0 1865773 erlang:timestamp/0 1856433 erlang:monotonic_time/0 1840571 erlang:system_time/0 1851965 erlang:time_offset/0 50446 erlang:unique_integer/0 44924 t:-test/0-fun-0-/0 1866288 os:timestamp/0 938257 os:system_time/0 896746 Looks better than on OSX. Still almost 2x times slower to "Determine Order of Events With Time of the Event" as described in "Time and Time Correction in Erlang". -- Cheers, Pavlo Martynenko -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmeiklejohn@REDACTED Tue Jul 7 16:13:14 2015 From: cmeiklejohn@REDACTED (Christopher Meiklejohn) Date: Tue, 7 Jul 2015 16:13:14 +0200 Subject: [erlang-questions] [ANN] Syn: a global process registry In-Reply-To: <20150707134318.GF72759@ferdair.local> References: <20150707134318.GF72759@ferdair.local> Message-ID: On Tue, Jul 7, 2015 at 3:43 PM, Fred Hebert wrote: > On 07/06, Roberto Ostinelli wrote: >> >> Syn is a process registry that has the following features: >> >> - Global (i.e. a process is uniquely identified with a Key across all >> the nodes of a cluster). >> - Any term can be used as Key. >> - Fast writes. >> - Automatically handles conflict resolution (such as net splits). >> - Configurable callbacks. >> Did you happen to evaluate our work on Riak PG? Riak PG provides a CRDT based solution for process group membership. There's a paper available from Erlang Workshop 2013. http://christophermeiklejohn.com/erlang/2013/06/05/erlang-gproc-failure-semantics.html http://christophermeiklejohn.com/erlang/2013/06/03/erlang-pg2-failure-semantics.html http://christophermeiklejohn.com/erlang/riak/crdt/2013/06/24/introducing-riak-pg-distributed-process-groups-for-erlang.html http://dl.acm.org/citation.cfm?id=2505309 - Christopher From roberto@REDACTED Tue Jul 7 16:25:31 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 7 Jul 2015 16:25:31 +0200 Subject: [erlang-questions] [ANN] Syn: a global process registry In-Reply-To: <20150707134318.GF72759@ferdair.local> References: <20150707134318.GF72759@ferdair.local> Message-ID: Hi Fred, Thank you for your input. Comments below. > One of the things mentioned in your article was that because you used > mostly unique device names, you didn't have to worry much about conflicts > in names, and could consequently relax the consistency properties to go for > eventual consistency. > > There is however no details about how this takes place. Attributes that > are fun to know are: > > - What's the conflict resolution mechanism > - how long does it take to detect a conflict > - how long does it take to resolve a conflict > > For example, I looked at the following code: > https://github.com/ostinelli/syn/blob/master/src/syn_consistency.erl#L255-L262 > > case CallbackModule of > undefined -> > error_logger:warning_msg("Found a double process for ~s, > killing it on local node ~p", [Key, node()]), > exit(LocalProcessPid, kill); > _ -> spawn(fun() -> > error_logger:warning_msg("Found a double process for ~s, about > to trigger callback on local node ~p", [Key, node()]), > CallbackModule:CallbackFunction(Key, LocalProcessPid) end) > end > > And this makes it look like it is possible for two nodes to find > conflicting pids, and if they find it at the same time, both processes are > killed at once. This can be worked-around by setting up a function that > always picks the same pid no matter who executes it (exit(max(P1,P2), > kill), for example), but killing the local pid always risks having all > nodes involved making that same decision and then having nobody left as > soon as there's a conflict. > When a node is disconnected from the cluster, the other nodes will remove from their mnesia tables all the pids (and hence the keys) that run on the disconnected node, and viceversa: https://github.com/ostinelli/syn/blob/master/src/syn_consistency.erl#L134 This means that the disconnected node *does not* have in its mnesia replica the keys of all the other nodes, and the other nodes *do not* have in their mnesia replicas the keys of the disconnected node. If the disconnected node was to merge back in right away (i.e. with no new registrations happening), there simply wouldn't be any conflicts and everything would be merged in. In a more realistic scenario, the nodes of the cluster and the disconnected node keep registering new pids. If, during the net split, there's no unique key that has been used both on the disconnected node and on the rest of the cluster, then we're back to the previous scenario: everything gets merged in. If the same unique key has been registered both on the disconnected node and on the cluster, then we have a conflict. In this case, if you scroll a little above in the code, you'll see that at that all of the merge code runs inside of a global lock: https://github.com/ostinelli/syn/blob/master/src/syn_consistency.erl#L180 When one node starts the merge, the other nodes are basically waiting. The risk of having both killed is therefore non-existent. Or, I might have forgotten something (it happens!), in which way I'd be delighted to know and improve the code :) Just to give you an example of what I've been observing: 2 nodes, 1 million connected (and registered) devices, a net split of 5 minutes, less than 10 conflicts, resolved in less than 500ms from the moments mnesia signalled an inconsistent database, to the moment the global lock is released). So what could be the impact of this on a cluster where the conflict rate is > higher, say 80%? Would an app like Syn mostly kill my entire cluster if I > don't configure it properly? Or maybe I misunderstood something from my > very brief reading of the code. > Please consider that as per the use-case defined (IoT applications), conflicts are extremely minor. Your example would mean that 80% of the devices, during a net split, connected both to the disconnected node and the rest of the cluster. It is weird to say the least. That being said, I have not benchmarked this case scenario, but here again we are talking about finding the conflicting keys, and sending an exit signal to 1 of the 2 conflicting pids: https://github.com/ostinelli/syn/blob/master/src/syn_consistency.erl#L238 These things are rather quick in the 7 digit numbers. > The speed boost is interesting, but without more details about the app's > handling of conflict when the uniqueness of names isn't guaranteed, it's > hard to make myself a solid idea of how it would go in the wild. > If you mean uniqueness of names in a precise given time, indeed. Syn is eventually consistent. Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Tue Jul 7 16:31:01 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 7 Jul 2015 16:31:01 +0200 Subject: [erlang-questions] [ANN] Syn: a global process registry In-Reply-To: References: <20150707134318.GF72759@ferdair.local> Message-ID: > > Did you happen to evaluate our work on Riak PG? Riak PG provides a > CRDT based solution for process group membership. There's a paper > available from Erlang Workshop 2013. > > > http://christophermeiklejohn.com/erlang/2013/06/05/erlang-gproc-failure-semantics.html > > http://christophermeiklejohn.com/erlang/2013/06/03/erlang-pg2-failure-semantics.html > > http://christophermeiklejohn.com/erlang/riak/crdt/2013/06/24/introducing-riak-pg-distributed-process-groups-for-erlang.html > http://dl.acm.org/citation.cfm?id=2505309 > > - Christopher > Hi Christopher, I did not. I'd be delighted to hear from you if you end up evaluating and comparing them to other existing solutions. Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue Jul 7 18:30:28 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 7 Jul 2015 19:30:28 +0300 Subject: [erlang-questions] What is the proper way of sending messages to pid from enif thread? Message-ID: Hi. I've created thread in nif function. When I create thread, I know pid of caller: ErlNifPid self; enif_self(env, &self); But later in thread I need to create some environment and send message to this pid. What is the proper way to save this ErlNifPid in C structure so that code in other thread will be able to send messages. And as far as I understand, I need to call enif_alloc_env in that thread? -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Tue Jul 7 18:40:47 2015 From: sergej.jurecko@REDACTED (Sergej Jurecko) Date: Tue, 07 Jul 2015 18:40:47 +0200 Subject: [erlang-questions] What is the proper way of sending messages to pid from enif thread? In-Reply-To: References: Message-ID: <5BC76465-FE66-4E0F-B68B-82B55973D36D@gmail.com> I just copy ErlNifPid to a C structure and use it from another thread. The environment used on that other thread was created with enif_alloc_env. Sergej From: on behalf of Max Lapshin Date: Tuesday 7 July 2015 18:30 To: Erlang-Questions Questions Subject: [erlang-questions] What is the proper way of sending messages to pid from enif thread? Hi. I've created thread in nif function. When I create thread, I know pid of caller: ErlNifPid self; enif_self(env, &self); But later in thread I need to create some environment and send message to this pid. What is the proper way to save this ErlNifPid in C structure so that code in other thread will be able to send messages. And as far as I understand, I need to call enif_alloc_env in that thread? _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Tue Jul 7 20:22:36 2015 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 07 Jul 2015 19:22:36 +0100 Subject: [erlang-questions] Cowboy and Bullet Message-ID: <559C18EC.5090707@gmail.com> Hey guys, So, I updated to the latest Bullet and Cowboy, but I'm seeing a disconnect. Is the latest Bullet even compatible with the latest Cowboy? When run, Cowboy expects a third-party handler to supply an init/2 function, while Bullet provides an init/3. Thoughts? Thanks, Lee From mjtruog@REDACTED Tue Jul 7 22:04:01 2015 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 07 Jul 2015 13:04:01 -0700 Subject: [erlang-questions] [ANN] Syn: a global process registry In-Reply-To: References: <20150707134318.GF72759@ferdair.local> Message-ID: <559C30B1.90906@gmail.com> On 07/07/2015 07:25 AM, Roberto Ostinelli wrote: > Hi Fred, > Thank you for your input. Comments below. > > One of the things mentioned in your article was that because you used mostly unique device names, you didn't have to worry much about conflicts in names, and could consequently relax the consistency properties to go for eventual consistency. > > There is however no details about how this takes place. Attributes that are fun to know are: > > - What's the conflict resolution mechanism > - how long does it take to detect a conflict > - how long does it take to resolve a conflict > > For example, I looked at the following code: https://github.com/ostinelli/syn/blob/master/src/syn_consistency.erl#L255-L262 > > case CallbackModule of > undefined -> > error_logger:warning_msg("Found a double process for ~s, killing it on local node ~p", [Key, node()]), > exit(LocalProcessPid, kill); > _ -> spawn(fun() -> > error_logger:warning_msg("Found a double process for ~s, about to trigger callback on local node ~p", [Key, node()]), > CallbackModule:CallbackFunction(Key, LocalProcessPid) end) > end > > And this makes it look like it is possible for two nodes to find conflicting pids, and if they find it at the same time, both processes are killed at once. This can be worked-around by setting up a function that always picks the same pid no matter who executes it (exit(max(P1,P2), kill), for example), but killing the local pid always risks having all nodes involved making that same decision and then having nobody left as soon as there's a conflict. > > > > When a node is disconnected from the cluster, the other nodes will remove from their mnesia tables all the pids (and hence the keys) that run on the disconnected node, and viceversa: > https://github.com/ostinelli/syn/blob/master/src/syn_consistency.erl#L134 > > This means that the disconnected node *does not* have in its mnesia replica the keys of all the other nodes, and the other nodes *do not* have in their mnesia replicas the keys of the disconnected node. > > If the disconnected node was to merge back in right away (i.e. with no new registrations happening), there simply wouldn't be any conflicts and everything would be merged in. > > In a more realistic scenario, the nodes of the cluster and the disconnected node keep registering new pids. > If, during the net split, there's no unique key that has been used both on the disconnected node and on the rest of the cluster, then we're back to the previous scenario: everything gets merged in. > If the same unique key has been registered both on the disconnected node and on the cluster, then we have a conflict. > > In this case, if you scroll a little above in the code, you'll see that at that all of the merge code runs inside of a global lock: > https://github.com/ostinelli/syn/blob/master/src/syn_consistency.erl#L180 > > When one node starts the merge, the other nodes are basically waiting. The risk of having both killed is therefore non-existent. Or, I might have forgotten something (it happens!), in which way I'd be delighted to know and improve the code :) > > Just to give you an example of what I've been observing: 2 nodes, 1 million connected (and registered) devices, a net split of 5 minutes, less than 10 conflicts, resolved in less than 500ms from the moments mnesia signalled an inconsistent database, to the moment the global lock is released). The handling of conflicts is important when classifying the system. I have seen in the code that "doubles" are purged which are likely when the same name exists in two separate network partitions that are attempting to merge back together. You have used the term "eventually consistent" to basically mean "consistent until a netsplit occurs", due to the loss of data when separate network partitions are merged. Due to using a global lock to resolve any conflicts that exist during the merge, you are losing availability during that time period, even if it is only 500ms for 2 nodes with a decent amount of processes. So, that means your system is partition tolerant all the time while losing both consistency and availability when a netsplit occurs. I understand this type of system matches your use case, but I think it is important to be clear about the impact of netsplits. Best Regards, Michael > > > So what could be the impact of this on a cluster where the conflict rate is higher, say 80%? Would an app like Syn mostly kill my entire cluster if I don't configure it properly? Or maybe I misunderstood something from my very brief reading of the code. > > > Please consider that as per the use-case defined (IoT applications), conflicts are extremely minor. > Your example would mean that 80% of the devices, during a net split, connected both to the disconnected node and the rest of the cluster. It is weird to say the least. > > That being said, I have not benchmarked this case scenario, but here again we are talking about finding the conflicting keys, and sending an exit signal to 1 of the 2 conflicting pids: > https://github.com/ostinelli/syn/blob/master/src/syn_consistency.erl#L238 > > These things are rather quick in the 7 digit numbers. > > The speed boost is interesting, but without more details about the app's handling of conflict when the uniqueness of names isn't guaranteed, it's hard to make myself a solid idea of how it would go in the wild. > > > If you mean uniqueness of names in a precise given time, indeed. Syn is eventually consistent. > > > Best, > r. > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From rickard@REDACTED Wed Jul 8 00:45:36 2015 From: rickard@REDACTED (Rickard Green) Date: Wed, 8 Jul 2015 00:45:36 +0200 Subject: [erlang-questions] OTP-18 os:timestamp 10 times slower on OSX compared to OTP-17 In-Reply-To: References: Message-ID: On Tue, Jul 7, 2015 at 2:48 PM, Pavel Martynenko wrote: > Linux 2.6.26-2-686 (hardware) > > Eshell V6.4 (abort with ^G) > 1> > erlang:now/0 1012080 > os:timestamp/0 779001 > > Eshell V7.0 (abort with ^G) > 1> > erlang:now/0 1865773 > erlang:timestamp/0 1856433 > erlang:monotonic_time/0 1840571 > erlang:system_time/0 1851965 > erlang:time_offset/0 50446 > erlang:unique_integer/0 44924 > t:-test/0-fun-0-/0 1866288 > os:timestamp/0 938257 > os:system_time/0 896746 > > Looks better than on OSX. Still almost 2x times slower to "Determine Order > of Events With Time of the Event" as described in "Time and Time Correction > in Erlang". > > -- > Cheers, > Pavlo Martynenko > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > Looking at MacOSX first: The biggest problem on MacOSX is the performance of clock_get_time(). The performance of os:timestamp() and os:system_time() can be improved by reverting to use of gettimeofday() for OS system time at the expense of worse accuracy, resolution and precision of both monotonic time and system time. This will however not help improving erlang:monotonic_time(), erlang:system_time(), or erlang:now(). clock_get_time() is needed for time correction of these, so unfortunately the only way to improve on these is to revert back to not using time correction. By reverting to gettimeofday() and disabling time correction you are more or less back to the implementation used on MacOSX in OTP 17 regarding retrieval of time. Regarding the Linux case: On newer kernels (newer than 2.6.39) the clock id CLOCK_BOOTTIME is used for monotonic time instead of CLOCK_MONOTONIC since OTP 18. This since CLOCK_BOOTTIME also include the time the machine is in a suspended state. CLOCK_BOOTTIME however proved to be much more expensive, so we will revert back to using CLOCK_MONOTONIC. Besides this we've also found a performance bug where unnecessary data is copied when retrieving corrected monotonic time. This effects all systems except Linux with multi time warp mode enabled (which performs better). When reverting to CLOCK_MONOTONIC and fixing the unnecessary copying, the difference shrinks a lot. There is however still a difference in a sequential benchmark. This due to overhead computing the time correction in such a way that it is possible to retrieve time in parallel without contention (the multi time warp mode implementation on Linux does not have to compute this correction which is why it performs better). However, as soon as you get contention on the time retrieval, OTP 17 falls behind. We will soon release a patch where: - the unnecessary copying has been removed - default monotonic clock id on linux has changed to CLOCK_MONOTONIC - you can force usage of gettimeofday() for OS system time when building the system. Regards, Rickard -- Rickard Green, Erlang/OTP, Ericsson AB From max.lapshin@REDACTED Wed Jul 8 11:11:53 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 8 Jul 2015 12:11:53 +0300 Subject: [erlang-questions] What is the proper way of sending messages to pid from enif thread? In-Reply-To: <5BC76465-FE66-4E0F-B68B-82B55973D36D@gmail.com> References: <5BC76465-FE66-4E0F-B68B-82B55973D36D@gmail.com> Message-ID: Sergej, can you share a snippet of how do you do the sending from a separate thread? -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Jul 8 11:21:06 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 08 Jul 2015 11:21:06 +0200 Subject: [erlang-questions] What is the proper way of sending messages to pid from enif thread? In-Reply-To: References: <5BC76465-FE66-4E0F-B68B-82B55973D36D@gmail.com> Message-ID: <559CEB82.4060007@ninenines.eu> I have something like this here: https://github.com/ninenines/esdl2/blob/master/c_src/nif_helpers.c#L102 nif_helpers.c and .h are utility functions I built allowing you to setup and use a separate thread and communicate with it using casts and calls (the C code just returns the message to be sent back, if any). On 07/08/2015 11:11 AM, Max Lapshin wrote: > Sergej, can you share a snippet of how do you do the sending from a > separate thread? > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From essen@REDACTED Wed Jul 8 11:21:47 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 08 Jul 2015 11:21:47 +0200 Subject: [erlang-questions] Cowboy and Bullet In-Reply-To: <559C18EC.5090707@gmail.com> References: <559C18EC.5090707@gmail.com> Message-ID: <559CEBAB.8000803@ninenines.eu> It's not compatible with Cowboy master. Nothing is compatible with master, it's too early to use it. :-) On 07/07/2015 08:22 PM, Lee Sylvester wrote: > Hey guys, > > So, I updated to the latest Bullet and Cowboy, but I'm seeing a > disconnect. Is the latest Bullet even compatible with the latest > Cowboy? When run, Cowboy expects a third-party handler to supply an > init/2 function, while Bullet provides an init/3. > > Thoughts? > > Thanks, > Lee > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From max.lapshin@REDACTED Wed Jul 8 11:38:09 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 8 Jul 2015 12:38:09 +0300 Subject: [erlang-questions] What is the proper way of sending messages to pid from enif thread? In-Reply-To: <559CEB82.4060007@ninenines.eu> References: <5BC76465-FE66-4E0F-B68B-82B55973D36D@gmail.com> <559CEB82.4060007@ninenines.eu> Message-ID: I've found my problem: I need to run enif_self in initializing function and use that pid in thread env. When I was calling enif_get_local_pid and etc, it was not working. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rickard@REDACTED Wed Jul 8 23:14:19 2015 From: rickard@REDACTED (Rickard Green) Date: Wed, 8 Jul 2015 23:14:19 +0200 Subject: [erlang-questions] Patch package OTP 18.0.2 released Message-ID: <559D92AB.1070309@erlang.org> Patch Package: OTP 18.0.2 Git Tag: OTP-18.0.2 Date: 2015-07-08 Trouble Report Id: OTP-12889, OTP-12890, OTP-12892, OTP-12894, OTP-12895, OTP-12896 Seq num: seq12885 System: OTP Release: 18 Application: erts-7.0.2, runtime_tools-1.9.1 Predecessor: OTP 18.0.1 Check out the git tag OTP-18.0.2, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- erts-7.0.2 ------------------------------------------------------ --------------------------------------------------------------------- The erts-7.0.2 application can be applied independently of other applications on a full OTP 18 installation. --- Fixed Bugs and Malfunctions --- OTP-12889 Application(s): erts Related Id(s): seq12885 A process could end up in an inconsistent half exited state in the runtime system without SMP support. This could occur if the processes was traced by a port that it also was linked to, and the port terminated abnormally while handling a trace message for the process. This bug has always existed in the runtime system without SMP support, but never in the runtime system with SMP support. OTP-12894 Application(s): erts Removed unnecessary copying of data when retrieving corrected Erlang monotonic time. OTP-12895 Application(s): erts *** POTENTIAL INCOMPATIBILITY *** Changed default OS monotonic clock source chosen at build time. This in order to improve performance. The behavior will now on most systems be that (both OS and Erlang) monotonic time stops when the system is suspended. If you prefer that monotonic time elapse during suspend of the machine, you can pass the command line argument --enable-prefer-elapsed-monotonic-time-during-suspend to configure when building Erlang/OTP. The configuration stage will try to find such a clock source, but might not be able to find it. Note that there might be a performance penalty associated with such a clock source. OTP-12896 Application(s): erts erlang:system_info(end_time) returned a faulty value on 32-bit architectures. --- Improvements and New Features --- OTP-12892 Application(s): erts The configure command line argument --enable-gettimeofday-as-os-system-time has been added which force usage of gettimeofday() for OS system time. This will improve performance of os:system_time() and os:timestamp() on MacOS X, at the expense of worse accuracy, resolution and precision of Erlang monotonic time, Erlang system time, and OS system time. Full runtime dependencies of erts-7.0.2: kernel-4.0, sasl-2.4, stdlib-2.5 --------------------------------------------------------------------- --- runtime_tools-1.9.1 --------------------------------------------- --------------------------------------------------------------------- The runtime_tools-1.9.1 application can be applied independently of other applications on a full OTP 18 installation. --- Fixed Bugs and Malfunctions --- OTP-12890 Application(s): runtime_tools Related Id(s): seq12885 The trace_file_drv did not handle EINTR correct which caused it to fail when the runtime system received a signal. Full runtime dependencies of runtime_tools-1.9.1: erts-7.0, kernel-3.0, mnesia-4.12, stdlib-2.0 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From raimo+erlang-questions@REDACTED Thu Jul 9 12:28:55 2015 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 9 Jul 2015 12:28:55 +0200 Subject: [erlang-questions] epmd behind firewall prevents observer starting up quickly In-Reply-To: <1436125690.2647902.315690921.3F050137@webmail.messagingengine.com> References: <1436125690.2647902.315690921.3F050137@webmail.messagingengine.com> Message-ID: <20150709102855.GB18276@erix.ericsson.se> This seems like a bug. I understand why net_adm:localhost() returns the external name since it is supposed to give you a canonical name, and it is documented so. But I do not understand why the node does not communicate with the local EPMD over the loopback interface, which should be logical. We should correct this. / Raimo Niskanen, Erlang/OTP On Sun, Jul 05, 2015 at 09:48:10PM +0200, Dave Cottlehuber wrote: > I use observer quite a bit, but when I start it in distributed node it > takes ~80 seconds to start up, on 1 computer only that has a valid > DNS-resolvable hostname but is sensibly firewalled. > > > time erl -s observer -sname foo > > ^C > > 0.85s user 0.24s system 1% cpu 1:19.43 total > > turns out that inside observer_wx.erl it calls (finally) > erl_epmd:names/1 which times out, even though epmd is running & > available on the right port. > > > epmd -d -d -d & > epmd: Sun Jul 5 21:34:20 2015: epmd running - daemon = 0 > epmd: Sun Jul 5 21:34:20 2015: try to initiate listening port 4369 > epmd: Sun Jul 5 21:34:20 2015: entering the main select() loop > > > erl -sname foo > > erl_epmd:names(net_adm:localhost()). %% hangs > > https://github.com/erlang/otp/blob/master/lib/kernel/src/erl_epmd.erl#L426-L432 > > The issue appears to be that net_adm:localhost() returns the hostname > for this computer, which resolves to its NATted external IP, which is > sensibly blocked from receiving external traffic for epmd's port range, > and therefore the function times out as the socket fails to connect. > > > erl -name foo > Erlang/OTP 18 [erts-7.0.1] [source] [64-bit] [smp:4:4] > [async-threads:10] [hipe] [kernel-poll:false] [dtrace] > > Eshell V7.0.1 (abort with ^G) > (foo@REDACTED)1> nodes(). > [] > (foo@REDACTED)2> net_adm:ping('shell@REDACTED'). > pong > (foo@REDACTED)3> nodes(). > ['shell@REDACTED'] > (foo@REDACTED)4> erl_epmd:open({9,1,1,1}). %% changed to > protect the innocent > {error,etimedout} > (foo@REDACTED)5> erl_epmd:open({127,0,0,1}). > {ok,#Port<0.760>} > (foo@REDACTED)6> > > observer eventually works correctly after the 60+ timeout, but I'd like > to know if there's any way to work around this? I'm hesitant to call it > a bug, but it does seem reasonable for observer & other tools to work on > systems that have FQDN set and are behind firewalls. > > A+ > Dave > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From max.lapshin@REDACTED Thu Jul 9 14:35:55 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 9 Jul 2015 15:35:55 +0300 Subject: [erlang-questions] GLIBC_2.12 not found when compile 18.0 on debian and run on SLES Message-ID: I've compiled erlang on debian 7 and launched in SLES. Tried to launch, speaking correctly because: *# */opt/flussonic/lib/erlang/erts-7.0/bin/beam.smp /opt/flussonic/lib/erlang/erts-7.0/bin/beam.smp: /lib64/libpthread.so.0: version `GLIBC_2.12' not found (required by /opt/flussonic/lib/erlang/erts-7.0/bin/beam.smp) Quick investigation with objdump -T beam.smp |grep GLIBC_2.12 showed that it is a pthread_setname_np erts/configure now is checking for pthread_setname_np and have found this function on "new" debian but it lacks on old suse. Is it possible to pass flags to erts/configure not to include pthread_setname_np? -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Thu Jul 9 15:02:45 2015 From: lukas@REDACTED (Lukas Larsson) Date: Thu, 9 Jul 2015 15:02:45 +0200 Subject: [erlang-questions] GLIBC_2.12 not found when compile 18.0 on debian and run on SLES In-Reply-To: References: Message-ID: Hello, On Thu, Jul 9, 2015 at 2:35 PM, Max Lapshin wrote: > Is it possible to pass flags to erts/configure not to include > pthread_setname_np? > I don't think is not possible. Your two options are (I think), manually comment out ETHR_HAVE_PTHREAD_SETNAME_NP_2 in erts/`config.guess`/config.h after configure, or patch configure to take an option. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From volkert@REDACTED Thu Jul 9 17:24:06 2015 From: volkert@REDACTED (Volkert) Date: Thu, 09 Jul 2015 17:24:06 +0200 Subject: [erlang-questions] Pipe Operator in Erlang? Message-ID: <559E9216.6000900@komponentenwerkstatt.de> Dear all, I really love the Pipe Operator |> in Elixir and F#. Are there any plans to have this in Erlang? Volkert From t@REDACTED Thu Jul 9 17:35:04 2015 From: t@REDACTED (Tristan Sloughter) Date: Thu, 09 Jul 2015 10:35:04 -0500 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: <559E9216.6000900@komponentenwerkstatt.de> References: <559E9216.6000900@komponentenwerkstatt.de> Message-ID: <1436456104.732571.319551801.48D54ED5@webmail.messagingengine.com> I hope not. Take this code for example f(X) -> g(Y) -> X + Y end. f(X, Y) -> X - Y. 3 |> f(2). 5 or 1? -- Tristan Sloughter t@REDACTED On Thu, Jul 9, 2015, at 10:24 AM, Volkert wrote: > Dear all, > > I really love the Pipe Operator |> in Elixir and F#. > > Are there any plans to have this in Erlang? > > Volkert > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From t@REDACTED Thu Jul 9 17:38:55 2015 From: t@REDACTED (Tristan Sloughter) Date: Thu, 09 Jul 2015 10:38:55 -0500 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: <1436456104.732571.319551801.48D54ED5@webmail.messagingengine.com> References: <559E9216.6000900@komponentenwerkstatt.de> <1436456104.732571.319551801.48D54ED5@webmail.messagingengine.com> Message-ID: <1436456335.733421.319554817.18EA4B0E@webmail.messagingengine.com> Yea yea, Fred pointed out a typo: f(X) -> fun(Y) -> X + Y end. f(X, Y) -> X - Y. 3 |> f(2). 5 or 1? -- Tristan Sloughter t@REDACTED On Thu, Jul 9, 2015, at 10:35 AM, Tristan Sloughter wrote: > I hope not. Take this code for example > > f(X) -> > g(Y) -> X + Y end. > > f(X, Y) -> > X - Y. > > 3 |> f(2). > > 5 or 1? > > -- > Tristan Sloughter > t@REDACTED > > On Thu, Jul 9, 2015, at 10:24 AM, Volkert wrote: > > Dear all, > > > > I really love the Pipe Operator |> in Elixir and F#. > > > > Are there any plans to have this in Erlang? > > > > Volkert > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From pierrefenoll@REDACTED Thu Jul 9 18:24:03 2015 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Thu, 9 Jul 2015 09:24:03 -0700 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: <1436456335.733421.319554817.18EA4B0E@webmail.messagingengine.com> References: <559E9216.6000900@komponentenwerkstatt.de> <1436456104.732571.319551801.48D54ED5@webmail.messagingengine.com> <1436456335.733421.319554817.18EA4B0E@webmail.messagingengine.com> Message-ID: I hope it will, but in addition to "curried" funs: fun f(2)/1 == fun(Y) -> f(2, Y) end. Thus: 3 |> fun f(2)/1 == -1 Cheers, -- Pierre Fenoll On 9 July 2015 at 08:38, Tristan Sloughter wrote: > Yea yea, Fred pointed out a typo: > > f(X) -> > fun(Y) -> X + Y end. > > f(X, Y) -> > X - Y. > > 3 |> f(2). > > 5 or 1? > > -- > Tristan Sloughter > t@REDACTED > > On Thu, Jul 9, 2015, at 10:35 AM, Tristan Sloughter wrote: > > I hope not. Take this code for example > > > > f(X) -> > > g(Y) -> X + Y end. > > > > f(X, Y) -> > > X - Y. > > > > 3 |> f(2). > > > > 5 or 1? > > > > -- > > Tristan Sloughter > > t@REDACTED > > > > On Thu, Jul 9, 2015, at 10:24 AM, Volkert wrote: > > > Dear all, > > > > > > I really love the Pipe Operator |> in Elixir and F#. > > > > > > Are there any plans to have this in Erlang? > > > > > > Volkert > > > > > > > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gianfranco.alongi@REDACTED Thu Jul 9 19:00:43 2015 From: gianfranco.alongi@REDACTED (Gianfranco Alongi) Date: Thu, 9 Jul 2015 19:00:43 +0200 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: References: <559E9216.6000900@komponentenwerkstatt.de> <1436456104.732571.319551801.48D54ED5@webmail.messagingengine.com> <1436456335.733421.319554817.18EA4B0E@webmail.messagingengine.com> Message-ID: Unless the language actually supports partial application, it feels odd to add this. BR G On Thu, Jul 9, 2015 at 6:24 PM, Pierre Fenoll wrote: > I hope it will, but in addition to "curried" funs: > > fun f(2)/1 == fun(Y) -> f(2, Y) end. > > Thus: > > 3 |> fun f(2)/1 == -1 > > > > > > Cheers, > -- > Pierre Fenoll > > > On 9 July 2015 at 08:38, Tristan Sloughter wrote: > >> Yea yea, Fred pointed out a typo: >> >> f(X) -> >> fun(Y) -> X + Y end. >> >> f(X, Y) -> >> X - Y. >> >> 3 |> f(2). >> >> 5 or 1? >> >> -- >> Tristan Sloughter >> t@REDACTED >> >> On Thu, Jul 9, 2015, at 10:35 AM, Tristan Sloughter wrote: >> > I hope not. Take this code for example >> > >> > f(X) -> >> > g(Y) -> X + Y end. >> > >> > f(X, Y) -> >> > X - Y. >> > >> > 3 |> f(2). >> > >> > 5 or 1? >> > >> > -- >> > Tristan Sloughter >> > t@REDACTED >> > >> > On Thu, Jul 9, 2015, at 10:24 AM, Volkert wrote: >> > > Dear all, >> > > >> > > I really love the Pipe Operator |> in Elixir and F#. >> > > >> > > Are there any plans to have this in Erlang? >> > > >> > > Volkert >> > > >> > > >> > > >> > > _______________________________________________ >> > > erlang-questions mailing list >> > > erlang-questions@REDACTED >> > > http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu Jul 9 19:12:17 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 9 Jul 2015 20:12:17 +0300 Subject: [erlang-questions] GLIBC_2.12 not found when compile 18.0 on debian and run on SLES In-Reply-To: References: Message-ID: Right now I've made dirty hack with sed. Will try to look if I can edit configure and add this patch. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Thu Jul 9 19:13:26 2015 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Thu, 9 Jul 2015 19:13:26 +0200 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: <559E9216.6000900@komponentenwerkstatt.de> References: <559E9216.6000900@komponentenwerkstatt.de> Message-ID: I agree with the general feelings of the thread. F# relies on currying. Elixir relies on macros (it is actually closer to the thread operator found in some lisps). Also F# has most of its standard library expecting the "subject" as last argument (due to currying). Elixir defaults to the first argument. Erlang, for better or worse, has them mixed (see binary and lists modules). For those reasons I don't think this feature makes much sense in Erlang. It is also hard to have elegant currying when functions are identified by name and arity. I believe Joxa is the only language in the Erlang VM with currying but they dropped the fun name/arity bit (citation needed). Language features are like a Jenga game, all the pieces need to fit together neatly, otherwise you have a crumbling mess. *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D On Thu, Jul 9, 2015 at 5:24 PM, Volkert wrote: > Dear all, > > I really love the Pipe Operator |> in Elixir and F#. > > Are there any plans to have this in Erlang? > > Volkert > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serge@REDACTED Thu Jul 9 19:44:14 2015 From: serge@REDACTED (Serge Aleynikov) Date: Thu, 9 Jul 2015 13:44:14 -0400 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: <559E9216.6000900@komponentenwerkstatt.de> References: <559E9216.6000900@komponentenwerkstatt.de> Message-ID: Though it's a bit off topic, but I'd love to see a different kind of application of '|' supported by syntax (yet probably ';' would be more in the "Erlang" spirit for the task): case SomeVar of a | b | c -> true; _ -> false end. On Thu, Jul 9, 2015 at 11:24 AM, Volkert wrote: > Dear all, > > I really love the Pipe Operator |> in Elixir and F#. > > Are there any plans to have this in Erlang? > > Volkert > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Thu Jul 9 19:44:55 2015 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 9 Jul 2015 13:44:55 -0400 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: References: <559E9216.6000900@komponentenwerkstatt.de> Message-ID: <20150709174454.GA24852@ferdair.local> On 07/09, Jos? Valim wrote: >I agree with the general feelings of the thread. F# relies on currying. >Elixir relies on macros (it is actually closer to the thread operator found >in some lisps). > >Also F# has most of its standard library expecting the "subject" as last >argument (due to currying). Elixir defaults to the first argument. Erlang, >for better or worse, has them mixed (see binary and lists modules). > I agree with this. The only workable form I could imagine could go something like this with macros, without major changes to the language: pipe(Init, a(_, X), b(Y, _), c(_, _, 41.12)), where '_' gets replaced by the threaded in 'Init' state. The obvious problem is lack of composition: pipe(Init, pipe(Init2, a(_))) where you can't know if a(_) refers to Init or Init2 at a glance without knowing precise evaluation rules. Generally I haven't felt I missed this feature too much in Erlang. In the cases where it could really be nice, I resorted to using: pipe(Init, Funs) -> lists:foldl(fun(F, State) -> F(State) end, Init, Funs). pipe(Init, [fun(S) -> set(S, 230) end, fun(S) -> update(S) end, fun(S) -> output(S), S end]). Obviously, one could easily come and swoop in with a macro: pipe(Init, $$, % $$ is replaced by 'Init' set($$, 230), update($$), fun() -> output($$), $$ end) The problem is that composition is not obvious: pipe(Init, $$, pipe($$, $_, set($_, 230), update($_), fun() -> output($$), $_ end)) In that case, the last 'output($$)' would try to print the literal '$$' value instead of the substituted one given underneath it all, it's equivalent to: pipe(Init, [fun(S) -> pipe(S, [...]) end]). Woops! But what's cool? Add in a maybe pipe! maybe_pipe(Init, Funs) -> %% use throws and catches if you wanna go faster lists:foldl(fun(F, {ok, State}) -> F(State) ; (F, {error, Err}) -> {error, Err} end, Init, Funs). And with the same set of parse transforms you can change: f(X0) -> case g(X0) of {ok, X1} -> case h(X1) of {ok, X2} -> {ok, X2}; Err = {error, _} -> Err end; Err = {error, _} -> Err end. Into: f(X0) -> maybe_pipe(X0, $$, g($$), h($$)). Which translates to: f(X0) -> maybe_pipe(X0, [fun(X) -> g(X) end, fun(X) -> h(X) end]). And all of this is doable today in library code for or from anyone, doesn't fundamentally change the shape of Erlang, although it will definitely be more confusing for newcomers or people not familiar with the concept. All it needs is someone angry enough to do it. I don't believe (from doing experiments like Hubble[1]) that it's particularly hard. Regards, Fred. [1]: https://github.com/ferd/hubble From eric.pailleau@REDACTED Thu Jul 9 20:34:30 2015 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Thu, 09 Jul 2015 20:34:30 +0200 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: Message-ID: <6cc8f74b-3dee-4fee-908b-8758cc9eb589@email.android.com> Hello, A bit off topic too, but I would really love pipe as synchronous message passing. A = Pid | {somemessage, somevalue }. Instead sending message with ! and wait with receive statement. Regards Le?9 juil. 2015 19:44, Serge Aleynikov a ?crit?: > > Though it's a bit off topic, but I'd love to see a different kind of application of '|' supported by syntax (yet probably ';' would be more in the "Erlang" spirit for the task): > > case SomeVar of > ? a | b | c -> > ? ? true; > ? _ -> > ? ? false > end. > > > > On Thu, Jul 9, 2015 at 11:24 AM, Volkert wrote: >> >> Dear all, >> >> I really love the Pipe Operator |> in Elixir and F#. >> >> Are there any plans to have this in Erlang? >> >> Volkert >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From henry@REDACTED Thu Jul 9 21:07:06 2015 From: henry@REDACTED (=?UTF-8?B?SGVucsO9IMOew7NyIEJhbGR1cnNzb24=?=) Date: Thu, 9 Jul 2015 19:07:06 +0000 Subject: [erlang-questions] QLC in stdlib-2.5 broken? Message-ID: Hello I'm trying to use QLC, but even including the header gives me a compiler error: ? ~ cat test.erl -module(test). -include_lib("stdlib/include/qlc.hrl"). -export([init/0]). init() -> ok. ? ~ erlc test.erl test.erl: error in parse transform 'qlc': {{case_clause,[{5}]}, [{qlc_pt,lc_loc,2, [{file,"qlc_pt.erl"},{line,341}]}, {qlc_pt,'-lc_messages/2-lc$^1/1-1-',2, [{file,"qlc_pt.erl"},{line,337}]}, {qlc_pt,'-lc_messages/2-lc$^0/1-0-',2, [{file,"qlc_pt.erl"},{line,337}]}, {qlc_pt,compile_messages,4, [{file,"qlc_pt.erl"},{line,305}]}, {qlc_pt,parse_transform,2, [{file,"qlc_pt.erl"},{line,95}]}, {compile,'-foldl_transform/2-anonymous-2-', 2, [{file,"compile.erl"},{line,932}]}, {compile,foldl_transform,2, [{file,"compile.erl"},{line,934}]}, {compile,'-internal_comp/4-anonymous-1-',2, [{file,"compile.erl"},{line,295}]}]} ? ~ I'm using the version erlang.org releases for OS/X: [{release,"Erlang/OTP","18","7.0", [{kernel,"4.0","/usr/local/lib/erlang/lib/kernel-4.0"}, {stdlib,"2.5","/usr/local/lib/erlang/lib/stdlib-2.5"}, {sasl,"2.5","/usr/local/lib/erlang/lib/sasl-2.5"}], permanent}]. Regards, Henry. Henr? ??r Baldursson / Programmer henry@REDACTED / +354 869-8264 <+3548698264> CrankWheel +354 534-5700 <+3545345700> ?rm?li 6, 108 Reykjav?k, Iceland http://crankwheel.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Jul 9 21:52:08 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Thu, 09 Jul 2015 21:52:08 +0200 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: References: <559E9216.6000900@komponentenwerkstatt.de> Message-ID: <559ED0E8.6090207@ninenines.eu> Closer to Erlang syntax would be something like: case SomeVar of V when V in [a, b, c] -> true; _ -> false end Or better yet: if SomeVar in [a, b, c] -> true; true -> false end You can generalize with variables (SomeVar in List) though the compiler probably won't be able to optimize those away. Of course the whole "returning a boolean" makes no sense, you probably want a function call in its place instead. On 07/09/2015 07:44 PM, Serge Aleynikov wrote: > Though it's a bit off topic, but I'd love to see a different kind of > application of '|' supported by syntax (yet probably ';' would be more > in the "Erlang" spirit for the task): > > case SomeVar of > a | b | c -> > true; > _ -> > false > end. > > > > On Thu, Jul 9, 2015 at 11:24 AM, Volkert > > wrote: > > Dear all, > > I really love the Pipe Operator |> in Elixir and F#. > > Are there any plans to have this in Erlang? > > Volkert > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From serge@REDACTED Thu Jul 9 22:03:36 2015 From: serge@REDACTED (Serge Aleynikov) Date: Thu, 9 Jul 2015 16:03:36 -0400 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: <559ED0E8.6090207@ninenines.eu> References: <559E9216.6000900@komponentenwerkstatt.de> <559ED0E8.6090207@ninenines.eu> Message-ID: I believe this is not quite the same thing as your proposition would require for each a, b, c to be expressions that could be used in guards whereas I was looking for a generic way to be able to join several matches in a single 'case' branch (in the spirit of the C's multiple subsequent case statements): case {Ctrl, Char} of {ctrl, _} -> ignore; {_, $\n} | {_, $\r} -> end_of_line; Other -> Other end. On Thu, Jul 9, 2015 at 3:52 PM, Lo?c Hoguin wrote: > Closer to Erlang syntax would be something like: > > case SomeVar of > V when V in [a, b, c] -> true; > _ -> false > end > > Or better yet: > > if > SomeVar in [a, b, c] -> true; > true -> false > end > > You can generalize with variables (SomeVar in List) though the compiler > probably won't be able to optimize those away. > > Of course the whole "returning a boolean" makes no sense, you probably > want a function call in its place instead. > > On 07/09/2015 07:44 PM, Serge Aleynikov wrote: > >> Though it's a bit off topic, but I'd love to see a different kind of >> application of '|' supported by syntax (yet probably ';' would be more >> in the "Erlang" spirit for the task): >> >> case SomeVar of >> a | b | c -> >> true; >> _ -> >> false >> end. >> >> >> >> On Thu, Jul 9, 2015 at 11:24 AM, Volkert >> > > wrote: >> >> Dear all, >> >> I really love the Pipe Operator |> in Elixir and F#. >> >> Are there any plans to have this in Erlang? >> >> Volkert >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > -------------- next part -------------- An HTML attachment was scrubbed... URL: From volkert@REDACTED Thu Jul 9 22:05:42 2015 From: volkert@REDACTED (Volkert) Date: Thu, 09 Jul 2015 22:05:42 +0200 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: References: <559E9216.6000900@komponentenwerkstatt.de> Message-ID: <559ED416.7030408@komponentenwerkstatt.de> yes, i see the problems .... may be my question was a bit silly ;-) Am 09.07.2015 um 19:13 schrieb Jos? Valim: > I agree with the general feelings of the thread. F# relies on > currying. Elixir relies on macros (it is actually closer to the thread > operator found in some lisps). > > Also F# has most of its standard library expecting the "subject" as > last argument (due to currying). Elixir defaults to the first > argument. Erlang, for better or worse, has them mixed (see binary and > lists modules). > > For those reasons I don't think this feature makes much sense in > Erlang. It is also hard to have elegant currying when functions are > identified by name and arity. I believe Joxa is the only language in > the Erlang VM with currying but they dropped the fun name/arity bit > (citation needed). > > Language features are like a Jenga game, all the pieces need to fit > together neatly, otherwise you have a crumbling mess. > > > > *Jos? Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Director of R&D > > On Thu, Jul 9, 2015 at 5:24 PM, Volkert > > wrote: > > Dear all, > > I really love the Pipe Operator |> in Elixir and F#. > > Are there any plans to have this in Erlang? > > Volkert > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Thu Jul 9 22:13:25 2015 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 9 Jul 2015 22:13:25 +0200 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: <6cc8f74b-3dee-4fee-908b-8758cc9eb589@email.android.com> References: <6cc8f74b-3dee-4fee-908b-8758cc9eb589@email.android.com> Message-ID: <7C878A2A-A9CB-4C90-A0FD-C9A57E72C645@feuerlabs.com> > On 09 Jul 2015, at 20:34, ?ric Pailleau wrote: > > A bit off topic too, but I would really love pipe as synchronous message passing. > A = Pid | {somemessage, somevalue }. > > Instead sending message with ! and wait with receive statement. That sounds like Joe Armstrongs ?!!? operator, which he proposed while working on the notion of ?conteptual integrity?. https://www.sics.se/~joe/talks/acm2003.pdf BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From anthonym@REDACTED Thu Jul 9 22:38:17 2015 From: anthonym@REDACTED (Anthony Molinaro) Date: Thu, 9 Jul 2015 13:38:17 -0700 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: References: <559E9216.6000900@komponentenwerkstatt.de> <559ED0E8.6090207@ninenines.eu> Message-ID: <59898F8B-6409-4DB1-B83E-1BA672D0FE6D@alumni.caltech.edu> > On Jul 9, 2015, at 1:03 PM, Serge Aleynikov wrote: > > case {Ctrl, Char} of > {ctrl, _} -> > ignore; > {_, $\n} | {_, $\r} -> > end_of_line; > Other -> > Other > end. > This can be achieved now with a guard like so case {Ctrl, Char} of {ctrl, _} -> ignore; {_, C} where C =:= $\n; C =:= $\r -> end_of_line; Other -> Other end Granted it's not as compact... -Anthony -------------- next part -------------- An HTML attachment was scrubbed... URL: From be.dmitry@REDACTED Thu Jul 9 23:26:56 2015 From: be.dmitry@REDACTED (Dmitry Belyaev) Date: Fri, 10 Jul 2015 07:26:56 +1000 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: <20150709174454.GA24852@ferdair.local> References: <559E9216.6000900@komponentenwerkstatt.de> <20150709174454.GA24852@ferdair.local> Message-ID: <4D110B2D-539F-4C6C-88F6-926BACAE132E@gmail.com> I'm surprised nobody mentioned erlando: https://github.com/rabbitmq/erlando -- Best wishes, Dmitry Belyaev On 10 July 2015 3:44:55 AM AEST, Fred Hebert wrote: >On 07/09, Jos? Valim wrote: >>I agree with the general feelings of the thread. F# relies on >currying. >>Elixir relies on macros (it is actually closer to the thread operator >found >>in some lisps). >> >>Also F# has most of its standard library expecting the "subject" as >last >>argument (due to currying). Elixir defaults to the first argument. >Erlang, >>for better or worse, has them mixed (see binary and lists modules). >> > >I agree with this. The only workable form I could imagine could go >something like this with macros, without major changes to the language: > >pipe(Init, a(_, X), b(Y, _), c(_, _, 41.12)), where '_' gets replaced >by >the threaded in 'Init' state. The obvious problem is lack of >composition: > >pipe(Init, pipe(Init2, a(_))) where you can't know if a(_) refers to >Init or Init2 at a glance without knowing precise evaluation rules. > >Generally I haven't felt I missed this feature too much in Erlang. In >the cases where it could really be nice, I resorted to using: > > pipe(Init, Funs) -> > lists:foldl(fun(F, State) -> F(State) end, Init, Funs). > > pipe(Init, > [fun(S) -> set(S, 230) end, > fun(S) -> update(S) end, > fun(S) -> output(S), S end]). > >Obviously, one could easily come and swoop in with a macro: > > pipe(Init, $$, % $$ is replaced by 'Init' > set($$, 230), > update($$), > fun() -> output($$), $$ end) > >The problem is that composition is not obvious: > > pipe(Init, $$, > pipe($$, $_, > set($_, 230), > update($_), > fun() -> output($$), $_ end)) > >In that case, the last 'output($$)' would try to print the literal '$$' > >value instead of the substituted one given underneath it all, it's >equivalent to: > > pipe(Init, [fun(S) -> pipe(S, [...]) end]). > >Woops! But what's cool? Add in a maybe pipe! > > maybe_pipe(Init, Funs) -> > %% use throws and catches if you wanna go faster > lists:foldl(fun(F, {ok, State}) -> F(State) > ; (F, {error, Err}) -> {error, Err} end, > Init, > Funs). > >And with the same set of parse transforms you can change: > > f(X0) -> > case g(X0) of > {ok, X1} -> > case h(X1) of > {ok, X2} -> {ok, X2}; > Err = {error, _} -> Err > end; > Err = {error, _} -> > Err > end. > >Into: > > f(X0) -> > maybe_pipe(X0, $$, g($$), h($$)). > >Which translates to: > > f(X0) -> > maybe_pipe(X0, > [fun(X) -> g(X) end, > fun(X) -> h(X) end]). > >And all of this is doable today in library code for or from anyone, >doesn't fundamentally change the shape of Erlang, although it will >definitely be more confusing for newcomers or people not familiar with >the concept. > >All it needs is someone angry enough to do it. I don't believe (from >doing experiments like Hubble[1]) that it's particularly hard. > >Regards, >Fred. > >[1]: https://github.com/ferd/hubble > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Fri Jul 10 00:24:42 2015 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Fri, 10 Jul 2015 00:24:42 +0200 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: <7C878A2A-A9CB-4C90-A0FD-C9A57E72C645@feuerlabs.com> Message-ID: <6ead097b-8f89-43b2-bb12-ca984c2f8424@email.android.com> Hi, Yes, I forgot this. Joe's approach is more general. My wish is more prosaic, to let Erlang code syntax more compact and readable. Erlang power is in asynchronous messaging, and ! is its operator. In the meantime some behaviours have synchronous call but need function calls, and no operator available. I suppose the reason is that not any Pid can answer back a synchronous answer. In such case an exception could be raised. A = catch Pid |? {somemessage, somevalue }. This would probably need a timeout. On other hand, ! + receive may wait forever. | could do the same. Le?9 juil. 2015 22:13, Ulf Wiger a ?crit?: > > > > On 09 Jul 2015, at 20:34, ?ric Pailleau wrote: > > > > A bit off topic too, but I would really love pipe as synchronous message passing. > > A = Pid |? {somemessage, somevalue }. > > > > Instead sending message with ! and wait with receive statement. > > That sounds like Joe Armstrongs ?!!? operator, which he proposed while working on the notion of ?conteptual integrity?. > > https://www.sics.se/~joe/talks/acm2003.pdf > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > From mononcqc@REDACTED Fri Jul 10 01:18:08 2015 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 9 Jul 2015 19:18:08 -0400 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: <6ead097b-8f89-43b2-bb12-ca984c2f8424@email.android.com> References: <7C878A2A-A9CB-4C90-A0FD-C9A57E72C645@feuerlabs.com> <6ead097b-8f89-43b2-bb12-ca984c2f8424@email.android.com> Message-ID: <20150709231805.GA1339@ferdmbp.local> On 07/10, ?ric Pailleau wrote: >A = catch Pid |? {somemessage, somevalue }. >This would probably need a timeout. On other hand, ! + receive may wait forever. | could do the same. A = [catch Pid | {somemessage, somevalue}]. Is A equal to: a) [Pid | {somemessage, somevalue}] (improper list) b) [Result] (the received message) It also ignores that two processes may have more than two communications going on at the same time, and that without a selective receive, it's not obvious that anything would match properly. From lloyd@REDACTED Fri Jul 10 02:05:08 2015 From: lloyd@REDACTED (lloyd@REDACTED) Date: Thu, 9 Jul 2015 20:05:08 -0400 (EDT) Subject: [erlang-questions] Exporting a record type Message-ID: <1436486708.04388403@apps.rackspace.com> Hi folks, I've asked Lo?c Hoguin directly about this, but I thought an elaboration and discussion might be of interest to many on this list. In Lo?c's new book, The Erlanger Playbook, he advocates against using including records through *.hrl. (pp 40-41) Instead, he recommends using types: -record(state, {}). -opaque state() :: #state{}. He leaves hanging the question of how to access this record from another module. The Erlang Types and Function Specifications doc suggests that the type can be exported using -export_type(...). But, to my feeble mind, the exposition is less than clear. My case and problem is as follows: -record(book, { ... yada yada (some seven fields) } I define the record, then: -opaque book() :: #book{}. -export_type([book/0]). Now, I access a remote db in a different module and want to instantiate a book record, but it's not at all clear to me how to do so. If I do the following in the console... wg_books:book(). I get: ** exception error: undefined function wg_dets_books:book/0 This seems to be an all-to-little documented corner of Erlang. Is there a kind soul who can help show the way? Many thanks, LRP From bob@REDACTED Fri Jul 10 02:17:08 2015 From: bob@REDACTED (Bob Ippolito) Date: Thu, 9 Jul 2015 17:17:08 -0700 Subject: [erlang-questions] Exporting a record type In-Reply-To: <1436486708.04388403@apps.rackspace.com> References: <1436486708.04388403@apps.rackspace.com> Message-ID: On Thu, Jul 9, 2015 at 5:05 PM, wrote: > > In Lo?c's new book, The Erlanger Playbook, he advocates against using > including records through *.hrl. (pp 40-41) > > Instead, he recommends using types: > > -record(state, {}). > -opaque state() :: #state{}. > This looks like the fairly common recommendation of: don't export records. Treat them as an opaque data type, and provide functions for working with them. > He leaves hanging the question of how to access this record from another > module. > > The Erlang Types and Function Specifications doc suggests that the type > can be exported using -export_type(...). But, to my feeble mind, the > exposition is less than clear. > > My case and problem is as follows: > > -record(book, { > ... yada yada (some seven fields) > } > > I define the record, then: > > -opaque book() :: #book{}. > > -export_type([book/0]). > > Now, I access a remote db in a different module and want to instantiate a > book record, but it's not at all clear to me how to do so. > You don't. You have a function in the module that defines book that can create a book record, and other functions to manipulate them. An example of this style would be the dict module. It uses records internally to implement a dictionary data structure, but you don't have direct access to the record(s) that it uses. Just functions to work with them. -bob -------------- next part -------------- An HTML attachment was scrubbed... URL: From elbrujohalcon@REDACTED Fri Jul 10 02:56:22 2015 From: elbrujohalcon@REDACTED (Fernando Benavides) Date: Thu, 9 Jul 2015 17:56:22 -0700 Subject: [erlang-questions] Exporting a record type In-Reply-To: References: <1436486708.04388403@apps.rackspace.com> Message-ID: Lo?c will correct me if I'm wrong but the idea is to use records *only* as an internal representation of your data types. For instance, let's say in your system you have shoes (for lack of a better thing). You'll then create a module called shoes.erl: -module(shoes). -export([new/2, id/1, color/1, color/2, size/1, size/2]). -record(shoe, {id = new_id(), color, size}). -opaque shoe() :: #shoe{}. -export_type([shoe/0]). new(Color, Size) -> #shoe{color = Color, size = Size}. id(#shoe{id = Id}) -> Id. color(#shoe{color = Color}) -> Color. color(Shoe, Color) -> Shoe#shoe{color = Color}. size(#shoe{size = Size}) -> Size. size(Shoe, Size) -> Shoe#shoe{size = Size}. new_id() -> implement:it(yourself). Whenever you need to work with shoes anywhere in your system, you just use the functions exported by the module. Let's say you have a list of shoes and you only want those of a particular size: filter_by_size(Shoes, Size) -> [Shoe || Shoe <- Shoes, shoes:size(Shoe) == Size]. Hope this helps. *Fernando Benavides * On Thu, Jul 9, 2015 at 5:17 PM, Bob Ippolito wrote: > On Thu, Jul 9, 2015 at 5:05 PM, wrote: > >> >> In Lo?c's new book, The Erlanger Playbook, he advocates against using >> including records through *.hrl. (pp 40-41) >> >> Instead, he recommends using types: >> >> -record(state, {}). >> -opaque state() :: #state{}. >> > > This looks like the fairly common recommendation of: don't export records. > Treat them as an opaque data type, and provide functions for working with > them. > > >> He leaves hanging the question of how to access this record from another >> module. >> >> The Erlang Types and Function Specifications doc suggests that the type >> can be exported using -export_type(...). But, to my feeble mind, the >> exposition is less than clear. >> >> My case and problem is as follows: >> >> -record(book, { >> ... yada yada (some seven fields) >> } >> >> I define the record, then: >> >> -opaque book() :: #book{}. >> >> -export_type([book/0]). >> >> Now, I access a remote db in a different module and want to instantiate a >> book record, but it's not at all clear to me how to do so. >> > > You don't. You have a function in the module that defines book that can > create a book record, and other functions to manipulate them. > > An example of this style would be the dict module. It uses records > internally to implement a dictionary data structure, but you don't have > direct access to the record(s) that it uses. Just functions to work with > them. > > -bob > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Fri Jul 10 03:51:25 2015 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Thu, 9 Jul 2015 21:51:25 -0400 Subject: [erlang-questions] Exporting a record type In-Reply-To: References: <1436486708.04388403@apps.rackspace.com> Message-ID: <39011972-D94D-416B-BDAE-2DA05CAC43A7@writersglen.com> Thanks Bob and Fernando, Fernando, your code example is a great help. Thanks again, Lloyd Sent from my iPad > On Jul 9, 2015, at 8:56 PM, Fernando Benavides wrote: > > Lo?c will correct me if I'm wrong but the idea is to use records *only* as an internal representation of your data types. > For instance, let's say in your system you have shoes (for lack of a better thing). You'll then create a module called shoes.erl: > > -module(shoes). > > -export([new/2, id/1, color/1, color/2, size/1, size/2]). > > -record(shoe, {id = new_id(), color, size}). > -opaque shoe() :: #shoe{}. > -export_type([shoe/0]). > > new(Color, Size) -> #shoe{color = Color, size = Size}. > id(#shoe{id = Id}) -> Id. > color(#shoe{color = Color}) -> Color. > color(Shoe, Color) -> Shoe#shoe{color = Color}. > size(#shoe{size = Size}) -> Size. > size(Shoe, Size) -> Shoe#shoe{size = Size}. > > new_id() -> implement:it(yourself). > > Whenever you need to work with shoes anywhere in your system, you just use the functions exported by the module. Let's say you have a list of shoes and you only want those of a particular size: > > filter_by_size(Shoes, Size) -> > [Shoe || Shoe <- Shoes, shoes:size(Shoe) == Size]. > > Hope this helps. > > > > Fernando Benavides > >> On Thu, Jul 9, 2015 at 5:17 PM, Bob Ippolito wrote: >>> On Thu, Jul 9, 2015 at 5:05 PM, wrote: >>> >>> In Lo?c's new book, The Erlanger Playbook, he advocates against using including records through *.hrl. (pp 40-41) >>> >>> Instead, he recommends using types: >>> >>> -record(state, {}). >>> -opaque state() :: #state{}. >> >> This looks like the fairly common recommendation of: don't export records. Treat them as an opaque data type, and provide functions for working with them. >> >>> He leaves hanging the question of how to access this record from another module. >>> >>> The Erlang Types and Function Specifications doc suggests that the type can be exported using -export_type(...). But, to my feeble mind, the exposition is less than clear. >>> >>> My case and problem is as follows: >>> >>> -record(book, { >>> ... yada yada (some seven fields) >>> } >>> >>> I define the record, then: >>> >>> -opaque book() :: #book{}. >>> >>> -export_type([book/0]). >>> >>> Now, I access a remote db in a different module and want to instantiate a book record, but it's not at all clear to me how to do so. >> >> You don't. You have a function in the module that defines book that can create a book record, and other functions to manipulate them. >> >> An example of this style would be the dict module. It uses records internally to implement a dictionary data structure, but you don't have direct access to the record(s) that it uses. Just functions to work with them. >> >> -bob >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jul 10 04:01:34 2015 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 10 Jul 2015 14:01:34 +1200 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: References: <559E9216.6000900@komponentenwerkstatt.de> <1436456104.732571.319551801.48D54ED5@webmail.messagingengine.com> <1436456335.733421.319554817.18EA4B0E@webmail.messagingengine.com> Message-ID: <821E61F3-A790-428E-B96B-EF651FE5E340@cs.otago.ac.nz> For what it's worth, I implemented |> in my Smalltalk system. It's astonishing how useful it isn't. I use o in ML and I use . in Haskell but in Smalltalk it just. does. not. fit. In the relevant respects (no built-in currying, distinction between named and anonymous functions) Erlang and Smalltalk are so similar that I'd expect |> to be just as useless in Erlang as I found it in Smalltalk. (When I say |> was useless, I'm being euphemistic.) From indicasloth@REDACTED Fri Jul 10 05:38:54 2015 From: indicasloth@REDACTED (Indica Sloth) Date: Thu, 9 Jul 2015 20:38:54 -0700 Subject: [erlang-questions] Issues with epmd_publish Message-ID: I am having a strange problem trying to bring up a C-Node Server. I am following the example from http://www.erlang.org/doc/tutorial/cnode.html. I found that I would fail at if (erl_publish(port) == -1) erl_err_quit("erl_publish"); Running epmd in debug mode showed the following message after the failure: got partial packet only on file descriptor 4 (0) gdb showed that the application was getting an error in epmd_publish.c when checking for a valid response. if (((res=get8(s)) != EI_EPMD_ALIVE2_RESP)) Have you seen this before? Do I have something configured in the system wrong? I am so confused. Thanks a bunch. -------------- next part -------------- An HTML attachment was scrubbed... URL: From henry@REDACTED Fri Jul 10 00:19:03 2015 From: henry@REDACTED (=?UTF-8?B?SGVucsO9IMOew7NyIEJhbGR1cnNzb24=?=) Date: Thu, 9 Jul 2015 22:19:03 +0000 Subject: [erlang-questions] QLC in stdlib-2.5 broken? In-Reply-To: References: Message-ID: Fixed it myself. Not sure what the problem was but a refresh of Erlang 18 from erlang solutions did the trick. - Henry Henr? ??r Baldursson / Programmer henry@REDACTED / +354 869-8264 <+3548698264> CrankWheel +354 534-5700 <+3545345700> ?rm?li 6, 108 Reykjav?k, Iceland http://crankwheel.com/ On Thu, Jul 9, 2015 at 6:55 PM, Henr? ??r Baldursson wrote: > Hello > > I'm trying to use QLC, but even including the header gives me a compiler > error: > ? ~ cat test.erl > -module(test). > -include_lib("stdlib/include/qlc.hrl"). > -export([init/0]). > > init() -> > ok. > ? ~ erlc test.erl > test.erl: error in parse transform 'qlc': {{case_clause,[{5}]}, > [{qlc_pt,lc_loc,2, > [{file,"qlc_pt.erl"},{line,341}]}, > {qlc_pt,'-lc_messages/2-lc$^1/1-1-',2, > [{file,"qlc_pt.erl"},{line,337}]}, > {qlc_pt,'-lc_messages/2-lc$^0/1-0-',2, > [{file,"qlc_pt.erl"},{line,337}]}, > {qlc_pt,compile_messages,4, > [{file,"qlc_pt.erl"},{line,305}]}, > {qlc_pt,parse_transform,2, > [{file,"qlc_pt.erl"},{line,95}]}, > > {compile,'-foldl_transform/2-anonymous-2-', > 2, > [{file,"compile.erl"},{line,932}]}, > {compile,foldl_transform,2, > [{file,"compile.erl"},{line,934}]}, > > {compile,'-internal_comp/4-anonymous-1-',2, > [{file,"compile.erl"},{line,295}]}]} > ? ~ > > I'm using the version erlang.org releases for OS/X: > [{release,"Erlang/OTP","18","7.0", > [{kernel,"4.0","/usr/local/lib/erlang/lib/kernel-4.0"}, > {stdlib,"2.5","/usr/local/lib/erlang/lib/stdlib-2.5"}, > {sasl,"2.5","/usr/local/lib/erlang/lib/sasl-2.5"}], > permanent}]. > > Regards, > Henry. > > > Henr? ??r Baldursson / Programmer > henry@REDACTED / +354 869-8264 <+3548698264> > > CrankWheel > +354 534-5700 <+3545345700> > ?rm?li 6, 108 Reykjav?k, Iceland > http://crankwheel.com/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierrefenoll@REDACTED Fri Jul 10 07:17:07 2015 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Thu, 9 Jul 2015 22:17:07 -0700 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: <821E61F3-A790-428E-B96B-EF651FE5E340@cs.otago.ac.nz> References: <559E9216.6000900@komponentenwerkstatt.de> <1436456104.732571.319551801.48D54ED5@webmail.messagingengine.com> <1436456335.733421.319554817.18EA4B0E@webmail.messagingengine.com> <821E61F3-A790-428E-B96B-EF651FE5E340@cs.otago.ac.nz> Message-ID: ROK, A use case I often bump into is modifying some state using the accessors already written for this structure. For example, modifying a record: PvtFuns = [ fun add_pvt_type/1 , fun add_pvt_vsn/1 , fun maybe_add_pvt_api_key/1 , fun maybe_add_pvt_tree/1 , fun add_pvt_enabled/1 ], NewRecord = lists:foldl(fun(F, C) -> F(C) end, Record, PvtFuns), Or some opaque structure, with accessors written more seriously: NewContext = cb_context:setters(Context, [ {fun cb_context:set_doc/2, []} , {fun cb_context:set_resp_status/2, 'success'} , {fun cb_context:set_resp_data/2, []} , {fun cb_context:set_resp_etag/2, 'undefined'} ]) Where setters/2 is a call to lists:foldl/3. A pipe operator would turn this structure-specific code into something more readable and editable too. Maybe like something like this: NewContext = Context |> cb_context:set_doc(_, []) |> cb_context:set_resp_status(_, success) |> cb_context:set_resp_data(_, []) |> cb_context:set_resp_etag(_, undefined) Where m:f(_, A2) is fun (_1) -> m:f(_1, A2) end; thus redefining _ for pipes (in non-matching contexts?). Scoping rules for nested pipes looks like a hard problem, but I am not sure nestedness would be a great thing to have? Cheers, -- Pierre Fenoll On 9 July 2015 at 19:01, Richard A. O'Keefe wrote: > For what it's worth, I implemented |> in my Smalltalk system. > It's astonishing how useful it isn't. > I use o in ML and I use . in Haskell but in Smalltalk it > just. does. not. fit. In the relevant respects (no built-in > currying, distinction between named and anonymous functions) > Erlang and Smalltalk are so similar that I'd expect |> to be > just as useless in Erlang as I found it in Smalltalk. > > (When I say |> was useless, I'm being euphemistic.) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Fri Jul 10 07:22:03 2015 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Fri, 10 Jul 2015 07:22:03 +0200 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: <20150709231805.GA1339@ferdmbp.local> Message-ID: Le?10 juil. 2015 01:18, Fred Hebert a ?crit?: > > On 07/10, ?ric Pailleau wrote: > >A = catch Pid |? {somemessage, somevalue }. > >This would probably need a timeout. On other hand,? ! + receive may wait forever. | could do the same. > > A = [catch Pid | {somemessage, somevalue}]. > > Is A equal to: > > a) [Pid | {somemessage, somevalue}] (improper list) > b) [Result] (the received message) > Yes, you are right. But I do not request that pipe must be the operator. If !! Is better, I do not care. The need is more important than the syntax itself. > It also ignores that two processes may have more than two communications > going on at the same time, and that without a selective receive, it's > not obvious that anything would match properly. I m not sure to understand there. We are talking about synchronous message. When using gen_server:call you do not receive answer to another request, but yours. From eric.pailleau@REDACTED Fri Jul 10 07:40:45 2015 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Fri, 10 Jul 2015 07:40:45 +0200 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: <1436456335.733421.319554817.18EA4B0E@webmail.messagingengine.com> Message-ID: Le?9 juil. 2015 17:38, Tristan Sloughter a ?crit?: > > Yea yea, Fred pointed out a typo: > > f(X) -> > ??? fun(Y) -> X + Y end. > > f(X, Y) -> > ??? X - Y. > > 3 |> f(2). Following Fred remark, [ 3 |> f(3)] would be difficult for a parser to understand between the new operator and a (faulty in such case) improper list. As far pipe is already used in list comprehension, this character shouldn't be used for another purpose. > > 5 or 1? > > -- > ? Tristan Sloughter > ? t@REDACTED > > On Thu, Jul 9, 2015, at 10:35 AM, Tristan Sloughter wrote: > > I hope not. Take this code for example > > > > f(X) -> > >???? g(Y) -> X + Y end. > > > > f(X, Y) -> > >???? X - Y. > > > > 3 |> f(2). > > > > 5 or 1? > > > > -- > >?? Tristan Sloughter > >?? t@REDACTED > > > > On Thu, Jul 9, 2015, at 10:24 AM, Volkert wrote: > > > Dear all, > > > > > > I really love the Pipe Operator |> in Elixir and F#. > > > > > > Are there any plans to have this in Erlang? > > > > > > Volkert > > > > > > > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Fri Jul 10 08:22:11 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 10 Jul 2015 09:22:11 +0300 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: References: <559E9216.6000900@komponentenwerkstatt.de> <1436456104.732571.319551801.48D54ED5@webmail.messagingengine.com> <1436456335.733421.319554817.18EA4B0E@webmail.messagingengine.com> <821E61F3-A790-428E-B96B-EF651FE5E340@cs.otago.ac.nz> Message-ID: > PvtFuns = [ fun add_pvt_type/1 > , fun add_pvt_vsn/1 > , fun maybe_add_pvt_api_key/1 > , fun maybe_add_pvt_tree/1 > , fun add_pvt_enabled/1 > ], > NewRecord = lists:foldl(fun(F, C) -> F(C) end, Record, PvtFuns), > I remember how I've got 10-20% speedup when I changed this code in flussonic to plain function calls =) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jul 10 08:27:45 2015 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 10 Jul 2015 18:27:45 +1200 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: References: <559E9216.6000900@komponentenwerkstatt.de> <1436456104.732571.319551801.48D54ED5@webmail.messagingengine.com> <1436456335.733421.319554817.18EA4B0E@webmail.messagingengine.com> <821E61F3-A790-428E-B96B-EF651FE5E340@cs.otago.ac.nz> Message-ID: On 10/07/2015, at 5:17 pm, Pierre Fenoll wrote: > A use case I often bump into is modifying some state using the accessors already written for this structure. > > For example, modifying a record: > > PvtFuns = [ fun add_pvt_type/1 > , fun add_pvt_vsn/1 > , fun maybe_add_pvt_api_key/1 > , fun maybe_add_pvt_tree/1 > , fun add_pvt_enabled/1 > ], > NewRecord = lists:foldl(fun(F, C) -> F(C) end, Record, PvtFuns), I'm aware of this style, but honestly, Pvt_Funs = add_pvt_enabled( maybe_add_pvt_tree( maybe_add_pvt_api_key( add_pvt_vsn( add_pvt_type( Record))))) is quite clear. There is nothing superfluous in that, unlike the folding version, and it copes neatly with multi-argument functions as long as the "subject" is the last argument. Of course in Smalltalk, this would be Pvt_Funs := Record add_pvt_type add_pvt_vsn maybe_add_pvt_api_key maybe_add_pvt_tree add_pvt_enabled because unary "functions" come after their argument in Smalltalk. Now suppose Erlang borrowed an idea from Pop-2 and said that e0.f[(e1,...,en)] -- with [] meaning optional -- meant the same as f(e0[,e1...,en]). Except that . is already taken, so let's try ? . Pvt_Funs = Record ?add_pvt_type() ?add_pvt_vsn() ?maybe_add_pvt_api_key() ?maybe_add_pvt_tree() ?add_pvt_enabled() What's the difference between this and the pipe operator? (Aside from the fact that it's based on a much older programming language?) Answer: the pipe operator is SEMANTICS: E |> F is a call to an actually existing "thing" called "|>". > (|>);; val it : ('a -> ('a -> 'b) -> 'b) = But suffix function calls are SYNTAX. In E?F, F is not an expression and is not evaluated by itself, and ? has no meaning outside this construction. > > Or some opaque structure, with accessors written more seriously: > > NewContext = > cb_context:setters(Context, [ {fun cb_context:set_doc/2, []} > , {fun cb_context:set_resp_status/2, 'success'} > , {fun cb_context:set_resp_data/2, []} > , {fun cb_context:set_resp_etag/2, 'undefined'} > ]) I'd rather see this as something like New_Context = cb_context:set(Context, [ {doc, []}, {resp_status, success}, {resp_data, []}, {resp_etag, undefined}]) However, the "suffix function call" *syntax* (remember, ? is NOT AN OPERATOR) would turn this into New_Context = Context ?cb_context:set_doc([]) ?cb_context:set_resp_status(success) ?cb_context:set_resp_data([]) ?cb_context:set_resp_etag(undefined). So if --> --> '?' --> ':' | ':' | --> | --> | '(' ')' | '(' ')' will satisfy you, suffix function calls could be implemented as a minor parser hack with no change to the AST and no implications for runtime. (Like I say, syntax, not semantics.) > > Scoping rules for nested pipes looks like a hard problem, but I am not sure nestedness would be a great thing to have? Suffix function calls have no scoping issues. If anyone wants to continue this thread, please be careful to distinguish between "the composition OPERATOR" and "suffix function call SYNTAX". From essen@REDACTED Fri Jul 10 10:11:22 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Fri, 10 Jul 2015 10:11:22 +0200 Subject: [erlang-questions] Exporting a record type In-Reply-To: <39011972-D94D-416B-BDAE-2DA05CAC43A7@writersglen.com> References: <1436486708.04388403@apps.rackspace.com> <39011972-D94D-416B-BDAE-2DA05CAC43A7@writersglen.com> Message-ID: <559F7E2A.7030804@ninenines.eu> I guess I need to expand on that in the next release. :-) Sorry I couldn't answer sooner, barely keeping up with everything. On 07/10/2015 03:51 AM, Lloyd R. Prentice wrote: > Thanks Bob and Fernando, > > Fernando, your code example is a great help. > > Thanks again, > > Lloyd > > Sent from my iPad > > On Jul 9, 2015, at 8:56 PM, Fernando Benavides > wrote: > >> Lo?c will correct me if I'm wrong but the idea is to use records >> *only* as an internal representation of your data types. >> For instance, let's say in your system you have shoes (for lack of a >> better thing). You'll then create a module called shoes.erl: >> >> -module(shoes). >> >> -export([new/2, id/1, color/1, color/2, size/1, size/2]). >> >> -record(shoe, {id = new_id(), color, size}). >> -opaque shoe() :: #shoe{}. >> -export_type([shoe/0]). >> >> new(Color, Size) -> #shoe{color = Color, size = Size}. >> id(#shoe{id = Id}) -> Id. >> color(#shoe{color = Color}) -> Color. >> color(Shoe, Color) -> Shoe#shoe{color = Color}. >> size(#shoe{size = Size}) -> Size. >> size(Shoe, Size) -> Shoe#shoe{size = Size}. >> >> new_id() -> implement:it(yourself). >> >> Whenever you need to work with shoes anywhere in your system, you just >> use the functions exported by the module. Let's say you have a list of >> shoes and you only want those of a particular size: >> >> filter_by_size(Shoes, Size) -> >> [Shoe || Shoe <- Shoes, shoes:size(Shoe) == Size]. >> >> Hope this helps. >> >> >> >> /*Fernando Benavides */ >> >> >> On Thu, Jul 9, 2015 at 5:17 PM, Bob Ippolito > > wrote: >> >> On Thu, Jul 9, 2015 at 5:05 PM, > > wrote: >> >> >> In Lo?c's new book, The Erlanger Playbook, he advocates >> against using including records through *.hrl. (pp 40-41) >> >> Instead, he recommends using types: >> >> -record(state, {}). >> -opaque state() :: #state{}. >> >> >> This looks like the fairly common recommendation of: don't export >> records. Treat them as an opaque data type, and provide functions >> for working with them. >> >> He leaves hanging the question of how to access this record >> from another module. >> >> The Erlang Types and Function Specifications doc suggests that >> the type can be exported using -export_type(...). But, to my >> feeble mind, the exposition is less than clear. >> >> My case and problem is as follows: >> >> -record(book, { >> ... yada yada (some seven fields) >> } >> >> I define the record, then: >> >> -opaque book() :: #book{}. >> >> -export_type([book/0]). >> >> Now, I access a remote db in a different module and want to >> instantiate a book record, but it's not at all clear to me how >> to do so. >> >> >> You don't. You have a function in the module that defines book >> that can create a book record, and other functions to manipulate them. >> >> An example of this style would be the dict module. It uses records >> internally to implement a dictionary data structure, but you don't >> have direct access to the record(s) that it uses. Just functions >> to work with them. >> >> -bob >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From essen@REDACTED Fri Jul 10 10:15:54 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Fri, 10 Jul 2015 10:15:54 +0200 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: References: <559E9216.6000900@komponentenwerkstatt.de> <1436456104.732571.319551801.48D54ED5@webmail.messagingengine.com> <1436456335.733421.319554817.18EA4B0E@webmail.messagingengine.com> <821E61F3-A790-428E-B96B-EF651FE5E340@cs.otago.ac.nz> Message-ID: <559F7F3A.1050709@ninenines.eu> On 07/10/2015 07:17 AM, Pierre Fenoll wrote: > A pipe operator would turn this structure-specific code into something > more readable and editable too. > Maybe like something like this: > > NewContext = Context > |> cb_context:set_doc(_, []) > |> cb_context:set_resp_status(_, success) > |> cb_context:set_resp_data(_, []) > |> cb_context:set_resp_etag(_, undefined) > > Where m:f(_, A2) is fun (_1) -> m:f(_1, A2) end; thus redefining _ for > pipes (in non-matching contexts?). What's wrong with this? NewContext = cb_context:set(#{doc=>[], resp_status=success...}, Context) Has less repetition/verbosity, is most likely faster, already works today, and you can also use a proplist if you're a map hater. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From pierrefenoll@REDACTED Fri Jul 10 10:23:29 2015 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Fri, 10 Jul 2015 01:23:29 -0700 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: <559F7F3A.1050709@ninenines.eu> References: <559E9216.6000900@komponentenwerkstatt.de> <1436456104.732571.319551801.48D54ED5@webmail.messagingengine.com> <1436456335.733421.319554817.18EA4B0E@webmail.messagingengine.com> <821E61F3-A790-428E-B96B-EF651FE5E340@cs.otago.ac.nz> <559F7F3A.1050709@ninenines.eu> Message-ID: Loic: what's wrong is you can't xref it, AFAIK. Cheers, -- Pierre Fenoll On 10 July 2015 at 01:15, Lo?c Hoguin wrote: > On 07/10/2015 07:17 AM, Pierre Fenoll wrote: > >> A pipe operator would turn this structure-specific code into something >> more readable and editable too. >> Maybe like something like this: >> >> NewContext = Context >> |> cb_context:set_doc(_, []) >> |> cb_context:set_resp_status(_, success) >> |> cb_context:set_resp_data(_, []) >> |> cb_context:set_resp_etag(_, undefined) >> >> Where m:f(_, A2) is fun (_1) -> m:f(_1, A2) end; thus redefining _ for >> pipes (in non-matching contexts?). >> > > What's wrong with this? > > NewContext = cb_context:set(#{doc=>[], resp_status=success...}, Context) > > Has less repetition/verbosity, is most likely faster, already works today, > and you can also use a proplist if you're a map hater. > > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Jul 10 10:31:27 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Fri, 10 Jul 2015 10:31:27 +0200 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: References: <559E9216.6000900@komponentenwerkstatt.de> <1436456104.732571.319551801.48D54ED5@webmail.messagingengine.com> <1436456335.733421.319554817.18EA4B0E@webmail.messagingengine.com> <821E61F3-A790-428E-B96B-EF651FE5E340@cs.otago.ac.nz> <559F7F3A.1050709@ninenines.eu> Message-ID: <559F82DF.9060900@ninenines.eu> :set/2 is a normal function, why wouldn't xref be able to work with it? Am I missing something? On 07/10/2015 10:23 AM, Pierre Fenoll wrote: > Loic: what's wrong is you can't xref it, AFAIK. > > > Cheers, > -- > Pierre Fenoll > > > On 10 July 2015 at 01:15, Lo?c Hoguin > wrote: > > On 07/10/2015 07:17 AM, Pierre Fenoll wrote: > > A pipe operator would turn this structure-specific code into > something > more readable and editable too. > Maybe like something like this: > > NewContext = Context > |> cb_context:set_doc(_, []) > |> cb_context:set_resp_status(_, success) > |> cb_context:set_resp_data(_, []) > |> cb_context:set_resp_etag(_, undefined) > > Where m:f(_, A2) is fun (_1) -> m:f(_1, A2) end; thus redefining > _ for > pipes (in non-matching contexts?). > > > What's wrong with this? > > NewContext = cb_context:set(#{doc=>[], resp_status=success...}, Context) > > Has less repetition/verbosity, is most likely faster, already works > today, and you can also use a proplist if you're a map hater. > > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From ulf@REDACTED Fri Jul 10 10:36:49 2015 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 10 Jul 2015 10:36:49 +0200 Subject: [erlang-questions] Exporting a record type In-Reply-To: <1436486708.04388403@apps.rackspace.com> References: <1436486708.04388403@apps.rackspace.com> Message-ID: <290FB57D-54D0-4DC4-AE8B-324C3E21B90A@feuerlabs.com> The -export_type/1 construct makes the type visible to other modules, so you can refer to it. It doesn?t correspond to a function call. In Bob?s example, you could have a module shoe_store, where: -spec buy(store(), shoes:shoe(), price()) -> receipt(). (Soon, Bob and I will launch our new chain?) The reason why records should not be ?exported? is the compile-time dependency: other modules may break if you change the record structure and they are not recompiled at the same time. There are a few ways to manage this: - Only use records inside one module - Only use records inside one application, since an application is normally built and upgraded as a unit anyway - Export accessor functions to manipulate the record structure The last point, accessor functions, may either be high-level or map one-to-one to the record arguments. There is parse_trans [1] for those who want to rely heavily on exported records and accessor functions. For a practical example, see e.g. [2] and [3]. One can also use exprecs as a low-level support for creating higher-level accessor functions. Exprecs does have some support for transforming old records to new. This is almost entirely undocumented, and I?ve always considered it incomplete. But as I?m writing this, I did think of one thing that might make it worth documenting - will see if I get a chance to experiment ? Meanwhile, [4] offers some hints. [1] https://github.com/uwiger/parse_trans/blob/master/doc/exprecs.md [2] http://blogs.openaether.org/?p=253 [3] https://github.com/uwiger/jobs/blob/master/src/jobs_info.erl (jobs_info:pp/1 converts a record to {Tag, Proplist}) [4] https://github.com/uwiger/parse_trans/blob/master/examples/test_exprecs_vsns.erl Eshell V5.10.4 (abort with ^G) 1> test_exprecs_vsns:f(). '#info-r'(fields) -> [a,b,c] '#info-r__1_1'(fields)' -> [a,b,c,d] (not exported) '#info-r__1_2'(fields)' -> [a,b] (not exported) '#convert-'("1_1", {r,1,2,3,4}) -> {{r,1,2,3},[{d,4}]} '#convert-'("1_2", {r,1,2}) -> {{r,1,2,undefined},[]} ok > On 10 Jul 2015, at 02:05, lloyd@REDACTED wrote: > > Hi folks, > > I've asked Lo?c Hoguin directly about this, but I thought an elaboration and discussion might be of interest to many on this list. > > In Lo?c's new book, The Erlanger Playbook, he advocates against using including records through *.hrl. (pp 40-41) > > Instead, he recommends using types: > > -record(state, {}). > -opaque state() :: #state{}. > > He leaves hanging the question of how to access this record from another module. > > The Erlang Types and Function Specifications doc suggests that the type can be exported using -export_type(...). But, to my feeble mind, the exposition is less than clear. > > My case and problem is as follows: > > -record(book, { > ... yada yada (some seven fields) > } > > I define the record, then: > > -opaque book() :: #book{}. > > -export_type([book/0]). > > Now, I access a remote db in a different module and want to instantiate a book record, but it's not at all clear to me how to do so. > > If I do the following in the console... > > wg_books:book(). I get: > > ** exception error: undefined function wg_dets_books:book/0 > > This seems to be an all-to-little documented corner of Erlang. Is there a kind soul who can help show the way? > > Many thanks, > > LRP > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From jesper.louis.andersen@REDACTED Fri Jul 10 10:48:57 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 10 Jul 2015 10:48:57 +0200 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: <559E9216.6000900@komponentenwerkstatt.de> References: <559E9216.6000900@komponentenwerkstatt.de> Message-ID: On Thu, Jul 9, 2015 at 5:24 PM, Volkert wrote: > Are there any plans to have this in Erlang? As already noted, the operator |> is hard to add to the language because it doesn't work that well. I'd much rather see a succinct notion for a monadic val/bind where I can set a monad M = #monad { val = Val, bind = Bind }, with M do X <- foo(), Y <- bar(X), return(Y) end and run inside the monad[0]. Since the language is dynamically typed, you would have to supply the monad you are building over, but you would be able to do a lot of the things people request with the |> operator, as it is a special case. [0] I've given this very little syntactical thought at the moment, but hopefully you get the idea. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From be.dmitry@REDACTED Fri Jul 10 12:07:30 2015 From: be.dmitry@REDACTED (Dmitry Belyaev) Date: Fri, 10 Jul 2015 20:07:30 +1000 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: References: <559E9216.6000900@komponentenwerkstatt.de> Message-ID: <6FB96114-9844-4A2B-8585-92365CA4A610@gmail.com> That is already done in erlando: https://github.com/rabbitmq/erlando -- Best wishes, Dmitry Belyaev On 10 July 2015 6:48:57 PM AEST, Jesper Louis Andersen wrote: >On Thu, Jul 9, 2015 at 5:24 PM, Volkert > >wrote: > >> Are there any plans to have this in Erlang? > > >As already noted, the operator |> is hard to add to the language >because it >doesn't work that well. > >I'd much rather see a succinct notion for a monadic val/bind where I >can >set a monad > >M = #monad { val = Val, bind = Bind }, >with M do > X <- foo(), > Y <- bar(X), > return(Y) >end > >and run inside the monad[0]. Since the language is dynamically typed, >you >would have to supply the monad you are building over, but you would be >able >to do a lot of the things people request with the |> operator, as it is >a >special case. > >[0] I've given this very little syntactical thought at the moment, but >hopefully you get the idea. > > >-- >J. > > >------------------------------------------------------------------------ > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Fri Jul 10 13:25:11 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 10 Jul 2015 13:25:11 +0200 Subject: [erlang-questions] Exporting a record type In-Reply-To: <1436486708.04388403@apps.rackspace.com> References: <1436486708.04388403@apps.rackspace.com> Message-ID: On Fri, Jul 10, 2015 at 2:05 AM, wrote: > I've asked Lo?c Hoguin directly about this, but I thought an elaboration > and discussion might be of interest to many on this list. The general idea is: "prefer loose coupling". Whenever you stuff a record into a header file, every user of that header file becomes tightly coupled to each other. This means, should the record change structure, that you have to go hunting for all the uses and make sure they do the right thing. If you, on the other hand, define a local record: -module(foo). -opaque t() :: #state{}. -export_type([t/0]). then type 't' is now local to the module, and thus users can only manipulate it with functions from the 'foo' module. This is far more modular, as you have a well-defined API in foo that explains "what you can do with the type 't'". Generally you want to avoid module 'foo' to contain getter/setter pairs and define the "transactional" states that can happen to 't'. That way, the semantics of 't'-transitions are rigidly defined in module 'foo' and you know where to change the semantics (in module 'foo'). Of course, there are exceptions to the rule where it is better to share a record. But my experience is that you want to keep the records as close to the using modules as possible. Another common thing is that people stuff their .hrl's into the "include" directory rather than in "src", so they are exportable and reachable by other applications. This is needed at times, but again: you are now tightly coupling applications together, not only modules. And you need to be sure this is what you want. General dictum: control the scope of records harshly, for they breed like rabbits in a grassy field. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Fri Jul 10 14:33:36 2015 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 10 Jul 2015 14:33:36 +0200 Subject: [erlang-questions] [ANN] leveldb_manager: a helper to eleveldb for in-flight backups Message-ID: In order to be able to snapshot and back up a leveldb instance without stopping the Erlang node (or shutting down eleveldb), we have written a manager application that lets you temporarily stop leveldb traffic and perform the snapshot (a very quick operation): https://github.com/klarna/leveldb_manager We would prefer if this made it into the eleveldb distribution itself, in some shape, but for now the cleanest solution was to make it a separate application with its own supervision. /Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Fri Jul 10 17:05:22 2015 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Fri, 10 Jul 2015 11:05:22 -0400 Subject: [erlang-questions] Exporting a record type In-Reply-To: References: <1436486708.04388403@apps.rackspace.com> Message-ID: <330A96FE-A329-4D8E-8DAF-8DABF86716BB@writersglen.com> Thanks Jesper and to all who have so generously responded. Jesper, can you please expand on the following: "Generally you want to avoid module 'foo' to contain getter/setter pairs and define the "transactional" states that can happen to 't'." Where would one define getter/setter pairs? What do you mean by "transactional" states? Many thanks, Lloyd Sent from my iPad > On Jul 10, 2015, at 7:25 AM, Jesper Louis Andersen wrote: > > Generally you want to avoid module 'foo' to contain getter/setter pairs and define the "transactional" states that can happen to 't'. From gguthrie@REDACTED Fri Jul 10 17:18:19 2015 From: gguthrie@REDACTED (Gordon Guthrie) Date: Fri, 10 Jul 2015 16:18:19 +0100 Subject: [erlang-questions] Exporting a record type In-Reply-To: <330A96FE-A329-4D8E-8DAF-8DABF86716BB@writersglen.com> References: <1436486708.04388403@apps.rackspace.com> <330A96FE-A329-4D8E-8DAF-8DABF86716BB@writersglen.com> Message-ID: <9A55AD73-6BE0-4FD7-9372-874E47826F2B@basho.com> Maybe an example would help. This is the riak_object from Basho?s riak and it is an encapsulated record with exported types and transactional functions that operate on the opaque object: https://github.com/basho/riak_kv/blob/develop/src/riak_object.erl It is written like this because it is passed around the whole system and persisted to the metal, so opacity and forward/backward changes need to be able to be dropped into the code base with rewriting *everything* Gordon > Le 10 juil. 2015 ? 16:05, Lloyd R. Prentice a ?crit : > > Thanks Jesper and to all who have so generously responded. > > Jesper, can you please expand on the following: > > "Generally you want to avoid module 'foo' to contain getter/setter pairs and define the "transactional" states that can happen to 't'." > > Where would one define getter/setter pairs? What do you mean by "transactional" states? > > Many thanks, > > Lloyd > > > Sent from my iPad > >> On Jul 10, 2015, at 7:25 AM, Jesper Louis Andersen wrote: >> >> Generally you want to avoid module 'foo' to contain getter/setter pairs and define the "transactional" states that can happen to 't'. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Fri Jul 10 18:11:18 2015 From: lloyd@REDACTED (lloyd@REDACTED) Date: Fri, 10 Jul 2015 12:11:18 -0400 (EDT) Subject: [erlang-questions] Exporting a record type In-Reply-To: <9A55AD73-6BE0-4FD7-9372-874E47826F2B@basho.com> References: <1436486708.04388403@apps.rackspace.com> <330A96FE-A329-4D8E-8DAF-8DABF86716BB@writersglen.com> <9A55AD73-6BE0-4FD7-9372-874E47826F2B@basho.com> Message-ID: <1436544678.40631215@apps.rackspace.com> Hi Gordon, The why is beginning to penetrate. The how is still murky. Examples definitely help. If I understand your riak example, it looks to me like the module riak_object.erl, among other things, defines a record called #r_object{...}, then employs the -opaque declaration to hide the structure of #r_object{} behind a veil, giving it an alias, riak_object() (line 67). But as I scan further down the module, I see riak_object() used in many -spec declarations but not in a single function definition; while #r_object{...} shows up in numerous functions (Lines 135, 144, 151, etc.). Is my understanding correct that one would strive to define all functions accessing #r_object{...) directly in and only in the module riak_object and it is these functions that would be exported? I see that the alias riak_object() is exported. How would it, then, be called in other modules to populate or access #r_object{}? Here's my case: In the module wg_dets_books.erl I've defined: -record(book, { isbn, member_id, date=calendar:universal_time(), title, category, gentre, author, publisher, pubDate, smallImageURL, largeImageURL, squib, coverCopy }). -opaque book() :: #book{}. -export_type([book/0]). In another module I query a remote db that returns: Title = get_title(Xml), Author = get_author(Xml), Publisher = get_publisher(Xml), PubDate = get_pub_date(Xml), SmallImage = get_small_image(Xml), LargeImage = get_large_image(Xml), CoverCopy = get_cover_copy(Xml), So, now what syntax would I use in this module to populate the book record? Apologies to all for my dense mentality, but this seems like a fundamental and important matter of Erlang craft that I want to understand thoroughly. All the best, Lloyd -----Original Message----- From: "Gordon Guthrie" Sent: Friday, July 10, 2015 11:18am To: "Lloyd R. Prentice" Cc: "Jesper Louis Andersen" , "erlang-questions" Subject: Re: [erlang-questions] Exporting a record type Maybe an example would help. This is the riak_object from Basho?s riak and it is an encapsulated record with exported types and transactional functions that operate on the opaque object: https://github.com/basho/riak_kv/blob/develop/src/riak_object.erl It is written like this because it is passed around the whole system and persisted to the metal, so opacity and forward/backward changes need to be able to be dropped into the code base with rewriting *everything* Gordon > Le 10 juil. 2015 ? 16:05, Lloyd R. Prentice a ?crit : > > Thanks Jesper and to all who have so generously responded. > > Jesper, can you please expand on the following: > > "Generally you want to avoid module 'foo' to contain getter/setter pairs and define the "transactional" states that can happen to 't'." > > Where would one define getter/setter pairs? What do you mean by "transactional" states? > > Many thanks, > > Lloyd > > > Sent from my iPad > >> On Jul 10, 2015, at 7:25 AM, Jesper Louis Andersen wrote: >> >> Generally you want to avoid module 'foo' to contain getter/setter pairs and define the "transactional" states that can happen to 't'. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From nico.kruber@REDACTED Sat Jul 11 14:33:48 2015 From: nico.kruber@REDACTED (Nico Kruber) Date: Sat, 11 Jul 2015 14:33:48 +0200 Subject: [erlang-questions] Erlang hangs in supervisor:do_terminate/2 Message-ID: <1794014.6vMsUISv8L@nico-pc> Hi, I'm having trouble with supervisor:do_terminate/2 in both, Erlang 18.0.1 and 18.0.2 which I haven't seen with earlier versions so far. I currently do not have a minimal use case but will try to come up with something soon. I'm using Scalaris and inside a single process, I'm starting its services (multiple processes in a supervisor tree) and stopping them again in a loop. Sometimes, although very rarely, stopping the services seems to hang. When I send the beam process a SIGUSR1 I can always see two processes being in "Running" state: 1) a supervisor in supervisor:do_terminate/2 (any of the present supervisors - not always the same!) 2) a child/worker of this supervisor handling a message (or at least, so it seems) Their stacktraces seem inconclusive, please find an example of the two processes from the crashdump_viewer below. Is there any known problem/change in Erlang 18 that could have caused this? Regards Nico 1) supervisor: '0x00002b6c8aec69a0' "Return addr 0x8942AFD8 (supervisor:do_terminate/2 + 240)" y0 <0.6955.0> '0x00002b6c8aec69b0' "Return addr 0x89423D28 (supervisor:handle_call/3 + 2408)" y0 {<0.6951.0>,sup_wpool} y1 permanent y2 {child,<0.6955.0>, {wpool_w,2}, {wpool_worker,start_link,[...]}, permanent,brutal_kill,worker,...} '0x00002b6c8aec69d0' "Return addr 0x89405070 (gen_server:try_handle_call/4 + 176)" y0 {state,{<0.6951.0>,sup_wpool}, one_for_one, [{child,[...],...},{child,...}], undefined,10,1,...} y1 one_for_one y2 {child,<0.6955.0>, {wpool_w,2}, {wpool_worker,start_link,[...]}, permanent,brutal_kill,worker,...} '0x00002b6c8aec69f0' "Return addr 0x894056B8 (gen_server:handle_msg/5 + 192)" y0 "Catch 0x894050D0 (gen_server:try_handle_call/4 + 272)" '0x00002b6c8aec6a00' "Return addr 0x7F6AF758 (proc_lib:init_p_do_apply/3 + 56)" y0 supervisor y1 {state,{<0.6951.0>,sup_wpool}, one_for_one, [{child,[...],...},{child,...}], undefined,10,1,...} y2 <0.6951.0> y3 <0.6941.0> y4 {terminate_child,{wpool_w,2}} y5 {<0.7140.0>,#Ref<10689.0.3.14414>} '0x00002b6c8aec6a38' "Return addr 0x9014E8 ()" y0 "Catch 0x7F6AF778 (proc_lib:init_p_do_apply/3 + 88)" 2) child worker: '0x00002b6c8aec39d0' 60000 '0x00002b6c8aec39d8' "Return addr 0x9014E8 ()" y0 {wpool_worker,#Fun,[],false,[],false,unknown,...} Click to expand above term y1 [...(Incomplete Heap)] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: This is a digitally signed message part. URL: From evnix.com@REDACTED Sun Jul 12 10:40:33 2015 From: evnix.com@REDACTED (avinash D'silva) Date: Sun, 12 Jul 2015 14:10:33 +0530 Subject: [erlang-questions] 1st day of Erlang: what is wrong with this code Message-ID: loop(Socket) -> case gen_tcp:recv(Socket, 0) of {ok, Data} -> gen_tcp:send(Socket, string:to_upper(Data)), loop(Socket); {error, closed} -> ok end. I get an error for string:to_upper() I am trying to create a simple echo server that converts text to upper case. original code: http://20bits.com/article/network-programming-in-erlang/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Sun Jul 12 11:58:05 2015 From: bob@REDACTED (Bob Ippolito) Date: Sun, 12 Jul 2015 02:58:05 -0700 Subject: [erlang-questions] 1st day of Erlang: what is wrong with this code In-Reply-To: References: Message-ID: Data is likely a binary(), not a string(). You can use the unicode module to go between binary data and lists of characters: http://www.erlang.org/doc/man/unicode.html On Sun, Jul 12, 2015 at 1:40 AM, avinash D'silva wrote: > loop(Socket) -> > case gen_tcp:recv(Socket, 0) of > {ok, Data} -> > gen_tcp:send(Socket, string:to_upper(Data)), > loop(Socket); > {error, closed} -> > ok > end. > > I get an error for string:to_upper() > > I am trying to create a simple echo server that converts text to upper > case. > > original code: http://20bits.com/article/network-programming-in-erlang/ > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sun Jul 12 12:03:10 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 12 Jul 2015 12:03:10 +0200 Subject: [erlang-questions] 1st day of Erlang: what is wrong with this code In-Reply-To: References: Message-ID: Hi! Your problem hides itself in the original code: -define(TCP_OPTIONS, [binary, {packet, 0}, {active, false}, {reuseaddr, true}]). This requests that the socket port is in 'binary' mode. This means that the result of `gen_tcp:recv(Socket, 0)` is {ok, Data} is a binary, not a string. One way around this is: case gen_tcp:recv(Socket, 0) of {ok, Data} when is_binary(Data) -> String = binary_to_list(Data), gen_tcp:send(Socket, string:to_upper(Data)), loop(Socket); ... This works for ASCII, but it fails blatantly for anything which is utf-8, where the definition of uppercasing is less well understood. The reason you can send a string() on a binary socket is that a string is a list of code_points, and if the code points are all less than 128, then you have a list of bytes. These are a subset of the iodata() type which `gen_tcp:send/2` accepts. On Sun, Jul 12, 2015 at 10:40 AM, avinash D'silva wrote: > loop(Socket) -> > case gen_tcp:recv(Socket, 0) of > {ok, Data} -> > gen_tcp:send(Socket, string:to_upper(Data)), > loop(Socket); > {error, closed} -> > ok > end. > > I get an error for string:to_upper() > > I am trying to create a simple echo server that converts text to upper > case. > > original code: http://20bits.com/article/network-programming-in-erlang/ > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Sun Jul 12 12:11:04 2015 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Sun, 12 Jul 2015 03:11:04 -0700 Subject: [erlang-questions] 1st day of Erlang: what is wrong with this code In-Reply-To: References: Message-ID: <55A23D38.8070905@gmail.com> On 07/12/2015 01:40 AM, avinash D'silva wrote: > I get an error for string:to_upper() It would have been useful if you'd pasted the error in your message. :) Anyway, the Data that is being returned by gen_tcp:recv/2 is being returned as a binary, rather than a list. (I might be wrong here, but a "list" is what Erlang calls non-Unicode strings.) string:to_upper/1 expects a list. If you look where TCP_OPTIONS is defined, you see that one of the options passed to gen_tcp:listen/2 is "binary". Change that to "list" and gen_tcp:recv will return lists, rather than binaries. (See http://erlang.org/doc/man/gen_tcp.html#listen-2 for info on the options you can pass to gen_tcp:listen/2 [and indeed, that entire page for info on the gen_tcp module]. The docs are arranged intelligently. This means that the reference manual for everything in the string module is at http://erlang.org/doc/man/string.html ) Also -just for fun- before you change TCP_OPTIONS, add io:format("~p~n", [Data]), before the line gen_tcp:send(Socket, string:to_upper(Data)), then rebuild and restart your program. If you look carefully at the output from the Erlang shell, you'll notice that binaries are printed <<"Like this">> and -after you change TCP_OPTIONS- that lists are printed "Like this". I'm no expert, but I remember hearing that -for large amounts of data- working with binaries is substantially faster than working with lists. As this is your first day with Erlang, you probably don't need to worry about that. :) Also, check out "Learn you some Erlang for Great Good!" ( http://learnyousomeerlang.com/ ). You might get some value out of it. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From jesper.louis.andersen@REDACTED Sun Jul 12 14:31:24 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 12 Jul 2015 14:31:24 +0200 Subject: [erlang-questions] Exporting a record type In-Reply-To: <1436544678.40631215@apps.rackspace.com> References: <1436486708.04388403@apps.rackspace.com> <330A96FE-A329-4D8E-8DAF-8DABF86716BB@writersglen.com> <9A55AD73-6BE0-4FD7-9372-874E47826F2B@basho.com> <1436544678.40631215@apps.rackspace.com> Message-ID: On Fri, Jul 10, 2015 at 6:11 PM, wrote: > So, now what syntax would I use in this module to populate the book record? So, we have a book record in a database. We want a representation of that book record in the system. In other words, we want to be able to reify a book in the database as a book in the system. Once we have the book in the system, we can manipulate that book, and then reflect the book back into the database for persistence. This is the general idea. The problem is that there are two subsystems which need knowledge about books: the database low level module, and the module representing or manipulating the book. One solution, which is the simple solution, is to put a book record into a header file, and then refer to the book record in all modules using it. The problem of this approach is that any user will be tightly coupled to the books representation, so it is harder to manipulate the representation of the book. Any other solution must put the representational knowledge in one module only. And a little thought shows that it is a bad idea to let the DB module contain the knowledge. So you construct a "book" module: -module(book). -opaque book() :: #book{}. -spec unmarshal({xml, Doc, Description} | {json, Doc, Description} | {postgres, ...}) -> {ok, book()} | {error, Reason}. -spec marshal(Book, xml | json | transit) -> Data. unmarshal takes a notion of a reification-location, i.e., xml, together with a document, Doc, and a "Description" which is a symbolic term describing how to obtain the book from the document. The unmarshaler can then utilize such a description in order to reify the book from the document. As a start, you may want to ignore the description and hard-code the notion of e.g., 'get_title(Doc)', but later on it may be necessary to generalize this why it can be nice to describe something about the document you received. A postgres connection would probably just return some abstract '${ Col => Val }' mapping ColNames to Values in a row. and the reification process is the obvious one. The point is: *someone* has to parse the data into a #book{} record, so it may be necessary to have a generic intermediate format inside the system for doing so. "unmarshal" is also used to construct a new book for later persistence. One can imagine adding a variant which is {post, Data, Description} for a C-in-CRUD creation, with parsing, input validation and all that fluff. One way of thinking of the description could be path-expressions inside the Xml document in XPath notation. So the Description is #{ title => "a path to the title", author => "a path to the author", ... } In turn, the DB code knows, and provides, something about the *location* of data inside the document, and the reifier inside 'book' only uses such location-descriptions to pick out stuff. If the position of the title suddenly changes in a later Document version, the DB code can simply supply a different location. Continuing, what can we do with books? Maybe we have a thumbnail overview of books. For this, we only need a few entries: repr(Book, {msgpack, View}) -> msgpack:encode(repr(Book, View)); repr(Book, {json, View}) -> jsx:encode(repr(Book, View)); repr(book#{ title = T, pubDate = PD, smallImageURL = SI }, thumbnail_overview) -> #{ thumbnail = SI, title = T, pub_date = PD }. Note that a caller can't possible know the internal representation. A caller gets a "generic" representation[0], which can then be cast in several forms. Any "update" on the #book{} record has to factor through the module as well. Now, a cowboy/webmachine RESTful server would load the book from the db and call unmarshall on it in the "resource_exists" callback (or perhaps earlier in the chain in some situations). And once the content-type is negotiated it would use repr/2 to cast such a generic book record into the desired content-type for the desired output-view. [0] In a real system, this will almost invariably get generalized further. Once you have a couple of these modules, there are commonality which can be factored out into a more general framework, I'm sure. The key is to avoid having to come "back" to code and change it whenever some other part of the system changes. And make sure that changes localize to one or a few modules. Beware architectures where you have to edit many modules to add a new "vertical" path in the system. This suggests you should be looking to "transpose" the code path. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wanglihe.programmer@REDACTED Sun Jul 12 15:19:06 2015 From: wanglihe.programmer@REDACTED (Lihe Wang) Date: Sun, 12 Jul 2015 21:19:06 +0800 Subject: [erlang-questions] Why application:start can not accept any args? Message-ID: Can any one give the explain about OTP? The application behaviour's call back "start" need args, but when I want start my code as a real application, I only can write "applicaion:start(mycode)", and can not pass any args to mycode. Why otp design like this? When I want start a application as background service from escript, for example: main([Arg1, Arg2, Arg3]) -> application:start(mycode1), %%with Arg1 application:start(mycode2), %%with Arg2 workcode(Arg3). %%workcode (escript) will run as a daemon, and need service supplied by mycode1 and mycode2. What is the right way to write these code? -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Sun Jul 12 17:06:01 2015 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Sun, 12 Jul 2015 11:06:01 -0400 Subject: [erlang-questions] Exporting a record type In-Reply-To: References: <1436486708.04388403@apps.rackspace.com> <330A96FE-A329-4D8E-8DAF-8DABF86716BB@writersglen.com> <9A55AD73-6BE0-4FD7-9372-874E47826F2B@basho.com> <1436544678.40631215@apps.rackspace.com> Message-ID: <927D610A-A519-4106-9D91-F53B5EC83485@writersglen.com> Hi Jesper, I was going to add you to my pantheon of Erlang heroes only to find that you were already there. Thank you! Your explication makes perfect sense. A basic point still mystifies me, however: > -opaque book() :: #book{}. Just exactly what is going on here? My naive interpretation is that we are defining a function name as an alias for a record and hiding the composition of the record. But I'm not sure this is correct since In the various code examples I see the xxxx() form used in -spec definitions, but not code. Also, in your repr/2 code (which is very suggestive of neat things one can do) what is the significance of the View variable? As I see it now, it's simply a tag like thumbnail_overview, but I'm not comfortable that my understanding is correct. Profound thanks again, Lloyd Sent from my iPad > On Jul 12, 2015, at 8:31 AM, Jesper Louis Andersen wrote: > > >> On Fri, Jul 10, 2015 at 6:11 PM, wrote: >> So, now what syntax would I use in this module to populate the book record? > > So, we have a book record in a database. We want a representation of that book record in the system. In other words, we want to be able to reify a book in the database as a book in the system. Once we have the book in the system, we can manipulate that book, and then reflect the book back into the database for persistence. This is the general idea. > > The problem is that there are two subsystems which need knowledge about books: the database low level module, and the module representing or manipulating the book. One solution, which is the simple solution, is to put a book record into a header file, and then refer to the book record in all modules using it. The problem of this approach is that any user will be tightly coupled to the books representation, so it is harder to manipulate the representation of the book. > > Any other solution must put the representational knowledge in one module only. And a little thought shows that it is a bad idea to let the DB module contain the knowledge. So you construct a "book" module: > > -module(book). > -opaque book() :: #book{}. > > -spec unmarshal({xml, Doc, Description} | {json, Doc, Description} | {postgres, ...}) -> {ok, book()} | {error, Reason}. > -spec marshal(Book, xml | json | transit) -> Data. > > unmarshal takes a notion of a reification-location, i.e., xml, together with a document, Doc, and a "Description" which is a symbolic term describing how to obtain the book from the document. The unmarshaler can then utilize such a description in order to reify the book from the document. As a start, you may want to ignore the description and hard-code the notion of e.g., 'get_title(Doc)', but later on it may be necessary to generalize this why it can be nice to describe something about the document you received. A postgres connection would probably just return some abstract '${ Col => Val }' mapping ColNames to Values in a row. and the reification process is the obvious one. The point is: *someone* has to parse the data into a #book{} record, so it may be necessary to have a generic intermediate format inside the system for doing so. > > "unmarshal" is also used to construct a new book for later persistence. One can imagine adding a variant which is {post, Data, Description} for a C-in-CRUD creation, with parsing, input validation and all that fluff. > > One way of thinking of the description could be path-expressions inside the Xml document in XPath notation. So the Description is > > #{ > title => "a path to the title", > author => "a path to the author", > ... > } > > In turn, the DB code knows, and provides, something about the *location* of data inside the document, and the reifier inside 'book' only uses such location-descriptions to pick out stuff. If the position of the title suddenly changes in a later Document version, the DB code can simply supply a different location. > > Continuing, what can we do with books? Maybe we have a thumbnail overview of books. For this, we only need a few entries: > > repr(Book, {msgpack, View}) -> msgpack:encode(repr(Book, View)); > repr(Book, {json, View}) -> jsx:encode(repr(Book, View)); > repr(book#{ title = T, pubDate = PD, smallImageURL = SI }, thumbnail_overview) -> > #{ > thumbnail = SI, > title = T, > pub_date = PD > }. > > Note that a caller can't possible know the internal representation. A caller gets a "generic" representation[0], which can then be cast in several forms. Any "update" on the #book{} record has to factor through the module as well. > > Now, a cowboy/webmachine RESTful server would load the book from the db and call unmarshall on it in the "resource_exists" callback (or perhaps earlier in the chain in some situations). And once the content-type is negotiated it would use repr/2 to cast such a generic book record into the desired content-type for the desired output-view. > > [0] In a real system, this will almost invariably get generalized further. Once you have a couple of these modules, there are commonality which can be factored out into a more general framework, I'm sure. The key is to avoid having to come "back" to code and change it whenever some other part of the system changes. And make sure that changes localize to one or a few modules. Beware architectures where you have to edit many modules to add a new "vertical" path in the system. This suggests you should be looking to "transpose" the code path. > > -- > J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Sun Jul 12 18:53:59 2015 From: bob@REDACTED (Bob Ippolito) Date: Sun, 12 Jul 2015 09:53:59 -0700 Subject: [erlang-questions] Exporting a record type In-Reply-To: <927D610A-A519-4106-9D91-F53B5EC83485@writersglen.com> References: <1436486708.04388403@apps.rackspace.com> <330A96FE-A329-4D8E-8DAF-8DABF86716BB@writersglen.com> <9A55AD73-6BE0-4FD7-9372-874E47826F2B@basho.com> <1436544678.40631215@apps.rackspace.com> <927D610A-A519-4106-9D91-F53B5EC83485@writersglen.com> Message-ID: On Sunday, July 12, 2015, Lloyd R. Prentice wrote: > > Thank you! Your explication makes perfect sense. A basic point still > mystifies me, however: > > -opaque book() :: #book{}. > > > Just exactly what is going on here? My naive interpretation is that we are > defining a function name as an alias for a record and hiding the > composition of the record. But I'm not sure this is correct since In the > various code examples I see the xxxx() form used in -spec definitions, but > not code. > -opaque is like -type but it does not export the implementation details of the type. This states that `book()` is a type alias for the `#book{}` record, but it's opaque so other modules are only allowed to pass around values of type `book()`, never create them directly. Your second thought here is basically spot on, this is a way to define a `book()` that can be used in `-spec` annotations. -bob > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Sun Jul 12 20:57:36 2015 From: lloyd@REDACTED (lloyd@REDACTED) Date: Sun, 12 Jul 2015 14:57:36 -0400 (EDT) Subject: [erlang-questions] Exporting a record type In-Reply-To: References: <1436486708.04388403@apps.rackspace.com> <330A96FE-A329-4D8E-8DAF-8DABF86716BB@writersglen.com> <9A55AD73-6BE0-4FD7-9372-874E47826F2B@basho.com> <1436544678.40631215@apps.rackspace.com> <927D610A-A519-4106-9D91-F53B5EC83485@writersglen.com> Message-ID: <1436727456.91269395@apps.rackspace.com> Thank you, Bob. Can't say how much I appreciate the support of this great community. One day, perhaps, enough will seep through my skull to contribute back. Best wishes to all, Lloyd -----Original Message----- From: "Bob Ippolito" Sent: Sunday, July 12, 2015 12:53pm To: "Lloyd R. Prentice" Cc: "Jesper Louis Andersen" , "erlang-questions" Subject: Re: [erlang-questions] Exporting a record type On Sunday, July 12, 2015, Lloyd R. Prentice wrote: > > Thank you! Your explication makes perfect sense. A basic point still > mystifies me, however: > > -opaque book() :: #book{}. > > > Just exactly what is going on here? My naive interpretation is that we are > defining a function name as an alias for a record and hiding the > composition of the record. But I'm not sure this is correct since In the > various code examples I see the xxxx() form used in -spec definitions, but > not code. > -opaque is like -type but it does not export the implementation details of the type. This states that `book()` is a type alias for the `#book{}` record, but it's opaque so other modules are only allowed to pass around values of type `book()`, never create them directly. Your second thought here is basically spot on, this is a way to define a `book()` that can be used in `-spec` annotations. -bob > From lukas@REDACTED Mon Jul 13 10:11:38 2015 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 13 Jul 2015 10:11:38 +0200 Subject: [erlang-questions] Erlang hangs in supervisor:do_terminate/2 In-Reply-To: <1794014.6vMsUISv8L@nico-pc> References: <1794014.6vMsUISv8L@nico-pc> Message-ID: Hello Nico, On Sat, Jul 11, 2015 at 2:33 PM, Nico Kruber wrote: > Hi, > I'm having trouble with supervisor:do_terminate/2 in both, Erlang 18.0.1 > and > 18.0.2 which I haven't seen with earlier versions so far. I currently do > not > have a minimal use case but will try to come up with something soon. > Would be great if you could manage that. Minimal examples are always much easier to work with. > I'm using Scalaris and inside a single process, I'm starting its services > (multiple processes in a supervisor tree) and stopping them again in a > loop. > Sometimes, although very rarely, stopping the services seems to hang. When > I > send the beam process a SIGUSR1 I can always see two processes being in > "Running" state: > 1) a supervisor in supervisor:do_terminate/2 (any of the present > supervisors - > not always the same!) > 2) a child/worker of this supervisor handling a message (or at least, so it > seems) > > Their stacktraces seem inconclusive, please find an example of the two > processes from the crashdump_viewer below. > Are the stack traces always the same for the supervisor? i.e. it is Running in do_terminate? Since there is a receive in do_terminate, if it is stuck I would have expected it to be in the Waiting state. But if it is Running and hanging, it could point to some kind of live lock. > Is there any known problem/change in Erlang 18 that could have caused this? > There are changes in supervisor for 18 (the introduction of maps as child specs), but it *should* not cause any problems like these. A smaller example demonstrating what is going wrong for you would help a lot in trying to understand what is going on. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Mon Jul 13 12:03:07 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 13 Jul 2015 12:03:07 +0200 Subject: [erlang-questions] Exporting a record type In-Reply-To: <927D610A-A519-4106-9D91-F53B5EC83485@writersglen.com> References: <1436486708.04388403@apps.rackspace.com> <330A96FE-A329-4D8E-8DAF-8DABF86716BB@writersglen.com> <9A55AD73-6BE0-4FD7-9372-874E47826F2B@basho.com> <1436544678.40631215@apps.rackspace.com> <927D610A-A519-4106-9D91-F53B5EC83485@writersglen.com> Message-ID: On Sun, Jul 12, 2015 at 5:06 PM, Lloyd R. Prentice wrote: > Also, in your repr/2 code (which is very suggestive of neat things one can > do) what is the significance of the View variable? As I see it now, it's > simply a tag like thumbnail_overview, but I'm not comfortable that my > understanding is correct. The idea, at least, is this: a "view" of some data, book, record, ..., is a way to cast that data into another structure by transforming it. I.e., "view" the data in another form. In statically typed languages, some languages support built-in view-types, so you can define these transformations formally, but in Erlang you have to make a more dynamic resolution. In its simple form, the view is just an atom(), requesting the view. But in a more advanced form, the view is a language which can be executed to construct a view of the data: ... repr(Book, {relation, R}) -> repr_rel(Book, R). repr_rel(#book { genre = G } = Book, genre) -> {G, Book}. Now, suppose you have a list of books, Bs: Rel = [repr(B, {relation, genre}) || B <- Bs], R = sofs:relation(Rel), F = sofs:relation_to_family(R), sofs:to_external(F). If for instance you have books [#book { genre = fantasy } = A, #book { genre = fantasy } = B, #book { genre = scifi } = C], Then the above would return [{fantasy, [A, B]}, {scifi, [C]}]. There are numerous tricks that can be played here, depending on what transformations you need. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nayibor@REDACTED Mon Jul 13 12:39:34 2015 From: nayibor@REDACTED (Nuku Ameyibor) Date: Mon, 13 Jul 2015 10:39:34 +0000 Subject: [erlang-questions] Why application:start can not accept any args? In-Reply-To: References: Message-ID: u can add environmental variables for your application in your .app file . there is a tuple with a key env which u can use to add environmental variables to your application which you can then access from your application eg . {application, test, [{vsn, "1.0.0"}, {description, "testing"}, {modules, [test, test_sup,test_start]}, {applications, [stdlib, kernel, crypto]}, {registered, [test,test_sup]}, {mod, {test, []}}, {env, [ {env1,b}, {env2,a}, {env3,c} ] } ]}. you can access them like so {ok, Env} = application:get_env(env1), etc.. there is also a set_env/3 ,set_env/4 which can be used to change environmental variables during run-time On Sun, Jul 12, 2015 at 1:19 PM, Lihe Wang wrote: > Can any one give the explain about OTP? The application behaviour's call > back "start" need args, but when I want start my code as a real > application, I only can write "applicaion:start(mycode)", and can not pass > any args to mycode. Why otp design like this? When I want start a > application as background service from escript, for example: > > main([Arg1, Arg2, Arg3]) -> > application:start(mycode1), %%with Arg1 > application:start(mycode2), %%with Arg2 > workcode(Arg3). %%workcode (escript) will run as a daemon, and need > service supplied by mycode1 and mycode2. > > What is the right way to write these code? > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From donpedrothird@REDACTED Mon Jul 13 15:01:46 2015 From: donpedrothird@REDACTED (John Doe) Date: Mon, 13 Jul 2015 16:01:46 +0300 Subject: [erlang-questions] reason of core dump (core.xxxx) Message-ID: One of my clients got a crash of erlang vm. The app has been running on this vps for several months with no problems, and still works after restart. It's not erl-crash.dump but a classic linux core dump (core.xxxx file 600Mb size). The load was quite heavy all the time, about 25M hits/day on a cheap VPS with 2Gb RAM, web app. The app has no nifs and anything like this, its pure erlang, including all deps. The only difference between standard erlang installation is that the crypto was built with static openssl. Both the app and the erlang VM were built on different machine though. R16B03 x64. What could be the reason of such crash without generating erl-crash.dump? Could it be because of VM run out of memory? Any other possible reasons of erlang vm crashing? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon@REDACTED Mon Jul 13 16:19:16 2015 From: jon@REDACTED (Jon Schneider) Date: Mon, 13 Jul 2015 15:19:16 +0100 Subject: [erlang-questions] reason of core dump (core.xxxx) In-Reply-To: References: Message-ID: >> What could be the reason of such crash without generating erl-crash.dump? There should be an accompanying kernel message. Also the core dump itself should tell you something. That's what they're for. My experience of virtual private servers is one of very very variable I/O performance. Another thing that bites quite often is Linux overcommitting on virtual memory and killing the wrong process when it goes wrong. But again logs should say so. Jon From henry@REDACTED Mon Jul 13 17:52:27 2015 From: henry@REDACTED (=?UTF-8?B?SGVucsO9IMOew7NyIEJhbGR1cnNzb24=?=) Date: Mon, 13 Jul 2015 15:52:27 +0000 Subject: [erlang-questions] Weird error with Cowboy on restart Message-ID: Hey guys So I got my project up and running. But I seem to be fighting some sort of corruption on restarting my process. If I stop my task (as it's still in staging) and start it up again, I get a bunch of these messages before my program exits: =ERROR REPORT==== 13-Jul-2015::12:19:31 === Error in process <0.328.0> on node 'erlfector@REDACTED' with exit value: {{case_clause,{error,closed}}, [{ranch_acceptor,loop,3, [{file,"/home/cw/erlfector/_build/default/lib/ranch/src/ranch_acceptor.erl"}, {line,28}]}]} The only way I've found to fix is it is to remove my rel directory and rebuild my application between restarts. Anyone have any ideas? Cowboy version: 1.0.1 Erlang: [{release,"Erlang/OTP","18","7.0", [{kernel,"4.0","/usr/lib/erlang/lib/kernel-4.0"}, {stdlib,"2.5","/usr/lib/erlang/lib/stdlib-2.5"}, {sasl,"2.5","/usr/lib/erlang/lib/sasl-2.5"}], permanent}]. Regards, Henry. Henr? ??r Baldursson / Programmer henry@REDACTED / +354 869-8264 <+3548698264> CrankWheel +354 534-5700 <+3545345700> ?rm?li 6, 108 Reykjav?k, Iceland http://crankwheel.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From henry@REDACTED Mon Jul 13 20:15:55 2015 From: henry@REDACTED (=?UTF-8?B?SGVucsO9IMOew7NyIEJhbGR1cnNzb24=?=) Date: Mon, 13 Jul 2015 18:15:55 +0000 Subject: [erlang-questions] Weird error with Cowboy on restart In-Reply-To: References: Message-ID: Seems that if I just delete the mnesia directory, I'm golden to start cowboy up again. Guess I need to die more gracefully when I'm testing? Regards, Henry. Henr? ??r Baldursson / Programmer henry@REDACTED / +354 869-8264 <+3548698264> CrankWheel +354 534-5700 <+3545345700> ?rm?li 6, 108 Reykjav?k, Iceland http://crankwheel.com/ On Mon, Jul 13, 2015 at 3:52 PM, Henr? ??r Baldursson wrote: > Hey guys > > So I got my project up and running. But I seem to be fighting some sort of > corruption on restarting my process. > > If I stop my task (as it's still in staging) and start it up again, I get > a bunch of these messages before my program exits: > =ERROR REPORT==== 13-Jul-2015::12:19:31 === > Error in process <0.328.0> on node 'erlfector@REDACTED' > with exit value: > {{case_clause,{error,closed}}, > [{ranch_acceptor,loop,3, > > [{file,"/home/cw/erlfector/_build/default/lib/ranch/src/ranch_acceptor.erl"}, > {line,28}]}]} > > The only way I've found to fix is it is to remove my rel directory and > rebuild my application between restarts. > > Anyone have any ideas? > > Cowboy version: 1.0.1 > Erlang: [{release,"Erlang/OTP","18","7.0", > [{kernel,"4.0","/usr/lib/erlang/lib/kernel-4.0"}, > > {stdlib,"2.5","/usr/lib/erlang/lib/stdlib-2.5"}, > > {sasl,"2.5","/usr/lib/erlang/lib/sasl-2.5"}], > > permanent}]. > > > > Regards, > Henry. > > > Henr? ??r Baldursson / Programmer > henry@REDACTED / +354 869-8264 <+3548698264> > > CrankWheel > +354 534-5700 <+3545345700> > ?rm?li 6, 108 Reykjav?k, Iceland > http://crankwheel.com/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Mon Jul 13 20:29:43 2015 From: vasdeveloper@REDACTED (Kannan) Date: Mon, 13 Jul 2015 23:59:43 +0530 Subject: [erlang-questions] eJabberd Loadbalancing Message-ID: Hi Team, We are working on a startup idea that uses eJabberd. We are trying to build the load balancing into the client, using a discovery call just before the login procedure. There will be a discovery assist node at the backend, which maps a chat user to a specific IP. This is some thing equivalent to DNS SRV lookup, but more at the application layer, and database shards are handled by the application itself. We are of course customizing the eJabberd to suit our needs. Application is handling the sharding, but will get the help of Mnesia for replication. Do you see any pitfalls in this method of loadbalancing. Regards, Theepan -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Mon Jul 13 21:22:55 2015 From: vasdeveloper@REDACTED (Kannan) Date: Tue, 14 Jul 2015 00:52:55 +0530 Subject: [erlang-questions] eJabberd Loadbalancing In-Reply-To: References: Message-ID: Hi Team, We are working on a startup idea that uses eJabberd. We are trying to build the load balancing into the client, using a discovery call just before the login procedure. There will be a discovery assist node at the backend, which maps a chat user to a specific IP. This is some thing equivalent to DNS SRV lookup, but more at the application layer, and database shards are handled by the application itself. We are of course customizing the eJabberd to suit our needs. Application is handling the sharding, but will get the help of Mnesia for replication. Do you see any pitfalls in this method of loadbalancing. Regards, Theepan On Mon, Jul 13, 2015 at 11:59 PM, Kannan wrote: > Hi Team, > > We are working on a startup idea that uses eJabberd. We are trying to > build the load balancing into the client, using a discovery call just > before the login procedure. There will be a discovery assist node at the > backend, which maps a chat user to a specific IP. This is some thing > equivalent to DNS SRV lookup, but more at the application layer, and > database shards are handled by the application itself. We are of course > customizing the eJabberd to suit our needs. > > Application is handling the sharding, but will get the help of Mnesia for > replication. > > > Do you see any pitfalls in this method of loadbalancing. > > Regards, > Theepan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Tue Jul 14 00:56:01 2015 From: rvirding@REDACTED (Robert Virding) Date: Tue, 14 Jul 2015 00:56:01 +0200 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: References: <20150709231805.GA1339@ferdmbp.local> Message-ID: On 10 July 2015 at 07:22, ?ric Pailleau wrote: > > Le 10 juil. 2015 01:18, Fred Hebert a ?crit : > > > > On 07/10, ?ric Pailleau wrote: > > >A = catch Pid | {somemessage, somevalue }. > > >This would probably need a timeout. On other hand, ! + receive may > wait forever. | could do the same. > > > > A = [catch Pid | {somemessage, somevalue}]. > > > > Is A equal to: > > > > a) [Pid | {somemessage, somevalue}] (improper list) > > b) [Result] (the received message) > > > Yes, you are right. But I do not request that pipe must be the operator. > If !! Is better, I do not care. The need is more important than the syntax > itself. > > > > It also ignores that two processes may have more than two communications > > going on at the same time, and that without a selective receive, it's > > not obvious that anything would match properly. > I m not sure to understand there. We are talking about synchronous > message. When using gen_server:call you do not receive answer to another > request, but yours. > It does that because the gen_server:call wraps your message with a tagged tuple containing for the sender pid and a unique ref. It then goes into a receive waiting for a message containing that unique ref. The receiving end understands this when it sends it reply it includes the unique ref so the sender will receive the right message. The reason for explicitly doing all this work is that all the receiving process sees is exactly the message that was sent, no more no less. There is no implicit information included at all. This means that if you were to add a synchronous message then you would have to either add a new message handling, not just in the message itself but maybe in receive as well. Or you could say that one of these messages will have a specific format which you can receive and you must reply with a specific format. Another problem is that synchronous message passing is much more complex and you need to define/handle many more things in the system than with async messages. For example how to handle you not getting a reply, do you use timeouts, or check if the other process has died, or a combination of both, etc? So you either end up with a set of special case operators/calls or a call with lots of options. And how should it work with distribution where today it is the same. No, I didn't like Joe's suggestion. :-) Robert -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Tue Jul 14 03:14:27 2015 From: lloyd@REDACTED (lloyd@REDACTED) Date: Mon, 13 Jul 2015 21:14:27 -0400 (EDT) Subject: [erlang-questions] Exporting a record type In-Reply-To: References: <1436486708.04388403@apps.rackspace.com> <330A96FE-A329-4D8E-8DAF-8DABF86716BB@writersglen.com> <9A55AD73-6BE0-4FD7-9372-874E47826F2B@basho.com> <1436544678.40631215@apps.rackspace.com> <927D610A-A519-4106-9D91-F53B5EC83485@writersglen.com> Message-ID: <1436836467.224211959@apps.rackspace.com> Hi Jesper et. al. This way cool stuff. The topic is much deeper than I thought. I need to play with it over the next few days to be sure I fully understand. But it feels like it deserves a fleshed out blog post or tutorial. I'd be happy to do the heavy lifting for such a post, but no doubt many more questions will flood my child-mind before I'm competent to do so. Do you folks mind if I continue in my role as Mickey-the_Dunce? Many thanks to all, Lloyd -----Original Message----- From: "Jesper Louis Andersen" Sent: Monday, July 13, 2015 6:03am To: "Lloyd R. Prentice" Cc: "Gordon Guthrie" , "erlang-questions" Subject: Re: [erlang-questions] Exporting a record type On Sun, Jul 12, 2015 at 5:06 PM, Lloyd R. Prentice wrote: > Also, in your repr/2 code (which is very suggestive of neat things one can > do) what is the significance of the View variable? As I see it now, it's > simply a tag like thumbnail_overview, but I'm not comfortable that my > understanding is correct. The idea, at least, is this: a "view" of some data, book, record, ..., is a way to cast that data into another structure by transforming it. I.e., "view" the data in another form. In statically typed languages, some languages support built-in view-types, so you can define these transformations formally, but in Erlang you have to make a more dynamic resolution. In its simple form, the view is just an atom(), requesting the view. But in a more advanced form, the view is a language which can be executed to construct a view of the data: ... repr(Book, {relation, R}) -> repr_rel(Book, R). repr_rel(#book { genre = G } = Book, genre) -> {G, Book}. Now, suppose you have a list of books, Bs: Rel = [repr(B, {relation, genre}) || B <- Bs], R = sofs:relation(Rel), F = sofs:relation_to_family(R), sofs:to_external(F). If for instance you have books [#book { genre = fantasy } = A, #book { genre = fantasy } = B, #book { genre = scifi } = C], Then the above would return [{fantasy, [A, B]}, {scifi, [C]}]. There are numerous tricks that can be played here, depending on what transformations you need. -- J. From wanglihe.programmer@REDACTED Tue Jul 14 01:54:25 2015 From: wanglihe.programmer@REDACTED (Lihe Wang) Date: Tue, 14 Jul 2015 07:54:25 +0800 Subject: [erlang-questions] Why application:start can not accept any args? In-Reply-To: References: Message-ID: Thank you Nuku. Yes it is a solution pass args to a application, but why not pass args directly by application:start(mycode1, Arg1)? Why otp design a api this way? And application behaviour's callback function start, it needs Args. It is seems like that something forgotten or something wrong. In application.erl or somewhere, it generate a atom 'normal' and blank list [], then passes them to mycode1:start(Type, Args). Why? 2015-07-13 18:39 GMT+08:00 Nuku Ameyibor : > u can add environmental variables for your application in your .app file > . > there is a tuple with a key env which u can use to add environmental > variables to your application which you can then access from your > application > eg . > {application, test, > [{vsn, "1.0.0"}, > {description, "testing"}, > {modules, [test, test_sup,test_start]}, > {applications, [stdlib, kernel, crypto]}, > {registered, [test,test_sup]}, > {mod, {test, []}}, > {env, > [ > {env1,b}, > {env2,a}, > {env3,c} > ] > } > ]}. > > you can access them like so > {ok, Env} = application:get_env(env1), etc.. > > there is also a set_env/3 ,set_env/4 which can be used to change > environmental variables during run-time > > > On Sun, Jul 12, 2015 at 1:19 PM, Lihe Wang > wrote: > >> Can any one give the explain about OTP? The application behaviour's call >> back "start" need args, but when I want start my code as a real >> application, I only can write "applicaion:start(mycode)", and can not pass >> any args to mycode. Why otp design like this? When I want start a >> application as background service from escript, for example: >> >> main([Arg1, Arg2, Arg3]) -> >> application:start(mycode1), %%with Arg1 >> application:start(mycode2), %%with Arg2 >> workcode(Arg3). %%workcode (escript) will run as a daemon, and need >> service supplied by mycode1 and mycode2. >> >> What is the right way to write these code? >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Tue Jul 14 07:44:09 2015 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Mon, 13 Jul 2015 22:44:09 -0700 Subject: [erlang-questions] 1st day of Erlang: what is wrong with this code In-Reply-To: References: <55A23D38.8070905@gmail.com> Message-ID: <55A4A1A9.1060806@gmail.com> On 07/12/2015 03:19 AM, avinash D'silva wrote: > Error in process <0.58.0> with exit value: > {function_clause,[{string,to_upper, > [<<"llll\r\n">>], > [{file,"string.erl"},{line,529}]}, > {echo,loop,1,[{file,"echo.erl"},{line,23}]}]} > > That was the error. But how would I find the cause of the error? > In java or other languages that I worked earlier, I would get a "Type > Mismatch Error". I'm fairly certain that this is how Erlang spells "Type Mismatch Error". To see this in action, try compiling this program that makes use of function guards: -module(test). -export([test/1]). test(Arg) when is_list(Arg) -> io:format("Is list~n"); test(Arg) when is_atom(Arg) -> io:format("Is atom~n"). If you run the function with a list, atom, and then binary, you can see that the first two are accepted, but there's no function to handle a binary, so the program dies: Erlang/OTP 18 [erts-7.0] [source] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] Eshell V7.0 (abort with ^G) 1> c(test). {ok,test} 2> spawn(test, test, ["hello"]). Is list <0.47.0> 3> spawn(test, test, [hello]). Is atom <0.49.0> 4> spawn(test, test, [<<"hello">>]). =ERROR REPORT==== 13-Jul-2015::22:34:24 === Error in process <0.53.0> with exit value: {function_clause,[{test,test,[<<"hello">>],[{file,"test.erl"},{line,4}]}]} <0.53.0> 5> (For information about spawn/3, check out http://www.erlang.org/doc/man/erlang.html#spawn-3 . I'm spawning the function, rather than calling it directly, as the shell does a bit of interpretation of the error, so it's less obvious that what is happening here is the same thing that was happening with the program that you were having trouble with.) Erlang error reports are hard to read at first. With more experience, they'll become easy to read. I'm also pretty sure that this is the place to ask questions when you have them. :) Good luck, and have fun! -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From essen@REDACTED Tue Jul 14 09:20:30 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 14 Jul 2015 09:20:30 +0200 Subject: [erlang-questions] Exporting a record type In-Reply-To: <1436836467.224211959@apps.rackspace.com> References: <1436486708.04388403@apps.rackspace.com> <330A96FE-A329-4D8E-8DAF-8DABF86716BB@writersglen.com> <9A55AD73-6BE0-4FD7-9372-874E47826F2B@basho.com> <1436544678.40631215@apps.rackspace.com> <927D610A-A519-4106-9D91-F53B5EC83485@writersglen.com> <1436836467.224211959@apps.rackspace.com> Message-ID: <55A4B83E.8090803@ninenines.eu> If you have a few months with nothing to do you can figure out sofs and add human-readable documentation in OTP. Then everyone can understand Jesper's post. :-) On 07/14/2015 03:14 AM, lloyd@REDACTED wrote: > Hi Jesper et. al. > > This way cool stuff. > > The topic is much deeper than I thought. I need to play with it over the next few days to be sure I fully understand. But it feels like it deserves a fleshed out blog post or tutorial. > > I'd be happy to do the heavy lifting for such a post, but no doubt many more questions will flood my child-mind before I'm competent to do so. Do you folks mind if I continue in my role as Mickey-the_Dunce? > > Many thanks to all, > > Lloyd > > -----Original Message----- > From: "Jesper Louis Andersen" > Sent: Monday, July 13, 2015 6:03am > To: "Lloyd R. Prentice" > Cc: "Gordon Guthrie" , "erlang-questions" > Subject: Re: [erlang-questions] Exporting a record type > > On Sun, Jul 12, 2015 at 5:06 PM, Lloyd R. Prentice > wrote: > >> Also, in your repr/2 code (which is very suggestive of neat things one can >> do) what is the significance of the View variable? As I see it now, it's >> simply a tag like thumbnail_overview, but I'm not comfortable that my >> understanding is correct. > > > The idea, at least, is this: a "view" of some data, book, record, ..., is a > way to cast that data into another structure by transforming it. I.e., > "view" the data in another form. In statically typed languages, some > languages support built-in view-types, so you can define these > transformations formally, but in Erlang you have to make a more dynamic > resolution. > > In its simple form, the view is just an atom(), requesting the view. But in > a more advanced form, the view is a language which can be executed to > construct a view of the data: > > ... > repr(Book, {relation, R}) -> > repr_rel(Book, R). > > repr_rel(#book { genre = G } = Book, genre) -> > {G, Book}. > > Now, suppose you have a list of books, Bs: > > Rel = [repr(B, {relation, genre}) || B <- Bs], > R = sofs:relation(Rel), > F = sofs:relation_to_family(R), > sofs:to_external(F). > > If for instance you have books [#book { genre = fantasy } = A, #book { > genre = fantasy } = B, #book { genre = scifi } = C], > > Then the above would return > > [{fantasy, [A, B]}, {scifi, [C]}]. > > There are numerous tricks that can be played here, depending on what > transformations you need. > > > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From jesper.louis.andersen@REDACTED Tue Jul 14 09:52:30 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 14 Jul 2015 09:52:30 +0200 Subject: [erlang-questions] Exporting a record type In-Reply-To: <55A4B83E.8090803@ninenines.eu> References: <1436486708.04388403@apps.rackspace.com> <330A96FE-A329-4D8E-8DAF-8DABF86716BB@writersglen.com> <9A55AD73-6BE0-4FD7-9372-874E47826F2B@basho.com> <1436544678.40631215@apps.rackspace.com> <927D610A-A519-4106-9D91-F53B5EC83485@writersglen.com> <1436836467.224211959@apps.rackspace.com> <55A4B83E.8090803@ninenines.eu> Message-ID: On Tue, Jul 14, 2015 at 9:20 AM, Lo?c Hoguin wrote: > If you have a few months with nothing to do you can figure out sofs and > add human-readable documentation in OTP. Then everyone can understand > Jesper's post. :-) It has already been written: "Th?orie des ensembles - Nicholas Bourbaki" http://www.amazon.com/Th%C3%A9orie-ensembles-French-Edition-Bourbaki/dp/3540340343 :) -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From coa@REDACTED Tue Jul 14 10:21:39 2015 From: coa@REDACTED (=?UTF-8?Q?Cl=C3=A1udio_Amaral?=) Date: Tue, 14 Jul 2015 09:21:39 +0100 Subject: [erlang-questions] Fwd: Automated Testing Tools in Erlang In-Reply-To: References: Message-ID: Hello all! We all know and love tools like eunit, quickcheck or proper, but there are more types of tools around that do different kinds of tests. I work at a place where I now have the opportunity to shift the technologies used on test automation and I would eventually like to have quickcheck/proper style of tests on our application models. Just doing it is a tough sell on my environment due to the time it may require to get things going (not many erlangers here, but enough to start making things happen, I guess). I thought it could be possible to build my way towards that and have all my automated test needs in Erlang. This way it would be easier to get people started with it. For this I need your help in finding such tools for BDD style tests and performance tests. I found references to cucumber implementations (which would be most interesting) but I find no tutorials or active developments (latest thing I found was from 3 years ago). I will continue to search, but I am sure you guys will show me a very cool set of resources to explore. Best Cl?udio Amaral -------------- next part -------------- An HTML attachment was scrubbed... URL: From mremond@REDACTED Tue Jul 14 13:15:56 2015 From: mremond@REDACTED (=?utf-8?b?TWlja2HDq2wgUsOpbW9uZA==?=) Date: Tue, 14 Jul 2015 13:15:56 +0200 Subject: [erlang-questions] eJabberd Loadbalancing In-Reply-To: References: Message-ID: <28721A92-4F25-456D-A4E2-F86C4F009208@process-one.net> Hello Kannan, On 13 Jul 2015, at 20:29, Kannan wrote: > Hi Team, > > We are working on a startup idea that uses eJabberd. We are trying to > build > the load balancing into the client, using a discovery call just before > the > login procedure. > > Do you see any pitfalls in this method of loadbalancing. No, this is a good approach that we have been using sometime. It should work well, as long as you can update the list of candidate fast enough to be very accurate. Cheers, -- Micka?l R?mond http://www.process-one.net From lloyd@REDACTED Tue Jul 14 15:34:30 2015 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Tue, 14 Jul 2015 09:34:30 -0400 Subject: [erlang-questions] Exporting a record type In-Reply-To: <55A4B83E.8090803@ninenines.eu> References: <1436486708.04388403@apps.rackspace.com> <330A96FE-A329-4D8E-8DAF-8DABF86716BB@writersglen.com> <9A55AD73-6BE0-4FD7-9372-874E47826F2B@basho.com> <1436544678.40631215@apps.rackspace.com> <927D610A-A519-4106-9D91-F53B5EC83485@writersglen.com> <1436836467.224211959@apps.rackspace.com> <55A4B83E.8090803@ninenines.eu> Message-ID: <1681CF11-AF85-4FB1-A95F-8A645BE9B5A3@writersglen.com> Howdy Lo?c, Well, I'm a pragmatist with a job to do. And, at my age I don't have many years left to pursue worthwhile projects. For this reason and others, I value greatly the insights and guidance that folks on this list have so generously provided. Perhaps Jesper is teasing us a bit. But to the extent that I understand the code that he has proposed, I sense serious intent and interesting ideas. Similarly, your book promises to be a cornucopia of hard-won practical insights about the craft of Erlang programming. But to my impatient mind your book and Jesper's code share the same frustrating attributes. They raise the issues without driving the point home. I'm not complaining. Your book is still a work-in-progress and Jesper is no doubt a busy man. So the best I can do is question the things I don't understand and, if able, help others grasp that that I do. But let's face it. Clear and simple explication of complicated matters is damned difficult. You can always spot those who really know by their talents in this regard. So my offer holds. If you, Jesper, and other Erlang wizards have the patience and will, perhaps through give-and-take discussion we can bring greater clarity to the subtleties of data representation and Erlang records. Each Erlang contributor stands on the shoulder of giants. Why should others have to live through the hair-pulling hours of frustration that you, Jesper, and every other Erlang programmer of note has experienced as they've sought to express their ideas and solve problems through Erlang? All the best, Lloyd Sent from my iPad > On Jul 14, 2015, at 3:20 AM, Lo?c Hoguin wrote: > > If you have a few months with nothing to do you can figure out sofs and add human-readable documentation in OTP. Then everyone can understand Jesper's post. :-) > >> On 07/14/2015 03:14 AM, lloyd@REDACTED wrote: >> Hi Jesper et. al. >> >> This way cool stuff. >> >> The topic is much deeper than I thought. I need to play with it over the next few days to be sure I fully understand. But it feels like it deserves a fleshed out blog post or tutorial. >> >> I'd be happy to do the heavy lifting for such a post, but no doubt many more questions will flood my child-mind before I'm competent to do so. Do you folks mind if I continue in my role as Mickey-the_Dunce? >> >> Many thanks to all, >> >> Lloyd >> >> -----Original Message----- >> From: "Jesper Louis Andersen" >> Sent: Monday, July 13, 2015 6:03am >> To: "Lloyd R. Prentice" >> Cc: "Gordon Guthrie" , "erlang-questions" >> Subject: Re: [erlang-questions] Exporting a record type >> >> On Sun, Jul 12, 2015 at 5:06 PM, Lloyd R. Prentice >> wrote: >> >>> Also, in your repr/2 code (which is very suggestive of neat things one can >>> do) what is the significance of the View variable? As I see it now, it's >>> simply a tag like thumbnail_overview, but I'm not comfortable that my >>> understanding is correct. >> >> >> The idea, at least, is this: a "view" of some data, book, record, ..., is a >> way to cast that data into another structure by transforming it. I.e., >> "view" the data in another form. In statically typed languages, some >> languages support built-in view-types, so you can define these >> transformations formally, but in Erlang you have to make a more dynamic >> resolution. >> >> In its simple form, the view is just an atom(), requesting the view. But in >> a more advanced form, the view is a language which can be executed to >> construct a view of the data: >> >> ... >> repr(Book, {relation, R}) -> >> repr_rel(Book, R). >> >> repr_rel(#book { genre = G } = Book, genre) -> >> {G, Book}. >> >> Now, suppose you have a list of books, Bs: >> >> Rel = [repr(B, {relation, genre}) || B <- Bs], >> R = sofs:relation(Rel), >> F = sofs:relation_to_family(R), >> sofs:to_external(F). >> >> If for instance you have books [#book { genre = fantasy } = A, #book { >> genre = fantasy } = B, #book { genre = scifi } = C], >> >> Then the above would return >> >> [{fantasy, [A, B]}, {scifi, [C]}]. >> >> There are numerous tricks that can be played here, depending on what >> transformations you need. > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang From lloyd@REDACTED Tue Jul 14 15:38:47 2015 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Tue, 14 Jul 2015 09:38:47 -0400 Subject: [erlang-questions] Exporting a record type In-Reply-To: <55A4B83E.8090803@ninenines.eu> References: <1436486708.04388403@apps.rackspace.com> <330A96FE-A329-4D8E-8DAF-8DABF86716BB@writersglen.com> <9A55AD73-6BE0-4FD7-9372-874E47826F2B@basho.com> <1436544678.40631215@apps.rackspace.com> <927D610A-A519-4106-9D91-F53B5EC83485@writersglen.com> <1436836467.224211959@apps.rackspace.com> <55A4B83E.8090803@ninenines.eu> Message-ID: <43633FBB-EF13-428C-920E-DA741A333E8E@writersglen.com> P.S. Nicolas Bourbaki was a committee. There's hope for us yet. LRP Sent from my iPad > On Jul 14, 2015, at 3:20 AM, Lo?c Hoguin wrote: > > If you have a few months with nothing to do you can figure out sofs and add human-readable documentation in OTP. Then everyone can understand Jesper's post. :-) > >> On 07/14/2015 03:14 AM, lloyd@REDACTED wrote: >> Hi Jesper et. al. >> >> This way cool stuff. >> >> The topic is much deeper than I thought. I need to play with it over the next few days to be sure I fully understand. But it feels like it deserves a fleshed out blog post or tutorial. >> >> I'd be happy to do the heavy lifting for such a post, but no doubt many more questions will flood my child-mind before I'm competent to do so. Do you folks mind if I continue in my role as Mickey-the_Dunce? >> >> Many thanks to all, >> >> Lloyd >> >> -----Original Message----- >> From: "Jesper Louis Andersen" >> Sent: Monday, July 13, 2015 6:03am >> To: "Lloyd R. Prentice" >> Cc: "Gordon Guthrie" , "erlang-questions" >> Subject: Re: [erlang-questions] Exporting a record type >> >> On Sun, Jul 12, 2015 at 5:06 PM, Lloyd R. Prentice >> wrote: >> >>> Also, in your repr/2 code (which is very suggestive of neat things one can >>> do) what is the significance of the View variable? As I see it now, it's >>> simply a tag like thumbnail_overview, but I'm not comfortable that my >>> understanding is correct. >> >> >> The idea, at least, is this: a "view" of some data, book, record, ..., is a >> way to cast that data into another structure by transforming it. I.e., >> "view" the data in another form. In statically typed languages, some >> languages support built-in view-types, so you can define these >> transformations formally, but in Erlang you have to make a more dynamic >> resolution. >> >> In its simple form, the view is just an atom(), requesting the view. But in >> a more advanced form, the view is a language which can be executed to >> construct a view of the data: >> >> ... >> repr(Book, {relation, R}) -> >> repr_rel(Book, R). >> >> repr_rel(#book { genre = G } = Book, genre) -> >> {G, Book}. >> >> Now, suppose you have a list of books, Bs: >> >> Rel = [repr(B, {relation, genre}) || B <- Bs], >> R = sofs:relation(Rel), >> F = sofs:relation_to_family(R), >> sofs:to_external(F). >> >> If for instance you have books [#book { genre = fantasy } = A, #book { >> genre = fantasy } = B, #book { genre = scifi } = C], >> >> Then the above would return >> >> [{fantasy, [A, B]}, {scifi, [C]}]. >> >> There are numerous tricks that can be played here, depending on what >> transformations you need. > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang From max.lapshin@REDACTED Tue Jul 14 16:20:34 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 14 Jul 2015 17:20:34 +0300 Subject: [erlang-questions] Openssh server replace by erlang ssh? Message-ID: Is it possible to launch bash as an erlang ssh shell? I don't know exactly what problems may be with it, but I suppose that it is not just a byte to byte copy, because there are some issues with opening pty. I've implemented very, very simple piece of code: https://gist.github.com/maxlapshin/db32ffdcc06a5fe21c40 It allows to connect and run bash, but when I press Ctrl+c on ssh client, it doesn't send ctrl+c to erlang ssh server, but just closes. I suppose that it happens because erlang ssh server doesn't tell to forward control sequences. Is it possible to fix it somehow? -------------- next part -------------- An HTML attachment was scrubbed... URL: From h87rguez@REDACTED Tue Jul 14 15:08:28 2015 From: h87rguez@REDACTED (=?UTF-8?Q?Humberto_Rodr=C3=ADguez_Avila?=) Date: Tue, 14 Jul 2015 15:08:28 +0200 Subject: [erlang-questions] erl_sytanx help Message-ID: Hello guys! I am a new programming in Erlang and I interested in doing things with meta-programming. I have been working with merl and I did a couple of examples (module generation). Nevertheless, I have been trying to do the same thing using erl_syntax, but I can?t found how to build the syntax tree of the directive -import :( For example: -import(mymdule, [test1/0, test2/0]) Any suggestion? Thanks in advance! --- Humberto -------------- next part -------------- An HTML attachment was scrubbed... URL: From andra.dinu@REDACTED Tue Jul 14 19:17:11 2015 From: andra.dinu@REDACTED (Andra Dinu) Date: Tue, 14 Jul 2015 18:17:11 +0100 Subject: [erlang-questions] Call For Talks: Berlin Erlang Factory Lite 1 Dec Message-ID: Berlin EFL is back for one day of talks on 1 December and three days of training courses on 2-4 December. Erlang & Elixir talks wanted, submit yours now! http://www.erlang-factory.com/berlin2015/home#home *Andra Dinu* Community & Social -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Tue Jul 14 22:06:12 2015 From: vasdeveloper@REDACTED (Kannan) Date: Wed, 15 Jul 2015 01:36:12 +0530 Subject: [erlang-questions] JSON Parser Message-ID: Hi Team, I come across many JSON libraries. Once from MochiWeb, Other one from Yaws. Third one from CouchDB. And some others through Googling. Which one is the fastest, if you had ever got a chance to evaluate them. Regards, Theepan -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Tue Jul 14 22:58:09 2015 From: vasdeveloper@REDACTED (Kannan) Date: Wed, 15 Jul 2015 02:28:09 +0530 Subject: [erlang-questions] CORS Supporting Erlang HTTP Server Message-ID: Hi Team, Is there a Erlang HTTP server, including Yaws, that supports CORS by configuration? Regards, Theepan -------------- next part -------------- An HTML attachment was scrubbed... URL: From radek@REDACTED Tue Jul 14 23:25:40 2015 From: radek@REDACTED (Radoslaw Gruchalski) Date: Tue, 14 Jul 2015 21:25:40 +0000 (UTC) Subject: [erlang-questions] CORS Supporting Erlang HTTP Server In-Reply-To: References: Message-ID: CORS is all based on headers. Cowboy can definitely do it. Sent from Outlook On Tue, Jul 14, 2015 at 1:58 PM -0700, "Kannan" wrote: Hi Team, Is there a Erlang HTTP server, including Yaws, that supports CORS by configuration? Regards,Theepan -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Wed Jul 15 00:02:37 2015 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 15 Jul 2015 00:02:37 +0200 Subject: [erlang-questions] erl_sytanx help In-Reply-To: References: Message-ID: This is a good opportunity to demonstrate the function erl_syntax:meta/1 (using merl in OTP 18 to simplify building the AST): 1> AST=merl:quote("-import(mymdule, [test1/0, test2/0])"). 2> io:put_chars(erl_prettypr:format(erl_syntax:meta(AST))). erl_syntax:make_tree(prefix_expr, [[erl_syntax:operator('-')], [erl_syntax:make_tree(application, [[erl_syntax:atom(import)], [erl_syntax:atom(mymdule), erl_syntax:list([erl_syntax:make_tree(infix_expr, [[erl_syntax:atom(test1)], [erl_syntax:operator('/')], [erl_syntax:integer(0)]]), erl_syntax:make_tree(infix_expr, [[erl_syntax:atom(test2)], [erl_syntax:operator('/')], [erl_syntax:integer(0)]])])]])]]) There's also the useful function merl:show/1 for inspecting the basic structure of an AST: 3> merl:show(AST). prefix_expr: -import(mymdule, [...]) operator: - + application: import(mymdule, [test1 / 0, test2 / 0]) atom: import + atom: mymdule list: [test1 / 0, test2 / 0] infix_expr: test1 / 0 atom: test1 + operator: / + integer: 0 infix_expr: test2 / 0 atom: test2 + operator: / + integer: 0 /Richard On Tue, Jul 14, 2015 at 3:08 PM, Humberto Rodr?guez Avila < h87rguez@REDACTED> wrote: > Hello guys! > > I am a new programming in Erlang and I interested in doing things with > meta-programming. I have been working with merl and I did a couple of > examples (module generation). Nevertheless, I have been trying to do the > same thing using erl_syntax, but I can?t found how to build the syntax > tree of the directive -import :( > > For example: -import(mymdule, [test1/0, test2/0]) > > Any suggestion? > > Thanks in advance! > > --- Humberto > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From be.dmitry@REDACTED Wed Jul 15 01:22:06 2015 From: be.dmitry@REDACTED (Dmitry Belyaev) Date: Wed, 15 Jul 2015 09:22:06 +1000 Subject: [erlang-questions] CORS Supporting Erlang HTTP Server In-Reply-To: References: Message-ID: <3560FC65-A67A-42BC-9F42-F298EC84DAC3@gmail.com> Not out of the box. There is https://github.com/danielwhite/cowboy_cors to do it. -- Best wishes, Dmitry Belyaev On 15 July 2015 7:25:40 AM AEST, Radoslaw Gruchalski wrote: >CORS is all based on headers. Cowboy can definitely do it. > >Sent from Outlook > > > > >On Tue, Jul 14, 2015 at 1:58 PM -0700, "Kannan" > wrote: > > > > > > > > > > >Hi Team, >Is there a Erlang HTTP server, including Yaws, that supports CORS by >configuration? >Regards,Theepan > >------------------------------------------------------------------------ > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Jul 15 11:03:56 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 15 Jul 2015 11:03:56 +0200 Subject: [erlang-questions] JSON Parser In-Reply-To: References: Message-ID: On Tue, Jul 14, 2015 at 10:06 PM, Kannan wrote: > I come across many JSON libraries. Once from MochiWeb, Other one from > Yaws. Third one from CouchDB. And some others through Googling. > There are two very popular JSON parsers in Erlang: jsx and jiffy. jsx is written in plain Erlang. It is fast, correct and since it is written in Erlang, it will also automatically yield for other processes in the system. jiffy is written as a C NIF. It is about 10 times faster than jsx, but the caveat is everything that has to do with C NIFs: blocking a scheduler, C code having errors, security considerations, etc. I tend to run with `jsx` in my projects, and then I switch away from JSON when it gets to slow. JSON is a bad format that should never have existed in the first place. We are stuck with it because it's historic alternative, XML, was far worse in every aspect. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nayibor@REDACTED Wed Jul 15 12:36:17 2015 From: nayibor@REDACTED (Nuku Ameyibor) Date: Wed, 15 Jul 2015 10:36:17 +0000 Subject: [erlang-questions] Why application:start can not accept any args? In-Reply-To: References: Message-ID: application:start/1,2 is intended for starting applications not loading configurations . it checks that all included applications are loaded b5 starting,plus some other stuff etc... there are multiple ways to pass arguments to an erlang application .some override others etc... have a look at http://www.erlang.org/doc/man/config.html . it will give u different ways to do that . On Mon, Jul 13, 2015 at 11:54 PM, Lihe Wang wrote: > Thank you Nuku. Yes it is a solution pass args to a application, but why > not pass args directly by application:start(mycode1, Arg1)? Why otp design > a api this way? And application behaviour's callback function start, it > needs Args. It is seems like that something forgotten or something wrong. > In application.erl or somewhere, it generate a atom 'normal' and blank list > [], then passes them to mycode1:start(Type, Args). Why? > > 2015-07-13 18:39 GMT+08:00 Nuku Ameyibor : > >> u can add environmental variables for your application in your .app >> file . >> there is a tuple with a key env which u can use to add environmental >> variables to your application which you can then access from your >> application >> eg . >> {application, test, >> [{vsn, "1.0.0"}, >> {description, "testing"}, >> {modules, [test, test_sup,test_start]}, >> {applications, [stdlib, kernel, crypto]}, >> {registered, [test,test_sup]}, >> {mod, {test, []}}, >> {env, >> [ >> {env1,b}, >> {env2,a}, >> {env3,c} >> ] >> } >> ]}. >> >> you can access them like so >> {ok, Env} = application:get_env(env1), etc.. >> >> there is also a set_env/3 ,set_env/4 which can be used to change >> environmental variables during run-time >> >> >> On Sun, Jul 12, 2015 at 1:19 PM, Lihe Wang > > wrote: >> >>> Can any one give the explain about OTP? The application behaviour's call >>> back "start" need args, but when I want start my code as a real >>> application, I only can write "applicaion:start(mycode)", and can not pass >>> any args to mycode. Why otp design like this? When I want start a >>> application as background service from escript, for example: >>> >>> main([Arg1, Arg2, Arg3]) -> >>> application:start(mycode1), %%with Arg1 >>> application:start(mycode2), %%with Arg2 >>> workcode(Arg3). %%workcode (escript) will run as a daemon, and need >>> service supplied by mycode1 and mycode2. >>> >>> What is the right way to write these code? >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ddosia@REDACTED Wed Jul 15 13:18:01 2015 From: ddosia@REDACTED (Daniil Churikov) Date: Wed, 15 Jul 2015 12:18:01 +0100 Subject: [erlang-questions] How to reproduce OOM. Message-ID: Recently I was searching for robust way to restart elang VM in case of sudden stop (like crash or OOM). And I found rather difficult to reproduce OOM conditions: I was trying to create gigantic list of integers, so VM would be killed. I did this: L1 = [I || I <- lists:seq(1, 1000)]. L2 = [L1 || _ <- lists:seq(1, 1000)]. L3 = [L2 || _ <- lists:seq(1, 1000)]. % here VM hangs But according to syslog it doesn't look like OOM death, I can't see usual log entries like "Out of memory: Kill process 3855 (beam.smp) score 909 or sacrifice child" and others. My current understanding is I asked more memory then I allowed to ask, and b/c OS is not willing to give this memory, erlang VM hangs. If somebody have ideas how to reproduce OOM I would appreciate for sharing. Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From dszoboszlay@REDACTED Wed Jul 15 13:24:09 2015 From: dszoboszlay@REDACTED (=?UTF-8?Q?D=C3=A1niel_Szoboszlay?=) Date: Wed, 15 Jul 2015 13:24:09 +0200 Subject: [erlang-questions] How to reproduce OOM. In-Reply-To: References: Message-ID: This one works for me quite reliably: Eshell V5.10.4.0.1 (abort with ^G) 1> <<1:8000000000000>>. Crash dump was written to: erl_crash.dump binary_alloc: Cannot allocate 1000000000031 bytes of memory (of type "binary"). Aborted (core dumped) 2015-07-15 13:18 GMT+02:00 Daniil Churikov : > Recently I was searching for robust way to restart elang VM in case of > sudden > stop (like crash or OOM). > And I found rather difficult to reproduce OOM conditions: I was trying to > create > gigantic list of integers, so VM would be killed. > I did this: > > L1 = [I || I <- lists:seq(1, 1000)]. > L2 = [L1 || _ <- lists:seq(1, 1000)]. > L3 = [L2 || _ <- lists:seq(1, 1000)]. % here VM hangs > > But according to syslog it doesn't look like OOM death, I can't see usual > log > entries like "Out of memory: Kill process 3855 (beam.smp) score 909 or > sacrifice > child" and others. > > My current understanding is I asked more memory then I allowed to ask, and > b/c > OS is not willing to give this memory, erlang VM hangs. > > If somebody have ideas how to reproduce OOM I would appreciate for sharing. > > Thanks! > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ddosia@REDACTED Wed Jul 15 13:33:37 2015 From: ddosia@REDACTED (Daniil Churikov) Date: Wed, 15 Jul 2015 12:33:37 +0100 Subject: [erlang-questions] How to reproduce OOM. In-Reply-To: References: Message-ID: Hello D?niel. Thanks for your suggestions, although it indeed creates memory allocation error, it does not do what I need. I want VM to be killed, but don't want to do it by myself. I am seeking for OOM killer's help, if you know what I mean ;) 2015-07-15 12:24 GMT+01:00 D?niel Szoboszlay : > This one works for me quite reliably: > > Eshell V5.10.4.0.1 (abort with ^G) > 1> <<1:8000000000000>>. > > Crash dump was written to: erl_crash.dump > binary_alloc: Cannot allocate 1000000000031 bytes of memory (of type > "binary"). > Aborted (core dumped) > > > 2015-07-15 13:18 GMT+02:00 Daniil Churikov : > >> Recently I was searching for robust way to restart elang VM in case of >> sudden >> stop (like crash or OOM). >> And I found rather difficult to reproduce OOM conditions: I was trying to >> create >> gigantic list of integers, so VM would be killed. >> I did this: >> >> L1 = [I || I <- lists:seq(1, 1000)]. >> L2 = [L1 || _ <- lists:seq(1, 1000)]. >> L3 = [L2 || _ <- lists:seq(1, 1000)]. % here VM hangs >> >> But according to syslog it doesn't look like OOM death, I can't see usual >> log >> entries like "Out of memory: Kill process 3855 (beam.smp) score 909 or >> sacrifice >> child" and others. >> >> My current understanding is I asked more memory then I allowed to ask, >> and b/c >> OS is not willing to give this memory, erlang VM hangs. >> >> If somebody have ideas how to reproduce OOM I would appreciate for >> sharing. >> >> Thanks! >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon@REDACTED Wed Jul 15 14:07:22 2015 From: jon@REDACTED (Jon Schneider) Date: Wed, 15 Jul 2015 13:07:22 +0100 Subject: [erlang-questions] How to reproduce OOM. In-Reply-To: References: Message-ID: <0050ba502be920e734b294dbb8ea9984.squirrel@mail.jschneider.net> The OOM killer is not a reliable thing and changes with the wind. If you want a ceiling on memory how about using ulimit possibly from the shell before starting beam ? Jon > Hello D??niel. Thanks for your suggestions, although it indeed creates > memory allocation error, > it does not do what I need. > I want VM to be killed, but don't want to do it by myself. I am seeking > for > OOM killer's help, > if you know what I mean ;) > > 2015-07-15 12:24 GMT+01:00 D??niel Szoboszlay : > >> This one works for me quite reliably: >> >> Eshell V5.10.4.0.1 (abort with ^G) >> 1> <<1:8000000000000>>. >> >> Crash dump was written to: erl_crash.dump >> binary_alloc: Cannot allocate 1000000000031 bytes of memory (of type >> "binary"). >> Aborted (core dumped) >> >> >> 2015-07-15 13:18 GMT+02:00 Daniil Churikov : >> >>> Recently I was searching for robust way to restart elang VM in case of >>> sudden >>> stop (like crash or OOM). >>> And I found rather difficult to reproduce OOM conditions: I was trying >>> to >>> create >>> gigantic list of integers, so VM would be killed. >>> I did this: >>> >>> L1 = [I || I <- lists:seq(1, 1000)]. >>> L2 = [L1 || _ <- lists:seq(1, 1000)]. >>> L3 = [L2 || _ <- lists:seq(1, 1000)]. % here VM hangs >>> >>> But according to syslog it doesn't look like OOM death, I can't see >>> usual >>> log >>> entries like "Out of memory: Kill process 3855 (beam.smp) score 909 or >>> sacrifice >>> child" and others. >>> >>> My current understanding is I asked more memory then I allowed to ask, >>> and b/c >>> OS is not willing to give this memory, erlang VM hangs. >>> >>> If somebody have ideas how to reproduce OOM I would appreciate for >>> sharing. >>> >>> Thanks! >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From garry@REDACTED Wed Jul 15 17:29:21 2015 From: garry@REDACTED (Garry Hodgson) Date: Wed, 15 Jul 2015 11:29:21 -0400 Subject: [erlang-questions] JSON Parser In-Reply-To: References: Message-ID: <55A67C51.6000201@research.att.com> On 7/15/15 5:03 AM, Jesper Louis Andersen wrote: > JSON is a bad format that should never have existed in the first > place. We are stuck with it because it's historic alternative, XML, > was far worse in every aspect. well, not *every* respect. i imagine it sold a lot of hard drives. From diparfitt@REDACTED Wed Jul 15 16:50:33 2015 From: diparfitt@REDACTED (Dave Parfitt) Date: Wed, 15 Jul 2015 10:50:33 -0400 Subject: [erlang-questions] [ANN] erln8, a sneaky version manager for Erlang, Elixir and Rebar/Rebar3 Message-ID: Hello - erln8 is set of tools that let you build and then set versions of Erlang, Elixir, Rebar and Rebar3 per directory. While an older version of erln8 has been around for a couple years, the latest rewrite *beta* now has Elixir support. Note, that as it's still a beta, there might be a couple rough spots here and there. Friendly pull requests welcome. Here's an example of building the latest version of Erlang + Rebar, and then setting versions to use in multiple directories: # build the latest tagged version of Erlang (OTP-18.0.2 at the time of writing) erln8 --build-latest # build OTP_R16B03-1 erln8 --build OTP_R16B03-1 # build the latest tagged version of Rebar reo --build-latest cd ~/some/project erln8 --use OTP-18.0.2 # any calls to erlang commands in this directory will be for OTP-18.0.2 cd ~/some/other/project erln8 --use OTP_R16B03-1 # any calls to erlang commands in this directory will be for OTP_R16B03-1 You can download erln8 source, docs and prebuilt binaries here: http://erln8.github.io/erln8/ Cheers - Dave https://github.com/metadave From vasdeveloper@REDACTED Wed Jul 15 20:00:12 2015 From: vasdeveloper@REDACTED (Kannan) Date: Wed, 15 Jul 2015 23:30:12 +0530 Subject: [erlang-questions] JSON Parser In-Reply-To: References: Message-ID: Is any of them supporting Erlang 'record' as their base for encoding/decoding. I see many of them are doing it with just list of tuples of binaries.Erlang records best match the structure of JSON format. JSON ---------- {"name": "Theepan", "work": "Coding", "salary": "0" } Matching Erlang record ----------------------------------- -record( json_record, { 'name' = "Theepan", 'work' = "Coding", 'salary' = "0" } } Thanks, Theepan On Wed, Jul 15, 2015 at 2:33 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Tue, Jul 14, 2015 at 10:06 PM, Kannan wrote: > >> I come across many JSON libraries. Once from MochiWeb, Other one from >> Yaws. Third one from CouchDB. And some others through Googling. >> > > There are two very popular JSON parsers in Erlang: jsx and jiffy. > > jsx is written in plain Erlang. It is fast, correct and since it is > written in Erlang, it will also automatically yield for other processes in > the system. > > jiffy is written as a C NIF. It is about 10 times faster than jsx, but the > caveat is everything that has to do with C NIFs: blocking a scheduler, C > code having errors, security considerations, etc. > > I tend to run with `jsx` in my projects, and then I switch away from JSON > when it gets to slow. JSON is a bad format that should never have existed > in the first place. We are stuck with it because it's historic alternative, > XML, was far worse in every aspect. > > > -- > J. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From garry@REDACTED Wed Jul 15 20:03:20 2015 From: garry@REDACTED (Garry Hodgson) Date: Wed, 15 Jul 2015 14:03:20 -0400 Subject: [erlang-questions] JSON Parser In-Reply-To: References: Message-ID: <55A6A068.4020703@research.att.com> I prefer mapping json objects to erlang maps, lists to lists, and strings to binaries. i use a wrapper around mochijson to do that. On 7/15/15 2:00 PM, Kannan wrote: > Is any of them supporting Erlang 'record' as their base for > encoding/decoding. I see many of them are doing it with just list of > tuples of binaries.Erlang records best match the structure of JSON > format. > > JSON > ---------- > {"name": "Theepan", > "work": "Coding", > "salary": "0" > } > > Matching Erlang record > ----------------------------------- > -record( json_record, > { > 'name' = "Theepan", > 'work' = "Coding", > 'salary' = "0" > } > } > > Thanks, > Theepan > > On Wed, Jul 15, 2015 at 2:33 PM, Jesper Louis Andersen > > wrote: > > > On Tue, Jul 14, 2015 at 10:06 PM, Kannan > wrote: > > I come across many JSON libraries. Once from MochiWeb, Other > one from Yaws. Third one from CouchDB. And some others through > Googling. > > > There are two very popular JSON parsers in Erlang: jsx and jiffy. > > jsx is written in plain Erlang. It is fast, correct and since it > is written in Erlang, it will also automatically yield for other > processes in the system. > > jiffy is written as a C NIF. It is about 10 times faster than jsx, > but the caveat is everything that has to do with C NIFs: blocking > a scheduler, C code having errors, security considerations, etc. > > I tend to run with `jsx` in my projects, and then I switch away > from JSON when it gets to slow. JSON is a bad format that should > never have existed in the first place. We are stuck with it > because it's historic alternative, XML, was far worse in every aspect. > > > -- > J. > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Wed Jul 15 20:04:13 2015 From: vasdeveloper@REDACTED (Kannan) Date: Wed, 15 Jul 2015 23:34:13 +0530 Subject: [erlang-questions] CORS Supporting Erlang HTTP Server In-Reply-To: <3560FC65-A67A-42BC-9F42-F298EC84DAC3@gmail.com> References: <3560FC65-A67A-42BC-9F42-F298EC84DAC3@gmail.com> Message-ID: Which one is the best? Yaws or Cowboy? I could not find any comparison matrix on the internet. Thanks, Theepan On Wed, Jul 15, 2015 at 4:52 AM, Dmitry Belyaev wrote: > Not out of the box. There is https://github.com/danielwhite/cowboy_cors > to do it. > -- > Best wishes, > Dmitry Belyaev > > On 15 July 2015 7:25:40 AM AEST, Radoslaw Gruchalski > wrote: > >> CORS is all based on headers. Cowboy can definitely do it. >> >> Sent from Outlook >> >> >> >> >> On Tue, Jul 14, 2015 at 1:58 PM -0700, "Kannan" >> wrote: >> >> Hi Team, >>> >>> Is there a Erlang HTTP server, including Yaws, that supports CORS by >>> configuration? >>> >>> Regards, >>> Theepan >>> >> ------------------------------ >> >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Wed Jul 15 20:24:39 2015 From: bob@REDACTED (Bob Ippolito) Date: Wed, 15 Jul 2015 11:24:39 -0700 Subject: [erlang-questions] JSON Parser In-Reply-To: References: Message-ID: Records are a compile time construct, so in order to parse as records you'd need to define the records in advance and provide a schema and record definition to the parser for the document. This could be done in a layer above the JSON parser. Maps are the data structure you're looking for. Most of the JSON implementations provide backwards compatibility to versions of Erlang before maps, so other data structures (such as dicts or proplists) are more common. On Wednesday, July 15, 2015, Kannan wrote: > Is any of them supporting Erlang 'record' as their base for > encoding/decoding. I see many of them are doing it with just list of tuples > of binaries.Erlang records best match the structure of JSON format. > > JSON > ---------- > {"name": "Theepan", > "work": "Coding", > "salary": "0" > } > > Matching Erlang record > ----------------------------------- > -record( json_record, > { > 'name' = "Theepan", > 'work' = "Coding", > 'salary' = "0" > } > } > > Thanks, > Theepan > > On Wed, Jul 15, 2015 at 2:33 PM, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED > > wrote: > >> >> On Tue, Jul 14, 2015 at 10:06 PM, Kannan > > wrote: >> >>> I come across many JSON libraries. Once from MochiWeb, Other one from >>> Yaws. Third one from CouchDB. And some others through Googling. >>> >> >> There are two very popular JSON parsers in Erlang: jsx and jiffy. >> >> jsx is written in plain Erlang. It is fast, correct and since it is >> written in Erlang, it will also automatically yield for other processes in >> the system. >> >> jiffy is written as a C NIF. It is about 10 times faster than jsx, but >> the caveat is everything that has to do with C NIFs: blocking a scheduler, >> C code having errors, security considerations, etc. >> >> I tend to run with `jsx` in my projects, and then I switch away from JSON >> when it gets to slow. JSON is a bad format that should never have existed >> in the first place. We are stuck with it because it's historic alternative, >> XML, was far worse in every aspect. >> >> >> -- >> J. >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Wed Jul 15 20:28:05 2015 From: vasdeveloper@REDACTED (Kannan) Date: Wed, 15 Jul 2015 23:58:05 +0530 Subject: [erlang-questions] JSON Parser In-Reply-To: <55A6A068.4020703@research.att.com> References: <55A6A068.4020703@research.att.com> Message-ID: I was out of Erlang programing for the last five years, I think maps has been introduced during that time. I was not aware of it earlier. Just went through the documentation, and yes it is the best format to match JSONs. The syntax is quite complicated though. Using two characters to map key and value, assignment, update, access and matching have different K-V separator symbols etc. However, as a declaration free K-V grouping structure, it is the best option. Regards, Theepan On Wed, Jul 15, 2015 at 11:33 PM, Garry Hodgson wrote: > I prefer mapping json objects to erlang maps, lists to lists, and > strings to binaries. i use a wrapper around mochijson to do that. > > > On 7/15/15 2:00 PM, Kannan wrote: > > Is any of them supporting Erlang 'record' as their base for > encoding/decoding. I see many of them are doing it with just list of tuples > of binaries.Erlang records best match the structure of JSON format. > > JSON > ---------- > {"name": "Theepan", > "work": "Coding", > "salary": "0" > } > > Matching Erlang record > ----------------------------------- > -record( json_record, > { > 'name' = "Theepan", > 'work' = "Coding", > 'salary' = "0" > } > } > > Thanks, > Theepan > > On Wed, Jul 15, 2015 at 2:33 PM, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > >> >> On Tue, Jul 14, 2015 at 10:06 PM, Kannan wrote: >> >>> I come across many JSON libraries. Once from MochiWeb, Other one from >>> Yaws. Third one from CouchDB. And some others through Googling. >>> >> >> There are two very popular JSON parsers in Erlang: jsx and jiffy. >> >> jsx is written in plain Erlang. It is fast, correct and since it is >> written in Erlang, it will also automatically yield for other processes in >> the system. >> >> jiffy is written as a C NIF. It is about 10 times faster than jsx, but >> the caveat is everything that has to do with C NIFs: blocking a scheduler, >> C code having errors, security considerations, etc. >> >> I tend to run with `jsx` in my projects, and then I switch away from >> JSON when it gets to slow. JSON is a bad format that should never have >> existed in the first place. We are stuck with it because it's historic >> alternative, XML, was far worse in every aspect. >> >> >> -- >> J. >> > > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Jul 15 20:51:34 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 15 Jul 2015 20:51:34 +0200 Subject: [erlang-questions] CORS Supporting Erlang HTTP Server In-Reply-To: References: <3560FC65-A67A-42BC-9F42-F298EC84DAC3@gmail.com> Message-ID: <55A6ABB6.60107@ninenines.eu> They have different goals and states. Cowboy is young and still sees fairly large changes (in part to accomodate to newer protocols like HTTP/2 or Websocket compression, and in part because there are still usability issues). However from version 2 onward changes should be minimal for a while. Cowboy aims to support the core protocols for Web development, and nothing more. This means HTTP 1 and 2, SPDY, Websocket, and so on. It also includes a REST machine (adapted from Webmachine). The main focus is on REST and Websocket first. It does not include anything related to formats like JSON or HTML, enterprise protocols like RPC/SOAP/.. or anything related to XML, or Flash and so on. There's a constant effort on keeping things small and tidy, and providing functionality that covers 99% of users. The changes are user driven, with unused features happily removed while new features get added after detecting patterns in the many user questions I receive (but only if they are in the scope, of course). Hope that helps. On 07/15/2015 08:04 PM, Kannan wrote: > Which one is the best? Yaws or Cowboy? I could not find any comparison > matrix on the internet. > > Thanks, > Theepan > > On Wed, Jul 15, 2015 at 4:52 AM, Dmitry Belyaev > wrote: > > Not out of the box. There is > https://github.com/danielwhite/cowboy_cors to do it. > -- > Best wishes, > Dmitry Belyaev > > On 15 July 2015 7:25:40 AM AEST, Radoslaw Gruchalski > > wrote: > > CORS is all based on headers. Cowboy can definitely do it. > > Sent from Outlook > > > > > On Tue, Jul 14, 2015 at 1:58 PM -0700, "Kannan" > > wrote: > > Hi Team, > > Is there a Erlang HTTP server, including Yaws, that supports > CORS by configuration? > > Regards, > Theepan > > ------------------------------------------------------------------------ > > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From rtrlists@REDACTED Wed Jul 15 20:54:28 2015 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 15 Jul 2015 19:54:28 +0100 Subject: [erlang-questions] Fwd: Automated Testing Tools in Erlang In-Reply-To: References: Message-ID: Have you had a look at the Common Test system that comes with Erlang? While it is not overtly BDD, the way you write tests are for the positive case, going with the usual "let it crash" philosophy, meaning the test failed if it crashed. This makes writing tests in a BDD fashion almost superfluous. /Robby On Jul 14, 2015 10:21 AM, "Cl?udio Amaral" wrote: > Hello all! > > We all know and love tools like eunit, quickcheck or proper, but there are > more types of tools around that do different kinds of tests. > > I work at a place where I now have the opportunity to shift the > technologies used on test automation and I would eventually like to have > quickcheck/proper style of tests on our application models. > > Just doing it is a tough sell on my environment due to the time it may > require to get things going (not many erlangers here, but enough to start > making things happen, I guess). > > I thought it could be possible to build my way towards that and have all > my automated test needs in Erlang. This way it would be easier to get > people started with it. > > For this I need your help in finding such tools for BDD style tests and > performance tests. I found references to cucumber implementations (which > would be most interesting) but I find no tutorials or active developments > (latest thing I found was from 3 years ago). I will continue to search, but > I am sure you guys will show me a very cool set of resources to explore. > > Best > Cl?udio Amaral > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Wed Jul 15 21:25:28 2015 From: vasdeveloper@REDACTED (Kannan) Date: Thu, 16 Jul 2015 00:55:28 +0530 Subject: [erlang-questions] CORS Supporting Erlang HTTP Server In-Reply-To: <55A6ABB6.60107@ninenines.eu> References: <3560FC65-A67A-42BC-9F42-F298EC84DAC3@gmail.com> <55A6ABB6.60107@ninenines.eu> Message-ID: Thanks for all the responses team. Just to put my requirements clear - * File upload/download * CORS - Better if this support can be configured. * Websockets * Dynamic web application * Performance * Embedding Which is the best? Cowboy/Yaws/Mochiweb On Thu, Jul 16, 2015 at 12:21 AM, Lo?c Hoguin wrote: > They have different goals and states. > > Cowboy is young and still sees fairly large changes (in part to accomodate > to newer protocols like HTTP/2 or Websocket compression, and in part > because there are still usability issues). However from version 2 onward > changes should be minimal for a while. > > Cowboy aims to support the core protocols for Web development, and nothing > more. This means HTTP 1 and 2, SPDY, Websocket, and so on. It also includes > a REST machine (adapted from Webmachine). The main focus is on REST and > Websocket first. > > It does not include anything related to formats like JSON or HTML, > enterprise protocols like RPC/SOAP/.. or anything related to XML, or Flash > and so on. > > There's a constant effort on keeping things small and tidy, and providing > functionality that covers 99% of users. The changes are user driven, with > unused features happily removed while new features get added after > detecting patterns in the many user questions I receive (but only if they > are in the scope, of course). > > Hope that helps. > > On 07/15/2015 08:04 PM, Kannan wrote: > >> Which one is the best? Yaws or Cowboy? I could not find any comparison >> matrix on the internet. >> >> Thanks, >> Theepan >> >> On Wed, Jul 15, 2015 at 4:52 AM, Dmitry Belyaev > > wrote: >> >> Not out of the box. There is >> https://github.com/danielwhite/cowboy_cors to do it. >> -- >> Best wishes, >> Dmitry Belyaev >> >> On 15 July 2015 7:25:40 AM AEST, Radoslaw Gruchalski >> > wrote: >> >> CORS is all based on headers. Cowboy can definitely do it. >> >> Sent from Outlook >> >> >> >> >> On Tue, Jul 14, 2015 at 1:58 PM -0700, "Kannan" >> > wrote: >> >> Hi Team, >> >> Is there a Erlang HTTP server, including Yaws, that supports >> CORS by configuration? >> >> Regards, >> Theepan >> >> >> ------------------------------------------------------------------------ >> >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Jul 15 21:28:05 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 15 Jul 2015 21:28:05 +0200 Subject: [erlang-questions] CORS Supporting Erlang HTTP Server In-Reply-To: References: <3560FC65-A67A-42BC-9F42-F298EC84DAC3@gmail.com> <55A6ABB6.60107@ninenines.eu> Message-ID: <55A6B445.20504@ninenines.eu> Cowboy with the cowboy_cors project that was linked to you fits all these requirements. On 07/15/2015 09:25 PM, Kannan wrote: > Thanks for all the responses team. Just to put my requirements clear - > > * File upload/download > * CORS - Better if this support can be configured. > * Websockets > * Dynamic web application > * Performance > * Embedding > > Which is the best? Cowboy/Yaws/Mochiweb > > > On Thu, Jul 16, 2015 at 12:21 AM, Lo?c Hoguin > wrote: > > They have different goals and states. > > Cowboy is young and still sees fairly large changes (in part to > accomodate to newer protocols like HTTP/2 or Websocket compression, > and in part because there are still usability issues). However from > version 2 onward changes should be minimal for a while. > > Cowboy aims to support the core protocols for Web development, and > nothing more. This means HTTP 1 and 2, SPDY, Websocket, and so on. > It also includes a REST machine (adapted from Webmachine). The main > focus is on REST and Websocket first. > > It does not include anything related to formats like JSON or HTML, > enterprise protocols like RPC/SOAP/.. or anything related to XML, or > Flash and so on. > > There's a constant effort on keeping things small and tidy, and > providing functionality that covers 99% of users. The changes are > user driven, with unused features happily removed while new features > get added after detecting patterns in the many user questions I > receive (but only if they are in the scope, of course). > > Hope that helps. > > On 07/15/2015 08:04 PM, Kannan wrote: > > Which one is the best? Yaws or Cowboy? I could not find any > comparison > matrix on the internet. > > Thanks, > Theepan > > On Wed, Jul 15, 2015 at 4:52 AM, Dmitry Belyaev > > >> wrote: > > Not out of the box. There is > https://github.com/danielwhite/cowboy_cors to do it. > -- > Best wishes, > Dmitry Belyaev > > On 15 July 2015 7:25:40 AM AEST, Radoslaw Gruchalski > > >> wrote: > > CORS is all based on headers. Cowboy can definitely do it. > > Sent from Outlook > > > > > On Tue, Jul 14, 2015 at 1:58 PM -0700, "Kannan" > > >> > wrote: > > Hi Team, > > Is there a Erlang HTTP server, including Yaws, that > supports > CORS by configuration? > > Regards, > Theepan > > > ------------------------------------------------------------------------ > > erlang-questions mailing list > erlang-questions@REDACTED > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From vasdeveloper@REDACTED Wed Jul 15 21:39:13 2015 From: vasdeveloper@REDACTED (Kannan) Date: Thu, 16 Jul 2015 01:09:13 +0530 Subject: [erlang-questions] CORS Supporting Erlang HTTP Server In-Reply-To: <55A6B445.20504@ninenines.eu> References: <3560FC65-A67A-42BC-9F42-F298EC84DAC3@gmail.com> <55A6ABB6.60107@ninenines.eu> <55A6B445.20504@ninenines.eu> Message-ID: Thanks Loic Hoguin, some where on the internet I found benchmark data. Cowboy had better data against its other Erlang counterparts. Since we are on a very tight timeline, would you recommend it for production with very high user traffic? Regards, Theepan On Thu, Jul 16, 2015 at 12:58 AM, Lo?c Hoguin wrote: > Cowboy with the cowboy_cors project that was linked to you fits all these > requirements. > > On 07/15/2015 09:25 PM, Kannan wrote: > >> Thanks for all the responses team. Just to put my requirements clear - >> >> * File upload/download >> * CORS - Better if this support can be configured. >> * Websockets >> * Dynamic web application >> * Performance >> * Embedding >> >> Which is the best? Cowboy/Yaws/Mochiweb >> >> >> On Thu, Jul 16, 2015 at 12:21 AM, Lo?c Hoguin > > wrote: >> >> They have different goals and states. >> >> Cowboy is young and still sees fairly large changes (in part to >> accomodate to newer protocols like HTTP/2 or Websocket compression, >> and in part because there are still usability issues). However from >> version 2 onward changes should be minimal for a while. >> >> Cowboy aims to support the core protocols for Web development, and >> nothing more. This means HTTP 1 and 2, SPDY, Websocket, and so on. >> It also includes a REST machine (adapted from Webmachine). The main >> focus is on REST and Websocket first. >> >> It does not include anything related to formats like JSON or HTML, >> enterprise protocols like RPC/SOAP/.. or anything related to XML, or >> Flash and so on. >> >> There's a constant effort on keeping things small and tidy, and >> providing functionality that covers 99% of users. The changes are >> user driven, with unused features happily removed while new features >> get added after detecting patterns in the many user questions I >> receive (but only if they are in the scope, of course). >> >> Hope that helps. >> >> On 07/15/2015 08:04 PM, Kannan wrote: >> >> Which one is the best? Yaws or Cowboy? I could not find any >> comparison >> matrix on the internet. >> >> Thanks, >> Theepan >> >> On Wed, Jul 15, 2015 at 4:52 AM, Dmitry Belyaev >> >> >> wrote: >> >> Not out of the box. There is >> https://github.com/danielwhite/cowboy_cors to do it. >> -- >> Best wishes, >> Dmitry Belyaev >> >> On 15 July 2015 7:25:40 AM AEST, Radoslaw Gruchalski >> >> >> >> wrote: >> >> CORS is all based on headers. Cowboy can definitely do >> it. >> >> Sent from Outlook >> >> >> >> >> On Tue, Jul 14, 2015 at 1:58 PM -0700, "Kannan" >> >> >> >> wrote: >> >> Hi Team, >> >> Is there a Erlang HTTP server, including Yaws, that >> supports >> CORS by configuration? >> >> Regards, >> Theepan >> >> >> >> ------------------------------------------------------------------------ >> >> erlang-questions mailing list >> erlang-questions@REDACTED >> >> > > >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> -- >> Lo?c Hoguin >> http://ninenines.eu >> Author of The Erlanger Playbook, >> A book about software development using Erlang >> >> >> > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Jul 15 21:41:48 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 15 Jul 2015 21:41:48 +0200 Subject: [erlang-questions] CORS Supporting Erlang HTTP Server In-Reply-To: References: <3560FC65-A67A-42BC-9F42-F298EC84DAC3@gmail.com> <55A6ABB6.60107@ninenines.eu> <55A6B445.20504@ninenines.eu> Message-ID: <55A6B77C.90301@ninenines.eu> All I can say is that it works very well for users who use it under those conditions. But I suspect the same would be true for most Erlang servers. On 07/15/2015 09:39 PM, Kannan wrote: > Thanks Loic Hoguin, some where on the internet I found benchmark data. > Cowboy had better data against its other Erlang counterparts. > > Since we are on a very tight timeline, would you recommend it for > production with very high user traffic? > > Regards, > Theepan > > On Thu, Jul 16, 2015 at 12:58 AM, Lo?c Hoguin > wrote: > > Cowboy with the cowboy_cors project that was linked to you fits all > these requirements. > > On 07/15/2015 09:25 PM, Kannan wrote: > > Thanks for all the responses team. Just to put my requirements > clear - > > * File upload/download > * CORS - Better if this support can be configured. > * Websockets > * Dynamic web application > * Performance > * Embedding > > Which is the best? Cowboy/Yaws/Mochiweb > > > On Thu, Jul 16, 2015 at 12:21 AM, Lo?c Hoguin > > >> wrote: > > They have different goals and states. > > Cowboy is young and still sees fairly large changes (in part to > accomodate to newer protocols like HTTP/2 or Websocket > compression, > and in part because there are still usability issues). > However from > version 2 onward changes should be minimal for a while. > > Cowboy aims to support the core protocols for Web > development, and > nothing more. This means HTTP 1 and 2, SPDY, Websocket, and > so on. > It also includes a REST machine (adapted from Webmachine). > The main > focus is on REST and Websocket first. > > It does not include anything related to formats like JSON > or HTML, > enterprise protocols like RPC/SOAP/.. or anything related > to XML, or > Flash and so on. > > There's a constant effort on keeping things small and tidy, and > providing functionality that covers 99% of users. The > changes are > user driven, with unused features happily removed while new > features > get added after detecting patterns in the many user questions I > receive (but only if they are in the scope, of course). > > Hope that helps. > > On 07/15/2015 08:04 PM, Kannan wrote: > > Which one is the best? Yaws or Cowboy? I could not find any > comparison > matrix on the internet. > > Thanks, > Theepan > > On Wed, Jul 15, 2015 at 4:52 AM, Dmitry Belyaev > > > > >>> wrote: > > Not out of the box. There is > https://github.com/danielwhite/cowboy_cors to do it. > -- > Best wishes, > Dmitry Belyaev > > On 15 July 2015 7:25:40 AM AEST, Radoslaw Gruchalski > > > >>> wrote: > > CORS is all based on headers. Cowboy can > definitely do it. > > Sent from Outlook > > > > > On Tue, Jul 14, 2015 at 1:58 PM -0700, "Kannan" > > > >>> > wrote: > > Hi Team, > > Is there a Erlang HTTP server, including > Yaws, that > supports > CORS by configuration? > > Regards, > Theepan > > > > ------------------------------------------------------------------------ > > erlang-questions mailing list > erlang-questions@REDACTED > > > > >> > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > > > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From josh.rubyist@REDACTED Wed Jul 15 22:13:52 2015 From: josh.rubyist@REDACTED (Josh Adams) Date: Wed, 15 Jul 2015 15:13:52 -0500 Subject: [erlang-questions] Erlang and Rules Engines / Production Systems Message-ID: Hi all, I'm in need of making a recommendation regarding a Business Rules Engine and am evaluating various open source and commercial projects for a customer. In the meantime, given the nature of this particular need, I'm realizing that it's possible I could build a DSL for generating an AST and pattern matching against it. I'm operating on the assumption that there are potentially 10s of thousands of rules in the system at scale, rather than hundreds or millions. My personal goals are to be able to sustainably handle a decent amount of traffic via a webservice that can resolve all of these rules for a given input in around 2ms. This is not a hard requirement, but it is the performance that the system needs. 10s of ms would be acceptable. 100s would be out of the question. I've not got anything more than tangential / play-style experience with any BREs. At present I am evaluating Drools, which is Java-based and uses the Rete algorithm [1]. I am not yet sure whether it will meet my performance "requirements". On the Erlang front, I've seen at least ERESEYE[2]/sereseye[3] which are RETE+CLIPS and RETE-based versions. I've not dug into them yet at all. I was curious if anyone on the list has experience with this in Erlang and had any pointers or real-life anecdotes regarding building such a system. It needn't be actually RETE-based, and in fact I would expect the newer algorithms to provide better performance. Anyone have anything to share with someone furiously filling their mind with this stuff at present? Thanks in advance, -Josh Adams [1] Apparently actually something called RETE-OO - http://ieeexplore.ieee.org/xpl/login.jsp?reload=true&tp=&arnumber=5551128&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D5551128 [2] https://www.erlang-solutions.com/sites/default/files/trimedia/eresye_0.pdf [3] https://github.com/afiniate/seresye -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Wed Jul 15 22:57:27 2015 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Wed, 15 Jul 2015 22:57:27 +0200 Subject: [erlang-questions] Pipe Operator in Erlang? In-Reply-To: References: <20150709231805.GA1339@ferdmbp.local> Message-ID: <55A6C937.5090700@wanadoo.fr> Hi Robert, thanks for your thoughts. mine inline below. > > > It does that because the gen_server:call wraps your message with a > tagged tuple containing for the sender pid and a unique ref. It then > goes into a receive waiting for a message containing that unique ref. > The receiving end understands this when it sends it reply it includes > the unique ref so the sender will receive the right message. > > The reason for explicitly doing all this work is that all the receiving > process sees is exactly the message that was sent, no more no less. > There is no implicit information included at all. This means that if you > were to add a synchronous message then you would have to either add a > new message handling, not just in the message itself but maybe in > receive as well. Or you could say that one of these messages will have a > specific format which you can receive and you must reply with a specific > format. Yes I know this. It is probably far more complicated than async messages. But even if Erlang power is in async messages, and if possible async messaging must be used, there is always 10% of case where sync message is necessary. Erlang would be a real killer language if sync messaging was as async level. I encounter the problem when adding the process state view in observer. Apart : BTW , State format is not standardized, well I mean not common to all kind of processes . /Apart. Some , well, most, processes do not even handle system messages . And as far I know there is no way to know it in advance, or to "ask" it to the process. The lonely way (?) is to send a system message, and wait an answer (or not) with a timeout. Result, you have to wait a timeout in most case. processes might, MHO, always answer to system messages, at least to tell that system messages cannot be handled, because no sys callback are available. On contrary, if the behavior allow it, the answer to a special system message may indicate if sending a sync message is possible. This way, it would be possible to know if a process can handle a sync request, obviously a cache must probably be used in order not to ask each time if a process can handle a sync message. > > Another problem is that synchronous message passing is much more complex > and you need to define/handle many more things in the system than with > async messages. For example how to handle you not getting a reply, do > you use timeouts, or check if the other process has died, or a > combination of both, etc? So you either end up with a set of special > case operators/calls or a call with lots of options. Either the process do not handle sync message (see my upper comment), and an error or exception must be raised immediately, or the process can, and the default sys module 5000 ms timeout apply to the !! (or whatever) operator. If a different timeout is needed, a VM flag can be created. And for any other case, old style ! + receive + after or plain gen_server:call for gen_servers are still possible. > > And how should it work with distribution where today it is the same. Version of Erlang VM must be verified in order to know if process on a node can answer to system messages. If no, the sync call must return immediately an error or an exception. > > No, I didn't like Joe's suggestion. :-) Yes I understood this :>) ... Regards From g@REDACTED Wed Jul 15 23:01:32 2015 From: g@REDACTED (Garrett Smith) Date: Wed, 15 Jul 2015 16:01:32 -0500 Subject: [erlang-questions] Erlang and Rules Engines / Production Systems In-Reply-To: References: Message-ID: I've used Erlang for quite a few rules engines - but they're all purpose built and not intended to be a generalized solution. That said, purpose built rules engines are super simple to build Erlang. If someone approached me with a problem like "we need an rules engine" I'd certainly point them to the Java space. That's full of management pleasing products that promise strategic value. Drools of course is the established player for open source. Before I started looking around for another job, I'd push back and ask the powers what problem do they actually have - the one that makes them think they need a rules engine. There's something that's real and not abstract. Use Erlang to tackle that directly. Just hard code the rules for the time being - it's easy to move them outside the program as config - and when people complain that they can't edit the cryptic Erlang term syntax, introduce a rules compile phase that takes Markdown defined rules - or better, Yaml - and generates the Erlang config. When the next real problem manifests, do that again. Over time you might start to see some patterns that you can generalize. You may run into performance problems that drive you to use esoteric data structures and algorithms. Or maybe you can get by with ridiculously naive, unclever code. I've found this approach leads to effective software that's very small for any given problem. I really try hard to avoid the need for something like a "rules engine" or a "business process enactment engine'. If a business user says, "hey this is great, but we really need to be able edit these rules ourselves and push to production without IT interference" - try one thing before looking for another job... assure them that they will get this capability, but the organization will get there incrementally rather than in one step. If they say, "thanks but we just got back from Garter BPM Summit and we need this yesterday" then use Drools, or iLog, or Pega. You don't want Erlang. The promise of Erlang is production awesomeness. If you haven't put an Erlang app into production and put it through a few maintenance phases, it's hard to easily communicate how much better it is - in terms of speed of implementation, service availability, and speed of change. But part of the Tao of Erlang is directness of code - it's harder than a lot of languages to create big layered abstractions. There are a lot of established rules engine/BPM products laying around and if that's what you really want, just pick something that's established and skip Erlang for this one. On Wed, Jul 15, 2015 at 3:13 PM, Josh Adams wrote: > Hi all, > > I'm in need of making a recommendation regarding a Business Rules Engine and > am evaluating various open source and commercial projects for a customer. > In the meantime, given the nature of this particular need, I'm realizing > that it's possible I could build a DSL for generating an AST and pattern > matching against it. > > I'm operating on the assumption that there are potentially 10s of thousands > of rules in the system at scale, rather than hundreds or millions. My > personal goals are to be able to sustainably handle a decent amount of > traffic via a webservice that can resolve all of these rules for a given > input in around 2ms. This is not a hard requirement, but it is the > performance that the system needs. 10s of ms would be acceptable. 100s > would be out of the question. > > I've not got anything more than tangential / play-style experience with any > BREs. At present I am evaluating Drools, which is Java-based and uses the > Rete algorithm [1]. I am not yet sure whether it will meet my performance > "requirements". > > On the Erlang front, I've seen at least ERESEYE[2]/sereseye[3] which are > RETE+CLIPS and RETE-based versions. I've not dug into them yet at all. > > I was curious if anyone on the list has experience with this in Erlang and > had any pointers or real-life anecdotes regarding building such a system. > It needn't be actually RETE-based, and in fact I would expect the newer > algorithms to provide better performance. Anyone have anything to share > with someone furiously filling their mind with this stuff at present? > > Thanks in advance, > > -Josh Adams > > [1] Apparently actually something called RETE-OO - > http://ieeexplore.ieee.org/xpl/login.jsp?reload=true&tp=&arnumber=5551128&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D5551128 > [2] > https://www.erlang-solutions.com/sites/default/files/trimedia/eresye_0.pdf > [3] https://github.com/afiniate/seresye > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From garret.smith@REDACTED Wed Jul 15 23:41:19 2015 From: garret.smith@REDACTED (Garret Smith) Date: Wed, 15 Jul 2015 14:41:19 -0700 Subject: [erlang-questions] JSON Parser In-Reply-To: References: <55A6A068.4020703@research.att.com> Message-ID: I've had a similar need, to map JSON to Erlang records. As Bob said, this should be a layer above the JSON parsing. aeon is a library that does this (JSON -> erlang terms -> erlang records) by making the record type information available at runtime with a parse transform, then using it to direct the translation from Erlang terms (as produced by jsx) to an Erlang record. It's light on documentation. If you think records are the way to go vs maps (I like that I can ensure fields are present and rely on their type) I'd be happy to add some examples. aeon - https://github.com/garret-smith/aeon.git -Garret On Wed, Jul 15, 2015 at 11:28 AM, Kannan wrote: > I was out of Erlang programing for the last five years, I think maps has > been introduced during that time. I was not aware of it earlier. Just went > through the documentation, and yes it is the best format to match JSONs. > > The syntax is quite complicated though. Using two characters to map key and > value, assignment, update, access and matching have different K-V separator > symbols etc. However, as a declaration free K-V grouping structure, it is > the best option. > > Regards, > Theepan > > On Wed, Jul 15, 2015 at 11:33 PM, Garry Hodgson > wrote: >> >> I prefer mapping json objects to erlang maps, lists to lists, and strings >> to binaries. i use a wrapper around mochijson to do that. >> >> >> On 7/15/15 2:00 PM, Kannan wrote: >> >> Is any of them supporting Erlang 'record' as their base for >> encoding/decoding. I see many of them are doing it with just list of tuples >> of binaries.Erlang records best match the structure of JSON format. >> >> JSON >> ---------- >> {"name": "Theepan", >> "work": "Coding", >> "salary": "0" >> } >> >> Matching Erlang record >> ----------------------------------- >> -record( json_record, >> { >> 'name' = "Theepan", >> 'work' = "Coding", >> 'salary' = "0" >> } >> } >> >> Thanks, >> Theepan >> >> On Wed, Jul 15, 2015 at 2:33 PM, Jesper Louis Andersen >> wrote: >>> >>> >>> On Tue, Jul 14, 2015 at 10:06 PM, Kannan wrote: >>>> >>>> I come across many JSON libraries. Once from MochiWeb, Other one from >>>> Yaws. Third one from CouchDB. And some others through Googling. >>> >>> >>> There are two very popular JSON parsers in Erlang: jsx and jiffy. >>> >>> jsx is written in plain Erlang. It is fast, correct and since it is >>> written in Erlang, it will also automatically yield for other processes in >>> the system. >>> >>> jiffy is written as a C NIF. It is about 10 times faster than jsx, but >>> the caveat is everything that has to do with C NIFs: blocking a scheduler, C >>> code having errors, security considerations, etc. >>> >>> I tend to run with `jsx` in my projects, and then I switch away from JSON >>> when it gets to slow. JSON is a bad format that should never have existed in >>> the first place. We are stuck with it because it's historic alternative, >>> XML, was far worse in every aspect. >>> >>> >>> -- >>> J. >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From josh.rubyist@REDACTED Wed Jul 15 23:43:34 2015 From: josh.rubyist@REDACTED (Josh Adams) Date: Wed, 15 Jul 2015 16:43:34 -0500 Subject: [erlang-questions] Erlang and Rules Engines / Production Systems In-Reply-To: References: Message-ID: On Wed, Jul 15, 2015 at 4:01 PM, Garrett Smith wrote: > I've used Erlang for quite a few rules engines - but they're all > purpose built and not intended to be a generalized solution. That > said, purpose built rules engines are super simple to build Erlang. > This was my feeling as soon as I started thinking of the problem, of course. Had some hope there was a generalized one, but nbd. Before I started looking around for another job I was so happy when I saw you replied, specifically because I knew I'd get a laugh out of it. They do have both very specific things they want to do with it now (where it totally does in fact make sense to have a non-programmer expert in their domain enter rules in a manageable format). Also starting out needing the ability for their domain expert to run tests easily is a big win. So I think I will end up going with Drools. Bright side - I have no need to maintain this, at all, so it's a very short-term involvement with Java from my POV, and dealing with a BRE at scale without maintenance means I'll get to learn fun stuff without the pain, I hope. I've put a decent number of very trivial Elixir systems into production. None too complicated. The problem I had with the most interesting one of them was that once we launched it it ran for its entire lifetime (9 months) without ever needing to change, so I didn't get to experience that bit. Then the company pivoted and it was either decommissioned or is still running in AWS pointlessly, subscribing to rabbitmq queues and doing jack-squat with the messages. -Josh They -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Thu Jul 16 01:02:42 2015 From: lloyd@REDACTED (lloyd@REDACTED) Date: Wed, 15 Jul 2015 19:02:42 -0400 (EDT) Subject: [erlang-questions] Exporting a record type In-Reply-To: References: <1436486708.04388403@apps.rackspace.com> <330A96FE-A329-4D8E-8DAF-8DABF86716BB@writersglen.com> <9A55AD73-6BE0-4FD7-9372-874E47826F2B@basho.com> <1436544678.40631215@apps.rackspace.com> <927D610A-A519-4106-9D91-F53B5EC83485@writersglen.com> Message-ID: <1437001362.18511345@apps.rackspace.com> Hi Jesper, Well, I finally found time to try out the code you proposed. It is so cool! I dawned on me that I could do: repr_rel(#book { isbn = I } = Book, isbn) -> {I, Book}; repr_rel(#book { title = T } = Book, title) -> {T, Book}; repr_rel(#book { author = A } = Book, author) -> {A, Book}; repr_rel(#book { publisher = P } = Book, publisher) -> {P, Book}; repr_rel(#book { pub_date = P } = Book, pub_date) -> {P, Book}; repr_rel(#book { genre = G } = Book, genre) -> {G, Book}. And thanks for introducing the sofs library. I've often wondered how I might use it.Your code opens the door. Three quick questions: 1) I'm guessing repr is your abbreviation for replica and rel for relation. Is this correct? 2) I've been trying to figure out a similarly compact way to set values, but can't without explicitly referencing the book record structure. Any suggestions? 3) Pardon my thick skull, but the book/0 declaration in the -opaque and -export_type attributes still puzzles me. Can I use book() in external modules? But without a parameter for a book instance, just exactly how would it be used? Much much appreciate your help. Lloyd -----Original Message----- From: "Jesper Louis Andersen" Sent: Monday, July 13, 2015 6:03am To: "Lloyd R. Prentice" Cc: "Gordon Guthrie" , "erlang-questions" Subject: Re: [erlang-questions] Exporting a record type On Sun, Jul 12, 2015 at 5:06 PM, Lloyd R. Prentice wrote: > Also, in your repr/2 code (which is very suggestive of neat things one can > do) what is the significance of the View variable? As I see it now, it's > simply a tag like thumbnail_overview, but I'm not comfortable that my > understanding is correct. The idea, at least, is this: a "view" of some data, book, record, ..., is a way to cast that data into another structure by transforming it. I.e., "view" the data in another form. In statically typed languages, some languages support built-in view-types, so you can define these transformations formally, but in Erlang you have to make a more dynamic resolution. In its simple form, the view is just an atom(), requesting the view. But in a more advanced form, the view is a language which can be executed to construct a view of the data: ... repr(Book, {relation, R}) -> repr_rel(Book, R). repr_rel(#book { genre = G } = Book, genre) -> {G, Book}. Now, suppose you have a list of books, Bs: Rel = [repr(B, {relation, genre}) || B <- Bs], R = sofs:relation(Rel), F = sofs:relation_to_family(R), sofs:to_external(F). If for instance you have books [#book { genre = fantasy } = A, #book { genre = fantasy } = B, #book { genre = scifi } = C], Then the above would return [{fantasy, [A, B]}, {scifi, [C]}]. There are numerous tricks that can be played here, depending on what transformations you need. -- J. From be.dmitry@REDACTED Thu Jul 16 01:37:27 2015 From: be.dmitry@REDACTED (Dmitry Belyaev) Date: Thu, 16 Jul 2015 09:37:27 +1000 Subject: [erlang-questions] Exporting a record type In-Reply-To: <1437001362.18511345@apps.rackspace.com> References: <1436486708.04388403@apps.rackspace.com> <330A96FE-A329-4D8E-8DAF-8DABF86716BB@writersglen.com> <9A55AD73-6BE0-4FD7-9372-874E47826F2B@basho.com> <1436544678.40631215@apps.rackspace.com> <927D610A-A519-4106-9D91-F53B5EC83485@writersglen.com> <1437001362.18511345@apps.rackspace.com> Message-ID: <4C1C4CDF-474B-4143-B8BD-248FDF906960@gmail.com> -- Best wishes, Dmitry Belyaev On 16 July 2015 9:02:42 AM AEST, lloyd@REDACTED wrote: > >1) I'm guessing repr is your abbreviation for replica and rel for >relation. Is this correct? I think repr means "represent", so the result is a relation representation. > >3) Pardon my thick skull, but the book/0 declaration in the -opaque and >-export_type attributes still puzzles me. Can I use book() in external >modules? But without a parameter for a book instance, just exactly how >would it be used? > Opaque means a value of the type must not be deconstructed in any other module. It only makes sense if you export it using "-export_type". You can freely use this exported type in any other module. If you try to peek inside such a value dialyzer will complain. So in another module you'd write: -spec author_name(book:book()) -> binary(). do(Book) -> Author = book:author(Book), book_author:name(Author). >Much much appreciate your help. > >Lloyd > > > >-----Original Message----- >From: "Jesper Louis Andersen" >Sent: Monday, July 13, 2015 6:03am >To: "Lloyd R. Prentice" >Cc: "Gordon Guthrie" , "erlang-questions" > >Subject: Re: [erlang-questions] Exporting a record type > >On Sun, Jul 12, 2015 at 5:06 PM, Lloyd R. Prentice > >wrote: > >> Also, in your repr/2 code (which is very suggestive of neat things >one can >> do) what is the significance of the View variable? As I see it now, >it's >> simply a tag like thumbnail_overview, but I'm not comfortable that my >> understanding is correct. > > >The idea, at least, is this: a "view" of some data, book, record, ..., >is a >way to cast that data into another structure by transforming it. I.e., >"view" the data in another form. In statically typed languages, some >languages support built-in view-types, so you can define these >transformations formally, but in Erlang you have to make a more dynamic >resolution. > >In its simple form, the view is just an atom(), requesting the view. >But in >a more advanced form, the view is a language which can be executed to >construct a view of the data: > >... >repr(Book, {relation, R}) -> > repr_rel(Book, R). > >repr_rel(#book { genre = G } = Book, genre) -> > {G, Book}. > >Now, suppose you have a list of books, Bs: > >Rel = [repr(B, {relation, genre}) || B <- Bs], >R = sofs:relation(Rel), >F = sofs:relation_to_family(R), >sofs:to_external(F). > >If for instance you have books [#book { genre = fantasy } = A, #book { >genre = fantasy } = B, #book { genre = scifi } = C], > >Then the above would return > >[{fantasy, [A, B]}, {scifi, [C]}]. > >There are numerous tricks that can be played here, depending on what >transformations you need. From rvirding@REDACTED Thu Jul 16 02:49:21 2015 From: rvirding@REDACTED (Robert Virding) Date: Thu, 16 Jul 2015 02:49:21 +0200 Subject: [erlang-questions] Erlang and Rules Engines / Production Systems In-Reply-To: References: Message-ID: Another alternative is a rules engine in prolog which might be an alternative to ereseye. Prolog is good with rules.:-) You can always access an prolog system using ports or c-nodes, or use erlog for a smaller internal prolog engine. There is no real problems in hiding the syntax with a DSL. Robert On 15 July 2015 at 23:43, Josh Adams wrote: > On Wed, Jul 15, 2015 at 4:01 PM, Garrett Smith wrote: > >> I've used Erlang for quite a few rules engines - but they're all >> purpose built and not intended to be a generalized solution. That >> said, purpose built rules engines are super simple to build Erlang. >> > > This was my feeling as soon as I started thinking of the problem, of > course. Had some hope there was a generalized one, but nbd. > > Before I started looking around for another job > > > I was so happy when I saw you replied, specifically because I knew I'd get > a laugh out of it. > > They do have both very specific things they want to do with it now (where > it totally does in fact make sense to have a non-programmer expert in their > domain enter rules in a manageable format). Also starting out needing the > ability for their domain expert to run tests easily is a big win. So I > think I will end up going with Drools. Bright side - I have no need to > maintain this, at all, so it's a very short-term involvement with Java from > my POV, and dealing with a BRE at scale without maintenance means I'll get > to learn fun stuff without the pain, I hope. > > I've put a decent number of very trivial Elixir systems into production. > None too complicated. The problem I had with the most interesting one of > them was that once we launched it it ran for its entire lifetime (9 months) > without ever needing to change, so I didn't get to experience that bit. > Then the company pivoted and it was either decommissioned or is still > running in AWS pointlessly, subscribing to rabbitmq queues and doing > jack-squat with the messages. > > -Josh > > They > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.fitzpatrick6012@REDACTED Thu Jul 16 00:07:16 2015 From: matthew.fitzpatrick6012@REDACTED (Matthew Fitzpatrick) Date: Wed, 15 Jul 2015 17:07:16 -0500 Subject: [erlang-questions] JSON Parser In-Reply-To: References: <55A6A068.4020703@research.att.com> Message-ID: I'm in a project that's stuck back on a very old version of Erlang. We don't have the capability of using maps so we use records as our "Erlang" model. I'm not sure if you're stuck using records. If you can use maps, I think that fits best. If you HAVE to use records though, We've had good luck utilizing jiffy along with an additional translation layer in our code. Your translation layer can be responsible for handling the transformation to and from EJSON. The key component is using `record_info(fields, _)` which will allow you to get your record keys at run-time. I've often done this with a macro: ```erlang -record(my_record, {id}). -define(RECORD_KEYS, record_info(fields, my_record)) %% ?RECORD_KEYS = [id]. ``` After you've got your keys, you should be able to do all sorts of shenanigans to make the mapping for your intermediary layer. Fair warning though, the intermediary layer starts to get a little weird if you use records that contain other records or if you want to make it generic. If you decide to go that route and have other questions feel free to hit me up off list. First mailing list reply btw, if i did something wrong/incorrect, just email me off list about it and I'll fix it for next time. On Wed, Jul 15, 2015 at 1:28 PM, Kannan wrote: > I was out of Erlang programing for the last five years, I think maps has > been introduced during that time. I was not aware of it earlier. Just went > through the documentation, and yes it is the best format to match JSONs. > > The syntax is quite complicated though. Using two characters to map key > and value, assignment, update, access and matching have different K-V > separator symbols etc. However, as a declaration free K-V grouping > structure, it is the best option. > > Regards, > Theepan > > On Wed, Jul 15, 2015 at 11:33 PM, Garry Hodgson > wrote: > >> I prefer mapping json objects to erlang maps, lists to lists, and >> strings to binaries. i use a wrapper around mochijson to do that. >> >> >> On 7/15/15 2:00 PM, Kannan wrote: >> >> Is any of them supporting Erlang 'record' as their base for >> encoding/decoding. I see many of them are doing it with just list of tuples >> of binaries.Erlang records best match the structure of JSON format. >> >> JSON >> ---------- >> {"name": "Theepan", >> "work": "Coding", >> "salary": "0" >> } >> >> Matching Erlang record >> ----------------------------------- >> -record( json_record, >> { >> 'name' = "Theepan", >> 'work' = "Coding", >> 'salary' = "0" >> } >> } >> >> Thanks, >> Theepan >> >> On Wed, Jul 15, 2015 at 2:33 PM, Jesper Louis Andersen < >> jesper.louis.andersen@REDACTED> wrote: >> >>> >>> On Tue, Jul 14, 2015 at 10:06 PM, Kannan wrote: >>> >>>> I come across many JSON libraries. Once from MochiWeb, Other one from >>>> Yaws. Third one from CouchDB. And some others through Googling. >>>> >>> >>> There are two very popular JSON parsers in Erlang: jsx and jiffy. >>> >>> jsx is written in plain Erlang. It is fast, correct and since it is >>> written in Erlang, it will also automatically yield for other processes in >>> the system. >>> >>> jiffy is written as a C NIF. It is about 10 times faster than jsx, but >>> the caveat is everything that has to do with C NIFs: blocking a scheduler, >>> C code having errors, security considerations, etc. >>> >>> I tend to run with `jsx` in my projects, and then I switch away from >>> JSON when it gets to slow. JSON is a bad format that should never have >>> existed in the first place. We are stuck with it because it's historic >>> alternative, XML, was far worse in every aspect. >>> >>> >>> -- >>> J. >>> >> >> >> >> _______________________________________________ >> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sid5@REDACTED Thu Jul 16 03:59:33 2015 From: sid5@REDACTED (Sid Muller) Date: Thu, 16 Jul 2015 03:59:33 +0200 Subject: [erlang-questions] shell crash on exit debug Message-ID: An HTML attachment was scrubbed... URL: From rtrlists@REDACTED Thu Jul 16 07:56:55 2015 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 16 Jul 2015 06:56:55 +0100 Subject: [erlang-questions] shell crash on exit debug In-Reply-To: References: Message-ID: Hi Sid, I think if you turn on sasl logs, you'll get more info. Either by starting your Erlang shell with sasl enabled (can't remember the magic switch offhand, sorry) or by starting it explicitly in the shell using application:start(sasl). Hth, Robby On Jul 16, 2015 5:50 AM, "Sid Muller" wrote: > Hi, > > does anyone have any pointers about debugging the shell crash? Recently my > shell started crashing after I type q(). in the shell: > > 4> q(). > ok > ** exception exit: shutdown > 5> > > It's not very verbose but I'd like to fix what ever the issue is, I start > the shell run my application and then stop it and then exit. It used to > work fine but recently as I added some dets code it has been crashing on > exit. > > I don't have any problems in the shell if I start, run, and stop and > start, run and stop my application. Only after I try to exit the shell it > crashes. I am closing the dets tables so I don't know what's causing it. It > would be helpfull if the shell crash was actually giving out more > information. > > Is there some kind of startup switch that I could pass to the shell to > give more information about the crash? > > Thank you! > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Thu Jul 16 08:17:46 2015 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 15 Jul 2015 23:17:46 -0700 Subject: [erlang-questions] JSON Parser In-Reply-To: References: Message-ID: <55A74C8A.9040906@gmail.com> On 07/15/2015 11:24 AM, Bob Ippolito wrote: > Records are a compile time construct, so in order to parse as records you'd need to define the records in advance and provide a schema and record definition to the parser for the document. This could be done in a layer above the JSON parser. > One approach which is possible for the record approach, is to use the parse transform https://github.com/okeuday/record_info_runtime/ with a function the parse transform adds: record_info_fieldtypes/1. So, then the Erlang type spec information is accessible along with the fields based on the record name, to determine any details for encoding into JSON or decoding from JSON. The nice part of that approach is then that the type spec information allows validation to catch problems, if there are any, like with nested record structures. Not sure about the best way to encapsulate that approach, but I know it has worked in the past. > > Maps are the data structure you're looking for. Most of the JSON implementations provide backwards compatibility to versions of Erlang before maps, so other data structures (such as dicts or proplists) are more common. > > On Wednesday, July 15, 2015, Kannan > wrote: > > Is any of them supporting Erlang 'record' as their base for encoding/decoding. I see many of them are doing it with just list of tuples of binaries.Erlang records best match the structure of JSON format. > > JSON > ---------- > {"name": "Theepan", > "work": "Coding", > "salary": "0" > } > > Matching Erlang record > ----------------------------------- > -record( json_record, > { > 'name' = "Theepan", > 'work' = "Coding", > 'salary' = "0" > } > } > > Thanks, > Theepan > > On Wed, Jul 15, 2015 at 2:33 PM, Jesper Louis Andersen > wrote: > > > On Tue, Jul 14, 2015 at 10:06 PM, Kannan > wrote: > > I come across many JSON libraries. Once from MochiWeb, Other one from Yaws. Third one from CouchDB. And some others through Googling. > > > There are two very popular JSON parsers in Erlang: jsx and jiffy. > > jsx is written in plain Erlang. It is fast, correct and since it is written in Erlang, it will also automatically yield for other processes in the system. > > jiffy is written as a C NIF. It is about 10 times faster than jsx, but the caveat is everything that has to do with C NIFs: blocking a scheduler, C code having errors, security considerations, etc. > > I tend to run with `jsx` in my projects, and then I switch away from JSON when it gets to slow. JSON is a bad format that should never have existed in the first place. We are stuck with it because it's historic alternative, XML, was far worse in every aspect. > > > -- > J. > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Jul 16 10:24:56 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 16 Jul 2015 10:24:56 +0200 Subject: [erlang-questions] Exporting a record type In-Reply-To: <1437001362.18511345@apps.rackspace.com> References: <1436486708.04388403@apps.rackspace.com> <330A96FE-A329-4D8E-8DAF-8DABF86716BB@writersglen.com> <9A55AD73-6BE0-4FD7-9372-874E47826F2B@basho.com> <1436544678.40631215@apps.rackspace.com> <927D610A-A519-4106-9D91-F53B5EC83485@writersglen.com> <1437001362.18511345@apps.rackspace.com> Message-ID: On Thu, Jul 16, 2015 at 1:02 AM, wrote: > 1) I'm guessing repr is your abbreviation for replica and rel for > relation. Is this correct? > Dmitry is right on both accounts here. `repr` is a shorthand for representation, i.e., how to represent the data. 3) Pardon my thick skull, but the book/0 declaration in the -opaque and > -export_type attributes still puzzles me. Can I use book() in external > modules? But without a parameter for a book instance, just exactly how > would it be used? > In languages with static types, you eventually have to make a choice for a type 'book'. If we export this type and use it in other modules, are these other modules allowed to the see the structure of a book and work with that structure or are they not? The former is called a transparent ascription and the latter is called an opaque ascription. The reason you may want to use opaque ascriptions is that you can change the internal representation. Say you have defined type 'a queue = 'a list let empty = [] let push x q = List.append q [x] let pop q = match q with | [] -> None | (x :: q) -> Some (x, q) In something like OCaml. Now, if another module abuses the fact that a queue is a list, then you can't change the implementation later on. So you mark the queue as opaque which means any user of the queue *has* to use the functions empty/push/pop defined in the module. We can thus later change the internal representation to, say, type 'a queue = { front : 'a list ; back : 'a list } and change the functions accordingly, without breaking any other user of the module as we forced them into a mode where they cannot inspect the internal representation of the queue. The -opaque annotation serves the same purpose in Erlang, and this is also, what Dmitry said :) -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alttagil@REDACTED Thu Jul 16 11:16:52 2015 From: alttagil@REDACTED (Alex Hudich) Date: Thu, 16 Jul 2015 12:16:52 +0300 Subject: [erlang-questions] bad certificate if trying to verify StartSsl certificate Message-ID: Hi! wget http://curl.haxx.se/ca/cacert.pem and then ssl:connect( "www.nicemine.ru", 443, [{verify,verify_peer},{server_name_indication,"www.nicemine.ru"},{depth,2},{cacertfile,"cacert.pem"}] ). gives me {error,{tls_alert,"bad certificate"}} Why? Site can be opened ok in the browser. Erlang/OTP 17 [erts-6.3] From carlsson.richard@REDACTED Thu Jul 16 14:20:58 2015 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 16 Jul 2015 14:20:58 +0200 Subject: [erlang-questions] erl_sytanx help In-Reply-To: <12A559D6-C2EB-462C-9FB7-C7ACA9AB398D@gmail.com> References: <12A559D6-C2EB-462C-9FB7-C7ACA9AB398D@gmail.com> Message-ID: There should in most cases be no overhead using merl compared to using erl_syntax directly. In particular, whatever code you _generate_ using merl will be no different from what you would construct through calls to erl_syntax. If there is any overhead, it will be during code generation. You can inspect the Erlang code that merl creates by using erlc +"'P'" (note the extra quotes) or calling compile:file(foo, ['P']). /Richard On Wed, Jul 15, 2015 at 12:23 AM, Humberto Rodriguez < humbert.rguez@REDACTED> wrote: > Richard, thanks for the example! > > I have another question: How much is the penalty (performance) of use this > way? I am thinking use erlang as base language to make a DSL on top of Riak > KV. > > Do you have some advice for me? What is the way that you recommend me, > merl or erl_syntax? > > Best regards, > --- Humberto > > > > On 15 Jul 2015, at 00:02, Richard Carlsson > wrote: > > This is a good opportunity to demonstrate the function erl_syntax:meta/1 > (using merl in OTP 18 to simplify building the AST): > > 1> AST=merl:quote("-import(mymdule, [test1/0, test2/0])"). > 2> io:put_chars(erl_prettypr:format(erl_syntax:meta(AST))). > > erl_syntax:make_tree(prefix_expr, > [[erl_syntax:operator('-')], > [erl_syntax:make_tree(application, > [[erl_syntax:atom(import)], > [erl_syntax:atom(mymdule), > > erl_syntax:list([erl_syntax:make_tree(infix_expr, > > [[erl_syntax:atom(test1)], > > [erl_syntax:operator('/')], > > [erl_syntax:integer(0)]]), > > erl_syntax:make_tree(infix_expr, > > [[erl_syntax:atom(test2)], > > [erl_syntax:operator('/')], > > [erl_syntax:integer(0)]])])]])]]) > > There's also the useful function merl:show/1 for inspecting the basic > structure of an AST: > > 3> merl:show(AST). > > prefix_expr: -import(mymdule, [...]) > operator: - > + > application: import(mymdule, [test1 / 0, test2 / 0]) > atom: import > + > atom: mymdule > list: [test1 / 0, test2 / 0] > infix_expr: test1 / 0 > atom: test1 > + > operator: / > + > integer: 0 > infix_expr: test2 / 0 > atom: test2 > + > operator: / > + > integer: 0 > > > /Richard > > On Tue, Jul 14, 2015 at 3:08 PM, Humberto Rodr?guez Avila < > h87rguez@REDACTED> wrote: > >> Hello guys! >> >> I am a new programming in Erlang and I interested in doing things with >> meta-programming. I have been working with merl and I did a couple of >> examples (module generation). Nevertheless, I have been trying to do the >> same thing using erl_syntax, but I can?t found how to build the syntax >> tree of the directive -import :( >> >> For example: -import(mymdule, [test1/0, test2/0]) >> >> Any suggestion? >> >> Thanks in advance! >> >> --- Humberto >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rtrlists@REDACTED Thu Jul 16 14:31:52 2015 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 16 Jul 2015 13:31:52 +0100 Subject: [erlang-questions] Why application:start can not accept any args? In-Reply-To: References: Message-ID: When you call application:start(myapp) then the corresponding application tuple from the .app file for that application is consulted to find which module contains the startup code. This is the line with the 'mod' key. This specifies the module and the arguments to pass as a second parameter to that module's start/2 function as {Mod, StartArg}. The reason for this is that when you are working in a (semi)interactive environment, you would simply call the start function directly. And this usually happens during development or testing. But when you have a finished system, you have automated startup using the application framework. In this case any arguments must be available via configuration. Thus you can provide the arguments as part of the application configuration in the .app file. There you can have an entry {mod, {myapp, some_erlang_term}} and this would invoke myapp:start(normal, some_erlang_term). I have not seen this used very often. Hope this explains it a little bit, Robby On 13 July 2015 at 11:39, Nuku Ameyibor wrote: > u can add environmental variables for your application in your .app file > . > there is a tuple with a key env which u can use to add environmental > variables to your application which you can then access from your > application > eg . > {application, test, > [{vsn, "1.0.0"}, > {description, "testing"}, > {modules, [test, test_sup,test_start]}, > {applications, [stdlib, kernel, crypto]}, > {registered, [test,test_sup]}, > {mod, {test, []}}, > {env, > [ > {env1,b}, > {env2,a}, > {env3,c} > ] > } > ]}. > > you can access them like so > {ok, Env} = application:get_env(env1), etc.. > > there is also a set_env/3 ,set_env/4 which can be used to change > environmental variables during run-time > > > On Sun, Jul 12, 2015 at 1:19 PM, Lihe Wang > wrote: > >> Can any one give the explain about OTP? The application behaviour's call >> back "start" need args, but when I want start my code as a real >> application, I only can write "applicaion:start(mycode)", and can not pass >> any args to mycode. Why otp design like this? When I want start a >> application as background service from escript, for example: >> >> main([Arg1, Arg2, Arg3]) -> >> application:start(mycode1), %%with Arg1 >> application:start(mycode2), %%with Arg2 >> workcode(Arg3). %%workcode (escript) will run as a daemon, and need >> service supplied by mycode1 and mycode2. >> >> What is the right way to write these code? >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Jul 16 14:55:48 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 16 Jul 2015 14:55:48 +0200 Subject: [erlang-questions] Erlang and Rules Engines / Production Systems In-Reply-To: References: Message-ID: On Wed, Jul 15, 2015 at 11:01 PM, Garrett Smith wrote: > Use Erlang to tackle that directly. Just hard code the > rules for the time being - it's easy to move them outside the program > as config - and when people complain that they can't edit the cryptic > Erlang term syntax, introduce a rules compile phase that takes > Markdown defined rules - or better, Yaml - and generates the Erlang > config. > Parse the Yaml to an AST. Compile the AST through merl to Erlang code. Compile the Erlang code to BEAM. Load module. Call into module. Revel in the fast execution :) -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From spylik@REDACTED Thu Jul 16 14:09:27 2015 From: spylik@REDACTED (Oleksii Semilietov) Date: Thu, 16 Jul 2015 15:09:27 +0300 Subject: [erlang-questions] The most effective way to check ETS-table created Message-ID: Did somebody make research about the most cheap and effective way to check is ets table exists? What should be faster ets:info(Table) or ets:first(Table) with exception? Any other suggestion? Best regards. -- Oleksii Semilietov -------------- next part -------------- An HTML attachment was scrubbed... URL: From alttagil@REDACTED Thu Jul 16 15:15:37 2015 From: alttagil@REDACTED (Alex Hudich) Date: Thu, 16 Jul 2015 16:15:37 +0300 Subject: [erlang-questions] bad certificate if trying to verify StartSsl certificate In-Reply-To: References: Message-ID: <3039E4C0-C74C-4E33-A05F-19375D52C5A3@gmail.com> When I tried to check connection with openssl command I?ve got w/o cacert.pem file: $ openssl s_client -connect nicemine.ru:443 -verify 99 verify depth is 99 CONNECTED(00000003) depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority verify error:num=19:self signed certificate in certificate chain verify return:1 depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority verify return:1 depth=1 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA verify return:1 depth=0 /C=KZ/CN=www.nicefiles.ru/emailAddress=webmaster@REDACTED verify return:1 and with it $ openssl s_client -connect nicemine.ru:443 -verify 99 -CAfile cacert.pem verify depth is 99 CONNECTED(00000003) depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority verify return:1 depth=1 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA verify return:1 depth=0 /C=KZ/CN=www.nicefiles.ru/emailAddress=webmaster@REDACTED verify return:1 so cacert.pem file contains enough info for StartCom certificates to be checked as valid. Also I?ve tried to dig it more in erlang and I?ve found that I get error in OTP 18 too. And the reason for bad certificate error is {bad_cert,invalid_issuer} I also tried to add https://www.startssl.com/certs/sub.class1.server.ca.pem file to cacert.pem but with no luck. > 16 ???? 2015 ?., ? 12:16, Alex Hudich ???????(?): > > Hi! > > > > wget http://curl.haxx.se/ca/cacert.pem > > and then > > ssl:connect( "www.nicemine.ru", 443, [{verify,verify_peer},{server_name_indication,"www.nicemine.ru"},{depth,2},{cacertfile,"cacert.pem"}] ). > > gives me {error,{tls_alert,"bad certificate"}} > > > > Why? Site can be opened ok in the browser. > > Erlang/OTP 17 [erts-6.3] > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From olivier.boudeville@REDACTED Thu Jul 16 15:28:01 2015 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Thu, 16 Jul 2015 15:28:01 +0200 Subject: [erlang-questions] A few type-related 18.0 questions In-Reply-To: References: Message-ID: Hello Tuncer, tuncer.ayaz@REDACTED a ?crit sur 26/06/2015 23:42:18 : [...] > > Using 18.0, is there a way of specifying to Dialyzer that warnings > > about calls to non-existing functions should be suppressed by > > telling that from outside of the module that is supposed to define > > these functions? > > Yes, Hans Bolinder was kind enough to add it on short notice during > the 18.0 cycle. > > We use it in rebar, which has to work on anything from R13B03 to 18.0. > https://github.com/rebar/rebar/blob/6cc18c931c/src/rebar_utils.erl#L75: > > -dialyzer({no_missing_calls, escript_foldl/3}). > > This tells Dialyzer to ignore calls to unknown functions when > analyzing escript_foldl/3. > > Does this work for you? Thanks for your reply and sorry for my late message. The suppression I was mentioning regarded a call from a plain module M1 to another module M2 that is generated at runtime (compile:form/2) and then loaded (code:load_binary/3): Dialyzer then sees in M1 an actual call to a module M2 that it cannot find (no source nor BEAM file, for obvious reasons). In this case I do not think we can tell Dialyzer to suppress this warning (since the -dialyzer attribute seems to be meant to be used in the sources *of the module itself*, i.e. it would have here to be specified in the sourceless M2). This is not a big issue (and these are edges cases indeed), but it would be interesting if we could define 'global' suppressions, or that would apply to a set of modules (similarly, Dialyzer would not be able to see that conditional calls could be done based on the actual availability of the exported function - using module_info/1 for that; Dialyzer would then detect a potential call to a non-existing function, it would be neat if we could specify that these warnings should be suppressed as we know that this cannot happen). Thanks again, Best, Olivier. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. -------------- next part -------------- An HTML attachment was scrubbed... URL: From e@REDACTED Thu Jul 16 15:59:47 2015 From: e@REDACTED (e@REDACTED) Date: Thu, 16 Jul 2015 15:59:47 +0200 Subject: [erlang-questions] WomENcourage 2015 conference Message-ID: <55A7B8D3.5020000@bestmx.net> Hi all. Here in the erlang-questions they have been posting their Call For Papers. So i hope that some of you may know how to contact them. They notified me that my paper is accepted and refuse to answer my questions ever since, leaving me no means to contact them. From eric.pailleau@REDACTED Thu Jul 16 19:29:33 2015 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Thu, 16 Jul 2015 19:29:33 +0200 Subject: [erlang-questions] bad certificate if trying to verify StartSsl certificate In-Reply-To: <3039E4C0-C74C-4E33-A05F-19375D52C5A3@gmail.com> Message-ID: Hi, try with depth = 3. Depth 0 to depth 2 is 3. Regards Le?16 juil. 2015 15:15, Alex Hudich a ?crit?: > > When I tried to check connection with openssl command I?ve got w/o cacert.pem file: > > $ openssl s_client -connect nicemine.ru:443 -verify 99? > verify depth is 99 > CONNECTED(00000003) > depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority > verify error:num=19:self signed certificate in certificate chain > verify return:1 > depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority > verify return:1 > depth=1 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA > verify return:1 > depth=0 /C=KZ/CN=www.nicefiles.ru/emailAddress=webmaster@REDACTED > verify return:1 > > > and with it > > $ openssl s_client -connect nicemine.ru:443 -verify 99 -CAfile cacert.pem > verify depth is 99 > CONNECTED(00000003) > depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority > verify return:1 > depth=1 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA > verify return:1 > depth=0 /C=KZ/CN=www.nicefiles.ru/emailAddress=webmaster@REDACTED > verify return:1 > > so cacert.pem file contains enough info for StartCom certificates to be checked as valid. > > > Also I?ve tried to dig it more in erlang and I?ve found that I get error in OTP 18 too. > > And the reason for bad certificate error is {bad_cert,invalid_issuer} > > > > I also tried to add ?https://www.startssl.com/certs/sub.class1.server.ca.pem?file to cacert.pem but with no luck. > > > > > >> 16 ???? 2015 ?., ? 12:16, Alex Hudich ???????(?): >> >> Hi! >> >> >> >> wget http://curl.haxx.se/ca/cacert.pem >> >> and then >> >> ssl:connect( "www.nicemine.ru", 443, [{verify,verify_peer},{server_name_indication,"www.nicemine.ru"},{depth,2},{cacertfile,"cacert.pem"}] ). >> >> gives me {error,{tls_alert,"bad certificate"}} >> >> >> >> Why? Site can be opened ok in the browser. >> >> Erlang/OTP 17 [erts-6.3] >> >> > From alttagil@REDACTED Thu Jul 16 19:54:20 2015 From: alttagil@REDACTED (Alex Hudich) Date: Thu, 16 Jul 2015 20:54:20 +0300 Subject: [erlang-questions] bad certificate if trying to verify StartSsl certificate In-Reply-To: References: Message-ID: <5C14B5C7-3026-4AED-BD2F-DE333AF4482D@gmail.com> Hi, It doesn?t help. Still {bad_cert,invalid_issuer} > 16 ???? 2015 ?., ? 20:29, ?ric Pailleau ???????(?): > > Hi, try with depth = 3. Depth 0 to depth 2 is 3. > Regards > > Le 16 juil. 2015 15:15, Alex Hudich a ?crit : >> >> When I tried to check connection with openssl command I?ve got w/o cacert.pem file: >> >> $ openssl s_client -connect nicemine.ru:443 -verify 99 >> verify depth is 99 >> CONNECTED(00000003) >> depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority >> verify error:num=19:self signed certificate in certificate chain >> verify return:1 >> depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority >> verify return:1 >> depth=1 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA >> verify return:1 >> depth=0 /C=KZ/CN=www.nicefiles.ru/emailAddress=webmaster@REDACTED >> verify return:1 >> >> >> and with it >> >> $ openssl s_client -connect nicemine.ru:443 -verify 99 -CAfile cacert.pem >> verify depth is 99 >> CONNECTED(00000003) >> depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority >> verify return:1 >> depth=1 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA >> verify return:1 >> depth=0 /C=KZ/CN=www.nicefiles.ru/emailAddress=webmaster@REDACTED >> verify return:1 >> >> so cacert.pem file contains enough info for StartCom certificates to be checked as valid. >> >> >> Also I?ve tried to dig it more in erlang and I?ve found that I get error in OTP 18 too. >> >> And the reason for bad certificate error is {bad_cert,invalid_issuer} >> >> >> >> I also tried to add https://www.startssl.com/certs/sub.class1.server.ca.pem file to cacert.pem but with no luck. >> >> >> >> >> >>> 16 ???? 2015 ?., ? 12:16, Alex Hudich ???????(?): >>> >>> Hi! >>> >>> >>> >>> wget http://curl.haxx.se/ca/cacert.pem >>> >>> and then >>> >>> ssl:connect( "www.nicemine.ru", 443, [{verify,verify_peer},{server_name_indication,"www.nicemine.ru"},{depth,2},{cacertfile,"cacert.pem"}] ). >>> >>> gives me {error,{tls_alert,"bad certificate"}} >>> >>> >>> >>> Why? Site can be opened ok in the browser. >>> >>> Erlang/OTP 17 [erts-6.3] >>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From santif@REDACTED Thu Jul 16 20:16:08 2015 From: santif@REDACTED (=?UTF-8?Q?Santiago_Fern=C3=A1ndez?=) Date: Thu, 16 Jul 2015 15:16:08 -0300 Subject: [erlang-questions] bad certificate if trying to verify StartSsl certificate In-Reply-To: <5C14B5C7-3026-4AED-BD2F-DE333AF4482D@gmail.com> References: <5C14B5C7-3026-4AED-BD2F-DE333AF4482D@gmail.com> Message-ID: can't reproduce: Erlang/OTP 17 [erts-6.4] [source] [64-bit] [smp:8:8] [async-threads:10] [kernel-poll:false] Eshell V6.4 (abort with ^G) 1> application:ensure_all_started(ssl). {ok,[crypto,asn1,public_key,ssl]} 2> ssl:connect( "www.nicemine.ru", 443, [{verify,verify_peer},{server_name_indication,"www.nicemine.ru"},{depth,2},{cacertfile,"cacert.pem"}] ). {ok,{sslsocket,{gen_tcp,#Port<0.821>,tls_connection, undefined}, <0.49.0>}} -- Santiago On Thu, Jul 16, 2015 at 2:54 PM, Alex Hudich wrote: > Hi, > > It doesn?t help. Still {bad_cert,invalid_issuer} > > > > 16 ???? 2015 ?., ? 20:29, ?ric Pailleau > ???????(?): > > Hi, try with depth = 3. Depth 0 to depth 2 is 3. > Regards > > Le 16 juil. 2015 15:15, Alex Hudich a ?crit : > > > When I tried to check connection with openssl command I?ve got w/o > cacert.pem file: > > $ openssl s_client -connect nicemine.ru:443 -verify 99 > verify depth is 99 > CONNECTED(00000003) > depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate > Signing/CN=StartCom Certification Authority > verify error:num=19:self signed certificate in certificate chain > verify return:1 > depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate > Signing/CN=StartCom Certification Authority > verify return:1 > depth=1 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate > Signing/CN=StartCom Class 1 Primary Intermediate Server CA > verify return:1 > depth=0 /C=KZ/CN=www.nicefiles.ru/emailAddress=webmaster@REDACTED > verify return:1 > > > and with it > > $ openssl s_client -connect nicemine.ru:443 -verify 99 -CAfile cacert.pem > verify depth is 99 > CONNECTED(00000003) > depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate > Signing/CN=StartCom Certification Authority > verify return:1 > depth=1 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate > Signing/CN=StartCom Class 1 Primary Intermediate Server CA > verify return:1 > depth=0 /C=KZ/CN=www.nicefiles.ru/emailAddress=webmaster@REDACTED > verify return:1 > > so cacert.pem file contains enough info for StartCom certificates to be > checked as valid. > > > Also I?ve tried to dig it more in erlang and I?ve found that I get error > in OTP 18 too. > > And the reason for bad certificate error is {bad_cert,invalid_issuer} > > > > I also tried to add > https://www.startssl.com/certs/sub.class1.server.ca.pem file to > cacert.pem but with no luck. > > > > > > 16 ???? 2015 ?., ? 12:16, Alex Hudich ???????(?): > > Hi! > > > > wget http://curl.haxx.se/ca/cacert.pem > > and then > > ssl:connect( "www.nicemine.ru", 443, > [{verify,verify_peer},{server_name_indication,"www.nicemine.ru"},{depth,2},{cacertfile,"cacert.pem"}] > ). > > gives me {error,{tls_alert,"bad certificate"}} > > > > Why? Site can be opened ok in the browser. > > Erlang/OTP 17 [erts-6.3] > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Thu Jul 16 21:02:58 2015 From: lloyd@REDACTED (lloyd@REDACTED) Date: Thu, 16 Jul 2015 15:02:58 -0400 (EDT) Subject: [erlang-questions] An Erlang 101 question Message-ID: <1437073378.164822464@apps.rackspace.com> Hello, I have several functions that look like this: get_contact(Email) -> open_db(), Result = dets:lookup(?contacts, Email), close_db(), Result. The implication is that the function returns and empty list, [], if it fails to find a record, otherwise it returns a list containing the record. This makes it easy to distinguish the failure-to-return a record condition, but means that every function that depends up on get_contact/1 needs to do so and know what to do in that case. But that begins to feel like defensive programming. I can see several ways of dealing with this problem, but it occurs to me that there must be a conventional approach. What might be... what? Thanks, LRP From alttagil@REDACTED Thu Jul 16 21:15:48 2015 From: alttagil@REDACTED (Alex Hudich) Date: Thu, 16 Jul 2015 22:15:48 +0300 Subject: [erlang-questions] bad certificate if trying to verify StartSsl certificate In-Reply-To: References: <5C14B5C7-3026-4AED-BD2F-DE333AF4482D@gmail.com> Message-ID: <010C00BC-8E15-4A2A-B301-99775DB96AB3@gmail.com> ubuntu 14.04 # wget http://curl.haxx.se/ca/cacert.pem --2015-07-16 19:11:50-- http://curl.haxx.se/ca/cacert.pem Resolving curl.haxx.se (curl.haxx.se)... 2a00:1a28:1200:9::2, 80.67.6.50 Connecting to curl.haxx.se (curl.haxx.se)|2a00:1a28:1200:9::2|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 258424 (252K) Saving to: 'cacert.pem' 100%[=============================================================================================================================================================================================>] 258,424 1.62MB/s in 0.2s 2015-07-16 19:11:50 (1.62 MB/s) - 'cacert.pem' saved [258424/258424] # erl Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] Eshell V7.0 (abort with ^G) 1> application:ensure_all_started(ssl). {ok,[crypto,asn1,public_key,ssl]} 2> ssl:connect( "www.nicemine.ru", 443, [{verify,verify_peer},{server_name_indication,"www.nicemine.ru"},{depth,2},{cacertfile,"cacert.pem"}] ). =ERROR REPORT==== 16-Jul-2015::19:12:18 === SSL: certify: ssl_handshake.erl:1476:Fatal error: bad certificate {error,{tls_alert,"bad certificate"}} 3> and Mac OS X $ wget http://curl.haxx.se/ca/cacert.pem --2015-07-16 22:09:02-- http://curl.haxx.se/ca/cacert.pem Resolving curl.haxx.se... 80.67.6.50, 2a00:1a28:1200:9::2 Connecting to curl.haxx.se|80.67.6.50|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 258424 (252K) Saving to: 'cacert.pem' 100%[=============================================================================================================================================================================================>] 258,424 --.-K/s in 0.1s 2015-07-16 22:09:02 (1.92 MB/s) - 'cacert.pem' saved [258424/258424] $ erl Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace] Eshell V6.3 (abort with ^G) 1> application:ensure_all_started(ssl). {ok,[crypto,asn1,public_key,ssl]} 2> ssl:connect( "www.nicemine.ru", 443, [{verify,verify_peer},{server_name_indication,"www.nicemine.ru"},{depth,2},{cacertfile,"cacert.pem"}] ). =ERROR REPORT==== 16-Jul-2015::22:09:23 === SSL: certify: ssl_handshake.erl:1389:Fatal error: bad certificate {error,{tls_alert,"bad certificate"}} 3> :(((((((((((( > 16 ???? 2015 ?., ? 21:16, Santiago Fern?ndez ???????(?): > > can't reproduce: > > Erlang/OTP 17 [erts-6.4] [source] [64-bit] [smp:8:8] [async-threads:10] [kernel-poll:false] > > Eshell V6.4 (abort with ^G) > 1> application:ensure_all_started(ssl). > {ok,[crypto,asn1,public_key,ssl]} > 2> ssl:connect( "www.nicemine.ru ", 443, [{verify,verify_peer},{server_name_indication,"www.nicemine.ru "},{depth,2},{cacertfile,"cacert.pem"}] ). > {ok,{sslsocket,{gen_tcp,#Port<0.821>,tls_connection, > undefined}, > <0.49.0>}} > > > > > > -- > Santiago > > On Thu, Jul 16, 2015 at 2:54 PM, Alex Hudich > wrote: > Hi, > > It doesn?t help. Still {bad_cert,invalid_issuer} > > > >> 16 ???? 2015 ?., ? 20:29, ?ric Pailleau > ???????(?): >> >> Hi, try with depth = 3. Depth 0 to depth 2 is 3. >> Regards >> >> Le 16 juil. 2015 15:15, Alex Hudich > a ?crit : >>> >>> When I tried to check connection with openssl command I?ve got w/o cacert.pem file: >>> >>> $ openssl s_client -connect nicemine.ru :443 -verify 99 >>> verify depth is 99 >>> CONNECTED(00000003) >>> depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority >>> verify error:num=19:self signed certificate in certificate chain >>> verify return:1 >>> depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority >>> verify return:1 >>> depth=1 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA >>> verify return:1 >>> depth=0 /C=KZ/CN=www.nicefiles.ru/emailAddress=webmaster@REDACTED >>> verify return:1 >>> >>> >>> and with it >>> >>> $ openssl s_client -connect nicemine.ru :443 -verify 99 -CAfile cacert.pem >>> verify depth is 99 >>> CONNECTED(00000003) >>> depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority >>> verify return:1 >>> depth=1 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA >>> verify return:1 >>> depth=0 /C=KZ/CN=www.nicefiles.ru/emailAddress=webmaster@REDACTED >>> verify return:1 >>> >>> so cacert.pem file contains enough info for StartCom certificates to be checked as valid. >>> >>> >>> Also I?ve tried to dig it more in erlang and I?ve found that I get error in OTP 18 too. >>> >>> And the reason for bad certificate error is {bad_cert,invalid_issuer} >>> >>> >>> >>> I also tried to add https://www.startssl.com/certs/sub.class1.server.ca.pem file to cacert.pem but with no luck. >>> >>> >>> >>> >>> >>>> 16 ???? 2015 ?., ? 12:16, Alex Hudich > ???????(?): >>>> >>>> Hi! >>>> >>>> >>>> >>>> wget http://curl.haxx.se/ca/cacert.pem >>>> >>>> and then >>>> >>>> ssl:connect( "www.nicemine.ru ", 443, [{verify,verify_peer},{server_name_indication,"www.nicemine.ru "},{depth,2},{cacertfile,"cacert.pem"}] ). >>>> >>>> gives me {error,{tls_alert,"bad certificate"}} >>>> >>>> >>>> >>>> Why? Site can be opened ok in the browser. >>>> >>>> Erlang/OTP 17 [erts-6.3] >>>> >>>> >>> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fernando.benavides@REDACTED Thu Jul 16 21:40:04 2015 From: fernando.benavides@REDACTED (Fernando 'Brujo' Benavides) Date: Thu, 16 Jul 2015 16:40:04 -0300 Subject: [erlang-questions] [ANN] Videos from ErlangBA 2015 Message-ID: <56C66231-FB6E-488A-A64A-BEA60405E987@inaka.net> The videos from ErlangBA 2015 talks are available online now :) WARNING: they?re in spanish :) Cheers! Fernando "Brujo" Benavides about.me/elbrujohalcon -------------- next part -------------- An HTML attachment was scrubbed... URL: From rtrlists@REDACTED Thu Jul 16 22:16:11 2015 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 16 Jul 2015 21:16:11 +0100 Subject: [erlang-questions] An Erlang 101 question In-Reply-To: <1437073378.164822464@apps.rackspace.com> References: <1437073378.164822464@apps.rackspace.com> Message-ID: This depends a bit in what you're trying to achieve. Do you need a valid result? Or would some kind of default value be sufficient? In the later case, you could then also provide a get_contact(Email, Default) that returns the default value if none was found in the db. If you require a valid result to appear in your db, then it could still depend on the rest of your app. Is that something normal you need to be able to react on? If yes, well, then you need to code for it. If not, then you might want to consider it an "exceptional" occurrence and simply crash the process and let the supervisor clean things up. HTH, Robby On Jul 16, 2015 9:03 PM, wrote: > Hello, > > I have several functions that look like this: > > get_contact(Email) -> > open_db(), > Result = dets:lookup(?contacts, Email), > close_db(), > Result. > > The implication is that the function returns and empty list, [], if it > fails to find a record, otherwise it returns a list containing the record. > > This makes it easy to distinguish the failure-to-return a record > condition, but means that every function that depends up on get_contact/1 > needs to do so and know what to do in that case. But that begins to feel > like defensive programming. > > I can see several ways of dealing with this problem, but it occurs to me > that there must be a conventional approach. What might be... what? > > Thanks, > > LRP > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Thu Jul 16 23:37:59 2015 From: g@REDACTED (Garrett Smith) Date: Thu, 16 Jul 2015 16:37:59 -0500 Subject: [erlang-questions] An Erlang 101 question In-Reply-To: <1437073378.164822464@apps.rackspace.com> References: <1437073378.164822464@apps.rackspace.com> Message-ID: Hey, I'm all for a direct solution, but this opening and closing of the db for each read feels a little cavalier, even for me ;) I would use a gen_server (e2 service) to maintain the open db, flushing on each write to guard against data loss if the VM is shutdown abruptly. But to answer your question, a more canonical interface here would be to return either {ok, Contact} or error (mapped from [Contact] or [] respectively) - or just return Contact and generate an exception if the contact doesn't exists. The form depends on what you want this function to do. See dict for an example of an interface that provide both interfaces - and lets the user decide. You're right though, exposing the dets interface here is wrong, morally speaking. On Thu, Jul 16, 2015 at 2:02 PM, wrote: > Hello, > > I have several functions that look like this: > > get_contact(Email) -> > open_db(), > Result = dets:lookup(?contacts, Email), > close_db(), > Result. > > The implication is that the function returns and empty list, [], if it fails to find a record, otherwise it returns a list containing the record. > > This makes it easy to distinguish the failure-to-return a record condition, but means that every function that depends up on get_contact/1 needs to do so and know what to do in that case. But that begins to feel like defensive programming. > > I can see several ways of dealing with this problem, but it occurs to me that there must be a conventional approach. What might be... what? > > Thanks, > > LRP > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From martink@REDACTED Thu Jul 16 23:44:41 2015 From: martink@REDACTED (Martin Karlsson) Date: Fri, 17 Jul 2015 09:44:41 +1200 Subject: [erlang-questions] An Erlang 101 question In-Reply-To: <1437073378.164822464@apps.rackspace.com> References: <1437073378.164822464@apps.rackspace.com> Message-ID: It boils down to what the caller wants to do with the result and if it makes sense to handle it at all. For database stuff I usually do: {ok, Result} = query(). and return something else for errors, or others, such as: {error, not_found} = query(). This way those who don't care about stuff going wrong do: {ok, Result} = query(). knowing that Result will always contain something useful. And those who do care: case query() of {ok, Result} -> 'hello world!'. {error, not_found} -> ooops end. If you expect your queries to always success (i.e the errors are not part of the normal flow) you can even throw on errors and just return the result. Result = query() and if you need to check it: try query() of Result -> Result catch {throw, not_found} -> oops end Your current case is just a variant of the first option but with less information. It is harder to pattern match meaning you'd basically always need: case query() of [] -> not_found; Result -> 'hello world!' end mnesia is a funny application which uses both of the above approaches depending on if you are using mnesia:transaction (returns {atomic, Result} or {aborted, Reason}}) vs mnesia:activity which throws on errors. From lloyd@REDACTED Fri Jul 17 00:46:10 2015 From: lloyd@REDACTED (lloyd@REDACTED) Date: Thu, 16 Jul 2015 18:46:10 -0400 (EDT) Subject: [erlang-questions] An Erlang 101 question In-Reply-To: References: <1437073378.164822464@apps.rackspace.com> Message-ID: <1437086770.567813759@apps.rackspace.com> Hi Garrett and Martin, Yes, I'm using dets temporarily until I decide what database to use. All the dets-exposed functions are going through an api, e.g.: contacts:get_contact(Email) -> contacts_api:get_contact(Email). contacts_api:get_contact(Email) -> dets_contacts:get_contact(Email). get_contact(Email) -> open_db(), Result = dets:lookup(?contacts, Email), close_db(), Result. Sort of in the spirit of your suggestion to get something working then sweat the edge cases. I'm not confident that the second level of indirection is necessary, but it reflects my current understanding of how to isolate record structures to one module only. I'm using Jesper's repr/2 code to access values. Re: the idea of exposing db calls through a gen_server or e2 service, would this apply only to dbs such as dets that don't have transactions? Or is it good practice regardless of of the db back-end? As my last few posts may have hinted, with the help of Jesper and others I'm struggling to understand conventional Erlang practices that I, at least, have not found well documented. That's the problem of being a one-man-band. I can't call in my supervisor when I'm stuck. But folks have been extraordinarily generous with tips and information for which I'm profoundly grateful. Martin--- thanks for your suggestion. I'll adopt it. Thanks to all, Lloyd -----Original Message----- From: "Garrett Smith" Sent: Thursday, July 16, 2015 5:37pm To: "Lloyd Prentice" Cc: "erlang-questions" Subject: Re: [erlang-questions] An Erlang 101 question Hey, I'm all for a direct solution, but this opening and closing of the db for each read feels a little cavalier, even for me ;) I would use a gen_server (e2 service) to maintain the open db, flushing on each write to guard against data loss if the VM is shutdown abruptly. But to answer your question, a more canonical interface here would be to return either {ok, Contact} or error (mapped from [Contact] or [] respectively) - or just return Contact and generate an exception if the contact doesn't exists. The form depends on what you want this function to do. See dict for an example of an interface that provide both interfaces - and lets the user decide. You're right though, exposing the dets interface here is wrong, morally speaking. On Thu, Jul 16, 2015 at 2:02 PM, wrote: > Hello, > > I have several functions that look like this: > > get_contact(Email) -> > open_db(), > Result = dets:lookup(?contacts, Email), > close_db(), > Result. > > The implication is that the function returns and empty list, [], if it fails to find a record, otherwise it returns a list containing the record. > > This makes it easy to distinguish the failure-to-return a record condition, but means that every function that depends up on get_contact/1 needs to do so and know what to do in that case. But that begins to feel like defensive programming. > > I can see several ways of dealing with this problem, but it occurs to me that there must be a conventional approach. What might be... what? > > Thanks, > > LRP > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From g@REDACTED Fri Jul 17 04:24:03 2015 From: g@REDACTED (Garrett Smith) Date: Thu, 16 Jul 2015 21:24:03 -0500 Subject: [erlang-questions] An Erlang 101 question In-Reply-To: <1437086770.567813759@apps.rackspace.com> References: <1437073378.164822464@apps.rackspace.com> <1437086770.567813759@apps.rackspace.com> Message-ID: A few observations... I think the problem you're looking at resembles the data server in the e2 tutorial: http://e2project.org/tutorial.html In the tutorial, you start out creating a super simple data interface for reading and writing values: http://e2project.org/tutorial.html#data-access-api Later, to make that interface available to the system, you wrap it in a registered process (e2 service): http://e2project.org/tutorial.html#data-server That's an indirection, but the step serves an important role: to implement a lifecycle for the data access layer. By lifecycle I mean, what do you need to do start the data facilities? What do you need to access them? The indirection gives you process isolation as well and a chance to recover from failure. It serves as the "data service" to your app, wrapping the details that consumers don't care about and working to make the facility as available as possible. In your case, you could simply rename the modules from the tutorial this way: mydb_db -> contacts_dets mydb_data -> contacts So call to contacts:get_contact/1 is a service front-end, which calls e2_service:call/2 (or gen_server:call/2 if you want to type more code). That sends a message from the client to the service process. The message handler (handle_msg/3 in e2, or handle_call/3 for gen_server) is called on the service process side and just passes the Db reference (managed by the service) and the additional args (provided by the client call) to contacts_dets, which implements the dets specific backend. Imagine this for your contact operations and you've got it! As for these ideas about hiding record definitions, I think for your case, right now, I would just forget everything there. It's not helping you. It's really not a problem to let a record propagate throughout your application. It might be a problem in time, but it's not now. It might never be. That said, working with records as a data structure IMO is a bit painful and I'd be _tempted_ to use maps for attribute data in and out of your contacts_dets module. By "attribute data" here I mean non key values that tend to change over time. So adding a contact might look like this: contacts_dets:update_contact(Key, Attrs) Attrs here is a map, which contacts_dets can coerce into whatever record it's using. But this takes work and code and what does it get you? It's nice that you don't have a gnarly record definition in use everywhere, but the map code will look similar. Is it easier to maintain? I doubt it. If I were you right now, I'd just let this record leak everywhere. You have more interesting problems to solve. So quick summary of my thinking: - Copy the e2 tutorial code and rename the modules as per above - Replace the get, set, and del functions with the operations that you want for contacts - Happily use this facility in your application for all your contact related needs! On Thu, Jul 16, 2015 at 5:46 PM, wrote: > Hi Garrett and Martin, > > Yes, I'm using dets temporarily until I decide what database to use. All the dets-exposed functions are going through an api, e.g.: > > contacts:get_contact(Email) -> > contacts_api:get_contact(Email). > > contacts_api:get_contact(Email) -> > dets_contacts:get_contact(Email). > > get_contact(Email) -> > open_db(), > Result = dets:lookup(?contacts, Email), > close_db(), > Result. > > Sort of in the spirit of your suggestion to get something working then sweat the edge cases. > > I'm not confident that the second level of indirection is necessary, but it reflects my current understanding of how to isolate record structures to one module only. I'm using Jesper's repr/2 code to access values. > > Re: the idea of exposing db calls through a gen_server or e2 service, would this apply only to dbs such as dets that don't have transactions? Or is it good practice regardless of of the db back-end? > > As my last few posts may have hinted, with the help of Jesper and others I'm struggling to understand conventional Erlang practices that I, at least, have not found well documented. That's the problem of being a one-man-band. I can't call in my supervisor when I'm stuck. But folks have been extraordinarily generous with tips and information for which I'm profoundly grateful. > > Martin--- thanks for your suggestion. I'll adopt it. > > Thanks to all, > > Lloyd > > > -----Original Message----- > From: "Garrett Smith" > Sent: Thursday, July 16, 2015 5:37pm > To: "Lloyd Prentice" > Cc: "erlang-questions" > Subject: Re: [erlang-questions] An Erlang 101 question > > Hey, I'm all for a direct solution, but this opening and closing of > the db for each read feels a little cavalier, even for me ;) > > I would use a gen_server (e2 service) to maintain the open db, > flushing on each write to guard against data loss if the VM is > shutdown abruptly. > > But to answer your question, a more canonical interface here would be > to return either {ok, Contact} or error (mapped from [Contact] or [] > respectively) - or just return Contact and generate an exception if > the contact doesn't exists. The form depends on what you want this > function to do. See dict for an example of an interface that provide > both interfaces - and lets the user decide. > > You're right though, exposing the dets interface here is wrong, > morally speaking. > > On Thu, Jul 16, 2015 at 2:02 PM, wrote: >> Hello, >> >> I have several functions that look like this: >> >> get_contact(Email) -> >> open_db(), >> Result = dets:lookup(?contacts, Email), >> close_db(), >> Result. >> >> The implication is that the function returns and empty list, [], if it fails to find a record, otherwise it returns a list containing the record. >> >> This makes it easy to distinguish the failure-to-return a record condition, but means that every function that depends up on get_contact/1 needs to do so and know what to do in that case. But that begins to feel like defensive programming. >> >> I can see several ways of dealing with this problem, but it occurs to me that there must be a conventional approach. What might be... what? >> >> Thanks, >> >> LRP >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From ruanbeihong@REDACTED Thu Jul 16 22:49:38 2015 From: ruanbeihong@REDACTED (ruanbeihong) Date: Fri, 17 Jul 2015 04:49:38 +0800 Subject: [erlang-questions] rebar eunit test pass but fail on cover test Message-ID: <3119103.jZzUViM4mz@debian> Hi everyone, I'm learning erlang for a while. Currently I'm using rebar to manage building and test work. I find the cover test provided in rebar (eunit?) quite convenient. But recently I come across a strange situation: when running test without '{cover_enabled, true}.' in rebar.conf, it works find and passed my test, but with it will result in a fail of test. And the crush info is IMO not quite useful. So I ask here for someone might come across same problem and to see if this is a bug. My code is at https://github.com/jamesruan/z85 commit: 6e0fc13667 simply git clone it and run: rebar eunit or rebar -C rebar.conf eunit The first will success in my environment ( Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async- threads:10] [kernel-poll:false]) But the second will fail. Thanks, James Ruan From sid5@REDACTED Fri Jul 17 02:24:33 2015 From: sid5@REDACTED (Sid Muller) Date: Fri, 17 Jul 2015 02:24:33 +0200 Subject: [erlang-questions] shell crash on exit debug In-Reply-To: References: , Message-ID: An HTML attachment was scrubbed... URL: From dleach@REDACTED Fri Jul 17 02:44:28 2015 From: dleach@REDACTED (David Leach) Date: Fri, 17 Jul 2015 00:44:28 +0000 Subject: [erlang-questions] How to reproduce OOM. In-Reply-To: <0050ba502be920e734b294dbb8ea9984.squirrel@mail.jschneider.net> References: , <0050ba502be920e734b294dbb8ea9984.squirrel@mail.jschneider.net> Message-ID: You could always try invoke OOM manually once memory has been used up. If sysrq trigger is enabled Alt + SysRq/PrintScreen + f or if supported on your system os:cmd("echo b > /proc/sysrq-trigger") However, it doesn't guarantee it's going to kill your process, just that it will kill something. See http://unix.stackexchange.com/questions/153585/how-oom-killer-decides-which-process-to-kill-first ________________________________________ From: erlang-questions-bounces@REDACTED on behalf of Jon Schneider Sent: Thursday, 16 July 2015 12:07 a.m. To: Daniil Churikov Cc: erlang questions Subject: Re: [erlang-questions] How to reproduce OOM. The OOM killer is not a reliable thing and changes with the wind. If you want a ceiling on memory how about using ulimit possibly from the shell before starting beam ? Jon > Hello D??niel. Thanks for your suggestions, although it indeed creates > memory allocation error, > it does not do what I need. > I want VM to be killed, but don't want to do it by myself. I am seeking > for > OOM killer's help, > if you know what I mean ;) > > 2015-07-15 12:24 GMT+01:00 D??niel Szoboszlay : > >> This one works for me quite reliably: >> >> Eshell V5.10.4.0.1 (abort with ^G) >> 1> <<1:8000000000000>>. >> >> Crash dump was written to: erl_crash.dump >> binary_alloc: Cannot allocate 1000000000031 bytes of memory (of type >> "binary"). >> Aborted (core dumped) >> >> >> 2015-07-15 13:18 GMT+02:00 Daniil Churikov : >> >>> Recently I was searching for robust way to restart elang VM in case of >>> sudden >>> stop (like crash or OOM). >>> And I found rather difficult to reproduce OOM conditions: I was trying >>> to >>> create >>> gigantic list of integers, so VM would be killed. >>> I did this: >>> >>> L1 = [I || I <- lists:seq(1, 1000)]. >>> L2 = [L1 || _ <- lists:seq(1, 1000)]. >>> L3 = [L2 || _ <- lists:seq(1, 1000)]. % here VM hangs >>> >>> But according to syslog it doesn't look like OOM death, I can't see >>> usual >>> log >>> entries like "Out of memory: Kill process 3855 (beam.smp) score 909 or >>> sacrifice >>> child" and others. >>> >>> My current understanding is I asked more memory then I allowed to ask, >>> and b/c >>> OS is not willing to give this memory, erlang VM hangs. >>> >>> If somebody have ideas how to reproduce OOM I would appreciate for >>> sharing. >>> >>> Thanks! >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From alttagil@REDACTED Fri Jul 17 11:17:52 2015 From: alttagil@REDACTED (Alex Hudich) Date: Fri, 17 Jul 2015 12:17:52 +0300 Subject: [erlang-questions] bad certificate if trying to verify StartSsl certificate In-Reply-To: References: <5C14B5C7-3026-4AED-BD2F-DE333AF4482D@gmail.com> Message-ID: It seems that it is version specific bug. It can be reproduced in 18 and 17.4 OTP versions and everything works fine in 17.5 # erl Erlang/OTP 17 [erts-6.4] [source] [64-bit] [async-threads:10] [hipe] [kernel-poll:false] Eshell V6.4 (abort with ^G) 1> application:ensure_all_started(ssl). {ok,[crypto,asn1,public_key,ssl]} 2> ssl:connect( "www.nicemine.ru", 443, [{verify,verify_peer},{server_name_indication,"www.nicemine.ru"},{depth,2},{cacertfile,"cacert.pem"}] ). {ok,{sslsocket,{gen_tcp,#Port<0.903>,tls_connection, undefined}, <0.48.0>}} 3> > 16 ???? 2015 ?., ? 21:16, Santiago Fern?ndez ???????(?): > > can't reproduce: > > Erlang/OTP 17 [erts-6.4] [source] [64-bit] [smp:8:8] [async-threads:10] [kernel-poll:false] > > Eshell V6.4 (abort with ^G) > 1> application:ensure_all_started(ssl). > {ok,[crypto,asn1,public_key,ssl]} > 2> ssl:connect( "www.nicemine.ru ", 443, [{verify,verify_peer},{server_name_indication,"www.nicemine.ru "},{depth,2},{cacertfile,"cacert.pem"}] ). > {ok,{sslsocket,{gen_tcp,#Port<0.821>,tls_connection, > undefined}, > <0.49.0>}} > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alttagil@REDACTED Fri Jul 17 11:31:45 2015 From: alttagil@REDACTED (Alex Hudich) Date: Fri, 17 Jul 2015 12:31:45 +0300 Subject: [erlang-questions] bad certificate if trying to verify StartSsl certificate In-Reply-To: References: <5C14B5C7-3026-4AED-BD2F-DE333AF4482D@gmail.com> Message-ID: But it seems to me thet there are some diffrernces between 17.4 and 17.5 which make 17.5 ?more buggy? I prepared two files. cacert.pem.1 is empty file with length 0 and cacert.pem which I?ve downloaded earlier. And there is an output of 17.5 which seems to me wrong. Line 2 and 3 is ok. Line 4 is ok. But why line 5 gave me no error?? Erlang/OTP 17 [erts-6.4] [source] [64-bit] [async-threads:10] [hipe] [kernel-poll:false] Eshell V6.4 (abort with ^G) 1> application:ensure_all_started(ssl). {ok,[crypto,asn1,public_key,ssl]} 2> ssl:connect( "www.nicemine.ru", 443, [{verify,verify_peer},{server_name_indication,"www.nicemine.ru"},{depth,2},{cacertfile,"cacert.pem.1"}] ). =ERROR REPORT==== 17-Jul-2015::13:26:45 === SSL: certify: ssl_handshake.erl:1401:Fatal error: unknown ca {error,{tls_alert,"unknown ca"}} 3> ssl:connect( "www.nicemine.ru", 443, [{verify,verify_peer},{server_name_indication,"www.nicemine.ru"},{depth,2},{cacertfile,"cacert.pem.1"}] ). =ERROR REPORT==== 17-Jul-2015::13:26:48 === SSL: certify: ssl_handshake.erl:1401:Fatal error: unknown ca {error,{tls_alert,"unknown ca"}} 4> ssl:connect( "www.nicemine.ru", 443, [{verify,verify_peer},{server_name_indication,"www.nicemine.ru"},{depth,2},{cacertfile,"cacert.pem"}] ). {ok,{sslsocket,{gen_tcp,#Port<0.1236>,tls_connection, undefined}, <0.53.0>}} 5> ssl:connect( "www.nicemine.ru", 443, [{verify,verify_peer},{server_name_indication,"www.nicemine.ru"},{depth,2},{cacertfile,"cacert.pem.1"}] ). {ok,{sslsocket,{gen_tcp,#Port<0.1243>,tls_connection, undefined}, <0.55.0>}} > 16 ???? 2015 ?., ? 21:16, Santiago Fern?ndez ???????(?): > > can't reproduce: > > Erlang/OTP 17 [erts-6.4] [source] [64-bit] [smp:8:8] [async-threads:10] [kernel-poll:false] > > Eshell V6.4 (abort with ^G) > 1> application:ensure_all_started(ssl). > {ok,[crypto,asn1,public_key,ssl]} > 2> ssl:connect( "www.nicemine.ru ", 443, [{verify,verify_peer},{server_name_indication,"www.nicemine.ru "},{depth,2},{cacertfile,"cacert.pem"}] ). > {ok,{sslsocket,{gen_tcp,#Port<0.821>,tls_connection, > undefined}, > <0.49.0>}} > > > > > > -- > Santiago > > On Thu, Jul 16, 2015 at 2:54 PM, Alex Hudich > wrote: > Hi, > > It doesn?t help. Still {bad_cert,invalid_issuer} > > > >> 16 ???? 2015 ?., ? 20:29, ?ric Pailleau > ???????(?): >> >> Hi, try with depth = 3. Depth 0 to depth 2 is 3. >> Regards >> >> Le 16 juil. 2015 15:15, Alex Hudich > a ?crit : >>> >>> When I tried to check connection with openssl command I?ve got w/o cacert.pem file: >>> >>> $ openssl s_client -connect nicemine.ru :443 -verify 99 >>> verify depth is 99 >>> CONNECTED(00000003) >>> depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority >>> verify error:num=19:self signed certificate in certificate chain >>> verify return:1 >>> depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority >>> verify return:1 >>> depth=1 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA >>> verify return:1 >>> depth=0 /C=KZ/CN=www.nicefiles.ru/emailAddress=webmaster@REDACTED >>> verify return:1 >>> >>> >>> and with it >>> >>> $ openssl s_client -connect nicemine.ru :443 -verify 99 -CAfile cacert.pem >>> verify depth is 99 >>> CONNECTED(00000003) >>> depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority >>> verify return:1 >>> depth=1 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA >>> verify return:1 >>> depth=0 /C=KZ/CN=www.nicefiles.ru/emailAddress=webmaster@REDACTED >>> verify return:1 >>> >>> so cacert.pem file contains enough info for StartCom certificates to be checked as valid. >>> >>> >>> Also I?ve tried to dig it more in erlang and I?ve found that I get error in OTP 18 too. >>> >>> And the reason for bad certificate error is {bad_cert,invalid_issuer} >>> >>> >>> >>> I also tried to add https://www.startssl.com/certs/sub.class1.server.ca.pem file to cacert.pem but with no luck. >>> >>> >>> >>> >>> >>>> 16 ???? 2015 ?., ? 12:16, Alex Hudich > ???????(?): >>>> >>>> Hi! >>>> >>>> >>>> >>>> wget http://curl.haxx.se/ca/cacert.pem >>>> >>>> and then >>>> >>>> ssl:connect( "www.nicemine.ru ", 443, [{verify,verify_peer},{server_name_indication,"www.nicemine.ru "},{depth,2},{cacertfile,"cacert.pem"}] ). >>>> >>>> gives me {error,{tls_alert,"bad certificate"}} >>>> >>>> >>>> >>>> Why? Site can be opened ok in the browser. >>>> >>>> Erlang/OTP 17 [erts-6.3] >>>> >>>> >>> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ddosia@REDACTED Fri Jul 17 15:47:04 2015 From: ddosia@REDACTED (Daniil Churikov) Date: Fri, 17 Jul 2015 14:47:04 +0100 Subject: [erlang-questions] How to reproduce OOM. In-Reply-To: References: <0050ba502be920e734b294dbb8ea9984.squirrel@mail.jschneider.net> Message-ID: Thanks everybody for your help, I will try your suggestions. 2015-07-17 1:44 GMT+01:00 David Leach : > You could always try invoke OOM manually once memory has been used up. If > sysrq trigger is enabled > Alt + SysRq/PrintScreen + f > > or if supported on your system > > os:cmd("echo b > /proc/sysrq-trigger") > > However, it doesn't guarantee it's going to kill your process, just that > it will kill something. See > http://unix.stackexchange.com/questions/153585/how-oom-killer-decides-which-process-to-kill-first > > > > ________________________________________ > From: erlang-questions-bounces@REDACTED < > erlang-questions-bounces@REDACTED> on behalf of Jon Schneider < > jon@REDACTED> > Sent: Thursday, 16 July 2015 12:07 a.m. > To: Daniil Churikov > Cc: erlang questions > Subject: Re: [erlang-questions] How to reproduce OOM. > > The OOM killer is not a reliable thing and changes with the wind. > > If you want a ceiling on memory how about using ulimit possibly from the > shell before starting beam ? > > Jon > > > > Hello D??niel. Thanks for your suggestions, although it indeed creates > > memory allocation error, > > it does not do what I need. > > I want VM to be killed, but don't want to do it by myself. I am seeking > > for > > OOM killer's help, > > if you know what I mean ;) > > > > 2015-07-15 12:24 GMT+01:00 D??niel Szoboszlay : > > > >> This one works for me quite reliably: > >> > >> Eshell V5.10.4.0.1 (abort with ^G) > >> 1> <<1:8000000000000>>. > >> > >> Crash dump was written to: erl_crash.dump > >> binary_alloc: Cannot allocate 1000000000031 bytes of memory (of type > >> "binary"). > >> Aborted (core dumped) > >> > >> > >> 2015-07-15 13:18 GMT+02:00 Daniil Churikov : > >> > >>> Recently I was searching for robust way to restart elang VM in case of > >>> sudden > >>> stop (like crash or OOM). > >>> And I found rather difficult to reproduce OOM conditions: I was trying > >>> to > >>> create > >>> gigantic list of integers, so VM would be killed. > >>> I did this: > >>> > >>> L1 = [I || I <- lists:seq(1, 1000)]. > >>> L2 = [L1 || _ <- lists:seq(1, 1000)]. > >>> L3 = [L2 || _ <- lists:seq(1, 1000)]. % here VM hangs > >>> > >>> But according to syslog it doesn't look like OOM death, I can't see > >>> usual > >>> log > >>> entries like "Out of memory: Kill process 3855 (beam.smp) score 909 or > >>> sacrifice > >>> child" and others. > >>> > >>> My current understanding is I asked more memory then I allowed to ask, > >>> and b/c > >>> OS is not willing to give this memory, erlang VM hangs. > >>> > >>> If somebody have ideas how to reproduce OOM I would appreciate for > >>> sharing. > >>> > >>> Thanks! > >>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >>> > >>> > >> > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Fri Jul 17 17:35:19 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 17 Jul 2015 17:35:19 +0200 Subject: [erlang-questions] A few type-related 18.0 questions In-Reply-To: References: Message-ID: On Thu, Jul 16, 2015 at 3:28 PM, Olivier BOUDEVILLE wrote: > Hello Tuncer, > > tuncer.ayaz@REDACTED a ?crit sur 26/06/2015 23:42:18 : > > [...] > > > Using 18.0, is there a way of specifying to Dialyzer that > > > warnings about calls to non-existing functions should be > > > suppressed by telling that from outside of the module that is > > > supposed to define these functions? > > > > Yes, Hans Bolinder was kind enough to add it on short notice > > during the 18.0 cycle. > > > > We use it in rebar, which has to work on anything from R13B03 to > > 18.0. > > https://github.com/rebar/rebar/blob/6cc18c931c/src/rebar_utils.erl#L75: > > > > -dialyzer({no_missing_calls, escript_foldl/3}). > > > > This tells Dialyzer to ignore calls to unknown functions when > > analyzing escript_foldl/3. > > > > Does this work for you? > > Thanks for your reply and sorry for my late message. > > The suppression I was mentioning regarded a call from a plain module > M1 to another module M2 that is generated at runtime > (compile:form/2) and then loaded (code:load_binary/3): Dialyzer then > sees in M1 an actual call to a module M2 that it cannot find (no > source nor BEAM file, for obvious reasons). Are you saying that (1) because the module doesn't exist as a .beam file Dialyzer doesn't know of its existence (2) and there's no way to make the ephemeral/volatile module known to Dialyzer as part of the analysis call ? > In this case I do not think we can tell Dialyzer to suppress this > warning (since the -dialyzer attribute seems to be meant to be used > in the sources *of the module itself*, i.e. it would have here to be > specified in the sourceless M2). > > This is not a big issue (and these are edges cases indeed), but it > would be interesting if we could define 'global' suppressions, or > that would apply to a set of modules (similarly, Dialyzer would not > be able to see that conditional calls could be done based on the > actual availability of the exported function - using module_info/1 > for that; Dialyzer would then detect a potential call to a > non-existing function, it would be neat if we could specify that > these warnings should be suppressed as we know that this cannot > happen). From olivier.boudeville@REDACTED Fri Jul 17 18:01:52 2015 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Fri, 17 Jul 2015 18:01:52 +0200 Subject: [erlang-questions] A few type-related 18.0 questions In-Reply-To: References: Message-ID: Hello Tuncer, [...] > Are you saying that > > (1) because the module doesn't exist as a .beam file Dialyzer doesn't > know of its existence In this particular case, the generated module exists neither as a source file (.erl) nor as a compiled one (.beam), as it exists only at runtime and in-memory - so I do not think Dialyzer has any chance to be aware of it and of its actual content (exports)? > > (2) and there's no way to make the ephemeral/volatile module known to > Dialyzer as part of the analysis call > ? Ah, maybe there is such a solution that I am not aware of, but apart from running a Dialyzer attached to the application (a bit like an attached gdb), I do not think Dialyzer can do much there. So I was looking for a way to silence it whenever it saw calls from anywhere in the code to any function defined in these pseudo-modules. To clarify the use case, this pseudo-module generation serves to create efficient look-up tables of immutable data that is known only at runtime (ex: after having read a data file). Have a nice week-end! Best, Olivier. > > In this case I do not think we can tell Dialyzer to suppress this > > warning (since the -dialyzer attribute seems to be meant to be used > > in the sources *of the module itself*, i.e. it would have here to be > > specified in the sourceless M2). > > > > This is not a big issue (and these are edges cases indeed), but it > > would be interesting if we could define 'global' suppressions, or > > that would apply to a set of modules (similarly, Dialyzer would not > > be able to see that conditional calls could be done based on the > > actual availability of the exported function - using module_info/1 > > for that; Dialyzer would then detect a potential call to a > > non-existing function, it would be neat if we could specify that > > these warnings should be suppressed as we know that this cannot > > happen). --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Fri Jul 17 19:00:04 2015 From: comptekki@REDACTED (Wes James) Date: Fri, 17 Jul 2015 11:00:04 -0600 Subject: [erlang-questions] Very slow download for otp 17.5 src Message-ID: I'm trying to download the 17.5 erlang sorce. wget says 4 days???? Anyone else have the 17.5 source? I looked on erlang solutions, but I could only see 18 install packages. Thanks, -wes -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Fri Jul 17 20:42:06 2015 From: felixgallo@REDACTED (Felix Gallo) Date: Fri, 17 Jul 2015 11:42:06 -0700 Subject: [erlang-questions] Very slow download for otp 17.5 src In-Reply-To: References: Message-ID: If you have git access, you could try git clone https://github.com/erlang/otp.git cd otp git checkout OTP-17.5.6.2 On Fri, Jul 17, 2015 at 10:00 AM, Wes James wrote: > I'm trying to download the 17.5 erlang sorce. wget says 4 days???? > Anyone else have the 17.5 source? I looked on erlang solutions, but I > could only see 18 install packages. > > Thanks, > > -wes > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Sat Jul 18 11:40:40 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 18 Jul 2015 11:40:40 +0200 Subject: [erlang-questions] A few type-related 18.0 questions In-Reply-To: References: Message-ID: On Fri, Jul 17, 2015 at 6:01 PM, Olivier BOUDEVILLE wrote: > Hello Tuncer, > > [...] > > > Are you saying that > > > > (1) because the module doesn't exist as a .beam file Dialyzer > > doesn't know of its existence > > In this particular case, the generated module exists neither as a > source file (.erl) nor as a compiled one (.beam), as it exists only > at runtime and in-memory - so I do not think Dialyzer has any chance > to be aware of it and of its actual content (exports)? The official API operates only on files, so I can't find anything either. Your use case isn't out of the ordinary and very valid, and maybe you can save the file, but it can unnecessarily complicate your project's structure or build process. > The suppression I was mentioning regarded a call from a plain module > M1 to another module M2 that is generated at runtime > (compile:form/2) and then loaded (code:load_binary/3): Dialyzer then > sees in M1 an actual call to a module M2 that it cannot find (no > source nor BEAM file, for obvious reasons). 1. M1 exists as a file 2. M1 call M2:gimme_data/1, say in M1:do_stuff/3 3. M2 only exists at runtime without a corresponding file 4. The warning is about a call to M2:gimme_data/1 which is unknown because M2 doesn't exist (yet). Assuming I got this right, you would add -dialyzer({no_missing_calls, do_stuff/3}) in M1.erl, to avoid the warning. From tuncer.ayaz@REDACTED Sat Jul 18 11:44:51 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 18 Jul 2015 11:44:51 +0200 Subject: [erlang-questions] Very slow download for otp 17.5 src In-Reply-To: References: Message-ID: On Fri, Jul 17, 2015 at 7:00 PM, Wes James wrote: > I'm trying to download the 17.5 erlang sorce. wget says 4 days???? > Anyone else have the 17.5 source? I looked on erlang solutions, but > I could only see 18 install packages. https://github.com/erlang/otp/archive/OTP-17.5.6.2.tar.gz kerl should use this as well, as it's broken since 17.0, because it only sees what's been published as a full release with Windows installer in erlang.org/download (by regex matching the dir listing). That means, you cannot get 17.2 or any of the minor versions with kerl. From icfp.publicity@REDACTED Sat Jul 18 18:46:23 2015 From: icfp.publicity@REDACTED (David Van Horn) Date: Sat, 18 Jul 2015 12:46:23 -0400 Subject: [erlang-questions] ICFP 2015 Call for Participation Message-ID: [ Early registration ends 3 August. ] ===================================================================== Call for Participation ICFP 2015 20th ACM SIGPLAN International Conference on Functional Programming and affiliated events August 30 - September 5, 2015 Vancouver, British Columbia, Canada http://icfpconference.org/icfp2015/ ===================================================================== ICFP provides a forum for researchers and developers to hear about the latest work on the design, implementations, principles, and uses of functional programming. The conference covers the entire spectrum of work, from practice to theory, including its peripheries. A full week dedicated to functional programming: 1 conference, 1 symposium, 11 workshops, tutorials, programming contest results, student research competition, and mentoring workshop * Program: http://icfpconference.org/icfp2015/program.html * Accepted Papers: http://icfpconference.org/icfp2015/accepted.html * Affiliated Events: http://icfpconference.org/icfp2015/affiliated.html * Local arrangements (including travel and accommodation): http://icfpconference.org/icfp2015/local.html * Registration is available via: https://regmaster4.com/2015conf/ICFP15/register.php Early registration is due 3 August, 2015. * Programming contest, 7-10 August, 2015: http://icfpcontest.org/ * Follow @icfp_conference on twitter for the latest news: http://twitter.com/icfp_conference There are several events affiliated with ICFP: Sunday, August 30 Haskell Implementors Workshop Workshop on Higher-order Programming with Effects Workshop on Generic Programming Programming Languages Mentoring Workshop Ally Skills Tutorial Monday, August 31 ? Wednesday, September 2 ICFP Thursday, September 3 Workshop on Functional High-Performance Computing Haskell Symposium ? Day 1 ML Family Workshop Commercial Users of Functional Programming ? Day 1 Friday, September 4 Erlang Workshop Haskell Symposium ? Day 2 OCaml Workshop Commercial Users of Functional Programming ? Day 2 Scheme and Functional Programming Workshop Saturday, September 5 Functional Art, Music, Modeling and Design Commercial Users of Functional Programming ? Day 3 Conference Organizers General Chair: Kathleen Fisher, Tufts University Program Chair: John Reppy, University of Chicago Local Arrangements Chair: Ronald Garcia, University of British Columbia Industrial Relations Chair: Anil Madhavapeddy, University of Cambridge Workshop Co-Chairs: Tom Schrijvers, KU Leuven Nicolas Wu, University of Bristol Programming Contest Chair: Joe Kiniry, Galois Student Research Competition Chair: Andrew Kennedy, Microsoft Research Mentoring Workshop Co-Chairs: Ronald Garcia, University of British Columbia Stephanie Weirich, University of Pennsylvania Publicity Chair: David Van Horn, University of Maryland Video Chair: Iavor Diatchki, Galois Student Volunteer Co-Chairs: Felipe Ba?ados Schwerter, University of British Columbia Gabriel Scherer, INRIA Mobile App Chair: Reid Holmes, University of Waterloo Industrial partners: Platinum partners Jane Street Capital Gold partners Anonymous donor Ahrefs Google Mozilla Research Oracle Labs Silver partners Bloomberg Tsuru Capital Galois The University of Chicago Bronze partners Erlang Solutions FireEye IntelliFactory PivotCloud Systor Vest ===================================================================== From benhsu@REDACTED Sat Jul 18 23:44:27 2015 From: benhsu@REDACTED (Ben Hsu) Date: Sat, 18 Jul 2015 17:44:27 -0400 Subject: [erlang-questions] running trace (or similar) Message-ID: Hello I am learning erlang by playing with a small webapp. The webapp is written in Cowboy with erlang.mk "make" creates a binary in _rel and I can use this binary to start a console. ./_rel/fiar/bin/fiar console I would like to debug this app using something like dbg:trace. dbg:trace doesn't work from the console, but dbg:tracer doesnt seem to work: (fiar@REDACTED)1> dbg:tracer(). ** exception error: undefined function dbg:tracer/0 1: how can I make dbg:tracer() work? 2: are there any other tools to look into the running app? What do you recommend? Thank you for helping me learn Erlang [1] In case you're wondering, its FIAR : https://github.com/inaka/fiar -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Sun Jul 19 20:37:05 2015 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Sun, 19 Jul 2015 11:37:05 -0700 Subject: [erlang-questions] running trace (or similar) In-Reply-To: References: Message-ID: <55ABEE51.2080002@gmail.com> On 07/18/2015 02:44 PM, Ben Hsu wrote: > > Hello > > I am learning erlang by playing with a small webapp. The webapp is > written in Cowboy with erlang.mk > > "make" creates a binary in _rel and I can use this binary to start a > console. > > ./_rel/fiar/bin/fiar console I can't get fiar to build because of erlang.mk's insistence that warnings be treated as errors, so I can't help you with your *real* problem. (erlang.mk devs: Any chance that this, and erlang.mk's utter inability to build with parallel make will ever get fixed? :) ) All I can tell you is that dbg:tracer/0 works for me in a newly started erl shell: Erlang/OTP 18 [erts-7.0] [source] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] Eshell V7.0 (abort with ^G) 1> dbg:tracer(). {ok,<0.36.0>} 2> Taking a wild guess: what if you added runtime_tools to the list of applications in src/fiar.app.src and then rebuilt? Does dbg:tracer/0 work in your fiar shell now? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From fernando.benavides@REDACTED Sun Jul 19 21:05:13 2015 From: fernando.benavides@REDACTED (Fernando 'Brujo' Benavides) Date: Sun, 19 Jul 2015 16:05:13 -0300 Subject: [erlang-questions] running trace (or similar) In-Reply-To: <55ABEE51.2080002@gmail.com> References: <55ABEE51.2080002@gmail.com> Message-ID: And, for development purposes, you can always start your app using make shell, instead of building a release and running it. Fernando "Brujo" Benavides about.me/elbrujohalcon > On Jul 19, 2015, at 15:37, Kenneth Lakin wrote: > > On 07/18/2015 02:44 PM, Ben Hsu wrote: >> >> Hello >> >> I am learning erlang by playing with a small webapp. The webapp is >> written in Cowboy with erlang.mk >> >> "make" creates a binary in _rel and I can use this binary to start a >> console. >> >> ./_rel/fiar/bin/fiar console > > I can't get fiar to build because of erlang.mk's insistence that > warnings be treated as errors, so I can't help you with your *real* > problem. (erlang.mk devs: Any chance that this, and erlang.mk's utter > inability to build with parallel make will ever get fixed? :) ) > > All I can tell you is that dbg:tracer/0 works for me in a newly started > erl shell: > > Erlang/OTP 18 [erts-7.0] [source] [smp:2:2] [async-threads:10] [hipe] > [kernel-poll:false] > > Eshell V7.0 (abort with ^G) > 1> dbg:tracer(). > {ok,<0.36.0>} > 2> > > Taking a wild guess: what if you added runtime_tools to the list of > applications in src/fiar.app.src and then rebuilt? Does dbg:tracer/0 > work in your fiar shell now? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vasdeveloper@REDACTED Sun Jul 19 21:54:31 2015 From: vasdeveloper@REDACTED (Kannan) Date: Mon, 20 Jul 2015 01:24:31 +0530 Subject: [erlang-questions] Clarification regarding improvements suggested by rr@whatsapp Message-ID: Team, Is there any chance Erlang/OTP team is considering the patches done by rr@REDACTED? At least that are related to Mnesia and async thread head line blocking for file io? Thanks, Theepan -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sun Jul 19 23:10:36 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 20 Jul 2015 00:10:36 +0300 Subject: [erlang-questions] Clarification regarding improvements suggested by rr@whatsapp In-Reply-To: References: Message-ID: What is a "async thread head line blocking" ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Mon Jul 20 09:21:13 2015 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 20 Jul 2015 09:21:13 +0200 Subject: [erlang-questions] Tricky Dialyzer type Message-ID: <20150720072113.GA7865@erix.ericsson.se> Hi all. I need to specify a tricky type for Dialyzer (because all other options as far as I can see are exhausted): a list with a type tag as first element, and then the rest of the elements of one specific type. Something like this where I use the atom 'uint8' as type tag for the type uint8(): -type uint8() :: 0..255. -type tagged_list_uint8() :: cons('uint8', [uint8()]). But there is no cons() type in the type language what I can see. I have tried with "list('uint8', uint8())", "[uint8, uint8()]" and those are also syntactically invalid. I find no support for this in the type language reference. The closest I have come is: -type tagged_list_uint8() :: nonempty_improper_list('uint8', [uint8()]). But that would as I understand it allow for any number >= 1 of tag atoms followed by uint8() elements. I want to allow exactly one tag atom! Ideas, anyone? -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From jesper.louis.andersen@REDACTED Mon Jul 20 10:06:32 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 20 Jul 2015 10:06:32 +0200 Subject: [erlang-questions] Tricky Dialyzer type In-Reply-To: <20150720072113.GA7865@erix.ericsson.se> References: <20150720072113.GA7865@erix.ericsson.se> Message-ID: On Mon, Jul 20, 2015 at 9:21 AM, Raimo Niskanen < raimo+erlang-questions@REDACTED> wrote: > Ideas, anyone? Your representation is icky at the type level. If possible, go for a tuple instead since it has type A*B as a product and then this is fairly easy to represent: -type tagged_list(A) :: {A, [A]}. This also opens up the possibility of sum-typing the tagged list: -type tagged_list() :: {tag_a, [..]} | {tag_b, [..]}. which in turn can hammer down the precision at which the dialyzer can work. In naive type systems, list are homogeneous in the sense that every element has to be drawn from the same ground type. Properly handling lists where the "first element drives the type of the remainder of the list" is nudging itself towards the land of dependent type theory, which the dialyzer definitely doesn't support. It may be possible to work with the notion of "list of two elements", [uint8(), ty(), ...], saying that the list is a cons of uint8() followed by one or more elements of type ty(), but I'm not sure if the dialyzer understands this at all. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From olivier.boudeville@REDACTED Mon Jul 20 11:19:37 2015 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Mon, 20 Jul 2015 11:19:37 +0200 Subject: [erlang-questions] A few type-related 18.0 questions In-Reply-To: References: Message-ID: Hello Tuncer, > 1. M1 exists as a file > > 2. M1 call M2:gimme_data/1, say in M1:do_stuff/3 > > 3. M2 only exists at runtime without a corresponding file > > 4. The warning is about a call to M2:gimme_data/1 which is > unknown because M2 doesn't exist (yet). > > Assuming I got this right, you would add > > -dialyzer({no_missing_calls, do_stuff/3}) > > in M1.erl, to avoid the warning. Indeed, you got it right, however, unless I made clumsy mistakes, in M1.erl (using your example functions), no combination of: -dialyzer( { no_missing_calls, do_stuff/3 } ). and/or -dialyzer( { nowarn_function, do_stuff/3 } ). prevents Dialyzer from telling that functions like M2:gimme_data/1 are unknown. I was thinking that functions specified in the -dialyzer attribute are the callee ones, not the caller ones (that's why I wanted to see whether we could specify a module for them, if they were not M1-local). Anyway, not a big problem. Maybe in the future separate suppression files (listing the warnings to be suppressed regardless of the call site) will be supported (supposing it is not supported yet). Thanks for your time! Best, Olivier. Olivier BOUDEVILLE EDF ? R&D D?partement SINETICS, Groupe ASICS, Bureau B-226 1, avenue du G?n?ral de Gaulle 92140 Clamart olivier.boudeville@REDACTED T?l. : +33 (0)1 47 65 59 58 T?l. mobile : +33 (0)6 16 83 37 22 Fax : +33 (0)1 47 65 27 13 Un geste simple pour l'environnement, n'imprimez ce message que si vous en avez l'utilit?. Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 1816 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 1151 bytes Desc: not available URL: From raimo+erlang-questions@REDACTED Mon Jul 20 11:49:42 2015 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 20 Jul 2015 11:49:42 +0200 Subject: [erlang-questions] Tricky Dialyzer type In-Reply-To: References: <20150720072113.GA7865@erix.ericsson.se> Message-ID: <20150720094942.GA22261@erix.ericsson.se> On Mon, Jul 20, 2015 at 10:06:32AM +0200, Jesper Louis Andersen wrote: > On Mon, Jul 20, 2015 at 9:21 AM, Raimo Niskanen < > raimo+erlang-questions@REDACTED> wrote: > > > Ideas, anyone? > > > Your representation is icky at the type level. If possible, go for a tuple > instead since it has type A*B as a product and then this is fairly easy to > represent: Unfortunately that is not possible since the tagged tuple already has another meaning in the code I am trying to add this feature to. This was the "exhausted option" I briefly mentioned. I should have stated that explicitly in my question... > > -type tagged_list(A) :: {A, [A]}. > > This also opens up the possibility of sum-typing the tagged list: > > -type tagged_list() :: > {tag_a, [..]} | {tag_b, [..]}. > > which in turn can hammer down the precision at which the dialyzer can work. > > In naive type systems, list are homogeneous in the sense that every element > has to be drawn from the same ground type. Properly handling lists where > the "first element drives the type of the remainder of the list" is nudging > itself towards the land of dependent type theory, which the dialyzer > definitely doesn't support. That's a pity. For me it is just about two different kinds of wrappers. Obviously I am not a type expert. > > It may be possible to work with the notion of "list of two elements", > [uint8(), ty(), ...], saying that the list is a cons of uint8() followed by > one or more elements of type ty(), but I'm not sure if the dialyzer > understands this at all. At least list(type(), type()) and [type(), type()] both gives syntax errors so it seams that Dialyzer does not understand that at all. > > > > -- > J. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From max.lapshin@REDACTED Mon Jul 20 13:24:33 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 20 Jul 2015 14:24:33 +0300 Subject: [erlang-questions] gen_tcp send_timeout and memory leak caveats Message-ID: Hi. Seems that I met rather rare situation when send_timeout doesn't help from memory leak. I have an architecture, when central process send messages with big binary blobs to subscribed processes via plain ! Each process takes messages and tries to write to socket which is configured with {send_timeout, 10000}. If socket replies with {error, timeout} then process is closed and no more messages are received. This architecture can work only if gen_tcp:send will 100% return after 10, maybe 15 or at least 60 seconds. After customer has masterly configured bonding on 4 ethernet ports and cut 2 of 4 cables, we've got rare situation with 50% of packet loss. It lead to very interesting situation when plenty of processes are locked in prim_inet send for many minutes. gen_tcp:send doesn't allow to pass nosuspend option to port_command and thus it is impossible to send data to socket with real timeout. If tcp socket is blocked, erlang:port_command will be blocked for a veeeryy long time. I had to refuse from gen_tcp to port_command and enable add missing timeout there: send(Socket, Data) -> try erlang:port_command(Socket, Data, [nosuspend]) of false -> {error, busy}; true -> receive {inet_reply, Socket, Status} -> Status after 20000 -> {error, timeout} end catch error:Error -> {error, Error} end. Another way to handle it is to launch a process that will scan erlang:processes() and check their message_queue_len. Very "elegant" and "good designed" solution, but it will work. What is the proper way to handle such situation? And yes, I cannot use gen_server:call from central process to clients, because I need it to scale to 2000-3000 secondary processes. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Mon Jul 20 15:19:02 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 20 Jul 2015 15:19:02 +0200 Subject: [erlang-questions] Tricky Dialyzer type In-Reply-To: <20150720094942.GA22261@erix.ericsson.se> References: <20150720072113.GA7865@erix.ericsson.se> <20150720094942.GA22261@erix.ericsson.se> Message-ID: On Mon, Jul 20, 2015 at 11:49 AM, Raimo Niskanen < raimo+erlang-questions@REDACTED> wrote: > That's a pity. For me it is just about two different kinds of wrappers. > Obviously I am not a type expert. > The reason it is somewhat uncommon to want a list in a programming language with type systems is that you can't get it to compile if the type check fails. So in such a language, you would have been forced into the product type with a tuple {uint8(), [ty()]} as the only way to represent this. The wrong program would never have existed. Hence it would not be a problem. Digging further, you eventually hit granite in the sense that you do need a heterogenous list type. When that happens you are either out of luck, or the type system is expressive enough to let you model it. It is, generally, hard to graft a type system unto old code. You may have to fall back to say that your data is [any()]. It is unsatisfactory, but at least it is correct. I think that if you ask the 'typer' program, you get the most specific type in the type lattice which dialyzer can figure out from looking at the code base. If you use this type, you are at least not incorrect, and you are at the limit of what the dialyzer understands, currently. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Mon Jul 20 18:56:23 2015 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 20 Jul 2015 18:56:23 +0200 Subject: [erlang-questions] Tricky Dialyzer type In-Reply-To: References: <20150720072113.GA7865@erix.ericsson.se> <20150720094942.GA22261@erix.ericsson.se> Message-ID: <20150720165623.GA8816@erix.ericsson.se> On Mon, Jul 20, 2015 at 03:19:02PM +0200, Jesper Louis Andersen wrote: > On Mon, Jul 20, 2015 at 11:49 AM, Raimo Niskanen < > raimo+erlang-questions@REDACTED> wrote: > > > That's a pity. For me it is just about two different kinds of wrappers. > > Obviously I am not a type expert. > > > > The reason it is somewhat uncommon to want a list in a programming language > with type systems is that you can't get it to compile if the type check > fails. So in such a language, you would have been forced into the product > type with a tuple {uint8(), [ty()]} as the only way to represent this. The > wrong program would never have existed. Hence it would not be a problem. > > Digging further, you eventually hit granite in the sense that you do need a > heterogenous list type. When that happens you are either out of luck, or > the type system is expressive enough to let you model it. > > It is, generally, hard to graft a type system unto old code. You may have > to fall back to say that your data is [any()]. It is unsatisfactory, but at > least it is correct. I think that if you ask the 'typer' program, you get > the most specific type in the type lattice which dialyzer can figure out > from looking at the code base. If you use this type, you are at least not > incorrect, and you are at the limit of what the dialyzer understands, > currently. Ok. Thank you for your time. I now know a bit more than before. Best Regards / Raimo > > > > > -- > J. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From tuncer.ayaz@REDACTED Mon Jul 20 20:13:59 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 20 Jul 2015 20:13:59 +0200 Subject: [erlang-questions] A few type-related 18.0 questions In-Reply-To: References: Message-ID: On Mon, Jul 20, 2015 at 11:19 AM, Olivier BOUDEVILLE wrote: > > Hello Tuncer, > > > 1. M1 exists as a file > > > > 2. M1 call M2:gimme_data/1, say in M1:do_stuff/3 > > > > 3. M2 only exists at runtime without a corresponding file > > > > 4. The warning is about a call to M2:gimme_data/1 which is > > unknown because M2 doesn't exist (yet). > > > > Assuming I got this right, you would add > > > > -dialyzer({no_missing_calls, do_stuff/3}) > > > > in M1.erl, to avoid the warning. > > Indeed, you got it right, however, unless I made clumsy mistakes, in > M1.erl (using your example functions), no combination of: > > -dialyzer( { no_missing_calls, do_stuff/3 } ). > > and/or > > -dialyzer( { nowarn_function, do_stuff/3 } ). > > prevents Dialyzer from telling that functions like M2:gimme_data/1 > are unknown. I haven't checked the implementation, but maybe with M2.beam non-existent, {no_missing_calls, do_stuff/3} makes no difference. That would explain why it works for rebar_utils and not M1. > I was thinking that functions specified in the -dialyzer attribute > are the callee ones, not the caller ones (that's why I wanted to see > whether we could specify a module for them, if they were not > M1-local). {no_missing_calls, do_stuff/3} tells Dialyzer to not raise a missing function warning for any call within do_stuff/3. > Anyway, not a big problem. Maybe in the future separate suppression > files (listing the warnings to be suppressed regardless of the call > site) will be supported (supposing it is not supported yet). > > Thanks for your time! From eriksoe@REDACTED Mon Jul 20 20:49:10 2015 From: eriksoe@REDACTED (=?UTF-8?Q?Erik_S=C3=B8e_S=C3=B8rensen?=) Date: Mon, 20 Jul 2015 20:49:10 +0200 Subject: [erlang-questions] Tricky Dialyzer type In-Reply-To: <20150720094942.GA22261@erix.ericsson.se> References: <20150720072113.GA7865@erix.ericsson.se> <20150720094942.GA22261@erix.ericsson.se> Message-ID: Depending on exactly what tagged tuples have already been assigned meaning, there may be other options. If only 2-tuples are "occupied", you can use {magic, Tag, List}. If only tuples with an initial atom are "occupied", you can use {{Tag}, List}. Are all tuple types indeed spoken for? On the other hand, it sounds like all list types are available. In that case, a term like [{Tag, List}] might work better - it has different trade-offs, at least, with respect to type analysis. /Erik Den 20/07/2015 11.49 skrev "Raimo Niskanen" < raimo+erlang-questions@REDACTED>: > On Mon, Jul 20, 2015 at 10:06:32AM +0200, Jesper Louis Andersen wrote: > > On Mon, Jul 20, 2015 at 9:21 AM, Raimo Niskanen < > > raimo+erlang-questions@REDACTED> wrote: > > > > > Ideas, anyone? > > > > > > Your representation is icky at the type level. If possible, go for a > tuple > > instead since it has type A*B as a product and then this is fairly easy > to > > represent: > > Unfortunately that is not possible since the tagged tuple already has > another meaning in the code I am trying to add this feature to. This was > the "exhausted option" I briefly mentioned. I should have stated that > explicitly in my question... > > > > > -type tagged_list(A) :: {A, [A]}. > > > > This also opens up the possibility of sum-typing the tagged list: > > > > -type tagged_list() :: > > {tag_a, [..]} | {tag_b, [..]}. > > > > which in turn can hammer down the precision at which the dialyzer can > work. > > > > In naive type systems, list are homogeneous in the sense that every > element > > has to be drawn from the same ground type. Properly handling lists where > > the "first element drives the type of the remainder of the list" is > nudging > > itself towards the land of dependent type theory, which the dialyzer > > definitely doesn't support. > > That's a pity. For me it is just about two different kinds of wrappers. > Obviously I am not a type expert. > > > > > It may be possible to work with the notion of "list of two elements", > > [uint8(), ty(), ...], saying that the list is a cons of uint8() followed > by > > one or more elements of type ty(), but I'm not sure if the dialyzer > > understands this at all. > > At least list(type(), type()) and [type(), type()] both gives syntax errors > so it seams that Dialyzer does not understand that at all. > > > > > > > > > -- > > J. > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From w@REDACTED Tue Jul 21 06:00:04 2015 From: w@REDACTED (Hugo Wang) Date: Tue, 21 Jul 2015 12:00:04 +0800 Subject: [erlang-questions] How to Update List Elements A Lot Message-ID: Hi all, I'm writing an encrypt/decrypt stuff in Erlang, the input and output are both lists (or binary). In other words, I need to generate a list of bytes based on another list of bytes. During the generating there are a lot of band, bor, bxor, bsl, bsr, etc operations. Every element in the new list (the output) would be updated many times. In other language, like C or Python, we can init an output list and then update its elements inline. In Erlang, what I currently do, would make a copy of the list every time when an element need to update. This looks not quite right. Any comments/ideas on this? I want to use pure Erlang to implement it, rather than making a C port. Thanks, Hugo -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Jul 21 09:55:28 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 21 Jul 2015 09:55:28 +0200 Subject: [erlang-questions] How to Update List Elements A Lot In-Reply-To: References: Message-ID: On Tue, Jul 21, 2015 at 6:00 AM, Hugo Wang wrote: > In other language, like C or Python, we can init an output list and then > update its elements inline. In Erlang, what I currently do, would make a > copy of the list every time when an element need to update. This looks not > quite right. > Do you have a small snippet of an inner loop somewhere? It may be easier to recommend an approach then. Do note that crypto-stuff in Erlang is perhaps not the best approach since the complexity of the VM makes it harder to account for side channels. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierrefenoll@REDACTED Tue Jul 21 12:17:57 2015 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Tue, 21 Jul 2015 19:17:57 +0900 Subject: [erlang-questions] How to Update List Elements A Lot In-Reply-To: References: Message-ID: <77959D8F-B594-4F94-9A51-FB5474A8550C@gmail.com> The process dictionary is the cheapest mutable structure in erlang. Have a try at it. Maybe a NIF handling a fixed size mutable tuple will do too.. No idea! > On 21 Jul 2015, at 16:55, Jesper Louis Andersen wrote: > > >> On Tue, Jul 21, 2015 at 6:00 AM, Hugo Wang wrote: >> In other language, like C or Python, we can init an output list and then update its elements inline. In Erlang, what I currently do, would make a copy of the list every time when an element need to update. This looks not quite right. > > Do you have a small snippet of an inner loop somewhere? It may be easier to recommend an approach then. Do note that crypto-stuff in Erlang is perhaps not the best approach since the complexity of the VM makes it harder to account for side channels. > > > -- > J. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From weoccc@REDACTED Tue Jul 21 07:48:47 2015 From: weoccc@REDACTED (weoccc) Date: Mon, 20 Jul 2015 22:48:47 -0700 Subject: [erlang-questions] running debugger fails Message-ID: Hi, I am new to erlang and I am running mongooseIM app and want to do some debugging on local server. I tried to follow the tutorial as show here : http://www.erlang.org/doc/apps/debugger/debugger_chapter.html but get some error: Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:8:8] [async-threads:10] [kernel-poll:false] Eshell V6.2 (abort with ^G) 1> debugger.start(). * 1: syntax error before: '.' 1> debugger:start(). {error,{undef,[{wx,new,[],[]}, {dbg_wx_mon,init,3,[{file,"dbg_wx_mon.erl"},{line,113}]}]}} Any idea how to fix those ? Thx, Weide -------------- next part -------------- An HTML attachment was scrubbed... URL: From gguthrie@REDACTED Tue Jul 21 13:43:57 2015 From: gguthrie@REDACTED (Gordon Guthrie) Date: Tue, 21 Jul 2015 12:43:57 +0100 Subject: [erlang-questions] running debugger fails In-Reply-To: References: Message-ID: Weide The syntax is: debugger:start(). debugger colon start () full stop debbuger:start means the function start exported from the module debugger Gordon > Le 21 juil. 2015 ? 06:48, weoccc a ?crit : > > Hi, > > I am new to erlang and I am running mongooseIM app and want to do some debugging on local server. > > I tried to follow the tutorial as show here : http://www.erlang.org/doc/apps/debugger/debugger_chapter.html > > but get some error: > > Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:8:8] [async-threads:10] [kernel-poll:false] > > Eshell V6.2 (abort with ^G) > 1> debugger.start(). > * 1: syntax error before: '.' > 1> debugger:start(). > {error,{undef,[{wx,new,[],[]}, > {dbg_wx_mon,init,3,[{file,"dbg_wx_mon.erl"},{line,113}]}]}} > > Any idea how to fix those ? > > Thx, > > Weide > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgolovan@REDACTED Tue Jul 21 13:52:15 2015 From: sgolovan@REDACTED (Sergei Golovan) Date: Tue, 21 Jul 2015 14:52:15 +0300 Subject: [erlang-questions] running debugger fails In-Reply-To: References: Message-ID: Hi! On Tue, Jul 21, 2015 at 8:48 AM, weoccc wrote: > 1> debugger:start(). > {error,{undef,[{wx,new,[],[]}, > {dbg_wx_mon,init,3,[{file,"dbg_wx_mon.erl"},{line,113}]}]}} > > Any idea how to fix those ? This likely means that you Erlang installation misses the Erlang wx application. I know that Erlang package prebuilt for Ubuntu Linux don't include wx. How did you install your Erlang? Cheers! -- Sergei Golovan From essen@REDACTED Tue Jul 21 16:13:42 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 21 Jul 2015 16:13:42 +0200 Subject: [erlang-questions] running trace (or similar) In-Reply-To: References: Message-ID: <55AE5396.4000802@ninenines.eu> On 07/18/2015 11:44 PM, Ben Hsu wrote: > > Hello > > I am learning erlang by playing with a small webapp. The webapp is > written in Cowboy with erlang.mk > > "make" creates a binary in _rel and I can use this binary to start a > console. > > ./_rel/fiar/bin/fiar console > > I would like to debug this app using something like dbg:trace. dbg:trace > doesn't work from the console, but dbg:tracer doesnt seem to work: > > (fiar@REDACTED )1> dbg:tracer(). > ** exception error: undefined function dbg:tracer/0 > > 1: how can I make dbg:tracer() work? You need to add 'runtime_tools' to the list of applications in your release configuration in relx.config (next to your own application). > 2: are there any other tools to look into the running app? What do you > recommend? Recon is probably a good idea for a production system. You can also add observer there, if you are a beginner to Erlang. > Thank you for helping me learn Erlang > > [1] In case you're wondering, its FIAR : https://github.com/inaka/fiar -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From essen@REDACTED Tue Jul 21 16:19:49 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 21 Jul 2015 16:19:49 +0200 Subject: [erlang-questions] running trace (or similar) In-Reply-To: <55ABEE51.2080002@gmail.com> References: <55ABEE51.2080002@gmail.com> Message-ID: <55AE5505.5000405@ninenines.eu> On 07/19/2015 08:37 PM, Kenneth Lakin wrote: > I can't get fiar to build because of erlang.mk's insistence that > warnings be treated as errors, so I can't help you with your *real* > problem. (erlang.mk devs: Any chance that this, and erlang.mk's utter > inability to build with parallel make will ever get fixed? :) ) About warnings as errors: this is a bug, it shouldn't be enabled for deps. I have a local patch I need to push that fix it when the dependency is an erlang.mk project. The option is automatically removed from rebar projects when used as deps already. About parallel make: it's complicated. There is some support for it though, so if you run into issues I really need to know what they are. Do open a ticket. There are a few things that aren't done in parallel today, but will be improved as part of the work I am doing to support Windows, mostly related to handling of dependencies. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From raimo+erlang-questions@REDACTED Tue Jul 21 16:22:36 2015 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 21 Jul 2015 16:22:36 +0200 Subject: [erlang-questions] Tricky Dialyzer type In-Reply-To: References: <20150720072113.GA7865@erix.ericsson.se> <20150720094942.GA22261@erix.ericsson.se> Message-ID: <20150721142236.GA5699@erix.ericsson.se> On Mon, Jul 20, 2015 at 08:49:10PM +0200, Erik S?e S?rensen wrote: > Depending on exactly what tagged tuples have already been assigned meaning, > there may be other options. > If only 2-tuples are "occupied", you can use {magic, Tag, List}. > If only tuples with an initial atom are "occupied", you can use {{Tag}, > List}. > Are all tuple types indeed spoken for? > On the other hand, it sounds like all list types are available. In that > case, a term like [{Tag, List}] might work better - it has different > trade-offs, at least, with respect to type analysis. Creative ideas, but [{Tag,List}] is already spoken for, and I thought {magic, Tag, List} was ugly as well as {{Tag}, List}. {Tag,undefined} is not spoken for and that is my best plan B. But now I have: [{Tag,Val}, {Tag,OtherVal}, ...] for a sequence with size >= 1 note that all Tags have to be equal or else there is an error [] for No Value i.e nonexistent {Tag,undefined} for a sequence with size 0 Ugly and inconsistent. With the tagged list: [Tag|[Val,...]] for a sequence with size >= 1 [Tag] for a sequence with size == 0 [] for No Value i.e nonexistent. which I think is much more consistent. / Raimo > > /Erik > Den 20/07/2015 11.49 skrev "Raimo Niskanen" < > raimo+erlang-questions@REDACTED>: > > > On Mon, Jul 20, 2015 at 10:06:32AM +0200, Jesper Louis Andersen wrote: > > > On Mon, Jul 20, 2015 at 9:21 AM, Raimo Niskanen < > > > raimo+erlang-questions@REDACTED> wrote: > > > > > > > Ideas, anyone? > > > > > > > > > Your representation is icky at the type level. If possible, go for a > > tuple > > > instead since it has type A*B as a product and then this is fairly easy > > to > > > represent: > > > > Unfortunately that is not possible since the tagged tuple already has > > another meaning in the code I am trying to add this feature to. This was > > the "exhausted option" I briefly mentioned. I should have stated that > > explicitly in my question... > > > > > > > > -type tagged_list(A) :: {A, [A]}. > > > > > > This also opens up the possibility of sum-typing the tagged list: > > > > > > -type tagged_list() :: > > > {tag_a, [..]} | {tag_b, [..]}. > > > > > > which in turn can hammer down the precision at which the dialyzer can > > work. > > > > > > In naive type systems, list are homogeneous in the sense that every > > element > > > has to be drawn from the same ground type. Properly handling lists where > > > the "first element drives the type of the remainder of the list" is > > nudging > > > itself towards the land of dependent type theory, which the dialyzer > > > definitely doesn't support. > > > > That's a pity. For me it is just about two different kinds of wrappers. > > Obviously I am not a type expert. > > > > > > > > It may be possible to work with the notion of "list of two elements", > > > [uint8(), ty(), ...], saying that the list is a cons of uint8() followed > > by > > > one or more elements of type ty(), but I'm not sure if the dialyzer > > > understands this at all. > > > > At least list(type(), type()) and [type(), type()] both gives syntax errors > > so it seams that Dialyzer does not understand that at all. > > > > > > > > > > > > > > -- > > > J. > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From w@REDACTED Tue Jul 21 14:46:34 2015 From: w@REDACTED (Hugo Wang) Date: Tue, 21 Jul 2015 20:46:34 +0800 Subject: [erlang-questions] How to Update List Elements A Lot In-Reply-To: References: Message-ID: Good points! Thank you Vance. On Tue, Jul 21, 2015 at 7:15 PM, Vance Shipley wrote: > On Tue, Jul 21, 2015 at 9:30 AM, Hugo Wang wrote: > > In other language, like C or Python, we can init an output list and then > > update its elements inline. In Erlang, what I currently do, would make a > > copy of the list every time when an element need to update. This looks > not > > quite right. > > If the problem is that it doesn't feel right than there isn't a > problem yet. Welcome to functional programming. > > The Erlang way is 1) make it work, 2) make it correct, 3) optimize. > Most of the time you go straight to production around 2) and never get > to 3). > > If you are going to worry about code performance the only answer is > measure, measure, measure. See if it does matter before working on an > imagined problem. > > Immutability helps greatly with 2) and that really is the most > important thing (presuming you got past 1). > > -- > -Vance > -------------- next part -------------- An HTML attachment was scrubbed... URL: From w@REDACTED Tue Jul 21 14:49:57 2015 From: w@REDACTED (Hugo Wang) Date: Tue, 21 Jul 2015 20:49:57 +0800 Subject: [erlang-questions] How to Update List Elements A Lot In-Reply-To: <77959D8F-B594-4F94-9A51-FB5474A8550C@gmail.com> References: <77959D8F-B594-4F94-9A51-FB5474A8550C@gmail.com> Message-ID: The process dictionary sounds a good option. I will try it. Thanks for the suggestions! On Tue, Jul 21, 2015 at 6:17 PM, Pierre Fenoll wrote: > The process dictionary is the cheapest mutable structure in erlang. Have a > try at it. > > Maybe a NIF handling a fixed size mutable tuple will do too.. No idea! > > > > On 21 Jul 2015, at 16:55, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > > > On Tue, Jul 21, 2015 at 6:00 AM, Hugo Wang wrote: > >> In other language, like C or Python, we can init an output list and then >> update its elements inline. In Erlang, what I currently do, would make a >> copy of the list every time when an element need to update. This looks not >> quite right. >> > > Do you have a small snippet of an inner loop somewhere? It may be easier > to recommend an approach then. Do note that crypto-stuff in Erlang is > perhaps not the best approach since the complexity of the VM makes it > harder to account for side channels. > > > -- > J. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eriksoe@REDACTED Tue Jul 21 19:16:51 2015 From: eriksoe@REDACTED (=?UTF-8?Q?Erik_S=C3=B8e_S=C3=B8rensen?=) Date: Tue, 21 Jul 2015 19:16:51 +0200 Subject: [erlang-questions] How to Update List Elements A Lot In-Reply-To: References: Message-ID: Are the updates so scattered? If they are a series of front-to-back or back-to-front updates, then there's not much of a problem. Certain other fixed patterns (eg, swapping each pair of values, or in general replacing ..., X, Y,... with ..., f(X, Y), g(X, Y),... ) are easy too, with lists or binaries. The really hard case is when the sequence of indexes of the updates is not fixed. Purely functional data structures don't do such patterns too well, performance-wise - although, as has been pointed out by others, it may perform well enough for the purpose. The "ramfile" driver may be a solution, although last time I checked it wasn't too fast. /Erik Den 21/07/2015 09.56 skrev "Jesper Louis Andersen" < jesper.louis.andersen@REDACTED>: > > On Tue, Jul 21, 2015 at 6:00 AM, Hugo Wang wrote: > >> In other language, like C or Python, we can init an output list and then >> update its elements inline. In Erlang, what I currently do, would make a >> copy of the list every time when an element need to update. This looks not >> quite right. >> > > Do you have a small snippet of an inner loop somewhere? It may be easier > to recommend an approach then. Do note that crypto-stuff in Erlang is > perhaps not the best approach since the complexity of the VM makes it > harder to account for side channels. > > > -- > J. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrallen1@REDACTED Tue Jul 21 22:57:33 2015 From: mrallen1@REDACTED (Mark Allen) Date: Tue, 21 Jul 2015 20:57:33 +0000 (UTC) Subject: [erlang-questions] Courtesy announcement: Lager 3.0 merge is nigh Message-ID: <1102186886.2132827.1437512253932.JavaMail.yahoo@mail.yahoo.com> Hello - this is a courtesy reminder that Real Soon Now (hopefully no later than Friday) the lager 3.0 work will be merged and tagged into the master branch on github.? While we have taken many pains to ensure that the 3.0 branch is mostly[1] backwards compatible with 2.x code, if you're "floating" on your dependencies by using the "master" branch in your rebar configuration (or other tooling) then you may want to consider additional testing to ensure nothing has broken vis a vis your application logging **especially** if you use tracing features of lager. If you want to continue using the 2.x semantics but with OTP 17 support and some bug fixes, consider using the 2.x maintenance branch or using the 2.2.0 tag in your build tool configuration. If you would like to preview your application behavior with lager 3.x code, please use the 3.0.0-RC2 tag. We expect that 3.0.0-RC2 will become '3.0.0' pending further code review activities. ? Thanks Mark [1]: Tracing has breaking changes. See?http://erlang.org/pipermail/erlang-questions/2015-June/084816.html?for context/information. -------------- next part -------------- An HTML attachment was scrubbed... URL: From w@REDACTED Wed Jul 22 03:01:48 2015 From: w@REDACTED (Hugo Wang) Date: Wed, 22 Jul 2015 09:01:48 +0800 Subject: [erlang-questions] How to Update List Elements A Lot In-Reply-To: References: Message-ID: Hi Jesper & Erik, The updates has inner loops and not that fixed patterned. The code will look like the following after a lot of simplicities: void decode(char * src, char * dst, ...) { ... hash_buffer(...); // random updates ... // other stuff has_buffer(...); ... } void hash_buffer(...) { ... shift_buffer(...); ... shift_buffer(...); ... } void shift_buffer(...) { ... // updating elements in dst } On Wed, Jul 22, 2015 at 1:16 AM, Erik S?e S?rensen wrote: > Are the updates so scattered? If they are a series of front-to-back or > back-to-front updates, then there's not much of a problem. Certain other > fixed patterns (eg, swapping each pair of values, or in general replacing > ..., X, Y,... with ..., f(X, Y), g(X, Y),... ) are easy too, with lists or > binaries. The really hard case is when the sequence of indexes of the > updates is not fixed. Purely functional data structures don't do such > patterns too well, performance-wise - although, as has been pointed out by > others, it may perform well enough for the purpose. > The "ramfile" driver may be a solution, although last time I checked it > wasn't too fast. > /Erik > Den 21/07/2015 09.56 skrev "Jesper Louis Andersen" < > jesper.louis.andersen@REDACTED>: > >> >> On Tue, Jul 21, 2015 at 6:00 AM, Hugo Wang wrote: >> >>> In other language, like C or Python, we can init an output list and then >>> update its elements inline. In Erlang, what I currently do, would make a >>> copy of the list every time when an element need to update. This looks not >>> quite right. >>> >> >> Do you have a small snippet of an inner loop somewhere? It may be easier >> to recommend an approach then. Do note that crypto-stuff in Erlang is >> perhaps not the best approach since the complexity of the VM makes it >> harder to account for side channels. >> >> >> -- >> J. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From weoccc@REDACTED Tue Jul 21 22:04:01 2015 From: weoccc@REDACTED (weoccc) Date: Tue, 21 Jul 2015 13:04:01 -0700 Subject: [erlang-questions] running debugger fails In-Reply-To: References: Message-ID: Hi Sergei, Yes, you are right. I use the prebuilt version of erlang on ubuntu using apt-get install erlang. Any idea on how to make it work for ubuntu ? Shall I build and install Erlang from source ? Weide On Tue, Jul 21, 2015 at 4:52 AM, Sergei Golovan wrote: > Hi! > > On Tue, Jul 21, 2015 at 8:48 AM, weoccc wrote: > > > 1> debugger:start(). > > {error,{undef,[{wx,new,[],[]}, > > > {dbg_wx_mon,init,3,[{file,"dbg_wx_mon.erl"},{line,113}]}]}} > > > > Any idea how to fix those ? > > This likely means that you Erlang installation misses the Erlang wx > application. > I know that Erlang package prebuilt for Ubuntu Linux don't include wx. > How did you > install your Erlang? > > Cheers! > > -- > Sergei Golovan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgolovan@REDACTED Wed Jul 22 07:40:19 2015 From: sgolovan@REDACTED (Sergei Golovan) Date: Wed, 22 Jul 2015 08:40:19 +0300 Subject: [erlang-questions] running debugger fails In-Reply-To: References: Message-ID: Hi! On Tue, Jul 21, 2015 at 11:04 PM, weoccc wrote: > Hi Sergei, > > Yes, you are right. I use the prebuilt version of erlang on ubuntu using > apt-get install erlang. Any idea on how to make it work for ubuntu ? Shall > I build and install Erlang from source ? I guess the easiest way is to install Erlang packages from Erlang solutions repository: https://www.erlang-solutions.com/downloads/download-erlang-otp As far as I can see, they are meant to be drop-in replacements for the stock packages. Alternatively, you can build Erlang from source, but this means that you'll not be able to install Ubuntu packages which depend on Erlang and you'll have to build them from source as well. Cheers! -- Sergei Golovan From nem@REDACTED Thu Jul 23 01:35:40 2015 From: nem@REDACTED (Geoff Cant) Date: Wed, 22 Jul 2015 16:35:40 -0700 Subject: [erlang-questions] SSL Client CA Certs/chain validation Message-ID: Hi all, I?m wondering if anyone has written a guide (or can link to example code) showing how they use OTP?s SSL library to connect to arbitrary TLS servers on the internet with x.509 cert chain validation. I know the default SSL library option is ?verify_none?, and that there is a ?cacertfile? option, but a) it?s 2015 and you should verify cert chains, and b) are people really bundling all the standard public CA certs into a single giant cacertfile? If you are bundling say all of ubuntu?s /etc/certs, do you have any tooling for this (cat /etc/certs/*.pem >> get_me_everyone.cacerts)? Am I missing something and OTP automatically uses the contents of /etc/certs ? Also, are people writing utility libraries/code to wrap ssl:* in order to setup the connect/listen options they use? (I know I wrote one to do certificate pinning) I?m generally curious about your OTP ssl client use - particularly around cert chain validation. Cheers, -Geoff From kaiduanx@REDACTED Thu Jul 23 02:28:56 2015 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Wed, 22 Jul 2015 20:28:56 -0400 Subject: [erlang-questions] SSL Client CA Certs/chain validation In-Reply-To: References: Message-ID: The following articles explain thing very clearly, http://security.stackexchange.com/questions/59566/ssl-certificate-chain-verification http://security.stackexchange.com/questions/56389/ssl-certificate-framework-101-how-does-the-browser-actually-verify-the-validity /Kaiduan On Wed, Jul 22, 2015 at 7:35 PM, Geoff Cant wrote: > Hi all, I?m wondering if anyone has written a guide (or can link to example code) showing how they use OTP?s SSL library to connect to arbitrary TLS servers on the internet with x.509 cert chain validation. > > I know the default SSL library option is ?verify_none?, and that there is a ?cacertfile? option, but a) it?s 2015 and you should verify cert chains, and b) are people really bundling all the standard public CA certs into a single giant cacertfile? If you are bundling say all of ubuntu?s /etc/certs, do you have any tooling for this (cat /etc/certs/*.pem >> get_me_everyone.cacerts)? Am I missing something and OTP automatically uses the contents of /etc/certs ? > > Also, are people writing utility libraries/code to wrap ssl:* in order to setup the connect/listen options they use? (I know I wrote one to do certificate pinning) > > > I?m generally curious about your OTP ssl client use - particularly around cert chain validation. > > Cheers, > -Geoff > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From eric.meadows.jonsson@REDACTED Thu Jul 23 03:15:22 2015 From: eric.meadows.jonsson@REDACTED (=?UTF-8?Q?Eric_Meadows=2DJ=C3=B6nsson?=) Date: Thu, 23 Jul 2015 03:15:22 +0200 Subject: [erlang-questions] SSL Client CA Certs/chain validation In-Reply-To: References: Message-ID: I maintain a http client using httpc as part of the Hex package manager and have been trying to do proper HTTPS connections with it. In my experience there is a lot of things you have to implement yourself if you want to do it correctly and I have had many of the same questions you have. Many things are still not clear for me but hopefully I can answer some of your questions. I am going to be linking to Elixir code but I think it will be easy for an Erlang programmer to understand it. These are the SSL options I use: https://github.com/hexpm/hex/blob/98ebb655a3e4b494795f510c07e6b16f16650e91/lib/hex/api.ex#L54-L55 . Interesting options are `verify_fun`, Erlang doesn't seem to support hostname verification so I use an Elixir port of https://github.com/deadtrickster/ssl_verify_hostname.erl for that, many thanks to Ilya Khaprov for creating that library. I pass in a CA certificate store via `cacerts`, it is generated from Mozilla's store with curl's mk-ca-bundle tool http://curl.haxx.se/docs/mk-ca-bundle.html, you can see how it is called here: https://github.com/hexpm/hex/blob/98ebb655a3e4b494795f510c07e6b16f16650e91/mix.exs#L61-L84 . Finally, unless you implement the `partial_chain` function many websites wont work because they do not send a complete certificate chain. Disclaimer: I do not know the correct (and secure) way to implement this function, I even received a security bug report recently https://github.com/hexpm/hex/issues/108 because the old implementation was even more wrong. As you can see from the linked issue we are still not confident that this is a correct implementation. Reviews of this function from the OTP team or someone intimate with Erlang SSL would be very much appreciated. There are also the new options for SNI in OTP 18 which I have not implemented yet. As you can tell there is a lot of code you have to write yourself and Erlang's ssl application does not have very exhaustive documentation or any guides at all AFAICT, so it is very hard to implement this without any bugs exposing security holes and I am not very confident in my own code because of this. On Thu, Jul 23, 2015 at 2:28 AM, Kaiduan Xie wrote: > The following articles explain thing very clearly, > > > http://security.stackexchange.com/questions/59566/ssl-certificate-chain-verification > > > http://security.stackexchange.com/questions/56389/ssl-certificate-framework-101-how-does-the-browser-actually-verify-the-validity > > /Kaiduan > > On Wed, Jul 22, 2015 at 7:35 PM, Geoff Cant wrote: > > Hi all, I?m wondering if anyone has written a guide (or can link to > example code) showing how they use OTP?s SSL library to connect to > arbitrary TLS servers on the internet with x.509 cert chain validation. > > > > I know the default SSL library option is ?verify_none?, and that there > is a ?cacertfile? option, but a) it?s 2015 and you should verify cert > chains, and b) are people really bundling all the standard public CA certs > into a single giant cacertfile? If you are bundling say all of ubuntu?s > /etc/certs, do you have any tooling for this (cat /etc/certs/*.pem >> > get_me_everyone.cacerts)? Am I missing something and OTP automatically uses > the contents of /etc/certs ? > > > > Also, are people writing utility libraries/code to wrap ssl:* in order > to setup the connect/listen options they use? (I know I wrote one to do > certificate pinning) > > > > > > I?m generally curious about your OTP ssl client use - particularly > around cert chain validation. > > > > Cheers, > > -Geoff > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Eric Meadows-J?nsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Jul 23 03:26:33 2015 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 23 Jul 2015 13:26:33 +1200 Subject: [erlang-questions] How to Update List Elements A Lot In-Reply-To: References: Message-ID: <8C797B4E-A9CB-4E48-BF9E-8AF40B9E780C@cs.otago.ac.nz> First, I have to echo Vance Shipley: make it work before you worry about making it fast. Second, I get slightly twitchy when people way "random updates", because they don't always *mean* "random", just "not sequential". This confusion goes deep into IT terminology and is most unfortunate. I wonder if you are processing the input in blocks of say 64 bytes? If so, had you considered representing a block by a spray of separate variables rather than a data structure? And maybe *computing* the Erlang code to do the shuffling? There is another issue to consider when translating bit-shuffling code from other languages to Erlang, and this includes apparently high level algorithms that appear to be language-neutral. Erlang gets integer arithmetic *right*. When you do a left shift, you do not lose bits off the top. When you add or subtract, the result is not taken modulo some weird inhumane number. There are two consequences. One is that an algorithm may run unexpectedly slowly in Erlang because Erlang is computing intermediate results as specified, not truncating them. The other is that explicit masking may be needed to simulate 32-bit or 64-bit arithmetic, whichever the algorithm assumes. From t@REDACTED Thu Jul 23 03:27:43 2015 From: t@REDACTED (Tristan Sloughter) Date: Wed, 22 Jul 2015 20:27:43 -0500 Subject: [erlang-questions] SSL Client CA Certs/chain validation In-Reply-To: References: Message-ID: <1437614863.3621125.330805153.3FD63531@webmail.messagingengine.com> For Erlang code I copied from Eric to implement the same for rebar3's hex client :) https://github.com/rebar/rebar3/blob/master/src/rebar_pkg_resource.erl#L127-L172 and https://github.com/rebar/rebar3/blob/master/src/rebar_cacerts.erl -- Tristan Sloughter t@REDACTED On Wed, Jul 22, 2015, at 08:15 PM, Eric Meadows-J?nsson wrote: > I maintain a http client using httpc as part of the Hex package manager and have been trying to do proper HTTPS connections with it. In my experience there is a lot of things you have to implement yourself if you want to do it correctly and I have had many of the same questions you have. Many things are still not clear for me but hopefully I can answer some of your questions. I am going to be linking to Elixir code but I think it will be easy for an Erlang programmer to understand it. > > These are the SSL options I use:?https://github.com/hexpm/hex/blob/98ebb655a3e4b494795f510c07e6b16f16650e91/lib/hex/api.ex#L54-L55. > > Interesting options are `verify_fun`, Erlang doesn't seem to support hostname verification so I use an Elixir port of?https://github.com/deadtrickster/ssl_verify_hostname.erl for that, many thanks to?Ilya Khaprov for creating that library. I pass in a CA certificate store via `cacerts`, it is generated from Mozilla's store with curl's mk-ca-bundle tool http://curl.haxx.se/docs/mk-ca-bundle.html, you can see how it is called here:?https://github.com/hexpm/hex/blob/98ebb655a3e4b494795f510c07e6b16f16650e91/mix.exs#L61-L84. > > Finally, unless you implement the `partial_chain` function many websites wont work because they do not send a complete certificate chain. Disclaimer: I do not know the correct (and secure) way to implement this function, I even received a security bug report recently https://github.com/hexpm/hex/issues/108 because the old implementation was even more wrong. As you can see from the linked issue we are still not confident that this is a correct implementation. Reviews of this function from the OTP team or someone intimate with Erlang SSL would be very much appreciated. > > There are also the new options for SNI in OTP 18 which I have not implemented yet. > > As you can tell there is a lot of code you have to write yourself and Erlang's ssl application does not have very exhaustive documentation or any guides at all AFAICT, so it is very hard to implement this without any bugs exposing security holes and I am not very confident in my own code because of this. > > On Thu, Jul 23, 2015 at 2:28 AM, Kaiduan Xie wrote: >> The following articles explain thing very clearly, >> >> http://security.stackexchange.com/questions/59566/ssl-certificate-chain-verification >> >> http://security.stackexchange.com/questions/56389/ssl-certificate-framework-101-how-does-the-browser-actually-verify-the-validity >> >> /Kaiduan >> >> On Wed, Jul 22, 2015 at 7:35 PM, Geoff Cant wrote: >> > Hi all, I?m wondering if anyone has written a guide (or can link to example code) showing how they use OTP?s SSL library to connect to arbitrary TLS servers on the internet with x.509 cert chain validation. >> > >> > I know the default SSL library option is ?verify_none?, and that there is a ?cacertfile? option, but a) it?s 2015 and you should verify cert chains, and b) are people really bundling all the standard public CA certs into a single giant cacertfile? If you are bundling say all of ubuntu?s /etc/certs, do you have any tooling for this (cat /etc/certs/*.pem >> get_me_everyone.cacerts)? Am I missing something and OTP automatically uses the contents of /etc/certs ? >> > >> > Also, are people writing utility libraries/code to wrap ssl:* in order to setup the connect/listen options they use? (I know I wrote one to do certificate pinning) >> > >> > >> > I?m generally curious about your OTP ssl client use - particularly around cert chain validation. >> > >> > Cheers, >> > -Geoff >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Eric Meadows-J?nsson > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From chan.sisowath@REDACTED Thu Jul 23 12:02:21 2015 From: chan.sisowath@REDACTED (chan sisowath) Date: Thu, 23 Jul 2015 18:02:21 +0800 Subject: [erlang-questions] [ANN] erln8, a sneaky version manager for Erlang, Elixir and Rebar/Rebar3 In-Reply-To: References: Message-ID: hi, nice tool should have use it before. thanks for sharing. 2015-07-15 22:50 GMT+08:00 Dave Parfitt : > Hello - > > erln8 is set of tools that let you build and then set versions of Erlang, > Elixir, Rebar and Rebar3 per directory. While an older version of erln8 > has been > around for a couple years, the latest rewrite *beta* now has Elixir > support. Note, > that as it's still a beta, there might be a couple rough spots here > and there. Friendly > pull requests welcome. > > Here's an example of building the latest version of Erlang + Rebar, > and then setting > versions to use in multiple directories: > > # build the latest tagged version of Erlang (OTP-18.0.2 at the time of > writing) > erln8 --build-latest > > # build OTP_R16B03-1 > erln8 --build OTP_R16B03-1 > > # build the latest tagged version of Rebar > reo --build-latest > > cd ~/some/project > erln8 --use OTP-18.0.2 > # any calls to erlang commands in this directory will be for OTP-18.0.2 > > cd ~/some/other/project > erln8 --use OTP_R16B03-1 > # any calls to erlang commands in this directory will be for OTP_R16B03-1 > > You can download erln8 source, docs and prebuilt binaries here: > http://erln8.github.io/erln8/ > > > Cheers - > Dave > https://github.com/metadave > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Thu Jul 23 12:11:59 2015 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 23 Jul 2015 10:11:59 +0000 Subject: [erlang-questions] SSL Client CA Certs/chain validation In-Reply-To: <1437614863.3621125.330805153.3FD63531@webmail.messagingengine.com> References: <1437614863.3621125.330805153.3FD63531@webmail.messagingengine.com> Message-ID: On Thu, Jul 23, 2015 at 3:27 AM Tristan Sloughter wrote: > For Erlang code I copied from Eric to implement the same for rebar3's > hex client :) > > > https://github.com/rebar/rebar3/blob/master/src/rebar_pkg_resource.erl#L127-L172 > > and > > https://github.com/rebar/rebar3/blob/master/src/rebar_cacerts.erl > > -- > Tristan Sloughter > t@REDACTED > > Hrmm Are you sure it is actually working? I tried similar code in hackney on the following URLS: https://api.gateway.evercam.io/v1 https://api.twilio.com/2010-04-01/Accounts/ And I get an "unkown_ca" error... Hackney changes are: https://github.com/benoitc/hackney/pull/217 On which URLS did you tested it? How do you generate your CA file? - benoit > > > On Wed, Jul 22, 2015, at 08:15 PM, Eric Meadows-J?nsson wrote: > > I maintain a http client using httpc as part of the Hex package manager > and have been trying to do proper HTTPS connections with it. In my > experience there is a lot of things you have to implement yourself if you > want to do it correctly and I have had many of the same questions you have. > Many things are still not clear for me but hopefully I can answer some of > your questions. I am going to be linking to Elixir code but I think it will > be easy for an Erlang programmer to understand it. > > These are the SSL options I use: > https://github.com/hexpm/hex/blob/98ebb655a3e4b494795f510c07e6b16f16650e91/lib/hex/api.ex#L54-L55 > . > > Interesting options are `verify_fun`, Erlang doesn't seem to support > hostname verification so I use an Elixir port of > https://github.com/deadtrickster/ssl_verify_hostname.erl for that, many > thanks to Ilya Khaprov for creating that library. I pass in a CA > certificate store via `cacerts`, it is generated from Mozilla's store with > curl's mk-ca-bundle tool http://curl.haxx.se/docs/mk-ca-bundle.html, you > can see how it is called here: > https://github.com/hexpm/hex/blob/98ebb655a3e4b494795f510c07e6b16f16650e91/mix.exs#L61-L84 > . > > Finally, unless you implement the `partial_chain` function many websites > wont work because they do not send a complete certificate chain. > Disclaimer: I do not know the correct (and secure) way to implement this > function, I even received a security bug report recently > https://github.com/hexpm/hex/issues/108 because the old implementation > was even more wrong. As you can see from the linked issue we are still not > confident that this is a correct implementation. Reviews of this function > from the OTP team or someone intimate with Erlang SSL would be very much > appreciated. > > There are also the new options for SNI in OTP 18 which I have not > implemented yet. > > As you can tell there is a lot of code you have to write yourself and > Erlang's ssl application does not have very exhaustive documentation or any > guides at all AFAICT, so it is very hard to implement this without any bugs > exposing security holes and I am not very confident in my own code because > of this. > > On Thu, Jul 23, 2015 at 2:28 AM, Kaiduan Xie wrote: > > The following articles explain thing very clearly, > > > http://security.stackexchange.com/questions/59566/ssl-certificate-chain-verification > > > http://security.stackexchange.com/questions/56389/ssl-certificate-framework-101-how-does-the-browser-actually-verify-the-validity > > /Kaiduan > > On Wed, Jul 22, 2015 at 7:35 PM, Geoff Cant wrote: > > Hi all, I?m wondering if anyone has written a guide (or can link to > example code) showing how they use OTP?s SSL library to connect to > arbitrary TLS servers on the internet with x.509 cert chain validation. > > > > I know the default SSL library option is ?verify_none?, and that there > is a ?cacertfile? option, but a) it?s 2015 and you should verify cert > chains, and b) are people really bundling all the standard public CA certs > into a single giant cacertfile? If you are bundling say all of ubuntu?s > /etc/certs, do you have any tooling for this (cat /etc/certs/*.pem >> > get_me_everyone.cacerts)? Am I missing something and OTP automatically uses > the contents of /etc/certs ? > > > > Also, are people writing utility libraries/code to wrap ssl:* in order > to setup the connect/listen options they use? (I know I wrote one to do > certificate pinning) > > > > > > I?m generally curious about your OTP ssl client use - particularly > around cert chain validation. > > > > Cheers, > > -Geoff > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Eric Meadows-J?nsson > *_______________________________________________* > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From 44290016@REDACTED Thu Jul 23 13:38:35 2015 From: 44290016@REDACTED (=?gb18030?B?vMW8xQ==?=) Date: Thu, 23 Jul 2015 19:38:35 +0800 Subject: [erlang-questions] [cowboy]Can content_types_accepted match all Cotent-Types? Message-ID: Dear Erlangers, I have a client sending POST request to my server, but I'm not sure what kind of Content-type in the POST header. so can I match all Content-Types in 'content_types_accepted' of my REST module? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Jul 23 14:04:22 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 23 Jul 2015 14:04:22 +0200 Subject: [erlang-questions] [cowboy]Can content_types_accepted match all Cotent-Types? In-Reply-To: References: Message-ID: It is documented (Cowboy 1.x): Cowboy will select the most appropriate content-type from the list. If any parameter is acceptable, then the tuple form should be used with parameters set to '*'. If the parameters value is set to [] only content-type values with no parameters will be accepted. All parameter values are treated in a case sensitive manner except the charset parameter, if present, which is case insensitive. So if you set the content type to '*' (that is the atom()), then any content type will be accepted by that resource, unless a more specific content type matches. On Thu, Jul 23, 2015 at 1:38 PM, ?? <44290016@REDACTED> wrote: > Dear Erlangers, > > I have a client sending POST request to my server, but I'm not sure what > kind of Content-type in the POST header. so can I match all Content-Types > in 'content_types_accepted' of my REST module? > > Thanks > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.meadows.jonsson@REDACTED Thu Jul 23 14:29:51 2015 From: eric.meadows.jonsson@REDACTED (=?UTF-8?Q?Eric_Meadows=2DJ=C3=B6nsson?=) Date: Thu, 23 Jul 2015 14:29:51 +0200 Subject: [erlang-questions] SSL Client CA Certs/chain validation In-Reply-To: References: <1437614863.3621125.330805153.3FD63531@webmail.messagingengine.com> Message-ID: https://api.twilio.com/2010-04-01/Accounts/ works for me with Hex's http client. https://api.gateway.evercam.io/v1 doesn't work and the issue seems to be that they only send their own certificate without any intermediate certificates in the chain. Since `partial_chain` is supposed to claim trust for an intermediate certificate (and only for an intermediate in the provided chain, you cannot return your own intermediate from the CA store) this is impossible to do. On Thu, Jul 23, 2015 at 12:11 PM, Benoit Chesneau wrote: > > > On Thu, Jul 23, 2015 at 3:27 AM Tristan Sloughter wrote: > >> For Erlang code I copied from Eric to implement the same for rebar3's >> hex client :) >> >> >> https://github.com/rebar/rebar3/blob/master/src/rebar_pkg_resource.erl#L127-L172 >> >> and >> >> https://github.com/rebar/rebar3/blob/master/src/rebar_cacerts.erl >> >> -- >> Tristan Sloughter >> t@REDACTED >> >> > > Hrmm Are you sure it is actually working? > > I tried similar code in hackney on the following URLS: > > https://api.gateway.evercam.io/v1 > https://api.twilio.com/2010-04-01/Accounts/ > > And I get an "unkown_ca" error... > > Hackney changes are: > https://github.com/benoitc/hackney/pull/217 > > On which URLS did you tested it? How do you generate your CA file? > > - benoit > > >> >> >> On Wed, Jul 22, 2015, at 08:15 PM, Eric Meadows-J?nsson wrote: >> >> I maintain a http client using httpc as part of the Hex package manager >> and have been trying to do proper HTTPS connections with it. In my >> experience there is a lot of things you have to implement yourself if you >> want to do it correctly and I have had many of the same questions you have. >> Many things are still not clear for me but hopefully I can answer some of >> your questions. I am going to be linking to Elixir code but I think it will >> be easy for an Erlang programmer to understand it. >> >> These are the SSL options I use: >> https://github.com/hexpm/hex/blob/98ebb655a3e4b494795f510c07e6b16f16650e91/lib/hex/api.ex#L54-L55 >> . >> >> Interesting options are `verify_fun`, Erlang doesn't seem to support >> hostname verification so I use an Elixir port of >> https://github.com/deadtrickster/ssl_verify_hostname.erl for that, many >> thanks to Ilya Khaprov for creating that library. I pass in a CA >> certificate store via `cacerts`, it is generated from Mozilla's store with >> curl's mk-ca-bundle tool http://curl.haxx.se/docs/mk-ca-bundle.html, you >> can see how it is called here: >> https://github.com/hexpm/hex/blob/98ebb655a3e4b494795f510c07e6b16f16650e91/mix.exs#L61-L84 >> . >> >> Finally, unless you implement the `partial_chain` function many websites >> wont work because they do not send a complete certificate chain. >> Disclaimer: I do not know the correct (and secure) way to implement this >> function, I even received a security bug report recently >> https://github.com/hexpm/hex/issues/108 because the old implementation >> was even more wrong. As you can see from the linked issue we are still not >> confident that this is a correct implementation. Reviews of this function >> from the OTP team or someone intimate with Erlang SSL would be very much >> appreciated. >> >> There are also the new options for SNI in OTP 18 which I have not >> implemented yet. >> >> As you can tell there is a lot of code you have to write yourself and >> Erlang's ssl application does not have very exhaustive documentation or any >> guides at all AFAICT, so it is very hard to implement this without any bugs >> exposing security holes and I am not very confident in my own code because >> of this. >> >> On Thu, Jul 23, 2015 at 2:28 AM, Kaiduan Xie wrote: >> >> The following articles explain thing very clearly, >> >> >> http://security.stackexchange.com/questions/59566/ssl-certificate-chain-verification >> >> >> http://security.stackexchange.com/questions/56389/ssl-certificate-framework-101-how-does-the-browser-actually-verify-the-validity >> >> /Kaiduan >> >> On Wed, Jul 22, 2015 at 7:35 PM, Geoff Cant wrote: >> > Hi all, I?m wondering if anyone has written a guide (or can link to >> example code) showing how they use OTP?s SSL library to connect to >> arbitrary TLS servers on the internet with x.509 cert chain validation. >> > >> > I know the default SSL library option is ?verify_none?, and that there >> is a ?cacertfile? option, but a) it?s 2015 and you should verify cert >> chains, and b) are people really bundling all the standard public CA certs >> into a single giant cacertfile? If you are bundling say all of ubuntu?s >> /etc/certs, do you have any tooling for this (cat /etc/certs/*.pem >> >> get_me_everyone.cacerts)? Am I missing something and OTP automatically uses >> the contents of /etc/certs ? >> > >> > Also, are people writing utility libraries/code to wrap ssl:* in order >> to setup the connect/listen options they use? (I know I wrote one to do >> certificate pinning) >> > >> > >> > I?m generally curious about your OTP ssl client use - particularly >> around cert chain validation. >> > >> > Cheers, >> > -Geoff >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> -- >> Eric Meadows-J?nsson >> *_______________________________________________* >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Eric Meadows-J?nsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Thu Jul 23 15:20:46 2015 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 23 Jul 2015 13:20:46 +0000 Subject: [erlang-questions] SSL Client CA Certs/chain validation In-Reply-To: References: <1437614863.3621125.330805153.3FD63531@webmail.messagingengine.com> Message-ID: On Thu, Jul 23, 2015 at 2:29 PM Eric Meadows-J?nsson < eric.meadows.jonsson@REDACTED> wrote: > https://api.twilio.com/2010-04-01/Accounts/ works for me with Hex's http > client. https://api.gateway.evercam.io/v1 doesn't work and the issue > seems to be that they only send their own certificate without any > intermediate certificates in the chain. Since `partial_chain` is supposed > to claim trust for an intermediate certificate (and only for an > intermediate in the provided chain, you cannot return your own intermediate > from the CA store) this is impossible to do. > I did a minimal case there to test: https://github.com/benoitc/test_ssl So indeed I confirm the first url " https://api.twilio.com/2010-04-01/Accounts/ " is working. But not the second one. not sure now why it's not working in hackney now :/ But the interresting thing is that curl is returning different results: [test_ssl] curl --cacert priv/ca-bundle.crt https://api.twilio.com/2010-04-01/Accounts/ curl: (51) SSL: certificate verification failed (result: 5) [test_ssl] curl --cacert priv/ca-bundle.crt https://api.gateway.evercam.io/v1 .... Probably because curl on osx is using keychain though. I will check for the intermediate certificates. - benoit > > On Thu, Jul 23, 2015 at 12:11 PM, Benoit Chesneau > wrote: > >> >> >> On Thu, Jul 23, 2015 at 3:27 AM Tristan Sloughter >> wrote: >> >>> For Erlang code I copied from Eric to implement the same for rebar3's >>> hex client :) >>> >>> >>> https://github.com/rebar/rebar3/blob/master/src/rebar_pkg_resource.erl#L127-L172 >>> >>> and >>> >>> https://github.com/rebar/rebar3/blob/master/src/rebar_cacerts.erl >>> >>> -- >>> Tristan Sloughter >>> t@REDACTED >>> >>> >> >> Hrmm Are you sure it is actually working? >> >> I tried similar code in hackney on the following URLS: >> >> https://api.gateway.evercam.io/v1 >> https://api.twilio.com/2010-04-01/Accounts/ >> >> And I get an "unkown_ca" error... >> >> Hackney changes are: >> https://github.com/benoitc/hackney/pull/217 >> >> On which URLS did you tested it? How do you generate your CA file? >> >> - benoit >> >> >>> >>> >>> On Wed, Jul 22, 2015, at 08:15 PM, Eric Meadows-J?nsson wrote: >>> >>> I maintain a http client using httpc as part of the Hex package manager >>> and have been trying to do proper HTTPS connections with it. In my >>> experience there is a lot of things you have to implement yourself if you >>> want to do it correctly and I have had many of the same questions you have. >>> Many things are still not clear for me but hopefully I can answer some of >>> your questions. I am going to be linking to Elixir code but I think it will >>> be easy for an Erlang programmer to understand it. >>> >>> These are the SSL options I use: >>> https://github.com/hexpm/hex/blob/98ebb655a3e4b494795f510c07e6b16f16650e91/lib/hex/api.ex#L54-L55 >>> . >>> >>> Interesting options are `verify_fun`, Erlang doesn't seem to support >>> hostname verification so I use an Elixir port of >>> https://github.com/deadtrickster/ssl_verify_hostname.erl for that, many >>> thanks to Ilya Khaprov for creating that library. I pass in a CA >>> certificate store via `cacerts`, it is generated from Mozilla's store with >>> curl's mk-ca-bundle tool http://curl.haxx.se/docs/mk-ca-bundle.html, >>> you can see how it is called here: >>> https://github.com/hexpm/hex/blob/98ebb655a3e4b494795f510c07e6b16f16650e91/mix.exs#L61-L84 >>> . >>> >>> Finally, unless you implement the `partial_chain` function many websites >>> wont work because they do not send a complete certificate chain. >>> Disclaimer: I do not know the correct (and secure) way to implement this >>> function, I even received a security bug report recently >>> https://github.com/hexpm/hex/issues/108 because the old implementation >>> was even more wrong. As you can see from the linked issue we are still not >>> confident that this is a correct implementation. Reviews of this function >>> from the OTP team or someone intimate with Erlang SSL would be very much >>> appreciated. >>> >>> There are also the new options for SNI in OTP 18 which I have not >>> implemented yet. >>> >>> As you can tell there is a lot of code you have to write yourself and >>> Erlang's ssl application does not have very exhaustive documentation or any >>> guides at all AFAICT, so it is very hard to implement this without any bugs >>> exposing security holes and I am not very confident in my own code because >>> of this. >>> >>> On Thu, Jul 23, 2015 at 2:28 AM, Kaiduan Xie wrote: >>> >>> The following articles explain thing very clearly, >>> >>> >>> http://security.stackexchange.com/questions/59566/ssl-certificate-chain-verification >>> >>> >>> http://security.stackexchange.com/questions/56389/ssl-certificate-framework-101-how-does-the-browser-actually-verify-the-validity >>> >>> /Kaiduan >>> >>> On Wed, Jul 22, 2015 at 7:35 PM, Geoff Cant wrote: >>> > Hi all, I?m wondering if anyone has written a guide (or can link to >>> example code) showing how they use OTP?s SSL library to connect to >>> arbitrary TLS servers on the internet with x.509 cert chain validation. >>> > >>> > I know the default SSL library option is ?verify_none?, and that >>> there is a ?cacertfile? option, but a) it?s 2015 and you should verify cert >>> chains, and b) are people really bundling all the standard public CA certs >>> into a single giant cacertfile? If you are bundling say all of ubuntu?s >>> /etc/certs, do you have any tooling for this (cat /etc/certs/*.pem >> >>> get_me_everyone.cacerts)? Am I missing something and OTP automatically uses >>> the contents of /etc/certs ? >>> > >>> > Also, are people writing utility libraries/code to wrap ssl:* in >>> order to setup the connect/listen options they use? (I know I wrote one to >>> do certificate pinning) >>> > >>> > >>> > I?m generally curious about your OTP ssl client use - particularly >>> around cert chain validation. >>> > >>> > Cheers, >>> > -Geoff >>> > _______________________________________________ >>> > erlang-questions mailing list >>> > erlang-questions@REDACTED >>> > http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >>> >>> -- >>> Eric Meadows-J?nsson >>> *_______________________________________________* >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > Eric Meadows-J?nsson > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Thu Jul 23 15:32:14 2015 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 23 Jul 2015 13:32:14 +0000 Subject: [erlang-questions] SSL Client CA Certs/chain validation In-Reply-To: References: <1437614863.3621125.330805153.3FD63531@webmail.messagingengine.com> Message-ID: and my bad... I was silently removing the partial_chain option ;). Anyway thanks for the explanation :) - benoit On Thu, Jul 23, 2015 at 3:20 PM Benoit Chesneau wrote: > On Thu, Jul 23, 2015 at 2:29 PM Eric Meadows-J?nsson < > eric.meadows.jonsson@REDACTED> wrote: > >> https://api.twilio.com/2010-04-01/Accounts/ works for me with Hex's http >> client. https://api.gateway.evercam.io/v1 doesn't work and the issue >> seems to be that they only send their own certificate without any >> intermediate certificates in the chain. Since `partial_chain` is supposed >> to claim trust for an intermediate certificate (and only for an >> intermediate in the provided chain, you cannot return your own intermediate >> from the CA store) this is impossible to do. >> > > I did a minimal case there to test: > https://github.com/benoitc/test_ssl > > So indeed I confirm the first url " > https://api.twilio.com/2010-04-01/Accounts/ " is working. But not the > second one. > > not sure now why it's not working in hackney now :/ > > But the interresting thing is that curl is returning different results: > > [test_ssl] curl --cacert priv/ca-bundle.crt > https://api.twilio.com/2010-04-01/Accounts/ > curl: (51) SSL: certificate verification failed (result: 5) > > [test_ssl] curl --cacert priv/ca-bundle.crt > https://api.gateway.evercam.io/v1 > > > .... > > Probably because curl on osx is using keychain though. I will check for > the intermediate certificates. > > - benoit > > >> >> On Thu, Jul 23, 2015 at 12:11 PM, Benoit Chesneau >> wrote: >> >>> >>> >>> On Thu, Jul 23, 2015 at 3:27 AM Tristan Sloughter >>> wrote: >>> >>>> For Erlang code I copied from Eric to implement the same for rebar3's >>>> hex client :) >>>> >>>> >>>> https://github.com/rebar/rebar3/blob/master/src/rebar_pkg_resource.erl#L127-L172 >>>> >>>> and >>>> >>>> https://github.com/rebar/rebar3/blob/master/src/rebar_cacerts.erl >>>> >>>> -- >>>> Tristan Sloughter >>>> t@REDACTED >>>> >>>> >>> >>> Hrmm Are you sure it is actually working? >>> >>> I tried similar code in hackney on the following URLS: >>> >>> https://api.gateway.evercam.io/v1 >>> https://api.twilio.com/2010-04-01/Accounts/ >>> >>> And I get an "unkown_ca" error... >>> >>> Hackney changes are: >>> https://github.com/benoitc/hackney/pull/217 >>> >>> On which URLS did you tested it? How do you generate your CA file? >>> >>> - benoit >>> >>> >>>> >>>> >>>> On Wed, Jul 22, 2015, at 08:15 PM, Eric Meadows-J?nsson wrote: >>>> >>>> I maintain a http client using httpc as part of the Hex package manager >>>> and have been trying to do proper HTTPS connections with it. In my >>>> experience there is a lot of things you have to implement yourself if you >>>> want to do it correctly and I have had many of the same questions you have. >>>> Many things are still not clear for me but hopefully I can answer some of >>>> your questions. I am going to be linking to Elixir code but I think it will >>>> be easy for an Erlang programmer to understand it. >>>> >>>> These are the SSL options I use: >>>> https://github.com/hexpm/hex/blob/98ebb655a3e4b494795f510c07e6b16f16650e91/lib/hex/api.ex#L54-L55 >>>> . >>>> >>>> Interesting options are `verify_fun`, Erlang doesn't seem to support >>>> hostname verification so I use an Elixir port of >>>> https://github.com/deadtrickster/ssl_verify_hostname.erl for that, >>>> many thanks to Ilya Khaprov for creating that library. I pass in a CA >>>> certificate store via `cacerts`, it is generated from Mozilla's store with >>>> curl's mk-ca-bundle tool http://curl.haxx.se/docs/mk-ca-bundle.html, >>>> you can see how it is called here: >>>> https://github.com/hexpm/hex/blob/98ebb655a3e4b494795f510c07e6b16f16650e91/mix.exs#L61-L84 >>>> . >>>> >>>> Finally, unless you implement the `partial_chain` function many >>>> websites wont work because they do not send a complete certificate chain. >>>> Disclaimer: I do not know the correct (and secure) way to implement this >>>> function, I even received a security bug report recently >>>> https://github.com/hexpm/hex/issues/108 because the old implementation >>>> was even more wrong. As you can see from the linked issue we are still not >>>> confident that this is a correct implementation. Reviews of this function >>>> from the OTP team or someone intimate with Erlang SSL would be very much >>>> appreciated. >>>> >>>> There are also the new options for SNI in OTP 18 which I have not >>>> implemented yet. >>>> >>>> As you can tell there is a lot of code you have to write yourself and >>>> Erlang's ssl application does not have very exhaustive documentation or any >>>> guides at all AFAICT, so it is very hard to implement this without any bugs >>>> exposing security holes and I am not very confident in my own code because >>>> of this. >>>> >>>> On Thu, Jul 23, 2015 at 2:28 AM, Kaiduan Xie >>>> wrote: >>>> >>>> The following articles explain thing very clearly, >>>> >>>> >>>> http://security.stackexchange.com/questions/59566/ssl-certificate-chain-verification >>>> >>>> >>>> http://security.stackexchange.com/questions/56389/ssl-certificate-framework-101-how-does-the-browser-actually-verify-the-validity >>>> >>>> /Kaiduan >>>> >>>> On Wed, Jul 22, 2015 at 7:35 PM, Geoff Cant >>>> wrote: >>>> > Hi all, I?m wondering if anyone has written a guide (or can link to >>>> example code) showing how they use OTP?s SSL library to connect to >>>> arbitrary TLS servers on the internet with x.509 cert chain validation. >>>> > >>>> > I know the default SSL library option is ?verify_none?, and that >>>> there is a ?cacertfile? option, but a) it?s 2015 and you should verify cert >>>> chains, and b) are people really bundling all the standard public CA certs >>>> into a single giant cacertfile? If you are bundling say all of ubuntu?s >>>> /etc/certs, do you have any tooling for this (cat /etc/certs/*.pem >> >>>> get_me_everyone.cacerts)? Am I missing something and OTP automatically uses >>>> the contents of /etc/certs ? >>>> > >>>> > Also, are people writing utility libraries/code to wrap ssl:* in >>>> order to setup the connect/listen options they use? (I know I wrote one to >>>> do certificate pinning) >>>> > >>>> > >>>> > I?m generally curious about your OTP ssl client use - particularly >>>> around cert chain validation. >>>> > >>>> > Cheers, >>>> > -Geoff >>>> > _______________________________________________ >>>> > erlang-questions mailing list >>>> > erlang-questions@REDACTED >>>> > http://erlang.org/mailman/listinfo/erlang-questions >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>>> >>>> >>>> -- >>>> Eric Meadows-J?nsson >>>> *_______________________________________________* >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> >> -- >> Eric Meadows-J?nsson >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mallen@REDACTED Thu Jul 23 19:55:44 2015 From: mallen@REDACTED (Mark Allen) Date: Thu, 23 Jul 2015 12:55:44 -0500 Subject: [erlang-questions] [ANN] Lager 3.0.0 is here Message-ID: As promised the long awaited lager 3.0.0 branch was pushed to master today. A new tag 3.0.0 is also available if you want to pin your deps. Our testing gives us good confidence that 3.0.0 should be mostly-backwards compatible with 2.x (please note that tracing semantics are NOT backward compatible. See the README in the repo for details.) https://github.com/basho/lager Highlights of the new features and improvements in 3.0 include: + multiple sinks for log messages (including configurable behavior about when and where to drop messages) + Several bug fixes with Unicode and log_root directives. + Additional lager configuration checking + New configurable group leader IO behavior + Support abbreviated severity levels in log output + If a record contains "undefined" fields optionally omit them from output + Several test suite bug fixes + Several new tests to cover multiple sinks + OTP 17 compatibility + Makes the "high watermark" load shedding logic available to backends (if desired) and is implemented in the file backend (if desired.) + Bring in goldrush 0.1.7 for additional operators which may be used in trace(s). There is also a 2.x maintenance branch and a 2.2.0 tag which has OTP 17 support and some minor bug fixes backported from 3.0 if you're looking for OTP 17 support but aren't ready to transition to 3.0 yet. We also have a triaged list of issues for 3.next on lager which you may review at https://github.com/basho/lager/labels/3.x%20future (but there are no firm release plans for a 3.next release right now.) We hope you enjoy all of these new features in lager. Thanks! Mark Allen -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Jul 24 01:06:05 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 24 Jul 2015 02:06:05 +0300 Subject: [erlang-questions] [ANN] Lager 3.0.0 is here In-Reply-To: References: Message-ID: Is it possible with lager3 to tell to move completely some metadata-marked stream of messages to one backend and don't put it to other backends? -------------- next part -------------- An HTML attachment was scrubbed... URL: From seancribbs@REDACTED Fri Jul 24 04:06:04 2015 From: seancribbs@REDACTED (Sean Cribbs) Date: Thu, 23 Jul 2015 21:06:04 -0500 Subject: [erlang-questions] [ANN] Lager 3.0.0 is here In-Reply-To: References: Message-ID: Max, That is the purpose of the "multiple sinks" [1] feature Mark described. You can dispatch log messages to an entirely different sink, at compile-time via the parse_transform. If the sink is not configured, nothing will be logged; if you configure it with specific backends, they will take an entirely separate path from the default sink. [1] https://github.com/basho/lager#sinks On Thu, Jul 23, 2015 at 6:06 PM, Max Lapshin wrote: > Is it possible with lager3 to tell to move completely some metadata-marked > stream of messages to one backend and don't put it to other backends? > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Jul 24 08:48:06 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 24 Jul 2015 09:48:06 +0300 Subject: [erlang-questions] [ANN] Lager 3.0.0 is here In-Reply-To: References: Message-ID: Thanks, I've read it. I don't see how can I tell this exact log mesage to go to audit sink, not to default sink. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Fri Jul 24 09:54:17 2015 From: sdl.web@REDACTED (Leo Liu) Date: Fri, 24 Jul 2015 15:54:17 +0800 Subject: [erlang-questions] How is graph like this generated? Message-ID: Hi there, There are a couple of graphs in this talk: Erlang Factory SF 2015 - Anthony Molinaro - How to pick a Pool in Erlang without Drowning https://youtu.be/GO_97_6w5lU?t=1101 I wonder how those graphs are generated? Thanks. Leo From jesper.louis.andersen@REDACTED Fri Jul 24 11:13:42 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 24 Jul 2015 11:13:42 +0200 Subject: [erlang-questions] How is graph like this generated? In-Reply-To: References: Message-ID: On Fri, Jul 24, 2015 at 9:54 AM, Leo Liu wrote: > I wonder how those graphs are generated? Thanks. I think those graphs are generated by a tool. Either manually, or automatically. The have many names, one of which is "Sequence Diagrams" and another one being "Message Sequence Chart (MSC)". Usually there is specific meaning attached to different shapes in them, but the general rule is that time flows downwards and you can see the interaction between several agents. In Erlang, we have a tool for generating these, which is the `et` application (short for event tracer). It allows you to add dummy function calls into your application and then record what the application is doing, automatically generating a diagram for you. My post is somewhat old, but I still think it is applicable: http://jlouisramblings.blogspot.dk/2011/10/using-event-tracer-tool-set-in-erlang.html That way, you can build diagrams automatically by attaching trace probes to the report_event function. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus@REDACTED Fri Jul 24 11:41:54 2015 From: magnus@REDACTED (Magnus Henoch) Date: Fri, 24 Jul 2015 10:41:54 +0100 Subject: [erlang-questions] How is graph like this generated? In-Reply-To: (Leo Liu's message of "Fri, 24 Jul 2015 15:54:17 +0800") References: Message-ID: Leo Liu writes: > Hi there, > > There are a couple of graphs in this talk: Erlang Factory SF > 2015 - Anthony Molinaro - How to pick a Pool in Erlang without > Drowning https://youtu.be/GO_97_6w5lU?t=1101 > > I wonder how those graphs are generated? Thanks. I'm not sure what tool was used to generate those graphs, but you can create such graphs online using https://www.websequencediagrams.com/ . Regards, Magnus From fernando.benavides@REDACTED Fri Jul 24 12:46:46 2015 From: fernando.benavides@REDACTED (Fernando 'Brujo' Benavides) Date: Fri, 24 Jul 2015 07:46:46 -0300 Subject: [erlang-questions] How is graph like this generated? In-Reply-To: References: Message-ID: <160532AE-2651-411B-A11F-B601D1CE0277@inaka.net> Based on that great example, and the easter egg I recently found (et:phone_home/4,5), I created this example module: https://github.com/elbrujohalcon/witchcraft/blob/master/src/ufo.erl To see it in action, do? 1> ufo:start(). 2> ufo:run(). You will see stuff like this going around: http://g.recordit.co/9KvFR7FRqN.gif Cheers! Fernando "Brujo" Benavides about.me/elbrujohalcon > On Jul 24, 2015, at 06:13, Jesper Louis Andersen wrote: > > > On Fri, Jul 24, 2015 at 9:54 AM, Leo Liu > wrote: > I wonder how those graphs are generated? Thanks. > > I think those graphs are generated by a tool. Either manually, or automatically. The have many names, one of which is "Sequence Diagrams" and another one being "Message Sequence Chart (MSC)". Usually there is specific meaning attached to different shapes in them, but the general rule is that time flows downwards and you can see the interaction between several agents. > > In Erlang, we have a tool for generating these, which is the `et` application (short for event tracer). It allows you to add dummy function calls into your application and then record what the application is doing, automatically generating a diagram for you. My post is somewhat old, but I still think it is applicable: > > http://jlouisramblings.blogspot.dk/2011/10/using-event-tracer-tool-set-in-erlang.html > > That way, you can build diagrams automatically by attaching trace probes to the report_event function. > > > > -- > J. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vychodil.hynek@REDACTED Fri Jul 24 13:43:18 2015 From: vychodil.hynek@REDACTED (Hynek Vychodil) Date: Fri, 24 Jul 2015 13:43:18 +0200 Subject: [erlang-questions] How is graph like this generated? In-Reply-To: <160532AE-2651-411B-A11F-B601D1CE0277@inaka.net> References: <160532AE-2651-411B-A11F-B601D1CE0277@inaka.net> Message-ID: There is example in official documentation showing graph generation without involving internal erlang tracing in tutorial http://www.erlang.org/doc/apps/et/et_tutorial.html -module(et_display_demo). -export([test/0]). test() -> {ok, Viewer} = et_viewer:start([{title,"Coffee Order"}, {max_actors,10}]), Drink = {drink,iced_chai_latte}, Size = {size,grande}, Milk = {milk,whole}, Flavor = {flavor,vanilla}, C = et_viewer:get_collector_pid(Viewer), et_collector:report_event(C,99,customer,barrista1,place_order,[Drink,Size,Milk,Flavor]), et_collector:report_event(C,80,barrista1,register,enter_order,[Drink,Size,Flavor]), et_collector:report_event(C,80,register,barrista1,give_total,"$5"), et_collector:report_event(C,80,barrista1,barrista1,get_cup,[Drink,Size]), et_collector:report_event(C,80,barrista1,barrista2,give_cup,[]), et_collector:report_event(C,90,barrista1,customer,request_money,"$5"), et_collector:report_event(C,90,customer,barrista1,pay_money,"$5"), et_collector:report_event(C,80,barrista2,barrista2,get_chai_mix,[]), et_collector:report_event(C,80,barrista2,barrista2,add_flavor,[Flavor]), et_collector:report_event(C,80,barrista2,barrista2,add_milk,[Milk]), et_collector:report_event(C,80,barrista2,barrista2,add_ice,[]), et_collector:report_event(C,80,barrista2,barrista2,swirl,[]), et_collector:report_event(C,80,barrista2,customer,give_tasty_beverage,[Drink,Size]), ok. I think it is a pretty descriptive demo. Cheers! H. On Fri, Jul 24, 2015 at 12:46 PM, Fernando 'Brujo' Benavides < fernando.benavides@REDACTED> wrote: > Based on that great example, and the easter egg I recently found > (et:phone_home/4,5), I created this example module: > https://github.com/elbrujohalcon/witchcraft/blob/master/src/ufo.erl > > To see it in action, do? > 1> ufo:start(). > 2> ufo:run(). > > You will see stuff like this going around: > http://g.recordit.co/9KvFR7FRqN.gif > > Cheers! > > [image: --] > Fernando "Brujo" Benavides > [image: http://]about.me/elbrujohalcon > > > > > > On Jul 24, 2015, at 06:13, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > > > On Fri, Jul 24, 2015 at 9:54 AM, Leo Liu wrote: > >> I wonder how those graphs are generated? Thanks. > > > I think those graphs are generated by a tool. Either manually, or > automatically. The have many names, one of which is "Sequence Diagrams" and > another one being "Message Sequence Chart (MSC)". Usually there is specific > meaning attached to different shapes in them, but the general rule is that > time flows downwards and you can see the interaction between several agents. > > In Erlang, we have a tool for generating these, which is the `et` > application (short for event tracer). It allows you to add dummy function > calls into your application and then record what the application is doing, > automatically generating a diagram for you. My post is somewhat old, but I > still think it is applicable: > > > http://jlouisramblings.blogspot.dk/2011/10/using-event-tracer-tool-set-in-erlang.html > > That way, you can build diagrams automatically by attaching trace probes > to the report_event function. > > > > -- > J. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Fri Jul 24 14:30:30 2015 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 24 Jul 2015 13:30:30 +0100 Subject: [erlang-questions] How is graph like this generated? In-Reply-To: References: Message-ID: On 24 July 2015 at 08:54, Leo Liu wrote: > Hi there, > > There are a couple of graphs in this talk: Erlang Factory SF 2015 - > Anthony Molinaro - How to pick a Pool in Erlang without Drowning > https://youtu.be/GO_97_6w5lU?t=1101 > > I wonder how those graphs are generated? Thanks. > > Have a look at PlantUML. It allows you to draw very nice message sequence diagrams by specifying the interactions textually. cheers Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From madhukars@REDACTED Fri Jul 24 12:29:06 2015 From: madhukars@REDACTED (Madhukar Sah) Date: Fri, 24 Jul 2015 15:59:06 +0530 Subject: [erlang-questions] snmp trap double oid problem Message-ID: <55B21372.1030505@utl.in> Hi, In case of sending a trap in erlang-snmp, a person is required to define two oid's in the MIB. This restriction exists only in erlang. In other agents like net-snmp only a single oid is used to represent the trap. Is it possible to have a single OID for trap in erlang as well. Regards, Madhukar Sah From sdl.web@REDACTED Sat Jul 25 01:37:26 2015 From: sdl.web@REDACTED (Leo Liu) Date: Sat, 25 Jul 2015 07:37:26 +0800 Subject: [erlang-questions] How is graph like this generated? References: Message-ID: Thanks all for the excellent answers especially for pointing out the OTP `et' application which looks like a nice tool. From mallen@REDACTED Sat Jul 25 03:14:14 2015 From: mallen@REDACTED (Mark Allen) Date: Fri, 24 Jul 2015 20:14:14 -0500 Subject: [erlang-questions] [ANN] Lager 3.0.0 is here In-Reply-To: References: Message-ID: Max, you'd have to explicitly create a new sink and then log messages to it. For example if you set up a sink named `foo' you could send messages into the foo sink by using foo:info("~p", [<<"I said foo">>]). The `foo' information would get picked up by the parse transformation code and processed at compile time to direct any output (depending on log level or traces active) to a gen_event process named foo_lager_event. Whatever backends happen to be installed in foo_lager_event would do the actual work of processing a message to disk, or the console or syslog, or whatever. On Fri, Jul 24, 2015 at 1:48 AM, Max Lapshin wrote: > Thanks, I've read it. > > I don't see how can I tell this exact log mesage to go to audit sink, not > to default sink. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonym@REDACTED Sat Jul 25 05:29:36 2015 From: anthonym@REDACTED (ANTHONY MOLINARO) Date: Fri, 24 Jul 2015 20:29:36 -0700 Subject: [erlang-questions] How is graph like this generated? In-Reply-To: References: Message-ID: <79932F3C-199D-42E1-8816-778769759C2A@alumni.caltech.edu> > On Jul 24, 2015, at 12:54 AM, Leo Liu wrote: > > There are a couple of graphs in this talk: Erlang Factory SF 2015 - > Anthony Molinaro - How to pick a Pool in Erlang without Drowning > https://youtu.be/GO_97_6w5lU?t=1101 I?m not sure if you mean the graphs which were generated with gnuplot or the sequence diagrams which were generated with a tool called mscgen (http://www.mcternan.me.uk/mscgen/ ). Either way, I have all the source files here (https://github.com/djnym/erlang-pool-research/blob/master /). The notes directory has the .msc files and the Makefile has the commands used to generate them. The root contains a gnuplot_in fie which are the commands sent to gnuplot to generate the graphs. Hope that helps, -Anthony -------------- next part -------------- An HTML attachment was scrubbed... URL: From evnix.com@REDACTED Sat Jul 25 14:22:37 2015 From: evnix.com@REDACTED (avinash D'silva) Date: Sat, 25 Jul 2015 17:52:37 +0530 Subject: [erlang-questions] How to access a Map returned by jiffy:decode() Message-ID: I have the following code. Tx=jiffy:decode(<<"{\"foo\": \"bar\",\"key\": \"val\"}">>). where Tx is: {[{<<"foo">>,<<"bar">>},{<<"key">>,<<"val">>}]} I tried the following: maps:get("foo",Tx). ** exception error: {badmap,{[{<<"foo">>,<<"bar">>},{<<"key">>,<<"val">>}]}} in function maps:get/2 called as maps:get("foo", {[{<<"foo">>,<<"bar">>},{<<"key">>,<<"val">>}]}) Any Idea on what I am doing wrong? any possible solutions? I Couldn't find any documentation to access this type of map. -------------- next part -------------- An HTML attachment was scrubbed... URL: From muhamed.huseinbasic@REDACTED Sat Jul 25 14:25:33 2015 From: muhamed.huseinbasic@REDACTED (=?iso-8859-2?B?TXVoYW1lZCBIdXNlaW5iYblp5g==?=) Date: Sat, 25 Jul 2015 14:25:33 +0200 Subject: [erlang-questions] Possible improvement of getting started tutorial Message-ID: Dear, I have recently started learning Erlang. Google search for 'Getting started with Erlang' gives this link: http://www.erlang.org/download/getting_started-5.4.pdf as one of the first results. I have started reading it. However, after some time, I have realized that there is this book as well: http://www.erlang.org/doc/pdf/otp-system-documentation.pdf. It is newer book and part of it treats the same content as previous link. My suggestion is to remove the previous version (5.4) with the new version (7.0). If that is not the good idea, then it would be good at least to change the pdf of the old version (5.4) and to put the note at the beginning of the book so that future readers are aware of the new version. Regards, Muhamed Huseinba?i? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rtrlists@REDACTED Sat Jul 25 15:28:36 2015 From: rtrlists@REDACTED (Robert Raschke) Date: Sat, 25 Jul 2015 14:28:36 +0100 Subject: [erlang-questions] How to access a Map returned by jiffy:decode() In-Reply-To: References: Message-ID: That's not a map. It's a tuple of one element, which is a list of two-tuples ({key, value}). Such a list is often called a proplist. You could get your results like this: {Tx} = jiffy:decode... proplists:get_value(<<"foo">>, Tx) Hth, Robby On Jul 25, 2015 2:02 PM, "avinash D'silva" wrote: > I have the following code. > Tx=jiffy:decode(<<"{\"foo\": \"bar\",\"key\": \"val\"}">>). > > where Tx is: > {[{<<"foo">>,<<"bar">>},{<<"key">>,<<"val">>}]} > > I tried the following: > > maps:get("foo",Tx). > ** exception error: > {badmap,{[{<<"foo">>,<<"bar">>},{<<"key">>,<<"val">>}]}} > in function maps:get/2 > called as maps:get("foo", > {[{<<"foo">>,<<"bar">>},{<<"key">>,<<"val">>}]}) > > Any Idea on what I am doing wrong? any possible solutions? > I Couldn't find any documentation to access this type of map. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.shneyderman@REDACTED Sat Jul 25 15:29:26 2015 From: a.shneyderman@REDACTED (Alex Shneyderman) Date: Sat, 25 Jul 2015 09:29:26 -0400 Subject: [erlang-questions] erl_tar limitation or a bug? Message-ID: This is form the erl_tar docs: "For maximum compatibility, it is safe to archive files with names up to 100 characters in length. Such tar files can generally be extracted by any tar program. If filenames exceed 100 characters in length, the resulting tar file can only be correctly extracted by a POSIX-compatible tar program (such as Solaris tar), not by GNU tar. File have longer names than 256 bytes cannot be stored at all. The filename of the file a symbolic link points is always limited to 100 characters." It is not clear to me from this description if erl_tar should be able to produce tar files that include file names longer than 100 chars long and shorter than 256? >From my experiments I know such files are not possible to add to tars with erl_tar. There is a problem with this setting. https://github.com/erlang/otp/blob/maint/lib/stdlib/src/erl_tar.erl#L266 that leads to an exception when call to split_filename(...) -> filenme:join([]) being made. So, the question I have is this an error in the docs or is this an error in code (the fix is to change that setting to 256, I think)? Cheers, Alex. From kennethlakin@REDACTED Sat Jul 25 18:58:03 2015 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Sat, 25 Jul 2015 09:58:03 -0700 Subject: [erlang-questions] How to access a Map returned by jiffy:decode() In-Reply-To: References: Message-ID: <55B3C01B.80703@gmail.com> On 07/25/2015 05:22 AM, avinash D'silva wrote: > I have the following code. > Tx=jiffy:decode(<<"{\"foo\": \"bar\",\"key\": \"val\"}">>). > > where Tx is: > {[{<<"foo">>,<<"bar">>},{<<"key">>,<<"val">>}]} You want to pass the return_maps option: Tx=jiffy:decode(<<"{\"foo\": \"bar\",\"key\": \"val\"}">>, [return_maps]). See: https://github.com/davisp/jiffy#jiffydecode12 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From matthew.fitzpatrick6012@REDACTED Sat Jul 25 15:30:12 2015 From: matthew.fitzpatrick6012@REDACTED (Matthew Fitzpatrick) Date: Sat, 25 Jul 2015 08:30:12 -0500 Subject: [erlang-questions] How to access a Map returned by jiffy:decode() In-Reply-To: References: Message-ID: Hey avinash, By default, Jiffy returns "EJSON", not a map. It looks liked you need to use an Option for Jiffy to return a map. ```erlang Tx=jiffy:decode(<<"{\"foo\": \"bar\",\"key\": \"val\"}">>, [return_maps]). ``` jiffy:decode/1,2 - jiffy:decode(IoData) - jiffy:decode(IoData, Options) The options for decode are: - return_maps - Tell Jiffy to return objects using the maps data type on VMs that support it. This raises an error on VMs that don't support maps. - {null_term, Term} - Returns the specified Term instead of null when decoding JSON. This is for people that wish to use undefined instead of null. - use_nil - Returns the atom nil instead of null when decoding JSON. This is a short hand for {null_term, nil}. - return_trailer - If any non-whitespace is found after the first JSON term is decoded the return value of decode/2 becomes {has_trailer, FirstTerm, RestData::iodata()}. This is useful to decode multiple terms in a single binary. - {bytes_per_red, N} where N >= 0 - This controls the number of bytes that Jiffy will process as an equivalent to a reduction. Each 20 reductions we consume 1% of our allocated time slice for the current process. When the Erlang VM indicates we need to return from the NIF. - {bytes_per_iter, N} where N >= 0 - Backwards compatible option that is converted into the bytes_per_red value. On Sat, Jul 25, 2015 at 7:22 AM, avinash D'silva wrote: > I have the following code. > Tx=jiffy:decode(<<"{\"foo\": \"bar\",\"key\": \"val\"}">>). > > where Tx is: > {[{<<"foo">>,<<"bar">>},{<<"key">>,<<"val">>}]} > > I tried the following: > > maps:get("foo",Tx). > ** exception error: > {badmap,{[{<<"foo">>,<<"bar">>},{<<"key">>,<<"val">>}]}} > in function maps:get/2 > called as maps:get("foo", > {[{<<"foo">>,<<"bar">>},{<<"key">>,<<"val">>}]}) > > Any Idea on what I am doing wrong? any possible solutions? > I Couldn't find any documentation to access this type of map. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jusfeel@REDACTED Sun Jul 26 15:11:27 2015 From: jusfeel@REDACTED (=?GBK?B?zfXquw==?=) Date: Sun, 26 Jul 2015 21:11:27 +0800 (CST) Subject: [erlang-questions] Is this proper way to convert a latin string to utf8 string Message-ID: <254a8012.24490.14eca7d5390.Coremail.jusfeel@163.com> Hi, Does anyone know if this is a proper way to convert latin string to utf-8 string? {ok, S} = asn1rt:utf8_binary_to_list(list_to_binary([232,191,153])). io:format("~ts~n",[S]). Thanks, Hao -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sun Jul 26 18:14:16 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 26 Jul 2015 18:14:16 +0200 Subject: [erlang-questions] Is this proper way to convert a latin string to utf8 string In-Reply-To: <254a8012.24490.14eca7d5390.Coremail.jusfeel@163.com> References: <254a8012.24490.14eca7d5390.Coremail.jusfeel@163.com> Message-ID: On Sun, Jul 26, 2015 at 3:11 PM, ?? wrote: > Hi, > Does anyone know if this is a proper way to convert latin string to utf-8 > string? > > {ok, S} = asn1rt:utf8_binary_to_list(list_to_binary([232,191,153])). > io:format("~ts~n",[S]). > Use the `unicode` module for character conversion: 1> unicode:characters_to_binary([232,191,153], latin1, utf8). <<195,168,194,191,194,153>> 2> io:format("~ts~n", [v(1)]). It prints as three characters: LATIN SMALL LETTER E WITH GRAVE INVERTED QUESTION MARK (unbound 0x0099 part of the Latin-1 supplement range) I don't know if this is correct for you. What are you trying to do generally? That is, what is the problem you are having. Perhaps we can give better help if we know your situation. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sun Jul 26 21:37:34 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 26 Jul 2015 21:37:34 +0200 Subject: [erlang-questions] Is this proper way to convert a latin string to utf8 string In-Reply-To: <61ad39.1b5a.14ecbd84a49.Coremail.jusfeel@163.com> References: <254a8012.24490.14eca7d5390.Coremail.jusfeel@163.com> <61ad39.1b5a.14ecbd84a49.Coremail.jusfeel@163.com> Message-ID: On Sun, Jul 26, 2015 at 9:30 PM, ?? wrote: > I am using a web framework(Chicagoboss). I posted the data into Erlang in > Chinese in utf8 encoded string from a web form. It is read by Erlang as > [232,191,153]. This is just one single Chinese character. But erlang read > it as [232,191,153]. So I want to consume via ajax later on on the client > side. Ah, you have three bytes, 232, 192, 153 in utf8 representation and want to convert those into a unicode codepoint. Here I have a binary with a bit of other utf8 characters in it (since ASCII and utf8 overlaps): 9> B = <<232, 191, 153, $h, $e, $l, $l, $o>>. <<232,191,153,104,101,108,108,111>> 10> [CP || <> <= B]. [36825,104,101,108,108,111] -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jusfeel@REDACTED Sun Jul 26 21:30:26 2015 From: jusfeel@REDACTED (=?GBK?B?zfXquw==?=) Date: Mon, 27 Jul 2015 03:30:26 +0800 (CST) Subject: [erlang-questions] Is this proper way to convert a latin string to utf8 string In-Reply-To: References: <254a8012.24490.14eca7d5390.Coremail.jusfeel@163.com> Message-ID: <61ad39.1b5a.14ecbd84a49.Coremail.jusfeel@163.com> I think my questions is wrong. It is not a latin1 encoded string. It is actually utf8 encoded string but be read by Erlang into [232,191,153] represented by a latin - charlist. A simple list_to_binary will give me back the UTF8 already. It's just I need a list to accommodate other part of the program. I am using a web framework(Chicagoboss). I posted the data into Erlang in Chinese in utf8 encoded string from a web form. It is read by Erlang as [232,191,153]. This is just one single Chinese character. But erlang read it as [232,191,153]. So I want to consume via ajax later on on the client side. But because this piece of information is in a blob of long json data and it is needed to be converted to binary before sending down the wire. So in order to make this piece of information to be correctly converted as one part of the whole assembled json, it needs to turn to a utf8 list first like this: asn1rt:utf8_binary_to_list(list_to_binary([232,191,153])), this will give me [36825] which represent the same Chinese character as <<232,191,153>>. You can test this by io:format("~ts~n",[[36825]]). and io:format("~ts~n",[<<232,191,153>>]). They all output the same character: ? then later, asn1rt:utf8_list_to_binary will convert all the json data together to binary. -- Hao ? 2015-07-27 00:14:16?"Jesper Louis Andersen" ??? On Sun, Jul 26, 2015 at 3:11 PM, ?? wrote: Hi, Does anyone know if this is a proper way to convert latin string to utf-8 string? {ok, S} = asn1rt:utf8_binary_to_list(list_to_binary([232,191,153])). io:format("~ts~n",[S]). Use the `unicode` module for character conversion: 1> unicode:characters_to_binary([232,191,153], latin1, utf8). <<195,168,194,191,194,153>> 2> io:format("~ts~n", [v(1)]). It prints as three characters: LATIN SMALL LETTER E WITH GRAVE INVERTED QUESTION MARK (unbound 0x0099 part of the Latin-1 supplement range) I don't know if this is correct for you. What are you trying to do generally? That is, what is the problem you are having. Perhaps we can give better help if we know your situation. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jusfeel@REDACTED Sun Jul 26 21:45:44 2015 From: jusfeel@REDACTED (=?GBK?B?zfXquw==?=) Date: Mon, 27 Jul 2015 03:45:44 +0800 (CST) Subject: [erlang-questions] Is this proper way to convert a latin string to utf8 string In-Reply-To: References: <254a8012.24490.14eca7d5390.Coremail.jusfeel@163.com> <61ad39.1b5a.14ecbd84a49.Coremail.jusfeel@163.com> Message-ID: <717e84b8.1d83.14ecbe64a68.Coremail.jusfeel@163.com> This is so cool. Thanks! -- Hao ? 2015-07-27 03:37:34?"Jesper Louis Andersen" ??? On Sun, Jul 26, 2015 at 9:30 PM, ?? wrote: I am using a web framework(Chicagoboss). I posted the data into Erlang in Chinese in utf8 encoded string from a web form. It is read by Erlang as [232,191,153]. This is just one single Chinese character. But erlang read it as [232,191,153]. So I want to consume via ajax later on on the client side. Ah, you have three bytes, 232, 192, 153 in utf8 representation and want to convert those into a unicode codepoint. Here I have a binary with a bit of other utf8 characters in it (since ASCII and utf8 overlaps): 9> B = <<232, 191, 153, $h, $e, $l, $l, $o>>. <<232,191,153,104,101,108,108,111>> 10> [CP || <> <= B]. [36825,104,101,108,108,111] -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Mon Jul 27 00:58:34 2015 From: sdl.web@REDACTED (Leo Liu) Date: Mon, 27 Jul 2015 06:58:34 +0800 Subject: [erlang-questions] How is graph like this generated? References: <79932F3C-199D-42E1-8816-778769759C2A@alumni.caltech.edu> Message-ID: On 2015-07-25 11:29 +0800, ANTHONY MOLINARO wrote: > I?m not sure if you mean the graphs which were generated with gnuplot > or the sequence diagrams which were generated with a tool called > mscgen (http://www.mcternan.me.uk/mscgen/ > ). The latter. `mscgen' looks interesting. > Either way, I have all the source files here > (https://github.com/djnym/erlang-pool-research/blob/master > /). The > notes directory has the .msc files and the Makefile has the commands > used to generate them. The root contains a gnuplot_in fie which are > the commands sent to gnuplot to generate the graphs. > > Hope that helps, Yes it helps. Leo From dominik_pawlak@REDACTED Mon Jul 27 16:10:10 2015 From: dominik_pawlak@REDACTED (Dominik Pawlak) Date: Mon, 27 Jul 2015 16:10:10 +0200 Subject: [erlang-questions] snmp trap double oid problem In-Reply-To: <55B21372.1030505@utl.in> References: <55B21372.1030505@utl.in> Message-ID: <55B63BC2.4020903@yahoo.co.uk> Hello, Could you please give a link to where that restriction is stated? I'm using something like this in 17.1 : in MIB: testMIBFirstTrap NOTIFICATION-TYPE OBJECTS { sysContact, sysName } STATUS current DESCRIPTION "First trap." ::= { testMIBTraps 1 } in code: snmpa:send_notification(snmp_master_agent, testMIBFirstTrap, no_receiver, NotifyName, []) Regards, Dominik On 24.07.2015 12:29, Madhukar Sah wrote: > Hi, > > In case of sending a trap in erlang-snmp, a person is required to > define two oid's in the MIB. This restriction exists only in erlang. > In other agents like net-snmp only a single oid is used to represent > the trap. > Is it possible to have a single OID for trap in erlang as well. > > > Regards, > Madhukar Sah > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mattevans123@REDACTED Mon Jul 27 16:25:53 2015 From: mattevans123@REDACTED (Matthew Evans) Date: Mon, 27 Jul 2015 10:25:53 -0400 Subject: [erlang-questions] gen_server and init Message-ID: Hi, We have used this pattern in gen_servers for a long time now: init(_) -> %% Some stuff {ok, #state{}, 0}. %%%%%% handle_info(timeout, State) -> %% Init stuff {noreply,State} Basically the idea is to prevent init from blocking and to have init/1 return with a timeout of 0 causing an immediate timeout message to be invoked in the handle_info. Here's what's odd. Sometimes this timeout does not fire. It doesn't appear that any message is getting sent, but I would imagine that since it's a registered gen_server there is no way that can happen. Does anyone have any ideas? We are running vsn 17.1, and ntp is enabled on the host (Linux). Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Mon Jul 27 16:34:58 2015 From: sergej.jurecko@REDACTED (Sergej =?UTF-8?B?SnVyZcSNa28=?=) Date: Mon, 27 Jul 2015 16:34:58 +0200 Subject: [erlang-questions] gen_server and init In-Reply-To: References: Message-ID: <25402A16-E81F-4ED9-939F-8F0C0C1E1486@gmail.com> If something is in the message queue before init is done executing, that timeout will never get called. Timeout means execute only if nothing else happens during. You should be using self() ! timeout Sergej From: on behalf of Matthew Evans Date: Monday 27 July 2015 16:25 To: "erlang-questions@REDACTED" Subject: [erlang-questions] gen_server and init Hi, We have used this pattern in gen_servers for a long time now: init(_) -> %% Some stuff {ok, #state{}, 0}. %%% %%% handle_info(timeout, State) -> %% Init stuff {noreply,State} Basically the idea is to prevent init from blocking and to have init/1 return with a timeout of 0 causing an immediate timeout message to be invoked in the handle_info. Here's what's odd. Sometimes this timeout does not fire. It doesn't appear that any message is getting sent, but I would imagine that since it's a registered gen_server there is no way that can happen. Does anyone have any ideas? We are running vsn 17.1, and ntp is enabled on the host (Linux). Thanks _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Jul 27 16:40:30 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Mon, 27 Jul 2015 16:40:30 +0200 Subject: [erlang-questions] gen_server and init In-Reply-To: References: Message-ID: <55B642DE.6040706@ninenines.eu> This and sending yourself a message is a bad idea. It will usually work, until it doesn't, and you will have a very hard time figuring out why. Instead, start the process using proc_lib:spawn_link or proc_lib:start_link (depending on whether it needs to be synchronous or not), then perform your initialization (calling proc_lib:init_ack where appropriate), and finally calling gen_server:enter_loop. What this gives you is pretty much the ability to customize the initialization of your gen_server process. This is the correct way to do it. Your solution can fail if you receive a message. Sending yourself a message is subject to race conditions where you receive a message before you could init. On 07/27/2015 04:25 PM, Matthew Evans wrote: > Hi, > > We have used this pattern in gen_servers for a long time now: > > init(_) -> > %% Some stuff > {ok, #state{}, 0}. > > %%% > %%% > > handle_info(timeout, State) -> > %% Init stuff > {noreply,State} > > Basically the idea is to prevent init from blocking and to have init/1 > return with a timeout of 0 causing an immediate timeout message to be > invoked in the handle_info. > > Here's what's odd. Sometimes this timeout does not fire. It doesn't > appear that any message is getting sent, but I would imagine that since > it's a registered gen_server there is no way that can happen. > > Does anyone have any ideas? > > We are running vsn 17.1, and ntp is enabled on the host (Linux). > > Thanks > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From mattevans123@REDACTED Mon Jul 27 16:44:20 2015 From: mattevans123@REDACTED (Matthew Evans) Date: Mon, 27 Jul 2015 10:44:20 -0400 Subject: [erlang-questions] gen_server and init In-Reply-To: <55B642DE.6040706@ninenines.eu> References: , <55B642DE.6040706@ninenines.eu> Message-ID: Thanks guys, I guess I was under the assumption that a process could not even get any messages until init had returned and the pid was registered. > Date: Mon, 27 Jul 2015 16:40:30 +0200 > From: essen@REDACTED > To: mattevans123@REDACTED; erlang-questions@REDACTED > Subject: Re: [erlang-questions] gen_server and init > > This and sending yourself a message is a bad idea. It will usually work, > until it doesn't, and you will have a very hard time figuring out why. > > Instead, start the process using proc_lib:spawn_link or > proc_lib:start_link (depending on whether it needs to be synchronous or > not), then perform your initialization (calling proc_lib:init_ack where > appropriate), and finally calling gen_server:enter_loop. > > What this gives you is pretty much the ability to customize the > initialization of your gen_server process. > > This is the correct way to do it. Your solution can fail if you receive > a message. Sending yourself a message is subject to race conditions > where you receive a message before you could init. > > On 07/27/2015 04:25 PM, Matthew Evans wrote: > > Hi, > > > > We have used this pattern in gen_servers for a long time now: > > > > init(_) -> > > %% Some stuff > > {ok, #state{}, 0}. > > > > %%% > > %%% > > > > handle_info(timeout, State) -> > > %% Init stuff > > {noreply,State} > > > > Basically the idea is to prevent init from blocking and to have init/1 > > return with a timeout of 0 causing an immediate timeout message to be > > invoked in the handle_info. > > > > Here's what's odd. Sometimes this timeout does not fire. It doesn't > > appear that any message is getting sent, but I would imagine that since > > it's a registered gen_server there is no way that can happen. > > > > Does anyone have any ideas? > > > > We are running vsn 17.1, and ntp is enabled on the host (Linux). > > > > Thanks > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Mon Jul 27 16:59:40 2015 From: rvirding@REDACTED (Robert Virding) Date: Mon, 27 Jul 2015 16:59:40 +0200 Subject: [erlang-questions] Erlounge in Bristol next week Message-ID: I will be in Bristol next week together with some colleagues to hold a course. Any local erlangers or LFEers or elixirers or functional programmers interested in an erlounge? We will be there 2-7/8. Robert -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Jul 27 17:03:10 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 27 Jul 2015 18:03:10 +0300 Subject: [erlang-questions] gen_server and init In-Reply-To: References: <55B642DE.6040706@ninenines.eu> Message-ID: Loic is right, but you should understand that you can receive message before your "self() ! init" message only if you explicitly tell your pid someone in init() because before the end of init your pid is unknown to others. -------------- next part -------------- An HTML attachment was scrubbed... URL: From donpedrothird@REDACTED Mon Jul 27 17:27:44 2015 From: donpedrothird@REDACTED (John Doe) Date: Mon, 27 Jul 2015 18:27:44 +0300 Subject: [erlang-questions] riak client best practices Message-ID: Hello, I wonder if there are any best practices on using Riak cluster from Erlang services, such as Cowboy web services. For example - use riakc protobuf client or join services and Riak ring in a common erlang cluster with the same cookie? - open a new connection from riakc to Riak node on each hit or keep a connection pool? I suppose it should be the pool as sockets are finite. - on every request connect to a random Riak node from the ring? Or use dedicated Riak nodes for each service instance, if there are many? - what to do with the new nodes which are joining the ring and the ones which are removed from the ring? What is the more or less standard way to notify clients about this? - what to do in a client if a Riak node temporarily goes offline? Keep trying to connect to such node until it gets online again? This would obviously hit latency unless there's a special dedicated process for this. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Mon Jul 27 18:45:33 2015 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Mon, 27 Jul 2015 18:45:33 +0200 Subject: [erlang-questions] gen_server and init In-Reply-To: Message-ID: <9906a4fa-e264-42a2-b579-4f30dbab7c00@email.android.com> Hi, To see this, do a sync call to your gen server in the init function. Your process will stuck forever, by launching observer, on your process detail, you will see your message waiting in its process message queue, until process is killed. Le?27 juil. 2015 16:44, Matthew Evans a ?crit?: > > Thanks guys, > > I guess I was under the assumption that a process could not even get any messages until init had returned and the pid was registered. > > > Date: Mon, 27 Jul 2015 16:40:30 +0200 > > From: essen@REDACTED > > To: mattevans123@REDACTED; erlang-questions@REDACTED > > Subject: Re: [erlang-questions] gen_server and init > > > > This and sending yourself a message is a bad idea. It will usually work, > > until it doesn't, and you will have a very hard time figuring out why. > > > > Instead, start the process using proc_lib:spawn_link or > > proc_lib:start_link (depending on whether it needs to be synchronous or > > not), then perform your initialization (calling proc_lib:init_ack where > > appropriate), and finally calling gen_server:enter_loop. > > > > What this gives you is pretty much the ability to customize the > > initialization of your gen_server process. > > > > This is the correct way to do it. Your solution can fail if you receive > > a message. Sending yourself a message is subject to race conditions > > where you receive a message before you could init. > > > > On 07/27/2015 04:25 PM, Matthew Evans wrote: > > > Hi, > > > > > > We have used this pattern in gen_servers for a long time now: > > > > > > init(_) -> > > > %% Some stuff > > > {ok, #state{}, 0}. > > > > > > %%% > > > %%% > > > > > > handle_info(timeout, State) -> > > > %% Init stuff > > > {noreply,State} > > > > > > Basically the idea is to prevent init from blocking and to have init/1 > > > return with a timeout of 0 causing an immediate timeout message to be > > > invoked in the handle_info. > > > > > > Here's what's odd. Sometimes this timeout does not fire. It doesn't > > > appear that any message is getting sent, but I would imagine that since > > > it's a registered gen_server there is no way that can happen. > > > > > > Does anyone have any ideas? > > > > > > We are running vsn 17.1, and ntp is enabled on the host (Linux). > > > > > > Thanks > > > > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > -- > > Lo?c Hoguin > > http://ninenines.eu > > Author of The Erlanger Playbook, > > A book about software development using Erlang From k.petrauskas@REDACTED Mon Jul 27 18:59:32 2015 From: k.petrauskas@REDACTED (Karolis Petrauskas) Date: Mon, 27 Jul 2015 19:59:32 +0300 Subject: [erlang-questions] gen_server and init In-Reply-To: References: <55B642DE.6040706@ninenines.eu> Message-ID: I think this is not the truth anymore, as pids are now reused. You can get a message that was meant for previously living process with the same PID. Karolis On Mon, Jul 27, 2015 at 6:03 PM, Max Lapshin wrote: > Loic is right, but you should understand that you can receive message before > your "self() ! init" message only if you explicitly tell your pid someone > in init() because before the end of init your pid is unknown to others. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From jesper.louis.andersen@REDACTED Mon Jul 27 20:29:17 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 27 Jul 2015 20:29:17 +0200 Subject: [erlang-questions] gen_server and init In-Reply-To: <55B642DE.6040706@ninenines.eu> References: <55B642DE.6040706@ninenines.eu> Message-ID: On Mon, Jul 27, 2015 at 4:40 PM, Lo?c Hoguin wrote: > This and sending yourself a message is a bad idea. It will usually work, > until it doesn't, and you will have a very hard time figuring out why. I'm quite curious. What is the scenario where this fails? I have never observed this in practice on very busy systems, and I think going through proc_lib for this is a detour I'd rather not we'd have to take. Here are two modules, z0 and z.erl. They are very boring gen servers: -module(z0). -behaviour(gen_server). -ifdef(PULSE). -include_lib("pulse_otp/include/pulse_otp.hrl"). -endif. -export([start_link/0]). %% Operational API -export([read/0, read_p/1]). %% gen_server API -export([ init/1, handle_cast/2, handle_call/3, terminate/2, code_change/3, handle_info/2 ]). %% API start_link() -> gen_server:start_link({local, ?MODULE}, ?MODULE, [], []). read() -> gen_server:call(?MODULE, read). read_p(P) -> gen_server:call(P, read). %% Callbacks init([]) -> {ok, initializing, 0}. handle_call(read, _From, State) -> {reply, State, State}. handle_cast(_M, State) -> {noreply, State}. handle_info(timeout, _State) -> {noreply, ready}. terminate(_How, _State) -> ok. code_change(_OldVsn, State, _Extra) -> {ok, State}. --------------------- -module(z). -behaviour(gen_server). -ifdef(PULSE). -include_lib("pulse_otp/include/pulse_otp.hrl"). -endif. -export([start_link/0]). %% Operational API -export([read/0, read_p/1]). %% gen_server API -export([ init/1, handle_cast/2, handle_call/3, terminate/2, code_change/3, handle_info/2 ]). %% API start_link() -> gen_server:start_link({local, ?MODULE}, ?MODULE, [], []). read() -> gen_server:call(?MODULE, read). read_p(P) -> gen_server:call(P, read). %% Callbacks init([]) -> self() ! timeout, {ok, initializing}. handle_call(read, _From, State) -> {reply, State, State}. handle_cast(_M, State) -> {noreply, State}. handle_info(timeout, _State) -> {noreply, ready}. terminate(_How, _State) -> ok. code_change(_OldVsn, State, _Extra) -> {ok, State}. --------------------- Ok, now with these down, we can use EQC and PULSE to generate the counterexample if any should be in there: -module(z_eqc). -include_lib("eqc/include/eqc.hrl"). -include_lib("eqc/include/eqc_statem.hrl"). -include_lib("pulse/include/pulse.hrl"). -include_lib("pulse_otp/include/pulse_otp.hrl"). -compile(export_all). -record(state, { init, ref }). initial_state() -> #state { init = false }. server_start() -> {ok, Pid} = z:start_link(), unlink(Pid), Pid. %% SERVER START_LINK server_start_pre(S) -> not initialized(S). server_start_args(_S) -> []. server_start_next(S, Ref, []) -> S#state { ref = Ref, init = true }. server_start_post(_S, [], Ref) -> is_pid(Ref). %% READ BY PID read_p(P) -> z:read_p(P). read_p_pre(S) -> initialized(S). read_p_args(#state { ref = Ref }) -> [Ref]. read_p_return(_S, [_Ref]) -> ready. %% READ read() -> try z:read() of M -> M catch _Class:_Err -> {error, undefined} end. read_args(_S) -> []. read_return(#state { init = false}, []) -> {error, undefined}; read_return(#state { init = true}, []) -> ready. %% Run a test under PULSE to randomize the process schedule as well. prop_model_pulse() -> ?SETUP(fun() -> setup(), fun() -> ok end end, ?LET(Shrinking, parameter(shrinking, false), ?FORALL(Cmds, parallel_commands(?MODULE), ?ALWAYS(if not Shrinking -> 1; Shrinking -> 20 end, ?PULSE(HSR={_, _, R}, begin ok = cleanup(), run_parallel_commands(?MODULE, Cmds) end, aggregate(command_names(Cmds), pretty_commands(?MODULE, Cmds, HSR, R == ok))))))). setup() -> error_logger:tty(false), ok. cleanup() -> case whereis(z) of undefined -> ok; Pid -> exit(Pid, kill), timer:sleep(3) end, ok. initialized(#state { init = I }) -> I. pulse_instrument() -> [ pulse_instrument(File) || File <- filelib:wildcard("*.erl") ], ok. pulse_instrument(File) -> io:format("Compiling: ~p~n", [File]), {ok, Mod} = compile:file(File, [{d, 'PULSE', true}, {d, 'WITH_PULSE', true}, {d, 'EQC_TESTING', true}, {parse_transform, pulse_instrument}]), code:purge(Mod), code:load_file(Mod), Mod. ----------------------------- However, when I run this, I get no errors, even though I'm trying to behave non-nicely: I won't call the Pid until I know about it. I will blindly call the name, z, and if I get an error on this, I'll verify that the possible linearization is that the process was not started yet. If I get a result, I force it to be ready: 60> eqc:module({testing_time, 30}, z_eqc). prop_model_pulse: ....................................................................................................(x10)....................................................................................................(x100)............................(x10)....... Time limit reached: 30.0 seconds. OK, passed 3970 tests 49.857% {z_eqc,read,0} 44.492% {z_eqc,read_p,1} 5.651% {z_eqc,server_start,0} [] ... So: What is the scenario where this approach fails? -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Jul 27 21:14:37 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Mon, 27 Jul 2015 21:14:37 +0200 Subject: [erlang-questions] gen_server and init In-Reply-To: References: <55B642DE.6040706@ninenines.eu> Message-ID: <55B6831D.60304@ninenines.eu> (Had problems pasting so hope it's all OK!) I'm not sure what you wrote there but I can give you two scenarios where it can fail off the top of my head. The first is very unlikely and can only fail if you use the 0 timeout, while the second is actually much easier to observe and can fail with both methods: P1 calls start_link P2 init (returns 0 timeout) P2 yields before calling receive P1 returns from start_link and sends P2 a message P2 receives message And: P1 calls start_link P2 in init subscribes to some kind of pubsub PS PS sends P2 message(s) P2 returns from init and receive those messages I'm not sure why your tool doesn't catch the first case, Concuerror *definitely does*, and without needing to write all this weird code too. :-) Here is the module: -module(z). -behaviour(gen_server). -export([start_link/0]). %% Operational API -export([read/0]). %% gen_server API -export([ init/1, handle_cast/2, handle_call/3, terminate/2, code_change/3, handle_info/2, test/0 ]). test() -> start_link(), ready = read(), stop(). %% API start_link() -> gen_server:start({local, ?MODULE}, ?MODULE, [], []). read() -> gen_server:call(?MODULE, read). stop() -> gen_server:call(?MODULE, stop). %% Callbacks init([]) -> {ok, initializing, 0}. handle_call(read, _From, State) -> {reply, State, State}; handle_call(stop, _, State) -> {stop, normal, ok, stop}. handle_cast(_M, State) -> {noreply, State}. handle_info(timeout, _State) -> {noreply, ready}. terminate(_How, _State) -> ok. code_change(_OldVsn, State, _Extra) -> {ok, State}. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Here is the Concuerror output: % erlc z.erl && ./concuerror -f z.erl -m z -k --after_timeout 1000 && cat concuerror_report.txt z.erl:42: Warning: variable 'State' is unused concuerror: WARNING: file z.erl shadows the default ./z.beam Concuerror started at 27 Jul 2015 19:08:58 Writing results in concuerror_report.txt Info: Instrumented z Info: Instrumented io_lib Info: Instrumented gen_server Info: Instrumented gen Info: Instrumented proc_lib Info: Instrumented erlang Info: Instrumented init Info: Instrumented sys Info: You can see pairs of racing instructions (in the report and --graph) with '--show_races true' Warning: Continuing after error Done! (Exit status: completed) Summary: 1 errors, 4/4 interleavings explored ################################################################################ Concuerror started with options: [{after_timeout,1000}, {assume_racing,true}, {depth_bound,5000}, {entry_point,{z,test,[]}}, {files,["z.erl"]}, {ignore_error,[]}, {instant_delivery,false}, {interleaving_bound,infinity}, {keep_going,true}, {non_racing_system,[]}, {optimal,true}, {print_depth,20}, {scheduling,round_robin}, {scheduling_bound,infinity}, {scheduling_bound_type,none}, {show_races,false}, {strict_scheduling,false}, {symbolic_names,true}, {timeout,1000}, {treat_as_normal,[]}] ################################################################################ Erroneous interleaving 1: * At step 24 process P exited abnormally Reason: {{badmatch,initializing},[{z,test,0,[{file,"z.erl"},{line,22}]}]} Stacktrace: [{z,test,0,[{file,"z.erl"},{line,22}]}] * Blocked at a 'receive' (when all other processes have exited): P.1 in gen_server.erl line 360 -------------------------------------------------------------------------------- Interleaving info: 1: P: undefined = erlang:whereis(z) in gen.erl line 284 2: P: [] = erlang:process_info(P, registered_name) in proc_lib.erl line 675 3: P: {P.1,#Ref<0.0.0.177>} = erlang:spawn_opt({proc_lib,init_p,[P,[],gen,init_it,[gen_server,P,self,{local,z},z,[],[]]],[monitor]}) in erlang.erl line 2496 4: P.1: undefined = erlang:put('$ancestors', [P]) in proc_lib.erl line 231 5: P.1: undefined = erlang:put('$initial_call', {z,init,1}) in proc_lib.erl line 232 6: P.1: true = erlang:register(z, P.1) in gen.erl line 287 7: P.1: {P.1,{get_argument,generic_debug}} = init ! {P.1,{get_argument,generic_debug}} in init.erl line 145 8: Message ({P.1,{get_argument,generic_debug}}) from P.1 reaches init 9: Message ({init,error}) from init reaches P.1 10: P.1: receives message ({init,error}) in init.erl line 146 11: P.1: {ack,P.1,{ok,P.1}} = P ! {ack,P.1,{ok,P.1}} in proc_lib.erl line 375 12: Message ({ack,P.1,{ok,P.1}}) from P.1 reaches P 13: P: receives message ({ack,P.1,{ok,P.1}}) in proc_lib.erl line 344 14: P: true = erlang:demonitor(#Ref<0.0.0.177>, [flush]) in proc_lib.erl line 346 15: P: P.1 = erlang:whereis(z) in gen.erl line 157 16: P: #Ref<0.0.0.223> = erlang:monitor(process, P.1) in gen.erl line 200 17: P: {'$gen_call',{P,#Ref<0.0.0.223>},read} = erlang:send(P.1, {'$gen_call',{P,#Ref<0.0.0.223>},read}, [noconnect]) in gen.erl line 211 18: Message ({'$gen_call',{P,#Ref<0.0.0.223>},read}) from P reaches P.1 19: P.1: receives message ({'$gen_call',{P,#Ref<0.0.0.223>},read}) in gen_server.erl line 360 20: P.1: {#Ref<0.0.0.223>,initializing} = P ! {#Ref<0.0.0.223>,initializing} in gen_server.erl line 219 21: Message ({#Ref<0.0.0.223>,initializing}) from P.1 reaches P 22: P: receives message ({#Ref<0.0.0.223>,initializing}) in gen.erl line 213 23: P: true = erlang:demonitor(#Ref<0.0.0.223>, [flush]) in gen.erl line 215 24: P: exits abnormally ({{badmatch,initializing},[{z,test,0,[{file,[122,46,101,114,108]},{line,22}]}]}) ################################################################################ Warnings: -------------------------------------------------------------------------------- Continuing after error ################################################################################ Info: -------------------------------------------------------------------------------- Instrumented z Instrumented io_lib Instrumented gen_server Instrumented gen Instrumented proc_lib Instrumented erlang Instrumented init Instrumented sys You can see pairs of racing instructions (in the report and --graph) with '--show_races true' ################################################################################ Done! (Exit status: completed) Summary: 1 errors, 4/4 interleavings explored %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% As you can see P.1 yields before getting to the receive, then P calls read, and so the "after 0" doesn't run because the call message is in the mailbox already. Enjoy! On 07/27/2015 08:29 PM, Jesper Louis Andersen wrote: > > On Mon, Jul 27, 2015 at 4:40 PM, Lo?c Hoguin > wrote: > > This and sending yourself a message is a bad idea. It will usually > work, until it doesn't, and you will have a very hard time figuring > out why. > > > I'm quite curious. What is the scenario where this fails? I have never > observed this in practice on very busy systems, and I think going > through proc_lib for this is a detour I'd rather not we'd have to take. > > Here are two modules, z0 and z.erl. They are very boring gen servers: > > -module(z0). > -behaviour(gen_server). > > -ifdef(PULSE). > -include_lib("pulse_otp/include/pulse_otp.hrl"). > -endif. > > -export([start_link/0]). > > %% Operational API > -export([read/0, read_p/1]). > > %% gen_server API > -export([ > init/1, > handle_cast/2, > handle_call/3, > terminate/2, > code_change/3, > handle_info/2 > ]). > > %% API > start_link() -> > gen_server:start_link({local, ?MODULE}, ?MODULE, [], []). > > read() -> > gen_server:call(?MODULE, read). > > read_p(P) -> > gen_server:call(P, read). > > %% Callbacks > init([]) -> > {ok, initializing, 0}. > > handle_call(read, _From, State) -> > {reply, State, State}. > handle_cast(_M, State) -> > {noreply, State}. > > handle_info(timeout, _State) -> > {noreply, ready}. > terminate(_How, _State) -> > ok. > code_change(_OldVsn, State, _Extra) -> > {ok, State}. > > --------------------- > > -module(z). > -behaviour(gen_server). > > -ifdef(PULSE). > -include_lib("pulse_otp/include/pulse_otp.hrl"). > -endif. > > -export([start_link/0]). > > %% Operational API > -export([read/0, read_p/1]). > > %% gen_server API > -export([ > init/1, > handle_cast/2, > handle_call/3, > terminate/2, > code_change/3, > handle_info/2 > ]). > > %% API > start_link() -> > gen_server:start_link({local, ?MODULE}, ?MODULE, [], []). > > read() -> > gen_server:call(?MODULE, read). > > read_p(P) -> > gen_server:call(P, read). > > %% Callbacks > init([]) -> > self() ! timeout, > {ok, initializing}. > > handle_call(read, _From, State) -> > {reply, State, State}. > handle_cast(_M, State) -> > {noreply, State}. > > handle_info(timeout, _State) -> > {noreply, ready}. > terminate(_How, _State) -> > ok. > code_change(_OldVsn, State, _Extra) -> > {ok, State}. > > --------------------- > > Ok, now with these down, we can use EQC and PULSE to generate the > counterexample if any should be in there: > > -module(z_eqc). > -include_lib("eqc/include/eqc.hrl"). > -include_lib("eqc/include/eqc_statem.hrl"). > > -include_lib("pulse/include/pulse.hrl"). > -include_lib("pulse_otp/include/pulse_otp.hrl"). > > -compile(export_all). > > -record(state, { init, ref }). > > initial_state() -> > #state { init = false }. > server_start() -> > {ok, Pid} = z:start_link(), > unlink(Pid), > Pid. > %% SERVER START_LINK > server_start_pre(S) -> not initialized(S). > server_start_args(_S) -> []. > > server_start_next(S, Ref, []) -> > S#state { ref = Ref, init = true }. > server_start_post(_S, [], Ref) -> is_pid(Ref). > > %% READ BY PID > read_p(P) -> > z:read_p(P). > read_p_pre(S) -> initialized(S). > read_p_args(#state { ref = Ref }) -> [Ref]. > > read_p_return(_S, [_Ref]) -> ready. > > %% READ > read() -> > try z:read() of > M -> M > catch > _Class:_Err -> > {error, undefined} > end. > read_args(_S) -> []. > read_return(#state { init = false}, []) -> {error, undefined}; > read_return(#state { init = true}, []) -> ready. > > %% Run a test under PULSE to randomize the process schedule as well. > prop_model_pulse() -> > ?SETUP(fun() -> > setup(), > fun() -> ok end > end, > ?LET(Shrinking, parameter(shrinking, false), > ?FORALL(Cmds, parallel_commands(?MODULE), > ?ALWAYS(if not Shrinking -> 1; Shrinking -> 20 end, > ?PULSE(HSR={_, _, R}, > begin > ok = cleanup(), > run_parallel_commands(?MODULE, Cmds) > end, > aggregate(command_names(Cmds), > pretty_commands(?MODULE, Cmds, HSR, R == ok))))))). > > setup() -> > error_logger:tty(false), > ok. > > cleanup() -> > case whereis(z) of > undefined -> ok; > Pid -> > exit(Pid, kill), > timer:sleep(3) > end, > ok. > > initialized(#state { init = I }) -> I. > > pulse_instrument() -> > [ pulse_instrument(File) || File <- filelib:wildcard("*.erl") ], > ok. > pulse_instrument(File) -> > io:format("Compiling: ~p~n", [File]), > {ok, Mod} = compile:file(File, [{d, 'PULSE', true}, {d, > 'WITH_PULSE', true}, > {d, 'EQC_TESTING', true}, > {parse_transform, pulse_instrument}]), > code:purge(Mod), > code:load_file(Mod), > Mod. > > ----------------------------- > > However, when I run this, I get no errors, even though I'm trying to > behave non-nicely: > > I won't call the Pid until I know about it. > I will blindly call the name, z, and if I get an error on this, I'll > verify that the possible linearization is that the process was not > started yet. If I get a result, I force it to be ready: > > 60> eqc:module({testing_time, 30}, z_eqc). > prop_model_pulse: > ....................................................................................................(x10)....................................................................................................(x100)............................(x10)....... > Time limit reached: 30.0 seconds. > > OK, passed 3970 tests > > 49.857% {z_eqc,read,0} > 44.492% {z_eqc,read_p,1} > 5.651% {z_eqc,server_start,0} > [] > > ... > > So: What is the scenario where this approach fails? > > > > -- > J. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From candres.bolanos@REDACTED Mon Jul 27 19:54:11 2015 From: candres.bolanos@REDACTED (=?UTF-8?Q?CARLOS_ANDRES_BOLA=C3=91OS_REALPE?=) Date: Mon, 27 Jul 2015 12:54:11 -0500 Subject: [erlang-questions] riak client best practices Message-ID: Hi. - use riakc protobuf client or join services and Riak ring in a common erlang cluster with the same cookie? R// Definitely use riak protobufs, is the mainstream, maintained and optimized by Basho - open a new connection from riakc to Riak node on each hit or keep a connection pool? I suppose it should be the pool as sockets are finite. R// Definitely you should use a connection pool - on every request connect to a random Riak node from the ring? Or use dedicated Riak nodes for each service instance, if there are many? R// I'd say: it depends on what you really need AND the option that fits better with your systemic properties. One common case could be use a load balancer, take a look at this link: " http://docs.basho.com/riak/latest/ops/advanced/configs/load-balancing-proxy". On other hand, depending of your context, is not bad to have a symetric topology as you mentioned, having a Riak node per Service/Server node, as long as you have a same symetric traffic over your resources, because in top of the server nodes I suppose that you will have a load balancer, so the traffic will be distributed symetrically over the Riak nodes too. And the other option is quite similar, picking a random Riak node from the client, using some balancing strategy. - what to do with the new nodes which are joining the ring and the ones which are removed from the ring? What is the more or less standard way to notify clients about this? R// The idea is don't do anything -- as far as possible. Riak handles this, it is an internal Riak process. Also it could be an admin task, but not to deal with it from your client (in this case is what you call service instance). You can read more about it: http://docs.basho.com/riak/latest/ops/running/handoff/ - what to do in a client if a Riak node temporarily goes offline? Keep trying to connect to such node until it gets online again? This would obviously hit latency unless there's a special dedicated process for this. R// This is one of the drawbacks when you have one-to-one connections (Riak node per Service node). And you would have to handle re-connection attempts as you mention, but the idea is the timeout between attempts be longer each time (e.g.: using a linear or exponential factor), and in this way avoid to hit the DB to much. Again, probably the recomendation here is to use the load balancer approach (mentioned previously), because your service instances are pointing to the LB not directly to the DB, so if some Riak node goes down, the upcoming requests will be distributed on the rest of the available Riak nodes. Finally, I strongly recommend to use 'sumo_db' ( https://github.com/inaka/sumo_db), which will make easier interact with Riak, and also avoid to deal with some things like connection pools, configuration, complex search, etc. C. Andres Bola?os -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Jul 28 10:05:08 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 28 Jul 2015 10:05:08 +0200 Subject: [erlang-questions] gen_server and init In-Reply-To: <55B6831D.60304@ninenines.eu> References: <55B642DE.6040706@ninenines.eu> <55B6831D.60304@ninenines.eu> Message-ID: On Mon, Jul 27, 2015 at 9:14 PM, Lo?c Hoguin wrote: > I'm not sure what you wrote there but I can give you two scenarios where > it can fail off the top of my head. The first is very unlikely and can only > fail if you use the 0 timeout, while the second is actually much easier to > observe and can fail with both methods: > > P1 calls start_link > P2 init (returns 0 timeout) > P2 yields before calling receive > P1 returns from start_link and sends P2 a message > P2 receives message > > Yes, this one is an obvious linearization problem. And indeed, using concuerror for this is the right tool for the job. But if you send yourself a message, concuerror reports, as expected that this is safe behaviour: Done! (Exit status: completed) Summary: 0 errors, 3/3 interleavings explored and if that was not the case, there is something about the semantics of Erlang I need to learn :) > And: > > P1 calls start_link > P2 in init subscribes to some kind of pubsub PS > PS sends P2 message(s) > P2 returns from init and receive those messages > The fact that you give other processes access to the Pid the guarantee is lost. In that case, you can capture messages arriving before initialization and do something sensible with them, or you can do the enter_loop dance you suggested. I don't think it is a bad solution, but I would prefer to be able to do something simpler in common cases, and indeed, sending yourself a message is one of those, if we are to believe concuerror. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue Jul 28 11:23:35 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 28 Jul 2015 11:23:35 +0200 Subject: [erlang-questions] gen_server and init In-Reply-To: References: <55B642DE.6040706@ninenines.eu> <55B6831D.60304@ninenines.eu> Message-ID: <55B74A17.10902@ninenines.eu> On 07/28/2015 10:05 AM, Jesper Louis Andersen wrote: > > On Mon, Jul 27, 2015 at 9:14 PM, Lo?c Hoguin > wrote: > > I'm not sure what you wrote there but I can give you two scenarios > where it can fail off the top of my head. The first is very unlikely > and can only fail if you use the 0 timeout, while the second is > actually much easier to observe and can fail with both methods: > > P1 calls start_link > P2 init (returns 0 timeout) > P2 yields before calling receive > P1 returns from start_link and sends P2 a message > P2 receives message > > > Yes, this one is an obvious linearization problem. And indeed, using > concuerror for this is the right tool for the job. But if you send > yourself a message, concuerror reports, as expected that this is safe > behaviour: > > Done! (Exit status: completed) > Summary: 0 errors, 3/3 interleavings explored > > and if that was not the case, there is something about the semantics of > Erlang I need to learn :) That is however assuming there are only two processes in your system, and everything is well written and pids don't wrap up (which they apparently do?) Not to forget human error when an operator uses the shell and the pid/3 function to test things out. If you send yourself a message, or use the 0 timeout, you will also introduce another issue in your gen_server: you can trigger the clause again by simply sending the wrong message to the process. You could use the state to make sure you are indeed initializing, but that makes things a lot more complex. The chances of a problem occuring are very low, but the chance of figuring out what happened when something does go wrong are much lower. > And: > > P1 calls start_link > P2 in init subscribes to some kind of pubsub PS > PS sends P2 message(s) > P2 returns from init and receive those messages > > > The fact that you give other processes access to the Pid the guarantee > is lost. In that case, you can capture messages arriving before > initialization and do something sensible with them, or you can do the > enter_loop dance you suggested. I don't think it is a bad solution, but > I would prefer to be able to do something simpler in common cases, and > indeed, sending yourself a message is one of those, if we are to believe > concuerror. I'm not sure where the complexity lies with enter_loop. Instead of gen_server:start_link use proc_lib:spawn_link. Instead of returning from init, call gen_server:enter_loop. Done. Then it depends if you need to have a synchronous start, in which case you will want to use proc_lib:start_link and proc_lib:init_ack. But in most cases this is not needed, especially since you now won't rely on hacks that require the start to be synchronous. :-) Perhaps it is more obscure because people have relied on hacks and documented those hacks a lot more than enter_loop. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From jesper.louis.andersen@REDACTED Tue Jul 28 11:41:53 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 28 Jul 2015 11:41:53 +0200 Subject: [erlang-questions] gen_server and init In-Reply-To: <55B74A17.10902@ninenines.eu> References: <55B642DE.6040706@ninenines.eu> <55B6831D.60304@ninenines.eu> <55B74A17.10902@ninenines.eu> Message-ID: On Tue, Jul 28, 2015 at 11:23 AM, Lo?c Hoguin wrote: > That is however assuming there are only two processes in your system, and > everything is well written and pids don't wrap up (which they apparently > do?) I don't think adding more processes can tickle a bug which is not found by two processes here. If I have N processes racing for this, only one has to go before. Once I know which, I can ignore all the other processes. So the problem can be reduced to a 2-process model I think. Pids internally have a 28 bit structure so wrap-around can occur if you manage to generate 268 million processes, and come around again. It depends a lot on your internal architecture if this will happen or not in a practical setting. The scenario is that an old process might have been holding on to an old Pid and tries to reuse it and now a new process exist in its name. 268 million sounds way too low on a 64 bit architecture however. There you have 60 bits rather than 28, so you would not hit any limit at all were these used. There is some effort in switching over however. If you've hit these limits, I think it is worthwhile to mention that you did. As systems grow ever more complicated, earlier ceilings will eventually be hit and knowing in advance it can happen is nice. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Tue Jul 28 12:15:07 2015 From: chandrashekhar.mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Tue, 28 Jul 2015 11:15:07 +0100 Subject: [erlang-questions] ets access from NIF thread? In-Reply-To: References: Message-ID: <093CA681-A29C-4800-A9D9-D50264620558@gmail.com> Dear OTP team, This seems to be a good idea. This would probably only work for public ETS tables, but would still be useful for some applications. Chandru > On 5 Jan 2012, at 22:01, Daniel Goertzen wrote: > > I was wondering if NIF threads will be able to access ets tables at some point in the future. Is this feasible in the current VM? > > A work around is of course to make my own shared table that provides both a NIF and C API, but leveraging ets seems convenient. > > > Dan. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Tue Jul 28 12:30:11 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 28 Jul 2015 12:30:11 +0200 Subject: [erlang-questions] Optional callbacks Message-ID: <55B759B3.6000808@ninenines.eu> Started trying to play with those, and not managing to get a warning from Dialyzer. When I change the body of the function to 'ok', I do get a warning for a normal callback, and I get nothing for an optional callback. Is that normal? The behaviour_info function does return the callback as optional. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From be.dmitry@REDACTED Tue Jul 28 13:03:49 2015 From: be.dmitry@REDACTED (Dmitry Belyaev) Date: Tue, 28 Jul 2015 21:03:49 +1000 Subject: [erlang-questions] gen_server and init In-Reply-To: References: <55B642DE.6040706@ninenines.eu> <55B6831D.60304@ninenines.eu> <55B74A17.10902@ninenines.eu> Message-ID: <9B4937EE-6AC4-4804-878C-A599B3F3CEEA@gmail.com> Using timeout is definitely a bad idea. Sending init message is a much better choice until the process is not registered under well known name. In this case the process might receive a message before the init message in the init function is put into the mailbox. -- Best wishes, Dmitry Belyaev On 28 July 2015 7:41:53 PM AEST, Jesper Louis Andersen wrote: >On Tue, Jul 28, 2015 at 11:23 AM, Lo?c Hoguin >wrote: > >> That is however assuming there are only two processes in your system, >and >> everything is well written and pids don't wrap up (which they >apparently >> do?) > > >I don't think adding more processes can tickle a bug which is not found >by >two processes here. If I have N processes racing for this, only one has >to >go before. Once I know which, I can ignore all the other processes. So >the >problem can be reduced to a 2-process model I think. > >Pids internally have a 28 bit structure so wrap-around can occur if you >manage to generate 268 million processes, and come around again. It >depends >a lot on your internal architecture if this will happen or not in a >practical setting. > >The scenario is that an old process might have been holding on to an >old >Pid and tries to reuse it and now a new process exist in its name. > >268 million sounds way too low on a 64 bit architecture however. There >you >have 60 bits rather than 28, so you would not hit any limit at all were >these used. There is some effort in switching over however. If you've >hit >these limits, I think it is worthwhile to mention that you did. As >systems >grow ever more complicated, earlier ceilings will eventually be hit and >knowing in advance it can happen is nice. > > >-- >J. > > >------------------------------------------------------------------------ > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From zkessin@REDACTED Tue Jul 28 13:19:04 2015 From: zkessin@REDACTED (Zachary Kessin) Date: Tue, 28 Jul 2015 12:19:04 +0100 Subject: [erlang-questions] Building a rpm from a relx release Message-ID: <55B76528.8010300@gmail.com> I am working on building a RPM from a relx release, and one question I have is what is the best way to setup init scripts. What I want is of course for linux to start the erlang package on boot. Simply linking the normal start file into /etc/init.d does not work as you get directory errors. So what have people had luck with -- Zachary Kessin Mostly Erlang Podcast Skype: zachkessin Twitter: @zkessin -------------- next part -------------- An HTML attachment was scrubbed... URL: From karol.urbanski@REDACTED Tue Jul 28 13:50:31 2015 From: karol.urbanski@REDACTED (Karol Urbanski) Date: Tue, 28 Jul 2015 13:50:31 +0200 Subject: [erlang-questions] Building a rpm from a relx release In-Reply-To: <55B76528.8010300@gmail.com> References: <55B76528.8010300@gmail.com> Message-ID: <20150728115030.GA2055@dex.erlang-solutions.com> The best way is to take a skeleton initscript (or edit an existing one) and fill out the start/stop functions and the necessary case statements; the initscript will act as a wrapper that calls the start file in a way that will avoid directory errors. After that, if you have a properly created initscript, all you need to do is just chkconfig to autostart your package. Don't package the /etc/rc.d/rcN.d symlink to the initscript; instead create a postinstall script that will use chkconfig to add it to autostart on package install. This is simply good practice so your rpm file listing doesn't desynch if someone removes the package from autostart. Best regards, Karol Urbanski On Tue, Jul 28, 2015 at 12:19:04PM +0100, Zachary Kessin wrote: > I am working on building a RPM from a relx release, and one question I have is what is the best way to setup > init scripts. What I want is of course for linux to start the erlang package on boot. Simply linking the normal > start file into /etc/init.d does not work as you get directory errors. > So what have people had luck with > > -- > Zachary Kessin > Mostly Erlang Podcast > Skype: zachkessin > Twitter: @zkessin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From seancribbs@REDACTED Tue Jul 28 15:53:07 2015 From: seancribbs@REDACTED (Sean Cribbs) Date: Tue, 28 Jul 2015 08:53:07 -0500 Subject: [erlang-questions] Building a rpm from a relx release In-Reply-To: <20150728115030.GA2055@dex.erlang-solutions.com> References: <55B76528.8010300@gmail.com> <20150728115030.GA2055@dex.erlang-solutions.com> Message-ID: Zack, https://github.com/basho/node_package has templates you can use, even if you don't use the entire thing (since it relies on reltool, not relx). On Tue, Jul 28, 2015 at 6:50 AM, Karol Urbanski < karol.urbanski@REDACTED> wrote: > The best way is to take a skeleton initscript (or edit an > existing one) and fill out the start/stop functions and the necessary > case statements; the initscript will act as a wrapper that calls > the start file in a way that will avoid directory errors. > > After that, if you have a properly created initscript, all you need to > do is just chkconfig to autostart your package. > > Don't package the /etc/rc.d/rcN.d symlink to the initscript; instead > create a postinstall script that will use chkconfig to add it to > autostart on package install. This is simply good practice so your rpm > file listing doesn't desynch if someone removes the package from > autostart. > > Best regards, > Karol Urbanski > On Tue, Jul 28, 2015 at 12:19:04PM +0100, Zachary Kessin wrote: > > I am working on building a RPM from a relx release, and one question > I have is what is the best way to setup > > init scripts. What I want is of course for linux to start the erlang > package on boot. Simply linking the normal > > start file into /etc/init.d does not work as you get directory errors. > > So what have people had luck with > > > > -- > > Zachary Kessin > > Mostly Erlang Podcast > > Skype: zachkessin > > Twitter: @zkessin > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andre.graf@REDACTED Tue Jul 28 16:01:02 2015 From: andre.graf@REDACTED (Andre Graf) Date: Tue, 28 Jul 2015 16:01:02 +0200 Subject: [erlang-questions] Building a rpm from a relx release In-Reply-To: References: <55B76528.8010300@gmail.com> <20150728115030.GA2055@dex.erlang-solutions.com> Message-ID: <55B78B1E.1050100@erl.io> We're currently using a patched version of node_package that uses rebar3 (and therefore relx) for generating the VerneMQ deb and rpm packages. you can find it in https://github.com/dergraf/node_package/tree/rebar3-support Cheers, Andre On 07/28/2015 03:53 PM, Sean Cribbs wrote: > Zack, > > https://github.com/basho/node_package has templates you can use, even if > you don't use the entire thing (since it relies on reltool, not relx). > > On Tue, Jul 28, 2015 at 6:50 AM, Karol Urbanski > > wrote: > > The best way is to take a skeleton initscript (or edit an > existing one) and fill out the start/stop functions and the necessary > case statements; the initscript will act as a wrapper that calls > the start file in a way that will avoid directory errors. > > After that, if you have a properly created initscript, all you need to > do is just chkconfig to autostart your package. > > Don't package the /etc/rc.d/rcN.d symlink to the initscript; instead > create a postinstall script that will use chkconfig to add it to > autostart on package install. This is simply good practice so your rpm > file listing doesn't desynch if someone removes the package from > autostart. > > Best regards, > Karol Urbanski > On Tue, Jul 28, 2015 at 12:19:04PM +0100, Zachary Kessin wrote: > > I am working on building a RPM from a relx release, and one question I have is what is the best way to setup > > init scripts. What I want is of course for linux to start the erlang package on boot. Simply linking the normal > > start file into /etc/init.d does not work as you get directory errors. > > So what have people had luck with > > > > -- > > Zachary Kessin > > Mostly Erlang Podcast > > Skype: zachkessin > > Twitter: @zkessin > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mrtndimitrov@REDACTED Tue Jul 28 17:03:22 2015 From: mrtndimitrov@REDACTED (Martin Koroudjiev) Date: Tue, 28 Jul 2015 18:03:22 +0300 Subject: [erlang-questions] Is supervisor notified when child is restarted? Message-ID: <55B799BA.5040805@gmail.com> Hello, I am wondering how can I get notified when a child process is restarted? Imagine this simple scenario - supervisor with one_for_one strategy and 3 gen_servers as workers. Each in its state has the PIDs of the other 2 and sends them messages. Now if one child gets restarted, the other 2 will have the old PID. How can I notify them? I can't restart the whole supervision tree because the number of child processes is buried deep in the initialization logic of the application. Regards, Martin From seancribbs@REDACTED Tue Jul 28 17:23:35 2015 From: seancribbs@REDACTED (Sean Cribbs) Date: Tue, 28 Jul 2015 10:23:35 -0500 Subject: [erlang-questions] Is supervisor notified when child is restarted? In-Reply-To: <55B799BA.5040805@gmail.com> References: <55B799BA.5040805@gmail.com> Message-ID: Hi Martin, It sounds like you need either to use either registered names, process groups, or the one_for_all strategy. Registered processes are the easiest way, you can simply refer to them by the atom -- but you must expect that sometimes messages will be lost if the process is restarting. Alternately, if you change this supervisor's strategy to one_for_all you will ensure that every process always has the correct state (assuming it already starts up in the correct state), but existing work from non-failed processes can be lost. For process groups, you can use pg2 [1] which comes with OTP, or there are several alternate implementations available on github, but messaging the other processes in this small group will incur additional overhead. Each solution has tradeoffs, I hope that helps you decide which one to use. On Tue, Jul 28, 2015 at 10:03 AM, Martin Koroudjiev wrote: > Hello, > > I am wondering how can I get notified when a child process is restarted? > Imagine this simple scenario - supervisor with one_for_one strategy and > 3 gen_servers as workers. Each in its state has the PIDs of the other 2 > and sends them messages. Now if one child gets restarted, the other 2 > will have the old PID. How can I notify them? > > I can't restart the whole supervision tree because the number of child > processes is buried deep in the initialization logic of the application. > > Regards, > Martin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From seancribbs@REDACTED Tue Jul 28 17:24:06 2015 From: seancribbs@REDACTED (Sean Cribbs) Date: Tue, 28 Jul 2015 10:24:06 -0500 Subject: [erlang-questions] Is supervisor notified when child is restarted? In-Reply-To: References: <55B799BA.5040805@gmail.com> Message-ID: Woops, forgot to include the link: [1] http://erlang.org/doc/man/pg2.html On Tue, Jul 28, 2015 at 10:23 AM, Sean Cribbs wrote: > Hi Martin, > > It sounds like you need either to use either registered names, process > groups, or the one_for_all strategy. Registered processes are the easiest > way, you can simply refer to them by the atom -- but you must expect that > sometimes messages will be lost if the process is restarting. Alternately, > if you change this supervisor's strategy to one_for_all you will ensure > that every process always has the correct state (assuming it already starts > up in the correct state), but existing work from non-failed processes can > be lost. For process groups, you can use pg2 [1] which comes with OTP, or > there are several alternate implementations available on github, but > messaging the other processes in this small group will incur additional > overhead. > > Each solution has tradeoffs, I hope that helps you decide which one to use. > > On Tue, Jul 28, 2015 at 10:03 AM, Martin Koroudjiev < > mrtndimitrov@REDACTED> wrote: > >> Hello, >> >> I am wondering how can I get notified when a child process is restarted? >> Imagine this simple scenario - supervisor with one_for_one strategy and >> 3 gen_servers as workers. Each in its state has the PIDs of the other 2 >> and sends them messages. Now if one child gets restarted, the other 2 >> will have the old PID. How can I notify them? >> >> I can't restart the whole supervision tree because the number of child >> processes is buried deep in the initialization logic of the application. >> >> Regards, >> Martin >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Tue Jul 28 18:44:18 2015 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 28 Jul 2015 09:44:18 -0700 Subject: [erlang-questions] gen_server and init In-Reply-To: <9B4937EE-6AC4-4804-878C-A599B3F3CEEA@gmail.com> References: <55B642DE.6040706@ninenines.eu> <55B6831D.60304@ninenines.eu> <55B74A17.10902@ninenines.eu> <9B4937EE-6AC4-4804-878C-A599B3F3CEEA@gmail.com> Message-ID: <55B7B162.4070701@gmail.com> On 07/28/2015 04:03 AM, Dmitry Belyaev wrote: > Using timeout is definitely a bad idea. Sending init message is a much better choice until the process is not registered under well known name. In this case the process might receive a message before the init message in the init function is put into the mailbox. If the first thing you do in the init/1 function is send a message to self(), and you are not registering the process with global/local/via using the start_link function of an Erlang/OTP behaviour, then the message you send should be the first message in the process' queue to be processed once the init/1 function returns successfully and the process gets to its loop. The gen module shows that name registration happens before the init/1 function is called, so that can create the race condition that was previously described, though there is nothing preventing you from doing the same registration within the body of the init/1 function. > -- > Best wishes, > Dmitry Belyaev > > On 28 July 2015 7:41:53 PM AEST, Jesper Louis Andersen wrote: > > > On Tue, Jul 28, 2015 at 11:23 AM, Lo?c Hoguin > wrote: > > That is however assuming there are only two processes in your system, and everything is well written and pids don't wrap up (which they apparently do?) > > > I don't think adding more processes can tickle a bug which is not found by two processes here. If I have N processes racing for this, only one has to go before. Once I know which, I can ignore all the other processes. So the problem can be reduced to a 2-process model I think. > > Pids internally have a 28 bit structure so wrap-around can occur if you manage to generate 268 million processes, and come ar ound again. It depends a lot on your internal architecture if this will happen or not in a practical setting. > > The scenario is that an old process might have been holding on to an old Pid and tries to reuse it and now a new process exist in its name. > > 268 million sounds way too low on a 64 bit architecture however. There you have 60 bits rather than 28, so you would not hit any limit at all were these used. There is some effort in switching over however. If you've hit these limits, I think it is worthwhile to mention that you did. As systems grow ever more complicated, earlier ceilings will eventually be hit and knowing in advance it can happen is nice. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue Jul 28 19:32:37 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 28 Jul 2015 20:32:37 +0300 Subject: [erlang-questions] Building a rpm from a relx release In-Reply-To: <55B78B1E.1050100@erl.io> References: <55B76528.8010300@gmail.com> <20150728115030.GA2055@dex.erlang-solutions.com> <55B78B1E.1050100@erl.io> Message-ID: JFYI: you can use erlang rpm builder https://github.com/flussonic/epm to make a deb or rpm. It does not support all features of rpm (librpm is the most horrible code I ever seen), but it works for us. It doesn't require any C rpm library and it may be used to make package from MacOS X. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ali.sabil@REDACTED Tue Jul 28 20:33:29 2015 From: ali.sabil@REDACTED (Ali Sabil) Date: Tue, 28 Jul 2015 18:33:29 +0000 Subject: [erlang-questions] gen_server and init In-Reply-To: <55B7B162.4070701@gmail.com> References: <55B642DE.6040706@ninenines.eu> <55B6831D.60304@ninenines.eu> <55B74A17.10902@ninenines.eu> <9B4937EE-6AC4-4804-878C-A599B3F3CEEA@gmail.com> <55B7B162.4070701@gmail.com> Message-ID: I would recommend taking a look at gen_server:enter_loop as an alternative to sending a message to the process itself. It's a much cleaner alternative imho. On Tue 28 Jul 2015 at 18:44 Michael Truog wrote: > On 07/28/2015 04:03 AM, Dmitry Belyaev wrote: > > Using timeout is definitely a bad idea. Sending init message is a much > better choice until the process is not registered under well known name. In > this case the process might receive a message before the init message in > the init function is put into the mailbox. > > If the first thing you do in the init/1 function is send a message to > self(), and you are not registering the process with global/local/via using > the start_link function of an Erlang/OTP behaviour, then the message you > send should be the first message in the process' queue to be processed once > the init/1 function returns successfully and the process gets to its loop. > The gen module shows that name registration happens before the init/1 > function is called, so that can create the race condition that was > previously described, though there is nothing preventing you from doing the > same registration within the body of the init/1 function. > > > -- > Best wishes, > Dmitry Belyaev > > On 28 July 2015 7:41:53 PM AEST, Jesper Louis Andersen > > wrote: >> >> >> On Tue, Jul 28, 2015 at 11:23 AM, Lo?c Hoguin wrote: >> >>> That is however assuming there are only two processes in your system, >>> and everything is well written and pids don't wrap up (which they >>> apparently do?) >> >> >> I don't think adding more processes can tickle a bug which is not found >> by two processes here. If I have N processes racing for this, only one has >> to go before. Once I know which, I can ignore all the other processes. So >> the problem can be reduced to a 2-process model I think. >> >> Pids internally have a 28 bit structure so wrap-around can occur if you >> manage to generate 268 million processes, and come ar ound again. It >> depends a lot on your internal architecture if this will happen or not in a >> practical setting. >> >> The scenario is that an old process might have been holding on to an old >> Pid and tries to reuse it and now a new process exist in its name. >> >> 268 million sounds way too low on a 64 bit architecture however. There >> you have 60 bits rather than 28, so you would not hit any limit at all were >> these used. There is some effort in switching over however. If you've hit >> these limits, I think it is worthwhile to mention that you did. As systems >> grow ever more complicated, earlier ceilings will eventually be hit and >> knowing in advance it can happen is nice. >> >> > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Tue Jul 28 21:24:48 2015 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 28 Jul 2015 12:24:48 -0700 Subject: [erlang-questions] gen_server and init In-Reply-To: References: <55B642DE.6040706@ninenines.eu> <55B6831D.60304@ninenines.eu> <55B74A17.10902@ninenines.eu> <9B4937EE-6AC4-4804-878C-A599B3F3CEEA@gmail.com> <55B7B162.4070701@gmail.com> Message-ID: <55B7D700.9040906@gmail.com> On 07/28/2015 11:33 AM, Ali Sabil wrote: > I would recommend taking a look at gen_server:enter_loop as an alternative to sending a message to the process itself. It's a much cleaner alternative imho. The gen_server:enter_loop function does not return and the init/1 function must return, to show initialization succeeded. > On Tue 28 Jul 2015 at 18:44 Michael Truog > wrote: > > On 07/28/2015 04:03 AM, Dmitry Belyaev wrote: >> Using timeout is definitely a bad idea. Sending init message is a much better choice until the process is not registered under well known name. In this case the process might receive a message before the init message in the init function is put into the mailbox. > If the first thing you do in the init/1 function is send a message to self(), and you are not registering the process with global/local/via using the start_link function of an Erlang/OTP behaviour, then the message you send should be the first message in the process' queue to be processed once the init/1 function returns successfully and the process gets to its loop. The gen module shows that name registration happens before the init/1 function is called, so that can create the race condition that was previously described, though there is nothing preventing you from doing the same registration within the body of the init/1 function. > > >> -- >> Best wishes, >> Dmitry Belyaev >> >> On 28 July 2015 7:41:53 PM AEST, Jesper Louis Andersen wrote: >> >> >> On Tue, Jul 28, 2015 at 11:23 AM, Lo?c Hoguin > wrote: >> >> That is however assuming there are only two processes in your system, and everything is well written and pids don't wrap up (which they apparently do?) >> >> >> I don't think adding more processes can tickle a bug which is not found by two processes here. If I have N processes racing for this, only one has to go before. Once I know which, I can ignore all the other processes. So the problem can be reduced to a 2-process model I think. >> >> Pids internally have a 28 bit structure so wrap-around can occur if you manage to generate 268 million processes, and come ar ound again. It depends a lot on your internal architecture if this will happen or not in a practical setting. >> >> The scenario is that an old process might have been holding on to an old Pid and tries to reuse it and now a new process exist in its name. >> >> 268 million sounds way too low on a 64 bit architecture however. There you have 60 bits rather than 28, so you would not hit any limit at all were these used. There is some effort in switching over however. If you've hit these limits, I think it is worthwhile to mention that you did. As systems grow ever more complicated, earlier ceilings will eventually be hit and knowing in advance it can happen is nice. >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrtndimitrov@REDACTED Tue Jul 28 22:25:21 2015 From: mrtndimitrov@REDACTED (Martin Koroudjiev) Date: Tue, 28 Jul 2015 23:25:21 +0300 Subject: [erlang-questions] Is supervisor notified when child is restarted? In-Reply-To: References: <55B799BA.5040805@gmail.com> Message-ID: <55B7E531.8000406@gmail.com> Thank you for your time and information. Process groups sound interesting. I will dig deeper. Regards, Martin On 7/28/2015 6:24 PM, Sean Cribbs wrote: > Woops, forgot to include the link: > > [1] http://erlang.org/doc/man/pg2.html > > On Tue, Jul 28, 2015 at 10:23 AM, Sean Cribbs > wrote: > > Hi Martin, > > It sounds like you need either to use either registered names, > process groups, or the one_for_all strategy. Registered processes > are the easiest way, you can simply refer to them by the atom -- > but you must expect that sometimes messages will be lost if the > process is restarting. Alternately, if you change this > supervisor's strategy to one_for_all you will ensure that every > process always has the correct state (assuming it already starts > up in the correct state), but existing work from non-failed > processes can be lost. For process groups, you can use pg2 [1] > which comes with OTP, or there are several alternate > implementations available on github, but messaging the other > processes in this small group will incur additional overhead. > > Each solution has tradeoffs, I hope that helps you decide which > one to use. > > On Tue, Jul 28, 2015 at 10:03 AM, Martin Koroudjiev > > wrote: > > Hello, > > I am wondering how can I get notified when a child process is > restarted? > Imagine this simple scenario - supervisor with one_for_one > strategy and > 3 gen_servers as workers. Each in its state has the PIDs of > the other 2 > and sends them messages. Now if one child gets restarted, the > other 2 > will have the old PID. How can I notify them? > > I can't restart the whole supervision tree because the number > of child > processes is buried deep in the initialization logic of the > application. > > Regards, > Martin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marlus.saraiva@REDACTED Wed Jul 29 21:04:05 2015 From: marlus.saraiva@REDACTED (Marlus Saraiva) Date: Wed, 29 Jul 2015 16:04:05 -0300 Subject: [erlang-questions] [ANN] Erlang/Elixir packages for Alpine Linux (musl libc + busybox) Message-ID: Hi all, Erlang 18.0.2 and Elixir 1.0.5 packages are now available for Alpine Linux. More info about the packages, minimal docker images and examples can be found at: https://github.com/msaraiva/alpine-erlang (WIP) Some examples of minimal docker images: Erlang: 16.78 MB Elixir: 21.44 MB Phoenix Chat Example (full web application): 23.52 MB Since building Erlang against musl + busybox requires some patching, it would be really nice to get more people involved, either by just providing feedback or, even better, by creating new packages, providing patches, docker images or anything related. So if you have something to share, fell free to open an issue or send me an email. Cheers, -- Marlus Saraiva https://github.com/msaraiva https://twitter.com/MarlusSaraiva -------------- next part -------------- An HTML attachment was scrubbed... URL: From semmitmondo@REDACTED Wed Jul 29 22:47:54 2015 From: semmitmondo@REDACTED (semmit mondo) Date: Wed, 29 Jul 2015 22:47:54 +0200 (CEST) Subject: [erlang-questions] telephony Message-ID: Hi List, Is there any open source application out there that provides an Erlanginterface for programming a voip router? Or is it the asterisk ecosystem(for instance with Erlang using AGI) that comes the closest to theoriginal mission? S. -------------- next part -------------- An HTML attachment was scrubbed... URL: From james@REDACTED Thu Jul 30 00:39:25 2015 From: james@REDACTED (James Aimonetti) Date: Wed, 29 Jul 2015 15:39:25 -0700 Subject: [erlang-questions] telephony In-Reply-To: References: Message-ID: <55B9561D.2080104@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Le 07/29/2015 01:47 PM, semmit mondo a ?crit : > Hi List, Is there any open source application out there that > provides an Erlanginterface for programming a voip router? Or is > it the asterisk ecosystem(for instance with Erlang using AGI) that > comes the closest to theoriginal mission? S. > > > > _______________________________________________ erlang-questions > mailing list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > FreeSWITCH has two modules, mod_erlang_event and mod_kazoo, which present FreeSWITCH as a C-Node to other Erlang VMs. We built and use mod_kazoo for Kazoo, but it is generic in purpose; I know several folks using mod_erlang_event in production as well. Different use cases for each. - -- James Aimonetti Lead Systems Architect / Impressionable Scallywag "I thought I fixed that" 2600Hz | http://2600hz.com sip:james@REDACTED tel:415.886.7905 irc:mc_ @ freenode -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJVuVYdAAoJENTKa+JPXCVgpJ0IALM8efQeSj6eNDJgJh+ecXfl gR9Mo1otgMkjCFUdQBN9z2IDzKqszX2YfmUf7IlKp02YOKDhMNDPIYzBYgrh2/4K s4bYjDAGZ9GeX2lPtGe/1fcg4ZLPJS/e0ZT4Te+j3QR0s1i3vGXQFwKSM7RZ3UIz +fY5kwY+EgYYe9fPKGMnY0dAC2z2Dw8jAbVHtVnUV1YX5xfv9Gcn8UDW4ly8ixuw mfiIlenNjJBoeMeNhwT+J75Xlu/4WtAmwis36CwjXOA58WhzdS/8AyZWe+UIMzAT tc+QML8urOW5P72x9G4kZ4f0BOH1JDIl/yYweJgyRKVY/37bxtTQs3RjxoBEvHw= =NAOI -----END PGP SIGNATURE----- From Tomas.Kukosa@REDACTED Thu Jul 30 09:34:49 2015 From: Tomas.Kukosa@REDACTED (=?Windows-1252?Q?Kukosa_Tom=E1=9A?=) Date: Thu, 30 Jul 2015 07:34:49 +0000 Subject: [erlang-questions] telephony In-Reply-To: <55B9561D.2080104@2600hz.com> References: <55B9561D.2080104@2600hz.com> Message-ID: <55B9D398.3030902@ixperta.com> On 30.7.2015 0:39, James Aimonetti wrote: >> Hi List, Is there any open source application out there that >> provides an Erlanginterface for programming a voip router? Or is >> it the asterisk ecosystem(for instance with Erlang using AGI) that >> comes the closest to theoriginal mission? S. > FreeSWITCH has two modules, mod_erlang_event and mod_kazoo, which > present FreeSWITCH as a C-Node to other Erlang VMs. We built and use > mod_kazoo for Kazoo, but it is generic in purpose; I know several > folks using mod_erlang_event in production as well. Different use > cases for each. Is there any simple description of difference between mod_kazoo and mod_erlang_event together with hints which use cases they are good for? From hanssv@REDACTED Thu Jul 30 10:40:29 2015 From: hanssv@REDACTED (Hans Svensson) Date: Thu, 30 Jul 2015 10:40:29 +0200 Subject: [erlang-questions] Erlang Workshop: Call for participation Message-ID: <55B9E2FD.5050101@gmail.com> [ NOTE: Early registration ends 3 Aug ] ==================================================================== CALL FOR PARTICIPATION ACM SIGPLAN Erlang Workshop 2015 Vancouver, Canada 4 September, 2015 http://www.erlang.org/workshop/2015/ ==================================================================== The purpose of the Erlang Workshop is to bring together the open source, academic, and industrial programming communities of Erlang. It provides a forum that enables participants to get the latest updates on recent developments, new techniques and tools tailored to Erlang, novel applications, lessons from users' experiences and current research problems relevant to the practice of Erlang and functional programming. These are the accepted papers and programme for this year (they should appear on the website shortly): 09:00 - 09:10 Opening & Welcome 09:10 - 10:00 Invited Keynote: ?Micro services in Erlang? by Eric Merritt 10:00 - 10:30 Coffee Break 10:30 - 11:30 SCALABILITY and DISTRIBUTION SESSION - The Implementation and Use of a Generic Dataflow Behaviour in Erlang by Christopher Meiklejohn and Peter Van Roy - Performance Portability through Semi-explicit Placement in Distributed Erlang by Kenneth MacKenzie, Natalia Chechina and Phil Trinder 11:30 - 11:50 Break 11:50 - 12:20 TESTING SESSION, part 1 - Attribute Grammars in Erlang by Ulf Norell and Alex Gerdes 12:30 - 14:00 Lunch 14:00 - 15:00 TESTING SESSION, part 2 - Smother - An MC/DC analysis tool for Erlang by Ramsay Taylor and John Derrick - Linking Unit Tests and Properties by Alex Gerdes, John Hughes, Nick Smallbone and Meng Wang 15:00 - 15:15 Break 15:15 - 16:30 Erlang research - Ongoing projects - RELEASE - SyncFree - Prowess - Paraphrase - Rephrase 16:30 - 17:00 Coffee Break 17:00 - 17:20 Erlang Latest News 17:20 - 17:30 Farewell & Closing REGISTRATION IS OPEN: https://regmaster4.com/2015conf/ICFP15/register.php Local arrangements (including travel and accommodation): http://www.icfpconference.org/icfp2015/local.html We hope to see you in Vancouver! -- Hans Svensson and Melinda T?th Erlang Workshop 2015 Chairs From mrtndimitrov@REDACTED Thu Jul 30 11:56:41 2015 From: mrtndimitrov@REDACTED (Martin Koroudjiev) Date: Thu, 30 Jul 2015 12:56:41 +0300 Subject: [erlang-questions] Is the PID known in MyModule:init/1 function? Message-ID: <55B9F4D9.1020606@gmail.com> Hello, I have to save the PID of the dynamically created gen_servers. Can I use self/0 in MyModule:init/1 function? Thanks, Martin From jesper.louis.andersen@REDACTED Thu Jul 30 12:38:08 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 30 Jul 2015 12:38:08 +0200 Subject: [erlang-questions] Is the PID known in MyModule:init/1 function? In-Reply-To: <55B9F4D9.1020606@gmail.com> References: <55B9F4D9.1020606@gmail.com> Message-ID: Yes, Module:init/1 runs in the context of the newly created process. On Thu, Jul 30, 2015 at 11:56 AM, Martin Koroudjiev wrote: > Hello, > > I have to save the PID of the dynamically created gen_servers. Can I use > self/0 in MyModule:init/1 function? > > Thanks, > Martin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From z@REDACTED Thu Jul 30 12:43:09 2015 From: z@REDACTED (Danil Zagoskin) Date: Thu, 30 Jul 2015 13:43:09 +0300 Subject: [erlang-questions] eJabberd Loadbalancing In-Reply-To: References: Message-ID: Hi! If you control the client's code, you may make use of XMPP see-other-host error: http://xmpp.org/rfcs/rfc6120.html#streams-error-conditions-see-other-host With it you may make a server which would redirect connecting clients to the proper server (if needed) just after client tells its JID. On Mon, Jul 13, 2015 at 9:29 PM, Kannan wrote: > Hi Team, > > We are working on a startup idea that uses eJabberd. We are trying to > build the load balancing into the client, using a discovery call just > before the login procedure. There will be a discovery assist node at the > backend, which maps a chat user to a specific IP. This is some thing > equivalent to DNS SRV lookup, but more at the application layer, and > database shards are handled by the application itself. We are of course > customizing the eJabberd to suit our needs. > > Application is handling the sharding, but will get the help of Mnesia for > replication. > > > Do you see any pitfalls in this method of loadbalancing. > > Regards, > Theepan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrtndimitrov@REDACTED Thu Jul 30 13:09:36 2015 From: mrtndimitrov@REDACTED (Martin Koroudjiev) Date: Thu, 30 Jul 2015 14:09:36 +0300 Subject: [erlang-questions] Is the PID known in MyModule:init/1 function? In-Reply-To: References: <55B9F4D9.1020606@gmail.com> Message-ID: <55BA05F0.4070802@gmail.com> Very cool! Thank you all. Regards, Martin On 7/30/2015 1:38 PM, Jesper Louis Andersen wrote: > Yes, > > Module:init/1 runs in the context of the newly created process. > > On Thu, Jul 30, 2015 at 11:56 AM, Martin Koroudjiev > > wrote: > > Hello, > > I have to save the PID of the dynamically created gen_servers. Can > I use > self/0 in MyModule:init/1 function? > > Thanks, > Martin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From z@REDACTED Thu Jul 30 17:33:30 2015 From: z@REDACTED (Danil Zagoskin) Date: Thu, 30 Jul 2015 18:33:30 +0300 Subject: [erlang-questions] [ANN] inet64_tcp: make the old stuff work in IPv6-only networks Message-ID: Hello! If you need to run random TCP applications in IPv6-only network you usually have to add inet6 option to the gen_tcp:connect or gen_tcp:listen call. This has some drawbacks: * If you edit a third-party code (e.g. database driver), you have to fork the entire project * You have to apply this fix to many places. * Simple fix breaks IPv4-only connectivity With inet64_tcp you can just add a dependency for your application, and this gen_tcp backend will automatically discover IPv4 and IPv6 addresses for a given hostname on connect. Also listening with default options will listen IPv6 too. -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From z@REDACTED Thu Jul 30 17:35:53 2015 From: z@REDACTED (Danil Zagoskin) Date: Thu, 30 Jul 2015 18:35:53 +0300 Subject: [erlang-questions] [ANN] inet64_tcp: make the old stuff work in IPv6-only networks In-Reply-To: References: Message-ID: Forgot to add the link: https://github.com/yandex/inet64_tcp On Thu, Jul 30, 2015 at 6:33 PM, Danil Zagoskin wrote: > Hello! > > If you need to run random TCP applications in IPv6-only network you > usually have to add inet6 option to the gen_tcp:connect or gen_tcp:listen > call. > This has some drawbacks: > * If you edit a third-party code (e.g. database driver), you have to > fork the entire project > * You have to apply this fix to many places. > * Simple fix breaks IPv4-only connectivity > > With inet64_tcp you can just add a dependency for your application, and > this gen_tcp backend will automatically discover IPv4 and IPv6 addresses > for a given hostname on connect. > > Also listening with default options will listen IPv6 too. > > -- > Danil Zagoskin | z@REDACTED > -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Thu Jul 30 19:46:28 2015 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 30 Jul 2015 13:46:28 -0400 Subject: [erlang-questions] gen_server and init In-Reply-To: References: <55B642DE.6040706@ninenines.eu> Message-ID: <20150730171334.GA21552@ferdmbp.local> On 07/27, Karolis Petrauskas wrote: >I think this is not the truth anymore, as pids are now reused. You can >get a message that was meant for previously living process with the >same PID. > Clarification here: it's only going to be a worry if somehow the process with the same pid before died and whoever held references to it kept them active and going (no monitor or links?) long enough for the entire pid space to wrap around to the point of the old process. It's much more likely you'll see this problem with registered processes and untimely handling of monitors; I wouldn't really worry about pids wrapping. From evnix.com@REDACTED Thu Jul 30 20:22:07 2015 From: evnix.com@REDACTED (avinash D'silva) Date: Thu, 30 Jul 2015 23:52:07 +0530 Subject: [erlang-questions] Need some advice on implementing a Queue Server Message-ID: Hi, I am trying to learn Erlang by implementing a small Queueing Server. What have I done so far? I have created a basic websocket server using cowboy. I am trying to implement a Queue which *all processes* can access. I have used lists which can act as queues. The basic insert function looks like this: Queue(Q) -> receive Msg -> Queue(lists:append(Q,[Msg])) end. To create a Queue called "Q_1" > pg2:create(<<"Q_1">>). > P = spawn(test,Queue,[[1]]). % spawn queue and insert 1. > pg2:join(<<"Q_1">>,P). > P ! 2. > P ! 3. % add some elements to queue. to retrieve the elements from the named Queue "Q_1": 1. I use [Px|_]=pg2:get_members() 2. then I would do something like: Px ! POP_ELEMENT Is there a better or more scalable way of doing this? Will this work when a lot of clients are connected( around 60), having a insert rate of 100k elements per hour? Any help is appreciated :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From james@REDACTED Thu Jul 30 20:38:28 2015 From: james@REDACTED (James Aimonetti) Date: Thu, 30 Jul 2015 11:38:28 -0700 Subject: [erlang-questions] telephony In-Reply-To: <55B9D398.3030902@ixperta.com> References: <55B9561D.2080104@2600hz.com> <55B9D398.3030902@ixperta.com> Message-ID: <55BA6F24.6070508@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Le 07/30/2015 12:34 AM, Kukosa Tom?? a ?crit : > On 30.7.2015 0:39, James Aimonetti wrote: >>> Hi List, Is there any open source application out there that >>> provides an Erlanginterface for programming a voip router? Or >>> is it the asterisk ecosystem(for instance with Erlang using >>> AGI) that comes the closest to theoriginal mission? S. > >> FreeSWITCH has two modules, mod_erlang_event and mod_kazoo, >> which present FreeSWITCH as a C-Node to other Erlang VMs. We >> built and use mod_kazoo for Kazoo, but it is generic in purpose; >> I know several folks using mod_erlang_event in production as >> well. Different use cases for each. > > Is there any simple description of difference between mod_kazoo and > mod_erlang_event together with hints which use cases they are good > for? _______________________________________________ > erlang-questions mailing list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > Its been a while that I've looked at mod_erlang_event, but the main differences I recall are: mod_kazoo supports multiple connections from multiple VMs; mod_erlang_event supports one. mod_kazoo opens many TCP sockets for streaming events out of FreeSWITCH to the Erlang VM; mod_erlang_event streams them over the same connection as everything else. mod_erlang_event allows you to take control during the dialplan processing; mod_kazoo does not. Both support being in control of the call from the start (similar to mod_xml_curl if you're familiar with that). mod_kazoo runs more threads and seems more performant internally. I would say (without having seen recent commit history) that mod_kazoo is a more maintained module with a company being behind it (2600Hz), while mod_erlang_event is more community supported at this point (again, could be heresy). We built Kazoo on mod_erlang_event initially, but decided we needed to rewrite it from scratch once we hit a certain level of call volume; mod_kazoo is the result of that effort. - -- James Aimonetti Lead Systems Architect / Impressionable Scallywag "I thought I fixed that" 2600Hz | http://2600hz.com sip:james@REDACTED tel:415.886.7905 irc:mc_ @ freenode -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJVum8kAAoJENTKa+JPXCVgSj8H/2zFV5JF6kqP9nMX5azd3lcG pIc1N9euxi8/aP7Kn0eug4cMCPf8Aik31oPS3OQzB7U/Mr743R8dQq9YeFtl/RPw DSBF++VQR1GhBnZwa7eToiVzBxylrFlSz4mOAbBueg0rflqr0qZLGARgGOMXUHi9 AYKurAoeT2hsehti83eiBpY6L+fqeTxy7SqkBA+t250MbuiW8AEdUZ9AgkrS2HHX eQ/pHqaWurnLME943JK3Z28AS5Nx7eSsmv1XyEfb6sKYyT8oi8aP64BmFBqsE2gM ObwslOJK4E3uQ5rb5tEGHIlF5/9ZjZWtbJStoit1p+MeyWWt+Svw5WOPpTXqGqA= =W3e/ -----END PGP SIGNATURE----- From comptekki@REDACTED Thu Jul 30 21:02:07 2015 From: comptekki@REDACTED (Wes James) Date: Thu, 30 Jul 2015 13:02:07 -0600 Subject: [erlang-questions] Need some advice on implementing a Queue Server In-Reply-To: References: Message-ID: On Thu, Jul 30, 2015 at 12:22 PM, avinash D'silva wrote: > > Hi, > > I am trying to learn Erlang by implementing a small Queueing Server. > > What have I done so far? > I have created a basic websocket server using cowboy. > > I am trying to implement a Queue which all processes can access. > > I have used lists which can act as queues. > > The basic insert function looks like this: > Queue(Q) -> > receive > Msg -> > Queue(lists:append(Q,[Msg])) > end. > > To create a Queue called "Q_1" > > > pg2:create(<<"Q_1">>). > > P = spawn(test,Queue,[[1]]). % spawn queue and insert 1. > > pg2:join(<<"Q_1">>,P). > > P ! 2. > > P ! 3. % add some elements to queue. > > to retrieve the elements from the named Queue "Q_1": > 1. I use [Px|_]=pg2:get_members() > 2. then I would do something like: Px ! POP_ELEMENT > > > Is there a better or more scalable way of doing this? > > Will this work when a lot of clients are connected( around 60), having a insert rate of 100k elements per hour? > > Any help is appreciated :) > I did a search on yahoo for "erlang queue example" and this was the first hit: http://www.erlang.org/doc/man/queue.html There were several other examples that might help. -wes -------------- next part -------------- An HTML attachment was scrubbed... URL: From co7eb@REDACTED Thu Jul 30 21:25:39 2015 From: co7eb@REDACTED (Gilberio Carmenates Garcia) Date: Thu, 30 Jul 2015 15:25:39 -0400 Subject: [erlang-questions] Need some advice on implementing a Queue Server In-Reply-To: References: Message-ID: <002d01d0cafd$8c0b45b0$a421d110$@co.cu> Regards, James, I think using queue module from otp and gen_server behavior, maintaining the Queue in the state, would help you to implement something like that. queue module have many interesting APIs that will do the job for you very faster. Just remember not to use queue:len/1 or any other O(n) order function if you don?t need it, for len you can just keep it in track in the gen_server state along with the queue. NOTE: some of the queue APIs, throws exceptions, use the simple API if you don want to handle exceptions. Ie: Init([])-> {ok, {queue:new(), 0}}. Handle_call({push, Item}, _From, {MyQueue, Len}) {reply, ok, {queue:in(Item, MyQueue), Len + 1}} Handle_call(pop, _From, {MyQueue, Len}) case queue:out(MyQueue) of {{value, Item}, Q} -> {reply, Item, {Q, Len - 1}} {empty, Q} -> {reply, no_item, {Q, Len}} If you want to retrieve some specific item you can always use queue:filter/2 function instead. Something like this: pop_item(Item, Queue)-> queue:filter(fun(I)-> I =/= Item end, Queue). but that is function of order O(n) so? NOTE: I just wrote the example without testing it, so I apologize for any mistake. Best regards, Ivan. De: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] En nombre de Wes James Enviado el: jueves, 30 de julio de 2015 15:02 Para: avinash D'silva CC: erlang-questions@REDACTED Asunto: Re: [erlang-questions] Need some advice on implementing a Queue Server On Thu, Jul 30, 2015 at 12:22 PM, avinash D'silva wrote: > > Hi, > > I am trying to learn Erlang by implementing a small Queueing Server. > > What have I done so far? > I have created a basic websocket server using cowboy. > > I am trying to implement a Queue which all processes can access. > > I have used lists which can act as queues. > > The basic insert function looks like this: > Queue(Q) -> > receive > Msg -> > Queue(lists:append(Q,[Msg])) > end. > > To create a Queue called "Q_1" > > > pg2:create(<<"Q_1">>). > > P = spawn(test,Queue,[[1]]). % spawn queue and insert 1. > > pg2:join(<<"Q_1">>,P). > > P ! 2. > > P ! 3. % add some elements to queue. > > to retrieve the elements from the named Queue "Q_1": > 1. I use [Px|_]=pg2:get_members() > 2. then I would do something like: Px ! POP_ELEMENT > > > Is there a better or more scalable way of doing this? > > Will this work when a lot of clients are connected( around 60), having a insert rate of 100k elements per hour? > > Any help is appreciated :) > I did a search on yahoo for "erlang queue example" and this was the first hit: http://www.erlang.org/doc/man/queue.html There were several other examples that might help. -wes -------------- next part -------------- An HTML attachment was scrubbed... URL: From josh.rubyist@REDACTED Fri Jul 31 02:03:09 2015 From: josh.rubyist@REDACTED (Josh Adams) Date: Thu, 30 Jul 2015 19:03:09 -0500 Subject: [erlang-questions] Compiling Elixir with HiPE fails in buggy-seeming ways Message-ID: So I've recently compiled Elixir using HiPE (because otherwise, I failed to get any performance benefit from compiling my elixir project itself with HiPE, due to context-switches to BEAM modules when using the Elixir standard library). I can work through it by fiddling a bit, but I thought it probably indicates a bug in HiPE. Can someone help me confirm this and tell me where / how I should best report the bug, if it's confirmed? I recorded my attempts in this gist: https://gist.github.com/knewter/533d22eb69ef1036c22a If I do the following in Elixir, it will succeed (but even here, it fails if I use o3, which I documented here: https://gist.github.com/knewter/f4e0a9ba82938ed520fa) For what it's worth, Jose confirmed my suspicions that this was a bug on twitter here: https://twitter.com/josevalim/status/626469188031377408 Anyway, I didn't think it would be good-citizenish of me to fail to report this, even though I have no concept of how to fix it. Thanks in advance for any help pointing me to the best place to report it! -- Josh Adams -------------- next part -------------- An HTML attachment was scrubbed... URL: From evnix.com@REDACTED Fri Jul 31 06:51:02 2015 From: evnix.com@REDACTED (avinash D'silva) Date: Fri, 31 Jul 2015 10:21:02 +0530 Subject: [erlang-questions] Need some advice on implementing a Queue Server In-Reply-To: <002d01d0cafd$8c0b45b0$a421d110$@co.cu> References: <002d01d0cafd$8c0b45b0$a421d110$@co.cu> Message-ID: Thank you both. Gilberio, your example taught me something new today :) BTW, if I want a named queue, should I use pg2:create(<<"Q_1">>). or is there a better way? On Fri, Jul 31, 2015 at 12:55 AM, Gilberio Carmenates Garcia < co7eb@REDACTED> wrote: > Regards, James, I think using queue module from otp and gen_server > behavior, maintaining the Queue in the state, would help you to implement > something like that. queue module have many interesting APIs that will do > the job for you very faster. Just remember not to use queue:len/1 or any > other O(n) order function if you don?t need it, for len you can just keep > it in track in the gen_server state along with the queue. NOTE: some of > the queue APIs, throws exceptions, use the simple API if you don want to > handle exceptions. > > > > Ie: > > Init([])-> > > {ok, {queue:new(), 0}}. > > > > Handle_call({push, Item}, _From, {MyQueue, Len}) > > {reply, ok, {queue:in(Item, MyQueue), Len + 1}} > > > > Handle_call(pop, _From, {MyQueue, Len}) > > case queue:out(MyQueue) of > > {{value, Item}, Q} -> > > {reply, Item, {Q, Len - 1}} > > {empty, Q} -> > > {reply, no_item, {Q, Len}} > > > > If you want to retrieve some specific item you can always use > queue:filter/2 function instead. > > Something like this: > > pop_item(Item, Queue)-> > > queue:filter(fun(I)-> I =/= Item end, Queue). > > > > but that is function of order O(n) so? > > > > NOTE: I just wrote the example without testing it, so I apologize for any > mistake. > > > > Best regards, > > Ivan. > > > > > > *De:* erlang-questions-bounces@REDACTED [mailto: > erlang-questions-bounces@REDACTED] *En nombre de *Wes James > *Enviado el:* jueves, 30 de julio de 2015 15:02 > *Para:* avinash D'silva > *CC:* erlang-questions@REDACTED > *Asunto:* Re: [erlang-questions] Need some advice on implementing a Queue > Server > > > > > > On Thu, Jul 30, 2015 at 12:22 PM, avinash D'silva > wrote: > > > > Hi, > > > > I am trying to learn Erlang by implementing a small Queueing Server. > > > > What have I done so far? > > I have created a basic websocket server using cowboy. > > > > I am trying to implement a Queue which all processes can access. > > > > I have used lists which can act as queues. > > > > The basic insert function looks like this: > > Queue(Q) -> > > receive > > Msg -> > > Queue(lists:append(Q,[Msg])) > > end. > > > > To create a Queue called "Q_1" > > > > > pg2:create(<<"Q_1">>). > > > P = spawn(test,Queue,[[1]]). % spawn queue and insert 1. > > > pg2:join(<<"Q_1">>,P). > > > P ! 2. > > > P ! 3. % add some elements to queue. > > > > to retrieve the elements from the named Queue "Q_1": > > 1. I use [Px|_]=pg2:get_members() > > 2. then I would do something like: Px ! POP_ELEMENT > > > > > > Is there a better or more scalable way of doing this? > > > > Will this work when a lot of clients are connected( around 60), having a > insert rate of 100k elements per hour? > > > > Any help is appreciated :) > > > > I did a search on yahoo for "erlang queue example" and this was the first > hit: > > http://www.erlang.org/doc/man/queue.html > > There were several other examples that might help. > > -wes > > > > -- Powered By codologic -------------- next part -------------- An HTML attachment was scrubbed... URL: From torben.hoffmann@REDACTED Fri Jul 31 09:35:05 2015 From: torben.hoffmann@REDACTED (Torben Hoffmann) Date: Fri, 31 Jul 2015 09:35:05 +0200 Subject: [erlang-questions] Need some advice on implementing a Queue Server In-Reply-To: References: <002d01d0cafd$8c0b45b0$a421d110$@co.cu> Message-ID: Slightly off topic, but I have used pg2 to create a simple pub/sub library for Elixir at https://github.com/lehoff/ogma Might be a source for inspiration. Keep in mind that pg2 merely provides a way to send a message to multiple processes in one go. All the semantics of your queue has to be dealt with in your code, but I'm sure you are taking this into account. Cheers, Torben avinash D'silva writes: > Thank you both. > > Gilberio, your example taught me something new today :) > > BTW, if I want a named queue, should I use pg2:create(<<"Q_1">>). or is > there a better way? > > On Fri, Jul 31, 2015 at 12:55 AM, Gilberio Carmenates Garcia < > co7eb@REDACTED> wrote: > >> Regards, James, I think using queue module from otp and gen_server >> behavior, maintaining the Queue in the state, would help you to implement >> something like that. queue module have many interesting APIs that will do >> the job for you very faster. Just remember not to use queue:len/1 or any >> other O(n) order function if you don?t need it, for len you can just keep >> it in track in the gen_server state along with the queue. NOTE: some of >> the queue APIs, throws exceptions, use the simple API if you don want to >> handle exceptions. >> >> >> >> Ie: >> >> Init([])-> >> >> {ok, {queue:new(), 0}}. >> >> >> >> Handle_call({push, Item}, _From, {MyQueue, Len}) >> >> {reply, ok, {queue:in(Item, MyQueue), Len + 1}} >> >> >> >> Handle_call(pop, _From, {MyQueue, Len}) >> >> case queue:out(MyQueue) of >> >> {{value, Item}, Q} -> >> >> {reply, Item, {Q, Len - 1}} >> >> {empty, Q} -> >> >> {reply, no_item, {Q, Len}} >> >> >> >> If you want to retrieve some specific item you can always use >> queue:filter/2 function instead. >> >> Something like this: >> >> pop_item(Item, Queue)-> >> >> queue:filter(fun(I)-> I =/= Item end, Queue). >> >> >> >> but that is function of order O(n) so? >> >> >> >> NOTE: I just wrote the example without testing it, so I apologize for any >> mistake. >> >> >> >> Best regards, >> >> Ivan. >> >> >> >> >> >> *De:* erlang-questions-bounces@REDACTED [mailto: >> erlang-questions-bounces@REDACTED] *En nombre de *Wes James >> *Enviado el:* jueves, 30 de julio de 2015 15:02 >> *Para:* avinash D'silva >> *CC:* erlang-questions@REDACTED >> *Asunto:* Re: [erlang-questions] Need some advice on implementing a Queue >> Server >> >> >> >> >> >> On Thu, Jul 30, 2015 at 12:22 PM, avinash D'silva >> wrote: >> > >> > Hi, >> > >> > I am trying to learn Erlang by implementing a small Queueing Server. >> > >> > What have I done so far? >> > I have created a basic websocket server using cowboy. >> > >> > I am trying to implement a Queue which all processes can access. >> > >> > I have used lists which can act as queues. >> > >> > The basic insert function looks like this: >> > Queue(Q) -> >> > receive >> > Msg -> >> > Queue(lists:append(Q,[Msg])) >> > end. >> > >> > To create a Queue called "Q_1" >> > >> > > pg2:create(<<"Q_1">>). >> > > P = spawn(test,Queue,[[1]]). % spawn queue and insert 1. >> > > pg2:join(<<"Q_1">>,P). >> > > P ! 2. >> > > P ! 3. % add some elements to queue. >> > >> > to retrieve the elements from the named Queue "Q_1": >> > 1. I use [Px|_]=pg2:get_members() >> > 2. then I would do something like: Px ! POP_ELEMENT >> > >> > >> > Is there a better or more scalable way of doing this? >> > >> > Will this work when a lot of clients are connected( around 60), having a >> insert rate of 100k elements per hour? >> > >> > Any help is appreciated :) >> > >> >> I did a search on yahoo for "erlang queue example" and this was the first >> hit: >> >> http://www.erlang.org/doc/man/queue.html >> >> There were several other examples that might help. >> >> -wes >> >> >> >> -- Torben Hoffmann CTO, erlang-solutions.com T: +45 25 14 05 38 From nico.kruber@REDACTED Fri Jul 31 18:15:52 2015 From: nico.kruber@REDACTED (Nico Kruber) Date: Fri, 31 Jul 2015 18:15:52 +0200 Subject: [erlang-questions] Erlang hangs in supervisor:do_terminate/2 In-Reply-To: References: <1794014.6vMsUISv8L@nico-pc> Message-ID: <3527639.eQQigZ7nU2@csr-pc40> On Monday 13 Jul 2015 10:11:38 Lukas Larsson wrote: > Hello Nico, > > On Sat, Jul 11, 2015 at 2:33 PM, Nico Kruber wrote: > > Hi, > > I'm having trouble with supervisor:do_terminate/2 in both, Erlang 18.0.1 > > and > > 18.0.2 which I haven't seen with earlier versions so far. I currently do > > not > > have a minimal use case but will try to come up with something soon. > > Would be great if you could manage that. Minimal examples are always much > easier to work with. > > > I'm using Scalaris and inside a single process, I'm starting its services > > (multiple processes in a supervisor tree) and stopping them again in a > > loop. > > Sometimes, although very rarely, stopping the services seems to hang. When > > I > > send the beam process a SIGUSR1 I can always see two processes being in > > "Running" state: > > 1) a supervisor in supervisor:do_terminate/2 (any of the present > > supervisors - > > not always the same!) > > 2) a child/worker of this supervisor handling a message (or at least, so > > it > > seems) > > > > Their stacktraces seem inconclusive, please find an example of the two > > processes from the crashdump_viewer below. > > Are the stack traces always the same for the supervisor? i.e. it is Running > in do_terminate? Since there is a receive in do_terminate, if it is stuck I > would have expected it to be in the Waiting state. But if it is Running and > hanging, it could point to some kind of live lock. > > > Is there any known problem/change in Erlang 18 that could have caused > > this? > > There are changes in supervisor for 18 (the introduction of maps as child > specs), but it *should* not cause any problems like these. > > A smaller example demonstrating what is going wrong for you would help a > lot in trying to understand what is going on. > > Lukas Hi Lukas, unfortunately, reproducing this error requires some effort. For now, I can safely reproduce it with Scalaris (see below) but the system needs to be overloaded for the bug to appear early. The supervisor stack traces are always the same (with do_terminate on top). Whenever the bug appears, everything is stuck in that erlang VM, even remote connections, e.g. for debugging, do not work anymore but the node is still listed in epmd. Please find the gdb backtraces attached. I created 2 snapshots each of 2 hanging erlang VMs - it does indeed look like a livelock since the traces only differ slightly. I also have the erlang crashdump files but they are too large to post here (3-4MiB) - I can send them to you personally if you like to have a look. Regards Nico ## Way to re-produce using Scalaris: wget https://github.com/scalaris-team/scalaris/archive/master.tar.gz -O - | tar -xz cd scalaris-master ./configure && make then run the following in (#CPU-cores*2) shells (e.g. 8 shells for the 4 CPU cores I have): read -r -d '' EVALCMD <<'EOF' log:set_log_level(none), [begin io:format(" ~B", [I]), admin:add_node([{first}]),admin:add_nodes(3), io:format("K "), Killed = api_vm:kill_nodes(4), 4 = length(Killed) end || I <- lists:seq(1, 10000)]. EOF SCALARIS_PORT=$RANDOM ERL_SCHED_FLAGS="" ./bin/scalarisctl -n node$RANDOM -m start -y $RANDOM -p $SCALARIS_PORT -t first_nostart -e "-noinput -scalaris mgmt_server \"{{127,0,0,1},${SCALARIS_PORT},mgmt_server}\" -scalaris known_hosts \"[{{127,0,0,1},${SCALARIS_PORT},service_per_vm}]\" -scalaris monitor_perf_interval \"0\" -scalaris lb_active_use_gossip \"false\" -scalaris gossip_load_number_of_buckets \"1\" -scalaris gossip_load_additional_modules \"[]\" -pa '$PWD/test' -eval '${EVALCMD}'" (if one start fails immediately, just try again - this hack uses $RANDOM but does not restrict its values accordingly...) anyway, after some time, at least one of these processes will stop creating any output -> this erlang VM hangs! FYI: gdb traces are created with gdb -ex "set pagination 0" -ex "thread apply all bt full" --batch -p -------------- next part -------------- A non-text attachment was scrubbed... Name: hang-1.tar.gz Type: application/x-compressed-tar Size: 9478 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: hang-2.tar.gz Type: application/x-compressed-tar Size: 9684 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: This is a digitally signed message part. URL: From nistrigunya@REDACTED Fri Jul 31 17:13:25 2015 From: nistrigunya@REDACTED (Avinash Dhumane) Date: Fri, 31 Jul 2015 20:43:25 +0530 Subject: [erlang-questions] Effectiveness of "selective receive" Message-ID: I have a need to lookup a tuple based on a 32-bit integer key and remove it from a set after it is found. The set is shared by 2 processes - one inserts the tuple and the other performs a lookup followed by a delete. So, I am considering ETS versus selective receive. Consideration for selective receive because I need the lookup call to block until the tuple becomes available in the set. I suppose selective receive will not be able to lookup in constant time whereas set-ETS will! But, I favour the selective receive because it makes the code so simple and intuitive. So, my question is this - till what point, in terms of number of tuples in the set, the selective receive will be effective. The insertion in the set is capped at 400 tuples per second, but the lookup (followed by a delete) will be considerably slower (by 20 to 70%) than the rate of inserts. Please advise on considerations that apply in this situation in order to employ a suitable scheme. Thanks Avinash -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Fri Jul 31 19:05:26 2015 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 31 Jul 2015 19:05:26 +0200 Subject: [erlang-questions] Effectiveness of "selective receive" In-Reply-To: References: Message-ID: If I understand you correctly, you would essentially busy-loop over the ETS table until the object is gone? This would not be constant time. Selective receive can be done largely in constant time if you take care, e.g. call(Pid, Req) -> Ref = erlang:monitor(process, Pid), Pid ! {?$call?, {self(), Ref}, Req}, receive {Ref, Reply} -> Reply; {?DOWN?, Ref, _, _, Reason} -> error(Reason) end. The above will be detected by the compiler as an optimizable construct, since all patterns in the receive clause contain the newly created unique reference Ref. Therefore, the compiler can ?mark? the end of the message queue before creating Ref, then jump directly to the mark for the receive. No messages before the mark can contain Ref, since it didn?t exist then. This optimization is built into gen_server:call(), but you can exploit it yourself by ensuring that the Ref creation (either monitor() or make_ref()) is done in the same scope (same function) as the receive, and the patterns in the receive clause all contain Ref. The cost of the receive would be proportional to the number of messages arriving to the receiving process while it?s waiting, since it has to pattern-match on each. BR, Ulf > On 31 Jul 2015, at 17:13, Avinash Dhumane > wrote: > > I have a need to lookup a tuple based on a 32-bit integer key and remove it from a set after it is found. The set is shared by 2 processes - one inserts the tuple and the other performs a lookup followed by a delete. > > So, I am considering ETS versus selective receive. Consideration for selective receive because I need the lookup call to block until the tuple becomes available in the set. > > I suppose selective receive will not be able to lookup in constant time whereas set-ETS will! But, I favour the selective receive because it makes the code so simple and intuitive. > > So, my question is this - till what point, in terms of number of tuples in the set, the selective receive will be effective. The insertion in the set is capped at 400 tuples per second, but the lookup (followed by a delete) will be considerably slower (by 20 to 70%) than the rate of inserts. > > Please advise on considerations that apply in this situation in order to employ a suitable scheme. > > Thanks > Avinash > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From co7eb@REDACTED Fri Jul 31 20:08:11 2015 From: co7eb@REDACTED (Gilberio Carmenates Garcia) Date: Fri, 31 Jul 2015 14:08:11 -0400 Subject: [erlang-questions] Need some advice on implementing a Queue Server In-Reply-To: References: <002d01d0cafd$8c0b45b0$a421d110$@co.cu> Message-ID: <000601d0cbbb$ded63ed0$9c82bc70$@co.cu> Hi, for that case you could also have a dictionary of queues then name them as key Queues = dict:new() dict:store(<<"Q_1">>, queue:new(), Queues), later you just do: dict:fetch(<<"Q_1">>, Queues) you can also store the Queues Dictionary in the gen_server state and retrieve the queue you want with a simple gen_server:call/2. get_queue(Name)-> gen_server:call(?MODULE, {get_queue, Name}). handle_call({get_queue, Name} _From, State)-> %% you can have a well defined data structure in State if you define it as a record you want. Queues = State#state.queues, R = dict:fetch(Name, Queues), %% this throws an error if no key was found so you can use %% dict:find/2 or handle the error. {reply, R, State}. Again: apologize for any typo. Best Regards, Ivan. De: avinash D'silva [mailto:evnix.com@REDACTED] Enviado el: viernes, 31 de julio de 2015 0:51 Para: Gilberio Carmenates Garcia CC: Wes James; erlang-questions@REDACTED Asunto: Re: [erlang-questions] Need some advice on implementing a Queue Server Thank you both. Gilberio, your example taught me something new today :) BTW, if I want a named queue, should I use pg2:create(<<"Q_1">>). or is there a better way? On Fri, Jul 31, 2015 at 12:55 AM, Gilberio Carmenates Garcia wrote: Regards, James, I think using queue module from otp and gen_server behavior, maintaining the Queue in the state, would help you to implement something like that. queue module have many interesting APIs that will do the job for you very faster. Just remember not to use queue:len/1 or any other O(n) order function if you don?t need it, for len you can just keep it in track in the gen_server state along with the queue. NOTE: some of the queue APIs, throws exceptions, use the simple API if you don want to handle exceptions. Ie: Init([])-> {ok, {queue:new(), 0}}. Handle_call({push, Item}, _From, {MyQueue, Len}) {reply, ok, {queue:in(Item, MyQueue), Len + 1}} Handle_call(pop, _From, {MyQueue, Len}) case queue:out(MyQueue) of {{value, Item}, Q} -> {reply, Item, {Q, Len - 1}} {empty, Q} -> {reply, no_item, {Q, Len}} If you want to retrieve some specific item you can always use queue:filter/2 function instead. Something like this: pop_item(Item, Queue)-> queue:filter(fun(I)-> I =/= Item end, Queue). but that is function of order O(n) so? NOTE: I just wrote the example without testing it, so I apologize for any mistake. Best regards, Ivan. De: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] En nombre de Wes James Enviado el: jueves, 30 de julio de 2015 15:02 Para: avinash D'silva CC: erlang-questions@REDACTED Asunto: Re: [erlang-questions] Need some advice on implementing a Queue Server On Thu, Jul 30, 2015 at 12:22 PM, avinash D'silva wrote: > > Hi, > > I am trying to learn Erlang by implementing a small Queueing Server. > > What have I done so far? > I have created a basic websocket server using cowboy. > > I am trying to implement a Queue which all processes can access. > > I have used lists which can act as queues. > > The basic insert function looks like this: > Queue(Q) -> > receive > Msg -> > Queue(lists:append(Q,[Msg])) > end. > > To create a Queue called "Q_1" > > > pg2:create(<<"Q_1">>). > > P = spawn(test,Queue,[[1]]). % spawn queue and insert 1. > > pg2:join(<<"Q_1">>,P). > > P ! 2. > > P ! 3. % add some elements to queue. > > to retrieve the elements from the named Queue "Q_1": > 1. I use [Px|_]=pg2:get_members() > 2. then I would do something like: Px ! POP_ELEMENT > > > Is there a better or more scalable way of doing this? > > Will this work when a lot of clients are connected( around 60), having a insert rate of 100k elements per hour? > > Any help is appreciated :) > I did a search on yahoo for "erlang queue example" and this was the first hit: http://www.erlang.org/doc/man/queue.html There were several other examples that might help. -wes -- Powered By codologic -------------- next part -------------- An HTML attachment was scrubbed... URL: